Fetch Items from an eBay Listing Page

Language: Crystal

The goal of this sample is to demonstrate fetching HTML data from a page (an eBay search listing), parsing the HTML content using a 3rd party library1, and looking for a specific attribute within the content (listingId). The elements that have it contain IDs to the products that appear on the given listing page. We would finally map those out, and turn the resulting array into a set (to_s) in order to remove duplicates.

require "http/client"
require "myhtml"

module Bayhunt
  res = HTTP::Client.get "https://www.ebay.com/sch/i.html?_nkw=iphone+xr"
  parser = Myhtml::Parser.new(res.body)

  listing_ids = parser.css("[listingId]")
    .to_a
    .map do |it|
      it.attributes["listingid"].to_i64
    end.to_s

  # or, use the shorter-form version (`&`) if there is only one argument passed to the block
  listing_ids = parser.css("[listingId]")
    .to_a
    .map(&.attributes["listingid"].to_i64)
    .to_s

  p listing_ids
end