Concept of inventory item and field
We’ll use the term
inventory item
for an item that we could potentially show in recommendations. For instance:We’ll use the term field for each piece of information about a given item. For instance, the typical fields for an article `inventory item` include title, URL, image, and description.
- On a typical news or viral content website, the `inventory items` are articles.
- On a video website, the `inventory items` are videos.
- On a retail website, the `inventory items` are the products being sold.
How our scraping works
Prior to sending you our Javascript snippet, we identify what inventory item
means in the context of your website, and set up our scraping to collect inventory information only for those items. We generally set up preliminary filters based on the pattern of the URL, and the type of the page as specified in the page metadata.
Once you install our Javascript, we start scraping your inventory. Every time a user visits a page on your website that meets our critera for an inventory item, we collect all the fields from that page and send it to our servers. If the item is not currently in our inventory, we add it. Otherwise we update it.
Almost-real-time inventory
Our inventory is up-to-date subject to the following caveats:
- We can only detect pages that have been visited at least once by a user while our Javascript was installed.
- We update fields for a page only when it is visited, and it gets deleted if nobody has accessed it for more than 30 days (we can remove the TTL depending on your use case).
- If a page gets deleted, and didn't clean up the data by using the inventory DELETE endpoint of our REST API. You can configure the TTL for items inserted through REST API [here](https://liftigniter.readme.io/docs/inventory).