{"_id":"57e1d1899e590717008c9191","githubsync":"","parentDoc":null,"version":{"_id":"5668fab608f90021008e8832","__v":19,"project":"5668fab608f90021008e882f","createdAt":"2015-12-10T04:08:22.769Z","releaseDate":"2015-12-10T04:08:22.769Z","categories":["5668fab708f90021008e8833","569740f124490c3700170a64","569742b58560a60d00e2c25d","569742bd0b09a41900b2446c","569742cd69393517000c82b3","569742f459a6692d003fad8f","569743020b09a41900b2446d","5697430b69393517000c82b5","56a17776470ae00d00c30642","56a2c48a831e2a0d0069b1ad","56b535757bccae0d00e9a1cd","56e1ff6aa49fdc0e005746b5","57e1c88115bf6522002a5e4e","57fa65275ba65a17008b988f","57fbeea34002550e004c032e","58474584889b6c2d00fb86e9","58475dcc64157f0f002f1907","587e7b5158666c2700965d4e","58a349fc30852819007ba083"],"is_deprecated":false,"is_hidden":false,"is_beta":false,"is_stable":true,"codename":"","version_clean":"1.18.0","version":"1.18"},"__v":0,"project":"5668fab608f90021008e882f","category":{"_id":"57e1c88115bf6522002a5e4e","project":"5668fab608f90021008e882f","__v":0,"version":"5668fab608f90021008e8832","sync":{"url":"","isSync":false},"reference":false,"createdAt":"2016-09-20T23:38:41.155Z","from_sync":false,"order":11,"slug":"metrics","title":"Metrics"},"user":"5668fa9755e4b32100935d41","updates":[],"next":{"pages":[],"description":""},"createdAt":"2016-09-21T00:17:13.680Z","link_external":false,"link_url":"","sync_unique":"","hidden":false,"api":{"results":{"codes":[]},"settings":"","auth":"required","params":[],"url":""},"isReference":false,"order":4,"body":"The widget shown event (called `widget_shown`) is triggered every time a set of recommendations (whether LiftIgniter's or any other algorithm's) is rendered. Developers can get a quick summary at the [Widget Events](doc:widget-events) page. This page is more for people interested in data science and analysis.\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"1. Conditions for triggering\"\n}\n[/block]\nThe widget shown event should be triggered whenever recommendations by LiftIgniter or any other recommendation system are rendered on the page. The definition of \"rendered\" can be variable and depends on your page layout. We receive the event through your implementation of our [Tracking Widgets](doc:tracking-widgets) logic. Note that if you haven't implemented tracking, we won't receive this event.\n\nOn standard HTML page layouts without pagination or lazy loading, all widgets should be rendered on page load. In this case, you should get one `widget_shown` event per widget per pageview. On paginated layouts (such as slideshows) or lazy loading recommendation scrolls, you have flexibility in deciding when to actually render the recommendations. For instance, you might decide to fetch, render, and track end-of-slideshow recommendations on page load, or you might do so only when the user has reached the last slide.\n\nThe `widget_shown` event can be triggered even in cases where the widget area does not actually enter the user's viewport. There is a similar event called [widget visible](doc:widget-visible) that triggers only on entering the user's viewport.\n\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"2. Fields in widget shown events\"\n}\n[/block]\nWidget shown events have the following special fields that play a key role for performance tracking, analytics, and machine learning:\n\n* Visible items (called `visibleItems`, and compactified in our JS as `vi`): This is a list of the recommended items shown in the widget. The list of visible items helps our machine learning systems keep track of how often specific items are getting shown, so that we can calculate item CTRs and our system can better gauge the performance of specific items.\n* Widget name (called `widgetName`, and compactified in our JS as `w`): This is the name of the widget. For instance, you might have difference widget names like `home-page-recommendations` and `article-recommendations`.\n* Source (called `source`): This gives information on the algorithm used as the source of recommendations. We use `LI` for LiftIgniter's recommendations (in our Analytics, this shows up as \"LiftIgniter\") and `base` for the baseline recommendations. You can use other names.\n\nNote that although we in principle \"know\" the recommendations we return in the case that the recommendations are powered by LiftIgniter, getting confirmation of what was actually shown in the form of visible items is important, since this leaves you with some flexibility to filter or edit the recommendations after we return them. It also makes for a cleaner implementation since the same machine learning logic can apply across different algorithms including those not powered by LiftIgniter.\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"3. Comparing widget shown event values in an A/B test\"\n}\n[/block]\n\n[block:image]\n{\n  \"images\": [\n    {\n      \"image\": [\n        \"https://files.readme.io/78d0dac-widget-shown-ab-example.png\",\n        \"widget-shown-ab-example.png\",\n        1712,\n        784,\n        \"#e0eaf8\"\n      ],\n      \"caption\": \"Widget shown values in a fair 50/50 A/B test. Although traffic varies a lot by day, values for LiftIgniter and the baseline are close to equal on any given day.\"\n    }\n  ]\n}\n[/block]\nLiftIgniter does A/B testing by user. This means that when we do a 50/50 A/B test, 50% of users see LiftIgniter's recommendations all the time, and the other 50% see the baseline recommendations all the time.\n\nThis means that, at the point in time that an A/B test is launched, LiftIgniter and the base should have equal numbers of `widget_shown`. However, differences in the algorithm can lead to differences in the total number of `widget_shown` events, for both direct and indirect reasons.\n\n* The **direct effect**: If the CTR is high (generally if it is over 5%) and the pages that users click to *also* show the same recommendation widget, then the slice with higher CTR will also end up getting more `widget_shown` events.\n* The **indirect effect** (which might manifest itself after several weeks): More engaging recommendations could drive up users' rate of return, and therefore result in more `widget_shown` events in one slice.\n\nIn other words, if you see a *slight* but statistically significant difference in the shown counts in a 50/50 A/B test, and this is in the same direction as that of differences in CTR, it is probably not a bug but a mix of the direct and indirect effects.\n[block:api-header]\n{\n  \"type\": \"basic\",\n  \"title\": \"4. Derived metrics\"\n}\n[/block]\n\n[block:parameters]\n{\n  \"data\": {\n    \"0-0\": \"Click-Through Rate (CTR)\",\n    \"h-0\": \"Derived metric\",\n    \"h-1\": \"Numerator\",\n    \"h-2\": \"Denominator\",\n    \"h-3\": \"Supported by LiftIgniter?\",\n    \"0-1\": \"Widget Click\",\n    \"0-2\": \"Widget Shown\",\n    \"0-3\": \"Yes, with [Tracking Widgets](doc:tracking-widgets) implemented.\",\n    \"1-0\": \"Engaged Click-Through Rate (ECTR)\",\n    \"1-1\": \"Engaged Click\",\n    \"1-2\": \"Widget Shown\",\n    \"1-3\": \"Yes, with [Tracking Widgets](doc:tracking-widgets) implemented and engagement definition implemented.\",\n    \"2-0\": \"Visibility\",\n    \"2-1\": \"Widget Visible\",\n    \"2-2\": \"Widget Shown\",\n    \"2-3\": \"Yes, with tracking implemented.\",\n    \"3-0\": \"Conversion-to-Shown Ratio\",\n    \"3-1\": \"Conversion (through click)\",\n    \"3-2\": \"Widget Shown\",\n    \"3-3\": \"Yes, with [Tracking Widgets](doc:tracking-widgets)  and [Tracking Conversion](doc:tracking-conversion) implemented.\"\n  },\n  \"cols\": 4,\n  \"rows\": 4\n}\n[/block]","excerpt":"","slug":"widget-shown","type":"basic","title":"Widget Shown"}
The widget shown event (called `widget_shown`) is triggered every time a set of recommendations (whether LiftIgniter's or any other algorithm's) is rendered. Developers can get a quick summary at the [Widget Events](doc:widget-events) page. This page is more for people interested in data science and analysis. [block:api-header] { "type": "basic", "title": "1. Conditions for triggering" } [/block] The widget shown event should be triggered whenever recommendations by LiftIgniter or any other recommendation system are rendered on the page. The definition of "rendered" can be variable and depends on your page layout. We receive the event through your implementation of our [Tracking Widgets](doc:tracking-widgets) logic. Note that if you haven't implemented tracking, we won't receive this event. On standard HTML page layouts without pagination or lazy loading, all widgets should be rendered on page load. In this case, you should get one `widget_shown` event per widget per pageview. On paginated layouts (such as slideshows) or lazy loading recommendation scrolls, you have flexibility in deciding when to actually render the recommendations. For instance, you might decide to fetch, render, and track end-of-slideshow recommendations on page load, or you might do so only when the user has reached the last slide. The `widget_shown` event can be triggered even in cases where the widget area does not actually enter the user's viewport. There is a similar event called [widget visible](doc:widget-visible) that triggers only on entering the user's viewport. [block:api-header] { "type": "basic", "title": "2. Fields in widget shown events" } [/block] Widget shown events have the following special fields that play a key role for performance tracking, analytics, and machine learning: * Visible items (called `visibleItems`, and compactified in our JS as `vi`): This is a list of the recommended items shown in the widget. The list of visible items helps our machine learning systems keep track of how often specific items are getting shown, so that we can calculate item CTRs and our system can better gauge the performance of specific items. * Widget name (called `widgetName`, and compactified in our JS as `w`): This is the name of the widget. For instance, you might have difference widget names like `home-page-recommendations` and `article-recommendations`. * Source (called `source`): This gives information on the algorithm used as the source of recommendations. We use `LI` for LiftIgniter's recommendations (in our Analytics, this shows up as "LiftIgniter") and `base` for the baseline recommendations. You can use other names. Note that although we in principle "know" the recommendations we return in the case that the recommendations are powered by LiftIgniter, getting confirmation of what was actually shown in the form of visible items is important, since this leaves you with some flexibility to filter or edit the recommendations after we return them. It also makes for a cleaner implementation since the same machine learning logic can apply across different algorithms including those not powered by LiftIgniter. [block:api-header] { "type": "basic", "title": "3. Comparing widget shown event values in an A/B test" } [/block] [block:image] { "images": [ { "image": [ "https://files.readme.io/78d0dac-widget-shown-ab-example.png", "widget-shown-ab-example.png", 1712, 784, "#e0eaf8" ], "caption": "Widget shown values in a fair 50/50 A/B test. Although traffic varies a lot by day, values for LiftIgniter and the baseline are close to equal on any given day." } ] } [/block] LiftIgniter does A/B testing by user. This means that when we do a 50/50 A/B test, 50% of users see LiftIgniter's recommendations all the time, and the other 50% see the baseline recommendations all the time. This means that, at the point in time that an A/B test is launched, LiftIgniter and the base should have equal numbers of `widget_shown`. However, differences in the algorithm can lead to differences in the total number of `widget_shown` events, for both direct and indirect reasons. * The **direct effect**: If the CTR is high (generally if it is over 5%) and the pages that users click to *also* show the same recommendation widget, then the slice with higher CTR will also end up getting more `widget_shown` events. * The **indirect effect** (which might manifest itself after several weeks): More engaging recommendations could drive up users' rate of return, and therefore result in more `widget_shown` events in one slice. In other words, if you see a *slight* but statistically significant difference in the shown counts in a 50/50 A/B test, and this is in the same direction as that of differences in CTR, it is probably not a bug but a mix of the direct and indirect effects. [block:api-header] { "type": "basic", "title": "4. Derived metrics" } [/block] [block:parameters] { "data": { "0-0": "Click-Through Rate (CTR)", "h-0": "Derived metric", "h-1": "Numerator", "h-2": "Denominator", "h-3": "Supported by LiftIgniter?", "0-1": "Widget Click", "0-2": "Widget Shown", "0-3": "Yes, with [Tracking Widgets](doc:tracking-widgets) implemented.", "1-0": "Engaged Click-Through Rate (ECTR)", "1-1": "Engaged Click", "1-2": "Widget Shown", "1-3": "Yes, with [Tracking Widgets](doc:tracking-widgets) implemented and engagement definition implemented.", "2-0": "Visibility", "2-1": "Widget Visible", "2-2": "Widget Shown", "2-3": "Yes, with tracking implemented.", "3-0": "Conversion-to-Shown Ratio", "3-1": "Conversion (through click)", "3-2": "Widget Shown", "3-3": "Yes, with [Tracking Widgets](doc:tracking-widgets) and [Tracking Conversion](doc:tracking-conversion) implemented." }, "cols": 4, "rows": 4 } [/block]