The widget shown event (called
widget_shown) is triggered every time a set of recommendations (whether LiftIgniter's or any other algorithm's) is rendered. Developers can get a quick summary at the Widget Events page. This page is more for people interested in data science and analysis.
1. Conditions for triggering
The widget shown event should be triggered whenever recommendations by LiftIgniter or any other recommendation system are rendered on the page. The definition of "rendered" can be variable and depends on your page layout. We receive the event through your implementation of our Tracking Widgets logic. Note that if you haven't implemented tracking, we won't receive this event.
On standard HTML page layouts without pagination or lazy loading, all widgets should be rendered on page load. In this case, you should get one
widget_shown event per widget per pageview. On paginated layouts (such as slideshows) or lazy loading recommendation scrolls, you have flexibility in deciding when to actually render the recommendations. For instance, you might decide to fetch, render, and track end-of-slideshow recommendations on page load, or you might do so only when the user has reached the last slide.
widget_shown event can be triggered even in cases where the widget area does not actually enter the user's viewport. There is a similar event called widget visible that triggers only on entering the user's viewport.
2. Fields in widget shown events
Widget shown events have the following special fields that play a key role for performance tracking, analytics, and machine learning:
- Visible items (called
visibleItems, and compactified in our JS as
vi): This is a list of the recommended items shown in the widget. The list of visible items helps our machine learning systems keep track of how often specific items are getting shown, so that we can calculate item CTRs and our system can better gauge the performance of specific items.
- Widget name (called
widgetName, and compactified in our JS as
w): This is the name of the widget. For instance, you might have difference widget names like
- Source (called
source): This gives information on the algorithm used as the source of recommendations. We use
LIfor LiftIgniter's recommendations (in our Analytics, this shows up as "LiftIgniter") and
basefor the baseline recommendations. You can use other names.
Note that although we in principle "know" the recommendations we return in the case that the recommendations are powered by LiftIgniter, getting confirmation of what was actually shown in the form of visible items is important, since this leaves you with some flexibility to filter or edit the recommendations after we return them. It also makes for a cleaner implementation since the same machine learning logic can apply across different algorithms including those not powered by LiftIgniter.
3. Comparing widget shown event values in an A/B test
LiftIgniter does A/B testing by user. This means that when we do a 50/50 A/B test, 50% of users see LiftIgniter's recommendations all the time, and the other 50% see the baseline recommendations all the time.
This means that, at the point in time that an A/B test is launched, LiftIgniter and the base should have equal numbers of
widget_shown. However, differences in the algorithm can lead to differences in the total number of
widget_shown events, for both direct and indirect reasons.
- The direct effect: If the CTR is high (generally if it is over 5%) and the pages that users click to also show the same recommendation widget, then the slice with higher CTR will also end up getting more
- The indirect effect (which might manifest itself after several weeks): More engaging recommendations could drive up users' rate of return, and therefore result in more
widget_shownevents in one slice.
In other words, if you see a slight but statistically significant difference in the shown counts in a 50/50 A/B test, and this is in the same direction as that of differences in CTR, it is probably not a bug but a mix of the direct and indirect effects.
4. Derived metrics
|Derived metric||Numerator||Denominator||Supported by LiftIgniter?|
|Click-Through Rate (CTR)||Widget Click||Widget Shown||Yes, with Tracking Widgets implemented.|
|Engaged Click-Through Rate (ECTR)||Engaged Click||Widget Shown||Yes, with Tracking Widgets implemented and engagement definition implemented.|
|Visibility||Widget Visible||Widget Shown||Yes, with tracking implemented.|
|Conversion-to-Shown Ratio||Conversion (through click)||Widget Shown||Yes, with Tracking Widgets and Tracking Conversion implemented.|