Attribution
Part of Basket and merchandising analytics
Measuring the effect of a product recommendation
Separate recommendation exposure and clicks from sales effect, then compare eligible groups on a defined commercial outcome.
To assess a recommendation's effect, compare a commercial outcome for eligible shoppers assigned to see it against the same outcome for a comparable group assigned a different experience. Record what was actually displayed. Clicks and tracked purchases describe engagement, but alone cannot show what would have happened without the recommendation.
Define what changes
Name the product-page placement, eligible audience, recommended products and comparison experience. State whether the change adds a block, substitutes an item or moves a placement. Record cases where no recommendation displays.
Shopify Search & Discovery recommendations need a suitable product-page theme section. Its display rules can suppress a product that is already in the visitor's basket. Complementary products must have stock above zero; related products may display with zero stock when continued selling is enabled. Saving a recommendation configuration therefore does not establish that a shopper saw it.
Key Considerations for Measuring Recommendation Effectiveness
- Required theme sectionShopify Search & Discovery recommendations need a suitable product-page theme section
- Stock requirementsComplementary products must have stock above zero; related products may display with zero stock when continued selling is enabled
- Display confirmationSaving a configuration does not guarantee a shopper saw the recommendation
Separate exposure, engagement and outcome
| Measure | Counting rule | Purpose |
|---|---|---|
| Exposure | Eligible sessions in which the block was displayed | Establish opportunity to interact. |
| Engagement | Exposed sessions with a recommendation click | Diagnose selection of the displayed item. |
| Commercial outcome | Qualifying orders, retained sales or defined contribution per assigned eligible session | Compare the overall result. |
Choose a primary commercial outcome before launch. If the aim is add-on sales, inspect purchases of the recommended item and total basket results; a recommendation could shift a purchase from another item. State order-status and adjustment rules for monetary outcomes, and mark recent orders provisional if later changes may affect them.
Shopify's behaviour reports include “Product recommendation conversions over time” and “Product recommendations with low engagement”. Check the definitions and coverage of any report before using it to assess a recommendation.
A tracked purchase rate is not an incremental sales rate. It describes observed activity, not what would have happened without the recommendation.
Make a fair comparison
Where feasible, assign eligible shoppers or sessions before they encounter the placement. Analyse the assigned groups, including people who never click, using the same outcome rule and observation period. Keep assignment consistent for returning shoppers. Record concurrent changes in price, promotion or product availability.
Report group sizes, the outcome in each group, their difference and missing-data limits. If a suitable comparison is unavailable, report displays, clicks and later orders as descriptive evidence and leave the incremental effect unestimated. Base the keep, revise or stop decision on the predefined commercial outcome rather than a click-rate change alone.
