Comparing forecast demand with actual order patterns

Compare a demand forecast with later orders using the version issued before the period began. Match variant, dates, channel and unit, then flag periods when stock may have blocked orders. A close match to constrained orders does not establish an accurate demand forecast.

Fix the forecast and its target

Keep the forecast’s issue date, version, horizon and expected units for each variant and period. State whether its target was customer demand or orders expected under a planned stock position. A forecast revised after orders arrive answers a different question from the plan available when the stock decision was made. Retain manual overrides and their dates.

Choose one actual-order population, such as orders placed or payment-confirmed orders, and apply it consistently. Shopify sales reports include pending, cancelled and unpaid orders. Shopify’s Inventory sold daily by product report tracks the quantity and percentage of inventory sold per day; inventory adjustments are covered in separate reports. Neither report label alone gives a custom completed-order population.

Use the same calendar and time zone for forecasts and actuals. A weekly forecast split into days after the week ended is a retrospective allocation, not a daily forecast issued in advance.

Key Data Points for Forecast Accuracy Assessment

Separate constrained periods

Add confirmed orderable time and blocked or uncertain intervals to each variant-period. Shopify’s available quantity excludes committed, unavailable and incoming units, but zero tracked stock does not always block orders: continued selling can be enabled. Check the selling rule and fulfilment route before classifying a period.

Show the forecast–order gap for every period, with its stock context. During a blocked period, that gap combines forecast uncertainty and any effect of unavailability. It is not a measured count of customers who wanted the missing units. Keep constrained and uncertain periods separate from an error summary intended to assess a demand forecast against order observations.

Stock Availability and Order Constraints Over Time

Read comparable errors

For a reasonably unconstrained variant-period, define signed error = actual qualifying units − forecast units. A positive value means orders exceeded the forecast; a negative value means they fell short. Mean absolute error averages the absolute differences and remains in units. Percentage errors are undefined when actual units are zero and can be unstable near zero.

Even in an orderable period, orders are only an observed proxy for demand. Traffic, price, promotion and assortment can change. Report how many variant-periods entered the error summary and how many were excluded or labelled uncertain. If none were sufficiently unconstrained, do not publish a demand-accuracy figure from order data alone.

Inspect the error pattern by variant and period before acting. Positive and negative errors can cancel in a category total while one size is repeatedly under-forecast. For a method comparison, issue each candidate forecast using only information available at its forecast date and assess it against later periods. Rolling-origin evaluation repeats that sequence through time.

Use the review to revise a documented ordering assumption, improve stock-history capture or wait for comparable observations.

Forecast Demand vs Actual Orders: Key Metrics by Period