Does your revenue forecast beat a simple baseline?

A detailed revenue forecast should earn the work it takes to maintain. Compare it with a simple alternative using forecasts made before the outcome was known. A model that reproduces last year's results after seeing them has not demonstrated that it would have helped management make an earlier decision.

The comparison needs a clear target: the revenue measure, business coverage and lead time that matter. Forecasting next month's revenue is a different task from forecasting revenue three months ahead, when a planned hire will reach full cost. Test the forecast at the lead time of that commitment, not only where its results look strongest.

Write a short specification. For example: predict monthly net revenue for the existing business, in the reporting currency, three months ahead, using information available at each forecast date. Name the actuals source and how later accounting corrections will be handled.

Keep bookings, invoices and collections out of a recognized-revenue target unless the model explicitly translates them under the company's accounting policy. If the desired prediction is cash receipts, test that measure separately. A forecast can be accurate for one and poor for the other.

Choose a modest comparison method. Carrying forward the latest observed value is one possibility; using the same month a year earlier is another when a seasonal comparison is appropriate and that history exists. These are established benchmark methods in Forecasting: Principles and Practice. Neither is a prediction that the business will remain unchanged. Each gives the more elaborate method a transparent alternative to beat.

For each historical forecast date, assemble the inputs then available. Save their cutoff and version. A CRM export downloaded today may show the final status of a deal that was uncertain three months ago. Feeding that later status into the old forecast gives the model information management did not have.

The same problem can enter through preparation steps. Selecting variables, fitting adjustments or choosing model settings using the later outcomes can contaminate the test even if the final formula only references earlier months. The scikit-learn documentation explains data leakage as the use of information unavailable at prediction time and warns that it produces overly optimistic evaluation.

Distinguish two exercises in the test record. A replay with archived inputs evaluates the historical process. A reconstruction using corrected historical data evaluates a model under a different information assumption. Both can be useful, but they should not share an unlabeled accuracy claim.

If the necessary source snapshots do not exist, say so. Begin saving forecasts and inputs now, while using a clearly limited historical test. Do not invent old pipeline stages to produce a complete-looking evaluation.

Run the candidate and baseline from the same successive dates. At each date, use only the permitted earlier information and produce the same future target. Chronological evaluation avoids training on future observations and testing on the past, the problem addressed by time-series splitting.

For a three-month-ahead test, a June 30 forecast can predict September, a July 31 forecast October, and an August 31 forecast November. Keep these separate from forecasts for the following month. Otherwise the summary can improve simply because it contains more short-lead predictions.

Document data availability, too. If the June accounts were not finalized on June 30, the historical forecast cannot silently receive those final numbers. Use the input that was available, or label a reconstruction that assumes otherwise.

The following numbers are invented solely to demonstrate a three-month-ahead comparison. Revenue is shown in thousands of dollars. The baseline uses the corresponding month of the prior year; the candidate represents a more detailed model. All predictions are assumed to have been saved at the stated date.

Absolute errors for the baseline are 5, 10 and 10, giving a mean absolute error of about $8,333. The candidate's errors are 8, 5 and 5, giving $6,000. Its average absolute error is 28% lower in this tiny illustration. Mean absolute error measures error magnitude; the calculation does not let positive and negative misses cancel. See the metric's official definition.

Direction tells a different story. Using actual minus forecast, the candidate's signed errors are −8, −5 and −5. It overpredicts every observation, by $6,000 on average. The baseline's signed errors are +5, +10 and −10: its smaller average signed miss conceals larger individual errors.

Three observations do not establish a dependable winner. They show why the review needs both magnitude and direction, plus the individual results behind the average. Include periods relevant to the business's decision, such as its peak season, rather than presenting only convenient months.

Inspect consequential misses. Did an overprediction influence an irreversible spending commitment? Did an underestimate leave delivery capacity unprepared? Those are questions for the decision record; the error percentage alone does not establish what the business lost or saved.

Use the same test dates when comparing methods. Mark missing predictions and explain why they are missing rather than quietly dropping difficult periods. Show error by material business segment where a good company total could conceal offsetting mistakes. Percentage measures also need care around zero actual revenue; keep the currency error visible.

After repeatedly changing a model to improve one historical test, evaluate the chosen version on later observations that did not guide those changes. Otherwise the test itself becomes part of the design process. Retain the baseline alongside it so the next review can distinguish sustained improvement from a fortunate period.

Finish with an explicit choice: retain the current method, simplify it, replace it or gather more evidence. Include the maintenance effort and input reliability in that choice. A useful forecast need not win every month, but its added complexity should have a documented purpose and a fair test.