NEXTMarketReaction

Methodology

How the analysis is built and tested

Where the data comes from, how the pre-release and release-update analyses work, how every model was tested, and the results — including where we found no advantage.

The results in brief

  • Pre-release (Forecast vs Previous): tested on 14–148 past releases per indicator — no indicator or horizon validated.
  • U.S. release updates: 0 of 11 models validated at a realistic decision time (30 s after the release).
  • Canada's Labour Force Survey (EUR/CAD): validated at 3m, 5m, 8m, 15m (26 out-of-sample releases).
  • In the current experimental phase the site also shows directions from models that are not validated at the decision time; they are labelled as estimates, and their records are in the table below.

Data sources

  • Release values (Actual, Forecast, Previous) come from economic-calendar data providers; each release page names the source and the time the Actual was received. The official statistics offices publish the figures themselves.
  • Historical research uses tick-level EUR/USD (and EUR/CAD) prices for past releases from 2022-01 to 2026-03. These research prices are not shown on the site.
  • Live analyses use only the Actual, the Forecast and the historical model — never the market move after the release. Experimental directions are evaluated after 15 minutes against public market trade data.

Forecast, Previous and Actual

Forecast is the economists' consensus published before the release; Previous is the last reading and may be revised; Actual is published at the release time. The models use the surprise — Actual minus Forecast. Previous is shown for context only.

Stage 1 — Pre-release

Before the Actual exists, the only information is Forecast versus Previous. We tested whether that difference predicted the EUR/USD move from the last price before the release.

No indicator and no horizon passed the validation bar (14–148 releases per indicator; Brier scores around the 0.25 coin-flip baseline). The site therefore shows "No clear directional advantage" before every release.

Stage 2 — Release update

When the Actual is published, a parameter-free rule reads the direction of the surprise for that indicator, and probabilities come from how EUR/USD moved after strictly earlier releases (walk-forward, 15 folds). Each indicator is tested on its own; simultaneous releases are never counted twice.

Results are measured from 30 seconds after the official release — the earliest moment an update can realistically reach you (the Actual has reached us 7.7–23 s after the release in production).

When is a horizon "validated"?

The test plan was registered before any result was computed. A horizon counts as validated only when it has at least 20 out-of-sample releases, a one-sided binomial test that stays below p = 0.05 after Holm correction across that indicator's horizons, and a Brier score below 0.25.

Out-of-sample results by indicator

Share of releases in which the rule's direction was right, measured from 30 s after the release. n = number of out-of-sample releases for that horizon.

Out-of-sample results by indicator
IndicatorOut-of-sample releases1m3m5m8m15mValidated
Initial Jobless Claims9552% n=9456% n=9554% n=9452% n=9556% n=95None
ADP Non-Farm Employment Change2441% n=2238% n=2433% n=2450% n=2446% n=24None
ISM Manufacturing PMI2357% n=2348% n=2335% n=2348% n=2348% n=23None
ISM Services PMI2365% n=2370% n=2361% n=2365% n=2361% n=23None
Non-Farm Payrolls (NFP)2352% n=2352% n=2361% n=2361% n=2357% n=23None
Producer Price Index (PPI)2245% n=2227% n=2236% n=2236% n=2232% n=22None
Retail Sales2259% n=2250% n=2241% n=2250% n=2255% n=22None
Unemployment Rate2152% n=2143% n=2157% n=2157% n=2138% n=21None
Consumer Price Index (CPI)1377% n=1385% n=1362% n=1369% n=1385% n=13None
Core CPI1136% n=1155% n=1136% n=1155% n=1155% n=11None
Gross Domestic Product (GDP)90% n=933% n=933% n=922% n=922% n=9None
Canada Labour Force Survey26–81% n=2685% n=2685% n=2677% n=263m, 5m, 8m, 15m

Small samples (for example n = 9–13) can show high or low percentages by chance — that is why the validation bar requires n ≥ 20 and a significance test. A percentage alone is not evidence of an edge.

When several modelled releases came out at the same moment, their directions conflicted at 32 of 78 such moments; the site then shows "Mixed data" or "No clear direction".

Probability and reliability

A score such as "DOWN 70%" is the historical share of comparable past releases in which EUR/USD moved that way. It describes the past, not a promise: any single release can go the other way. How much to trust a score depends on the out-of-sample record above, not on the size of the percentage.

When no direction is shown

  • Before the Actual is published (Stage 1 for every release).
  • When the Actual equals the Forecast, or a value is missing.
  • When a report's headline figure is in line with its forecast and only secondary components differ.
  • When releases published at the same moment point in opposite directions.
  • When the experimental phase is switched off, every release without a validated horizon shows "No clear direction".
  • For events without a model, such as central bank decisions and speeches: they are marked as volatility events.

Limitations

  • Most indicators have only 9–24 out-of-sample releases per horizon in the test; Jobless Claims has about 95.
  • Market behaviour changes; relationships found in 2022–2026 can weaken or stop working.
  • For U.S. releases the market usually prices the surprise within seconds, before an update can reach you.
  • Economic data can be delayed or revised; an Actual is shown only when it can be verified.
  • This is statistical information, not financial advice or a personal recommendation. Trading involves risk.

How to read a release page All guides

Validation results generated on 2026-10-13 from the pre-registered study; the page updates when the study is re-run.

Methodology and validation results | NEXTMarketReaction