Model calibration
When the model says 70%, does it happen 70% of the time? Every graded pick is grouped by its predicted probability and compared with how often that group was actually right.
Reliability diagram
Each point is one probability band. Horizontal position: the average probability the model gave. Vertical position: the share of those picks that were correct. Points on the dashed diagonal are perfectly calibrated; above it the model was too cautious, below it too confident. Higher bands beat lower ones in 7 of 7 steps.
Observed hit rate (point size = number of picks) Perfect calibration
| Band | Picks | Predicted | Observed | Gap |
|---|---|---|---|---|
| Under 55% | 2050 | 52% | 52% | +0 |
| 55–59% | 3160 | 57% | 53% | -4 |
| 60–64% | 6067 | 62% | 58% | -4 |
| 65–69% | 5154 | 67% | 65% | -2 |
| 70–74% | 2812 | 72% | 73% | +1 |
| 75–79% | 2689 | 77% | 76% | -1 |
| 80–84% | 1980 | 82% | 77% | -5 |
| 85+% | 961 | 86% | 79% | -7 |
Calibration by market
| Market | Picks | Predicted | Observed |
|---|---|---|---|
| Both teams to score: yes | 7607 | 61% | 57% |
| Match result (1X2) | 5912 | 65% | 62% |
| Double chance | 3102 | 83% | 77% |
| Over 2.5 goals | 2308 | 65% | 68% |
| Over 1.5 goals | 2157 | 75% | 75% |
| Under 3.5 goals | 1890 | 72% | 72% |
| Under 2.5 goals | 1269 | 61% | 60% |
| Both teams to score: no | 595 | 53% | 56% |
| Over 3.5 goals | 31 | 64% | 55% |
Weekly drift
Why we publish this
Hit rate alone can flatter any tipster: back only short favourites and you will be right most of the time while losing money. Calibration asks a harder question – are the probabilities themselves honest? If the 60% band lands 60% of the time and the 80% band 80%, the numbers can be trusted when you compare them with bookmaker odds.
Single weeks wobble; the 90-day view is the one to judge. Profit at level stakes, market by market, is on the leaderboard, every graded pick on the track record, and the method in our methodology.
Model calibration: do our probabilities mean what they say?
Calibration is the most demanding test a prediction model can face. It asks whether the probabilities are honest: when xGaura says 60%, does the outcome happen about 60 times in every 100? This page groups thousands of graded forecasts by the probability we gave them and plots that against how often they actually landed.
How the chart works
Forecasts are sorted into bands – 50–55%, 55–60%, 60–65% and so on. For each band, the chart shows the average predicted probability on one axis and the observed win rate on the other. A perfectly calibrated model produces points along the diagonal line. Points above the line mean the model was too cautious; points below it mean it was overconfident.
Why calibration matters more than accuracy
A model that only picks heavy favourites can boast a high hit rate while being badly calibrated – it might call every pick 90% when only 75% land. Calibration exposes that. It also matters for betting: expected value depends on the probability being right. If our 60% picks really land 54% of the time, apparent value at odds of 1.85 would be an illusion.
The Brier score
Alongside the chart, the page reports a Brier score – the average squared difference between each forecast and the outcome (1 if it happened, 0 if not). Lower is better. A forecaster who always says 50% scores 0.25; a perfect forecaster scores 0. Good football models usually land somewhere between 0.18 and 0.22 for binary markets, because football outcomes are genuinely uncertain.
Reading small bands carefully
Bands with few forecasts are noisy. If only thirty picks fell in the 85–90% band, an observed rate of 80% could easily be chance. The page shows how many forecasts sit in each band, and you should give more weight to bands with hundreds of entries.
What we do with the results
Calibration feeds back into the model. If one band consistently runs below the diagonal, it suggests the ratings or the market blend are pushing probabilities too far in one direction, and the weights can be adjusted. Because the data are public, you can watch that process over time rather than taking our word for it.
Calibration and your own betting
Even with a well-calibrated model, short-term results will wander. A 70% band will have weeks at 60% and weeks at 80%. Calibration is a long-run property, which is exactly why it is useful: it tells you whether to trust the numbers over the next few hundred bets, not the next few. Read alongside the track record and the methodology. Over 18s only.
A worked example
Suppose 412 forecasts fell in the 65–70% band, averaging 67.3%. If 272 of them won, the observed rate is 66.0% – just below the diagonal and well within normal variation for a sample that size. If only 240 had won (58.3%), the gap would be large enough to suggest overconfidence in that band and a reason to revisit the weights.
Frequently asked questions
What does calibration mean?
A model is calibrated when its probabilities match reality: of all the picks it rates at 70%, about 70% should be correct. Calibration is the fairest single test of a prediction model.
Is xGaura calibrated?
Across 24873 graded picks in the last 90 days, the mean predicted probability was 67% and the observed hit rate 64% – a difference of -3 points. The band-by-band table shows where the model runs hot or cold.
Why can a band look off?
Small samples. A band with a dozen picks can miss its target by 15 points through luck alone. Trust the bands with hundreds of picks most.
How are picks graded?
On the 90-minute score. Extra time and penalties are excluded; postponed or abandoned games are void and left out of the test.