Week Three — Foundation model vs Brownian motion. Kronos on five-minute BTC.

Recent testing shows that the foundation model Kronos does not outperform the traditional Brownian motion model in predicting short-term Bitcoin price movements in out-of-sample data, challenging assumptions about the superiority of modern learned models in trading applications.

Over the past week, researchers tested Kronos, an open-source foundation model trained on millions of candlestick data from global exchanges, against a geometric Brownian motion baseline in predicting whether Bitcoin would close above its open price over five-minute intervals. The test involved reconstructing market conditions from historical trade logs and running multiple forecast paths to estimate the probability of upward movement.

Results indicated that, across 497 trades, the Brownian motion model achieved a lower Brier score (0.193) and log-loss (0.567) than Kronos (0.213 and 1.080, respectively). In practical terms, Brownian motion’s predictions were more accurate and less overconfident. When tested on the last 249 trades, which Kronos had not seen during training, the performance difference was statistically insignificant, with a Brier score difference of only 0.0011, well within random variation.

Why It Matters

This finding suggests that, despite the sophistication of modern foundation models like Kronos, traditional mathematical models such as Brownian motion remain competitive or superior in short-term, out-of-sample trading predictions. This has implications for algorithmic trading strategies that rely on complex models, highlighting the importance of rigorous validation and the potential limitations of learned models in volatile markets.

Background

Previous two-week testing of a variety of strategies revealed that most ‘edges’ in short-term crypto trading were mechanical artifacts that did not persist over time. The experiment aimed to determine whether a learned foundation model could provide a genuine advantage over the classical Brownian baseline, which has been a staple in financial modeling since the early 20th century. Kronos, developed by a research team and trained on extensive market data, was chosen for its credibility and open-source availability.

“Our tests show that Kronos does not outperform Brownian motion in out-of-sample predictions, raising questions about the practical benefits of modern foundation models for short-term trading.”

— Thorsten Meyer, researcher

“Kronos is designed as a research tool, not a trading system, and its performance in this context highlights the challenges of applying large models directly to live trading without further refinement.”

— Kronos model developers

What Remains Unclear

It remains unclear whether further tuning, larger models, or different training data could improve Kronos’s out-of-sample performance. Additionally, the potential benefits of Kronos in longer-term or different market conditions are still untested and uncertain.

What’s Next

Next steps include exploring larger or more specialized models, testing in live trading environments, and analyzing whether integrating Kronos with other signals could yield better results. Further research is needed to determine if model improvements can translate into practical trading advantages.

Key Questions

Why did Kronos not outperform Brownian motion in these tests?

The tests suggest that, at least in the short-term and out-of-sample context, Kronos’s predictions did not significantly improve over the simple, classical Brownian baseline. This may be due to the inherent unpredictability of short-term crypto price movements or the current limitations of the model’s training and architecture.

Can foundation models like Kronos be improved for trading?

Potentially, yes. Further tuning, larger datasets, or combining models with other signals might enhance performance. However, as of now, the evidence indicates they do not yet outperform traditional models in out-of-sample short-term predictions.

Does this mean traditional models are better for crypto trading?

Not necessarily, but this testing shows that classical models like Brownian motion remain competitive, especially in out-of-sample scenarios. More research is needed to evaluate the full potential of modern learned models in different trading contexts.

What are the implications for traders using AI models?

Traders should be cautious about overestimating the predictive power of complex models. Rigorous backtesting and out-of-sample validation are essential to avoid false confidence in model predictions.

Source: Thorsten Meyer AI