Chronos: What Happens When You Turn Time Series into Words?


Could a language model trained to predict the next word also learn to predict what happens next in a sequence of numbers? It sounds like an unusual idea. Language models are designed to understand patterns in text, while time series forecasting involves predicting future values from historical data. Yet both tasks share something fundamental: using what came before to make an informed prediction about what comes next.
In Chronos: Learning the Language of Time Series, researchers at AWS AI Labs and Amazon Supply Chain explore whether the same approach used to train language models can be applied to numerical data. Rather than building a completely new forecasting architecture, they convert numbers into tokens and train an existing language model to predict the next one.
The results are promising, particularly when forecasting data similar to what the model encountered during training. However, the approach also raises some important questions about how numerical information is represented, what the model can learn and where its predictions begin to break down. In this Passion Academy, Dr Fabio Rodriguez takes a closer look at the research, examining both its contributions and the assumptions behind it.
Time series data appears in almost every industry. Retailers use historical sales to forecast demand, energy companies predict electricity consumption and financial institutions analyse market movements. Traditionally, forecasting has relied on statistical methods or specialised machine learning models designed specifically for numerical sequences. These approaches can work very well but they often require models to be trained or adapted for particular datasets.
The researchers behind Chronos asked a different question: could a general purpose language model learn to forecast time series by treating numerical values in the same way that a language model treats words?
The underlying idea is relatively simple. A language model learns patterns in sequences of tokens and predicts which token is likely to come next. Chronos applies this principle to numerical data by converting each observation into a discrete token. Instead of predicting the next word in a sentence, the model predicts the next numerical value in a sequence. This means the researchers can use an existing language model architecture, without making changes to its fundamental structure.
The most important part of Chronos is how it converts numerical values into tokens. This process involves two stages: scaling and quantisation.
Time series can have very different numerical ranges. One business might record daily sales of around 20 units, while another records thousands. To make these series easier to compare, Chronos divides every value by the absolute mean of the historical sequence.
This preserves the overall shape of the data while bringing different numerical scales closer together. For example, two businesses with very different sales volumes could end up with the same normalised sequence if their sales follow the same pattern.
Once the values have been scaled, Chronos maps them to one of 4,094 numerical bins, covering a range from minus 15 to plus 15. Each bin represents a possible numerical value. The model then treats these bins as tokens, much like the words in a language model's vocabulary.
For example, a sequence of daily sales might be converted into a series of tokens representing the values 10, 20, 30, 40, 30, 10 and 0. Rather than processing the original numbers directly, the model learns relationships between these tokens and uses them to predict what comes next.
One important consequence is that Chronos discards the actual timestamps. It keeps the order of the observations but not the dates or the time intervals between them. A sequence of daily values and a sequence of hourly values are therefore represented in the same way if their numerical patterns are identical. This makes the approach flexible but it also means the model has no direct understanding of weekdays, holidays, seasonal calendars or the frequency of the data.
With the numerical data converted into tokens, the researchers can train a standard language model to predict the next token in a sequence.
Chronos uses the T5 architecture, an encoder and decoder model originally developed for natural language processing. The researchers trained four different model sizes, ranging from 20 million to 710 million parameters.
During training, the model receives a sequence of historical values and learns to predict what comes next. It uses the same general next token prediction objective found in language modelling.
However, forecasting is not just about producing one likely number. There is often uncertainty about what might happen next and a useful forecasting system should be able to represent that uncertainty. Chronos addresses this by sampling multiple possible futures. For each prediction, it generates 20 possible continuations, converts the tokens back into numerical values and uses the resulting forecasts to produce a distribution of possible outcomes.
Rather than returning just one answer, it can provide a range of predictions. This is particularly useful in situations where businesses need to understand not only what is likely to happen, but also how much uncertainty surrounds that prediction.
A major part of the research is how the model is trained. The researchers assembled 28 public time series datasets, containing approximately 84 billion observations. These covered a range of domains and different types of numerical patterns. They also introduced two data augmentation techniques to expand the training data.
TSMixup takes between one and three real time series and combines them using randomly selected weights.
For example, a new sequence might contain 60% of one series, 30% of another and 10% of a third. The aim is to expose the model to more combinations of existing patterns. Using this technique, the researchers expanded approximately 890,000 original series into around 10 million mixed series. However, this method has an important limitation: it can combine patterns that already exist in the data but it cannot introduce entirely new types of behaviour.
The second technique creates artificial time series by combining simple mathematical patterns such as trends, seasonality and noise. Unlike TSMixup, which blends real data, KernelSynth generates new sequences from these underlying patterns.
The researchers found that including around 10% synthetic data improved forecasting performance in their zero shot experiments. Adding more than this, however, made performance worse.
There is also a limitation. The synthetic series tend to be smooth and Gaussian in nature. They do not naturally capture sudden jumps, sharp changes in direction or unusual events such as a financial crash. This becomes important when considering how well Chronos can handle situations that are very different from its training data.
The researchers evaluated Chronos across 42 datasets, comparing it against 18 existing forecasting baselines. They split the evaluation into two main categories.
The first benchmark tested Chronos on 15 datasets that were included in its training data.
Here, the larger Chronos models performed strongly. Chronos T5 Large achieved a normalised forecasting error of 0.564, outperforming several established forecasting approaches in this benchmark. This is an encouraging result, although it is worth remembering that these datasets were part of the model's training process.
The second benchmark tested the model on 27 datasets it had never encountered during training.
This is a more challenging test because the model must apply what it has learned to entirely new data, without being specifically trained on those datasets.
Chronos T5 Large achieved a score of 0.645. This was competitive with several of the other models but it did not establish a clear lead. For comparison, N HiTS scored 0.672 and PatchTST scored 0.684.
The researchers also found that fine tuning Chronos T5 Small on the individual datasets improved its performance, bringing the score down to 0.597. The scoring metric is based on forecasting error relative to a seasonal naive baseline, where lower scores indicate better performance. A score of 1 means the model performs no better than simply repeating the previous season's values.
The distinction between these two benchmarks matters. Chronos demonstrates that a single model can learn useful patterns across different datasets. However, its zero shot performance does not mean it can consistently outperform models trained specifically for each forecasting task.
One of the most interesting parts of the paper is not simply the overall benchmark results but the experiments investigating which design choices actually matter.
The researchers tested different model sizes, training approaches, data augmentation techniques and context lengths. A few findings stand out:
Increasing the model size from 20 million to 710 million parameters improved zero shot forecasting performance. This suggests that the model benefits from having more capacity to learn patterns across different time series.
Despite the central idea of treating numbers as words, the researchers found that initialising the model with pre trained language model weights did not provide a clear advantage. Training the models from scratch performed as well as, or better than, starting with existing language model weights.
This is an interesting distinction. Chronos borrows the architecture and training approach of language models but that does not mean it needs to inherit their previously learned linguistic knowledge.
The researchers also tested how much historical data the model should receive before making a prediction. Increasing the context length helped performance up to around 1,024 observations. Beyond that, there was no clear improvement in the experiments.
The KernelSynth experiments showed that synthetic data can improve forecasting performance but only up to a point. Around 10% of the training data produced the best results, while larger proportions made performance worse.
These experiments show that the quality and composition of training data can be just as important as the size of the model itself.
Despite the promising results, the paper also highlights some important limitations. These are particularly relevant when considering whether Chronos could be used in real world forecasting systems.
Chronos is trained largely on historical patterns, with synthetic data designed around smooth trends, seasonality and noise. This makes it less naturally suited to situations involving sudden changes, such as a financial crash, an unexpected supply chain disruption or a rapid shift in consumer demand.
The model can only learn behaviours that are sufficiently represented in its training data. Combining existing patterns does not necessarily teach it how to respond to entirely new events.
The quantisation process uses a fixed range between minus 15 and plus 15 after scaling. Any value outside this range is clipped to the nearest boundary. This means unusually large spikes can be underestimated or lost entirely.
For example, imagine a business that normally sells one unit per day but suddenly sells 100. After scaling, that spike could exceed the maximum value represented by the bins. Chronos would therefore be unable to represent the full size of the event.
The scaling method also creates a resolution problem. As the bins have equal widths, a dataset with a much higher average value can lose precision when smaller fluctuations are converted into tokens.
Chronos receives numerical values in sequence but it does not directly receive information about when those values occurred. It does not know whether a value represents a Monday, a weekend or a public holiday. Nor does it receive additional variables such as weather, pricing or promotional activity.
For many forecasting problems, these factors are essential. A retailer predicting demand, for example, may need to account for seasonal events and promotions that cannot be understood from historical sales values alone.
The model is trained to predict the correct token using a classification objective called cross entropy. However, the numerical bins have an order, while the loss function does not directly account for the distance between them. Predicting a value one bin away from the correct answer is penalised in the same way as predicting a value thousands of bins away. This is a slightly unusual choice for a numerical forecasting task, where the difference between two predictions is clearly meaningful.
Chronos is an interesting example of how ideas from one area of machine learning can be applied to another.
Its central contribution is not necessarily that language models are better at forecasting than specialised models. Instead, it demonstrates that a relatively simple tokenisation approach can allow a standard language model architecture to learn useful patterns from time series data.
The research also provides a valuable benchmark, with open weights, code and data that other researchers can use to investigate the approach further. However, there is an important distinction between a model that can produce plausible forecasts and one that can be trusted in a real operational environment.
For businesses, forecasting often involves more than recognising patterns in historical numbers. It requires understanding the context behind those numbers, accounting for external factors and being able to respond when conditions change.
Chronos offers a promising foundation for general purpose forecasting, but its limitations around time, numerical precision and unusual events show why the design of the data representation matters so much.
One direction explored by Dr Fabio Rodríguez is applying Chronos to stock market data.
Financial markets provide a particularly interesting test because they contain many of the behaviours that the original approach struggles with: sudden movements, regime changes and values that can vary significantly over time.
The proposed experiment would compare zero shot and fine tuned Chronos models against simpler forecasting approaches, including the assumption that tomorrow's price will be the same as today's. It would also examine whether the model's predicted ranges genuinely capture uncertainty, particularly during periods of market disruption.
This is not part of the original Chronos paper but a proposed direction for further experimentation. It raises a useful question: can a model trained on general time series data learn enough about financial markets to make reliable predictions, or will its limitations become more apparent when faced with genuinely unusual events?
The answer could tell us a lot about the strengths and weaknesses of general purpose forecasting models.