Skip to content

Preliminaries: From Autoregression to Flow Matching ​

Sections 1 and 2 of the paper mention a long list of prior work. After finishing this section, you will be able to sort them into a few categories, explain how KiT differs from each one, and not get overwhelmed by names when reading the introduction.

Category 1: Training a separate model for each market ​

Early deep learning approaches: LSTM, GRU, or feeding candlestick charts to convolutional networks as images. The problem is that a separate model must be trained for each market and each forecast horizon, so switching scenarios means starting over.

Later Transformer forecasting models (iTransformer, PatchTST, TFT) are stronger on general benchmarks, and TFT can output quantiles to express uncertainty, but they too were not designed for "one model covering all markets."

Category 2: General time series foundation models ​

Borrowing the idea of large language models: pretrain a model on massive amounts of time series from all kinds of domains, ready to use out of the box. The ones compared in the paper include Chronos, TimesFM, and Sundial.

Their problems are very concrete on candlestick data:

  • They only look at a single curve (usually the closing price), and know nothing about the relationships among open, high, low, and close (the high must be ≥ the open and the close)
  • Many produce only a single forecast (Chronos and TimesFM output only one path in the paper's comparison, K=1), with no distribution over paths

Category 3: Kronos, a finance-specific foundation model ​

Kronos is pretrained on large-scale multi-market candlestick data, making it the closest to what KiT aims to do. The paper points out two problems with it:

  1. Discretization is lossy. It first uses a learned quantizer to turn each candlestick into discrete tokens (like compressing colors down to 256 colors), then processes them like a language model. Quantization loses information, and at decoding time it may produce impossible candlesticks like "high lower than the close."
  2. Autoregression accumulates errors. It generates one candlestick after another, each conditioned on the previously generated ones. If candlestick 1 is slightly off, candlestick 2 is off a bit more on top of that error, drifting further as it goes.
Line them upAn autoregressive model predicts 3 bars — in what order?
  1. 1?
  2. 2?
  3. 3?

Click them one by one, in order.

Category 4: Diffusion-style generative models ​

The idea of diffusion models: during training, noise is gradually added to the data until it becomes pure noise, teaching the model to "denoise"; during generation, start from pure noise and denoise step by step to obtain a sample. Image generation (the Stable Diffusion family) works this way.

Those applied to time series include TimeGrad, CSDI, TSDiff, Diffusion-TS, and NsDiff; flow matching variants that replace the stochastic diffusion process with deterministic ODE paths for faster sampling include FlowTS and TSFlow; in finance there is DiffSTOCK.

The paper's assessment: these methods were all proposed and tested on small-scale, single-domain benchmarks, and none has yet been built as a foundation model for financial markets. That is the gap KiT aims to fill.

Flow matching: the one concept you most need to understand in this lesson ​

KiT uses rectified flow, a form of flow matching. You already saw the formula in the training chapter; here we take a different angle:

  • Imagine two piles of points: one pile is the real "future 20 candlesticks" (the data), the other is standard Gaussian noise.
  • Pair them up arbitrarily (one data point, one noise point), and draw a straight line between them.
  • Train a network: give it any point on the line along with that point's position t, and have it output the direction of the line (noise − data).
  • Once learned, starting from any noise point and walking backward along the direction the network points, you arrive at a point that "looks like real data."

Compared with early diffusion models, the paths are straight lines, so a few dozen steps suffice; compared with autoregression, the entire future segment emerges simultaneously from the noise, so there is no "an earlier candlestick going wrong leads later ones astray."

Quiz timeWhat are the two improvements KiT makes over Kronos, as stated in the paper?

Check yourself ​

  • I can sort the prior work mentioned in the paper into four categories
  • I can explain why autoregression accumulates errors
  • I can use "drawing straight lines between two piles of points" to explain what flow matching is learning

Next section: Close Reading 3.2: One Candlestick Becomes Five Numbers.

gaodong.aiAboutPrivacyContact

Independent tutorials, not affiliated with any product mentioned. Product names and trademarks belong to their owners. Check official pages for prices and policies.