Skip to content

Close Reading of 3.5 and Appendix D: Training Objectives and Dual Guidance ​

After finishing this section, you will be able to write down KiT's training loss and the guidance formula at inference time, and, looking at Table 8, explain which direction each of the two guidance knobs pulls the 64 paths.

3.1 + 3.5: Training Objective ​

Combining the preliminaries from Section 3.1 with Section 3.5, there are only three formulas:

  • (1) Noising: z_t = (1 − t) x₀ + t ε
  • (2) Target velocity: v* = ε − x₀, and the loss is the mean squared error
  • (8) KiT's version: the mean squared error is averaged only over the Lh future positions; history participates in attention but does not enter the loss

t is drawn from a logit-normal distribution: t = σ(N(0, 1)). This is borrowed from the Stable Diffusion 3 paper (Esser et al., 2024), concentrating training on intermediate noise levels. For code details, see the training chapter.

Three Kinds of Condition Dropout: For the Sake of Guidance ​

During training, three kinds of substitutions are made with certain probabilities:

  1. The instrument and industry embeddings are replaced with a shared [UNK] ("unknown")
  2. All identity signals (market, industry, instrument, granularity) are replaced with [NULL]
  3. The entire history segment is zeroed out

This way the same network learns three kinds of predictions: with full conditioning, without identity, and without history. In the official configuration these correspond to inst_dropout, allcond_dropout, and hist_dropout.

Equation (9): Dual Classifier-Free Guidance ​

Classifier-free guidance (CFG) is the most commonly used inference knob for conditional generative models. Equation (10) in Appendix D is the standard form:

v = v∅ + w (v_c − v∅)

w = 1 is just an ordinary conditional model; w > 1 amplifies the difference between "conditioned" and "unconditioned", making samples stick closer to the condition but reducing diversity. Turning up CFG in image generation makes the picture match the prompt better but more monotonous—it's the same principle.

KiT splits it into two independent axes, Equation (9):

v̂ = v_θ + (w_id − 1)(v_θ − v∅id) + (w_hist − 1)(v_θ − v∅hist)

  • Identity guidance w_id: amplifies the effect of "which instrument this is". The instrument's identity carries its "slow personality": typical drift, volatility, and candlestick shape.
  • History guidance w_hist: amplifies the effect of "the recent stretch of movement".

The --cfg-w and --hist-cfg-w in the inference script are exactly these two w's; the code is in kit_v1/infer/sampler.py, and the three predictions are computed together in a single batched forward pass.

Appendix D: Each Knob Pulls in One Direction ​

The paper picked 100 instruments, sampled 64 paths for each from the same initial noise, adjusted only one w at a time (from 1 to 4), and looked at three metrics (smaller means "closer"):

  • Prior gap: how far the path's volatility is from the instrument's long-term volatility
  • History gap: how far the path's average terminal return is from the return of the recent stretch
  • Spread: how dispersed the endpoints of the 64 paths are
Drag itTwo kinds of guidance, each pulling in one directionData: paper Appendix D, Table 8
Distance from the instrument's nature (Prior gap)
0.41
Distance from recent trends (History gap)
0.52
Path dispersion (Spread)
1.08

w = 1 means no guidance; this is the setting used for the paper's main results.

How to read it:

  • w_id from 1 to 4, Prior gap 0.41 → 0.15, History gap stays around 0.5: identity guidance only pulls the paths toward the instrument's true character.
  • w_hist from 1 to 4, History gap 0.52 → 0.19, Prior gap stays around 0.4: history guidance only pulls the paths toward the recent movement.
  • Both make Spread smaller (1.08 → 0.69 or 0.72): the tighter the fit, the less diverse the paths.

So why is the default 1? The paper also compared the IC and RankIC of return prediction: turning up CFG helps for some instruments (especially some high-volatility ones), but the overall average barely changes. So in the main results both w's stay at 1, i.e., guidance is off.

This is also the background for the inference script's help text saying "--cfg-w harms calibration, use with caution": the paths concentrate, the distribution looks more confident, but not necessarily more accurate.

Quiz timeIf you want the predicted volatility to look more like this stock's usual behavior, which should you tune?

Check Yourself ​

  • I can write down KiT's training loss and state which segment it is computed on
  • I know the correspondence between the three kinds of condition dropout and the two guidance knobs
  • Using Table 8, I can explain which direction w_id and w_hist each pull, and why the default for both is 1

Next section: Close Reading of Section 4: How to Read the Experiments.

gaodong.aiAboutPrivacyContact

Independent tutorials, not affiliated with any product mentioned. Product names and trademarks belong to their owners. Check official pages for prices and policies.