Backtests, Conclusions, and Questions to Ask When Reading Papers
After finishing this section, you will have read the entire paper. You will be able to evaluate its backtests, know what the authors say the next steps are, and take away a checklist of questions you can use when reading similar papers in the future.
4.3 Backtests: How They Are Designed
- Market and time: A-shares, one year (from year 2025, month 5 to year 2026, month 4)
- Signal: rank instruments by predicted endpoint return
- Portfolio: long only, buy the top N ranked
- How parameters are set: the number of holdings N and rebalancing frequency are chosen on the validation period and then frozen, not tuned on the test period
- Limit-up buys are skipped: buy orders that would close at the limit-up price are skipped
- Costs: 12.5 basis points for buys, 22.5 basis points for sells (commission 2.5, stamp duty 10, slippage 10 per side)
Results
After deducting costs, the net annualized return on the test period:
| Granularity | Net annualized return | Sharpe ratio |
|---|---|---|
| 5-minute bars | +65.2% | 1.70 |
| 2-hour bars | +64.4% | 2.46 |
In Figure 4, KiT's net value curve ultimately ends above the equal-weighted portfolio of the same stock pool, above the median of buy-and-hold for individual stocks, and also outperforms the CSI 300 and CSI 500.
Sharpe ratio means "how much excess return you get per unit of volatility taken on." With the same annualized return, the 2-hour bars have a higher Sharpe, indicating a smoother net value curve.
How to View the Backtests
The term the paper itself uses in its conclusions is "theoretical backtests," to show that the predictions "have economic significance." When reading, you can compare against the rules of the open-source backtest script from the Evaluation and Backtesting lesson and ask a few questions:
- Liquidity and market impact: the open-source script explicitly does not consider the constraint of 100-share lots and the impact of large orders. With a large amount of capital, you may not be able to buy that much of the small-cap stocks at the top of the ranking.
- Cost assumptions: is slippage of 10 basis points per side enough under high-frequency rebalancing (5-minute bars)?
- Is one year long enough: was this year's market environment unusual?
- Stock pool: which instruments are selected into the pool itself affects the results.
These points don't mean the backtests are wrong; they are boundaries that any backtest has. The paper's main evidence is the RankIC in Table 1 and Table 2; the backtests serve as supporting illustration.
Section 5: Conclusions and Next Steps
The authors summarize three points: a flow-matching diffusion Transformer that models K-line paths, a five-dimensional representation that strictly preserves K-line geometry, and conditional injection designed according to the nature of the signal.
Two directions proposed:
- Higher-dimensional inputs: since no discrete tokenizer is needed, KiT can directly incorporate data of more dimensions; the examples given in the paper are the Level-3 order book and alpha factors.
- Fine-tuning for downstream tasks: as a foundation model, fine-tuning for specific tasks is a direction worth exploring. You already tried the simplest form of this in the Fine-tuning section.
A General Checklist of Questions
When reading any "forecasting-type" paper, you can ask these questions. Let's practice with KiT:
Look it upquestions
Click one to see.
You've Finished Reading
Look back at the table from the first section: three challenges, three answers — for each cell you can now state the formula, code location, and ablation numbers.
Quiz timeIn the KiT paper, which table is the most direct evidence that ”the five-dimensional representation is useful”?
Check Yourself
- I can describe the backtest design and the two sets of annualized return and Sharpe numbers
- I can list at least three real-world factors the backtests did not cover
- I went through KiT using the checklist of questions
Return to the course homepage, or go to Errors and FAQ.