01 · Quant Toolchain
For individual programmers / traders who want to get into quant: first assemble a toolchain that is "good enough, no fuss". This article does not chase an engineering team's arms race — it just wants you running the research loop on your own machine at the lowest possible cost.
Disclaimer: All content on this site is for learning and research only and does not constitute investment advice. Markets are risky; invest with caution.
1. Language Choice: Why Python Is Primary
The core personal quant workflow is write strategy → fetch data → run backtest → read results, and almost none of those steps depends on extreme performance. Python's dominance in the quant ecosystem comes from:
| Reason | Explanation |
|---|---|
| Complete ecosystem | Ready-made libraries for data (AKShare/Tushare/CCXT), computation (pandas/numpy), backtesting (backtrader/vectorbt), performance analysis (pyfolio) |
| Fast iteration | From strategy idea to verification usually takes a few dozen lines; changing parameters and viewing curves is second-level feedback |
| Abundant learning material | Almost all quant tutorials, communities, and open-source strategies are Python |
| Team/career alignment | Python is the mainstream language for quant roles in China; what you learn transfers directly |
When is Python not enough? When the bottleneck is no longer "how fast you write code" but "how fast it runs, how low the latency is":
- Data volume in the tens of millions of rows or more, single factor computations taking minutes;
- Millisecond-level execution needed (high-frequency T+0, market making);
- Backtests sweeping hundreds of thousands of parameter combinations, or requiring multiprocess/multi-machine parallelism.
Only then consider rewriting hot modules (e.g., factor computation, matching simulation) in C++ or Rust — not rewriting everything.
1.1 When Do You Need C++ / Rust
| Scenario | Language | Notes |
|---|---|---|
| High-frequency execution | C++ | Exchange gateway APIs (e.g., CTP) are natively C++ DLLs; market making/HFT must stay close to the metal |
| Heavy compute engines | C++ / Rust | Self-built backtest matching engines, large factor libraries; Rust is safer with no GC pauses |
| Regular personal quant | Not needed | Daily/hourly strategies + data at the ten-thousand-row scale: Python has zero performance pressure |
💡 Language Choice Verdict
Do not start personal quant with C++. Wait until you have "a concrete problem Python cannot solve", then rewrite the hot spots — don't use C++ for C++'s sake.
2. Core Library Checklist
| Library | Purpose | Role in Personal Quant |
|---|---|---|
| numpy | Numeric computing, array ops | The foundation of all computation; pandas' engine underneath |
| pandas | Tabular data, time series | The standard container for market data; the main backtest battlefield |
| matplotlib | Plotting | Equity curves, drawdowns, indicator charts |
| TA-Lib | 150+ technical indicators | MACD/RSI/Bollinger etc. in one line (pip install TA-Lib requires the C library first) |
| requests | HTTP requests | Fetching REST market data, order endpoints |
| websockets / websocket-client | WebSocket | Real-time market data subscriptions |
| backtrader | Event-driven backtest framework | A mature option for complex backtests (see also 03-Your First Backtest) |
| vn.py | Trading + backtest framework | The heavyweight option when you want both research and live connectivity |
| pyfolio | Performance analysis | Sharpe ratio/drawdown/monthly heatmaps from return series in one call |
| statsmodels / scipy | Statistical tests | Cointegration tests, regression — must-haves for statistical arbitrage |
| sqlite3 (stdlib) | Local storage | Zero-config database at personal data volumes |
Minimal install (example code is for teaching only; live-trading risk is yours):
# Use venv or uv to manage environments; do not install into system Python
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install pandas numpy matplotlib \
ta-lib backtrader pyfolio \
requests websocket-client \
akshare tushare ccxt⚠️ Avoiding ta-lib Install Pitfalls
On macOS, ta-lib requires brew install ta-lib first; on Windows prefer conda install -c conda-forge ta-lib from Anaconda — otherwise compilation often fails.
3. Development Environment
3.1 Environment Management: Anaconda vs venv
| Option | Best For | Caveats |
|---|---|---|
| venv + pip | Keep the system clean; good enough | Lowest learning cost; recommended here |
| uv | pip-like simplicity + very fast installs | Modern alternative; uv venv creates an environment in one line |
| Anaconda | Avoiding compile hassles (TA-Lib fails often) | Large footprint, environment chaos risk; not ideal for production scripts |
💡 One Rule for Environments
One virtual environment per project, dependencies pinned in requirements.txt or pyproject.toml, so any machine can reproduce your setup.
3.2 Editor: VS Code First
- Must-have extensions: Python, Jupyter (notebook support), Pylance.
- Open the project root; switch the interpreter to the project
.venvfrom the bottom-right corner. - If backtest performance becomes a bottleneck, profile with
%timeitor a profiler before reaching for another language.
3.3 Jupyter vs Scripts: Which to Use When
| Work Mode | Use Jupyter | Use .py Scripts |
|---|---|---|
| Exploring data | ✅ See tables as you go | |
| Trying indicators/plots | ✅ Instant charts | |
| Strategy drafts | ✅ Edit one line, rerun one cell | |
| Scheduled jobs/incremental updates | ✅ Call directly via cron / Task Scheduler | |
| Backtests/parameter sweeps (long-running) | ✅ No UI dependency; runs in background | |
| Deploying to servers | ✅ Scripts are the only option |
💡 Recommended Workflow
Think it through in Jupyter → consolidate into .py modules → cron runs the data scripts. Notebooks are scratch paper; scripts are the product. Never let a notebook become your production environment.
4. Data Source Comparison
| Data Source | Type | Coverage | Main Limitations |
|---|---|---|---|
| AKShare | Free/open source | China A-shares, futures, macro, funds, some US stocks/crypto | Interfaces and fields shift with upstream websites; watch updates; no commercial guarantees |
| Tushare Pro | Free (credit-based) | Mostly China A-shares (market/financial data) | Advanced endpoints need credits earned via registration age/donations/tasks |
| Binance API | Free | Crypto spot/futures market data and trading | Rate limits by weight; geographic and compliance restrictions |
| CCXT | Free/open source | Unified interface to hundreds of crypto exchanges | Unified abstraction sacrifices platform-specific features; some endpoints need a proxy |
| Wind | Paid | Full coverage: A-shares/HK/US/futures/macro | Expensive (unrealistic for individuals); data quality and support come standard |
| JoinQuant | Paid / free within platform | A-shares/futures/funds | Convenient inside the platform; local export restricted |
| RiceQuant | Paid | A-shares/futures/HK-US stocks | Same as above; tied to their quant platform |
Recommendations for the individual stage:
- A-share daily research: start with AKShare → move to Tushare Pro when you need stable financial data or cleaner fields.
- Futures: AKShare (Sina/Eastmoney sources) is enough for dominant-contract dailies.
- Crypto: use exchange official REST/WebSocket directly (Binance/OKX); skip third parties.
- Never depend on a single source: cross-check core data between two sources (see the data quality checks in 02-Data Acquisition).
5. Suggested Research Directory Layout
quant-lab/ # one directory per strategy project
├── .venv/ # virtual environment (not committed to git)
├── data/ # raw data (never edited after download; incremental append only)
│ ├── raw/ # raw market data
│ └── processed/ # cleaned research data
├── strategy/ # strategy code (one module per strategy)
│ ├── dual_ma.py
│ ├── boll_revert.py
│ └── common.py # indicators, utilities
├── backtest/ # backtest scripts and results
│ ├── run_dual_ma.py
│ └── results/ # exported equity/details per run
├── research/ # Jupyter drafts, exploration notes
├── scripts/ # scheduled jobs like incremental updates
├── config.yaml # parameters and data source config
└── requirements.txtdata/is a read-only asset: scripts only append increments; never rewrite history.strategy/is your code asset: strategies are only a few hundred lines; the value accrues here.- Export one result file per backtest into
backtest/results/, named with parameters and timestamp (e.g.,dual_ma_f5_s20_20260816.csv).
6. Version Control and Reproducibility
6.1 What git Should Track
| Track | Don't Track |
|---|---|
| Strategy code, data scripts | data/ (raw data stays out of git; refetchable by script) |
requirements.txt | .venv/, Jupyter outputs, intermediate backtest artifacts |
Template of config.yaml (real tokens live in env vars) | Any keys/tokens (see key security in 04-Live Automation) |
git init && git add strategy scripts requirements.txt
git commit -m "feat: initial dual MA strategy, params f5/s20"Commit after every strategy change, with message explaining what and why — three days later, reading your history beats rereading the code.
6.2 Archiving Experiment Results
Backtest reproducibility rests on three aligned things: code version (git commit) + parameters + data date range. After each backtest, append a row to an experiment log:
| Date | commit | Strategy | Params | Data Range | Annualized | Max Drawdown | Sharpe | Notes |
|---|---|---|---|---|---|---|---|---|
| 2026-08-16 | a3f9c2 | dual_ma | 5/20 | 2020-2024 | 18.2% | -22% | 1.1 | Clear decay after adding fees (fictional example) |
A single CSV or Notion table suffices. Never rely on memory.
7. Common Pitfalls: The Three Every Beginner Hits
7.1 Time Zone Handling
⚠️ Mixed Time Zones Are a Silent Trap
Crypto timestamps are UTC milliseconds; some China A-share APIs hand over Beijing-time strings. When mixing them, normalize to one time zone (store everything in UTC, convert to local only for display):
import pandas as pd
# Binance klines open_time is a UTC millisecond timestamp
df = pd.read_csv("data/btcusdt_1h.csv")
df["open_time"] = pd.to_datetime(df["open_time"], unit="ms", utc=True)
df["open_time"] = df["open_time"].dt.tz_convert("Asia/Shanghai") # for display only7.2 Data Alignment
Indexes from different sources or contracts must be explicitly aligned before merging. pandas aligns on index by default and silently produces NaN on mismatch — the number-one source of backtest bugs:
a = pd.Series([1, 2, 3], index=[1, 2, 3]) # instrument A close prices
b = pd.Series([9, 8, 7], index=[2, 3, 4]) # another instrument's prices
pd.DataFrame({"a": a, "b": b}) # NaN at indexes 1/4 — always check💀 Look-Ahead Bias Is the Top Cause of Distorted Backtests
Using an indicator that can only be computed after today's close to fill today's order is peeking at the future; the backtest will be wildly inflated. Signals must be shifted with shift(1) so they take effect the next bar — this is rule number one of personal quant research.
7.3 Look-Ahead Bias
Using an indicator that can only be computed after today's close to participate in that same day's fill is peeking at the future, and inflates the backtest badly. Two most common forms:
- Filling on the same day's MA crossover signal → should fill at next day's open (or the bar after the signal);
- Computing means/volatility over the full dataset (e.g., zscore using the entire series) → use only history up to today (
rolling, not a full-samplemean()).
Correct approach (example code is for teaching only; live-trading risk is yours):
df["signal"] = (df["ma_fast"] > df["ma_slow"]).astype(int)
df["position"] = df["signal"].shift(1) # key: shift(1), effective next bar8. Next Step
With the toolchain ready, move on to 02-Data Acquisition in Practice: get China A-share, futures, and crypto data truly clean.
⚠️ Risk Warning
Tools and data are only the starting point of research and guarantee nothing about returns. Interface changes, missing data, and wrong price adjustment in free sources can all distort backtest results — paid data is not guaranteed accurate either. Always treat "data quality checks + multi-source cross-validation" as your research baseline. This article is for learning and research only and does not constitute investment advice.