How the agent decides
A quick read on what the research agent measures, why a good backtest isn't enough, and how a winning configuration is actually determined. Every number referenced here is visible on the corresponding run — this page explains what they mean.
The research loop
You state a testable idea and the success criteria (Sharpe, win rate, drawdown, trade count, recent out-of-sample Sharpe). Goals are evaluated deterministically — the AI never grades its own homework.
The agent writes an experiment plan: which levers to test, in what order, and what would falsify the hypothesis.
Each round runs a real action — backtest, parameter sweep, walk-forward validation, or a config change — and is judged by deterministic quality checks (overfit signals, bankrupt parameter regions, degenerate sweeps).
Before concluding, the trades are audited (exit mix, R-multiples, monthly consistency) and the best config is scored against the deployment-readiness gates below.
The run ends with an evidence-cited verdict and machine-readable follow-up suggestions — each must cite a concrete artifact (a failed gate, a quality flag, a fold result). Campaigns chain these automatically, pausing for your approval.
How a winner is picked
One number answering "how deployable is this config?" — not just how profitable it looked.
A composite that starts from stability-adjusted risk-adjusted return, is rewarded by profit factor and reward:risk, and is reduced by penalties for boundary-hugging parameters, unprofitable months, and churn-heavy exit profiles. Exact weights are proprietary.
Selection is constraint-first (configs that fail hard constraints are out regardless of score) and high-water-marked: a later round can never silently replace a better config found earlier.
Why a good backtest isn't enough
A single backtest shows one path through one history — it rewards configurations tuned to that exact path. Every promising config is therefore re-tested with walk-forward validation (train on one window, test on unseen data, repeat across windows) and a Monte-Carlo bootstrap (thousands of reshuffles of the actual trades to estimate how the strategy fares when luck runs against it). Deployment decisions are made on the unlucky tail of those simulations, not the average.
The training data the strategy was developed on — like studying for an exam with the textbook. Good IS results are expected but don't prove the strategy works on new data.
Fresh data the strategy has never seen — the actual exam. If OOS results hold up, the strategy likely isn't overfit to historical quirks.
Instead of one IS/OOS split, the data is divided into 4+ overlapping time windows. Each window trains on IS and tests on OOS independently. If the strategy works across all windows, it's robust across different market conditions — not just lucky timing.
The actual trade sequence is randomly reshuffled 1,000 times to build confidence intervals. Even if average performance is good, Monte Carlo shows the realistic range of outcomes — including worst-case scenarios. A strategy where the 5th-percentile Sharpe is still positive is genuinely robust.
Measures return relative to risk. A Sharpe of 1.0 means you earn 1 unit of return per unit of risk. Above 1.5 is strong; above 2.0 is excellent. Below 0.5 is generally not worth the risk.
The largest peak-to-trough decline during the test period. If your account hit $10,000 then dropped to $7,500 before recovering, that's a 25% max drawdown. Lower is better — high drawdown means high pain.
OOS Sharpe divided by IS Sharpe. A ratio of 1.0 means the strategy performs equally well on unseen data. Above 0.7 is healthy. Below 0.5 suggests the strategy may be overfit to the training period.
The deployment-readiness gates
A config is deployableonly when every gate passes. Current thresholds and each run's actual values are shown on the run's Deployment Checklist and the campaign Gate matrix.
| Gate | What it protects you from |
|---|---|
Worst-case tail wf_mc_p5 | Even in the unlucky bottom slice of simulated futures, the strategy should still earn a solid risk-adjusted return. |
No boundary-hugging no_boundary_hits | The best parameters shouldn't sit at the edge of the range that was searched — that usually means the optimizer wanted to go further and the result is fragile. |
Profit factor profit_factor | Gross profits should meaningfully outweigh gross losses. |
Reward : risk effective_rr | The average winner should be meaningfully larger than the average loser. |
Exit mix trailing_exit_share | Wins shouldn't depend almost entirely on one exit type — trail-dominated profiles tend to churn and break in live conditions. |
Consistency months_profitable | Most calendar months should be profitable, not one lucky streak carrying the year. |
Even activity fold_trade_cv | Trading activity should be spread across the whole period — a strategy that only fires in one regime is regime-dependent. |
Same data, same config wf_hash_match | The walk-forward validation must have run on exactly the config and data being judged — no swapped-in results. |
Sweep objectives
Sweeps search for the most deployable config overall.
The sampler and the selector both optimize Deploy Score, so search matches selection.
For hardening: after the normal sweep, the top finalists are re-tested with real walk-forward validation and re-ranked by their worst-case tail — the winner is the config that holds up when unlucky, not the one with the prettiest average.
Two-stage: Deploy Score screen, then a walk-forward Monte-Carlo re-rank of finalists keyed on tail Sharpe. Used when the worst-case-tail gate is the blocker.
Run verdicts
The hypothesis held: goals met and the result survived validation.
The run improved the config but didn't fully meet the goals — the most common productive outcome; learnings compound into the strategy's ledger.
The hypothesis was contradicted by the evidence. Recorded so future runs don't retry it.
Not enough evidence either way — usually too few trades or noisy results.
The run hit its round or LLM-call budget before reaching a conclusion.
Exact scoring weights, threshold calibration, and indicator internals are proprietary and intentionally not documented here.
Last reviewed: 2026-07-07