QUANTITATIVE RESEARCH / MACHINE LEARNING / FULL-STACK
KodaQuant
Walk-Forward FX Research Terminal
A walk-forward FX research terminal: chronological validation, cost-aware execution, eighteen registered model families, overfitting diagnostics, and a committee pipeline — packaged as a local-first Electron desktop application.
View releaseAt a glance
- Role
- Quantitative research · ML and full-stack engineering
- Timeline
- University thesis → v1.0.0 release
- Platforms
- Web · Electron desktop (Ubuntu, Windows)
- Scope
- 6 FX pairs · 18 model families · OANDA thesis corpus, demo + CSV/Parquet datasets
- State
- v1.0.0 released · private source
Dashboard
The dashboard is the entry point to the KodaQuant research workspace.
Why this screen
Keeps market context visible while research, execution, and monitoring stay separate concerns.
Engineering highlights
- Chronological walk-forward splits with purge and embargo
- Eighteen registered model families behind one registry
- Cost-aware execution: spread, slippage and a one-bar delay
- Overfitting diagnostics: CSCV/PBO, DSR, PSR, HAC, trust score
- FastAPI with Celery and Redis workers, or in-process on desktop
- Public v1.0.0 Ubuntu and Windows builds with SHA-256 checksums
Why it exists
A backtest is easy to flatter: let a future candle into a feature, ignore the spread, refit on the whole history, and the equity curve stops meaning anything. KodaQuant treats chronology, cost and execution delay as part of the system rather than as caveats in a footnote, so a result reflects the conditions a strategy would actually have faced.
Where it started
Original objective
The original objective was academic. It began with the logistic-regression exercise from Alexander Hegerman's machine-learning course — which came with its own warning that the baseline was not something to trade — and turned it into a thesis question at HAW: does a model trained on FX candles still hold up once chronology, trading costs and execution delay are respected?
The technique
Walk-forward evaluation. Train on one window, purge the training rows whose forward label overlaps the validation window, embargo the boundary, evaluate out of sample, advance the window, and repeat. Preprocessing, feature selection and tuning are fitted inside the training window only, and every model is scored on held-out folds rather than on how well it fit.
Final product
A local-first FX research terminal: install a dataset, configure a study, run walk-forward evaluation across eighteen model families, read the overfitting diagnostics, and replay a committee in paper mode. Version 1.0.0 ships as an Ubuntu AppImage and a Windows installer with SHA-256 checksums. It is research and paper-replay software — the stable release does not place real orders.
How the scope changed
Thesis notebook
A logistic regression over FX candles in Jupyter, hand-built features, and the first chronological split — plus the course's own warning that the baseline was not tradeable.
Static prototype
A simple HTML and CSS page so results could be read outside a notebook. It proved the problem was not presentation but trust in the numbers.
Research pipeline
A Python data layer, feature engineering with disk caching, a walk-forward backtester, execution simulation and a model registry.
Product layer
FastAPI for the REST and WebSocket surface, a React workbench, and Celery with Redis so a long study never blocks the interface.
Committee stage
Optuna hyperparameter search, CSCV/PBO and deflated Sharpe diagnostics, regime and trust scoring, and a selection committee with an optional LLM proposer.
Desktop release
An Electron shell with a bundled Python backend, published for Ubuntu and Windows with update metadata and checksums — and a deliberate boundary: research and paper replay, not live execution.
How it is built
Datasets are installed, not re-seeded on demand. Canonical Parquet fragments are the source of truth, a SQLite catalog describes them, a Python research pipeline does feature engineering and walk-forward evaluation, and a FastAPI layer coordinates jobs and serves results to a React workbench — wrapped in an Electron shell on the desktop.
Data flow
Modules
Engineering decisions
Chronological walk-forward splits instead of random cross-validation
- Why
- Financial series are ordered; shuffling rows leaks the future into training and inflates every metric
- Trade-off accepted
- Fewer effective samples, and evaluation runs that are far longer
Purge overlapping labels and embargo the train/validation boundary
- Why
- A forward-looking label that spans the boundary is still leakage, even when the split is chronological
- Trade-off accepted
- More discarded rows near every fold edge and more complex split logic
Model spread, slippage and a one-bar execution delay inside the backtest
- Why
- A strategy that ignores trading costs is not evidence of anything
- Trade-off accepted
- More simulation code, slower runs, and results that look worse — which is the point
Canonical Parquet as the source of truth, SQLite only as a catalog
- Why
- Immutable, checksummed fragments are reproducible, and a corrupt file can fail loudly instead of silently
- Trade-off accepted
- Two stores to keep in sync, plus a one-time migration from the legacy candle table
One model registry instead of a hard-coded pipeline per family
- Why
- Eighteen families are only comparable if they are built, tuned and scored through the same contract
- Trade-off accepted
- Registry metadata and hyperparameter tiers to maintain as models evolve
Desktop mode can run jobs in-process instead of requiring Redis and Celery
- Why
- A packaged desktop application must not depend on Docker or a broker to run its first backtest
- Trade-off accepted
- Two execution paths that must stay behaviourally aligned, plus recovery logic for both
Data decisions
What was hard
News and sentiment features can quietly leak the future
Long research jobs would otherwise block the interface
Eighteen model families have to stay comparable
A packaged desktop app cannot assume Redis, Docker or a broker
Corrupt or partial market data must not silently degrade a study
Model selection can overfit the selection process itself
AI engineering
Two parts of the system use a model that is not a market predictor: an optional news-sentiment layer and the Factory proposer that suggests committee configurations. Both are optional, both are schema-checked, and both degrade to a deterministic path when the model is absent — no study depends on an API being reachable.
A proposer can invent a configuration the engine cannot run
- Control
- The proposal is validated against the same schemas and registry constraints as a hand-written configuration; an unusable proposal falls back to the deterministic baseline instead of failing the study.
- Effect
- A study always produces a runnable committee. The model is an assistant to the search, never an authority over it.
Hosted keys are easy to leak and easy to assume configured
- Control
- Keys live in the OS keychain — or a Docker secret file for containers — and are read through one store. News sentiment defaults to VADER; the hosted path is opt-in. Tests assert key material never reaches configuration output or logs.
- Effect
- No key means no call, not a broken feature. A key is never written to a queryable file, even locally.
Sentiment features are expensive and chronology-sensitive
- Control
- Article scores are cached, merged with forward-fill only, and default to the free lexicon backend; the hosted model is a weighted, explicitly enabled addition rather than the default path.
- Effect
- The cheap path is the default path, and enabling the hosted one is a deliberate, auditable choice.
A remote GPU is an unknown machine billed by the minute
- Control
- The Vast.ai integration only manages instances the user already rents, mirrors remote job state into the local job record, and keeps that local record authoritative. It is an optional path, not a dependency.
- Effect
- Remote compute is a convenience layer. Losing it does not change what the local application can do.
Guardrails
- Provider adapters are read-only and path-allowlisted; there is no execution endpoint to reach by accident.
- Commercial use is a separate, evidenced approval — the demo sync path asserts it stays disabled by default.
- The news LLM and the Factory proposer are optional, and both degrade to deterministic behaviour.
- Committee output is persisted with its inputs, fold diagnostics and seed policy, so a decision can be audited after the fact.
- The stable release registers paper replay only; broker execution is not exposed.
How it is verified
The gate is a documented methodology plus a layered suite, not a headline number. Heavy HPO, full-cycle, benchmark and GPU tests are marked slow and excluded from CI; the fast, CI-scoped suite is what gates a change.
- CI runs Ruff undefined-name checks, Python tests excluding the slow and api markers, and frontend lint, typecheck, unit tests and production build
- CI-scoped Python suite: 1,926 passed and 9 skipped; combined non-slow suite: 2,025 passed and 9 skipped
- 16 walk-forward integrity tests covering train/test overlap, execution delay and chronological ordering
- Committee and snapshot regression, news, and API plus price-stream suites run as separate targeted gates
- Build validation and Electron artifact tests verify the PyInstaller spec, imports, data files and entry points
- Frontend Vitest suites plus lint, typecheck and a production build on every push
- The release workflow verifies the unpacked Electron app, installer, update metadata and SHA-256 checksums before publishing
Where it stands
KodaQuant reached v1.0.0 and publishes an Ubuntu AppImage and a Windows installer with SHA-256 checksums. The stable release is a local research terminal: install a dataset, run a chronological study, compare eighteen model families, read the overfitting diagnostics, and replay a committee in paper mode. It does not place real orders. The source repository stays private, so the release binaries and this write-up are the public surface.
Known limitations
- The stable release is research and paper-replay software: it does not place real orders and broker execution is not exposed.
- The public build ships credential-free demo data and user-owned CSV/Parquet imports; provider access is gated, and OANDA is disabled by default.
- Remote GPU rental through Vast.ai is implemented and unit-tested, but instance provisioning is not a completed production deployment.
- Some settings still differ between backtest and paper paths — trailing stops are backtest-only, and the newer paper path does not yet consume every execution setting.
- Deep-model inference shapes and the CNN sequence path are known sharp edges documented in the repository rather than silent behaviour.
- CatBoost is registered but not declared in the requirements files; finBERT and PyTorch are deliberately excluded from the packaged build.
- Source is private; only the release binaries and this write-up are public.
Next
Near-term work is provider onboarding behind the approval registry, aligning the remaining execution settings across backtest and paper replay, closing the deep-model shape defects, and hardening remote GPU provisioning before it is described as a supported path.
In short
KodaQuant is a research instrument first and an application second. The engineering was never a model's accuracy — it was refusing to let the future touch the past: installed datasets instead of silent re-seeding, purge and embargo instead of convenient shuffling, costs and delays inside the simulation, and a committee that has to survive out-of-sample folds, regime coverage and seed variance before it is allowed an opinion.