Irrational Signals: an ML product, end-to-end — and the call to retire it
An intraday trading-signals product for US equities, built and operated solo across data, model, API, and go-to-market. It ran live, then was deliberately wound down. Archived build case study.
- End-to-enddata · model · API · GTM
- Provenanceper-signal win-rate honesty
- Retireddeliberate, ordered wind-down
Irrational Signals was an intraday trading-signals product for US equities — a tiered, API-first SaaS with a Python SDK. Statistical models scanned hundreds of equities each market hour and surfaced short-horizon opportunities across three sectors (Technology, Consumer Cyclical, Communication Services), six times per trading day. It was built and operated end-to-end: data pipeline, model, API, billing, brand, and infrastructure.
It is no longer a commercial product. The model validated across volatility regimes and performed live for a stretch, but the commercial thesis didn't close and the market eventually moved somewhere the data had never been. It was decommissioned deliberately and in order. This page is an archived build case study — the methodology and the judgement, not a live service.
Treat technical honesty as a product feature. The hard part of a signals product isn't producing a number — it's being honest about how much to trust it. Every layer was built to make the provenance and uncertainty of a signal explicit rather than hidden behind a headline percentage.
- Data / ML: an hourly R + Python pipeline — market data → staged feature-engineering 'context' tables → a LightGBM ranker gated at AUC ≥ 0.70 → empirically-bounded expected-return and exit targets. Retrained weekly, with daily monitoring for distribution shift and paper-vs-actual win-rate gaps.
- The honesty layer (the standout): every signal declared a win_rate_type — per_symbol, all_symbol, or pooled — stating how trustworthy its own win rate was. It surfaced the provenance of a number, not just the number; most live signals were individually validated (per_symbol).
- Product: tiering mapped to operational cost and conviction, not arbitrary feature-gating. Proportional sampling across win-rate buckets, with live preflight checks reserved for the top tier.
- Operational surface: JWT auth, tiered API keys, rate limits, RFC 7807 problem-details errors, and security headers. Whop as merchant-of-record — a deliberate choice to offload US sales-tax and EU VAT compliance. GA4 wired to authenticated user IDs to measure acquisition-to-retention.
- Stack: R + Python, PostgreSQL, FastAPI, Cloudflare (Tunnel / Worker / KV), LightGBM.
Market data → staged feature tables → LightGBM ranker (AUC ≥ 0.70) → REST API + Python SDK. Every signal carried the provenance of its own win rate.
A complete ML product shipped and operated solo — across data engineering, modelling, API design, billing, and go-to-market. It ran live, validated against paper-forward returns, and held up across several volatility regimes before the wind-down.
Two things ended it, stated plainly. The model broke down under rare, out-of-distribution shocks — geopolitical and commodity events (US–Iran tensions, oil spikes) that no calibration window can anticipate; it worked until the market went somewhere the data had never been. And the commercial thesis didn't close: a retail audience paying recurring for a statistical edge, net of their own execution costs, never materialised. So it was decommissioned deliberately and in order.
The takeaway isn't the model — it's the judgement around it. The win_rate_type provenance field mattered more than any single accuracy number: trading customers assume the best about your numbers unless you tell them what to assume, and saying 'this one is per-symbol with ≥10 samples, that one is pooled' is more credible than a shiny percentage. The original site led with a '70%+ win rate' headline; that was the positioning mistake. The other half of the lesson is knowing when to stop — shipping end-to-end is a skill, and so is decommissioning something in an orderly way once the evidence says the thesis won't close.