# AI Trading Signal Lab — Complete Project Documentation

Generated per docs/QA_AUDIT_PROMPTS.md (Prompt 2). Audience: an engineer or AI
model joining the project cold.

## Table of Contents

1. [Executive Overview](#1-executive-overview)
2. [System Architecture](#2-system-architecture)
3. [Folder Structure](#3-folder-structure)
4. [AI Agent Documentation](#4-ai-agent-documentation)
5. [API Documentation](#5-api-documentation)
6. [Database Documentation](#6-database-documentation)
7. [Workflows](#7-workflows)
8. [Configuration Reference](#8-configuration-reference)
9. [Deployment](#9-deployment)
10. [Monitoring](#10-monitoring)
11. [Security](#11-security)
12. [Maintenance Guide](#12-maintenance-guide)

---

## 1. Executive Overview

**Purpose:** AI-powered trading *research* platform. Scans US equities, scores
CALL/PUT options setups and BUY/SELL stock swing setups through a multi-agent
pipeline, enforces hard risk rules, tracks everything through paper trading,
and grades its own strategies on evidence.

**Business goal:** private research tool now → proven track record → possible
commercial/SaaS platform (Phase 3). **Live trading is deliberately impossible
in this codebase** — no order-execution paths exist (see docs/AUDIT_STATUS.md).

**Governing documents:** docs/MASTER_AUDIT_PROMPT.md (pre-production gates),
docs/AI_AGENT_ARCHITECTURE.md (meta-evaluation design),
docs/DASHBOARD_SPEC.md (UX spec), docs/QA_AUDIT_PROMPTS.md (review process).
Current audit verdict: **APPROVED FOR PAPER TRADING ONLY**.

## 2. System Architecture

```
Next.js dashboard (frontend/, port 3000)
   │  fetch w/ mock-data fallback (src/lib/api.ts)
   ▼
FastAPI (backend/app, port 8000)
   ├─ security middleware (headers → rate limit → auth gate)
   ├─ routers (auth, dashboard, scanner, signals, paper_trades,
   │           risk, reports, scheduler, analytics)
   ├─ agent pipeline (orchestrator → scanner → regime → per-candidate:
   │      volatility → technical → options → sentiment → probability
   │      (+calibration) → risk manager;  swing agent in parallel)
   ├─ services (market_data mock/live, live_data Alpaca+Tradier,
   │      options_pricing BS, paper_trading, position_mgmt, alerts,
   │      earnings, learning, performance, briefing, backtest,
   │      reporting, llm, scheduler)
   └─ SQLAlchemy → SQLite (dev) / MySQL (prod), additive migrations
```

**Data flow:** provider bars → freshness circuit breaker → indicators →
agent scores → ensemble confidence/probability (calibrated by past outcomes)
→ risk gate (approve/reject with reasons) → Signal row → (user or nothing) →
PaperTrade → mark-to-market loop (position mgmt: trail/scale/time) →
closed trade → learning/performance/benchmark → daily report + alerts.

**Design principles:** every agent = single responsibility returning a
structured `AgentResult`; every threshold in env vars; every safety rule
centralized and non-bypassable; mock-first with live upgrades env-gated;
failures degrade to mock/fallback rather than break.

## 3. Folder Structure

| Path | Purpose |
|---|---|
| `frontend/src/app/*` | Pages: dashboard (/), scanner, signals(+[id]), paper-trades, risk, backtest, performance, reports, settings |
| `frontend/src/components/` | Sidebar, PageHeader, StatCard, Badge (Direction/Type/Status/Risk/ConfidenceBar), Sparkline, SignalCard |
| `frontend/src/lib/` | types.ts (all API shapes), api.ts (client + mock fallback), mock.ts, useApi.ts hook, format.ts |
| `backend/app/agents/` | base, indicators (pure-python TA), market_scanner, volatility, technical, options, sentiment, swing, regime, probability, risk_manager, orchestrator |
| `backend/app/services/` | market_data (+mock, freshness), live_data (Alpaca/Tradier), options_pricing (Black-Scholes), paper_trading, position_mgmt, alerts (email+telegram), earnings (Finnhub), learning (calibration), performance (scorekeeper/grades), briefing (CIO), backtest, reporting, scheduler, llm |
| `backend/app/routers/` | REST endpoints, thin over services |
| `backend/app/` | main.py (app+middleware+startup), config.py (Settings), database.py, models.py, schemas.py, auth.py (JWT/PBKDF2), security.py (middleware), migrations.py (additive columns) |
| `backend/scanner/run_scan.py` | Cron entry: scan + mark + alerts + daily report |
| `backend/tests/` | ~75 tests: agents, api, providers, swing, options pricing, backtest+learning, risk gates, performance, briefing, alerts+scheduler, security |
| `database/schema.sql` | MySQL DDL mirror (models.py is source of truth) |
| `deployment/` | DEPLOYMENT.md, SECURITY.md (go-live checklist), nginx/systemd/cron examples |
| `docs/` | All governing docs + this file + AUDIT_STATUS.md |
| `setup.ps1` / `start.ps1` | Windows install+test / run both servers |

## 4. AI Agent Documentation

All agents implement `Agent.run(context: dict) -> AgentResult(agent, score
0-100, direction, data, notes)`. Failure mode for every agent: exceptions in
vendor calls are caught and degraded (mock fallback / None), never raised
into the pipeline.

| Agent | Purpose | Key inputs | Key outputs | Tunables |
|---|---|---|---|---|
| MarketScanner | Universe → top candidates by activity | universe quotes | ranked rows, candidates | max_candidates |
| Regime | Classify market (bull/bear/chop, high-vol) from SPY | 90d index bars | regime, risk_scale (0.5 in high vol) | REGIME_SYMBOL, REGIME_HIGH_VOL_ATR_PCT |
| Volatility | Tradability score from ATR%, rel-volume | bars, quote | score, atr | ideal ATR% band 2–6 hard-coded |
| Technical | RSI/MACD/EMA/SMA50/Bollinger/momentum → direction | bars | direction CALL/PUT, indicators dict | — |
| Options | Best contract: delta fit ~0.55, OI, spread, IV | chain, direction | contract dict, chain-quality score | — |
| Sentiment | Mock (price-proxy) or LLM (Anthropic/OpenAI key) | bars, symbol | sentiment −1..1, label, source | ANTHROPIC/OPENAI_API_KEY |
| Swing | Trend-pullback stock setups (BUY/SELL) | bars | direction, setup score | MIN_SWING_SCORE 65, MIN_TREND_STRENGTH 0.25, STOP_ATR_MULT 2.0, RSI 38–62 (agents/swing.py) |
| Probability | Weighted ensemble → confidence + calibrated probability | agent results, calibration | confidence ≤98, probability 5–90 | WEIGHTS in probability.py; learning.BLEND 0.35 |
| RiskManager | Final gate: entry/stop/targets/size, approve/reject | price, atr(+mult), confidence, probability, open risk, risk_scale, days_to_earnings | full trade plan + rejection_reasons | MIN_* thresholds in .env |

**Guarantees:** the AI layer cannot change risk limits, position caps, or
enable live trading; calibration only nudges probability within 5–90 with a
35% blend cap; rejected signals cannot be paper-traded (HTTP 400).

## 5. API Documentation

Base `http://localhost:8000`; interactive docs at `/docs` (disabled in
production). Auth: JWT bearer; with `REQUIRE_AUTH=true` all `/api/*` except
`/api/auth/*` require it (middleware, non-bypassable).

| Method+Path | Purpose |
|---|---|
| GET /health | liveness + mode flag |
| POST /api/auth/register · POST /api/auth/login · GET /api/auth/me | JWT auth |
| GET /api/dashboard/summary · GET /api/dashboard/briefing | portfolio stats · CIO briefing + readiness-ranked opportunities |
| POST /api/scanner/run · GET /api/scanner/results | run pipeline · latest scan |
| GET /api/signals?status=&limit= · GET /api/signals/{id} · DELETE /api/signals/history | list/detail/clear (traded signals kept for learning) |
| GET/POST /api/paper-trades · POST /api/paper-trades/mark · POST /api/paper-trades/{id}/close | open (approved only), mark-to-market (auto-close + position mgmt), manual close |
| GET /api/risk/rules · GET /api/risk/summary | thresholds · utilization |
| GET /api/reports/daily · POST /api/reports/generate | daily reports |
| GET/POST /api/scheduler/status, /run-now | auto-scan control |
| POST /api/backtest/run | walk-forward backtest (days, min_score, horizon, slippage_bps, commission_per_share) |
| GET /api/learning/calibration · GET /api/analytics/performance · /benchmark · /regime | learning + scorekeeper + benchmark + regime |

Errors: FastAPI standard `{"detail": ...}`; 400 invalid ops, 401 auth, 404
missing, 429 rate limit.

## 6. Database Documentation

Source of truth: `backend/app/models.py`. `migrations.py` adds missing
columns additively at startup (never drops/alters).

| Table | Key columns | Notes |
|---|---|---|
| users | email unique, password_hash (PBKDF2) | |
| scan_runs | universe_size, candidates, signals_created, results JSON | |
| signals | symbol, direction (CALL/PUT/BUY/SELL), signal_type (options/swing), status (approved/rejected), confidence, probability, entry/stop/targets, risk_reward, position_size, max_risk_usd, scores/rationale/rejection_reasons/option_contract/indicators/history JSON | FK scan_run_id |
| paper_trades | signal_id FK, direction, quantity, entry_price, stop_loss, **initial_stop** (R anchor), **scaled_out**, target, status, instrument (shares/option), option_* (contract, premiums, contracts, pnl), pnl, **mfe_r/mae_r**, exit_reason (stop_loss/target/manual/time_stop/scale_out), opened/closed_at | |
| alert_log | symbol+direction+alert_date+kind unique-ish dedupe | prevents alert spam |
| daily_reports | report_date unique, summary text, payload JSON | |

## 7. Workflows

**Scan cycle** (manual POST /api/scanner/run, auto-scheduler, or cron):
regime classify → for each fresh candidate: agents → ensemble → calibration →
risk gate (earnings blackout, regime risk_scale, daily budget) → Signal rows
(options + swing) → mark-to-market open trades (excursions, trail, scale-out,
time stop, stop/target closes) → alerts (email/Telegram, deduped) → scheduler
status updated.

**Paper trade lifecycle:** approved signal → open (entry pays
PAPER_SLIPPAGE_BPS; option contract attached + sized to risk budget) → marked
every cycle → closed by stop/target/time/scale-out/manual → feeds learning
(calibration), performance (grades), benchmark, reports.

**Backtest:** walk-forward per symbol, entries next-open with slippage,
stop-first conservative fills, costs modeled, R-multiple stats + equity curve.

## 8. Configuration Reference (.env)

| Variable | Default | Purpose |
|---|---|---|
| ENV | development | `production` = fail-closed boot checks + /docs off |
| REQUIRE_AUTH | false | mandatory true in production |
| RATE_LIMIT_PER_MINUTE | 240 | 0 disables (tests) |
| SECRET_KEY / ACCESS_TOKEN_EXPIRE_MINUTES | — / 1440 | JWT |
| DATABASE_URL | sqlite:///./trading_lab.db | MySQL string for prod |
| ACCOUNT_EQUITY / MAX_RISK_PER_TRADE_PCT / MAX_DAILY_RISK_PCT / MIN_RISK_REWARD / MIN_CONFIDENCE / MIN_PROBABILITY | 100000 / 1.0 / 6.0 / 2.0 / 65 / 55 | risk rules |
| DATA_PROVIDER | mock | `live` + keys = real data |
| ALPACA_API_KEY/SECRET, ALPACA_DATA_FEED | — / sip | bars + real-time quotes |
| TRADIER_API_TOKEN | — | real options chains |
| ANTHROPIC_API_KEY / OPENAI_API_KEY (+_MODEL) | — | LLM sentiment |
| FINNHUB_API_KEY / EARNINGS_BLACKOUT_DAYS | — / 5 | earnings guard |
| DATA_FRESHNESS_MAX_DAYS | 5 | circuit breaker |
| POSITION_MGMT_ENABLED / TRAIL_TRIGGER_R / SCALE_OUT_R / MAX_HOLD_DAYS / PAPER_SLIPPAGE_BPS | true / 1.0 / 1.5 / 10 / 5 | trade management |
| REGIME_SYMBOL / REGIME_HIGH_VOL_ATR_PCT | SPY / 2.5 | regime |
| AUTO_SCAN_ENABLED / AUTO_SCAN_INTERVAL_MINUTES / AUTO_SCAN_ALWAYS | true / 15 / false | scheduler |
| EMAIL_ALERTS_ENABLED + SMTP_* + ALERT_EMAIL_* | — | email alerts (Gmail app password) |
| TELEGRAM_BOT_TOKEN / TELEGRAM_CHAT_ID | — | Telegram alerts |
| FRONTEND_ORIGIN / NEXT_PUBLIC_API_URL | localhost:3000 / :8000 | CORS / client |

## 9. Deployment

Local: `setup.ps1` (venv+pip+pytest, npm install+build) → `start.ps1`.
Production: deployment/DEPLOYMENT.md (HostGator VPS) + deployment/SECURITY.md
(mandatory checklist) + nginx/systemd/cron examples. Staged progression per
MASTER_AUDIT_PROMPT Phase 18 — currently at "paper trading" stage.

## 10. Monitoring

Current: python logging to console (uvicorn window); scheduler status
endpoint (last_run, last_result, runs); circuit-breaker and feed-fallback
warnings logged. Gaps (AUDIT H-4): no centralized/tamper-resistant logs, no
alerting on scheduler stall — see SECURITY.md monitoring section for the
interim uptime-monitor approach.

## 11. Security

JWT auth (PBKDF2 200k iters), middleware auth gate on all /api routes,
per-IP rate limiting, security headers, HSTS in prod, fail-closed production
boot (strong secret + auth required), /docs disabled in prod, CORS pinned to
frontend origin, secrets only in .env (git-ignored). Known gaps tracked in
docs/AUDIT_STATUS.md (C-1 single-user model, C-2 plaintext .env, M-1 no
login backoff).

## 12. Maintenance Guide

- **Add an agent:** subclass `Agent` in `backend/app/agents/`, return
  `AgentResult`, wire into orchestrator, add tests, document here + CODEX.
- **Tune strategy:** change constants in agents/swing.py or .env thresholds;
  ALWAYS run a before/after backtest and record both in CODEX_TASKS.
- **Add an endpoint:** router + schema + test; never bypass services.
- **Schema change:** add column to models.py AND migrations.py AND
  database/schema.sql (additive only).
- **Update models/prompts:** LLM model ids via ANTHROPIC_MODEL/OPENAI_MODEL.
- **Backup:** copy `backend/trading_lab.db` (SQLite) or mysqldump; restore =
  replace file / import dump; migrations self-heal missing columns.
- **Troubleshooting:** backend window logs; `GET /api/scheduler/status`;
  `python diagnose_alpaca.py SYMBOL` for data-feed issues; delete
  `trading_lab.db` for a factory reset (loses history/learning).
- **Golden rules:** tests must pass before merge (Rule 4); paper-only until
  the MASTER_AUDIT_PROMPT process approves anything more; every safety gate
  gets a test.