crypto_trader/docs/architecture/autopilot.md

Autopilot Architecture

This document describes the architecture of the Autopilot system, including both Pattern-Based and ML-Based modes.

Overview

The Autopilot system provides autonomous trading signal generation using an Intelligent Autopilot powered by machine learning-based strategy selection.

Architecture Diagram

┌─────────────────────────────────────────────────────────────┐
│                    Unified Autopilot API                    │
│                  (/api/autopilot/start-unified)            │
└───────────────────────┬─────────────────────────────────────┘
                        │
        ┌───────────────┴───────────────┐
        │                               │
        ▼                               ▼
┌───────────────────┐         ┌───────────────────┐
│ ML-Based          │         │  Signal Display   │
│ Autopilot         │         │  (Dashboard)      │
├───────────────────┤         └───────────────────┘
│ MarketAnalyzer    │
│ StrategySelector  │
│ PerformanceTracker│
└───────────────────┘
        │
        ▼
┌───────────────────────┐
│   Signal Generation   │
│   (StrategySignal)    │
└───────────────────────┘
        │
        ▼
┌───────────────────┐
│  Auto-Execution   │
│  (Optional)       │
└───────────────────┘

ML-Based Autopilot Flow

Note

Global Model Architecture: The system uses a single "Global Model" trained on data from all configured symbols. This allows for shared learning (patterns from BTC help predict ETH) and efficient resource usage.

Important

Single-Asset Focus: The current ML model is designed for single-asset strategies (e.g., Momentum, RSI, Grid). It analyzes each coin in isolation. It does NOT currently support multi-asset strategies like Arbitrage, which require analyzing correlations between multiple coins simultaneously.

Market Data (OHLCV)
        │
        ▼
┌───────────────────┐
│ MarketAnalyzer    │
│  - Analyze        │
│    conditions     │
│  - Determine      │
│    regime         │
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐
│ StrategySelector   │
│  - Global ML Model │
│  - Select best     │
│    strategy        │
│  - Calculate       │
│    confidence      │
└─────────┬─────────┘
          │
          ▼
┌───────────────────┐
│ Selected Strategy │
│  - Generate signal│
│  - Calculate size  │
└─────────┬─────────┘
          │
          ▼
    StrategySignal

API Endpoints

Unified Endpoints

• POST /api/autopilot/start-unified - Start autopilot
• POST /api/autopilot/stop-unified - Stop autopilot
• GET /api/autopilot/status-unified/{symbol} - Get autopilot status

Legacy Endpoints (Removed)

• POST /api/autopilot/start - (Removed)
• POST /api/autopilot/intelligent/start - (Removed)

Mode Selection Logic

The unified endpoints now exclusively support the "intelligent" mode. Code handling other modes has been removed.

def start_unified_autopilot(config: UnifiedAutopilotConfig):
    # Always starts intelligent autopilot
    autopilot = get_intelligent_autopilot(
        symbol=config.symbol,
        exchange_id=config.exchange_id,
        timeframe=config.timeframe,
        interval=config.interval,
        paper_trading=config.paper_trading
    )
    
    if config.auto_execute:
        autopilot.enable_auto_execution = True
    
    autopilot.start()

Auto-Execution Integration

Auto-execution is integrated directly into the ML-based workflow:

# Auto-Execution
if auto_execute and signal and confidence > threshold:
    trading_engine.execute_order(
        symbol=signal.symbol,
        side=signal.signal_type,
        quantity=strategy.calculate_position_size(signal, balance),
        ...
    )

Pre-Flight Order Validation

Before submitting orders, the autopilot validates that the order will succeed:

async def _can_execute_order(side, quantity, price) -> Tuple[bool, str]:
    # 1. Check minimum order value ($1 USD)
    if order_value < 1.0:
        return False, "Order value below minimum"
    
    # 2. For BUY: check sufficient balance (with fee buffer)
    if side == BUY:
        if balance < (order_value + fee_estimate):
            return False, "Insufficient funds"
    
    # 3. For SELL: check position exists
    if side == SELL:
        if no_position:
            return False, "No position to sell"
    
    return True, "OK"

Benefits:

• Prevents creation of PENDING orders that would be REJECTED
• Cleaner order history (no garbage orders)
• Faster execution (skip validation at execution layer)

Smart Order Type Selection

The autopilot intelligently chooses between LIMIT and MARKET orders:

Scenario	Order Type	Rationale
Strong signal (>80%)	MARKET	High confidence, execute now
Normal BUY signal	LIMIT (-0.1%)	Get better entry, pay maker fees
Take-profit SELL	LIMIT (+0.1%)	Get better exit price
Stop-loss SELL	MARKET	Urgent exit, protect capital

def _determine_order_type_and_price(side, signal_strength, price, is_stop_loss):
    # Strong signals or stop-losses use MARKET
    if signal_strength > 0.8 or is_stop_loss:
        return MARKET, None
    
    # Normal signals use LIMIT with 0.1% better price
    if side == BUY:
        limit_price = price * 0.999  # 0.1% below market
    else:
        limit_price = price * 1.001  # 0.1% above market
    
    return LIMIT, limit_price

Stop-Loss vs Take-Profit Detection:

• If current_price < entry_price: Stop-loss (use MARKET)
• If current_price > entry_price: Take-profit (use LIMIT)

Data Flow

1. Market Data Collection: OHLCV data from database
2. Market Analysis: MarketAnalyzer determines conditions
3. Strategy Selection: ML model selects best strategy
4. Signal Generation: Selected strategy generates signal
5. Opportunity Evaluation: Check confidence and criteria
6. Execution (if enabled): Execute trade
7. Learning: Record trade for model improvement

Component Responsibilities

• MarketAnalyzer: Analyzes market conditions and determines regime
• StrategySelector: ML model for selecting optimal strategy
• PerformanceTracker: Tracks strategy performance for learning
• IntelligentAutopilot: Orchestrates ML-based selection and execution

Configuration

{
    "symbol": "BTC/USD",
    "exchange_id": 1,
    "timeframe": "1h",
    "interval": 60.0,  # Analysis cycle in seconds
    "paper_trading": True,
    "min_confidence_threshold": 0.75,
    "max_trades_per_day": 10,
    "auto_execute": False
}

State Management

• _running: Boolean flag
• _last_analysis: Last analysis result
• _selected_strategy: Currently selected strategy
• _trades_today: Trade count for daily limit
• _ohlcv_data: Current market data
• _strategy_instances: Cached strategy instances
• _intelligent_autopilots: Global registry of active instances

Error Handling

Both modes implement error handling:

• Market Data Errors: Fallback to cached data or skip cycle
• Pattern Detection Errors: Return no pattern, continue monitoring
• Sentiment Analysis Errors: Use neutral sentiment, continue
• ML Model Errors: Fallback to rule-based selection
• Execution Errors: Log error, continue monitoring

Performance Considerations

• Moderate: ML model adds overhead but is optimized for speed
• Adaptive: Performance improves with more training data
• Scalable: Model can handle multiple symbols with shared learning

ML Model Training

The ML model training system supports background training via Celery with configurable parameters.

Training Configuration

Training is configured via the Bootstrap Config API and persisted in the application config:

Parameter	Description	Default
days	Historical data (days) to fetch	90
timeframe	OHLCV candle timeframe	1h
min_samples_per_strategy	Minimum samples needed per strategy	10
symbols	Cryptocurrencies to train on	["BTC/USD", "ETH/USD"]

API Endpoints

GET  /api/autopilot/bootstrap-config     # Get current config
PUT  /api/autopilot/bootstrap-config     # Update config
POST /api/autopilot/intelligent/retrain  # Trigger retraining
GET  /api/autopilot/tasks/{task_id}      # Poll training status
GET  /api/autopilot/intelligent/model-info  # Get model info
POST /api/autopilot/intelligent/reset    # Reset model

Background Training Architecture

┌─────────────────┐     ┌──────────────┐     ┌─────────────────┐
│ Settings Page   │────▶│ FastAPI      │────▶│ Celery Worker   │
│ (React)         │     │ Backend      │     │ (train_task)    │
└─────────────────┘     └──────────────┘     └─────────────────┘
        │                      │                      │
        │ Poll status          │ get_config()         │ Reset singletons
        │◀─────────────────────│                      │ Bootstrap data
        │                      │                      │ Train model
        │                      │                      │ Save to disk
        │                      │◀─────────────────────│
        │ SUCCESS + metrics    │                      │
        │◀─────────────────────│                      │
        │                      │                      │
        ▼                      ▼                      ▼
  UI shows results      Model auto-loads       Model file saved

Training Task Flow

1. Singleton Reset: Reset StrategySelector, PerformanceTracker, and Database singletons to prevent async conflicts
2. Check Existing Data: Query performance tracker for existing training samples
3. Bootstrap (if needed): Fetch OHLCV data for each symbol with progress updates
4. Train Model: Train ML model with cross-validation and walk-forward validation
5. Save Model: Persist trained model to ~/.local/share/crypto_trader/models/
6. Return Metrics: Return training accuracy and model metadata

Model Auto-Reload

The StrategySelector.get_model_info() method automatically checks for new model files:

def get_model_info(self):
    # Always check if a newer model is available on disk
    self._try_load_saved_model()  # Compare timestamps, reload if newer
    return {...}

This ensures the API process picks up newly trained models without restart.

Error Handling

• Celery Task Errors: Serialized to JSON with traceback for frontend display
• Polling Errors: Frontend stops polling after 3 consecutive failures
• Connection Errors: Singletons reset to prevent asyncpg.InterfaceError

Strategy Grouping

To improve ML accuracy, individual strategies are grouped into 5 logical categories. The model predicts the best group, then rule-based logic selects the optimal strategy within that group.

Strategy Groups

Group	Strategies	Market Conditions
Trend Following	moving_average, macd, confirmed	Strong trends (ADX > 25)
Mean Reversion	rsi, bollinger_mean_reversion, grid, divergence	Ranging markets, oversold/overbought
Momentum	momentum, volatility_breakout	High volume spikes, breakouts
Market Making	market_making, dca	Low volatility, stable markets
Sentiment Based	sentiment, pairs_trading, consensus	External signals available

Benefits

• Reduced Classes: 5 groups vs 14 strategies (~20% random baseline vs ~7%)
• Better Generalization: Model learns group characteristics rather than memorizing individual strategies
• Combined Confidence: group_confidence × strategy_confidence for final score

Improved Bootstrap Sampling

Training data is collected using intelligent sampling rather than fixed intervals:

Sampling Strategies

1. Regime-Change Detection: Sample when market regime changes (e.g., trending → ranging)
2. Minimum Time Spacing: 24-hour gaps between samples for 1h timeframe
3. Periodic Sampling: Every 48 hours regardless of regime changes

Timeframe Spacing

Timeframe	Min Spacing (candles)	Actual Time
1m	1440	24 hours
15m	96	24 hours
1h	24	24 hours
4h	6	24 hours
1d	1	24 hours

Security Considerations

• Auto-Execution: Requires explicit user confirmation
• Paper Trading: Default mode for safety
• Risk Limits: Enforced regardless of mode
• API Keys: Encrypted storage
• Audit Logging: All autopilot actions logged

Future Enhancements

• Hybrid mode combining both approaches
• Real-time mode switching
• Multi-symbol autopilot management
• Advanced risk management integration
• Performance analytics dashboard
• Persistent training configuration (file/database storage)