crypto_trader/docs/architecture/data_flow.md

Data Flow Architecture

This document describes how data flows through the Crypto Trader system.

Real-Time Data Flow

Exchange WebSocket
        │
        ▼
Exchange Adapter
        │
        ▼
Data Collector
        │
        ├──► Data Quality Validation
        │
        ├──► Data Storage (Database)
        │
        └──► Timeframe Manager
                │
                ├──► Strategy 1 (1h timeframe)
                ├──► Strategy 2 (15m timeframe)
                └──► Strategy 3 (1d timeframe)
                        │
                        ▼
                    Signal Generation
                        │
                        ▼
                    Trading Engine

Trading Signal Flow

Market Data Update
        │
        ▼
Strategy.on_data()
        │
        ▼
Indicator Calculation
        │
        ▼
Signal Generation
        │
        ▼
Trading Engine
        │
        ├──► Risk Manager (Pre-trade Check)
        │         │
        │         ├──► Position Sizing
        │         ├──► Drawdown Check
        │         └──► Daily Loss Check
        │
        ▼
Order Manager
        │
        ├──► Paper Trading (if enabled)
        │         │
        │         └──► Paper Trading Simulator
        │
        └──► Live Trading
                │
                ▼
            Exchange Adapter
                │
                ▼
            Exchange API
                │
                ▼
            Order Execution
                │
                ▼
            Order Manager (Update Status)
                │
                ▼
            Position Tracker
                │
                ▼
            Portfolio Analytics

Backtesting Data Flow

Historical Data Provider
        │
        ▼
Backtesting Engine
        │
        ├──► Data Replay (Time-based)
        │
        ├──► Strategy Execution
        │         │
        │         ├──► Signal Generation
        │         │
        │         └──► Order Simulation
        │                 │
        │                 ├──► Slippage Model
        │                 ├──► Fee Model
        │                 └──► Order Book Simulation
        │
        └──► Performance Calculation
                │
                ├──► Metrics Calculation
                │         │
                │         ├──► Returns
                │         ├──► Sharpe Ratio
                │         ├──► Sortino Ratio
                │         └──► Drawdown
                │
                └──► Results Storage

Portfolio Update Flow

Trade Execution
        │
        ▼
Position Update
        │
        ├──► Position Tracker
        │         │
        │         ├──► Update Quantity
        │         ├──► Update Entry Price
        │         └──► Calculate P&L
        │
        └──► Portfolio Analytics
                │
                ├──► Recalculate Metrics
                │         │
                │         ├──► Total Value
                │         ├──► Unrealized P&L
                │         ├──► Realized P&L
                │         └──► Performance Metrics
                │
                └──► Alert System
                        │
                        └──► Risk Alerts (if triggered)

Data Storage Flow

Data Collection
        │
        ▼
Data Quality Check
        │
        ├──► Valid Data
        │         │
        │         └──► Database Storage
        │                 │
        │                 ├──► Market Data Table
        │                 ├──► Trades Table
        │                 └──► Positions Table
        │
        └──► Invalid Data
                │
                └──► Error Logging
                        │
                        └──► Gap Detection
                                │
                                └──► Gap Filling (if enabled)

Multi-Timeframe Synchronization

1m Data Stream
        │
        ├──► Strategy (1m)
        │
        └──► Resample to 5m
                │
                ├──► Strategy (5m)
                │
                └──► Resample to 1h
                        │
                        ├──► Strategy (1h)
                        │
                        └──► Resample to 1d
                                │
                                └──► Strategy (1d)

Alert Flow

Data Update / Event
        │
        ▼
Alert Engine
        │
        ├──► Check Alert Conditions
        │         │
        │         ├──► Price Alerts
        │         ├──► Indicator Alerts
        │         ├──► Risk Alerts
        │         └──► System Alerts
        │
        └──► Trigger Alert (if condition met)
                │
                ├──► Desktop Notification
                ├──► Sound Alert
                └──► Email Notification (if configured)

Error Recovery Flow

Error Detected
        │
        ▼
Error Recovery System
        │
        ├──► Log Error
        │
        ├──► Attempt Recovery
        │         │
        │         ├──► Retry Operation
        │         ├──► Fallback Mechanism
        │         └──► State Restoration
        │
        └──► Health Monitor
                │
                ├──► Check System Health
                │
                └──► Auto-restart (if critical)