Local changes: Updated model training, removed debug instrumentation, and configuration improvements
This commit is contained in:
kfox
185
docs/architecture/overview.md
Normal file
185
docs/architecture/overview.md
Normal file
@@ -0,0 +1,185 @@
|
||||
# Architecture Overview
|
||||
|
||||
This document provides a high-level overview of the Crypto Trader architecture.
|
||||
|
||||
## System Architecture
|
||||
|
||||
Crypto Trader follows a modular architecture with clear separation of concerns:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Frontend (React + Vite) │
|
||||
│ Dashboard | Strategies | Portfolio | Backtest | Settings│
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Backend API (FastAPI) │
|
||||
│ Autopilot | Trading | Market Data | WebSocket │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌──────────────┐ ┌──────────────┐ ┌──────────────────┐
|
||||
│ Redis │ │ Celery │ │ PostgreSQL │
|
||||
│ State/Cache │←───│ Workers │ │ Database │
|
||||
└──────────────┘ └──────────────┘ └──────────────────┘
|
||||
▲ │
|
||||
│ ▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Trading Engine │
|
||||
│ Order Management | Position Tracking | Risk Management │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
│
|
||||
┌───────────────────┼───────────────────┐
|
||||
▼ ▼ ▼
|
||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ Strategies │ │ Exchanges │ │ Portfolio │
|
||||
│ Framework │ │ Adapters │ │ Tracker │
|
||||
└──────────────┘ └──────────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
## Core Components
|
||||
|
||||
### 1. Frontend Layer
|
||||
|
||||
- **Framework**: React with TypeScript
|
||||
- **Build Tool**: Vite
|
||||
- **Components**: Dashboard, Strategy Manager, Portfolio View, Backtest View, Settings
|
||||
- **State Management**: React Query, Context API
|
||||
- **Styling**: Material-UI (MUI)
|
||||
|
||||
### 2. Backend API
|
||||
|
||||
- **Framework**: FastAPI (async Python)
|
||||
- **Features**: RESTful API, WebSocket support, auto-docs (Swagger/OpenAPI)
|
||||
- **Authentication**: JWT tokens (planned)
|
||||
- **Responsibilities**: Orchestrates all business logic
|
||||
|
||||
### 3. Redis (State Management)
|
||||
|
||||
- **Purpose**: Distributed state management and caching
|
||||
- **Use Cases**:
|
||||
- Autopilot registry (prevents multiple instances)
|
||||
- Daily trade count persistence (survives restarts)
|
||||
- Session caching (planned)
|
||||
- Real-time data caching (planned)
|
||||
- **Configuration**: `src/core/config.py` → `redis` section
|
||||
|
||||
### 4. Celery (Background Tasks)
|
||||
|
||||
- **Purpose**: Offload CPU-intensive tasks from the API
|
||||
- **Use Cases**:
|
||||
- ML model training (`train_model_task`)
|
||||
- Data bootstrapping (`bootstrap_task`)
|
||||
- Report generation (`generate_report_task`)
|
||||
- **Broker**: Redis
|
||||
- **Worker Command**: `celery -A src.worker.app worker --loglevel=info`
|
||||
|
||||
### 5. Trading Engine
|
||||
|
||||
- **Responsibilities**: Order execution, position management, risk checks
|
||||
- **Components**: Trading engine, order manager, paper trading simulator
|
||||
- **Integration**: Connects strategies to exchanges
|
||||
|
||||
### 6. Strategy Framework
|
||||
|
||||
- **Base Class**: `BaseStrategy` provides common interface
|
||||
- **Registry**: Manages available strategies (RSI, MACD, Bollinger, etc.)
|
||||
- **ML Selection**: `IntelligentAutopilot` uses ML to select optimal strategies
|
||||
- **Features**: Multi-timeframe support, scheduling, parameter management
|
||||
|
||||
### 7. Exchange Adapters
|
||||
|
||||
- **Pattern**: Adapter pattern for unified interface
|
||||
- **Factory**: Dynamic exchange instantiation
|
||||
- **Current**: Coinbase Advanced Trade API, Binance Public
|
||||
- **Data Providers**: CCXT-based providers with automatic failover
|
||||
|
||||
### 8. Risk Management
|
||||
|
||||
- **Components**: Risk manager, stop-loss, position sizing, limits
|
||||
- **Integration**: Pre-trade checks, real-time monitoring
|
||||
- **Features**: Drawdown limits, daily loss limits, position limits
|
||||
|
||||
### 9. Backtesting Engine
|
||||
|
||||
- **Features**: Historical data replay, realistic simulation
|
||||
- **Components**: Engine, metrics, slippage model, fee model
|
||||
- **Optimization**: Parameter optimization support
|
||||
|
||||
### 10. Portfolio Management
|
||||
|
||||
- **Tracking**: Real-time position tracking
|
||||
- **Analytics**: Performance metrics, risk analysis
|
||||
- **Rebalancing**: Automatic portfolio rebalancing (planned)
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Trading Flow
|
||||
|
||||
1. User starts autopilot → API receives request
|
||||
2. Redis lock checked/set → Prevents duplicate instances
|
||||
3. Strategy generates signal
|
||||
4. Risk manager validates trade
|
||||
5. Order manager creates order
|
||||
6. Exchange adapter executes order
|
||||
7. Position tracker updates positions
|
||||
8. Redis updates trade count
|
||||
|
||||
### ML Training Flow (Background)
|
||||
|
||||
1. User triggers `/retrain` API
|
||||
2. API queues `train_model_task` to Celery
|
||||
3. API returns task ID immediately (non-blocking)
|
||||
4. Celery worker picks up task
|
||||
5. Worker bootstraps data if needed
|
||||
6. Worker trains model
|
||||
7. User polls `/tasks/{task_id}` for status
|
||||
|
||||
## Technology Stack
|
||||
|
||||
- **Language**: Python 3.11+
|
||||
- **Frontend**: React 18, TypeScript, Vite
|
||||
- **Backend**: FastAPI, Uvicorn
|
||||
- **Database**: PostgreSQL with SQLAlchemy
|
||||
- **Cache/State**: Redis
|
||||
- **Task Queue**: Celery
|
||||
- **Exchange Library**: CCXT
|
||||
- **Data Analysis**: Pandas, NumPy
|
||||
- **Machine Learning**: LightGBM, scikit-learn
|
||||
- **Technical Analysis**: pandas-ta, TA-Lib
|
||||
- **Async**: asyncio
|
||||
- **Testing**: pytest, React Testing Library
|
||||
|
||||
## Design Patterns
|
||||
|
||||
- **Adapter Pattern**: Exchange adapters
|
||||
- **Factory Pattern**: Exchange and strategy creation
|
||||
- **Strategy Pattern**: Trading strategies
|
||||
- **Observer Pattern**: Data updates to strategies
|
||||
- **Singleton Pattern**: Configuration, database connections, Redis client
|
||||
|
||||
## Security Architecture
|
||||
|
||||
- **API Key Encryption**: Fernet encryption
|
||||
- **Secure Storage**: Keyring integration
|
||||
- **Audit Logging**: All security events logged
|
||||
- **Permission Management**: Read-only vs trading modes
|
||||
|
||||
## Scalability Considerations
|
||||
|
||||
- **Async Operations**: Non-blocking I/O throughout
|
||||
- **Redis State**: Enables horizontal scaling of API workers
|
||||
- **Celery Workers**: Can scale independently for heavy workloads
|
||||
- **Database Optimization**: Indexed queries, connection pooling
|
||||
- **Data Retention**: Configurable retention policies
|
||||
|
||||
## Extension Points
|
||||
|
||||
- **Exchange Adapters**: Add new exchanges via adapter interface
|
||||
- **Strategies**: Create custom strategies via base class
|
||||
- **Indicators**: Add custom indicators
|
||||
- **Order Types**: Extend advanced order types
|
||||
- **Risk Rules**: Add custom risk management rules
|
||||
- **Celery Tasks**: Add new background tasks in `src/worker/tasks.py`
|
||||
|
||||
Reference in New Issue
Block a user