Local changes: Updated model training, removed debug instrumentation, and configuration improvements
This commit is contained in:
kfox
116
docs/deployment/README.md
Normal file
116
docs/deployment/README.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# Deployment Guide
|
||||
|
||||
This guide covers deploying Crypto Trader in various environments.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [AppImage Deployment](appimage.md) - Building and distributing AppImage
|
||||
2. [Bluefin Linux](bluefin.md) - Bluefin Linux specific instructions
|
||||
3. [PostgreSQL Setup](postgresql.md) - PostgreSQL configuration
|
||||
4. [Updates](updates.md) - Update mechanism and versioning
|
||||
|
||||
## Deployment Options
|
||||
|
||||
### AppImage (Recommended)
|
||||
|
||||
- Single executable file
|
||||
- No installation required
|
||||
- Portable across Linux distributions
|
||||
- Includes all dependencies
|
||||
|
||||
### From Source
|
||||
|
||||
- Full control over installation
|
||||
- Customizable configuration
|
||||
- Development and production use
|
||||
|
||||
## System Requirements
|
||||
|
||||
- **OS**: Linux (Bluefin recommended), macOS, Windows
|
||||
- **Python**: 3.11+ (for source installation)
|
||||
- **Node.js**: 18+ (for frontend)
|
||||
- **Memory**: 4GB minimum, 8GB recommended
|
||||
- **Storage**: 1GB+ for application and data
|
||||
- **Network**: Internet connection required
|
||||
- **Redis**: Version 5.0+ (for state management)
|
||||
- **PostgreSQL**: Version 14+ (for database)
|
||||
|
||||
## Quick Deployment
|
||||
|
||||
### AppImage
|
||||
|
||||
1. Download AppImage
|
||||
2. Make executable: `chmod +x crypto_trader-*.AppImage`
|
||||
3. Run: `./crypto_trader-*.AppImage`
|
||||
|
||||
### From Source
|
||||
|
||||
1. Clone repository
|
||||
2. Install dependencies: `pip install -r requirements.txt`
|
||||
3. Install frontend dependencies: `cd frontend && npm install`
|
||||
4. Start Redis (see Redis section below for options)
|
||||
5. Start Celery worker: `celery -A src.worker.app worker --loglevel=info &`
|
||||
6. Start backend: `uvicorn backend.main:app --reload --host 0.0.0.0 --port 8000 &`
|
||||
7. Start frontend: `cd frontend && npm run dev`
|
||||
|
||||
Or use the helper script:
|
||||
```bash
|
||||
./scripts/start_all.sh
|
||||
```
|
||||
|
||||
## Required Services
|
||||
|
||||
### Redis
|
||||
|
||||
Redis is required for distributed state management and Celery background tasks (e.g., ML model training).
|
||||
|
||||
```bash
|
||||
# Install (Ubuntu/Debian)
|
||||
sudo apt-get install redis-server
|
||||
```
|
||||
|
||||
**Starting Redis**:
|
||||
|
||||
```bash
|
||||
# Option 1: Using system service (requires sudo)
|
||||
sudo service redis-server start
|
||||
|
||||
# Option 2: Direct daemon mode (for containers/restricted environments)
|
||||
redis-server --daemonize yes
|
||||
|
||||
# Verify
|
||||
redis-cli ping # Should return PONG
|
||||
```
|
||||
|
||||
> **Note**: In containerized environments (Toolbox, Distrobox, etc.) where `sudo` is not available, use the direct daemon mode option.
|
||||
|
||||
### Celery Worker
|
||||
|
||||
Celery handles background tasks like ML training:
|
||||
|
||||
```bash
|
||||
# Start worker
|
||||
celery -A src.worker.app worker --loglevel=info
|
||||
|
||||
# Start with specific queues
|
||||
celery -A src.worker.app worker -Q ml_training,celery --loglevel=info
|
||||
```
|
||||
|
||||
## Post-Deployment
|
||||
|
||||
After deployment:
|
||||
|
||||
1. Configure exchanges
|
||||
2. Set up risk management
|
||||
3. Verify Redis connection: `python scripts/verify_redis.py`
|
||||
4. Test with paper trading
|
||||
5. Review configuration
|
||||
6. Start with small positions
|
||||
|
||||
## Production Considerations
|
||||
|
||||
- Use a process manager (systemd, supervisor) for services
|
||||
- Configure Redis persistence (AOF or RDB)
|
||||
- Set up monitoring and alerting
|
||||
- Enable HTTPS for the API
|
||||
- Configure proper firewall rules
|
||||
171
docs/deployment/postgresql.md
Normal file
171
docs/deployment/postgresql.md
Normal file
@@ -0,0 +1,171 @@
|
||||
# PostgreSQL Setup
|
||||
|
||||
This guide covers optional PostgreSQL database configuration.
|
||||
|
||||
## When to Use PostgreSQL
|
||||
|
||||
PostgreSQL is recommended for:
|
||||
|
||||
- Large datasets (millions of trades)
|
||||
- Multiple users
|
||||
- Advanced queries
|
||||
- Production deployments
|
||||
- High-performance requirements
|
||||
|
||||
## Installation
|
||||
|
||||
### Install PostgreSQL
|
||||
|
||||
**Fedora/Bluefin**:
|
||||
```bash
|
||||
sudo dnf install postgresql postgresql-server
|
||||
sudo postgresql-setup --initdb
|
||||
sudo systemctl enable postgresql
|
||||
sudo systemctl start postgresql
|
||||
```
|
||||
|
||||
**Ubuntu/Debian**:
|
||||
```bash
|
||||
sudo apt-get install postgresql postgresql-contrib
|
||||
sudo systemctl enable postgresql
|
||||
sudo systemctl start postgresql
|
||||
```
|
||||
|
||||
## Database Setup
|
||||
|
||||
### Create Database
|
||||
|
||||
```bash
|
||||
sudo -u postgres psql
|
||||
```
|
||||
|
||||
```sql
|
||||
CREATE DATABASE crypto_trader;
|
||||
CREATE USER crypto_trader_user WITH PASSWORD 'your_password';
|
||||
GRANT ALL PRIVILEGES ON DATABASE crypto_trader TO crypto_trader_user;
|
||||
\q
|
||||
```
|
||||
|
||||
### Configure Connection
|
||||
|
||||
Update `config.yaml`:
|
||||
|
||||
```yaml
|
||||
database:
|
||||
type: postgresql
|
||||
host: localhost
|
||||
port: 5432
|
||||
database: crypto_trader
|
||||
user: crypto_trader_user
|
||||
password: ${DB_PASSWORD} # Use environment variable
|
||||
```
|
||||
|
||||
### Set Environment Variable
|
||||
|
||||
```bash
|
||||
export DB_PASSWORD='your_password'
|
||||
```
|
||||
|
||||
Or add to `~/.bashrc`:
|
||||
```bash
|
||||
echo 'export DB_PASSWORD="your_password"' >> ~/.bashrc
|
||||
```
|
||||
|
||||
## Migration from SQLite
|
||||
|
||||
### Export from SQLite
|
||||
|
||||
```bash
|
||||
sqlite3 trading.db .dump > dump.sql
|
||||
```
|
||||
|
||||
### Import to PostgreSQL
|
||||
|
||||
```bash
|
||||
psql -U crypto_trader_user -d crypto_trader -f dump.sql
|
||||
```
|
||||
|
||||
## Performance Tuning
|
||||
|
||||
### PostgreSQL Configuration
|
||||
|
||||
Edit `/etc/postgresql/*/main/postgresql.conf`:
|
||||
|
||||
```ini
|
||||
shared_buffers = 256MB
|
||||
effective_cache_size = 1GB
|
||||
maintenance_work_mem = 64MB
|
||||
checkpoint_completion_target = 0.9
|
||||
wal_buffers = 16MB
|
||||
default_statistics_target = 100
|
||||
random_page_cost = 1.1
|
||||
effective_io_concurrency = 200
|
||||
work_mem = 4MB
|
||||
min_wal_size = 1GB
|
||||
max_wal_size = 4GB
|
||||
```
|
||||
|
||||
### Indexes
|
||||
|
||||
Key indexes are created automatically. For custom queries, add indexes:
|
||||
|
||||
```sql
|
||||
CREATE INDEX idx_trades_symbol_date ON trades(symbol, executed_at);
|
||||
CREATE INDEX idx_market_data_symbol_timeframe ON market_data(symbol, timeframe, timestamp);
|
||||
```
|
||||
|
||||
## Backup and Recovery
|
||||
|
||||
### Backup
|
||||
|
||||
```bash
|
||||
pg_dump -U crypto_trader_user crypto_trader > backup.sql
|
||||
```
|
||||
|
||||
### Restore
|
||||
|
||||
```bash
|
||||
psql -U crypto_trader_user crypto_trader < backup.sql
|
||||
```
|
||||
|
||||
### Automated Backups
|
||||
|
||||
Set up cron job:
|
||||
|
||||
```bash
|
||||
0 2 * * * pg_dump -U crypto_trader_user crypto_trader > /backup/crypto_trader_$(date +\%Y\%m\%d).sql
|
||||
```
|
||||
|
||||
## Security
|
||||
|
||||
### Connection Security
|
||||
|
||||
- Use strong passwords
|
||||
- Restrict network access
|
||||
- Use SSL connections for remote access
|
||||
- Regular security updates
|
||||
|
||||
### User Permissions
|
||||
|
||||
- Use dedicated database user
|
||||
- Grant only necessary permissions
|
||||
- Don't use superuser for application
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Connection refused?**
|
||||
- Check PostgreSQL is running: `sudo systemctl status postgresql`
|
||||
- Verify connection settings
|
||||
- Check firewall rules
|
||||
|
||||
**Authentication failed?**
|
||||
- Verify username and password
|
||||
- Check `pg_hba.conf` configuration
|
||||
- Review PostgreSQL logs
|
||||
|
||||
**Performance issues?**
|
||||
- Check PostgreSQL configuration
|
||||
- Review query performance
|
||||
- Add appropriate indexes
|
||||
- Monitor resource usage
|
||||
|
||||
131
docs/deployment/updates.md
Normal file
131
docs/deployment/updates.md
Normal file
@@ -0,0 +1,131 @@
|
||||
# Update Mechanism
|
||||
|
||||
This guide covers the built-in update mechanism for Crypto Trader.
|
||||
|
||||
## Update System
|
||||
|
||||
Crypto Trader includes a built-in update checker and installer for AppImage deployments.
|
||||
|
||||
## Update Check
|
||||
|
||||
### Automatic Check
|
||||
|
||||
Updates are checked on application startup (if enabled):
|
||||
|
||||
```yaml
|
||||
updates:
|
||||
check_on_startup: true
|
||||
repository_url: "https://github.com/user/crypto_trader"
|
||||
```
|
||||
|
||||
### Manual Check
|
||||
|
||||
Check for updates from the application:
|
||||
|
||||
1. Navigate to Help > Check for Updates
|
||||
2. Application checks GitHub releases
|
||||
3. Notifies if update available
|
||||
|
||||
## Update Process
|
||||
|
||||
### Automatic Update
|
||||
|
||||
1. **Check for Updates**
|
||||
- Compares current version with latest release
|
||||
- Downloads update information
|
||||
|
||||
2. **Download Update**
|
||||
- Downloads new AppImage
|
||||
- Shows progress
|
||||
- Verifies download
|
||||
|
||||
3. **Install Update**
|
||||
- Creates backup of current version
|
||||
- Replaces with new version
|
||||
- Makes executable
|
||||
|
||||
4. **Restart**
|
||||
- Prompts to restart
|
||||
- Launches new version
|
||||
|
||||
### Manual Update
|
||||
|
||||
1. Download new AppImage
|
||||
2. Replace old file
|
||||
3. Make executable: `chmod +x crypto_trader-*.AppImage`
|
||||
4. Run new version
|
||||
|
||||
## Version Management
|
||||
|
||||
### Version Format
|
||||
|
||||
Follows semantic versioning: `MAJOR.MINOR.PATCH`
|
||||
|
||||
Example: `1.2.3`
|
||||
|
||||
### Version Comparison
|
||||
|
||||
- **Major**: Breaking changes
|
||||
- **Minor**: New features (backward compatible)
|
||||
- **Patch**: Bug fixes (backward compatible)
|
||||
|
||||
## Update Configuration
|
||||
|
||||
### Disable Auto-Check
|
||||
|
||||
```yaml
|
||||
updates:
|
||||
check_on_startup: false
|
||||
```
|
||||
|
||||
### Custom Repository
|
||||
|
||||
```yaml
|
||||
updates:
|
||||
repository_url: "https://github.com/your-org/crypto_trader"
|
||||
```
|
||||
|
||||
## Update Notifications
|
||||
|
||||
Users are notified when:
|
||||
|
||||
- Update is available on startup
|
||||
- Manual check finds update
|
||||
- Critical security update available
|
||||
|
||||
## Rollback
|
||||
|
||||
If update causes issues:
|
||||
|
||||
1. Locate backup: `crypto_trader-*.AppImage.backup`
|
||||
2. Restore backup:
|
||||
```bash
|
||||
mv crypto_trader-*.AppImage.backup crypto_trader-*.AppImage
|
||||
chmod +x crypto_trader-*.AppImage
|
||||
```
|
||||
3. Run previous version
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Test Updates**: Test updates in development first
|
||||
2. **Backup**: Always backup before updating
|
||||
3. **Release Notes**: Review release notes before updating
|
||||
4. **Staged Rollout**: Consider staged rollout for major updates
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Update check fails?**
|
||||
- Check internet connection
|
||||
- Verify repository URL
|
||||
- Review application logs
|
||||
|
||||
**Download fails?**
|
||||
- Check disk space
|
||||
- Verify download URL
|
||||
- Check network connection
|
||||
|
||||
**Installation fails?**
|
||||
- Check file permissions
|
||||
- Verify AppImage integrity
|
||||
- Review error logs
|
||||
|
||||
262
docs/deployment/web_architecture.md
Normal file
262
docs/deployment/web_architecture.md
Normal file
@@ -0,0 +1,262 @@
|
||||
# Web Architecture Deployment Guide
|
||||
|
||||
This guide covers deploying the new web-based architecture for Crypto Trader.
|
||||
|
||||
## Overview
|
||||
|
||||
The application has been migrated from PyQt6 desktop app to a modern web-based architecture:
|
||||
|
||||
- **Frontend**: React + TypeScript + Material-UI
|
||||
- **Backend**: FastAPI (Python)
|
||||
- **Deployment**: Docker
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Web Browser (Client) │
|
||||
│ React + Material-UI + TypeScript │
|
||||
└─────────────────┬───────────────────────┘
|
||||
│ HTTP/WebSocket
|
||||
┌─────────────────▼───────────────────────┐
|
||||
│ Python Backend API (FastAPI) │
|
||||
│ - Trading Engine (existing code) │
|
||||
│ - Strategy Framework (existing code) │
|
||||
│ - Portfolio Management (existing code) │
|
||||
└─────────────────┬───────────────────────┘
|
||||
│
|
||||
┌─────────────────▼───────────────────────┐
|
||||
│ Database (SQLite/PostgreSQL) │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Development
|
||||
|
||||
1. **Start Backend**:
|
||||
```bash
|
||||
cd backend
|
||||
python -m uvicorn main:app --reload --port 8000
|
||||
```
|
||||
|
||||
2. **Start Frontend**:
|
||||
```bash
|
||||
cd frontend
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
|
||||
3. **Access Application**:
|
||||
- Frontend: http://localhost:3000
|
||||
- API Docs: http://localhost:8000/docs
|
||||
- API: http://localhost:8000/api
|
||||
|
||||
### Docker Deployment
|
||||
|
||||
1. **Build and Run**:
|
||||
```bash
|
||||
docker-compose up --build
|
||||
```
|
||||
|
||||
2. **Access Application**:
|
||||
- Application: http://localhost:8000
|
||||
- API Docs: http://localhost:8000/docs
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Trading
|
||||
- `POST /api/trading/orders` - Create order
|
||||
- `GET /api/trading/orders` - List orders
|
||||
- `GET /api/trading/orders/{id}` - Get order
|
||||
- `POST /api/trading/orders/{id}/cancel` - Cancel order
|
||||
- `GET /api/trading/positions` - Get positions
|
||||
- `GET /api/trading/balance` - Get balance
|
||||
|
||||
### Portfolio
|
||||
- `GET /api/portfolio/current` - Get current portfolio
|
||||
- `GET /api/portfolio/history` - Get portfolio history
|
||||
- `POST /api/portfolio/positions/update-prices` - Update prices
|
||||
|
||||
### Strategies
|
||||
- `GET /api/strategies/` - List strategies
|
||||
- `GET /api/strategies/available` - List available strategy types
|
||||
- `POST /api/strategies/` - Create strategy
|
||||
- `GET /api/strategies/{id}` - Get strategy
|
||||
- `PUT /api/strategies/{id}` - Update strategy
|
||||
- `DELETE /api/strategies/{id}` - Delete strategy
|
||||
- `POST /api/strategies/{id}/start` - Start strategy
|
||||
- `POST /api/strategies/{id}/stop` - Stop strategy
|
||||
|
||||
### Backtesting
|
||||
- `POST /api/backtesting/run` - Run backtest
|
||||
- `GET /api/backtesting/results/{id}` - Get backtest results
|
||||
|
||||
### Exchanges
|
||||
- `GET /api/exchanges/` - List exchanges
|
||||
- `GET /api/exchanges/{id}` - Get exchange
|
||||
|
||||
### WebSocket
|
||||
- `WS /ws/` - WebSocket connection for real-time updates
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Create `.env` file:
|
||||
|
||||
```env
|
||||
# API Configuration
|
||||
VITE_API_URL=http://localhost:8000
|
||||
VITE_WS_URL=ws://localhost:8000/ws/
|
||||
|
||||
# Database (optional)
|
||||
DATABASE_URL=sqlite:///./data/crypto_trader.db
|
||||
```
|
||||
|
||||
### Docker Environment
|
||||
|
||||
Environment variables can be set in `docker-compose.yml`:
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
- DATABASE_URL=sqlite:///./data/crypto_trader.db
|
||||
- PYTHONPATH=/app
|
||||
```
|
||||
|
||||
## Data Persistence
|
||||
|
||||
Data is stored in the `./data` directory, which is mounted as a volume in Docker:
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Backend Development
|
||||
|
||||
1. Install dependencies:
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
pip install -r backend/requirements.txt
|
||||
```
|
||||
|
||||
2. Run with hot-reload:
|
||||
```bash
|
||||
python -m uvicorn backend.main:app --reload
|
||||
```
|
||||
|
||||
### Frontend Development
|
||||
|
||||
1. Install dependencies:
|
||||
```bash
|
||||
cd frontend
|
||||
npm install
|
||||
```
|
||||
|
||||
2. Run development server:
|
||||
```bash
|
||||
npm run dev
|
||||
```
|
||||
|
||||
3. Build for production:
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
## Production Deployment
|
||||
|
||||
### Docker Production
|
||||
|
||||
1. Build production image:
|
||||
```bash
|
||||
docker build -t crypto-trader:latest .
|
||||
```
|
||||
|
||||
2. Run container:
|
||||
```bash
|
||||
docker run -d \
|
||||
-p 8000:8000 \
|
||||
-v $(pwd)/data:/app/data \
|
||||
-v $(pwd)/config:/app/config \
|
||||
--name crypto-trader \
|
||||
crypto-trader:latest
|
||||
```
|
||||
|
||||
### Reverse Proxy (Nginx)
|
||||
|
||||
Example Nginx configuration:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name crypto-trader.example.com;
|
||||
|
||||
location / {
|
||||
proxy_pass http://localhost:8000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
}
|
||||
|
||||
location /ws/ {
|
||||
proxy_pass http://localhost:8000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Migration from PyQt6
|
||||
|
||||
The existing Python code has been preserved:
|
||||
|
||||
- Trading engine (`src/trading/`)
|
||||
- Strategy framework (`src/strategies/`)
|
||||
- Portfolio tracker (`src/portfolio/`)
|
||||
- Backtesting engine (`src/backtesting/`)
|
||||
- All other core modules
|
||||
|
||||
Only the UI layer has been replaced with a web interface.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Backend Issues
|
||||
|
||||
- **Port already in use**: Change port in `docker-compose.yml` or use `--port` flag
|
||||
- **Database errors**: Check database file permissions in `./data` directory
|
||||
- **Import errors**: Ensure `PYTHONPATH=/app` is set
|
||||
|
||||
### Frontend Issues
|
||||
|
||||
- **API connection errors**: Check `VITE_API_URL` in `.env` file
|
||||
- **WebSocket connection fails**: Verify WebSocket URL and backend is running
|
||||
- **Build errors**: Clear `node_modules` and reinstall: `rm -rf node_modules && npm install`
|
||||
|
||||
### Docker Issues
|
||||
|
||||
- **Build fails**: Check Dockerfile syntax and dependencies
|
||||
- **Container won't start**: Check logs: `docker-compose logs`
|
||||
- **Volume permissions**: Ensure `./data` directory is writable
|
||||
|
||||
## Benefits of Web Architecture
|
||||
|
||||
1. **Modern UI**: Access to entire web ecosystem (Material-UI, charts, etc.)
|
||||
2. **Cross-platform**: Works on any device with a browser
|
||||
3. **Easier deployment**: Docker is simpler than AppImage
|
||||
4. **Better development**: Hot-reload, better tooling
|
||||
5. **Maintainability**: Easier to update and deploy
|
||||
6. **Accessibility**: Access from anywhere via browser
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Add authentication (JWT tokens)
|
||||
2. Implement WebSocket price updates
|
||||
3. Add more charting capabilities
|
||||
4. Enhance strategy management UI
|
||||
5. Add mobile-responsive design
|
||||
Reference in New Issue
Block a user