crypto_trader/docs/developer/setup.md

# Development Environment Setup

This guide will help you set up a development environment for Crypto Trader.

## Prerequisites

- Python 3.11 or higher
- Node.js 18 or higher
- Git
- Virtual environment tool (venv or virtualenv)
- Code editor (VS Code, PyCharm, etc.)
- PostgreSQL 14 or higher
- Redis 5.0 or higher

## Initial Setup

### 1. Clone the Repository

```bash
git clone <repository-url>
cd crypto_trader
```

### 2. Create Virtual Environment

```bash
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
```

### 3. Install Python Dependencies

```bash
# Install main dependencies
pip install -r requirements.txt

# Install development dependencies
pip install -r tests/requirements.txt

# Install documentation dependencies
pip install -r docs/requirements.txt
```

### 4. Install Frontend Dependencies

```bash
cd frontend
npm install
cd ..
```

### 5. Install Pre-commit Hooks (Optional)

```bash
pip install pre-commit
pre-commit install
```

## Development Tools

### Recommended IDE Setup

**VS Code**:
- Python extension
- Pylance for type checking
- Python Test Explorer
- ESLint and Prettier for frontend
- YAML extension

**PyCharm**:
- Configure Python interpreter
- Set up test runner
- Configure code style
- Enable JavaScript/TypeScript support

### Code Quality Tools

```bash
# Install linting and formatting tools
pip install black flake8 mypy pylint

# Format code
black src/ tests/

# Lint code
flake8 src/ tests/

# Type checking
mypy src/
```

## Database Setup

### PostgreSQL (Required)

You must have a PostgreSQL instance running for development.

```bash
# Install PostgreSQL
# Create development database
createdb crypto_trader_dev

# Update config.yaml or set env var
# DATABASE_URL=postgresql+asyncpg://user:password@localhost/crypto_trader_dev
```

### SQLite (Internal)

Used internally for unit tests (in-memory) only. No setup required.

## Redis Setup

Redis is required for distributed state management and Celery background tasks (e.g., ML model retraining).

```bash
# Install Redis (Ubuntu/Debian)
sudo apt-get install redis-server
```

**Starting Redis**:

```bash
# Option 1: Using system service (requires sudo)
sudo service redis-server start

# Option 2: Direct daemon mode (for containers/restricted environments)
redis-server --daemonize yes

# Verify
redis-cli ping  # Should return PONG
```

> **Note**: In containerized environments (Toolbox, Distrobox, Docker, etc.) where `sudo` is not available, use the direct daemon mode option. If you see "Connection refused" errors when using features like "Retrain Model", Redis is likely not running.

### Default Configuration

Redis defaults to `localhost:6379`. Override in `config.yaml`:

```yaml
redis:
  host: "127.0.0.1"
  port: 6379
  db: 0
```

## Running the Application

### Start All Services (Recommended)

Use the helper script to start all services:

```bash
./scripts/start_all.sh
```

### Start Services Manually

```bash
# 1. Start Redis
# Use sudo if available, otherwise direct daemon mode
sudo service redis-server start  # OR: redis-server --daemonize yes

# 2. Start Celery Worker (background tasks)
celery -A src.worker.app worker --loglevel=info &

# 3. Start Backend API
uvicorn backend.main:app --reload --host 0.0.0.0 --port 8000 &

# 4. Start Frontend
cd frontend && npm run dev
```

### Access Points

- **Frontend**: http://localhost:3000
- **Backend API**: http://localhost:8000
- **API Docs**: http://localhost:8000/docs

### Verify Redis/Celery Integration

```bash
python scripts/verify_redis.py
```

## Running Tests

```bash
# Run all tests
pytest

# Run with coverage
pytest --cov=src --cov-report=html

# Run specific test file
pytest tests/unit/core/test_redis.py

# Run Redis/Celery tests
pytest tests/unit/core/test_redis.py tests/unit/worker/test_tasks.py -v

# Run with verbose output
pytest -v
```

## Project Structure

```
crypto_trader/
├── src/                    # Backend source code
│   ├── autopilot/          # Intelligent autopilot
│   ├── core/               # Core utilities (config, redis, logging)
│   ├── strategies/         # Trading strategies
│   ├── trading/            # Trading engine
│   └── worker/             # Celery tasks
├── backend/                # FastAPI application
│   ├── api/                # API endpoints
│   └── main.py             # Application entry point
├── frontend/               # React frontend
│   ├── src/                # React source
│   └── package.json        # Frontend dependencies
├── tests/                  # Test suite
├── scripts/                # Utility scripts
│   ├── start_all.sh        # Start all services
│   └── verify_redis.py     # Verify Redis/Celery
├── docs/                   # Documentation
└── requirements.txt        # Python dependencies
```

## Common Development Tasks

### Adding a New Celery Task

1. Add task function in `src/worker/tasks.py`
2. Configure task routing in `src/worker/app.py` (optional)
3. Create API endpoint in `backend/api/`
4. Write unit tests in `tests/unit/worker/`

### Adding a New API Endpoint

1. Create/update router in `backend/api/`
2. Register router in `backend/main.py`
3. Add frontend API client in `frontend/src/api/`
4. Write tests

### Running Documentation

```bash
cd docs/api
make html
# Open build/html/index.html
```

### Building AppImage

```bash
./packaging/build_appimage.sh
```

## Debugging

### Enable Debug Logging

Set in `config.yaml`:
```yaml
logging:
  level: DEBUG
```

### Using Debugger

```bash
# VS Code: Set breakpoints and use debugger
# PyCharm: Configure debug configuration
# Command line: Use pdb
python -m pdb -c continue backend/main.py
```

### Checking Celery Tasks

```bash
# Monitor tasks
celery -A src.worker.app events

# Inspect active tasks
celery -A src.worker.app inspect active
```

## Troubleshooting

**Import errors?**
- Verify virtual environment is activated
- Check PYTHONPATH
- Reinstall dependencies

**Redis connection failed / "Connection refused" error?**
- Ensure Redis is running: `redis-cli ping` (should return `PONG`)
- Start Redis if not running:
  - With sudo: `sudo service redis-server start`
  - Without sudo: `redis-server --daemonize yes`
- Check host/port configuration in `config.yaml`
- Verify firewall rules

**Celery tasks not executing?**
- Ensure worker is running: `ps aux | grep celery`
- Check worker logs: `tail -f celery.log`
- Verify Redis is accessible

**Tests failing?**
- Check test database setup
- Verify test fixtures
- Review test logs

**Frontend not connecting?**
- Check API is running on port 8000
- Verify CORS settings in backend
- Check browser console for errors