quality_app/DATABASE_DOCKER_SETUP.md

# Database Setup for Docker Deployment

## Overview
The Recticel Quality Application uses a **dual-database approach**:
1. **MariaDB** (Primary) - Production data, users, permissions, orders
2. **SQLite** (Backup/Legacy) - Local user authentication fallback

## Database Configuration Flow

### 1. Docker Environment Variables → Database Connection

```
Docker .env file
    ↓
docker-compose.yml (environment section)
    ↓
Docker container environment variables
    ↓
setup_complete_database.py (reads from env)
    ↓
external_server.conf file (generated)
    ↓
Application runtime (reads conf file)
```

### 2. Environment Variables Used

| Variable | Default | Purpose | Used By |
|----------|---------|---------|---------|
| `DB_HOST` | `db` | Database server hostname | All DB operations |
| `DB_PORT` | `3306` | MariaDB port | All DB operations |
| `DB_NAME` | `trasabilitate` | Database name | All DB operations |
| `DB_USER` | `trasabilitate` | Database username | All DB operations |
| `DB_PASSWORD` | `Initial01!` | Database password | All DB operations |
| `MYSQL_ROOT_PASSWORD` | `rootpassword` | MariaDB root password | DB initialization |
| `INIT_DB` | `true` | Run schema setup | docker-entrypoint.sh |
| `SEED_DB` | `true` | Create superadmin user | docker-entrypoint.sh |

### 3. Database Initialization Process

#### Phase 1: MariaDB Container Startup
```bash
# docker-compose.yml starts MariaDB container
# init-db.sql runs automatically:
1. CREATE DATABASE trasabilitate
2. CREATE USER 'trasabilitate'@'%'
3. GRANT ALL PRIVILEGES
```

#### Phase 2: Application Container Waits
```bash
# docker-entrypoint.sh:
1. Waits for MariaDB to be ready (health check)
2. Tests connection with credentials
3. Retries up to 60 times (2s intervals = 120s timeout)
```

#### Phase 3: Configuration File Generation
```bash
# docker-entrypoint.sh creates:
/app/instance/external_server.conf
    server_domain=db          # From DB_HOST
    port=3306                  # From DB_PORT
    database_name=trasabilitate # From DB_NAME
    username=trasabilitate     # From DB_USER
    password=Initial01!        # From DB_PASSWORD
```

#### Phase 4: Schema Creation (if INIT_DB=true)
```bash
# setup_complete_database.py creates:
- scan1_orders (quality scans - station 1)
- scanfg_orders (quality scans - finished goods)
- order_for_labels (production orders for labels)
- warehouse_locations (warehouse management)
- users (user authentication)
- roles (user roles)
- permissions (permission definitions)
- role_permissions (role-permission mappings)
- role_hierarchy (role inheritance)
- permission_audit_log (permission change tracking)

# Also creates triggers:
- increment_approved_quantity (auto-count approved items)
- increment_approved_quantity_fg (auto-count finished goods)
```

#### Phase 5: Data Seeding (if SEED_DB=true)
```bash
# seed.py creates:
- Superadmin user (username: superadmin, password: superadmin123)

# setup_complete_database.py also creates:
- Default permission set (35+ permissions)
- Role hierarchy (7 roles: superadmin → admin → manager → workers)
- Role-permission mappings
```

### 4. How Application Connects to Database

#### A. Settings Module (app/settings.py)
```python
def get_external_db_connection():
    # Reads /app/instance/external_server.conf
    # Returns mariadb.connect() using conf values
```

#### B. Other Modules (order_labels.py, print_module.py, warehouse.py)
```python
def get_db_connection():
    # Also reads external_server.conf
    # Each module manages its own connections
```

#### C. SQLAlchemy (app/__init__.py)
```python
# Currently hardcoded to SQLite (NOT DOCKER-FRIENDLY!)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///users.db'
```

## Current Issues & Recommendations

### ❌ Problem 1: Hardcoded SQLite in __init__.py
**Issue:** `app/__init__.py` uses hardcoded SQLite connection
```python
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///users.db'
```

**Impact:** 
- Not using environment variables
- SQLAlchemy not connected to MariaDB
- Inconsistent with external_server.conf approach

**Solution:** Update to read from environment:
```python
import os

def create_app():
    app = Flask(__name__)
    
    # Database configuration from environment
    db_user = os.getenv('DB_USER', 'trasabilitate')
    db_pass = os.getenv('DB_PASSWORD', 'Initial01!')
    db_host = os.getenv('DB_HOST', 'localhost')
    db_port = os.getenv('DB_PORT', '3306')
    db_name = os.getenv('DB_NAME', 'trasabilitate')
    
    # Use MariaDB/MySQL connection
    app.config['SQLALCHEMY_DATABASE_URI'] = (
        f'mysql+mariadb://{db_user}:{db_pass}@{db_host}:{db_port}/{db_name}'
    )
```

### ❌ Problem 2: Dual Connection Methods
**Issue:** Application uses two different connection methods:
1. SQLAlchemy ORM (for User model)
2. Direct mariadb.connect() (for everything else)

**Impact:**
- Complexity in maintenance
- Potential connection pool exhaustion
- Inconsistent transaction handling

**Recommendation:** Standardize on one approach:
- **Option A:** Use SQLAlchemy for everything (preferred)
- **Option B:** Use direct mariadb connections everywhere

### ❌ Problem 3: external_server.conf Redundancy
**Issue:** Configuration is duplicated:
1. Environment variables → external_server.conf
2. Application reads external_server.conf

**Impact:**
- Unnecessary file I/O
- Potential sync issues
- Not 12-factor app compliant

**Recommendation:** Read directly from environment variables

## Docker Deployment Database Schema

### MariaDB Container Configuration
```yaml
# docker-compose.yml
db:
  image: mariadb:11.3
  environment:
    MYSQL_ROOT_PASSWORD: rootpassword
    MYSQL_DATABASE: trasabilitate
    MYSQL_USER: trasabilitate
    MYSQL_PASSWORD: Initial01!
  volumes:
    - /srv/docker-test/mariadb:/var/lib/mysql  # Persistent storage
    - ./init-db.sql:/docker-entrypoint-initdb.d/01-init.sql
```

### Database Tables Created

| Table | Purpose | Records |
|-------|---------|---------|
| `scan1_orders` | Quality scan records (station 1) | 1000s |
| `scanfg_orders` | Finished goods scan records | 1000s |
| `order_for_labels` | Production orders needing labels | 100s |
| `warehouse_locations` | Warehouse location codes | 50-200 |
| `users` | User accounts | 10-50 |
| `roles` | Role definitions | 7 |
| `permissions` | Permission definitions | 35+ |
| `role_permissions` | Role-permission mappings | 100+ |
| `role_hierarchy` | Role inheritance tree | 7 |
| `permission_audit_log` | Permission change audit trail | Growing |

### Default Users & Roles

**Superadmin User:**
- Username: `superadmin`
- Password: `superadmin123`
- Role: `superadmin`
- Access: Full system access

**Role Hierarchy:**
```
superadmin (level 1)
  └─ admin (level 2)
      └─ manager (level 3)
          ├─ quality_manager (level 4)
          │   └─ quality_worker (level 5)
          └─ warehouse_manager (level 4)
              └─ warehouse_worker (level 5)
```

## Production Deployment Checklist

- [ ] Change `MYSQL_ROOT_PASSWORD` from default
- [ ] Change `DB_PASSWORD` from default (Initial01!)
- [ ] Change superadmin password from default (superadmin123)
- [ ] Set `INIT_DB=false` after first deployment
- [ ] Set `SEED_DB=false` after first deployment
- [ ] Set strong `SECRET_KEY` in environment
- [ ] Backup MariaDB data directory regularly
- [ ] Enable MariaDB binary logging for point-in-time recovery
- [ ] Configure proper `DB_MAX_RETRIES` and `DB_RETRY_INTERVAL`
- [ ] Monitor database connections and performance
- [ ] Set up database user with minimal required privileges

## Troubleshooting

### Database Connection Failed
```bash
# Check if MariaDB container is running
docker-compose ps

# Check MariaDB logs
docker-compose logs db

# Test connection from app container
docker-compose exec web python3 -c "
import mariadb
conn = mariadb.connect(
    user='trasabilitate',
    password='Initial01!',
    host='db',
    port=3306,
    database='trasabilitate'
)
print('Connection successful!')
"
```

### Tables Not Created
```bash
# Run setup script manually
docker-compose exec web python3 /app/app/db_create_scripts/setup_complete_database.py

# Check tables
docker-compose exec db mysql -utrasabilitate -pInitial01! trasabilitate -e "SHOW TABLES;"
```

### external_server.conf Not Found
```bash
# Verify file exists
docker-compose exec web cat /app/instance/external_server.conf

# Recreate if missing (entrypoint should do this automatically)
docker-compose restart web
```

## Migration from Non-Docker to Docker

If migrating from a non-Docker deployment:

1. **Backup existing MariaDB database:**
   ```bash
   mysqldump -u trasabilitate -p trasabilitate > backup.sql
   ```

2. **Update docker-compose.yml paths to existing data:**
   ```yaml
   db:
     volumes:
       - /path/to/existing/mariadb:/var/lib/mysql
   ```

3. **Or restore to new Docker MariaDB:**
   ```bash
   docker-compose exec -T db mysql -utrasabilitate -pInitial01! trasabilitate < backup.sql
   ```

4. **Verify data:**
   ```bash
   docker-compose exec db mysql -utrasabilitate -pInitial01! trasabilitate -e "SELECT COUNT(*) FROM users;"
   ```

## Environment Variable Examples

### Development (.env)
```bash
DB_HOST=db
DB_PORT=3306
DB_NAME=trasabilitate
DB_USER=trasabilitate
DB_PASSWORD=Initial01!
MYSQL_ROOT_PASSWORD=rootpassword
INIT_DB=true
SEED_DB=true
FLASK_ENV=development
GUNICORN_LOG_LEVEL=debug
```

### Production (.env)
```bash
DB_HOST=db
DB_PORT=3306
DB_NAME=trasabilitate
DB_USER=trasabilitate
DB_PASSWORD=SuperSecurePassword123!@#
MYSQL_ROOT_PASSWORD=SuperSecureRootPass456!@#
INIT_DB=false
SEED_DB=false
FLASK_ENV=production
GUNICORN_LOG_LEVEL=info
SECRET_KEY=your-super-secret-key-change-this
```