quality_app/documentation/DATABASE_RESTORE_GUIDE.md

# Database Restore Guide

## Overview
The database restore functionality allows superadmins to restore the entire database from a backup file. This is essential for:
- **Server Migration**: Moving the application to a new server
- **Disaster Recovery**: Recovering from data corruption or loss
- **Testing/Development**: Restoring production data to test environment
- **Rollback**: Reverting to a previous state after issues

## ⚠️ CRITICAL WARNINGS

### Data Loss Risk
- **ALL CURRENT DATA WILL BE PERMANENTLY DELETED**
- The restore operation is **IRREVERSIBLE**
- Once started, it cannot be stopped
- No "undo" functionality exists

### Downtime Requirements
- Users may experience brief downtime during restore
- All database connections will be terminated
- Active sessions may be invalidated
- Plan restores during maintenance windows

### Access Requirements
- **SUPERADMIN ACCESS ONLY**
- No other role has restore permissions
- This is by design for safety

## Large Database Support

### Supported File Sizes
The backup system is optimized for databases of all sizes:

- ✅ **Small databases** (< 100MB): Full validation, fast operations
- ✅ **Medium databases** (100MB - 2GB): Partial validation (first 10MB), normal operations
- ✅ **Large databases** (2GB - 10GB): Basic validation only, longer operations
- ✅ **Very large databases** (> 10GB): Can be configured by increasing limits

### Upload Limits
- **Maximum upload size**: 10GB
- **Warning threshold**: 1GB (user confirmation required)
- **Timeout**: 30 minutes for upload + validation + restore

### Performance Estimates

| Database Size | Backup Creation | Upload Time* | Validation | Restore Time |
|--------------|----------------|-------------|-----------|--------------|
| 100MB | ~5 seconds | ~10 seconds | ~1 second | ~15 seconds |
| 500MB | ~15 seconds | ~1 minute | ~2 seconds | ~45 seconds |
| 1GB | ~30 seconds | ~2 minutes | ~3 seconds | ~2 minutes |
| 5GB | ~2-3 minutes | ~10-15 minutes | ~1 second | ~10 minutes |
| 10GB | ~5-7 minutes | ~25-35 minutes | ~1 second | ~20 minutes |

*Upload times assume 100Mbps network connection

### Smart Validation
The system intelligently adjusts validation based on file size:

**Small Files (< 100MB)**:
- Full line-by-line validation
- Checks for users table, INSERT statements, database structure
- Detects suspicious commands

**Medium Files (100MB - 2GB)**:
- Validates only first 10MB in detail
- Quick structure check
- Performance optimized (~1-3 seconds)

**Large Files (2GB - 10GB)**:
- Basic validation only (file size, extension)
- Skips detailed content check for performance
- Validation completes in ~1 second
- Message: "Large backup file accepted - detailed validation skipped for performance"

### Memory Efficiency
All backup operations use **streaming** - no memory concerns:
- ✅ **Backup creation**: mysqldump streams directly to disk
- ✅ **File upload**: Saved directly to disk (no RAM buffering)
- ✅ **Restore**: mysql reads from disk in chunks
- ✅ **Memory usage**: < 100MB regardless of database size

### System Requirements

**For 5GB Database**:
- **Disk space**: 10GB free (2x database size)
- **Memory**: < 100MB (streaming operations)
- **Network**: 100Mbps or faster recommended
- **Time**: ~30 minutes total (upload + restore)

**For 10GB Database**:
- **Disk space**: 20GB free
- **Memory**: < 100MB
- **Network**: 1Gbps recommended
- **Time**: ~1 hour total

## How to Restore Database

### Step 1: Access Settings Page
1. Log in as **superadmin**
2. Navigate to **Settings** page
3. Scroll down to **Database Backup Management** section
4. Find the **⚠️ Restore Database** section (orange warning box)

### Step 2: Upload or Select Backup File

**Option A: Upload External Backup**
1. Click **"📁 Choose File"** in the Upload section
2. Select your .sql backup file (up to 10GB)
3. If file is > 1GB, confirm the upload warning
4. Click **"⬆️ Upload File"** button
5. Wait for upload and validation (shows progress)
6. File appears in restore dropdown once complete

**Option B: Use Existing Backup**
1. Skip upload if backup already exists on server
2. Proceed directly to dropdown selection

### Step 3: Select Backup from Dropdown
1. Click the dropdown: **"Select Backup to Restore"**
2. Choose from available backup files
   - Files are listed with size and creation date
   - Example: `backup_trasabilitate_20251103_212929.sql (318 KB - 2025-11-03 21:29:29)`
   - Uploaded files: `backup_uploaded_20251103_214500_mybackup.sql (5.2 GB - ...)`
3. The **Restore Database** button will enable once selected

### Step 4: Confirm Restore (Double Confirmation)

#### First Confirmation Dialog
```
⚠️ CRITICAL WARNING ⚠️

You are about to RESTORE the database from:
backup_trasabilitate_20251103_212929.sql

This will PERMANENTLY DELETE all current data and replace it with the backup data.

This action CANNOT be undone!

Do you want to continue?
```
- Click **OK** to proceed or **Cancel** to abort

#### Second Confirmation (Type-to-Confirm)
```
⚠️ FINAL CONFIRMATION ⚠️

Type "RESTORE" in capital letters to confirm you understand:
• All current database data will be PERMANENTLY DELETED
• This action is IRREVERSIBLE
• Users may experience downtime during restore

Type RESTORE to continue:
```
- Type exactly: **RESTORE** (all capitals)
- Any other text will cancel the operation

### Step 4: Restore Process
1. Button changes to: **"⏳ Restoring database... Please wait..."**
2. Backend performs restore operation:
   - Drops existing database
   - Creates new empty database
   - Imports backup SQL file
   - Verifies restoration
3. On success:
   - Success message displays
   - Page automatically reloads
   - All data is now from the backup file

## UI Features

### Visual Safety Indicators
- **Orange Warning Box**: Highly visible restore section
- **Warning Icons**: ⚠️ symbols throughout
- **Explicit Text**: Clear warnings about data loss
- **Color Coding**: Orange (#ff9800) for danger

### Dark Mode Support
- Restore section adapts to dark theme
- Warning colors remain visible in both modes
- Light mode: Light orange background (#fff3e0)
- Dark mode: Dark brown background (#3a2a1f) with orange text

### Button States
- **Disabled**: Grey button when no backup selected
- **Enabled**: Red button (#ff5722) when backup selected
- **Processing**: Loading indicator during restore

## Technical Implementation

### API Endpoint
```
POST /api/backup/restore/<filename>
```

**Access Control**: `@superadmin_only` decorator

**Parameters**:
- `filename`: Name of backup file to restore (in URL path)

**Response**:
```json
{
    "success": true,
    "message": "Database restored successfully from backup_trasabilitate_20251103_212929.sql"
}
```

### Backend Process (DatabaseBackupManager.restore_backup)

```python
def restore_backup(self, filename: str) -> dict:
    """
    Restore database from a backup file
    
    Process:
    1. Verify backup file exists
    2. Drop existing database
    3. Create new database
    4. Import SQL dump
    5. Grant permissions
    6. Verify restoration
    """
```

**Commands Executed**:
```sql
-- Drop existing database
DROP DATABASE IF EXISTS trasabilitate;

-- Create new database
CREATE DATABASE trasabilitate;

-- Import backup (via mysql command)
mysql trasabilitate < /srv/quality_app/backups/backup_trasabilitate_20251103_212929.sql

-- Grant permissions
GRANT ALL PRIVILEGES ON trasabilitate.* TO 'your_user'@'localhost';
FLUSH PRIVILEGES;
```

### Security Features
1. **Double Confirmation**: Prevents accidental restores
2. **Type-to-Confirm**: Requires typing "RESTORE" exactly
3. **Superadmin Only**: No other roles can access
4. **Audit Trail**: All restores logged in error.log
5. **Session Check**: Requires valid superadmin session

## Server Migration Procedure

### Migrating to New Server

#### On Old Server:
1. **Create Final Backup**
   - Go to Settings → Database Backup Management
   - Click **⚡ Backup Now**
   - Wait for backup to complete (see performance estimates above)
   - Download the backup file (⬇️ Download button)
   - Save file securely (e.g., `backup_trasabilitate_20251103.sql`)
   - **Note**: Large databases (5GB+) will take 5-10 minutes to backup

2. **Stop Application** (optional but recommended)
   ```bash
   cd /srv/quality_app/py_app
   bash stop_production.sh
   ```

#### On New Server:
1. **Install Application**
   - Clone repository
   - Set up Python environment
   - Install dependencies
   - Configure `external_server.conf`

2. **Initialize Empty Database**
   ```bash
   sudo mysql -e "CREATE DATABASE trasabilitate;"
   sudo mysql -e "GRANT ALL PRIVILEGES ON trasabilitate.* TO 'your_user'@'localhost';"
   ```

3. **Transfer Backup File**

   **Option A: Direct Upload via UI** (Recommended for files < 5GB)
   - Start application
   - Login as superadmin → Settings
   - Use **"Upload Backup File"** section
   - Select your backup file (up to 10GB supported)
   - System will validate and add to restore list automatically
   - **Estimated time**: 10-30 minutes for 5GB file on 100Mbps network

   **Option B: Manual Copy** (Faster for very large files)
   - Copy backup file directly to server: `scp backup_file.sql user@newserver:/srv/quality_app/backups/`
   - Or use external storage/USB drive
   - Ensure permissions: `chmod 644 /srv/quality_app/backups/backup_*.sql`
   - File appears in restore dropdown immediately

4. **Start Application** (if not already running)
   ```bash
   cd /srv/quality_app/py_app
   bash start_production.sh
   ```

5. **Restore Database via UI**
   - Log in as superadmin
   - Go to Settings → Database Backup Management
   - **Upload Section**: Upload file OR skip if already copied
   - **Restore Section**: Select backup from dropdown
   - Click **Restore Database**
   - Complete double-confirmation
   - Wait for restore to complete
   - **Estimated time**: 5-20 minutes for 5GB database

6. **Verify Migration**
   - Check that all users exist
   - Verify data integrity
   - Test all modules (Quality, Warehouse, Labels, Daily Mirror)
   - Confirm permissions are correct

### Large Database Migration Tips

**For Databases > 5GB**:
1. ✅ Use **Manual Copy** (Option B) instead of upload - Much faster
2. ✅ Schedule migration during **off-hours** to avoid user impact
3. ✅ Expect **30-60 minutes** total time for 10GB database
4. ✅ Ensure **sufficient disk space** (2x database size)
5. ✅ Monitor progress in logs: `tail -f /srv/quality_app/logs/error.log`
6. ✅ Keep old server running until verification complete

**Network Transfer Time Examples**:
- 5GB @ 100Mbps network: ~7 minutes via scp, ~15 minutes via browser upload
- 5GB @ 1Gbps network: ~40 seconds via scp, ~2 minutes via browser upload
- 10GB @ 100Mbps network: ~14 minutes via scp, ~30 minutes via browser upload

### Alternative: Command-Line Restore

If UI is not available, restore manually:

```bash
# Stop application
cd /srv/quality_app/py_app
bash stop_production.sh

# Drop and recreate database
sudo mysql -e "DROP DATABASE IF EXISTS trasabilitate;"
sudo mysql -e "CREATE DATABASE trasabilitate;"

# Restore from backup
sudo mysql trasabilitate < /srv/quality_app/backups/backup_trasabilitate_20251103.sql

# Grant permissions
sudo mysql -e "GRANT ALL PRIVILEGES ON trasabilitate.* TO 'your_user'@'localhost';"
sudo mysql -e "FLUSH PRIVILEGES;"

# Restart application
bash start_production.sh
```

## Troubleshooting

### Error: "Backup file not found"
**Cause**: Selected backup file doesn't exist in backup directory

**Solution**:
```bash
# Check backup directory
ls -lh /srv/quality_app/backups/

# Verify file exists and is readable
ls -l /srv/quality_app/backups/backup_trasabilitate_*.sql
```

### Error: "Permission denied"
**Cause**: Insufficient MySQL privileges

**Solution**:
```bash
# Grant all privileges to database user
sudo mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'your_user'@'localhost';"
sudo mysql -e "FLUSH PRIVILEGES;"
```

### Error: "Database connection failed"
**Cause**: MySQL server not running or wrong credentials

**Solution**:
```bash
# Check MySQL status
sudo systemctl status mariadb

# Verify credentials in external_server.conf
cat /srv/quality_app/py_app/instance/external_server.conf

# Test connection
mysql -u your_user -p -e "SELECT 1;"
```

### Error: "Restore partially completed"
**Cause**: SQL syntax errors in backup file

**Solution**:
1. Check error logs:
   ```bash
   tail -f /srv/quality_app/logs/error.log
   ```
2. Try manual restore to see specific errors:
   ```bash
   sudo mysql trasabilitate < backup_file.sql
   ```
3. Fix issues in backup file if possible
4. Create new backup from source database

### Application Won't Start After Restore
**Cause**: Database structure mismatch or missing tables

**Solution**:
```bash
# Verify all tables exist
mysql trasabilitate -e "SHOW TABLES;"

# Check for specific required tables
mysql trasabilitate -e "SELECT COUNT(*) FROM users;"

# If tables missing, restore from a known-good backup
```

## Best Practices

### Before Restoring
1. ✅ **Create a current backup** before restoring older one
2. ✅ **Notify users** of planned downtime
3. ✅ **Test restore** in development environment first
4. ✅ **Verify backup integrity** (download and check file)
5. ✅ **Plan rollback strategy** if restore fails

### During Restore
1. ✅ **Monitor logs** in real-time:
   ```bash
   tail -f /srv/quality_app/logs/error.log
   ```
2. ✅ **Don't interrupt** the process
3. ✅ **Keep backup window** as short as possible

### After Restore
1. ✅ **Verify data** integrity
2. ✅ **Test all features** (login, modules, reports)
3. ✅ **Check user permissions** are correct
4. ✅ **Monitor application** for errors
5. ✅ **Document restore** in change log

## Related Documentation
- [DATABASE_BACKUP_GUIDE.md](DATABASE_BACKUP_GUIDE.md) - Creating backups
- [DATABASE_DOCKER_SETUP.md](DATABASE_DOCKER_SETUP.md) - Database configuration
- [DOCKER_DEPLOYMENT.md](../old%20code/DOCKER_DEPLOYMENT.md) - Deployment procedures

## Summary
The restore functionality provides a safe and reliable way to restore database backups for server migration and disaster recovery. The double-confirmation system prevents accidental data loss, while the UI provides clear visibility into available backups. Always create a current backup before restoring, and test the restore process in a non-production environment when possible.