Fixed the scan error and backup problems
This commit is contained in:
Quality System Admin
484
documentation/BACKUP_SCHEDULE_FEATURE.md
Normal file
484
documentation/BACKUP_SCHEDULE_FEATURE.md
Normal file
@@ -0,0 +1,484 @@
|
||||
# Backup Schedule Feature - Complete Guide
|
||||
|
||||
## Overview
|
||||
|
||||
The backup schedule feature allows administrators to configure automated backups that run at specified times with customizable frequency. This ensures regular, consistent backups without manual intervention.
|
||||
|
||||
**Added:** November 5, 2025
|
||||
**Version:** 1.1.0
|
||||
|
||||
---
|
||||
|
||||
## Key Features
|
||||
|
||||
### 1. Automated Scheduling
|
||||
- **Daily Backups:** Run every day at specified time
|
||||
- **Weekly Backups:** Run once per week
|
||||
- **Monthly Backups:** Run once per month
|
||||
- **Custom Time:** Choose exact time (24-hour format)
|
||||
|
||||
### 2. Backup Type Selection ✨ NEW
|
||||
- **Full Backup:** Complete database with schema, triggers, and data
|
||||
- **Data-Only Backup:** Only table data (faster, smaller files)
|
||||
|
||||
### 3. Retention Management
|
||||
- **Automatic Cleanup:** Delete backups older than X days
|
||||
- **Configurable Period:** Keep backups from 1 to 365 days
|
||||
- **Smart Storage:** Prevents disk space issues
|
||||
|
||||
### 4. Easy Management
|
||||
- **Enable/Disable:** Toggle scheduled backups on/off
|
||||
- **Visual Interface:** Clear, intuitive settings panel
|
||||
- **Status Tracking:** See current schedule at a glance
|
||||
|
||||
---
|
||||
|
||||
## Configuration Options
|
||||
|
||||
### Schedule Settings
|
||||
|
||||
| Setting | Options | Default | Description |
|
||||
|---------|---------|---------|-------------|
|
||||
| **Enabled** | On/Off | Off | Enable or disable scheduled backups |
|
||||
| **Time** | 00:00 - 23:59 | 02:00 | Time to run backup (24-hour format) |
|
||||
| **Frequency** | Daily, Weekly, Monthly | Daily | How often to run backup |
|
||||
| **Backup Type** | Full, Data-Only | Full | Type of backup to create |
|
||||
| **Retention** | 1-365 days | 30 | Days to keep old backups |
|
||||
|
||||
---
|
||||
|
||||
## Recommended Configurations
|
||||
|
||||
### Configuration 1: Daily Data Snapshots
|
||||
**Best for:** Production environments with frequent data changes
|
||||
|
||||
```json
|
||||
{
|
||||
"enabled": true,
|
||||
"time": "02:00",
|
||||
"frequency": "daily",
|
||||
"backup_type": "data-only",
|
||||
"retention_days": 7
|
||||
}
|
||||
```
|
||||
|
||||
**Why:**
|
||||
- ✅ Fast daily backups (data-only is 30-40% faster)
|
||||
- ✅ Smaller file sizes
|
||||
- ✅ 7-day retention keeps recent history without filling disk
|
||||
- ✅ Schema changes handled separately
|
||||
|
||||
### Configuration 2: Weekly Full Backups
|
||||
**Best for:** Stable environments, comprehensive safety
|
||||
|
||||
```json
|
||||
{
|
||||
"enabled": true,
|
||||
"time": "03:00",
|
||||
"frequency": "weekly",
|
||||
"backup_type": "full",
|
||||
"retention_days": 60
|
||||
}
|
||||
```
|
||||
|
||||
**Why:**
|
||||
- ✅ Complete database backup with schema and triggers
|
||||
- ✅ Less frequent (lower storage usage)
|
||||
- ✅ 60-day retention for long-term recovery
|
||||
- ✅ Safe for disaster recovery
|
||||
|
||||
### Configuration 3: Hybrid Approach (Recommended)
|
||||
**Best for:** Most production environments
|
||||
|
||||
**Schedule 1 - Daily Data:**
|
||||
```json
|
||||
{
|
||||
"enabled": true,
|
||||
"time": "02:00",
|
||||
"frequency": "daily",
|
||||
"backup_type": "data-only",
|
||||
"retention_days": 7
|
||||
}
|
||||
```
|
||||
|
||||
**Schedule 2 - Weekly Full (manual or separate scheduler):**
|
||||
- Run manual full backup every Sunday
|
||||
- Keep for 90 days
|
||||
|
||||
**Why:**
|
||||
- ✅ Daily data snapshots for quick recovery
|
||||
- ✅ Weekly full backups for complete safety
|
||||
- ✅ Balanced storage usage
|
||||
- ✅ Multiple recovery points
|
||||
|
||||
---
|
||||
|
||||
## How to Configure
|
||||
|
||||
### Via Web Interface
|
||||
|
||||
1. **Navigate to Settings:**
|
||||
- Log in as Admin or Superadmin
|
||||
- Go to **Settings** page
|
||||
- Scroll to **Database Backup Management** section
|
||||
|
||||
2. **Configure Schedule:**
|
||||
- Check **"Enable Scheduled Backups"** checkbox
|
||||
- Set **Backup Time** (e.g., 02:00)
|
||||
- Choose **Frequency** (Daily/Weekly/Monthly)
|
||||
- Select **Backup Type:**
|
||||
- **Full Backup** for complete safety
|
||||
- **Data-Only Backup** for faster, smaller backups
|
||||
- Set **Retention Days** (1-365)
|
||||
|
||||
3. **Save Configuration:**
|
||||
- Click **💾 Save Schedule** button
|
||||
- Confirm settings in alert message
|
||||
|
||||
### Via Configuration File
|
||||
|
||||
**File Location:** `/srv/quality_app/backups/backup_schedule.json`
|
||||
|
||||
**Example:**
|
||||
```json
|
||||
{
|
||||
"enabled": true,
|
||||
"time": "02:00",
|
||||
"frequency": "daily",
|
||||
"backup_type": "data-only",
|
||||
"retention_days": 30
|
||||
}
|
||||
```
|
||||
|
||||
**Note:** Changes take effect on next scheduled run.
|
||||
|
||||
---
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
### 1. Schedule Storage
|
||||
- **File:** `backup_schedule.json` in backups directory
|
||||
- **Format:** JSON
|
||||
- **Persistence:** Survives application restarts
|
||||
|
||||
### 2. Backup Execution
|
||||
The schedule configuration is stored, but actual execution requires a cron job or scheduler:
|
||||
|
||||
**Recommended: Use system cron**
|
||||
```bash
|
||||
# Edit crontab
|
||||
crontab -e
|
||||
|
||||
# Add entry for 2 AM daily
|
||||
0 2 * * * cd /srv/quality_app/py_app && /srv/quality_recticel/recticel/bin/python3 -c "from app.database_backup import DatabaseBackupManager; from app import create_app; app = create_app(); app.app_context().push(); mgr = DatabaseBackupManager(); schedule = mgr.get_backup_schedule(); mgr.create_data_only_backup() if schedule['backup_type'] == 'data-only' else mgr.create_backup()"
|
||||
```
|
||||
|
||||
**Alternative: APScheduler (application-level)**
|
||||
```python
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
|
||||
scheduler = BackgroundScheduler()
|
||||
|
||||
def scheduled_backup():
|
||||
schedule = backup_manager.get_backup_schedule()
|
||||
if schedule['enabled']:
|
||||
if schedule['backup_type'] == 'data-only':
|
||||
backup_manager.create_data_only_backup()
|
||||
else:
|
||||
backup_manager.create_backup()
|
||||
backup_manager.cleanup_old_backups(schedule['retention_days'])
|
||||
|
||||
# Schedule based on configuration
|
||||
scheduler.add_job(scheduled_backup, 'cron', hour=2, minute=0)
|
||||
scheduler.start()
|
||||
```
|
||||
|
||||
### 3. Cleanup Process
|
||||
Automated cleanup runs after each backup:
|
||||
- Scans backup directory
|
||||
- Identifies files older than retention_days
|
||||
- Deletes old backups
|
||||
- Logs deletion activity
|
||||
|
||||
---
|
||||
|
||||
## Backup Type Comparison
|
||||
|
||||
### Full Backup (Schema + Data + Triggers)
|
||||
|
||||
**mysqldump command:**
|
||||
```bash
|
||||
mysqldump \
|
||||
--single-transaction \
|
||||
--skip-lock-tables \
|
||||
--force \
|
||||
--routines \
|
||||
--triggers \ # ✅ Included
|
||||
--events \
|
||||
--add-drop-database \
|
||||
--databases trasabilitate
|
||||
```
|
||||
|
||||
**Typical size:** 1-2 MB (schema) + data size
|
||||
**Backup time:** ~15-30 seconds
|
||||
**Restore:** Complete replacement
|
||||
|
||||
### Data-Only Backup
|
||||
|
||||
**mysqldump command:**
|
||||
```bash
|
||||
mysqldump \
|
||||
--no-create-info \ # ❌ Skip CREATE TABLE
|
||||
--skip-triggers \ # ❌ Skip triggers
|
||||
--no-create-db \ # ❌ Skip CREATE DATABASE
|
||||
--complete-insert \
|
||||
--extended-insert \
|
||||
--single-transaction \
|
||||
trasabilitate
|
||||
```
|
||||
|
||||
**Typical size:** Data size only
|
||||
**Backup time:** ~10-20 seconds (30-40% faster)
|
||||
**Restore:** Data only (schema must exist)
|
||||
|
||||
---
|
||||
|
||||
## Understanding the UI
|
||||
|
||||
### Schedule Form Fields
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ ☑ Enable Scheduled Backups │
|
||||
├─────────────────────────────────────────────┤
|
||||
│ Backup Time: [02:00] │
|
||||
│ Frequency: [Daily ▼] │
|
||||
│ Backup Type: [Full Backup ▼] │
|
||||
│ Keep backups for: [30] days │
|
||||
├─────────────────────────────────────────────┤
|
||||
│ [💾 Save Schedule] │
|
||||
└─────────────────────────────────────────────┘
|
||||
|
||||
💡 Recommendation: Use Full Backup for weekly/
|
||||
monthly schedules (complete safety), and
|
||||
Data-Only for daily schedules (faster,
|
||||
smaller files).
|
||||
```
|
||||
|
||||
### Success Message Format
|
||||
When saving schedule:
|
||||
```
|
||||
✅ Backup schedule saved successfully
|
||||
|
||||
Scheduled [Full/Data-Only] backups will run
|
||||
[daily/weekly/monthly] at [HH:MM].
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Get Current Schedule
|
||||
```
|
||||
GET /api/backup/schedule
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"schedule": {
|
||||
"enabled": true,
|
||||
"time": "02:00",
|
||||
"frequency": "daily",
|
||||
"backup_type": "data-only",
|
||||
"retention_days": 30
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Save Schedule
|
||||
```
|
||||
POST /api/backup/schedule
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"enabled": true,
|
||||
"time": "02:00",
|
||||
"frequency": "daily",
|
||||
"backup_type": "data-only",
|
||||
"retention_days": 30
|
||||
}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "Backup schedule saved successfully"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Monitoring and Logs
|
||||
|
||||
### Check Backup Files
|
||||
```bash
|
||||
ls -lh /srv/quality_app/backups/*.sql | tail -10
|
||||
```
|
||||
|
||||
### Verify Schedule Configuration
|
||||
```bash
|
||||
cat /srv/quality_app/backups/backup_schedule.json
|
||||
```
|
||||
|
||||
### Check Application Logs
|
||||
```bash
|
||||
tail -f /srv/quality_app/logs/error.log | grep -i backup
|
||||
```
|
||||
|
||||
### Monitor Disk Usage
|
||||
```bash
|
||||
du -sh /srv/quality_app/backups/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: Scheduled backups not running
|
||||
|
||||
**Check 1:** Is schedule enabled?
|
||||
```bash
|
||||
cat /srv/quality_app/backups/backup_schedule.json | grep enabled
|
||||
```
|
||||
|
||||
**Check 2:** Is cron job configured?
|
||||
```bash
|
||||
crontab -l | grep backup
|
||||
```
|
||||
|
||||
**Check 3:** Are there permission issues?
|
||||
```bash
|
||||
ls -la /srv/quality_app/backups/
|
||||
```
|
||||
|
||||
**Solution:** Ensure cron job exists and has proper permissions.
|
||||
|
||||
---
|
||||
|
||||
### Issue: Backup files growing too large
|
||||
|
||||
**Check disk usage:**
|
||||
```bash
|
||||
du -sh /srv/quality_app/backups/
|
||||
ls -lh /srv/quality_app/backups/*.sql | wc -l
|
||||
```
|
||||
|
||||
**Solutions:**
|
||||
1. Reduce retention_days (e.g., from 30 to 7)
|
||||
2. Use data-only backups (smaller files)
|
||||
3. Store old backups on external storage
|
||||
4. Compress backups: `gzip /srv/quality_app/backups/*.sql`
|
||||
|
||||
---
|
||||
|
||||
### Issue: Data-only restore fails
|
||||
|
||||
**Error:** "Table doesn't exist"
|
||||
|
||||
**Cause:** Database schema not present
|
||||
|
||||
**Solution:**
|
||||
1. Run full backup restore first, OR
|
||||
2. Ensure database structure exists via setup script
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
### ✅ DO:
|
||||
1. **Enable scheduled backups** - Automate for consistency
|
||||
2. **Use data-only for daily** - Faster, smaller files
|
||||
3. **Use full for weekly** - Complete safety net
|
||||
4. **Test restore regularly** - Verify backups work
|
||||
5. **Monitor disk space** - Prevent storage issues
|
||||
6. **Store off-site copies** - Disaster recovery
|
||||
7. **Adjust retention** - Balance safety vs. storage
|
||||
|
||||
### ❌ DON'T:
|
||||
1. **Don't disable all backups** - Always have some backup
|
||||
2. **Don't set retention too low** - Keep at least 7 days
|
||||
3. **Don't ignore disk warnings** - Monitor storage
|
||||
4. **Don't forget to test restores** - Untested backups are useless
|
||||
5. **Don't rely only on scheduled** - Manual backups before major changes
|
||||
|
||||
---
|
||||
|
||||
## Security and Access
|
||||
|
||||
### Required Roles
|
||||
- **View Schedule:** Admin, Superadmin
|
||||
- **Edit Schedule:** Admin, Superadmin
|
||||
- **Execute Manual Backup:** Admin, Superadmin
|
||||
- **Restore Database:** Superadmin only
|
||||
|
||||
### File Permissions
|
||||
```bash
|
||||
# Backup directory
|
||||
drwxrwxr-x /srv/quality_app/backups/
|
||||
|
||||
# Schedule file
|
||||
-rw-rw-r-- backup_schedule.json
|
||||
|
||||
# Backup files
|
||||
-rw-rw-r-- *.sql
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### Upgrading from Previous Version (without backup_type)
|
||||
|
||||
**Automatic:** Schedule automatically gets `backup_type: "full"` on first load
|
||||
|
||||
**Manual update:**
|
||||
```bash
|
||||
cd /srv/quality_app/backups/
|
||||
# Backup current schedule
|
||||
cp backup_schedule.json backup_schedule.json.bak
|
||||
|
||||
# Add backup_type field
|
||||
cat backup_schedule.json | jq '. + {"backup_type": "full"}' > backup_schedule_new.json
|
||||
mv backup_schedule_new.json backup_schedule.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [DATA_ONLY_BACKUP_FEATURE.md](DATA_ONLY_BACKUP_FEATURE.md) - Data-only backup details
|
||||
- [BACKUP_SYSTEM.md](BACKUP_SYSTEM.md) - Complete backup system overview
|
||||
- [QUICK_BACKUP_REFERENCE.md](QUICK_BACKUP_REFERENCE.md) - Quick reference guide
|
||||
|
||||
---
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
### Planned Features:
|
||||
- [ ] Multiple schedules (daily data + weekly full)
|
||||
- [ ] Email notifications on backup completion
|
||||
- [ ] Backup to remote storage (S3, FTP)
|
||||
- [ ] Backup compression (gzip)
|
||||
- [ ] Backup encryption
|
||||
- [ ] Web-based backup browsing
|
||||
- [ ] Automatic restore testing
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** November 5, 2025
|
||||
**Module:** `app/database_backup.py`
|
||||
**UI Template:** `app/templates/settings.html`
|
||||
**Application:** Quality Recticel - Trasabilitate System
|
||||
312
documentation/DATA_ONLY_BACKUP_FEATURE.md
Normal file
312
documentation/DATA_ONLY_BACKUP_FEATURE.md
Normal file
@@ -0,0 +1,312 @@
|
||||
# Data-Only Backup and Restore Feature
|
||||
|
||||
## Overview
|
||||
|
||||
The data-only backup and restore feature allows you to backup and restore **only the data** from the database, without affecting the database schema, triggers, or structure. This is useful for:
|
||||
|
||||
- **Quick data transfers** between identical database structures
|
||||
- **Data refreshes** without changing the schema
|
||||
- **Faster backups** when you only need to save data
|
||||
- **Testing scenarios** where you want to swap data but keep the structure
|
||||
|
||||
---
|
||||
|
||||
## Key Features
|
||||
|
||||
### 1. Data-Only Backup
|
||||
Creates a backup file containing **only INSERT statements** for all tables.
|
||||
|
||||
**What's included:**
|
||||
- ✅ All table data (INSERT statements)
|
||||
- ✅ Column names in INSERT statements (complete-insert format)
|
||||
- ✅ Multi-row INSERT for efficiency
|
||||
|
||||
**What's NOT included:**
|
||||
- ❌ CREATE TABLE statements (no schema)
|
||||
- ❌ CREATE DATABASE statements
|
||||
- ❌ Trigger definitions
|
||||
- ❌ Stored procedures or functions
|
||||
- ❌ Views
|
||||
|
||||
**File naming:** `data_only_trasabilitate_YYYYMMDD_HHMMSS.sql`
|
||||
|
||||
### 2. Data-Only Restore
|
||||
Restores data from a data-only backup file into an **existing database**.
|
||||
|
||||
**What happens during restore:**
|
||||
1. **Truncates all tables** (deletes all current data)
|
||||
2. **Disables foreign key checks** temporarily
|
||||
3. **Inserts data** from the backup file
|
||||
4. **Re-enables foreign key checks**
|
||||
5. **Preserves** existing schema, triggers, and structure
|
||||
|
||||
**⚠️ Important:** The database schema must already exist and match the backup structure.
|
||||
|
||||
---
|
||||
|
||||
## Usage
|
||||
|
||||
### Creating a Data-Only Backup
|
||||
|
||||
#### Via Web Interface:
|
||||
1. Navigate to **Settings** page
|
||||
2. Scroll to **Database Backup Management** section
|
||||
3. Click **📦 Data-Only Backup** button
|
||||
4. Backup file will be created and added to the backup list
|
||||
|
||||
#### Via API:
|
||||
```bash
|
||||
curl -X POST http://localhost:8781/api/backup/create-data-only \
|
||||
-H "Content-Type: application/json" \
|
||||
--cookie "session=your_session_cookie"
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "Data-only backup created successfully",
|
||||
"filename": "data_only_trasabilitate_20251105_160000.sql",
|
||||
"size": "12.45 MB",
|
||||
"timestamp": "20251105_160000"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Restoring from Data-Only Backup
|
||||
|
||||
#### Via Web Interface:
|
||||
1. Navigate to **Settings** page
|
||||
2. Scroll to **Restore Database** section (Superadmin only)
|
||||
3. Select a backup file from the dropdown
|
||||
4. Choose **"Data-Only Restore"** radio button
|
||||
5. Click **🔄 Restore Database** button
|
||||
6. Confirm twice (with typing "RESTORE DATA")
|
||||
|
||||
#### Via API:
|
||||
```bash
|
||||
curl -X POST http://localhost:8781/api/backup/restore-data-only/data_only_trasabilitate_20251105_160000.sql \
|
||||
-H "Content-Type: application/json" \
|
||||
--cookie "session=your_session_cookie"
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "Data restored successfully from data_only_trasabilitate_20251105_160000.sql"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Comparison: Full Backup vs Data-Only Backup
|
||||
|
||||
| Feature | Full Backup | Data-Only Backup |
|
||||
|---------|-------------|------------------|
|
||||
| **Database Schema** | ✅ Included | ❌ Not included |
|
||||
| **Triggers** | ✅ Included | ❌ Not included |
|
||||
| **Stored Procedures** | ✅ Included | ❌ Not included |
|
||||
| **Table Data** | ✅ Included | ✅ Included |
|
||||
| **File Size** | Larger | Smaller |
|
||||
| **Backup Speed** | Slower | Faster |
|
||||
| **Use Case** | Complete migration, disaster recovery | Data refresh, testing |
|
||||
| **Restore Requirements** | None (creates everything) | Database schema must exist |
|
||||
|
||||
---
|
||||
|
||||
## Use Cases
|
||||
|
||||
### ✅ When to Use Data-Only Backup:
|
||||
|
||||
1. **Daily Data Snapshots**
|
||||
- You want to backup data frequently without duplicating schema
|
||||
- Faster backups for large databases
|
||||
|
||||
2. **Data Transfer Between Servers**
|
||||
- Both servers have identical database structure
|
||||
- You only need to copy the data
|
||||
|
||||
3. **Testing and Development**
|
||||
- Load production data into test environment
|
||||
- Test environment already has correct schema
|
||||
|
||||
4. **Data Refresh**
|
||||
- Replace old data with new data
|
||||
- Keep existing triggers and procedures
|
||||
|
||||
### ❌ When NOT to Use Data-Only Backup:
|
||||
|
||||
1. **Complete Database Migration**
|
||||
- Use full backup to ensure all structures are migrated
|
||||
|
||||
2. **Disaster Recovery**
|
||||
- Use full backup to restore everything
|
||||
|
||||
3. **Schema Changes**
|
||||
- If schema has changed, data-only restore will fail
|
||||
- Use full backup and restore
|
||||
|
||||
4. **Fresh Database Setup**
|
||||
- No existing schema to restore into
|
||||
- Use full backup or database setup script
|
||||
|
||||
---
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
### mysqldump Command for Data-Only Backup
|
||||
```bash
|
||||
mysqldump \
|
||||
--host=localhost \
|
||||
--port=3306 \
|
||||
--user=trasabilitate \
|
||||
--password=password \
|
||||
--no-create-info # Skip CREATE TABLE statements
|
||||
--skip-triggers # Skip trigger definitions
|
||||
--no-create-db # Skip CREATE DATABASE statement
|
||||
--complete-insert # Include column names in INSERT
|
||||
--extended-insert # Multi-row INSERTs for efficiency
|
||||
--single-transaction # Consistent snapshot
|
||||
--skip-lock-tables # Avoid table locks
|
||||
trasabilitate
|
||||
```
|
||||
|
||||
### Data-Only Restore Process
|
||||
```python
|
||||
# 1. Disable foreign key checks
|
||||
SET FOREIGN_KEY_CHECKS = 0;
|
||||
|
||||
# 2. Get all tables
|
||||
SHOW TABLES;
|
||||
|
||||
# 3. Truncate each table (except system tables)
|
||||
TRUNCATE TABLE `table_name`;
|
||||
|
||||
# 4. Execute the data-only backup SQL file
|
||||
# (Contains INSERT statements)
|
||||
|
||||
# 5. Re-enable foreign key checks
|
||||
SET FOREIGN_KEY_CHECKS = 1;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security and Permissions
|
||||
|
||||
- **Data-Only Backup Creation:** Requires `admin` or `superadmin` role
|
||||
- **Data-Only Restore:** Requires `superadmin` role only
|
||||
- **API Access:** Requires valid session authentication
|
||||
- **File Access:** Backups stored in `/srv/quality_app/backups` (configurable)
|
||||
|
||||
---
|
||||
|
||||
## Safety Features
|
||||
|
||||
### Confirmation Process for Restore:
|
||||
1. **First Confirmation:** Dialog explaining what will happen
|
||||
2. **Second Confirmation:** Requires typing "RESTORE DATA" in capital letters
|
||||
3. **Type Detection:** Warns if trying to do full restore on data-only file
|
||||
|
||||
### Data Integrity:
|
||||
- **Foreign key checks** disabled during restore to avoid constraint errors
|
||||
- **Transaction-based** backup for consistent snapshots
|
||||
- **Table truncation** ensures clean data without duplicates
|
||||
- **Automatic re-enabling** of foreign key checks after restore
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Create Data-Only Backup
|
||||
```
|
||||
POST /api/backup/create-data-only
|
||||
```
|
||||
**Access:** Admin+
|
||||
**Response:** Backup filename and size
|
||||
|
||||
### Restore Data-Only Backup
|
||||
```
|
||||
POST /api/backup/restore-data-only/<filename>
|
||||
```
|
||||
**Access:** Superadmin only
|
||||
**Response:** Success/failure message
|
||||
|
||||
---
|
||||
|
||||
## File Naming Convention
|
||||
|
||||
### Data-Only Backups:
|
||||
- Format: `data_only_<database>_<timestamp>.sql`
|
||||
- Example: `data_only_trasabilitate_20251105_143022.sql`
|
||||
|
||||
### Full Backups:
|
||||
- Format: `backup_<database>_<timestamp>.sql`
|
||||
- Example: `backup_trasabilitate_20251105_143022.sql`
|
||||
|
||||
The `data_only_` prefix helps identify backup type at a glance.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Error: "Data restore failed: Table 'X' doesn't exist"
|
||||
**Cause:** Database schema not present or incomplete
|
||||
**Solution:** Run full backup restore or database setup script first
|
||||
|
||||
### Error: "Column count doesn't match"
|
||||
**Cause:** Schema structure has changed since backup was created
|
||||
**Solution:** Use a newer data-only backup or update schema first
|
||||
|
||||
### Error: "Foreign key constraint fails"
|
||||
**Cause:** Foreign key checks not properly disabled
|
||||
**Solution:** Check MariaDB user has SUPER privilege
|
||||
|
||||
### Warning: "Could not truncate table"
|
||||
**Cause:** Table has special permissions or is a view
|
||||
**Solution:** Non-critical warning; restore will continue
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Always keep full backups** for complete disaster recovery
|
||||
2. **Use data-only backups** for frequent snapshots
|
||||
3. **Test restores** in non-production environment first
|
||||
4. **Document schema changes** that affect data structure
|
||||
5. **Schedule both types** of backups (e.g., full weekly, data-only daily)
|
||||
|
||||
---
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Backup Speed:
|
||||
- **Full backup (17 tables):** ~15-30 seconds
|
||||
- **Data-only backup (17 tables):** ~10-20 seconds (faster by 30-40%)
|
||||
|
||||
### File Size:
|
||||
- **Full backup:** Includes schema (~1-2 MB) + data
|
||||
- **Data-only backup:** Only data (smaller by 1-2 MB)
|
||||
|
||||
### Restore Speed:
|
||||
- **Full restore:** Drops and recreates everything
|
||||
- **Data-only restore:** Only truncates and inserts (faster on large schemas)
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [BACKUP_SYSTEM.md](BACKUP_SYSTEM.md) - Complete backup system overview
|
||||
- [DATABASE_RESTORE_GUIDE.md](DATABASE_RESTORE_GUIDE.md) - Detailed restore procedures
|
||||
- [DATABASE_STRUCTURE.md](DATABASE_STRUCTURE.md) - Database schema reference
|
||||
|
||||
---
|
||||
|
||||
## Implementation Date
|
||||
|
||||
**Feature Added:** November 5, 2025
|
||||
**Version:** 1.1.0
|
||||
**Python Module:** `app/database_backup.py`
|
||||
**API Routes:** `app/routes.py` (lines 3800-3835)
|
||||
**UI Template:** `app/templates/settings.html`
|
||||
199
documentation/QUICK_BACKUP_REFERENCE.md
Normal file
199
documentation/QUICK_BACKUP_REFERENCE.md
Normal file
@@ -0,0 +1,199 @@
|
||||
# Quick Backup Reference Guide
|
||||
|
||||
## When to Use Which Backup Type?
|
||||
|
||||
### 🔵 Full Backup (Schema + Data + Triggers)
|
||||
|
||||
**Use when:**
|
||||
- ✅ Setting up a new database server
|
||||
- ✅ Complete disaster recovery
|
||||
- ✅ Migrating to a different server
|
||||
- ✅ Database schema has changed
|
||||
- ✅ You need everything (safest option)
|
||||
|
||||
**Creates:**
|
||||
- Database structure (CREATE TABLE, CREATE DATABASE)
|
||||
- All triggers and stored procedures
|
||||
- All data (INSERT statements)
|
||||
|
||||
**File:** `backup_trasabilitate_20251105_190632.sql`
|
||||
|
||||
---
|
||||
|
||||
### 🟢 Data-Only Backup (Data Only)
|
||||
|
||||
**Use when:**
|
||||
- ✅ Quick daily data snapshots
|
||||
- ✅ Both databases have identical structure
|
||||
- ✅ You want to load different data into existing database
|
||||
- ✅ Faster backups for large databases
|
||||
- ✅ Testing with production data
|
||||
|
||||
**Creates:**
|
||||
- Only INSERT statements for all tables
|
||||
- No schema, no triggers, no structure
|
||||
|
||||
**File:** `data_only_trasabilitate_20251105_190632.sql`
|
||||
|
||||
---
|
||||
|
||||
## Quick Command Reference
|
||||
|
||||
### Web Interface
|
||||
|
||||
**Location:** Settings → Database Backup Management
|
||||
|
||||
#### Create Backups:
|
||||
- **Full Backup:** Click `⚡ Full Backup (Schema + Data)` button
|
||||
- **Data-Only:** Click `📦 Data-Only Backup` button
|
||||
|
||||
#### Restore Database (Superadmin Only):
|
||||
1. Select backup file from dropdown
|
||||
2. Choose restore type:
|
||||
- **Full Restore:** Replace entire database
|
||||
- **Data-Only Restore:** Replace only data
|
||||
3. Click `🔄 Restore Database` button
|
||||
4. Confirm twice
|
||||
|
||||
---
|
||||
|
||||
## Backup Comparison
|
||||
|
||||
| Feature | Full Backup | Data-Only Backup |
|
||||
|---------|-------------|------------------|
|
||||
| **Speed** | Slower | ⚡ Faster (30-40% quicker) |
|
||||
| **File Size** | Larger | 📦 Smaller (~1-2 MB less) |
|
||||
| **Schema** | ✅ Yes | ❌ No |
|
||||
| **Triggers** | ✅ Yes | ❌ No |
|
||||
| **Data** | ✅ Yes | ✅ Yes |
|
||||
| **Use Case** | Complete recovery | Data refresh |
|
||||
| **Restore Requirement** | None | Schema must exist |
|
||||
|
||||
---
|
||||
|
||||
## Safety Features
|
||||
|
||||
### Full Restore
|
||||
- **Confirmation:** Type "RESTORE" in capital letters
|
||||
- **Effect:** Replaces EVERYTHING
|
||||
- **Warning:** All data, schema, triggers deleted
|
||||
|
||||
### Data-Only Restore
|
||||
- **Confirmation:** Type "RESTORE DATA" in capital letters
|
||||
- **Effect:** Replaces only data
|
||||
- **Warning:** All data deleted, schema preserved
|
||||
|
||||
### Smart Detection
|
||||
- System warns if you try to do full restore on data-only file
|
||||
- System warns if you try to do data-only restore on full file
|
||||
|
||||
---
|
||||
|
||||
## Common Scenarios
|
||||
|
||||
### Scenario 1: Daily Backups
|
||||
**Recommendation:**
|
||||
- Monday: Full backup (keeps everything)
|
||||
- Tuesday-Sunday: Data-only backups (faster, smaller)
|
||||
|
||||
### Scenario 2: Database Migration
|
||||
**Recommendation:**
|
||||
- Use full backup (safest, includes everything)
|
||||
|
||||
### Scenario 3: Load Test Data
|
||||
**Recommendation:**
|
||||
- Use data-only backup (preserve your test triggers)
|
||||
|
||||
### Scenario 4: Disaster Recovery
|
||||
**Recommendation:**
|
||||
- Use full backup (complete restoration)
|
||||
|
||||
### Scenario 5: Data Refresh
|
||||
**Recommendation:**
|
||||
- Use data-only backup (quick data swap)
|
||||
|
||||
---
|
||||
|
||||
## File Naming Convention
|
||||
|
||||
### Identify Backup Type by Filename:
|
||||
|
||||
```
|
||||
backup_trasabilitate_20251105_143022.sql
|
||||
└─┬─┘ └─────┬──────┘ └────┬─────┘
|
||||
│ │ └─ Timestamp
|
||||
│ └─ Database name
|
||||
└─ Full backup
|
||||
|
||||
data_only_trasabilitate_20251105_143022.sql
|
||||
└───┬───┘ └─────┬──────┘ └────┬─────┘
|
||||
│ │ └─ Timestamp
|
||||
│ └─ Database name
|
||||
└─ Data-only backup
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Table doesn't exist" during data-only restore
|
||||
**Solution:** Run full backup restore first, or use database setup script
|
||||
|
||||
### "Column count doesn't match" during data-only restore
|
||||
**Solution:** Schema has changed. Update schema or use newer backup
|
||||
|
||||
### "Foreign key constraint fails" during restore
|
||||
**Solution:** Database user needs SUPER privilege
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. ✅ Keep both types of backups
|
||||
2. ✅ Test restores in non-production first
|
||||
3. ✅ Schedule full backups weekly
|
||||
4. ✅ Schedule data-only backups daily
|
||||
5. ✅ Keep backups for 30+ days
|
||||
6. ✅ Store backups off-server for disaster recovery
|
||||
|
||||
---
|
||||
|
||||
## Access Requirements
|
||||
|
||||
| Action | Required Role |
|
||||
|--------|--------------|
|
||||
| Create Full Backup | Admin or Superadmin |
|
||||
| Create Data-Only Backup | Admin or Superadmin |
|
||||
| View Backup List | Admin or Superadmin |
|
||||
| Download Backup | Admin or Superadmin |
|
||||
| Delete Backup | Admin or Superadmin |
|
||||
| **Full Restore** | **Superadmin Only** |
|
||||
| **Data-Only Restore** | **Superadmin Only** |
|
||||
|
||||
---
|
||||
|
||||
## Quick Tips
|
||||
|
||||
💡 **Tip 1:** Data-only backups are 30-40% faster than full backups
|
||||
|
||||
💡 **Tip 2:** Use data-only restore to quickly swap between production and test data
|
||||
|
||||
💡 **Tip 3:** Always keep at least one full backup for disaster recovery
|
||||
|
||||
💡 **Tip 4:** Data-only backups are perfect for automated daily snapshots
|
||||
|
||||
💡 **Tip 5:** Test your restore process regularly (at least quarterly)
|
||||
|
||||
---
|
||||
|
||||
## Support
|
||||
|
||||
For detailed information, see:
|
||||
- [DATA_ONLY_BACKUP_FEATURE.md](DATA_ONLY_BACKUP_FEATURE.md) - Complete feature documentation
|
||||
- [BACKUP_SYSTEM.md](BACKUP_SYSTEM.md) - Overall backup system
|
||||
- [DATABASE_RESTORE_GUIDE.md](DATABASE_RESTORE_GUIDE.md) - Restore procedures
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** November 5, 2025
|
||||
**Application:** Quality Recticel - Trasabilitate System
|
||||
Reference in New Issue
Block a user