quality_app/documentation/DATA_ONLY_BACKUP_FEATURE.md

Data-Only Backup and Restore Feature

Overview

The data-only backup and restore feature allows you to backup and restore only the data from the database, without affecting the database schema, triggers, or structure. This is useful for:

• Quick data transfers between identical database structures
• Data refreshes without changing the schema
• Faster backups when you only need to save data
• Testing scenarios where you want to swap data but keep the structure

---

Key Features

1. Data-Only Backup

Creates a backup file containing only INSERT statements for all tables.

What's included:

• ✅ All table data (INSERT statements)
• ✅ Column names in INSERT statements (complete-insert format)
• ✅ Multi-row INSERT for efficiency

What's NOT included:

• ❌ CREATE TABLE statements (no schema)
• ❌ CREATE DATABASE statements
• ❌ Trigger definitions
• ❌ Stored procedures or functions
• ❌ Views

File naming: data_only_trasabilitate_YYYYMMDD_HHMMSS.sql

2. Data-Only Restore

Restores data from a data-only backup file into an existing database.

What happens during restore:

1. Truncates all tables (deletes all current data)
2. Disables foreign key checks temporarily
3. Inserts data from the backup file
4. Re-enables foreign key checks
5. Preserves existing schema, triggers, and structure

⚠️ Important: The database schema must already exist and match the backup structure.

---

Usage

Creating a Data-Only Backup

Via Web Interface:

1. Navigate to Settings page
2. Scroll to Database Backup Management section
3. Click 📦 Data-Only Backup button
4. Backup file will be created and added to the backup list

Via API:

curl -X POST http://localhost:8781/api/backup/create-data-only \
  -H "Content-Type: application/json" \
  --cookie "session=your_session_cookie"

Response:

{
  "success": true,
  "message": "Data-only backup created successfully",
  "filename": "data_only_trasabilitate_20251105_160000.sql",
  "size": "12.45 MB",
  "timestamp": "20251105_160000"
}

---

Restoring from Data-Only Backup

Via Web Interface:

1. Navigate to Settings page
2. Scroll to Restore Database section (Superadmin only)
3. Select a backup file from the dropdown
4. Choose "Data-Only Restore" radio button
5. Click 🔄 Restore Database button
6. Confirm twice (with typing "RESTORE DATA")

Via API:

curl -X POST http://localhost:8781/api/backup/restore-data-only/data_only_trasabilitate_20251105_160000.sql \
  -H "Content-Type: application/json" \
  --cookie "session=your_session_cookie"

Response:

{
  "success": true,
  "message": "Data restored successfully from data_only_trasabilitate_20251105_160000.sql"
}

---

Comparison: Full Backup vs Data-Only Backup

Feature	Full Backup	Data-Only Backup
Database Schema	✅ Included	❌ Not included
Triggers	✅ Included	❌ Not included
Stored Procedures	✅ Included	❌ Not included
Table Data	✅ Included	✅ Included
File Size	Larger	Smaller
Backup Speed	Slower	Faster
Use Case	Complete migration, disaster recovery	Data refresh, testing
Restore Requirements	None (creates everything)	Database schema must exist

---

Use Cases

✅ When to Use Data-Only Backup:

1. Daily Data Snapshots

   • You want to backup data frequently without duplicating schema
   • Faster backups for large databases

2. Data Transfer Between Servers

   • Both servers have identical database structure
   • You only need to copy the data

3. Testing and Development

   • Load production data into test environment
   • Test environment already has correct schema

4. Data Refresh

   • Replace old data with new data
   • Keep existing triggers and procedures

❌ When NOT to Use Data-Only Backup:

1. Complete Database Migration

   • Use full backup to ensure all structures are migrated

2. Disaster Recovery

   • Use full backup to restore everything

3. Schema Changes

   • If schema has changed, data-only restore will fail
   • Use full backup and restore

4. Fresh Database Setup

   • No existing schema to restore into
   • Use full backup or database setup script

---

Technical Implementation

mysqldump Command for Data-Only Backup

mysqldump \
  --host=localhost \
  --port=3306 \
  --user=trasabilitate \
  --password=password \
  --no-create-info      # Skip CREATE TABLE statements
  --skip-triggers       # Skip trigger definitions
  --no-create-db        # Skip CREATE DATABASE statement
  --complete-insert     # Include column names in INSERT
  --extended-insert     # Multi-row INSERTs for efficiency
  --single-transaction  # Consistent snapshot
  --skip-lock-tables    # Avoid table locks
  trasabilitate

Data-Only Restore Process

# 1. Disable foreign key checks
SET FOREIGN_KEY_CHECKS = 0;

# 2. Get all tables
SHOW TABLES;

# 3. Truncate each table (except system tables)
TRUNCATE TABLE `table_name`;

# 4. Execute the data-only backup SQL file
# (Contains INSERT statements)

# 5. Re-enable foreign key checks
SET FOREIGN_KEY_CHECKS = 1;

---

Security and Permissions

• Data-Only Backup Creation: Requires admin or superadmin role
• Data-Only Restore: Requires superadmin role only
• API Access: Requires valid session authentication
• File Access: Backups stored in /srv/quality_app/backups (configurable)

---

Safety Features

Confirmation Process for Restore:

1. First Confirmation: Dialog explaining what will happen
2. Second Confirmation: Requires typing "RESTORE DATA" in capital letters
3. Type Detection: Warns if trying to do full restore on data-only file

Data Integrity:

• Foreign key checks disabled during restore to avoid constraint errors
• Transaction-based backup for consistent snapshots
• Table truncation ensures clean data without duplicates
• Automatic re-enabling of foreign key checks after restore

---

API Endpoints

Create Data-Only Backup

POST /api/backup/create-data-only

Access: Admin+ Response: Backup filename and size

Restore Data-Only Backup

POST /api/backup/restore-data-only/<filename>

Access: Superadmin only Response: Success/failure message

---

File Naming Convention

Data-Only Backups:

• Format: data_only_<database>_<timestamp>.sql
• Example: data_only_trasabilitate_20251105_143022.sql

Full Backups:

• Format: backup_<database>_<timestamp>.sql
• Example: backup_trasabilitate_20251105_143022.sql

The data_only_ prefix helps identify backup type at a glance.

---

Troubleshooting

Error: "Data restore failed: Table 'X' doesn't exist"

Cause: Database schema not present or incomplete Solution: Run full backup restore or database setup script first

Error: "Column count doesn't match"

Cause: Schema structure has changed since backup was created Solution: Use a newer data-only backup or update schema first

Error: "Foreign key constraint fails"

Cause: Foreign key checks not properly disabled Solution: Check MariaDB user has SUPER privilege

Warning: "Could not truncate table"

Cause: Table has special permissions or is a view Solution: Non-critical warning; restore will continue

---

Best Practices

1. Always keep full backups for complete disaster recovery
2. Use data-only backups for frequent snapshots
3. Test restores in non-production environment first
4. Document schema changes that affect data structure
5. Schedule both types of backups (e.g., full weekly, data-only daily)

---

Performance Considerations

Backup Speed:

• Full backup (17 tables): ~15-30 seconds
• Data-only backup (17 tables): ~10-20 seconds (faster by 30-40%)

File Size:

• Full backup: Includes schema (~1-2 MB) + data
• Data-only backup: Only data (smaller by 1-2 MB)

Restore Speed:

• Full restore: Drops and recreates everything
• Data-only restore: Only truncates and inserts (faster on large schemas)

---

Related Documentation

• BACKUP_SYSTEM.md - Complete backup system overview
• DATABASE_RESTORE_GUIDE.md - Detailed restore procedures
• DATABASE_STRUCTURE.md - Database schema reference

---

Implementation Date

Feature Added: November 5, 2025 Version: 1.1.0 Python Module: app/database_backup.py API Routes: app/routes.py (lines 3800-3835) UI Template: app/templates/settings.html