313 lines
8.6 KiB
Markdown
313 lines
8.6 KiB
Markdown
# Data-Only Backup and Restore Feature
|
|
|
|
## Overview
|
|
|
|
The data-only backup and restore feature allows you to backup and restore **only the data** from the database, without affecting the database schema, triggers, or structure. This is useful for:
|
|
|
|
- **Quick data transfers** between identical database structures
|
|
- **Data refreshes** without changing the schema
|
|
- **Faster backups** when you only need to save data
|
|
- **Testing scenarios** where you want to swap data but keep the structure
|
|
|
|
---
|
|
|
|
## Key Features
|
|
|
|
### 1. Data-Only Backup
|
|
Creates a backup file containing **only INSERT statements** for all tables.
|
|
|
|
**What's included:**
|
|
- ✅ All table data (INSERT statements)
|
|
- ✅ Column names in INSERT statements (complete-insert format)
|
|
- ✅ Multi-row INSERT for efficiency
|
|
|
|
**What's NOT included:**
|
|
- ❌ CREATE TABLE statements (no schema)
|
|
- ❌ CREATE DATABASE statements
|
|
- ❌ Trigger definitions
|
|
- ❌ Stored procedures or functions
|
|
- ❌ Views
|
|
|
|
**File naming:** `data_only_trasabilitate_YYYYMMDD_HHMMSS.sql`
|
|
|
|
### 2. Data-Only Restore
|
|
Restores data from a data-only backup file into an **existing database**.
|
|
|
|
**What happens during restore:**
|
|
1. **Truncates all tables** (deletes all current data)
|
|
2. **Disables foreign key checks** temporarily
|
|
3. **Inserts data** from the backup file
|
|
4. **Re-enables foreign key checks**
|
|
5. **Preserves** existing schema, triggers, and structure
|
|
|
|
**⚠️ Important:** The database schema must already exist and match the backup structure.
|
|
|
|
---
|
|
|
|
## Usage
|
|
|
|
### Creating a Data-Only Backup
|
|
|
|
#### Via Web Interface:
|
|
1. Navigate to **Settings** page
|
|
2. Scroll to **Database Backup Management** section
|
|
3. Click **📦 Data-Only Backup** button
|
|
4. Backup file will be created and added to the backup list
|
|
|
|
#### Via API:
|
|
```bash
|
|
curl -X POST http://localhost:8781/api/backup/create-data-only \
|
|
-H "Content-Type: application/json" \
|
|
--cookie "session=your_session_cookie"
|
|
```
|
|
|
|
**Response:**
|
|
```json
|
|
{
|
|
"success": true,
|
|
"message": "Data-only backup created successfully",
|
|
"filename": "data_only_trasabilitate_20251105_160000.sql",
|
|
"size": "12.45 MB",
|
|
"timestamp": "20251105_160000"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### Restoring from Data-Only Backup
|
|
|
|
#### Via Web Interface:
|
|
1. Navigate to **Settings** page
|
|
2. Scroll to **Restore Database** section (Superadmin only)
|
|
3. Select a backup file from the dropdown
|
|
4. Choose **"Data-Only Restore"** radio button
|
|
5. Click **🔄 Restore Database** button
|
|
6. Confirm twice (with typing "RESTORE DATA")
|
|
|
|
#### Via API:
|
|
```bash
|
|
curl -X POST http://localhost:8781/api/backup/restore-data-only/data_only_trasabilitate_20251105_160000.sql \
|
|
-H "Content-Type: application/json" \
|
|
--cookie "session=your_session_cookie"
|
|
```
|
|
|
|
**Response:**
|
|
```json
|
|
{
|
|
"success": true,
|
|
"message": "Data restored successfully from data_only_trasabilitate_20251105_160000.sql"
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Comparison: Full Backup vs Data-Only Backup
|
|
|
|
| Feature | Full Backup | Data-Only Backup |
|
|
|---------|-------------|------------------|
|
|
| **Database Schema** | ✅ Included | ❌ Not included |
|
|
| **Triggers** | ✅ Included | ❌ Not included |
|
|
| **Stored Procedures** | ✅ Included | ❌ Not included |
|
|
| **Table Data** | ✅ Included | ✅ Included |
|
|
| **File Size** | Larger | Smaller |
|
|
| **Backup Speed** | Slower | Faster |
|
|
| **Use Case** | Complete migration, disaster recovery | Data refresh, testing |
|
|
| **Restore Requirements** | None (creates everything) | Database schema must exist |
|
|
|
|
---
|
|
|
|
## Use Cases
|
|
|
|
### ✅ When to Use Data-Only Backup:
|
|
|
|
1. **Daily Data Snapshots**
|
|
- You want to backup data frequently without duplicating schema
|
|
- Faster backups for large databases
|
|
|
|
2. **Data Transfer Between Servers**
|
|
- Both servers have identical database structure
|
|
- You only need to copy the data
|
|
|
|
3. **Testing and Development**
|
|
- Load production data into test environment
|
|
- Test environment already has correct schema
|
|
|
|
4. **Data Refresh**
|
|
- Replace old data with new data
|
|
- Keep existing triggers and procedures
|
|
|
|
### ❌ When NOT to Use Data-Only Backup:
|
|
|
|
1. **Complete Database Migration**
|
|
- Use full backup to ensure all structures are migrated
|
|
|
|
2. **Disaster Recovery**
|
|
- Use full backup to restore everything
|
|
|
|
3. **Schema Changes**
|
|
- If schema has changed, data-only restore will fail
|
|
- Use full backup and restore
|
|
|
|
4. **Fresh Database Setup**
|
|
- No existing schema to restore into
|
|
- Use full backup or database setup script
|
|
|
|
---
|
|
|
|
## Technical Implementation
|
|
|
|
### mysqldump Command for Data-Only Backup
|
|
```bash
|
|
mysqldump \
|
|
--host=localhost \
|
|
--port=3306 \
|
|
--user=trasabilitate \
|
|
--password=password \
|
|
--no-create-info # Skip CREATE TABLE statements
|
|
--skip-triggers # Skip trigger definitions
|
|
--no-create-db # Skip CREATE DATABASE statement
|
|
--complete-insert # Include column names in INSERT
|
|
--extended-insert # Multi-row INSERTs for efficiency
|
|
--single-transaction # Consistent snapshot
|
|
--skip-lock-tables # Avoid table locks
|
|
trasabilitate
|
|
```
|
|
|
|
### Data-Only Restore Process
|
|
```python
|
|
# 1. Disable foreign key checks
|
|
SET FOREIGN_KEY_CHECKS = 0;
|
|
|
|
# 2. Get all tables
|
|
SHOW TABLES;
|
|
|
|
# 3. Truncate each table (except system tables)
|
|
TRUNCATE TABLE `table_name`;
|
|
|
|
# 4. Execute the data-only backup SQL file
|
|
# (Contains INSERT statements)
|
|
|
|
# 5. Re-enable foreign key checks
|
|
SET FOREIGN_KEY_CHECKS = 1;
|
|
```
|
|
|
|
---
|
|
|
|
## Security and Permissions
|
|
|
|
- **Data-Only Backup Creation:** Requires `admin` or `superadmin` role
|
|
- **Data-Only Restore:** Requires `superadmin` role only
|
|
- **API Access:** Requires valid session authentication
|
|
- **File Access:** Backups stored in `/srv/quality_app/backups` (configurable)
|
|
|
|
---
|
|
|
|
## Safety Features
|
|
|
|
### Confirmation Process for Restore:
|
|
1. **First Confirmation:** Dialog explaining what will happen
|
|
2. **Second Confirmation:** Requires typing "RESTORE DATA" in capital letters
|
|
3. **Type Detection:** Warns if trying to do full restore on data-only file
|
|
|
|
### Data Integrity:
|
|
- **Foreign key checks** disabled during restore to avoid constraint errors
|
|
- **Transaction-based** backup for consistent snapshots
|
|
- **Table truncation** ensures clean data without duplicates
|
|
- **Automatic re-enabling** of foreign key checks after restore
|
|
|
|
---
|
|
|
|
## API Endpoints
|
|
|
|
### Create Data-Only Backup
|
|
```
|
|
POST /api/backup/create-data-only
|
|
```
|
|
**Access:** Admin+
|
|
**Response:** Backup filename and size
|
|
|
|
### Restore Data-Only Backup
|
|
```
|
|
POST /api/backup/restore-data-only/<filename>
|
|
```
|
|
**Access:** Superadmin only
|
|
**Response:** Success/failure message
|
|
|
|
---
|
|
|
|
## File Naming Convention
|
|
|
|
### Data-Only Backups:
|
|
- Format: `data_only_<database>_<timestamp>.sql`
|
|
- Example: `data_only_trasabilitate_20251105_143022.sql`
|
|
|
|
### Full Backups:
|
|
- Format: `backup_<database>_<timestamp>.sql`
|
|
- Example: `backup_trasabilitate_20251105_143022.sql`
|
|
|
|
The `data_only_` prefix helps identify backup type at a glance.
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Error: "Data restore failed: Table 'X' doesn't exist"
|
|
**Cause:** Database schema not present or incomplete
|
|
**Solution:** Run full backup restore or database setup script first
|
|
|
|
### Error: "Column count doesn't match"
|
|
**Cause:** Schema structure has changed since backup was created
|
|
**Solution:** Use a newer data-only backup or update schema first
|
|
|
|
### Error: "Foreign key constraint fails"
|
|
**Cause:** Foreign key checks not properly disabled
|
|
**Solution:** Check MariaDB user has SUPER privilege
|
|
|
|
### Warning: "Could not truncate table"
|
|
**Cause:** Table has special permissions or is a view
|
|
**Solution:** Non-critical warning; restore will continue
|
|
|
|
---
|
|
|
|
## Best Practices
|
|
|
|
1. **Always keep full backups** for complete disaster recovery
|
|
2. **Use data-only backups** for frequent snapshots
|
|
3. **Test restores** in non-production environment first
|
|
4. **Document schema changes** that affect data structure
|
|
5. **Schedule both types** of backups (e.g., full weekly, data-only daily)
|
|
|
|
---
|
|
|
|
## Performance Considerations
|
|
|
|
### Backup Speed:
|
|
- **Full backup (17 tables):** ~15-30 seconds
|
|
- **Data-only backup (17 tables):** ~10-20 seconds (faster by 30-40%)
|
|
|
|
### File Size:
|
|
- **Full backup:** Includes schema (~1-2 MB) + data
|
|
- **Data-only backup:** Only data (smaller by 1-2 MB)
|
|
|
|
### Restore Speed:
|
|
- **Full restore:** Drops and recreates everything
|
|
- **Data-only restore:** Only truncates and inserts (faster on large schemas)
|
|
|
|
---
|
|
|
|
## Related Documentation
|
|
|
|
- [BACKUP_SYSTEM.md](BACKUP_SYSTEM.md) - Complete backup system overview
|
|
- [DATABASE_RESTORE_GUIDE.md](DATABASE_RESTORE_GUIDE.md) - Detailed restore procedures
|
|
- [DATABASE_STRUCTURE.md](DATABASE_STRUCTURE.md) - Database schema reference
|
|
|
|
---
|
|
|
|
## Implementation Date
|
|
|
|
**Feature Added:** November 5, 2025
|
|
**Version:** 1.1.0
|
|
**Python Module:** `app/database_backup.py`
|
|
**API Routes:** `app/routes.py` (lines 3800-3835)
|
|
**UI Template:** `app/templates/settings.html`
|