b15cc93b9d
- Fixed 3 JavaScript syntax errors in fg_scan.html (lines 951, 840-950, 1175-1215) - Restored form field validation with proper null safety checks - Re-enabled auto-advance between form fields - Re-enabled CP code auto-complete with hyphen detection - Updated validation error messages with clear format specifications and examples - Added autocomplete='off' to all input fields - Removed auto-prefix correction feature - Updated warehouse routes and modules for box assignment workflow - Added/improved database initialization scripts - Updated requirements.txt dependencies Format specifications implemented: - Operator Code: OP + 2 digits (example: OP01, OP99) - CP Code: CP + 8 digits + hyphen + 4 digits (example: CP00000000-0001) - OC1/OC2 Codes: OC + 2 digits (example: OC01, OC99) - Defect Code: 3 digits only
330 lines
12 KiB
Markdown
330 lines
12 KiB
Markdown
# Database Initialization Strategy Analysis
|
|
|
|
**Analysis Date:** January 28, 2026
|
|
|
|
---
|
|
|
|
## 🎯 Overall Strategy
|
|
|
|
You're **absolutely correct**! The application uses a **two-tier intelligent database initialization strategy**:
|
|
|
|
1. **init_db.py** → Basic initialization for fresh databases
|
|
2. **initialize_db.py** → Comprehensive initialization with automatic schema verification & repair
|
|
|
|
---
|
|
|
|
## 📊 Architecture Overview
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────┐
|
|
│ Application Startup/Deployment │
|
|
└─────────────────────┬───────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌──────────────────────────────┐
|
|
│ initialize_db.py runs │
|
|
│ (Main initialization) │
|
|
└────┬─────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────┐
|
|
│ Step 0: CHECK EXISTING DB │
|
|
│ check_and_repair_database() │
|
|
└────┬────────────────────────┘
|
|
│
|
|
├─── Database EXISTS? ──YES──┐
|
|
│ │
|
|
│ ▼───────────────┐
|
|
│ ┌─────────────────────────┤
|
|
│ │ RUN SCHEMA VERIFIER │
|
|
│ │ (db_schema_verifier.py)│
|
|
│ └─────────────────────────┘
|
|
│ │
|
|
│ ┌─────────┴─────────────┐
|
|
│ │ │
|
|
│ ▼ ▼
|
|
│ ┌────────────┐ ┌──────────────────┐
|
|
│ │ Check: │ │ REPAIR: │
|
|
│ │ - Tables │ │ - Add missing │
|
|
│ │ - Columns │ │ tables │
|
|
│ │ - Data │ │ - Add missing │
|
|
│ └────────────┘ │ columns │
|
|
│ │ - Add missing │
|
|
│ │ data │
|
|
│ └──────────────────┘
|
|
│
|
|
└─── Database NEW? ──NO──┐
|
|
│
|
|
▼
|
|
┌──────────────────────┐
|
|
│ Skip verification │
|
|
│ (start from scratch) │
|
|
└──────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ Step 1: Create Database (if not exists) │
|
|
│ CREATE DATABASE IF NOT EXISTS quality_db │
|
|
└─────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ Step 2: Create ALL Tables (with FK & Indexes) │
|
|
│ - 18+ tables including: │
|
|
│ ✅ boxes_crates │
|
|
│ ✅ box_contents │
|
|
│ ✅ scanfg_orders (WITH location_id & box_id) │
|
|
│ ✅ cp_location_history │
|
|
│ ✅ warehouse_locations │
|
|
│ + 13 more... │
|
|
└─────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ Step 3: Insert Default Data │
|
|
│ - Create default roles │
|
|
│ - Create admin user │
|
|
│ - Create warehouse locations │
|
|
│ - Initialize permissions │
|
|
└─────────────────────────────────────────────────────────┘
|
|
│
|
|
▼
|
|
┌─────────────────────────────────────────────────────────┐
|
|
│ ✅ Database Ready for Application │
|
|
└─────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## 🔍 How Schema Verification Works
|
|
|
|
### Located in: `/srv/quality_app-v2/app/db_schema_verifier.py`
|
|
|
|
The **SchemaVerifier** class automatically:
|
|
|
|
1. **Checks if database exists**
|
|
- If NEW: Skip verification, create from scratch
|
|
- If EXISTING: Run verification and repair
|
|
|
|
2. **Verifies Tables**
|
|
```python
|
|
def verify_tables(self):
|
|
"""Verify all required tables exist"""
|
|
# For each required table:
|
|
# - Check if it exists
|
|
# - If missing, CREATE it
|
|
# - If exists, verify structure
|
|
```
|
|
|
|
3. **Verifies Columns**
|
|
```python
|
|
def verify_columns(self):
|
|
"""Verify all required columns exist in each table"""
|
|
# For each table:
|
|
# - Get existing columns
|
|
# - Compare with required columns
|
|
# - If missing, ADD them with ALTER TABLE
|
|
# - If type mismatch, UPDATE column type
|
|
```
|
|
|
|
4. **Verifies Reference Data**
|
|
```python
|
|
def verify_reference_data(self):
|
|
"""Ensure required data exists"""
|
|
# - Check roles exist
|
|
# - Check admin user exists
|
|
# - Check warehouse locations exist
|
|
# - Add missing data
|
|
```
|
|
|
|
---
|
|
|
|
## 📋 Comparison: init_db.py vs initialize_db.py
|
|
|
|
| Feature | init_db.py | initialize_db.py |
|
|
|---------|-----------|------------------|
|
|
| **Purpose** | Basic initialization | Comprehensive with verification |
|
|
| **Database Check** | ❌ Creates new only | ✅ Checks & verifies existing |
|
|
| **Schema Repair** | ❌ NO | ✅ YES (via SchemaVerifier) |
|
|
| **Add Missing Tables** | ❌ NO | ✅ YES |
|
|
| **Add Missing Columns** | ❌ NO | ✅ YES (location_id, box_id) |
|
|
| **Add Missing Data** | ❌ NO | ✅ YES |
|
|
| **Tables Created** | 9 | 18+ |
|
|
| **Has scanfg_orders** | ❌ NO | ✅ YES (with location_id) |
|
|
| **Deployment Ready** | ⚠️ Partial | ✅ Full |
|
|
| **Handles Upgrades** | ❌ NO | ✅ YES |
|
|
|
|
---
|
|
|
|
## 🚀 Deployment Flow
|
|
|
|
### Scenario 1: Fresh Database (NEW Installation)
|
|
```
|
|
1. Run initialize_db.py
|
|
2. check_and_repair_database() → Database doesn't exist yet
|
|
3. Skip verification (no existing db to check)
|
|
4. Create fresh database
|
|
5. Create all 18+ tables
|
|
6. Insert default data
|
|
7. Application starts with complete schema
|
|
✅ Status: Ready
|
|
```
|
|
|
|
### Scenario 2: Existing Database (UPGRADE/PATCH)
|
|
```
|
|
1. Run initialize_db.py (again, for updates)
|
|
2. check_and_repair_database() → Database exists
|
|
3. Connect to existing database
|
|
4. Run SchemaVerifier.verify_and_repair()
|
|
5. Check all tables:
|
|
- scanfg_orders exists? ✅ (YES)
|
|
- location_id column exists? ✅ (YES, if added before)
|
|
- box_id column exists? ✅ (YES, if added before)
|
|
6. If missing:
|
|
- ADD location_id column
|
|
- ADD indexes
|
|
- CREATE missing tables
|
|
7. Application starts with enhanced schema
|
|
✅ Status: Updated
|
|
```
|
|
|
|
### Scenario 3: Partial Update (Some New Columns Added)
|
|
```
|
|
1. Database has scanfg_orders but NO location_id
|
|
2. Run initialize_db.py
|
|
3. SchemaVerifier detects missing location_id
|
|
4. Automatically runs:
|
|
ALTER TABLE scanfg_orders ADD COLUMN location_id BIGINT;
|
|
ALTER TABLE scanfg_orders ADD INDEX idx_location_id (location_id);
|
|
ALTER TABLE scanfg_orders ADD FOREIGN KEY...;
|
|
5. Application starts with complete schema
|
|
✅ Status: Patched
|
|
```
|
|
|
|
---
|
|
|
|
## 🔧 How It Handles location_id Field
|
|
|
|
### When location_id is Missing:
|
|
|
|
**In db_schema_verifier.py verify_columns():**
|
|
```python
|
|
# For scanfg_orders table:
|
|
required_columns = {
|
|
'location_id': {
|
|
'type': 'BIGINT',
|
|
'nullable': True,
|
|
'key': 'MUL'
|
|
},
|
|
'box_id': {
|
|
'type': 'BIGINT',
|
|
'nullable': True,
|
|
'key': 'MUL'
|
|
},
|
|
# ... other columns
|
|
}
|
|
|
|
# Check existing columns
|
|
existing = get_table_columns('scanfg_orders')
|
|
|
|
# If location_id missing:
|
|
if 'location_id' not in existing:
|
|
# Automatically add it!
|
|
ALTER TABLE scanfg_orders
|
|
ADD COLUMN location_id BIGINT;
|
|
|
|
self.changes_made.append('Added location_id column to scanfg_orders')
|
|
```
|
|
|
|
---
|
|
|
|
## ✅ Current Status (Your Database)
|
|
|
|
**Deployment Method:** ✅ initialize_db.py (confirmed)
|
|
|
|
**Verification Results:**
|
|
```
|
|
✅ scanfg_orders table EXISTS
|
|
✅ location_id column EXISTS
|
|
✅ box_id column EXISTS
|
|
✅ Foreign key constraints EXISTS
|
|
✅ Indexes EXISTS
|
|
✅ Ready for production
|
|
```
|
|
|
|
---
|
|
|
|
## 🎯 Why This Two-File Strategy?
|
|
|
|
### **init_db.py (Legacy/Minimal)**
|
|
- ✅ Simple, quick initialization
|
|
- ✅ Creates core user/role tables
|
|
- ❌ Doesn't support box tracking
|
|
- ❌ No upgrade path
|
|
- ❌ No schema verification
|
|
|
|
### **initialize_db.py (Production)**
|
|
- ✅ Complete application setup
|
|
- ✅ Supports box tracking features
|
|
- ✅ Auto-detects and repairs schema
|
|
- ✅ Upgradeable
|
|
- ✅ Safe for existing databases
|
|
- ✅ Professional deployment
|
|
|
|
---
|
|
|
|
## 🔄 Upgrade Path Example
|
|
|
|
**Suppose you deployed with init_db.py 6 months ago...**
|
|
|
|
```
|
|
Initial state:
|
|
- Database exists with basic tables
|
|
- scanfg_orders table MISSING
|
|
- location_id field MISSING
|
|
|
|
Today, you run initialize_db.py:
|
|
|
|
Step 1: check_and_repair_database()
|
|
↓
|
|
Database exists → Run SchemaVerifier
|
|
↓
|
|
Check scanfg_orders → Missing!
|
|
↓
|
|
Create scanfg_orders table with all fields
|
|
↓
|
|
Create location_id column
|
|
↓
|
|
Create foreign key constraint
|
|
↓
|
|
Create indexes
|
|
|
|
Result:
|
|
✅ Database upgraded safely
|
|
✅ No data loss
|
|
✅ New features available
|
|
✅ Ready for box tracking
|
|
```
|
|
|
|
---
|
|
|
|
## 📝 Key Takeaway
|
|
|
|
**Your understanding is correct!**
|
|
|
|
The architecture uses:
|
|
1. **initialize_db.py** as the main deployment script
|
|
2. **check_and_repair_database()** to detect existing databases
|
|
3. **SchemaVerifier** class to verify and repair schema
|
|
4. **db_schema_verifier.py** to handle missing tables, columns, and data
|
|
|
|
This allows the application to:
|
|
- ✅ Work with fresh databases
|
|
- ✅ Work with existing databases
|
|
- ✅ Automatically repair missing schema elements (like location_id)
|
|
- ✅ Support upgrades without data loss
|
|
- ✅ Add new features incrementally
|
|
|
|
**Always use `initialize_db.py` for deployment**, not `init_db.py`.
|
|
|