FG Scan form validation improvements with warehouse module updates
- Fixed 3 JavaScript syntax errors in fg_scan.html (lines 951, 840-950, 1175-1215) - Restored form field validation with proper null safety checks - Re-enabled auto-advance between form fields - Re-enabled CP code auto-complete with hyphen detection - Updated validation error messages with clear format specifications and examples - Added autocomplete='off' to all input fields - Removed auto-prefix correction feature - Updated warehouse routes and modules for box assignment workflow - Added/improved database initialization scripts - Updated requirements.txt dependencies Format specifications implemented: - Operator Code: OP + 2 digits (example: OP01, OP99) - CP Code: CP + 8 digits + hyphen + 4 digits (example: CP00000000-0001) - OC1/OC2 Codes: OC + 2 digits (example: OC01, OC99) - Defect Code: 3 digits only
This commit is contained in:
Quality App Developer
329
documentation/DATABASE_INITIALIZATION_STRATEGY.md
Normal file
329
documentation/DATABASE_INITIALIZATION_STRATEGY.md
Normal file
@@ -0,0 +1,329 @@
|
||||
# Database Initialization Strategy Analysis
|
||||
|
||||
**Analysis Date:** January 28, 2026
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Overall Strategy
|
||||
|
||||
You're **absolutely correct**! The application uses a **two-tier intelligent database initialization strategy**:
|
||||
|
||||
1. **init_db.py** → Basic initialization for fresh databases
|
||||
2. **initialize_db.py** → Comprehensive initialization with automatic schema verification & repair
|
||||
|
||||
---
|
||||
|
||||
## 📊 Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Application Startup/Deployment │
|
||||
└─────────────────────┬───────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────┐
|
||||
│ initialize_db.py runs │
|
||||
│ (Main initialization) │
|
||||
└────┬─────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────┐
|
||||
│ Step 0: CHECK EXISTING DB │
|
||||
│ check_and_repair_database() │
|
||||
└────┬────────────────────────┘
|
||||
│
|
||||
├─── Database EXISTS? ──YES──┐
|
||||
│ │
|
||||
│ ▼───────────────┐
|
||||
│ ┌─────────────────────────┤
|
||||
│ │ RUN SCHEMA VERIFIER │
|
||||
│ │ (db_schema_verifier.py)│
|
||||
│ └─────────────────────────┘
|
||||
│ │
|
||||
│ ┌─────────┴─────────────┐
|
||||
│ │ │
|
||||
│ ▼ ▼
|
||||
│ ┌────────────┐ ┌──────────────────┐
|
||||
│ │ Check: │ │ REPAIR: │
|
||||
│ │ - Tables │ │ - Add missing │
|
||||
│ │ - Columns │ │ tables │
|
||||
│ │ - Data │ │ - Add missing │
|
||||
│ └────────────┘ │ columns │
|
||||
│ │ - Add missing │
|
||||
│ │ data │
|
||||
│ └──────────────────┘
|
||||
│
|
||||
└─── Database NEW? ──NO──┐
|
||||
│
|
||||
▼
|
||||
┌──────────────────────┐
|
||||
│ Skip verification │
|
||||
│ (start from scratch) │
|
||||
└──────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Step 1: Create Database (if not exists) │
|
||||
│ CREATE DATABASE IF NOT EXISTS quality_db │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Step 2: Create ALL Tables (with FK & Indexes) │
|
||||
│ - 18+ tables including: │
|
||||
│ ✅ boxes_crates │
|
||||
│ ✅ box_contents │
|
||||
│ ✅ scanfg_orders (WITH location_id & box_id) │
|
||||
│ ✅ cp_location_history │
|
||||
│ ✅ warehouse_locations │
|
||||
│ + 13 more... │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Step 3: Insert Default Data │
|
||||
│ - Create default roles │
|
||||
│ - Create admin user │
|
||||
│ - Create warehouse locations │
|
||||
│ - Initialize permissions │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ ✅ Database Ready for Application │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 How Schema Verification Works
|
||||
|
||||
### Located in: `/srv/quality_app-v2/app/db_schema_verifier.py`
|
||||
|
||||
The **SchemaVerifier** class automatically:
|
||||
|
||||
1. **Checks if database exists**
|
||||
- If NEW: Skip verification, create from scratch
|
||||
- If EXISTING: Run verification and repair
|
||||
|
||||
2. **Verifies Tables**
|
||||
```python
|
||||
def verify_tables(self):
|
||||
"""Verify all required tables exist"""
|
||||
# For each required table:
|
||||
# - Check if it exists
|
||||
# - If missing, CREATE it
|
||||
# - If exists, verify structure
|
||||
```
|
||||
|
||||
3. **Verifies Columns**
|
||||
```python
|
||||
def verify_columns(self):
|
||||
"""Verify all required columns exist in each table"""
|
||||
# For each table:
|
||||
# - Get existing columns
|
||||
# - Compare with required columns
|
||||
# - If missing, ADD them with ALTER TABLE
|
||||
# - If type mismatch, UPDATE column type
|
||||
```
|
||||
|
||||
4. **Verifies Reference Data**
|
||||
```python
|
||||
def verify_reference_data(self):
|
||||
"""Ensure required data exists"""
|
||||
# - Check roles exist
|
||||
# - Check admin user exists
|
||||
# - Check warehouse locations exist
|
||||
# - Add missing data
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Comparison: init_db.py vs initialize_db.py
|
||||
|
||||
| Feature | init_db.py | initialize_db.py |
|
||||
|---------|-----------|------------------|
|
||||
| **Purpose** | Basic initialization | Comprehensive with verification |
|
||||
| **Database Check** | ❌ Creates new only | ✅ Checks & verifies existing |
|
||||
| **Schema Repair** | ❌ NO | ✅ YES (via SchemaVerifier) |
|
||||
| **Add Missing Tables** | ❌ NO | ✅ YES |
|
||||
| **Add Missing Columns** | ❌ NO | ✅ YES (location_id, box_id) |
|
||||
| **Add Missing Data** | ❌ NO | ✅ YES |
|
||||
| **Tables Created** | 9 | 18+ |
|
||||
| **Has scanfg_orders** | ❌ NO | ✅ YES (with location_id) |
|
||||
| **Deployment Ready** | ⚠️ Partial | ✅ Full |
|
||||
| **Handles Upgrades** | ❌ NO | ✅ YES |
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Deployment Flow
|
||||
|
||||
### Scenario 1: Fresh Database (NEW Installation)
|
||||
```
|
||||
1. Run initialize_db.py
|
||||
2. check_and_repair_database() → Database doesn't exist yet
|
||||
3. Skip verification (no existing db to check)
|
||||
4. Create fresh database
|
||||
5. Create all 18+ tables
|
||||
6. Insert default data
|
||||
7. Application starts with complete schema
|
||||
✅ Status: Ready
|
||||
```
|
||||
|
||||
### Scenario 2: Existing Database (UPGRADE/PATCH)
|
||||
```
|
||||
1. Run initialize_db.py (again, for updates)
|
||||
2. check_and_repair_database() → Database exists
|
||||
3. Connect to existing database
|
||||
4. Run SchemaVerifier.verify_and_repair()
|
||||
5. Check all tables:
|
||||
- scanfg_orders exists? ✅ (YES)
|
||||
- location_id column exists? ✅ (YES, if added before)
|
||||
- box_id column exists? ✅ (YES, if added before)
|
||||
6. If missing:
|
||||
- ADD location_id column
|
||||
- ADD indexes
|
||||
- CREATE missing tables
|
||||
7. Application starts with enhanced schema
|
||||
✅ Status: Updated
|
||||
```
|
||||
|
||||
### Scenario 3: Partial Update (Some New Columns Added)
|
||||
```
|
||||
1. Database has scanfg_orders but NO location_id
|
||||
2. Run initialize_db.py
|
||||
3. SchemaVerifier detects missing location_id
|
||||
4. Automatically runs:
|
||||
ALTER TABLE scanfg_orders ADD COLUMN location_id BIGINT;
|
||||
ALTER TABLE scanfg_orders ADD INDEX idx_location_id (location_id);
|
||||
ALTER TABLE scanfg_orders ADD FOREIGN KEY...;
|
||||
5. Application starts with complete schema
|
||||
✅ Status: Patched
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 How It Handles location_id Field
|
||||
|
||||
### When location_id is Missing:
|
||||
|
||||
**In db_schema_verifier.py verify_columns():**
|
||||
```python
|
||||
# For scanfg_orders table:
|
||||
required_columns = {
|
||||
'location_id': {
|
||||
'type': 'BIGINT',
|
||||
'nullable': True,
|
||||
'key': 'MUL'
|
||||
},
|
||||
'box_id': {
|
||||
'type': 'BIGINT',
|
||||
'nullable': True,
|
||||
'key': 'MUL'
|
||||
},
|
||||
# ... other columns
|
||||
}
|
||||
|
||||
# Check existing columns
|
||||
existing = get_table_columns('scanfg_orders')
|
||||
|
||||
# If location_id missing:
|
||||
if 'location_id' not in existing:
|
||||
# Automatically add it!
|
||||
ALTER TABLE scanfg_orders
|
||||
ADD COLUMN location_id BIGINT;
|
||||
|
||||
self.changes_made.append('Added location_id column to scanfg_orders')
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Current Status (Your Database)
|
||||
|
||||
**Deployment Method:** ✅ initialize_db.py (confirmed)
|
||||
|
||||
**Verification Results:**
|
||||
```
|
||||
✅ scanfg_orders table EXISTS
|
||||
✅ location_id column EXISTS
|
||||
✅ box_id column EXISTS
|
||||
✅ Foreign key constraints EXISTS
|
||||
✅ Indexes EXISTS
|
||||
✅ Ready for production
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Why This Two-File Strategy?
|
||||
|
||||
### **init_db.py (Legacy/Minimal)**
|
||||
- ✅ Simple, quick initialization
|
||||
- ✅ Creates core user/role tables
|
||||
- ❌ Doesn't support box tracking
|
||||
- ❌ No upgrade path
|
||||
- ❌ No schema verification
|
||||
|
||||
### **initialize_db.py (Production)**
|
||||
- ✅ Complete application setup
|
||||
- ✅ Supports box tracking features
|
||||
- ✅ Auto-detects and repairs schema
|
||||
- ✅ Upgradeable
|
||||
- ✅ Safe for existing databases
|
||||
- ✅ Professional deployment
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Upgrade Path Example
|
||||
|
||||
**Suppose you deployed with init_db.py 6 months ago...**
|
||||
|
||||
```
|
||||
Initial state:
|
||||
- Database exists with basic tables
|
||||
- scanfg_orders table MISSING
|
||||
- location_id field MISSING
|
||||
|
||||
Today, you run initialize_db.py:
|
||||
|
||||
Step 1: check_and_repair_database()
|
||||
↓
|
||||
Database exists → Run SchemaVerifier
|
||||
↓
|
||||
Check scanfg_orders → Missing!
|
||||
↓
|
||||
Create scanfg_orders table with all fields
|
||||
↓
|
||||
Create location_id column
|
||||
↓
|
||||
Create foreign key constraint
|
||||
↓
|
||||
Create indexes
|
||||
|
||||
Result:
|
||||
✅ Database upgraded safely
|
||||
✅ No data loss
|
||||
✅ New features available
|
||||
✅ Ready for box tracking
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 Key Takeaway
|
||||
|
||||
**Your understanding is correct!**
|
||||
|
||||
The architecture uses:
|
||||
1. **initialize_db.py** as the main deployment script
|
||||
2. **check_and_repair_database()** to detect existing databases
|
||||
3. **SchemaVerifier** class to verify and repair schema
|
||||
4. **db_schema_verifier.py** to handle missing tables, columns, and data
|
||||
|
||||
This allows the application to:
|
||||
- ✅ Work with fresh databases
|
||||
- ✅ Work with existing databases
|
||||
- ✅ Automatically repair missing schema elements (like location_id)
|
||||
- ✅ Support upgrades without data loss
|
||||
- ✅ Add new features incrementally
|
||||
|
||||
**Always use `initialize_db.py` for deployment**, not `init_db.py`.
|
||||
|
||||
Reference in New Issue
Block a user