quality_app/DEPLOYMENT_QUICK_REFERENCE.md

Quick Reference - Connection Pooling & Logging

✅ What Was Fixed

Problem: Database timeout after 20-30 minutes on fgscan page
Solution: DBUtils connection pooling + comprehensive logging
Result: Max 20 connections, proper resource cleanup, full operation visibility

---

📊 Configuration Summary

Connection Pool

Maximum Connections: 20
Minimum Cached: 3
Maximum Cached: 10
Max Shared: 5
Blocking: True
Health Check: On-demand ping

Log Files

/srv/quality_app/py_app/logs/
├── application_YYYYMMDD.log    - All DEBUG+ events
├── errors_YYYYMMDD.log         - ERROR+ events only
├── database_YYYYMMDD.log       - DB operations
├── routes_YYYYMMDD.log         - HTTP routes + login attempts
└── settings_YYYYMMDD.log       - Permission checks

Docker Configuration

Data Root: /srv/docker
Old Root: /var/lib/docker (was 48% full)
Available Space: 209GB in /srv

---

🔍 How to Monitor

View Live Logs

# Application logs
tail -f /srv/quality_app/py_app/logs/application_*.log

# Error logs
tail -f /srv/quality_app/py_app/logs/errors_*.log

# Database operations
tail -f /srv/quality_app/py_app/logs/database_*.log

# Container logs
docker logs -f quality-app

Check Container Status

# List containers
docker ps

# Check Docker info
docker info | grep "Docker Root Dir"

# Check resource usage
docker stats quality-app

# Inspect app container
docker inspect quality-app

Verify Connection Pool

Look for these log patterns:

✅ Log message shows: "Database connection pool initialized successfully (max 20 connections)"
✅ Every database operation shows: "Acquiring database connection from pool"
✅ After operation: "Database connection closed"
✅ No "pool initialization failed" errors

---

🧪 Testing the Fix

Test 1: Login with Logging

curl -X POST http://localhost:8781/ -d "username=superadmin&password=superadmin123"
# Check routes_YYYYMMDD.log for login attempt entry

Test 2: Extended Session (User Testing)

1. Login to application
2. Navigate to fgscan page
3. Submit data multiple times over 30+ minutes
4. Verify:
   • No timeout errors
   • Data saves correctly
   • Application remains responsive
   • No connection errors in logs

Test 3: Monitor Logs

# In terminal 1 - watch logs
tail -f /srv/quality_app/py_app/logs/application_*.log

# In terminal 2 - generate traffic
for i in {1..10}; do curl -s http://localhost:8781/ > /dev/null; sleep 5; done

# Verify: Should see multiple connection acquire/release cycles

---

🚨 Troubleshooting

No logs being written

Check:

• ls -la /srv/quality_app/py_app/logs/ - files exist?
• docker exec quality-app ls -la /app/logs/ - inside container?
• docker logs quality-app - any permission errors?

Connection pool errors

Check logs for:

• charset' is an invalid keyword argument → Fixed in db_pool.py line 84
• Failed to get connection from pool → Database unreachable
• pool initialization failed → Config file issue

Docker disk space errors

Check:

df -h /srv           # Should have 209GB available
df -h /             # Should no longer be 48% full
docker system df    # Show Docker space usage

Application not starting

Check:

docker logs quality-app            # Full startup output
docker inspect quality-app         # Container health
docker compose ps                  # Service status

---

📈 Expected Behavior After Fix

Before Pooling

• Random timeout errors after 20-30 minutes
• New database connection per operation
• Unlimited connections accumulating
• MariaDB max_connections (150) reached
• Page becomes unresponsive
• Data save failures

After Pooling

• Stable performance indefinitely
• Connection reuse from pool
• Max 20 connections always
• No connection exhaustion
• Page remains responsive
• Data saves reliably
• Full operational logging

---

🔧 Key Files Modified

File	Change	Impact
app/db_pool.py	NEW - Connection pool	Eliminates connection exhaustion
app/logging_config.py	NEW - Logging setup	Full operation visibility
app/routes.py	Added logging + context mgr	Route-level operation tracking
app/settings.py	Added logging + context mgr	Permission check logging
app/init.py	Init logging first	Proper initialization order
requirements.txt	Added DBUtils==3.1.2	Connection pooling library
/etc/docker/daemon.json	NEW - data-root=/srv/docker	209GB available disk space

---

📞 Contact Points for Issues

1. Application Logs: /srv/quality_app/py_app/logs/application_*.log
2. Error Logs: /srv/quality_app/py_app/logs/errors_*.log
3. Docker Status: docker ps, docker stats
4. Container Logs: docker logs quality-app

---

✨ Success Indicators

After deploying, you should see:

✅ Application responds consistently (no timeouts)
✅ Logs show "Successfully obtained connection from pool"
✅ Docker root is at /srv/docker
✅ /srv/docker has 209GB available
✅ No connection exhaustion errors
✅ Logs show complete operation lifecycle

---

Deployed: January 22, 2026
Status: ✅ Production Ready