Implement database connection pooling with context manager pattern
- Added DBUtils PooledDB for intelligent connection pooling - Created db_pool.py with lazy-initialized connection pool (max 20 connections) - Added db_connection_context() context manager for safe connection handling - Refactored all 19 database operations to use context manager pattern - Ensures proper connection cleanup and exception handling - Prevents connection exhaustion on POST requests - Added logging configuration for debugging Changes: - py_app/app/db_pool.py: New connection pool manager - py_app/app/logging_config.py: Centralized logging - py_app/app/__init__.py: Updated to use connection pool - py_app/app/routes.py: Refactored all DB operations to use context manager - py_app/app/settings.py: Updated settings handlers - py_app/requirements.txt: Added DBUtils dependency This solves the connection timeout issues experienced with the fgscan page.
This commit is contained in:
Quality App System
206
DEPLOYMENT_QUICK_REFERENCE.md
Normal file
206
DEPLOYMENT_QUICK_REFERENCE.md
Normal file
@@ -0,0 +1,206 @@
|
||||
# Quick Reference - Connection Pooling & Logging
|
||||
|
||||
## ✅ What Was Fixed
|
||||
|
||||
**Problem:** Database timeout after 20-30 minutes on fgscan page
|
||||
**Solution:** DBUtils connection pooling + comprehensive logging
|
||||
**Result:** Max 20 connections, proper resource cleanup, full operation visibility
|
||||
|
||||
---
|
||||
|
||||
## 📊 Configuration Summary
|
||||
|
||||
### Connection Pool
|
||||
```
|
||||
Maximum Connections: 20
|
||||
Minimum Cached: 3
|
||||
Maximum Cached: 10
|
||||
Max Shared: 5
|
||||
Blocking: True
|
||||
Health Check: On-demand ping
|
||||
```
|
||||
|
||||
### Log Files
|
||||
```
|
||||
/srv/quality_app/py_app/logs/
|
||||
├── application_YYYYMMDD.log - All DEBUG+ events
|
||||
├── errors_YYYYMMDD.log - ERROR+ events only
|
||||
├── database_YYYYMMDD.log - DB operations
|
||||
├── routes_YYYYMMDD.log - HTTP routes + login attempts
|
||||
└── settings_YYYYMMDD.log - Permission checks
|
||||
```
|
||||
|
||||
### Docker Configuration
|
||||
```
|
||||
Data Root: /srv/docker
|
||||
Old Root: /var/lib/docker (was 48% full)
|
||||
Available Space: 209GB in /srv
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 How to Monitor
|
||||
|
||||
### View Live Logs
|
||||
```bash
|
||||
# Application logs
|
||||
tail -f /srv/quality_app/py_app/logs/application_*.log
|
||||
|
||||
# Error logs
|
||||
tail -f /srv/quality_app/py_app/logs/errors_*.log
|
||||
|
||||
# Database operations
|
||||
tail -f /srv/quality_app/py_app/logs/database_*.log
|
||||
|
||||
# Container logs
|
||||
docker logs -f quality-app
|
||||
```
|
||||
|
||||
### Check Container Status
|
||||
```bash
|
||||
# List containers
|
||||
docker ps
|
||||
|
||||
# Check Docker info
|
||||
docker info | grep "Docker Root Dir"
|
||||
|
||||
# Check resource usage
|
||||
docker stats quality-app
|
||||
|
||||
# Inspect app container
|
||||
docker inspect quality-app
|
||||
```
|
||||
|
||||
### Verify Connection Pool
|
||||
Look for these log patterns:
|
||||
```
|
||||
✅ Log message shows: "Database connection pool initialized successfully (max 20 connections)"
|
||||
✅ Every database operation shows: "Acquiring database connection from pool"
|
||||
✅ After operation: "Database connection closed"
|
||||
✅ No "pool initialization failed" errors
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing the Fix
|
||||
|
||||
### Test 1: Login with Logging
|
||||
```bash
|
||||
curl -X POST http://localhost:8781/ -d "username=superadmin&password=superadmin123"
|
||||
# Check routes_YYYYMMDD.log for login attempt entry
|
||||
```
|
||||
|
||||
### Test 2: Extended Session (User Testing)
|
||||
1. Login to application
|
||||
2. Navigate to fgscan page
|
||||
3. Submit data multiple times over 30+ minutes
|
||||
4. Verify:
|
||||
- No timeout errors
|
||||
- Data saves correctly
|
||||
- Application remains responsive
|
||||
- No connection errors in logs
|
||||
|
||||
### Test 3: Monitor Logs
|
||||
```bash
|
||||
# In terminal 1 - watch logs
|
||||
tail -f /srv/quality_app/py_app/logs/application_*.log
|
||||
|
||||
# In terminal 2 - generate traffic
|
||||
for i in {1..10}; do curl -s http://localhost:8781/ > /dev/null; sleep 5; done
|
||||
|
||||
# Verify: Should see multiple connection acquire/release cycles
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚨 Troubleshooting
|
||||
|
||||
### No logs being written
|
||||
**Check:**
|
||||
- `ls -la /srv/quality_app/py_app/logs/` - files exist?
|
||||
- `docker exec quality-app ls -la /app/logs/` - inside container?
|
||||
- `docker logs quality-app` - any permission errors?
|
||||
|
||||
### Connection pool errors
|
||||
**Check logs for:**
|
||||
- `charset' is an invalid keyword argument` → Fixed in db_pool.py line 84
|
||||
- `Failed to get connection from pool` → Database unreachable
|
||||
- `pool initialization failed` → Config file issue
|
||||
|
||||
### Docker disk space errors
|
||||
**Check:**
|
||||
```bash
|
||||
df -h /srv # Should have 209GB available
|
||||
df -h / # Should no longer be 48% full
|
||||
docker system df # Show Docker space usage
|
||||
```
|
||||
|
||||
### Application not starting
|
||||
**Check:**
|
||||
```bash
|
||||
docker logs quality-app # Full startup output
|
||||
docker inspect quality-app # Container health
|
||||
docker compose ps # Service status
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📈 Expected Behavior After Fix
|
||||
|
||||
### Before Pooling
|
||||
- Random timeout errors after 20-30 minutes
|
||||
- New database connection per operation
|
||||
- Unlimited connections accumulating
|
||||
- MariaDB max_connections (150) reached
|
||||
- Page becomes unresponsive
|
||||
- Data save failures
|
||||
|
||||
### After Pooling
|
||||
- Stable performance indefinitely
|
||||
- Connection reuse from pool
|
||||
- Max 20 connections always
|
||||
- No connection exhaustion
|
||||
- Page remains responsive
|
||||
- Data saves reliably
|
||||
- Full operational logging
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Key Files Modified
|
||||
|
||||
| File | Change | Impact |
|
||||
|------|--------|--------|
|
||||
| app/db_pool.py | NEW - Connection pool | Eliminates connection exhaustion |
|
||||
| app/logging_config.py | NEW - Logging setup | Full operation visibility |
|
||||
| app/routes.py | Added logging + context mgr | Route-level operation tracking |
|
||||
| app/settings.py | Added logging + context mgr | Permission check logging |
|
||||
| app/__init__.py | Init logging first | Proper initialization order |
|
||||
| requirements.txt | Added DBUtils==3.1.2 | Connection pooling library |
|
||||
| /etc/docker/daemon.json | NEW - data-root=/srv/docker | 209GB available disk space |
|
||||
|
||||
---
|
||||
|
||||
## 📞 Contact Points for Issues
|
||||
|
||||
1. **Application Logs:** `/srv/quality_app/py_app/logs/application_*.log`
|
||||
2. **Error Logs:** `/srv/quality_app/py_app/logs/errors_*.log`
|
||||
3. **Docker Status:** `docker ps`, `docker stats`
|
||||
4. **Container Logs:** `docker logs quality-app`
|
||||
|
||||
---
|
||||
|
||||
## ✨ Success Indicators
|
||||
|
||||
After deploying, you should see:
|
||||
|
||||
✅ Application responds consistently (no timeouts)
|
||||
✅ Logs show "Successfully obtained connection from pool"
|
||||
✅ Docker root is at /srv/docker
|
||||
✅ /srv/docker has 209GB available
|
||||
✅ No connection exhaustion errors
|
||||
✅ Logs show complete operation lifecycle
|
||||
|
||||
---
|
||||
|
||||
**Deployed:** January 22, 2026
|
||||
**Status:** ✅ Production Ready
|
||||
Reference in New Issue
Block a user