619 lines
14 KiB
Markdown
619 lines
14 KiB
Markdown
# Production Startup Guide
|
||
|
||
## Overview
|
||
This guide covers starting, stopping, and managing the Quality Recticel application in production using the provided management scripts.
|
||
|
||
## Quick Start
|
||
|
||
### Start Application
|
||
```bash
|
||
cd /srv/quality_app/py_app
|
||
bash start_production.sh
|
||
```
|
||
|
||
### Stop Application
|
||
```bash
|
||
cd /srv/quality_app/py_app
|
||
bash stop_production.sh
|
||
```
|
||
|
||
### Check Status
|
||
```bash
|
||
cd /srv/quality_app/py_app
|
||
bash status_production.sh
|
||
```
|
||
|
||
## Management Scripts
|
||
|
||
### start_production.sh
|
||
Production startup script that launches the application using Gunicorn WSGI server.
|
||
|
||
**Features**:
|
||
- ✅ Validates prerequisites (virtual environment, Gunicorn)
|
||
- ✅ Tests database connection before starting
|
||
- ✅ Auto-detects project location (quality_app vs quality_recticel)
|
||
- ✅ Creates PID file for process management
|
||
- ✅ Starts Gunicorn in daemon mode (background)
|
||
- ✅ Displays comprehensive startup information
|
||
|
||
**Prerequisites Checked**:
|
||
1. Virtual environment exists (`../recticel`)
|
||
2. Gunicorn is installed
|
||
3. Database connection is working
|
||
4. No existing instance running
|
||
|
||
**Configuration**:
|
||
- **Workers**: CPU count × 2 + 1 (default: 9 workers)
|
||
- **Port**: 8781
|
||
- **Bind**: 0.0.0.0 (all interfaces)
|
||
- **Config**: gunicorn.conf.py
|
||
- **Timeout**: 1800 seconds (30 minutes)
|
||
- **Max Upload**: 10GB
|
||
|
||
**Output Example**:
|
||
```
|
||
🚀 Trasabilitate Application - Production Startup
|
||
==============================================
|
||
|
||
📋 Checking Prerequisites
|
||
----------------------------------------
|
||
✅ Virtual environment found
|
||
✅ Gunicorn is available
|
||
✅ Database connection verified
|
||
|
||
📋 Starting Production Server
|
||
----------------------------------------
|
||
Starting Gunicorn WSGI server...
|
||
Configuration: gunicorn.conf.py
|
||
Workers: 9
|
||
Binding to: 0.0.0.0:8781
|
||
|
||
✅ Application started successfully!
|
||
|
||
==============================================
|
||
🎉 PRODUCTION SERVER RUNNING
|
||
==============================================
|
||
|
||
📋 Server Information:
|
||
• Process ID: 402172
|
||
• Configuration: gunicorn.conf.py
|
||
• Project: quality_app
|
||
• Access Log: /srv/quality_app/logs/access.log
|
||
• Error Log: /srv/quality_app/logs/error.log
|
||
|
||
🌐 Application URLs:
|
||
• Local: http://127.0.0.1:8781
|
||
• Network: http://192.168.0.205:8781
|
||
|
||
👤 Default Login:
|
||
• Username: superadmin
|
||
• Password: superadmin123
|
||
|
||
🔧 Management Commands:
|
||
• Stop server: kill 402172 && rm ../run/trasabilitate.pid
|
||
• View logs: tail -f /srv/quality_app/logs/error.log
|
||
• Monitor access: tail -f /srv/quality_app/logs/access.log
|
||
• Server status: ps -p 402172
|
||
|
||
⚠️ Server is running in daemon mode (background)
|
||
```
|
||
|
||
### stop_production.sh
|
||
Gracefully stops the running application.
|
||
|
||
**Features**:
|
||
- ✅ Reads PID from file
|
||
- ✅ Sends SIGTERM (graceful shutdown)
|
||
- ✅ Waits 3 seconds for graceful exit
|
||
- ✅ Falls back to SIGKILL if needed
|
||
- ✅ Cleans up PID file
|
||
|
||
**Process**:
|
||
1. Checks if PID file exists
|
||
2. Verifies process is running
|
||
3. Sends SIGTERM signal
|
||
4. Waits for graceful shutdown
|
||
5. Uses SIGKILL if process doesn't stop
|
||
6. Removes PID file
|
||
|
||
**Output Example**:
|
||
```
|
||
🛑 Trasabilitate Application - Production Stop
|
||
==============================================
|
||
Stopping Trasabilitate application (PID: 402172)...
|
||
✅ Application stopped successfully
|
||
|
||
✅ Trasabilitate application has been stopped
|
||
```
|
||
|
||
### status_production.sh
|
||
Displays current application status and useful information.
|
||
|
||
**Features**:
|
||
- ✅ Auto-detects project location
|
||
- ✅ Shows process information (CPU, memory, uptime)
|
||
- ✅ Tests web server connectivity
|
||
- ✅ Displays log file locations
|
||
- ✅ Provides quick command reference
|
||
|
||
**Output Example**:
|
||
```
|
||
📊 Trasabilitate Application - Status Check
|
||
==============================================
|
||
✅ Application is running (PID: 402172)
|
||
|
||
📋 Process Information:
|
||
402172 1 3.3 0.5 00:58 gunicorn --config gunicorn.conf.py
|
||
|
||
🌐 Server Information:
|
||
• Project: quality_app
|
||
• Listening on: 0.0.0.0:8781
|
||
• Local URL: http://127.0.0.1:8781
|
||
• Network URL: http://192.168.0.205:8781
|
||
|
||
📁 Log Files:
|
||
• Access Log: /srv/quality_app/logs/access.log
|
||
• Error Log: /srv/quality_app/logs/error.log
|
||
|
||
🔧 Quick Commands:
|
||
• Stop server: ./stop_production.sh
|
||
• Restart server: ./stop_production.sh && ./start_production.sh
|
||
• View error log: tail -f /srv/quality_app/logs/error.log
|
||
• View access log: tail -f /srv/quality_app/logs/access.log
|
||
|
||
🌐 Connection Test:
|
||
✅ Web server is responding
|
||
```
|
||
|
||
## File Locations
|
||
|
||
### Script Locations
|
||
```
|
||
/srv/quality_app/py_app/
|
||
├── start_production.sh # Start the application
|
||
├── stop_production.sh # Stop the application
|
||
├── status_production.sh # Check status
|
||
├── gunicorn.conf.py # Gunicorn configuration
|
||
├── wsgi.py # WSGI entry point
|
||
└── run.py # Flask application entry
|
||
```
|
||
|
||
### Runtime Files
|
||
```
|
||
/srv/quality_app/
|
||
├── py_app/
|
||
│ └── run/
|
||
│ └── trasabilitate.pid # Process ID file
|
||
├── logs/
|
||
│ ├── access.log # Access logs
|
||
│ └── error.log # Error logs
|
||
└── backups/ # Database backups
|
||
```
|
||
|
||
### Virtual Environment
|
||
```
|
||
/srv/quality_recticel/recticel/ # Shared virtual environment
|
||
```
|
||
|
||
## Log Monitoring
|
||
|
||
### View Real-Time Logs
|
||
|
||
**Error Log** (application errors, debugging):
|
||
```bash
|
||
tail -f /srv/quality_app/logs/error.log
|
||
```
|
||
|
||
**Access Log** (HTTP requests):
|
||
```bash
|
||
tail -f /srv/quality_app/logs/access.log
|
||
```
|
||
|
||
**Filter for Errors**:
|
||
```bash
|
||
grep ERROR /srv/quality_app/logs/error.log
|
||
grep "500\|404" /srv/quality_app/logs/access.log
|
||
```
|
||
|
||
### Log Rotation
|
||
|
||
Logs grow over time. To prevent disk space issues:
|
||
|
||
**Manual Rotation**:
|
||
```bash
|
||
# Backup current logs
|
||
mv /srv/quality_app/logs/error.log /srv/quality_app/logs/error.log.$(date +%Y%m%d)
|
||
mv /srv/quality_app/logs/access.log /srv/quality_app/logs/access.log.$(date +%Y%m%d)
|
||
|
||
# Restart to create new logs
|
||
cd /srv/quality_app/py_app
|
||
bash stop_production.sh && bash start_production.sh
|
||
```
|
||
|
||
**Setup Logrotate** (recommended):
|
||
```bash
|
||
sudo nano /etc/logrotate.d/trasabilitate
|
||
```
|
||
|
||
Add:
|
||
```
|
||
/srv/quality_app/logs/*.log {
|
||
daily
|
||
rotate 30
|
||
compress
|
||
delaycompress
|
||
notifempty
|
||
missingok
|
||
create 0644 ske087 ske087
|
||
postrotate
|
||
kill -HUP `cat /srv/quality_app/py_app/run/trasabilitate.pid 2>/dev/null` 2>/dev/null || true
|
||
endscript
|
||
}
|
||
```
|
||
|
||
## Process Management
|
||
|
||
### Check if Running
|
||
```bash
|
||
ps aux | grep gunicorn | grep trasabilitate
|
||
```
|
||
|
||
### Get Process ID
|
||
```bash
|
||
cat /srv/quality_app/py_app/run/trasabilitate.pid
|
||
```
|
||
|
||
### View Process Tree
|
||
```bash
|
||
pstree -p $(cat /srv/quality_app/py_app/run/trasabilitate.pid)
|
||
```
|
||
|
||
### Monitor Resources
|
||
```bash
|
||
# CPU and Memory usage
|
||
top -p $(cat /srv/quality_app/py_app/run/trasabilitate.pid)
|
||
|
||
# Detailed stats
|
||
ps -p $(cat /srv/quality_app/py_app/run/trasabilitate.pid) -o pid,ppid,cmd,%cpu,%mem,vsz,rss,etime
|
||
```
|
||
|
||
### Kill Process (Emergency)
|
||
```bash
|
||
# Graceful
|
||
kill $(cat /srv/quality_app/py_app/run/trasabilitate.pid)
|
||
|
||
# Force kill
|
||
kill -9 $(cat /srv/quality_app/py_app/run/trasabilitate.pid)
|
||
|
||
# Clean up PID file
|
||
rm /srv/quality_app/py_app/run/trasabilitate.pid
|
||
```
|
||
|
||
## Common Tasks
|
||
|
||
### Restart Application
|
||
```bash
|
||
cd /srv/quality_app/py_app
|
||
bash stop_production.sh && bash start_production.sh
|
||
```
|
||
|
||
### Deploy Code Changes
|
||
```bash
|
||
# 1. Stop application
|
||
cd /srv/quality_app/py_app
|
||
bash stop_production.sh
|
||
|
||
# 2. Pull latest code (if using git)
|
||
cd /srv/quality_app
|
||
git pull
|
||
|
||
# 3. Update dependencies if needed
|
||
source /srv/quality_recticel/recticel/bin/activate
|
||
pip install -r py_app/requirements.txt
|
||
|
||
# 4. Start application
|
||
cd py_app
|
||
bash start_production.sh
|
||
```
|
||
|
||
### Change Port or Workers
|
||
|
||
Edit `gunicorn.conf.py` or set environment variables:
|
||
|
||
```bash
|
||
# Temporary (current session)
|
||
export GUNICORN_BIND="0.0.0.0:8080"
|
||
export GUNICORN_WORKERS="16"
|
||
cd /srv/quality_app/py_app
|
||
bash start_production.sh
|
||
|
||
# Permanent (edit config file)
|
||
nano gunicorn.conf.py
|
||
# Change: bind = "0.0.0.0:8781"
|
||
# Restart application
|
||
```
|
||
|
||
### Update Configuration
|
||
|
||
**Database Settings**:
|
||
```bash
|
||
nano /srv/quality_app/py_app/instance/external_server.conf
|
||
# Restart required
|
||
```
|
||
|
||
**Application Settings**:
|
||
```bash
|
||
nano /srv/quality_app/py_app/app/__init__.py
|
||
# Restart required
|
||
```
|
||
|
||
## Troubleshooting
|
||
|
||
### Application Won't Start
|
||
|
||
**1. Check if already running**:
|
||
```bash
|
||
bash status_production.sh
|
||
```
|
||
|
||
**2. Check database connection**:
|
||
```bash
|
||
mysql -u trasabilitate -p -e "SELECT 1;"
|
||
```
|
||
|
||
**3. Check virtual environment**:
|
||
```bash
|
||
ls -l /srv/quality_recticel/recticel/bin/python3
|
||
```
|
||
|
||
**4. Check permissions**:
|
||
```bash
|
||
ls -l /srv/quality_app/py_app/*.sh
|
||
chmod +x /srv/quality_app/py_app/*.sh
|
||
```
|
||
|
||
**5. Check error logs**:
|
||
```bash
|
||
tail -100 /srv/quality_app/logs/error.log
|
||
```
|
||
|
||
### Application Crashes
|
||
|
||
**View crash logs**:
|
||
```bash
|
||
tail -100 /srv/quality_app/logs/error.log | grep -i "error\|exception\|traceback"
|
||
```
|
||
|
||
**Check system resources**:
|
||
```bash
|
||
df -h # Disk space
|
||
free -h # Memory
|
||
top # CPU usage
|
||
```
|
||
|
||
**Check for out of memory**:
|
||
```bash
|
||
dmesg | grep -i "out of memory"
|
||
```
|
||
|
||
### Workers Dying
|
||
|
||
Workers restart automatically after max_requests (1000). If workers crash frequently:
|
||
|
||
**1. Check error logs for exceptions**
|
||
**2. Increase worker timeout** (edit gunicorn.conf.py)
|
||
**3. Reduce number of workers**
|
||
**4. Check for memory leaks**
|
||
|
||
### Port Already in Use
|
||
|
||
```bash
|
||
# Find process using port 8781
|
||
sudo lsof -i :8781
|
||
|
||
# Kill the process
|
||
sudo kill -9 <PID>
|
||
|
||
# Or change port in gunicorn.conf.py
|
||
```
|
||
|
||
### Stale PID File
|
||
|
||
```bash
|
||
# Remove stale PID file
|
||
rm /srv/quality_app/py_app/run/trasabilitate.pid
|
||
|
||
# Start application
|
||
bash start_production.sh
|
||
```
|
||
|
||
## Performance Tuning
|
||
|
||
### Worker Configuration
|
||
|
||
**Calculate optimal workers**:
|
||
```
|
||
Workers = (2 × CPU cores) + 1
|
||
```
|
||
|
||
For 4-core CPU: 9 workers (default)
|
||
For 8-core CPU: 17 workers
|
||
|
||
Edit `gunicorn.conf.py`:
|
||
```python
|
||
workers = int(os.getenv("GUNICORN_WORKERS", "17"))
|
||
```
|
||
|
||
### Timeout Configuration
|
||
|
||
**For large database operations**:
|
||
```python
|
||
timeout = int(os.getenv("GUNICORN_TIMEOUT", "1800")) # 30 minutes
|
||
```
|
||
|
||
**For normal operations**:
|
||
```python
|
||
timeout = int(os.getenv("GUNICORN_TIMEOUT", "120")) # 2 minutes
|
||
```
|
||
|
||
### Memory Management
|
||
|
||
**Worker recycling**:
|
||
```python
|
||
max_requests = 1000 # Restart after 1000 requests
|
||
max_requests_jitter = 100 # Add randomness to prevent simultaneous restarts
|
||
```
|
||
|
||
### Connection Pooling
|
||
|
||
Configure in application code for better database performance.
|
||
|
||
## Security Considerations
|
||
|
||
### Change Default Credentials
|
||
```sql
|
||
-- Connect to database
|
||
mysql trasabilitate
|
||
|
||
-- Update superadmin password
|
||
UPDATE users SET password = '<hashed_password>' WHERE username = 'superadmin';
|
||
```
|
||
|
||
### Firewall Configuration
|
||
```bash
|
||
# Allow only from specific IPs
|
||
sudo ufw allow from 192.168.0.0/24 to any port 8781
|
||
|
||
# Or use reverse proxy (nginx/apache)
|
||
```
|
||
|
||
### SSL/HTTPS
|
||
|
||
Use a reverse proxy (nginx) for SSL:
|
||
|
||
```nginx
|
||
server {
|
||
listen 443 ssl;
|
||
server_name your-domain.com;
|
||
|
||
ssl_certificate /path/to/cert.pem;
|
||
ssl_certificate_key /path/to/key.pem;
|
||
|
||
location / {
|
||
proxy_pass http://127.0.0.1:8781;
|
||
proxy_set_header Host $host;
|
||
proxy_set_header X-Real-IP $remote_addr;
|
||
}
|
||
}
|
||
```
|
||
|
||
## Systemd Service (Optional)
|
||
|
||
For automatic startup on boot, create a systemd service:
|
||
|
||
**Create service file**:
|
||
```bash
|
||
sudo nano /etc/systemd/system/trasabilitate.service
|
||
```
|
||
|
||
**Service configuration**:
|
||
```ini
|
||
[Unit]
|
||
Description=Trasabilitate Quality Management Application
|
||
After=network.target mariadb.service
|
||
|
||
[Service]
|
||
Type=forking
|
||
User=ske087
|
||
Group=ske087
|
||
WorkingDirectory=/srv/quality_app/py_app
|
||
Environment="PATH=/srv/quality_recticel/recticel/bin:/usr/local/bin:/usr/bin:/bin"
|
||
ExecStart=/srv/quality_app/py_app/start_production.sh
|
||
ExecStop=/srv/quality_app/py_app/stop_production.sh
|
||
PIDFile=/srv/quality_app/py_app/run/trasabilitate.pid
|
||
Restart=on-failure
|
||
RestartSec=10
|
||
|
||
[Install]
|
||
WantedBy=multi-user.target
|
||
```
|
||
|
||
**Enable and start**:
|
||
```bash
|
||
sudo systemctl daemon-reload
|
||
sudo systemctl enable trasabilitate
|
||
sudo systemctl start trasabilitate
|
||
sudo systemctl status trasabilitate
|
||
```
|
||
|
||
**Manage with systemctl**:
|
||
```bash
|
||
sudo systemctl start trasabilitate
|
||
sudo systemctl stop trasabilitate
|
||
sudo systemctl restart trasabilitate
|
||
sudo systemctl status trasabilitate
|
||
```
|
||
|
||
## Monitoring and Alerts
|
||
|
||
### Basic Health Check Script
|
||
|
||
Create `/srv/quality_app/py_app/healthcheck.sh`:
|
||
```bash
|
||
#!/bin/bash
|
||
RESPONSE=$(curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:8781)
|
||
|
||
if [ "$RESPONSE" = "200" ] || [ "$RESPONSE" = "302" ]; then
|
||
echo "OK: Application is running"
|
||
exit 0
|
||
else
|
||
echo "ERROR: Application not responding (HTTP $RESPONSE)"
|
||
exit 1
|
||
fi
|
||
```
|
||
|
||
### Scheduled Health Checks (Cron)
|
||
```bash
|
||
crontab -e
|
||
# Add: Check every 5 minutes
|
||
*/5 * * * * /srv/quality_app/py_app/healthcheck.sh || /srv/quality_app/py_app/start_production.sh
|
||
```
|
||
|
||
## Summary
|
||
|
||
**Start Application**:
|
||
```bash
|
||
cd /srv/quality_app/py_app && bash start_production.sh
|
||
```
|
||
|
||
**Stop Application**:
|
||
```bash
|
||
cd /srv/quality_app/py_app && bash stop_production.sh
|
||
```
|
||
|
||
**Check Status**:
|
||
```bash
|
||
cd /srv/quality_app/py_app && bash status_production.sh
|
||
```
|
||
|
||
**View Logs**:
|
||
```bash
|
||
tail -f /srv/quality_app/logs/error.log
|
||
```
|
||
|
||
**Restart**:
|
||
```bash
|
||
cd /srv/quality_app/py_app && bash stop_production.sh && bash start_production.sh
|
||
```
|
||
|
||
For more information, see:
|
||
- [DATABASE_RESTORE_GUIDE.md](DATABASE_RESTORE_GUIDE.md) - Backup and restore procedures
|
||
- [DATABASE_BACKUP_GUIDE.md](DATABASE_BACKUP_GUIDE.md) - Backup management
|
||
- [DOCKER_DEPLOYMENT.md](../old%20code/DOCKER_DEPLOYMENT.md) - Docker deployment options
|
||
|
||
---
|
||
|
||
**Last Updated**: November 3, 2025
|
||
**Application**: Quality Recticel Traceability System
|
||
**Version**: 1.0.0
|