Deployment runbook

Production Deployment Runbook

Quick reference guide for deploying Dashtray to production.

Quick Links

• Full Deployment Guide
• Monitoring Setup
• Production Checklist

Prerequisites

• All tests passing
• Staging environment verified
• Environment variables configured
• Team notified of deployment window

Deployment Steps

1. Pre-Deployment Check

# Run pre-deployment checks
./scripts/pre-deployment-check.sh

# Verify Convex environment
npx convex env list --prod

2. Deploy Convex Backend

# Deploy Convex functions
npx convex deploy --prod

# Verify deployment
npx convex logs --prod --tail

3. Deploy Frontend

# Merge to main (triggers GitHub Actions)
git checkout main
git pull origin main
git merge staging
git push origin main

# Monitor deployment
# https://github.com/your-org/dashtray/actions

4. Verify Deployment

# Run post-deployment checks
PRODUCTION_URL=https://dashtray.com \
CONVEX_URL=https://your-project.convex.site \
./scripts/post-deployment-verify.sh

5. Smoke Tests

# Test homepage
curl -I https://dashtray.com

# Test authentication
curl -X POST https://dashtray.com/api/auth/signin \
  -H "Content-Type: application/json" \
  -d '{"email":"[e-mail]","password":"testpass"}'

# Test Convex health
curl https://your-project.convex.site/api/health

6. Monitor

# Watch Convex logs
npx convex logs --prod --watch

# Check error rate in monitoring dashboard
# Check response times
# Verify webhooks are working

Rollback Procedure

Rollback Convex

# List deployments
npx convex deployments list --prod

# Rollback to previous
npx convex deployments rollback <deployment-id> --prod

Rollback Cloudflare Pages

1. Go to Cloudflare Pages dashboard
2. Navigate to Deployments
3. Find last working deployment
4. Click “Rollback to this deployment”

Rollback Git

# Revert last commit
git revert HEAD
git push origin main

# Or reset to specific commit
git reset --hard <commit-hash>
git push origin main --force

Emergency Contacts

• On-Call Engineer: [phone/slack]
• Convex Support: [e-mail]
• Cloudflare Support: https://dash.cloudflare.com/support
• DodoPayments Support: [e-mail]

Common Issues

Issue: Build Failure

Symptoms: GitHub Actions build fails

Solution:

1. Check build logs in GitHub Actions
2. Verify environment variables are set
3. Test build locally: pnpm run build
4. Check Node.js version (should be 20)

Issue: Convex Functions Not Deploying

Symptoms: Functions not updating in production

Solution:

1. Check convex.json configuration
2. Verify Convex CLI is authenticated: npx convex dev
3. Check for syntax errors in Convex functions
4. Try manual deployment: npx convex deploy --prod

Issue: Environment Variables Not Loading

Symptoms: Application errors related to missing config

Solution:

1. Verify variables in Convex: npx convex env list --prod
2. Check Cloudflare Pages environment variables
3. Verify GitHub secrets and variables
4. Rebuild application after updating variables

Issue: Webhooks Not Received

Symptoms: DodoPayments events not processing

Solution:

1. Check webhook URL in DodoPayments dashboard
2. Verify webhook signature validation
3. Check Convex logs: npx convex logs --prod | grep webhook
4. Test webhook manually using DodoPayments test mode

Issue: SSL Certificate Errors

Symptoms: HTTPS not working or certificate warnings

Solution:

1. Verify DNS records are correct
2. Wait for DNS propagation (up to 24 hours)
3. Check Cloudflare SSL/TLS settings (should be “Full (strict)”)
4. Verify custom domain is added in Cloudflare Pages

Issue: High Response Times

Symptoms: Application is slow

Solution:

1. Check Cloudflare Analytics for traffic spikes
2. Review Convex function performance
3. Check for database query issues
4. Verify CDN caching is working
5. Consider scaling Convex plan if needed

Monitoring Checklist

After deployment, verify:

• Uptime monitoring is active
• Error tracking is receiving events
• Performance monitoring shows normal metrics
• Webhook delivery is working
• Email delivery is working
• Integration syncs are running
• Cron jobs are executing
• Rate limiting is active
• API endpoints are responding

Post-Deployment Tasks

Within 24 hours:

• Monitor error logs
• Check user feedback
• Verify all integrations working
• Review performance metrics
• Update team on deployment status
• Document any issues encountered
• Plan next release

Deployment Schedule

Recommended deployment windows:

• Best: Tuesday-Thursday, 10 AM - 2 PM (local time)
• Avoid: Fridays, weekends, holidays, late nights
• Maintenance Window: Announce 24 hours in advance

Version Tracking

Document each deployment:

Date: 2025-01-15
Version: v1.2.0
Deployed By: [name]
Commit: abc123
Changes:
- Feature: Added new dashboard widget
- Fix: Resolved webhook timeout issue
- Improvement: Optimized database queries
Issues: None
Rollback: Not required

Success Criteria

Deployment is successful when:

• All smoke tests pass
• Error rate < 1%
• Response time < 2s (p95)
• No critical bugs reported
• All integrations working
• Webhooks processing correctly
• Monitoring shows normal metrics

Additional Resources

• Convex Deployment Docs
• Cloudflare Pages Docs
• SvelteKit Deployment
• GitHub Actions Docs