Environment Configuration Guide

This document explains the environment configuration system in the project, which uses a base + override approach to eliminate duplication.

Overview

The project uses a merge-based configuration system where:

1. .env.example - Contains all default configuration values
2. .env.ci - Contains only CI-specific overrides (minimal file)
3. bin/merge_env.sh - Merges base + overrides to create .env

This approach eliminates code duplication and makes it easy to maintain environment-specific configurations.

Files Overview

.env.example (Base Configuration)

• Purpose: Complete base configuration with all environment variables
• Network: Docker Compose service names (for local development)
• Database: MYSQL_SERVER_HOST=db
• Redis: redis://valkey-lru:6382, redis://valkey-persistence:6380
• Contains: ~200 environment variables with sensible defaults

.env.ci (CI/CD Overrides Only)

• Purpose: Contains only the differences needed for CI/CD environments
• Size: ~13 overrides instead of duplicating ~200 variables
• Network: localhost with specific ports
• Database: MYSQL_SERVER_HOST=127.0.0.1, MYSQL_USER=hh_user
• Redis: redis://localhost:6382, redis://localhost:6380

Usage

Quick Setup (Recommended)

# For local development (Docker Compose)
./bin/setup_env.sh local

# For CI/Test environment
./bin/setup_env.sh ci

# For staging environment
./bin/setup_env.sh staging

# Clean base configuration only
./bin/setup_env.sh clean

Manual Merge

# Merge .env.example + .env.ci → .env
./bin/merge_env.sh .env.ci

# Use .env.example only (no overrides)
./bin/merge_env.sh

Traditional Approach (Still Works)

# For local development
cp .env.example .env

# For CI/CD (manual)
cp .env.example .env
# Then manually edit the few values that differ

GitHub Actions Integration

The CI/CD workflow automatically merges configurations:

- name: Setup configuration files
  run: |
    ./bin/merge_env.sh .env.ci
    cp config/database.example.yml config/database.yml
    # ... other config files

VS Code Dev Container Integration

The dev container automatically sets up the local development environment:

Dev Container Configuration

The .devcontainer/devcontainer.json uses our merge system:

"initializeCommand": [
  "bash", "-c",
  "cd ${localWorkspaceFolder} && chmod +x bin/setup_env.sh bin/merge_env.sh && ./bin/setup_env.sh local"
]

What Happens When You Open in Dev Container

1. Before container starts: ./bin/setup_env.sh local runs on the host
2. Inside container: Services start with proper Docker Compose networking
3. Result: .env file has the correct service names like:
   • MYSQL_SERVER_HOST=db (Docker service name)
   • REDIS_LRU_URL=redis://valkey-lru:6382/2
   • MEMCACHED_HOST=memcached

Dev Container vs CI Environment

Benefits of the New System

✅ Advantages

1. No Duplication: .env.ci has only 13 lines instead of 200+
2. Easy Maintenance: Update .env.example once, all environments benefit
3. Clear Overrides: Instantly see what’s different in each environment
4. Flexible: Can create .env.staging, .env.local with minimal overrides
5. Backward Compatible: Old cp .env.example .env still works

📊 Comparison

Key Differences by Environment

Creating New Environment Overrides

To create a new environment-specific configuration:

Example: Staging Environment

# Create .env.staging with only the differences
cat > .env.staging << 'EOF'
# Staging Environment Overrides
MYSQL_SERVER_HOST=staging-db.example.com
MYSQL_PASSWORD=staging-secure-password
REDIS_LRU_URL=redis://staging-redis.example.com:6379/2
ENABLE_ROLLBAR=true
ROLLBAR_TOKEN=staging-rollbar-token
EOF

# Use the staging configuration
./bin/setup_env.sh staging

Example: Local Development Overrides

# Create .env.local for personal development preferences
cat > .env.local << 'EOF'
# Personal Development Overrides
ENABLE_GOOGLE_ANALYTICS=true
ENABLE_ROLLBAR=true
ROLLBAR_TOKEN=my-dev-rollbar-token
LOG_LEVEL=debug
EOF

# Use your local configuration
./bin/setup_env.sh local

Adding the Override File to .gitignore

Add environment-specific files to .gitignore to prevent accidental commits:

# Add to .gitignore
echo ".env.local" >> .gitignore
echo ".env.staging" >> .gitignore
echo ".env.production" >> .gitignore

Troubleshooting

Verify Configuration Merge

# Check what overrides were applied
./bin/merge_env.sh .env.ci | grep "Override:"

# Verify specific values in the merged .env
grep -E "(MYSQL_SERVER_HOST|REDIS_LRU_URL)" .env

Debug Merge Issues

# See the merge process step by step
bash -x ./bin/merge_env.sh .env.ci

Reset to Clean State

# Start fresh with base configuration
./bin/setup_env.sh clean

Dev Container Issues

Problem: Dev container fails to start or services can’t connect

Solution: Check environment configuration:

# Verify dev container uses Docker service names
grep "MYSQL_SERVER_HOST" .env  # Should show "db"
grep "REDIS_LRU_URL" .env      # Should show "valkey-lru"

# If showing localhost instead of service names:
./bin/setup_env.sh local

Problem: Database connection fails in dev container

Solution:

1. Check if services are running: docker-compose ps
2. Verify database configuration matches dev container setup
3. Check MySQL credentials in .env and config/database.yml

Problem: Environment setup scripts not executable

Solution:

chmod +x bin/setup_env.sh bin/merge_env.sh

| Redis Persistent | redis://valkey-persistence:6380 | redis://localhost:6380 | | Redis Inventory | redis://valkey-inventory:6381 | redis://localhost:6381 | | Memcached | MEMCACHED_HOST=memcached | MEMCACHED_HOST=localhost | | MongoDB | mongodb://root:root@mongodb:27017/... | mongodb://root:root@localhost:27017/... | | Sidekiq | SIDEKIQ_HOSTNAME=valkey-persistence | SIDEKIQ_HOSTNAME=localhost |

Maintenance

When to Update .env.ci

• When service ports change in GitHub Actions workflow
• When new environment variables are added that affect CI/CD
• When database credentials change for CI/CD environment

When to Update .env.example

• When new environment variables are added for development
• When Docker Compose service names change
• When new external services are integrated

Important Notes

1. Never commit .env - This file contains your local overrides
2. Always commit .env.ci - This is needed for CI/CD to work
3. Keep .env.example updated - This is the template for new developers
4. Test both environments - Changes affecting one often affect the other

Troubleshooting

Common CI/CD Issues

• Connection refused: Check if service hostnames use localhost instead of service names
• Wrong ports: Verify ports match those defined in GitHub Actions workflow
• Database not found: Ensure database credentials match the test environment setup

Common Development Issues

• Service not found: Check if Docker Compose services are running
• Wrong network: Ensure you’re using service names, not localhost
• Port conflicts: Check if ports are already in use on your host machine