Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Infrastructure Repository Guide: hungryhub-terraform vs platform-infra

Overview

HungryHub maintains two infrastructure-as-code repositories. This document explains why each exists, what belongs where, and how the LiteLLM migration demonstrates the boundary.

hungryhub-terraform

Purpose: Customer-facing HungryHub product infrastructure — everything that directly serves end-users (restaurants, diners, partners).

AttributeDetail
AWS Accountsproduction (202255947274), sandbox/dev (079994049689)
Domains*.hungryhub.com (prod), *.hhstaging.dev (dev/staging)
EKS Clusterseks-prod-21 (prod), eks-dev-262 (dev)
Scale~1.8B THB GMV target, ~1M RPM peak, ~489 pods
ServicesRails app, Sidekiq workers, pricing engine, Pegasus web, relay, menu, vendor hub, mobile APIs, Aurora MySQL, ElastiCache, ClickHouse, Cloudflare, Route 53, self-hosted GitHub runners

What belongs here:

  • All production and staging environments for customer-facing services
  • Database infrastructure (Aurora, ElastiCache, RDS)
  • CDN and DNS (Cloudflare, Route 53 for hungryhub.com)
  • CI/CD runners (Hetzner k3s ARC runners)
  • Monitoring and observability for product services

What does NOT belong here:

  • Internal developer tools
  • AI/ML platform services used only by the engineering team
  • Backstage, LiteLLM, Langfuse, or other internal tooling

platform-infra

Purpose: Internal/shared platform capabilities and independently operated tool workloads — services that support the engineering team, not end-users.

AttributeDetail
AWS Accountplatform-prod (534120921623)
Domain*.hh-bee.my.id
EKS Clusterbackstage-blueprint
Managed byArgoCD (gitops/) + Terraform (terraform/)
ServicesBackstage developer portal, LiteLLM AI gateway, Langfuse observability, ClickHouse (analytics), ArgoCD

What belongs here:

  • Developer productivity tools (Backstage, internal dashboards)
  • AI/ML gateway (LiteLLM) and observability (Langfuse)
  • Internal analytics infrastructure (ClickHouse for ClickStack)
  • Platform team automation and tooling
  • Cloudflare Zero Trust management (planned)

What does NOT belong here:

  • Customer-facing services or their staging environments
  • Production databases serving end-users

Why LiteLLM Moved from hungryhub-terraform to platform-infra

The Problem

LiteLLM was originally deployed in the sandbox (dev) AWS account alongside customer-facing staging services. This created several issues:

  1. Shared blast radius — Infrastructure changes to the dev cluster for product staging could affect the AI gateway that the entire engineering team depends on.
  2. Wrong ownership boundary — LiteLLM is an internal tool, not a customer-facing service. It was managed by Terraform Helm releases mixed in with product service deployments.
  3. Cost attribution — LiteLLM costs were blended into the dev account, making it hard to track platform team spending separately.
  4. Scaling independence — The dev cluster’s sizing and scaling policies are optimized for staging workloads, not always-on internal tools.

The Solution

LiteLLM now runs on the platform-infra EKS cluster (backstage-blueprint) in the dedicated platform-prod account:

  • Isolated from product deployments — No risk of staging changes breaking the AI gateway.
  • ArgoCD-managed — Helm values live in gitops/services/internal-ai/litellm/values.yaml with auto-sync, self-heal, and pruning.
  • Dedicated RDS — Own PostgreSQL instance (internal-ai-postgres) with per-workload databases.
  • Clear cost boundary — All costs attributed to the platform-prod account.
  • Same API keys — Keys were migrated so existing integrations work without changes.

New Endpoint

OldNew
URLhttps://litellm.hhstaging.devhttps://litellm.hh-bee.my.id
UIhttps://litellm.hhstaging.dev/ui/https://litellm.hh-bee.my.id/ui/
APIhttps://litellm.hhstaging.dev/v1https://litellm.hh-bee.my.id/v1

Decision Rule

If a service directly serves HungryHub customers (restaurants, diners, partners), it belongs in hungryhub-terraform.

If a service serves the engineering team (developer tools, AI gateway, observability, analytics), it belongs in platform-infra.