Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Hunger Games — Architecture and Operations

Hunger Games is the web-scraper backend that powers Eagle Eye. It fetches 3rd-party competitor pricing data via Apify and official APIs (Facebook, Eatigo, etc.) and exposes it to Eagle Eye through a REST API.


Architecture

External sources
  Apify (scraper platform)
  Facebook Graph API
  Eatigo + other provider APIs
        │
        ▼
  go-service/  (Go + Fiber + Clean Architecture)
        │
        ├── cmd/api          REST API server
        ├── cmd/worker        Asynq background worker
        └── cmd/scheduler     Cron scheduler
        │
        ├── PostgreSQL        Primary data store
        ├── MongoDB           Document store (scraper output)
        └── Redis             Cache + job queue (Asynq)

Tech stack

CategoryTechnology
LanguageGo 1.24+
HTTP frameworkFiber v2
ORMGORM (PostgreSQL)
AuthenticationJWT (RSA key pair)
DatabasesPostgreSQL, MongoDB
Cache / queueRedis + Asynq
AI scrapingOpen Router
API docsSwagger (via Swag)
TestingGo standard testing + Mockery
Migrationsgolang-migrate
LoggingZap

Repository structure

_app_services/go-service/
  cmd/
    api/                HTTP API entrypoint
    worker/             Asynq background worker entrypoint
    scheduler/          Cron scheduler entrypoint
  internal/
    http/
      auth/             JWT authentication middleware
      handler/          HTTP request handlers
      middleware/       Fiber middlewares
    usecase/            Business logic layer
    repository/
      postgre/          PostgreSQL repositories
      mongodb/          MongoDB repositories
      facebook_api/     Facebook Graph API client
      apify/            Apify integration
    presenter/          JSON response formatting
    parser/             Request parsing
    helper/             Shared utilities
    queue/              Asynq consumers and publishers
  config/               DB, Redis, Logger configuration
  entity/               Domain entities and DTOs
  deploy/               Docker and deployment configs
_architecture_diagram/  Excalidraw + Mermaid diagrams
api-docs/               Bruno API collections
manifest/               Deployment manifests
docs/                   Developer documentation

Common development tasks

Run the API server locally

cd _app_services/go-service
cp .env.example .env   # fill in DB/Redis/API credentials
go run cmd/api/main.go

Swagger UI: http://localhost:<port>/swagger/index.html

Run the background worker

go run cmd/worker/main.go

Run database migrations

# Up
migrate -path internal/migrations -database "$DATABASE_URL" up

# Down (one step)
migrate -path internal/migrations -database "$DATABASE_URL" down 1

Regenerate Swagger docs

swag init -g cmd/api/main.go

Regenerate mocks

mockery --all

Run tests

go test ./...

Scraper integrations

IntegrationPurpose
ApifyManaged web scraping platform — runs scrapers for competitor sites
Facebook Graph APIFetches Facebook page offers and promotions
Open RouterAI-assisted scraping and data extraction
Eatigo / othersDirect API integrations for structured competitor data

  • Eagle Eye: the frontend + BFF that consumes Hunger Games data — hungryhub-team/eagle-eye
  • API collection: api-docs/ (Bruno)
  • Architecture diagrams: _architecture_diagram/