Tiger And Pegasus Producer Rollout Plan For HH-Lion

Goal

hh-lion does not need a new event model. The main work on this side is producer onboarding, validation observability, canary rollout, and downstream quality checks so Tiger and Pegasus can safely become upstream producers.

What HH-Lion Already Expects

The contract is already defined:

• Topic: hh.lion.interaction.events (or <NAMESPACE>.hh.lion.interaction.events where environment namespacing is used)
• Canonical event types:
  • impression
  • click
  • view
  • favorite_added
  • package_selected
  • begin_checkout
  • booking_confirmed
  • search

The biggest ingestion dependencies are:

• valid Nano ID identity fields for anonymous_id and session_id
• request/list/section attribution for search and browsing events
• package and booking fields for commerce funnel events
• reliable context fields for language, device, and referrer category

HH-Lion Responsibilities

1. Keep the contract stable

Avoid changing the contract during producer rollout unless a real ambiguity is discovered. The producers should align to the current event contract rather than hh-lion loosening semantics for legacy compatibility.

2. Support staged producer onboarding

hh-lion should support a staged rollout path:

• local validation with sample events
• engineering producer canary
• production dual-write canary
• full rollout after quality checks pass

If Tiger needs a short transitional period, JSON fallback can be tolerated during rollout, but note that the current consumer fallback behavior is not environment-gated and may also apply outside engineering on Avro deserialization errors. The target production state should remain Avro plus Schema Registry, and enforcing Avro-only ingestion in production would require an explicit config or environment guard in the consumer.

3. Make ingestion failures easy to debug

Before production rollout, confirm that dashboards and alerts exist for:

• accepted event count by event_type
• rejected event count by validation reason
• DLQ volume
• anonymous/session coverage
• missing request_id
• missing package_id on package events
• per-source acceptance rate if Tiger identifies source application

4. Add source-level rollout checks

Each new producer should be validated on three dimensions:

• contract validity
• attribution completeness
• downstream feature usefulness

Recommended checks:

• search events have search_query and request_id
• impression/click events preserve position and section metadata
• checkout and booking events carry package_id
• booking-confirmed events join cleanly into item and member feature jobs

Rollout Stages

Stage 1. Contract test fixtures

• Build a Tiger-generated fixture set for all eight canonical event types.
• Validate fixtures against the same quality rules used in ingestion.
• Include guest and member examples.

Stage 2. Engineering dual write

• Pegasus sends canonical payloads to Tiger in parallel with the current legacy path.
• Tiger publishes canonical events to hh-lion engineering.
• Monitor rejection reasons and fix upstream payload issues first.

Stage 3. Production canary

• Enable a small percentage of Pegasus traffic for canonical publish.
• Compare event volumes against expected search, click, checkout, and booking traffic.
• Watch DLQ and validation failures closely.

Stage 4. Production scale-up

• Increase traffic gradually.
• Confirm member and item feature pipelines show sane counts.
• Confirm no severe skew between click, checkout, and booking-confirmed coverage.

Quality Gates

The rollout should not move forward if any of these remain materially broken:

• guest traffic lacks anonymous_id
• session_id is not a valid Nano ID
• search results do not preserve request_id and position
• package events are missing package_id
• booking-confirmed events cannot be linked to a valid booking value
• DLQ rate is persistently elevated

Backfill Position

Fresh collection should be the primary plan. Historical backfill is secondary and should be conservative:

• db1 can help with booking and package enrichment
• OpenSearch can help with restaurant metadata enrichment
• db2 is not sufficient to recreate canonical browsing behavior

In practice, hh-lion should treat the new canonical stream as the trustworthy start of high-quality behavioral data.

Deliverables

1. Engineering fixture pack for all canonical event types.
2. Producer-specific validation dashboard for Tiger. Reference: Canonical Ingestion Canary Dashboard Spec
3. Canary checklist and rollback conditions.
4. Acceptance report covering event validity, attribution completeness, and feature-pipeline sanity.