Deployment Plan Document (Kafka hh-server)

Pre-Deployment Steps

•  Update `KARAFKA_BOOTSTRAP_SERVERS` env on AWS Secret Managers `/hungryhub/prod` with the correct value
  

KARAFKA_BOOTSTRAP_SERVERS=kafka-hh-production.a.aivencloud.com:25942 

•  Add `KARAFKA_BOOTSTRAP_SERVERS` env on AWS System Managers => Parameter Store `/prod/hh_server_build2`
  

KARAFKA_BOOTSTRAP_SERVERS=kafka-hh-production.a.aivencloud.com:25942

•  Add `KARAFKA_BOOTSTRAP_SERVERS` & `NAMESPACE` env keys & values on ConfigMaps `hungryhub-server-config` production
  

NAMESPACE=production
KARAFKA_BOOTSTRAP_SERVERS=kafka-hh-production.a.aivencloud.com:25942

•  Create topics for production on Aiven ([https://console.aiven.io/project/hh-production/services/kafka/topics](https://console.aiven.io/project/hh-production/services/kafka/topics))
  

# List of the Topics
- production.hh.search.restaurants
- production.hh.search.restaurants.availability
- production.hh.search.restaurants.tags
- production.hh.search.restaurantTags

•  Create or check (if exist) these karafka configuration keys on AWS System Managers => Parameter Store for production env
  

• /prod/karafka_ssl_ca

 -----BEGIN CERTIFICATE-----
MIIEQTCCAqmgAwIBAgIUfU4Ht+xMJg4gqR8TEpFvhJO2T3cwDQYJKoZIhvcNAQEM
BQAwOjE4MDYGA1UEAwwvODIzN2U5MDctYzAwZC00Y2NmLWIxODktNDBiYmMxZjhi
MWU1IFByb2plY3QgQ0EwHhcNMjMwMjAyMDIzOTQ3WhcNMzMwMTMwMDIzOTQ3WjA6
MTgwNgYDVQQDDC84MjM3ZTkwNy1jMDBkLTRjY2YtYjE4OS00MGJiYzFmOGIxZTUg
UHJvamVjdCBDQTCCAaIwDQYJKoZIhvcNAQEBBQADggGPADCCAYoCggGBAK8yCxGh
HRUh692Mc6xkTlmB3AWTCZ6QF9etwPbU886rS67rrZbaJ2Ysq29Ubt3gJy448zw/
iYWrsq0TTCKgZhRFZwz4JzjlnOROnLTcBPOkQSZgiC6usho3sWWQa0NbtdR4b2XD
OjNZzmZ4CoFJDuwdr/54JW2PnVjL8chGfTORoZdLlL3zzUcdujp5TLLW/rCakJU4
eTMqAbFEPJhbzQrPiJRsPa0HlpcRnO0HrUuIp2HRjbJt1SA4Db3bMcO3JxTb5FZK
xyIQgc5rbUxxRDV6DeHjIucybMH53DJh4EYanmYTeDVYsMkVv6yG2kvyWiwJhzJE
PX1QvnkesBKrP0HwPlECjoDGa3xOzj+jFaaUUNGy7ClVirAr9QA2kDhwp0bix+AH
B4NHVns2UE9JHmBtTZD1LtJXvm8MyuAy1Aa71BmMWkKsY1HhQnUN1Mqo1kBBeOFM
/PM/AmZ+slBmqxyT0krXE0UmTxZW+M94VbzzN6/ffIFg9P4qtit6PdpnpwIDAQAB
oz8wPTAdBgNVHQ4EFgQUmGfSZEIdxkhXhBg1u3ycrlPI+7gwDwYDVR0TBAgwBgEB
/wIBADALBgNVHQ8EBAMCAQYwDQYJKoZIhvcNAQEMBQADggGBADuZMOhsMc6RbyJ9
DLE63NMzHix0dH14FdJSd73G7HpDwYr8CdIgIoGf0/bJR2cl+PXd0hIK6F2lu02/
vE32Xk8ufy/uEvi1U3HqQ1i5kSsr5crrK2pC713ksAJBh1+R8Gg3L9Qu1uz5vgtE
KuHxEgKb6DVWuxZST6GEiT9MN3Qng5ySDFAIPOuQ49P7zJii9xDGW9kLQOrtvGTI
k3xXIZzCdVQtdJr+ct0KYUfRSwBHsHd8lVEuVXsvvXyxUcdxcez63O1HJ7kCMnKm
qo+Hzy952PoRMJWxesnK+hch31sNwYO3YsP31w2AHTxtdX+nezb06jTNBtiC6Xv4
S5wSL6MNIiOUehDWXAiX4CxLXHsW3KIoQwv2RBCsCgvLxfE24OrrFuwkZbzlHYwT
Q11fSHY8a3QHol7QZtKwYlBNR3i+rFHEAG5dfb4eetzfccbID6neizyMDV0Zbf0J
iviUGHVgjLUEsaRjJXwBpRpMJ9qx8fEpGDwHT/KinRIU8nI/vg==
-----END CERTIFICATE-----

• /prod/karafka_ssl_cert

-----BEGIN CERTIFICATE-----
MIIEWDCCAsCgAwIBAgIUbQiyAAGFDW30eg1RJL5iaZFX+RgwDQYJKoZIhvcNAQEM
BQAwOjE4MDYGA1UEAwwvODIzN2U5MDctYzAwZC00Y2NmLWIxODktNDBiYmMxZjhi
MWU1IFByb2plY3QgQ0EwHhcNMjQwMTI1MDcwMzUxWhcNMjYwNDI0MDcwMzUxWjA2
MQ4wDAYDVQQKDAVrYWZrYTERMA8GA1UECwwIdTZ6NncxcjQxETAPBgNVBAMMCGF2
bmFkbWluMIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEAo+5DkqPQxQk8
3EWO8l+hj8ip4dsZfPG+9IjJu/KTjrJcsuL5C+qRFZ2V1i76WyDeJwHnhSj6RMsY
wc0qETJTKFvuAuVOh3yHLAFC1+zdrO+fubkPqij+86NIhEIcIIWRn1oJYPv2qDMr
l8RfDelyYLOkZKO//YHmQra//ulKd2Me8J2bC+xgFPxAKf0RAW7E8klHHqxDXYS8
8f0SHMDWiTwfliQaTT4e+oseBB2sSgP8HSJd/CwggzhtxF6bryK9EpFH/RTO20ZY
i+mnDRNReCcUIdKgf5MmGDs8gJjdcMgkSZbkL7Mr8+OaQTayPLLMMPVzgHuBy7KS
cKJ7N1EQKwTMHh9SFT88eHvaz0iJ/D9cYaB987JwcX4uQiSzat1arnqBkzwoXvza
N/d375rjg0mu94Ays15ttaI5qrKWo7rVklt80n4y8+ZXHEDLcG/Ga2HPzGMmnlUP
/REkk6hf8ac9cX+R7FlkJPbO65NfNvISEKbp+3yxLrhmaeirLnxnAgMBAAGjWjBY
MB0GA1UdDgQWBBRq0/BU28ClB5MRC0sTX065FH/UuTAJBgNVHRMEAjAAMAsGA1Ud
DwQEAwIFoDAfBgNVHSMEGDAWgBSYZ9JkQh3GSFeEGDW7fJyuU8j7uDANBgkqhkiG
9w0BAQwFAAOCAYEADUQGWi+wIhaLhRmkfa9eHi4P5GPkM1Jyr9FHWbk0Gt0Ed5Sf
w5Ih1GXcgfrvG7EwFYIfRlJIrHD2qqy7SU9FY4ppCAUhaV0G+1oVcCvg2oa/jfcr
WCbmhfwG4tYqYK2Q3TOvtI63bKCH+eNxLWt7zPofezWTEFbero0DlHKT4BZaHdVW
GVnUTJ45ugDpJUT4l+ei/dxtXkoUPYHqjtnair1hcjPMBJG5H45stqBpWVCAWCbd
wHXymkwPTmDNt/jEpbNzprQaaIuJwWI5Ert+Mqc4sKaVp/QmtrvFaJxfBAZpOAMB
TbMdlTiD2bG0PN2UsWVj3BFyN7EsjEX5DvjKAW8DwVkdFoGIt2h/E9Yhqvf7CHVd
UAxX/Gs7cpLuD7jiGdKglK4AnXZT1OkeJF1kQrfnEpqc0wy+COFv61BKPjXmF1fx
3idAfulGTlHq1ihl3FavMuu/zHgfh4sMoaZZ017ov7dIF3wUAmRoZZELUhXUjjYV
JUF1v0gTD4lN9hqs
-----END CERTIFICATE----- 

• /prod/karafka_ssl_key

-----BEGIN PRIVATE KEY-----
MIIG/QIBADANBgkqhkiG9w0BAQEFAASCBucwggbjAgEAAoIBgQCj7kOSo9DFCTzc
RY7yX6GPyKnh2xl88b70iMm78pOOslyy4vkL6pEVnZXWLvpbIN4nAeeFKPpEyxjB
zSoRMlMoW+4C5U6HfIcsAULX7N2s75+5uQ+qKP7zo0iEQhwghZGfWglg+/aoMyuX
xF8N6XJgs6Rko7/9geZCtr/+6Up3Yx7wnZsL7GAU/EAp/REBbsTySUcerENdhLzx
/RIcwNaJPB+WJBpNPh76ix4EHaxKA/wdIl38LCCDOG3EXpuvIr0SkUf9FM7bRliL
6acNE1F4JxQh0qB/kyYYOzyAmN1wyCRJluQvsyvz45pBNrI8ssww9XOAe4HLspJw
ons3URArBMweH1IVPzx4e9rPSIn8P1xhoH3zsnBxfi5CJLNq3VqueoGTPChe/No3
93fvmuODSa73gDKzXm21ojmqspajutWSW3zSfjLz5lccQMtwb8ZrYc/MYyaeVQ/9
ESSTqF/xpz1xf5HsWWQk9s7rk1828hIQpun7fLEuuGZp6KsufGcCAwEAAQKCAYBQ
nGKoIlwP+CnwW1gewX3t+YvoX1haQ3pWP0mMXY3H9td0s/4pqPAtQ++hgjpkQ3Za
kioqJetjlseWzW+pQyj3SLVywRXrtAW4UupuybB8UN0nc+RLFZvBbQLE3sl8CJKC
XMEN6vjw3zDJkav4aFpxSQVG4+sOGymCluE01b1jld9qDOZoZ5Ub3Kskuqf5H2JD
jlslryn1O8e562tTrlQQ6frSVvCEGU8cfvqEpZ7KSslX/kVfMsfOHxqDHM7LwXWU
NLefID1dBY6HqTQSAfHfK6KyJJGDkTZMFW8ylkt+o6QQGCTkRcTnsUlYL2vEv4gO
7OtM0nq+KNcRfw/bHIrdiONPyM+ROiJMJfxbDOaCyMB3nyNWMAKKzQB2ow48RVvs
g2yfBuRlEjnZ/K8aOcvdOYaF60RzHBROhd7l7LHn7kTvlV7JBXQczeeD9tmGPZBC
Db/HUhbhEfOoAq4ATkWnmWM2Fb5O6CHCvNSRwGxfdb4Co/M7mcct31OEDmYQnnEC
gcEA3QKs5vAbdue2AQbWvG9oF0d/4u+nMj9ggS+Rat1UDv6MCFXk3AwTeYejBkHo
zK+HoqnxWgoCWPtk8En8s91uscwTOhnoWKtjb/W++5yavwKtKja7KjymEQuMsPNS
uY+wyAY1830CIF2VX1QQvyODZprkXhuxVSCBOqSVYP21vpudtf7N5TkJn58QctUx
QlNxguthz8pZ1zuDkHR+Kt6cllYjHqFm/TxpOip996+UL1vc+dSNizApjB6a6m5h
yO7/AoHBAL3iNMYGWiShZZztmnSTxWaUr/qLmK6v0OQ4Y4lI8N9T7YYt5qR9cA2T
zy6/qPyEhTzC8oPg2PGytitO0OdU+/2k6HOybEU17w0EiSu/r5jhf04quG0EYZXr
jBktFMnWKnIDdG1M6I31WNCcZ22oNE+C11hgIB51R5z24NwgjE5Znvh00C0EHB3r
fhf4HRUy3SUKc/OrVLNEDqMKBKMTk2jsh5/jy/BPJ6Ye3TuhBIzvlelH9bsYpMZ2
Wu7zm+5amQKBwAqv026DU8CnKazZCM8pz+24Pl3Z2sraN69nfm+dbjAfbOQGSz+8
scBhp6doznn/Tt61uv7wlvZ4iYP4/+gun+inBpL7p7mJ6BAE4QxHoU/1MCxnYxuC
vUD7vvzbdzeT1CeH7VntxImOHOf0J68mZ3cxrDNEwYltPfJ4/OQowH4MBk9Dl+8q
l+WJCoGS/4PGhymeeidTEI8QyKbWizdFdmrqlV4ozdMW9bujTF4Ktm6n0YKfHY0K
cifdXv5RAhd2PwKBwBplFVzpiSJKZSuZQ70sxWXrCUNUoFa4DduLkxqETydcLUpX
mxYGccsRXt1quvzPFw5JMxjt6no+k2AVxqhBgf3SLJXYmYBNbPtsQ/0W2jvwLJgb
DCPrVf1XDtR5FciwwpmyrT58UF9bWYs2EpKFrS/YSkL4qb5SIzBIiR1QUPCbOKPk
6QM47QejspV2YeSOe3soptGaYV3B+h7jx507nWQabQfmzHSNLD/FTo95ZH6K11Xk
MN5fIJiN3zoUFvrdEQKBwQDWjOMl1lYITP/t6lUn8xImOfhIzgnHcN+Cm++3fvNm
xmAz7S4l6cnNzCrCiW86w4ydVYqyx6YPyHPW1JDWeNKXhwxCBvbX5kRpq0giqRh5
tr6mdx8xxyGtbez1Z/C8DHcYEEnpiYdT92ZmiqIdEx9tzuDsPoxX7mm2iBs5mr+z
3Ac6PZLo3cmBDDJiBe4EaCNvLqMBR17Cbj2GSgmqHBVdDTy7EkXeloiI2HBoARSU
ZEFi5kaZZhEVtJO8rbUsb24=
-----END PRIVATE KEY----- 

Deployment Steps

•  Get reviews from other Devs and make sure there is no conflict in the Pull Request
  

[

github.com

https://github.com/hungryhub-team/hh-server/pull/5549

](https://github.com/hungryhub-team/hh-server/pull/5549)

•  Merge the [PR](https://github.com/hungryhub-team/hh-server/pull/5549) and make sure all of the steps on #Pre-Deployment-Steps are completed
  

•  Waiting & monitoring the deployment process to be done
  

Post-Deployment Steps

•  Check if the `hungryhub-sidekiq-kafka` server are present and running properly
  

•  Check inside the pods of `hungryhub-server` the `.karafka` folder must be present and there is these 3 file
  

.karafka/ca.pem
.karafka/service.cert
.karafka/service.key 

•  Reindex all schema from rails console
  

• restaurants schema

# generate payload for all restaurants
# wait until the finish
bin/rake reindex_schema:generate_restaurant_payload

# check the if the payload is generated successfully
# the total restaurants on kafka_reindex_restaurants table should be the same with the Restaurant.active.not_expired data
bin/rails console
KafkaReindexRestaurant.count
Restaurant.active.not_expired.count

# reindex all restaurants data
bin/rake reindex_schema:restaurants

• restaurants availabilities schema

# generate availability (inventory) payload for all restaurants
bin/rake reindex_schema:generate_availability_payload

# check the if the availability payload is generated successfully
# the total restaurants on kafka_reindex_availabilities table should be the same with the Restaurant.active.not_expired data
bin/rails console
KafkaReindexAvailability.count
Restaurant.active.not_expired.count

# reindex availability (inventory) for all restaurant
bin/rake reindex_schema:inventories

• restaurant tags schema

# reindex all restaurant tags data
bin/rake reindex_schema:restaurant_tags

•  Add new feature `kafka_producer_hhsearch` to Flipper and then make it Fully Enable ([https://hungryhub.com/admin/flipper/features](https://hungryhub.com/admin/flipper/features))
  

•  Done