Grafana — DevLake Datasource Setup

A checklist for setting up the Aiven Grafana → Aiven MySQL datasource for the HungryHub DevLake DORA dashboards. Skipping any field silently breaks dashboards with a generic “db query error” that doesn’t say which field is wrong.

TL;DR

The Aiven Grafana datasource devlake (uid dfo08jdx29z40f) has these required fields:

The Database field is the one that breaks things silently — if it’s empty, the datasource passes its health check (Database Connection OK) but every table query returns db query error: query failed - please inspect Grafana server log for details.

Full setup procedure (UI)

1. Open https://public-grafana-hh-production.f.aivencloud.com/connections/datasources/edit/15 (or your Aiven Grafana base URL)
2. Name: devlake
3. Type: mysql (from the dropdown, NOT “MySQL-compatible” or similar)
4. Connection section:
   • Host devlake-mysql-hh-development.f.aivencloud.com:15939 (note: NO /lake suffix here — that’s the Database field)
   • Database lake ← this is the field that breaks silently if empty
   • User avnadmin
   • Password paste the value from Aiven console
5. TLS/SSL Mode: require (encrypt without cert verify). For verify-ca, you need the Aiven CA cert — paste its contents into the TLS CA Cert textarea below
6. TLS Skip Verify: true (Aiven’s recommendation for self-signed certs)
7. Scroll to the bottom and click Save & test — you should see “Database Connection OK” in green

Setting up the datasource via API (requires Editor role)

# Get a service-account token with Editor role from Aiven Grafana
# (Admin → Service accounts → Create)

curl -X PUT "https://public-grafana-hh-production.f.aivencloud.com/api/datasources/15" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "devlake",
    "type": "mysql",
    "url": "devlake-mysql-hh-development.f.aivencloud.com:15939",
    "user": "avnadmin",
    "database": "lake",
    "access": "proxy",
    "jsonData": {
      "tlsSkipVerify": true,
      "connMaxLifetime": 14400,
      "maxIdleConns": 100,
      "maxIdleConnsAuto": true,
      "maxOpenConns": 100
    }
  }'

The secureJsonData.password field must be set separately (or via the secure config endpoint) — Aiven Grafana returns a password field already masked in the GET response.

Verifying the setup

# Test a query that requires the database field
curl -X POST "https://public-grafana-hh-production.f.aivencloud.com/api/ds/query" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "queries": [{
      "refId": "A",
      "datasourceId": 15,
      "rawSql": "SELECT COUNT(*) FROM projects",
      "format": "table"
    }],
    "from": "0", "to": "0"
  }'

If database is correctly set, you get back a frames[0].data.values array. If empty, you get db query error: query failed - please inspect Grafana server log for details with HTTP 200 (the API doesn’t fail; the query does).

Smoke test (CI)

Add a step to the DevLake smoke test pipeline that runs a query against the Grafana datasource proxy and asserts a non-empty values array. This would have caught the empty-database issue in minutes instead of days.

import requests

resp = requests.post(
    f"{GRAFANA_URL}/api/ds/query",
    headers={"Authorization": f"Bearer {TOKEN}"},
    json={
        "queries": [{
            "refId": "smoke",
            "datasourceId": 15,
            "rawSql": "SELECT COUNT(*) FROM projects",
            "format": "table",
        }],
        "from": "0", "to": "0",
    },
)
data = resp.json()["results"]["smoke"]
assert "error" not in data, f"Grafana query failed: {data.get('error')}"
assert data["frames"][0]["data"]["values"][0][0] > 0, "Expected projects table to have data"

What can go wrong

Related

• Postmortem: DevLake_Incident_2026-06-04_GraphQL_Regression.md
• DORA SQL bug: DORA_Dashboard_SQL_Bug.md
• DevLake runbook: DevLake_Runbook.md
• DevLake architecture: DevLake_Architecture_and_Operations.md
• Grafana MCP for AI agents: Grafana_MCP_for_AI_Agents.md