fix(dashboard-ui): upgrade Grafana to 11.6.16 and pin plugins

[satriadhikara]

Description

Dashboards that read from the dbrepo-json (Infinity) datasource show "No data" and this error:

Error: 404 Not Found, loading react/jsx-runtime from http://localhost/dashboard/public/plugins/yesoreyeram-infinity-datasource/module.js?_cache=4.0.0

GF_INSTALL_PLUGINS installs the newest Infinity release every time the container is created. Infinity 4.0.0 needs Grafana >=11.6.11, but bitnamilegacy/grafana:11.4.0 actually reports Grafana 11.3.1. So the plugin frontend fails to load.

Pinning the plugin alone keeps us on an old Infinity. Upgrading alone doesn't work while we stay on Bitnami. bitnamilegacy/grafana stopped getting updates in August 2025, and its newest tags are 11.6.1 and 12.1.1, which are both too old for Infinity 4.x.

Docker Compose

In docker-compose.yml, .docker/docker-compose.yml and the .live skel compose:

• Switches to the official grafana/grafana:11.6.16 image. I stayed on 11.6 to avoid the Grafana 12 breaking changes.
• Moves the config mounts from /opt/bitnami/grafana/conf/... to /etc/grafana/.... The dashboards are already provisioned from /app/dashboards, so they don't change.
• Pins yesoreyeram-infinity-datasource to 4.0.0 and grafana-opensearch-datasource to 2.34.4.
• Sets GF_PLUGINS_PREINSTALL_DISABLED. Otherwise the official image installs unpinned default apps (grafana-lokiexplore-app, grafana-pyroscope-app, ...) in the background on every start.

Helm

On Helm the dashboards are broken for one more reason. Since dfb2f97 dashboardui.plugins is empty, so Infinity and OpenSearch are never installed, and dashboardui.datasources only defines Prometheus. templates/dashboard-ui-datasources-secret.yaml (which defines dbrepojson0 and dbrepoopensearch0, used by the dashboards and the Dashboard Service) is not referenced anywhere. On a v1.14.1 install both datasources report "Plugin not registered".

The Bitnami Grafana chart only ships bitnamilegacy images, so this switches the dashboardui dependency to the official chart, matching Compose:

• grafana chart 8.15.0 from https://grafana-community.github.io/helm-charts (grafana/helm-charts is frozen at 10.5.15), with image grafana/grafana:11.6.16.
• Same plugin pins and GF_PLUGINS_PREINSTALL_DISABLED.
• Loads env from dashboard-ui-secret, which now also holds admin-user/admin-password for admin.existingSecret.
• Mounts dashboard-ui-datasources-secret at /etc/grafana/provisioning/datasources.
• Runs Grafana as a StatefulSet, so helm upgrade from a Bitnami-chart release needs no manual step (see Existing deployments). No persistence (Grafana's state is in dashboard-db), test framework off, and the chart's RBAC off: it only serves the sidecars, which are not used, and would create cluster-wide roles. Keeps the PodDisruptionBudget the Bitnami chart had (maxUnavailable: 1). README table and values.schema.json updated.
• Adds templates/dashboard-ui-legacy-pvc.yaml, which keeps the dashboard-ui PVC of the old Bitnami Grafana instead of letting Helm delete it.

Also

• Uses the same image in the Dashboard Service test fixture.
• Adds .scripts/test-grafana-plugins.py, which checks that every Compose file and the Helm values use exactly grafana/grafana:11.6.16 with Infinity 4.0.0 and OpenSearch 2.34.4, and disable preinstalls. CI runs it in the lint-compose job.

Deployment evidence

We hit the same bug in our ITB Docker Compose deployment of v1.13.4. Infinity 3.10.1 crashed the query editor with React error #130, and 3.7.1 (its manifest says >=10.4.8) still crashed on the Bitnami image. We have been running a pinned Infinity on ITB since then.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes, no API changes)
• Dependency update

Related Issue

Closes #106

Checklist

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing tests passed.

Additional Notes

Validation: Docker Compose

On a local Docker Compose stack using .docker/docker-compose.yml (v1.14.1 images):

• Before (main): a dashboard with one Infinity table panel querying /api/v1/image showed "No data" and the exact error above.
• After: I recreated only dbrepo-dashboard-ui with this change on the same dashboard database. The panel shows the data, and the Infinity query editor renders with no console errors (checked in Chrome).
• Grafana migrated the existing PostgreSQL dashboard database. All system dashboards and the test dashboard were kept.
• Datasource health is OK for dbrepo-json, Prometheus and OpenSearch.
• The Keycloak dashboard uses grafana-piechart-panel, which the Bitnami image bundled. Angular is already disabled on the current image, and those panels are migrated to the core piechart on both images. No "panel plugin not found".
• pytest tests/test_integration_app.py in dbrepo-dashboard-service against grafana/grafana:11.6.16: 14 passed.
• python3 .scripts/test-grafana-plugins.py: passes, and fails on main.
• docker compose config on all three Compose files: passed.

Validation: Helm

The chart on main (and the published 1.14.1 chart) does not render with its own default values.yaml (gatewayservice containerPorts.http is the first error). That comes from the values rewrite in be186a7 and is not related to Grafana, so I will send it as a separate PR. To test this change I used the pre-be186a7e6 values with the dashboardui block from this PR:

• helm template renders 169 objects and helm lint passes. The render is identical to this branch's chart apart from generated random secrets.
• Installed on a single-node OrbStack Kubernetes cluster (v1.14.1 images, small test presets): Grafana reports 11.6.16, only Infinity 4.0.0 and OpenSearch 2.34.4 are installed, health is OK for dbrepojson0, dbrepoopensearch0 and dbrepometrics0, the Infinity module.js loads (200), and an inline Infinity query through /api/ds/query returns data.
• Before (1.14.1 chart): Grafana 11.3.1, dbrepojson0 and dbrepoopensearch0 health "Plugin not registered", no plugins installed.

Validation: Helm upgrade from the Bitnami chart

Tested on OrbStack Kubernetes 1.35 with Helm 4.3.0. To change only the Grafana part of a release, I used a small umbrella chart with the released subcharts (Bitnami grafana 11.4.2, grafana 8.15.0, postgresql 16.1.0) and DBRepo's own dashboard templates. Each source release was seeded with a private dashboard (Viewer permission removed, as update_anonymous_read_access does), a public dashboard, a hand-made dashboard and a service account.

Sources:

• 1.13.5 values: Grafana stores its state in dashboard-db, no PVC.
• 1.14.1 values: with the shipped values the Grafana pod never starts. The ConfigMap dashboard-service-config does not exist, and admin.userKey/passwordKey are keys of the official chart, so the Bitnami chart reads the key password from dashboard-service-secret, which is not there. With those two fixed, Grafana runs on SQLite on the dashboard-ui PVC, and that is the source I used.

Results:

• Previous commit of this PR, from 1.14.1: helm upgrade fails (spec.selector: field is immutable, plus spec.strategy.rollingUpdate: Forbidden for the switch to Recreate). After the documented kubectl delete deployment, the upgrade succeeds, but Helm deletes the dashboard-ui PVC and its volume, and Grafana starts empty.
• This PR, from 1.14.1: the upgrade succeeds with no manual step. Helm removes the old Deployment, creates the StatefulSet and keeps the PVC. Grafana 11.6.16 runs on dashboard-db. A second upgrade changes nothing. helm rollback to the Bitnami revision brings back Grafana 11.3.1 with all three dashboards and the service account from SQLite.
• This PR, from 1.13.5: the upgrade succeeds with no manual step. Dashboards, the service account and permissions survive the 11.3.1 to 11.6.16 schema migration (anonymous request for the private dashboard: 403, public: 200). Rollback to the Bitnami revision works on the migrated database.
• Both with server-side apply (the Helm 4 default) and with --server-side=false (client-side apply, as in Helm 3). I did not run a Helm 3 binary.
• The PodDisruptionBudget dashboard-ui is taken over with the new selector and covers both pods (1 disruption allowed). Rollback restores the old selector.
• Fresh install: StatefulSet, no PVC rendered by the new template, no ClusterRole.
• Real chart: helm lint passes, and helm template with the pre-be186a7e6 values and this dashboardui block renders the StatefulSet and its headless Service and no Grafana ClusterRole. I have not upgraded a full DBRepo stack with this commit.

Alternatives I tried and dropped: a pre-upgrade hook that deletes the old Deployment (the upgrade works, but helm rollback fails on the same selector error in reverse, and it needs RBAC and a kubectl image), renaming the Grafana objects and adding a dashboard-ui Service alias (works, but renames every Grafana object), and staying on SQLite on the same PVC (works, but needs the hook, allows one replica only, and would leave the dashboard-db data of 1.13 installs unused).

Existing deployments

• Compose: Grafana migrates its database schema forward on the first start of 11.6. Going back to the Bitnami 11.4.0 image would need a restore of the dashboard database.
• Helm: helm upgrade from a Bitnami-chart release needs no manual step. The old Deployment selects pods by app.kubernetes.io/component: grafana, which the official chart does not set, and selectors cannot be changed in place. Because Grafana is now a StatefulSet, Helm creates it and removes the old Deployment. helm rollback to the Bitnami release works the same way back.
• Helm storage: Grafana reads GF_DATABASE_* from dashboard-ui-secret and stores its state in dashboard-db, as on 1.13 and earlier. The values rewrite in be186a7 (1.14.0) dropped grafana.extraEnvVarsSecret: dashboard-ui-secret and persistence.enabled: false, so a 1.14 install whose Grafana was running used SQLite on the dashboard-ui PVC while dashboard-db stayed unused. That PVC (10Gi by default) is now kept (helm.sh/resource-policy: keep) and never deleted by Helm, so a rollback finds its data, but it is not migrated. The Dashboard Service setup job recreates the dashboards of existing databases. Dashboards made by hand in that SQLite database are not carried over. Once it is no longer needed: kubectl delete pvc dashboard-ui -n <namespace>.
• Helm with helm template, helm-diff, a plain --dry-run or Argo CD: lookup returns nothing there, so their output shows the PVC as deleted, and Argo CD would really delete it. Annotate it before upgrading: kubectl annotate pvc dashboard-ui helm.sh/resource-policy=keep -n <namespace>.
• Helm values: dashboardui.grafana.replicaCount / dashboardui.replicaCount is now dashboardui.replicas (official chart naming), and dashboardui.admin.user/password are now set in the chart values.

[maxlastglance]

The Grafana image and plugin changes look right, and the dashboard tests pass. One thing to fix before taking this out of draft: .scripts/test-grafana-plugins.py is not invoked anywhere in CI, so the new regression check can silently stop working.

Please run it from the existing lint-compose job, which already has Docker Compose available. The three red Java jobs are the unrelated MinIO failure addressed by #113.

[maxlastglance]

Thanks for adding the Helm changes and wiring the plugin check into CI. The Compose changes look good. I see two problems for existing Helm installations:

• The old Bitnami setup uses SQLite on the dashboard-ui PVC. The new chart loads the PostgreSQL settings from dashboard-ui-secret and disables persistence. Existing dashboards, users and credentials are not migrated, and the old PVC can disappear from the release. We should either preserve the current database for upgrades or provide a tested migration before dropping the claim.
• The chart change also changes the selector of Deployment/dashboard-ui. Kubernetes will reject that during a normal helm upgrade because deployment selectors are immutable. Recreate only changes the rollout strategy; it does not recreate the Deployment. This needs a compatible selector or a controlled replacement step.

Small thing: .scripts/test-grafana-plugins.py checks the image repository but not the tag, so grafana:latest still passes. The plugin checks also do not verify the expected versions. It would be better to assert the actual pins.

Fresh installs should be fine. My concern is the upgrade path for installations that already have Grafana data.

[satriadhikara]

Thanks, I could reproduce both upgrade problems. They are fixed in 5ffa279:

• Selector: Grafana now runs as a StatefulSet. Helm creates it and removes the old Deployment, so helm upgrade from a Bitnami-chart release needs no manual step, and helm rollback to that release works too. I also tried a pre-upgrade hook that deletes the Deployment, but then the rollback fails on the same selector error in reverse.

• Storage: the SQLite PVC comes from the values rewrite in be186a7. It dropped grafana.extraEnvVarsSecret: dashboard-ui-secret and persistence.enabled: false, so 1.14 Grafana fell back to SQLite on a PVC while dashboard-db stayed unused. 1.13 and earlier stored Grafana's state in dashboard-db, and this PR goes back to that, so 1.13 installs keep their dashboards and permissions (tested across the 11.3.1 to 11.6.16 migration). For 1.14 installs, the dashboard-ui PVC is now kept with helm.sh/resource-policy: keep instead of being deleted, and a rollback finds its data. The setup job recreates the dashboards of existing databases.

  One thing I noticed: with the shipped 1.14.1 values the Grafana pod never starts (the dashboard-service-config ConfigMap does not exist, and the admin secret keys are the official chart's names). So SQLite data only exists where someone fixed that by hand. Do you know of such installations with hand-made dashboards worth migrating? If so, I can add a migration step. Otherwise I would leave it at keeping the claim.

• Plugin check: it now asserts grafana/grafana:11.6.16 and the exact Infinity and OpenSearch versions in all Compose files and the Helm values.

I also turned off the chart's RBAC (it only serves the sidecars, which we don't use, and it created cluster-wide roles) and kept the PodDisruptionBudget the Bitnami chart had. The test setup and results are in the PR description.