fix(docker): make Compose health checks reflect service state

[satriadhikara]

Description

Three Docker Compose health checks don't report the real state of their service. Two of them are always unhealthy and one is always healthy. This PR changes only the test: lines. The <<: *healthcheck-params anchors and all other service settings are unchanged.

Service(s)	Before	Effect	After
dbrepo-metadata-db-metrics, dbrepo-data-db-metrics	test: -h	Compose runs /bin/sh -c "-h", which fails with Illegal option -h. Always unhealthy.	Fetch the exporter's own /metrics over bash /dev/tcp and require mysql_up 1
dbrepo-auth-db, dbrepo-dashboard-db	psql -U … -h 127.0.0.1 … -c 'select version();'	TCP auth needs a password. dashboard-db has no PGPASSWORD, so it gets fe_sendauth: no password. Always unhealthy.	pg_isready -U … -h 127.0.0.1 -p 5432 -d …
dbrepo-gateway-service	curl -sSL 127.0.0.1:8080/health | grep 'UP' | exit	The pipeline ends in exit, which returns 0. Always healthy, even when /health can't be reached.	curl -fsS 127.0.0.1:8080/health | grep -q 'UP'

Files changed (same edits in each, where the service/check exists):

• .docker/docker-compose.yml: all three fixes
• docker-compose.yml: Postgres and gateway (this file has no exporter services)
• .live/config/includes.chroot_after_packages/etc/skel/docker-compose.yml: exporters and Postgres (the gateway in this file has no health check)

Why these commands

• Exporters: bitnamilegacy/mysqld-exporter:0.15.1 has bash but no curl or wget, which I checked with command -v. mysqld_exporter --version would only prove the binary exists. The /dev/tcp request checks that the exporter is serving on :9104 and can reach its database (mysql_up 1). That's the state Prometheus depends on.
• Postgres: pg_isready is in bitnamilegacy/postgresql:17.0.0 at /opt/bitnami/postgresql/bin/pg_isready. It checks whether the server accepts connections without logging in, so it doesn't need a password and doesn't log auth failures every interval. On current main, auth-db passes because it sets PGPASSWORD. I changed it too so both Postgres services use the same password-independent check.
• Gateway: nginx:1.28 includes curl. With -f, an HTTP error makes curl fail. With grep -q at the end of the pipeline, the exit status becomes the check result.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes, no API changes)
• Dependency update

Related Issue

None filed. Found while deploying DBRepo with Docker Compose at ITB (see Additional Notes).

Checklist

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing tests passed.

Additional Notes

Field evidence (ITB deployment, DBRepo 1.13.4, Docker Compose)

In a production-style Compose deployment at ITB, both exporters were unhealthy because of the invalid -h command, and both Postgres services stayed unhealthy because of the password-less psql check. They became healthy after the checks were replaced on the server. That deployment used mysqld_exporter --version for the exporters. This PR uses a stronger probe instead (see above).

Reproduction on main (3862655)

I ran the affected services in an isolated Compose project (-p hcrepro). An override file reset container_name, removed host ports, and set healthcheck interval: 5s / retries: 3 so results came in quickly. The test: commands were the ones from .docker/docker-compose.yml. The gateway used a minimal nginx conf containing only the location = /health block from dbrepo-gateway-service/dbrepo.conf, because the full conf won't start without its upstream hosts.

Before (docker inspect --format '{{json .State.Health}}', summarized):

hcrepro-dbrepo-auth-db-1              healthy    exit=0
hcrepro-dbrepo-dashboard-db-1         unhealthy  exit=2  'psql: error: connection to server at "127.0.0.1", port 5432 failed: fe_sendauth: no password'
hcrepro-dbrepo-data-db-metrics-1      unhealthy  exit=2  '/bin/sh: 0: Illegal option -h'
hcrepro-dbrepo-metadata-db-metrics-1  unhealthy  exit=2  '/bin/sh: 0: Illegal option -h'
hcrepro-dbrepo-gateway-service-1      healthy    exit=0

After (same project, recreated from this branch):

hcrepro-dbrepo-auth-db-1              healthy    exit=0  '127.0.0.1:5432 - accepting connections'
hcrepro-dbrepo-dashboard-db-1         healthy    exit=0  '127.0.0.1:5432 - accepting connections'
hcrepro-dbrepo-data-db-metrics-1      healthy    exit=0
hcrepro-dbrepo-metadata-db-metrics-1  healthy    exit=0
hcrepro-dbrepo-gateway-service-1      healthy    exit=0

Failure cases are now detected

Gateway check against a dead port, run inside the gateway container:

$ curl -sSL 127.0.0.1:8081/health | grep 'UP' | exit      # old
curl: (7) Failed to connect to 127.0.0.1 port 8081 after 0 ms: Could not connect to server
exit=0
$ curl -fsS 127.0.0.1:8081/health | grep -q UP            # new
curl: (7) Failed to connect to 127.0.0.1 port 8081 after 0 ms: Could not connect to server
exit=1

Exporter with its database stopped (docker stop on data-db):

hcrepro-dbrepo-data-db-metrics-1      unhealthy  streak=3 exit=1
hcrepro-dbrepo-metadata-db-metrics-1  healthy    streak=0 exit=0
# /metrics on data-db-metrics: mysql_up 0

pg_isready against a closed port returns no attempt with exit 3.

Not tested

• I didn't bring up the full stack, and I didn't run the root docker-compose.yml or the .live copy. They have the same test: strings as .docker/docker-compose.yml, which was the file I tested.
• I tested the gateway check with a minimal nginx conf instead of the full dbrepo.conf, as described above.
• I didn't build or boot the .live ISO.
• Exporters are now unhealthy while their database is unreachable. Nothing in the Compose files depends on an exporter with condition: service_healthy, so startup order is unchanged.

Helm

Helm doesn't have these bugs. I rendered the vendored Bitnami subcharts (mariadb-galera-13.2.7, postgresql-16.1.0, keycloak-25.2.0, nginx-20.0.2) with DBRepo's values for them and read the probes:

• The mysqld-exporter sidecars use httpGet / on the metrics port.
• dashboard-db and the Keycloak auth-db use pg_isready.
• The gateway uses tcpSocket for liveness and httpGet / for readiness. Unrelated to this PR: in data-deployment.yaml, consumer-deployment.yaml and metadata-deployment.yaml, the livenessProbe calls /actuator/health/readiness and the readinessProbe calls /actuator/health/liveness, so they appear to be swapped. I left that out of this PR.