Skip to content

fix(history): keep working measurements when a device is offline - #1592

Merged
frahlg merged 3 commits into
masterfrom
codex/history-partial-offline-20261012
Oct 12, 2026
Merged

frahlg merged 3 commits into
masterfrom
codex/history-partial-offline-20261012

Conversation

@frahlg

@frahlg frahlg commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

A fresh native install can show current meter readings while an offline battery prevents all dashboard history. The published v0.141.0-beta.1 reproduced this: meter samples kept writing, but the dashboard stayed empty and day/week totals read as zero.

Keep each working flow in chart and energy history. Mark missing power and state of charge as unknown, keep gaps in Live/day/week views, and flag incomplete measurements or an invalid household balance. Household load remains unknown until all inputs form a valid balance. Forecast learning and dispatch keep their existing strict checks.

Keep state schema 7 and put partial chart/energy rows and daily caches in separate additive tables, with missing-field evidence in the same transaction. Older Cores read only complete rows in the original tables and their cursor cannot bridge a partial interval after rollback. The newer Core reads both, including partial rows retained across the old return. The native schema/order release preflight passes. Quality survives averaging, retention, restart and full backup/restore. No launcher or schema-change guard is bypassed.

Validation:

  • Regression fails before the change; make verify and focused race tests pass on top of the merged timezone, EV departure and protocol PRs.
  • All 713 web tests pass.
  • Three review regressions fail before correction and pass afterward: configured solar with no measurement remains unknown, site battery SoC keeps its capacity weighting, and offline battery cards clear the stale bat_soc field. Final make verify and focused race tests pass on c0fdaa5.
  • Three isolated Linux ARM64 Core runs cover a working meter with a working battery, an offline battery, and an invalid balance. They keep meter history and have zero rejected ticks/write failures; missing battery/load values are null. These use a local development binary, not a published candidate. An additional copied-data return runs the actual published v0.141.0-beta.1, verifies all 18 partial evidence rows stay unchanged and out of its charts, then runs the new Core again and sees the saved partial history.
  • Browser check shows the meter curve, unknown battery/load/totals and the visible warning. Human inspection remains deferred to the owner's beta review.

This fixes the reproduced offline-device history defect. It does not establish the cause of the earlier storage deadlines or complete general migration qualification.

Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T18:20:49.540403Z c0fdaa5 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@frahlg
frahlg marked this pull request as draft October 12, 2026 18:04

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 90817e4b73

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread go/cmd/ftw/main.go
Comment on lines +3941 to 3943
if opts.PVInvalidReason != "" {
p.Missing |= state.MissingPV
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Treat configured but unmeasured PV as missing

When weather.pv_rated_w or PV arrays declare solar but no driver provides a PV flow, forecastMeasurementTopology sets HouseholdInvalidReason to configured_pv_without_measurement_source; it does not set PVInvalidReason. Because this condition checks only the latter and Known[DerPV] remains true when there is no flow, every live-meter tick is persisted with PV equal to a measured zero. The daily history then permanently reports 0 Wh rather than unknown solar generation, so this household topology needs to set MissingPV as well.

AGENTS.md reference: AGENTS.md:L22-L23

Useful? React with 👍 / 👎.

Comment thread go/internal/api/api_history_quality.go Outdated
Comment on lines +70 to +71
if count > 0 {
sum /= float64(count)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve capacity weighting for site SoC

On sites with multiple batteries of unequal capacity, this replaces the capacity-weighted avgSoC already computed by handleStatus with an unweighted mean of telemetry sources. For example, a 10 kWh battery at 20% and a 2 kWh battery at 80% changes from the existing 30% site value to 50%, making the dashboard and API report the wrong aggregate SoC. Keep the existing weighted value when all required SoCs are known, and only null it when qualification fails.

Useful? React with 👍 / 👎.

Comment thread go/internal/api/api_history_quality.go Outdated
Comment on lines +49 to +50
if source.Flow.DerType == telemetry.DerBattery {
d["soc"] = source.SoC

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clear the battery SoC field consumed by clients

When a battery becomes stale or offline, source.SoC is nil, but the driver map already contains the last reading under bat_soc from handleStatus. Assigning nil to a new soc key leaves that stale bat_soc intact; existing UI paths read d.bat_soc, so driver cards can continue showing the old percentage even though power and site-level SoC are unknown. Update or clear bat_soc instead.

AGENTS.md reference: AGENTS.md:L22-L23

Useful? React with 👍 / 👎.

Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
Signed-off-by: Fredrik Ahlgren <fredrik@sourceful-labs.com>
@frahlg
frahlg marked this pull request as ready for review October 12, 2026 18:17
@frahlg
frahlg merged commit afaa603 into master Oct 12, 2026
18 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c0fdaa5521

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread web/app.js
pts.push({ x: tsToX(ts), y: valToY(sr.data[j]) });
}
if (pts.length) segments.push(pts);
segments.forEach(function (pts) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep live pulses on the latest segment only

When a series contains one or more null gaps, this loop adds the endpoint of every historical segment to liveTips; the later pulse renderer then gives all of those old endpoints a static halo and a fresh ripple on every new data push. This makes measurements immediately before an outage look live, so only cache the final segment's endpoint when it reaches the newest sample.

AGENTS.md reference: AGENTS.md:L22-L23

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant