diff --git a/README.md b/README.md index 9f0bb7d..bb8fa45 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ HAdoc keeps floor plans, cabling, power circuits, controller I/O (PLC channels, ESP pins) and the link to your home automation software in one place, as plain YAML files you keep in git. -> Status: early. The viewer, data format, validation, trace, WAGO import and editing in the app work; the other importers and the AREI diagrams are on the [roadmap](#roadmap). +> Status: early. The viewer, data format, validation, trace, editing in the app, the WAGO and ESPHome imports and the live connections to openHAB, UniFi, Tasmota and MQTT work; the PLC logic viewer and the AREI diagrams are on the [roadmap](#roadmap). ## What you get @@ -13,6 +13,9 @@ HAdoc keeps floor plans, cabling, power circuits, controller I/O (PLC channels, - **Cores and terminals**: list the cores of a cable and the terminal each one lands on; the cable then shows up on every port it touches. - **Edit mode**: drag points, place new ones, draw cable routes and link PLC channels to points. Edits are written back into your YAML files (comments and layout kept) and can be committed to git. - **Tap anything** to see its trace: the circuit and breaker that feed it, the cables and terminals it lands on, the device or PLC channel behind it, and the MQTT topic or openHAB item that reads it. +- **Live status** from UniFi, Tasmota and MQTT: online or offline and the IP address of every documented device. +- **openHAB**: the current state of each item and the rules that use it (by trigger, group trigger or in the script), with the script one tap away. +- **"Not documented yet"** report: openHAB items linked to a channel that no signal uses, unknown clients on your IoT subnet, Tasmota and ESPHome nodes on MQTT, and ESPHome pins without a signal. - **"If it stops working"** notes in plain language, for everyone in the house. - **Checks** that flag broken references, IDs that don't follow your scheme, and what is still undocumented. - **Search** by room, ID, device, PLC name or openHAB item. @@ -100,6 +103,36 @@ The export is read on every load, so re-exporting after a change in CODESYS upda Imported signals that are not linked to a point yet are listed under "To check". +### Importing ESPHome configs + +Point `imports.esphome.dir` at the ESPHome config folder (mount it into the container, e.g. `-v /opt/esphome/config:/esphome:ro`). Each node becomes a device with its node name as ID, and every pin the config uses becomes a port with what it is used for. Add the location, MAC and power in `devices.yaml` under the same ID. + +```yaml +imports: + esphome: + dir: /esphome +``` + +### Live connections + +```yaml +integrations: + poll_seconds: 60 + openhab: { url: "http://192.168.2.31:8080" } + unifi: { url: "https://unifi.example.com", subnets: [192.168.20.0/24] } + tasmota: {} + mqtt: { url: "mqtt://192.168.2.9:1883" } +``` + +| Source | What it adds | Secret (environment variable) | +| --- | --- | --- | +| openHAB | Item state, the rules that use an item, items no signal uses | `HADOC_OPENHAB_TOKEN`, an API token (optional when anonymous access is allowed) | +| UniFi | Online and IP by MAC address (or by name), unknown clients | `HADOC_UNIFI_KEY`, an Integration API key from Settings > Control Plane > Integrations | +| Tasmota | Online, IP, MAC and firmware of every device with `firmware: Tasmota` and a `hostname` | `HADOC_TASMOTA_USER`, `HADOC_TASMOTA_PASSWORD` when a web password is set | +| MQTT | Tasmota (LWT, discovery) and ESPHome (`/status`) nodes that announce themselves | `HADOC_MQTT_USER`, `HADOC_MQTT_PASSWORD` | + +Each source can name another variable with `token_env`, `key_env`, `user_env`/`password_env` or `username_env`/`password_env`. For UniFi OS consoles set `api_path: /proxy/network/integration/v1`. A source that fails shows its error in `GET /api/live`; the plan keeps working without it. + ### Validate ```bash @@ -131,8 +164,10 @@ In Docker the app runs as uid 1000, so the data folder must be writable for that | --- | --- | | `GET /api/house` | All data plus error and warning counts | | `GET /api/trace/:id` | The trace for a point, device or enclosure | -| `GET /api/issues` | Every error and warning | +| `GET /api/issues` | Every error and warning, and what is not documented yet (`info`) | | `GET /api/health` | Liveness and when the data was last loaded | +| `GET /api/live` | Status per source, live data per device, state and rules per openHAB item | +| `GET /api/live/rules/:uid` | The script of an openHAB rule | | `PATCH /api/points/:id` | Move a point: `{ "pos": [x, y] }` or `{ "pos": null }` | | `POST /api/points` | New point: `{ type, level, room, name?, pos?, signal? }`; `signal` links that signal to it | | `PUT /api/cables/:id/route` | Route on one floor: `{ level, path: [[x, y], …] }`, or `path: null` to remove it | @@ -145,7 +180,7 @@ The server reloads the data when files in the data folder change. 1. **MVP**: plan underlay, rooms, points, point pages, trace, checks *(done)*. 2. **WAGO import** from the CODESYS 2.3 PLC configuration export *(done)*. 3. **Cables and circuits**: drawing routes, terminals and cores, breaker colouring, editing in the app with git commits *(done)*. -4. **Importers**: openHAB, UniFi (status and IP), ESPHome, Tasmota, MQTT, plus the "undocumented" report and the rules behind each item. +4. **Importers**: openHAB, UniFi (status and IP), ESPHome, Tasmota, MQTT, plus the "undocumented" report and the rules behind each item *(done)*. 5. **PLC logic viewer** from POU exports, wall and enclosure views, QR labels. 6. **AREI** single-line diagram (eendraadschema) and situation sketch (situatieschets). 7. Importers for other systems (Home Assistant, KNX, Loxone, Shelly). diff --git a/docker-compose.yml b/docker-compose.yml index 1870c41..8775c8e 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -14,5 +14,8 @@ services: HADOC_GIT_COMMIT: "1" # Set to "1" to turn editing off. HADOC_READONLY: "0" + # Secrets for the live connections in settings.yaml (integrations). + HADOC_OPENHAB_TOKEN: ${HADOC_OPENHAB_TOKEN:-} + HADOC_UNIFI_KEY: ${HADOC_UNIFI_KEY:-} # The app writes edits as the "node" user (uid 1000); ./house must be writable for it. user: "1000:1000" diff --git a/examples/demo-house/devices.yaml b/examples/demo-house/devices.yaml index e1fa759..b5952bd 100644 --- a/examples/demo-house/devices.yaml +++ b/examples/demo-house/devices.yaml @@ -14,8 +14,7 @@ hostname: esp-dressing mac: null powered_by: sonoff-mirror - ports: - - { name: GPIO14, signal: GPIO } + # Ports come from imports/esphome/esp-dressing.yaml. - id: sonoff-mirror kind: relay diff --git a/examples/demo-house/imports/esphome/esp-dressing.yaml b/examples/demo-house/imports/esphome/esp-dressing.yaml new file mode 100644 index 0000000..ed2ca5a --- /dev/null +++ b/examples/demo-house/imports/esphome/esp-dressing.yaml @@ -0,0 +1,40 @@ +# A copy of the ESPHome config, as an example for imports.esphome. +substitutions: + name: esp-dressing + +esphome: + name: ${name} + +esp8266: + board: nodemcuv2 + +wifi: + ssid: !secret wifi_ssid + password: !secret wifi_password + +mqtt: + broker: 192.168.2.9 + +binary_sensor: + - platform: gpio + pin: GPIO14 + name: Motion + device_class: motion + + - platform: gpio + pin: + number: GPIO12 + inverted: true + mode: INPUT_PULLUP + name: Door contact + +i2c: + sda: GPIO4 + scl: GPIO5 + +sensor: + - platform: bme280_i2c + temperature: + name: Dressing temperature + humidity: + name: Dressing humidity diff --git a/examples/demo-house/imports/esphome/secrets.yaml b/examples/demo-house/imports/esphome/secrets.yaml new file mode 100644 index 0000000..b492e6d --- /dev/null +++ b/examples/demo-house/imports/esphome/secrets.yaml @@ -0,0 +1,2 @@ +wifi_ssid: demo +wifi_password: demo diff --git a/examples/demo-house/settings.yaml b/examples/demo-house/settings.yaml index 36aaca6..4937db0 100644 --- a/examples/demo-house/settings.yaml +++ b/examples/demo-house/settings.yaml @@ -26,3 +26,15 @@ imports: wago: config: imports/plc-configuration.exp controller: plc + # ESPHome node configs; every pin they use becomes a port on the ESP. + esphome: + dir: imports/esphome + +# Live connections, polled while the app runs. Secrets come from environment +# variables (HADOC_OPENHAB_TOKEN, HADOC_UNIFI_KEY, ...), never from this file. +# integrations: +# poll_seconds: 60 +# openhab: { url: "http://192.168.2.31:8080" } +# unifi: { url: "https://unifi.example.com", subnets: [192.168.20.0/24] } +# tasmota: {} +# mqtt: { url: "mqtt://192.168.2.9:1883" } diff --git a/package-lock.json b/package-lock.json index 6f43dde..e4e7bd4 100644 --- a/package-lock.json +++ b/package-lock.json @@ -16,6 +16,15 @@ "node": ">=22" } }, + "node_modules/@babel/runtime": { + "version": "7.29.10", + "resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.29.10.tgz", + "integrity": "sha512-NpugTzbKgGrNe45Ui32cN8vCFFMxVtP/9rtfwv0nP5FRDAjvYBxJcLVpYA9mr1jxP6cnIT7x9LUzuDTbWYYNRA==", + "license": "MIT", + "engines": { + "node": ">=6.9.0" + } + }, "node_modules/@esbuild/aix-ppc64": { "version": "0.28.2", "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.2.tgz", @@ -1112,12 +1121,29 @@ "version": "22.20.5", "resolved": "https://registry.npmjs.org/@types/node/-/node-22.20.5.tgz", "integrity": "sha512-U2+DNr+wSjpsTS/wZGYHq7GcwfuSmKiKvoPvK22zwTlRhU91yOniN4qRR5KhIjvif7ysw/dz/hKmfDH0Ris4aA==", - "dev": true, "license": "MIT", "dependencies": { "undici-types": "~6.21.0" } }, + "node_modules/@types/readable-stream": { + "version": "4.0.25", + "resolved": "https://registry.npmjs.org/@types/readable-stream/-/readable-stream-4.0.25.tgz", + "integrity": "sha512-7GsxL8Wd5X3jh73/qgW7WCnbBvGSdD8U3E4Jiuus3/8I+PN20DtlkJtwFhr/sImXfDDQ2Z03i/9wNiIF3TVSsw==", + "license": "MIT", + "dependencies": { + "@types/node": "*" + } + }, + "node_modules/@types/ws": { + "version": "8.18.2", + "resolved": "https://registry.npmjs.org/@types/ws/-/ws-8.18.2.tgz", + "integrity": "sha512-67MQl+fpWKVTT1NYdnmo3U4sc/xPo/zQBncVnI74qmQa0z/b+1g6iYqNmGCPbxO+zz2aklb08a0oHfegiVd0/w==", + "license": "MIT", + "dependencies": { + "@types/node": "*" + } + }, "node_modules/@vitest/mocker": { "version": "5.0.3", "resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-5.0.3.tgz", @@ -1156,6 +1182,18 @@ "url": "https://opencollective.com/vitest" } }, + "node_modules/abort-controller": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/abort-controller/-/abort-controller-3.0.0.tgz", + "integrity": "sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg==", + "license": "MIT", + "dependencies": { + "event-target-shim": "^5.0.0" + }, + "engines": { + "node": ">=6.5" + } + }, "node_modules/abstract-logging": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/abstract-logging/-/abstract-logging-2.0.1.tgz", @@ -1276,6 +1314,38 @@ "node": "18 || 20 || >=22" } }, + "node_modules/base64-js": { + "version": "1.5.1", + "resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz", + "integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT" + }, + "node_modules/bl": { + "version": "6.1.6", + "resolved": "https://registry.npmjs.org/bl/-/bl-6.1.6.tgz", + "integrity": "sha512-jLsPgN/YSvPUg9UX0Kd73CXpm2Psg9FxMeCSXnk3WBO3CMT10JMwijubhGfHCnFu6TPn1ei3b975dxv7K2pWVg==", + "license": "MIT", + "dependencies": { + "@types/readable-stream": "^4.0.0", + "buffer": "^6.0.3", + "inherits": "^2.0.4", + "readable-stream": "^4.2.0" + } + }, "node_modules/brace-expansion": { "version": "5.0.12", "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.12.tgz", @@ -1288,6 +1358,48 @@ "node": "20 || >=22" } }, + "node_modules/broker-factory": { + "version": "3.1.15", + "resolved": "https://registry.npmjs.org/broker-factory/-/broker-factory-3.1.15.tgz", + "integrity": "sha512-ko+aWvgNuP49meGrdjUu7rC+Y+Wai3cCPxP3xWwHsHfehFjOh5ZQM2yC4gEB2UddeZ/YXhm0K1eG/L6fxym2Og==", + "license": "MIT", + "dependencies": { + "@babel/runtime": "^7.29.7", + "fast-unique-numbers": "^9.0.27", + "tslib": "^2.8.1", + "worker-factory": "^7.0.50" + } + }, + "node_modules/buffer": { + "version": "6.0.3", + "resolved": "https://registry.npmjs.org/buffer/-/buffer-6.0.3.tgz", + "integrity": "sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT", + "dependencies": { + "base64-js": "^1.3.1", + "ieee754": "^1.2.1" + } + }, + "node_modules/buffer-from": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.2.tgz", + "integrity": "sha512-E+XQCRwSbaaiChtv6k6Dwgc+bx+Bs6vuKJHHl5kox/BaKbhiXzqQOwK4cO22yElGp2OCmjwVhT3HmxgyPGnJfQ==", + "license": "MIT" + }, "node_modules/chai": { "version": "6.3.0", "resolved": "https://registry.npmjs.org/chai/-/chai-6.3.0.tgz", @@ -1324,6 +1436,41 @@ "node": ">=6" } }, + "node_modules/commist": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/commist/-/commist-3.2.0.tgz", + "integrity": "sha512-4PIMoPniho+LqXmpS5d3NuGYncG6XWlkBSVGiWycL22dd42OYdUGil2CWuzklaJoNxyxUSpO4MKIBU94viWNAw==", + "license": "MIT" + }, + "node_modules/concat-stream": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/concat-stream/-/concat-stream-2.0.0.tgz", + "integrity": "sha512-MWufYdFw53ccGjCA+Ol7XJYpAlW6/prSMzuPOTRnJGcGzuhLn4Scrz7qf6o8bROZ514ltazcIFJZevcfbo0x7A==", + "engines": [ + "node >= 6.0" + ], + "license": "MIT", + "dependencies": { + "buffer-from": "^1.0.0", + "inherits": "^2.0.3", + "readable-stream": "^3.0.2", + "typedarray": "^0.0.6" + } + }, + "node_modules/concat-stream/node_modules/readable-stream": { + "version": "3.6.2", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz", + "integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==", + "license": "MIT", + "dependencies": { + "inherits": "^2.0.3", + "string_decoder": "^1.1.1", + "util-deprecate": "^1.0.1" + }, + "engines": { + "node": ">= 6" + } + }, "node_modules/content-disposition": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-3.0.0.tgz", @@ -1350,6 +1497,23 @@ "url": "https://opencollective.com/express" } }, + "node_modules/debug": { + "version": "4.4.3", + "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", + "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==", + "license": "MIT", + "dependencies": { + "ms": "^2.1.3" + }, + "engines": { + "node": ">=6.0" + }, + "peerDependenciesMeta": { + "supports-color": { + "optional": true + } + } + }, "node_modules/deepmerge": { "version": "4.3.1", "resolved": "https://registry.npmjs.org/deepmerge/-/deepmerge-4.3.1.tgz", @@ -1485,6 +1649,24 @@ "@types/estree": "^1.0.0" } }, + "node_modules/event-target-shim": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/event-target-shim/-/event-target-shim-5.0.1.tgz", + "integrity": "sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ==", + "license": "MIT", + "engines": { + "node": ">=6" + } + }, + "node_modules/events": { + "version": "3.3.0", + "resolved": "https://registry.npmjs.org/events/-/events-3.3.0.tgz", + "integrity": "sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==", + "license": "MIT", + "engines": { + "node": ">=0.8.x" + } + }, "node_modules/expect-type": { "version": "1.4.0", "resolved": "https://registry.npmjs.org/expect-type/-/expect-type-1.4.0.tgz", @@ -1556,6 +1738,19 @@ "fast-decode-uri-component": "^1.0.1" } }, + "node_modules/fast-unique-numbers": { + "version": "9.0.27", + "resolved": "https://registry.npmjs.org/fast-unique-numbers/-/fast-unique-numbers-9.0.27.tgz", + "integrity": "sha512-nDA9ADeINN8SA2u2wCtU+siWFTTDqQR37XvgPIDDmboWQeExz7X0mImxuaN+kJddliIqy2FpVRmnvRZ+j8i1/A==", + "license": "MIT", + "dependencies": { + "@babel/runtime": "^7.29.2", + "tslib": "^2.8.1" + }, + "engines": { + "node": ">=18.2.0" + } + }, "node_modules/fast-uri": { "version": "3.1.8", "resolved": "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.8.tgz", @@ -1694,6 +1889,12 @@ "url": "https://github.com/sponsors/isaacs" } }, + "node_modules/help-me": { + "version": "5.0.0", + "resolved": "https://registry.npmjs.org/help-me/-/help-me-5.0.0.tgz", + "integrity": "sha512-7xgomUX6ADmcYzFik0HzAxh/73YlKR9bmFzf51CZwR+b6YtzU2m0u49hQCqV6SvlqIqsaxovfwdvbnsw3b/zpg==", + "license": "MIT" + }, "node_modules/http-errors": { "version": "2.0.1", "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz", @@ -1714,12 +1915,41 @@ "url": "https://opencollective.com/express" } }, + "node_modules/ieee754": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz", + "integrity": "sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "BSD-3-Clause" + }, "node_modules/inherits": { "version": "2.0.4", "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz", "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==", "license": "ISC" }, + "node_modules/ip-address": { + "version": "10.7.3", + "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.7.3.tgz", + "integrity": "sha512-A1kdq/tSb5QjvKvAMgIoEvDBIgL7qaqVP/jkvSwYYRZ9iEzvPpopxp2wQfu3SuZRHtpHNxMn8Fs0bS+gf5Xmwg==", + "license": "MIT", + "engines": { + "node": ">= 12" + } + }, "node_modules/ipaddr.js": { "version": "2.5.0", "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-2.5.0.tgz", @@ -2100,6 +2330,15 @@ "url": "https://github.com/sponsors/isaacs" } }, + "node_modules/minimist": { + "version": "1.2.8", + "resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz", + "integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/ljharb" + } + }, "node_modules/minipass": { "version": "7.1.3", "resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz", @@ -2109,6 +2348,54 @@ "node": ">=16 || 14 >=14.17" } }, + "node_modules/mqtt": { + "version": "5.16.0", + "resolved": "https://registry.npmjs.org/mqtt/-/mqtt-5.16.0.tgz", + "integrity": "sha512-G0VsEtuJz5sJUgZJaaADMVfW7SIFPzkIuHR+d6rk+tBeSwdCfd9BaIRycRfd2AbyFDEmLlFM+SDQeti/+IbOnw==", + "license": "MIT", + "dependencies": { + "@types/readable-stream": "^4.0.21", + "@types/ws": "^8.18.1", + "commist": "^3.2.0", + "concat-stream": "^2.0.0", + "debug": "^4.4.1", + "help-me": "^5.0.0", + "lru-cache": "^10.4.3", + "minimist": "^1.2.8", + "mqtt-packet": "^9.0.2", + "readable-stream": "^4.7.0", + "rfdc": "^1.4.1", + "socks": "^2.8.6", + "split2": "^4.2.0", + "worker-timers": "^8.0.23", + "ws": "^8.18.3" + }, + "bin": { + "mqtt": "build/bin/mqtt.js", + "mqtt_pub": "build/bin/pub.js", + "mqtt_sub": "build/bin/sub.js" + }, + "engines": { + "node": ">=16.0.0" + } + }, + "node_modules/mqtt-packet": { + "version": "9.0.2", + "resolved": "https://registry.npmjs.org/mqtt-packet/-/mqtt-packet-9.0.2.tgz", + "integrity": "sha512-MvIY0B8/qjq7bKxdN1eD+nrljoeaai+qjLJgfRn3TiMuz0pamsIWY2bFODPZMSNmabsLANXsLl4EMoWvlaTZWA==", + "license": "MIT", + "dependencies": { + "bl": "^6.0.8", + "debug": "^4.3.4", + "process-nextick-args": "^2.0.1" + } + }, + "node_modules/mqtt/node_modules/lru-cache": { + "version": "10.4.3", + "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-10.4.3.tgz", + "integrity": "sha512-JNAzZcXrCt42VGLuYz0zfAzDfAvJWW6AfYlDBQyDV5DClI2m5sAmK+OIO7s59XfsRsWHp02jAJrRadPRGTt6SQ==", + "license": "ISC" + }, "node_modules/mri": { "version": "1.2.0", "resolved": "https://registry.npmjs.org/mri/-/mri-1.2.0.tgz", @@ -2119,6 +2406,12 @@ "node": ">=4" } }, + "node_modules/ms": { + "version": "2.1.3", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", + "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==", + "license": "MIT" + }, "node_modules/nanoid": { "version": "3.3.20", "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.20.tgz", @@ -2263,6 +2556,21 @@ "node": "^10 || ^12 || >=14" } }, + "node_modules/process": { + "version": "0.11.10", + "resolved": "https://registry.npmjs.org/process/-/process-0.11.10.tgz", + "integrity": "sha512-cdGef/drWFoydD1JsMzuFf8100nZl+GT+yacc2bEced5f9Rjk4z+WtFUTBu9PhOi9j/jfmBPu0mMEY4wIdAF8A==", + "license": "MIT", + "engines": { + "node": ">= 0.6.0" + } + }, + "node_modules/process-nextick-args": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/process-nextick-args/-/process-nextick-args-2.0.1.tgz", + "integrity": "sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==", + "license": "MIT" + }, "node_modules/process-warning": { "version": "5.1.1", "resolved": "https://registry.npmjs.org/process-warning/-/process-warning-5.1.1.tgz", @@ -2285,6 +2593,22 @@ "integrity": "sha512-tYC1Q1hgyRuHgloV/YXs2w15unPVh8qfu/qCTfhTYamaw7fyhumKa2yGpdSo87vY32rIclj+4fWYQXUMs9EHvg==", "license": "MIT" }, + "node_modules/readable-stream": { + "version": "4.8.0", + "resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-4.8.0.tgz", + "integrity": "sha512-2jhTTU3Eilp/2DMT71gbcoqC35eQHasMJH0RHkYxwlxIv3+FR6Y5xAFZIwEGLxKkjZYEFlsiMNIdwUqjK2lLHQ==", + "license": "MIT", + "dependencies": { + "abort-controller": "^3.0.0", + "buffer": "^6.0.3", + "events": "^3.3.0", + "process": "^0.11.10", + "string_decoder": "^1.3.0" + }, + "engines": { + "node": "^12.22.0 || ^14.17.0 || >=16.0.0" + } + }, "node_modules/readdirp": { "version": "4.1.2", "resolved": "https://registry.npmjs.org/readdirp/-/readdirp-4.1.2.tgz", @@ -2386,6 +2710,26 @@ "node": ">=6" } }, + "node_modules/safe-buffer": { + "version": "5.2.1", + "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz", + "integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/feross" + }, + { + "type": "patreon", + "url": "https://www.patreon.com/feross" + }, + { + "type": "consulting", + "url": "https://feross.org/support" + } + ], + "license": "MIT" + }, "node_modules/safe-regex2": { "version": "5.1.2", "resolved": "https://registry.npmjs.org/safe-regex2/-/safe-regex2-5.1.2.tgz", @@ -2457,6 +2801,30 @@ "integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==", "license": "ISC" }, + "node_modules/smart-buffer": { + "version": "4.2.0", + "resolved": "https://registry.npmjs.org/smart-buffer/-/smart-buffer-4.2.0.tgz", + "integrity": "sha512-94hK0Hh8rPqQl2xXc3HsaBoOXKV20MToPkcXvwbISWLEs+64sBq5kFgn2kJDHb1Pry9yrP0dxrCI9RRci7RXKg==", + "license": "MIT", + "engines": { + "node": ">= 6.0.0", + "npm": ">= 3.0.0" + } + }, + "node_modules/socks": { + "version": "2.8.10", + "resolved": "https://registry.npmjs.org/socks/-/socks-2.8.10.tgz", + "integrity": "sha512-e0VyvkVTwVYViNovRkZ9aodhxVlyoMn7eJhVUPxZ+eK9P/7CBkxvvsBOHqFPEH416726W8tLXXXjKwqgTErrCQ==", + "license": "MIT", + "dependencies": { + "ip-address": "^10.1.1", + "smart-buffer": "^4.2.0" + }, + "engines": { + "node": ">= 10.0.0", + "npm": ">= 3.0.0" + } + }, "node_modules/sonic-boom": { "version": "4.2.1", "resolved": "https://registry.npmjs.org/sonic-boom/-/sonic-boom-4.2.1.tgz", @@ -2501,6 +2869,15 @@ "dev": true, "license": "MIT" }, + "node_modules/string_decoder": { + "version": "1.3.0", + "resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz", + "integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==", + "license": "MIT", + "dependencies": { + "safe-buffer": "~5.2.0" + } + }, "node_modules/svelte": { "version": "5.57.2", "resolved": "https://registry.npmjs.org/svelte/-/svelte-5.57.2.tgz", @@ -2630,6 +3007,12 @@ "node": ">=0.6" } }, + "node_modules/tslib": { + "version": "2.8.1", + "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz", + "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==", + "license": "0BSD" + }, "node_modules/tsx": { "version": "4.23.15", "resolved": "https://registry.npmjs.org/tsx/-/tsx-4.23.15.tgz", @@ -2649,6 +3032,12 @@ "fsevents": "~2.3.3" } }, + "node_modules/typedarray": { + "version": "0.0.6", + "resolved": "https://registry.npmjs.org/typedarray/-/typedarray-0.0.6.tgz", + "integrity": "sha512-/aCDEGatGvZ2BIk+HmLf4ifCJFwvKFNb9/JeZPMulfgFracn9QFcAf5GO8B/mweUjSoblS5In0cWhqpfs/5PQA==", + "license": "MIT" + }, "node_modules/typescript": { "version": "6.0.3", "resolved": "https://registry.npmjs.org/typescript/-/typescript-6.0.3.tgz", @@ -2667,7 +3056,12 @@ "version": "6.21.0", "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz", "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==", - "dev": true, + "license": "MIT" + }, + "node_modules/util-deprecate": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz", + "integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==", "license": "MIT" }, "node_modules/vite": { @@ -2864,6 +3258,74 @@ "node": ">=20.11" } }, + "node_modules/worker-factory": { + "version": "7.0.50", + "resolved": "https://registry.npmjs.org/worker-factory/-/worker-factory-7.0.50.tgz", + "integrity": "sha512-hhwc0G+sFwM4qBuhJIUBn2p1Jf8v/FwmLUANBf/Q+Lt2uI8mfIZQhXaZQACodQD4R7Zp6cn/6702bIvNn2puJQ==", + "license": "MIT", + "dependencies": { + "@babel/runtime": "^7.29.7", + "fast-unique-numbers": "^9.0.27", + "tslib": "^2.8.1" + } + }, + "node_modules/worker-timers": { + "version": "8.0.34", + "resolved": "https://registry.npmjs.org/worker-timers/-/worker-timers-8.0.34.tgz", + "integrity": "sha512-WXL+Dqsm0G6dnC66rQsvM3tPT2adhbqSirWUCZGglkALOr8ocS0KlpU0iuKbjflJOlu3ydZikMEVrYnHOM9suw==", + "license": "MIT", + "dependencies": { + "@babel/runtime": "^7.29.7", + "tslib": "^2.8.1", + "worker-timers-broker": "^8.0.18", + "worker-timers-worker": "^9.0.15" + } + }, + "node_modules/worker-timers-broker": { + "version": "8.0.18", + "resolved": "https://registry.npmjs.org/worker-timers-broker/-/worker-timers-broker-8.0.18.tgz", + "integrity": "sha512-FrjzDVX1wKfZN0gRbCFqv8VHuTncG4sbI/WGEg4tSSQeIsnwqg4YBYWMAHYLJtUDEYYmiK65UKFrVMVk2irDSg==", + "license": "MIT", + "dependencies": { + "@babel/runtime": "^7.29.7", + "broker-factory": "^3.1.15", + "fast-unique-numbers": "^9.0.27", + "tslib": "^2.8.1", + "worker-timers-worker": "^9.0.15" + } + }, + "node_modules/worker-timers-worker": { + "version": "9.0.15", + "resolved": "https://registry.npmjs.org/worker-timers-worker/-/worker-timers-worker-9.0.15.tgz", + "integrity": "sha512-KKUe7lZ/Aignr51H6hOUik8LwTnIgojH/1lwhli8A8qIEIyewogZTpNpMW5B6BF7nmwBOkUoTYgf1H3QShcjSA==", + "license": "MIT", + "dependencies": { + "@babel/runtime": "^7.29.7", + "tslib": "^2.8.1", + "worker-factory": "^7.0.50" + } + }, + "node_modules/ws": { + "version": "8.22.0", + "resolved": "https://registry.npmjs.org/ws/-/ws-8.22.0.tgz", + "integrity": "sha512-Ydggc987+RO0AnWtZ/7Wq9FtNvcrL1b/RO0ud9mWjUPgDrsAAwQSF51sm2hm1XofbU/4jkpGEsLFsZZxU+1DOg==", + "license": "MIT", + "engines": { + "node": ">=10.0.0" + }, + "peerDependencies": { + "bufferutil": "^4.0.1", + "utf-8-validate": ">=5.0.2" + }, + "peerDependenciesMeta": { + "bufferutil": { + "optional": true + }, + "utf-8-validate": { + "optional": true + } + } + }, "node_modules/yaml": { "version": "2.9.1", "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.1.tgz", @@ -2894,6 +3356,7 @@ "@fastify/static": "^10.1.6", "ajv": "^8.20.0", "fastify": "^5.12.5", + "mqtt": "^5.16.0", "yaml": "^2.9.1" }, "bin": { diff --git a/schema/devices.schema.json b/schema/devices.schema.json index 46522f4..62f1c85 100644 --- a/schema/devices.schema.json +++ b/schema/devices.schema.json @@ -26,7 +26,8 @@ "required": ["name"], "properties": { "name": { "type": "string", "description": "Physical label as printed, e.g. 'DI 01.3', 'GPIO14', 'L'." }, - "signal": { "type": "string", "description": "e.g. 'DI 24V', 'DO 24V', '230V', 'GPIO'." } + "signal": { "type": "string", "description": "e.g. 'DI 24V', 'DO 24V', '230V', 'GPIO'." }, + "use": { "type": "string", "description": "What the firmware config uses it for, e.g. 'binary_sensor: Motion'. Set by imports." } } } }, diff --git a/schema/settings.schema.json b/schema/settings.schema.json index ca8ea5f..81045c3 100644 --- a/schema/settings.schema.json +++ b/schema/settings.schema.json @@ -121,6 +121,120 @@ "description": "Prefix for module device IDs." } } + }, + "esphome": { + "type": "object", + "additionalProperties": false, + "required": [ + "dir" + ], + "description": "Folder with ESPHome node configs (the ESPHome dashboard's config folder). Pins become device ports.", + "properties": { + "dir": { + "type": "string", + "description": "Path relative to the data folder, or absolute, e.g. /esphome." + } + } + } + } + }, + "integrations": { + "type": "object", + "additionalProperties": false, + "description": "Live connections to other systems, polled while the app runs. Secrets come from environment variables, never from this file.", + "properties": { + "poll_seconds": { + "type": "number", + "minimum": 10, + "default": 60 + }, + "openhab": { + "type": "object", + "additionalProperties": false, + "required": [ + "url" + ], + "properties": { + "url": { + "type": "string", + "description": "e.g. http://192.168.2.31:8080" + }, + "token_env": { + "type": "string", + "description": "Variable holding an API token. Default HADOC_OPENHAB_TOKEN." + } + } + }, + "unifi": { + "type": "object", + "additionalProperties": false, + "required": [ + "url" + ], + "properties": { + "url": { + "type": "string", + "description": "Network application URL, e.g. https://unifi.example.com" + }, + "api_path": { + "type": "string", + "default": "/integration/v1", + "description": "'/integration/v1' for a self-hosted Network application, '/proxy/network/integration/v1' on a UniFi OS console." + }, + "site": { + "type": "string", + "default": "default", + "description": "Site name or ID." + }, + "key_env": { + "type": "string", + "description": "Variable holding the Integration API key. Default HADOC_UNIFI_KEY." + }, + "subnets": { + "type": "array", + "items": { + "type": "string", + "pattern": "^\\d+\\.\\d+\\.\\d+\\.\\d+/\\d+$" + }, + "description": "Only report unknown clients in these subnets, e.g. ['192.168.20.0/24'] for the IoT VLAN. Default: every client." + } + } + }, + "tasmota": { + "type": "object", + "additionalProperties": false, + "description": "Polls every device with firmware Tasmota and a hostname.", + "properties": { + "user_env": { + "type": "string", + "description": "Variable with the web user, when a password is set. Default HADOC_TASMOTA_USER." + }, + "password_env": { + "type": "string", + "description": "Variable with the web password. Default HADOC_TASMOTA_PASSWORD." + } + } + }, + "mqtt": { + "type": "object", + "additionalProperties": false, + "required": [ + "url" + ], + "properties": { + "url": { + "type": "string", + "description": "e.g. mqtt://192.168.2.9:1883" + }, + "username_env": { + "type": "string", + "description": "Default HADOC_MQTT_USER." + }, + "password_env": { + "type": "string", + "description": "Default HADOC_MQTT_PASSWORD." + } + } } } } diff --git a/server/package.json b/server/package.json index cfe5d63..4a4a676 100644 --- a/server/package.json +++ b/server/package.json @@ -18,6 +18,7 @@ "@fastify/static": "^10.1.6", "ajv": "^8.20.0", "fastify": "^5.12.5", + "mqtt": "^5.16.0", "yaml": "^2.9.1" }, "devDependencies": { diff --git a/server/src/app.ts b/server/src/app.ts index 88aef41..86e3711 100644 --- a/server/src/app.ts +++ b/server/src/app.ts @@ -4,6 +4,7 @@ import Fastify from "fastify"; import fastifyStatic from "@fastify/static"; import { circuitOf } from "./circuits.js"; import { EditError, editor } from "./edit.js"; +import { liveService, undocumentedPins } from "./live/index.js"; import { indexHouse, type HouseIndex } from "./index-house.js"; import { loadHouse } from "./load.js"; import type { House } from "./model.js"; @@ -20,6 +21,8 @@ export interface AppOptions { readOnly?: boolean; /** Commit every edit in the data folder's git repository. */ gitCommit?: boolean; + /** Replaces global fetch for the live integrations, for tests. */ + fetch?: typeof fetch; } const xy = { type: "array", items: { type: "number" }, minItems: 2, maxItems: 2 } as const; @@ -32,10 +35,16 @@ export async function buildApp(options: AppOptions) { let index: HouseIndex = indexHouse(house); let loadedAt = new Date(); + const live = liveService(() => house, { fetch: options.fetch, log: (msg) => app.log.warn(msg) }); + live.configure(house.settings.integrations); + app.addHook("onClose", async () => live.stop()); + const allIssues = () => [...house.issues, ...undocumentedPins(house), ...live.report()]; + const reload = async () => { house = await loadHouse(dataDir); index = indexHouse(house); loadedAt = new Date(); + live.configure(house.settings.integrations); app.log.info({ issues: house.issues.length }, "data reloaded"); }; @@ -51,7 +60,8 @@ export async function buildApp(options: AppOptions) { app.get("/api/health", async () => ({ ok: true, loadedAt: loadedAt.toISOString() })); app.get("/api/house", async () => { - const { issues, ...rest } = house; + const { issues: _, ...rest } = house; + const issues = allIssues(); return { ...rest, circuitOf: circuitOf(house, index), @@ -59,11 +69,20 @@ export async function buildApp(options: AppOptions) { issueCount: { errors: issues.filter((i) => i.severity === "error").length, warnings: issues.filter((i) => i.severity === "warning").length, + info: issues.filter((i) => i.severity === "info").length, }, }; }); - app.get("/api/issues", async () => house.issues); + app.get("/api/issues", async () => allIssues()); + + app.get("/api/live", async () => live.snapshot()); + + app.get<{ Params: { uid: string } }>("/api/live/rules/:uid", async (req, reply) => { + const rule = live.rule(req.params.uid); + if (!rule) return reply.code(404).send({ error: `No rule "${req.params.uid}"` }); + return rule; + }); app.get<{ Params: { id: string } }>("/api/trace/:id", async (req, reply) => { const trace = traceThing(house, index, req.params.id); diff --git a/server/src/importers/esphome.ts b/server/src/importers/esphome.ts new file mode 100644 index 0000000..ea36145 --- /dev/null +++ b/server/src/importers/esphome.ts @@ -0,0 +1,90 @@ +// Reads ESPHome node configs and turns every pin they use into a device port. + +import { parse } from "yaml"; +import type { Device } from "../model.js"; + +export interface EsphomePin { + pin: string; + /** What uses it, e.g. "binary_sensor/gpio: Motion" or "i2c: sda". */ + use: string; +} + +export interface EsphomeNode { + name: string; + board?: string; + platform?: string; + pins: EsphomePin[]; +} + +/** Top-level keys that never hold pins of their own. */ +const SKIP = new Set(["esphome", "substitutions", "packages", "wifi", "api", "ota", "logger", "web_server", "captive_portal", "mqtt", "time"]); +const PLATFORMS = ["esp8266", "esp32", "rp2040", "bk72xx", "rtl87xx", "libretiny"]; +const PIN_KEY = /^(pin|.*_pin|sda|scl|pin_[a-z])$/; + +function substitute(value: string, subs: Record): string { + return value.replace(/\$\{?([A-Za-z0-9_]+)\}?/g, (m, key: string) => subs[key] ?? m); +} + +function pinName(value: unknown, subs: Record): string | null { + if (typeof value === "string" || typeof value === "number") return substitute(String(value), subs); + if (value && typeof value === "object" && "number" in value) return pinName((value as { number: unknown }).number, subs); + return null; +} + +/** Parses one node config. Returns null for files that are not a node (secrets, packages). */ +export function parseEsphomeNode(text: string): EsphomeNode | null { + // !secret, !include and !lambda are custom tags; unknown tags parse as plain strings. + const doc = parse(text, { logLevel: "silent" }) as Record | null; + if (!doc || typeof doc !== "object" || !doc.esphome) return null; + const subs = Object.fromEntries( + Object.entries((doc.substitutions as Record) ?? {}).map(([k, v]) => [k, String(v)]), + ); + const esphome = doc.esphome as Record; + const name = substitute(String(esphome.name ?? ""), subs); + if (!name) return null; + + const platform = PLATFORMS.find((p) => p in doc); + const board = platform ? (doc[platform] as Record | null)?.board : esphome.board; + + const pins: EsphomePin[] = []; + const walk = (node: unknown, component: string, label: string) => { + if (Array.isArray(node)) { + for (const child of node) walk(child, component, label); + return; + } + if (!node || typeof node !== "object") return; + const entry = node as Record; + const what = [component + (entry.platform ? `/${String(entry.platform)}` : ""), entry.name ?? entry.id ?? label] + .filter(Boolean) + .map((p) => substitute(String(p), subs)); + for (const [key, value] of Object.entries(entry)) { + if (PIN_KEY.test(key)) { + const pin = pinName(value, subs); + if (pin) pins.push({ pin, use: key === "pin" ? what.join(": ") : `${what.join(": ")} (${key})` }); + } else if (typeof value === "object") { + walk(value, component, label); + } + } + }; + for (const [key, value] of Object.entries(doc)) { + if (SKIP.has(key) || PLATFORMS.includes(key)) continue; + walk(value, key, ""); + } + return { name, board: board ? String(board) : undefined, platform, pins }; +} + +export function esphomeToHadoc(nodes: EsphomeNode[]): Device[] { + return nodes.map((node) => { + const ports = new Map(); + for (const p of node.pins) ports.set(p.pin, ports.has(p.pin) ? `${ports.get(p.pin)}; ${p.use}` : p.use); + return { + id: node.name, + kind: "esp", + firmware: "ESPHome", + hostname: node.name, + ...(node.board ? { model: node.board } : {}), + ports: [...ports].map(([name, use]) => ({ name, signal: "GPIO", use })), + source: "esphome", + }; + }); +} diff --git a/server/src/importers/index.ts b/server/src/importers/index.ts index 0bd1c09..b5bdecb 100644 --- a/server/src/importers/index.ts +++ b/server/src/importers/index.ts @@ -1,7 +1,8 @@ -import { readFile } from "node:fs/promises"; +import { readFile, readdir } from "node:fs/promises"; import { existsSync } from "node:fs"; import path from "node:path"; import type { Device, Issue, Settings, Signal } from "../model.js"; +import { esphomeToHadoc, parseEsphomeNode, type EsphomeNode } from "./esphome.js"; import { parseWagoConfig, wagoToHadoc } from "./wago.js"; /** Runs the file imports configured in settings.imports. */ @@ -26,5 +27,24 @@ export async function runImports(dataDir: string, settings: Settings, issues: Is } } + const esphome = settings.imports?.esphome; + if (esphome) { + const dir = path.resolve(dataDir, esphome.dir); + if (!existsSync(dir)) { + issues.push({ severity: "error", file: "settings.yaml", message: `imports.esphome.dir: ${esphome.dir} does not exist` }); + } else { + const nodes: EsphomeNode[] = []; + for (const f of (await readdir(dir)).filter((f) => /\.ya?ml$/.test(f) && f !== "secrets.yaml").sort()) { + try { + const node = parseEsphomeNode(await readFile(path.join(dir, f), "utf8")); + if (node) nodes.push(node); + } catch (err) { + issues.push({ severity: "warning", file: `${esphome.dir}/${f}`, message: `Cannot read: ${(err as Error).message}` }); + } + } + devices.push(...esphomeToHadoc(nodes)); + } + } + return { devices, signals }; } diff --git a/server/src/live/index.ts b/server/src/live/index.ts new file mode 100644 index 0000000..adf715a --- /dev/null +++ b/server/src/live/index.ts @@ -0,0 +1,263 @@ +// Live data from the systems around the house, polled while the app runs and +// matched to the documented devices and signals. + +import type { Device, House, Integrations, Issue } from "../model.js"; +import { connectMqtt, type MqttNode } from "./mqtt.js"; +import { fetchOpenhab, type OpenhabData, type RuleUse } from "./openhab.js"; +import { fetchTasmota, type TasmotaStatus } from "./tasmota.js"; +import { fetchUnifi, inSubnet, type NetClient } from "./unifi.js"; + +export interface LiveDevice { + online?: boolean; + ip?: string; + mac?: string; + firmware?: string; + /** Which systems said so, e.g. ["unifi", "mqtt"]. */ + from: string[]; +} + +export interface LiveItem { + label?: string; + type: string; + state?: string; + rules: RuleUse[]; +} + +export interface SourceStatus { + ok: boolean; + error?: string; + at?: string; +} + +export interface LiveSnapshot { + sources: Record; + /** By HAdoc device ID. */ + devices: Record; + /** openHAB items that a signal refers to, by item name. */ + items: Record; +} + +const normMac = (mac: string | null | undefined) => mac?.toLowerCase().replace(/-/g, ":"); +const env = (name: string | undefined, fallback: string) => process.env[name ?? fallback] || undefined; + +export function liveService(house: () => House, options: { fetch?: typeof fetch; log?: (msg: string) => void } = {}) { + const fetchFn = options.fetch ?? fetch; + const log = options.log ?? (() => {}); + + let config: Integrations = {}; + let configKey = ""; + let timer: NodeJS.Timeout | undefined; + let mqtt: ReturnType | undefined; + + const sources: Record = {}; + let openhab: OpenhabData | undefined; + let clients: NetClient[] | undefined; + const tasmota = new Map(); + + const done = (name: string, error?: unknown) => { + sources[name] = { ok: !error, error: error ? (error as Error).message : undefined, at: new Date().toISOString() }; + if (error) log(`${name}: ${(error as Error).message}`); + }; + + async function poll() { + const tasks: Promise[] = []; + if (config.openhab) { + const cfg = config.openhab; + tasks.push( + fetchOpenhab(cfg.url, env(cfg.token_env, "HADOC_OPENHAB_TOKEN"), fetchFn).then( + (data) => ((openhab = data), done("openhab")), + (err) => done("openhab", err), + ), + ); + } + if (config.unifi) { + const cfg = config.unifi; + const key = env(cfg.key_env, "HADOC_UNIFI_KEY"); + if (!key) done("unifi", new Error(`Set ${cfg.key_env ?? "HADOC_UNIFI_KEY"} to an Integration API key`)); + else { + tasks.push( + fetchUnifi(cfg, key, fetchFn).then( + (data) => ((clients = data), done("unifi")), + (err) => done("unifi", err), + ), + ); + } + } + if (config.tasmota) { + const auth = { user: env(config.tasmota.user_env, "HADOC_TASMOTA_USER"), password: env(config.tasmota.password_env, "HADOC_TASMOTA_PASSWORD") }; + const hosts = house().devices.filter((d) => /tasmota/i.test(d.firmware ?? "") && d.hostname); + tasks.push( + Promise.all( + hosts.map((d) => + fetchTasmota(d.hostname!, auth, fetchFn).then( + (s) => void tasmota.set(d.id, s), + () => void tasmota.set(d.id, null), + ), + ), + ).then(() => { + const failed = hosts.filter((d) => tasmota.get(d.id) === null).length; + done("tasmota", failed === hosts.length && hosts.length > 0 ? new Error("No Tasmota device answered") : undefined); + }), + ); + } + await Promise.all(tasks); + } + + function stop() { + clearInterval(timer); + timer = undefined; + mqtt?.close(); + mqtt = undefined; + } + + /** (Re)starts polling when the integrations in settings change. */ + function configure(next: Integrations | undefined) { + const key = JSON.stringify(next ?? {}); + if (key === configKey) return; + stop(); + config = next ?? {}; + configKey = key; + for (const k of Object.keys(sources)) delete sources[k]; + openhab = undefined; + clients = undefined; + tasmota.clear(); + + if (config.mqtt) { + mqtt = connectMqtt( + config.mqtt.url, + { username: env(config.mqtt.username_env, "HADOC_MQTT_USER"), password: env(config.mqtt.password_env, "HADOC_MQTT_PASSWORD") }, + (msg) => log(`mqtt: ${msg}`), + ); + } + if (config.openhab || config.unifi || config.tasmota) { + void poll(); + timer = setInterval(() => void poll(), (config.poll_seconds ?? 60) * 1000); + timer.unref(); + } + } + + function mqttNodes(): MqttNode[] { + return mqtt ? [...mqtt.nodes.values()] : []; + } + + /** The live view of each documented device, from every source that knows it. */ + function devices(h: House): Record { + const out: Record = {}; + const add = (d: Device, from: string, patch: Omit) => { + const cur = out[d.id] ?? { from: [] }; + out[d.id] = { + ...cur, + ...Object.fromEntries(Object.entries(patch).filter(([, v]) => v !== undefined)), + // Any source that sees it online wins over one that does not. + online: cur.online || patch.online, + from: [...cur.from, from], + }; + }; + const byMac = new Map(clients?.map((c) => [c.mac, c])); + const nodes = mqttNodes(); + for (const d of h.devices) { + const names = new Set([d.id, d.hostname].filter(Boolean).map((n) => n!.toLowerCase())); + const mac = normMac(d.mac); + if (clients) { + const c = (mac && byMac.get(mac)) || clients.find((c) => c.name && names.has(c.name.toLowerCase())); + if (c) add(d, "unifi", { online: true, ip: c.ip, mac: c.mac }); + else if (mac) add(d, "unifi", { online: false }); + } + const t = tasmota.get(d.id); + if (t) add(d, "tasmota", { online: true, ip: t.ip, mac: t.mac, firmware: t.firmware }); + else if (t === null) add(d, "tasmota", { online: false }); + const n = nodes.find((n) => names.has(n.name.toLowerCase()) || (mac && n.mac === mac)); + if (n) add(d, "mqtt", { online: n.online, ip: n.ip, mac: n.mac }); + } + return out; + } + + function snapshot(): LiveSnapshot { + const h = house(); + const items: Record = {}; + if (openhab) { + for (const s of h.signals) { + const name = s.openhab?.item; + const item = name ? openhab.items.get(name) : undefined; + if (name && item) items[name] = { label: item.label, type: item.type, state: item.state, rules: openhab.uses.get(name) ?? [] }; + } + } + const status = { ...sources }; + if (mqtt) status.mqtt = mqtt.status(); + return { sources: status, devices: devices(h), items }; + } + + /** What the other systems know about that HAdoc does not, and documented devices that are offline. */ + function report(): Issue[] { + const h = house(); + const issues: Issue[] = []; + const info = (file: string, id: string, message: string) => issues.push({ severity: "info", file, id, message }); + + const live = devices(h); + for (const d of h.devices) { + if (live[d.id]?.online === false) { + issues.push({ severity: "warning", file: "live", id: d.id, message: `Offline according to ${live[d.id]!.from.join(", ")}` }); + } + } + + if (openhab) { + const used = new Set(h.signals.map((s) => s.openhab?.item).filter(Boolean)); + for (const item of openhab.items.values()) { + if (item.channels.length && !used.has(item.name)) { + info("openHAB", item.name, `${item.label ? `"${item.label}" ` : ""}is linked to ${item.channels[0]} but no signal uses it`); + } + } + for (const s of h.signals) { + if (s.openhab?.item && !openhab.items.has(s.openhab.item)) { + issues.push({ severity: "warning", file: "openHAB", id: s.id, message: `Item ${s.openhab.item} does not exist in openHAB` }); + } + } + } + + if (clients) { + const known = new Set(h.devices.map((d) => normMac(d.mac)).filter(Boolean)); + const names = new Set(h.devices.flatMap((d) => [d.id, d.hostname]).filter(Boolean).map((n) => n!.toLowerCase())); + const subnets = config.unifi?.subnets; + for (const c of clients) { + if (c.type === "DEVICE" || known.has(c.mac) || (c.name && names.has(c.name.toLowerCase()))) continue; + if (subnets?.length && !(c.ip && subnets.some((s) => inSubnet(c.ip!, s)))) continue; + info("UniFi", c.name ?? c.mac, `${[c.ip, c.mac].filter(Boolean).join(", ")} is on the network but not documented`); + } + } + + const names = new Set(h.devices.flatMap((d) => [d.id, d.hostname]).filter(Boolean).map((n) => n!.toLowerCase())); + const macs = new Set(h.devices.map((d) => normMac(d.mac)).filter(Boolean)); + for (const n of mqttNodes()) { + if (names.has(n.name.toLowerCase()) || (n.hostname && names.has(n.hostname.toLowerCase())) || (n.mac && macs.has(n.mac))) continue; + info("MQTT", n.name, `${n.firmware} node${n.ip ? ` at ${n.ip}` : ""} announces itself but is not documented`); + } + + return issues; + } + + return { + configure, + poll, + stop, + snapshot, + report, + /** Script source of an openHAB rule, for showing it next to the item. */ + rule: (uid: string) => openhab?.rules.get(uid), + }; +} + +export type LiveService = ReturnType; + +/** ESPHome pins that no signal documents yet. Static, so it works without any live connection. */ +export function undocumentedPins(h: House): Issue[] { + const issues: Issue[] = []; + for (const d of h.devices) { + if (d.source !== "esphome") continue; + for (const p of d.ports ?? []) { + const ref = `${d.id}:${p.name}`; + if (h.signals.some((s) => s.port === ref) || h.cables.some((c) => c.from === ref || c.to === ref)) continue; + issues.push({ severity: "info", file: "ESPHome", id: ref, message: `${p.use ?? "Pin"} is not documented as a signal or cable` }); + } + } + return issues; +} diff --git a/server/src/live/mqtt.ts b/server/src/live/mqtt.ts new file mode 100644 index 0000000..385b619 --- /dev/null +++ b/server/src/live/mqtt.ts @@ -0,0 +1,66 @@ +// Listens on the MQTT broker for the nodes that announce themselves: +// Tasmota (LWT and discovery) and ESPHome (availability topic). + +import mqtt from "mqtt"; + +export interface MqttNode { + /** Tasmota topic or ESPHome node name. */ + name: string; + firmware: "Tasmota" | "ESPHome"; + online?: boolean; + ip?: string; + mac?: string; + hostname?: string; + seen: string; +} + +export const MQTT_TOPICS = ["tele/+/LWT", "+/status", "tasmota/discovery/+/config"]; + +/** Updates the node list for one message. Pure, so it can be tested without a broker. */ +export function observe(nodes: Map, topic: string, payload: string, now = new Date()) { + const parts = topic.split("/"); + const upsert = (name: string, firmware: MqttNode["firmware"], patch: Partial) => { + const node = nodes.get(name) ?? { name, firmware, seen: now.toISOString() }; + nodes.set(name, { ...node, ...patch, seen: now.toISOString() }); + }; + + if (parts.length === 3 && parts[0] === "tele" && parts[2] === "LWT") { + upsert(parts[1]!, "Tasmota", { online: payload.trim() === "Online" }); + } else if (parts.length === 4 && parts[0] === "tasmota" && parts[1] === "discovery") { + try { + const d = JSON.parse(payload) as { t?: string; ip?: string; mac?: string; hn?: string }; + if (d.t) { + const mac = d.mac?.toLowerCase().replace(/(..)(?!$)/g, "$1:"); + upsert(d.t, "Tasmota", { ip: d.ip, mac, hostname: d.hn }); + } + } catch { + // Not a discovery message we understand. + } + } else if (parts.length === 2 && parts[1] === "status" && /^(online|offline)$/.test(payload.trim())) { + upsert(parts[0]!, "ESPHome", { online: payload.trim() === "online" }); + } +} + +export function connectMqtt( + url: string, + auth: { username?: string; password?: string }, + onError: (message: string) => void, +) { + const nodes = new Map(); + let lastError: string | undefined; + const client = mqtt.connect(url, { ...auth, clientId: `hadoc-${Math.random().toString(16).slice(2, 8)}`, reconnectPeriod: 30000 }); + client.on("connect", () => { + lastError = undefined; + client.subscribe(MQTT_TOPICS); + }); + client.on("error", (err) => { + lastError = err.message; + onError(err.message); + }); + client.on("message", (topic, payload) => observe(nodes, topic, payload.toString())); + return { + nodes, + status: () => ({ ok: client.connected, error: client.connected ? undefined : (lastError ?? "Not connected") }), + close: () => client.end(true), + }; +} diff --git a/server/src/live/openhab.ts b/server/src/live/openhab.ts new file mode 100644 index 0000000..3d66c62 --- /dev/null +++ b/server/src/live/openhab.ts @@ -0,0 +1,94 @@ +// openHAB REST API: items, their channel links, and the rules that use each item. + +export interface OhItem { + name: string; + label?: string; + type: string; + state?: string; + groupNames: string[]; + /** Channels the item is linked to; empty for virtual items. */ + channels: string[]; +} + +export interface OhRule { + uid: string; + name: string; + /** Script source for rules that have one (DSL, JS, Blockly's generated code). */ + script?: string; +} + +export interface RuleUse { + uid: string; + name: string; + /** Why the rule counts: it triggers on the item, on a group it is in, or names it in its script. */ + how: "trigger" | "group trigger" | "script"; +} + +export interface OpenhabData { + items: Map; + rules: Map; + uses: Map; +} + +interface RestRule { + uid: string; + name?: string; + triggers?: { configuration?: { itemName?: string; groupName?: string } }[]; + actions?: { configuration?: { script?: string } }[]; +} + +export async function fetchOpenhab(url: string, token: string | undefined, fetchFn: typeof fetch): Promise { + const base = url.replace(/\/$/, ""); + const get = async (p: string): Promise => { + const res = await fetchFn(`${base}${p}`, { + headers: { accept: "application/json", ...(token ? { authorization: `Bearer ${token}` } : {}) }, + signal: AbortSignal.timeout(15000), + }); + if (!res.ok) throw new Error(`${p}: ${res.status} ${res.statusText}`); + return (await res.json()) as T; + }; + + const [rawItems, links, rawRules] = await Promise.all([ + get<{ name: string; label?: string; type: string; state?: string; groupNames?: string[] }[]>( + "/rest/items?recursive=false&fields=name,label,type,state,groupNames", + ), + get<{ itemName: string; channelUID: string }[]>("/rest/links"), + get("/rest/rules"), + ]); + + const items = new Map(); + for (const i of rawItems) { + items.set(i.name, { name: i.name, label: i.label, type: i.type, state: i.state, groupNames: i.groupNames ?? [], channels: [] }); + } + for (const l of links) items.get(l.itemName)?.channels.push(l.channelUID); + + const rules = new Map(); + const uses = new Map(); + const use = (item: string, u: RuleUse) => { + const list = uses.get(item) ?? []; + if (!list.some((x) => x.uid === u.uid)) list.push(u); + uses.set(item, list); + }; + + for (const r of rawRules) { + const name = r.name ?? r.uid; + const script = (r.actions ?? []).map((a) => a.configuration?.script).filter(Boolean).join("\n\n") || undefined; + rules.set(r.uid, { uid: r.uid, name, script }); + + for (const t of r.triggers ?? []) { + const { itemName, groupName } = t.configuration ?? {}; + if (itemName) use(itemName, { uid: r.uid, name, how: "trigger" }); + if (groupName) { + for (const i of items.values()) if (i.groupNames.includes(groupName)) use(i.name, { uid: r.uid, name, how: "group trigger" }); + } + } + if (script) { + // Item names are identifiers, so a whole-word match is a good enough "this rule uses it". + for (const word of new Set(script.match(/[A-Za-z_][A-Za-z0-9_]*/g) ?? [])) { + if (items.has(word)) use(word, { uid: r.uid, name, how: "script" }); + } + } + } + + return { items, rules, uses }; +} diff --git a/server/src/live/tasmota.ts b/server/src/live/tasmota.ts new file mode 100644 index 0000000..2aee936 --- /dev/null +++ b/server/src/live/tasmota.ts @@ -0,0 +1,35 @@ +// Tasmota's HTTP command API: "Status 0" returns name, firmware, IP and MAC in one call. + +export interface TasmotaStatus { + ip?: string; + mac?: string; + firmware?: string; + name?: string; + topic?: string; +} + +export async function fetchTasmota( + host: string, + auth: { user?: string; password?: string }, + fetchFn: typeof fetch, +): Promise { + const q = new URLSearchParams({ cmnd: "Status 0" }); + if (auth.password) { + q.set("user", auth.user ?? "admin"); + q.set("password", auth.password); + } + const res = await fetchFn(`http://${host}/cm?${q}`, { signal: AbortSignal.timeout(5000) }); + if (!res.ok) throw new Error(`${res.status} ${res.statusText}`); + const s = (await res.json()) as { + Status?: { DeviceName?: string; Topic?: string }; + StatusFWR?: { Version?: string }; + StatusNET?: { IPAddress?: string; Mac?: string }; + }; + return { + ip: s.StatusNET?.IPAddress, + mac: s.StatusNET?.Mac?.toLowerCase(), + firmware: s.StatusFWR?.Version ? `Tasmota ${s.StatusFWR.Version}` : undefined, + name: s.Status?.DeviceName, + topic: s.Status?.Topic, + }; +} diff --git a/server/src/live/unifi.ts b/server/src/live/unifi.ts new file mode 100644 index 0000000..7659394 --- /dev/null +++ b/server/src/live/unifi.ts @@ -0,0 +1,69 @@ +// UniFi Network Integration API (Network application 9.x and later). + +export interface NetClient { + mac: string; + ip?: string; + name?: string; + /** "WIRED", "WIRELESS", or "DEVICE" for UniFi's own gear. */ + type?: string; + connectedAt?: string; +} + +interface Page { + offset: number; + count: number; + totalCount: number; + data: T[]; +} + +export async function fetchUnifi( + cfg: { url: string; api_path?: string; site?: string }, + key: string, + fetchFn: typeof fetch, +): Promise { + const base = `${cfg.url.replace(/\/$/, "")}${cfg.api_path ?? "/integration/v1"}`; + const get = async (p: string): Promise => { + const res = await fetchFn(`${base}${p}`, { + headers: { accept: "application/json", "x-api-key": key }, + signal: AbortSignal.timeout(15000), + }); + if (!res.ok) throw new Error(`${p}: ${res.status} ${res.statusText}`); + return (await res.json()) as T; + }; + const all = async (p: string): Promise => { + const out: T[] = []; + for (let offset = 0; ; ) { + const page = await get>(`${p}?offset=${offset}&limit=200`); + out.push(...page.data); + offset += page.count; + if (page.count === 0 || offset >= page.totalCount) return out; + } + }; + + const wanted = cfg.site ?? "default"; + const sites = await all<{ id: string; internalReference?: string; name?: string }>("/sites"); + const site = sites.find((s) => s.id === wanted || s.internalReference === wanted || s.name === wanted); + if (!site) throw new Error(`Site "${wanted}" not found; there are ${sites.map((s) => s.internalReference ?? s.name).join(", ")}`); + + const [clients, devices] = await Promise.all([ + all<{ type?: string; name?: string; ipAddress?: string; macAddress?: string; connectedAt?: string }>(`/sites/${site.id}/clients`), + all<{ name?: string; ipAddress?: string; macAddress?: string; state?: string }>(`/sites/${site.id}/devices`), + ]); + + return [ + ...clients + .filter((c) => c.macAddress) + .map((c) => ({ mac: c.macAddress!.toLowerCase(), ip: c.ipAddress, name: c.name, type: c.type, connectedAt: c.connectedAt })), + ...devices + .filter((d) => d.macAddress && d.state === "ONLINE") + .map((d) => ({ mac: d.macAddress!.toLowerCase(), ip: d.ipAddress, name: d.name, type: "DEVICE" })), + ]; +} + +/** True when an IPv4 address is inside a CIDR subnet like "192.168.20.0/24". */ +export function inSubnet(ip: string, cidr: string): boolean { + const [net, bits] = cidr.split("/"); + const toInt = (a: string) => a.split(".").reduce((n, o) => (n << 8) + Number(o), 0) >>> 0; + const mask = Number(bits) === 0 ? 0 : (~0 << (32 - Number(bits))) >>> 0; + return (toInt(ip) & mask) === (toInt(net!) & mask); +} diff --git a/server/src/model.ts b/server/src/model.ts index 50496c6..419ad57 100644 --- a/server/src/model.ts +++ b/server/src/model.ts @@ -11,7 +11,17 @@ export interface Settings { types: { code: string; name: string; symbol?: string; layer?: Layer }[]; imports?: { wago?: { config: string; controller?: string; id_prefix?: string }; + esphome?: { dir: string }; }; + integrations?: Integrations; +} + +export interface Integrations { + poll_seconds?: number; + openhab?: { url: string; token_env?: string }; + unifi?: { url: string; api_path?: string; site?: string; key_env?: string; subnets?: string[] }; + tasmota?: { user_env?: string; password_env?: string }; + mqtt?: { url: string; username_env?: string; password_env?: string }; } export type Layer = "power" | "data" | "automation" | "hidden"; @@ -62,6 +72,8 @@ export interface Enclosure { export interface Port { name: string; signal?: string; + /** What the firmware config uses it for; set by imports such as ESPHome. */ + use?: string; } export interface Device { @@ -116,7 +128,8 @@ export interface Signal { } export interface Issue { - severity: "error" | "warning"; + /** "info" is for what is found elsewhere but not documented here yet. */ + severity: "error" | "warning" | "info"; file: string; id?: string; message: string; diff --git a/server/src/trace.ts b/server/src/trace.ts index dd431dd..81b7328 100644 --- a/server/src/trace.ts +++ b/server/src/trace.ts @@ -19,6 +19,7 @@ export interface CableEnd { export interface PortUse { name: string; signal?: string; + use?: string; /** Signals wired to this port, e.g. the PLC variable reading it. */ signals: Signal[]; /** Cables landing on this port, with their other end. */ @@ -129,6 +130,7 @@ export function traceThing(house: House, index: HouseIndex, id: string): Trace | return { name: p.name, signal: p.signal, + use: p.use, signals: house.signals.filter((s) => s.port === ref), cables: cables.filter((c) => c.here === ref || c.cable.cores?.some((k) => k.from === ref || k.to === ref)), }; diff --git a/server/test/live.test.ts b/server/test/live.test.ts new file mode 100644 index 0000000..fcf9468 --- /dev/null +++ b/server/test/live.test.ts @@ -0,0 +1,171 @@ +import { cp, mkdtemp, readFile, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { describe, expect, it } from "vitest"; +import { buildApp } from "../src/app.js"; +import { esphomeToHadoc, parseEsphomeNode } from "../src/importers/esphome.js"; +import { observe, type MqttNode } from "../src/live/mqtt.js"; +import { inSubnet } from "../src/live/unifi.js"; + +const DEMO = fileURLToPath(new URL("../../examples/demo-house/", import.meta.url)); + +// What the systems on the LAN answer, as recorded from their APIs. +const FIXTURES: Record = { + "http://openhab.test/rest/items?recursive=false&fields=name,label,type,state,groupNames": [ + { name: "Dressing_Motion", label: "Dressing motion", type: "Switch", state: "OFF", groupNames: ["gMotion"] }, + { name: "Hall_Light", label: "Hall light", type: "Switch", state: "ON", groupNames: [] }, + { name: "Away_Mode", label: "Away", type: "Switch", state: "OFF", groupNames: [] }, + ], + "http://openhab.test/rest/links": [ + { itemName: "Dressing_Motion", channelUID: "mqtt:topic:broker:dressing:motion" }, + { itemName: "Hall_Light", channelUID: "modbus:data:plc:hall" }, + ], + "http://openhab.test/rest/rules": [ + { uid: "r1", name: "Dressing light on motion", triggers: [{ configuration: { itemName: "Dressing_Motion" } }], actions: [] }, + { + uid: "r2", + name: "Night mode", + triggers: [{ configuration: { itemName: "Away_Mode" } }], + actions: [{ configuration: { script: 'if (Dressing_Motion.state == ON) { logInfo("x", "y") }' } }], + }, + { uid: "r3", name: "Any motion", triggers: [{ configuration: { groupName: "gMotion" } }], actions: [] }, + ], + "https://unifi.test/integration/v1/sites?offset=0&limit=200": { + offset: 0, count: 1, totalCount: 1, data: [{ id: "s1", internalReference: "default", name: "Default" }], + }, + "https://unifi.test/integration/v1/sites/s1/clients?offset=0&limit=200": { + offset: 0, count: 3, totalCount: 3, + data: [ + { type: "WIRELESS", name: "esp-dressing", ipAddress: "192.168.20.41", macAddress: "A4:CF:12:00:00:01" }, + { type: "WIRELESS", name: "shelly1-ab12", ipAddress: "192.168.20.55", macAddress: "C4:5B:BE:00:00:02" }, + { type: "WIRELESS", name: "phone", ipAddress: "192.168.1.10", macAddress: "11:22:33:44:55:66" }, + ], + }, + "https://unifi.test/integration/v1/sites/s1/devices?offset=0&limit=200": { offset: 0, count: 0, totalCount: 0, data: [] }, + "http://sonoff-mirror/cm?cmnd=Status+0": { + Status: { DeviceName: "Mirror", Topic: "sonoff-mirror" }, + StatusFWR: { Version: "14.2.0(release-tasmota)" }, + StatusNET: { IPAddress: "192.168.20.30", Mac: "DC:4F:22:00:00:03" }, + }, +}; + +const fakeFetch = (async (url: string | URL, init?: RequestInit) => { + const key = String(url); + if (key.startsWith("https://unifi.test") && (init?.headers as Record)["x-api-key"] !== "secret") { + return new Response("{}", { status: 401, statusText: "Unauthorized" }); + } + if (!(key in FIXTURES)) return new Response("{}", { status: 404, statusText: `No fixture for ${key}` }); + return new Response(JSON.stringify(FIXTURES[key]), { headers: { "content-type": "application/json" } }); +}) as typeof fetch; + +async function liveDemo() { + const dir = await mkdtemp(path.join(tmpdir(), "hadoc-live-")); + await cp(DEMO, dir, { recursive: true }); + const settings = await readFile(path.join(dir, "settings.yaml"), "utf8"); + await writeFile( + path.join(dir, "settings.yaml"), + `${settings} +integrations: + openhab: { url: http://openhab.test } + unifi: { url: https://unifi.test, subnets: [192.168.20.0/24] } + tasmota: {} +`, + ); + process.env.HADOC_UNIFI_KEY = "secret"; + const app = await buildApp({ dataDir: dir, fetch: fakeFetch }); + // The first poll starts on load; wait for it. + for (let i = 0; i < 50; i++) { + const live = (await app.inject({ url: "/api/live" })).json(); + if (live.sources.openhab && live.sources.unifi && live.sources.tasmota) break; + await new Promise((r) => setTimeout(r, 20)); + } + return app; +} + +describe("live integrations", () => { + it("matches network, Tasmota and openHAB data to the documented house", async () => { + const app = await liveDemo(); + const live = (await app.inject({ url: "/api/live" })).json(); + expect(live.sources).toMatchObject({ openhab: { ok: true }, unifi: { ok: true }, tasmota: { ok: true } }); + expect(live.devices["esp-dressing"]).toMatchObject({ online: true, ip: "192.168.20.41", from: ["unifi"] }); + expect(live.devices["sonoff-mirror"]).toMatchObject({ online: true, ip: "192.168.20.30", firmware: "Tasmota 14.2.0(release-tasmota)" }); + expect(live.items.Dressing_Motion.state).toBe("OFF"); + expect(live.items.Dressing_Motion.rules).toEqual([ + { uid: "r1", name: "Dressing light on motion", how: "trigger" }, + { uid: "r2", name: "Night mode", how: "script" }, + { uid: "r3", name: "Any motion", how: "group trigger" }, + ]); + const rule = (await app.inject({ url: "/api/live/rules/r2" })).json(); + expect(rule.script).toContain("Dressing_Motion.state"); + await app.close(); + }); + + it("reports what the other systems know and HAdoc does not", async () => { + const app = await liveDemo(); + const issues = (await app.inject({ url: "/api/issues" })).json() as { severity: string; file: string; id: string; message: string }[]; + const info = issues.filter((i) => i.severity === "info").map((i) => `${i.file} ${i.id}: ${i.message}`); + expect(info).toContain('openHAB Hall_Light: "Hall light" is linked to modbus:data:plc:hall but no signal uses it'); + expect(info).toContain("UniFi shelly1-ab12: 192.168.20.55, c4:5b:be:00:00:02 is on the network but not documented"); + // Outside the configured subnet, so not reported. + expect(info.some((i) => i.includes("phone"))).toBe(false); + // Virtual items have no channel and are not expected to be documented. + expect(info.some((i) => i.includes("Away_Mode"))).toBe(false); + expect(info).toContain("ESPHome esp-dressing:GPIO12: binary_sensor/gpio: Door contact is not documented as a signal or cable"); + const house = (await app.inject({ url: "/api/house" })).json(); + expect(house.issueCount.info).toBe(info.length); + await app.close(); + }); + + it("says why a source failed", async () => { + process.env.HADOC_UNIFI_KEY = "wrong"; + const dir = await mkdtemp(path.join(tmpdir(), "hadoc-live-")); + await cp(DEMO, dir, { recursive: true }); + const settings = await readFile(path.join(dir, "settings.yaml"), "utf8"); + await writeFile(path.join(dir, "settings.yaml"), `${settings}\nintegrations:\n unifi: { url: https://unifi.test }\n`); + const app = await buildApp({ dataDir: dir, fetch: fakeFetch }); + await new Promise((r) => setTimeout(r, 50)); + const live = (await app.inject({ url: "/api/live" })).json(); + expect(live.sources.unifi).toMatchObject({ ok: false, error: "/sites?offset=0&limit=200: 401 Unauthorized" }); + await app.close(); + }); +}); + +describe("mqtt", () => { + it("tracks Tasmota and ESPHome nodes from their announcements", () => { + const nodes = new Map(); + observe(nodes, "tele/sonoff-mirror/LWT", "Online"); + observe(nodes, "tasmota/discovery/DC4F22000003/config", JSON.stringify({ t: "sonoff-mirror", ip: "192.168.20.30", mac: "DC4F22000003", hn: "sonoff-mirror-0003" })); + observe(nodes, "esp-garage/status", "offline"); + observe(nodes, "zigbee2mqtt/status", '{"state":"online"}'); + expect([...nodes.values()].map(({ seen: _, ...n }) => n)).toEqual([ + { name: "sonoff-mirror", firmware: "Tasmota", online: true, ip: "192.168.20.30", mac: "dc:4f:22:00:00:03", hostname: "sonoff-mirror-0003" }, + { name: "esp-garage", firmware: "ESPHome", online: false }, + ]); + }); + + it("checks subnets", () => { + expect(inSubnet("192.168.20.55", "192.168.20.0/24")).toBe(true); + expect(inSubnet("192.168.21.1", "192.168.20.0/24")).toBe(false); + expect(inSubnet("10.1.2.3", "0.0.0.0/0")).toBe(true); + }); +}); + +describe("esphome import", () => { + it("reads every pin with what uses it, also through substitutions and pin objects", async () => { + const text = await readFile(path.join(DEMO, "imports/esphome/esp-dressing.yaml"), "utf8"); + const node = parseEsphomeNode(text)!; + expect(node).toMatchObject({ name: "esp-dressing", board: "nodemcuv2", platform: "esp8266" }); + expect(node.pins).toEqual([ + { pin: "GPIO14", use: "binary_sensor/gpio: Motion" }, + { pin: "GPIO12", use: "binary_sensor/gpio: Door contact" }, + { pin: "GPIO4", use: "i2c (sda)" }, + { pin: "GPIO5", use: "i2c (scl)" }, + ]); + expect(esphomeToHadoc([node])[0]).toMatchObject({ id: "esp-dressing", kind: "esp", source: "esphome" }); + }); + + it("skips files that are not a node", () => { + expect(parseEsphomeNode("wifi_ssid: x\n")).toBeNull(); + }); +}); diff --git a/web/src/App.svelte b/web/src/App.svelte index fb73ec6..3319aaf 100644 --- a/web/src/App.svelte +++ b/web/src/App.svelte @@ -1,12 +1,13 @@ @@ -312,6 +335,7 @@
select(null)} /> {:else if showIssues} - (showIssues = false)} /> + (showIssues = false)} /> {/if} diff --git a/web/src/lib/Details.svelte b/web/src/lib/Details.svelte index 1f5431e..c74d2a7 100644 --- a/web/src/lib/Details.svelte +++ b/web/src/lib/Details.svelte @@ -1,11 +1,12 @@ +{#snippet status(d: LiveDevice | undefined)} + {#if d && d.online !== undefined} + + {d.online ? "online" : "offline"}{#if d.ip}{d.ip}{/if} + + {/if} +{/snippet} + +{#snippet item(s: Signal)} + {@const name = s.openhab?.item} + {@const it = name ? live?.items[name] : undefined} + {#if name} +
openHAB
+
+ {name} + {#if it?.state !== undefined}{it.state}{/if} + {#if it?.rules.length} + {#each it.rules as r (r.uid)} +
(e.currentTarget as HTMLDetailsElement).open && loadScript(r.uid)}> + {r.name} {r.how === "script" ? "uses it" : `on ${r.how}`} + 
{scripts[r.uid] ?? "Loading…"}
+
+ {/each} + {:else if it} +
No rules use it
+ {/if} +
+ {/if} +{/snippet} + {#snippet link(ref: ThingRef | null)} {#if ref === null} not documented {:else if canOpen(ref)} {#if ref.port}{ref.port}{/if} + >{/if}{#if ref.kind === "device"}{@render status(live?.devices[ref.id])}{/if} {:else} {ref.label} {/if} @@ -92,12 +134,14 @@ {/if} {#if device} + {@const ld = live?.devices[device.id]} + {#if ld}

{@render status(ld)}

{/if}
Kind
{device.kind}
{#if device.model}
Model
{device.model}
{/if} - {#if device.firmware}
Firmware
{device.firmware}
{/if} + {#if ld?.firmware ?? device.firmware}
Firmware
{ld?.firmware ?? device.firmware}
{/if} {#if device.hostname}
Hostname
{device.hostname}
{/if} - {#if device.mac}
MAC
{device.mac}
{/if} + {#if device.mac ?? ld?.mac}
MAC
{device.mac ?? ld?.mac}
{/if}
{#if device.notes}

{device.notes}

{/if} {/if} @@ -128,6 +172,9 @@
{#if s.plc?.name}{s.plc.name}{/if} {#if s.name}{s.name}{/if} + {#if s.openhab?.item}{s.openhab.item}{#if live?.items[s.openhab.item]?.state !== undefined}{live.items[s.openhab.item]!.state}{/if}{/if} {#if s.point}→ {#if editing && actions}{/if} {/if} @@ -139,7 +186,7 @@ {#each port.cables as end (end.cable.id)}
{end.cable.id} to {@render link(end.there)}
{/each} - {#if !port.signals.length && !port.cables.length}spare{/if} + {#if !port.signals.length && !port.cables.length}{port.use ? `${port.use} (not documented)` : "spare"}{/if} {/each} @@ -229,7 +276,7 @@
{#if s.plc?.name}
PLC
{[s.plc.name, s.plc.address].filter(Boolean).join(" at ")}
{/if} {#if s.mqtt}
MQTT
{s.mqtt}
{/if} - {#if s.openhab?.item}
openHAB
{s.openhab.item}
{/if} + {@render item(s)}
{/each} @@ -393,6 +440,42 @@ margin-right: 6px; vertical-align: -1px; } + .status { + font-size: 13px; + color: var(--muted); + margin-left: 6px; + white-space: nowrap; + } + p > .status:first-child { + margin-left: 0; + } + .status .mono { + margin-left: 6px; + } + .led { + display: inline-block; + width: 8px; + height: 8px; + border-radius: 50%; + background: var(--error); + margin-right: 4px; + } + .status.online .led { + background: #16a34a; + } + .rule summary { + cursor: pointer; + font-size: 14px; + } + .rule pre { + font-size: 12px; + background: var(--surface); + border-radius: 6px; + padding: 6px 8px; + overflow-x: auto; + margin: 4px 0; + max-height: 300px; + } .tag { font-size: 12px; border: 1px solid var(--border); diff --git a/web/src/lib/Issues.svelte b/web/src/lib/Issues.svelte index ec7ce7b..f6c448f 100644 --- a/web/src/lib/Issues.svelte +++ b/web/src/lib/Issues.svelte @@ -3,22 +3,24 @@ interface Props { issues: Issue[]; + /** Whether an issue's ID is something the app can open. */ + canOpen: (id: string) => boolean; onselect: (id: string) => void; onclose: () => void; } - let { issues, onselect, onclose }: Props = $props(); + let { issues, canOpen, onselect, onclose }: Props = $props(); - // Errors first, then one group per file or import, largest first. + const RANK = { error: 0, warning: 1, info: 2 }; + + // Errors, then warnings, then what is undocumented; one group per file or source, largest first. const groups = $derived.by(() => { const map = new Map(); for (const issue of issues) { const key = `${issue.severity}\u0000${issue.file}`; map.set(key, [...(map.get(key) ?? []), issue]); } - return [...map.values()].sort( - (a, b) => (a[0]!.severity === b[0]!.severity ? b.length - a.length : a[0]!.severity === "error" ? -1 : 1), - ); + return [...map.values()].sort((a, b) => RANK[a[0]!.severity] - RANK[b[0]!.severity] || b.length - a.length); }); @@ -30,13 +32,17 @@ {#if groups.length === 0}

Everything is documented and consistent.

{:else} - {#each groups as group (`${group[0]!.severity}${group[0]!.file}`)} + {#each groups as group, g (`${group[0]!.severity}${group[0]!.file}`)} + {#if group[0]!.severity === "info" && groups[g - 1]?.[0]!.severity !== "info"} +

Found elsewhere, not documented yet

+ {/if}
{group[0]!.file} {group.length}
    {#each group as issue, i (i)}
  • - {#if issue.id}{/if} + {#if issue.id && canOpen(issue.id)} + {:else if issue.id}{issue.id}{/if} {issue.message}
    • {/each} @@ -75,6 +81,20 @@ details.error { border-color: var(--error); } + details.info { + border-color: var(--border); + } + h3 { + margin: 20px 0 4px; + font-size: 12px; + text-transform: uppercase; + letter-spacing: 0.04em; + color: var(--muted); + } + .mono { + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + font-size: 13px; + } summary { cursor: pointer; } diff --git a/web/src/lib/api.ts b/web/src/lib/api.ts index 0e1e8c0..dd43a89 100644 --- a/web/src/lib/api.ts +++ b/web/src/lib/api.ts @@ -1,11 +1,14 @@ import type { House, Issue, XY } from "../../../server/src/model.ts"; import type { Trace } from "../../../server/src/trace.ts"; +import type { LiveSnapshot } from "../../../server/src/live/index.ts"; +import type { OhRule } from "../../../server/src/live/openhab.ts"; export type { Cable, Circuit, Device, Floor, House, Issue, Point, Settings, Signal, XY } from "../../../server/src/model.ts"; export type { Trace, ThingRef } from "../../../server/src/trace.ts"; +export type { LiveDevice, LiveItem, LiveSnapshot } from "../../../server/src/live/index.ts"; export type HouseData = Omit & { - issueCount: { errors: number; warnings: number }; + issueCount: { errors: number; warnings: number; info: number }; /** Point or device ID to the circuit that feeds it. */ circuitOf: Record; /** False when the server refuses edits. */ @@ -48,6 +51,8 @@ type Saved = { id: string; file: string; committed: boolean }; export const api = { house: () => getJson("/api/house"), issues: () => getJson("/api/issues"), + live: () => getJson("/api/live"), + rule: (uid: string) => getJson(`/api/live/rules/${encodeURIComponent(uid)}`), trace: (id: string) => getJson(`/api/trace/${encodeURIComponent(id)}`), movePoint: (id: string, pos: XY | null) => send("PATCH", `/api/points/${encodeURIComponent(id)}`, { pos }),