feat(data): add passive versioned raw capture independent of control

2026-10-02 14:55:43 +00:00
parent e0e52d2454
commit 76f8b067cc
7 changed files with 603 additions and 0 deletions
@@ -0,0 +1,67 @@
+# Passive V4 data capture - user inventory and raw acquisition
+
+Status 2026-10-02: source implementation, CLI/mocked tests and staged user installation.
+Not part of the live manager, not a control or training release.
+
+## Lihrenmoos user-reported inventory (not applied to running configuration)
+
+- GoodWe 1/2: each 50 kW AC, each 156 kWh reported physical battery capacity.
+- SolarEdge: 10 kW AC, 10 kWh reported physical battery capacity.
+- EV allocation: 30 kW / 160 kWh total, understood as 10 kW EV per inverter.
+- PV roof: approximately 20 kWp east and 20 kWp west. Tilt/individual strings unknown.
+- Nominal battery sum: 322 kWh. 162 kWh nominal difference to EV allocation is NOT a
+  verified usable SDL capacity. AC ratings are not verified battery charge/discharge ratings.
+- Exact types, firmware, common AC meter boundary and device watchdog remain unverified.
+- Current saved manager topology still reports 161.44 kWh / 39 kW. Gateway currently
+  lists 130/130/8 kWh and 57/57/5 kW. These may use a different usable/nominal basis.
+  Do not rebase virtual SOC or change current controls based on these notes alone.
+
+## Why collect before boundary approval
+
+Record the actual original numerical channels now, with source metadata and timestamps.
+The algebraic base-load candidate is useful for diagnosis, but NEVER a certified base-load
+training sample until the physical AC/DC boundary and historical recipe are verified.
+No dependence on hardware-watchdog approval for passive recording.
+
+21 channels: PCC power, legacy displayed PCC power, PV for three inverters, three physical
+battery powers, EV/SDL virtual powers and SOCs, three physical SOCs, dynamic EV limits,
+T1/T2 import counters, EV/SDL requested setpoints (READ ONLY).
+A cyclic script reads values already present in Symcon every 30 seconds. It does not issue
+additional Modbus polls. Original VariableUpdated/VariableChanged are retained; these are
+Symcon timestamps, not independent device-measurement timestamps. Two-pass reads detect
+concurrent changes; missing/non-numeric/re-used channels remain explicit quality failures.
+
+## Storage and interpretation
+
+Daily append-only raw-YYYYMMDD.jsonl in the separate capture stage data directory, UTC
+capture times plus original source timestamps, immutable mapping/inventory snapshots by
+SHA256. Raw errors and quality_hold are recorded too; nothing is invented as zero.
+The raw observations are a source for later documented 5-minute aggregation, not themselves
+aligned 5-minute means. No interpolation, training publication or future SDL assumption.
+
+A 512 MiB journal quota, 64 MiB daily limit, 256 KiB record cap and 100 MiB free-space
+reserve bound recording. Limits fail visibly without deleting old data. File permissions
+0640; no upload credentials, request bodies, complete instance configurations or secrets.
+A separate status variable and STATUS.json show latest capture and failures. Process restart
+retains files; an incomplete final JSONL row is preserved for manual review, not extended.
+
+## Install / stop
+
+Staged user entry point: /srv/agent/netplan-v4-data-capture-stage/install.php
+Execute in the Symcon PHP editor (not a Linux shell):
+
+    require '/srv/agent/netplan-v4-data-capture-stage/install.php';
+
+Installer checks exact source hashes and numeric source identities before creating a NEW
+root category ENELIX_V4_PASSIVE_CAPTURE, a status string, collector script and stop script.
+IPS_SetScriptTimer enables only this new script at 30 seconds. Repeating installation is
+idempotent. Unrelated/edited scripts or object identity collisions are not overwritten.
+
+Stop using the newly created script 'Nur diese Messaufnahme stoppen'; it disables only
+this collection timer, retains collected files, and does not stop the manager or battery.
+No module reload, existing archive change, existing timer/poller change, service restart,
+forecast selection, battery reservation or actuator command is included.
+
+Current tests: 36 core/file tests + 19 mocked native lifecycle tests with PHP CLI on the
+test host. First real capture and future timer execution require the user activation.
+These tests do not certify hardware mappings or a Symcon-kernel run.