Durable Telegram Inbox Records

Thesis

Inbound Telegram updates are not operationally useful until they become durable inbox records. The durable-inbox work completes the first inbound path by writing raw payloads, metadata, and readable summaries to disk, then advancing the listener offset only after persistence succeeds.

This is the difference between a polling loop and a channel server. A polling loop sees updates. A channel server leaves records that another process, agent, or operator can inspect later.

The Inbox Shape

Each inbound update lands under:

<org-root>/.matic/channels/telegram/inbox/<update-id>/

The record contains three files:

raw.json
metadata.yaml
summary.md

raw.json is the Telegram payload exactly as Matic received it. metadata.yaml is the structured operational view: update id, kind, summary, chat information, received timestamp, and next update offset. summary.md is a short human summary for inspection.

The folder is named by Telegram update_id. That makes it stable across process restarts and easy to compare with listener state.

Why Three Files

The three-file shape is not redundant. Each file serves a different reader.

The raw JSON is for truth. Telegram payloads vary, and the normalizer will never understand every future shape on the first pass. Keeping the raw payload means the system can improve later without losing evidence.

The metadata YAML is for machine-oriented Matic state. It gives tests, diagnostics, and later routing logic a stable structure without requiring them to parse prose.

The markdown summary is for humans. An operator should not have to open a raw Telegram payload just to understand that a text message arrived or a service event happened.

This is the filesystem-first pattern in miniature: raw evidence, structured metadata, and readable context live side by side.

Offset Advancement

The listener tracks progress with update ids:

• last_update_id
• next_update_offset
• next_offset

last_update_id records the highest update written to disk. next_update_offset records the next Telegram offset to request. next_offset is the runtime property used by the listener and intake code.

The important rule is ordering: the offset advances only after inbox writes succeed. That prevents a dangerous failure mode. If state advanced before the record existed, a crash could permanently skip an update. The inbound-intake work keeps acknowledgement tied to persistence.

Restart Safety

On restart, intake reads listener state and polls Telegram with the saved next offset. Updates below that offset are skipped. New updates are written, then state advances again.

The tests cover this explicitly. One test proves the saved offset is used before polling again. Another proves an update below the saved offset does not rewrite an existing inbox record, while the next update is written and advances state.

That gives the listener a practical high-water mark. It is not a complete recovery story yet, because later work still needs stale lease handling and richer diagnostics. But it is enough to prevent the basic duplicate-processing bug that appears when a listener forgets what it has already acknowledged.

The Foreground Shell Still Matters

The inbound-intake work did not replace the foreground shell. It inserted intake into it.

The foreground listener still checks its stop conditions on every loop. If a bot token is available, it creates a default intake boundary and polls Telegram. If no token is available, the listener can still maintain lifecycle state and respond to close.

That means runtime control and protocol intake remain separate. Operators can reason about start and stop behavior without understanding Telegram payloads. Developers can improve normalization without changing how the process exits.

What Existing Behavior Stayed Stable

The inbound work did not break outbound channel behavior. channels send still uses the Telegram sender, channels status still reports channel filesystem counts, and the Telegram implementation remains under src/pkgs/core/channels/telegram/.

The inbound-intake changes add files and behavior around the existing contract:

• api.py for getUpdates,
• update.py for normalization,
• inbox.py for durable records,
• intake.py for poll-write-advance behavior,
• extra state fields for update progress,
• and tests plus fixtures for representative payloads.

Why This Is Production-Shaped

The implementation is still an MVP slice, but the shape is production-minded. It has an explicit network boundary. It keeps secrets in environment variables. It writes durable records. It advances offsets after persistence. It preserves raw payloads. It runs inside a stoppable foreground shell. It is covered by unit tests and a loopback HTTP test.

Those choices make the next production work tractable. The recovery work can focus on recovery and observability. The packaging work can focus on end-to-end smoke and packaging. The core inbound path now has receipts.

The Design Rule

Do not treat inbound messages as logs. Logs describe what a process said while it was running. Inbox records are product state. They are the durable evidence that a channel delivered work into the org.

The inbound-intake work turns Telegram updates into that kind of state.