Stream Processing Expert

# Stream Processing Expert ## Overview **Stream Processing Expert** implements scalable real-time data pipelines for event-driven analytics and real-time insight. It designs topologies that are correct under failure and explicit about semantics — event-time windowing, watermarks, exactly-once processing, back-pressure handling, and stream-batch hybrid processing (Kafka Streams / Flink / Storm / Kinesis class). ## Core Capabilities - ✅ **Event-driven architecture** — durable topics, explicit event contracts, producer/consumer decoupling - ✅ **Delivery semantics** — exactly-once where it matters, mechanism-specified per path - ✅ **Time semantics** — event-time watermarking, allowed lateness, side outputs for late data - ✅ **Fault tolerance** — checkpointing, durable state, replay from offsets - ✅ **Scaling & flow control** — partitioning/parallelism tuned without breaking ordering; back-pressure handling - ✅ **Observability** — lag, throughput, latency percentiles, checkpoint health, alerting ## Usage 1. Load the `stream-processing-expert` agent into the Markus workspace. 2. Provide the event sources, ordering/latency requirements, and existing job configs. 3. The agent produces the streaming architecture, pipeline config, delivery-semantics spec, late-data policy, scaling plan, and monitoring runbook — registered via `deliverable_create`, with build stages tracked via `task_create` (using `blocked_by`). ## License & Source Based on [stream-processing-expert](https://github.com/gensecaihq/Claude-Code-Subagents-Collection/blob/main/subagents/data-analytics/stream-processing-expert.md) (MIT). --- **Attribution:** Based on [stream-processing-expert](https://github.com/gensecaihq/Claude-Code-Subagents-Collection/blob/main/subagents/data-analytics/stream-processing-expert.md) from [Claude-Code-Subagents-Collection](https://github.com/gensecaihq/Claude-Code-Subagents-Collection) - Original author: gensecaihq contributors - License: MIT