Webhooks and Events

Cycles services emit events from implemented budget, reservation, tenant, API-key, webhook, and system hooks. The schema also registers planned event types that are not emitted yet; the Event Payloads Reference tracks that status. Webhooks deliver emitted events to external endpoints via HTTP POST with HMAC-SHA256 signatures.

Core Concepts

Events

An event is an immutable record of an emitted decision, lifecycle outcome, or state change. Every event has:

• event_id — unique identifier; the key consumers dedupe on
• event_type — dotted format like budget.exhausted or reservation.denied
• category — one of: budget, reservation, tenant, api_key, policy, webhook, system
• timestamp — when the event occurred
• tenant_id — which tenant is affected
• source — which service emitted it (an open string; current reference values include cycles-server, cycles-admin, and cycles-events)
• data — optional event-specific payload (varies by type)

Events are stored in Redis with a 90-day TTL (configurable).

Webhook Subscriptions

A subscription defines which events to deliver and where:

• url — HTTPS endpoint to receive HTTP POST requests
• event_types — specific events to receive (e.g., ["budget.exhausted", "reservation.denied"])
• event_categories — receive all events in a category (additive with event_types)
• scope_filter — optional scope-path filter; only events whose scope matches are delivered (see Webhook Scope Filter Syntax)
• signing_secret — HMAC-SHA256 key for payload verification

Delivery Semantics

• At-least-once — events may be delivered more than once. Deduplicate using event_id.
• No delivery-order guarantee — concurrent delivery workers and retries can reorder deliveries. Use event_id only for deduplication; reconstruct chronology from event timestamps, the stored event log, and current domain state.
• Non-blocking — webhook delivery never blocks the API operation that produced the event.
• Retry with backoff — failed deliveries retry with exponential backoff (default: 5 retries).
• Auto-disable — subscriptions are disabled after consecutive failures (default: 10).

Architecture

Admin ServerCRUD operations

Runtime Serverreserve / commit

event:{id} + delivery:{id} + LPUSH dispatch:pending

Redisdispatch:pending queue

BRPOP

Cycles Events ServicePort 7980

HTTP POST + X-Cycles-Signature

Your Webhook Endpoint

Event flow architecture: 1. Both the Admin Server (CRUD operations) and Runtime Server (reserve/commit) write events to Redis. Each event creates an event:{id} record, a delivery:{id} record, and is pushed to the dispatch:pending queue via LPUSH. 2. Redis holds the dispatch:pending queue. 3. The Cycles Events Service (port 7980) consumes events via BRPOP from the dispatch:pending queue. 4. The Events Service delivers webhooks via HTTP POST with X-Cycles-Signature header to your webhook endpoint. The events service is optional. If not deployed, events accumulate in Redis with TTL and are delivered when the service starts.

The events service is optional. If not deployed, events accumulate in Redis with TTL and are delivered when the service starts.

51 Registered Event Types

Category	Count	Registered examples (some remain planned)
budget	17	budget.exhausted, budget.threshold_crossed, budget.over_limit_entered, budget.funded, budget.closed_via_tenant_cascade
reservation	6	reservation.denied, reservation.commit_overage, reservation.released_via_tenant_cascade
tenant	6	tenant.created, tenant.suspended, tenant.closed
api_key	7	api_key.created, api_key.revoked, api_key.revoked_via_tenant_cascade
policy	3	policy.created, policy.updated, policy.deleted
webhook	7	webhook.created, webhook.paused, webhook.disabled_via_tenant_cascade
system	5	system.store_connection_lost, system.webhook_delivery_failed

The four *_via_tenant_cascade types were added to the enum in governance spec revision v0.1.25.35.

Tenant Self-Service

Tenants can create their own webhook subscriptions via /v1/webhooks (requires webhooks:write permission). Tenant webhooks are restricted to budget, reservation, and tenant events: 29 of the 51 registered event types — including the budget.* and reservation.* _via_tenant_cascade fan-out events emitted during a tenant close (see Tenant-Close Cascade Semantics). API key, policy, webhook lifecycle, and system events are admin-only: a tenant-owned subscription can neither carry them nor receive them from the event stream, enforced at write, dispatch, and last-mile delivery (governance WEBHOOK SUBSCRIPTION INVARIANT 2; the one exception is the owner-triggered /test probe — see Tenant-accessible events).

Security

• HMAC-SHA256 — every delivery includes X-Cycles-Signature: sha256=<hex> for payload verification
• Encryption at rest — signing secrets are encrypted in Redis with AES-256-GCM using WEBHOOK_SECRET_ENCRYPTION_KEY; admin and events fail startup without the key unless plaintext is explicitly enabled for local development
• SSRF prevention — private IP ranges blocked by default, HTTPS required in production

Learn More

• Webhook Integrations Guide — PagerDuty, Slack, ServiceNow examples with code
• Security Hardening — webhook URL security and secret rotation
• Production Operations — events service deployment and failure handling