Cycles

Appearance

← Blog

· Cycles Team · 7 min read

patternsbudgetsarchitectureguide
Share Share

AI Agent Budget Patterns: A Practical Guide ​

Every team running AI agents in production eventually faces the same question: how should we structure our budgets? Too coarse and a single runaway agent burns through the allocation. Too granular and the overhead of managing hundreds of micro-budgets becomes its own problem. This guide covers the six patterns we see most often, with concrete examples and trade-offs for each.

These patterns aren't mutually exclusive — most production systems combine two or three. The common budget patterns page in our docs covers the Cycles-specific implementation details; this post focuses on the architectural thinking behind each approach.

Note: Code examples in this post are simplified pseudocode to illustrate the pattern intent. For production-ready implementations using the actual Cycles SDK, see the Python quickstart, TypeScript quickstart, or common budget patterns.

Pattern 1: Tenant Isolation Budgets ​

When to use: Multi-tenant platforms where each customer or team gets their own AI agent access and you need separate runtime budget ceilings.

The simplest and most common starting point. Each tenant gets an independent budget, so one tenant's submitted reservations do not consume another tenant's allocation. The ceiling applies to operations routed through the mandatory integration boundary and to the estimates the caller submits.

python
# Tenant isolation: each tenant has a completely independent budget
tenant_budget = cycles.create_budget(
    scope=f"tenant:{tenant_id}",
    limit_dollars=500.00,
    period="monthly",
    on_exhausted="reject"
)

# Every agent call for this tenant checks against their budget
async def run_agent_for_tenant(tenant_id, task):
    budget = cycles.get_budget(scope=f"tenant:{tenant_id}")
    result = await budget.execute(
        agent.run(task),
        estimated_cost=estimate_task_cost(task)
    )
    return result

Trade-offs:

  • Provides separate ledgers, so one tenant's protected calls cannot consume another tenant's allocation
  • Simple to reason about and explain to customers
  • Can lead to underutilization: if Tenant A uses 10% of their budget and Tenant B hits 100%, there's no sharing
  • Requires careful initial sizing — set too low and legitimate workloads get blocked

This pattern maps directly to how tenant, workflow, and run budgets work in Cycles.

Pattern 2: Workflow-Level Caps ​

When to use: When different agent workflows have different cost profiles and risk levels, and you want to cap each independently.

A code review agent and a deep research agent have very different cost characteristics. Workflow-level caps let you set appropriate limits for each.

python
# Different workflows get different budgets
workflow_budgets = {
    "code-review":     {"limit": 2.00,  "per": "run"},
    "deep-research":   {"limit": 25.00, "per": "run"},
    "summarization":   {"limit": 5.00,  "per": "run"},
    "chat":            {"limit": 1.00,  "per": "session"},
}

async def run_workflow(workflow_type, input_data):
    config = workflow_budgets[workflow_type]
    budget = cycles.create_budget(
        scope=f"workflow:{workflow_type}:{run_id}",
        limit_dollars=config["limit"],
        period=config["per"]
    )
    return await budget.execute(agent.run(input_data))

Trade-offs:

  • Right-sized limits for each use case reduce both waste and false denials
  • Makes cost profiles explicit and auditable
  • Requires understanding the cost distribution of each workflow upfront
  • New workflows need budget configuration before deployment

Pattern 3: Per-Run Budgets with Graceful Degradation ​

When to use: When you want agents to produce some result even when they hit budget limits, rather than failing entirely.

This is the pattern that separates production-grade agent systems from prototypes. Instead of a hard stop at budget exhaustion, the agent downgrades its approach.

python
async def research_with_degradation(query, budget_dollars=10.00):
    budget = cycles.create_budget(
        scope=f"run:{run_id}",
        limit_dollars=budget_dollars
    )

    # Phase 1: Use the best model
    remaining = budget.remaining()
    if remaining > 5.00:
        result = await budget.execute(
            agent.run(query, model="claude-opus-4-8")
        )
    # Phase 2: Fall back to a cheaper model
    elif remaining > 1.00:
        result = await budget.execute(
            agent.run(query, model="claude-sonnet-4-6")
        )
    # Phase 3: Return cached/partial results
    else:
        result = get_cached_or_partial_result(query)
        result.metadata["degraded"] = True

    return result

Trade-offs:

  • Users get a result instead of an error, improving perceived reliability
  • Requires designing multiple quality tiers for each workflow
  • The "degraded" signal needs to propagate to the user — silent degradation erodes trust
  • More complex to test: you need to validate each fallback tier

We cover degradation strategies in detail in How to Think About Degradation Paths.

Pattern 4: Shared Pool with Priority Tiers ​

When to use: When you want to maximize utilization of a fixed budget across multiple agents or users while preferring high-priority work.

Instead of giving each consumer a fixed allocation, you share a pool and have the application defer lower-priority work when the pool runs low. Cycles does not natively order reservations by priority. A read-then-act threshold is advisory and can race under concurrency; if capacity for critical work must be guaranteed, provision a separate protected ledger for that work.

python
# Shared pool with priority tiers
pool = cycles.create_budget(
    scope="org:engineering",
    limit_dollars=5000.00,
    period="monthly"
)

# Application policy decides which work to submit as the pool gets low.
# This pre-check is a scheduling hint, not an atomic priority guarantee.
PRIORITY_THRESHOLDS = {
    "critical":  0.0,   # Only denied at $0 remaining
    "high":      0.10,  # Denied below 10% remaining
    "normal":    0.25,  # Denied below 25% remaining
    "low":       0.50,  # Denied below 50% remaining
    "bulk":      0.70,  # Denied below 70% remaining (off-peak only)
}

async def execute_with_priority(task, priority="normal"):
    remaining_fraction = pool.remaining() / pool.limit
    threshold = PRIORITY_THRESHOLDS[priority]

    if remaining_fraction <= threshold:
        raise BudgetExhaustedError(
            f"Pool at {remaining_fraction:.0%}, "
            f"threshold for '{priority}' is {threshold:.0%}"
        )

    return await pool.execute(task)

Trade-offs:

  • Higher overall utilization — no budget sits idle while another is exhausted
  • Application scheduling can preserve more headroom for critical work
  • Harder to predict per-team or per-user costs for billing purposes
  • Requires agreement on what constitutes "critical" vs. "low" priority
  • Risk of low-priority work getting permanently starved in busy periods

Pattern 5: Shadow Mode Rollout ​

When to use: When you're introducing budget controls to an existing system and need to validate limits before enforcing them.

This is less a budget structure and more a deployment pattern, but it's essential for any team that isn't starting from scratch. Shadow mode tracks what would have been denied without actually denying anything.

python
# Evaluate the same request without creating a reservation.
decision = await cycles.reserve(
    subject={"tenant": tenant_id, "workflow": workflow_id},
    estimate=estimated_microcents,
    unit="USD_MICROCENTS",
    dry_run=True,
)

# Dry run creates no reservation or balance mutation. The application records
# every decision and actual outcome; denied evaluations may also emit
# reservation.denied from the current reference server.
app_log.write({
    "decision": decision,
    "estimate": estimated_microcents,
    "actual": await run_and_measure(task),
})

Trade-offs:

  • The budget result does not block the workflow; the application still owns logging, execution, and other failure handling
  • Generates real data for sizing budgets accurately
  • Adds latency (the budget check still happens, just without enforcement)
  • Teams sometimes stay in shadow mode too long, delaying the value of enforcement

Our shadow mode rollout guide walks through the full process, including how to retain and analyze dry-run responses and choose enforcement cutover criteria.

Pattern 6: Hybrid Model (Tokens + Dollars) ​

When to use: When you need to track both the raw resource consumption (tokens) and the monetary cost (dollars), because they don't always move in lockstep.

Token counts and dollar costs diverge when you use multiple models, when pricing changes, or when non-LLM tools (web search, code execution) are part of the agent's toolkit.

python
# Cycles budgets and reservation amounts each use one unit. Provision separate
# ledgers for TOKENS and USD_MICROCENTS, then have the application acquire both
# reservations before execution. If the second reserve fails, release the first.
token_hold = await reserve(unit="TOKENS", estimate=estimate_tokens(task))
try:
    cost_hold = await reserve(
        unit="USD_MICROCENTS",
        estimate=estimate_cost_microcents(task),
    )
except Exception:
    await token_hold.release("cost_budget_not_available")
    raise

# Execute only after both holds succeed, then settle each measured amount.
result, actual_tokens, actual_microcents = await run_and_measure(task)
await token_hold.commit(actual_tokens)
await cost_hold.commit(actual_microcents)

Trade-offs:

  • Catches scenarios that a single-dimension budget misses (e.g., a cheap model looping uses few dollars but millions of tokens)
  • Useful for capacity planning beyond just cost
  • More complex to configure and explain to users
  • Requires accurate estimation for both dimensions
  • The two unit-specific reservations are coordinated by the application; Cycles does not make them one cross-unit atomic transaction

Combining Patterns ​

Most production systems layer two or three of these patterns. A common combination:

  1. Tenant isolation (Pattern 1) as the outer boundary
  2. Workflow caps (Pattern 2) within each tenant
  3. Graceful degradation (Pattern 3) within each workflow run
  4. Shadow mode (Pattern 5) for rollout

For instrumented paths, this gives you separate customer ceilings, right-sized limits per use case, user-friendly behavior at the limits, and a measured path to enforcement.

Tenant Budget ($500/mo)
├── Code Review Workflow ($2/run)
│   └── Per-run with degradation
├── Research Workflow ($25/run)
│   └── Per-run with degradation
└── Chat Workflow ($1/session)
    └── Hard deny at limit

The budget allocation and management guide covers how to implement these hierarchies in Cycles, and the cost estimation cheat sheet helps with initial sizing for each tier.

Choosing Your Starting Point ​

If you're unsure where to begin:

  • Single-tenant, single-agent: Start with Pattern 3 (per-run with degradation)
  • Multi-tenant SaaS: Start with Pattern 1 (tenant isolation) + Pattern 5 (shadow mode)
  • Internal platform with multiple teams: Start with Pattern 4 (shared pool with priority)
  • Migrating from no controls: Start with Pattern 5 (shadow mode) to gather data first

The most important step isn't picking the perfect pattern — it's having any budget boundary in the execution path. You can always refine the structure later. You can't un-spend money that an uncontrolled agent already burned.

Next steps ​

Last updated: