We build in this tutorial a cost aware planning agent which balances out the quality of output against actual constraints, like token usage, latencies, and budgets on tool-calls. The agent is designed to create multiple candidates actions, calculate their costs and benefits and select the execution plan which maximizes value within strict budget constraints. We demonstrate that agentic systems are capable of moving beyond “always use the LLM” It is important to avoid a reactive behavior by focusing on trade-offs and resource awareness. Click here to see the FULL CODES here.
import os, time, math, json, random
Dataclasses can be imported to fields
Typing import List, Dict Optional, Tuple or Any
From getpass import Getpass
USE_OPENAI is True
If USE_OPENAI is true, then the following will be displayed:
If not, os.getenv ("OPENAI_API_KEY"):
os.environ["OPENAI_API_KEY"] = getpass("Enter OPENAI_API_KEY (hidden): ").strip()
try:
OpenAI import OpenAI
OpenAI client()
Except Exception As e.
print("OpenAI SDK import failed. Falling back to offline mode.nError:", e)
USE_OPENAI is False
We configure the environment for execution and load the OpenAI API keys securely at runtime, without hardcoding them. Also, we initialize the client to ensure that the agent will gracefully switch over to offline mode when the API becomes unavailable. Visit the FULL CODES here.
def approx_tokens(text: str) -> int:
return max(1, math.ceil(len(text) / 4))
@dataclass
A Budget for Class:
max_tokens: int
max_latency_ms: int
Maximum number of tool calls: Int
@dataclass
Spending on class:
tokens: int = 0
latency_ms: int = 0
Integer = 0 for tool_calls
def within(self, b: Budget) -> bool:
return (self.tokens <= b.max_tokens and
self.latency_ms <= b.max_latency_ms and
self.tool_calls <= b.max_tool_calls)
Def add (self, other) "Spend") -> "Spend":
Return Spending
tokens=self.tokens + other.tokens,
latency_ms=self.latency_ms + other.latency_ms,
tool_calls=self.tool_calls + other.tool_calls
)
We provide the budgeting abstractions to enable agents to think explicitly about costs. Modeling token usage, tool calls, latency and other quantities as first class quantities, we provide methods for accumulating and validating spend. We can enforce the constraints in planning and implementation. Click here to see the FULL CODES here.
@dataclass
Step Option:
name: str
description: str
est_spend: Spend
est_value: float
executor: str
Payload: dict[str, Any] = field(default_factory=dict)
@dataclass
Class PlanCandidate
Steps:[StepOption]
Spend:
Value: float
Hash: str ""
def llm_text(prompt: str, *, model: str = "gpt-5"The effort is: "low") -> str:
If not USE_OPENAI
Return to the Homepage ""
t0 = time.time()
resp = client.responses.create(
model=model,
reasoning={"effort": effort},
input=prompt,
)
_ = (time.time() - t0)
If you want to return the output text, please click on this link. ""
The data structures represent both individual actions and complete plan options. Also, we define a lightweight LLM wrapping that standardizes the way text is measured and generated. This allows planners to think about action abstractly, without becoming too tied down to the execution details. Take a look at the FULL CODES here.
def generate_step_options(task: str) -> List[StepOption]:
Base = [
StepOption(
name="Clarify deliverables (local)",
description="Extract deliverable checklist + acceptance criteria from the task.",
est_spend=Spend(tokens=60, latency_ms=20, tool_calls=0),
est_value=6.0,
executor="local",
),
StepOption(
name="Outline plan (LLM)",
description="Create a structured outline with sections, constraints, and assumptions.",
est_spend=Spend(tokens=600, latency_ms=1200, tool_calls=1),
est_value=10.0,
executor="llm",
payload={"prompt_kind":"outline"}
),
StepOption(
name="Outline plan (local)",
description="Create a rough outline using templates (no LLM).",
est_spend=Spend(tokens=120, latency_ms=40, tool_calls=0),
est_value=5.5,
executor="local",
),
StepOption(
name="Risk register (LLM)",
description="Generate risks, mitigations, owners, and severity.",
est_spend=Spend(tokens=700, latency_ms=1400, tool_calls=1),
est_value=9.0,
executor="llm",
payload={"prompt_kind":"risks"}
),
StepOption(
name="Risk register (local)",
description="Generate a standard risk register from a reusable template.",
est_spend=Spend(tokens=160, latency_ms=60, tool_calls=0),
est_value=5.0,
executor="local",
),
StepOption(
name="Timeline (LLM)",
description="Draft a realistic milestone timeline with dependencies.",
est_spend=Spend(tokens=650, latency_ms=1300, tool_calls=1),
est_value=8.5,
executor="llm",
payload={"prompt_kind":"timeline"}
),
StepOption(
name="Timeline (local)",
description="Draft a simple timeline from a generic milestone template.",
est_spend=Spend(tokens=150, latency_ms=60, tool_calls=0),
est_value=4.8,
executor="local",
),
StepOption(
name="Quality pass (LLM)",
description="Rewrite for clarity, consistency, and formatting.",
est_spend=Spend(tokens=900, latency_ms=1600, tool_calls=1),
est_value=8.0,
executor="llm",
payload={"prompt_kind":"polish"}
),
StepOption(
name="Quality pass (local)",
description="Light formatting + consistency checks without LLM.",
est_spend=Spend(tokens=120, latency_ms=50, tool_calls=0),
est_value=3.5,
executor="local",
),
]
Use Open if USE_OPENAI
meta_prompt = f"""
You're a planning assistant. You are a planning assistant.
Like checks, validations, stakeholder tailored or tailoring. Shorten each step.
TASK:
{task}
List of JSON values with the following fields: Name, Description, Est_Value(1-10).
"""
txt = llm_text(meta_prompt, model="gpt-5", effort="low")
try:
Items = json.loads (txt.strip())
It is available in all items[:5]:
base.append(
StepOption(
name=str(it.get("name","Extra step (local)"))[:60],
description=str(it.get("description",""))[:200],
est_spend=Spend(tokens=120, latency_ms=60, tool_calls=0),
est_value=float(it.get("est_value", 5.0)),
executor="local",
)
)
Except Exception
pass
Return base
We focus on generating a diverse set of candidate steps, including both LLM-based and local alternatives with different cost–quality trade-offs. We can optionally use our model itself to suggest low-cost additional improvements, while still controlling its impact on budget. We can enrich our action area without compromising efficiency. Take a look at the FULL CODES here.
Def plan_under_budget
Choose from the list[StepOption],
budget: Budget,
*,
Maximum number of steps: 6
The beam width is int 12
Diversity_penalty float = 0.
) -> PlanCandidate:
def redundancy_cost(chosen: List[StepOption], new: StepOption) -> float:
key_new = new.name.split("(")[0].strip().lower()
Overlap = 0.
For s, choose:
key_s = s.name.split("(")[0].strip().lower()
If key_s >= key_new
1 + overlap
return overlap * diversity_penalty
Beams[PlanCandidate] = [PlanCandidate(steps=[], spend=Spend(), value=0.0, rationale="")]
For _ within range (max_steps),:
Expanded List[PlanCandidate] = []
Cand in beams
For opt-in options:
If you opt-in cand.steps
Continue reading
new_spend = cand.spend.add(opt.est_spend)
if not new_spend.within(budget):
Continue reading
new_value = cand.value + opt.est_value - redundancy_cost(cand.steps, opt)
expanded.append(
PlanCandidate(
steps=cand.steps + [opt],
spend=new_spend,
value=new_value,
rationale=cand.rationale
)
)
If not expanded
Breaking News
expanded.sort(key=lambda c: c.value, reverse=True)
Beams = Expanded[:beam_width]
Best = max(beams; key=lambda C: c.value);
return best
Budget-constrained logic is implemented to find the best combination of actions within strict limitations. In order to prevent wasteful actions, we use a beam search style and redundancy penalty. The agent is now cost-aware, as they are optimizing the value of a task within constraints. Look at the FULL CODES here.
Run_local_step (task: str; step: StepOption and working: Dict)[str, Any]) -> str:
Name = lowercase name()
If you want to know more about if "clarify deliverables" In name:
Return (
"Deliverables checklist:n"
"- Executive summaryn- Scope & assumptionsn- Workplan + milestonesn"
"- Risk register (risk, impact, likelihood, mitigation, owner)n"
"- Next steps + data neededn"
)
If you want to know more about if "outline plan" In name:
Return (
"Outline:n1) Context & objectiven2) Scopen3) Approachn4) Timelinen5) Risksn6) Next stepsn"
)
If you want to know more about if "risk register" In name:
Return (
"Risk register (template):n"
"1) Data access delays | High | Mitigation: agree data list + ownersn"
"2) Stakeholder alignment | Med | Mitigation: weekly reviewn"
"3) Tooling constraints | Med | Mitigation: phased rolloutn"
)
If you want to know more about if "timeline" In name:
Return (
"Timeline (template):n"
"Week 1: discovery + requirementsnWeek 2: prototype + feedbackn"
"Week 3: pilot + metricsnWeek 4: rollout + handovern"
)
If you want to know more about if "quality pass" In name:
draft = working.get("draft", "")
Return to the Homepage "Light quality pass done (headings normalized, bullets aligned).n" Draft
The return of f"Completed: {step.name}n"
Run_llm_step (task: str; step: StepOption; working: Dict[str, Any]) -> str:
Get the kind of step.value.get ("prompt_kind", "generic")
context = working.get("draft", "")
prompts = {
"outline"The f"Create a crisp, structured outline for the task below.nTASK:n{task}nReturn a numbered outline.",
"risks"The f"Create a risk register for the task below. Include: Risk | Impact | Likelihood | Mitigation | Owner.nTASK:n{task}",
"timeline"The f"Create a realistic milestone timeline with dependencies for the task below.nTASK:n{task}",
"polish"""Rewrite and polish the following draft for clarity and consistency.nDRAFT:n{context}",
"generic"The f"Help with this step: {step.description}nTASK:n{task}nCURRENT:n{context}",
}
return llm_text(prompts.get(kind, prompts["generic"]), model="gpt-5", effort="low")
def execute_plan(task: str, plan: PlanCandidate) -> Tuple[str, Spend]:
working = {"draft": ""}
Real = Spend()
for i, step in enumerate(plan.steps, 1):
t0 = time.time()
if step.executor == "llm" USE_OPENAI
Run_llm_step (task, step and working)
tool_calls = 1
else:
Out = run_local_step() (task, Step, Working)
tool_calls = 0
dt_ms = int((time.time() - t0) * 1000)
tok = approx_tokens(out)
actual = actual.add(Spend(tokens=tok, latency_ms=dt_ms, tool_calls=tool_calls))
working["draft"] += = f"nn### Step {i}: {step.name}n{out}n"
return working["draft"].strip()Current
Task = "Draft a 1-page project proposal for a logistics dashboard + fleet optimization pilot, including scope, timeline, and risks."
BUDGET = Budget(
max_tokens=2200,
max_latency_ms=3500,
max_tool_calls=2
)
options = generate_step_options(TASK)
best_plan = plan_under_budget(options, BUDGET, max_steps=6, beam_width=14)
print("=== SELECTED PLAN (budget-aware) ===")
For s, see best_plan.steps
print(f"- {s.name} | est_spend={s.est_spend} | est_value={s.est_value}")
print("nEstimated spend:", best_plan.spend)
print("Budget:", BUDGET)
print("n=== EXECUTING PLAN ===")
Draft = actual + execute_plan (TASK, best_plan).
print("n=== OUTPUT DRAFT ===n")
print(draft[:6000])
print("n=== ACTUAL SPEND (approx) ===")
print(actual)
print("nWithin budget?", actual.within(BUDGET))
Step by step, we execute and monitor the chosen plan. Dynamically, we choose the local or LLM execution path and combine it into one coherent document. Comparing estimated and actual expenditure, we show how planning assumptions can validated and fine-tuned in practice.
We concluded by showing how an agent that is aware of its cost can adapt in real-time to its consumption. The planning assumptions were validated by tracking actual expenditures. We only executed the steps within the budget that was pre-defined. This closed the loop between estimation, and execution. We also showed how AI agent systems could become more controllable and scalable if they treated cost, tool use, and latency as primary decision variables, rather than secondary ones.
Take a look at the FULL CODES here. Also, feel free to follow us on Twitter Join our Facebook group! 100k+ ML SubReddit Subscribe Now our Newsletter. Wait! Are you using Telegram? now you can join us on telegram as well.
Post How an AI Agent Chooses What to Do Under Tokens, Latency, and Tool-Call Budget Constraints? First on MarkTechPost.

