Fresh402 Agent Efficiency Benchmark -> November 5, 2026
Register a baseline. Refresh deliberately. Compare structured observations. Recover the original result when a response is lost. Four agent workflows bring the existing Web and Base tools together.
Start with a walletless Check -> One account, all seven tools
Real model decisions -> controlled data
Five paired trials. Same GPT-6.1 Sol model, same 65-operation OpenAPI JSON fixtures, same required compatibility findings. The comparison is against oasdiff1.34.1, a maintained open-source analyzer, rather than a model reading unparsed JSON alone.
| Metric | Free oasdiff workflow | Fresh402 comparison functions |
|---|---|---|
| Total model tokens | 50,113 | 42,537 |
| Correct required findings | 5 of 5 | 5 of 5 |
| End-to-end local trial time | 21.5s | 21.9s |
| Model API-price equivalent | $0.03909 | $0.03150 |
| With Starter consumed-credit equivalent | $0.03909 | $0.04400 |
The token denominator is the baseline median of 50,113; reduction is 1 - 42,537/50,113. This is a context-efficiency result, not a lower-total-cost or faster-production claim. Cache rates, model decisions, service fees and network latency affect economics. All 10 completions passed the scoped semantic checks.
Download all 10 API trial measurements and cost bases (JSON) -> Read the method and limits ->
Register once, Check for freshness, then use scoped Extract or Smart Diff when appropriate. Ignore documented noise; preserve watches and original operation keys.
Check 2 -> Extract 3 -> Smart Diff 4 credits. Calls are separate; no automatic monitoring scheduler or replacement payment.
Documentation Guardian workflow ->Compare a saved OpenAPI JSON baseline with a current contract. Read directional findings and explicit uncertainty. YAML and arbitrary remote references are unsupported.
OpenAPI Contract Diff 5 credits or$0.020 direct USDC. The result is bounded compatibility evidence, not proof that every client will work.
Review the tool contract ->Combine Contract Watch, Wallet Watch and Token Facts with pinned finalized-block provenance. Reuse saved observations while distinguishing replay from a fresh read.
4 / 8 / 4 eligible prepaid credits. Wallet Watch remains a restricted pilot: selected provider-reported ranges are not complete history or an independent completeness proof.
Base observation limits ->Connect website notes, API changes and selected Base observations into one brief. Explain conflicts and missing evidence; correlation does not prove a deployment or causality.
Authorize each chosen call independently within your own finite budget. Retain the full results alongside the compact brief.
Explore the existing flagship workflows ->These commands only read discovery. They do not connect a wallet, sign, buy credits or run a paid tool.
curl --fail https://api.fresh402.com/v2/prepaid/plans
curl --fail https://api.fresh402.com/v3/agent-trial
curl --fail https://api.fresh402.com/openapi.jsonNew Starter ($2.50 / 1,000 credits) and Developer ($15 / 10,000) purchases cover all seven tools after explicit All-Services terms acceptance and finalized funding. One-time credits do not expire. Existing purchases retain their rights. Trial credentials are separate; exhaustion never authorizes a purchase.
Review the exact purchase flow -> Public npm SDK -> Connect MCP -> Live All-Services terms
The seven-tool machine trial needs an unfunded EOA ownership signature and available per-tool/shared capacity. The walletless entry is one bounded Check, not signerless access to costly Base operations. Inspect live quotas; no allowance promises guaranteed admission.
Recorded November 5, 2026. Five paired fresh-model contexts per flagship, 20 frozen held-out tasks across four workflows. The featured API subset contains all five pairs, without failed or removed API trials. Four separate interrupted contexts in the broader study remain in the private run record and are not presented as completed measurements.
For the featured API case, each fixture has 65 operations: one GET removed, one cursor query parameter becomes required, and parameter ordering changes. Both conditions may inspect raw sources and keep local state. Correctness requires the exact affected paths/parameter and no ordering false positive. GPT-6.1 Sol made real tool choices; these are controlled AI evaluations, not customer testimonials.
Fresh402 measurements run the actual production comparison functions in local workerd against synthetic sources. They include model context, source inspection and baseline parsing, but exclude production gateway/network/cold-start latency, settlement, deployment and pre-existing package installation. oasdiff uses the installed 1.34.1 executable with external references disabled. These observations do not establish complete production workflow savings or all-workload superiority.
Model costs use the official standard API rates observed November 5: $2 /million uncached input tokens,$0.10 /million cached input,$10 /million output. Reasoning tokens are already included in output. The actual runtime used an authorized ChatGPT/Codex session; the dollar values are API equivalents, not a subscription invoice. Direct service equivalent is$0.020; Starter consumed-credit equivalent is5 * $0.0025=$0.0125; Developer is5 * $0.0015=$0.0075. Local tests made no USDC payment and debited no real credits. Unused package value remains stored value.
All paired values, distributions and pricing bases make the published claim auditable. Reproduce the compatibility findings using oasdiff on your own before/after OpenAPI JSON; a hosted Fresh402 comparison requires an eligible public watch and the documented explicit access method. This page does not authorize spending.