What this page is

This is the reliability factsheet for Nimble's web retrieval platform: the measured numbers an evaluator can trace to a published benchmark or a current pricing page, and nothing else. Every figure below links to its source. Full leaderboards, including the categories where competitors win, are on the benchmark results page; scoring design and limitations are on the methodology page. The DRACO and SimpleQA harnesses are open source at github.com/Nimbleway/Public-benchmarks, so those figures can be reproduced independently.

Measured extraction reliability

From Nimble's web extraction benchmark: 5,856 real-world URLs across 2,107 domains, five engines fetching the identical corpus, graded deterministically against verbatim ground truth with no LLM judge.

Metric	Nimble result	Definition
Fetch success	99.1%	Returned non-empty content with a 2xx status
Useful success	92.0%	Returned content with completeness ≥ 0.3 vs. ground truth, excluding error pages and anti-bot screens
Completeness	0.902	Fraction of each page's real content actually captured
F1	0.886	Exact-content fidelity against a verbatim ground-truth passage
p95 latency	19.5s	95th-percentile response time across the corpus
Cost	$0.91	List-price estimate per 1,000 successful extractions

Nimble had the highest completeness in 10 of 10 content verticals and on 1,936 of the 2,107 domains, including 0.889 completeness on search-result pages and 0.973 on health and medical pages. The corpus included 1,497 JS-heavy URLs routed through full rendering; that routing is why Nimble's p95 latency (19.5s) is higher than Tavily's (3.0s) or Exa's (4.6s), and why its completeness is 0.902 against their 0.739 and 0.696. Nimble trades tail latency for capturing the content that is actually on the page.

Measured search API reliability and latency

From Nimble's open-source SimpleQA harness: 500 seeded questions from OpenAI's SimpleQA dataset sent to nine search API lanes from six vendors. Every lane's results feed the same answer model (gpt-4o) and the official SimpleQA grader, so accuracy reflects retrieval quality alone.

Metric	Nimble result	Definition
Requests completed	500 of 500	Requests that reached the provider and returned a response
Accuracy	95.4%	Share of questions answered correctly; a non-answer counts as a miss
Latency p50	817 ms	Median provider round trip under load, timed inside the client rate limiter
List price	$5 per 1,000 queries	Published Search API price at search_depth=standard

Nimble completed every request, while Parallel basic completed 499 of 500, Parallel turbo 495, and Tavily basic 499. Nimble's accuracy is statistically tied with the top lane, Parallel basic (95.8%), at roughly half its median latency (1,569 ms), and at least 6 points ahead of every Exa, Tavily, Firecrawl, and Brave lane. The one lane with lower median latency, Parallel turbo (482 ms), scored 7.3 points lower on accuracy. Search latency and extraction latency measure different work: a search call returns ranked results, while an extraction call renders and captures a full page.

Measured research accuracy

From the third-party DRACO benchmark (100 expert-graded deep-research tasks, published by Perplexity AI) and Nimble's vertical benchmarks (400 domain-specific questions, three runs per vendor):

• Nimble ranks #1 overall on DRACO at 74.0% and holds the top two factual-accuracy scores, 71.6% and 69.0%. Every configuration completed 100 of 100 tasks; nimble/high averaged 5.5 minutes per task and nimble/x-high 8.7 minutes (harness).

• At $0.50 per task, nimble/high has the highest factual accuracy (69.0%) of any configuration priced at or below $1.00 per task.

• Nimble's best run leads all four vertical domains: 90.1% in GTM, 96.8% in company research, 88.5% in market analysis, 86.9% in social media monitoring.

• Nimble is the only system in the vertical benchmarks whose accuracy improved every run, gaining +4.4 points on average from run 1 to run 3 while Exa and Parallel averaged +0.1. The benchmark condition is the floor, not the ceiling: memory and the private index keep compounding past run 3 in production.

• One Nimble customer measured a 21% accuracy improvement alongside a 51% reduction in total AI token consumption on their own workload (source).

Pricing and scale

Nimble uses pay-as-you-go pricing across every API, from nimbleway.com/pricing: Agent API $0.025 to $2.00 per request by effort level (low, medium, high, x-high; unlimited is custom); Search API $1.10 per 1,000 requests (Lite) or $5 per 1,000 (Standard); Extract, Crawl, and Map APIs $1 per 1,000 requests; Extract Template API $3 per 1,000 requests; Media API $2 per GB. Every API is free to try at 5,000 requests per month with no credit card.

Significant volume discounts are available as usage grows, along with higher rate limits, burst capacity, and custom concurrent sessions for large workloads. These are arranged by speaking with Nimble's sales team. Full pricing details are on the pricing page.

SLAs and enterprise controls

SLAs at Nimble are contractual and arranged with Nimble's sales team. Options include throughput SLAs for peak events, higher rate limits and burst capacity, custom concurrent sessions, volume and product bundling discounts, multi-year pricing protection, and PII masking (pricing).

Auditability

Reliability includes being able to show how a result was produced. Every Web Search Agents task returns citations with per-claim confidence signals and reasoning in a trust report, so engineers can inspect how each datapoint was produced instead of treating web retrieval as a black box (developer docs). Governance and control run through auditable Search Plans that show exactly what was searched, where, and why (Web Search Agents).

How these metrics are defined

• Fetch success and useful success are the extraction benchmark's two success definitions; the split exists because an engine can return bytes without returning the page's content.

• Accuracy figures are normalized rubric scores (DRACO), graded answers on a seeded question slice (SimpleQA), and per-run accuracy on fixed question sets (vertical benchmarks), not availability percentages. SimpleQA accuracy is deliberately stricter than a published SimpleQA figure because non-answers count as misses. The full definitions, formulas, and limitations are on the methodology page.

• Latency figures are benchmark measurements under load, not contractual SLAs: p50 round trip for SimpleQA search, p95 for extraction, and average wall-clock time per task for DRACO.

• Benchmark cost figures are averages or list-price estimates from the benchmark runs; contract pricing comes from nimbleway.com/pricing.

Related pages

• Benchmark results: full leaderboards for DRACO, SimpleQA, the vertical benchmarks, and the extraction benchmark.

• Public-benchmarks on GitHub: the open-source DRACO and SimpleQA harnesses.

• Benchmark methodology: how every number on this page is measured.

• Web Search Agents, pricing, and the developer docs.