loading
All projects
2025Production Engineering & Research at Adda Education14 pages
Parallel Agentic Audits & High-Throughput LLM Systems
Architecture and optimization strategies for scaling multi-agent evaluations across 326,000+ real-world audio streams under strict rate and cost constraints
abstract
Deploying large language models for high-volume enterprise auditing requires balancing reasoning accuracy against strict API rate limits and severe cost ceilings. This technical breakdown details the architecture of a parallel agentic audit system that evaluated 326,000+ customer and educational calls at ~85% accuracy. It covers prompt caching, tiered agent routing (fast evaluation vs. deep arbitration), and asynchronous batching that reduced per-call costs by 58% while tripling audit throughput.
Developed and deployed in production at Adda Education, powering enterprise quality assurance and candidate scoring.
key findings
- Tiered routing between lightweight classification models and deep reasoning arbiters achieved ~85% accuracy while preventing costly unnecessary reasoning passes
- Asynchronous worker pool orchestration with rate-aware token bucket scheduling achieved a 3x speedup under strict provider concurrency limits
- Per-call cost decreased by 58%, saving tens of thousands in inference expenditure at six-figure call scale
- DynamoDB to BigQuery analytics migration reduced data pipeline infrastructure costs by ~95%
cite this work
APA
Srivastava, S. P. (2025). Parallel Agentic Audits: Scaling High-Throughput LLM Reasoning Systems Under Production Rate and Cost Limits. Adda Education AI Technical Series.
BibTeX
@misc{srivastava2025agenticaudits,
author = {Srivastava, Satyarth Prakash},
title = {Parallel Agentic Audits: Scaling High-Throughput LLM Reasoning Systems Under Production Rate and Cost Limits},
year = {2025},
note = {Adda Education AI Systems}
}