Batch API: Pay 50% Less

Batch processing is the largest single discount in the LLM market, and it is the one most teams never turn on. OpenAI, Anthropic, Google and xAI all h
Onsp

Batch processing is the largest single discount in the LLM market, and it is the one most teams never turn on. OpenAI, Anthropic, Google and xAI all halve both input and output rates for requests that can wait up to 24 hours. Not a promotional rate, not a tier you have to qualify for — a permanent 50% cut on the same models, the same context windows and the same outputs, in exchange for latency.

The arithmetic is brutal in the best way. A job that costs $400 a night costs $200 in batch. Run it for a year and the difference is $73,000. Most teams that run nightly jobs already have the workload; they simply submit it synchronously because that is what the tutorial showed.

Prompt caching as an investment with payback period
Batch and caching are both payback decisions — batch pays back on the first request.

Who offers batch, and at what price

ProviderBatch supportDiscountNotes
OpenAIAll 13 models50% off input and outputResults within 24 hours
AnthropicAll 5 models50% off input and outputResults within 24 hours
GoogleAll 6 models50% off input and outputResults within 24 hours
xAIBoth Grok models50% off input and outputResults within 24 hours
DeepSeekNone—Off-peak hours bill at 50% instead

Four of the five providers in this dataset treat batch as a first-class tier. DeepSeek is the exception: it has no batch API, and instead discounts requests outside peak hours. The economics are similar, but the trigger is different — you cannot submit a batch at 2 p.m. and get the discount, you have to wait for the window.

The annual saving on one nightly job

Take a realistic nightly workload: 10,000 requests, each with 10,000 input tokens and 2,000 output tokens. That is 100 million input tokens and 20 million output tokens per night.

ModelList price / nightBatch / nightAnnual listAnnual batchSaved
GPT-6 Astra$2,000$1,000$730,000$365,000$365,000
Claude Fable 5.1$2,000$1,000$730,000$365,000$365,000
GPT-6 Sol$400$200$146,000$73,000$73,000
Claude Sonnet 5$400$200$146,000$73,000$73,000
Gemini 3.8 Flash$150$75$54,750$27,375$27,375
Grok 4.3$175$87.50$63,875$31,937$31,937
GPT-6 Luna$20$10$7,300$3,650$3,650

The saving scales with the model tier, which produces a counter-intuitive conclusion: batch on an expensive model often costs less than synchronous calls on a cheap one. GPT-6 Astra in batch is $1,000 a night; GPT-6 Sol at list is $400. The expensive model stays expensive — but the gap between tiers narrows by half, which changes which tier you should use for the job.

When batch is the wrong tool

The discount is real, and it is unusable for anything a human is waiting on. Batch fails for:

  • Interactive chat — a 24-hour wait is not a response time.
  • Agents and tool loops — each step depends on the previous output, so nothing can be queued in advance.
  • Streaming interfaces — batch returns results at the end, not token by token.
  • Anything with a deadline under 24 hours — including most reporting and alerting.

The test is simple: if the result is still useful tomorrow morning, it belongs in batch. If it is not, it does not.

Workloads that should already be in batch

WorkloadTypical volumeWhy it fits
Nightly enrichment10K–1M recordsResults needed tomorrow, not now
Classification backfillAny historical setOne-off, no user waiting
Embedding refreshCorpus-sizedRuns on a schedule
Evaluation runsHundreds to thousandsOffline by definition
Report generationDaily or weeklyDeadline is the morning
Synthetic data creationLarge, burstyNo latency requirement at all
Content moderation sweepsRetrospectiveBatch by nature

How to structure a batch pipeline

1. Make requests idempotent. Attach your own request ID to every line so a retried job does not double-write results. Batch jobs fail in the middle, not at the start.

2. Split by failure domain. A single 500,000-line job that fails at line 400,000 is a bad afternoon. Chunk into 10,000-line jobs so a failure costs one chunk, not the night.

3. Budget for the tail. Most batch results return in minutes to hours, but the contract is 24 hours. Do not schedule a downstream job that depends on batch output to start one hour after submission.

4. Track cost per chunk. Batch bills per token like everything else; a chunk that silently grows its prompt size will show up as a cost anomaly, not a latency one.

5. Keep a synchronous fallback. When a batch job misses its window, the fallback is either a delayed report or a synchronous retry for the affected records — decide which before you need it.

Does batch stack with caching?

Both discounts exist on the same request, but the safe planning assumption is the simpler one: batch halves the list rate. Build the forecast on that number, and treat any cache saving as upside. Teams that budget for stacked discounts tend to discover the interaction the hard way, in a month where the invoice is 30% above plan.

FAQ

How much does the Batch API save?

50% on both input and output tokens for OpenAI, Anthropic, Google and xAI. DeepSeek offers no batch API but discounts off-peak requests by 50% instead.

How long does a batch job take?

Most results return within hours, but the contractual window is 24 hours. Plan downstream jobs around the window, not the average.

Can batch requests stream?

No. Batch returns complete results at the end of processing, which is why it cannot serve interactive interfaces.

Is batch available on every model?

Yes on OpenAI, Anthropic, Google and xAI — every model in this dataset from those four providers supports it. DeepSeek supports none.

Related reading

Model your own nightly job in the free LLM cost calculator — it applies the batch discount per model, and flags the ones that do not offer it.

إرسال تعليق

Cookie Consent
We serve cookies on this site to analyze traffic, remember your preferences, and optimize your experience.
Oops!
It seems there is something wrong with your internet connection. Please connect to the internet and start browsing again.
AdBlock Detected!
We have detected that you are using adblocking plugin in your browser.
The revenue we earn by the advertisements is used to manage this website, we request you to whitelist our website in your adblocking plugin.
Site is Blocked
Sorry! This site is not available in your country.