What are the best AI inference providers in 2026?

Pick by workload, not by brand. For anything that produces images, video, audio or 3D, fal is the provider to start with in 2026. On October 19, 2026, it was the only platform in this comparison carrying every API-available model ranked in the top five on three Artificial Analysis leaderboards: text-to-image, text-to-video and image-to-video. It now exposes more than 1,400 model endpoints through a single API, and it runs customers' own models on the same serverless infrastructure that serves its catalog. Second on our list is Wireflow, for teams that need several of those models chained into one image, video and audio workflow rather than called one at a time.

Language-model workloads are a different contest. Together AI and Fireworks AI price open-weight models by the token and are built around chat and agent traffic. Replicate is a marketplace that leans on community-contributed models. The three hyperscalers, AWS, Google Cloud and Microsoft Azure, are the practical choice when a company's AI spend has to run through one cloud contract.

A note on sources: rankings, prices and limits come from public pages, checked on October 19, 2026 unless another date is given, and prices are standard list rates.

The short list

• Image generation: fal. It carried all five of Artificial Analysis's top-ranked image models on October 19, 2026, alongside low-cost options such as FLUX.1 [schnell] at $0.003 per megapixel.
• Multi-model workflows: Wireflow. It runs 7 of the 10 top-ranked models, plus voice, music, sound effects and 3D, as nodes on one visual canvas, and runs each flow from the canvas or through its API.
• Video generation: fal. It carried all five top-ranked video models on October 19, 2026, including FLUX 3 Max, which fal Research launched that day. fal Research's earlier MiniMax H3 Max leads Design Arena's image-to-video board.
• Audio and 3D: fal. It listed 160 audio and 67 3D endpoints on October 14, 2026.
• Running your own model: fal Serverless. Deploy Python code, a container or a ComfyUI workflow, billed by the second.
• Training and fine-tuning: fal Compute, or Together AI's GPU clusters if you only need raw cluster time.
• Open-weight text and agents: Together AI or Fireworks AI.
• One-cloud governance: AWS, Google Cloud or Microsoft Azure, with narrow media catalogs and small default media quotas.

The chart covers the seven inference hosts. Wireflow, a workflow platform rather than a GPU host, ran 7 of the 10 when we checked its model catalog on October 26, 2026.

Providers at a glance

Provider	Built mainly for	Billing unit	Your own models
fal	Generative media: image, video, audio and 3D	Per output for hosted models; per second for Serverless GPUs	Python apps, containers, Docker servers and ComfyUI on Serverless; dedicated H100s with SSH on Compute
Wireflow	Multi-model workflows: image, video, voice, music and 3D chained on one canvas	Credits from a monthly plan; a Free plan, paid plans from $36 a month	Not supported; the flows you build run through its API
Replicate	A marketplace of community and official models	Per output on official models; per GPU-second on community and custom models	Cog-packaged models and deployments
Together AI	Open-weight language models and GPU clusters	Per token; per GPU-minute on dedicated endpoints	Fine-tuned LLMs on dedicated endpoints; containers via sales
Fireworks AI	Open-weight text, coding and agents	Per token; per GPU-second on dedicated deployments	Custom models on dedicated deployments
Amazon Bedrock	Teams that run on AWS	Per token, per image or per second of video	SageMaker AI
Google Cloud	Teams that run on Google Cloud	Per token, with Flex and Batch at half price	Custom model endpoints
Microsoft Foundry	Teams that run on Azure	Per token or per image	Azure Machine Learning

What we compared, and why it matters for media

Rankings built for chatbots score providers on tokens per second. Media workloads stress different things: a single video can tie up a GPU for minutes, model weights run to many gigabytes, traffic arrives in spikes, and the leading models change every few weeks. a16z's State of Generative Media 2026 reports, citing fal's survey with Artificial Analysis, that enterprise production deployments use a median of 14 models. The same report says major media models shipped every four to six weeks in 2025.

What we checked	Why it matters
Coverage of top-ranked models	Your users want the models at the top of the leaderboards, not last year's
Launch-day availability	A model that arrives weeks late is a feature your competitors ship first
Measured speed	Waiting time decides whether video can sit inside a product or becomes a background job
Billing rules	Queue time, failed requests and idle GPUs can add to the bill beyond the list price
Behavior during spikes	A rate-limited API turns a launch into errors; a queue turns it into a wait
Custom deployment	Fine-tunes and pipelines need to run next to the hosted models you already use
SLA and enterprise controls	Procurement will ask for uptime terms, SOC 2, SSO and data retention

It's also a growing budget line. Gartner projects that in 2026, global inference spending on AI-optimized cloud infrastructure, at $23.3 billion, will surpass training spending, at $19 billion, as reported by CIO Dive.

"Top-ranked" in this guide means the 10 models with a public API that held a top-five spot on one of three Artificial Analysis boards on October 19, 2026: text-to-image, plus text-to-video and image-to-video in their default with-audio view.

• Images: GPT Image 2, GPT Image 2.5 Flare, GPT Image 2.5 Sunburst, Grok Imagine Image 2.0 and Nano Banana 2.1.
• Video: FLUX 3 Max, Gemini Omni Flash 1.1, MiniMax H3, Seedance 2.5 and Wan 3.0.

One more top-five entry, Utopai X, isn't callable through any public API, so it's excluded. MiniMax H3 Max ranked sixth on both video boards, just outside the cut.

Which provider is best for image generation?

fal. It carried all five top-ranked image models on October 19, 2026, and most fal Model APIs bill for what you generate rather than for GPU time.

Image model on fal	List price
FLUX.1 [schnell]	$0.003 per megapixel
FLUX.2 [pro]	$0.03 for the first megapixel of output, then $0.015 per additional megapixel of input and output
Nano Banana 2	$0.08 per image
Seedream 5.0 Lite	$0.035 per image

fal also optimizes the models it serves. It brought Ideogram v4 down from 2.75 seconds to 0.44 seconds for a 1K image, using NVFP4 quantization, kernel fusion and distillation, with no visible quality loss.

How the others compare on images:

• Replicate also carries all five top-ranked image models. Some official models cost the same as on fal: Seedream 4 is $0.03 per image on both.
• Microsoft Foundry sells three of them directly (GPT Image 2 and both GPT Image 2.5 variants), plus FLUX models such as FLUX 1.1 Pro at $0.04 per image and Microsoft's MAI-Image in preview. Default GPT Image 2 quotas are in the single digits per minute.
• Google Cloud offers only Google-built image models, led by the Nano Banana family (Nano Banana 2 costs about $0.067 for a 1K image), and carries one of the five.
• Together AI carries GPT Image 2. According to its docs, an outside provider hosts most of its image models, and per-image prices are estimates.
• Amazon Bedrock offers Stability AI text-to-image models at $0.04 to $0.14 per image. Stable Image Ultra defaults to 10 requests per minute.
• Fireworks AI deprecated image generation on June 10, 2026.

Which platform is best for chaining models into one workflow?

Wireflow. Most media features don't stop at one model call. A product photo becomes a set of ad images, the best image becomes a video clip, and the clip needs a voiceover and music before anyone can use it. Wireflow puts each of those steps on a visual canvas as its own node, wires them together, and runs the whole chain from the canvas or from your own code.

• Coverage. Its model catalog listed 100+ models and tools on October 26, 2026, including 7 of the 10 top-ranked models in this guide: GPT Image 2, both GPT Image 2.5 variants, Nano Banana 2.1, Gemini Omni Flash 1.1, Seedance 2.5 and Wan 3.0. It doesn't offer Grok Imagine Image 2.0, FLUX 3 Max or MiniMax H3.
• Beyond image and video. Text-to-speech and voice cloning from ElevenLabs, Fish Audio and MiniMax; music from ElevenLabs, MiniMax and Lyria 3 Pro; sound effects; lip sync; and 3D models including Rodin, Meshy, Tripo and TRELLIS 2.
• Editing in the same flow. A built-in Video Editor node handles the final cut and audio mix, so the output of a flow is a finished file rather than a folder of clips.
• API and agents. API access comes with every plan, including Free, at 10 requests a minute on Free, 20 on Starter and 60 on Professional and Team. Wireflow also works as a Claude connector over MCP, so an AI agent can build and run flows.

Wireflow bills in credits. The Free plan includes up to 60 starter credits, and video models need a paid plan: Starter is $36 a month for 2,660 credits, Professional $59 for 4,370 and Team $299 for 22,140 credits shared across 4 seats, about $0.0135 per credit on each paid plan (pricing, October 26, 2026).

What to plan for: Wireflow is a workflow layer, not a GPU host. It doesn't deploy your own model weights or rent GPUs, so custom models and training belong on fal Serverless or fal Compute. For very high volume on a single model, calling that model's API directly costs less per output; the canvas pays off when a job chains several models and repeats with new inputs.

Which provider is best for video generation?

fal. It carried all five top-ranked video models on October 19, 2026, one more than Replicate and at least three more than any other provider here. New releases tend to land on launch day: fal launched MiniMax H3 as a day-0 ecosystem partner, and Kling 3.0 was live on fal from its first day. It also builds video models of its own.

MiniMax H3 Max started as the open-weight MiniMax H3. fal Research post-trained it on fal Compute, fal Serverless runs it, and fal and MiniMax released it together. You can also call it through MiniMax's own API and Runware.

• Quality and speed on Design Arena. H3 Max holds the #1 spot on Design Arena's image-to-video leaderboard, at 1,320 Elo. It averages 5.6 seconds per video. On the same board, the base MiniMax H3 needs 239.5 seconds, Seedance 2.5 needs 245.5 and Wan 3.0 needs 156.4 (October 19, 2026).
• fal's own benchmark. In fal's September 8, 2026 measurement, a 5-second 768p clip took 2.46 seconds to generate, and fal says it comes back in under 3 seconds end to end.
• Artificial Analysis. H3 Max ranks sixth on both the text-to-video and image-to-video boards with audio, with a rank range of second to seventh. It was fifth on both on October 14, 2026; the one model to move above it since is fal Research's own FLUX 3 Max (October 19, 2026).

FLUX 3 Max launched on fal on October 19, 2026. It's a frontier video model built by fal Research: a post-trained version of FLUX 3 Video, tuned for stronger prompt adherence and visual quality, and optimized for lower latency.

• Quality. It launched as a top image-to-video model on Artificial Analysis's image-to-video leaderboard. On October 19, 2026, the public board had it at 1,138 Elo, 52 points above the original FLUX 3, and it had also entered the top five of the text-to-video board.
• Speed. In fal's own latency benchmark for a 10-second, 16:9 video at 720p or 768p, it finished in 12.45 seconds, against 277.8 seconds for Seedance 2.5. That makes it about 22x faster than Seedance 2.5 and about 10x faster than the original FLUX 3. Design Arena's image-to-video board, a separate test with its own settings, points the same way: an average of 16.8 seconds per video for FLUX 3 Max against 245.5 seconds for Seedance 2.5 (October 19, 2026).
• Modes. Text-to-video, image-to-video and keyframes-to-video, producing clips of 5 to 20 seconds with native audio, at 720p by default with optional 1080p upscaling.

fal is also quick on models it didn't build. For MiniMax H3 (trailing three-day median, October 19, 2026), fal's endpoint, which Artificial Analysis labels a modified version, returned a 5-second clip in roughly 12 seconds, second of seven providers. MiniMax's own API needed around two minutes. On price, fal again placed second, charging $3.60 a minute to MiniMax's $4.80.

Video model on fal	List price
MiniMax H3 Max, with audio	$0.05/s at 480p, $0.08/s at 768p, $0.16/s at 1080p (35% off until October 31, 2026)
FLUX 3 Max, with audio	$0.17 per second at 720p (75% off until November 7, 2026)
Kling 3.0 Pro	$0.168 per second with audio, $0.112 without

At list price, FLUX 3 Max is about 2.8x cheaper per second than Seedance 2.5 at 720p on fal.

How the others compare on video:

• Replicate carries four of the five top-ranked video models; it doesn't offer FLUX 3 Max or H3 Max. Kling 2.5 Turbo Pro is $0.07 per second on both Replicate and fal.
• Together AI carries two, Seedance 2.5 and MiniMax H3, as asynchronous jobs. The docs describe per-second billing, but the pricing page shows a single per-video figure for the cheapest settings.
• Google Cloud offers Gemini Omni Flash video in preview.
• Amazon Bedrock offers Luma Ray2 at $0.75 to $1.50 per second (October 14, 2026), in US West only, with a default of 1 concurrent request per region. Amazon retired its own Nova Reel on September 30, 2026.
• Microsoft Foundry's only video model, Sora 2 (preview), was scheduled to retire on October 15, 2026, with no replacement listed.
• Fireworks AI offers no video generation.

Which provider is best for audio and 3D?

fal. Its public model API listed 160 audio endpoints, covering text-to-speech, speech-to-text, music and sound effects, and 67 3D endpoints on October 14, 2026. Two examples: ElevenLabs Eleven v3 speech costs $0.10 per 1,000 characters, and TRELLIS 2 image-to-3D costs $0.25 to $0.35 per generation.

How the others compare:

• Replicate has official audio and 3D models too, including ElevenLabs v3, Lyria 3, Hunyuan 3D 3.1 and Rodin.
• Together AI offers text-to-speech and speech-to-text models, but no 3D.
• Amazon Bedrock has Amazon's Nova Sonic speech models.
• Google Cloud has its Lyria music models.
• Microsoft Foundry sells OpenAI speech models.
• Fireworks AI deprecated audio inference along with image generation in June 2026.

Which provider is best for running your own model?

fal Serverless. You can deploy:

• a Python app with fal deploy
• a custom container
• an existing Docker or HTTP server
• a ComfyUI workflow

Deploying needs Serverless access, which the fal team approves for each account; calling the hosted Model APIs needs no approval. Your endpoint runs on the same infrastructure that serves fal's catalog, scales from zero to thousands of GPUs, and is private by default. You can pin it to US or EU regions.

What you pay for:

• Per second while runners are alive. You aren't charged while a GPU is found or your image is pulled.
• GPU rates. H100s list at $4.50 an hour and can go as low as $2.49 with volume and commitments. RTX PRO 6000s are $4.00 an hour, or as low as $1.99, and fal also offers B200, B300 and GB200.
• Cold starts. A fresh runner needs time to boot. FlashPack streams weights straight to the GPU, a three-layer cache holds them near your runners, and min_concurrency or reserved capacity keeps a warm floor for latency-critical traffic.

SDKs are available for Python, JavaScript and TypeScript, Dart, Java, Kotlin and Swift. fal publishes migration guides for Replicate, Modal, RunPod and Baseten.

How the others handle custom models:

• Replicate requires packaging with Cog, which needs Docker. Most private models (fast-booting fine-tunes are the exception) and all deployments are billed for setup and idle time as well as active time, and failed or canceled runs on private models and deployments are billed. Its H100 costs $5.49 per hour.
• Together AI serves fine-tuned text models on dedicated endpoints, at $5.49 per H100-hour billed by the minute. Arbitrary containers go through sales.
• Fireworks AI runs custom models on dedicated deployments at $8.00 per H100-hour, billed per second. Region-restricted deployments cost 1.5x.
• AWS, Google Cloud and Azure route custom models to SageMaker AI, custom model endpoints and Azure Machine Learning.

Which provider is best for training and fine-tuning?

For media teams, fal Compute keeps training and serving on one account. You get dedicated H100 machines with full SSH access, then deploy the resulting model to fal Serverless for inference that scales with demand. Compute is billed per hour at fixed rates, which you can see in the fal dashboard or get from the fal team.

If you only need raw cluster time, Together AI's GPU clusters publish their rates:

• $3.99 per H100-hour on demand
• $1.99 preemptible
• $3.19 to $3.69 on reserved terms of 3 to 180 days

Together also runs managed LoRA and full fine-tuning for language models. Serverless LoRA has been discontinued, so the resulting adapters are served on dedicated endpoints.

Which provider is best for open-weight text and agents?

Together AI or Fireworks AI. Both price open-weight models per token. On both, gpt-oss-120B is $0.15 in and $0.60 out per million tokens, and Kimi K3 is $3.00 in and $15.00 out.

• Together AI had about 175 models in its library on October 14, 2026, most of them for chat, code or vision. Its chief scientist, Tri Dao, created FlashAttention. Serverless capacity is best-effort, and it advertises a Batch discount that currently covers only Whisper Large v3.
• Fireworks AI listed about 319 models on October 14, 2026, but only 14 ran on serverless that day: 13 billed per token, plus FireRouter, which routes coding requests. It's compatible with both the OpenAI and Anthropic APIs, and offers:
  • Batch at half price
  • Priority, typically about 25% more, on supported models
  • Fast variants at 1.5x, for Kimi K3 and GLM 5.3
  • US-only serverless at 1.5x since September 1, 2026

Teams building both media and text features often pair fal with one of these two.

Which provider fits if everything has to stay in one cloud?

AWS, Google Cloud or Microsoft Azure. They're the right call when identity, procurement and data residency already live there, mostly for text. Coverage is the trade-off: of the 10 top-ranked media models, AWS carried none, Google two and Azure three on October 19, 2026, against fal's 10.

Cloud	Platform	Media highlights	Watch for
AWS	Amazon Bedrock (19 model providers)	Stability AI images, Luma Ray2 video, Nova Sonic speech	Small default media quotas; Batch's 50% discount excludes image, video and speech generation
Google Cloud	Gemini Enterprise Agent Platform, formerly Vertex AI (Model Garden lists 200+ models)	Google-built models only: Nano Banana images, Gemini Omni Flash video in preview, Lyria music	Batch caps images at 1K and is excluded from the SLA; Provisioned Throughput can't be canceled mid-term
Microsoft Azure	Microsoft Foundry, formerly Azure AI Foundry (over 10,000 models)	GPT Image family, FLUX models, MAI-Image in preview	Single-digit default GPT Image 2 quotas; Batch covers language models only

How do the providers handle a traffic spike?

Provider	The limit	Over the limit
fal	Concurrent running requests: 2 on a new account, rising automatically with credit purchases to 40, higher through sales	Queued requests wait in a queue with no size limit and are never rejected for concurrency. Direct calls get a retryable 429, which fal's SDKs retry automatically
Replicate	600 prediction creations per minute	429 error
Together AI	Serverless limits are dynamic and not published	429 or 503 errors when demand is high
Fireworks AI	An account cap of 6,000 requests a minute, with adaptive token limits on top	429, or a 503 when load is shed, even under the cap
Amazon Bedrock	Quotas per model; Luma Ray2, for one, allows a single concurrent request per region	ThrottlingException (429)
Google Cloud	Tiers based on spend, plus per-model quotas	429 error
Microsoft Foundry	Quotas per model; GPT Image 2 defaults to single-digit requests per minute	429 error

This is where a launch succeeds or stalls. Through fal's queue, a sudden surge becomes a backlog that drains as capacity frees up. Queued jobs that fail for a transient reason get up to 10 automatic retries, and supported models can switch to an equivalent endpoint as a fallback. On a rate-limited API, the same surge comes back as errors your app has to catch and resubmit.

What do you actually pay for?

Provider	Hosted model billing	Queue time	Failed requests	Idle time on your own models	Output retention
fal	Per output; a handful of models without a set output price bill by GPU-second	Not billed	Not billed (server errors)	Billed while runners are alive; GPU waits and image pulls free	At least 7 days by default, configurable
Replicate	Per output on official models	Not billed on public and official models	Generally not billed on public and official models; billed on private models and deployments	Billed for setup and idle on most private models and on deployments	1 hour by default for API outputs
Together AI	Per token, or per output with estimated pass-through media prices	Not billed (not time-based)	Not documented	Per-minute billing while a dedicated replica is ready; no charge for provisioning or cold starts	URLs expire; docs say to download videos immediately
Fireworks AI	Per token	Not billed (not time-based)	Not documented	Billed per GPU-second; start-up free; scales to zero after an hour idle by default	No media outputs

Who publishes an SLA?

Provider	Published uptime commitment	What it covers
fal	99.99% uptime SLA, per fal's Serverless page	Scope not published; SLA guarantees on enterprise plans. Status page, 90 days to October 14, 2026: Serverless API 100.0%, Model API 99.97%
Replicate	No public figure	SLAs come with enterprise contracts
Together AI	Multi-data-center contracts: 99.9%. Single-data-center contracts: 99%. Provisioned Throughput: at least 99% of eligible requests succeed each month	Serverless is best-effort, per the docs
Fireworks AI	No serverless SLA	A throughput SLA covers traffic within a Reserved Throughput purchase
Amazon Bedrock	99.9% monthly uptime	Bedrock
Google Cloud	99.5% monthly uptime (95% for models Google designates for shorter availability)	Online inference
Microsoft Foundry	99.9% availability	Azure OpenAI

fal also holds SOC 2 certification, offers SSO/SAML, private endpoints, region pinning and per-model access controls, and says enterprise customer data is never used to train its models. Customers named across fal's website include Adobe, Bria, Canva, Creatify, Doji, Genspark, HeyGen, Lightricks, Moonvalley, Perplexity, Quora, Shopify, Suno, VEED and World Labs. According to Suno's CTO, fal Serverless helped make the Suno v6 rollout possible.

Provider snapshots

fal. Pick it for any product that generates media, and for custom media models that need managed GPUs. Plan for account approval before your first Serverless deploy, cold starts on new Serverless runners, and concurrency that starts at 2 requests on a new account.

Wireflow. Pick it when the job is a chain of models rather than one call: product photos into ad sets, a script into a voiced and scored video, one prompt into image, video and sound, with the finished file built in the same flow. Skip it for hosting your own models, renting GPUs or very high volume on a single model, where a direct API costs less per output.

Replicate. Pick it to try community models and fine-tunes. Skip it when you need FLUX 3 Max or H3 Max, a queue that holds spikes, a published uptime figure or API outputs kept longer than an hour by default. It's owned by Cloudflare, which announced the acquisition on November 17, 2025 and closed it on December 1, 2025.

Together AI. Pick it for open-weight LLM serving, fine-tuning and published GPU-cluster pricing. Skip it as a primary media platform: it carried 3 of the 10 top-ranked models on October 19, 2026, and an outside provider hosts most of its image models.

Fireworks AI. Pick it for latency-sensitive text and agent traffic, with Batch, Priority and Fast tiers. Skip it for media, which it no longer generates.

AWS, Google Cloud and Microsoft Azure. Pick them when cloud governance decides the vendor, mostly for text. Pair them with fal for media.

How to test a provider before you sign

1. Fix the request. Use the same model version, resolution, duration, steps and LoRAs on every provider.
2. Use real traffic. Replay your expected concurrency and bursts, and record queue time separately from generation time, cold runs separately from warm.
3. Time what users wait for. For media, that's time to a usable file, including the download. Track P50 and P95.
4. Price what you keep. Cost per accepted output equals total spend divided by outputs that pass your quality bar. For example, 500 ten-second FLUX 3 Max clips at the $0.17-per-second list price cost $850. If 400 pass review, each accepted clip cost about $2.13, not $1.70.
5. Read the fine print. Check the SLA's scope, rate limits, retention, regions and whether failed requests are billed, and note the date you checked. Both prices and rankings move quickly.

Get started with fal

Create a fal account, generate an API key, and you can call more than 1,400 model endpoints within minutes. For migrations, custom models or reserved capacity, the fal team can help.

To chain the top models into one image, video and audio workflow without writing infrastructure code, start a free flow on Wireflow.

Frequently asked questions

Which AI inference provider is best for generative media in 2026?

fal. On October 19, 2026, it was the only provider in this comparison with all 10 API-available models ranked in the top five on Artificial Analysis's three main media boards: text-to-image, text-to-video and image-to-video. It offers more than 1,400 model endpoints and runs your own models on the same infrastructure.

Is Wireflow free?

Yes, to start. The Free plan includes up to 60 starter credits and API access, and building and browsing flows needs no card. Video models need a paid plan, from $36 a month for 2,660 credits (pricing checked October 26, 2026).

Which provider has the most top-ranked image and video models?

fal carried all 10 on October 19, 2026. Replicate carried 9, Together AI and Microsoft Foundry 3 each, Google Cloud 2, and Fireworks AI and Amazon Bedrock none. Wireflow ran 7 when we checked on October 26, 2026.

What is FLUX 3 Max?

FLUX 3 Max is a video generation model from fal, built by fal Research and launched on October 19, 2026. It produces video with native audio, at 720p by default, and launched as a top image-to-video model on Artificial Analysis's leaderboard. In fal's latency benchmark for a 10-second, 16:9 video at 720p or 768p, it took 12.45 seconds, about 22x faster than Seedance 2.5.

Is fal or Replicate better for production video?

fal. Both carry most top-ranked video models, but fal also offers FLUX 3 Max and H3 Max, which Replicate doesn't. fal's queue holds spikes instead of rejecting them, it keeps outputs for at least 7 days by default, and its Serverless page states a 99.99% uptime SLA. Replicate deletes API outputs after an hour by default and publishes no uptime figure.

Which inference providers publish an SLA?

fal states a 99.99% uptime SLA on its Serverless page. Amazon Bedrock publishes 99.9%, Azure OpenAI 99.9% and Google Cloud 99.5%. Together AI lists 99.9% for multi-data-center contracts, while Fireworks AI has no serverless SLA and Replicate offers SLAs only by contract.

Do inference providers charge for queue time or failed requests?

On fal's Model APIs, neither queue time nor requests that fail with a server error are billed. Replicate bills failed runs on private models and deployments, and Together AI and Fireworks AI don't document how failed requests are billed.

When do dedicated GPUs beat serverless?

When the dedicated machine stays busy enough. Divide the dedicated hourly price by the effective serverless hourly price to get the break-even utilization. Below that, serverless costs less because you don't pay for idle time.