NVIDIA Free NIM Endpoints for Agent Workflows

NVIDIA opened its NIM endpoints to anyone with a free Developer Program account. You sign up at build.nvidia.com, confirm a phone number, and receive an nvapi- key that works with any OpenAI-compatible client after swapping the base URL to https://integrate.api.nvidia.com/v1. Over 100 models sit behind those endpoints, from Llama variants and Gemma 4 31B to DeepSeek, Qwen3.5-397B, and Nemotron Super 49B. The stated goal includes agent development with tool calling, and the service runs on DGX Cloud without requiring a credit card.

First steps with the free tier

Account creation takes minutes. Once the key appears in settings you can test a simple chat completion immediately. Model IDs follow patterns like meta/llama-3.1-8b or zhipuai/glm-4. No daily token ceiling is published, yet the hard floor sits at 40 requests per minute per model. New users once received 1,000 to 5,000 credits; current reports point to rate limits alone.

Every endpoint stays OpenAI compatible, so existing LangChain or CrewAI code needs only the base_url change. Embeddings, vision, and retrieval endpoints sit alongside chat models, which matters when an agent needs to fetch documents before answering.

Measured speed versus agent reality

Lab numbers look strong. Llama 3.1 8B on one H100 SXM hits roughly 1,201 tokens per second in FP8 with NIM optimizations, about double the 613 tokens per second seen without them. TensorRT-LLM paired with ReDrafter has shown 2.7 times speedups on larger production models. Those figures come from controlled benchmarks.

Agent workloads tell a different story. Developers running planning loops, tool calls, validation steps, and retries report DeepSeek R1 throughput dropping from over 100 tokens per second to 5 or timing out entirely. Gemma 4 often settles around 20 tokens per second under the same load. GLM 5.1 throws repeated 504 errors once the call volume rises. The gap appears because a single user task can generate dozens of backend requests instead of one.

Model	Lab TPS (H100)	Reported Agent TPS	Common Failure
Llama 3.1 8B	1201	40-60	429 after 30+ calls
DeepSeek R1	100+	5 or timeout	Extended cooldowns
Gemma 4 31B	~80	~20	Queue delays
GLM 5.1	Not published	Variable	504 errors

AgentIQ and Blueprints as practical starting points

NVIDIA supplies the AgentIQ library and a set of AI Blueprints built with partners including CrewAI and LangChain. These give pre-wired patterns for multi-step orchestration and RAG pipelines. GitHub examples show complete agent workflows that call tools, reflect on results, and loop until a goal is met. The hosted endpoints accept the same tool-calling schema used by paid providers, so swapping to a paid tier later requires minimal code changes.

Internal NVIDIA teams have used similar setups for software engineering agents and hardware design agents, claiming the equivalent of 30 years of work completed in a single year. Supply-chain agents reportedly cut daily planning time by more than 95 percent. Those results sit on enterprise hardware, yet the same patterns run on the free endpoints during early testing.

Where throttling appears first

Free-tier limits vary with overall demand and the specific model. NVIDIA states there is no fixed way to raise the cap. Frameworks that issue many sequential calls, such as OpenClaw or Hermes-style agents, hit 429 responses and multi-minute cooldowns faster than single-turn chat applications. Monitoring request counts per task becomes essential. A summarization agent that plans, retrieves, validates, and retries can exceed 40 requests before the first useful token returns.

Some models still consume legacy credits while others do not; checking the current free status before committing an agent loop saves surprises. Latency metrics like time-to-first-token and time-per-output-token also shift with output length, so a Q&A agent behaves differently from a long-form summarizer even on the same model.

Self-hosting when the free tier runs out

NIM containers can be downloaded and run locally or on up to two nodes and 16 GPUs. The same OpenAI-compatible API surface appears, so code written against the hosted endpoints transfers directly. Self-hosting removes the 40 RPM ceiling but shifts cost and maintenance to the developer. For teams that outgrow the free tier, this path keeps the workflow intact while moving to owned hardware or larger DGX Cloud instances.

Real production signals from partners

Pegatron deployed NVIDIA AI Enterprise plus DGX systems and recorded a 400 percent acceleration in agent development time along with a 7 percent labor-cost reduction per assembly line. The agents ingest robot sensor data through Isaac Sim and camera feeds via Metropolis for quality inspection. Klarna’s agent replaced work equivalent to 853 full-time employees and saved $60 million by Q3 2025. JPMorgan runs more than 450 agentic use cases on similar infrastructure. These deployments started with prototyping and later moved to paid capacity once volume and reliability requirements rose.

Choosing models for agent tasks

Catalog updates continue. MiniMax M2.7, a 230B mixture-of-experts model, joined on April 11. Gemma 4 31B arrived April 2. OpenRouter lists Nemotron 3 Ultra (55B active parameters, 550B total, 1 million token context) on its free tier. When picking models, test both raw speed and behavior under repeated tool calls rather than relying on leaderboard numbers alone. Models that maintain stable throughput after 20 or 30 requests per task tend to survive longer in agent loops.

Embedding and retrieval endpoints matter equally. An agent that cannot fetch context quickly will stall even if the chat model itself is fast. The free tier covers these categories, letting developers build complete retrieval-augmented agents before deciding on paid scale.

Production workloads eventually migrate. The free endpoints serve well for validation and early iteration, yet sustained agent traffic pushes teams toward paid tiers at $0.10 to $10 per million tokens or self-hosted NIM containers. Architecture decisions made early, such as request batching or fallback routing, determine how cleanly that transition happens.