Mark RadarMARK RADAR
About
中 EN
Sign in
Event File AI OpenAI Anthropic NVIDIA Large Language Models (LLMs)

NVIDIA Unveils Switchyard to Route and Translate LLM Traffic

1 reports · First detected 2026-09-03 · Last active 2026-09-03

AI agents and inference backends often rely on incompatible interfaces: Claude Code uses Anthropic’s Messages API, while Codex CLI speaks OpenAI formats, even as companies may want to serve models through vLLM, NVIDIA NIM or Ollama. NVIDIA’s Switchyard aims to remove that integration bottleneck by placing protocol translation and model routing in a common layer, allowing developers to change providers or backends without rewriting the agent itself.

NVIDIA released Switchyard on Sept. 2, 2026, as an Apache 2.0-licensed Rust proxy and library. It accepts OpenAI Chat Completions, OpenAI Responses and Anthropic Messages, translates requests and streaming responses between formats, and offers four routing modes, including passthrough, random, LLM-classifier and stage-router algorithms. The software can be installed from crates.io or PyPI and self-hosted, but NVIDIA labels it pre-alpha and experimental, warning that it is not ready for production and may change substantially before version 1.0.

All Coverage

1 original reports
MARKTECHPOST.COM 2026-09-03
Meet Switchyard: A Rust Proxy and Library That Routes and Translates LLM Traffic Across OpenAI and Anthropic APIs

The Backstory

The history behind this event
Nvidia Open-Sources Nemotron 3.5 Lightning and Switchyard Router2026-08-12 · 3 reports · similarity 0.82

AI agents repeatedly plan, call tools and verify results, making latency and inference spending a growing concern when every step relies on a large frontier model. Nvidia is addressing that bottleneck with Nemotron 3.5 Lightning for high-volume execution and NeMo Switchyard for assigning work across models. The pairing supports Nvidia’s broader effort to drive enterprise adoption of open AI software while strengthening demand for its DGX and RTX platforms.

Nvidia open-sourced Nemotron 3.5 Lightning on Aug. 11, 2026. The mixture-of-experts model has 30 billion parameters but activates 3 billion for each token, and Nvidia says it can generate output as much as four times faster than similarly sized models. The company also released NeMo Switchyard, which routes requests to the model best suited to each stage of a task, potentially reducing execution time and operating costs. Nvidia disclosed no fixed dollar price or standardized savings estimate.

Mark Radar｜MARK RADAR
Past Issues・by MarkReadFintech

If you search news on Google, you can set Mark Radar as a preferred source—our coverage will show up more often in your results. Set as preferred source on Google →

All times are in Taipei time (GMT+8)