How much can a looping AI agent burn? One test ran 41 times: 4 guards to set
OpenRouter ran six models, three times each, on a billing task built to fail. Without a guard, 4 of 18 runs retried…
The Deployment Engineering topic discusses how to run models faster and cheaper: inference acceleration, memory and cost optimization, serving, monitoring, and observability. For engineers responsible for launch and operations to compare concrete solutions and measured data before choosing a stack.
How to run models faster and more cost-effectively: inference acceleration, VRAM and costs, service deployment, and infrastructure selection.
OpenRouter ran six models, three times each, on a billing task built to fail. Without a guard, 4 of 18 runs retried…
OpenRouter has launched Beta localization in 7 languages including Simplified Chinese and Japanese, and detailed the…
On 11 November 2026, OpenRouter published official measurements of the same GitHub task using the GitHub MCP server,…
OpenRouter published an explainer on quantization, covering the principles of LLM quantization and reporting tests…
OpenRouter released a tutorial explaining three ways to obtain reliable structured output from any LLM: sending a…
Sarvam AI launched Token Factory, an inference service that provides open-weight models including Sarvam 105B, GLM…
OpenClaw has refactored how it obtains model lists, spanning about five weeks and 80 PRs, so new models no longer wait…
On Nov 5, 2026, OpenAI published a National Community Compact for future compute campuses it owns and operates, plus a…
The SGLang team and Cloudflare coordinated disclosure of a cross-request KVCache exposure issue affecting…
Wiz disclosed that a malicious version of LiteLLM, 1.107.0, was uploaded to PyPI at 11:39 UTC on October 21 and…
Ant Group and the vLLM community have open-sourced Humming, a general-purpose high-performance quantized inference…
Google and Meta jointly open-sourced TorchTPU, a native PyTorch backend for Google TPU that lets models run out of the…
llm-d, an open-source inference project led by IBM, Red Hat and Google, released llm-d 1.0 at the PyTorch Conference…
DeepSeek announced that its internal inference framework DeepInfer is now fully open-source, alongside the…
The ModelBest MiniCPM-V team detailed the lookahead expert scheduling (LES) technique behind on-device MiniCPM-V 4.7,…
Google launched the TorchTitan TPU fork (github.com/google-pytorch/torchtitan), which works with the TorchTPU native…
In a PyTorch-related announcement, Google introduced native PyTorch inference engines for vLLM and SGLang built on…
Modal released Sticky Sessions, built on Modal Servers, to route all requests in the same session to the same…
The n8n blog published a tutorial on deploying seven local models on a single NVIDIA DGX Spark (128GB unified memory)…
Meta introduced Prometheus, its gigawatt-scale supercluster in New Albany, Ohio. The first phase is complete and has…
AWS announced TorchNeuron, an open-source native PyTorch backend for Trainium, entering public beta on October 20…
The SGLang team launched Low-Latency Mode, which introduces specialized kernels through a plugin framework to…
Digital life blogger Kazike shared a week of in-depth use of the ASUS ProArt 16 (RTX Spark N1X chip, 128GB unified…
vLLM announced Day 0 support for Reflection's Beam model, a 501B-parameter MoE model with 23B activated per token,…
AMD and Embedded LLM detailed in a vLLM official blog post the inference optimization of Kimi K3 on eight AMD Instinct…
Ant Ling officially open-sourced Ling-3.1-flash, with about 560B total parameters, about 25B activated per token and…
Apple's machine learning research team demonstrated how to run frontier models with nearly a trillion parameters, such…
The SGLang team announced Day-0 inference support for Meta's open-source multimodal MoE model Muse Spark Open (816B…
Databricks demonstrated systematic performance optimizations that reduced Qwen 3.8 27B fine-tuning time on AI Runtime…
vLLM extended confidence scoring to all draft-model-based speculative types, using an online-fitted logistic…
Modal announced that its annualized revenue has surpassed $1 billion and that it has closed a $750 million Series D…
LangChain released the langchain-scaledown integration and added support for ScaleDown's constrained decoding models…
Anthropic published an engineering retrospective: because Claude drove average daily merged code from about 2,500 in…
Microsoft announced that starting next year it will expand its Maia chip family to multi-gigawatt deployments as the…
A ComfyUI blog author used an agent to collect various optimizations and compressed MiniMax H3 real-time video…
Goodfire Research built a monitoring cascade based on activation probes plus an LLM judge for Kimi K3 and GLM 5.3, and…
Inferact and the vLLM community completed optimizations within three weeks of the DeepSeek-V4.1-Flash release,…
GitHub announced a rebuild of its Git infrastructure to support the high-concurrency read and write loads of…
Together AI released Together Link, which connects the coding agent tools teams already use to open-source models on…
A Baseten engineer, referencing the MetaInfer paper, had Claude Code (Fable 5) automatically build an inference…
Sponsored
Orange Relay (pro.nat6.net) is a premium AI API relay — an excellent transfer station with competitive pricing, reliable quality and broad model coverage including Claude, GPT and Gemini.
Visit Orange Relay