Red Hat Puts Safety and Observability at the Core of Enterprise AI with Red Hat AI 3.5

Red Hat announced significant updates across the Red Hat AI portfolio with the release of Red Hat AI 3.5. As enterprise teams move past early experimentation and pilot successes, IT and platform engineering leaders face the challenge of running AI with the same operational rigor as mission-critical infrastructure. By providing the scalable foundation required to control, secure and observe these workloads across the hybrid cloud, Red Hat AI 3.5 bridges the gap between isolated AI pilots and a fully governed enterprise architecture.

What is Red Hat AI 3.5?

Red Hat AI 3.5 delivers the operational foundation organizations need to scale AI in production and extend it across hybrid environments through new safety and observability capabilities. With this release, organizations can verify models before deployment through EvalHub, enabling risk-focused safety benchmarking and the creation of regulatory compliance certifications. New observability dashboards give platform teams comprehensive metrics to gain real-time insight into inference health, GPU utilization, and AI model performance. Non-admin users can access dashboards for per-user token consumption showback, and distributed inference workloads.

In addition, Red Hat AI 3.5 expands on proven enterprise platform capabilities to deliver enhanced multi-tenancy for AI service providers and AI use cases that require complete hardware-to-software isolation as well as priority-aware serving with native multi-tenancy for shared GPU infrastructure. For organizations that require stronger isolation between tenants, Red Hat AI now officially supports running on Red Hat OpenShift hosted control planes deployed on Red Hat OpenShift Virtualization. Hosted control planes give every tenant a dedicated cluster control plane while consolidating the hardware beneath them. Running AI workloads in Red Hat OpenShift Virtualization virtual machines adds robust VM-level isolation across shared, GPU-enabled infrastructure. Together, these capabilities let infrastructure providers operate and upgrade the entire underlying environment from a single point of control.

Red Hat AI 3.5 also accelerates the path to governed AI agents. AutoRAG links enterprise data repositories straight to agentic applications, introducing advanced capabilities such as multilingual document support, conversational testing, and contextual retrieval. A visual pipeline gives teams confidence in their RAG configurations before deploying. Agent templates deliver pre-configured implementations for common patterns like code review, document processing, and research workflows. Once deployed, Inference-Time Scaling optimizes GPU spend by adjusting compute dynamically based on query complexity.

Why does Red Hat AI 3.5 matter?

As enterprise AI pilots succeed and initial results show promising returns, IT teams must then address the need for delivery at scale. But scaling AI across the business demands the same operational rigor as any mission-critical infrastructure: verified safety before deployment, precise resource controls across shared GPU environments, governed agent behavior and transparent usage metrics. Most organizations today face these challenges with fragmented tooling and manual processes that don't scale. Red Hat AI 3.5 addresses this gap by unifying safety, multi-tenancy, agentic development and observability into a single enterprise platform. The result is AI that operates as an accountable, governed AI architecture — not an isolated experiment — across hybrid cloud environments.

What Red Hat is saying

“The conversation has moved from getting AI into production to running it at scale as trusted enterprise infrastructure, which requires safety evidence, governed agents, cost attribution and multi-tenancy,” said Joe Fernandes, vice president and general manager, AI Business Unit, Red Hat. “With Red Hat AI 3.5, we are delivering the operational controls, verifiable trust and agentic foundations IT leaders need to run AI as a safe, controlled and accountable enterprise AI architecture across the hybrid cloud.”

• Verifiable pre-deployment safety and evaluation: Evaluated catalog models feature built-in Garak benchmark scores, while the general availability of EvalHub automates safety and auditable compliance reporting for custom models, RAG and agents.
• Shared GPU control for multi-tenant inference: Fair-share GPU scheduling manages resource allocation across tenants, while priority-aware serving provides admission control and priority-based request routing to protect real-time inference and allows background workloads to use available capacity.
• Agent APIs and gateway security: General availability support for the Responses API and built-in RAG provides a unified open-source interface for multi-turn agent conversations, reinforced by integrated NeMo Guardrails that intercept malicious tool calls.
• Enterprise data grounding and efficient reasoning: AutoRAG with pgvector support, native AutoML, and Inference-Time Scaling (ITS) allow models to adapt compute usage dynamically based on query difficulty.
• Built-in observability and MaaS showback: Delivers per-user token metering, performance dashboards for models and agents, MLflow visual agentic tracing, and GPU utilization dashboards for clear operational and usage transparency.
• Pre-built agent templates for faster development: AI Hub introduces agent templates and starter kits with pre-configured reference implementations for common enterprise patterns, including code review, document processing, and research workflows.

Deeper details

• Hosted control planes and OpenShift Virtualization support for better resource utilization with multitenancy: Red Hat AI now officially supports hosted control planes, giving each tenant a dedicated control plane and better hardware consolidation. Support for AI workloads in OpenShift Virtualization VMs allows multiple tenants to share physical servers with strong VM-level isolation.
• New batch of validated models with AI safety scores: More than 20 new validated models added to the Catalog including models from Google (Gemma 4), NVIDIA (Nemotron 3), Alibaba Cloud (Qwen) and more. These validated models have full performance benchmarking and now include integrated Garak safety, PII exposure, and toxicity risk scores, to provide full transparency to enterprises assessing AI model risk.
• Validated tool-calling models: A select group of validated models have been tagged as validated for tool-calling in the Catalog to give customers confidence when choosing a model they wish to use for agentic use cases.
• Traffic management and safe model updates: Priority-aware serving provides admission control and priority-based request routing to protect real-time inference while allowing background workloads to use available capacity and Controlled model rollout manages traffic during model updates, enabling safe transitions with minimal service disruption.
• Efficient GPU memory management: The general availability of CPU offloading and developer preview of storage offloading allow models to handle longer conversations and larger documents without requiring additional GPU hardware.
• Multimodal serving: Serves text, audio, and image generation within a unified serving layer via vLLM Omni (available in early access).
• Multi-cloud Kubernetes serving: Extends llm-d distributed inference further beyond OpenShift onto third-party Kubernetes services, offering a consistent model serving experience across clouds. Now generally available on CoreWeave CKS and Microsoft Azure and with Amazon EKS joining as technology preview.
• Agent development kits: AI hub introduces pre-configured agent templates and starter kits for common enterprise patterns such as code review, document processing, and research workflows, integrating frameworks, tools, and deployment configurations to run in sandboxed environments with operational controls and security policies intact from first deployment through production.
• Enterprise data workbench: Kubeflow Spark Operator, available as a developer preview, brings distributed data processing directly into the active workbench environment, unifying data preparation and model serving on a single platform.
• GPU-as-a-Service and multi-tenancy: Features namespace isolation, hosted control plane support, and real-time dashboard visibility into hardware inventories, active utilization, and dynamic GPU capacity borrowing.

Availability

Red Hat AI 3.5 is now generally available. Red Hat AI 3.5 is also now available as part of Red Hat AI Factory with NVIDIA.