Deliver and secure enterprise AI at the speed of inference

Most enterprise GPU clusters run far below their capacity. To improve GPU utilization, organizations should implement runtime efficiency techniques like continuous batching, speculative decoding, and quantization, which extract substantially more throughput from hardware. In addition, they can use intelligent inference routing to send simple queries to smaller models while also caching repeated answers so they are not recomputed. If organizations feed those GPUs properly and instrument the full stack, they can lower the cost per token.

GPUs consume data faster than the pipeline can deliver it, leaving these expensive accelerators idle while they wait. Legacy storage was not designed to provide the sustained, high-throughput access to data that modern training and inference demand. The data movement problem is compounded when access patterns are unpredictable, when preprocessing is handled by an overloaded CPU, and when data is scattered across silos with no fast, unified path to compute. Organizations can address slow data movement by treating data delivery as engineered infrastructure. They should employ prefetching, caching, parallel loading, and high-throughput storage that places data closer to the GPUs. With these techniques they can keep smaller clusters consistently fed rather than maintaining larger clusters that are constantly starved.

Traditional API and data security protects APIs and sensitive information through controls such as authentication, authorization, schema validation, threat detection, and data protection. However, these controls are not designed to reliably interpret the semantic meaning and intent of natural-language prompts, which creates additional risks for AI systems. AI runtime security protects systems during interactions with users. It interprets natural-language input to detect threats instead of fixed rules, and it examines output before it reaches users to ensure the model isn’t delivering harmful content or exposing sensitive data.

According to the non-profit OWASP (the Open Worldwide Application Security Project), the top threat to AI systems is prompt injection, where attackers craft inputs to manipulate model behavior. The next highly ranked threat is sensitive information disclosure, where models leak personal data, system prompts, or intellectual property through outputs. AI guardrails can help stop these and other types of threats. Input guardrails screen for prompt injection and system override patterns to prevent models from seeing malicious instructions. Output guardrails screen and validate model outputs before they reach users or trigger downstream actions, helping prevent sensitive data from being exposed through model outputs.

There are three key reasons why enterprises are repatriating data and AI workloads from the cloud to sovereign or air-gapped environments. First, many need to enhance control over regulated or sensitive data, ensuring that they comply with data sovereignty regulations while also keeping proprietary training data out of external cloud data centers. Second, organizations want to avoid high egress fees and unpredictable bills from cloud providers. Finally, organizations want to improve performance of AI inference by eliminating the overhead of shared-tenant cloud infrastructure.

To protect data leakage when using AI agents and generative AI (GenAI) tools, enterprise SecOps teams need to implement a multi-layered defense strategy. That strategy should include implementing real-time, inline guardrails for inputs and outputs; limiting AI agent access privileges; securing AI object storage used for retrieval-augmented generation (RAG); safeguarding API access; and employing continuous red teaming and remediation for uncovered vulnerabilities. At the same time, teams should deploy tools to help discover, classify, and govern workforce AI usage.

F5 AI Guardrails helps enforce runtime policies and protect against risks such as prompt injection, sensitive data leakage, harmful outputs, and unsafe agent actions. F5 AI Red Team uses adversarial testing to identify vulnerabilities and weaknesses before and during deployment. F5 AI Gateway provides a centralized control point for governing access to models, agents, and tools, while F5 Workforce AI Security provides visibility into sanctioned and unsanctioned AI use across the enterprise. Together, these capabilities form part of the F5 AI Security Platform. For broader protection and delivery across applications, APIs, and distributed environments, the F5 Application Delivery and Security Platform (ADSP) provides capabilities including web application and API protection, bot management, DDoS mitigation, and application delivery.