loading
loading
Cohere Locks Enterprise AI Prompts Behind Hardware-Encrypted Model Vault Cohere's Model Vault now runs on hardware-isolated NVIDIA H100 enclaves on Azure, letting enterprises verify their data stays private even from Cohere. - Cohere launched Confidential Computing on Model Vault in public beta. - Inference runs inside hardware-isolated enclaves on NVIDIA H100 GPUs on Microsoft Azure. - Remote attestation lets clients verify the environment before sending any sensitive prompts. - Payloads stay encrypted end-to-end, decryptable only inside the attested confidential environment. - Targets regulated workloads: financial fraud, legal due diligence, public sector, source code, patents. - Available now via standard Cohere APIs at vault.cohere.com. Cohere has opened a public beta for confidential computing on Model Vault, its dedicated managed inference platform. Requests remain encrypted until they reach a hardware-isolated environment spanning the CPU and GPU, reducing the ability of cloud or Cohere administrators to inspect prompts and responses during model execution. The service combines Model Vault with NVIDIA H100 GPUs running in Microsoft Azure confidential virtual machines. According to Cohere’s launch post, applications can retain their existing Cohere APIs and request formats while adding client-side encryption and verification. Plaintext stays inside the enclave Cloud platforms routinely encrypt stored data and network traffic, but inference requires a model to process usable prompt data in memory. Azure’s confidential virtual machines extend a trusted execution environment across the VM and its attached H100 GPU, protecting that processing window with hardware-backed controls. Attestation becomes the gate Before an application sends a sensitive payload, customer-side software checks signed evidence from the confidential environment against an approved policy. A mismatch in the expected hardware, firmware, or software measurements causes the client to withhold the request. Attestation adds a technical control to contractual commitments and provider access policies. Its assurance depends on the measurements a customer approves, the integrity of the verification client, and the process used to review software updates when those measurements change. API compatibility, plus client work Cohere says the model endpoints and request schemas remain unchanged, which limits changes to application logic. Developers still need to integrate the confidential-computing flow around those calls: - Create an encrypted vault through the beta console. - Obtain attestation evidence from the target environment. - Validate that evidence against the approved policy. - Encrypt and send the standard Cohere API request after successful verification. - Receive the protected response and decrypt it inside the customer environment. This flow is most relevant to retrieval systems, agents, document search, and analysis pipelines that handle sensitive source material. Any component that records plaintext before encryption, including application logs, tracing systems, prompt stores, or retrieval middleware, remains outside the confidential boundary. Protection stops at the enclave The confidential boundary protects data during model execution and transport into that environment. Customers remain responsible for controls elsewhere in the system, including: - Application authentication and authorization - Prompt, response, and retrieval logging - Endpoint and client security - Data retention and residency policies - Compliance reviews and audit evidence - Model-output validation and abuse controls - Protection against authorized users misusing data Confidential computing also leaves model quality and output safety unchanged. It narrows the infrastructure threat model by restricting access to plaintext during inference. H100 makes GPU isolation practical NVIDIA’s H100 introduced hardware support for confidential GPU computing. Earlier trusted execution environments largely covered CPU workloads, leaving large generative models without the accelerator performance required for practical serving. Microsoft later made Azure H100 availability generally available, giving AI vendors a production infrastructure layer on which to build confidential inference services. Model Vault already provides dedicated managed deployments for enterprises. Adding attested execution addresses a separate concern: infrastructure operators with privileged access should be unable to inspect customer data while the model runs. The target workloads Cohere is positioning the beta for data that security policies, contracts, or regulation have kept away from managed model endpoints. Examples include: - Fraud investigations and financial records - Privileged legal documents and due-diligence material - Sensitive public-sector information - Patent applications and unpublished research - Proprietary engineering and architecture documents - Private source-code search and analysis Cohere’s place in the stack Microsoft has published a broader Azure design for confidential AI inference and demonstrated workloads such as Whisper on the same class of confidential H100 virtual machines. Cohere packages those infrastructure primitives into its existing enterprise inference service, with standard Cohere endpoints and customer-side attestation. The product’s dedicated deployment model aligns with Cohere’s focus on banks, telecommunications companies, governments, and other organizations that require stronger isolation and auditability than conventional managed endpoints provide. Open questions for production Cohere’s beta announcement leaves several deployment details for customers to establish during evaluation: - Supported models, Azure regions, and data-residency options - Latency and throughput overhead from encryption and attestation - Pricing, quotas, service-level commitments, and support terms - Attestation policy ownership and measurement-update procedures - Key-management and recovery responsibilities - Audit artifacts available for regulatory reviews The strongest fit is managed inference over data whose policies require controls against provider and infrastructure access. The beta addresses confidentiality during model execution; application security, governance, operational performance, and cost remain separate production-review items.
document fetched 2026-10-24 15:22