Cohere’s UK subsidiary started an on-premises pilot this week that embeds City law-firm matter files into a private vector index, according to statements of work circulated to litigation partners and reviewed by InfoHandle. The engagement, with a Magic Circle alumni firm that InfoHandle agreed not to name because of client confidentiality clauses, targets seven decades of scanned pleadings and due-diligence binders that partners refuse to upload to US-hosted retrieval APIs.
What the appliance actually does
Cohere shipped a rack-mounted GPU server to the firm’s Bishopsgate archive floor running its Command embedding model weights locally. Nightly jobs OCR newly closed matters, chunk text into 512-token windows, and write vectors to an OpenSearch cluster that stays on the firm’s LAN. Associates query through an Outlook sidebar plugin that returns paragraph citations with Bates numbers, not generative summaries that could hallucinate holdings.
The statement of work caps model retraining on firm data: weights are frozen; only adapters for practice-group vocabulary are allowed. That constraint came from risk partners who watched US litigation over training-data provenance.
Why on-prem beat the cloud pitch
The firm’s clients include sovereign wealth funds and UK regulated utilities whose outside counsel guidelines prohibit transatlantic processing of unpublished deal terms. Even Microsoft 365 Copilot deployments with EU Data Boundary still route some telemetry through US analytics pipelines the firm’s information barrier committee distrusts.
Cohere’s London enterprise lead argued that on-prem embeddings let the firm offer semantic search to barred partners without crossing ethical walls—each practice group gets a separate index namespace with hardware-level VLAN isolation.
Evaluation and what nobody has verified
The UK AI Safety Institute has not reviewed this deployment; the firm relied on internal bias tests comparing embedding clusters for employment versus banking matters. Early results show clustering by governing law rather than partner name, which matters for anti-discrimination audits.
Cohere published benchmark scores for legal retrieval on public MTEB leaderboards, but those scores cover US case law. The firm’s knowledge management team ran a blind review: associates found correct clauses in 82 percent of synthetic queries against historic UK acquisition agreements—good enough for diligence prep, not for court filings without human read-through.
Regulatory hooks this month
ICO guidance on international transfers still pushes firms to document supplementary measures when using US AI vendors. Keeping embeddings in a basement rack is the most blunt measure, though GDPR accountability remains: the firm must log who queries which matter numbers.
Solicitors Regulation Authority technology principles due for consultation this autumn ask whether AI search tools require explicit client consent in engagement letters. The pilot firm is updating templates to mention semantic search on archived PDFs.
Competitive landscape
Harvey AI and Thomson Reuters CoCounsel sell hosted legal assistants; iManage and NetDocuments added vector search in cloud tenants. Cohere’s bet is that Magic Circle and silver-circle firms with fortress archives will pay capex for air-gapped search rather than negotiate data processing agreements with another US SaaS vendor.
Open-source alternatives using Llama embeddings were trialled and rejected after IT calculated GPU support costs exceeding Cohere’s three-year licence.
What associates feel in practice
Trainee solicitors used to keyword-hunt “change of control” across SharePoint trees; the plugin returns ranked hits from 1990s fax scans with redacted counterparties. Partners warned that speed gains must not skip human privilege review—every result set exports to a review log for ethics partners.
Facilities teams had to upgrade basement cooling because the GPU server dumps 6 kW into an aisle previously used only for paper storage.
Commercial terms and what happens if it fails
The pilot is priced as a six-month proof with an option to expand to Frankfurt and Singapore archive rooms if UK success criteria hit. If recall rates fall below 75 percent on banking matters, the firm can return hardware without penalty—a clause Cohere accepted to win the logo.
For Cohere’s UK push, the City pilot is a reference story in a market sceptical of generative hype: embeddings without chat reduce hallucination risk while still answering the question partners ask—where did we last negotiate a MAC clause like this? Whether that story scales beyond one basement depends on associates trusting citations more than they trust memory, and on regulators accepting that not every AI workflow needs a chat box.
