Open-Weight AI Runs on the Honesty Box
Consider a quietly subversive question: a hedge fund downloads an open-weight model, isolates it inside its own infrastructure, and uses it to analyse proprietary positions. Its licence says commercial use above one million dollars of annual revenue requires an enterprise agreement. Not one inference request ever returns to the model’s creator. How would anyone find out? That question sits underneath the fastest-growing distribution channel in AI, and the answer says something uncomfortable about how “open” model economics actually work.
Start with the muddle the word “open” now conceals – I have conflated these categories myself, and I study them for a living. xAI’s Grok-1 shipped under Apache 2.0, a genuinely permissive open-source licence with no commercial ceiling. Stability AI’s Community License is free only until an organisation crosses one million dollars in annual revenue, at which point the licence terminates and an enterprise agreement is required. Meta’s Llama licence is free until a company exceeds seven hundred million monthly active users. The Open Source Initiative’s definition of open-source AI requires the freedom to use a system for any purpose without asking permission – a test the revenue- and scale-gated licences fail, which is why “open weight” is the honest description for most of them.
The deeper problem is architectural. Cloud software polices itself: the supplier operates the servers, meters every call and can suspend an account in minutes. Downloadable weights reverse the visibility. The customer takes possession of the productive asset on day one; it can run air-gapped forever; the creator sees nothing. Enforcement by telemetry is simply unavailable.
Hard to detect is not the same as legally binding, however – and here the law adds a twist. In the Indiana Law Journal, Peter Henderson and Mark Lemley argue that AI terms-of-use restrictions may themselves be built on sand: model weights and outputs are largely not copyrightable, which leaves it unclear whether there is anything to license, and beyond account suspensions no model creator has yet pursued monetary penalties or injunctions to test the terms in court. Yet enterprises should take little comfort. They leak provenance through every channel that matters commercially: procurement records, model registries, vendor due diligence, employee disclosures, M&A data rooms, and downstream distribution. The practical enforcement mechanism for open-weight licences is not packet inspection but paperwork – the compliance finding in a funding round or acquisition that suddenly prices years of quiet usage.
The strategy follows for both sides of the transaction. Model providers face a genuine choice: permissive licensing maximises adoption; revenue-gated terms capture enterprise value but surrender observability the moment the weights leave the building – so design licences for the customer’s compliance department rather than for a courtroom, with clean thresholds, self-certification, and an easy conversion path to enterprise terms. Enterprise leaders should treat model licences the way software leaders learnt to treat open-source code two decades ago: maintain a registry of every model in use, record each licence and its revenue or scale trigger, and gate procurement and deal diligence on it – because the real cost of non-compliance is rarely the licence fee, it is the diligence finding. Open-weight AI remains a curious commercial product. The customer receives the valuable asset first. The invoice depends partly on what they later admit doing with it.
Personal analysis. Views are mine, not my employer's.