When Does Distilling GPT Become Theft?

The law of distillation: How API scraping and open-weight model provenance expose developers to trade secret claims and the exact steps to protect your AI stack. explained for builders.

Jul 23, 2026

TL;DR:

Training a model on a competitor’s outputs, whether you call it distillation, synthetic data, or fine-tuning, is ordinary engineering. So what creates the legal exposure? How you acquire the outputs defines the legal picture here. Copyright law does not work here due to human authorship requirements. What makes this question interesting is trade secret law, which reaches exactly what copyright terms cannot: it moves the case from what you built to how you got it, so the terms you clicked and the way you accessed the model decide your legal exposure. Some of this is settled, some is contested, and this piece is clear about which is which. If you are training on a competitor’s outputs, feeding your model another model’s outputs, or building on an open-weight model whose provenance you cannot vouch for, this piece is for you.

Legal Layer Angle.

In this piece I will talk about what a developer training on another’s models’ outputs needs to know: why copyright is the weak claim, how the anti-distillation terms they clicked can flip reverse engineering into misappropriation, and the checkpoints to decide whether they are exposed.

In early 2025, DeepSeek released an AI model that matched the best American systems and said it had trained the thing for about 5.6 million dollars, against the roughly 100 million dollars OpenAI spent on ChatGPT. When the market worked out what that implied, Nvidia lost 589 billion dollars in a single day, the largest single-day wipeout any public company has ever taken in stock market history. This historic 17% plunge was triggered by the release of R1 by DeepSeek, whose highly efficient, low-cost AI model raised Wall Street concerns about future corporate spending on Nvidia’s expensive hardware.

Almost immediately, OpenAI accused DeepSeek of training its model on OpenAI’s own outputs. Part of what fed the suspicion came from DeepSeek’s own chatbot, which, asked how it was made, would sometimes point users to OpenAI’s interface. The shortcut has a name, distillation, and the same accusation now hangs over much of the industry.

Here is why it reaches past those two labs and lands on you. A version of what DeepSeek is accused of is something ordinary teams do every week:

fine-tuning on a competitor’s outputs,
generating synthetic training data from another model,
or distilling a large model into a small one.

So let’s set DeepSeek aside for a moment. The question worth your attention is whether the thing you may already be doing is legal, and the answer turns on details almost no builder has checked.

Which is why the framing everyone is using is the wrong one. The public fight treats distillation itself as the thing to judge, guilty or innocent. But learning from another model’s outputs is ordinary engineering. Geoffrey Hinton’s group described the teacher-student method in 2015, and every serious lab has used some form of it since. The technique cannot be what makes conduct unlawful. What makes it unlawful is how you got the outputs and whether you were allowed to hold them for the purpose you held them for. The whole legal fight lives in how the outputs were acquired, well before the training run begins.

The accusation

In February 2026, OpenAI told the House Select Committee on China that a rival was free-riding on models it had spent billions to build and that it had detected obfuscated methods designed to hide when its outputs were being harvested. Anthropic’s own disclosure put numbers on the same pattern: roughly 24,000 fraudulent accounts and more than 16 million exchanges aimed at pulling out Claude’s reasoning, coding, and agentic capabilities. The coverage stopped at the accusation. The law starts one layer down, at what turns output harvesting from permitted into actionable, and that layer is what this piece is about.

Reverse engineering with an edge

Reverse engineering is a right. The Supreme Court said so in Kewanee Oil in 1974, treating the practice of starting with a lawfully obtained product and working backward to its secrets as a fair and honest means of acquisition. Trade secret law has protected it ever since, and the Defend Trade Secrets Act writes the protection into federal law by excluding reverse engineering from its definition of improper means. That means you can buy a competitor’s product, take it apart, and the secrets you find are yours to use.

This privilege also has an edge. It protects work that begins with a product you possess free of restriction. A contract can move that edge. Courts enforce terms that forbid reverse engineering, and they let those terms override the background privilege.

ProCD v. Zeidenberg upheld a shrinkwrap license restricting the use of data that copyright alone would not have protected.
Bowers v. Baystate went further, holding that a contractual ban on reverse engineering was enforceable and not swept aside by federal copyright law.

The lesson both cases teach is that the privilege was never a license to ignore the terms under which you were given access.

So, where is the line?

Here is the position I will defend: a knowing breach of an accepted anti-distillation term defeats the reverse-engineering privilege, and access obtained through circumvented controls turns the conduct into misappropriation by improper means.

Put plainly: the more the extraction looks like something you agreed not to do and hid while doing, the less it looks like reverse engineering and the closer it gets to theft.

Three moves hold that position up.

The first is that API access under anti-distillation terms is possession with strings, a different thing from buying a boxed product and taking it home. When you call a model through an API, you never possess the product free of restriction. You hold a license to use outputs on stated conditions, and every major provider, OpenAI, Anthropic, Mistral, and SpaceXAI, states a condition barring you from using those outputs to build a competing model. Reverse engineering protects the person who owns the thing outright. It was never written for the person who accepted terms and then broke them.

The second is that the federal definition of improper means already reaches this conduct. The Defend Trade Secrets Act lists breach of a duty to maintain secrecy and espionage through electronic means among the improper means of acquiring a secret. Fake accounts, shared credentials, defeated rate limits, and routers that mask a query’s origin sit at the legal heart of the DeepSeek story. The same statute that carves out honest reverse engineering condemns acquisition that hides itself.

The third is that the closest thing we have to a decided case points the same way. In Compulife v. Newman, a competitor pointed bots at a public insurance-quote engine and pulled tens of millions of quotes, far past what any human could gather, to rebuild the database behind it.

So what matters is, even though each individual quote was free to obtain, the compilation stayed a trade secret because it was not practically obtainable by legitimate means without that massive automated effort. Scale and method turned lawful single requests into misappropriation, and on remand the liability for the scraping was found and affirmed on appeal in 2024.

The court reached back to E.I. duPont v. Christopher, the 1970 case that treated aerial photographs of a plant under construction as an improper mean even though flying over it broke no law, for the principle that a method that defeats the normal limits on access can be improper on its own. That is the instinct a court carries into a case about 16 million exchanges run through obfuscated routers.

Compulife is not a clean fit, and the gap is worth stating plainly, because the strongest argument against everything I have just said lives in it. Compulife concerned a structured factual database, where the compilation itself was the secret and the scraper was rebuilding that compilation record by record. Distillation does not work that way. You are not exactly reconstructing a dataset. It is more like estimating the probability distribution the model encodes, inferring from what a system does how the system behaves. Opposing counsel will say that inferring the workings of a product from its observable outputs is the definition of reverse engineering, the thing Kewanee protects and the DTSA carves out, and that calling it misappropriation collapses the distinction the statute deliberately drew. That argument is a good one. Nothing has tested it. Applying a database-scraping case to statistical extraction is an untested extension of trade secret law, and any lawyer telling you the outcome is settled is selling certainty that does not exist. What Compulife supports is narrower and still useful: volume and method can convert individually lawful requests into an actionable taking. Whether a court carries that from quote records to model weights is the open question this whole area turns on.

That analogy also pins down what the trade secret actually is. The trade secret is the model’s learned behavior, the weights, and the training distribution they encode. Capability in the abstract will not do, since no court protects a competitor’s general ability to be good at something. Here, the outputs are the extraction vector, the pipe the secret is pulled through, and the role the individual quotes played in Compulife while the compilation was the secret. Keeping that distinction straight is what matters.

When people tried to work out how DeepSeek actually did it, they kept landing on three scenarios, and the three sort themselves along the exact line that decides liability.

In the first, someone gains improper access to the model itself through espionage or stolen credentials, which is trade secret theft in its oldest form.
In the second, the model is queried at scale through its public interface during training, which breaks no lock but breaches the terms accepted to get the key, a contract problem that can harden into misappropriation once the scale and the concealment show intent.
In the third, the training runs on outputs already dumped and circulating on the open internet, gathered without touching the provider at all, which is the hardest of the three to reach with any claim.

Same technique in all three, three different legal fates, and the method of acquisition is the thing that separates them.

There is an irony in all of this. The same OpenAI that told Congress DeepSeek had free-ridden on its models is now, as of July 2026, a trade secret defendant itself. Apple has sued it, alleging that departing Apple engineers carried confidential hardware designs into OpenAI through an exploited authentication bug and prototypes brought to job interviews, on a scale Apple says discovery will only widen. OpenAI denies it. However that case resolves, it shows the law’s attention tracking the manner of acquisition rather than the identity of the taker. The accuser and the accused can be the same company because acquisition is the through-line of every one of these fights, and everyone building here sits somewhere along it.

From all of this comes a test you can run on your own conduct. Ask three questions, in order.

Did you acquire the outputs through access you were entitled to, without fake accounts, shared or purchased credentials, circumvented rate limits, or defeated technical barriers?
Did the terms you accepted permit training on those outputs, or did they carry an anti-distillation clause you clicked past?
Does the volume, automation, and any concealment show a purpose of reconstructing the model’s learned behavior, rather than incidental use?

If the first answer is no, you are likely in misappropriation, whatever the other two say, because the acquisition itself was improper.
If the acquisition was clean but the use was prohibited, you are standing on contested ground, and this is where I plant the flag: breach of an accepted term is enough to strip the reverse-engineering privilege, so prohibited use plus lawful access still exposes you, in contract at a minimum and plausibly in trade secret.
If acquisition was clean and the use was permitted, then, and only then, “everyone distills” may be a real defense.

Most builders who feel safe are relying on that third outcome while their facts sit in the second.

The strongest objection to this deserves a place, because the lawyers most likely to argue with you will reach for it in the first thirty seconds. It runs like this. What I have described is a breach of contract while wearing a trade secret costume. Congress wrote the reverse-engineering carve-out into the Defend Trade Secrets Act on purpose, a policy choice that keeps lawful-access study out of trade secret law, and letting a terms-of-service breach flip that same study into misappropriation lets private contract rewrite a federal statute through the back door. On that view the model owner has a contract claim and no more, and the trade secret theory is an overreach.

I take the objection seriously, and here is where it runs out. The carve-out protects reverse engineering of a lawfully acquired product held free of restriction, the bought good on the workbench that Congress had in mind. It was never a rule that any access dressed up as a study is immune. The same statute’s definition of improper means already includes breach of a duty to maintain secrecy and acquisition by electronic espionage, and both of those live in the statute itself, with no help from any contract.

So the real question is which statutory bucket the conduct falls into, honest reverse engineering or improper acquisition, and that turns on the facts in the test: whether access was clean, and whether the taking was concealed and run at industrial scale.

Where access was circumvented, the trade secret claim stands on the espionage prong on its own, with no need to borrow from contract law. Where access was clean but the accepted term forbade the use, the contract breach is doing real work, and the honest version of my position gets narrower there: the breach is strong evidence that the acquisition was not the arm’s-length study the carve-out was written to protect, enough to put the reverse-engineering defense in serious doubt, though a court could still rest on contract alone. That narrower claim is the one worth defending on my end.

The copyright claim

The instinct is to argue this as reverse engineering versus fair use, a copyright framework. The copyright frame is the weak one for the model owner, and saying so is the useful part.

Raw model outputs strain the human-authorship requirement, so it is unclear the owner holds a copyright in the outputs at all. Louis Tompros, who teaches intellectual property at Harvard Law, made the point plainly in the DeepSeek context: copyright protects human creations, so a provider has little recourse in copyright over what its own model produced. And the providers have undercut themselves further in their own terms, since OpenAI assigns the rights in output content to the user, which makes it awkward to then claim the user infringed by using it. A company cannot easily give you the outputs and sue you for having them.

Run the fair-use question from the other side, and it does not settle in the distiller’s favor either. Fair use is the defense the large labs lean on to justify training on the open web, and a distiller reaches for the same shield by reflex. The case law is split. In Thomson Reuters v. Ross, decided February 2025 in Delaware, Judge Bibas found that training a legal-research tool on Westlaw’s headnotes was neither transformative nor fair, because the point was to build a competing product from the material it meant to displace. Within months, two other courts cut the other way: Bartz v. Anthropic and Kadrey v. Meta both held that training a generative model on copyrighted works was highly transformative fair use, and Ross, a non-generative search tool, is now on appeal at the Third Circuit. So fair use is unsettled, and what settling has happened leans toward protecting training, which is thin comfort for a model owner hoping copyright will stop a distiller. It is no safe harbor for the distiller either, since Ross shows that competitive purpose can still sink the defense on the right facts. That is the fact your anti-distillation exposure turns on.

So fair use is close to a distraction. The claims with teeth are contract and trade secret, the two I spent the last section on. Assessing your own exposure, spend your worry on those, and set the copyright theory aside.

Which one are you?

Two readers have a real problem here, and neither is DeepSeek.

The first is building on a competitor’s outputs right now, telling themselves it is just distillation. The test above is for them.
The second is the builder who fine-tuned an open-weight model of uncertain provenance or shipped a product on top of one and has no idea what the upstream extraction looked like.

The harder truth for the second reader is that inherited provenance means inherited exposure. A model you did not build can still be a model you cannot prove is clean, and “we just downloaded the weights” is no answer to holding a secret that reached you through improper means. Trade secret liability can follow a downstream user who knew, or had reason to know, that the material was tainted.

If Lab A harvests OpenAI's data improperly but dumps it publicly, and Lab B trains on that open dataset, Lab B has a powerful "we didn't agree to OpenAI's ToS, nor did we hack them" defense. Proving Lab B knew or had reason to know the data was a trade secret obtained via improper means is an incredibly high bar in practice, especially if the data has been cleaned, mixed, and repackaged.

If you download an open-weight model from Hugging Face that was trained or fine-tuned by someone else, you inherit their license violations. For example, if a developer fine-tuned a model using “distilled Llama 3 outputs” in mid-2024, that model violates the original Llama 3 license.

If your startup builds on top of that fine-tune, you are running a “tainted” model. In trade secret law, a downstream user can be held liable if they knew or had reason to know that the intellectual property they are using was acquired via improper means. Ignorance of upstream licensing is rarely a successful defense in a corporate dispute.

Just because you can run the code on your own hardware does not mean you own the intellectual property free of restriction. If you are distilling from an open-weight giant, you either have to accept Meta’s branding “tax,” or risk a breach-of-contract suit that turns your “legitimate reverse engineering” into misappropriation.

The defensive posture is not complicated. Assume the terms you clicked bind you. Assume extraction at scale reads as intent, because Compulife says it can. Keep a provenance trail you would be willing to produce in discovery, which means knowing where every model and every training signal in your stack came from. And run the question in the mirror, because the same analysis you are afraid of is the one you want protecting your own outputs. If a competitor is quietly distilling you, the strength of your position depends entirely on terms you wrote and controls you enforced before it happened.

Two clauses

Everything above was simply analysis. Now let’s look at how these work on an actual contract. There are mainly two clauses: one to protect what you ship, one to push risk back up the chain when you build on someone else’s model.

The first goes in your own terms of service, so that if someone distills you, you are standing where OpenAI and Anthropic are standing, on a written prohibition.

Restricted uses. You may not use the Services, or any Output, to train, fine-tune, distill, or otherwise develop or improve any machine-learning model or competing product, including by using Output as training data or as a target for model distillation. You may not access the Services through automated means, shared or fraudulent credentials, or any method that circumvents access, rate, or usage controls. Any such use is a material breach and constitutes use of the Services by improper means.

The phrase about “improper means” is doing deliberate work here. It ties the contract to the trade secret standard so that a breach is also evidence of improper acquisition. Where it can still fail: a court may treat the clause as an unread term of adhesion or read it narrowly, and none of it helps if you cannot show the other side actually accepted it. Your acceptance flow matters as much as your wording.

The second clause goes in any agreement where you acquire, license, or fine-tune someone else’s model so that provenance risk sits with the party who actually knows the answer.

Provenance representation and warranty. Provider represents and warrants that the Model, including its weights and all training and fine-tuning data, was developed without use of any third party’s outputs, services, or data in violation of that third party’s terms of use or applicable trade secret law, and that Provider holds all rights necessary to grant the license herein. Provider will indemnify Customer against claims arising from breach of this representation.

This one can still fail because a warranty is only as good as the counterparty’s solvency and honesty, an indemnity from a thinly capitalized model vendor will not do, and a provider who does not know its own upstream provenance cannot give you a truthful representation anyway. You can use it to force the conversation and treat a provider who will not sign it as having answered your question.

How does Europe see it?

The European route to the same conduct runs through different doctrines, but there is a difference. The EU Trade Secrets Directive, like US law, treats reverse engineering as lawful acquisition, and, like US law, lets a contract take that permission away. So the contract analysis travels, but the training-data question does not travel so cleanly.

Europe runs it through the text-and-data-mining rules of the 2019 Copyright Directive. Article 4 lets anyone mine lawfully accessible works for any purpose, including commercial model training, but only where the rightholder has not reserved its rights, and a machine-readable reservation in robots.txt or metadata is enough to switch the exception off. So the European default flips toward an opt-out the model owner may already have exercised. Add the sui generis database right, which has no real US equivalent and which the same opt-out reaches, and the same extraction that is a contract-and-secrets problem in New York becomes a mining-and-reservation problem in Frankfurt.

One requirement in the European rules deserves the last word, because it brings the whole argument home. Article 4 covers lawfully accessible works only. It does not reach circumvented paywalls, defeated access controls, or fraudulent credentials. So even in the more permissive regime, the exception dies the moment acquisition turns improper. Both systems, for all their doctrinal distance, put the same question first: how you got the outputs. What you did with them comes second.

A Model Comparison on Terms of Service

Comparing Google’s Gemma terms of use to Meta’s Llama licenses reveals a fascinating, shifting tactical battle. Both companies have taken completely different legal routes to handle the exact same anxiety: preventing competitors from using their open-weight models to train rival systems.

However, Google’s strategy is split into two distinct eras depending on which generation of Gemma you are building on.

1. The Classic Gemma License (Gemma 1, 2, and 3)

If you are using earlier Gemma weights, you are bound by a highly restrictive custom document called the Gemma Terms of Use. This license uses a unique, highly aggressive mechanism to lock down distillation: the concept of “Model Derivatives.”

How Gemma Defines a “Model Derivative”

Google defines a “Model Derivative” incredibly broadly. It explicitly covers:

“...any other machine learning model which is created by transfer of patterns of the weights, parameters, operations, or Output of Gemma... including distillation methods... or methods based on the generation of synthetic data Outputs by Gemma.”

This definition creates a highly viral licensing loop:

Under the terms, raw text outputs are not considered derivatives on their own. You are legally allowed to distribute a dataset of Gemma’s translations under a permissive license (like Creative Commons).
The moment a third party uses that CC-licensed dataset of Gemma outputs to train or distill a new model to replicate Gemma’s performance, that new model suddenly qualifies as a “Model Derivative.”
Because it is now a Model Derivative, that new model is instantly subject to Google’s entire original Terms of Use and Prohibited Use Policy.

While Meta’s Llama 3.1+ allows distillation as long as you pay a branding tax (naming your model “Llama-[Name]”), Google’s classic Gemma license quietly forces any distilled model to inherit Google’s restrictive terms of use and safety policies entirely.

Gemma 4
Google completely rewrote its strategy when it released Gemma 4. Moving away from custom agreements, Google released Gemma 4 under a standard, commercially permissive Apache 2.0 license.

Apache 2.0
By adopting Apache 2.0, Google completely removed the custom legal guardrails protecting Gemma 4’s weights:

You are legally allowed to fine-tune, restructure, and distill Gemma 4.
Models trained on Gemma 4 outputs do not become “Model Derivatives” bound by proprietary Google restrictions.
Unlike Meta’s Llama 3.1+, Google cannot force you to put “Gemma” in your distilled model’s name.

Meta Llama vs. Google Gemma

If you are a developer looking to build a lean, proprietary model via distillation:

Llama 3.3 is a powerful teacher, but you must accept that you are building free marketing for Meta by renaming your product.
Gemma 2 or 3 represents a quiet compliance risk. Even if you get the training data cleanly, your final model may legally belong to Google’s “Model Derivative” classification, forcing you to inherit their safety audits and termination clauses.
Gemma 4 is currently the safest corporate harbor. Because of the Apache 2.0 license, you can cleanly extract, distill, and package your final model as a completely closed-source, proprietary asset with zero upstream strings attached.

Discussion about this post

Ready for more?