# What Is AI Distillation? The Copying Fight Behind GPT-6 Astra, Explained

URL: https://whatisagenticai.net/blog/what-is-ai-distillation/
Published: 2026-10-29
Publisher: Little Tough LLC, maker of AI Agent Assistant: Agentic AI (https://apps.apple.com/us/app/ai-agent-assistant-agentic-ai/id6761501260)

AI distillation trains a small model on a big model’s answers. What it means, why OpenAI and Anthropic call some of it an attack, and the GPT-6 Astra report.

On October 28, 2026, a report said two Chinese AI labs built their newest models by learning from OpenAI’s GPT-6 Astra. The word in every headline was "distillation." If you have never heard it used about AI, it sounds like something from a whiskey factory.

It is actually a simple idea, and a very old one in AI terms. This guide explains what AI distillation means, why it is normal in one setting and called an "attack" in another, and what the Astra report does and does not prove.

**TL;DR:** AI distillation is training a smaller "student" model on the answers of a bigger "teacher" model, so the student learns to act like the teacher at a fraction of the size and cost. Labs do it to their own models all the time, and it is legal. It becomes a fight when one company quietly collects another company’s model outputs at scale to train a rival, which breaks the big labs’ terms of service. OpenAI and Anthropic call that a "distillation attack." On October 28, The Information reported, citing unnamed sources, that DeepSeek and Moonshot AI used GPT-6 Astra this way. Neither company had responded.

## What is AI distillation?

Picture a top chef and a new cook. The new cook does not go to culinary school for ten years. They just watch the chef plate hundreds of dishes and copy the results until their own plates look almost the same. That is distillation.

In AI, the "chef" is a large, expensive model, called the **teacher**. The "new cook" is a smaller model, called the **student**. You ask the teacher thousands or millions of questions, save its answers, and train the student to give the same answers. Anthropic’s own one-line definition is "training a less capable model on the outputs of a stronger one."

The idea is not new. The well-known paper that named it, "Distilling the Knowledge in a Neural Network," came from Geoffrey Hinton, Oriol Vinyals and Jeff Dean at Google in 2015.

## AI distillation explained: how it works, step by step

- **Pick a teacher.** A strong model that already does the job well.

- **Collect its answers.** Send it a huge, varied set of prompts and save every reply. Some methods also save the teacher’s step-by-step reasoning, which is very valuable.

- **Train the student.** Feed those prompt and answer pairs into a smaller model until it learns to respond the same way.

- **Test and ship.** The student is checked against the teacher. If it is close enough, it goes live, often as the "mini," "flash" or "lite" version of a model.

Why bother? A smaller model is faster and much cheaper to run. A classic example is DistilBERT from Hugging Face in 2019: its authors reported it was 40% smaller and 60% faster than Google’s BERT while keeping 97% of its language understanding.

## AI distillation vs fine-tuning vs quantization

These three words get mixed up a lot. Here is the difference in plain terms.

| Method | What it does | What it learns from | Typical goal |
| --- | --- | --- | --- |
| Distillation | Trains a new, usually smaller model to copy a bigger one | Another model’s answers | A cheaper model that acts like the expensive one |
| Fine-tuning | Keeps training an existing model on extra data | Your own examples, from people or from a model | Teach a model a specific job or style |
| Quantization | Stores the same model’s numbers with less precision | Nothing new, no extra training needed | Shrink the model so it fits on smaller hardware |

The lines can blur. Fine-tuning a model on another model’s answers is, in practice, a form of distillation. That is exactly why the practice is so hard to police. See also [fine-tuning in our glossary](https://whatisagenticai.net/agentic-ai-glossary/).

## What are AI distillation attacks?

A distillation attack is the same technique done without permission. Instead of distilling its own model, a company opens many accounts on a rival’s chatbot or API, sends it millions of carefully chosen questions, and uses the answers to train its own model. The rival spent huge sums to build the teacher. The copier pays only for the questions.

Two public reports show what this looks like:

|  | Anthropic report | OpenAI report |
| --- | --- | --- |
| Published | February 23, 2026 | September 30, 2026 |
| Model targeted | Claude | OpenAI’s reasoning models |
| Scale | More than 16 million exchanges through about 24,000 fake accounts | 16,000 requests from more than 4,000 users on July 24 and 25 alone, inside a wider cluster of over 15,000 users |
| Who was named | DeepSeek (150,000+ exchanges), Moonshot AI (3.4 million+), MiniMax (13 million+) | A "core cluster" tied to people associated with Moonshot AI |
| What they were after | Agentic coding, tool use and reasoning skills | The hidden reasoning OpenAI models produce before answering |
| Response | Detection classifiers, account checks, sharing signals with other labs | Banned accounts, tighter sign-up checks, findings shared via the Frontier Model Forum |

OpenAI said the campaign ran from July 1 to July 28. The operators copied encrypted reasoning from one chat and asked a model in a different chat to decode it. OpenAI said they did not break its encryption or reach stored user conversations.

## The GPT-6 Astra distillation report

On October 28, 2026, The Information reported that DeepSeek and Moonshot AI relied heavily on outputs from [GPT-6 Astra](https://whatisagenticai.net/blog/what-is-gpt-6-astra/), OpenAI’s flagship model, to train their newest releases. Reuters and others carried the story. The two models named were Moonshot’s [Kimi K3.1 Pro](https://whatisagenticai.net/blog/what-is-kimi-k3-1-pro/), released October 23, and DeepSeek’s V4.1-Flash 1017, released October 17.

Keep three things in mind:

- **It is a report, not a ruling.** The Information cited people with direct knowledge of the matter. No documents or named sources were made public.

- **No response yet.** OpenAI, DeepSeek and Moonshot AI did not immediately respond to Reuters’ requests for comment.

- **It fits a pattern.** It follows OpenAI’s September 30 post and Anthropic’s February report, which named the same two labs.

If you use those models, nothing changes overnight. They still work. The open question is whether US labs push for tougher rules on who can use their models, and how.

## Is AI distillation illegal?

Distillation as a method is legal and common. Labs distill their own big models into small ones, and anyone can distill an open model whose license allows it.

The trouble starts with someone else’s closed model. The major US labs’ terms of service forbid using their outputs to build competing models. So a distillation campaign is, at minimum, a contract violation that gets accounts banned. OpenAI’s Caroline Zier told Bloomberg the company’s concern is "violation of our terms of service, not open models or legitimate distillation." Whether it also breaks any law is a separate and unsettled question, and no court ruling on it has been reported in these cases.

## What this means for you

For everyday users, distillation is mostly good news. It is why fast, cheap models exist at all, and why AI keeps getting better on phones. The fight is about who gets to profit from a teacher model, not about whether your answers are safe.

The bigger lesson is that the "best model" changes every few weeks, and some of the cheap ones may be partly copies of the expensive ones. You should not have to track that. An agent app on your iPhone lets you hand off the task, plan a trip, draft an email, research a purchase, and get it done, without becoming an expert in model family trees. To see how agents work under the hood, start with [what agentic AI is](https://whatisagenticai.net/what-is-agentic-ai/).

## Quick answers

**What is AI distillation in simple terms?** Teaching a small AI model by having it copy the answers of a big one. The big model is the teacher, the small one is the student.

**What does distillation mean in AI models?** It means compressing what a large model knows into a smaller model by training the smaller one on the large model’s outputs, so it is cheaper and faster to run.

**Is AI distillation illegal?** The technique itself is legal. Using a rival’s closed model outputs to train a competitor breaks the terms of service of OpenAI, Anthropic and other major labs. Whether it breaks a law is not settled.

**What is an AI distillation attack?** A coordinated campaign that uses many accounts to pull huge numbers of answers from someone else’s model, without permission, to train a rival model.

**Did DeepSeek distill GPT-6 Astra?** The Information reported on October 28, 2026, citing unnamed sources, that DeepSeek and Moonshot AI relied heavily on Astra outputs. The companies had not commented at the time, and no public evidence has been released.

**AI distillation vs fine-tuning?** Distillation trains a model to copy another model. Fine-tuning keeps training a model on extra examples for a specific job. Fine-tuning on a model’s answers is a kind of distillation.

**Sources:** [Anthropic: Detecting and preventing distillation attacks](https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks) · [The Next Web on OpenAI’s September 30 report](https://thenextweb.com/news/openai-moonshot-distillation-campaign-hidden-reasoning) · [Decrypt](https://decrypt.co/379890) · [Reuters via TradingView](https://www.tradingview.com/news/reuters.com,2026:newsml_L6N46F02M:0-deepseek-moonshot-used-openai-s-gpt-6-astra-to-train-latest-models-the-information-reports/) · [Investing.com](https://www.investing.com/news/stock-market-news/openais-astra-used-by-chinese-ai-firms-to-train-models-the-information-93CH-4981331) · [Crypto Briefing](https://cryptobriefing.com/moonshot-deepseek-openai-astra-models/) · [Hinton, Vinyals, Dean (2015)](https://arxiv.org/abs/1503.02531) · [DistilBERT paper](https://arxiv.org/abs/1910.01108)