There is a quiet category mistake running through the AI training-data conversation, and it is costing the industry the next two years.
The mistake is treating annotation, expert routing, and judgement authoring as if they were the same product at different price points. They are not. They are different layers of the stack, doing different work, producing different outputs, and the layer the frontier labs actually need next is the one almost nobody is building.
I want to be precise about this, because the imprecision is where the money is being wasted.
What Annotation Actually Is
Annotation is labelling. A worker is shown an artefact (an image, a string of text, a model output) and asked to tag it, rank it, or choose between options against a rubric somebody else wrote. Scale AI built a multi-billion-dollar business on this. Surge AI refined it for language. Outlier and Invisible run variants of the same model with different quality tiers and recruitment strategies.
Annotation works. It is how the last decade of supervised learning happened, and how the RLHF layer underneath every frontier chatbot was tuned. If you need a million labels at high speed and reasonable quality, the annotation market is a real market and a useful one.
But annotation has a ceiling, and the ceiling is structural, not financial. The activity sits at the bottom three rungs of Bloom's taxonomy. Remember. Understand. Apply. A labeller recognises, distinguishes, and routes. They do not analyse, evaluate, or create. They are responding to a prompt written by somebody else against a rubric written by somebody else.
This is why throwing more annotation budget at frontier judgement problems produces diminishing returns. You are paying more people to do the same cognitive activity. You are not buying access to a different kind of thinking.
What Expert Routing Is, and What It Is Not
Mercor is the cleanest example of the second category. The thesis is sound: not all human input is equal, so let us match AI labs to vetted individual experts for paid task work and interviews. The marketplace adds a curation layer the annotation firms do not.
Expert routing is a real improvement over generic annotation. The output quality lifts. The questions you can ask get more sophisticated. The conversations get closer to the boundary of actual human cognition.
What it does not produce is a corpus.
Marketplaces route individual hours. They do not author a structured, persistent body of human judgement that an AI lab can fine-tune against, evaluate against, and return to. Every engagement is a fresh transaction. There is no compounding asset. The knowledge leaves with the expert at the end of the call.
For an AI lab building a long-running model that needs to reason across thousands of judgement scenarios on inference, that is the wrong shape of supply.
What Expert Judgement Authoring Is
There is a third category, and it is the one the frontier needs next.
Expert judgement authoring is the activity of a senior practitioner writing original scenarios, original reasoning, and original grading rubrics inside a structured corpus that an AI model can train on and be evaluated against. Not labelling somebody else's prompt. Not answering somebody else's interview question. Authoring the prompt, the response, the alternative responses, and the rubric in their own voice, against their own twenty years of judgement.
This sits at the top three rungs of Bloom. Analyze. Evaluate. Create. The activity is synthesis, not selection.
The output is also different. An annotation session produces labels. An expert engagement produces deliverables for one buyer. A judgement authoring session produces a permanent, licensable asset that any AI lab can train on, evaluate against, and cite. The contributor's judgement becomes an addressable node in a corpus, not a one-time payment.
This is what GLEAC has been building since 2018. The Judgement Corpus contains 250,000 authored records across 10 human-skill domains and all 5 Bloom's levels, written by 2,500+ senior operators, advisors, executives, and domain experts in 90 countries. Every entry is original. Every entry is attributable. Payment is guaranteed for micro-practices scoring above 8/10 completed without any AI usage: $100 per year for up to 10 years per qualifying third-party dataset licence; no separate revenue share applies.
Why This Is the Next Layer
Three structural reasons authored expert judgement is the layer frontier labs need now.
1. Provenance
The legal exposure on contested training data is no longer theoretical. Publishers, image archives, code platforms, and rights holders have moved from lawsuits-as-noise to lawsuits-as-strategy. The moat for a frontier model in 2027 will be the ability to show, on paper, where every piece of judgement-layer training data came from, who authored it, and under what licensing terms.
Annotation pipelines, by design, blur this. The worker is anonymous. The prompt source is often scraped. The chain of custody from raw artefact to fine-tuning dataset is hard to audit. Expert judgement authoring inverts the default. Every entry has a named contributor, a signed contract, and a clean licence. That is a structural advantage that compounds with every regulatory cycle.
2. Cognitive depth
Frontier labs have largely solved Remember, Understand, and Apply. The next generation of model value, the part the market is willing to pay for, is at Analyze, Evaluate, and Create. Diagnosis. Strategy. Original synthesis. Boundary judgement.
Those rungs cannot be annotated. You cannot have a worker on piecework rates tag an artefact for "good strategic judgement." The judgement is not in the artefact. The judgement is in the head of the person who would have produced the artefact. Authoring is the only way to get it onto the training distribution.
3. Contributor continuity
Annotators churn. Marketplaces re-source. Both are designed around interchangeable supply.
Authored corpora are designed around persistent contributors with skin in the game. A community member whose micro-practice scores above 8/10 and was completed without any AI usage is guaranteed $100 per year for up to 10 years when a qualifying data-lake dataset is licensed to a third party. That defined incentive aligns contributors to author more and better; no separate revenue share applies.
What This Means If You Are Buying
If you are a research lead at a frontier lab, a head of data at an agent company, or running model evaluation at any organisation that ships into regulated environments, the practical implication is straightforward.
Keep the annotation contract. You still need labels at scale.
Keep the expert marketplace. You still need individual depth for one-off calls.
But add the third layer. A licensed corpus of authored expert judgement, with documented provenance, top-of-Bloom cognitive depth, and a contributor base eligible for defined payments when qualifying datasets are licensed. That is the layer your next model release will be measured on, and it is not the same product as the first two.
GLEAC's entry point is a fixed-fee 14-day pilot. We deliver 80 fine-tuning examples in the Judgement and Decision Making skill. You write 50 evaluation scenarios. Both the baseline and the fine-tuned model are scored under blind human preference rating. The success bar is a 15 percentage-point or greater preference lift. If we do not clear it, you do not graduate to the licence.
Direct outreach: sallyann@gleac.com.
Frequently Asked Questions
Is GLEAC competing with Scale AI or Surge AI?
No. Scale and Surge sell annotation labour for labelling and RLHF pipelines. GLEAC sells an authored corpus of original expert judgement, written by named senior practitioners at the top of Bloom's taxonomy. Most serious AI programmes will buy from both categories. They solve different problems.
Is GLEAC competing with Mercor?
No. Mercor routes individual experts to AI labs for paid task work and interviews. The output is hours and deliverables. GLEAC produces a licensable corpus of authored judgement. Different supply shape. Different unit of value. Buying one does not replace the other.
Why does authoring work where annotation does not?
Annotation activity sits at the bottom three rungs of Bloom (Remember, Understand, Apply). Authoring sits at the top three (Analyze, Evaluate, Create). Frontier labs need data at the top three. The labour pool, the workflow, and the unit economics required to produce that data are entirely different.
Who authors the Judgement Corpus?
2,500+ vetted members of the Lovely Humans community across 90 countries. Founders, executives, lawyers, doctors, organisational psychologists, behavioural scientists, board directors, designers, military officers, educators, coaches. Every contributor is application-vetted; payment is guaranteed for micro-practices scoring above 8/10 completed without any AI usage: $100 per year for up to 10 years per qualifying third-party dataset licence, with no separate revenue share.
How do I run a pilot?
Email sallyann@gleac.com or book a 15-minute conversation at calendly.com/callwithsal-gleac. The standard pilot is 14 days, fixed fee, with a 15 percentage-point preference lift as the graduation bar.
