Compare
Annotation, expert routing, and authored judgement are three different products solving three different problems. The frontier teams who get this right buy from more than one of these categories. The teams who do not, overpay for the wrong layer.
Below is a category-by-category comparison, written from GLEAC's perspective but stated factually. Scale AI, Surge AI, and Mercor are excellent at what they do. They are not what GLEAC does.
The quick take
Annotation (Scale, Surge, Outlier, Invisible)
Labels and ratings at the bottom three rungs of Bloom. Volume, throughput, and consistency. The right tool for supervised learning and RLHF preference data at scale.
Expert routing (Mercor)
Vetted individual experts matched to AI labs for paid task work and interviews. The right tool when you need a specific human in the loop for a specific task.
Authored judgement (GLEAC)
A persistent licensable corpus of original human judgement written by named senior practitioners at the top of Bloom. The right tool for fine-tuning and evaluating models on judgement, strategy, and original synthesis.
Most frontier teams use all three
The categories are complementary, not substitutes. Buy annotation for volume, expert routing for one-off depth, and an authored corpus for the permanent judgement layer your model is measured on.
Side by side
| Dimension | GLEAC / Lovely HumansAuthored judgement corpus | Scale AIAnnotation at scale | Surge AIAnnotation, language-focused | MercorExpert routing marketplace |
|---|---|---|---|---|
| What they sell | A licensable corpus of original human judgement, authored by named senior experts. | Annotation and RLHF labour at industrial scale, run through trained crowd workers. | Higher-quality language annotation and human feedback, run through vetted contributors. | A marketplace that matches AI labs with vetted individual experts for paid task work and interviews. |
| Primary use case | Fine-tuning and evaluating frontier and agent models on top-of-Bloom human judgement. | Supervised data labelling, RLHF preference data, model evaluation at high volume. | RLHF, instruction-tuning data, and language quality assurance. | On-demand expert hours for task work, interviews, and short engagements. |
| Unit of output | A persistent, licensable corpus node, citable across every future training and eval run. | Labels, ratings, preference pairs delivered by SKU. | Labels, ratings, written responses delivered by SKU. | Hours of expert time per engagement, plus deliverables. |
Based on publicly available product descriptions and category positioning at the time of writing. Vendors evolve. Confirm specifics with each provider before procurement.
For agent companies
Every agent team hits the same wall. The agent works in demos, then meets production, and the only way to keep quality up is to put a human in the loop. That human is now the bottleneck. Latency goes up, headcount goes up, queues form, and the agent's economics break.
With a licensed GLEAC corpus inside the agent's reasoning loop, the agent reaches for a suggested human judgement first, written by a senior practitioner who already authored the answer for that class of scenario, and only escalates to a live human when the situation is genuinely outside the corpus.
The licensed corpus becomes the first reviewer. Live humans become the exception, not the bottleneck. Throughput goes up, cost per decision goes down, and your agent finally scales without losing the judgement quality it needed humans for in the first place.
Per-vendor view
What they do well
Scale AI is the category leader in annotation at industrial scale. If you need millions of labels delivered on a programmatic SLA across image, text, audio, sensor, or video, Scale has the contracts, the workforce, and the tooling. They also run substantial defence and government programmes.
Where GLEAC sits
Scale's output is labels and preference pairs delivered by SKU. GLEAC's output is a licensable corpus of authored judgement. Different unit of value. Most teams that license GLEAC also keep their Scale contract for the volume layer underneath.
What they do well
Surge AI pushed annotation quality up by recruiting a smaller, more curated contributor pool and focusing on language. They are a strong default for RLHF preference data, instruction tuning, and language quality assurance, especially when the artefact is closer to natural prose than to a labelled image.
Where GLEAC sits
Surge's workflow is typically organised around labelling and rating against rubrics provided by the buyer. GLEAC's workflow is organised around contributors authoring the prompt, the response, the alternative responses, and the rubric themselves. The two activities produce different shapes of data, and the difference matters most when the target task sits at Analyze, Evaluate, or Create on Bloom.
What they do well
Mercor solved a real coordination problem: AI labs need access to vetted individual experts on demand, and most experts do not want to negotiate one-off contracts. Mercor turned that into a marketplace with clean matching, interviews, and short engagements.
Where GLEAC sits
Mercor sells hours. GLEAC sells a corpus. A Mercor engagement ends when the call ends. A GLEAC contribution becomes a permanent, citable node that any future model can train and evaluate against. Most labs that take judgement seriously will buy both, for different needs.
How to choose
If you need millions of labels at predictable cost and throughput
Start with Scale AI or Surge AI. Annotation marketplaces are the right tool for volume work at the bottom three rungs of Bloom.
If you need a specific expert for a specific task this week
Use an expert marketplace like Mercor. The unit economics are designed for one-off depth.
If your model is being measured on judgement, strategy, or original synthesis
License an authored judgement corpus. GLEAC is the layer purpose-built for top-of-Bloom capability.
If you need a permanent, citable training and eval asset, not a stream of labour
License a corpus. Annotation programmes and expert marketplaces both produce labour. Only a corpus produces a permanent, addressable node you can return to every training run.
If provenance and audit-ready licensing matter
Choose a vendor with named contributors, signed contracts, and clean chain of custody. Anonymous workforces work for many things; they do not work for regulatory and procurement contexts where you have to show where every entry came from.
Frequently asked
14 days. 80 fine-tuning examples in Judgement and Decision Making, 50 evaluation scenarios you write, scored under blind human preference rating. Graduation bar: a 15 percentage-point or greater preference lift.