Work · Verification programme
Annotator, verification programme
The verification programme checks the entries that Ackren's answers are derived from. It pays fifteen pounds an hour, per verified hour rather than per item, so the incentive is not to rush the unsure cases. The quality regime is public before you apply.
The work
What you would do
- 01
Verify entries against the guideline
You check lexicon and qualia entries: whether a sense is right, whether a relation holds, whether an example supports it. These are the parts of the knowledge base that are safe to verify at scale.
- 02
Mark the unsure, and be paid for it
'Unsure' is a first-class answer, not a failure. Every unsure item goes to expert adjudication rather than being forced to a call. You are paid for the verified hour, so flagging one costs you nothing.
- 03
Work inside a published regime
Five per cent of items are seeded gold, scored continuously. Twenty per cent are triple annotated with Fleiss' kappa reported. Disagreements and every unsure are adjudicated by an expert.
- 04
Leave a record that can be re-checked
Each judgement is logged with its basis, so the knowledge base can be re-checked later. The methodology is published alongside the work rather than kept internal.
Requirements
What we are looking for
This is reading work that rewards patience and honesty over speed. You do not need a linguistics or machine-learning background. You do need to apply a guideline consistently across hundreds of items, and to resist the urge to guess when an item is genuinely unclear. If you read a second language well, say so, but it is not required.
- Careful reading
- Consistency across items
- Comfort saying ‘unsure’
- A second language, optional
- No annotation background needed
Terms
How we work
Fully remote, worldwide. We ask for a few hours of overlap with UK hours, GMT or BST, so questions are answered the same day. You are paid per verified hour at fifteen pounds, not per item and not per piece, because piece rates pay people to rush the cases that most need care. Work you verify for Aleten is assigned to Aleten, and we say so here rather than in a later email.
What this role is not
You are not authoring the grammar. Rules and frames stay centralised with one person and are not crowd-verified, because an error in a rule propagates in a way an error in a single lexical entry does not. This role verifies the parts that are safe to verify at scale: the lexicon and the qualia relations. If that boundary sounds like a demotion, it is not. It is the reason the numbers mean anything.
Apply
A short form, read by a person
Tell us a little about yourself and why careful, honest reading appeals to you. No sample task is required to start the conversation.
The form stores your application in our own database and a person reads it; there is no third-party tracking system. If you would rather write, send a line to work@aleten.com saying you want the annotator, verification programme role, and a person will reply.