LLM evaluation services, run by embedded senior engineers.

Awana offers LLM evaluation services for AI labs and enterprise AI teams: dedicated cells of calibrated senior software engineers who work inside your own evaluation harness, Slack, GitHub and rubrics, in UTC-3 to UTC-6. It is an engineering capacity partner, not a data vendor or a platform: Awana does not build, broker or sell datasets, and every output is your IP.

We took a frontier AI lab to 50 engineers on AI-training and evaluation work in three months.

A cell, inside your environment

Tasks come in, engineers work them, one lead answers for the result, graded work goes back. The cell works inside your Slack, GitHub and evaluation harness.

What a cell does

  • Repo-level coding tasks: write, solve, grade
  • Anti-reward-hacking audits of passing solutions
  • Multi-step agent trajectory evaluation
  • OWASP-mapped red teaming and production stress-testing
  • Rubric grading and preference data, with written rationales
  • Day-zero onboarding of open-weight releases

An engineering capacity partner, not a data vendor.

Most labs already run several kinds of supplier. As an AI evaluation partner, Awana adds the senior-engineering layer beside them, in the same harness.

Supplier typeThe job it doesHow a cell sits beside it
Annotation and labeling platformsVolume: large pools of annotators on well-specified tasksThe cell takes the code and agent tasks in the same platform. The platform keeps its work.
Student and graduate networksBreadth: many subjects and languages covered at volumeThe cell covers software and agentic engineering, under the same rubrics.
Expert networksDepth: credentialed specialists by the hour in medicine, law, finance, scienceThe cell brings production engineering judgment to the same evaluation program.
Awana evaluation cellCalibrated senior software engineers, dedicated to your project, in US hoursPlugs into your harness and vendors. Builds, brokers and sells no datasets.

Code, rubrics and evaluation outputs stay 100% your IP. Public vendors, from their own material: Mercor alternatives.

Prove the signal before you commit.

Three steps: a free blind calibration round, a paid one-week trial pod, then a dedicated pod retainer. The cell is the standing team on your work; its first week runs as the trial pod.

  1. Step 1 · Free

    Blind calibration round

    Vetted engineers work a small sample of your real repo-level tasks or red-team prompts, blind. You judge the signal before you pay.

  2. Step 2 · One week, paid

    Trial pod

    Deployed beside your existing vendors on the same tasks, so you benchmark quality and velocity on your own work.

  3. End of the week

    Read-out

    Acceptance rate on every task, rationale samples, and agreement across the pod. No obligation after the week.

  4. Step 3 · Ongoing

    Dedicated pod retainer

    A multi-engineer cell on your project, funded from your post-training or evaluation budget.

Sample · invented numbers

Trial read-out: coding-benchmark pod, week 1

  • 42tasks shipped
  • 86%accepted on first review
  • 86%agreement with your gold set, floor 80%
  • Engineer A16 tasks · 94% accepted first time · 91% agreementKeep. Strongest rationales in the pod.
  • Engineer B15 tasks · 87% accepted first time · 88% agreementKeep.
  • Engineer C11 tasks · 73% accepted first time · 76% agreementBelow your floor. We swap this seat within the week.
The format a lab receives after a one-week trial pod. Full sample read-out (PDF)

How an engineer qualifies for your cell.

  1. Step 1

    Four vetting gates

    A live technical interview, a take-home built for the role, an English screen and reference checks.

  2. Step 2

    Blind calibration

    Graded against your gold set. Passing is required to join the cell.

  3. Step 3

    Onboarding

    Your guidelines, your rubrics, your Slack, GitHub and harness, and your access rules.

  4. Every batch

    Live cell

    Agreement tracked against the floor you set, with second-pass review on a sample.

Your environment, your rules.

You control

  • Your Slack, GitHub and evaluation harness
  • Your SSO or OAuth, least privilege, logged sessions
  • Your access tiers and redaction rules, with PII masked before human review
  • Access removed at offboarding
  • Your code, rubrics and evaluation outputs: 100% your IP, in your environment

Awana guarantees

  • NDA and a standard DPA before any data is shared
  • A background check on every engineer before they touch your systems
  • United States-led, working UTC-3 to UTC-6
  • One accountable lead for each cell
  • No datasets of our own: Awana does not build, broker or sell them

The full list for a security or procurement review: Security and contracting.

A frontier AI lab: 50 engineers on AI-training and evaluation work in three months.

We do not publish the lab's name.

  • 5 business days to a vetted shortlist
  • About three weeks to staff most pods, calibration included
  • The same day an engineer gets access, a first PR
  • Roughly 1 in 40 engineers make it through our vetting
  • UTC-3 to UTC-6: full overlap with the US workday

Who you talk to.

Calvin Sedao

Calvin Sedao

CEO and Co-Founder, Seattle

Founded Awana in 2020 after building a point-of-sale company with a Venezuelan engineering team.

Javier Dutan

Javier Dutan

Co-Founder and COO, New York

Former White House US Digital Service.

Your first call is with Calvin Sedao, our CEO, and one lead runs your cell end to end. Awana's leadership holds an active US security clearance.

Questions a lab asks first.

What is an engineering evaluation cell?

An engineering evaluation cell is a dedicated team of calibrated senior software engineers, with one accountable lead, that works inside your Slack, GitHub and evaluation harness, under your rubrics and access rules, in UTC-3 to UTC-6. Awana supplies the engineering capacity. The tools, data and outputs stay yours.

We already run an annotation platform and a harness. Does a cell replace them?

No. A cell is built to complement your stack. Annotation platforms, crowd vendors, student networks and expert networks cover volume and breadth. A cell adds senior engineering capacity for code and agent work, inside the same harness and beside the same vendors.

How are engineers calibrated?

Each passes the four vetting gates, then a blind calibration round against your gold set before touching live work. When your rubric changes, the cell re-calibrates before the next batch.

What is an anti-reward-hacking audit?

Engineers check whether a model's passing solutions really solve the task or game the grader: weakened or special-cased tests, hard-coded expected outputs, edits to the test harness, or exploits of an under-specified task. Then they harden the tasks and graders.

How do you handle confidential models and data?

Awana signs an NDA and a standard DPA before any data is shared, background-checks every engineer before they touch your systems, and keeps all work inside your environment, under your SSO, access tiers and redaction rules. Security and contracting.

How does an engagement start, and how is it priced?

With a free blind calibration round. Next is a paid one-week trial pod beside your existing vendors, with no obligation after the week. Then a dedicated pod retainer, funded from your post-training or evaluation budget. You get a quote after the first call.

Does Awana sell datasets?

No. Awana is an engineering capacity partner, not a data vendor: it does not build, broker or sell datasets. Every piece of code, every rubric and every evaluation output a cell produces is 100% your IP and stays in your environment.

What do you not do?

Awana does not run bio, chemical, radiological or nuclear (CBRN) risk testing. Adversarial evaluation cells are scoped to software and agentic engineering. Dangerous-capability evaluation of frontier models is led by specialist evaluators, and Awana supplies trained capacity to them when they need it.

More questions are answered in the text version of this page.

Scope your first cell.

Thirty minutes on the tasks, the rubric and your harness. You leave with a cell design and a calibration plan. The blind calibration round is free.

We do not build, broker or sell datasets. Code, rubrics and evaluation outputs stay yours.