Your AI, evaluated by the professionals who do the job.
We test your assistant or agent with practising lawyers, tax advisers, physicians and engineers, in your language and under the rules that apply to you.
Human evaluation at three points in the life of an AI system.
Before launch
Experts from your field test the assistant on the questions your customers ask, and check every answer for substance.
Choosing a model or vendor
Two or more systems, graded on your own tasks by the same panel against the same rubric.
After every release
The same test set on each new version, with every change in quality listed.
An LLM evaluation by people who know the subject.
Test tasks from your use case
Written by professionals from the relevant field, in your language.
Two independent readers per answer
Each answer graded against a rubric agreed with you.
A list of failures
Every failure with an example, its severity and the rule it breaks.
Expert data for further training
Corrected answers and rubrics, if you want to improve the system.
Checked under the rules that apply to you.
The same question has a different correct answer in Munich, Lyon or Milan. Examples of task types:
A tax answer under German law
Checked by a licensed tax adviser in Munich.
An answer on French employment law
Checked by an employment lawyer in Lyon.
Patient information in Italian
Checked by a pharmacist in Milan.
From your use case to a regular evaluation.
Brief
Use case, languages, risks and volume.
Panel
Professionals from the relevant field, verified and briefed.
Pilot
A paid first evaluation with a summary of findings.
Regular evaluation
The same test set on every release.
EU data protection by default.
Data is processed and stored in the EU.
Your data is never used to train models.
Named, verified experts under NDA, with access per project.
Under Art. 28 GDPR, for every project.
Experts can work inside your systems and under your access rules.
Frequently asked questions.
Further questions: hello@vaialabs.ai
Do you need access to our model?
Not necessarily. We can evaluate outputs that you export, or work through an interface or environment that you provide.
Can the experts work in our environment?
Yes. They work in your tools under your access rules, or in ours.
How long does an evaluation take?
It depends on the number of tasks, languages and fields. You receive a timeline after the scoping call.
What does it cost?
A fixed price per evaluation or billing per expert hour. Every engagement starts with a paid pilot.
Do you help with the EU AI Act?
We provide expert evaluations and document who evaluated what, and when. We do not give legal advice.
Which fields and languages do you cover?
Medicine, law, tax, finance, engineering and education, among others, in the languages our panel covers. Need another field? We find the experts on our platforms.
