Human–Machine Relations Lab
DEContact

Evaluation methods

Make the conditions of a result as visible as the result itself.

Read more

Scope and provenance

Name the model, dataset, version, task population and source. Keep reproduced measurements separate from figures transcribed from a supplied report. The w⁴iya page currently uses the latter.

Security and usefulness

Attack success and normal-task completion answer different questions. Read both, with wrong behaviour per 1,000 calls, total cost per case and the domain-specific damage measure. A lower attack rate alone does not describe overall usefulness.

Estimates and uncertainty

The OAuth row is an estimate and must stay outside measured best-value comparisons. The deck describes overlapping uncertainty ranges for some results but does not provide those intervals in the table. We do not invent them or make a sole-winner claim.

Humans and coverage

w⁴iya C2 uses a simulated human. Human waiting time and w⁴iya questions to humans are not included in the reported check time. The four-world average is not a per-world breakdown or evidence of full production coverage.