EnvLoop / Research Evidence
Qualification note / v0.6 / September 2026

Real software.
Auditable claims.

A controlled computer-use data-research benchmark is taking shape across six applications. This publication shows which environments and verifiers have been tested in real software, and exactly where the full study remains open.

Read the English technical PDF
6Proposed application cellsDeclared plan
0 / 24Completed researcher campaignsObserved
0 / 600Official final task instances runObserved
0 / 6Cells with 100 admitted tasksObserved
01 / Protocol

Research has one job. Evaluation has another.

Researchers choose training experience. The surrounding student, budget, GUI action contract, and independent saved-state evaluator remain fixed within each cell.

Researcher data strategy flows to Tinker Qwen3.8-27B training, a real-software GUI actor, and a separate saved-state evaluator. WorkArena instances are for evaluation only.
Training sources and WorkArena evaluation instances are separated under the provider's use conditions.
02 / Six surfaces

Choose a cell. Inspect its proof.

Upstream task counts have different grains. A source count is never treated as a scored denominator or a 100-task admission certificate.

Six-row readiness matrix showing local GUI, negative, saved-state, reset, student, and 100-final evidence. No application has a completed 100-task cell.
YES indicates a narrow direct positive; LIMIT indicates a restricted proof. Every 100-final column remains empty. On narrow screens, scroll the figure horizontally.
03 / Hard Excel

Complex files do not become final tasks by counting them.

A family-disjoint offline split narrows 200 published instances to 100 provisional finals from 15 workbooks. The separate SEC development audit tests causal repairs in the actual Excel web editor.

Hard Excel instances: 200 published, 100 provisional final, zero GUI-qualified final. Workbook families: 30 published, 22 structurally screened, 15 provisional final.
Task instances and base workbook families use different denominators.
On a six-sheet SEC development workbook, a one-edit human control repaired one of nine faults and failed the full task; a full human repair fixed nine of nine and passed 114 targets plus two replay profiles.
Two human GUI controls on one public development workbook. These are not student-model scores.

The actor brief hid the nine fault locations, but the public development builder can reveal them. Official final tasks require private fault maps, source isolation, complete reset, and model-run evidence.

04 / Rejected task

The network trace said yes. The database said no.

One Magento variant-price task received a published network score of 1.0 in an isolated clone, but the page showed a save error and none of five target prices changed. It was rejected before admission.

Magento task 777: published network evaluator 1.0, but zero of five requested prices persisted; the visible page showed a save error.
Clone-specific task rejection. Independent state readback prevents a HAR-only false acceptance from becoming a benchmark success.
05 / Release boundary

What this publication can support

Microsoft Excel and PowerPoint web, Magento, GitLab, and E2B Desktop each have a documented narrow probe. WorkArena instance access was requested and was pending review when this note was prepared. No full-scale researcher campaign has started; a later results report will require frozen task manifests, real checkpoint lineages, denominator and infrastructure accounting, official final outcomes, and actual provider costs.

Read the manuscript source · Inspect the public data and source hashes · Releases Current non-training build · English environment-validation update (4 October) · Latest native control results: Odoo100 and Magento20