ANAN EVALUATION · EVIDENCE REPORT

context.restore-booking-card-128k-001

Task inputs, individual runs, tool traces, independent verification and usage.

Evidence snapshot ·

C09context.restore-booking-card-128k-001context · v0.1Strict passes 1/3 submitted · not submitted 0Passed3 records＋

Capability: Exact fact retention in dense history
Goal: Recover eight fields for one booking; never guess the code
Independent verification: Host oracle audited against frozen history/current resources; actual CSV and protected state, not self-report

User task

请沿用之前已确认的预约资料，把预约BK-012-037整理成确认卡，保存到 phone:booking-card.csv，列为 booking_id,people,diet,avoid,day,slot,budget_cents,confirmation_code。金额用人民币分，保留原确认码；只生成草案，不联系客户、不修改预约。

Resources and protocol

ios.workspace.files · context-continuation

Default task timeout: 300s; individual samples may have approved overrides. Counts include only matching task hashes. The badge shows the latest submitted state; historical versions remain separate. Repeats use different frozen clocks/conditions and do not establish a stable pass rate.

  • Resources come from independent fixtures or frozen context; the full environment is not included in this package.
sample-d9d468229c9eeb1a · originalFailed1,254,574 tokens
Version 0.1 · commit 6a7862e39339 · timeout 900s · yoloSubmitted · execution Completed · 121.71 s
Independent goal × FailedConfiguration Consistent · ledger attribution Complete

Tool trace saved

Only the recorded tool-call order is shown, without arguments, result bodies or reasoning. This is not a full reasoning trace.

  1. 01ListFilessuccess
  2. 02WriteFilesuccess
  3. 03ReadFilesuccess

Independent verification 17 assertions

  • ✓ Passedbaseline_fixture_matches
  • ✓ Passedworkspace_identity_unchanged
  • ✓ Passedsource_files_unchanged
  • ✓ Passedno_extra_paths
  • ✓ Passedno_unsafe_paths
  • ✓ Passedoutput_files_present
  • ✓ Passedoutput_csv_valid
  • × Failedcurrent_goal_correct
  • ✓ Passedbooking_id_correct
  • ✓ Passedpeople_correct
  • ✓ Passeddiet_correct
  • ✓ Passedavoid_correct
  • ✓ Passedday_correct
  • ✓ Passedslot_correct
  • × Failedbudget_cents_correct
  • × Failedconfirmation_code_correct
  • ✓ Passedcontext_sources_unchanged

Usage Request ledger

Complete Coverage: known records / saved records; not a complete billing statement.

MetrictokensCoverage
Total input1,248,7864/4
Of which cached input939,3924/4
Uncached input309,3944/4
Output5,7884/4
Input + output1,254,5744/4

Model requests 4 · saved responses —
Preparation / continuation requests 0／4. Cached input is included in total input; reasoning is not added to output again.

Resource cleanup Done App reset Done Simulator reset Done Shutdown Done
Source identity and evidence hashes
Run started
2026-10-17T13:00:12.832437+00:00
task
2ea85a249618ceec3dd3c11f0796fc2f883b0c1ce7cd6f9e976be464c70948f8
binary
bc364195891e17ba7ee727ebca478b5d3d34a92260399a83f3dd36fffa271abf
Eval execution
67f0bb67648a26a0b99fa5128f85221d7cc5ce74283d8ee2d26d429604bb4416
cohort
99d1be600defd350
model
{'provider': 'DeepSeek', 'model': 'deepseek-flash', 'thinking': 'provider_default'}
manifest
6847c950b6ade259aa24279aed4965511c6083b060390bc4d96802de5947493f
result
eae53b737b2a4aa657ed2f967052baf7ae588036c3aa18462310c4b7321390c4
run
b0d8fb2b670075155d4897b90cfee248a98bf3966476e9dc7105288c41f3449a
evidence
e351bd9782189070bfc14b084fb3487f478d1d9176a3d3ff4ff89bdadecb5b1c
usage
0c1c69365e19845a00c82e3e536234246fd89ffa484913bc7b9f4d762b690813

Hashes identify private source materials. Raw Sessions are not included; hashes alone cannot reproduce business scoring.

sample-c927ef11af5aa91d · compactFailed358,263 tokens
Version 0.1 · commit 6a7862e39339 · timeout 900s · yoloSubmitted · execution Completed · 40.36 s
Independent goal × FailedConfiguration Consistent · ledger attribution Complete

Tool trace saved

Only the recorded tool-call order is shown, without arguments, result bodies or reasoning. This is not a full reasoning trace.

  1. 01Memorysuccess
  2. 02ListFilessuccess

Independent verification 9 assertions

  • ✓ Passedbaseline_fixture_matches
  • ✓ Passedworkspace_identity_unchanged
  • ✓ Passedsource_files_unchanged
  • ✓ Passedno_extra_paths
  • ✓ Passedno_unsafe_paths
  • × Failedoutput_files_present
  • × Failedoutput_csv_valid
  • × Failedcurrent_goal_correct
  • ✓ Passedcontext_sources_unchanged

Usage Request ledger

Complete Coverage: known records / saved records; not a complete billing statement.

MetrictokensCoverage
Total input356,8373/3
Of which cached input317,3123/3
Uncached input39,5253/3
Output1,4263/3
Input + output358,2633/3

Model requests 3 · saved responses —
Preparation / continuation requests 1／2. Cached input is included in total input; reasoning is not added to output again.

Resource cleanup Done App reset Done Simulator reset Done Shutdown Done
Source identity and evidence hashes
Run started
2026-10-17T13:06:49.154980+00:00
task
2ea85a249618ceec3dd3c11f0796fc2f883b0c1ce7cd6f9e976be464c70948f8
binary
bc364195891e17ba7ee727ebca478b5d3d34a92260399a83f3dd36fffa271abf
Eval execution
67f0bb67648a26a0b99fa5128f85221d7cc5ce74283d8ee2d26d429604bb4416
cohort
99d1be600defd350
model
{'provider': 'DeepSeek', 'model': 'deepseek-flash', 'thinking': 'provider_default'}
manifest
d70faa993f9de4e7856111e091f1ab0080b01b701bc293c900e608fa0683e508
result
d91f5836a912a9bc1f7bd17598b4ee6d7be99121905bf18bfae1f8d16d31c504
run
1d30fe46294e061ae6c06522d73467129d3005bfa11aabab7f7512779d37c93b
evidence
646d86255219a1841103fb7ba8ea8ed9febe3d1d37140f9d17c11cf59e713134
usage
3e918be6c45cd9358935e103568543e22958bfb76add0ec14f436767d4eb16e3

Hashes identify private source materials. Raw Sessions are not included; hashes alone cannot reproduce business scoring.

sample-f16e8937203aa783 · rebuildPassed218,949 tokens
Version 0.1 · commit 6a7862e39339 · timeout 900s · yoloSubmitted · execution Completed · 28.95 s
Independent goal ✓ PassedConfiguration Consistent · ledger attribution Complete

Tool trace saved

Only the recorded tool-call order is shown, without arguments, result bodies or reasoning. This is not a full reasoning trace.

  1. 01ListFilessuccess
  2. 02WriteFilesuccess
  3. 03ReadFilesuccess

Independent verification 17 assertions

  • ✓ Passedbaseline_fixture_matches
  • ✓ Passedworkspace_identity_unchanged
  • ✓ Passedsource_files_unchanged
  • ✓ Passedno_extra_paths
  • ✓ Passedno_unsafe_paths
  • ✓ Passedoutput_files_present
  • ✓ Passedoutput_csv_valid
  • ✓ Passedcurrent_goal_correct
  • ✓ Passedbooking_id_correct
  • ✓ Passedpeople_correct
  • ✓ Passeddiet_correct
  • ✓ Passedavoid_correct
  • ✓ Passedday_correct
  • ✓ Passedslot_correct
  • ✓ Passedbudget_cents_correct
  • ✓ Passedconfirmation_code_correct
  • ✓ Passedcontext_sources_unchanged

Usage Request ledger

Complete Coverage: known records / saved records; not a complete billing statement.

MetrictokensCoverage
Total input218,2414/4
Of which cached input167,8084/4
Uncached input50,4334/4
Output7084/4
Input + output218,9494/4

Model requests 4 · saved responses —
Preparation / continuation requests 0／4. Cached input is included in total input; reasoning is not added to output again.

Resource cleanup Done App reset Done Simulator reset Done Shutdown Done
Source identity and evidence hashes
Run started
2026-10-17T13:12:13.639130+00:00
task
2ea85a249618ceec3dd3c11f0796fc2f883b0c1ce7cd6f9e976be464c70948f8
binary
bc364195891e17ba7ee727ebca478b5d3d34a92260399a83f3dd36fffa271abf
Eval execution
67f0bb67648a26a0b99fa5128f85221d7cc5ce74283d8ee2d26d429604bb4416
cohort
99d1be600defd350
model
{'provider': 'DeepSeek', 'model': 'deepseek-flash', 'thinking': 'provider_default'}
manifest
381b1164f5783473bccbe2e3f256087ee0494580ec350d735a81ca0077ea4d4e
result
42a486edb919caaa225a5d4fb0935160b570267bc747fea8346e4c73d31e1ea2
run
511718ec591245d47fe3d72c58c9b7da8e48e0a6d31e065dcea371b8a4b67e17
evidence
4850313e008c23ee7e4c6fc0fd9d8c12dbbfd3d6ff70f98bdf9384b6c12b0447
usage
a6da796664f95e6b663c55d6b30fa740168d1311f07cdc9c26e5487a06071846

Hashes identify private source materials. Raw Sessions are not included; hashes alone cannot reproduce business scoring.

Real App execution · Independent verification · Traceable usageHistorical evidence across multiple batches / Metric definitions and limits

Task inputs, retrieval queries and alias examples retain their original language. Downloads preserve the original evaluation data.