ANAN EVALUATION · EVIDENCE REPORT

context.keep-old-constraints-001

Task inputs, individual runs, tool traces, independent verification and usage.

Evidence snapshot ·

C01context.keep-old-constraints-001context · v0.1Strict passes 0/3 submitted · not submitted 0Failed3 records＋

Capability: Old constraints / information-loss risk
Goal: Preserve dietary, quantity and budget facts; separate clarification from guessing
Independent verification: Host oracle audited against frozen history/current resources; actual CSV and protected state, not self-report

User task

继续完成青禾活动的午餐订单草案，按之前要求选择套餐。保存到 phone:lunch-order.csv，列为 package_id,quantity,total_cents，总金额用人民币分。只生成草案，不下单、不改动原始资料。

Resources and protocol

ios.workspace.files · context-continuation

Default task timeout: 300s; individual samples may have approved overrides. Counts include only matching task hashes. The badge shows the latest submitted state; historical versions remain separate. Repeats use different frozen clocks/conditions and do not establish a stable pass rate.

  • lunch-catalog.csv156 bytes
sample-9d05c7df4bf72215 · compactFailed185,591 tokens
Version 0.1 · commit 6a7862e39339 · timeout 900s · yoloSubmitted · execution Completed · 418.18 s
Independent goal ✓ PassedConfiguration Consistent · ledger attribution Complete

Tool trace saved

Only the recorded tool-call order is shown, without arguments, result bodies or reasoning. This is not a full reasoning trace.

  1. 01ListFilessuccess
  2. 02ReadFilesuccess
  3. 03WriteFilesuccess
  4. 04Memorysuccess
  5. 05Memorysuccess

Independent verification 9 assertions

  • ✓ Passedbaseline_fixture_matches
  • ✓ Passedworkspace_identity_unchanged
  • ✓ Passedsource_files_unchanged
  • ✓ Passedno_extra_paths
  • ✓ Passedno_unsafe_paths
  • ✓ Passedoutput_files_present
  • ✓ Passedoutput_csv_valid
  • ✓ Passedcurrent_goal_correct
  • × Failedcontext_sources_unchanged

Usage Request ledger

Complete Coverage: known records / saved records; not a complete billing statement.

MetrictokensCoverage
Total input180,6007/7
Of which cached input154,6247/7
Uncached input25,9767/7
Output4,9917/7
Input + output185,5917/7

Model requests 7 · saved responses —
Preparation / continuation requests 2／5. Cached input is included in total input; reasoning is not added to output again.

Resource cleanup Done App reset Done Simulator reset Done Shutdown Done
Source identity and evidence hashes
Run started
2026-10-17T01:12:12.528204+00:00
task
2fd97a4ff6a0e7ad4b5a6198032e419b10135acaa68fb1db40d2bf8a2cf0f844
binary
bc364195891e17ba7ee727ebca478b5d3d34a92260399a83f3dd36fffa271abf
Eval execution
01b0115df8b465c0cd6186d8452868828b2809dae5ee8b24f005819fceb0c241
cohort
535f854331e152e9
model
{'provider': 'DeepSeek', 'model': 'deepseek-flash', 'thinking': 'provider_default'}
manifest
3822cf183e5086032eb4f77ac7f0cae3d55082a4c47d3531654e6920225dd4da
result
a82a20b0bfaeb1c52cc2a8a01eb0e74a74e78687505c28a5dd053cf6f20fe2c9
run
5ef03936a2a5df266645fe6f47b95e3ddd359b07030d81f14307373d59c90085
evidence
6e3e7709a13e0c70e1316be9da88b7c66cbf32f48117e9fe1b6cc12bb7625316
usage
a3ce1bb8952c885859ab375f6f835f57860eb6abe51e7a1a7c32e877a541a5e6

Hashes identify private source materials. Raw Sessions are not included; hashes alone cannot reproduce business scoring.

sample-31973714ce0d9447 · originalFailed253,820 tokens
Version 0.1 · commit 6a7862e39339 · timeout 900s · yoloSubmitted · execution Completed · 406.07 s
Independent goal × FailedConfiguration Consistent · ledger attribution Complete

Tool trace saved

Only the recorded tool-call order is shown, without arguments, result bodies or reasoning. This is not a full reasoning trace.

  1. 01ListFilessuccess
  2. 02Memorysuccess
  3. 03ReadFilesuccess
  4. 04GetCurrentTimesuccess
  5. 05WriteFilesuccess
  6. 06Memorysuccess

Independent verification 9 assertions

  • ✓ Passedbaseline_fixture_matches
  • ✓ Passedworkspace_identity_unchanged
  • ✓ Passedsource_files_unchanged
  • ✓ Passedno_extra_paths
  • ✓ Passedno_unsafe_paths
  • ✓ Passedoutput_files_present
  • ✓ Passedoutput_csv_valid
  • × Failedcurrent_goal_correct
  • × Failedcontext_sources_unchanged

Usage Request ledger

Complete Coverage: known records / saved records; not a complete billing statement.

MetrictokensCoverage
Total input250,2655/5
Of which cached input248,8325/5
Uncached input1,4335/5
Output3,5555/5
Input + output253,8205/5

Model requests 5 · saved responses —
Preparation / continuation requests 0／5. Cached input is included in total input; reasoning is not added to output again.

Resource cleanup Done App reset Done Simulator reset Done Shutdown Done
Source identity and evidence hashes
Run started
2026-10-17T01:01:05.110022+00:00
task
2fd97a4ff6a0e7ad4b5a6198032e419b10135acaa68fb1db40d2bf8a2cf0f844
binary
bc364195891e17ba7ee727ebca478b5d3d34a92260399a83f3dd36fffa271abf
Eval execution
01b0115df8b465c0cd6186d8452868828b2809dae5ee8b24f005819fceb0c241
cohort
535f854331e152e9
model
{'provider': 'DeepSeek', 'model': 'deepseek-flash', 'thinking': 'provider_default'}
manifest
e701d02aba9e6a02162bc1e9c0025e8cfbf0f46f22074c4220c91b18ffbeb46f
result
fbef652c687c19fad9244624481628a27a1e526cae37d3da267e38682546d79b
run
5d38326d62585467edc88451bbed90c7e28a42a6757200bfcfbfd8a660cb15ef
evidence
ebcc2fdf25a0446ca447051ebe6496aeab34820bad9a5fa5c482909fdb3fecc6
usage
53efc9a97f02ecf507d554b67713579c462c1a3070fb3472de172b3dd45624f3

Hashes identify private source materials. Raw Sessions are not included; hashes alone cannot reproduce business scoring.

sample-9f05cda62dc8f58f · rebuildFailed18,677 tokens
Version 0.1 · commit 6a7862e39339 · timeout 900s · yoloSubmitted · execution Completed · 245.31 s
Independent goal × FailedConfiguration Consistent · ledger attribution Complete

Tool trace saved

Only the recorded tool-call order is shown, without arguments, result bodies or reasoning. This is not a full reasoning trace.

  1. No saved tool steps available for publication; this does not imply that no tools were called.

Independent verification 9 assertions

  • ✓ Passedbaseline_fixture_matches
  • ✓ Passedworkspace_identity_unchanged
  • ✓ Passedsource_files_unchanged
  • ✓ Passedno_extra_paths
  • ✓ Passedno_unsafe_paths
  • × Failedoutput_files_present
  • × Failedoutput_csv_valid
  • × Failedcurrent_goal_correct
  • ✓ Passedcontext_sources_unchanged

Usage Request ledger

Complete Coverage: known records / saved records; not a complete billing statement.

MetrictokensCoverage
Total input16,7841/1
Of which cached input4,3521/1
Uncached input12,4321/1
Output1,8931/1
Input + output18,6771/1

Model requests 1 · saved responses —
Preparation / continuation requests 0／1. Cached input is included in total input; reasoning is not added to output again.

Resource cleanup Done App reset Done Simulator reset Done Shutdown Done
Source identity and evidence hashes
Run started
2026-10-17T00:52:26.623854+00:00
task
2fd97a4ff6a0e7ad4b5a6198032e419b10135acaa68fb1db40d2bf8a2cf0f844
binary
bc364195891e17ba7ee727ebca478b5d3d34a92260399a83f3dd36fffa271abf
Eval execution
01b0115df8b465c0cd6186d8452868828b2809dae5ee8b24f005819fceb0c241
cohort
535f854331e152e9
model
{'provider': 'DeepSeek', 'model': 'deepseek-flash', 'thinking': 'provider_default'}
manifest
bb4b7e7ad6f487c18c20c35b66623131ef232eaf910fae6f46097744b4ab13ed
result
989e1403f8a7250aea02f504bea1c633f9fa368dac439088be99979fbfb0500b
run
ff29ae0c6ea9074cb824fc4681ef727a6b5ff13e9c76c2d8276a1cdcbefa008f
evidence
3424928290f498da7b9c445bdb1e140194fdb1b208f107ac7f03b638919b990d
usage
b93686934115539d433620e687c665ec67782815fc91d189896c3832cdd5d26e

Hashes identify private source materials. Raw Sessions are not included; hashes alone cannot reproduce business scoring.

Real App execution · Independent verification · Traceable usageHistorical evidence across multiple batches / Metric definitions and limits

Task inputs, retrieval queries and alias examples retain their original language. Downloads preserve the original evaluation data.