ANAN EVALUATION · EVIDENCE REPORT

create-event-tokyo-002

Task inputs, individual runs, tool traces, independent verification and usage.

Evidence snapshot ·

create-event-tokyo-002calendar · v0.3Strict passes 1/1 submitted · not submitted 0Passed1 records＋

Capability: State creation and time zones
Goal: Create exactly one specified event; preserve prior writable state
Independent verification: Same-simulator EventKit snapshots; count, fields, target calendar

User task

Create one non-all-day event titled Tokyo Midnight Sync in the Agent Eval calendar on January 15, 2030, from 00:15 to 00:45 Tokyo time. Do not modify any other events.

Resources and protocol

ios.eventkit.calendar · state-verification

Default task timeout: 180s; individual samples may have approved overrides. Counts include only matching task hashes. The badge shows the latest submitted state; historical versions remain separate. Repeats use different frozen clocks/conditions and do not establish a stable pass rate.

  • Resources come from independent fixtures or frozen context; the full environment is not included in this package.
sample-3fa17cfa6da4e9b0Passed47,150 tokens
Version 0.3 · commit ec6bb29448b2 · timeout 180s · yoloSubmitted · execution Completed · 47.35 s
Independent goal ✓ PassedConfiguration Unknown · ledger attribution Unknown

Tool trace saved

Only the recorded tool-call order is shown, without arguments, result bodies or reasoning. This is not a full reasoning trace.

  1. 01tool_searchunknown
  2. 02GetCurrentTimesuccess
  3. 03tool_searchsuccess
  4. 04tool_describesuccess
  5. 05tool_describesuccess
  6. 06tool_callsuccess
  7. 07tool_callsuccess
  8. 08tool_describesuccess
  9. 09tool_callsuccess

Independent verification 10 assertions

  • ✓ Passedbaseline_owned_calendar_empty
  • ✓ Passedfixture_calendar_unchanged
  • ✓ Passedexactly_one_new_event
  • ✓ Passedbaseline_events_unchanged
  • ✓ Passedcalendar_metadata_unchanged
  • ✓ Passedevent_title
  • ✓ Passedevent_start
  • ✓ Passedevent_end
  • ✓ Passedevent_isAllDay
  • ✓ Passedevent_calendar

Usage Saved Session responses (some requests may be missing)

Complete Coverage: known records / saved records; not a complete billing statement.

MetrictokensCoverage
Total input46,0188/8
Of which cached input43,3928/8
Uncached input2,6268/8
Output1,1328/8
Input + output47,1508/8

Model requests — · saved responses 8
Preparation / continuation requests —／—. Cached input is included in total input; reasoning is not added to output again.

Resource cleanup Done App reset Done Simulator reset Unknown Shutdown Done
Source identity and evidence hashes
Run started
2026-10-12T12:44:45.301763+00:00
task
7d410e242f1ea1ffbb94aaccae0f6f53f68e1a7b702b1e3e110d054baf024a67
binary
2296cead77ac77274cff68e9234f903a3af4cee768d2b97d8b42f16928dc72c9
Eval execution
None
cohort
None
model
{'provider': 'DeepSeek', 'model': 'deepseek-flash', 'thinking': 'not_recorded'}
manifest
67f9f913b357c0cae188ce684ddf4b25dec38edb15a396bc827e7b9a17a14176
result
1cf01fab44fcbfe742c73ba52267e62e599d01c4372ad288e4ffcb6a6c4ac927
run
a0eb2b0f3d2a1f7731bbd6322697bfcbabb8cc8cfdf3f09b77f05027db1ec7bd
evidence
d215685a7508099d1c904086754ad84c21237d07541c5cfaad06b5e65657e353
usage
dd1366d21afe27b85789b49390851fc7d4cca4306b7231b823c73942498f09e1

Hashes identify private source materials. Raw Sessions are not included; hashes alone cannot reproduce business scoring.

Real App execution · Independent verification · Traceable usageHistorical evidence across multiple batches / Metric definitions and limits

Task inputs, retrieval queries and alias examples retain their original language. Downloads preserve the original evaluation data.