ANAN EVALUATION · EVIDENCE REPORT

Tasks

Browse inputs, outcomes and evidence by task ID. Repeated runs and historical versions remain separate.

Evidence snapshot ·

Calendarcreate-event-tokyo-002

Create one non-all-day event titled Tokyo Midnight Sync in the Agent Eval calendar on January 15, 2030, from 00:15 to 00:45 Tokyo time. Do not modify any other events.

Strict passes 1 / 1 submissions

Calendarcalendar.create-event

Create a calendar event titled Benchmark Planning in the Agent Eval calendar on January 15, 2030, from 15:00 to 15:30 UTC. Do not modify any other events.

Strict passes 1 / 1 submissions

Calendardelete-exact-event-001

Delete only the Benchmark Planning event in the Agent Eval calendar on January 15, 2030, starting at 15:00 UTC. Preserve every other event, including same-title events on other dates, at other times and in other calendars.

Strict passes 1 / 1 submissions

Calendarfind-free-slot-calendar-scope-003

Considering only events in the Agent Eval calendar, find the earliest continuous 30-minute free slot on January 15, 2030, between 09:00 and 12:00 UTC. Create one non-all-day event titled Focus Time in that slot in Agent Eval. A slot may start exactly when another event ends, or end exactly when another event starts. Do not modify any existing events.

Strict passes 1 / 1 submissions

Calendarfind-free-slot-overlap-002

Considering only events in the Agent Eval calendar, find the earliest continuous 30-minute free slot on January 15, 2030, between 09:00 and 12:00 UTC. Create one non-all-day event titled Focus Time in that slot in Agent Eval. A slot may start exactly when another event ends, or end exactly when another event starts. Do not modify any existing events.

Strict passes 1 / 1 submissions

Calendarfind-free-slot-001

Considering only events in the Agent Eval calendar, find the earliest continuous 30-minute free slot on January 15, 2030, between 09:00 and 12:00 UTC. Create one non-all-day event titled Focus Time in that slot in Agent Eval. A slot may start exactly when another event ends, or end exactly when another event starts. Do not modify any existing events.

Strict passes 1 / 1 submissions

Calendarupdate-exact-event-001

In the Agent Eval calendar, move the Benchmark Planning event on January 15, 2030, from 15:00 UTC to 16:00 UTC. Keep its 30-minute duration, title, calendar, location and notes. Preserve all other events, including same-title events on other dates, at other times and in other calendars.

Strict passes 1 / 1 submissions

Contextcontext.distinguish-work-outcomes-001

继续整理这批工作的剩余事项,结合此前处理进度与当前工作日志,写入 phone:remaining.csv,列为 work_id,reason。reason用retry_failed/reopened/pending标记失败待重试、重新打开、尚未处理。只整理清单,不执行交付、不重写已完成文件、不改动原始资料。

Strict passes 3 / 3 submissions

Contextcontext.keep-old-constraints-001

继续完成青禾活动的午餐订单草案,按之前要求选择套餐。保存到 phone:lunch-order.csv,列为 package_id,quantity,total_cents,总金额用人民币分。只生成草案,不下单、不改动原始资料。

Strict passes 0 / 3 submissions

Contextcontext.prefer-current-file-001

继续整理青禾项目交接单,以当前 project.csv 为准,把项目编号、负责人、开始时间和预算保存到 phone:handoff.csv,列为 project_id,owner,start_at,budget_cents;预算用人民币分,不改动原始资料。

Strict passes 2 / 3 submissions

Contextcontext.reconcile-event-revisions-128k-001

请继续完成EV-108的活动方案,沿用此前最终生效的确认。根据 phone:event-options.csv 的当前报价,选择满足全部已确认条件且总价最低的餐食与场地组合,保存到 phone:event-plan.csv,列为 event_id,people,day,slot,address_id,budget_cents,meal_id,room_id,total_cents。总价为人数×餐食单价+餐食固定费+场地固定费,金额均用人民币分。只生成草案,不联系客户、不下单或预订,不修改原始报价。

Strict passes 3 / 3 submissions

Contextcontext.reconcile-event-revisions-001

请继续完成EV-108的活动方案,沿用此前最终生效的确认。根据 phone:event-options.csv 的当前报价,选择满足全部已确认条件且总价最低的餐食与场地组合,保存到 phone:event-plan.csv,列为 event_id,people,day,slot,address_id,budget_cents,meal_id,room_id,total_cents。总价为人数×餐食单价+餐食固定费+场地固定费,金额均用人民币分。只生成草案,不联系客户、不下单或预订,不修改原始报价。

Strict passes 3 / 3 submissions

Contextcontext.reconcile-revised-receipts-001

沿用之前确认的报销口径,根据当前票据完成QH-02报销汇总,保存到 phone:reimbursement.csv,列为 project_id,receipt_count,net_cents。receipt_count包含纳入的退款票据;金额用人民币分。不得修改原始票据或规则。

Strict passes 3 / 3 submissions

Contextcontext.recover-current-memory-001

接着整理青禾取件安排,把当前订单、取件地点和取件码保存到 phone:pickup-card.csv,列为 order_id,location,pickup_code。不改动原始资料。

Strict passes 3 / 3 submissions

Contextcontext.resolve-final-delivery-001

继续整理DL-061最终确认的交付安排,保存到 phone:dispatch.csv,列为 order_id,address_id,weekday,expedited,shipping_cents。weekday用Mon/Tue/Wed/Thu/Fri/Sat/Sun,expedited用true/false,费用用人民币分;不实际发货,不改动原始资料。

Strict passes 3 / 3 submissions

Contextcontext.restore-booking-card-128k-001

请沿用之前已确认的预约资料,把预约BK-012-037整理成确认卡,保存到 phone:booking-card.csv,列为 booking_id,people,diet,avoid,day,slot,budget_cents,confirmation_code。金额用人民币分,保留原确认码;只生成草案,不联系客户、不修改预约。

Strict passes 1 / 3 submissions

Contextcontext.restore-booking-card-001

请沿用之前已确认的预约资料,把预约BK-012-037整理成确认卡,保存到 phone:booking-card.csv,列为 booking_id,people,diet,avoid,day,slot,budget_cents,confirmation_code。金额用人民币分,保留原确认码;只生成草案,不联系客户、不修改预约。

Strict passes 2 / 3 submissions

Contextcontext.retain-procurement-exceptions-001

继续完成青禾采购草案,沿用此前确认的采购限制,买20件,从库存足够的单一批次中选总价最低的。保存到 phone:purchase.csv,列为 batch_id,quantity,total_cents;总价为20件商品金额,用人民币分,不含其他费用。只写草案,不采购,不改动原始资料。

Not run

Contextcontext.skip-completed-work-001

继续整理青禾交付检查清单,把还没完成的事项保存到 phone:remaining.csv,列为 item_id,title。已完成的不要重做或改写,也不要改动已有资料。

Strict passes 3 / 3 submissions

Files and planningplanning.allocate-workshops-002

Read brief.md and all six CSVs under inputs/ in your local phone workspace and allocate every workshop under the stated constraints and optimization priorities. Save schedule.json as one object with exactly the keys schedule, completion_time, total_cost. schedule is an array containing exactly one entry per workshop, in any order. Each entry has exactly workshop_id, candidate_id, room_id, start, end; times must be offset-bearing RFC3339 timestamps and agree with the selected candidate and duration. completion_time is the final workshop end as an offset-bearing RFC3339 timestamp, and total_cost is the amount in CNY yuan, expressed as a JSON number or decimal string with at most two fractional digits. For example, CNY 12.50 is 12.50 or "12.50", never 1250. Do not encode an integer count of cents. Respect the stated budget, participant/room capacity, availability, occupied intervals, 15-minute gaps and dependencies. Choose the earliest program completion, then the lowest cost at that completion; any tied optimum is accepted. Preserve all existing files and directories. Create only schedule.json, with no scratch files or directories. Do not book meetings or perform external actions.

Strict passes 1 / 1 submissions

Files and planningplanning.allocate-workshops-001

Read brief.md and all six CSVs under inputs/ in your local phone workspace and allocate every workshop under the stated constraints and optimization priorities. Save schedule.json as one object with exactly the keys schedule, completion_time, total_cost. schedule is an array containing exactly one entry per workshop, in any order. Each entry has exactly workshop_id, candidate_id, room_id, start, end; times must be offset-bearing RFC3339 timestamps and agree with the selected candidate and duration. completion_time is the final workshop end as an offset-bearing RFC3339 timestamp, and total_cost is exact CNY representing integer cents. Respect the stated budget, participant/room capacity, availability, occupied intervals, 15-minute gaps and dependencies. Choose the earliest program completion, then the lowest cost at that completion; any tied optimum is accepted. Preserve all existing files and directories. Create only schedule.json, with no scratch files or directories. Do not book meetings or perform external actions.

Strict passes 1 / 1 submissions

Files and planningfiles.apply-project-amendment-001

Apply the approved amendment-v4.json to register.md in your local phone workspace. archive/amendment-v3.json is superseded and must not be applied. Locate each project by its project ID; change only the four approved field values to the exact new text in the amendment. Leave every other byte, including notes, other projects, spacing and line endings, unchanged. Preserve all other existing files and directories. Then create only change-report.json with exactly source_revision, applied_change_ids and projects. source_revision is the approved integer revision; applied_change_ids is an array of all four applied change ID strings, without duplicates, in any order. projects is an object with exactly keys P02 and P04: P02 has exactly deadline (timestamp string) and capacity (integer number), P04 has exactly owner and status (strings). These values must describe the actual resulting register. Create no scratch files or directories and perform no external actions.

Strict passes 1 / 1 submissions

Files and planningfiles.audit-release-log-002

Audit all approved open blocker tickets for release R7 using brief.md and release-log.jsonl in your local phone workspace. Create only release-blockers.json as an object with exactly release, blocker_ids and count. release is "R7"; blocker_ids is an array of unique ticket ID strings in any order, and count is an integer equal to the number of selected IDs. Include every eligible ticket and exclude all others. Preserve every existing file and directory byte for byte. Do not create scratch files or perform external actions.

Strict passes 1 / 1 submissions

Files and planningfiles.audit-release-log-001

Audit all approved open blocker tickets for release R7 using brief.md and release-log.jsonl in your local phone workspace. Create only release-blockers.json as an object with exactly release, blocker_ids and count. release is "R7"; blocker_ids is an array of unique ticket ID strings in any order, and count is an integer equal to the number of selected IDs. Include every eligible ticket and exclude all others. Preserve every existing file and directory byte for byte. Do not create scratch files or perform external actions.

Strict passes 1 / 1 submissions

Files and planningfiles.close-project-ledger-001

Close this project ledger in your local phone workspace using policy/current.md, the three inputs/receipts-*.csv files, inputs/charges.csv, inputs/refunds.csv and inputs/fx.csv. Apply the active policy rather than archived rules. Save reconciliation.csv with exactly these columns: invoice_id,currency,expected_amount,charged_amount,refunded_amount,net_amount,excess_amount,shortfall_amount,net_cny. Include one row per distinct invoice in any order; all amount columns except net_cny use the invoice currency. Save summary.json with exactly these keys: currency, invoice_count, expected_total, charged_total, refunded_total, net_total, excess_total, shortfall_total, duplicate_receipt_ids, excluded_charge_ids, excluded_refund_ids. currency must be CNY and totals are the per-invoice converted and rounded amounts summed under the policy. invoice_count is an integer; each ID array is an unordered set encoded as an array with no duplicates. Use exact amounts representing integer cents. Preserve every existing file/directory, including protected/keep.txt. Create only reconciliation.csv and summary.json, no scratch files or directories. Do not issue refunds or perform external actions.

Strict passes 0 / 1 submissions

Files and planningfiles.extract-inspection-receipts-001

Using brief.md and inspection-log.jsonl in your local phone workspace, create only inspection-receipts.json. Include every inspection record exactly once, copying its final_status and confirmation_code exactly from the source. The output must be an object with exactly count and records: count is an integer; records is an array of objects with exactly id, final_status and confirmation_code, in any order. Do not infer missing values. Preserve every existing file and directory byte for byte. Do not create scratch files or perform external actions.

Strict passes 1 / 1 submissions

Files and planningfiles.reconcile-expenses-001

Reconcile receipts.csv and charges.csv in your local phone workspace. Identical receipts with the same invoice_id count once: compare all fields except receipt_id and retain the first row in file order. Each invoice should be charged once; when charged multiple times, retain the earliest transaction by charged_at and flag later transaction IDs as excess. Save reconciliation.csv with columns invoice_id,expected_amount,charged_amount,excess_amount, one row per invoice in any order. Save summary.json as one object with exactly these keys: currency, expected_total, charged_total, excess_total, duplicate_receipt_ids, and excess_transaction_ids. Use exact CNY amounts in decimal strings or numbers representing integer cents; ID arrays must have no duplicates. Preserve every existing file and directory, including protected/keep.txt. Create only these two output files and no other files or directories. Do not issue refunds or perform external actions.

Strict passes 1 / 1 submissions

GitHub MCPmcp.github-owned-public-stars

请统计当前 GitHub 账号本人拥有的公开、非 fork 仓库收到的 star 总数。不要计入私有仓库、fork、所属组织的仓库或我收藏的别人的仓库。请告诉我 star 总数。

Strict passes 0 / 1 submissions

GitHub MCPmcp.github-stars

我的github有多少star

Cost observation · answer unscored

Task inputs, retrieval queries and alias examples retain their original language. Downloads preserve the original evaluation data.