TOOL RETRIEVAL · OFFLINE SWIFT · FIXED COMPARISON

Tool retrieval: four core methods

Four methods, one tool catalog, the same queries: from name and description retrieval to Schema and native aliases, then extended retrieval aliases. All failures are retained; real Agent outcomes are kept separate.

Evidence snapshot ·

55Real tool definitions · excludes MCP
100Development / regression · 90 positives + 10 negatives
60Fixed comparison · 50 positives + 10 negatives
4Compact methods · 3 reused + 1 newly run

Findings first

① Four fixed method definitions.Keyword matching weights the original four fields. Simplified BM25 reads only names and descriptions. Native aliases uses four fields from Anan c586608: name / identifier, description, Schema and native aliases. Extended aliases keeps the same scoring and adds direct-tool aliases and action synonyms. The actual app was not modified.

② Fixed-comparison performance.Hit@1 across the four methods: 46%, 42%, 48%, 68%. Hit@5: 70%, 58%, 74%, 90%. Extended aliases adds 10 positive Top1 matches over native aliases.

③ Scope of conclusions.Native vs. extended aliases measures vocabulary additions. Name + description vs. native aliases also changes field coverage and normalization, so differences cannot be attributed solely to the BM25 formula. None of the four measures Agent success, and extended aliases is not claimed to be universally best.

Combined results: 60 fixed-comparison queries

Hit@1 / 3 / 5 and MRR use only the 50 positive queries. The other 10 negatives measure false retrievals only. Capability-group MRR averages within groups first to avoid overweighting paraphrases.

MethodHit@1Hit@3Hit@5MRRCapability-group mean MRRNegative false matches
Keyword matchingOriginal four fields, weighted word hits46.0%58.0%70.0%0.55920.603610/10
BM25 · name＋descriptionName and description in one document42.0%50.0%58.0%0.47800.504610/10
BM25 · name + description + Schema + native aliasesgroup + name, description, schemaText, existing native aliases48.0%70.0%74.0%0.59480.650110/10
BM25 · name + description + Schema + extended aliasesSame four fields; aliases include native aliases, direct-tool aliases and action synonyms68.0%84.0%90.0%0.77070.788310/10

Hit@1

Correct tool ranked first · 50 positives

100%0%
46%
42%
48%
68%
KeywordsBM25 name + descriptionBM25 name + description + schema + native aliasesBM25 name + description + schema + extended aliases

Hit@5

Correct tool in top 5 · 50 positives

100%0%
70%
58%
74%
90%
KeywordsBM25 name + descriptionBM25 name + description + schema + native aliasesBM25 name + description + schema + extended aliases

Charts use a fixed 0–100% axis and the same 50 positives. Periwinkle indicates extended aliases. MRR is in the table; raw algorithm scores are not compared across methods.

What do Hit@1, Hit@5 and MRR measure?

Hit@1：Is an acceptable tool ranked first?Hit@5：Is at least one acceptable tool in the first five?

MRR (Mean Reciprocal Rank):For each positive query, take the reciprocal rank of its first acceptable tool; then average across positives. A miss contributes 0. Negatives are excluded from MRR.

MRR = 平均值(1 / 第一个正确工具的排名)

Example: queries rank 1st, 2nd and not retrieved. MRR = (1 + 0.5 + 0) / 3 = 0.5. It distinguishes 1st from 5th more clearly than Hit@5. It is neither the inverse of mean rank nor accuracy.

Separate capability transfer, new phrasings and negatives
30 capability-isolated queries / 19 capability groups
MethodHit@1Hit@3Hit@5MRRCapability-group mean MRRNegative false matches
Keyword matchingOriginal four fields, weighted word hits70.0%83.3%93.3%0.79390.77460/0
BM25 · name＋descriptionName and description in one document56.7%63.3%76.7%0.63000.60130/0
BM25 · name + description + Schema + native aliasesgroup + name, description, schemaText, existing native aliases70.0%93.3%96.7%0.81780.80440/0
BM25 · name + description + Schema + extended aliasesSame four fields; aliases include native aliases, direct-tool aliases and action synonyms80.0%96.7%100.0%0.88440.89560/0
10 unsupported requests
MethodHit@1Hit@3Hit@5MRRCapability-group mean MRRNegative false matches
Keyword matchingOriginal four fields, weighted word hits—————10/10
BM25 · name＋descriptionName and description in one document—————10/10
BM25 · name + description + Schema + native aliasesgroup + name, description, schemaText, existing native aliases—————10/10
BM25 · name + description + Schema + extended aliasesSame four fields; aliases include native aliases, direct-tool aliases and action synonyms—————10/10
20 new phrasings / 8 existing direct tools
MethodHit@1Hit@3Hit@5MRRCapability-group mean MRRNegative false matches
Keyword matchingOriginal four fields, weighted word hits10.0%20.0%35.0%0.20710.19760/0
BM25 · name＋descriptionName and description in one document20.0%30.0%30.0%0.25000.27500/0
BM25 · name + description + Schema + native aliasesgroup + name, description, schemaText, existing native aliases15.0%35.0%40.0%0.26040.28370/0
BM25 · name + description + Schema + extended aliasesSame four fields; aliases include native aliases, direct-tool aliases and action synonyms50.0%65.0%75.0%0.60000.53330/0
Same-criteria results for the 100-query development / regression set
MethodHit@1Hit@3Hit@5MRRCapability-group mean MRRNegative false matches
Keyword matchingOriginal four fields, weighted word hits61.1%73.3%77.8%0.67610.767210/10
BM25 · name＋descriptionName and description in one document56.7%63.3%68.9%0.61520.686310/10
BM25 · name + description + Schema + native aliasesgroup + name, description, schemaText, existing native aliases60.0%73.3%76.7%0.66990.759810/10
BM25 · name + description + Schema + extended aliasesSame four fields; aliases include native aliases, direct-tool aliases and action synonyms78.9%91.1%97.8%0.85940.879310/10

Data and methods

Native and extended alias examples

Tool / actionNative aliases in frozen sourceAdditional experimental terms (excerpt)
calendar.events.createcalendar → 日历、日程、活动、安排
create → 创建、新建、添加		create → add、make、put、insert、new、新增
WriteFileNo aliases generated by the native category/action dictionaries abovesave、document、保存、文档
MemoryNo aliases generated by the native category/action dictionaries aboveremember、preferences、记住、偏好

Native aliases come from dictionaries in the frozen Anan source, not iOS fields. The extended group retains those terms and adds direct-tool aliases/action synonyms without changing names, descriptions, schemas or field weights. Aliases affect internal retrieval only, not returned tool descriptions. Including WriteFile and Memory in this offline catalog does not mean product tool_search indexes these direct tools.

The 160 queries are split into 100 development and 60 fixed-comparison queries; metrics are not pooled. The 60 were previously held out, but their results are now known. This rerun is a regression / exploratory comparison, not a new unseen test.

The comparison set has 30 queries covering 19 native capabilities outside the development labels, plus 20 new phrasings for 8 existing direct tools. Metadata for all 55 tools was already known. Paraphrases of one capability are not independent samples, and manual data does not represent usage frequency.

MethodMetadataGrade
Keyword matchingOriginal four fields, weighted word hitsDistinct query-word hits × field weight
BM25 · name＋descriptionName and description in one documentSingle-document BM25; no field weights / Schema / aliases
BM25 · name + description + Schema + native aliasesgroup + name, description, schemaText, existing native aliasesField-weighted BM25, c586608
BM25 · name + description + Schema + extended aliasesSame four fields; aliases include native aliases, direct-tool aliases and action synonymsSame field-weighted BM25, expanded vocabulary only; not ported into Anan
Get visible tool catalog→Tokenization and normalization→Build retrieval documents→Score, sort, paginate

Shared preprocessing splits camelCase and punctuation, normalizes case, uses system NLTokenizer segmentation, a small stopword list and English singular / plural handling, and retains existing native Chinese query aliases. All four methods prioritize exact tool-name matches, then score, with stable group / name tie-breaking.

1 · Keyword matching

This is weighted keyword matching, not the earliest simple substring match. Distinct query-word hits are counted separately in name / identifier, description, Schema metadata and native aliases. Repeating a word within one field earns no additional score.

score = Σ field (field weight × distinct query-word hits)
weights = [3, 1, 0.5, 1.5]
No IDF, term-frequency saturation or length normalization

2 · BM25：name＋description

Only the tool name and original description form a single document. Term frequencies and length are pooled. It reads no group, Schema or document-side aliases and has no separate field weights.

document = bounded(name) + " " + bounded(description)
score = BM25(query, document)
k1 = 1.2；b = 0.75

3 · BM25: name + description + Schema + native aliases

Frozen commit c586608. BM25 is computed per field, then weighted and summed: rarer terms generally discriminate more; repeated-term gains saturate; field length is normalized against its average.

fields = [group + name, description, schemaText, nativeAliases]
weights = [3, 1, 0.5, 1.5]
score = Σ field (weight × BM25 field contribution)
k1 = 1.2; b = 0.75

Native aliases come from keywords / verbs vocabularies in the search code, for internal indexing only. They are not a standard aliases property of tools or MCP.

4 · BM25: name + description + Schema + extended aliases

Two term categories are appended to internal aliases in the same four fields. Names, descriptions, Schema, field weights and BM25 parameters are unchanged.

直接工具词表：
WriteFile → save / document / 保存 / 文档
Memory → remember / preferences / 记住 / 偏好
操作同义词：
create → add / put / 新增
update → change / edit / 修改

No extra model calls; added terms are not returned in search results. This remains lexical retrieval and does not understand every synonym or negation. The final candidate was not ported into Anan.

How to read these four methods

Keyword matching weights the original four fields. The name + description method uses one document containing only those fields. The current method adds Schema, existing native aliases and field weights. The final candidate only adds two alias categories to the current method. Differences in field coverage cannot be attributed solely to the BM25 formula. “Final” means the candidate in this experiment; the real Anan app was not changed.

Keyword, current and candidate results reuse the previously verified run; name + description was newly executed. All metrics were reaggregated. Historical methods and artifacts not shown here are retained.

Local Swift retrieval only: no model, simulator, UI or MCP service was run, and Anan was not modified. Eight tools, including file and Memory tools, are directly exposed in production; this experiment only adds them to a unified experimental catalog. Retrieval hit rates are not Agent task success rates. Real CPU latency and token cost were not measured. The keyword experiment computes BM25 before overriding scores, so elapsed time cannot compare algorithm speed.

Individual queries, targets and rankings

Defaults to the 60 fixed-comparison queries. Expand each to see Top5, full rankings and within-method scores for all four methods.

N001 ListFiles

dev · exact · target ['builtin/ListFiles']

Exact declared name identifies this tool.

MethodFirst acceptable rankTop5
Keyword matching1builtin/ListFiles → apple/reminders.list → builtin/EditFile → builtin/WriteFile → apple/alarms.list
BM25 · name＋description1builtin/ListFiles → builtin/Memory → builtin/WriteFile → builtin/ReadFile → builtin/EditFile
BM25 · name + description + Schema + native aliases1builtin/ListFiles → builtin/WriteFile → builtin/EditFile → builtin/ReadFile → builtin/Memory
BM25 · name + description + Schema + extended aliases1builtin/ListFiles → builtin/WriteFile → builtin/ReadFile → builtin/EditFile → builtin/Memory
Full rankings, within-method scores

Keyword matching

[('builtin/ListFiles', 7), ('apple/reminders.list', 4.5), ('builtin/EditFile', 4.5), ('builtin/WriteFile', 4.5), ('apple/alarms.list', 4), ('apple/calendar.events.list', 4), ('apple/calendar.list', 4), ('apple/contacts.list', 4), ('apple/photos.albums.list', 4), ('apple/reminders.lists', 4), ('builtin/ReadFile', 4), ('apple/health.workouts.list', 3), ('builtin/Memory', 3), ('apple/alarms.cancel', 1.5), ('apple/calendar.events.delete', 1.5), ('apple/calendar.events.update', 1.5), ('apple/photos.search', 1.5), ('apple/reminders.create', 1.5), ('apple/shortcuts.run', 1.5), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.create', 1), ('apple/calendar.events.get', 1), ('apple/clipboard.read', 1), ('apple/calendar.events.create', 0.5), ('apple/contacts.delete', 0.5), ('apple/contacts.get', 0.5), ('apple/contacts.update', 0.5), ('apple/health.workouts.get', 0.5)]

BM25 · name＋description

[('builtin/ListFiles', 5.04384), ('builtin/Memory', 4.63922), ('builtin/WriteFile', 3.71445), ('builtin/ReadFile', 3.47241), ('builtin/EditFile', 3.37407), ('apple/reminders.lists', 2.1669), ('apple/calendar.list', 1.97043), ('apple/clipboard.read', 1.93144), ('apple/calendar.events.list', 1.6149), ('apple/contacts.list', 1.49249), ('apple/reminders.list', 1.49249), ('apple/photos.search', 1.47053), ('apple/alarms.list', 1.4702), ('apple/calendar.events.get', 1.458), ('apple/alarms.create', 1.37763), ('apple/photos.albums.list', 1.34012), ('apple/health.workouts.list', 1.33902), ('apple/reminders.create', 1.28652), ('apple/calendar.events.delete', 1.12484), ('apple/alarms.cancel', 1.04154), ('apple/shortcuts.run', 0.97937), ('apple/calendar.events.update', 0.92421), ('builtin/ScheduledTasks', 0.89939)]

BM25 · name + description + Schema + native aliases

[('builtin/ListFiles', 9.64347), ('builtin/WriteFile', 10.32191), ('builtin/EditFile', 9.25924), ('builtin/ReadFile', 8.54967), ('builtin/Memory', 5.45328), ('apple/reminders.lists', 4.12289), ('apple/reminders.list', 3.82967), ('apple/calendar.list', 3.8154), ('apple/contacts.list', 3.34463), ('apple/alarms.list', 3.32653), ('apple/calendar.events.list', 3.16618), ('apple/photos.albums.list', 2.93791), ('apple/health.workouts.list', 2.24993), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.65154), ('apple/shortcuts.run', 1.49016), ('apple/reminders.create', 1.46978), ('apple/calendar.events.delete', 1.41756), ('apple/alarms.cancel', 1.37363), ('builtin/ScheduledTasks', 1.26107), ('apple/calendar.events.get', 1.12534), ('apple/alarms.create', 1.02045), ('apple/calendar.events.update', 0.97819), ('apple/contacts.delete', 0.60095), ('apple/contacts.get', 0.60095), ('apple/health.workouts.get', 0.58448), ('apple/contacts.update', 0.39141), ('apple/calendar.events.create', 0.30692)]

BM25 · name + description + Schema + extended aliases

[('builtin/ListFiles', 10.84359), ('builtin/WriteFile', 12.98291), ('builtin/ReadFile', 11.13883), ('builtin/EditFile', 9.25924), ('builtin/Memory', 5.45328), ('apple/reminders.lists', 4.12289), ('apple/reminders.list', 3.82967), ('apple/calendar.list', 3.8154), ('apple/contacts.list', 3.34463), ('apple/alarms.list', 3.32653), ('apple/calendar.events.list', 3.16618), ('apple/photos.albums.list', 2.93791), ('apple/health.workouts.list', 2.24993), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.65154), ('apple/shortcuts.run', 1.49016), ('apple/reminders.create', 1.46978), ('apple/calendar.events.delete', 1.41756), ('apple/alarms.cancel', 1.37363), ('builtin/ScheduledTasks', 1.26107), ('apple/calendar.events.get', 1.12534), ('apple/alarms.create', 1.02045), ('apple/calendar.events.update', 0.97819), ('apple/contacts.delete', 0.60095), ('apple/contacts.get', 0.60095), ('apple/health.workouts.get', 0.58448), ('apple/contacts.update', 0.39141), ('apple/calendar.events.create', 0.30692)]

N002 ReadFile

dev · exact · target ['builtin/ReadFile']

Exact declared name identifies this tool.

MethodFirst acceptable rankTop5
Keyword matching1builtin/ReadFile → builtin/EditFile → apple/clipboard.read → builtin/WriteFile → builtin/ListFiles
BM25 · name＋description1builtin/ReadFile → builtin/Memory → builtin/EditFile → builtin/WriteFile → apple/clipboard.read
BM25 · name + description + Schema + native aliases1builtin/ReadFile → builtin/EditFile → builtin/WriteFile → builtin/Memory → apple/clipboard.read
BM25 · name + description + Schema + extended aliases1builtin/ReadFile → builtin/WriteFile → builtin/EditFile → builtin/ListFiles → builtin/Memory
Full rankings, within-method scores

Keyword matching

[('builtin/ReadFile', 8), ('builtin/EditFile', 5.5), ('apple/clipboard.read', 5), ('builtin/WriteFile', 4.5), ('builtin/ListFiles', 3), ('builtin/Memory', 2.5), ('apple/photos.search', 2), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.list', 1), ('apple/battery.get', 1), ('apple/calendar.events.list', 1), ('apple/contacts.get', 1), ('apple/device.get', 1), ('apple/health.sleep.query', 1), ('apple/health.statistics', 1), ('apple/health.workouts.get', 1), ('apple/health.workouts.list', 1), ('apple/maps.eta', 1), ('apple/photos.get', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1), ('apple/shortcuts.run', 0.5)]

BM25 · name＋description

[('builtin/ReadFile', 5.20811), ('builtin/Memory', 5.09531), ('builtin/EditFile', 4.73732), ('builtin/WriteFile', 3.71445), ('apple/clipboard.read', 3.27541), ('builtin/ListFiles', 3.05349), ('apple/photos.search', 2.20558), ('apple/contacts.get', 1.59613), ('apple/health.workouts.get', 1.26659), ('apple/weather.current', 1.24126), ('apple/health.workouts.list', 1.21491), ('apple/calendar.events.list', 1.0957), ('apple/device.get', 1.08242), ('apple/health.sleep.query', 1.08242), ('apple/battery.get', 1.06946), ('apple/maps.eta', 1.06946), ('builtin/ScheduledTasks', 1.03334), ('apple/alarms.list', 0.95511), ('apple/photos.get', 0.89753), ('apple/weather.hourly', 0.8465), ('apple/weather.daily', 0.78685), ('apple/health.statistics', 0.71734)]

BM25 · name + description + Schema + native aliases

[('builtin/ReadFile', 13.10696), ('builtin/EditFile', 10.6499), ('builtin/WriteFile', 10.32191), ('builtin/Memory', 5.95572), ('apple/clipboard.read', 5.85716), ('builtin/ListFiles', 5.80426), ('apple/photos.search', 2.07452), ('apple/contacts.get', 1.60914), ('apple/health.workouts.get', 1.30421), ('apple/health.workouts.list', 1.24611), ('apple/weather.current', 1.23137), ('builtin/ScheduledTasks', 1.14906), ('apple/calendar.events.list', 1.11381), ('apple/health.sleep.query', 1.09922), ('apple/device.get', 1.08501), ('apple/battery.get', 1.07116), ('apple/maps.eta', 1.07116), ('apple/alarms.list', 0.94991), ('apple/photos.get', 0.88951), ('apple/weather.hourly', 0.83632), ('apple/weather.daily', 0.77457), ('apple/shortcuts.run', 0.7661), ('apple/health.statistics', 0.70318)]

BM25 · name + description + Schema + extended aliases

[('builtin/ReadFile', 15.34061), ('builtin/WriteFile', 12.98291), ('builtin/EditFile', 10.24338), ('builtin/ListFiles', 5.80426), ('builtin/Memory', 5.48475), ('apple/clipboard.read', 4.67189), ('apple/contacts.get', 2.14254), ('apple/photos.search', 1.86366), ('apple/health.workouts.get', 1.82646), ('apple/device.get', 1.80027), ('apple/battery.get', 1.79046), ('apple/health.workouts.list', 1.72237), ('apple/calendar.events.list', 1.69172), ('apple/alarms.list', 1.64891), ('apple/photos.get', 1.63326), ('apple/calendar.events.get', 0.97668), ('apple/location.get', 0.97668), ('apple/reminders.get', 0.951), ('apple/contacts.list', 0.92664), ('apple/photos.albums.list', 0.92664), ('apple/calendar.list', 0.9035), ('apple/reminders.list', 0.88148), ('apple/weather.current', 0.87142), ('builtin/ScheduledTasks', 0.81317), ('apple/health.sleep.query', 0.77789), ('apple/shortcuts.run', 0.7661), ('apple/maps.eta', 0.75804), ('apple/weather.hourly', 0.59185), ('apple/weather.daily', 0.54815), ('apple/health.statistics', 0.49762)]

N003 WriteFile

dev · exact · target ['builtin/WriteFile']

Exact declared name identifies this tool.

MethodFirst acceptable rankTop5
Keyword matching1builtin/WriteFile → apple/clipboard.write → builtin/EditFile → builtin/ReadFile → builtin/ListFiles
BM25 · name＋description1builtin/WriteFile → apple/clipboard.write → builtin/Memory → builtin/ReadFile → builtin/EditFile
BM25 · name + description + Schema + native aliases1builtin/WriteFile → apple/clipboard.write → builtin/EditFile → builtin/ReadFile → builtin/ListFiles
BM25 · name + description + Schema + extended aliases1builtin/WriteFile → apple/clipboard.write → builtin/ReadFile → builtin/EditFile → builtin/ListFiles
Full rankings, within-method scores

Keyword matching

[('builtin/WriteFile', 7.5), ('apple/clipboard.write', 4.5), ('builtin/EditFile', 4.5), ('builtin/ReadFile', 4), ('builtin/ListFiles', 3), ('builtin/Memory', 2), ('apple/clipboard.read', 1), ('apple/photos.search', 1), ('builtin/ScheduledTasks', 1), ('apple/shortcuts.run', 0.5)]

BM25 · name＋description

[('builtin/WriteFile', 7.59405), ('apple/clipboard.write', 3.79364), ('builtin/Memory', 3.47446), ('builtin/ReadFile', 3.47241), ('builtin/EditFile', 3.37407), ('builtin/ListFiles', 3.05349), ('apple/clipboard.read', 1.93144), ('apple/photos.search', 1.47053), ('builtin/ScheduledTasks', 1.38049)]

BM25 · name + description + Schema + native aliases

[('builtin/WriteFile', 18.0842), ('apple/clipboard.write', 11.75425), ('builtin/EditFile', 9.25924), ('builtin/ReadFile', 8.54967), ('builtin/ListFiles', 5.80426), ('builtin/Memory', 5.10546), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.35321), ('builtin/ScheduledTasks', 1.21464), ('apple/shortcuts.run', 0.7661)]

BM25 · name + description + Schema + extended aliases

[('builtin/WriteFile', 24.30388), ('apple/clipboard.write', 11.75425), ('builtin/ReadFile', 11.13883), ('builtin/EditFile', 9.25924), ('builtin/ListFiles', 5.80426), ('builtin/Memory', 5.10546), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.35321), ('builtin/ScheduledTasks', 1.21464), ('apple/shortcuts.run', 0.7661)]

N004 EditFile

dev · exact · target ['builtin/EditFile']

Exact declared name identifies this tool.

MethodFirst acceptable rankTop5
Keyword matching1builtin/EditFile → builtin/WriteFile → builtin/ReadFile → builtin/ListFiles → builtin/Memory
BM25 · name＋description1builtin/EditFile → builtin/Memory → builtin/WriteFile → builtin/ReadFile → builtin/ListFiles
BM25 · name + description + Schema + native aliases1builtin/EditFile → builtin/WriteFile → builtin/Memory → builtin/ReadFile → builtin/ListFiles
BM25 · name + description + Schema + extended aliases1builtin/EditFile → builtin/WriteFile → builtin/ReadFile → builtin/Memory → builtin/ListFiles
Full rankings, within-method scores

Keyword matching

[('builtin/EditFile', 7.5), ('builtin/WriteFile', 4.5), ('builtin/ReadFile', 4), ('builtin/ListFiles', 3), ('builtin/Memory', 3), ('apple/clipboard.read', 1), ('apple/photos.search', 1), ('apple/shortcuts.run', 0.5)]

BM25 · name＋description

[('builtin/EditFile', 7.5917), ('builtin/Memory', 6.74403), ('builtin/WriteFile', 3.71445), ('builtin/ReadFile', 3.47241), ('builtin/ListFiles', 3.05349), ('apple/clipboard.read', 1.93144), ('apple/photos.search', 1.47053)]

BM25 · name + description + Schema + native aliases

[('builtin/EditFile', 18.83116), ('builtin/WriteFile', 10.32191), ('builtin/Memory', 9.39359), ('builtin/ReadFile', 8.54967), ('builtin/ListFiles', 5.80426), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.35321), ('apple/shortcuts.run', 0.7661)]

BM25 · name + description + Schema + extended aliases

[('builtin/EditFile', 19.59073), ('builtin/WriteFile', 12.98291), ('builtin/ReadFile', 11.13883), ('builtin/Memory', 8.11316), ('builtin/ListFiles', 5.80426), ('apple/contacts.update', 2.94821), ('apple/calendar.events.update', 2.87636), ('apple/reminders.update', 2.80794), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.35321), ('apple/shortcuts.run', 0.7661)]

N005 Memory

dev · exact · target ['builtin/Memory']

Exact declared name identifies this tool.

MethodFirst acceptable rankTop5
Keyword matching1builtin/Memory → builtin/ScheduledTasks
BM25 · name＋description1builtin/Memory → builtin/ScheduledTasks
BM25 · name + description + Schema + native aliases1builtin/Memory → builtin/ScheduledTasks
BM25 · name + description + Schema + extended aliases1builtin/Memory → builtin/ScheduledTasks
Full rankings, within-method scores

Keyword matching

[('builtin/Memory', 4.5), ('builtin/ScheduledTasks', 1)]

BM25 · name＋description

[('builtin/Memory', 4.42428), ('builtin/ScheduledTasks', 1.54802)]

BM25 · name + description + Schema + native aliases

[('builtin/Memory', 15.18709), ('builtin/ScheduledTasks', 1.49781)]

BM25 · name + description + Schema + extended aliases

[('builtin/Memory', 20.99573), ('builtin/ScheduledTasks', 1.49781)]

N006 GetCurrentTime

dev · exact · target ['builtin/GetCurrentTime']

Exact declared name identifies this tool.

MethodFirst acceptable rankTop5
Keyword matching1builtin/GetCurrentTime → apple/location.get → apple/calendar.events.get → apple/reminders.get → apple/weather.current
BM25 · name＋description1builtin/GetCurrentTime → apple/location.get → apple/maps.eta → apple/music.status → apple/maps.route
BM25 · name + description + Schema + native aliases1builtin/GetCurrentTime → apple/location.get → apple/reminders.get → apple/calendar.events.get → apple/weather.current
BM25 · name + description + Schema + extended aliases1builtin/GetCurrentTime → apple/location.get → apple/reminders.get → apple/calendar.events.get → apple/weather.current
Full rankings, within-method scores

Keyword matching

[('builtin/GetCurrentTime', 12), ('apple/location.get', 5), ('apple/calendar.events.get', 4), ('apple/reminders.get', 4), ('apple/weather.current', 4), ('apple/battery.get', 3), ('apple/contacts.get', 3), ('apple/device.get', 3), ('apple/health.workouts.get', 3), ('apple/photos.get', 3), ('builtin/ScheduledTasks', 2.5), ('apple/calendar.events.update', 2), ('apple/maps.eta', 2), ('apple/maps.route', 2), ('apple/music.status', 2), ('apple/weather.hourly', 2), ('apple/alarms.create', 1.5), ('apple/calendar.events.delete', 1.5), ('apple/calendar.events.list', 1.5), ('apple/health.statistics', 1.5), ('apple/reminders.list', 1.5), ('builtin/EditFile', 1.5), ('apple/alarms.cancel', 1), ('apple/location.geocode', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.search', 1), ('apple/maps.show', 1), ('apple/music.playback', 1), ('apple/weather.daily', 1), ('apple/calendar.events.create', 0.5), ('apple/health.sleep.query', 0.5), ('apple/health.workouts.list', 0.5), ('apple/reminders.create', 0.5), ('apple/reminders.update', 0.5)]

BM25 · name＋description

[('builtin/GetCurrentTime', 7.64193), ('apple/location.get', 4.58959), ('apple/maps.eta', 3.83931), ('apple/music.status', 3.10782), ('apple/maps.route', 2.87521), ('apple/reminders.get', 2.69485), ('apple/weather.hourly', 2.54825), ('apple/calendar.events.get', 2.47013), ('apple/health.workouts.get', 1.8901), ('apple/contacts.get', 1.86368), ('builtin/EditFile', 1.83299), ('apple/calendar.events.list', 1.82518), ('apple/weather.current', 1.66897), ('apple/location.reverseGeocode', 1.63355), ('apple/reminders.list', 1.62576), ('apple/device.get', 1.61526), ('apple/battery.get', 1.59592), ('apple/alarms.cancel', 1.57416), ('apple/weather.daily', 1.56055), ('apple/calendar.events.delete', 1.52299), ('apple/location.geocode', 1.5103), ('apple/alarms.create', 1.45133), ('apple/photos.get', 1.33936), ('apple/maps.show', 1.28422), ('apple/calendar.events.update', 1.25134), ('builtin/ScheduledTasks', 1.21774), ('apple/maps.search', 1.2068), ('apple/health.statistics', 0.96452), ('apple/music.playback', 0.94183)]

BM25 · name + description + Schema + native aliases

[('builtin/GetCurrentTime', 16.16533), ('apple/location.get', 7.84037), ('apple/reminders.get', 6.96445), ('apple/calendar.events.get', 6.12286), ('apple/weather.current', 5.04538), ('apple/battery.get', 4.61692), ('apple/contacts.get', 4.61692), ('apple/device.get', 4.61692), ('apple/photos.get', 4.61692), ('apple/health.workouts.get', 4.08145), ('apple/maps.eta', 2.95971), ('builtin/ScheduledTasks', 2.87555), ('apple/calendar.events.delete', 2.5715), ('apple/music.status', 2.45572), ('apple/calendar.events.update', 2.364), ('builtin/EditFile', 2.354), ('apple/maps.route', 2.26114), ('apple/weather.hourly', 1.99075), ('apple/calendar.events.list', 1.87013), ('apple/location.reverseGeocode', 1.5928), ('apple/alarms.create', 1.57832), ('apple/health.statistics', 1.53421), ('apple/reminders.list', 1.51429), ('apple/weather.daily', 1.46638), ('apple/location.geocode', 1.44283), ('apple/maps.show', 1.21419), ('apple/maps.search', 1.13698), ('apple/alarms.cancel', 1.03523), ('apple/music.playback', 0.87677), ('apple/reminders.create', 0.86688), ('apple/reminders.update', 0.86688), ('apple/health.sleep.query', 0.6728), ('apple/health.workouts.list', 0.62158), ('apple/calendar.events.create', 0.61489)]

BM25 · name + description + Schema + extended aliases

[('builtin/GetCurrentTime', 19.63803), ('apple/location.get', 7.84037), ('apple/reminders.get', 6.96445), ('apple/calendar.events.get', 6.12286), ('apple/weather.current', 5.04538), ('apple/battery.get', 4.61692), ('apple/contacts.get', 4.61692), ('apple/device.get', 4.61692), ('apple/photos.get', 4.61692), ('apple/health.workouts.get', 4.08145), ('apple/maps.eta', 2.95971), ('builtin/ScheduledTasks', 2.87555), ('apple/calendar.events.delete', 2.5715), ('apple/music.status', 2.45572), ('apple/calendar.events.update', 2.364), ('builtin/EditFile', 2.354), ('apple/maps.route', 2.26114), ('apple/weather.hourly', 1.99075), ('apple/calendar.events.list', 1.87013), ('apple/location.reverseGeocode', 1.5928), ('apple/alarms.create', 1.57832), ('apple/health.statistics', 1.53421), ('apple/reminders.list', 1.51429), ('apple/weather.daily', 1.46638), ('apple/location.geocode', 1.44283), ('apple/maps.show', 1.21419), ('apple/maps.search', 1.13698), ('apple/alarms.cancel', 1.03523), ('apple/music.playback', 0.87677), ('apple/reminders.create', 0.86688), ('apple/reminders.update', 0.86688), ('apple/health.sleep.query', 0.6728), ('apple/health.workouts.list', 0.62158), ('apple/calendar.events.create', 0.61489)]

N007 AskUserQuestions

dev · exact · target ['builtin/AskUserQuestions']

Exact declared name identifies this tool.

MethodFirst acceptable rankTop5
Keyword matching1builtin/AskUserQuestions → builtin/ScheduledTasks → apple/alarms.cancel → apple/calendar.events.delete → apple/device.get
BM25 · name＋description1builtin/AskUserQuestions → apple/shortcuts.run → apple/device.get → builtin/Memory → apple/calendar.events.delete
BM25 · name + description + Schema + native aliases1builtin/AskUserQuestions → apple/shortcuts.run → apple/device.get → builtin/Memory → apple/calendar.events.delete
BM25 · name + description + Schema + extended aliases1builtin/AskUserQuestions → apple/shortcuts.run → apple/device.get → builtin/Memory → apple/calendar.events.delete
Full rankings, within-method scores

Keyword matching

[('builtin/AskUserQuestions', 12.5), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.cancel', 1), ('apple/calendar.events.delete', 1), ('apple/device.get', 1), ('apple/maps.show', 1), ('apple/shortcuts.run', 1), ('builtin/Memory', 1)]

BM25 · name＋description

[('builtin/AskUserQuestions', 13.01363), ('apple/shortcuts.run', 2.74917), ('apple/device.get', 2.16548), ('builtin/Memory', 2.11424), ('apple/calendar.events.delete', 2.04177), ('apple/maps.show', 1.91078), ('apple/alarms.cancel', 1.89057), ('builtin/ScheduledTasks', 1.00102)]

BM25 · name + description + Schema + native aliases

[('builtin/AskUserQuestions', 37.14831), ('apple/shortcuts.run', 2.7231), ('apple/device.get', 2.17066), ('builtin/Memory', 2.08961), ('apple/calendar.events.delete', 2.06392), ('apple/maps.show', 1.90039), ('apple/alarms.cancel', 1.87912), ('builtin/ScheduledTasks', 1.24668)]

BM25 · name + description + Schema + extended aliases

[('builtin/AskUserQuestions', 47.20399), ('apple/shortcuts.run', 2.7231), ('apple/device.get', 2.17066), ('builtin/Memory', 2.08961), ('apple/calendar.events.delete', 2.06392), ('apple/maps.show', 1.90039), ('apple/alarms.cancel', 1.87912), ('builtin/ScheduledTasks', 1.24668)]

N008 ScheduledTasks

dev · exact · target ['builtin/ScheduledTasks']

Exact declared name identifies this tool.

MethodFirst acceptable rankTop5
Keyword matching1builtin/ScheduledTasks → apple/alarms.list → apple/links.open → apple/location.get → apple/music.playback
BM25 · name＋description1builtin/ScheduledTasks → apple/location.get → apple/alarms.list → apple/photos.get → apple/shortcuts.run
BM25 · name + description + Schema + native aliases1builtin/ScheduledTasks → apple/location.get → apple/alarms.list → apple/photos.get → apple/shortcuts.run
BM25 · name + description + Schema + extended aliases1builtin/ScheduledTasks → apple/location.get → apple/alarms.list → apple/photos.get → apple/shortcuts.run
Full rankings, within-method scores

Keyword matching

[('builtin/ScheduledTasks', 8.5), ('apple/alarms.list', 2), ('apple/links.open', 2), ('apple/location.get', 2), ('apple/music.playback', 2), ('apple/photos.albums.list', 2), ('apple/photos.get', 2), ('apple/photos.search', 2), ('apple/shortcuts.run', 2), ('apple/music.search', 1), ('apple/music.status', 1)]

BM25 · name＋description

[('builtin/ScheduledTasks', 3.60352), ('apple/location.get', 3.61593), ('apple/alarms.list', 3.19064), ('apple/photos.get', 2.99829), ('apple/shortcuts.run', 2.96847), ('apple/photos.albums.list', 2.80127), ('apple/photos.search', 2.4555), ('apple/music.playback', 2.33997), ('apple/links.open', 1.89434), ('apple/music.status', 1.62625), ('apple/music.search', 1.41383)]

BM25 · name + description + Schema + native aliases

[('builtin/ScheduledTasks', 13.74962), ('apple/location.get', 3.62458), ('apple/alarms.list', 3.17329), ('apple/photos.get', 2.97149), ('apple/shortcuts.run', 2.94032), ('apple/photos.albums.list', 2.79381), ('apple/photos.search', 2.40961), ('apple/music.playback', 2.29144), ('apple/links.open', 1.84022), ('apple/music.status', 1.62511), ('apple/music.search', 1.40119)]

BM25 · name + description + Schema + extended aliases

[('builtin/ScheduledTasks', 18.36004), ('apple/location.get', 3.62458), ('apple/alarms.list', 3.17329), ('apple/photos.get', 2.97149), ('apple/shortcuts.run', 2.94032), ('apple/photos.albums.list', 2.79381), ('apple/photos.search', 2.40961), ('apple/music.playback', 2.29144), ('apple/links.open', 1.84022), ('apple/music.status', 1.62511), ('apple/music.search', 1.40119)]

N009 calendar.events.create

dev · exact · target ['apple/calendar.events.create']

Exact declared name identifies this tool.

MethodFirst acceptable rankTop5
Keyword matching1apple/calendar.events.create → apple/calendar.events.get → apple/calendar.events.list → apple/calendar.events.update → apple/calendar.events.delete
BM25 · name＋description1apple/calendar.events.create → apple/calendar.events.get → apple/calendar.events.list → apple/calendar.events.delete → apple/calendar.events.update
BM25 · name + description + Schema + native aliases1apple/calendar.events.create → apple/calendar.events.get → apple/calendar.events.list → apple/calendar.events.update → apple/calendar.events.delete
BM25 · name + description + Schema + extended aliases1apple/calendar.events.create → apple/calendar.events.get → apple/calendar.events.list → apple/calendar.events.update → apple/calendar.events.delete
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.create', 13), ('apple/calendar.events.get', 8.5), ('apple/calendar.events.list', 8.5), ('apple/calendar.events.update', 8.5), ('apple/calendar.events.delete', 7.5), ('apple/alarms.create', 4), ('apple/calendar.list', 4), ('apple/contacts.create', 4), ('apple/reminders.create', 4), ('builtin/ScheduledTasks', 2.5), ('apple/links.open', 2), ('builtin/Memory', 1.5), ('apple/health.statistics', 1), ('apple/weather.daily', 1), ('builtin/WriteFile', 1)]

BM25 · name＋description

[('apple/calendar.events.create', 8.85655), ('apple/calendar.events.get', 7.47709), ('apple/calendar.events.list', 6.6899), ('apple/calendar.events.delete', 5.59086), ('apple/calendar.events.update', 5.58446), ('apple/calendar.list', 3.35399), ('apple/reminders.create', 3.05666), ('apple/contacts.create', 3.03074), ('builtin/WriteFile', 2.81316), ('apple/alarms.create', 2.50063), ('builtin/Memory', 2.11424), ('builtin/ScheduledTasks', 2.0673), ('apple/weather.daily', 1.47616), ('apple/health.statistics', 1.34576)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.create', 24.65544), ('apple/calendar.events.get', 18.39189), ('apple/calendar.events.list', 16.54769), ('apple/calendar.events.update', 15.55721), ('apple/calendar.events.delete', 15.25534), ('apple/reminders.create', 8.5428), ('apple/contacts.create', 8.51028), ('apple/alarms.create', 7.9143), ('apple/calendar.list', 7.72596), ('builtin/ScheduledTasks', 4.01713), ('builtin/Memory', 3.04182), ('builtin/WriteFile', 2.8775), ('apple/weather.daily', 1.29025), ('apple/health.statistics', 1.17133), ('apple/links.open', 0.66587)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.create', 24.65544), ('apple/calendar.events.get', 18.39189), ('apple/calendar.events.list', 16.54769), ('apple/calendar.events.update', 15.55721), ('apple/calendar.events.delete', 15.25534), ('apple/reminders.create', 8.5428), ('apple/contacts.create', 8.51028), ('apple/alarms.create', 7.9143), ('apple/calendar.list', 7.72596), ('builtin/WriteFile', 5.71516), ('builtin/ScheduledTasks', 4.01713), ('builtin/Memory', 3.04182), ('apple/weather.daily', 1.29025), ('apple/health.statistics', 1.17133), ('apple/links.open', 1.1424)]

N010 contacts.get

dev · exact · target ['apple/contacts.get']

Exact declared name identifies this tool.

MethodFirst acceptable rankTop5
Keyword matching1apple/contacts.get → apple/contacts.delete → apple/contacts.update → apple/calendar.events.get → apple/contacts.create
BM25 · name＋description1apple/contacts.get → apple/contacts.create → apple/contacts.delete → apple/contacts.list → apple/contacts.update
BM25 · name + description + Schema + native aliases1apple/contacts.get → apple/contacts.delete → apple/contacts.update → apple/contacts.create → apple/contacts.list
BM25 · name + description + Schema + extended aliases1apple/contacts.get → apple/contacts.delete → apple/contacts.update → apple/contacts.create → apple/contacts.list
Full rankings, within-method scores

Keyword matching

[('apple/contacts.get', 7.5), ('apple/contacts.delete', 4.5), ('apple/contacts.update', 4.5), ('apple/calendar.events.get', 4), ('apple/contacts.create', 4), ('apple/contacts.list', 4), ('apple/location.get', 4), ('apple/reminders.get', 4), ('builtin/GetCurrentTime', 4), ('apple/battery.get', 3), ('apple/device.get', 3), ('apple/health.workouts.get', 3), ('apple/photos.get', 3), ('builtin/ScheduledTasks', 2), ('apple/calendar.events.delete', 1.5), ('apple/calendar.events.update', 1.5)]

BM25 · name＋description

[('apple/contacts.get', 5.926), ('apple/contacts.create', 3.90839), ('apple/contacts.delete', 3.62112), ('apple/contacts.list', 3.59088), ('apple/contacts.update', 3.05754), ('apple/reminders.get', 2.69485), ('apple/calendar.events.get', 2.47013), ('builtin/GetCurrentTime', 2.44746), ('apple/location.get', 2.16872), ('apple/health.workouts.get', 1.8901), ('apple/device.get', 1.61526), ('apple/battery.get', 1.59592), ('apple/calendar.events.delete', 1.52299), ('apple/photos.get', 1.33936), ('apple/calendar.events.update', 1.25134), ('builtin/ScheduledTasks', 1.21774)]

BM25 · name + description + Schema + native aliases

[('apple/contacts.get', 16.58065), ('apple/contacts.delete', 11.14669), ('apple/contacts.update', 10.0687), ('apple/contacts.create', 9.89098), ('apple/contacts.list', 9.52374), ('apple/reminders.get', 6.96445), ('apple/location.get', 6.23604), ('apple/calendar.events.get', 6.12286), ('builtin/GetCurrentTime', 6.09011), ('apple/battery.get', 4.61692), ('apple/device.get', 4.61692), ('apple/photos.get', 4.61692), ('apple/health.workouts.get', 4.08145), ('builtin/ScheduledTasks', 3.15289), ('apple/calendar.events.delete', 2.5715), ('apple/calendar.events.update', 1.77448)]

BM25 · name + description + Schema + extended aliases

[('apple/contacts.get', 16.58065), ('apple/contacts.delete', 11.14669), ('apple/contacts.update', 10.0687), ('apple/contacts.create', 9.89098), ('apple/contacts.list', 9.52374), ('apple/reminders.get', 6.96445), ('apple/location.get', 6.23604), ('apple/calendar.events.get', 6.12286), ('builtin/GetCurrentTime', 6.09011), ('apple/battery.get', 4.61692), ('apple/device.get', 4.61692), ('apple/photos.get', 4.61692), ('apple/health.workouts.get', 4.08145), ('builtin/ScheduledTasks', 3.15289), ('apple/calendar.events.delete', 2.5715), ('apple/calendar.events.update', 1.77448)]

N011 list directory files

dev · phrase · target ['builtin/ListFiles']

Lists one directory level, not text contents.

MethodFirst acceptable rankTop5
Keyword matching1builtin/ListFiles → apple/reminders.list → builtin/EditFile → builtin/WriteFile → apple/alarms.list
BM25 · name＋description1builtin/ListFiles → builtin/Memory → builtin/WriteFile → builtin/ReadFile → builtin/EditFile
BM25 · name + description + Schema + native aliases1builtin/ListFiles → builtin/WriteFile → builtin/EditFile → builtin/Memory → builtin/ReadFile
BM25 · name + description + Schema + extended aliases1builtin/ListFiles → builtin/WriteFile → builtin/ReadFile → builtin/EditFile → builtin/Memory
Full rankings, within-method scores

Keyword matching

[('builtin/ListFiles', 8), ('apple/reminders.list', 4.5), ('builtin/EditFile', 4.5), ('builtin/WriteFile', 4.5), ('apple/alarms.list', 4), ('apple/calendar.events.list', 4), ('apple/calendar.list', 4), ('apple/contacts.list', 4), ('apple/photos.albums.list', 4), ('apple/reminders.lists', 4), ('builtin/Memory', 4), ('builtin/ReadFile', 4), ('apple/health.workouts.list', 3), ('apple/alarms.cancel', 1.5), ('apple/calendar.events.delete', 1.5), ('apple/calendar.events.update', 1.5), ('apple/photos.search', 1.5), ('apple/reminders.create', 1.5), ('apple/shortcuts.run', 1.5), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.create', 1), ('apple/calendar.events.get', 1), ('apple/clipboard.read', 1), ('apple/calendar.events.create', 0.5), ('apple/contacts.delete', 0.5), ('apple/contacts.get', 0.5), ('apple/contacts.update', 0.5), ('apple/health.workouts.get', 0.5)]

BM25 · name＋description

[('builtin/ListFiles', 9.76592), ('builtin/Memory', 7.90879), ('builtin/WriteFile', 3.71445), ('builtin/ReadFile', 3.47241), ('builtin/EditFile', 3.37407), ('apple/reminders.lists', 2.1669), ('apple/calendar.list', 1.97043), ('apple/clipboard.read', 1.93144), ('apple/calendar.events.list', 1.6149), ('apple/contacts.list', 1.49249), ('apple/reminders.list', 1.49249), ('apple/photos.search', 1.47053), ('apple/alarms.list', 1.4702), ('apple/calendar.events.get', 1.458), ('apple/alarms.create', 1.37763), ('apple/photos.albums.list', 1.34012), ('apple/health.workouts.list', 1.33902), ('apple/reminders.create', 1.28652), ('apple/calendar.events.delete', 1.12484), ('apple/alarms.cancel', 1.04154), ('apple/shortcuts.run', 0.97937), ('apple/calendar.events.update', 0.92421), ('builtin/ScheduledTasks', 0.89939)]

BM25 · name + description + Schema + native aliases

[('builtin/ListFiles', 14.51045), ('builtin/WriteFile', 10.32191), ('builtin/EditFile', 9.25924), ('builtin/Memory', 8.68476), ('builtin/ReadFile', 8.54967), ('apple/reminders.lists', 4.12289), ('apple/reminders.list', 3.82967), ('apple/calendar.list', 3.8154), ('apple/contacts.list', 3.34463), ('apple/alarms.list', 3.32653), ('apple/calendar.events.list', 3.16618), ('apple/photos.albums.list', 2.93791), ('apple/health.workouts.list', 2.24993), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.65154), ('apple/shortcuts.run', 1.49016), ('apple/reminders.create', 1.46978), ('apple/calendar.events.delete', 1.41756), ('apple/alarms.cancel', 1.37363), ('builtin/ScheduledTasks', 1.26107), ('apple/calendar.events.get', 1.12534), ('apple/alarms.create', 1.02045), ('apple/calendar.events.update', 0.97819), ('apple/contacts.delete', 0.60095), ('apple/contacts.get', 0.60095), ('apple/health.workouts.get', 0.58448), ('apple/contacts.update', 0.39141), ('apple/calendar.events.create', 0.30692)]

BM25 · name + description + Schema + extended aliases

[('builtin/ListFiles', 20.22412), ('builtin/WriteFile', 12.98291), ('builtin/ReadFile', 11.13883), ('builtin/EditFile', 9.25924), ('builtin/Memory', 8.68476), ('apple/reminders.lists', 4.12289), ('apple/reminders.list', 3.82967), ('apple/calendar.list', 3.8154), ('apple/contacts.list', 3.34463), ('apple/alarms.list', 3.32653), ('apple/calendar.events.list', 3.16618), ('apple/photos.albums.list', 2.93791), ('apple/health.workouts.list', 2.24993), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.65154), ('apple/shortcuts.run', 1.49016), ('apple/reminders.create', 1.46978), ('apple/calendar.events.delete', 1.41756), ('apple/alarms.cancel', 1.37363), ('builtin/ScheduledTasks', 1.26107), ('apple/calendar.events.get', 1.12534), ('apple/alarms.create', 1.02045), ('apple/calendar.events.update', 0.97819), ('apple/contacts.delete', 0.60095), ('apple/contacts.get', 0.60095), ('apple/health.workouts.get', 0.58448), ('apple/contacts.update', 0.39141), ('apple/calendar.events.create', 0.30692)]

N012 read UTF-8 file line range

dev · phrase · target ['builtin/ReadFile']

Reads UTF-8 with line offset and limit.

MethodFirst acceptable rankTop5
Keyword matching1builtin/ReadFile → builtin/WriteFile → builtin/EditFile → apple/clipboard.read → builtin/ListFiles
BM25 · name＋description1builtin/ReadFile → builtin/WriteFile → builtin/Memory → apple/calendar.events.list → builtin/EditFile
BM25 · name + description + Schema + native aliases1builtin/ReadFile → builtin/WriteFile → builtin/EditFile → builtin/Memory → apple/clipboard.read
BM25 · name + description + Schema + extended aliases1builtin/ReadFile → builtin/WriteFile → builtin/EditFile → builtin/ListFiles → builtin/Memory
Full rankings, within-method scores

Keyword matching

[('builtin/ReadFile', 12.5), ('builtin/WriteFile', 6.5), ('builtin/EditFile', 5.5), ('apple/clipboard.read', 5), ('builtin/ListFiles', 3), ('builtin/Memory', 2.5), ('apple/calendar.events.list', 2), ('apple/photos.search', 2), ('apple/weather.daily', 2), ('apple/weather.hourly', 2), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.list', 1), ('apple/battery.get', 1), ('apple/contacts.get', 1), ('apple/device.get', 1), ('apple/health.sleep.query', 1), ('apple/health.statistics', 1), ('apple/health.workouts.get', 1), ('apple/health.workouts.list', 1), ('apple/maps.eta', 1), ('apple/photos.get', 1), ('apple/weather.current', 1), ('apple/shortcuts.run', 0.5)]

BM25 · name＋description

[('builtin/ReadFile', 22.77865), ('builtin/WriteFile', 12.41528), ('builtin/Memory', 5.09531), ('apple/calendar.events.list', 4.77181), ('builtin/EditFile', 4.73732), ('apple/clipboard.read', 3.27541), ('builtin/ListFiles', 3.05349), ('apple/weather.hourly', 2.97029), ('apple/weather.daily', 2.76097), ('apple/photos.search', 2.20558), ('apple/contacts.get', 1.59613), ('apple/health.workouts.get', 1.26659), ('apple/weather.current', 1.24126), ('apple/health.workouts.list', 1.21491), ('apple/device.get', 1.08242), ('apple/health.sleep.query', 1.08242), ('apple/battery.get', 1.06946), ('apple/maps.eta', 1.06946), ('builtin/ScheduledTasks', 1.03334), ('apple/alarms.list', 0.95511), ('apple/photos.get', 0.89753), ('apple/health.statistics', 0.71734)]

BM25 · name + description + Schema + native aliases

[('builtin/ReadFile', 33.5644), ('builtin/WriteFile', 19.22174), ('builtin/EditFile', 10.6499), ('builtin/Memory', 5.95572), ('apple/clipboard.read', 5.85716), ('builtin/ListFiles', 5.80426), ('apple/calendar.events.list', 4.83033), ('apple/weather.hourly', 2.93457), ('apple/weather.daily', 2.71789), ('apple/photos.search', 2.07452), ('apple/contacts.get', 1.60914), ('apple/health.workouts.get', 1.30421), ('apple/health.workouts.list', 1.24611), ('apple/weather.current', 1.23137), ('builtin/ScheduledTasks', 1.14906), ('apple/health.sleep.query', 1.09922), ('apple/device.get', 1.08501), ('apple/battery.get', 1.07116), ('apple/maps.eta', 1.07116), ('apple/alarms.list', 0.94991), ('apple/photos.get', 0.88951), ('apple/shortcuts.run', 0.7661), ('apple/health.statistics', 0.70318)]

BM25 · name + description + Schema + extended aliases

[('builtin/ReadFile', 40.76941), ('builtin/WriteFile', 21.88274), ('builtin/EditFile', 10.24338), ('builtin/ListFiles', 5.80426), ('builtin/Memory', 5.48475), ('apple/calendar.events.list', 5.40824), ('apple/clipboard.read', 4.67189), ('apple/weather.hourly', 2.6901), ('apple/weather.daily', 2.49147), ('apple/contacts.get', 2.14254), ('apple/photos.search', 1.86366), ('apple/health.workouts.get', 1.82646), ('apple/device.get', 1.80027), ('apple/battery.get', 1.79046), ('apple/health.workouts.list', 1.72237), ('apple/alarms.list', 1.64891), ('apple/photos.get', 1.63326), ('apple/calendar.events.get', 0.97668), ('apple/location.get', 0.97668), ('apple/reminders.get', 0.951), ('apple/contacts.list', 0.92664), ('apple/photos.albums.list', 0.92664), ('apple/calendar.list', 0.9035), ('apple/reminders.list', 0.88148), ('apple/weather.current', 0.87142), ('builtin/ScheduledTasks', 0.81317), ('apple/health.sleep.query', 0.77789), ('apple/shortcuts.run', 0.7661), ('apple/maps.eta', 0.75804), ('apple/health.statistics', 0.49762)]

N013 create UTF-8 text file

dev · phrase · target ['builtin/WriteFile']

Creates a text file; full replacement requires explicit overwrite.

MethodFirst acceptable rankTop5
Keyword matching1builtin/WriteFile → builtin/ReadFile → builtin/EditFile → apple/alarms.create → apple/calendar.events.create
BM25 · name＋description1builtin/WriteFile → builtin/ReadFile → builtin/EditFile → builtin/Memory → apple/clipboard.read
BM25 · name + description + Schema + native aliases1builtin/WriteFile → builtin/ReadFile → builtin/EditFile → builtin/Memory → apple/reminders.create
BM25 · name + description + Schema + extended aliases1builtin/WriteFile → builtin/ReadFile → builtin/EditFile → builtin/Memory → apple/reminders.create
Full rankings, within-method scores

Keyword matching

[('builtin/WriteFile', 7.5), ('builtin/ReadFile', 7), ('builtin/EditFile', 6), ('apple/alarms.create', 4), ('apple/calendar.events.create', 4), ('apple/contacts.create', 4), ('apple/reminders.create', 4), ('builtin/Memory', 3.5), ('builtin/ListFiles', 3), ('apple/clipboard.read', 2), ('apple/shortcuts.run', 2), ('apple/clipboard.write', 1.5), ('builtin/ScheduledTasks', 1.5), ('apple/contacts.update', 1), ('apple/photos.search', 1), ('builtin/AskUserQuestions', 1)]

BM25 · name＋description

[('builtin/WriteFile', 15.22843), ('builtin/ReadFile', 15.17054), ('builtin/EditFile', 6.10137), ('builtin/Memory', 5.5887), ('apple/clipboard.read', 5.02443), ('apple/clipboard.write', 3.14747), ('apple/reminders.create', 3.05666), ('builtin/ListFiles', 3.05349), ('apple/contacts.create', 3.03074), ('apple/calendar.events.create', 2.94033), ('apple/alarms.create', 2.50063), ('builtin/AskUserQuestions', 2.16548), ('builtin/ScheduledTasks', 2.0673), ('apple/contacts.update', 1.89057), ('apple/shortcuts.run', 1.77773), ('apple/photos.search', 1.47053)]

BM25 · name + description + Schema + native aliases

[('builtin/WriteFile', 22.09924), ('builtin/ReadFile', 20.35009), ('builtin/EditFile', 13.38991), ('builtin/Memory', 8.74272), ('apple/reminders.create', 8.5428), ('apple/contacts.create', 8.51028), ('apple/calendar.events.create', 7.99065), ('apple/alarms.create', 7.9143), ('builtin/ListFiles', 5.80426), ('apple/clipboard.read', 4.69546), ('apple/clipboard.write', 4.5613), ('apple/shortcuts.run', 3.7391), ('builtin/ScheduledTasks', 2.2988), ('builtin/AskUserQuestions', 2.06218), ('apple/contacts.update', 1.76213), ('apple/photos.search', 1.35321)]

BM25 · name + description + Schema + extended aliases

[('builtin/WriteFile', 30.10191), ('builtin/ReadFile', 25.21369), ('builtin/EditFile', 15.58262), ('builtin/Memory', 8.6627), ('apple/reminders.create', 8.5428), ('apple/contacts.create', 8.51028), ('apple/calendar.events.create', 7.99065), ('apple/alarms.create', 7.9143), ('builtin/ListFiles', 5.80426), ('apple/clipboard.read', 4.52478), ('apple/clipboard.write', 4.2922), ('apple/shortcuts.run', 3.5637), ('builtin/ScheduledTasks', 2.2988), ('builtin/AskUserQuestions', 1.94052), ('apple/contacts.update', 1.65817), ('apple/photos.search', 1.35321)]

N014 replace exact text segment

dev · phrase · target ['builtin/EditFile']

Replaces one unique nonempty text segment.

MethodFirst acceptable rankTop5
Keyword matching1builtin/EditFile → apple/clipboard.write → apple/shortcuts.run → builtin/Memory → apple/contacts.update
BM25 · name＋description1builtin/EditFile → apple/clipboard.write → builtin/Memory → apple/contacts.update → apple/shortcuts.run
BM25 · name + description + Schema + native aliases1builtin/EditFile → apple/clipboard.write → builtin/Memory → apple/shortcuts.run → apple/contacts.update
BM25 · name + description + Schema + extended aliases1builtin/EditFile → apple/clipboard.write → apple/contacts.update → builtin/Memory → apple/shortcuts.run
Full rankings, within-method scores

Keyword matching

[('builtin/EditFile', 4), ('apple/clipboard.write', 3), ('apple/shortcuts.run', 3), ('builtin/Memory', 3), ('apple/contacts.update', 2), ('apple/links.open', 1.5), ('apple/clipboard.read', 1), ('apple/music.playback', 1), ('apple/photos.albums.list', 1), ('builtin/AskUserQuestions', 1), ('builtin/ReadFile', 1), ('apple/maps.search', 0.5)]

BM25 · name＋description

[('builtin/EditFile', 10.78592), ('apple/clipboard.write', 5.45158), ('builtin/Memory', 5.09184), ('apple/contacts.update', 4.0728), ('apple/shortcuts.run', 4.00715), ('apple/clipboard.read', 3.093), ('builtin/ReadFile', 2.85815), ('builtin/AskUserQuestions', 2.16548), ('apple/photos.albums.list', 2.10385), ('apple/music.playback', 1.61753), ('apple/links.open', 1.42271)]

BM25 · name + description + Schema + native aliases

[('builtin/EditFile', 13.25357), ('apple/clipboard.write', 8.05155), ('builtin/Memory', 6.46464), ('apple/shortcuts.run', 5.55083), ('apple/contacts.update', 3.93115), ('apple/clipboard.read', 2.89298), ('builtin/ReadFile', 2.74541), ('builtin/AskUserQuestions', 2.06218), ('apple/photos.albums.list', 1.67313), ('apple/music.playback', 1.58398), ('apple/links.open', 1.31248), ('apple/maps.search', 1.00735)]

BM25 · name + description + Schema + extended aliases

[('builtin/EditFile', 17.77812), ('apple/clipboard.write', 7.47537), ('apple/contacts.update', 6.09147), ('builtin/Memory', 6.06225), ('apple/shortcuts.run', 5.37543), ('builtin/ReadFile', 5.01984), ('apple/clipboard.read', 2.7223), ('builtin/WriteFile', 2.50401), ('apple/calendar.events.update', 2.49193), ('apple/reminders.update', 2.43265), ('builtin/AskUserQuestions', 1.94052), ('apple/photos.albums.list', 1.67313), ('apple/music.playback', 1.37228), ('apple/links.open', 1.31248), ('apple/maps.search', 1.00735)]

N015 lasting user memories Markdown

dev · phrase · target ['builtin/Memory']

Maintains persistent Markdown user memories.

MethodFirst acceptable rankTop5
Keyword matching1builtin/Memory → builtin/AskUserQuestions → builtin/ScheduledTasks → apple/alarms.cancel → apple/calendar.events.delete
BM25 · name＋description1builtin/Memory → builtin/AskUserQuestions → builtin/ScheduledTasks → apple/device.get → apple/calendar.events.delete
BM25 · name + description + Schema + native aliases1builtin/Memory → builtin/AskUserQuestions → builtin/ScheduledTasks → apple/device.get → apple/calendar.events.delete
BM25 · name + description + Schema + extended aliases1builtin/Memory → builtin/AskUserQuestions → builtin/ScheduledTasks → apple/device.get → apple/calendar.events.delete
Full rankings, within-method scores

Keyword matching

[('builtin/Memory', 8), ('builtin/AskUserQuestions', 4), ('builtin/ScheduledTasks', 2.5), ('apple/alarms.cancel', 1), ('apple/calendar.events.delete', 1), ('apple/device.get', 1), ('apple/maps.show', 1)]

BM25 · name＋description

[('builtin/Memory', 14.15204), ('builtin/AskUserQuestions', 3.28237), ('builtin/ScheduledTasks', 2.54904), ('apple/device.get', 2.16548), ('apple/calendar.events.delete', 2.04177), ('apple/maps.show', 1.91078), ('apple/alarms.cancel', 1.89057)]

BM25 · name + description + Schema + native aliases

[('builtin/Memory', 25.89397), ('builtin/AskUserQuestions', 8.40935), ('builtin/ScheduledTasks', 2.74449), ('apple/device.get', 2.17066), ('apple/calendar.events.delete', 2.06392), ('apple/maps.show', 1.90039), ('apple/alarms.cancel', 1.87912)]

BM25 · name + description + Schema + extended aliases

[('builtin/Memory', 36.41929), ('builtin/AskUserQuestions', 8.40935), ('builtin/ScheduledTasks', 2.74449), ('apple/device.get', 2.17066), ('apple/calendar.events.delete', 2.06392), ('apple/maps.show', 1.90039), ('apple/alarms.cancel', 1.87912)]

N016 read events overlapping time range

dev · phrase · target ['apple/calendar.events.list']

Returns events overlapping start/end; no get ID required.

MethodFirst acceptable rankTop5
Keyword matching1apple/calendar.events.list → apple/calendar.events.create → apple/calendar.events.update → builtin/ReadFile → apple/calendar.events.delete
BM25 · name＋description1apple/calendar.events.list → builtin/ReadFile → apple/health.sleep.query → apple/weather.hourly → apple/calendar.events.get
BM25 · name + description + Schema + native aliases1apple/calendar.events.list → apple/calendar.events.get → apple/calendar.events.delete → apple/calendar.events.update → apple/calendar.events.create
BM25 · name + description + Schema + extended aliases1apple/calendar.events.list → apple/calendar.events.get → apple/calendar.events.delete → apple/calendar.events.update → apple/calendar.events.create
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.list', 8.5), ('apple/calendar.events.create', 5), ('apple/calendar.events.update', 5), ('builtin/ReadFile', 5), ('apple/calendar.events.delete', 4.5), ('apple/calendar.events.get', 4.5), ('apple/clipboard.read', 4), ('builtin/GetCurrentTime', 4), ('apple/weather.hourly', 3), ('apple/health.sleep.query', 2.5), ('builtin/ScheduledTasks', 2.5), ('apple/maps.eta', 2), ('apple/weather.daily', 2), ('apple/alarms.create', 1.5), ('apple/health.statistics', 1.5), ('apple/health.workouts.list', 1.5), ('apple/reminders.list', 1.5), ('apple/alarms.cancel', 1), ('apple/alarms.list', 1), ('apple/battery.get', 1), ('apple/contacts.get', 1), ('apple/device.get', 1), ('apple/health.workouts.get', 1), ('apple/location.get', 1), ('apple/maps.route', 1), ('apple/music.status', 1), ('apple/photos.get', 1), ('apple/photos.search', 1), ('apple/weather.current', 1), ('builtin/EditFile', 1), ('builtin/Memory', 1), ('apple/reminders.create', 0.5), ('apple/reminders.update', 0.5)]

BM25 · name＋description

[('apple/calendar.events.list', 13.3704), ('builtin/ReadFile', 5.32006), ('apple/health.sleep.query', 4.43122), ('apple/weather.hourly', 4.38035), ('apple/calendar.events.get', 4.3717), ('apple/calendar.events.delete', 3.6762), ('apple/maps.eta', 3.47079), ('apple/calendar.events.update', 3.30335), ('builtin/GetCurrentTime', 2.98924), ('apple/calendar.events.create', 2.90674), ('apple/weather.daily', 2.76097), ('apple/location.get', 2.42087), ('apple/music.status', 1.7197), ('apple/reminders.list', 1.62576), ('builtin/Memory', 1.62085), ('apple/contacts.get', 1.59613), ('apple/maps.route', 1.59099), ('apple/alarms.cancel', 1.57416), ('apple/alarms.create', 1.45133), ('builtin/EditFile', 1.36324), ('apple/clipboard.read', 1.34398), ('apple/health.workouts.get', 1.26659), ('apple/weather.current', 1.24126), ('apple/health.workouts.list', 1.21491), ('apple/device.get', 1.08242), ('apple/battery.get', 1.06946), ('builtin/ScheduledTasks', 1.03334), ('apple/alarms.list', 0.95511), ('apple/photos.get', 0.89753), ('apple/photos.search', 0.73505), ('apple/health.statistics', 0.71734)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.list', 18.39446), ('apple/calendar.events.get', 11.55716), ('apple/calendar.events.delete', 10.69936), ('apple/calendar.events.update', 10.19764), ('apple/calendar.events.create', 9.16186), ('builtin/ReadFile', 8.22885), ('apple/health.sleep.query', 5.1728), ('builtin/GetCurrentTime', 4.85869), ('apple/clipboard.read', 4.05468), ('apple/weather.hourly', 3.85632), ('apple/weather.daily', 2.71789), ('apple/maps.eta', 2.66169), ('builtin/ScheduledTasks', 2.28654), ('apple/health.workouts.list', 1.86769), ('builtin/Memory', 1.61114), ('apple/contacts.get', 1.60914), ('apple/location.get', 1.60433), ('apple/alarms.create', 1.57832), ('apple/reminders.list', 1.51429), ('builtin/EditFile', 1.39066), ('apple/health.statistics', 1.33858), ('apple/health.workouts.get', 1.30421), ('apple/weather.current', 1.23137), ('apple/music.status', 1.13704), ('apple/device.get', 1.08501), ('apple/battery.get', 1.07116), ('apple/maps.route', 1.04695), ('apple/alarms.cancel', 1.03523), ('apple/alarms.list', 0.94991), ('apple/photos.get', 0.88951), ('apple/reminders.create', 0.86688), ('apple/reminders.update', 0.86688), ('apple/photos.search', 0.72131)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.list', 18.97237), ('apple/calendar.events.get', 12.53384), ('apple/calendar.events.delete', 10.69936), ('apple/calendar.events.update', 10.19764), ('apple/calendar.events.create', 9.16186), ('builtin/ReadFile', 7.87333), ('builtin/GetCurrentTime', 6.46661), ('apple/health.sleep.query', 4.85147), ('apple/weather.hourly', 3.61185), ('apple/clipboard.read', 2.86941), ('apple/location.get', 2.58101), ('apple/weather.daily', 2.49147), ('apple/reminders.list', 2.39578), ('apple/maps.eta', 2.34857), ('apple/health.workouts.list', 2.34395), ('apple/contacts.get', 2.14254), ('builtin/ScheduledTasks', 1.95065), ('apple/health.workouts.get', 1.82646), ('apple/device.get', 1.80027), ('apple/battery.get', 1.79046), ('apple/alarms.list', 1.64891), ('apple/photos.get', 1.63326), ('apple/alarms.create', 1.57832), ('builtin/Memory', 1.14017), ('apple/music.status', 1.13704), ('apple/health.statistics', 1.13302), ('apple/maps.route', 1.04695), ('apple/alarms.cancel', 1.03523), ('builtin/EditFile', 0.98414), ('apple/reminders.get', 0.951), ('apple/contacts.list', 0.92664), ('apple/photos.albums.list', 0.92664), ('apple/calendar.list', 0.9035), ('apple/weather.current', 0.87142), ('apple/reminders.create', 0.86688), ('apple/reminders.update', 0.86688), ('apple/photos.search', 0.51046)]

N017 mark reminder completion

dev · phrase · target ['apple/reminders.complete']

Sets completed on an existing reminder.

MethodFirst acceptable rankTop5
Keyword matching1apple/reminders.complete → apple/reminders.delete → apple/reminders.get → apple/reminders.update → apple/reminders.create
BM25 · name＋description1apple/reminders.complete → apple/reminders.get → apple/reminders.lists → apple/reminders.delete → apple/reminders.update
BM25 · name + description + Schema + native aliases1apple/reminders.complete → apple/reminders.get → apple/reminders.delete → apple/reminders.update → apple/reminders.lists
BM25 · name + description + Schema + extended aliases1apple/reminders.complete → apple/reminders.get → apple/reminders.delete → apple/reminders.update → apple/reminders.lists
Full rankings, within-method scores

Keyword matching

[('apple/reminders.complete', 5.5), ('apple/reminders.delete', 4.5), ('apple/reminders.get', 4.5), ('apple/reminders.update', 4.5), ('apple/reminders.create', 4), ('apple/reminders.list', 4), ('apple/reminders.lists', 4), ('apple/links.open', 2), ('apple/shortcuts.run', 1), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('apple/reminders.complete', 7.68922), ('apple/reminders.get', 3.61281), ('apple/reminders.lists', 3.54122), ('apple/reminders.delete', 3.2513), ('apple/reminders.update', 3.13715), ('apple/reminders.list', 3.11095), ('apple/reminders.create', 3.05666), ('apple/shortcuts.run', 2.74917)]

BM25 · name + description + Schema + native aliases

[('apple/reminders.complete', 13.66018), ('apple/reminders.get', 9.60564), ('apple/reminders.delete', 9.13089), ('apple/reminders.update', 8.37383), ('apple/reminders.lists', 8.13862), ('apple/reminders.list', 7.8455), ('apple/reminders.create', 7.53834), ('apple/shortcuts.run', 2.7231), ('builtin/ScheduledTasks', 0.90407), ('apple/links.open', 0.70568)]

BM25 · name + description + Schema + extended aliases

[('apple/reminders.complete', 13.66018), ('apple/reminders.get', 9.60564), ('apple/reminders.delete', 9.13089), ('apple/reminders.update', 8.37383), ('apple/reminders.lists', 8.13862), ('apple/reminders.list', 7.8455), ('apple/reminders.create', 7.53834), ('apple/shortcuts.run', 2.7231), ('apple/links.open', 1.2107), ('builtin/ScheduledTasks', 0.90407)]

N018 find contacts by phone email

dev · phrase · target ['apple/contacts.list']

List query matches name, organization, phone and email.

MethodFirst acceptable rankTop5
Keyword matching2apple/contacts.update → apple/contacts.list → apple/contacts.create → apple/contacts.delete → apple/contacts.get
BM25 · name＋description1apple/contacts.list → apple/contacts.update → apple/contacts.get → apple/contacts.create → apple/contacts.delete
BM25 · name + description + Schema + native aliases2apple/contacts.update → apple/contacts.list → apple/contacts.create → apple/contacts.get → apple/contacts.delete
BM25 · name + description + Schema + extended aliases2apple/contacts.update → apple/contacts.list → apple/contacts.create → apple/contacts.get → apple/contacts.delete
Full rankings, within-method scores

Keyword matching

[('apple/contacts.update', 7.5), ('apple/contacts.list', 6), ('apple/contacts.create', 5), ('apple/contacts.delete', 4.5), ('apple/contacts.get', 4.5), ('builtin/EditFile', 0.5), ('builtin/ListFiles', 0.5), ('builtin/ReadFile', 0.5), ('builtin/ScheduledTasks', 0.5), ('builtin/WriteFile', 0.5)]

BM25 · name＋description

[('apple/contacts.list', 9.62989), ('apple/contacts.update', 8.90489), ('apple/contacts.get', 4.06233), ('apple/contacts.create', 3.90839), ('apple/contacts.delete', 3.62112)]

BM25 · name + description + Schema + native aliases

[('apple/contacts.update', 18.19025), ('apple/contacts.list', 14.14966), ('apple/contacts.create', 13.7313), ('apple/contacts.get', 11.96374), ('apple/contacts.delete', 11.14669), ('builtin/ListFiles', 1.0922), ('builtin/ReadFile', 0.98816), ('builtin/WriteFile', 0.95189), ('builtin/EditFile', 0.90222), ('builtin/ScheduledTasks', 0.86526)]

BM25 · name + description + Schema + extended aliases

[('apple/contacts.update', 18.19025), ('apple/contacts.list', 15.59283), ('apple/contacts.create', 13.7313), ('apple/contacts.get', 13.52704), ('apple/contacts.delete', 11.14669), ('apple/music.search', 1.70526), ('apple/photos.search', 1.70526), ('apple/battery.get', 1.60792), ('apple/device.get', 1.60792), ('apple/maps.search', 1.60792), ('apple/photos.get', 1.5633), ('apple/alarms.list', 1.5211), ('apple/calendar.events.get', 1.5211), ('apple/location.get', 1.5211), ('apple/reminders.get', 1.48111), ('apple/photos.albums.list', 1.44317), ('apple/calendar.events.list', 1.40712), ('apple/calendar.list', 1.40712), ('apple/health.workouts.get', 1.40712), ('apple/reminders.list', 1.37284), ('apple/health.workouts.list', 1.30904), ('builtin/ListFiles', 1.0922), ('builtin/ReadFile', 0.98816), ('builtin/WriteFile', 0.95189), ('builtin/EditFile', 0.90222), ('builtin/ScheduledTasks', 0.86526)]

N019 route distance travel time ordered steps

dev · phrase · target ['apple/maps.route']

Route returns ordered steps; eta does not.

MethodFirst acceptable rankTop5
Keyword matching1apple/maps.route → apple/maps.eta → builtin/GetCurrentTime → apple/health.statistics → apple/weather.hourly
BM25 · name＋description1apple/maps.route → apple/maps.eta → apple/weather.hourly → apple/health.workouts.get → builtin/GetCurrentTime
BM25 · name + description + Schema + native aliases1apple/maps.route → apple/maps.eta → builtin/GetCurrentTime → apple/health.statistics → apple/health.workouts.get
BM25 · name + description + Schema + extended aliases1apple/maps.route → apple/maps.eta → builtin/GetCurrentTime → apple/health.statistics → apple/health.workouts.get
Full rankings, within-method scores

Keyword matching

[('apple/maps.route', 9), ('apple/maps.eta', 4), ('builtin/GetCurrentTime', 4), ('apple/health.statistics', 2), ('apple/weather.hourly', 2), ('apple/alarms.create', 1.5), ('apple/calendar.events.list', 1.5), ('apple/reminders.list', 1.5), ('apple/alarms.cancel', 1), ('apple/health.workouts.get', 1), ('apple/location.get', 1), ('apple/music.status', 1), ('apple/weather.daily', 1), ('builtin/ScheduledTasks', 1), ('apple/calendar.events.create', 0.5), ('apple/calendar.events.update', 0.5), ('apple/health.sleep.query', 0.5), ('apple/health.workouts.list', 0.5), ('apple/reminders.create', 0.5), ('apple/reminders.update', 0.5)]

BM25 · name＋description

[('apple/maps.route', 18.69891), ('apple/maps.eta', 11.96939), ('apple/weather.hourly', 3.74555), ('apple/health.workouts.get', 3.49453), ('builtin/GetCurrentTime', 2.98924), ('apple/location.get', 2.42087), ('apple/health.statistics', 2.21932), ('apple/weather.daily', 2.1709), ('apple/calendar.events.list', 1.82518), ('apple/music.status', 1.7197), ('apple/reminders.list', 1.62576), ('apple/alarms.cancel', 1.57416), ('apple/alarms.create', 1.45133)]

BM25 · name + description + Schema + native aliases

[('apple/maps.route', 25.54105), ('apple/maps.eta', 10.81515), ('builtin/GetCurrentTime', 4.85869), ('apple/health.statistics', 4.01729), ('apple/health.workouts.get', 3.59831), ('apple/weather.hourly', 3.22915), ('apple/weather.daily', 2.13703), ('apple/calendar.events.list', 1.87013), ('apple/location.get', 1.60433), ('apple/alarms.create', 1.57832), ('apple/reminders.list', 1.51429), ('apple/music.status', 1.13704), ('apple/alarms.cancel', 1.03523), ('builtin/ScheduledTasks', 0.95366), ('apple/reminders.create', 0.86688), ('apple/reminders.update', 0.86688), ('apple/health.sleep.query', 0.6728), ('apple/health.workouts.list', 0.62158), ('apple/calendar.events.create', 0.61489), ('apple/calendar.events.update', 0.58952)]

BM25 · name + description + Schema + extended aliases

[('apple/maps.route', 25.54105), ('apple/maps.eta', 10.81515), ('builtin/GetCurrentTime', 6.46661), ('apple/health.statistics', 4.01729), ('apple/health.workouts.get', 3.59831), ('apple/weather.hourly', 3.22915), ('apple/weather.daily', 2.13703), ('apple/calendar.events.list', 1.87013), ('apple/location.get', 1.60433), ('apple/alarms.create', 1.57832), ('apple/reminders.list', 1.51429), ('apple/music.status', 1.13704), ('apple/alarms.cancel', 1.03523), ('builtin/ScheduledTasks', 0.95366), ('apple/reminders.create', 0.86688), ('apple/reminders.update', 0.86688), ('apple/health.sleep.query', 0.6728), ('apple/health.workouts.list', 0.62158), ('apple/calendar.events.create', 0.61489), ('apple/calendar.events.update', 0.58952)]

N020 hourly forecast explicit coordinates

dev · phrase · target ['apple/weather.hourly']

Hourly forecast with explicit coordinates.

MethodFirst acceptable rankTop5
Keyword matching1apple/weather.hourly → apple/weather.current → apple/weather.daily → apple/location.reverseGeocode → apple/maps.eta
BM25 · name＋description1apple/weather.hourly → apple/weather.daily → apple/weather.current → apple/location.reverseGeocode → apple/maps.eta
BM25 · name + description + Schema + native aliases1apple/weather.hourly → apple/weather.daily → apple/weather.current → apple/location.reverseGeocode → apple/maps.eta
BM25 · name + description + Schema + extended aliases1apple/weather.hourly → apple/weather.daily → apple/weather.current → apple/location.reverseGeocode → apple/maps.eta
Full rankings, within-method scores

Keyword matching

[('apple/weather.hourly', 7), ('apple/weather.current', 3), ('apple/weather.daily', 3), ('apple/location.reverseGeocode', 2), ('apple/maps.eta', 2), ('apple/maps.show', 2), ('apple/maps.search', 1.5), ('apple/calendar.events.create', 1), ('apple/calendar.events.update', 1), ('apple/links.open', 1), ('apple/location.geocode', 1), ('apple/maps.route', 1), ('apple/alarms.create', 0.5), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('apple/weather.hourly', 9.77233), ('apple/weather.daily', 6.01578), ('apple/weather.current', 5.46601), ('apple/location.reverseGeocode', 4.34433), ('apple/maps.eta', 3.82421), ('apple/maps.show', 3.41531), ('apple/location.geocode', 1.76939), ('apple/maps.route', 1.50453), ('apple/maps.search', 1.41383), ('apple/calendar.events.create', 1.38598), ('apple/calendar.events.update', 1.32092), ('apple/links.open', 1.13446)]

BM25 · name + description + Schema + native aliases

[('apple/weather.hourly', 17.38789), ('apple/weather.daily', 5.87061), ('apple/weather.current', 5.33488), ('apple/location.reverseGeocode', 4.3525), ('apple/maps.eta', 3.74141), ('apple/maps.show', 3.31792), ('apple/maps.search', 2.07878), ('apple/location.geocode', 1.68446), ('apple/maps.route', 1.41753), ('apple/calendar.events.create', 1.31346), ('apple/calendar.events.update', 1.24802), ('apple/links.open', 1.10205), ('apple/alarms.create', 0.85051), ('builtin/ScheduledTasks', 0.76414)]

BM25 · name + description + Schema + extended aliases

[('apple/weather.hourly', 17.38789), ('apple/weather.daily', 5.87061), ('apple/weather.current', 5.33488), ('apple/location.reverseGeocode', 4.3525), ('apple/maps.eta', 3.74141), ('apple/maps.show', 3.31792), ('apple/maps.search', 2.07878), ('apple/location.geocode', 1.68446), ('apple/maps.route', 1.41753), ('apple/calendar.events.create', 1.31346), ('apple/calendar.events.update', 1.24802), ('apple/links.open', 1.10205), ('apple/alarms.create', 0.85051), ('builtin/ScheduledTasks', 0.76414)]

N021 sleep stage intervals source

dev · phrase · target ['apple/health.sleep.query']

Returns sleep intervals and source, not a summed night.

MethodFirst acceptable rankTop5
Keyword matching1apple/health.sleep.query → apple/weather.current → builtin/ScheduledTasks
BM25 · name＋description1apple/health.sleep.query → apple/weather.current
BM25 · name + description + Schema + native aliases1apple/health.sleep.query → apple/weather.current → builtin/ScheduledTasks
BM25 · name + description + Schema + extended aliases1apple/health.sleep.query → apple/weather.current → builtin/ScheduledTasks
Full rankings, within-method scores

Keyword matching

[('apple/health.sleep.query', 7), ('apple/weather.current', 1), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('apple/health.sleep.query', 20.20127), ('apple/weather.current', 2.6695)]

BM25 · name + description + Schema + native aliases

[('apple/health.sleep.query', 26.98398), ('apple/weather.current', 2.64004), ('builtin/ScheduledTasks', 0.76414)]

BM25 · name + description + Schema + extended aliases

[('apple/health.sleep.query', 26.98398), ('apple/weather.current', 2.64004), ('builtin/ScheduledTasks', 0.76414)]

N022 list photo albums metadata

dev · phrase · target ['apple/photos.albums.list']

Lists regular and smart albums metadata.

MethodFirst acceptable rankTop5
Keyword matching1apple/photos.albums.list → apple/photos.search → apple/photos.get → apple/alarms.list → apple/reminders.list
BM25 · name＋description1apple/photos.albums.list → apple/photos.search → apple/photos.get → apple/alarms.list → apple/contacts.get
BM25 · name + description + Schema + native aliases1apple/photos.albums.list → apple/photos.search → apple/photos.get → apple/alarms.list → apple/reminders.lists
BM25 · name + description + Schema + extended aliases1apple/photos.albums.list → apple/photos.search → apple/photos.get → apple/alarms.list → builtin/ListFiles
Full rankings, within-method scores

Keyword matching

[('apple/photos.albums.list', 13), ('apple/photos.search', 6.5), ('apple/photos.get', 5.5), ('apple/alarms.list', 5), ('apple/reminders.list', 4.5), ('apple/calendar.events.list', 4), ('apple/calendar.list', 4), ('apple/contacts.list', 4), ('apple/reminders.lists', 4), ('builtin/ListFiles', 4), ('apple/health.workouts.list', 3), ('apple/links.open', 2), ('apple/alarms.cancel', 1.5), ('apple/calendar.events.delete', 1.5), ('apple/calendar.events.update', 1.5), ('apple/contacts.get', 1.5), ('apple/reminders.create', 1.5), ('builtin/Memory', 1.5), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.create', 1), ('apple/calendar.events.get', 1), ('apple/shortcuts.run', 1), ('apple/calendar.events.create', 0.5), ('apple/contacts.delete', 0.5), ('apple/contacts.update', 0.5), ('apple/health.workouts.get', 0.5)]

BM25 · name＋description

[('apple/photos.albums.list', 10.92031), ('apple/photos.search', 5.96244), ('apple/photos.get', 5.45506), ('apple/alarms.list', 3.86649), ('apple/contacts.get', 3.13334), ('apple/reminders.lists', 2.1669), ('builtin/ListFiles', 1.99034), ('apple/calendar.list', 1.97043), ('apple/calendar.events.list', 1.6149), ('apple/contacts.list', 1.49249), ('apple/reminders.list', 1.49249), ('apple/calendar.events.get', 1.458), ('apple/alarms.create', 1.37763), ('apple/health.workouts.list', 1.33902), ('apple/reminders.create', 1.28652), ('builtin/Memory', 1.16476), ('apple/calendar.events.delete', 1.12484), ('apple/alarms.cancel', 1.04154), ('apple/shortcuts.run', 0.97937), ('apple/calendar.events.update', 0.92421), ('builtin/ScheduledTasks', 0.89939)]

BM25 · name + description + Schema + native aliases

[('apple/photos.albums.list', 25.49962), ('apple/photos.search', 14.35404), ('apple/photos.get', 13.0272), ('apple/alarms.list', 5.70979), ('apple/reminders.lists', 4.12289), ('builtin/ListFiles', 3.83921), ('apple/reminders.list', 3.82967), ('apple/calendar.list', 3.8154), ('apple/contacts.get', 3.52188), ('apple/contacts.list', 3.34463), ('apple/calendar.events.list', 3.16618), ('apple/health.workouts.list', 2.24993), ('apple/reminders.create', 1.46978), ('apple/calendar.events.delete', 1.41756), ('apple/alarms.cancel', 1.37363), ('builtin/ScheduledTasks', 1.26107), ('apple/calendar.events.get', 1.12534), ('apple/links.open', 1.12377), ('builtin/Memory', 1.10871), ('apple/alarms.create', 1.02045), ('apple/calendar.events.update', 0.97819), ('apple/shortcuts.run', 0.72406), ('apple/contacts.delete', 0.60095), ('apple/health.workouts.get', 0.58448), ('apple/contacts.update', 0.39141), ('apple/calendar.events.create', 0.30692)]

BM25 · name + description + Schema + extended aliases

[('apple/photos.albums.list', 25.49962), ('apple/photos.search', 14.35404), ('apple/photos.get', 13.0272), ('apple/alarms.list', 5.70979), ('builtin/ListFiles', 5.03933), ('apple/reminders.lists', 4.12289), ('apple/reminders.list', 3.82967), ('apple/calendar.list', 3.8154), ('apple/contacts.get', 3.52188), ('apple/contacts.list', 3.34463), ('apple/calendar.events.list', 3.16618), ('apple/health.workouts.list', 2.24993), ('apple/links.open', 1.78438), ('apple/reminders.create', 1.46978), ('apple/calendar.events.delete', 1.41756), ('apple/alarms.cancel', 1.37363), ('builtin/ScheduledTasks', 1.26107), ('apple/calendar.events.get', 1.12534), ('builtin/Memory', 1.10871), ('apple/alarms.create', 1.02045), ('apple/calendar.events.update', 0.97819), ('apple/shortcuts.run', 0.72406), ('apple/contacts.delete', 0.60095), ('apple/health.workouts.get', 0.58448), ('apple/contacts.update', 0.39141), ('apple/calendar.events.create', 0.30692)]

N023 看看这个目录里有哪些文件

dev · chinese · target ['builtin/ListFiles']

Directory enumeration is ListFiles.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliases1builtin/ListFiles → builtin/WriteFile → builtin/ReadFile
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[('builtin/ListFiles', 10.84583), ('builtin/WriteFile', 3.91339), ('builtin/ReadFile', 3.80773)]

N024 读取文件第十行开始的二十行

dev · chinese · target ['builtin/ReadFile']

Line offset/limit belong to ReadFile.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedapple/battery.get → apple/calendar.events.get → apple/contacts.get → apple/device.get → apple/health.workouts.get
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrievedapple/battery.get → apple/device.get → apple/contacts.get → apple/photos.get → apple/calendar.events.get
BM25 · name + description + Schema + extended aliases1builtin/ReadFile → builtin/ListFiles → builtin/WriteFile → apple/battery.get → apple/device.get
Full rankings, within-method scores

Keyword matching

[('apple/battery.get', 1.5), ('apple/calendar.events.get', 1.5), ('apple/contacts.get', 1.5), ('apple/device.get', 1.5), ('apple/health.workouts.get', 1.5), ('apple/location.get', 1.5), ('apple/photos.get', 1.5), ('apple/reminders.get', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/battery.get', 3.06136), ('apple/device.get', 3.06136), ('apple/contacts.get', 2.85558), ('apple/photos.get', 2.85558), ('apple/calendar.events.get', 2.67572), ('apple/location.get', 2.67572), ('apple/reminders.get', 2.37637), ('apple/health.workouts.get', 2.25048)]

BM25 · name + description + Schema + extended aliases

[('builtin/ReadFile', 11.21551), ('builtin/ListFiles', 5.5907), ('builtin/WriteFile', 3.91339), ('apple/battery.get', 3.57724), ('apple/device.get', 3.57724), ('apple/contacts.get', 3.50772), ('apple/photos.get', 3.50772), ('apple/calendar.events.get', 3.44085), ('apple/location.get', 3.44085), ('apple/reminders.get', 3.37647), ('apple/health.workouts.get', 3.25469)]

N025 帮我新建一个文本文件

dev · chinese · target ['builtin/WriteFile']

Creates UTF-8 file.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedapple/alarms.create → apple/calendar.events.create → apple/contacts.create → apple/reminders.create
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrievedapple/alarms.create → apple/contacts.create → apple/calendar.events.create → apple/reminders.create
BM25 · name + description + Schema + extended aliases5apple/alarms.create → apple/contacts.create → apple/calendar.events.create → apple/reminders.create → builtin/WriteFile
Full rankings, within-method scores

Keyword matching

[('apple/alarms.create', 1.5), ('apple/calendar.events.create', 1.5), ('apple/contacts.create', 1.5), ('apple/reminders.create', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/alarms.create', 4.41203), ('apple/contacts.create', 3.81889), ('apple/calendar.events.create', 3.57836), ('apple/reminders.create', 3.17802)]

BM25 · name + description + Schema + extended aliases

[('apple/alarms.create', 4.87245), ('apple/contacts.create', 4.6793), ('apple/calendar.events.create', 4.58836), ('apple/reminders.create', 4.50088), ('builtin/WriteFile', 3.27543)]

N026 把文件里唯一的一段文字替换掉

dev · chinese · target ['builtin/EditFile']

Exact one-segment edit is EditFile.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliases4builtin/ListFiles → builtin/WriteFile → builtin/ReadFile → builtin/EditFile → apple/contacts.update
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[('builtin/ListFiles', 5.5907), ('builtin/WriteFile', 3.91339), ('builtin/ReadFile', 3.80773), ('builtin/EditFile', 3.46259), ('apple/contacts.update', 3.20315), ('apple/calendar.events.update', 3.12509), ('apple/reminders.update', 3.05076)]

N027 记住我长期偏好使用中文交流

dev · chinese · target ['builtin/Memory']

Lasting user preference belongs in Memory.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliases1builtin/Memory
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[('builtin/Memory', 9.43336)]

N028 查一下以前记下来的我的偏好

dev · chinese · target ['builtin/Memory']

Views lasting user memories.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliasesNot retrieved无召回
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[]

N029 现在几点，今天星期几

dev · chinese · target ['builtin/GetCurrentTime']

Device current time/date/weekday.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliases1builtin/GetCurrentTime
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[('builtin/GetCurrentTime', 15.76542)]

N030 用一个问题卡片让我选择方案

dev · chinese · target ['builtin/AskUserQuestions']

Question card with choices.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliases1builtin/AskUserQuestions
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[('builtin/AskUserQuestions', 10.81905)]

N031 每天早上定时帮我做个只读检查

dev · chinese · target ['builtin/ScheduledTasks']

Daily best-effort read-only scheduled task.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliases1builtin/ScheduledTasks
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[('builtin/ScheduledTasks', 4.97137)]

N032 查一下下周日历有哪些安排

dev · chinese · target ['apple/calendar.events.list']

Reads events over a range.

MethodFirst acceptable rankTop5
Keyword matching4apple/calendar.events.create → apple/calendar.events.delete → apple/calendar.events.get → apple/calendar.events.list → apple/calendar.events.update
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliases5apple/calendar.events.delete → apple/calendar.events.update → apple/calendar.events.create → apple/calendar.events.get → apple/calendar.events.list
BM25 · name + description + Schema + extended aliases4apple/calendar.events.delete → apple/calendar.events.create → apple/calendar.events.get → apple/calendar.events.list → apple/calendar.list
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.create', 3), ('apple/calendar.events.delete', 3), ('apple/calendar.events.get', 3), ('apple/calendar.events.list', 3), ('apple/calendar.events.update', 3), ('apple/calendar.list', 3), ('apple/links.open', 2)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.delete', 6.30707), ('apple/calendar.events.update', 6.30707), ('apple/calendar.events.create', 5.90981), ('apple/calendar.events.get', 5.90981), ('apple/calendar.events.list', 5.55964), ('apple/calendar.list', 5.55964), ('apple/links.open', 0.79971)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.delete', 6.82409), ('apple/calendar.events.create', 5.87731), ('apple/calendar.events.get', 5.71863), ('apple/calendar.events.list', 5.29014), ('apple/calendar.list', 5.29014), ('apple/calendar.events.update', 5.16123), ('apple/links.open', 1.37203)]

N033 把这条待办标记为完成

dev · chinese · target ['apple/reminders.complete']

Updates completion, not delete.

MethodFirst acceptable rankTop5
Keyword matching1apple/reminders.complete → apple/reminders.create → apple/reminders.delete → apple/reminders.get → apple/reminders.list
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliases1apple/reminders.complete → apple/reminders.lists → apple/reminders.delete → apple/reminders.update → apple/reminders.create
BM25 · name + description + Schema + extended aliases1apple/reminders.complete → apple/reminders.lists → apple/reminders.delete → apple/reminders.create → apple/reminders.get
Full rankings, within-method scores

Keyword matching

[('apple/reminders.complete', 3), ('apple/reminders.create', 3), ('apple/reminders.delete', 3), ('apple/reminders.get', 3), ('apple/reminders.list', 3), ('apple/reminders.lists', 3), ('apple/reminders.update', 3), ('apple/links.open', 0.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/reminders.complete', 5.90074), ('apple/reminders.lists', 5.90074), ('apple/reminders.delete', 5.20146), ('apple/reminders.update', 5.20146), ('apple/reminders.create', 4.9105), ('apple/reminders.get', 4.9105), ('apple/reminders.list', 4.65037), ('apple/links.open', 0.61404)]

BM25 · name + description + Schema + extended aliases

[('apple/reminders.complete', 13.16515), ('apple/reminders.lists', 7.61042), ('apple/reminders.delete', 5.99796), ('apple/reminders.create', 5.35021), ('apple/reminders.get', 5.20955), ('apple/reminders.list', 4.82872), ('apple/reminders.update', 4.71386), ('apple/links.open', 0.61404)]

N034 查找名字里包含小王的联系人

dev · chinese · target ['apple/contacts.list']

Substring contact search.

MethodFirst acceptable rankTop5
Keyword matching1apple/contacts.list → apple/alarms.list → apple/calendar.events.list → apple/calendar.list → apple/contacts.create
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliases1apple/contacts.list → apple/contacts.delete → apple/contacts.update → apple/contacts.create → apple/contacts.get
BM25 · name + description + Schema + extended aliases1apple/contacts.list → apple/contacts.delete → apple/music.search → apple/photos.search → apple/maps.search
Full rankings, within-method scores

Keyword matching

[('apple/contacts.list', 3), ('apple/alarms.list', 1.5), ('apple/calendar.events.list', 1.5), ('apple/calendar.list', 1.5), ('apple/contacts.create', 1.5), ('apple/contacts.delete', 1.5), ('apple/contacts.get', 1.5), ('apple/contacts.update', 1.5), ('apple/health.workouts.list', 1.5), ('apple/maps.search', 1.5), ('apple/music.search', 1.5), ('apple/photos.albums.list', 1.5), ('apple/photos.search', 1.5), ('apple/reminders.list', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/contacts.list', 5.66937), ('apple/contacts.delete', 3.76824), ('apple/contacts.update', 3.76824), ('apple/contacts.create', 3.51494), ('apple/contacts.get', 3.51494), ('apple/alarms.list', 2.71823), ('apple/music.search', 2.53552), ('apple/photos.search', 2.53552), ('apple/photos.albums.list', 2.37582), ('apple/calendar.events.list', 2.23504), ('apple/calendar.list', 2.23504), ('apple/maps.search', 2.23504), ('apple/reminders.list', 1.99824), ('apple/health.workouts.list', 1.89771)]

BM25 · name + description + Schema + extended aliases

[('apple/contacts.list', 6.15121), ('apple/contacts.delete', 3.92968), ('apple/music.search', 3.51476), ('apple/photos.search', 3.51476), ('apple/maps.search', 3.37543), ('apple/contacts.create', 3.36892), ('apple/contacts.get', 3.27543), ('apple/alarms.list', 3.24673), ('apple/photos.albums.list', 3.12748), ('apple/calendar.events.list', 3.07108), ('apple/calendar.list', 3.07108), ('apple/reminders.list', 3.01668), ('apple/contacts.update', 2.94821), ('apple/health.workouts.list', 2.91346)]

N035 按给定起终点坐标计算步行路线

dev · chinese · target ['apple/maps.route']

Explicit-coordinate route calculation.

MethodFirst acceptable rankTop5
Keyword matching5apple/location.geocode → apple/location.get → apple/location.reverseGeocode → apple/maps.eta → apple/maps.route
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliases4apple/location.geocode → apple/location.reverseGeocode → apple/maps.eta → apple/maps.route → apple/maps.show
BM25 · name + description + Schema + extended aliases4apple/location.geocode → apple/location.reverseGeocode → apple/maps.eta → apple/maps.route → apple/maps.show
Full rankings, within-method scores

Keyword matching

[('apple/location.geocode', 1.5), ('apple/location.get', 1.5), ('apple/location.reverseGeocode', 1.5), ('apple/maps.eta', 1.5), ('apple/maps.route', 1.5), ('apple/maps.search', 1.5), ('apple/maps.show', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/location.geocode', 4.85181), ('apple/location.reverseGeocode', 4.85181), ('apple/maps.eta', 4.09409), ('apple/maps.route', 4.09409), ('apple/maps.show', 4.09409), ('apple/location.get', 3.93504), ('apple/maps.search', 3.36633)]

BM25 · name + description + Schema + extended aliases

[('apple/location.geocode', 5.86696), ('apple/location.reverseGeocode', 5.86696), ('apple/maps.eta', 5.12209), ('apple/maps.route', 5.12209), ('apple/maps.show', 4.13194), ('apple/location.get', 3.80773), ('apple/maps.search', 3.66024)]

N036 查未来七天每天的天气

dev · chinese · target ['apple/weather.daily']

Daily forecast, not hourly/current.

MethodFirst acceptable rankTop5
Keyword matching2apple/weather.current → apple/weather.daily → apple/weather.hourly
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliases2apple/weather.current → apple/weather.daily → apple/weather.hourly
BM25 · name + description + Schema + extended aliases2apple/weather.current → apple/weather.daily → apple/weather.hourly → builtin/ScheduledTasks
Full rankings, within-method scores

Keyword matching

[('apple/weather.current', 1.5), ('apple/weather.daily', 1.5), ('apple/weather.hourly', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/weather.current', 5.26033), ('apple/weather.daily', 5.26033), ('apple/weather.hourly', 5.26033)]

BM25 · name + description + Schema + extended aliases

[('apple/weather.current', 6.12162), ('apple/weather.daily', 6.12162), ('apple/weather.hourly', 6.12162), ('builtin/ScheduledTasks', 4.97137)]

N037 暂停正在播放的歌曲

dev · chinese · target ['apple/music.playback']

pause action controls Anan player.

MethodFirst acceptable rankTop5
Keyword matching1apple/music.playback → apple/music.search → apple/music.status
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliases1apple/music.playback → apple/music.status → apple/music.search
BM25 · name + description + Schema + extended aliases1apple/music.playback → apple/music.status → apple/music.search
Full rankings, within-method scores

Keyword matching

[('apple/music.playback', 3), ('apple/music.search', 3), ('apple/music.status', 3)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/music.playback', 10.52065), ('apple/music.status', 10.52065), ('apple/music.search', 8.39909)]

BM25 · name + description + Schema + extended aliases

[('apple/music.playback', 12.24325), ('apple/music.status', 12.24325), ('apple/music.search', 8.53748)]

N038 I forgot the filename; show names in the current folder

dev · intent · target ['builtin/ListFiles']

Only directory enumeration requested.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedapple/maps.show → apple/weather.current → builtin/GetCurrentTime → apple/clipboard.read → apple/contacts.create
BM25 · name＋descriptionNot retrievedapple/maps.show → apple/clipboard.read → apple/shortcuts.run → apple/device.get → apple/contacts.create
BM25 · name + description + Schema + native aliasesNot retrievedapple/maps.show → builtin/GetCurrentTime → apple/weather.current → apple/clipboard.read → apple/shortcuts.run
BM25 · name + description + Schema + extended aliases2apple/maps.show → builtin/ListFiles → builtin/GetCurrentTime → apple/weather.current → apple/shortcuts.run
Full rankings, within-method scores

Keyword matching

[('apple/maps.show', 6.5), ('apple/weather.current', 4), ('builtin/GetCurrentTime', 4), ('apple/clipboard.read', 2), ('apple/contacts.create', 1.5), ('apple/contacts.update', 1.5), ('apple/shortcuts.run', 1.5), ('builtin/EditFile', 1.5), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.cancel', 1), ('apple/alarms.create', 1), ('apple/alarms.list', 1), ('apple/calendar.events.list', 1), ('apple/contacts.list', 1), ('apple/device.get', 1), ('apple/health.statistics', 1), ('apple/links.open', 1), ('apple/location.geocode', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.eta', 1), ('apple/maps.route', 1), ('apple/maps.search', 1), ('apple/music.playback', 1), ('apple/music.search', 1), ('apple/music.status', 1), ('apple/photos.search', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1)]

BM25 · name＋description

[('apple/maps.show', 7.45798), ('apple/clipboard.read', 4.69121), ('apple/shortcuts.run', 3.16698), ('apple/device.get', 3.11442), ('apple/contacts.create', 2.46922), ('builtin/GetCurrentTime', 2.20523), ('apple/contacts.list', 2.09152), ('apple/contacts.update', 2.02514), ('apple/calendar.events.list', 1.9343), ('builtin/EditFile', 1.83299), ('apple/alarms.list', 1.68611), ('apple/weather.current', 1.66897), ('apple/alarms.cancel', 1.66828), ('apple/location.reverseGeocode', 1.63355), ('apple/music.search', 1.58446), ('apple/weather.daily', 1.56055), ('apple/alarms.create', 1.5381), ('apple/location.geocode', 1.5103), ('apple/maps.eta', 1.43797), ('apple/music.status', 1.38811), ('apple/photos.search', 1.29763), ('apple/maps.route', 1.28422), ('apple/maps.search', 1.2068), ('apple/weather.hourly', 1.13818), ('apple/links.open', 1.00107), ('apple/health.statistics', 0.96452), ('apple/music.playback', 0.94183), ('builtin/ScheduledTasks', 0.88332)]

BM25 · name + description + Schema + native aliases

[('apple/maps.show', 17.58396), ('builtin/GetCurrentTime', 5.21653), ('apple/weather.current', 5.04538), ('apple/clipboard.read', 4.66865), ('apple/shortcuts.run', 4.42835), ('apple/contacts.create', 4.00923), ('apple/contacts.update', 3.43853), ('apple/device.get', 3.11942), ('builtin/EditFile', 2.354), ('apple/contacts.list', 2.08281), ('apple/calendar.events.list', 1.96627), ('apple/alarms.list', 1.67694), ('apple/alarms.cancel', 1.65817), ('apple/location.reverseGeocode', 1.5928), ('apple/music.search', 1.5703), ('apple/alarms.create', 1.52191), ('apple/weather.daily', 1.46638), ('apple/location.geocode', 1.44283), ('apple/maps.eta', 1.36917), ('apple/music.status', 1.31868), ('apple/photos.search', 1.27337), ('apple/maps.route', 1.21419), ('builtin/ScheduledTasks', 1.17037), ('apple/maps.search', 1.13698), ('apple/weather.hourly', 1.069), ('apple/links.open', 0.97247), ('apple/health.statistics', 0.89881), ('apple/music.playback', 0.87677)]

BM25 · name + description + Schema + extended aliases

[('apple/maps.show', 12.21195), ('builtin/ListFiles', 10.51028), ('builtin/GetCurrentTime', 7.0813), ('apple/weather.current', 5.04538), ('apple/shortcuts.run', 4.42835), ('apple/contacts.list', 4.39439), ('apple/calendar.events.list', 4.22012), ('apple/alarms.list', 4.11335), ('apple/contacts.create', 4.00923), ('apple/contacts.update', 3.43853), ('apple/clipboard.read', 3.39228), ('apple/device.get', 3.11942), ('builtin/EditFile', 2.354), ('apple/photos.albums.list', 2.31158), ('apple/calendar.list', 2.25385), ('apple/reminders.list', 2.19893), ('apple/health.workouts.list', 2.09674), ('apple/alarms.cancel', 1.65817), ('apple/location.reverseGeocode', 1.5928), ('apple/music.search', 1.5703), ('apple/alarms.create', 1.52191), ('apple/weather.daily', 1.46638), ('apple/location.geocode', 1.44283), ('apple/maps.eta', 1.36917), ('apple/music.status', 1.31868), ('apple/photos.search', 1.27337), ('apple/maps.route', 1.21419), ('builtin/ScheduledTasks', 1.17037), ('apple/maps.search', 1.13698), ('apple/weather.hourly', 1.069), ('apple/links.open', 0.97247), ('apple/health.statistics', 0.89881), ('apple/music.playback', 0.87677)]

N039 I need the contents of notes.txt, not just its name

dev · intent · target ['builtin/ReadFile']

Read an existing text file.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedapple/links.open → builtin/ScheduledTasks → apple/clipboard.write → apple/contacts.list → apple/maps.search
BM25 · name＋descriptionNot retrievedbuiltin/ScheduledTasks → apple/clipboard.write → apple/clipboard.read → apple/calendar.events.get → apple/health.workouts.get
BM25 · name + description + Schema + native aliasesNot retrievedbuiltin/ScheduledTasks → apple/clipboard.write → apple/maps.show → apple/shortcuts.run → apple/clipboard.read
BM25 · name + description + Schema + extended aliases5builtin/ScheduledTasks → builtin/WriteFile → apple/clipboard.write → apple/maps.show → builtin/ReadFile
Full rankings, within-method scores

Keyword matching

[('apple/links.open', 4.5), ('builtin/ScheduledTasks', 3), ('apple/clipboard.write', 2.5), ('apple/contacts.list', 2.5), ('apple/maps.search', 2.5), ('apple/maps.show', 2.5), ('apple/alarms.create', 2), ('apple/alarms.list', 2), ('apple/clipboard.read', 2), ('apple/music.search', 2), ('apple/calendar.events.list', 1.5), ('apple/calendar.events.update', 1.5), ('apple/contacts.create', 1.5), ('apple/contacts.update', 1.5), ('apple/health.sleep.query', 1.5), ('apple/photos.albums.list', 1.5), ('apple/photos.search', 1.5), ('apple/reminders.create', 1.5), ('apple/reminders.list', 1.5), ('apple/reminders.update', 1.5), ('apple/shortcuts.run', 1.5), ('apple/alarms.cancel', 1), ('apple/battery.get', 1), ('apple/calendar.events.get', 1), ('apple/contacts.get', 1), ('apple/device.get', 1), ('apple/health.statistics', 1), ('apple/health.workouts.get', 1), ('apple/location.geocode', 1), ('apple/location.get', 1), ('apple/maps.eta', 1), ('apple/music.playback', 1), ('apple/music.status', 1), ('apple/photos.get', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/calendar.events.create', 0.5), ('apple/health.workouts.list', 0.5), ('builtin/Memory', 0.5), ('builtin/WriteFile', 0.5)]

BM25 · name＋description

[('builtin/ScheduledTasks', 6.08987), ('apple/clipboard.write', 4.3968), ('apple/clipboard.read', 4.12652), ('apple/calendar.events.get', 3.31895), ('apple/health.workouts.get', 3.17777), ('apple/shortcuts.run', 3.16698), ('apple/contacts.get', 3.13334), ('apple/device.get', 3.11442), ('apple/maps.search', 3.10681), ('apple/reminders.update', 3.04811), ('apple/contacts.list', 3.02124), ('apple/alarms.list', 2.95683), ('apple/maps.show', 2.95663), ('apple/music.search', 2.8007), ('apple/contacts.create', 2.46922), ('apple/reminders.list', 2.44865), ('apple/alarms.create', 2.36807), ('apple/music.playback', 2.16711), ('apple/calendar.events.update', 2.10385), ('apple/contacts.update', 2.02514), ('apple/calendar.events.list', 1.9343), ('apple/links.open', 1.86091), ('apple/alarms.cancel', 1.66828), ('apple/health.sleep.query', 1.56294), ('apple/weather.current', 1.38488), ('apple/photos.albums.list', 1.3627), ('apple/photos.search', 1.29763), ('apple/health.statistics', 1.24325), ('apple/photos.get', 1.21623), ('apple/reminders.create', 1.11196), ('apple/location.geocode', 1.07002), ('apple/location.get', 1.03112), ('apple/battery.get', 1.01877), ('apple/maps.eta', 1.01877), ('apple/music.status', 0.98345), ('apple/weather.daily', 0.74955)]

BM25 · name + description + Schema + native aliases

[('builtin/ScheduledTasks', 5.88177), ('apple/clipboard.write', 5.06624), ('apple/maps.show', 4.60259), ('apple/shortcuts.run', 4.42835), ('apple/clipboard.read', 4.10668), ('apple/contacts.create', 4.00923), ('apple/contacts.update', 3.43853), ('apple/calendar.events.get', 3.43216), ('apple/contacts.list', 3.33281), ('apple/maps.search', 3.32218), ('apple/reminders.update', 3.30002), ('apple/links.open', 3.27998), ('apple/health.workouts.get', 3.27215), ('apple/device.get', 3.11942), ('apple/calendar.events.update', 2.74321), ('apple/alarms.list', 2.7172), ('apple/music.search', 2.56338), ('apple/contacts.get', 2.53054), ('apple/calendar.events.list', 2.27618), ('apple/reminders.list', 2.25912), ('apple/alarms.create', 2.19677), ('apple/reminders.create', 1.76432), ('apple/music.playback', 1.72095), ('builtin/WriteFile', 1.66909), ('apple/alarms.cancel', 1.65817), ('apple/health.sleep.query', 1.62443), ('apple/photos.albums.list', 1.61938), ('apple/photos.search', 1.55727), ('builtin/Memory', 1.19415), ('apple/weather.current', 1.13096), ('apple/calendar.events.create', 1.11613), ('apple/health.statistics', 1.00933), ('apple/photos.get', 0.99308), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/music.status', 0.8076), ('apple/weather.daily', 0.60635), ('apple/health.workouts.list', 0.29633)]

BM25 · name + description + Schema + extended aliases

[('builtin/ScheduledTasks', 5.88177), ('builtin/WriteFile', 4.81168), ('apple/clipboard.write', 4.74852), ('apple/maps.show', 4.60259), ('builtin/ReadFile', 4.50088), ('apple/shortcuts.run', 4.42835), ('apple/links.open', 4.1746), ('apple/clipboard.read', 4.10668), ('apple/contacts.create', 4.00923), ('apple/contacts.update', 3.43853), ('apple/calendar.events.get', 3.43216), ('apple/contacts.list', 3.33281), ('apple/maps.search', 3.32218), ('apple/reminders.update', 3.30002), ('apple/health.workouts.get', 3.27215), ('apple/device.get', 3.11942), ('apple/calendar.events.update', 2.74321), ('apple/alarms.list', 2.7172), ('apple/music.search', 2.56338), ('apple/contacts.get', 2.53054), ('apple/calendar.events.list', 2.27618), ('apple/reminders.list', 2.25912), ('apple/alarms.create', 2.19677), ('apple/reminders.create', 1.76432), ('apple/alarms.cancel', 1.65817), ('apple/health.sleep.query', 1.62443), ('apple/photos.albums.list', 1.61938), ('apple/music.playback', 1.58398), ('apple/photos.search', 1.55727), ('apple/weather.current', 1.13096), ('apple/calendar.events.create', 1.11613), ('builtin/Memory', 1.09911), ('apple/health.statistics', 1.00933), ('apple/photos.get', 0.99308), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/music.status', 0.8076), ('apple/weather.daily', 0.60635), ('apple/health.workouts.list', 0.29633)]

N040 Save this entire draft as a new text document

dev · intent · target ['builtin/WriteFile']

Whole new file, not patch.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedapple/contacts.update → apple/links.open → builtin/EditFile → apple/clipboard.write → apple/shortcuts.run
BM25 · name＋descriptionNot retrievedapple/contacts.update → apple/clipboard.write → apple/clipboard.read → builtin/ScheduledTasks → builtin/ReadFile
BM25 · name + description + Schema + native aliasesNot retrievedbuiltin/EditFile → apple/contacts.update → apple/clipboard.write → apple/shortcuts.run → apple/clipboard.read
BM25 · name + description + Schema + extended aliases1builtin/WriteFile → builtin/EditFile → builtin/ReadFile → apple/contacts.update → apple/clipboard.write
Full rankings, within-method scores

Keyword matching

[('apple/contacts.update', 2), ('apple/links.open', 2), ('builtin/EditFile', 2), ('apple/clipboard.write', 1.5), ('apple/shortcuts.run', 1.5), ('builtin/ScheduledTasks', 1.5), ('apple/clipboard.read', 1), ('builtin/AskUserQuestions', 1), ('builtin/Memory', 1), ('builtin/ReadFile', 1), ('apple/calendar.events.update', 0.5)]

BM25 · name＋description

[('apple/contacts.update', 5.29461), ('apple/clipboard.write', 3.14747), ('apple/clipboard.read', 3.093), ('builtin/ScheduledTasks', 2.93947), ('builtin/ReadFile', 2.85815), ('builtin/EditFile', 2.72729), ('builtin/AskUserQuestions', 2.16548), ('apple/shortcuts.run', 1.77773)]

BM25 · name + description + Schema + native aliases

[('builtin/EditFile', 5.37492), ('apple/contacts.update', 5.14556), ('apple/clipboard.write', 4.5613), ('apple/shortcuts.run', 2.973), ('apple/clipboard.read', 2.89298), ('builtin/ScheduledTasks', 2.8876), ('builtin/ReadFile', 2.74541), ('builtin/Memory', 2.66951), ('builtin/AskUserQuestions', 2.06218), ('apple/calendar.events.update', 1.78238), ('apple/links.open', 1.50559)]

BM25 · name + description + Schema + extended aliases

[('builtin/WriteFile', 14.18772), ('builtin/EditFile', 10.97717), ('builtin/ReadFile', 8.82758), ('apple/contacts.update', 5.0416), ('apple/clipboard.write', 4.2922), ('apple/alarms.create', 2.90263), ('builtin/ScheduledTasks', 2.8876), ('apple/shortcuts.run', 2.7976), ('apple/contacts.create', 2.73695), ('apple/clipboard.read', 2.7223), ('apple/calendar.events.create', 2.661), ('apple/reminders.create', 2.58916), ('apple/links.open', 2.39064), ('builtin/Memory', 2.16924), ('builtin/AskUserQuestions', 1.94052), ('apple/calendar.events.update', 1.21197)]

N041 Fix a single typo in an existing document without replacing everything

dev · intent · target ['builtin/EditFile']

One exact substring replacement.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedbuiltin/WriteFile → apple/battery.get → apple/location.geocode → apple/location.reverseGeocode → apple/maps.search
BM25 · name＋descriptionNot retrievedbuiltin/WriteFile → apple/location.reverseGeocode → apple/location.geocode → apple/battery.get → apple/maps.search
BM25 · name + description + Schema + native aliasesNot retrievedbuiltin/WriteFile → apple/location.reverseGeocode → apple/location.geocode → apple/battery.get → apple/maps.search
BM25 · name + description + Schema + extended aliases2builtin/WriteFile → builtin/EditFile → apple/calendar.events.update → builtin/ReadFile → apple/contacts.update
Full rankings, within-method scores

Keyword matching

[('builtin/WriteFile', 3), ('apple/battery.get', 1), ('apple/location.geocode', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.search', 1), ('apple/photos.search', 1), ('apple/calendar.events.delete', 0.5), ('apple/calendar.events.update', 0.5), ('apple/clipboard.write', 0.5)]

BM25 · name＋description

[('builtin/WriteFile', 10.13039), ('apple/location.reverseGeocode', 2.80551), ('apple/location.geocode', 2.59384), ('apple/battery.get', 2.46962), ('apple/maps.search', 2.0726), ('apple/photos.search', 1.69739)]

BM25 · name + description + Schema + native aliases

[('builtin/WriteFile', 11.69735), ('apple/location.reverseGeocode', 2.87756), ('apple/location.geocode', 2.60663), ('apple/battery.get', 2.47355), ('apple/maps.search', 2.05407), ('apple/calendar.events.delete', 1.73506), ('apple/photos.search', 1.66567), ('apple/clipboard.write', 1.49569), ('apple/calendar.events.update', 0.88513)]

BM25 · name + description + Schema + extended aliases

[('builtin/WriteFile', 15.61074), ('builtin/EditFile', 12.2417), ('apple/calendar.events.update', 4.01022), ('builtin/ReadFile', 3.80773), ('apple/contacts.update', 3.20315), ('apple/reminders.update', 3.05076), ('apple/location.reverseGeocode', 2.87756), ('apple/location.geocode', 2.60663), ('apple/battery.get', 2.47355), ('apple/maps.search', 2.05407), ('apple/calendar.events.delete', 1.73506), ('apple/photos.search', 1.66567), ('apple/clipboard.write', 1.49569)]

N042 Remember that I prefer quiet restaurants for future conversations

dev · intent · target ['builtin/Memory']

Persist future-facing preference.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedapple/alarms.create → apple/links.open → apple/alarms.cancel → apple/alarms.list → apple/calendar.events.list
BM25 · name＋descriptionNot retrievedapple/alarms.create → apple/health.workouts.get → apple/health.workouts.list → apple/links.open → apple/health.sleep.query
BM25 · name + description + Schema + native aliasesNot retrievedapple/alarms.create → apple/health.workouts.get → apple/health.workouts.list → apple/links.open → apple/health.sleep.query
BM25 · name + description + Schema + extended aliases2apple/alarms.create → builtin/Memory → apple/health.workouts.get → apple/health.workouts.list → apple/links.open
Full rankings, within-method scores

Keyword matching

[('apple/alarms.create', 2.5), ('apple/links.open', 2), ('apple/alarms.cancel', 1), ('apple/alarms.list', 1), ('apple/calendar.events.list', 1), ('apple/clipboard.read', 1), ('apple/health.sleep.query', 1), ('apple/health.statistics', 1), ('apple/health.workouts.get', 1), ('apple/health.workouts.list', 1), ('apple/music.search', 1), ('apple/photos.search', 1), ('builtin/ScheduledTasks', 1)]

BM25 · name＋description

[('apple/alarms.create', 4.67653), ('apple/health.workouts.get', 3.17777), ('apple/health.workouts.list', 3.04811), ('apple/links.open', 3.04371), ('apple/health.sleep.query', 2.71569), ('apple/calendar.events.list', 1.9343), ('apple/health.statistics', 1.79974), ('apple/clipboard.read', 1.70434), ('apple/alarms.list', 1.68611), ('apple/alarms.cancel', 1.66828), ('apple/music.search', 1.58446), ('apple/photos.search', 1.29763), ('builtin/ScheduledTasks', 0.88332)]

BM25 · name + description + Schema + native aliases

[('apple/alarms.create', 6.68029), ('apple/health.workouts.get', 3.27215), ('apple/health.workouts.list', 3.12639), ('apple/links.open', 2.95676), ('apple/health.sleep.query', 2.75784), ('apple/calendar.events.list', 1.96627), ('apple/health.statistics', 1.76421), ('apple/clipboard.read', 1.69614), ('apple/alarms.list', 1.67694), ('apple/alarms.cancel', 1.65817), ('apple/music.search', 1.5703), ('apple/photos.search', 1.27337), ('builtin/ScheduledTasks', 0.85466)]

BM25 · name + description + Schema + extended aliases

[('apple/alarms.create', 6.68029), ('builtin/Memory', 4.71668), ('apple/health.workouts.get', 3.27215), ('apple/health.workouts.list', 3.12639), ('apple/links.open', 2.95676), ('apple/health.sleep.query', 2.75784), ('apple/calendar.events.list', 1.96627), ('apple/health.statistics', 1.76421), ('apple/clipboard.read', 1.69614), ('apple/alarms.list', 1.67694), ('apple/alarms.cancel', 1.65817), ('apple/music.search', 1.5703), ('apple/photos.search', 1.27337), ('builtin/ScheduledTasks', 0.85466)]

N043 Remove the outdated preference stored in my lasting memories

dev · intent · target ['builtin/Memory']

Memory delete is supported, ordinary file deletion is not.

MethodFirst acceptable rankTop5
Keyword matching1builtin/Memory → builtin/ScheduledTasks
BM25 · name＋description1builtin/Memory → builtin/ScheduledTasks
BM25 · name + description + Schema + native aliases1builtin/Memory → builtin/ScheduledTasks
BM25 · name + description + Schema + extended aliases1builtin/Memory → apple/contacts.delete → apple/calendar.events.delete → apple/reminders.delete → builtin/ScheduledTasks
Full rankings, within-method scores

Keyword matching

[('builtin/Memory', 6.5), ('builtin/ScheduledTasks', 1)]

BM25 · name＋description

[('builtin/Memory', 12.03781), ('builtin/ScheduledTasks', 1.54802)]

BM25 · name + description + Schema + native aliases

[('builtin/Memory', 22.71193), ('builtin/ScheduledTasks', 1.49781)]

BM25 · name + description + Schema + extended aliases

[('builtin/Memory', 36.81206), ('apple/contacts.delete', 4.2695), ('apple/calendar.events.delete', 4.13194), ('apple/reminders.delete', 3.88181), ('builtin/ScheduledTasks', 1.49781)]

N044 I have not told you which option I want; ask me to choose

dev · intent · target ['builtin/AskUserQuestions']

Collect missing choice through question card.

MethodFirst acceptable rankTop5
Keyword matching1builtin/AskUserQuestions → apple/links.open → apple/alarms.create → apple/alarms.list → apple/music.search
BM25 · name＋description1builtin/AskUserQuestions → apple/alarms.list → apple/music.search → apple/shortcuts.run → apple/alarms.create
BM25 · name + description + Schema + native aliases1builtin/AskUserQuestions → apple/shortcuts.run → apple/alarms.list → apple/music.search → apple/calendar.events.list
BM25 · name + description + Schema + extended aliases1builtin/AskUserQuestions → apple/shortcuts.run → apple/alarms.list → apple/music.search → apple/calendar.events.list
Full rankings, within-method scores

Keyword matching

[('builtin/AskUserQuestions', 5.5), ('apple/links.open', 3), ('apple/alarms.create', 2), ('apple/alarms.list', 2), ('apple/music.search', 2), ('builtin/ScheduledTasks', 2), ('apple/calendar.events.list', 1.5), ('apple/contacts.list', 1.5), ('apple/health.sleep.query', 1.5), ('apple/maps.search', 1.5), ('apple/photos.albums.list', 1.5), ('apple/photos.search', 1.5), ('apple/alarms.cancel', 1), ('apple/battery.get', 1), ('apple/clipboard.read', 1), ('apple/clipboard.write', 1), ('apple/health.statistics', 1), ('apple/location.geocode', 1), ('apple/location.get', 1), ('apple/maps.eta', 1), ('apple/maps.show', 1), ('apple/music.status', 1), ('apple/photos.get', 1), ('apple/reminders.create', 1), ('apple/shortcuts.run', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/health.workouts.list', 0.5), ('apple/reminders.list', 0.5)]

BM25 · name＋description

[('builtin/AskUserQuestions', 8.39528), ('apple/alarms.list', 2.95683), ('apple/music.search', 2.8007), ('apple/shortcuts.run', 2.74917), ('apple/alarms.create', 2.36807), ('apple/calendar.events.list', 1.9343), ('apple/links.open', 1.86091), ('apple/clipboard.read', 1.70434), ('apple/alarms.cancel', 1.66828), ('apple/health.sleep.query', 1.56294), ('apple/weather.current', 1.38488), ('apple/photos.albums.list', 1.3627), ('builtin/ScheduledTasks', 1.35996), ('apple/clipboard.write', 1.30984), ('apple/photos.search', 1.29763), ('apple/health.statistics', 1.24325), ('apple/photos.get', 1.21623), ('apple/reminders.create', 1.11196), ('apple/location.geocode', 1.07002), ('apple/location.get', 1.03112), ('apple/battery.get', 1.01877), ('apple/maps.eta', 1.01877), ('apple/music.status', 0.98345), ('apple/contacts.list', 0.92972), ('apple/maps.show', 0.90984), ('apple/maps.search', 0.85499), ('apple/weather.daily', 0.74955)]

BM25 · name + description + Schema + native aliases

[('builtin/AskUserQuestions', 18.31318), ('apple/shortcuts.run', 2.7231), ('apple/alarms.list', 2.7172), ('apple/music.search', 2.56338), ('apple/calendar.events.list', 2.27618), ('apple/alarms.create', 2.19677), ('apple/links.open', 2.19216), ('apple/clipboard.read', 1.69614), ('apple/alarms.cancel', 1.65817), ('apple/health.sleep.query', 1.62443), ('apple/photos.albums.list', 1.61938), ('apple/photos.search', 1.55727), ('apple/contacts.list', 1.25), ('builtin/ScheduledTasks', 1.23365), ('apple/weather.current', 1.13096), ('apple/maps.search', 1.09049), ('apple/clipboard.write', 1.07428), ('apple/health.statistics', 1.00933), ('apple/photos.get', 0.99308), ('apple/reminders.create', 0.92079), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/music.status', 0.8076), ('apple/maps.show', 0.74361), ('apple/weather.daily', 0.60635), ('apple/reminders.list', 0.31471), ('apple/health.workouts.list', 0.29633)]

BM25 · name + description + Schema + extended aliases

[('builtin/AskUserQuestions', 33.01502), ('apple/shortcuts.run', 2.7231), ('apple/alarms.list', 2.7172), ('apple/music.search', 2.56338), ('apple/calendar.events.list', 2.27618), ('apple/alarms.create', 2.19677), ('apple/links.open', 2.13869), ('apple/clipboard.read', 1.69614), ('apple/alarms.cancel', 1.65817), ('apple/health.sleep.query', 1.62443), ('apple/photos.albums.list', 1.61938), ('apple/photos.search', 1.55727), ('apple/contacts.list', 1.25), ('builtin/ScheduledTasks', 1.23365), ('apple/weather.current', 1.13096), ('apple/maps.search', 1.09049), ('apple/clipboard.write', 1.07428), ('apple/health.statistics', 1.00933), ('apple/photos.get', 0.99308), ('apple/reminders.create', 0.92079), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/music.status', 0.8076), ('apple/maps.show', 0.74361), ('apple/weather.daily', 0.60635), ('apple/reminders.list', 0.31471), ('apple/health.workouts.list', 0.29633)]

N045 Tomorrow run a read-only weather check while I am away

dev · intent · target ['builtin/ScheduledTasks']

Once scheduled read task; best effort, not alarm.

MethodFirst acceptable rankTop5
Keyword matching1builtin/ScheduledTasks → apple/clipboard.read → apple/weather.current → apple/weather.daily → apple/weather.hourly
BM25 · name＋description1builtin/ScheduledTasks → apple/weather.current → apple/shortcuts.run → apple/weather.hourly → apple/weather.daily
BM25 · name + description + Schema + native aliases3apple/weather.current → apple/shortcuts.run → builtin/ScheduledTasks → apple/weather.hourly → apple/weather.daily
BM25 · name + description + Schema + extended aliases3apple/shortcuts.run → apple/weather.current → builtin/ScheduledTasks → apple/weather.hourly → apple/weather.daily
Full rankings, within-method scores

Keyword matching

[('builtin/ScheduledTasks', 7), ('apple/clipboard.read', 5), ('apple/weather.current', 5), ('apple/weather.daily', 5), ('apple/weather.hourly', 5), ('apple/shortcuts.run', 4), ('builtin/ReadFile', 4), ('apple/alarms.list', 3), ('apple/photos.search', 3), ('apple/links.open', 2.5), ('apple/alarms.cancel', 2), ('apple/alarms.create', 2), ('apple/calendar.events.list', 2), ('apple/music.search', 2), ('apple/photos.get', 2), ('apple/music.playback', 1.5), ('builtin/Memory', 1.5), ('apple/battery.get', 1), ('apple/calendar.events.create', 1), ('apple/calendar.events.update', 1), ('apple/contacts.get', 1), ('apple/device.get', 1), ('apple/health.sleep.query', 1), ('apple/health.statistics', 1), ('apple/health.workouts.get', 1), ('apple/health.workouts.list', 1), ('apple/maps.eta', 1), ('apple/maps.show', 1), ('apple/music.status', 1), ('apple/photos.albums.list', 1), ('builtin/EditFile', 1)]

BM25 · name＋description

[('builtin/ScheduledTasks', 9.51877), ('apple/weather.current', 5.62775), ('apple/shortcuts.run', 4.45367), ('apple/weather.hourly', 4.22419), ('apple/weather.daily', 3.989), ('apple/alarms.list', 3.92544), ('apple/photos.search', 3.51612), ('apple/music.search', 3.30114), ('apple/clipboard.read', 3.04831), ('apple/calendar.events.list', 3.02999), ('apple/alarms.cancel', 2.93891), ('apple/calendar.events.create', 2.72209), ('apple/alarms.create', 2.70959), ('apple/links.open', 2.51291), ('apple/photos.get', 2.10433), ('apple/music.status', 1.8925), ('builtin/ReadFile', 1.73569), ('apple/photos.albums.list', 1.63489), ('builtin/Memory', 1.62085), ('apple/contacts.get', 1.59613), ('builtin/EditFile', 1.36324), ('apple/maps.show', 1.28422), ('apple/health.workouts.get', 1.26659), ('apple/health.workouts.list', 1.21491), ('apple/calendar.events.update', 1.1275), ('apple/device.get', 1.08242), ('apple/health.sleep.query', 1.08242), ('apple/battery.get', 1.06946), ('apple/maps.eta', 1.06946), ('apple/music.playback', 0.94183), ('apple/health.statistics', 0.71734)]

BM25 · name + description + Schema + native aliases

[('apple/weather.current', 12.61836), ('apple/shortcuts.run', 11.1813), ('builtin/ScheduledTasks', 10.9609), ('apple/weather.hourly', 10.69686), ('apple/weather.daily', 10.48018), ('apple/clipboard.read', 5.75082), ('builtin/ReadFile', 4.55729), ('apple/alarms.list', 3.84105), ('apple/photos.search', 3.38504), ('apple/links.open', 3.28532), ('apple/music.search', 3.19184), ('apple/calendar.events.list', 3.08008), ('apple/alarms.cancel', 2.85878), ('apple/alarms.create', 2.62385), ('apple/calendar.events.create', 2.4284), ('apple/photos.get', 2.02648), ('builtin/Memory', 1.99879), ('apple/music.status', 1.79823), ('apple/music.playback', 1.62165), ('apple/contacts.get', 1.60914), ('apple/photos.albums.list', 1.5512), ('builtin/EditFile', 1.39066), ('apple/health.workouts.get', 1.30421), ('apple/health.workouts.list', 1.24611), ('apple/maps.show', 1.21419), ('apple/health.sleep.query', 1.09922), ('apple/device.get', 1.08501), ('apple/battery.get', 1.07116), ('apple/maps.eta', 1.07116), ('apple/calendar.events.update', 1.069), ('apple/health.statistics', 0.70318)]

BM25 · name + description + Schema + extended aliases

[('apple/shortcuts.run', 17.25493), ('apple/weather.current', 12.2584), ('builtin/ScheduledTasks', 10.625), ('apple/weather.hourly', 10.45239), ('apple/weather.daily', 10.25376), ('apple/clipboard.read', 4.56555), ('apple/alarms.list', 4.54005), ('builtin/ReadFile', 4.20178), ('apple/calendar.events.list', 3.65799), ('apple/links.open', 3.28532), ('apple/music.search', 3.19184), ('apple/photos.search', 3.17419), ('apple/alarms.cancel', 2.85878), ('apple/photos.get', 2.77024), ('apple/alarms.create', 2.62385), ('apple/photos.albums.list', 2.47784), ('apple/calendar.events.create', 2.4284), ('apple/contacts.get', 2.14254), ('apple/health.workouts.get', 1.82646), ('apple/device.get', 1.80027), ('apple/music.status', 1.79823), ('apple/battery.get', 1.79046), ('apple/health.workouts.list', 1.72237), ('apple/music.playback', 1.62165), ('builtin/Memory', 1.52782), ('apple/maps.show', 1.21419), ('apple/calendar.events.update', 1.069), ('builtin/EditFile', 0.98414), ('apple/calendar.events.get', 0.97668), ('apple/location.get', 0.97668), ('apple/reminders.get', 0.951), ('apple/contacts.list', 0.92664), ('apple/calendar.list', 0.9035), ('apple/reminders.list', 0.88148), ('apple/health.sleep.query', 0.77789), ('apple/maps.eta', 0.75804), ('apple/health.statistics', 0.49762)]

N046 Put a thirty minute meeting on my calendar

dev · intent · target ['apple/calendar.events.create']

Create timed non-recurring event.

MethodFirst acceptable rankTop5
Keyword matching1apple/calendar.events.create → apple/calendar.events.list → apple/calendar.events.get → apple/calendar.events.update → apple/calendar.list
BM25 · name＋description4apple/calendar.list → apple/calendar.events.list → apple/calendar.events.get → apple/calendar.events.create → apple/calendar.events.update
BM25 · name + description + Schema + native aliases2apple/calendar.events.list → apple/calendar.events.create → apple/calendar.list → apple/calendar.events.get → apple/calendar.events.update
BM25 · name + description + Schema + extended aliases1apple/calendar.events.create → apple/calendar.events.list → apple/calendar.list → apple/calendar.events.get → apple/calendar.events.update
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.create', 4.5), ('apple/calendar.events.list', 4.5), ('apple/calendar.events.get', 4), ('apple/calendar.events.update', 4), ('apple/calendar.list', 4), ('apple/calendar.events.delete', 3), ('apple/links.open', 2), ('builtin/ScheduledTasks', 2), ('apple/health.statistics', 1), ('apple/weather.daily', 1)]

BM25 · name＋description

[('apple/calendar.list', 3.35399), ('apple/calendar.events.list', 3.30636), ('apple/calendar.events.get', 3.10538), ('apple/calendar.events.create', 3.00948), ('apple/calendar.events.update', 2.28111), ('apple/calendar.events.delete', 1.91466), ('builtin/ScheduledTasks', 1.80237), ('apple/weather.daily', 1.47616), ('apple/health.statistics', 1.34576)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.list', 8.2996), ('apple/calendar.events.create', 8.11782), ('apple/calendar.list', 7.72596), ('apple/calendar.events.get', 6.83473), ('apple/calendar.events.update', 5.94909), ('apple/calendar.events.delete', 4.55598), ('builtin/ScheduledTasks', 3.48666), ('apple/weather.daily', 1.29025), ('apple/health.statistics', 1.17133), ('apple/links.open', 0.66587)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.create', 11.67649), ('apple/calendar.events.list', 8.2996), ('apple/calendar.list', 7.72596), ('apple/calendar.events.get', 6.83473), ('apple/calendar.events.update', 5.94909), ('apple/calendar.events.delete', 4.55598), ('apple/alarms.create', 3.88181), ('apple/contacts.create', 3.66024), ('builtin/ScheduledTasks', 3.48666), ('apple/reminders.create', 3.46259), ('apple/weather.daily', 1.29025), ('apple/health.statistics', 1.17133), ('apple/links.open', 1.1424)]

N047 Add a new item to my shopping reminder list

dev · intent · target ['apple/reminders.create']

Creates reminder, not calendar event.

MethodFirst acceptable rankTop5
Keyword matching3apple/reminders.list → apple/reminders.lists → apple/reminders.create → apple/contacts.list → apple/reminders.complete
BM25 · name＋description4apple/reminders.list → apple/reminders.lists → builtin/Memory → apple/reminders.create → apple/clipboard.write
BM25 · name + description + Schema + native aliases6apple/reminders.list → apple/reminders.lists → apple/reminders.get → apple/reminders.complete → apple/reminders.delete
BM25 · name + description + Schema + extended aliases1apple/reminders.create → apple/reminders.list → apple/reminders.lists → apple/reminders.get → apple/reminders.complete
Full rankings, within-method scores

Keyword matching

[('apple/reminders.list', 9.5), ('apple/reminders.lists', 8), ('apple/reminders.create', 5.5), ('apple/contacts.list', 5), ('apple/reminders.complete', 4.5), ('apple/reminders.delete', 4.5), ('apple/reminders.get', 4.5), ('apple/reminders.update', 4.5), ('apple/alarms.list', 4), ('apple/calendar.events.list', 4), ('apple/calendar.list', 4), ('apple/photos.albums.list', 4), ('builtin/ListFiles', 4), ('apple/health.workouts.list', 3), ('builtin/Memory', 3), ('apple/links.open', 2.5), ('apple/calendar.events.update', 2), ('builtin/ScheduledTasks', 2), ('apple/alarms.cancel', 1.5), ('apple/calendar.events.delete', 1.5), ('apple/clipboard.write', 1.5), ('apple/alarms.create', 1), ('apple/calendar.events.get', 1), ('apple/health.sleep.query', 1), ('apple/location.geocode', 1), ('apple/shortcuts.run', 1), ('apple/calendar.events.create', 0.5), ('apple/contacts.delete', 0.5), ('apple/contacts.get', 0.5), ('apple/contacts.update', 0.5), ('apple/health.workouts.get', 0.5), ('apple/photos.search', 0.5), ('builtin/EditFile', 0.5)]

BM25 · name＋description

[('apple/reminders.list', 7.05209), ('apple/reminders.lists', 5.70812), ('builtin/Memory', 4.43433), ('apple/reminders.create', 4.34318), ('apple/clipboard.write', 3.9472), ('apple/contacts.list', 3.94114), ('apple/reminders.get', 3.61281), ('apple/reminders.complete', 3.40623), ('apple/health.sleep.query', 3.3488), ('apple/reminders.delete', 3.2513), ('apple/reminders.update', 3.13715), ('apple/location.geocode', 2.81814), ('builtin/ListFiles', 1.99034), ('apple/calendar.list', 1.97043), ('apple/calendar.events.list', 1.6149), ('apple/alarms.list', 1.4702), ('apple/calendar.events.get', 1.458), ('apple/alarms.create', 1.37763), ('apple/photos.albums.list', 1.34012), ('apple/health.workouts.list', 1.33902), ('apple/calendar.events.delete', 1.12484), ('apple/alarms.cancel', 1.04154), ('apple/shortcuts.run', 0.97937), ('apple/calendar.events.update', 0.92421), ('builtin/ScheduledTasks', 0.89939)]

BM25 · name + description + Schema + native aliases

[('apple/reminders.list', 14.11362), ('apple/reminders.lists', 12.26152), ('apple/reminders.get', 9.60564), ('apple/reminders.complete', 9.28524), ('apple/reminders.delete', 9.13089), ('apple/reminders.create', 9.00812), ('apple/reminders.update', 8.37383), ('apple/contacts.list', 5.78309), ('apple/clipboard.write', 5.43702), ('builtin/Memory', 5.30365), ('builtin/ListFiles', 3.83921), ('apple/calendar.list', 3.8154), ('apple/alarms.list', 3.32653), ('apple/calendar.events.list', 3.16618), ('apple/health.sleep.query', 3.03274), ('apple/photos.albums.list', 2.93791), ('apple/location.geocode', 2.83203), ('apple/calendar.events.update', 2.76057), ('apple/health.workouts.list', 2.24993), ('builtin/ScheduledTasks', 2.16514), ('apple/calendar.events.delete', 1.41756), ('apple/alarms.cancel', 1.37363), ('builtin/EditFile', 1.24425), ('apple/calendar.events.get', 1.12534), ('apple/alarms.create', 1.02045), ('apple/links.open', 0.99589), ('apple/shortcuts.run', 0.72406), ('apple/contacts.delete', 0.60095), ('apple/contacts.get', 0.60095), ('apple/health.workouts.get', 0.58448), ('apple/contacts.update', 0.39141), ('apple/calendar.events.create', 0.30692), ('apple/photos.search', 0.29834)]

BM25 · name + description + Schema + extended aliases

[('apple/reminders.create', 14.35832), ('apple/reminders.list', 14.11362), ('apple/reminders.lists', 12.26152), ('apple/reminders.get', 9.60564), ('apple/reminders.complete', 9.28524), ('apple/reminders.delete', 9.13089), ('apple/reminders.update', 8.37383), ('apple/alarms.create', 7.01841), ('apple/calendar.events.create', 5.80559), ('apple/contacts.list', 5.78309), ('apple/contacts.create', 5.6556), ('apple/clipboard.write', 5.43702), ('builtin/ListFiles', 5.03933), ('builtin/Memory', 4.09125), ('apple/calendar.list', 3.8154), ('apple/alarms.list', 3.32653), ('apple/calendar.events.list', 3.16618), ('apple/photos.albums.list', 2.93791), ('apple/location.geocode', 2.83203), ('builtin/WriteFile', 2.661), ('apple/health.workouts.list', 2.24993), ('apple/health.sleep.query', 2.19909), ('apple/calendar.events.update', 2.19016), ('builtin/ScheduledTasks', 2.16514), ('apple/links.open', 1.42113), ('apple/calendar.events.delete', 1.41756), ('apple/alarms.cancel', 1.37363), ('apple/calendar.events.get', 1.12534), ('builtin/EditFile', 0.84605), ('apple/shortcuts.run', 0.72406), ('apple/contacts.delete', 0.60095), ('apple/contacts.get', 0.60095), ('apple/health.workouts.get', 0.58448), ('apple/contacts.update', 0.39141), ('apple/photos.search', 0.29834)]

N048 Add a new person and their email to my contacts

dev · intent · target ['apple/contacts.create']

Creates contact with name/email.

MethodFirst acceptable rankTop5
Keyword matching3apple/contacts.update → apple/contacts.list → apple/contacts.create → apple/contacts.delete → apple/contacts.get
BM25 · name＋description5apple/contacts.list → apple/contacts.update → apple/calendar.list → apple/contacts.get → apple/contacts.create
BM25 · name + description + Schema + native aliases3apple/contacts.update → apple/contacts.list → apple/contacts.create → apple/contacts.get → apple/contacts.delete
BM25 · name + description + Schema + extended aliases1apple/contacts.create → apple/contacts.update → apple/contacts.list → apple/contacts.get → apple/contacts.delete
Full rankings, within-method scores

Keyword matching

[('apple/contacts.update', 6), ('apple/contacts.list', 5), ('apple/contacts.create', 4.5), ('apple/contacts.delete', 4.5), ('apple/contacts.get', 4.5), ('builtin/Memory', 1.5), ('apple/calendar.list', 1), ('apple/health.sleep.query', 1), ('builtin/AskUserQuestions', 1), ('apple/calendar.events.update', 0.5), ('apple/links.open', 0.5), ('builtin/EditFile', 0.5), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('apple/contacts.list', 6.61038), ('apple/contacts.update', 5.98121), ('apple/calendar.list', 4.64275), ('apple/contacts.get', 4.06233), ('apple/contacts.create', 3.90839), ('apple/contacts.delete', 3.62112), ('apple/health.sleep.query', 3.3488), ('builtin/AskUserQuestions', 3.3488), ('builtin/Memory', 3.26957)]

BM25 · name + description + Schema + native aliases

[('apple/contacts.update', 14.77653), ('apple/contacts.list', 12.20525), ('apple/contacts.create', 12.11711), ('apple/contacts.get', 11.96374), ('apple/contacts.delete', 11.14669), ('apple/calendar.list', 4.77743), ('builtin/Memory', 4.19494), ('builtin/AskUserQuestions', 3.40078), ('apple/health.sleep.query', 3.03274), ('apple/calendar.events.update', 1.78238), ('builtin/EditFile', 1.24425), ('builtin/ScheduledTasks', 0.86526), ('apple/links.open', 0.2902)]

BM25 · name + description + Schema + extended aliases

[('apple/contacts.create', 17.7727), ('apple/contacts.update', 14.77653), ('apple/contacts.list', 12.20525), ('apple/contacts.get', 11.96374), ('apple/contacts.delete', 11.14669), ('apple/alarms.create', 5.99796), ('apple/calendar.events.create', 5.49867), ('apple/reminders.create', 5.35021), ('apple/calendar.list', 4.77743), ('builtin/AskUserQuestions', 3.40078), ('builtin/Memory', 2.98254), ('builtin/WriteFile', 2.661), ('apple/health.sleep.query', 2.19909), ('apple/calendar.events.update', 1.21197), ('builtin/ScheduledTasks', 0.86526), ('builtin/EditFile', 0.84605), ('apple/links.open', 0.21043)]

N049 How long would driving between these two coordinates take?

dev · intent · target ['apple/maps.eta', 'apple/maps.route']

Both return travel-time estimate; detailed steps not required.

MethodFirst acceptable rankTop5
Keyword matching1apple/maps.eta → apple/maps.route → apple/health.statistics → apple/links.open → apple/location.reverseGeocode
BM25 · name＋description1apple/maps.eta → apple/maps.route → apple/health.statistics → apple/location.reverseGeocode → apple/maps.show
BM25 · name + description + Schema + native aliases1apple/maps.eta → apple/maps.route → apple/health.statistics → apple/location.reverseGeocode → apple/maps.show
BM25 · name + description + Schema + extended aliases1apple/maps.eta → apple/maps.route → builtin/Memory → apple/health.statistics → apple/location.reverseGeocode
Full rankings, within-method scores

Keyword matching

[('apple/maps.eta', 3.5), ('apple/maps.route', 2.5), ('apple/health.statistics', 1), ('apple/links.open', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.show', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1)]

BM25 · name＋description

[('apple/maps.eta', 8.75697), ('apple/maps.route', 5.90987), ('apple/health.statistics', 2.58396), ('apple/location.reverseGeocode', 2.43055), ('apple/maps.show', 1.91078), ('apple/weather.current', 1.72621), ('apple/weather.hourly', 1.6935), ('apple/weather.daily', 1.57416), ('apple/links.open', 1.13446)]

BM25 · name + description + Schema + native aliases

[('apple/maps.eta', 10.80209), ('apple/maps.route', 7.90892), ('apple/health.statistics', 2.53295), ('apple/location.reverseGeocode', 2.49296), ('apple/maps.show', 1.90039), ('apple/weather.current', 1.70716), ('apple/weather.hourly', 1.67313), ('apple/weather.daily', 1.5496), ('apple/links.open', 1.10205)]

BM25 · name + description + Schema + extended aliases

[('apple/maps.eta', 10.80209), ('apple/maps.route', 7.90892), ('builtin/Memory', 4.71668), ('apple/health.statistics', 2.53295), ('apple/location.reverseGeocode', 2.49296), ('apple/maps.show', 1.90039), ('apple/weather.current', 1.70716), ('apple/weather.hourly', 1.67313), ('apple/weather.daily', 1.5496), ('apple/links.open', 1.10205)]

N050 Open the supported app using its registered URL

dev · intent · target ['apple/links.open']

Hands off supported URL/app; not arbitrary UI operation.

MethodFirst acceptable rankTop5
Keyword matching1apple/links.open → apple/alarms.create → apple/calendar.events.get → apple/clipboard.read → apple/contacts.list
BM25 · name＋description1apple/links.open → apple/maps.show → apple/contacts.list → apple/calendar.events.get → apple/health.workouts.get
BM25 · name + description + Schema + native aliases1apple/links.open → apple/maps.show → apple/contacts.list → apple/calendar.events.get → apple/health.workouts.get
BM25 · name + description + Schema + extended aliases1apple/links.open → apple/maps.show → builtin/ReadFile → apple/contacts.list → apple/calendar.events.get
Full rankings, within-method scores

Keyword matching

[('apple/links.open', 10), ('apple/alarms.create', 1), ('apple/calendar.events.get', 1), ('apple/clipboard.read', 1), ('apple/contacts.list', 1), ('apple/health.workouts.get', 1), ('apple/maps.eta', 1), ('apple/maps.route', 1), ('apple/maps.search', 1), ('apple/maps.show', 1), ('apple/music.status', 1), ('apple/shortcuts.run', 1), ('builtin/ScheduledTasks', 1)]

BM25 · name＋description

[('apple/links.open', 13.46148), ('apple/maps.show', 4.12697), ('apple/contacts.list', 3.51562), ('apple/calendar.events.get', 3.31895), ('apple/health.workouts.get', 3.17777), ('apple/maps.eta', 2.95063), ('apple/maps.route', 2.63514), ('apple/clipboard.read', 2.42218), ('apple/music.status', 2.384), ('apple/maps.search', 2.25182), ('apple/shortcuts.run', 2.05198), ('apple/alarms.create', 2.01195), ('builtin/ScheduledTasks', 1.15544)]

BM25 · name + description + Schema + native aliases

[('apple/links.open', 25.20381), ('apple/maps.show', 4.11127), ('apple/contacts.list', 3.50098), ('apple/calendar.events.get', 3.43216), ('apple/health.workouts.get', 3.27215), ('apple/maps.eta', 2.95532), ('apple/maps.route', 2.62081), ('apple/clipboard.read', 2.41054), ('apple/music.status', 2.38233), ('apple/maps.search', 2.23169), ('apple/shortcuts.run', 2.03253), ('apple/alarms.create', 1.99077), ('builtin/ScheduledTasks', 1.11796)]

BM25 · name + description + Schema + extended aliases

[('apple/links.open', 24.5332), ('apple/maps.show', 8.21014), ('builtin/ReadFile', 3.80773), ('apple/contacts.list', 3.50098), ('apple/calendar.events.get', 3.43216), ('apple/health.workouts.get', 3.27215), ('apple/maps.eta', 2.95532), ('apple/maps.route', 2.62081), ('apple/clipboard.read', 2.41054), ('apple/music.status', 2.38233), ('apple/maps.search', 2.23169), ('apple/shortcuts.run', 2.03253), ('apple/alarms.create', 1.99077), ('builtin/ScheduledTasks', 1.11796)]

N051 Run my saved shortcut by its exact name

dev · intent · target ['apple/shortcuts.run']

Executes named shortcut handoff.

MethodFirst acceptable rankTop5
Keyword matching1apple/shortcuts.run → apple/health.workouts.get → apple/contacts.create → apple/contacts.update → apple/links.open
BM25 · name＋description1apple/shortcuts.run → apple/health.workouts.get → apple/calendar.events.get → apple/device.get → builtin/Memory
BM25 · name + description + Schema + native aliases1apple/shortcuts.run → apple/health.workouts.get → apple/contacts.create → apple/maps.show → apple/contacts.update
BM25 · name + description + Schema + extended aliases1apple/shortcuts.run → apple/health.workouts.get → apple/contacts.create → apple/maps.show → apple/contacts.update
Full rankings, within-method scores

Keyword matching

[('apple/shortcuts.run', 12), ('apple/health.workouts.get', 2), ('apple/contacts.create', 1.5), ('apple/contacts.update', 1.5), ('apple/links.open', 1.5), ('apple/maps.search', 1.5), ('apple/maps.show', 1.5), ('builtin/Memory', 1.5), ('apple/calendar.events.create', 1), ('apple/calendar.events.delete', 1), ('apple/calendar.events.get', 1), ('apple/calendar.events.update', 1), ('apple/clipboard.read', 1), ('apple/contacts.delete', 1), ('apple/contacts.list', 1), ('apple/device.get', 1), ('apple/health.sleep.query', 1), ('apple/health.statistics', 1), ('apple/health.workouts.list', 1), ('apple/music.playback', 1), ('apple/photos.albums.list', 1), ('builtin/ScheduledTasks', 1), ('apple/clipboard.write', 0.5), ('builtin/EditFile', 0.5)]

BM25 · name＋description

[('apple/shortcuts.run', 15.43026), ('apple/health.workouts.get', 5.28763), ('apple/calendar.events.get', 3.31895), ('apple/device.get', 3.11442), ('builtin/Memory', 2.65144), ('builtin/ScheduledTasks', 2.52466), ('apple/contacts.create', 2.46922), ('apple/clipboard.read', 2.42218), ('apple/maps.search', 2.25182), ('apple/photos.albums.list', 2.10385), ('apple/contacts.list', 2.09152), ('apple/maps.show', 2.04679), ('apple/contacts.update', 2.02514), ('apple/contacts.delete', 2.02377), ('apple/health.workouts.list', 2.02377), ('apple/health.sleep.query', 1.80306), ('apple/calendar.events.delete', 1.70006), ('apple/calendar.events.create', 1.46562), ('apple/links.open', 1.42271), ('apple/calendar.events.update', 1.39683), ('apple/health.statistics', 1.19492), ('apple/music.playback', 1.16681)]

BM25 · name + description + Schema + native aliases

[('apple/shortcuts.run', 37.31269), ('apple/health.workouts.get', 5.44466), ('apple/contacts.create', 4.00923), ('apple/maps.show', 3.85898), ('apple/contacts.update', 3.43853), ('apple/calendar.events.get', 3.43216), ('apple/maps.search', 3.23905), ('apple/device.get', 3.11942), ('builtin/Memory', 2.69634), ('builtin/ScheduledTasks', 2.45748), ('apple/clipboard.read', 2.41054), ('apple/contacts.list', 2.08281), ('apple/health.workouts.list', 2.07574), ('apple/contacts.delete', 2.04537), ('apple/health.sleep.query', 1.83104), ('apple/calendar.events.delete', 1.7185), ('apple/photos.albums.list', 1.67313), ('apple/calendar.events.create', 1.46617), ('apple/calendar.events.update', 1.39312), ('apple/links.open', 1.31248), ('apple/clipboard.write', 1.19265), ('apple/health.statistics', 1.17133), ('apple/music.playback', 1.14261), ('builtin/EditFile', 0.90222)]

BM25 · name + description + Schema + extended aliases

[('apple/shortcuts.run', 43.38632), ('apple/health.workouts.get', 5.44466), ('apple/contacts.create', 4.00923), ('apple/maps.show', 3.85898), ('apple/contacts.update', 3.43853), ('apple/calendar.events.get', 3.43216), ('apple/maps.search', 3.23905), ('apple/device.get', 3.11942), ('builtin/Memory', 2.69634), ('builtin/ScheduledTasks', 2.45748), ('apple/clipboard.read', 2.41054), ('apple/contacts.list', 2.08281), ('apple/health.workouts.list', 2.07574), ('apple/contacts.delete', 2.04537), ('apple/health.sleep.query', 1.83104), ('apple/calendar.events.delete', 1.7185), ('apple/photos.albums.list', 1.67313), ('apple/calendar.events.create', 1.46617), ('apple/calendar.events.update', 1.39312), ('apple/links.open', 1.31248), ('apple/clipboard.write', 1.19265), ('apple/health.statistics', 1.17133), ('apple/music.playback', 1.14261), ('builtin/EditFile', 0.90222)]

N052 Show recorded running workouts

dev · intent · target ['apple/health.workouts.list']

Lists workouts with running activityType.

MethodFirst acceptable rankTop5
Keyword matching1apple/health.workouts.list → apple/health.workouts.get → apple/maps.show → apple/clipboard.read → builtin/ScheduledTasks
BM25 · name＋description1apple/health.workouts.list → apple/health.workouts.get → apple/maps.show → apple/clipboard.read
BM25 · name + description + Schema + native aliases1apple/health.workouts.list → apple/health.workouts.get → apple/maps.show → apple/clipboard.read → builtin/ScheduledTasks
BM25 · name + description + Schema + extended aliases1apple/health.workouts.list → apple/health.workouts.get → apple/maps.show → apple/alarms.list → apple/contacts.list
Full rankings, within-method scores

Keyword matching

[('apple/health.workouts.list', 6.5), ('apple/health.workouts.get', 5.5), ('apple/maps.show', 4), ('apple/clipboard.read', 1), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('apple/health.workouts.list', 12.98646), ('apple/health.workouts.get', 8.90126), ('apple/maps.show', 4.12697), ('apple/clipboard.read', 2.98687)]

BM25 · name + description + Schema + native aliases

[('apple/health.workouts.list', 20.69154), ('apple/health.workouts.get', 17.5659), ('apple/maps.show', 12.51078), ('apple/clipboard.read', 2.97251), ('builtin/ScheduledTasks', 1.11398)]

BM25 · name + description + Schema + extended aliases

[('apple/health.workouts.list', 22.78828), ('apple/health.workouts.get', 17.5659), ('apple/maps.show', 7.13877), ('apple/alarms.list', 2.43641), ('apple/contacts.list', 2.31158), ('apple/photos.albums.list', 2.31158), ('apple/calendar.events.list', 2.25385), ('apple/calendar.list', 2.25385), ('apple/reminders.list', 2.19893), ('apple/clipboard.read', 1.69614), ('builtin/ScheduledTasks', 1.11398)]

N053 env path offset limit UTF-8

dev · schema · target ['builtin/ReadFile']

offset/limit plus text identify line reader.

MethodFirst acceptable rankTop5
Keyword matching1builtin/ReadFile → apple/photos.search → builtin/WriteFile → apple/maps.search → apple/music.search
BM25 · name＋description1builtin/ReadFile → builtin/WriteFile → apple/photos.search → builtin/GetCurrentTime → apple/maps.search
BM25 · name + description + Schema + native aliases1builtin/ReadFile → builtin/WriteFile → apple/photos.search → builtin/ListFiles → builtin/EditFile
BM25 · name + description + Schema + extended aliases1builtin/ReadFile → builtin/WriteFile → apple/photos.search → builtin/ListFiles → builtin/EditFile
Full rankings, within-method scores

Keyword matching

[('builtin/ReadFile', 4), ('apple/photos.search', 3), ('builtin/WriteFile', 3), ('apple/maps.search', 1.5), ('apple/music.search', 1.5), ('apple/weather.daily', 1.5), ('apple/weather.hourly', 1.5), ('builtin/ListFiles', 1.5), ('apple/calendar.events.list', 1), ('apple/contacts.list', 1), ('apple/health.sleep.query', 1), ('apple/health.workouts.list', 1), ('apple/photos.albums.list', 1), ('apple/reminders.list', 1), ('builtin/EditFile', 1), ('builtin/GetCurrentTime', 1), ('apple/alarms.create', 0.5), ('apple/calendar.events.create', 0.5), ('apple/calendar.events.update', 0.5), ('apple/health.statistics', 0.5), ('apple/links.open', 0.5), ('builtin/Memory', 0.5), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('builtin/ReadFile', 8.83998), ('builtin/WriteFile', 8.70082), ('apple/photos.search', 4.92185), ('builtin/GetCurrentTime', 4.03298), ('apple/maps.search', 2.25182), ('apple/music.search', 2.25182), ('apple/weather.hourly', 2.12379), ('apple/weather.daily', 1.97413)]

BM25 · name + description + Schema + native aliases

[('builtin/ReadFile', 13.02362), ('builtin/WriteFile', 11.4245), ('apple/photos.search', 4.06938), ('builtin/ListFiles', 3.60483), ('builtin/EditFile', 2.41306), ('apple/weather.hourly', 2.27993), ('apple/music.search', 2.26512), ('apple/weather.daily', 2.19251), ('apple/photos.albums.list', 2.00896), ('apple/maps.search', 1.9721), ('apple/contacts.list', 1.97053), ('apple/health.sleep.query', 1.82898), ('builtin/GetCurrentTime', 1.80986), ('apple/calendar.events.list', 1.75578), ('apple/health.workouts.list', 1.70467), ('apple/reminders.list', 1.37527), ('builtin/Memory', 1.17529), ('apple/health.statistics', 0.77516), ('apple/alarms.create', 0.76633), ('apple/calendar.events.create', 0.75014), ('apple/calendar.events.update', 0.71919), ('apple/links.open', 0.21043), ('builtin/ScheduledTasks', 0.18693)]

BM25 · name + description + Schema + extended aliases

[('builtin/ReadFile', 13.02362), ('builtin/WriteFile', 11.4245), ('apple/photos.search', 4.06938), ('builtin/ListFiles', 3.60483), ('builtin/EditFile', 2.41306), ('apple/weather.hourly', 2.27993), ('apple/music.search', 2.26512), ('apple/weather.daily', 2.19251), ('apple/photos.albums.list', 2.00896), ('apple/maps.search', 1.9721), ('apple/contacts.list', 1.97053), ('apple/health.sleep.query', 1.82898), ('builtin/GetCurrentTime', 1.80986), ('apple/calendar.events.list', 1.75578), ('apple/health.workouts.list', 1.70467), ('apple/reminders.list', 1.37527), ('builtin/Memory', 1.17529), ('apple/health.statistics', 0.77516), ('apple/alarms.create', 0.76633), ('apple/calendar.events.create', 0.75014), ('apple/calendar.events.update', 0.71919), ('apple/links.open', 0.21043), ('builtin/ScheduledTasks', 0.18693)]

N054 oldText newText unique nonempty text segment

dev · schema · target ['builtin/EditFile']

Exact edit schema fields.

MethodFirst acceptable rankTop5
Keyword matching1builtin/EditFile → apple/clipboard.write → apple/shortcuts.run → builtin/Memory → apple/clipboard.read
BM25 · name＋description1builtin/EditFile → apple/clipboard.write → apple/clipboard.read → builtin/ReadFile → builtin/AskUserQuestions
BM25 · name + description + Schema + native aliases1builtin/EditFile → apple/clipboard.write → builtin/Memory → apple/shortcuts.run → apple/clipboard.read
BM25 · name + description + Schema + extended aliases1builtin/EditFile → builtin/WriteFile → builtin/ReadFile → apple/clipboard.write → builtin/Memory
Full rankings, within-method scores

Keyword matching

[('builtin/EditFile', 5), ('apple/clipboard.write', 1.5), ('apple/shortcuts.run', 1.5), ('builtin/Memory', 1.5), ('apple/clipboard.read', 1), ('apple/contacts.update', 1), ('builtin/AskUserQuestions', 1), ('builtin/ReadFile', 1), ('apple/calendar.events.update', 0.5), ('apple/links.open', 0.5)]

BM25 · name＋description

[('builtin/EditFile', 12.54847), ('apple/clipboard.write', 3.14747), ('apple/clipboard.read', 3.093), ('builtin/ReadFile', 2.85815), ('builtin/AskUserQuestions', 2.16548), ('apple/contacts.update', 1.89057), ('apple/shortcuts.run', 1.77773)]

BM25 · name + description + Schema + native aliases

[('builtin/EditFile', 17.70641), ('apple/clipboard.write', 4.5613), ('builtin/Memory', 3.60779), ('apple/shortcuts.run', 2.973), ('apple/clipboard.read', 2.89298), ('builtin/ReadFile', 2.74541), ('builtin/AskUserQuestions', 2.06218), ('apple/calendar.events.update', 1.78238), ('apple/contacts.update', 1.76213), ('apple/links.open', 0.32542)]

BM25 · name + description + Schema + extended aliases

[('builtin/EditFile', 19.50093), ('builtin/WriteFile', 5.16501), ('builtin/ReadFile', 5.01984), ('apple/clipboard.write', 4.2922), ('builtin/Memory', 3.10751), ('apple/alarms.create', 2.90263), ('apple/shortcuts.run', 2.7976), ('apple/contacts.create', 2.73695), ('apple/clipboard.read', 2.7223), ('apple/calendar.events.create', 2.661), ('apple/reminders.create', 2.58916), ('builtin/AskUserQuestions', 1.94052), ('apple/contacts.update', 1.65817), ('apple/calendar.events.update', 1.21197), ('apple/links.open', 0.32542)]

N055 overwrite content path UTF-8

dev · schema · target ['builtin/WriteFile']

overwrite is explicit full-file write option.

MethodFirst acceptable rankTop5
Keyword matching1builtin/WriteFile → builtin/ReadFile → apple/clipboard.write → apple/music.playback → apple/photos.search
BM25 · name＋description2builtin/ReadFile → builtin/WriteFile → apple/clipboard.write → apple/photos.search → apple/music.playback
BM25 · name + description + Schema + native aliases1builtin/WriteFile → builtin/ReadFile → apple/clipboard.write → builtin/Memory → apple/music.playback
BM25 · name + description + Schema + extended aliases1builtin/WriteFile → builtin/ReadFile → apple/clipboard.write → builtin/Memory → apple/music.playback
Full rankings, within-method scores

Keyword matching

[('builtin/WriteFile', 3.5), ('builtin/ReadFile', 2.5), ('apple/clipboard.write', 2), ('apple/music.playback', 1), ('apple/photos.search', 1), ('builtin/Memory', 1), ('apple/links.open', 0.5), ('builtin/EditFile', 0.5), ('builtin/ListFiles', 0.5)]

BM25 · name＋description

[('builtin/ReadFile', 8.83998), ('builtin/WriteFile', 8.70082), ('apple/clipboard.write', 3.08696), ('apple/photos.search', 2.64774), ('apple/music.playback', 2.16711)]

BM25 · name + description + Schema + native aliases

[('builtin/WriteFile', 13.37189), ('builtin/ReadFile', 10.42098), ('apple/clipboard.write', 5.83634), ('builtin/Memory', 2.36945), ('apple/music.playback', 1.72095), ('builtin/ListFiles', 1.46224), ('apple/photos.search', 1.44305), ('builtin/EditFile', 1.28159), ('apple/links.open', 0.21043)]

BM25 · name + description + Schema + extended aliases

[('builtin/WriteFile', 16.51448), ('builtin/ReadFile', 14.92187), ('apple/clipboard.write', 5.51862), ('builtin/Memory', 2.2744), ('apple/music.playback', 1.58398), ('builtin/ListFiles', 1.46224), ('apple/photos.search', 1.44305), ('builtin/EditFile', 1.28159), ('apple/links.open', 0.21043)]

N056 directory level limit 200

dev · schema · target ['builtin/ListFiles']

One level directory listing with bounded item limit.

MethodFirst acceptable rankTop5
Keyword matching1builtin/ListFiles → apple/maps.search → apple/music.search → apple/weather.daily → apple/weather.hourly
BM25 · name＋description1builtin/ListFiles → apple/battery.get → builtin/Memory → apple/maps.search → apple/music.search
BM25 · name + description + Schema + native aliases1builtin/ListFiles → apple/battery.get → builtin/Memory → apple/weather.hourly → apple/music.search
BM25 · name + description + Schema + extended aliases1builtin/ListFiles → apple/battery.get → builtin/Memory → apple/weather.hourly → apple/music.search
Full rankings, within-method scores

Keyword matching

[('builtin/ListFiles', 2.5), ('apple/maps.search', 1.5), ('apple/music.search', 1.5), ('apple/weather.daily', 1.5), ('apple/weather.hourly', 1.5), ('apple/battery.get', 1), ('builtin/Memory', 1), ('apple/calendar.events.list', 0.5), ('apple/contacts.list', 0.5), ('apple/health.sleep.query', 0.5), ('apple/health.workouts.list', 0.5), ('apple/photos.albums.list', 0.5), ('apple/photos.search', 0.5), ('apple/reminders.list', 0.5), ('builtin/ReadFile', 0.5)]

BM25 · name＋description

[('builtin/ListFiles', 9.44416), ('apple/battery.get', 4.45998), ('builtin/Memory', 3.26957), ('apple/maps.search', 2.25182), ('apple/music.search', 2.25182), ('apple/weather.hourly', 2.12379), ('apple/weather.daily', 1.97413)]

BM25 · name + description + Schema + native aliases

[('builtin/ListFiles', 10.50684), ('apple/battery.get', 4.46475), ('builtin/Memory', 3.23148), ('apple/weather.hourly', 2.27993), ('apple/music.search', 2.26512), ('apple/weather.daily', 2.19251), ('apple/maps.search', 1.9721), ('apple/photos.albums.list', 1.148), ('apple/contacts.list', 1.13032), ('apple/health.sleep.query', 0.86419), ('apple/reminders.list', 0.83471), ('apple/calendar.events.list', 0.82533), ('apple/health.workouts.list', 0.7984), ('apple/photos.search', 0.77317), ('builtin/ReadFile', 0.69926)]

BM25 · name + description + Schema + extended aliases

[('builtin/ListFiles', 15.02039), ('apple/battery.get', 4.46475), ('builtin/Memory', 3.23148), ('apple/weather.hourly', 2.27993), ('apple/music.search', 2.26512), ('apple/weather.daily', 2.19251), ('apple/maps.search', 1.9721), ('apple/photos.albums.list', 1.148), ('apple/contacts.list', 1.13032), ('apple/health.sleep.query', 0.86419), ('apple/reminders.list', 0.83471), ('apple/calendar.events.list', 0.82533), ('apple/health.workouts.list', 0.7984), ('apple/photos.search', 0.77317), ('builtin/ReadFile', 0.69926)]

N057 command view create edit delete Markdown

dev · schema · target ['builtin/Memory']

Memory command enum and Markdown scope.

MethodFirst acceptable rankTop5
Keyword matching1builtin/Memory → apple/alarms.create → apple/calendar.events.create → apple/calendar.events.delete → apple/contacts.create
BM25 · name＋description1builtin/Memory → builtin/EditFile → builtin/ScheduledTasks → apple/reminders.delete → apple/contacts.delete
BM25 · name + description + Schema + native aliases1builtin/Memory → apple/reminders.delete → apple/contacts.delete → builtin/EditFile → apple/calendar.events.delete
BM25 · name + description + Schema + extended aliases1builtin/Memory → builtin/EditFile → apple/reminders.delete → apple/contacts.delete → apple/calendar.events.delete
Full rankings, within-method scores

Keyword matching

[('builtin/Memory', 8), ('apple/alarms.create', 4), ('apple/calendar.events.create', 4), ('apple/calendar.events.delete', 4), ('apple/contacts.create', 4), ('apple/contacts.delete', 4), ('apple/reminders.create', 4), ('apple/reminders.delete', 4), ('builtin/EditFile', 3), ('builtin/ScheduledTasks', 3), ('builtin/WriteFile', 1)]

BM25 · name＋description

[('builtin/Memory', 15.43774), ('builtin/EditFile', 4.21763), ('builtin/ScheduledTasks', 3.95171), ('apple/reminders.delete', 3.75288), ('apple/contacts.delete', 3.62112), ('apple/calendar.events.delete', 3.22484), ('apple/reminders.create', 3.05666), ('apple/contacts.create', 3.03074), ('apple/calendar.events.create', 2.94033), ('builtin/WriteFile', 2.81316), ('apple/alarms.create', 2.50063)]

BM25 · name + description + Schema + native aliases

[('builtin/Memory', 23.30976), ('apple/reminders.delete', 10.1562), ('apple/contacts.delete', 9.97994), ('builtin/EditFile', 9.57192), ('apple/calendar.events.delete', 8.6982), ('apple/reminders.create', 8.5428), ('apple/contacts.create', 8.51028), ('apple/calendar.events.create', 7.99065), ('apple/alarms.create', 7.9143), ('builtin/ScheduledTasks', 4.4541), ('builtin/WriteFile', 2.8775)]

BM25 · name + description + Schema + extended aliases

[('builtin/Memory', 21.16015), ('builtin/EditFile', 10.33148), ('apple/reminders.delete', 10.1562), ('apple/contacts.delete', 9.97994), ('apple/calendar.events.delete', 8.6982), ('apple/reminders.create', 8.5428), ('apple/contacts.create', 8.51028), ('apple/calendar.events.create', 7.99065), ('apple/alarms.create', 7.9143), ('builtin/WriteFile', 5.71516), ('builtin/ScheduledTasks', 4.4541), ('builtin/ReadFile', 4.26983), ('apple/contacts.update', 2.94821), ('apple/calendar.events.update', 2.87636), ('apple/reminders.update', 2.80794)]

N058 draft kind weekly weekdays timeZone requiredCapabilities

dev · schema · target ['builtin/ScheduledTasks']

Scheduled task draft weekly/requiredCapabilities schema.

MethodFirst acceptable rankTop5
Keyword matching1builtin/ScheduledTasks → builtin/GetCurrentTime → apple/links.open → apple/reminders.list → apple/alarms.cancel
BM25 · name＋description2builtin/GetCurrentTime → builtin/ScheduledTasks → apple/reminders.list → apple/maps.eta → apple/weather.current
BM25 · name + description + Schema + native aliases2builtin/GetCurrentTime → builtin/ScheduledTasks → apple/reminders.list → apple/links.open → apple/weather.current
BM25 · name + description + Schema + extended aliases2builtin/GetCurrentTime → builtin/ScheduledTasks → apple/links.open → apple/reminders.list → apple/weather.current
Full rankings, within-method scores

Keyword matching

[('builtin/ScheduledTasks', 7), ('builtin/GetCurrentTime', 6), ('apple/links.open', 4.5), ('apple/reminders.list', 3.5), ('apple/alarms.cancel', 2), ('apple/calendar.events.list', 2), ('apple/calendar.events.update', 2), ('apple/maps.eta', 2), ('apple/maps.route', 2), ('apple/weather.current', 2), ('apple/weather.hourly', 2), ('apple/alarms.create', 1.5), ('builtin/Memory', 1.5), ('apple/calendar.events.create', 1), ('apple/health.sleep.query', 1), ('apple/health.statistics', 1), ('apple/health.workouts.list', 1), ('apple/location.get', 1), ('apple/music.status', 1), ('apple/reminders.create', 1), ('apple/reminders.update', 1), ('apple/music.playback', 0.5)]

BM25 · name＋description

[('builtin/GetCurrentTime', 11.71783), ('builtin/ScheduledTasks', 7.11169), ('apple/reminders.list', 4.64526), ('apple/maps.eta', 4.28932), ('apple/weather.current', 3.90385), ('apple/maps.route', 3.27711), ('apple/alarms.cancel', 3.24244), ('apple/weather.hourly', 2.90444), ('apple/links.open', 2.5656), ('apple/location.get', 2.42087), ('builtin/Memory', 1.86565), ('apple/calendar.events.list', 1.82518), ('apple/music.status', 1.7197), ('apple/calendar.events.update', 1.48035), ('apple/alarms.create', 1.45133)]

BM25 · name + description + Schema + native aliases

[('builtin/GetCurrentTime', 11.14346), ('builtin/ScheduledTasks', 10.42714), ('apple/reminders.list', 4.60738), ('apple/links.open', 4.18337), ('apple/weather.current', 3.69852), ('apple/maps.eta', 3.27787), ('apple/calendar.events.list', 2.78848), ('apple/calendar.events.update', 2.74949), ('apple/maps.route', 2.5433), ('apple/alarms.cancel', 2.51483), ('apple/weather.hourly', 2.23916), ('builtin/Memory', 2.12306), ('apple/reminders.create', 1.82848), ('apple/reminders.update', 1.82848), ('apple/health.sleep.query', 1.63439), ('apple/location.get', 1.60433), ('apple/alarms.create', 1.57832), ('apple/health.statistics', 1.54354), ('apple/health.workouts.list', 1.50996), ('apple/calendar.events.create', 1.49372), ('apple/music.status', 1.13704), ('apple/music.playback', 0.91797)]

BM25 · name + description + Schema + extended aliases

[('builtin/GetCurrentTime', 17.26493), ('builtin/ScheduledTasks', 15.39851), ('apple/links.open', 5.06842), ('apple/reminders.list', 4.60738), ('apple/weather.current', 3.69852), ('apple/maps.eta', 3.27787), ('apple/calendar.events.list', 2.78848), ('apple/calendar.events.update', 2.74949), ('apple/maps.route', 2.5433), ('apple/alarms.cancel', 2.51483), ('apple/weather.hourly', 2.23916), ('builtin/Memory', 2.12306), ('apple/reminders.create', 1.82848), ('apple/reminders.update', 1.82848), ('apple/health.sleep.query', 1.63439), ('apple/location.get', 1.60433), ('apple/alarms.create', 1.57832), ('apple/health.statistics', 1.54354), ('apple/health.workouts.list', 1.50996), ('apple/calendar.events.create', 1.49372), ('apple/music.status', 1.13704), ('apple/music.playback', 0.91797)]

N059 questions options free text question card

dev · schema · target ['builtin/AskUserQuestions']

Question card schema allows free text.

MethodFirst acceptable rankTop5
Keyword matching1builtin/AskUserQuestions → apple/clipboard.write → apple/shortcuts.run → builtin/EditFile → apple/clipboard.read
BM25 · name＋description1builtin/AskUserQuestions → apple/clipboard.write → apple/clipboard.read → builtin/ReadFile → builtin/EditFile
BM25 · name + description + Schema + native aliases1builtin/AskUserQuestions → apple/clipboard.write → builtin/EditFile → apple/shortcuts.run → apple/clipboard.read
BM25 · name + description + Schema + extended aliases1builtin/AskUserQuestions → builtin/EditFile → builtin/ReadFile → apple/clipboard.write → apple/shortcuts.run
Full rankings, within-method scores

Keyword matching

[('builtin/AskUserQuestions', 9), ('apple/clipboard.write', 1.5), ('apple/shortcuts.run', 1.5), ('builtin/EditFile', 1.5), ('apple/clipboard.read', 1), ('apple/contacts.update', 1), ('builtin/ReadFile', 1), ('builtin/Memory', 0.5)]

BM25 · name＋description

[('builtin/AskUserQuestions', 19.09754), ('apple/clipboard.write', 3.14747), ('apple/clipboard.read', 3.093), ('builtin/ReadFile', 2.85815), ('builtin/EditFile', 2.72729), ('apple/contacts.update', 1.89057), ('apple/shortcuts.run', 1.77773)]

BM25 · name + description + Schema + native aliases

[('builtin/AskUserQuestions', 33.30829), ('apple/clipboard.write', 4.5613), ('builtin/EditFile', 4.13067), ('apple/shortcuts.run', 2.973), ('apple/clipboard.read', 2.89298), ('builtin/ReadFile', 2.74541), ('apple/contacts.update', 1.76213), ('builtin/Memory', 1.35633)]

BM25 · name + description + Schema + extended aliases

[('builtin/AskUserQuestions', 49.41521), ('builtin/EditFile', 6.32338), ('builtin/ReadFile', 5.01984), ('apple/clipboard.write', 4.2922), ('apple/shortcuts.run', 2.7976), ('apple/clipboard.read', 2.7223), ('builtin/WriteFile', 2.50401), ('apple/contacts.update', 1.65817), ('builtin/Memory', 1.27631)]

N060 calendarIds start end timeZone offset

dev · schema · target ['apple/calendar.events.list']

Calendar list range/filter/pagination fields.

MethodFirst acceptable rankTop5
Keyword matching1apple/calendar.events.list → apple/calendar.events.update → apple/calendar.events.create → builtin/GetCurrentTime → apple/calendar.events.get
BM25 · name＋description4builtin/GetCurrentTime → apple/photos.search → apple/calendar.events.get → apple/calendar.events.list → apple/reminders.list
BM25 · name + description + Schema + native aliases1apple/calendar.events.list → apple/calendar.events.create → apple/calendar.events.update → apple/calendar.events.get → builtin/GetCurrentTime
BM25 · name + description + Schema + extended aliases1apple/calendar.events.list → apple/calendar.events.create → apple/calendar.events.update → apple/calendar.events.get → builtin/GetCurrentTime
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.list', 8.5), ('apple/calendar.events.update', 7.5), ('apple/calendar.events.create', 7), ('builtin/GetCurrentTime', 6), ('apple/calendar.events.get', 5.5), ('apple/health.statistics', 4.5), ('apple/photos.search', 4.5), ('apple/calendar.list', 4), ('apple/reminders.list', 4), ('apple/calendar.events.delete', 3), ('apple/health.sleep.query', 2.5), ('apple/health.workouts.list', 2.5), ('apple/alarms.create', 2), ('apple/links.open', 2), ('builtin/ScheduledTasks', 2), ('apple/alarms.cancel', 1), ('apple/location.get', 1), ('apple/maps.eta', 1), ('apple/maps.route', 1), ('apple/maps.show', 1), ('apple/music.status', 1), ('apple/reminders.create', 1), ('apple/reminders.update', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1), ('apple/contacts.list', 0.5), ('apple/photos.albums.list', 0.5), ('builtin/Memory', 0.5), ('builtin/ReadFile', 0.5)]

BM25 · name＋description

[('builtin/GetCurrentTime', 11.0552), ('apple/photos.search', 8.362), ('apple/calendar.events.get', 6.75516), ('apple/calendar.events.list', 5.13154), ('apple/reminders.list', 4.64526), ('apple/calendar.events.update', 4.59467), ('apple/calendar.list', 3.35399), ('apple/health.statistics', 3.32489), ('apple/calendar.events.create', 3.00948), ('apple/maps.show', 2.63514), ('apple/location.get', 2.42087), ('apple/maps.eta', 2.40134), ('apple/calendar.events.delete', 1.91466), ('apple/music.status', 1.7197), ('apple/maps.route', 1.59099), ('apple/alarms.cancel', 1.57416), ('apple/weather.daily', 1.47616), ('apple/alarms.create', 1.45133), ('apple/weather.hourly', 1.41007)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.list', 14.68493), ('apple/calendar.events.create', 11.72029), ('apple/calendar.events.update', 10.954), ('apple/calendar.events.get', 10.70402), ('builtin/GetCurrentTime', 8.7889), ('apple/calendar.list', 7.72596), ('apple/photos.search', 7.13465), ('apple/health.statistics', 6.22913), ('apple/reminders.list', 5.75845), ('apple/calendar.events.delete', 4.55598), ('apple/health.sleep.query', 4.13455), ('apple/health.workouts.list', 3.79468), ('apple/alarms.create', 2.34465), ('apple/reminders.create', 1.82848), ('apple/reminders.update', 1.82848), ('builtin/ScheduledTasks', 1.70129), ('apple/maps.show', 1.67694), ('apple/location.get', 1.60433), ('apple/maps.eta', 1.59054), ('apple/weather.daily', 1.29025), ('apple/music.status', 1.13704), ('apple/maps.route', 1.04695), ('apple/alarms.cancel', 1.03523), ('apple/weather.hourly', 0.92175), ('apple/photos.albums.list', 0.86096), ('apple/contacts.list', 0.84021), ('apple/links.open', 0.66587), ('builtin/ReadFile', 0.66413), ('builtin/Memory', 0.56895)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.list', 14.68493), ('apple/calendar.events.create', 11.72029), ('apple/calendar.events.update', 10.954), ('apple/calendar.events.get', 10.70402), ('builtin/GetCurrentTime', 10.39683), ('apple/calendar.list', 7.72596), ('apple/photos.search', 7.13465), ('apple/health.statistics', 6.22913), ('apple/reminders.list', 5.75845), ('apple/calendar.events.delete', 4.55598), ('apple/health.sleep.query', 4.13455), ('apple/health.workouts.list', 3.79468), ('apple/alarms.create', 2.34465), ('apple/reminders.create', 1.82848), ('apple/reminders.update', 1.82848), ('builtin/ScheduledTasks', 1.70129), ('apple/maps.show', 1.67694), ('apple/location.get', 1.60433), ('apple/maps.eta', 1.59054), ('apple/weather.daily', 1.29025), ('apple/links.open', 1.1424), ('apple/music.status', 1.13704), ('apple/maps.route', 1.04695), ('apple/alarms.cancel', 1.03523), ('apple/weather.hourly', 0.92175), ('apple/photos.albums.list', 0.86096), ('apple/contacts.list', 0.84021), ('builtin/ReadFile', 0.66413), ('builtin/Memory', 0.56895)]

N061 completed reminderId

dev · schema · target ['apple/reminders.complete']

Completion setter fields.

MethodFirst acceptable rankTop5
Keyword matching3apple/reminders.get → apple/reminders.list → apple/reminders.complete → apple/reminders.create → apple/reminders.delete
BM25 · name＋description5apple/reminders.get → apple/reminders.create → apple/reminders.list → apple/reminders.lists → apple/reminders.complete
BM25 · name + description + Schema + native aliases1apple/reminders.complete → apple/reminders.get → apple/reminders.list → apple/reminders.delete → apple/reminders.create
BM25 · name + description + Schema + extended aliases1apple/reminders.complete → apple/reminders.get → apple/reminders.list → apple/reminders.delete → apple/reminders.create
Full rankings, within-method scores

Keyword matching

[('apple/reminders.get', 6), ('apple/reminders.list', 6), ('apple/reminders.complete', 5.5), ('apple/reminders.create', 5.5), ('apple/reminders.delete', 5), ('apple/reminders.update', 5), ('apple/reminders.lists', 4), ('apple/links.open', 2), ('builtin/ScheduledTasks', 2), ('apple/calendar.events.create', 1.5), ('apple/calendar.events.delete', 1.5), ('apple/calendar.events.update', 1.5), ('apple/music.playback', 1.5), ('apple/photos.get', 1.5), ('apple/alarms.cancel', 0.5), ('apple/calendar.events.get', 0.5), ('apple/calendar.events.list', 0.5), ('apple/contacts.delete', 0.5), ('apple/contacts.get', 0.5), ('apple/contacts.update', 0.5), ('apple/health.workouts.get', 0.5), ('apple/photos.search', 0.5), ('builtin/AskUserQuestions', 0.5)]

BM25 · name＋description

[('apple/reminders.get', 6.30728), ('apple/reminders.create', 5.11734), ('apple/reminders.list', 4.8339), ('apple/reminders.lists', 3.54122), ('apple/reminders.complete', 3.40623), ('apple/reminders.delete', 3.2513), ('apple/reminders.update', 3.13715), ('apple/photos.get', 2.25391), ('apple/music.playback', 1.8781), ('apple/calendar.events.delete', 1.8017), ('apple/calendar.events.create', 1.55325), ('apple/calendar.events.update', 1.48035), ('builtin/ScheduledTasks', 1.44059)]

BM25 · name + description + Schema + native aliases

[('apple/reminders.complete', 12.46336), ('apple/reminders.get', 11.97165), ('apple/reminders.list', 10.39813), ('apple/reminders.delete', 9.99833), ('apple/reminders.create', 9.24035), ('apple/reminders.update', 8.95534), ('apple/reminders.lists', 8.13862), ('apple/photos.get', 2.01898), ('builtin/ScheduledTasks', 1.89603), ('apple/calendar.events.delete', 1.79327), ('apple/music.playback', 1.7244), ('apple/calendar.events.create', 1.36992), ('apple/calendar.events.update', 1.30621), ('apple/calendar.events.get', 0.78475), ('apple/links.open', 0.70568), ('apple/alarms.cancel', 0.69591), ('apple/contacts.delete', 0.69591), ('apple/contacts.get', 0.69591), ('apple/health.workouts.get', 0.67683), ('builtin/AskUserQuestions', 0.65878), ('apple/photos.search', 0.52026), ('apple/contacts.update', 0.45326), ('apple/calendar.events.list', 0.37713)]

BM25 · name + description + Schema + extended aliases

[('apple/reminders.complete', 12.46336), ('apple/reminders.get', 11.97165), ('apple/reminders.list', 10.39813), ('apple/reminders.delete', 9.99833), ('apple/reminders.create', 9.24035), ('apple/reminders.update', 8.95534), ('apple/reminders.lists', 8.13862), ('apple/photos.get', 2.01898), ('builtin/ScheduledTasks', 1.89603), ('apple/calendar.events.delete', 1.79327), ('apple/music.playback', 1.7244), ('apple/calendar.events.create', 1.36992), ('apple/calendar.events.update', 1.30621), ('apple/links.open', 1.2107), ('apple/calendar.events.get', 0.78475), ('apple/alarms.cancel', 0.69591), ('apple/contacts.delete', 0.69591), ('apple/contacts.get', 0.69591), ('apple/health.workouts.get', 0.67683), ('builtin/AskUserQuestions', 0.65878), ('apple/photos.search', 0.52026), ('apple/contacts.update', 0.45326), ('apple/calendar.events.list', 0.37713)]

N062 assetId opaque image video metadata

dev · schema · target ['apple/photos.get']

Fetch one metadata item by opaque asset ID.

MethodFirst acceptable rankTop5
Keyword matching1apple/photos.get → apple/photos.search → apple/photos.albums.list → apple/calendar.events.create → apple/calendar.events.delete
BM25 · name＋description1apple/photos.get → apple/photos.albums.list → apple/photos.search → apple/reminders.get → apple/clipboard.read
BM25 · name + description + Schema + native aliases1apple/photos.get → apple/photos.search → apple/photos.albums.list → apple/clipboard.read → apple/alarms.list
BM25 · name + description + Schema + extended aliases1apple/photos.get → apple/photos.search → apple/photos.albums.list → apple/clipboard.read → apple/alarms.list
Full rankings, within-method scores

Keyword matching

[('apple/photos.get', 7.5), ('apple/photos.search', 6), ('apple/photos.albums.list', 4), ('apple/calendar.events.create', 1.5), ('apple/calendar.events.delete', 1.5), ('apple/calendar.events.update', 1.5), ('apple/music.playback', 1.5), ('apple/reminders.create', 1.5), ('apple/reminders.get', 1.5), ('apple/reminders.list', 1.5), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.list', 1), ('apple/clipboard.read', 1), ('apple/alarms.cancel', 0.5), ('apple/calendar.events.get', 0.5), ('apple/calendar.events.list', 0.5), ('apple/contacts.delete', 0.5), ('apple/contacts.get', 0.5), ('apple/contacts.update', 0.5), ('apple/health.workouts.get', 0.5), ('apple/reminders.complete', 0.5), ('apple/reminders.delete', 0.5), ('apple/reminders.update', 0.5), ('builtin/AskUserQuestions', 0.5)]

BM25 · name＋description

[('apple/photos.get', 18.0433), ('apple/photos.albums.list', 9.87597), ('apple/photos.search', 8.66819), ('apple/reminders.get', 2.69446), ('apple/clipboard.read', 2.42218), ('apple/alarms.list', 2.39629), ('apple/reminders.create', 2.06068), ('apple/music.playback', 1.8781), ('apple/calendar.events.delete', 1.8017), ('apple/reminders.list', 1.72296), ('apple/calendar.events.create', 1.55325), ('apple/calendar.events.update', 1.48035), ('builtin/ScheduledTasks', 1.44059)]

BM25 · name + description + Schema + native aliases

[('apple/photos.get', 21.73325), ('apple/photos.search', 12.07167), ('apple/photos.albums.list', 9.85214), ('apple/clipboard.read', 2.41054), ('apple/alarms.list', 2.38325), ('apple/reminders.get', 2.36601), ('apple/calendar.events.delete', 1.79327), ('apple/music.playback', 1.7244), ('apple/reminders.create', 1.70201), ('apple/calendar.events.create', 1.36992), ('apple/reminders.list', 1.30883), ('apple/calendar.events.update', 1.30621), ('builtin/ScheduledTasks', 0.99196), ('apple/reminders.delete', 0.86744), ('apple/reminders.complete', 0.85247), ('apple/calendar.events.get', 0.78475), ('apple/alarms.cancel', 0.69591), ('apple/contacts.delete', 0.69591), ('apple/contacts.get', 0.69591), ('apple/health.workouts.get', 0.67683), ('builtin/AskUserQuestions', 0.65878), ('apple/reminders.update', 0.58151), ('apple/contacts.update', 0.45326), ('apple/calendar.events.list', 0.37713)]

BM25 · name + description + Schema + extended aliases

[('apple/photos.get', 21.73325), ('apple/photos.search', 12.07167), ('apple/photos.albums.list', 9.85214), ('apple/clipboard.read', 2.41054), ('apple/alarms.list', 2.38325), ('apple/reminders.get', 2.36601), ('apple/calendar.events.delete', 1.79327), ('apple/music.playback', 1.7244), ('apple/reminders.create', 1.70201), ('apple/calendar.events.create', 1.36992), ('apple/reminders.list', 1.30883), ('apple/calendar.events.update', 1.30621), ('builtin/ScheduledTasks', 0.99196), ('apple/reminders.delete', 0.86744), ('apple/reminders.complete', 0.85247), ('apple/calendar.events.get', 0.78475), ('apple/alarms.cancel', 0.69591), ('apple/contacts.delete', 0.69591), ('apple/contacts.get', 0.69591), ('apple/health.workouts.get', 0.67683), ('builtin/AskUserQuestions', 0.65878), ('apple/reminders.update', 0.58151), ('apple/contacts.update', 0.45326), ('apple/calendar.events.list', 0.37713)]

N063 metric heartRateVariabilitySDNN start end

dev · schema · target ['apple/health.statistics']

Statistics metric enum includes HRV.

MethodFirst acceptable rankTop5
Keyword matching1apple/health.statistics → apple/photos.search → apple/calendar.events.update → apple/calendar.events.get → apple/calendar.events.create
BM25 · name＋description1apple/health.statistics → apple/photos.search → apple/calendar.events.get → apple/maps.show → apple/calendar.events.update
BM25 · name + description + Schema + native aliases1apple/health.statistics → apple/photos.search → apple/calendar.events.get → apple/calendar.events.update → apple/maps.show
BM25 · name + description + Schema + extended aliases1apple/health.statistics → apple/photos.search → apple/calendar.events.get → apple/calendar.events.update → apple/maps.show
Full rankings, within-method scores

Keyword matching

[('apple/health.statistics', 8.5), ('apple/photos.search', 3), ('apple/calendar.events.update', 2), ('apple/calendar.events.get', 1.5), ('apple/calendar.events.create', 1), ('apple/calendar.events.list', 1), ('apple/health.sleep.query', 1), ('apple/health.workouts.list', 1), ('apple/maps.show', 1), ('builtin/Memory', 0.5)]

BM25 · name＋description

[('apple/health.statistics', 18.94015), ('apple/photos.search', 6.0879), ('apple/calendar.events.get', 3.64978), ('apple/maps.show', 2.63514), ('apple/calendar.events.update', 2.31356)]

BM25 · name + description + Schema + native aliases

[('apple/health.statistics', 28.64008), ('apple/photos.search', 5.28148), ('apple/calendar.events.get', 3.86929), ('apple/calendar.events.update', 2.85363), ('apple/maps.show', 1.67694), ('apple/health.sleep.query', 1.53537), ('apple/calendar.events.list', 1.44159), ('apple/health.workouts.list', 1.37845), ('apple/calendar.events.create', 1.35861), ('builtin/Memory', 0.56895)]

BM25 · name + description + Schema + extended aliases

[('apple/health.statistics', 28.64008), ('apple/photos.search', 5.28148), ('apple/calendar.events.get', 3.86929), ('apple/calendar.events.update', 2.85363), ('apple/maps.show', 1.67694), ('apple/health.sleep.query', 1.53537), ('apple/calendar.events.list', 1.44159), ('apple/health.workouts.list', 1.37845), ('apple/calendar.events.create', 1.35861), ('builtin/Memory', 0.56895)]

N064 action pause resume previous next songId

dev · schema · target ['apple/music.playback']

Player action enum; songId for play.

MethodFirst acceptable rankTop5
Keyword matching1apple/music.playback → builtin/ScheduledTasks → apple/calendar.events.create → apple/calendar.events.delete → apple/calendar.events.update
BM25 · name＋description1apple/music.playback → builtin/ScheduledTasks → apple/music.search → apple/reminders.get → apple/shortcuts.run
BM25 · name + description + Schema + native aliases1apple/music.playback → apple/music.search → builtin/ScheduledTasks → apple/shortcuts.run → apple/reminders.get
BM25 · name + description + Schema + extended aliases1apple/music.playback → apple/music.search → builtin/ScheduledTasks → apple/shortcuts.run → apple/reminders.get
Full rankings, within-method scores

Keyword matching

[('apple/music.playback', 10.5), ('builtin/ScheduledTasks', 3), ('apple/calendar.events.create', 1.5), ('apple/calendar.events.delete', 1.5), ('apple/calendar.events.update', 1.5), ('apple/music.search', 1.5), ('apple/photos.get', 1.5), ('apple/reminders.create', 1.5), ('apple/reminders.get', 1.5), ('apple/reminders.list', 1.5), ('apple/shortcuts.run', 1), ('apple/alarms.cancel', 0.5), ('apple/calendar.events.get', 0.5), ('apple/calendar.events.list', 0.5), ('apple/contacts.delete', 0.5), ('apple/contacts.get', 0.5), ('apple/contacts.update', 0.5), ('apple/health.workouts.get', 0.5), ('apple/photos.search', 0.5), ('apple/reminders.complete', 0.5), ('apple/reminders.delete', 0.5), ('apple/reminders.update', 0.5), ('builtin/AskUserQuestions', 0.5)]

BM25 · name＋description

[('apple/music.playback', 20.19458), ('builtin/ScheduledTasks', 2.82108), ('apple/music.search', 2.77679), ('apple/reminders.get', 2.69446), ('apple/shortcuts.run', 2.45165), ('apple/photos.get', 2.25391), ('apple/reminders.create', 2.06068), ('apple/calendar.events.delete', 1.8017), ('apple/reminders.list', 1.72296), ('apple/calendar.events.create', 1.55325), ('apple/calendar.events.update', 1.48035)]

BM25 · name + description + Schema + native aliases

[('apple/music.playback', 32.09136), ('apple/music.search', 4.95015), ('builtin/ScheduledTasks', 2.71124), ('apple/shortcuts.run', 2.4284), ('apple/reminders.get', 2.36601), ('apple/photos.get', 2.01898), ('apple/calendar.events.delete', 1.79327), ('apple/reminders.create', 1.70201), ('apple/calendar.events.create', 1.36992), ('apple/reminders.list', 1.30883), ('apple/calendar.events.update', 1.30621), ('apple/reminders.delete', 0.86744), ('apple/reminders.complete', 0.85247), ('apple/calendar.events.get', 0.78475), ('apple/alarms.cancel', 0.69591), ('apple/contacts.delete', 0.69591), ('apple/contacts.get', 0.69591), ('apple/health.workouts.get', 0.67683), ('builtin/AskUserQuestions', 0.65878), ('apple/reminders.update', 0.58151), ('apple/photos.search', 0.52026), ('apple/contacts.update', 0.45326), ('apple/calendar.events.list', 0.37713)]

BM25 · name + description + Schema + extended aliases

[('apple/music.playback', 32.09136), ('apple/music.search', 4.95015), ('builtin/ScheduledTasks', 2.71124), ('apple/shortcuts.run', 2.4284), ('apple/reminders.get', 2.36601), ('apple/photos.get', 2.01898), ('apple/calendar.events.delete', 1.79327), ('apple/reminders.create', 1.70201), ('apple/calendar.events.create', 1.36992), ('apple/reminders.list', 1.30883), ('apple/calendar.events.update', 1.30621), ('apple/reminders.delete', 0.86744), ('apple/reminders.complete', 0.85247), ('apple/calendar.events.get', 0.78475), ('apple/alarms.cancel', 0.69591), ('apple/contacts.delete', 0.69591), ('apple/contacts.get', 0.69591), ('apple/health.workouts.get', 0.67683), ('builtin/AskUserQuestions', 0.65878), ('apple/reminders.update', 0.58151), ('apple/photos.search', 0.52026), ('apple/contacts.update', 0.45326), ('apple/calendar.events.list', 0.37713)]

N065 List file names only; do not read their contents

dev · interference · target ['builtin/ListFiles']

Listing names differs from reading text.

MethodFirst acceptable rankTop5
Keyword matching4builtin/ReadFile → apple/photos.albums.list → apple/alarms.list → builtin/ListFiles → apple/contacts.list
BM25 · name＋description7apple/photos.albums.list → apple/calendar.list → builtin/Memory → apple/shortcuts.run → apple/health.sleep.query
BM25 · name + description + Schema + native aliases4builtin/ReadFile → builtin/WriteFile → builtin/EditFile → builtin/ListFiles → builtin/Memory
BM25 · name + description + Schema + extended aliases3builtin/ReadFile → builtin/WriteFile → builtin/ListFiles → builtin/EditFile → apple/calendar.list
Full rankings, within-method scores

Keyword matching

[('builtin/ReadFile', 8), ('apple/photos.albums.list', 7.5), ('apple/alarms.list', 7), ('builtin/ListFiles', 7), ('apple/contacts.list', 6.5), ('apple/calendar.events.list', 5.5), ('builtin/EditFile', 5.5), ('builtin/ScheduledTasks', 5.5), ('apple/calendar.list', 5), ('apple/clipboard.read', 5), ('apple/reminders.list', 5), ('builtin/Memory', 5), ('builtin/WriteFile', 5), ('apple/health.workouts.list', 4.5), ('apple/links.open', 4), ('apple/photos.search', 4), ('apple/reminders.lists', 4), ('apple/shortcuts.run', 4), ('apple/health.sleep.query', 3.5), ('apple/maps.show', 3.5), ('apple/alarms.create', 3), ('apple/photos.get', 3), ('apple/alarms.cancel', 2.5), ('apple/calendar.events.update', 2.5), ('apple/clipboard.write', 2.5), ('apple/music.playback', 2.5), ('apple/reminders.create', 2.5), ('apple/battery.get', 2), ('apple/contacts.update', 2), ('apple/device.get', 2), ('apple/health.statistics', 2), ('apple/maps.eta', 2), ('apple/music.search', 2), ('apple/music.status', 2), ('apple/weather.current', 2), ('apple/weather.daily', 2), ('apple/calendar.events.delete', 1.5), ('apple/contacts.create', 1.5), ('apple/contacts.get', 1.5), ('apple/health.workouts.get', 1.5), ('apple/maps.search', 1.5), ('apple/calendar.events.get', 1), ('apple/location.geocode', 1), ('apple/location.get', 1), ('apple/weather.hourly', 1), ('builtin/AskUserQuestions', 1), ('apple/calendar.events.create', 0.5), ('apple/contacts.delete', 0.5)]

BM25 · name＋description

[('apple/photos.albums.list', 6.65128), ('apple/calendar.list', 6.61318), ('builtin/Memory', 6.26007), ('apple/shortcuts.run', 5.85086), ('apple/health.sleep.query', 5.63174), ('builtin/ReadFile', 5.20811), ('builtin/ListFiles', 5.04384), ('apple/alarms.list', 4.98025), ('builtin/EditFile', 4.73732), ('apple/contacts.list', 4.51373), ('apple/clipboard.write', 4.3968), ('apple/maps.show', 4.24085), ('apple/device.get', 4.19684), ('apple/links.open', 3.9362), ('builtin/ScheduledTasks', 3.79879), ('builtin/WriteFile', 3.71445), ('apple/photos.search', 3.68903), ('apple/alarms.create', 3.37909), ('builtin/AskUserQuestions', 3.3488), ('apple/photos.get', 3.32056), ('apple/clipboard.read', 3.27541), ('apple/music.playback', 3.10894), ('apple/music.search', 2.93291), ('apple/music.status', 2.87595), ('apple/calendar.events.list', 2.71059), ('apple/weather.current', 2.62614), ('apple/health.workouts.list', 2.55393), ('apple/contacts.create', 2.46922), ('apple/reminders.create', 2.39849), ('apple/alarms.cancel', 2.31217), ('apple/reminders.lists', 2.1669), ('apple/battery.get', 2.08823), ('apple/maps.eta', 2.08823), ('apple/calendar.events.update', 2.05171), ('apple/contacts.update', 2.02514), ('apple/health.statistics', 1.96058), ('apple/contacts.get', 1.59613), ('apple/weather.daily', 1.5364), ('apple/reminders.list', 1.49249), ('apple/calendar.events.get', 1.458), ('apple/health.workouts.get', 1.26659), ('apple/calendar.events.delete', 1.12484), ('apple/location.geocode', 1.07002), ('apple/location.get', 1.03112), ('apple/maps.search', 0.85499), ('apple/weather.hourly', 0.8465)]

BM25 · name + description + Schema + native aliases

[('builtin/ReadFile', 13.10696), ('builtin/WriteFile', 11.991), ('builtin/EditFile', 10.6499), ('builtin/ListFiles', 9.64347), ('builtin/Memory', 8.64623), ('apple/calendar.list', 8.59282), ('apple/photos.albums.list', 8.41589), ('apple/shortcuts.run', 7.52788), ('apple/contacts.list', 6.67745), ('apple/alarms.list', 6.5309), ('apple/clipboard.read', 5.85716), ('apple/maps.show', 5.81678), ('apple/health.sleep.query', 5.75639), ('apple/clipboard.write', 5.06624), ('apple/calendar.events.list', 4.58989), ('builtin/ScheduledTasks', 4.39586), ('apple/device.get', 4.20442), ('apple/reminders.list', 4.14438), ('apple/reminders.lists', 4.12289), ('apple/links.open', 4.06064), ('apple/photos.search', 4.04711), ('apple/contacts.create', 4.00923), ('apple/contacts.update', 3.82994), ('apple/health.workouts.list', 3.79238), ('builtin/AskUserQuestions', 3.40078), ('apple/music.playback', 3.34261), ('apple/photos.get', 3.01957), ('apple/alarms.create', 2.79725), ('apple/music.search', 2.61463), ('apple/music.status', 2.60583), ('apple/alarms.cancel', 2.57423), ('apple/reminders.create', 2.39057), ('apple/weather.current', 2.36233), ('apple/contacts.get', 2.2101), ('apple/calendar.events.update', 2.04719), ('apple/battery.get', 1.90968), ('apple/maps.eta', 1.90968), ('apple/health.workouts.get', 1.88869), ('apple/health.statistics', 1.71251), ('apple/calendar.events.delete', 1.41756), ('apple/weather.daily', 1.38091), ('apple/calendar.events.get', 1.12534), ('apple/maps.search', 1.09049), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/weather.hourly', 0.83632), ('apple/contacts.delete', 0.60095), ('apple/calendar.events.create', 0.30692)]

BM25 · name + description + Schema + extended aliases

[('builtin/ReadFile', 19.84149), ('builtin/WriteFile', 17.79459), ('builtin/ListFiles', 10.84359), ('builtin/EditFile', 10.24338), ('apple/calendar.list', 9.49632), ('apple/photos.albums.list', 9.34253), ('builtin/Memory', 8.08021), ('apple/contacts.list', 7.60409), ('apple/shortcuts.run', 7.52788), ('apple/alarms.list', 7.2299), ('apple/maps.show', 5.81678), ('apple/health.sleep.query', 5.43506), ('apple/calendar.events.list', 5.1678), ('apple/reminders.list', 5.02586), ('apple/device.get', 4.91968), ('apple/clipboard.write', 4.74852), ('apple/clipboard.read', 4.67189), ('apple/health.workouts.list', 4.26863), ('apple/reminders.lists', 4.12289), ('apple/links.open', 4.06064), ('builtin/ScheduledTasks', 4.05996), ('apple/contacts.create', 4.00923), ('apple/photos.search', 3.83626), ('apple/contacts.update', 3.82994), ('apple/photos.get', 3.76333), ('builtin/AskUserQuestions', 3.40078), ('apple/music.playback', 3.20564), ('apple/alarms.create', 2.79725), ('apple/contacts.get', 2.74349), ('apple/battery.get', 2.62899), ('apple/music.search', 2.61463), ('apple/music.status', 2.60583), ('apple/alarms.cancel', 2.57423), ('apple/health.workouts.get', 2.41094), ('apple/reminders.create', 2.39057), ('apple/calendar.events.get', 2.10202), ('apple/calendar.events.update', 2.04719), ('apple/weather.current', 2.00237), ('apple/location.get', 1.82604), ('apple/maps.eta', 1.59656), ('apple/health.statistics', 1.50695), ('apple/calendar.events.delete', 1.41756), ('apple/weather.daily', 1.15449), ('apple/maps.search', 1.09049), ('apple/reminders.get', 0.951), ('apple/location.geocode', 0.88363), ('apple/contacts.delete', 0.60095), ('apple/weather.hourly', 0.59185), ('apple/calendar.events.create', 0.30692)]

N066 Read file contents, not lasting user memories

dev · interference · target ['builtin/ReadFile']

Ordinary file text, not Memory namespace.

MethodFirst acceptable rankTop5
Keyword matching2builtin/Memory → builtin/ReadFile → builtin/EditFile → apple/clipboard.read → builtin/ScheduledTasks
BM25 · name＋description2builtin/Memory → builtin/ReadFile → builtin/EditFile → apple/clipboard.write → builtin/ScheduledTasks
BM25 · name + description + Schema + native aliases2builtin/Memory → builtin/ReadFile → builtin/WriteFile → builtin/EditFile → builtin/AskUserQuestions
BM25 · name + description + Schema + extended aliases2builtin/Memory → builtin/ReadFile → builtin/WriteFile → builtin/EditFile → builtin/AskUserQuestions
Full rankings, within-method scores

Keyword matching

[('builtin/Memory', 9.5), ('builtin/ReadFile', 8), ('builtin/EditFile', 5.5), ('apple/clipboard.read', 5), ('builtin/ScheduledTasks', 5), ('builtin/WriteFile', 5), ('builtin/AskUserQuestions', 4), ('builtin/ListFiles', 3), ('apple/clipboard.write', 2.5), ('apple/health.sleep.query', 2.5), ('apple/photos.search', 2.5), ('apple/alarms.list', 2), ('apple/battery.get', 2), ('apple/device.get', 2), ('apple/health.statistics', 2), ('apple/maps.eta', 2), ('apple/maps.show', 2), ('apple/photos.get', 2), ('apple/weather.current', 2), ('apple/weather.daily', 2), ('apple/calendar.events.list', 1.5), ('apple/contacts.list', 1.5), ('apple/health.workouts.list', 1.5), ('apple/links.open', 1.5), ('apple/maps.search', 1.5), ('apple/photos.albums.list', 1.5), ('apple/alarms.cancel', 1), ('apple/alarms.create', 1), ('apple/calendar.events.delete', 1), ('apple/contacts.get', 1), ('apple/health.workouts.get', 1), ('apple/location.geocode', 1), ('apple/location.get', 1), ('apple/music.playback', 1), ('apple/music.search', 1), ('apple/music.status', 1), ('apple/reminders.create', 1), ('apple/weather.hourly', 1), ('apple/reminders.list', 0.5), ('apple/shortcuts.run', 0.5)]

BM25 · name＋description

[('builtin/Memory', 15.44059), ('builtin/ReadFile', 5.20811), ('builtin/EditFile', 4.73732), ('apple/clipboard.write', 4.3968), ('builtin/ScheduledTasks', 4.05903), ('builtin/WriteFile', 3.71445), ('builtin/AskUserQuestions', 3.28237), ('apple/clipboard.read', 3.27541), ('apple/device.get', 3.2479), ('builtin/ListFiles', 3.05349), ('apple/maps.show', 2.82063), ('apple/health.sleep.query', 2.64535), ('apple/weather.current', 2.62614), ('apple/alarms.list', 2.22583), ('apple/photos.search', 2.20558), ('apple/music.playback', 2.16711), ('apple/photos.get', 2.11376), ('apple/battery.get', 2.08823), ('apple/maps.eta', 2.08823), ('apple/calendar.events.delete', 2.04177), ('apple/health.statistics', 1.96058), ('apple/alarms.cancel', 1.89057), ('apple/contacts.get', 1.59613), ('apple/weather.daily', 1.5364), ('apple/photos.albums.list', 1.3627), ('apple/health.workouts.get', 1.26659), ('apple/music.search', 1.21623), ('apple/health.workouts.list', 1.21491), ('apple/reminders.create', 1.11196), ('apple/calendar.events.list', 1.0957), ('apple/location.geocode', 1.07002), ('apple/location.get', 1.03112), ('apple/music.status', 0.98345), ('apple/contacts.list', 0.92972), ('apple/links.open', 0.85983), ('apple/maps.search', 0.85499), ('apple/weather.hourly', 0.8465), ('apple/alarms.create', 0.82997)]

BM25 · name + description + Schema + native aliases

[('builtin/Memory', 28.18899), ('builtin/ReadFile', 13.10696), ('builtin/WriteFile', 11.991), ('builtin/EditFile', 10.6499), ('builtin/AskUserQuestions', 8.40935), ('apple/clipboard.read', 5.85716), ('builtin/ListFiles', 5.80426), ('apple/clipboard.write', 5.06624), ('builtin/ScheduledTasks', 4.27253), ('apple/device.get', 3.25566), ('apple/health.sleep.query', 2.72365), ('apple/maps.show', 2.644), ('apple/weather.current', 2.36233), ('apple/photos.search', 2.35842), ('apple/calendar.events.delete', 2.06392), ('apple/alarms.list', 1.99017), ('apple/battery.get', 1.90968), ('apple/maps.eta', 1.90968), ('apple/photos.get', 1.88259), ('apple/alarms.cancel', 1.87912), ('apple/music.playback', 1.72095), ('apple/health.statistics', 1.71251), ('apple/photos.albums.list', 1.61938), ('apple/contacts.get', 1.60914), ('apple/health.workouts.list', 1.54245), ('apple/calendar.events.list', 1.42372), ('apple/weather.daily', 1.38091), ('apple/health.workouts.get', 1.30421), ('apple/contacts.list', 1.25), ('apple/maps.search', 1.09049), ('apple/music.search', 0.99308), ('apple/reminders.create', 0.92079), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/links.open', 0.84079), ('apple/weather.hourly', 0.83632), ('apple/music.status', 0.8076), ('apple/shortcuts.run', 0.7661), ('apple/alarms.create', 0.67486), ('apple/reminders.list', 0.31471)]

BM25 · name + description + Schema + extended aliases

[('builtin/Memory', 38.1483), ('builtin/ReadFile', 19.84149), ('builtin/WriteFile', 17.79459), ('builtin/EditFile', 10.24338), ('builtin/AskUserQuestions', 8.40935), ('builtin/ListFiles', 5.80426), ('apple/clipboard.write', 4.74852), ('apple/clipboard.read', 4.67189), ('apple/device.get', 3.97092), ('builtin/ScheduledTasks', 3.93664), ('apple/alarms.list', 2.68917), ('apple/maps.show', 2.644), ('apple/battery.get', 2.62899), ('apple/photos.get', 2.62635), ('apple/photos.albums.list', 2.54602), ('apple/health.sleep.query', 2.40232), ('apple/contacts.list', 2.17665), ('apple/photos.search', 2.14756), ('apple/contacts.get', 2.14254), ('apple/calendar.events.delete', 2.06392), ('apple/health.workouts.list', 2.0187), ('apple/weather.current', 2.00237), ('apple/calendar.events.list', 2.00162), ('apple/alarms.cancel', 1.87912), ('apple/health.workouts.get', 1.82646), ('apple/location.get', 1.82604), ('apple/maps.eta', 1.59656), ('apple/music.playback', 1.58398), ('apple/health.statistics', 1.50695), ('apple/reminders.list', 1.1962), ('apple/weather.daily', 1.15449), ('apple/maps.search', 1.09049), ('apple/music.search', 0.99308), ('apple/calendar.events.get', 0.97668), ('apple/reminders.get', 0.951), ('apple/reminders.create', 0.92079), ('apple/calendar.list', 0.9035), ('apple/location.geocode', 0.88363), ('apple/links.open', 0.84079), ('apple/music.status', 0.8076), ('apple/shortcuts.run', 0.7661), ('apple/alarms.create', 0.67486), ('apple/weather.hourly', 0.59185)]

N067 Replace the entire file with this content, allowing overwrite

dev · interference · target ['builtin/WriteFile']

Whole replacement uses WriteFile, not one-segment EditFile.

MethodFirst acceptable rankTop5
Keyword matching2builtin/EditFile → builtin/WriteFile → builtin/ReadFile → apple/clipboard.write → builtin/ListFiles
BM25 · name＋description6builtin/EditFile → builtin/Memory → apple/contacts.update → apple/clipboard.write → apple/music.playback
BM25 · name + description + Schema + native aliases1builtin/WriteFile → builtin/EditFile → builtin/ReadFile → apple/clipboard.write → builtin/Memory
BM25 · name + description + Schema + extended aliases1builtin/WriteFile → builtin/ReadFile → builtin/EditFile → apple/contacts.update → builtin/Memory
Full rankings, within-method scores

Keyword matching

[('builtin/EditFile', 5.5), ('builtin/WriteFile', 5.5), ('builtin/ReadFile', 4), ('apple/clipboard.write', 3), ('builtin/ListFiles', 3), ('builtin/Memory', 3), ('apple/contacts.update', 2), ('apple/music.playback', 2), ('apple/clipboard.read', 1), ('apple/photos.search', 1), ('apple/shortcuts.run', 0.5)]

BM25 · name＋description

[('builtin/EditFile', 6.52211), ('builtin/Memory', 5.91486), ('apple/contacts.update', 5.58627), ('apple/clipboard.write', 5.39107), ('apple/music.playback', 3.78464), ('builtin/WriteFile', 3.71445), ('builtin/ReadFile', 3.47241), ('builtin/ListFiles', 3.05349), ('apple/clipboard.read', 1.93144), ('apple/photos.search', 1.47053)]

BM25 · name + description + Schema + native aliases

[('builtin/WriteFile', 13.46306), ('builtin/EditFile', 12.47058), ('builtin/ReadFile', 8.54967), ('apple/clipboard.write', 8.13394), ('builtin/Memory', 7.95071), ('builtin/ListFiles', 5.80426), ('apple/contacts.update', 5.55244), ('apple/music.playback', 3.30494), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.35321), ('apple/shortcuts.run', 0.7661)]

BM25 · name + description + Schema + extended aliases

[('builtin/WriteFile', 19.26665), ('builtin/ReadFile', 15.63971), ('builtin/EditFile', 14.80243), ('apple/contacts.update', 7.81672), ('builtin/Memory', 7.53329), ('apple/clipboard.write', 7.50913), ('builtin/ListFiles', 5.80426), ('apple/music.playback', 2.95626), ('apple/calendar.events.update', 2.49193), ('apple/reminders.update', 2.43265), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.35321), ('apple/shortcuts.run', 0.7661)]

N068 Edit exactly one matched passage, preserving the rest of the file

dev · interference · target ['builtin/EditFile']

Unique segment edit, not full write.

MethodFirst acceptable rankTop5
Keyword matching1builtin/EditFile → builtin/ReadFile → builtin/WriteFile → builtin/ListFiles → builtin/Memory
BM25 · name＋description1builtin/EditFile → builtin/Memory → builtin/ListFiles → builtin/WriteFile → builtin/ReadFile
BM25 · name + description + Schema + native aliases1builtin/EditFile → builtin/Memory → builtin/WriteFile → builtin/ReadFile → builtin/ListFiles
BM25 · name + description + Schema + extended aliases1builtin/EditFile → builtin/WriteFile → builtin/ReadFile → builtin/Memory → builtin/ListFiles
Full rankings, within-method scores

Keyword matching

[('builtin/EditFile', 9.5), ('builtin/ReadFile', 4.5), ('builtin/WriteFile', 4.5), ('builtin/ListFiles', 4), ('builtin/Memory', 4), ('apple/alarms.cancel', 1), ('apple/alarms.create', 1), ('apple/calendar.events.create', 1), ('apple/calendar.events.delete', 1), ('apple/calendar.events.update', 1), ('apple/clipboard.read', 1), ('apple/clipboard.write', 1), ('apple/contacts.create', 1), ('apple/contacts.delete', 1), ('apple/contacts.update', 1), ('apple/health.workouts.get', 1), ('apple/location.get', 1), ('apple/maps.route', 1), ('apple/music.playback', 1), ('apple/photos.get', 1), ('apple/photos.search', 1), ('apple/reminders.delete', 1), ('apple/shortcuts.run', 0.5)]

BM25 · name＋description

[('builtin/EditFile', 14.00479), ('builtin/Memory', 7.90879), ('builtin/ListFiles', 4.7357), ('builtin/WriteFile', 3.71445), ('builtin/ReadFile', 3.47241), ('apple/clipboard.read', 1.93144), ('apple/contacts.create', 1.66967), ('apple/photos.search', 1.47053), ('apple/contacts.update', 1.45931), ('apple/reminders.delete', 1.41606), ('apple/health.workouts.get', 1.39598), ('apple/calendar.events.update', 1.34012), ('apple/contacts.delete', 1.33902), ('apple/location.get', 1.19299), ('apple/calendar.events.delete', 1.12484), ('apple/clipboard.write', 1.09971), ('apple/maps.route', 1.05267), ('apple/alarms.cancel', 1.04154), ('apple/photos.get', 0.98921), ('apple/calendar.events.create', 0.96973), ('apple/alarms.create', 0.96027), ('apple/music.playback', 0.77202)]

BM25 · name + description + Schema + native aliases

[('builtin/EditFile', 25.30036), ('builtin/Memory', 10.49006), ('builtin/WriteFile', 10.32191), ('builtin/ReadFile', 9.06819), ('builtin/ListFiles', 7.45568), ('apple/clipboard.read', 1.80248), ('apple/contacts.create', 1.59734), ('apple/contacts.update', 1.38404), ('apple/health.workouts.get', 1.36911), ('apple/reminders.delete', 1.36911), ('apple/photos.search', 1.35321), ('apple/contacts.delete', 1.28899), ('apple/calendar.events.update', 1.27396), ('apple/location.get', 1.139), ('apple/calendar.events.delete', 1.083), ('apple/clipboard.write', 1.04448), ('apple/maps.route', 0.99719), ('apple/alarms.cancel', 0.98603), ('apple/photos.get', 0.93377), ('apple/calendar.events.create', 0.92398), ('apple/alarms.create', 0.905), ('apple/shortcuts.run', 0.7661), ('apple/music.playback', 0.72007)]

BM25 · name + description + Schema + extended aliases

[('builtin/EditFile', 31.0313), ('builtin/WriteFile', 12.98291), ('builtin/ReadFile', 11.65734), ('builtin/Memory', 9.20963), ('builtin/ListFiles', 7.45568), ('apple/contacts.update', 4.33224), ('apple/calendar.events.update', 4.15033), ('apple/reminders.update', 2.80794), ('apple/clipboard.read', 1.80248), ('apple/contacts.create', 1.59734), ('apple/health.workouts.get', 1.36911), ('apple/reminders.delete', 1.36911), ('apple/photos.search', 1.35321), ('apple/contacts.delete', 1.28899), ('apple/location.get', 1.139), ('apple/calendar.events.delete', 1.083), ('apple/clipboard.write', 1.04448), ('apple/maps.route', 0.99719), ('apple/alarms.cancel', 0.98603), ('apple/photos.get', 0.93377), ('apple/calendar.events.create', 0.92398), ('apple/alarms.create', 0.905), ('apple/shortcuts.run', 0.7661), ('apple/music.playback', 0.72007)]

N069 Delete a lasting memory file, not a normal workspace file

dev · interference · target ['builtin/Memory']

Memory explicitly supports delete within its namespace.

MethodFirst acceptable rankTop5
Keyword matching1builtin/Memory → builtin/EditFile → builtin/WriteFile → builtin/ReadFile → apple/calendar.events.delete
BM25 · name＋description1builtin/Memory → builtin/ScheduledTasks → apple/reminders.delete → builtin/WriteFile → apple/contacts.delete
BM25 · name + description + Schema + native aliases1builtin/Memory → builtin/WriteFile → builtin/EditFile → apple/reminders.delete → apple/contacts.delete
BM25 · name + description + Schema + extended aliases1builtin/Memory → builtin/WriteFile → builtin/ReadFile → builtin/EditFile → apple/reminders.delete
Full rankings, within-method scores

Keyword matching

[('builtin/Memory', 8.5), ('builtin/EditFile', 5), ('builtin/WriteFile', 5), ('builtin/ReadFile', 4.5), ('apple/calendar.events.delete', 4), ('apple/contacts.delete', 4), ('apple/reminders.delete', 4), ('builtin/ListFiles', 3.5), ('builtin/ScheduledTasks', 3.5), ('apple/contacts.list', 1.5), ('apple/health.sleep.query', 1.5), ('apple/links.open', 1.5), ('apple/maps.search', 1.5), ('apple/photos.albums.list', 1.5), ('apple/photos.search', 1.5), ('apple/alarms.create', 1), ('apple/alarms.list', 1), ('apple/battery.get', 1), ('apple/clipboard.read', 1), ('apple/clipboard.write', 1), ('apple/health.statistics', 1), ('apple/location.geocode', 1), ('apple/location.get', 1), ('apple/maps.eta', 1), ('apple/maps.show', 1), ('apple/music.search', 1), ('apple/music.status', 1), ('apple/photos.get', 1), ('apple/reminders.create', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/calendar.events.list', 0.5), ('apple/health.workouts.list', 0.5), ('apple/reminders.list', 0.5), ('apple/shortcuts.run', 0.5)]

BM25 · name＋description

[('builtin/Memory', 14.14591), ('builtin/ScheduledTasks', 3.90907), ('apple/reminders.delete', 3.75288), ('builtin/WriteFile', 3.71445), ('apple/contacts.delete', 3.62112), ('builtin/ReadFile', 3.47241), ('builtin/EditFile', 3.37407), ('apple/calendar.events.delete', 3.22484), ('builtin/ListFiles', 3.05349), ('apple/clipboard.read', 1.93144), ('apple/health.sleep.query', 1.56294), ('apple/photos.search', 1.47053), ('apple/weather.current', 1.38488), ('apple/photos.albums.list', 1.3627), ('apple/clipboard.write', 1.30984), ('apple/alarms.list', 1.27072), ('apple/health.statistics', 1.24325), ('apple/music.search', 1.21623), ('apple/photos.get', 1.21623), ('apple/reminders.create', 1.11196), ('apple/location.geocode', 1.07002), ('apple/location.get', 1.03112), ('apple/battery.get', 1.01877), ('apple/maps.eta', 1.01877), ('apple/music.status', 0.98345), ('apple/contacts.list', 0.92972), ('apple/maps.show', 0.90984), ('apple/links.open', 0.85983), ('apple/maps.search', 0.85499), ('apple/alarms.create', 0.82997), ('apple/weather.daily', 0.74955)]

BM25 · name + description + Schema + native aliases

[('builtin/Memory', 26.40639), ('builtin/WriteFile', 11.51567), ('builtin/EditFile', 10.39071), ('apple/reminders.delete', 10.1562), ('apple/contacts.delete', 9.97994), ('builtin/ReadFile', 9.78891), ('apple/calendar.events.delete', 8.6982), ('builtin/ListFiles', 7.17397), ('builtin/ScheduledTasks', 4.0321), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.63711), ('apple/health.sleep.query', 1.62443), ('apple/photos.albums.list', 1.61938), ('apple/contacts.list', 1.25), ('apple/weather.current', 1.13096), ('apple/maps.search', 1.09049), ('apple/clipboard.write', 1.07428), ('apple/alarms.list', 1.04026), ('apple/health.statistics', 1.00933), ('apple/music.search', 0.99308), ('apple/photos.get', 0.99308), ('apple/reminders.create', 0.92079), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/links.open', 0.84079), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/music.status', 0.8076), ('apple/shortcuts.run', 0.7661), ('apple/maps.show', 0.74361), ('apple/alarms.create', 0.67486), ('apple/weather.daily', 0.60635), ('apple/reminders.list', 0.31471), ('apple/calendar.events.list', 0.30991), ('apple/health.workouts.list', 0.29633)]

BM25 · name + description + Schema + extended aliases

[('builtin/Memory', 36.93171), ('builtin/WriteFile', 14.17667), ('builtin/ReadFile', 12.37807), ('builtin/EditFile', 10.39071), ('apple/reminders.delete', 10.1562), ('apple/contacts.delete', 9.97994), ('apple/calendar.events.delete', 8.6982), ('builtin/ListFiles', 7.17397), ('builtin/ScheduledTasks', 4.0321), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.63711), ('apple/health.sleep.query', 1.62443), ('apple/photos.albums.list', 1.61938), ('apple/contacts.list', 1.25), ('apple/weather.current', 1.13096), ('apple/maps.search', 1.09049), ('apple/clipboard.write', 1.07428), ('apple/alarms.list', 1.04026), ('apple/health.statistics', 1.00933), ('apple/music.search', 0.99308), ('apple/photos.get', 0.99308), ('apple/reminders.create', 0.92079), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/links.open', 0.84079), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/music.status', 0.8076), ('apple/shortcuts.run', 0.7661), ('apple/maps.show', 0.74361), ('apple/alarms.create', 0.67486), ('apple/weather.daily', 0.60635), ('apple/reminders.list', 0.31471), ('apple/calendar.events.list', 0.30991), ('apple/health.workouts.list', 0.29633)]

N070 Current date and time zone, not a weather forecast

dev · interference · target ['builtin/GetCurrentTime']

Current clock metadata.

MethodFirst acceptable rankTop5
Keyword matching2apple/weather.current → builtin/GetCurrentTime → apple/weather.daily → apple/weather.hourly → apple/reminders.list
BM25 · name＋description1builtin/GetCurrentTime → apple/weather.daily → apple/weather.current → apple/weather.hourly → apple/reminders.list
BM25 · name + description + Schema + native aliases3apple/weather.current → apple/weather.daily → builtin/GetCurrentTime → apple/weather.hourly → apple/reminders.list
BM25 · name + description + Schema + extended aliases1builtin/GetCurrentTime → apple/weather.current → apple/weather.daily → apple/weather.hourly → apple/reminders.list
Full rankings, within-method scores

Keyword matching

[('apple/weather.current', 10), ('builtin/GetCurrentTime', 10), ('apple/weather.daily', 8), ('apple/weather.hourly', 7), ('apple/reminders.list', 4.5), ('apple/reminders.create', 3.5), ('apple/alarms.create', 3), ('apple/health.statistics', 3), ('apple/maps.eta', 3), ('apple/music.status', 3), ('builtin/ScheduledTasks', 3), ('apple/calendar.events.list', 2.5), ('apple/health.sleep.query', 2.5), ('apple/maps.search', 2.5), ('apple/calendar.events.update', 2), ('apple/location.geocode', 2), ('apple/location.get', 2), ('apple/maps.route', 2), ('apple/maps.show', 2), ('apple/photos.search', 2), ('apple/contacts.list', 1.5), ('apple/health.workouts.list', 1.5), ('apple/links.open', 1.5), ('apple/photos.albums.list', 1.5), ('apple/reminders.update', 1.5), ('builtin/EditFile', 1.5), ('apple/alarms.cancel', 1), ('apple/alarms.list', 1), ('apple/battery.get', 1), ('apple/calendar.events.create', 1), ('apple/clipboard.write', 1), ('apple/location.reverseGeocode', 1), ('apple/music.playback', 1), ('apple/music.search', 1), ('apple/photos.get', 1)]

BM25 · name＋description

[('builtin/GetCurrentTime', 12.02097), ('apple/weather.daily', 11.2016), ('apple/weather.current', 9.82095), ('apple/weather.hourly', 8.26143), ('apple/reminders.list', 6.73678), ('apple/maps.eta', 4.85808), ('apple/music.status', 4.09127), ('apple/reminders.create', 3.61345), ('apple/location.get', 3.45199), ('apple/photos.search', 3.15446), ('apple/maps.route', 2.87521), ('apple/location.geocode', 2.58031), ('apple/alarms.create', 2.2813), ('apple/health.statistics', 2.20776), ('apple/maps.show', 2.19406), ('apple/maps.search', 2.06179), ('builtin/EditFile', 1.83299), ('apple/calendar.events.list', 1.82518), ('apple/calendar.events.update', 1.79701), ('apple/location.reverseGeocode', 1.63355), ('apple/alarms.cancel', 1.57416), ('apple/health.sleep.query', 1.56294), ('apple/photos.albums.list', 1.3627), ('apple/clipboard.write', 1.30984), ('apple/alarms.list', 1.27072), ('apple/music.search', 1.21623), ('apple/photos.get', 1.21623), ('apple/battery.get', 1.01877), ('apple/music.playback', 0.94183), ('apple/contacts.list', 0.92972), ('apple/links.open', 0.85983), ('builtin/ScheduledTasks', 0.47665)]

BM25 · name + description + Schema + native aliases

[('apple/weather.current', 19.91765), ('apple/weather.daily', 17.09571), ('builtin/GetCurrentTime', 14.7208), ('apple/weather.hourly', 14.15869), ('apple/reminders.list', 6.11216), ('apple/reminders.create', 5.74698), ('apple/photos.search', 4.04393), ('apple/maps.eta', 3.79823), ('apple/health.statistics', 3.45168), ('apple/alarms.create', 3.32241), ('apple/music.status', 3.26331), ('apple/health.sleep.query', 3.25882), ('apple/calendar.events.list', 3.09838), ('apple/calendar.events.update', 3.00106), ('builtin/ScheduledTasks', 2.64083), ('apple/reminders.update', 2.61949), ('apple/location.get', 2.45369), ('builtin/EditFile', 2.354), ('apple/location.geocode', 2.32647), ('apple/maps.route', 2.26114), ('apple/maps.search', 2.22747), ('apple/maps.show', 1.9578), ('apple/health.workouts.list', 1.8063), ('apple/photos.albums.list', 1.61938), ('apple/location.reverseGeocode', 1.5928), ('apple/calendar.events.create', 1.49372), ('apple/contacts.list', 1.25), ('apple/clipboard.write', 1.07428), ('apple/alarms.list', 1.04026), ('apple/alarms.cancel', 1.03523), ('apple/music.search', 0.99308), ('apple/photos.get', 0.99308), ('apple/music.playback', 0.87677), ('apple/links.open', 0.84079), ('apple/battery.get', 0.83852)]

BM25 · name + description + Schema + extended aliases

[('builtin/GetCurrentTime', 20.93045), ('apple/weather.current', 19.91765), ('apple/weather.daily', 17.09571), ('apple/weather.hourly', 14.15869), ('apple/reminders.list', 6.11216), ('apple/reminders.create', 5.74698), ('apple/photos.search', 4.04393), ('apple/maps.eta', 3.79823), ('apple/health.statistics', 3.45168), ('apple/alarms.create', 3.32241), ('apple/music.status', 3.26331), ('apple/health.sleep.query', 3.25882), ('apple/calendar.events.list', 3.09838), ('apple/calendar.events.update', 3.00106), ('builtin/ScheduledTasks', 2.64083), ('apple/reminders.update', 2.61949), ('apple/location.get', 2.45369), ('builtin/EditFile', 2.354), ('apple/location.geocode', 2.32647), ('apple/maps.route', 2.26114), ('apple/maps.search', 2.22747), ('apple/maps.show', 1.9578), ('apple/health.workouts.list', 1.8063), ('apple/photos.albums.list', 1.61938), ('apple/location.reverseGeocode', 1.5928), ('apple/calendar.events.create', 1.49372), ('apple/contacts.list', 1.25), ('apple/clipboard.write', 1.07428), ('apple/alarms.list', 1.04026), ('apple/alarms.cancel', 1.03523), ('apple/music.search', 0.99308), ('apple/photos.get', 0.99308), ('apple/music.playback', 0.87677), ('apple/links.open', 0.84079), ('apple/battery.get', 0.83852)]

N071 Delete this non-recurring calendar event, not a reminder

dev · interference · target ['apple/calendar.events.delete']

Calendar-specific delete.

MethodFirst acceptable rankTop5
Keyword matching1apple/calendar.events.delete → apple/calendar.events.create → apple/calendar.events.update → apple/reminders.delete → apple/calendar.events.get
BM25 · name＋description1apple/calendar.events.delete → apple/reminders.delete → apple/calendar.events.get → apple/calendar.events.update → apple/calendar.events.create
BM25 · name + description + Schema + native aliases1apple/calendar.events.delete → apple/reminders.delete → apple/calendar.events.get → apple/calendar.events.create → apple/calendar.events.update
BM25 · name + description + Schema + extended aliases1apple/calendar.events.delete → apple/reminders.delete → apple/calendar.events.get → apple/calendar.events.create → apple/calendar.events.update
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.delete', 13.5), ('apple/calendar.events.create', 11), ('apple/calendar.events.update', 10.5), ('apple/reminders.delete', 10.5), ('apple/calendar.events.get', 9.5), ('apple/calendar.events.list', 9), ('apple/links.open', 5.5), ('apple/reminders.complete', 5.5), ('apple/reminders.update', 5.5), ('apple/reminders.create', 5), ('apple/reminders.get', 4.5), ('apple/reminders.list', 4.5), ('apple/calendar.list', 4), ('apple/contacts.delete', 4), ('apple/reminders.lists', 4), ('builtin/ScheduledTasks', 4), ('apple/health.statistics', 2), ('apple/weather.daily', 2), ('apple/contacts.list', 1.5), ('apple/health.sleep.query', 1.5), ('apple/maps.search', 1.5), ('apple/photos.albums.list', 1.5), ('builtin/Memory', 1.5), ('apple/alarms.create', 1), ('apple/alarms.list', 1), ('apple/battery.get', 1), ('apple/clipboard.write', 1), ('apple/location.geocode', 1), ('apple/location.get', 1), ('apple/maps.eta', 1), ('apple/maps.show', 1), ('apple/music.search', 1), ('apple/music.status', 1), ('apple/photos.get', 1), ('apple/weather.current', 1), ('apple/health.workouts.list', 0.5), ('apple/photos.search', 0.5)]

BM25 · name＋description

[('apple/calendar.events.delete', 14.17009), ('apple/reminders.delete', 12.79805), ('apple/calendar.events.get', 10.1236), ('apple/calendar.events.update', 10.12086), ('apple/calendar.events.create', 9.8839), ('apple/calendar.events.list', 6.6899), ('apple/reminders.complete', 6.17579), ('apple/reminders.update', 5.56769), ('apple/reminders.create', 4.16862), ('apple/contacts.delete', 3.62112), ('apple/reminders.get', 3.61281), ('apple/reminders.lists', 3.54122), ('apple/calendar.list', 3.35399), ('apple/reminders.list', 3.11095), ('apple/health.statistics', 2.589), ('builtin/Memory', 2.44041), ('builtin/ScheduledTasks', 2.36105), ('apple/weather.daily', 2.22571), ('apple/health.sleep.query', 1.56294), ('apple/weather.current', 1.38488), ('apple/photos.albums.list', 1.3627), ('apple/clipboard.write', 1.30984), ('apple/alarms.list', 1.27072), ('apple/music.search', 1.21623), ('apple/photos.get', 1.21623), ('apple/location.geocode', 1.07002), ('apple/location.get', 1.03112), ('apple/battery.get', 1.01877), ('apple/maps.eta', 1.01877), ('apple/music.status', 0.98345), ('apple/contacts.list', 0.92972), ('apple/maps.show', 0.90984), ('apple/links.open', 0.85983), ('apple/maps.search', 0.85499), ('apple/alarms.create', 0.82997)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.delete', 29.35636), ('apple/reminders.delete', 25.16843), ('apple/calendar.events.get', 21.12868), ('apple/calendar.events.create', 20.63395), ('apple/calendar.events.update', 20.08332), ('apple/calendar.events.list', 16.8576), ('apple/reminders.complete', 12.11426), ('apple/reminders.update', 10.83032), ('apple/contacts.delete', 9.97994), ('apple/reminders.get', 9.60564), ('apple/reminders.create', 8.45913), ('apple/reminders.list', 8.16022), ('apple/reminders.lists', 8.13862), ('apple/calendar.list', 7.72596), ('builtin/ScheduledTasks', 5.15669), ('builtin/Memory', 3.1123), ('apple/links.open', 2.21235), ('apple/health.statistics', 2.18066), ('apple/weather.daily', 1.8966), ('apple/health.sleep.query', 1.62443), ('apple/photos.albums.list', 1.61938), ('apple/contacts.list', 1.25), ('apple/weather.current', 1.13096), ('apple/maps.search', 1.09049), ('apple/clipboard.write', 1.07428), ('apple/alarms.list', 1.04026), ('apple/music.search', 0.99308), ('apple/photos.get', 0.99308), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/music.status', 0.8076), ('apple/maps.show', 0.74361), ('apple/alarms.create', 0.67486), ('apple/health.workouts.list', 0.29633), ('apple/photos.search', 0.2839)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.delete', 29.18114), ('apple/reminders.delete', 25.00599), ('apple/calendar.events.get', 20.95829), ('apple/calendar.events.create', 20.52432), ('apple/calendar.events.update', 19.93217), ('apple/calendar.events.list', 16.8576), ('apple/reminders.complete', 11.93813), ('apple/reminders.update', 10.67739), ('apple/contacts.delete', 9.97994), ('apple/reminders.get', 9.60564), ('apple/reminders.create', 8.45913), ('apple/reminders.list', 8.16022), ('apple/reminders.lists', 8.13862), ('builtin/ScheduledTasks', 7.74585), ('apple/calendar.list', 7.72596), ('apple/links.open', 3.1939), ('builtin/Memory', 3.1123), ('apple/health.statistics', 2.18066), ('apple/weather.daily', 1.8966), ('apple/health.sleep.query', 1.62443), ('apple/photos.albums.list', 1.61938), ('apple/contacts.list', 1.25), ('apple/weather.current', 1.13096), ('apple/maps.search', 1.09049), ('apple/clipboard.write', 1.07428), ('apple/alarms.list', 1.04026), ('apple/music.search', 0.99308), ('apple/photos.get', 0.99308), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/music.status', 0.8076), ('apple/maps.show', 0.74361), ('apple/alarms.create', 0.67486), ('apple/health.workouts.list', 0.29633), ('apple/photos.search', 0.2839)]

N072 Delete this non-recurring reminder, not a calendar event

dev · interference · target ['apple/reminders.delete']

Reminder-specific delete.

MethodFirst acceptable rankTop5
Keyword matching4apple/calendar.events.delete → apple/calendar.events.create → apple/calendar.events.update → apple/reminders.delete → apple/calendar.events.get
BM25 · name＋description2apple/calendar.events.delete → apple/reminders.delete → apple/calendar.events.get → apple/calendar.events.update → apple/calendar.events.create
BM25 · name + description + Schema + native aliases2apple/calendar.events.delete → apple/reminders.delete → apple/calendar.events.get → apple/calendar.events.create → apple/calendar.events.update
BM25 · name + description + Schema + extended aliases2apple/calendar.events.delete → apple/reminders.delete → apple/calendar.events.get → apple/calendar.events.create → apple/calendar.events.update
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.delete', 13.5), ('apple/calendar.events.create', 11), ('apple/calendar.events.update', 10.5), ('apple/reminders.delete', 10.5), ('apple/calendar.events.get', 9.5), ('apple/calendar.events.list', 9), ('apple/links.open', 5.5), ('apple/reminders.complete', 5.5), ('apple/reminders.update', 5.5), ('apple/reminders.create', 5), ('apple/reminders.get', 4.5), ('apple/reminders.list', 4.5), ('apple/calendar.list', 4), ('apple/contacts.delete', 4), ('apple/reminders.lists', 4), ('builtin/ScheduledTasks', 4), ('apple/health.statistics', 2), ('apple/weather.daily', 2), ('apple/contacts.list', 1.5), ('apple/health.sleep.query', 1.5), ('apple/maps.search', 1.5), ('apple/photos.albums.list', 1.5), ('builtin/Memory', 1.5), ('apple/alarms.create', 1), ('apple/alarms.list', 1), ('apple/battery.get', 1), ('apple/clipboard.write', 1), ('apple/location.geocode', 1), ('apple/location.get', 1), ('apple/maps.eta', 1), ('apple/maps.show', 1), ('apple/music.search', 1), ('apple/music.status', 1), ('apple/photos.get', 1), ('apple/weather.current', 1), ('apple/health.workouts.list', 0.5), ('apple/photos.search', 0.5)]

BM25 · name＋description

[('apple/calendar.events.delete', 14.17009), ('apple/reminders.delete', 12.79805), ('apple/calendar.events.get', 10.1236), ('apple/calendar.events.update', 10.12086), ('apple/calendar.events.create', 9.8839), ('apple/calendar.events.list', 6.6899), ('apple/reminders.complete', 6.17579), ('apple/reminders.update', 5.56769), ('apple/reminders.create', 4.16862), ('apple/contacts.delete', 3.62112), ('apple/reminders.get', 3.61281), ('apple/reminders.lists', 3.54122), ('apple/calendar.list', 3.35399), ('apple/reminders.list', 3.11095), ('apple/health.statistics', 2.589), ('builtin/Memory', 2.44041), ('builtin/ScheduledTasks', 2.36105), ('apple/weather.daily', 2.22571), ('apple/health.sleep.query', 1.56294), ('apple/weather.current', 1.38488), ('apple/photos.albums.list', 1.3627), ('apple/clipboard.write', 1.30984), ('apple/alarms.list', 1.27072), ('apple/music.search', 1.21623), ('apple/photos.get', 1.21623), ('apple/location.geocode', 1.07002), ('apple/location.get', 1.03112), ('apple/battery.get', 1.01877), ('apple/maps.eta', 1.01877), ('apple/music.status', 0.98345), ('apple/contacts.list', 0.92972), ('apple/maps.show', 0.90984), ('apple/links.open', 0.85983), ('apple/maps.search', 0.85499), ('apple/alarms.create', 0.82997)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.delete', 29.35636), ('apple/reminders.delete', 25.16843), ('apple/calendar.events.get', 21.12868), ('apple/calendar.events.create', 20.63395), ('apple/calendar.events.update', 20.08332), ('apple/calendar.events.list', 16.8576), ('apple/reminders.complete', 12.11426), ('apple/reminders.update', 10.83032), ('apple/contacts.delete', 9.97994), ('apple/reminders.get', 9.60564), ('apple/reminders.create', 8.45913), ('apple/reminders.list', 8.16022), ('apple/reminders.lists', 8.13862), ('apple/calendar.list', 7.72596), ('builtin/ScheduledTasks', 5.15669), ('builtin/Memory', 3.1123), ('apple/links.open', 2.21235), ('apple/health.statistics', 2.18066), ('apple/weather.daily', 1.8966), ('apple/health.sleep.query', 1.62443), ('apple/photos.albums.list', 1.61938), ('apple/contacts.list', 1.25), ('apple/weather.current', 1.13096), ('apple/maps.search', 1.09049), ('apple/clipboard.write', 1.07428), ('apple/alarms.list', 1.04026), ('apple/music.search', 0.99308), ('apple/photos.get', 0.99308), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/music.status', 0.8076), ('apple/maps.show', 0.74361), ('apple/alarms.create', 0.67486), ('apple/health.workouts.list', 0.29633), ('apple/photos.search', 0.2839)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.delete', 29.18114), ('apple/reminders.delete', 25.00599), ('apple/calendar.events.get', 20.95829), ('apple/calendar.events.create', 20.52432), ('apple/calendar.events.update', 19.93217), ('apple/calendar.events.list', 16.8576), ('apple/reminders.complete', 11.93813), ('apple/reminders.update', 10.67739), ('apple/contacts.delete', 9.97994), ('apple/reminders.get', 9.60564), ('apple/reminders.create', 8.45913), ('apple/reminders.list', 8.16022), ('apple/reminders.lists', 8.13862), ('builtin/ScheduledTasks', 7.74585), ('apple/calendar.list', 7.72596), ('apple/links.open', 3.1939), ('builtin/Memory', 3.1123), ('apple/health.statistics', 2.18066), ('apple/weather.daily', 1.8966), ('apple/health.sleep.query', 1.62443), ('apple/photos.albums.list', 1.61938), ('apple/contacts.list', 1.25), ('apple/weather.current', 1.13096), ('apple/maps.search', 1.09049), ('apple/clipboard.write', 1.07428), ('apple/alarms.list', 1.04026), ('apple/music.search', 0.99308), ('apple/photos.get', 0.99308), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/music.status', 0.8076), ('apple/maps.show', 0.74361), ('apple/alarms.create', 0.67486), ('apple/health.workouts.list', 0.29633), ('apple/photos.search', 0.2839)]

N073 Read the one contact with this identifier, do not search all contacts

dev · interference · target ['apple/contacts.get']

Read by ID, not contact-list query.

MethodFirst acceptable rankTop5
Keyword matching1apple/contacts.get → apple/contacts.list → apple/contacts.delete → apple/contacts.update → apple/maps.search
BM25 · name＋description1apple/contacts.get → apple/contacts.list → apple/health.sleep.query → apple/contacts.create → apple/contacts.delete
BM25 · name + description + Schema + native aliases1apple/contacts.get → apple/contacts.delete → apple/contacts.list → apple/contacts.update → apple/music.search
BM25 · name + description + Schema + extended aliases1apple/contacts.get → apple/contacts.list → apple/contacts.delete → apple/contacts.update → apple/contacts.create
Full rankings, within-method scores

Keyword matching

[('apple/contacts.get', 7), ('apple/contacts.list', 6.5), ('apple/contacts.delete', 6), ('apple/contacts.update', 6), ('apple/maps.search', 6), ('apple/music.search', 5.5), ('apple/contacts.create', 5), ('apple/links.open', 4.5), ('apple/photos.search', 4.5), ('builtin/ReadFile', 4.5), ('apple/clipboard.read', 4), ('apple/health.sleep.query', 3.5), ('apple/photos.get', 3.5), ('builtin/ScheduledTasks', 3.5), ('apple/photos.albums.list', 2.5), ('builtin/Memory', 2.5), ('apple/alarms.create', 2), ('apple/alarms.list', 2), ('apple/battery.get', 2), ('apple/calendar.events.create', 2), ('apple/calendar.events.update', 2), ('apple/clipboard.write', 2), ('apple/device.get', 2), ('apple/health.statistics', 2), ('apple/health.workouts.get', 2), ('apple/location.get', 2), ('apple/maps.eta', 2), ('apple/music.playback', 2), ('apple/weather.current', 2), ('apple/weather.daily', 2), ('builtin/EditFile', 2), ('apple/calendar.events.list', 1.5), ('apple/health.workouts.list', 1.5), ('apple/location.geocode', 1.5), ('apple/alarms.cancel', 1), ('apple/calendar.events.delete', 1), ('apple/calendar.list', 1), ('apple/maps.route', 1), ('apple/maps.show', 1), ('apple/music.status', 1), ('apple/reminders.create', 1), ('apple/reminders.delete', 1), ('apple/weather.hourly', 1), ('builtin/ListFiles', 1), ('apple/location.reverseGeocode', 0.5), ('apple/reminders.list', 0.5)]

BM25 · name＋description

[('apple/contacts.get', 8.7918), ('apple/contacts.list', 6.96925), ('apple/health.sleep.query', 5.63174), ('apple/contacts.create', 5.57806), ('apple/contacts.delete', 4.96014), ('apple/music.search', 4.94455), ('apple/calendar.events.update', 4.69483), ('apple/maps.search', 4.58331), ('apple/contacts.update', 4.51685), ('apple/links.open', 3.84707), ('apple/device.get', 3.79811), ('apple/calendar.list', 3.76501), ('apple/music.playback', 3.70721), ('apple/photos.albums.list', 3.67626), ('apple/calendar.events.create', 3.39722), ('apple/photos.get', 3.10297), ('builtin/EditFile', 2.86574), ('builtin/Memory', 2.78561), ('apple/health.workouts.get', 2.66257), ('apple/weather.current', 2.62614), ('apple/photos.search', 2.57922), ('apple/clipboard.write', 2.40955), ('apple/alarms.list', 2.22583), ('apple/location.get', 2.22411), ('apple/battery.get', 2.08823), ('apple/maps.eta', 2.08823), ('apple/health.statistics', 1.96058), ('apple/alarms.create', 1.79024), ('builtin/ReadFile', 1.73569), ('builtin/ListFiles', 1.68221), ('apple/weather.daily', 1.5364), ('builtin/ScheduledTasks', 1.50999), ('apple/reminders.delete', 1.41606), ('apple/clipboard.read', 1.34398), ('apple/health.workouts.list', 1.21491), ('apple/calendar.events.delete', 1.12484), ('apple/reminders.create', 1.11196), ('apple/calendar.events.list', 1.0957), ('apple/location.geocode', 1.07002), ('apple/maps.route', 1.05267), ('apple/alarms.cancel', 1.04154), ('apple/music.status', 0.98345), ('apple/maps.show', 0.90984), ('apple/weather.hourly', 0.8465)]

BM25 · name + description + Schema + native aliases

[('apple/contacts.get', 17.31638), ('apple/contacts.delete', 13.80618), ('apple/contacts.list', 12.5971), ('apple/contacts.update', 12.34537), ('apple/music.search', 11.86447), ('apple/maps.search', 11.51832), ('apple/contacts.create', 11.48832), ('apple/photos.search', 7.63538), ('apple/health.sleep.query', 5.75639), ('builtin/ReadFile', 5.07581), ('apple/links.open', 4.39943), ('apple/calendar.events.update', 4.3187), ('apple/photos.get', 4.29836), ('apple/clipboard.read', 4.05468), ('apple/photos.albums.list', 3.92678), ('builtin/Memory', 3.4685), ('apple/music.playback', 3.34696), ('apple/calendar.events.create', 3.13226), ('apple/device.get', 3.12053), ('builtin/ScheduledTasks', 2.9226), ('apple/calendar.list', 2.89696), ('builtin/EditFile', 2.85052), ('apple/health.workouts.get', 2.67332), ('apple/weather.current', 2.36233), ('apple/clipboard.write', 2.11876), ('apple/location.geocode', 2.00204), ('apple/alarms.list', 1.99017), ('apple/location.get', 1.98836), ('apple/battery.get', 1.90968), ('apple/maps.eta', 1.90968), ('apple/health.statistics', 1.71251), ('builtin/ListFiles', 1.65142), ('apple/alarms.create', 1.57986), ('apple/health.workouts.list', 1.54245), ('apple/calendar.events.list', 1.42372), ('apple/weather.daily', 1.38091), ('apple/reminders.delete', 1.36911), ('apple/location.reverseGeocode', 1.14472), ('apple/calendar.events.delete', 1.083), ('apple/maps.route', 0.99719), ('apple/alarms.cancel', 0.98603), ('apple/reminders.create', 0.92079), ('apple/weather.hourly', 0.83632), ('apple/music.status', 0.8076), ('apple/maps.show', 0.74361), ('apple/reminders.list', 0.31471)]

BM25 · name + description + Schema + extended aliases

[('apple/contacts.get', 17.84977), ('apple/contacts.list', 15.37745), ('apple/contacts.delete', 13.80618), ('apple/contacts.update', 12.34537), ('apple/contacts.create', 11.48832), ('apple/music.search', 8.17486), ('apple/maps.search', 7.97925), ('apple/photos.albums.list', 6.70714), ('apple/calendar.list', 5.60787), ('apple/health.sleep.query', 5.43506), ('apple/photos.search', 5.17433), ('builtin/ReadFile', 4.72029), ('apple/alarms.list', 4.64298), ('apple/photos.get', 4.53915), ('apple/calendar.events.update', 4.3187), ('apple/device.get', 3.83578), ('apple/calendar.events.list', 3.80904), ('apple/links.open', 3.80598), ('apple/health.workouts.list', 3.70013), ('apple/music.playback', 3.34696), ('apple/health.workouts.get', 3.19557), ('apple/calendar.events.create', 3.13226), ('builtin/Memory', 2.99753), ('apple/location.get', 2.96504), ('apple/reminders.list', 2.95957), ('apple/clipboard.read', 2.86941), ('apple/battery.get', 2.62899), ('builtin/EditFile', 2.444), ('builtin/ScheduledTasks', 2.40707), ('apple/clipboard.write', 2.11876), ('apple/weather.current', 2.00237), ('apple/location.geocode', 2.00204), ('builtin/ListFiles', 1.65142), ('apple/maps.eta', 1.59656), ('apple/alarms.create', 1.57986), ('apple/health.statistics', 1.50695), ('apple/reminders.delete', 1.36911), ('apple/weather.daily', 1.15449), ('apple/location.reverseGeocode', 1.14472), ('apple/calendar.events.delete', 1.083), ('apple/maps.route', 0.99719), ('apple/alarms.cancel', 0.98603), ('apple/calendar.events.get', 0.97668), ('apple/reminders.get', 0.951), ('apple/reminders.create', 0.92079), ('apple/music.status', 0.8076), ('apple/maps.show', 0.74361), ('apple/weather.hourly', 0.59185)]

N074 Need ordered route steps rather than just ETA

dev · interference · target ['apple/maps.route']

Only route provides ordered steps.

MethodFirst acceptable rankTop5
Keyword matching1apple/maps.route → apple/maps.eta → builtin/ScheduledTasks → apple/health.statistics → apple/calendar.events.list
BM25 · name＋description1apple/maps.route → apple/maps.eta → builtin/ScheduledTasks → apple/calendar.events.list → apple/weather.hourly
BM25 · name + description + Schema + native aliases1apple/maps.route → apple/maps.eta → builtin/ScheduledTasks → apple/calendar.events.list → apple/health.statistics
BM25 · name + description + Schema + extended aliases1apple/maps.route → apple/maps.eta → builtin/ScheduledTasks → apple/calendar.events.list → apple/health.statistics
Full rankings, within-method scores

Keyword matching

[('apple/maps.route', 6), ('apple/maps.eta', 4), ('builtin/ScheduledTasks', 2), ('apple/health.statistics', 1.5), ('apple/calendar.events.list', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1)]

BM25 · name＋description

[('apple/maps.route', 11.51784), ('apple/maps.eta', 7.16105), ('builtin/ScheduledTasks', 4.72991), ('apple/calendar.events.list', 3.94685), ('apple/weather.hourly', 2.33548), ('apple/health.statistics', 2.21932), ('apple/weather.daily', 2.1709)]

BM25 · name + description + Schema + native aliases

[('apple/maps.route', 18.93443), ('apple/maps.eta', 12.52724), ('builtin/ScheduledTasks', 6.0937), ('apple/calendar.events.list', 4.01209), ('apple/health.statistics', 3.38189), ('apple/weather.hourly', 2.3074), ('apple/weather.daily', 2.13703)]

BM25 · name + description + Schema + extended aliases

[('apple/maps.route', 18.93443), ('apple/maps.eta', 12.52724), ('builtin/ScheduledTasks', 6.0937), ('apple/calendar.events.list', 4.01209), ('apple/health.statistics', 3.38189), ('apple/weather.hourly', 2.3074), ('apple/weather.daily', 2.13703)]

N075 Current conditions only, not hourly or daily forecasts

dev · interference · target ['apple/weather.current']

Current conditions endpoint.

MethodFirst acceptable rankTop5
Keyword matching1apple/weather.current → apple/weather.daily → apple/weather.hourly → builtin/GetCurrentTime → builtin/ScheduledTasks
BM25 · name＋description2apple/weather.daily → apple/weather.current → apple/weather.hourly → apple/health.statistics → apple/music.status
BM25 · name + description + Schema + native aliases3apple/weather.daily → apple/weather.hourly → apple/weather.current → builtin/GetCurrentTime → builtin/ScheduledTasks
BM25 · name + description + Schema + extended aliases3apple/weather.daily → apple/weather.hourly → apple/weather.current → builtin/ScheduledTasks → builtin/GetCurrentTime
Full rankings, within-method scores

Keyword matching

[('apple/weather.current', 7), ('apple/weather.daily', 7), ('apple/weather.hourly', 6), ('builtin/GetCurrentTime', 4), ('builtin/ScheduledTasks', 4), ('apple/health.statistics', 3), ('apple/maps.show', 3), ('apple/music.status', 3), ('apple/links.open', 2.5), ('apple/maps.search', 2.5), ('apple/music.playback', 2.5), ('apple/photos.albums.list', 2.5), ('apple/alarms.create', 2), ('apple/alarms.list', 2), ('apple/location.geocode', 2), ('apple/maps.eta', 2), ('apple/music.search', 2), ('apple/photos.get', 2), ('apple/contacts.list', 1.5), ('apple/health.sleep.query', 1.5), ('apple/photos.search', 1.5), ('builtin/EditFile', 1.5), ('apple/alarms.cancel', 1), ('apple/battery.get', 1), ('apple/calendar.events.update', 1), ('apple/clipboard.write', 1), ('apple/location.get', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.route', 1), ('apple/reminders.create', 1), ('apple/shortcuts.run', 1), ('apple/calendar.events.list', 0.5), ('apple/health.workouts.list', 0.5), ('apple/reminders.list', 0.5), ('builtin/Memory', 0.5)]

BM25 · name＋description

[('apple/weather.daily', 9.103), ('apple/weather.current', 8.54256), ('apple/weather.hourly', 7.88358), ('apple/health.statistics', 4.42708), ('apple/music.status', 4.26407), ('apple/maps.show', 3.47828), ('apple/photos.albums.list', 2.9976), ('apple/music.search', 2.93291), ('apple/location.geocode', 2.58031), ('apple/alarms.list', 2.55494), ('apple/maps.eta', 2.45674), ('apple/photos.get', 2.42303), ('apple/links.open', 2.37167), ('builtin/GetCurrentTime', 2.20523), ('apple/maps.search', 2.06179), ('apple/alarms.create', 2.00146), ('apple/music.playback', 1.88366), ('builtin/ScheduledTasks', 1.86606), ('builtin/EditFile', 1.83299), ('apple/shortcuts.run', 1.7045), ('apple/location.reverseGeocode', 1.63355), ('apple/health.sleep.query', 1.56294), ('apple/photos.search', 1.48345), ('apple/clipboard.write', 1.30984), ('apple/maps.route', 1.28422), ('apple/alarms.cancel', 1.27063), ('apple/calendar.events.update', 1.1275), ('apple/reminders.create', 1.11196), ('apple/location.get', 1.03112), ('apple/battery.get', 1.01877), ('apple/contacts.list', 0.92972)]

BM25 · name + description + Schema + native aliases

[('apple/weather.daily', 15.91093), ('apple/weather.hourly', 15.53574), ('apple/weather.current', 11.60447), ('builtin/GetCurrentTime', 5.21653), ('builtin/ScheduledTasks', 3.98502), ('apple/music.status', 3.92451), ('apple/health.statistics', 3.84821), ('apple/maps.show', 3.172), ('apple/photos.albums.list', 3.17058), ('apple/music.search', 2.61463), ('apple/music.playback', 2.49843), ('builtin/EditFile', 2.354), ('apple/location.geocode', 2.32647), ('apple/alarms.list', 2.25445), ('apple/links.open', 2.2506), ('apple/maps.search', 2.22747), ('apple/maps.eta', 2.20769), ('apple/photos.get', 2.13006), ('apple/alarms.create', 1.7768), ('apple/photos.search', 1.67426), ('apple/health.sleep.query', 1.62443), ('apple/shortcuts.run', 1.60938), ('apple/location.reverseGeocode', 1.5928), ('apple/contacts.list', 1.25), ('apple/maps.route', 1.21419), ('apple/alarms.cancel', 1.2006), ('apple/clipboard.write', 1.07428), ('apple/calendar.events.update', 1.069), ('apple/reminders.create', 0.92079), ('apple/location.get', 0.84936), ('apple/battery.get', 0.83852), ('builtin/Memory', 0.38765), ('apple/reminders.list', 0.31471), ('apple/calendar.events.list', 0.30991), ('apple/health.workouts.list', 0.29633)]

BM25 · name + description + Schema + extended aliases

[('apple/weather.daily', 15.91093), ('apple/weather.hourly', 15.53574), ('apple/weather.current', 11.60447), ('builtin/ScheduledTasks', 7.79275), ('builtin/GetCurrentTime', 7.0813), ('apple/music.status', 3.92451), ('apple/health.statistics', 3.84821), ('apple/maps.show', 3.172), ('apple/photos.albums.list', 3.17058), ('apple/music.search', 2.61463), ('apple/music.playback', 2.49843), ('builtin/EditFile', 2.354), ('apple/location.geocode', 2.32647), ('apple/alarms.list', 2.25445), ('apple/links.open', 2.2506), ('apple/maps.search', 2.22747), ('apple/maps.eta', 2.20769), ('apple/photos.get', 2.13006), ('apple/alarms.create', 1.7768), ('apple/photos.search', 1.67426), ('apple/health.sleep.query', 1.62443), ('apple/shortcuts.run', 1.60938), ('apple/location.reverseGeocode', 1.5928), ('apple/contacts.list', 1.25), ('apple/maps.route', 1.21419), ('apple/alarms.cancel', 1.2006), ('apple/clipboard.write', 1.07428), ('apple/calendar.events.update', 1.069), ('apple/reminders.create', 0.92079), ('apple/location.get', 0.84936), ('apple/battery.get', 0.83852), ('builtin/Memory', 0.38765), ('apple/reminders.list', 0.31471), ('apple/calendar.events.list', 0.30991), ('apple/health.workouts.list', 0.29633)]

N076 Show Anan player status without changing playback

dev · interference · target ['apple/music.status']

Read-only ApplicationMusicPlayer snapshot.

MethodFirst acceptable rankTop5
Keyword matching1apple/music.status → apple/music.playback → apple/maps.show → apple/clipboard.read → apple/alarms.cancel
BM25 · name＋description1apple/music.status → apple/music.playback → apple/clipboard.read → apple/maps.show → apple/location.reverseGeocode
BM25 · name + description + Schema + native aliases1apple/music.status → apple/music.playback → apple/maps.show → apple/clipboard.read → apple/location.reverseGeocode
BM25 · name + description + Schema + extended aliases1apple/music.status → apple/music.playback → apple/maps.show → apple/alarms.list → apple/clipboard.read
Full rankings, within-method scores

Keyword matching

[('apple/music.status', 7), ('apple/music.playback', 6), ('apple/maps.show', 4), ('apple/clipboard.read', 2), ('apple/alarms.cancel', 1), ('apple/alarms.create', 1), ('apple/alarms.list', 1), ('apple/battery.get', 1), ('apple/calendar.events.update', 1), ('apple/contacts.delete', 1), ('apple/location.geocode', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.search', 1), ('apple/music.search', 1), ('apple/photos.search', 1)]

BM25 · name＋description

[('apple/music.status', 14.6578), ('apple/music.playback', 7.62793), ('apple/clipboard.read', 4.91831), ('apple/maps.show', 4.12697), ('apple/location.reverseGeocode', 2.80551), ('apple/calendar.events.update', 2.59432), ('apple/location.geocode', 2.59384), ('apple/music.search', 2.47628), ('apple/battery.get', 2.46962), ('apple/contacts.delete', 2.43055), ('apple/maps.search', 2.0726), ('apple/alarms.list', 1.91078), ('apple/alarms.cancel', 1.89057), ('apple/alarms.create', 1.74305), ('apple/photos.search', 1.69739)]

BM25 · name + description + Schema + native aliases

[('apple/music.status', 23.06153), ('apple/music.playback', 15.04009), ('apple/maps.show', 12.51078), ('apple/clipboard.read', 4.89466), ('apple/location.reverseGeocode', 2.87756), ('apple/location.geocode', 2.60663), ('apple/calendar.events.update', 2.58742), ('apple/battery.get', 2.47355), ('apple/contacts.delete', 2.45649), ('apple/music.search', 2.45414), ('apple/maps.search', 2.05407), ('apple/alarms.list', 1.90039), ('apple/alarms.cancel', 1.87912), ('apple/alarms.create', 1.7247), ('apple/photos.search', 1.66567)]

BM25 · name + description + Schema + extended aliases

[('apple/music.status', 23.06153), ('apple/music.playback', 15.04009), ('apple/maps.show', 7.13877), ('apple/alarms.list', 4.3368), ('apple/clipboard.read', 3.61829), ('apple/location.reverseGeocode', 2.87756), ('apple/location.geocode', 2.60663), ('apple/calendar.events.update', 2.58742), ('apple/battery.get', 2.47355), ('apple/contacts.delete', 2.45649), ('apple/music.search', 2.45414), ('apple/contacts.list', 2.31158), ('apple/photos.albums.list', 2.31158), ('apple/calendar.events.list', 2.25385), ('apple/calendar.list', 2.25385), ('apple/reminders.list', 2.19893), ('apple/health.workouts.list', 2.09674), ('apple/maps.search', 2.05407), ('apple/alarms.cancel', 1.87912), ('apple/alarms.create', 1.7247), ('apple/photos.search', 1.66567)]

N077 Show what is inside this directory

dev · paraphrase · target ['builtin/ListFiles']

Directory names, not recursive content.

MethodFirst acceptable rankTop5
Keyword matching3apple/maps.show → apple/clipboard.read → builtin/ListFiles → builtin/Memory
BM25 · name＋description1builtin/ListFiles → apple/maps.show → builtin/Memory → apple/clipboard.read
BM25 · name + description + Schema + native aliases2apple/maps.show → builtin/ListFiles → builtin/Memory → apple/clipboard.read
BM25 · name + description + Schema + extended aliases1builtin/ListFiles → apple/maps.show → builtin/Memory → apple/alarms.list → apple/contacts.list
Full rankings, within-method scores

Keyword matching

[('apple/maps.show', 4), ('apple/clipboard.read', 1), ('builtin/ListFiles', 1), ('builtin/Memory', 1)]

BM25 · name＋description

[('builtin/ListFiles', 4.72208), ('apple/maps.show', 4.12697), ('builtin/Memory', 3.26957), ('apple/clipboard.read', 2.98687)]

BM25 · name + description + Schema + native aliases

[('apple/maps.show', 12.51078), ('builtin/ListFiles', 4.86698), ('builtin/Memory', 3.23148), ('apple/clipboard.read', 2.97251)]

BM25 · name + description + Schema + extended aliases

[('builtin/ListFiles', 9.38053), ('apple/maps.show', 7.13877), ('builtin/Memory', 3.23148), ('apple/alarms.list', 2.43641), ('apple/contacts.list', 2.31158), ('apple/photos.albums.list', 2.31158), ('apple/calendar.events.list', 2.25385), ('apple/calendar.list', 2.25385), ('apple/reminders.list', 2.19893), ('apple/health.workouts.list', 2.09674), ('apple/clipboard.read', 1.69614)]

N078 能把这个文件打开给我看文字吗

dev · paraphrase · target ['builtin/ReadFile']

Ordinary UTF-8 text reading.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedapple/links.open
BM25 · name＋descriptionNot retrievedapple/links.open
BM25 · name + description + Schema + native aliasesNot retrievedapple/links.open
BM25 · name + description + Schema + extended aliases1builtin/ReadFile → builtin/ListFiles → apple/maps.show → builtin/WriteFile → apple/links.open
Full rankings, within-method scores

Keyword matching

[('apple/links.open', 2.5)]

BM25 · name＋description

[('apple/links.open', 2.04264)]

BM25 · name + description + Schema + native aliases

[('apple/links.open', 3.04531)]

BM25 · name + description + Schema + extended aliases

[('builtin/ReadFile', 7.61547), ('builtin/ListFiles', 5.5907), ('apple/maps.show', 4.5438), ('builtin/WriteFile', 3.91339), ('apple/links.open', 3.12177)]

N079 Write this paragraph into a brand-new file

dev · paraphrase · target ['builtin/WriteFile']

New complete file creation.

MethodFirst acceptable rankTop5
Keyword matching1builtin/WriteFile → builtin/EditFile → apple/clipboard.write → builtin/ReadFile → builtin/ListFiles
BM25 · name＋description1builtin/WriteFile → apple/clipboard.write → builtin/Memory → builtin/ReadFile → builtin/EditFile
BM25 · name + description + Schema + native aliases1builtin/WriteFile → apple/clipboard.write → builtin/EditFile → builtin/ReadFile → builtin/Memory
BM25 · name + description + Schema + extended aliases1builtin/WriteFile → apple/clipboard.write → builtin/ReadFile → builtin/EditFile → builtin/Memory
Full rankings, within-method scores

Keyword matching

[('builtin/WriteFile', 7.5), ('builtin/EditFile', 5), ('apple/clipboard.write', 4.5), ('builtin/ReadFile', 4), ('builtin/ListFiles', 3), ('builtin/Memory', 2.5), ('apple/clipboard.read', 1), ('apple/photos.search', 1), ('builtin/ScheduledTasks', 1), ('apple/calendar.events.update', 0.5), ('apple/shortcuts.run', 0.5)]

BM25 · name＋description

[('builtin/WriteFile', 7.59405), ('apple/clipboard.write', 3.79364), ('builtin/Memory', 3.47446), ('builtin/ReadFile', 3.47241), ('builtin/EditFile', 3.37407), ('builtin/ListFiles', 3.05349), ('apple/clipboard.read', 1.93144), ('apple/photos.search', 1.47053), ('builtin/ScheduledTasks', 1.38049)]

BM25 · name + description + Schema + native aliases

[('builtin/WriteFile', 18.0842), ('apple/clipboard.write', 11.75425), ('builtin/EditFile', 10.50349), ('builtin/ReadFile', 8.54967), ('builtin/Memory', 6.41865), ('builtin/ListFiles', 5.80426), ('apple/clipboard.read', 1.80248), ('apple/calendar.events.update', 1.78238), ('apple/photos.search', 1.35321), ('builtin/ScheduledTasks', 1.21464), ('apple/shortcuts.run', 0.7661)]

BM25 · name + description + Schema + extended aliases

[('builtin/WriteFile', 26.96488), ('apple/clipboard.write', 11.75425), ('builtin/ReadFile', 11.13883), ('builtin/EditFile', 10.1053), ('builtin/Memory', 5.99839), ('builtin/ListFiles', 5.80426), ('apple/alarms.create', 2.90263), ('apple/contacts.create', 2.73695), ('apple/calendar.events.create', 2.661), ('apple/reminders.create', 2.58916), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.35321), ('builtin/ScheduledTasks', 1.21464), ('apple/calendar.events.update', 1.21197), ('apple/shortcuts.run', 0.7661)]

N080 只改这一处错别字，其余内容别动

dev · paraphrase · target ['builtin/EditFile']

One matched edit.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliases1builtin/EditFile → builtin/ReadFile
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[('builtin/EditFile', 4.97137), ('builtin/ReadFile', 4.97137)]

N081 下次聊天也要记得我喜欢早上运动

dev · paraphrase · target ['builtin/Memory']

Cross-conversation lasting preference.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedapple/health.sleep.query → apple/health.statistics → apple/health.workouts.get → apple/health.workouts.list
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrievedapple/health.sleep.query → apple/health.statistics → apple/health.workouts.get → apple/health.workouts.list
BM25 · name + description + Schema + extended aliasesNot retrievedapple/health.sleep.query → apple/health.statistics → apple/health.workouts.get → apple/health.workouts.list
Full rankings, within-method scores

Keyword matching

[('apple/health.sleep.query', 1.5), ('apple/health.statistics', 1.5), ('apple/health.workouts.get', 1.5), ('apple/health.workouts.list', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/health.sleep.query', 3.57836), ('apple/health.statistics', 3.17802), ('apple/health.workouts.get', 3.00966), ('apple/health.workouts.list', 2.85825)]

BM25 · name + description + Schema + extended aliases

[('apple/health.sleep.query', 4.74321), ('apple/health.statistics', 4.41653), ('apple/health.workouts.get', 3.20315), ('apple/health.workouts.list', 2.97987)]

N082 What do you remember about my preferences?

dev · paraphrase · target ['builtin/Memory']

Inspect stored lasting memory.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedapple/links.open → apple/health.sleep.query → apple/photos.albums.list
BM25 · name＋descriptionNot retrievedapple/health.sleep.query → apple/photos.albums.list → apple/links.open
BM25 · name + description + Schema + native aliasesNot retrievedapple/health.sleep.query → apple/photos.albums.list → apple/links.open
BM25 · name + description + Schema + extended aliases1builtin/Memory → apple/health.sleep.query → apple/photos.albums.list → apple/links.open
Full rankings, within-method scores

Keyword matching

[('apple/links.open', 1.5), ('apple/health.sleep.query', 1), ('apple/photos.albums.list', 1)]

BM25 · name＋description

[('apple/health.sleep.query', 2.98639), ('apple/photos.albums.list', 2.31356), ('apple/links.open', 1.56453)]

BM25 · name + description + Schema + native aliases

[('apple/health.sleep.query', 3.03274), ('apple/photos.albums.list', 2.3074), ('apple/links.open', 1.81003)]

BM25 · name + description + Schema + extended aliases

[('builtin/Memory', 9.43336), ('apple/health.sleep.query', 3.03274), ('apple/photos.albums.list', 2.3074), ('apple/links.open', 1.81003)]

N083 What weekday is it on this device right now?

dev · paraphrase · target ['builtin/GetCurrentTime']

Current device weekday.

MethodFirst acceptable rankTop5
Keyword matching2apple/device.get → builtin/GetCurrentTime → apple/battery.get → apple/calendar.events.create → apple/clipboard.write
BM25 · name＋description1builtin/GetCurrentTime → apple/device.get → apple/battery.get → apple/clipboard.write → apple/calendar.events.create
BM25 · name + description + Schema + native aliases2apple/device.get → builtin/GetCurrentTime → apple/battery.get → apple/clipboard.write → apple/calendar.events.create
BM25 · name + description + Schema + extended aliases1builtin/GetCurrentTime → apple/device.get → apple/battery.get → apple/clipboard.write → apple/calendar.events.create
Full rankings, within-method scores

Keyword matching

[('apple/device.get', 4), ('builtin/GetCurrentTime', 2), ('apple/battery.get', 1), ('apple/calendar.events.create', 1), ('apple/clipboard.write', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('builtin/ScheduledTasks', 1), ('apple/links.open', 0.5)]

BM25 · name＋description

[('builtin/GetCurrentTime', 7.3035), ('apple/device.get', 3.28237), ('apple/battery.get', 2.13955), ('apple/clipboard.write', 1.99616), ('apple/calendar.events.create', 1.76022), ('apple/weather.current', 1.72621), ('apple/weather.daily', 1.57416)]

BM25 · name + description + Schema + native aliases

[('apple/device.get', 8.03155), ('builtin/GetCurrentTime', 6.54068), ('apple/battery.get', 1.89098), ('apple/clipboard.write', 1.75647), ('apple/calendar.events.create', 1.55383), ('apple/weather.current', 1.50643), ('apple/weather.daily', 1.36739), ('builtin/ScheduledTasks', 0.86615), ('apple/links.open', 0.57781)]

BM25 · name + description + Schema + extended aliases

[('builtin/GetCurrentTime', 16.30937), ('apple/device.get', 8.03155), ('apple/battery.get', 1.89098), ('apple/clipboard.write', 1.75647), ('apple/calendar.events.create', 1.55383), ('apple/weather.current', 1.50643), ('apple/weather.daily', 1.36739), ('builtin/ScheduledTasks', 0.86615), ('apple/links.open', 0.57781)]

N084 给我两个选项让我点一下再继续

dev · paraphrase · target ['builtin/AskUserQuestions']

Interactive question card (no model run in experiment).

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliases1builtin/AskUserQuestions
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[('builtin/AskUserQuestions', 5.40953)]

N085 Set up a weekly read-only summary check

dev · paraphrase · target ['builtin/ScheduledTasks']

Weekly scheduled task.

MethodFirst acceptable rankTop5
Keyword matching1builtin/ScheduledTasks → apple/clipboard.read → builtin/ReadFile → apple/alarms.list → apple/health.statistics
BM25 · name＋description1builtin/ScheduledTasks → apple/reminders.complete → apple/location.reverseGeocode → apple/calendar.events.create → apple/location.geocode
BM25 · name + description + Schema + native aliases1builtin/ScheduledTasks → apple/reminders.complete → builtin/ReadFile → apple/clipboard.read → apple/location.reverseGeocode
BM25 · name + description + Schema + extended aliases1builtin/ScheduledTasks → apple/reminders.complete → builtin/ReadFile → apple/location.reverseGeocode → apple/clipboard.read
Full rankings, within-method scores

Keyword matching

[('builtin/ScheduledTasks', 5), ('apple/clipboard.read', 4), ('builtin/ReadFile', 4), ('apple/alarms.list', 2), ('apple/health.statistics', 2), ('apple/photos.get', 2), ('apple/photos.search', 2), ('apple/links.open', 1.5), ('apple/music.playback', 1.5), ('builtin/Memory', 1.5), ('apple/alarms.cancel', 1), ('apple/alarms.create', 1), ('apple/battery.get', 1), ('apple/calendar.events.create', 1), ('apple/calendar.events.list', 1), ('apple/calendar.events.update', 1), ('apple/contacts.get', 1), ('apple/device.get', 1), ('apple/health.sleep.query', 1), ('apple/health.workouts.get', 1), ('apple/health.workouts.list', 1), ('apple/location.geocode', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.eta', 1), ('apple/maps.route', 1), ('apple/maps.search', 1), ('apple/maps.show', 1), ('apple/music.search', 1), ('apple/music.status', 1), ('apple/photos.albums.list', 1), ('apple/reminders.complete', 1), ('apple/shortcuts.run', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1), ('builtin/EditFile', 1)]

BM25 · name＋description

[('builtin/ScheduledTasks', 6.1108), ('apple/reminders.complete', 4.98669), ('apple/location.reverseGeocode', 2.80551), ('apple/calendar.events.create', 2.72209), ('apple/location.geocode', 2.59384), ('apple/health.statistics', 2.37384), ('apple/alarms.list', 2.23933), ('apple/photos.search', 2.2185), ('apple/maps.route', 2.20556), ('apple/photos.get', 2.10433), ('apple/maps.search', 2.0726), ('apple/music.status', 1.8925), ('builtin/ReadFile', 1.73569), ('apple/music.search', 1.71668), ('apple/shortcuts.run', 1.7045), ('apple/photos.albums.list', 1.63489), ('builtin/Memory', 1.62085), ('apple/contacts.get', 1.59613), ('apple/links.open', 1.51184), ('builtin/EditFile', 1.36324), ('apple/clipboard.read', 1.34398), ('apple/maps.show', 1.28422), ('apple/alarms.cancel', 1.27063), ('apple/health.workouts.get', 1.26659), ('apple/weather.current', 1.24126), ('apple/health.workouts.list', 1.21491), ('apple/alarms.create', 1.17149), ('apple/calendar.events.update', 1.1275), ('apple/calendar.events.list', 1.0957), ('apple/device.get', 1.08242), ('apple/health.sleep.query', 1.08242), ('apple/battery.get', 1.06946), ('apple/maps.eta', 1.06946), ('apple/music.playback', 0.94183), ('apple/weather.hourly', 0.8465), ('apple/weather.daily', 0.78685)]

BM25 · name + description + Schema + native aliases

[('builtin/ScheduledTasks', 7.25359), ('apple/reminders.complete', 5.09374), ('builtin/ReadFile', 4.55729), ('apple/clipboard.read', 4.05468), ('apple/location.reverseGeocode', 2.87756), ('apple/location.geocode', 2.60663), ('apple/calendar.events.create', 2.4284), ('apple/health.statistics', 2.32698), ('apple/links.open', 2.31285), ('apple/maps.route', 2.19357), ('apple/alarms.list', 2.16411), ('apple/photos.search', 2.11167), ('apple/maps.search', 2.05407), ('apple/photos.get', 2.02648), ('builtin/Memory', 1.99879), ('apple/music.status', 1.79823), ('apple/music.playback', 1.62165), ('apple/music.search', 1.62155), ('apple/shortcuts.run', 1.60938), ('apple/contacts.get', 1.60914), ('apple/photos.albums.list', 1.5512), ('builtin/EditFile', 1.39066), ('apple/health.workouts.get', 1.30421), ('apple/health.workouts.list', 1.24611), ('apple/weather.current', 1.23137), ('apple/maps.show', 1.21419), ('apple/alarms.cancel', 1.2006), ('apple/calendar.events.list', 1.11381), ('apple/alarms.create', 1.10194), ('apple/health.sleep.query', 1.09922), ('apple/device.get', 1.08501), ('apple/battery.get', 1.07116), ('apple/maps.eta', 1.07116), ('apple/calendar.events.update', 1.069), ('apple/weather.hourly', 0.83632), ('apple/weather.daily', 0.77457)]

BM25 · name + description + Schema + extended aliases

[('builtin/ScheduledTasks', 11.88907), ('apple/reminders.complete', 5.09374), ('builtin/ReadFile', 4.20178), ('apple/location.reverseGeocode', 2.87756), ('apple/clipboard.read', 2.86941), ('apple/alarms.list', 2.86311), ('apple/photos.get', 2.77024), ('apple/location.geocode', 2.60663), ('apple/photos.albums.list', 2.47784), ('apple/calendar.events.create', 2.4284), ('apple/links.open', 2.31285), ('apple/maps.route', 2.19357), ('apple/contacts.get', 2.14254), ('apple/health.statistics', 2.12142), ('apple/maps.search', 2.05407), ('apple/photos.search', 1.90081), ('apple/health.workouts.get', 1.82646), ('apple/device.get', 1.80027), ('apple/music.status', 1.79823), ('apple/battery.get', 1.79046), ('apple/health.workouts.list', 1.72237), ('apple/calendar.events.list', 1.69172), ('apple/music.playback', 1.62165), ('apple/music.search', 1.62155), ('apple/shortcuts.run', 1.60938), ('builtin/Memory', 1.52782), ('apple/maps.show', 1.21419), ('apple/alarms.cancel', 1.2006), ('apple/alarms.create', 1.10194), ('apple/calendar.events.update', 1.069), ('builtin/EditFile', 0.98414), ('apple/calendar.events.get', 0.97668), ('apple/location.get', 0.97668), ('apple/reminders.get', 0.951), ('apple/contacts.list', 0.92664), ('apple/calendar.list', 0.9035), ('apple/reminders.list', 0.88148), ('apple/weather.current', 0.87142), ('apple/health.sleep.query', 0.77789), ('apple/maps.eta', 0.75804), ('apple/weather.hourly', 0.59185), ('apple/weather.daily', 0.54815)]

N086 Move this existing non-recurring meeting to a different start and end

dev · paraphrase · target ['apple/calendar.events.update']

Update event time fields.

MethodFirst acceptable rankTop5
Keyword matching1apple/calendar.events.update → apple/calendar.events.create → apple/photos.search → apple/calendar.events.delete → apple/calendar.events.get
BM25 · name＋description1apple/calendar.events.update → apple/calendar.events.get → apple/photos.search → apple/reminders.delete → apple/calendar.events.delete
BM25 · name + description + Schema + native aliases1apple/calendar.events.update → apple/calendar.events.delete → apple/calendar.events.get → apple/reminders.delete → apple/calendar.events.create
BM25 · name + description + Schema + extended aliases1apple/calendar.events.update → apple/calendar.events.delete → apple/calendar.events.get → apple/reminders.delete → apple/photos.search
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.update', 4.5), ('apple/calendar.events.create', 3), ('apple/photos.search', 3), ('apple/calendar.events.delete', 2.5), ('apple/calendar.events.get', 2.5), ('apple/health.statistics', 2), ('apple/reminders.delete', 2), ('builtin/WriteFile', 1.5), ('apple/calendar.events.list', 1), ('apple/health.sleep.query', 1), ('apple/health.workouts.list', 1), ('apple/maps.show', 1), ('apple/reminders.complete', 1), ('apple/reminders.update', 1), ('apple/clipboard.write', 0.5), ('builtin/Memory', 0.5)]

BM25 · name＋description

[('apple/calendar.events.update', 6.84996), ('apple/calendar.events.get', 6.29629), ('apple/photos.search', 6.0879), ('apple/reminders.delete', 5.79386), ('apple/calendar.events.delete', 5.35439), ('builtin/WriteFile', 5.06519), ('apple/calendar.events.create', 3.96768), ('apple/reminders.complete', 2.76956), ('apple/maps.show', 2.63514), ('apple/reminders.update', 2.43055), ('apple/health.statistics', 1.97914)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.update', 8.26487), ('apple/calendar.events.delete', 7.13789), ('apple/calendar.events.get', 6.60608), ('apple/reminders.delete', 5.88134), ('apple/calendar.events.create', 5.32777), ('apple/photos.search', 5.28148), ('builtin/WriteFile', 4.80239), ('apple/reminders.complete', 2.82901), ('apple/health.statistics', 2.7391), ('apple/reminders.update', 2.45649), ('apple/maps.show', 1.67694), ('apple/health.sleep.query', 1.53537), ('apple/clipboard.write', 1.49569), ('apple/calendar.events.list', 1.44159), ('apple/health.workouts.list', 1.37845), ('builtin/Memory', 0.56895)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.update', 8.11372), ('apple/calendar.events.delete', 6.96267), ('apple/calendar.events.get', 6.43569), ('apple/reminders.delete', 5.7189), ('apple/photos.search', 5.28148), ('apple/calendar.events.create', 5.21814), ('builtin/WriteFile', 4.80239), ('apple/health.statistics', 2.7391), ('apple/reminders.complete', 2.65289), ('builtin/ScheduledTasks', 2.58916), ('apple/reminders.update', 2.30356), ('apple/maps.show', 1.67694), ('apple/health.sleep.query', 1.53537), ('apple/clipboard.write', 1.49569), ('apple/calendar.events.list', 1.44159), ('apple/health.workouts.list', 1.37845), ('builtin/Memory', 0.56895)]

N087 Rename an existing non-recurring to-do item

dev · paraphrase · target ['apple/reminders.update']

Update reminder title.

MethodFirst acceptable rankTop5
Keyword matching15apple/calendar.events.delete → apple/calendar.events.update → apple/calendar.events.create → apple/clipboard.write → apple/reminders.delete
BM25 · name＋description13apple/reminders.delete → apple/calendar.events.delete → builtin/WriteFile → apple/calendar.events.update → apple/calendar.events.create
BM25 · name + description + Schema + native aliases11apple/calendar.events.delete → apple/clipboard.write → apple/reminders.delete → apple/calendar.events.update → builtin/WriteFile
BM25 · name + description + Schema + extended aliases5apple/calendar.events.update → apple/calendar.events.delete → apple/clipboard.write → apple/reminders.delete → apple/reminders.update
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.delete', 2.5), ('apple/calendar.events.update', 2.5), ('apple/calendar.events.create', 2), ('apple/clipboard.write', 2), ('apple/reminders.delete', 2), ('apple/links.open', 1.5), ('builtin/WriteFile', 1.5), ('apple/calendar.events.get', 1), ('apple/contacts.list', 1), ('apple/health.sleep.query', 1), ('apple/location.geocode', 1), ('apple/photos.albums.list', 1), ('apple/reminders.complete', 1), ('apple/reminders.list', 1), ('apple/reminders.update', 1)]

BM25 · name＋description

[('apple/reminders.delete', 5.79386), ('apple/calendar.events.delete', 5.35439), ('builtin/WriteFile', 5.06519), ('apple/calendar.events.update', 4.5364), ('apple/calendar.events.create', 3.96768), ('apple/clipboard.write', 3.9472), ('apple/health.sleep.query', 2.98639), ('apple/location.geocode', 2.81814), ('apple/reminders.complete', 2.76956), ('apple/calendar.events.get', 2.64651), ('apple/contacts.list', 2.44865), ('apple/reminders.list', 2.44865), ('apple/reminders.update', 2.43055), ('apple/photos.albums.list', 2.31356), ('apple/links.open', 1.56453)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.delete', 7.13789), ('apple/clipboard.write', 6.9327), ('apple/reminders.delete', 5.88134), ('apple/calendar.events.update', 5.41124), ('builtin/WriteFile', 4.80239), ('apple/calendar.events.create', 3.96916), ('apple/health.sleep.query', 3.03274), ('apple/location.geocode', 2.83203), ('apple/reminders.complete', 2.82901), ('apple/calendar.events.get', 2.73678), ('apple/reminders.update', 2.45649), ('apple/contacts.list', 2.43845), ('apple/reminders.list', 2.43845), ('apple/photos.albums.list', 2.3074), ('apple/links.open', 1.81003)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.update', 8.69668), ('apple/calendar.events.delete', 6.96267), ('apple/clipboard.write', 6.9327), ('apple/reminders.delete', 5.7189), ('apple/reminders.update', 5.6584), ('builtin/WriteFile', 4.80239), ('apple/calendar.events.create', 3.85953), ('apple/contacts.update', 3.52243), ('apple/health.sleep.query', 3.03274), ('apple/location.geocode', 2.83203), ('apple/reminders.complete', 2.65289), ('builtin/ScheduledTasks', 2.58916), ('apple/calendar.events.get', 2.5664), ('apple/contacts.list', 2.43845), ('apple/reminders.list', 2.43845), ('apple/photos.albums.list', 2.3074), ('apple/links.open', 1.81003)]

N088 Change the email addresses on an existing contact

dev · paraphrase · target ['apple/contacts.update']

Supplied email array replaces contact emails.

MethodFirst acceptable rankTop5
Keyword matching2apple/contacts.list → apple/contacts.update → apple/contacts.get → apple/contacts.create → apple/contacts.delete
BM25 · name＋description3apple/contacts.list → apple/contacts.get → apple/contacts.update → builtin/WriteFile → apple/contacts.create
BM25 · name + description + Schema + native aliases1apple/contacts.update → apple/contacts.list → apple/contacts.get → apple/contacts.create → apple/contacts.delete
BM25 · name + description + Schema + extended aliases1apple/contacts.update → apple/contacts.list → apple/contacts.get → apple/contacts.create → apple/contacts.delete
Full rankings, within-method scores

Keyword matching

[('apple/contacts.list', 7), ('apple/contacts.update', 6.5), ('apple/contacts.get', 5.5), ('apple/contacts.create', 5), ('apple/contacts.delete', 4.5), ('apple/calendar.events.update', 1.5), ('apple/location.geocode', 1.5), ('builtin/ScheduledTasks', 1.5), ('builtin/WriteFile', 1.5), ('apple/calendar.events.list', 1), ('apple/music.playback', 1), ('apple/photos.albums.list', 1), ('apple/reminders.complete', 1), ('apple/reminders.list', 1), ('apple/reminders.update', 1), ('apple/calendar.events.delete', 0.5), ('apple/clipboard.write', 0.5), ('apple/maps.search', 0.5)]

BM25 · name＋description

[('apple/contacts.list', 11.02606), ('apple/contacts.get', 7.50799), ('apple/contacts.update', 5.98121), ('builtin/WriteFile', 5.06519), ('apple/contacts.create', 3.90839), ('apple/contacts.delete', 3.62112), ('apple/location.geocode', 3.09904), ('apple/reminders.complete', 2.44391), ('apple/calendar.events.update', 2.14653), ('apple/reminders.update', 2.14476), ('apple/calendar.events.list', 1.9343), ('apple/reminders.list', 1.72296), ('apple/photos.albums.list', 1.48035), ('apple/music.playback', 1.23657), ('builtin/ScheduledTasks', 0.88332)]

BM25 · name + description + Schema + native aliases

[('apple/contacts.update', 16.20218), ('apple/contacts.list', 16.00384), ('apple/contacts.get', 14.6744), ('apple/contacts.create', 13.64048), ('apple/contacts.delete', 11.14669), ('builtin/WriteFile', 4.80239), ('apple/location.geocode', 4.07889), ('apple/calendar.events.update', 3.02752), ('apple/reminders.complete', 2.49638), ('apple/reminders.update', 2.16765), ('apple/calendar.events.list', 1.96627), ('apple/calendar.events.delete', 1.73506), ('builtin/ScheduledTasks', 1.71992), ('apple/reminders.list', 1.71579), ('apple/clipboard.write', 1.49569), ('apple/photos.albums.list', 1.47641), ('apple/music.playback', 1.21093), ('apple/maps.search', 1.07906)]

BM25 · name + description + Schema + extended aliases

[('apple/contacts.update', 18.21331), ('apple/contacts.list', 15.81906), ('apple/contacts.get', 14.6744), ('apple/contacts.create', 13.64048), ('apple/contacts.delete', 11.14669), ('builtin/WriteFile', 4.80239), ('apple/calendar.events.update', 4.75891), ('apple/location.geocode', 4.07889), ('apple/reminders.update', 3.84966), ('apple/reminders.complete', 2.22753), ('builtin/EditFile', 2.17402), ('apple/calendar.events.list', 1.75452), ('apple/calendar.events.delete', 1.73506), ('builtin/ScheduledTasks', 1.62788), ('apple/reminders.list', 1.53101), ('apple/clipboard.write', 1.49569), ('apple/photos.albums.list', 1.31741), ('apple/music.playback', 1.08052), ('apple/maps.search', 1.07906)]

N089 What percentage of battery is left?

dev · paraphrase · target ['apple/battery.get']

Snapshot levelPercent, null when unavailable.

MethodFirst acceptable rankTop5
Keyword matching1apple/battery.get → builtin/ScheduledTasks
BM25 · name＋description1apple/battery.get
BM25 · name + description + Schema + native aliases1apple/battery.get → builtin/ScheduledTasks
BM25 · name + description + Schema + extended aliases1apple/battery.get → builtin/ScheduledTasks
Full rankings, within-method scores

Keyword matching

[('apple/battery.get', 4), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('apple/battery.get', 5.87406)]

BM25 · name + description + Schema + native aliases

[('apple/battery.get', 14.03667), ('builtin/ScheduledTasks', 0.76414)]

BM25 · name + description + Schema + extended aliases

[('apple/battery.get', 14.03667), ('builtin/ScheduledTasks', 0.76414)]

N090 Find images created between two explicit dates

dev · paraphrase · target ['apple/photos.search']

Photo metadata search by creation time range.

MethodFirst acceptable rankTop5
Keyword matching1apple/photos.search → apple/calendar.events.update → apple/maps.eta → apple/maps.route → apple/weather.daily
BM25 · name＋description1apple/photos.search → apple/maps.eta → apple/maps.route → apple/photos.get → apple/weather.daily
BM25 · name + description + Schema + native aliases1apple/photos.search → apple/maps.eta → apple/maps.route → apple/photos.get → apple/weather.daily
BM25 · name + description + Schema + extended aliases1apple/photos.search → apple/photos.get → builtin/GetCurrentTime → apple/maps.eta → apple/maps.route
Full rankings, within-method scores

Keyword matching

[('apple/photos.search', 3), ('apple/calendar.events.update', 2), ('apple/maps.eta', 2), ('apple/maps.route', 2), ('apple/weather.daily', 2), ('apple/maps.search', 1.5), ('apple/reminders.create', 1.5), ('apple/alarms.create', 1), ('apple/calendar.events.create', 1), ('apple/clipboard.read', 1), ('apple/location.geocode', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.show', 1), ('apple/photos.albums.list', 1), ('apple/photos.get', 1), ('apple/reminders.list', 1), ('apple/weather.current', 1), ('apple/weather.hourly', 1), ('builtin/GetCurrentTime', 1), ('apple/reminders.update', 0.5)]

BM25 · name＋description

[('apple/photos.search', 5.9225), ('apple/maps.eta', 4.99337), ('apple/maps.route', 4.45946), ('apple/photos.get', 3.72832), ('apple/weather.daily', 3.72668), ('apple/calendar.events.update', 3.11793), ('builtin/GetCurrentTime', 2.79352), ('apple/reminders.create', 2.50148), ('apple/clipboard.read', 2.42218), ('apple/photos.albums.list', 2.10385), ('apple/reminders.list', 2.09152), ('apple/location.reverseGeocode', 1.91378), ('apple/location.geocode', 1.76939), ('apple/maps.show', 1.50453), ('apple/maps.search', 1.41383), ('apple/calendar.events.create', 1.38598), ('apple/weather.current', 1.3592), ('apple/weather.hourly', 1.33344)]

BM25 · name + description + Schema + native aliases

[('apple/photos.search', 7.39895), ('apple/maps.eta', 4.91243), ('apple/maps.route', 4.35639), ('apple/photos.get', 3.70985), ('apple/weather.daily', 3.30808), ('apple/reminders.create', 2.99771), ('apple/calendar.events.update', 2.81699), ('builtin/GetCurrentTime', 2.52524), ('apple/clipboard.read', 2.41054), ('apple/photos.albums.list', 2.09825), ('apple/maps.search', 2.07878), ('apple/alarms.create', 1.91974), ('apple/location.reverseGeocode', 1.85954), ('apple/reminders.list', 1.82336), ('apple/location.geocode', 1.68446), ('apple/maps.show', 1.41753), ('apple/calendar.events.create', 1.31346), ('apple/weather.current', 1.2734), ('apple/weather.hourly', 1.24802), ('apple/reminders.update', 0.79102)]

BM25 · name + description + Schema + extended aliases

[('apple/photos.search', 9.10421), ('apple/photos.get', 5.27315), ('builtin/GetCurrentTime', 5.26219), ('apple/maps.eta', 4.91243), ('apple/maps.route', 4.35639), ('apple/maps.search', 3.6867), ('apple/photos.albums.list', 3.54142), ('apple/weather.daily', 3.30808), ('apple/reminders.list', 3.19619), ('apple/reminders.create', 2.99771), ('apple/calendar.events.update', 2.81699), ('apple/clipboard.read', 2.41054), ('apple/alarms.create', 1.91974), ('apple/location.reverseGeocode', 1.85954), ('apple/music.search', 1.70526), ('apple/location.geocode', 1.68446), ('apple/battery.get', 1.60792), ('apple/device.get', 1.60792), ('apple/contacts.get', 1.5633), ('apple/alarms.list', 1.5211), ('apple/calendar.events.get', 1.5211), ('apple/location.get', 1.5211), ('apple/reminders.get', 1.48111), ('apple/contacts.list', 1.44317), ('apple/maps.show', 1.41753), ('apple/calendar.events.list', 1.40712), ('apple/calendar.list', 1.40712), ('apple/health.workouts.get', 1.40712), ('apple/calendar.events.create', 1.31346), ('apple/health.workouts.list', 1.30904), ('apple/weather.current', 1.2734), ('apple/weather.hourly', 1.24802), ('apple/reminders.update', 0.79102)]

N091 Delete an ordinary workspace file permanently

dev · unsupported · target []

No ordinary DeleteFile tool; Memory delete is limited to lasting memory.

MethodFirst acceptable rankTop5
Keyword matching—builtin/EditFile → builtin/WriteFile → builtin/ReadFile → apple/calendar.events.delete → apple/contacts.delete
BM25 · name＋description—builtin/Memory → apple/reminders.delete → builtin/WriteFile → apple/contacts.delete → builtin/ReadFile
BM25 · name + description + Schema + native aliases—builtin/WriteFile → builtin/EditFile → apple/reminders.delete → apple/contacts.delete → builtin/ReadFile
BM25 · name + description + Schema + extended aliases—builtin/WriteFile → builtin/ReadFile → builtin/EditFile → apple/reminders.delete → apple/contacts.delete
Full rankings, within-method scores

Keyword matching

[('builtin/EditFile', 5), ('builtin/WriteFile', 5), ('builtin/ReadFile', 4.5), ('apple/calendar.events.delete', 4), ('apple/contacts.delete', 4), ('apple/reminders.delete', 4), ('builtin/ListFiles', 3.5), ('builtin/Memory', 3), ('builtin/ScheduledTasks', 1.5), ('apple/clipboard.read', 1), ('apple/links.open', 1), ('apple/photos.search', 1), ('apple/shortcuts.run', 0.5)]

BM25 · name＋description

[('builtin/Memory', 5.91486), ('apple/reminders.delete', 3.75288), ('builtin/WriteFile', 3.71445), ('apple/contacts.delete', 3.62112), ('builtin/ReadFile', 3.47241), ('builtin/EditFile', 3.37407), ('apple/calendar.events.delete', 3.22484), ('builtin/ListFiles', 3.05349), ('apple/links.open', 2.04264), ('apple/clipboard.read', 1.93144), ('builtin/ScheduledTasks', 1.88441), ('apple/photos.search', 1.47053)]

BM25 · name + description + Schema + native aliases

[('builtin/WriteFile', 11.51567), ('builtin/EditFile', 10.39071), ('apple/reminders.delete', 10.1562), ('apple/contacts.delete', 9.97994), ('builtin/ReadFile', 9.78891), ('apple/calendar.events.delete', 8.6982), ('builtin/Memory', 7.45688), ('builtin/ListFiles', 7.17397), ('builtin/ScheduledTasks', 2.1553), ('apple/links.open', 1.98428), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.35321), ('apple/shortcuts.run', 0.7661)]

BM25 · name + description + Schema + extended aliases

[('builtin/WriteFile', 14.17667), ('builtin/ReadFile', 12.37807), ('builtin/EditFile', 10.39071), ('apple/reminders.delete', 10.1562), ('apple/contacts.delete', 9.97994), ('apple/calendar.events.delete', 8.6982), ('builtin/Memory', 7.45688), ('builtin/ListFiles', 7.17397), ('builtin/ScheduledTasks', 2.1553), ('apple/links.open', 1.98428), ('apple/clipboard.read', 1.80248), ('apple/photos.search', 1.35321), ('apple/shortcuts.run', 0.7661)]

N092 Run a Python program with a local shell

dev · unsupported · target []

Shell is excluded when Sandbox is off.

MethodFirst acceptable rankTop5
Keyword matching—apple/shortcuts.run → apple/calendar.events.delete → apple/calendar.events.update → apple/clipboard.write → apple/device.get
BM25 · name＋description—apple/shortcuts.run → builtin/ScheduledTasks → apple/device.get → apple/maps.eta → apple/calendar.events.delete
BM25 · name + description + Schema + native aliases—apple/shortcuts.run → builtin/ScheduledTasks → apple/device.get → apple/maps.eta → apple/calendar.events.delete
BM25 · name + description + Schema + extended aliases—apple/shortcuts.run → builtin/ScheduledTasks → apple/device.get → apple/maps.eta → apple/calendar.events.delete
Full rankings, within-method scores

Keyword matching

[('apple/shortcuts.run', 3), ('apple/calendar.events.delete', 1), ('apple/calendar.events.update', 1), ('apple/clipboard.write', 1), ('apple/device.get', 1), ('apple/health.statistics', 1), ('apple/maps.eta', 1), ('apple/maps.route', 1), ('apple/weather.daily', 1), ('builtin/ScheduledTasks', 1), ('apple/reminders.create', 0.5), ('apple/reminders.update', 0.5)]

BM25 · name＋description

[('apple/shortcuts.run', 2.74917), ('builtin/ScheduledTasks', 2.52466), ('apple/device.get', 2.03066), ('apple/maps.eta', 2.00635), ('apple/calendar.events.delete', 1.91466), ('apple/clipboard.write', 1.87188), ('apple/maps.route', 1.79183), ('apple/calendar.events.update', 1.57316), ('apple/weather.daily', 1.47616), ('apple/health.statistics', 1.34576)]

BM25 · name + description + Schema + native aliases

[('apple/shortcuts.run', 9.57192), ('builtin/ScheduledTasks', 2.45748), ('apple/device.get', 1.80737), ('apple/maps.eta', 1.7843), ('apple/calendar.events.delete', 1.7185), ('apple/clipboard.write', 1.65738), ('apple/maps.route', 1.58234), ('apple/calendar.events.update', 1.39312), ('apple/weather.daily', 1.29025), ('apple/health.statistics', 1.17133), ('apple/reminders.create', 0.70236), ('apple/reminders.update', 0.70236)]

BM25 · name + description + Schema + extended aliases

[('apple/shortcuts.run', 15.64555), ('builtin/ScheduledTasks', 2.45748), ('apple/device.get', 1.80737), ('apple/maps.eta', 1.7843), ('apple/calendar.events.delete', 1.7185), ('apple/clipboard.write', 1.65738), ('apple/maps.route', 1.58234), ('apple/calendar.events.update', 1.39312), ('apple/weather.daily', 1.29025), ('apple/health.statistics', 1.17133), ('apple/reminders.create', 0.70236), ('apple/reminders.update', 0.70236)]

N093 Copy a binary file between phone and remote sandbox

dev · unsupported · target []

CopyFile and remote environment are disabled in this profile.

MethodFirst acceptable rankTop5
Keyword matching—builtin/EditFile → builtin/WriteFile → builtin/ReadFile → builtin/ListFiles → apple/contacts.update
BM25 · name＋description—builtin/WriteFile → builtin/Memory → builtin/ReadFile → builtin/EditFile → apple/maps.eta
BM25 · name + description + Schema + native aliases—builtin/WriteFile → builtin/EditFile → builtin/ReadFile → builtin/ListFiles → builtin/Memory
BM25 · name + description + Schema + extended aliases—builtin/WriteFile → builtin/ReadFile → builtin/EditFile → builtin/ListFiles → builtin/Memory
Full rankings, within-method scores

Keyword matching

[('builtin/EditFile', 5), ('builtin/WriteFile', 5), ('builtin/ReadFile', 4.5), ('builtin/ListFiles', 3.5), ('apple/contacts.update', 1.5), ('builtin/Memory', 1.5), ('apple/clipboard.read', 1), ('apple/contacts.list', 1), ('apple/maps.eta', 1), ('apple/maps.route', 1), ('apple/photos.search', 1), ('apple/contacts.create', 0.5), ('apple/shortcuts.run', 0.5)]

BM25 · name＋description

[('builtin/WriteFile', 3.71445), ('builtin/Memory', 3.47446), ('builtin/ReadFile', 3.47241), ('builtin/EditFile', 3.37407), ('apple/maps.eta', 3.30871), ('builtin/ListFiles', 3.05349), ('apple/contacts.list', 3.01951), ('apple/maps.route', 2.95493), ('apple/contacts.update', 2.92367), ('apple/clipboard.read', 1.93144), ('apple/photos.search', 1.47053)]

BM25 · name + description + Schema + native aliases

[('builtin/WriteFile', 11.27381), ('builtin/EditFile', 10.16147), ('builtin/ReadFile', 9.53783), ('builtin/ListFiles', 6.89646), ('builtin/Memory', 4.34457), ('apple/contacts.update', 3.41372), ('apple/maps.eta', 3.31397), ('apple/maps.route', 2.93886), ('apple/contacts.list', 1.94441), ('apple/clipboard.read', 1.80248), ('apple/contacts.create', 1.6142), ('apple/photos.search', 1.35321), ('apple/shortcuts.run', 0.7661)]

BM25 · name + description + Schema + extended aliases

[('builtin/WriteFile', 13.93481), ('builtin/ReadFile', 12.12699), ('builtin/EditFile', 10.16147), ('builtin/ListFiles', 6.89646), ('builtin/Memory', 4.34457), ('apple/contacts.update', 3.41372), ('apple/maps.eta', 3.31397), ('apple/maps.route', 2.93886), ('apple/contacts.list', 1.94441), ('apple/clipboard.read', 1.80248), ('apple/contacts.create', 1.6142), ('apple/photos.search', 1.35321), ('apple/shortcuts.run', 0.7661)]

N094 Write a new heart-rate sample into HealthKit

dev · unsupported · target []

Health declarations are read-only.

MethodFirst acceptable rankTop5
Keyword matching—apple/health.statistics → apple/clipboard.write → apple/health.workouts.get → apple/health.sleep.query → apple/health.workouts.list
BM25 · name＋description—apple/health.statistics → builtin/WriteFile → apple/clipboard.write → apple/location.get → apple/health.workouts.get
BM25 · name + description + Schema + native aliases—apple/health.statistics → apple/clipboard.write → apple/health.workouts.get → builtin/WriteFile → apple/health.sleep.query
BM25 · name + description + Schema + extended aliases—apple/health.statistics → builtin/WriteFile → apple/clipboard.write → apple/health.workouts.get → apple/health.sleep.query
Full rankings, within-method scores

Keyword matching

[('apple/health.statistics', 9), ('apple/clipboard.write', 4.5), ('apple/health.workouts.get', 3.5), ('apple/health.sleep.query', 3), ('apple/health.workouts.list', 3), ('builtin/WriteFile', 3), ('builtin/ScheduledTasks', 2.5), ('apple/alarms.create', 1), ('apple/alarms.list', 1), ('apple/location.get', 1), ('apple/maps.search', 1), ('apple/photos.albums.list', 1), ('apple/photos.get', 1), ('apple/weather.current', 1), ('builtin/Memory', 1), ('apple/calendar.events.update', 0.5), ('builtin/EditFile', 0.5)]

BM25 · name＋description

[('apple/health.statistics', 16.63098), ('builtin/WriteFile', 3.8796), ('apple/clipboard.write', 3.79364), ('apple/location.get', 3.3488), ('apple/health.workouts.get', 2.92485), ('apple/health.workouts.list', 2.80551), ('apple/alarms.list', 2.66867), ('builtin/ScheduledTasks', 2.53593), ('apple/health.sleep.query', 2.49955), ('apple/maps.search', 1.79559), ('apple/photos.get', 1.79559), ('apple/alarms.create', 1.74305), ('apple/weather.current', 1.72621), ('apple/photos.albums.list', 1.6776)]

BM25 · name + description + Schema + native aliases

[('apple/health.statistics', 27.66135), ('apple/clipboard.write', 11.75425), ('apple/health.workouts.get', 7.9566), ('builtin/WriteFile', 7.76229), ('apple/health.sleep.query', 6.31587), ('apple/health.workouts.list', 6.31587), ('builtin/ScheduledTasks', 3.69847), ('apple/location.get', 3.35681), ('apple/alarms.list', 2.65852), ('builtin/Memory', 2.07407), ('apple/calendar.events.update', 1.78238), ('apple/maps.search', 1.77954), ('apple/photos.get', 1.77954), ('apple/alarms.create', 1.7247), ('apple/weather.current', 1.70716), ('apple/photos.albums.list', 1.67313), ('builtin/EditFile', 1.24425)]

BM25 · name + description + Schema + extended aliases

[('apple/health.statistics', 27.66135), ('builtin/WriteFile', 13.98196), ('apple/clipboard.write', 11.75425), ('apple/health.workouts.get', 7.9566), ('apple/health.sleep.query', 6.31587), ('apple/health.workouts.list', 6.31587), ('apple/alarms.create', 4.62733), ('builtin/ScheduledTasks', 3.69847), ('apple/location.get', 3.35681), ('apple/contacts.create', 2.73695), ('apple/calendar.events.create', 2.661), ('apple/alarms.list', 2.65852), ('apple/reminders.create', 2.58916), ('apple/maps.search', 1.77954), ('apple/photos.get', 1.77954), ('apple/weather.current', 1.70716), ('apple/photos.albums.list', 1.67313), ('builtin/Memory', 1.65382), ('apple/calendar.events.update', 1.21197), ('builtin/EditFile', 0.84605)]

N095 Create a recurring calendar meeting every Monday

dev · unsupported · target []

Calendar create explicitly supports non-recurring events only.

MethodFirst acceptable rankTop5
Keyword matching—apple/calendar.events.create → apple/calendar.events.get → apple/calendar.events.update → apple/calendar.events.list → apple/alarms.create
BM25 · name＋description—apple/calendar.events.create → apple/calendar.events.get → apple/calendar.events.update → apple/calendar.events.delete → builtin/ScheduledTasks
BM25 · name + description + Schema + native aliases—apple/calendar.events.create → apple/calendar.events.get → apple/reminders.create → apple/contacts.create → apple/calendar.events.update
BM25 · name + description + Schema + extended aliases—apple/calendar.events.create → apple/calendar.events.get → apple/reminders.create → apple/contacts.create → apple/calendar.events.list
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.create', 9.5), ('apple/calendar.events.get', 5), ('apple/calendar.events.update', 5), ('apple/calendar.events.list', 4.5), ('apple/alarms.create', 4), ('apple/calendar.events.delete', 4), ('apple/calendar.list', 4), ('apple/contacts.create', 4), ('apple/reminders.create', 4), ('builtin/ScheduledTasks', 3.5), ('apple/links.open', 2), ('builtin/Memory', 1.5), ('apple/health.statistics', 1), ('apple/photos.get', 1), ('apple/reminders.complete', 1), ('apple/reminders.delete', 1), ('apple/reminders.update', 1), ('apple/weather.daily', 1), ('builtin/WriteFile', 1)]

BM25 · name＋description

[('apple/calendar.events.create', 7.71003), ('apple/calendar.events.get', 5.7519), ('apple/calendar.events.update', 4.71366), ('apple/calendar.events.delete', 4.70849), ('builtin/ScheduledTasks', 3.61532), ('apple/calendar.list', 3.35399), ('apple/calendar.events.list', 3.30636), ('apple/reminders.create', 3.05666), ('apple/contacts.create', 3.03074), ('builtin/WriteFile', 2.81316), ('apple/photos.get', 2.77679), ('apple/reminders.complete', 2.76956), ('apple/reminders.delete', 2.57038), ('apple/alarms.create', 2.50063), ('apple/reminders.update', 2.43055), ('builtin/Memory', 2.11424), ('apple/weather.daily', 1.47616), ('apple/health.statistics', 1.34576)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.create', 17.86935), ('apple/calendar.events.get', 9.57151), ('apple/reminders.create', 8.5428), ('apple/contacts.create', 8.51028), ('apple/calendar.events.update', 8.37695), ('apple/calendar.events.list', 8.2996), ('apple/alarms.create', 7.9143), ('apple/calendar.list', 7.72596), ('apple/calendar.events.delete', 7.37048), ('builtin/ScheduledTasks', 5.15047), ('builtin/Memory', 3.04182), ('builtin/WriteFile', 2.8775), ('apple/reminders.complete', 2.82901), ('apple/photos.get', 2.75197), ('apple/reminders.delete', 2.60919), ('apple/reminders.update', 2.45649), ('apple/weather.daily', 1.29025), ('apple/health.statistics', 1.17133), ('apple/links.open', 0.66587)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.create', 17.75972), ('apple/calendar.events.get', 9.40113), ('apple/reminders.create', 8.5428), ('apple/contacts.create', 8.51028), ('apple/calendar.events.list', 8.2996), ('apple/calendar.events.update', 8.2258), ('apple/alarms.create', 7.9143), ('builtin/ScheduledTasks', 7.73963), ('apple/calendar.list', 7.72596), ('apple/calendar.events.delete', 7.19526), ('builtin/WriteFile', 5.71516), ('builtin/Memory', 3.04182), ('apple/photos.get', 2.75197), ('apple/reminders.complete', 2.65289), ('apple/reminders.delete', 2.44675), ('apple/reminders.update', 2.30356), ('apple/weather.daily', 1.29025), ('apple/health.statistics', 1.17133), ('apple/links.open', 1.1424)]

N096 Read the full content of my Apple Notes documents

dev · unsupported · target []

links.open handoff cannot read Notes content; no Notes read tool.

MethodFirst acceptable rankTop5
Keyword matching—apple/links.open → apple/clipboard.read → apple/alarms.list → apple/contacts.get → apple/music.playback
BM25 · name＋description—apple/contacts.get → apple/shortcuts.run → apple/links.open → apple/music.playback → apple/weather.current
BM25 · name + description + Schema + native aliases—apple/links.open → apple/contacts.get → apple/clipboard.read → builtin/ReadFile → apple/clipboard.write
BM25 · name + description + Schema + extended aliases—builtin/ReadFile → builtin/WriteFile → apple/links.open → apple/contacts.get → builtin/EditFile
Full rankings, within-method scores

Keyword matching

[('apple/links.open', 10), ('apple/clipboard.read', 7), ('apple/alarms.list', 5), ('apple/contacts.get', 5), ('apple/music.playback', 5), ('apple/shortcuts.run', 5), ('apple/weather.current', 5), ('apple/weather.daily', 5), ('apple/weather.hourly', 5), ('apple/calendar.events.update', 4.5), ('apple/clipboard.write', 4.5), ('apple/reminders.update', 4.5), ('apple/alarms.cancel', 4), ('apple/alarms.create', 4), ('apple/battery.get', 4), ('apple/calendar.events.list', 4), ('apple/device.get', 4), ('apple/health.sleep.query', 4), ('apple/health.statistics', 4), ('apple/health.workouts.get', 4), ('apple/health.workouts.list', 4), ('apple/maps.eta', 4), ('apple/maps.route', 4), ('apple/maps.search', 4), ('apple/maps.show', 4), ('apple/music.search', 4), ('apple/music.status', 4), ('apple/photos.get', 4), ('apple/photos.search', 4), ('apple/reminders.list', 4), ('builtin/ReadFile', 4), ('apple/calendar.events.create', 3.5), ('apple/reminders.create', 3.5), ('apple/calendar.events.delete', 3), ('apple/calendar.events.get', 3), ('apple/calendar.list', 3), ('apple/contacts.create', 3), ('apple/contacts.delete', 3), ('apple/contacts.list', 3), ('apple/contacts.update', 3), ('apple/location.geocode', 3), ('apple/location.get', 3), ('apple/location.reverseGeocode', 3), ('apple/photos.albums.list', 3), ('apple/reminders.complete', 3), ('apple/reminders.delete', 3), ('apple/reminders.get', 3), ('apple/reminders.lists', 3), ('builtin/ScheduledTasks', 2.5), ('builtin/Memory', 2), ('builtin/EditFile', 1), ('builtin/WriteFile', 0.5)]

BM25 · name＋description

[('apple/contacts.get', 4.72947), ('apple/shortcuts.run', 3.45174), ('apple/links.open', 3.44228), ('apple/music.playback', 3.27051), ('apple/weather.current', 3.19655), ('apple/clipboard.write', 3.08696), ('apple/reminders.update', 3.04811), ('apple/alarms.list', 2.45964), ('apple/reminders.list', 2.44865), ('apple/music.status', 2.384), ('builtin/ScheduledTasks', 2.18879), ('apple/weather.hourly', 2.17994), ('apple/calendar.events.update', 2.10385), ('apple/maps.show', 2.10128), ('apple/weather.daily', 2.02632), ('apple/alarms.create', 2.01195), ('builtin/ReadFile', 1.73569), ('builtin/Memory', 1.62085), ('apple/maps.route', 1.50453), ('apple/alarms.cancel', 1.48861), ('apple/maps.search', 1.41383), ('apple/music.search', 1.41383), ('builtin/EditFile', 1.36324), ('apple/clipboard.read', 1.34398), ('apple/health.workouts.get', 1.26659), ('apple/health.workouts.list', 1.21491), ('apple/calendar.events.list', 1.0957), ('apple/device.get', 1.08242), ('apple/health.sleep.query', 1.08242), ('apple/battery.get', 1.06946), ('apple/maps.eta', 1.06946), ('apple/photos.get', 0.89753), ('apple/photos.search', 0.73505), ('apple/health.statistics', 0.71734)]

BM25 · name + description + Schema + native aliases

[('apple/links.open', 6.89973), ('apple/contacts.get', 4.64651), ('apple/clipboard.read', 4.5615), ('builtin/ReadFile', 4.55729), ('apple/clipboard.write', 4.49879), ('builtin/Memory', 3.89773), ('apple/reminders.update', 3.80685), ('apple/calendar.events.update', 3.19126), ('apple/music.status', 2.88915), ('apple/shortcuts.run', 2.68354), ('apple/alarms.create', 2.49759), ('apple/reminders.list', 2.45123), ('apple/music.playback', 2.34015), ('builtin/ScheduledTasks', 2.26702), ('apple/weather.current', 1.93992), ('apple/health.workouts.get', 1.75225), ('apple/health.workouts.list', 1.69416), ('builtin/WriteFile', 1.66909), ('apple/alarms.list', 1.61235), ('apple/device.get', 1.59183), ('apple/battery.get', 1.57798), ('apple/maps.eta', 1.57798), ('apple/calendar.events.create', 1.56417), ('apple/calendar.events.list', 1.56185), ('apple/health.sleep.query', 1.54726), ('apple/weather.hourly', 1.48015), ('apple/weather.daily', 1.40828), ('apple/photos.get', 1.39633), ('builtin/EditFile', 1.39066), ('apple/reminders.create', 1.35036), ('apple/photos.search', 1.22813), ('apple/health.statistics', 1.21), ('apple/maps.show', 0.72451), ('apple/maps.route', 0.66243), ('apple/alarms.cancel', 0.66069), ('apple/maps.search', 0.65254), ('apple/music.search', 0.65254), ('apple/calendar.list', 0.50682), ('apple/contacts.create', 0.50682), ('apple/contacts.delete', 0.50682), ('apple/contacts.list', 0.50682), ('apple/contacts.update', 0.50682), ('apple/location.geocode', 0.50682), ('apple/location.get', 0.50682), ('apple/reminders.complete', 0.50682), ('apple/reminders.delete', 0.50682), ('apple/reminders.get', 0.50682), ('apple/reminders.lists', 0.50682), ('apple/calendar.events.delete', 0.44804), ('apple/calendar.events.get', 0.44804), ('apple/location.reverseGeocode', 0.44804), ('apple/photos.albums.list', 0.44804)]

BM25 · name + description + Schema + extended aliases

[('builtin/ReadFile', 12.5104), ('builtin/WriteFile', 8.72507), ('apple/links.open', 8.50181), ('apple/contacts.get', 5.1799), ('builtin/EditFile', 4.79187), ('apple/clipboard.write', 4.18106), ('apple/reminders.update', 3.80685), ('apple/clipboard.read', 3.37623), ('apple/reminders.list', 3.33272), ('builtin/Memory', 3.33171), ('apple/calendar.events.update', 3.19126), ('apple/music.status', 2.88915), ('apple/shortcuts.run', 2.68354), ('apple/alarms.create', 2.49759), ('apple/alarms.list', 2.31135), ('apple/device.get', 2.30709), ('apple/battery.get', 2.29729), ('apple/health.workouts.get', 2.2745), ('apple/music.playback', 2.20317), ('apple/health.workouts.list', 2.17041), ('apple/photos.get', 2.14009), ('apple/calendar.events.list', 2.13976), ('builtin/ScheduledTasks', 1.93113), ('apple/weather.current', 1.57996), ('apple/calendar.events.create', 1.56417), ('apple/location.get', 1.4835), ('apple/reminders.get', 1.45783), ('apple/contacts.list', 1.43347), ('apple/calendar.events.get', 1.42472), ('apple/calendar.list', 1.41032), ('apple/photos.albums.list', 1.37469), ('apple/reminders.create', 1.35036), ('apple/maps.eta', 1.26486), ('apple/weather.hourly', 1.23567), ('apple/health.sleep.query', 1.22594), ('apple/weather.daily', 1.18186), ('apple/photos.search', 1.01728), ('apple/health.statistics', 1.00445), ('apple/maps.show', 0.72451), ('apple/maps.route', 0.66243), ('apple/alarms.cancel', 0.66069), ('apple/maps.search', 0.65254), ('apple/music.search', 0.65254), ('apple/contacts.create', 0.50682), ('apple/contacts.delete', 0.50682), ('apple/contacts.update', 0.50682), ('apple/location.geocode', 0.50682), ('apple/reminders.complete', 0.50682), ('apple/reminders.delete', 0.50682), ('apple/reminders.lists', 0.50682), ('apple/calendar.events.delete', 0.44804), ('apple/location.reverseGeocode', 0.44804)]

N097 Send an SMS message to a phone number

dev · unsupported · target []

No SMS-send capability; opening an app does not send a message.

MethodFirst acceptable rankTop5
Keyword matching—apple/contacts.update → apple/contacts.create → apple/contacts.list → apple/links.open → apple/shortcuts.run
BM25 · name＋description—apple/contacts.list → apple/contacts.update
BM25 · name + description + Schema + native aliases—apple/contacts.update → apple/contacts.create → apple/contacts.list → apple/shortcuts.run → builtin/ListFiles
BM25 · name + description + Schema + extended aliases—apple/contacts.update → apple/contacts.create → apple/contacts.list → apple/shortcuts.run → builtin/ListFiles
Full rankings, within-method scores

Keyword matching

[('apple/contacts.update', 2), ('apple/contacts.create', 1), ('apple/contacts.list', 1), ('apple/links.open', 0.5), ('apple/shortcuts.run', 0.5), ('builtin/EditFile', 0.5), ('builtin/ListFiles', 0.5), ('builtin/ReadFile', 0.5), ('builtin/WriteFile', 0.5)]

BM25 · name＋description

[('apple/contacts.list', 3.01951), ('apple/contacts.update', 2.92367)]

BM25 · name + description + Schema + native aliases

[('apple/contacts.update', 5.24918), ('apple/contacts.create', 3.57547), ('apple/contacts.list', 1.94441), ('apple/shortcuts.run', 1.47097), ('builtin/ListFiles', 1.0922), ('builtin/ReadFile', 0.98816), ('builtin/WriteFile', 0.95189), ('builtin/EditFile', 0.90222), ('apple/links.open', 0.2902)]

BM25 · name + description + Schema + extended aliases

[('apple/contacts.update', 5.24918), ('apple/contacts.create', 3.57547), ('apple/contacts.list', 1.94441), ('apple/shortcuts.run', 1.47097), ('builtin/ListFiles', 1.0922), ('builtin/ReadFile', 0.98816), ('builtin/WriteFile', 0.95189), ('builtin/EditFile', 0.90222), ('apple/links.open', 0.2902)]

N098 Transcribe an audio recording into text

dev · unsupported · target []

No transcription tool in frozen catalog.

MethodFirst acceptable rankTop5
Keyword matching—apple/clipboard.write → apple/shortcuts.run → builtin/EditFile → apple/clipboard.read → apple/contacts.update
BM25 · name＋description—apple/clipboard.write → apple/clipboard.read → builtin/ReadFile → builtin/EditFile → builtin/AskUserQuestions
BM25 · name + description + Schema + native aliases—apple/clipboard.write → builtin/EditFile → apple/shortcuts.run → apple/clipboard.read → builtin/ReadFile
BM25 · name + description + Schema + extended aliases—builtin/EditFile → builtin/ReadFile → apple/clipboard.write → apple/shortcuts.run → apple/clipboard.read
Full rankings, within-method scores

Keyword matching

[('apple/clipboard.write', 1.5), ('apple/shortcuts.run', 1.5), ('builtin/EditFile', 1.5), ('apple/clipboard.read', 1), ('apple/contacts.update', 1), ('builtin/AskUserQuestions', 1), ('builtin/ReadFile', 1), ('builtin/Memory', 0.5)]

BM25 · name＋description

[('apple/clipboard.write', 3.14747), ('apple/clipboard.read', 3.093), ('builtin/ReadFile', 2.85815), ('builtin/EditFile', 2.72729), ('builtin/AskUserQuestions', 2.16548), ('apple/contacts.update', 1.89057), ('apple/shortcuts.run', 1.77773)]

BM25 · name + description + Schema + native aliases

[('apple/clipboard.write', 4.5613), ('builtin/EditFile', 4.13067), ('apple/shortcuts.run', 2.973), ('apple/clipboard.read', 2.89298), ('builtin/ReadFile', 2.74541), ('builtin/AskUserQuestions', 2.06218), ('apple/contacts.update', 1.76213), ('builtin/Memory', 1.35633)]

BM25 · name + description + Schema + extended aliases

[('builtin/EditFile', 6.32338), ('builtin/ReadFile', 5.01984), ('apple/clipboard.write', 4.2922), ('apple/shortcuts.run', 2.7976), ('apple/clipboard.read', 2.7223), ('builtin/WriteFile', 2.50401), ('builtin/AskUserQuestions', 1.94052), ('apple/contacts.update', 1.65817), ('builtin/Memory', 1.27631)]

N099 Tap arbitrary buttons in another app by screenshot coordinates

dev · unsupported · target []

Optional phone/screen UI tools are excluded.

MethodFirst acceptable rankTop5
Keyword matching—apple/links.open → apple/alarms.create → apple/location.reverseGeocode → apple/maps.eta → apple/maps.show
BM25 · name＋description—apple/links.open → apple/location.reverseGeocode → apple/music.status → apple/maps.eta → apple/shortcuts.run
BM25 · name + description + Schema + native aliases—apple/links.open → apple/location.reverseGeocode → apple/music.status → apple/maps.eta → apple/shortcuts.run
BM25 · name + description + Schema + extended aliases—apple/links.open → apple/location.reverseGeocode → apple/music.status → apple/maps.eta → apple/shortcuts.run
Full rankings, within-method scores

Keyword matching

[('apple/links.open', 5.5), ('apple/alarms.create', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.eta', 1), ('apple/maps.show', 1), ('apple/music.status', 1), ('apple/shortcuts.run', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1), ('builtin/ScheduledTasks', 1)]

BM25 · name＋description

[('apple/links.open', 6.61938), ('apple/location.reverseGeocode', 2.43055), ('apple/music.status', 2.384), ('apple/maps.eta', 2.13955), ('apple/shortcuts.run', 2.05198), ('apple/alarms.create', 2.01195), ('apple/maps.show', 1.91078), ('apple/weather.current', 1.72621), ('apple/weather.hourly', 1.6935), ('apple/weather.daily', 1.57416), ('builtin/ScheduledTasks', 1.15544)]

BM25 · name + description + Schema + native aliases

[('apple/links.open', 8.29973), ('apple/location.reverseGeocode', 2.49296), ('apple/music.status', 2.38233), ('apple/maps.eta', 2.14295), ('apple/shortcuts.run', 2.03253), ('apple/alarms.create', 1.99077), ('apple/maps.show', 1.90039), ('apple/weather.current', 1.70716), ('apple/weather.hourly', 1.67313), ('apple/weather.daily', 1.5496), ('builtin/ScheduledTasks', 1.11796)]

BM25 · name + description + Schema + extended aliases

[('apple/links.open', 8.96033), ('apple/location.reverseGeocode', 2.49296), ('apple/music.status', 2.38233), ('apple/maps.eta', 2.14295), ('apple/shortcuts.run', 2.03253), ('apple/alarms.create', 1.99077), ('apple/maps.show', 1.90039), ('apple/weather.current', 1.70716), ('apple/weather.hourly', 1.67313), ('apple/weather.daily', 1.5496), ('builtin/ScheduledTasks', 1.11796)]

N100 Fetch the complete HTML content of an arbitrary website

dev · unsupported · target []

Web tools disabled; links.open only hands off URL, not content retrieval.

MethodFirst acceptable rankTop5
Keyword matching—apple/reminders.complete → apple/clipboard.write → apple/music.playback → builtin/ScheduledTasks → builtin/WriteFile
BM25 · name＋description—apple/clipboard.write → apple/reminders.complete → apple/music.playback → builtin/ScheduledTasks
BM25 · name + description + Schema + native aliases—apple/reminders.complete → apple/clipboard.write → builtin/WriteFile → apple/music.playback → builtin/ScheduledTasks
BM25 · name + description + Schema + extended aliases—apple/reminders.complete → apple/clipboard.write → builtin/WriteFile → builtin/ReadFile → apple/battery.get
Full rankings, within-method scores

Keyword matching

[('apple/reminders.complete', 3), ('apple/clipboard.write', 2.5), ('apple/music.playback', 1), ('builtin/ScheduledTasks', 1), ('builtin/WriteFile', 1), ('builtin/Memory', 0.5)]

BM25 · name＋description

[('apple/clipboard.write', 5.83984), ('apple/reminders.complete', 3.81947), ('apple/music.playback', 2.16711), ('builtin/ScheduledTasks', 1.38049)]

BM25 · name + description + Schema + native aliases

[('apple/reminders.complete', 7.76229), ('apple/clipboard.write', 6.48824), ('builtin/WriteFile', 2.86284), ('apple/music.playback', 1.72095), ('builtin/ScheduledTasks', 1.21464), ('builtin/Memory', 1.19415)]

BM25 · name + description + Schema + extended aliases

[('apple/reminders.complete', 12.17882), ('apple/clipboard.write', 6.17051), ('builtin/WriteFile', 6.00543), ('builtin/ReadFile', 4.50088), ('apple/battery.get', 2.73695), ('apple/device.get', 2.73695), ('apple/contacts.get', 2.661), ('apple/photos.get', 2.661), ('apple/calendar.events.get', 2.58916), ('apple/location.get', 2.58916), ('apple/reminders.get', 2.52109), ('apple/health.workouts.get', 2.39516), ('apple/music.playback', 1.58398), ('builtin/ScheduledTasks', 1.21464), ('builtin/Memory', 1.09911)]

H001 Which calendars are accessible, and what are their identifiers?

comparison · intent_transfer · target ['apple/calendar.list']

Enumerates accessible calendars, not events.

MethodFirst acceptable rankTop5
Keyword matching1apple/calendar.list → apple/calendar.events.create → apple/calendar.events.list → apple/calendar.events.get → apple/calendar.events.update
BM25 · name＋description1apple/calendar.list → apple/contacts.get → apple/contacts.list → builtin/AskUserQuestions → apple/calendar.events.list
BM25 · name + description + Schema + native aliases1apple/calendar.list → apple/calendar.events.list → apple/calendar.events.create → apple/calendar.events.get → apple/calendar.events.update
BM25 · name + description + Schema + extended aliases1apple/calendar.list → apple/calendar.events.list → apple/calendar.events.create → apple/calendar.events.get → apple/calendar.events.update
Full rankings, within-method scores

Keyword matching

[('apple/calendar.list', 7), ('apple/calendar.events.create', 4.5), ('apple/calendar.events.list', 4.5), ('apple/calendar.events.get', 4), ('apple/calendar.events.update', 4), ('apple/calendar.events.delete', 3), ('apple/contacts.get', 2.5), ('apple/contacts.list', 2), ('apple/links.open', 2), ('apple/contacts.delete', 1.5), ('apple/contacts.update', 1.5), ('apple/photos.search', 1.5), ('apple/device.get', 1), ('apple/health.statistics', 1), ('apple/photos.albums.list', 1), ('apple/photos.get', 1), ('apple/reminders.get', 1), ('apple/reminders.lists', 1), ('apple/weather.daily', 1), ('builtin/AskUserQuestions', 1), ('apple/location.geocode', 0.5), ('apple/location.reverseGeocode', 0.5), ('builtin/EditFile', 0.5), ('builtin/ListFiles', 0.5), ('builtin/ReadFile', 0.5), ('builtin/ScheduledTasks', 0.5), ('builtin/WriteFile', 0.5)]

BM25 · name＋description

[('apple/calendar.list', 14.2615), ('apple/contacts.get', 5.2137), ('apple/contacts.list', 4.07441), ('builtin/AskUserQuestions', 3.3488), ('apple/calendar.events.list', 3.30636), ('apple/calendar.events.get', 3.10538), ('apple/calendar.events.create', 3.00948), ('apple/device.get', 2.71569), ('apple/reminders.get', 2.54246), ('apple/reminders.lists', 2.45844), ('apple/calendar.events.update', 2.28111), ('apple/contacts.delete', 2.02377), ('apple/calendar.events.delete', 1.91466), ('apple/contacts.update', 1.57416), ('apple/photos.get', 1.49508), ('apple/weather.daily', 1.47616), ('apple/photos.albums.list', 1.39683), ('apple/health.statistics', 1.34576), ('apple/photos.search', 1.22442)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.list', 17.47662), ('apple/calendar.events.list', 8.2996), ('apple/calendar.events.create', 8.11782), ('apple/calendar.events.get', 6.83473), ('apple/calendar.events.update', 5.94909), ('apple/contacts.get', 5.44425), ('apple/calendar.events.delete', 4.55598), ('builtin/AskUserQuestions', 3.40078), ('apple/contacts.list', 3.13018), ('apple/contacts.delete', 3.02148), ('apple/contacts.update', 2.15557), ('apple/reminders.get', 2.1152), ('apple/reminders.lists', 2.03876), ('apple/device.get', 2.03552), ('apple/photos.search', 1.45749), ('apple/weather.daily', 1.29025), ('apple/photos.get', 1.19601), ('apple/health.statistics', 1.17133), ('apple/location.reverseGeocode', 1.14472), ('apple/photos.albums.list', 1.1245), ('apple/location.geocode', 1.1184), ('builtin/ScheduledTasks', 0.85307), ('builtin/ListFiles', 0.73406), ('apple/links.open', 0.66587), ('builtin/ReadFile', 0.66413), ('builtin/WriteFile', 0.63976), ('builtin/EditFile', 0.60637)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.list', 17.47662), ('apple/calendar.events.list', 8.2996), ('apple/calendar.events.create', 8.11782), ('apple/calendar.events.get', 6.83473), ('apple/calendar.events.update', 5.94909), ('apple/contacts.get', 5.44425), ('apple/calendar.events.delete', 4.55598), ('builtin/AskUserQuestions', 3.40078), ('apple/contacts.list', 3.13018), ('apple/contacts.delete', 3.02148), ('apple/contacts.update', 2.15557), ('apple/reminders.get', 2.1152), ('apple/reminders.lists', 2.03876), ('apple/device.get', 2.03552), ('apple/photos.search', 1.45749), ('apple/weather.daily', 1.29025), ('apple/photos.get', 1.19601), ('apple/health.statistics', 1.17133), ('apple/location.reverseGeocode', 1.14472), ('apple/links.open', 1.1424), ('apple/photos.albums.list', 1.1245), ('apple/location.geocode', 1.1184), ('builtin/ScheduledTasks', 0.85307), ('builtin/ListFiles', 0.73406), ('builtin/ReadFile', 0.66413), ('builtin/WriteFile', 0.63976), ('builtin/EditFile', 0.60637)]

H002 I have an eventId already; fetch that single event.

comparison · intent_transfer · target ['apple/calendar.events.get']

Get by existing event ID.

MethodFirst acceptable rankTop5
Keyword matching5apple/calendar.events.create → apple/calendar.events.delete → apple/calendar.events.update → apple/calendar.events.list → apple/calendar.events.get
BM25 · name＋description5apple/calendar.events.delete → apple/calendar.events.list → apple/calendar.events.update → apple/calendar.events.create → apple/calendar.events.get
BM25 · name + description + Schema + native aliases2apple/calendar.events.delete → apple/calendar.events.get → apple/calendar.events.update → apple/calendar.events.list → apple/calendar.events.create
BM25 · name + description + Schema + extended aliases1apple/calendar.events.get → apple/calendar.events.delete → apple/calendar.events.update → apple/calendar.events.list → apple/calendar.events.create
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.create', 6), ('apple/calendar.events.delete', 6), ('apple/calendar.events.update', 6), ('apple/calendar.events.list', 5.5), ('apple/calendar.events.get', 5), ('builtin/ScheduledTasks', 3), ('apple/alarms.cancel', 1.5), ('apple/music.playback', 1.5), ('apple/photos.get', 1.5), ('apple/photos.search', 1.5), ('apple/reminders.create', 1.5), ('apple/reminders.get', 1.5), ('apple/reminders.list', 1.5), ('apple/alarms.create', 1), ('apple/alarms.list', 1), ('apple/clipboard.read', 1), ('apple/links.open', 1), ('apple/music.search', 1), ('apple/contacts.delete', 0.5), ('apple/contacts.get', 0.5), ('apple/contacts.update', 0.5), ('apple/health.workouts.get', 0.5), ('apple/reminders.complete', 0.5), ('apple/reminders.delete', 0.5), ('apple/reminders.update', 0.5), ('builtin/AskUserQuestions', 0.5)]

BM25 · name＋description

[('apple/calendar.events.delete', 5.4779), ('apple/calendar.events.list', 5.31783), ('apple/calendar.events.update', 4.7837), ('apple/calendar.events.create', 4.45999), ('apple/calendar.events.get', 4.3717), ('apple/reminders.get', 2.69446), ('builtin/ScheduledTasks', 2.32391), ('apple/photos.get', 2.25391), ('apple/reminders.create', 2.06068), ('apple/music.playback', 1.8781), ('apple/reminders.list', 1.72296), ('apple/clipboard.read', 1.70434), ('apple/alarms.list', 1.68611), ('apple/alarms.cancel', 1.66828), ('apple/music.search', 1.58446), ('apple/alarms.create', 1.5381), ('apple/photos.search', 1.29763), ('apple/links.open', 1.00107)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.delete', 12.49262), ('apple/calendar.events.get', 12.34192), ('apple/calendar.events.update', 10.91433), ('apple/calendar.events.list', 10.59148), ('apple/calendar.events.create', 9.91689), ('builtin/ScheduledTasks', 2.71188), ('apple/reminders.get', 2.36601), ('apple/alarms.cancel', 2.35408), ('apple/photos.get', 2.01898), ('apple/photos.search', 1.79364), ('apple/music.playback', 1.7244), ('apple/reminders.create', 1.70201), ('apple/clipboard.read', 1.69614), ('apple/alarms.list', 1.67694), ('apple/music.search', 1.5703), ('apple/alarms.create', 1.52191), ('apple/reminders.list', 1.30883), ('apple/links.open', 0.97247), ('apple/reminders.delete', 0.86744), ('apple/reminders.complete', 0.85247), ('apple/contacts.delete', 0.69591), ('apple/contacts.get', 0.69591), ('apple/health.workouts.get', 0.67683), ('builtin/AskUserQuestions', 0.65878), ('apple/reminders.update', 0.58151), ('apple/contacts.update', 0.45326)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.get', 14.93107), ('apple/calendar.events.delete', 12.49262), ('apple/calendar.events.update', 10.91433), ('apple/calendar.events.list', 10.59148), ('apple/calendar.events.create', 9.91689), ('apple/reminders.get', 4.8871), ('apple/photos.get', 4.67998), ('apple/contacts.get', 3.35691), ('apple/health.workouts.get', 3.07199), ('apple/battery.get', 2.73695), ('apple/device.get', 2.73695), ('builtin/ScheduledTasks', 2.71188), ('apple/location.get', 2.58916), ('apple/alarms.cancel', 2.35408), ('apple/photos.search', 1.79364), ('apple/music.playback', 1.7244), ('apple/reminders.create', 1.70201), ('apple/clipboard.read', 1.69614), ('apple/alarms.list', 1.67694), ('apple/music.search', 1.5703), ('apple/alarms.create', 1.52191), ('apple/reminders.list', 1.30883), ('apple/links.open', 0.97247), ('apple/reminders.delete', 0.86744), ('apple/reminders.complete', 0.85247), ('apple/contacts.delete', 0.69591), ('builtin/AskUserQuestions', 0.65878), ('apple/reminders.update', 0.58151), ('apple/contacts.update', 0.45326)]

H003 Look up the recorded workout with this UUID.

comparison · intent_transfer · target ['apple/health.workouts.get']

One recorded workout by UUID.

MethodFirst acceptable rankTop5
Keyword matching1apple/health.workouts.get → apple/health.workouts.list → apple/alarms.cancel → apple/alarms.list → apple/health.statistics
BM25 · name＋description1apple/health.workouts.get → apple/health.workouts.list → apple/location.reverseGeocode → apple/alarms.list → apple/alarms.cancel
BM25 · name + description + Schema + native aliases1apple/health.workouts.get → apple/health.workouts.list → apple/alarms.cancel → apple/location.reverseGeocode → apple/alarms.list
BM25 · name + description + Schema + extended aliases1apple/health.workouts.get → apple/health.workouts.list → apple/alarms.cancel → apple/location.reverseGeocode → apple/alarms.list
Full rankings, within-method scores

Keyword matching

[('apple/health.workouts.get', 7), ('apple/health.workouts.list', 5), ('apple/alarms.cancel', 1.5), ('apple/alarms.list', 1), ('apple/health.statistics', 1), ('apple/location.geocode', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.route', 1), ('apple/maps.search', 1), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('apple/health.workouts.get', 12.39579), ('apple/health.workouts.list', 8.61017), ('apple/location.reverseGeocode', 2.80551), ('apple/alarms.list', 2.63514), ('apple/alarms.cancel', 2.60726), ('apple/location.geocode', 2.59384), ('apple/maps.route', 2.20556), ('apple/maps.search', 2.0726), ('apple/health.statistics', 1.6565)]

BM25 · name + description + Schema + native aliases

[('apple/health.workouts.get', 23.1245), ('apple/health.workouts.list', 14.83928), ('apple/alarms.cancel', 4.607), ('apple/location.reverseGeocode', 2.87756), ('apple/alarms.list', 2.62081), ('apple/location.geocode', 2.60663), ('apple/maps.route', 2.19357), ('apple/maps.search', 2.05407), ('apple/health.statistics', 1.6238), ('builtin/ScheduledTasks', 1.11398)]

BM25 · name + description + Schema + extended aliases

[('apple/health.workouts.get', 23.1245), ('apple/health.workouts.list', 14.83928), ('apple/alarms.cancel', 4.607), ('apple/location.reverseGeocode', 2.87756), ('apple/alarms.list', 2.62081), ('apple/location.geocode', 2.60663), ('apple/maps.route', 2.19357), ('apple/maps.search', 2.05407), ('apple/health.statistics', 1.6238), ('builtin/ScheduledTasks', 1.11398)]

H004 列出我可用的提醒事项清单，并告诉我默认清单。

comparison · intent_transfer · target ['apple/reminders.lists']

List reminder containers and default, not reminder items.

MethodFirst acceptable rankTop5
Keyword matching6apple/reminders.list → apple/reminders.complete → apple/reminders.create → apple/reminders.delete → apple/reminders.get
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliases3apple/reminders.list → apple/reminders.complete → apple/reminders.lists → apple/reminders.delete → apple/reminders.update
BM25 · name + description + Schema + extended aliases1apple/reminders.lists → apple/reminders.delete → apple/reminders.list → apple/reminders.complete → apple/reminders.create
Full rankings, within-method scores

Keyword matching

[('apple/reminders.list', 6), ('apple/reminders.complete', 4.5), ('apple/reminders.create', 4.5), ('apple/reminders.delete', 4.5), ('apple/reminders.get', 4.5), ('apple/reminders.lists', 4.5), ('apple/reminders.update', 4.5), ('apple/links.open', 4), ('apple/alarms.list', 1.5), ('apple/calendar.events.list', 1.5), ('apple/calendar.list', 1.5), ('apple/contacts.list', 1.5), ('apple/health.workouts.list', 1.5), ('apple/photos.albums.list', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/reminders.list', 10.35558), ('apple/reminders.complete', 9.8152), ('apple/reminders.lists', 9.8152), ('apple/reminders.delete', 8.78566), ('apple/reminders.update', 8.78566), ('apple/reminders.create', 8.34962), ('apple/reminders.get', 8.34962), ('apple/alarms.list', 3.2646), ('apple/contacts.list', 2.85336), ('apple/photos.albums.list', 2.85336), ('apple/calendar.events.list', 2.68429), ('apple/calendar.list', 2.68429), ('apple/health.workouts.list', 2.27915), ('apple/links.open', 2.12533)]

BM25 · name + description + Schema + extended aliases

[('apple/reminders.lists', 13.97068), ('apple/reminders.delete', 11.32252), ('apple/reminders.list', 9.28689), ('apple/reminders.complete', 7.83635), ('apple/reminders.create', 6.41762), ('apple/reminders.get', 6.27663), ('apple/reminders.update', 5.771), ('apple/links.open', 4.10647), ('apple/alarms.list', 3.65657), ('apple/contacts.list', 3.52227), ('apple/photos.albums.list', 3.52227), ('apple/calendar.events.list', 3.45875), ('apple/calendar.list', 3.45875), ('apple/health.workouts.list', 3.28123), ('builtin/ListFiles', 2.73695)]

H005 Find my incomplete reminders whose titles contain dentist.

comparison · intent_transfer · target ['apple/reminders.list']

Title substring search among reminders.

MethodFirst acceptable rankTop5
Keyword matching1apple/reminders.list → apple/reminders.update → apple/reminders.complete → apple/reminders.create → apple/reminders.delete
BM25 · name＋description1apple/reminders.list → apple/reminders.get → apple/reminders.lists → apple/reminders.complete → apple/reminders.delete
BM25 · name + description + Schema + native aliases1apple/reminders.list → apple/reminders.get → apple/reminders.update → apple/reminders.complete → apple/reminders.delete
BM25 · name + description + Schema + extended aliases1apple/reminders.list → apple/reminders.get → apple/reminders.update → apple/reminders.complete → apple/reminders.delete
Full rankings, within-method scores

Keyword matching

[('apple/reminders.list', 6), ('apple/reminders.update', 5), ('apple/reminders.complete', 4.5), ('apple/reminders.create', 4.5), ('apple/reminders.delete', 4.5), ('apple/reminders.get', 4.5), ('apple/reminders.lists', 4), ('apple/links.open', 2), ('builtin/ScheduledTasks', 2), ('apple/alarms.create', 1.5), ('apple/alarms.list', 1), ('apple/music.search', 1), ('apple/calendar.events.create', 0.5), ('apple/calendar.events.update', 0.5)]

BM25 · name＋description

[('apple/reminders.list', 8.88032), ('apple/reminders.get', 3.61281), ('apple/reminders.lists', 3.54122), ('apple/reminders.complete', 3.40623), ('apple/reminders.delete', 3.2513), ('apple/reminders.update', 3.13715), ('apple/reminders.create', 3.05666), ('apple/alarms.list', 2.20556), ('apple/music.search', 2.0726), ('apple/alarms.create', 2.01195), ('builtin/ScheduledTasks', 1.15544)]

BM25 · name + description + Schema + native aliases

[('apple/reminders.list', 13.06227), ('apple/reminders.get', 9.60564), ('apple/reminders.update', 9.45148), ('apple/reminders.complete', 9.28524), ('apple/reminders.delete', 9.13089), ('apple/reminders.create', 8.61599), ('apple/reminders.lists', 8.13862), ('apple/alarms.create', 2.84964), ('builtin/ScheduledTasks', 2.19476), ('apple/alarms.list', 1.67694), ('apple/music.search', 1.5703), ('apple/calendar.events.create', 0.98489), ('apple/calendar.events.update', 0.94426), ('apple/links.open', 0.70568)]

BM25 · name + description + Schema + extended aliases

[('apple/reminders.list', 14.4351), ('apple/reminders.get', 11.08674), ('apple/reminders.update', 9.45148), ('apple/reminders.complete', 9.28524), ('apple/reminders.delete', 9.13089), ('apple/reminders.create', 8.61599), ('apple/reminders.lists', 8.13862), ('apple/music.search', 3.27556), ('apple/alarms.list', 3.19804), ('apple/alarms.create', 2.84964), ('builtin/ScheduledTasks', 2.19476), ('apple/photos.search', 1.70526), ('apple/battery.get', 1.60792), ('apple/device.get', 1.60792), ('apple/maps.search', 1.60792), ('apple/contacts.get', 1.5633), ('apple/photos.get', 1.5633), ('apple/calendar.events.get', 1.5211), ('apple/location.get', 1.5211), ('apple/contacts.list', 1.44317), ('apple/photos.albums.list', 1.44317), ('apple/calendar.events.list', 1.40712), ('apple/calendar.list', 1.40712), ('apple/health.workouts.get', 1.40712), ('apple/health.workouts.list', 1.30904), ('apple/links.open', 1.2107), ('apple/calendar.events.create', 0.98489), ('apple/calendar.events.update', 0.94426)]

H006 按已知reminderId读取这一条提醒的详情。

comparison · intent_transfer · target ['apple/reminders.get']

One reminder by identifier.

MethodFirst acceptable rankTop5
Keyword matching1apple/reminders.get → apple/reminders.create → apple/reminders.list → apple/reminders.complete → apple/reminders.delete
BM25 · name＋description1apple/reminders.get → apple/reminders.create → apple/reminders.list → apple/reminders.lists → apple/reminders.complete
BM25 · name + description + Schema + native aliases1apple/reminders.get → apple/reminders.complete → apple/reminders.delete → apple/reminders.create → apple/reminders.update
BM25 · name + description + Schema + extended aliases1apple/reminders.get → apple/reminders.complete → apple/reminders.delete → apple/reminders.create → apple/reminders.lists
Full rankings, within-method scores

Keyword matching

[('apple/reminders.get', 9), ('apple/reminders.create', 7), ('apple/reminders.list', 7), ('apple/reminders.complete', 6.5), ('apple/reminders.delete', 6.5), ('apple/reminders.update', 6.5), ('apple/reminders.lists', 5.5), ('apple/links.open', 4), ('apple/photos.get', 3), ('apple/calendar.events.get', 2), ('apple/contacts.get', 2), ('apple/health.workouts.get', 2), ('builtin/ScheduledTasks', 2), ('apple/battery.get', 1.5), ('apple/calendar.events.create', 1.5), ('apple/calendar.events.delete', 1.5), ('apple/calendar.events.update', 1.5), ('apple/device.get', 1.5), ('apple/location.get', 1.5), ('apple/music.playback', 1.5), ('apple/alarms.cancel', 0.5), ('apple/calendar.events.list', 0.5), ('apple/contacts.delete', 0.5), ('apple/contacts.update', 0.5), ('apple/photos.search', 0.5), ('builtin/AskUserQuestions', 0.5)]

BM25 · name＋description

[('apple/reminders.get', 6.30728), ('apple/reminders.create', 5.11734), ('apple/reminders.list', 4.8339), ('apple/reminders.lists', 3.54122), ('apple/reminders.complete', 3.40623), ('apple/reminders.delete', 3.2513), ('apple/reminders.update', 3.13715), ('apple/photos.get', 2.25391), ('apple/music.playback', 1.8781), ('apple/calendar.events.delete', 1.8017), ('apple/calendar.events.create', 1.55325), ('apple/calendar.events.update', 1.48035), ('builtin/ScheduledTasks', 1.44059)]

BM25 · name + description + Schema + native aliases

[('apple/reminders.get', 17.78713), ('apple/reminders.complete', 14.05217), ('apple/reminders.delete', 13.58252), ('apple/reminders.create', 12.67947), ('apple/reminders.update', 12.53954), ('apple/reminders.list', 12.45965), ('apple/reminders.lists', 12.05308), ('apple/photos.get', 4.87456), ('apple/contacts.get', 3.55148), ('apple/calendar.events.get', 3.46047), ('apple/battery.get', 3.06136), ('apple/device.get', 3.06136), ('apple/health.workouts.get', 2.92731), ('apple/location.get', 2.67572), ('apple/links.open', 2.08109), ('builtin/ScheduledTasks', 1.89603), ('apple/calendar.events.delete', 1.79327), ('apple/music.playback', 1.7244), ('apple/calendar.events.create', 1.36992), ('apple/calendar.events.update', 1.30621), ('apple/alarms.cancel', 0.69591), ('apple/contacts.delete', 0.69591), ('builtin/AskUserQuestions', 0.65878), ('apple/photos.search', 0.52026), ('apple/contacts.update', 0.45326), ('apple/calendar.events.list', 0.37713)]

BM25 · name + description + Schema + extended aliases

[('apple/reminders.get', 18.93628), ('apple/reminders.complete', 14.45235), ('apple/reminders.delete', 13.95677), ('apple/reminders.create', 12.89692), ('apple/reminders.lists', 12.76551), ('apple/reminders.list', 12.55181), ('apple/reminders.update', 12.29369), ('apple/photos.get', 5.5267), ('apple/calendar.events.get', 4.2256), ('apple/contacts.get', 4.20362), ('apple/health.workouts.get', 3.93153), ('apple/battery.get', 3.57724), ('apple/device.get', 3.57724), ('apple/location.get', 3.44085), ('apple/links.open', 3.42503), ('builtin/ReadFile', 2.43641), ('builtin/ScheduledTasks', 1.89603), ('apple/calendar.events.delete', 1.79327), ('apple/music.playback', 1.7244), ('apple/calendar.events.create', 1.36992), ('apple/calendar.events.update', 1.30621), ('apple/alarms.cancel', 0.69591), ('apple/contacts.delete', 0.69591), ('builtin/AskUserQuestions', 0.65878), ('apple/photos.search', 0.52026), ('apple/contacts.update', 0.45326), ('apple/calendar.events.list', 0.37713)]

H007 Erase this existing contact from the address book.

comparison · intent_transfer · target ['apple/contacts.delete']

Deletion capability, subject to approval; not update.

MethodFirst acceptable rankTop5
Keyword matching5apple/contacts.list → apple/contacts.get → apple/contacts.update → apple/contacts.create → apple/contacts.delete
BM25 · name＋description5apple/contacts.list → apple/contacts.get → builtin/WriteFile → apple/contacts.create → apple/contacts.delete
BM25 · name + description + Schema + native aliases5apple/contacts.list → apple/contacts.get → apple/contacts.update → apple/contacts.create → apple/contacts.delete
BM25 · name + description + Schema + extended aliases1apple/contacts.delete → apple/contacts.list → apple/contacts.get → apple/contacts.update → apple/contacts.create
Full rankings, within-method scores

Keyword matching

[('apple/contacts.list', 6), ('apple/contacts.get', 5.5), ('apple/contacts.update', 5), ('apple/contacts.create', 4.5), ('apple/contacts.delete', 4.5), ('apple/location.geocode', 1.5), ('builtin/WriteFile', 1.5), ('apple/calendar.events.delete', 0.5), ('apple/calendar.events.update', 0.5), ('apple/clipboard.write', 0.5), ('apple/maps.search', 0.5), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('apple/contacts.list', 9.79922), ('apple/contacts.get', 7.50799), ('builtin/WriteFile', 5.06519), ('apple/contacts.create', 3.90839), ('apple/contacts.delete', 3.62112), ('apple/location.geocode', 3.09904), ('apple/contacts.update', 3.05754)]

BM25 · name + description + Schema + native aliases

[('apple/contacts.list', 15.10752), ('apple/contacts.get', 14.6744), ('apple/contacts.update', 11.49435), ('apple/contacts.create', 11.41436), ('apple/contacts.delete', 11.14669), ('builtin/WriteFile', 4.80239), ('apple/location.geocode', 4.07889), ('apple/calendar.events.delete', 1.73506), ('apple/clipboard.write', 1.49569), ('apple/maps.search', 1.07906), ('apple/calendar.events.update', 0.88513), ('builtin/ScheduledTasks', 0.86526)]

BM25 · name + description + Schema + extended aliases

[('apple/contacts.delete', 15.84177), ('apple/contacts.list', 15.10752), ('apple/contacts.get', 14.6744), ('apple/contacts.update', 11.49435), ('apple/contacts.create', 11.41436), ('apple/calendar.events.delete', 6.27886), ('builtin/WriteFile', 4.80239), ('apple/reminders.delete', 4.26874), ('apple/location.geocode', 4.07889), ('apple/clipboard.write', 1.49569), ('apple/maps.search', 1.07906), ('apple/calendar.events.update', 0.88513), ('builtin/ScheduledTasks', 0.86526)]

H008 获取我当前的前台位置，经纬度即可。

comparison · intent_transfer · target ['apple/location.get']

Foreground current location acquisition.

MethodFirst acceptable rankTop5
Keyword matching2apple/location.geocode → apple/location.get → apple/location.reverseGeocode
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliases3apple/location.geocode → apple/location.reverseGeocode → apple/location.get
BM25 · name + description + Schema + extended aliases1apple/location.get → apple/location.geocode → apple/location.reverseGeocode → builtin/GetCurrentTime → apple/battery.get
Full rankings, within-method scores

Keyword matching

[('apple/location.geocode', 1.5), ('apple/location.get', 1.5), ('apple/location.reverseGeocode', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/location.geocode', 4.85181), ('apple/location.reverseGeocode', 4.85181), ('apple/location.get', 3.93504)]

BM25 · name + description + Schema + extended aliases

[('apple/location.get', 6.39689), ('apple/location.geocode', 5.86696), ('apple/location.reverseGeocode', 5.86696), ('builtin/GetCurrentTime', 5.25514), ('apple/battery.get', 2.73695), ('apple/device.get', 2.73695), ('apple/contacts.get', 2.661), ('apple/photos.get', 2.661), ('apple/calendar.events.get', 2.58916), ('apple/reminders.get', 2.52109), ('apple/health.workouts.get', 2.39516)]

H009 Turn an explicitly supplied street address into coordinates.

comparison · intent_transfer · target ['apple/location.geocode']

Forward geocoding of supplied address.

MethodFirst acceptable rankTop5
Keyword matching2apple/contacts.update → apple/location.geocode → apple/calendar.events.update → apple/contacts.get → apple/contacts.list
BM25 · name＋description4builtin/WriteFile → builtin/Memory → apple/contacts.get → apple/location.geocode → apple/reminders.update
BM25 · name + description + Schema + native aliases2builtin/WriteFile → apple/location.geocode → builtin/Memory → apple/contacts.update → apple/reminders.update
BM25 · name + description + Schema + extended aliases2builtin/WriteFile → apple/location.geocode → builtin/Memory → apple/contacts.update → apple/reminders.update
Full rankings, within-method scores

Keyword matching

[('apple/contacts.update', 1.5), ('apple/location.geocode', 1.5), ('apple/calendar.events.update', 1), ('apple/contacts.get', 1), ('apple/contacts.list', 1), ('apple/links.open', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.eta', 1), ('apple/maps.show', 1), ('apple/photos.search', 1), ('apple/reminders.list', 1), ('apple/reminders.update', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1), ('builtin/Memory', 1), ('builtin/ScheduledTasks', 1), ('builtin/WriteFile', 1), ('apple/contacts.create', 0.5), ('apple/maps.search', 0.5)]

BM25 · name＋description

[('builtin/WriteFile', 4.35041), ('builtin/Memory', 3.80676), ('apple/contacts.get', 3.44567), ('apple/location.geocode', 3.09904), ('apple/reminders.update', 2.80551), ('apple/contacts.list', 2.69272), ('apple/location.reverseGeocode', 2.43055), ('apple/reminders.list', 2.25376), ('apple/contacts.update', 2.18223), ('apple/maps.eta', 2.13955), ('apple/calendar.events.update', 1.9364), ('apple/maps.show', 1.91078), ('apple/weather.current', 1.72621), ('apple/photos.search', 1.69739), ('apple/weather.hourly', 1.6935), ('apple/weather.daily', 1.57416), ('builtin/ScheduledTasks', 1.54802), ('apple/links.open', 1.13446)]

BM25 · name + description + Schema + native aliases

[('builtin/WriteFile', 4.44991), ('apple/location.geocode', 4.07889), ('builtin/Memory', 3.76242), ('apple/contacts.update', 3.59467), ('apple/reminders.update', 2.83545), ('apple/contacts.get', 2.71066), ('apple/location.reverseGeocode', 2.49296), ('apple/reminders.list', 2.24438), ('apple/maps.eta', 2.14295), ('apple/contacts.list', 2.08281), ('apple/calendar.events.update', 1.93125), ('apple/maps.show', 1.90039), ('apple/weather.current', 1.70716), ('apple/weather.hourly', 1.67313), ('apple/photos.search', 1.66567), ('apple/weather.daily', 1.5496), ('apple/contacts.create', 1.52337), ('builtin/ScheduledTasks', 1.49781), ('apple/links.open', 1.10205), ('apple/maps.search', 1.07906)]

BM25 · name + description + Schema + extended aliases

[('builtin/WriteFile', 4.44991), ('apple/location.geocode', 4.07889), ('builtin/Memory', 3.76242), ('apple/contacts.update', 3.59467), ('apple/reminders.update', 2.83545), ('apple/contacts.get', 2.71066), ('apple/location.reverseGeocode', 2.49296), ('apple/reminders.list', 2.24438), ('apple/maps.eta', 2.14295), ('apple/contacts.list', 2.08281), ('apple/calendar.events.update', 1.93125), ('apple/maps.show', 1.90039), ('apple/weather.current', 1.70716), ('apple/weather.hourly', 1.67313), ('apple/photos.search', 1.66567), ('apple/weather.daily', 1.5496), ('apple/contacts.create', 1.52337), ('builtin/ScheduledTasks', 1.49781), ('apple/links.open', 1.10205), ('apple/maps.search', 1.07906)]

H010 把给定的经纬度转换为街道地址候选。

comparison · intent_transfer · target ['apple/location.reverseGeocode']

Reverse geocode explicit coordinates.

MethodFirst acceptable rankTop5
Keyword matching3apple/location.geocode → apple/location.get → apple/location.reverseGeocode
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliases2apple/location.geocode → apple/location.reverseGeocode → apple/location.get
BM25 · name + description + Schema + extended aliases2apple/location.geocode → apple/location.reverseGeocode → apple/location.get
Full rankings, within-method scores

Keyword matching

[('apple/location.geocode', 1.5), ('apple/location.get', 1.5), ('apple/location.reverseGeocode', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/location.geocode', 4.85181), ('apple/location.reverseGeocode', 4.85181), ('apple/location.get', 3.93504)]

BM25 · name + description + Schema + extended aliases

[('apple/location.geocode', 5.86696), ('apple/location.reverseGeocode', 5.86696), ('apple/location.get', 3.80773)]

H011 Search Apple Maps for pizza places inside this supplied region.

comparison · intent_transfer · target ['apple/maps.search']

Explicit query and optional bounding region.

MethodFirst acceptable rankTop5
Keyword matching1apple/maps.search → apple/links.open → apple/music.search → apple/maps.route → apple/maps.show
BM25 · name＋description1apple/maps.search → apple/maps.show → apple/music.search → apple/links.open → apple/maps.route
BM25 · name + description + Schema + native aliases1apple/maps.search → apple/music.search → apple/maps.show → apple/maps.route → apple/links.open
BM25 · name + description + Schema + extended aliases1apple/maps.search → apple/maps.show → apple/links.open → apple/maps.route → apple/music.search
Full rankings, within-method scores

Keyword matching

[('apple/maps.search', 14.5), ('apple/links.open', 11.5), ('apple/music.search', 8.5), ('apple/maps.route', 8), ('apple/maps.show', 8), ('apple/photos.search', 7), ('apple/maps.eta', 6), ('apple/shortcuts.run', 5), ('apple/alarms.cancel', 4), ('apple/alarms.create', 4), ('apple/alarms.list', 4), ('apple/calendar.events.update', 4), ('apple/contacts.update', 4), ('apple/music.playback', 4), ('apple/music.status', 4), ('apple/reminders.list', 4), ('apple/reminders.update', 4), ('apple/weather.current', 4), ('apple/weather.daily', 4), ('apple/weather.hourly', 4), ('apple/photos.get', 3.5), ('apple/battery.get', 3), ('apple/calendar.events.create', 3), ('apple/calendar.events.delete', 3), ('apple/calendar.events.get', 3), ('apple/calendar.events.list', 3), ('apple/calendar.list', 3), ('apple/clipboard.read', 3), ('apple/clipboard.write', 3), ('apple/contacts.create', 3), ('apple/contacts.delete', 3), ('apple/contacts.get', 3), ('apple/contacts.list', 3), ('apple/device.get', 3), ('apple/health.sleep.query', 3), ('apple/health.statistics', 3), ('apple/health.workouts.get', 3), ('apple/health.workouts.list', 3), ('apple/location.geocode', 3), ('apple/location.get', 3), ('apple/location.reverseGeocode', 3), ('apple/photos.albums.list', 3), ('apple/reminders.complete', 3), ('apple/reminders.create', 3), ('apple/reminders.delete', 3), ('apple/reminders.get', 3), ('apple/reminders.lists', 3), ('builtin/ScheduledTasks', 2)]

BM25 · name＋description

[('apple/maps.search', 13.46947), ('apple/maps.show', 6.27593), ('apple/music.search', 5.14214), ('apple/links.open', 4.86499), ('apple/maps.route', 4.85127), ('apple/photos.search', 3.54156), ('apple/shortcuts.run', 3.45174), ('apple/reminders.update', 2.80551), ('apple/maps.eta', 2.68318), ('apple/music.status', 2.384), ('apple/reminders.list', 2.25376), ('apple/contacts.update', 2.18223), ('apple/alarms.create', 2.01195), ('apple/weather.current', 1.95529), ('apple/calendar.events.update', 1.9364), ('apple/alarms.list', 1.50453), ('apple/alarms.cancel', 1.48861), ('apple/weather.hourly', 1.33344), ('apple/weather.daily', 1.23947), ('builtin/ScheduledTasks', 1.15544), ('apple/music.playback', 1.1034)]

BM25 · name + description + Schema + native aliases

[('apple/maps.search', 29.3049), ('apple/music.search', 11.52393), ('apple/maps.show', 10.6392), ('apple/maps.route', 9.32826), ('apple/links.open', 9.0425), ('apple/photos.search', 8.80266), ('apple/maps.eta', 7.13699), ('apple/reminders.update', 3.34228), ('apple/music.status', 2.88915), ('apple/reminders.list', 2.7512), ('builtin/ScheduledTasks', 2.74472), ('apple/shortcuts.run', 2.68354), ('apple/contacts.update', 2.67584), ('apple/alarms.create', 2.49759), ('apple/calendar.events.update', 2.37929), ('apple/photos.get', 1.98883), ('apple/weather.current', 0.70854), ('apple/alarms.list', 0.66243), ('apple/alarms.cancel', 0.66069), ('apple/weather.hourly', 0.64383), ('apple/weather.daily', 0.63371), ('apple/music.playback', 0.61919), ('apple/battery.get', 0.50682), ('apple/calendar.list', 0.50682), ('apple/clipboard.read', 0.50682), ('apple/clipboard.write', 0.50682), ('apple/contacts.create', 0.50682), ('apple/contacts.delete', 0.50682), ('apple/contacts.get', 0.50682), ('apple/contacts.list', 0.50682), ('apple/device.get', 0.50682), ('apple/health.statistics', 0.50682), ('apple/location.geocode', 0.50682), ('apple/location.get', 0.50682), ('apple/reminders.complete', 0.50682), ('apple/reminders.create', 0.50682), ('apple/reminders.delete', 0.50682), ('apple/reminders.get', 0.50682), ('apple/reminders.lists', 0.50682), ('apple/calendar.events.create', 0.44804), ('apple/calendar.events.delete', 0.44804), ('apple/calendar.events.get', 0.44804), ('apple/calendar.events.list', 0.44804), ('apple/health.sleep.query', 0.44804), ('apple/health.workouts.get', 0.44804), ('apple/health.workouts.list', 0.44804), ('apple/location.reverseGeocode', 0.44804), ('apple/photos.albums.list', 0.44804)]

BM25 · name + description + Schema + extended aliases

[('apple/maps.search', 25.76583), ('apple/maps.show', 10.6392), ('apple/links.open', 10.1148), ('apple/maps.route', 9.32826), ('apple/music.search', 7.83432), ('apple/maps.eta', 7.13699), ('apple/photos.search', 6.55247), ('apple/reminders.list', 4.51458), ('apple/reminders.update', 3.34228), ('apple/music.status', 2.88915), ('apple/shortcuts.run', 2.68354), ('apple/contacts.update', 2.67584), ('apple/alarms.list', 2.61625), ('builtin/ScheduledTasks', 2.56509), ('apple/alarms.create', 2.49759), ('apple/calendar.events.update', 2.37929), ('apple/contacts.list', 2.36054), ('apple/calendar.list', 2.31424), ('apple/photos.albums.list', 2.30176), ('apple/calendar.events.list', 2.25546), ('apple/health.workouts.list', 2.12947), ('apple/photos.get', 1.48585), ('apple/weather.current', 0.70854), ('apple/alarms.cancel', 0.66069), ('apple/weather.hourly', 0.64383), ('apple/weather.daily', 0.63371), ('apple/music.playback', 0.61919), ('apple/battery.get', 0.50682), ('apple/clipboard.read', 0.50682), ('apple/clipboard.write', 0.50682), ('apple/contacts.create', 0.50682), ('apple/contacts.delete', 0.50682), ('apple/contacts.get', 0.50682), ('apple/device.get', 0.50682), ('apple/health.statistics', 0.50682), ('apple/location.geocode', 0.50682), ('apple/location.get', 0.50682), ('apple/reminders.complete', 0.50682), ('apple/reminders.create', 0.50682), ('apple/reminders.delete', 0.50682), ('apple/reminders.get', 0.50682), ('apple/reminders.lists', 0.50682), ('apple/calendar.events.create', 0.44804), ('apple/calendar.events.delete', 0.44804), ('apple/calendar.events.get', 0.44804), ('apple/health.sleep.query', 0.44804), ('apple/health.workouts.get', 0.44804), ('apple/location.reverseGeocode', 0.44804)]

H012 在Apple地图里展示这一个坐标点，不规划路线。

comparison · intent_transfer · target ['apple/maps.show']

Foreground map pin handoff; no navigation promise.

MethodFirst acceptable rankTop5
Keyword matching4apple/links.open → apple/maps.route → apple/maps.search → apple/maps.show → apple/maps.eta
BM25 · name＋description4apple/shortcuts.run → apple/links.open → apple/music.status → apple/maps.show → apple/alarms.create
BM25 · name + description + Schema + native aliases1apple/maps.show → apple/maps.route → apple/maps.eta → apple/links.open → apple/maps.search
BM25 · name + description + Schema + extended aliases1apple/maps.show → apple/maps.route → apple/maps.eta → apple/links.open → apple/maps.search
Full rankings, within-method scores

Keyword matching

[('apple/links.open', 10), ('apple/maps.route', 7), ('apple/maps.search', 7), ('apple/maps.show', 7), ('apple/maps.eta', 6), ('apple/shortcuts.run', 5), ('apple/location.geocode', 4.5), ('apple/location.get', 4.5), ('apple/location.reverseGeocode', 4.5), ('apple/alarms.cancel', 4), ('apple/alarms.create', 4), ('apple/alarms.list', 4), ('apple/music.playback', 4), ('apple/music.search', 4), ('apple/music.status', 4), ('apple/weather.current', 4), ('apple/weather.daily', 4), ('apple/weather.hourly', 4), ('apple/battery.get', 3), ('apple/calendar.events.create', 3), ('apple/calendar.events.delete', 3), ('apple/calendar.events.get', 3), ('apple/calendar.events.list', 3), ('apple/calendar.events.update', 3), ('apple/calendar.list', 3), ('apple/clipboard.read', 3), ('apple/clipboard.write', 3), ('apple/contacts.create', 3), ('apple/contacts.delete', 3), ('apple/contacts.get', 3), ('apple/contacts.list', 3), ('apple/contacts.update', 3), ('apple/device.get', 3), ('apple/health.sleep.query', 3), ('apple/health.statistics', 3), ('apple/health.workouts.get', 3), ('apple/health.workouts.list', 3), ('apple/photos.albums.list', 3), ('apple/photos.get', 3), ('apple/photos.search', 3), ('apple/reminders.complete', 3), ('apple/reminders.create', 3), ('apple/reminders.delete', 3), ('apple/reminders.get', 3), ('apple/reminders.list', 3), ('apple/reminders.lists', 3), ('apple/reminders.update', 3), ('builtin/ScheduledTasks', 1)]

BM25 · name＋description

[('apple/shortcuts.run', 3.45174), ('apple/links.open', 3.44228), ('apple/music.status', 2.384), ('apple/maps.show', 2.10128), ('apple/alarms.create', 2.01195), ('apple/weather.current', 1.95529), ('apple/alarms.list', 1.50453), ('apple/maps.route', 1.50453), ('apple/alarms.cancel', 1.48861), ('apple/maps.search', 1.41383), ('apple/music.search', 1.41383), ('apple/weather.hourly', 1.33344), ('apple/weather.daily', 1.23947), ('builtin/ScheduledTasks', 1.15544), ('apple/music.playback', 1.1034)]

BM25 · name + description + Schema + native aliases

[('apple/maps.show', 8.58684), ('apple/maps.route', 8.52476), ('apple/maps.eta', 8.36915), ('apple/links.open', 7.77795), ('apple/maps.search', 7.11726), ('apple/location.geocode', 5.35864), ('apple/location.reverseGeocode', 5.29986), ('apple/location.get', 4.44186), ('apple/music.status', 2.88915), ('apple/shortcuts.run', 2.68354), ('apple/alarms.create', 2.49759), ('builtin/ScheduledTasks', 1.11796), ('apple/weather.current', 0.70854), ('apple/alarms.list', 0.66243), ('apple/alarms.cancel', 0.66069), ('apple/music.search', 0.65254), ('apple/weather.hourly', 0.64383), ('apple/weather.daily', 0.63371), ('apple/music.playback', 0.61919), ('apple/battery.get', 0.50682), ('apple/calendar.list', 0.50682), ('apple/clipboard.read', 0.50682), ('apple/clipboard.write', 0.50682), ('apple/contacts.create', 0.50682), ('apple/contacts.delete', 0.50682), ('apple/contacts.get', 0.50682), ('apple/contacts.list', 0.50682), ('apple/contacts.update', 0.50682), ('apple/device.get', 0.50682), ('apple/health.statistics', 0.50682), ('apple/photos.get', 0.50682), ('apple/photos.search', 0.50682), ('apple/reminders.complete', 0.50682), ('apple/reminders.create', 0.50682), ('apple/reminders.delete', 0.50682), ('apple/reminders.get', 0.50682), ('apple/reminders.list', 0.50682), ('apple/reminders.lists', 0.50682), ('apple/reminders.update', 0.50682), ('apple/calendar.events.create', 0.44804), ('apple/calendar.events.delete', 0.44804), ('apple/calendar.events.get', 0.44804), ('apple/calendar.events.list', 0.44804), ('apple/calendar.events.update', 0.44804), ('apple/health.sleep.query', 0.44804), ('apple/health.workouts.get', 0.44804), ('apple/health.workouts.list', 0.44804), ('apple/photos.albums.list', 0.44804)]

BM25 · name + description + Schema + extended aliases

[('apple/maps.show', 14.59191), ('apple/maps.route', 10.49894), ('apple/maps.eta', 10.34333), ('apple/links.open', 9.73183), ('apple/maps.search', 7.68169), ('apple/location.geocode', 6.37379), ('apple/location.reverseGeocode', 6.31501), ('apple/location.get', 4.31456), ('apple/music.status', 2.88915), ('apple/shortcuts.run', 2.68354), ('apple/alarms.create', 2.49759), ('builtin/ScheduledTasks', 1.11796), ('apple/weather.current', 0.70854), ('apple/alarms.list', 0.66243), ('apple/alarms.cancel', 0.66069), ('apple/music.search', 0.65254), ('apple/weather.hourly', 0.64383), ('apple/weather.daily', 0.63371), ('apple/music.playback', 0.61919), ('apple/battery.get', 0.50682), ('apple/calendar.list', 0.50682), ('apple/clipboard.read', 0.50682), ('apple/clipboard.write', 0.50682), ('apple/contacts.create', 0.50682), ('apple/contacts.delete', 0.50682), ('apple/contacts.get', 0.50682), ('apple/contacts.list', 0.50682), ('apple/contacts.update', 0.50682), ('apple/device.get', 0.50682), ('apple/health.statistics', 0.50682), ('apple/photos.get', 0.50682), ('apple/photos.search', 0.50682), ('apple/reminders.complete', 0.50682), ('apple/reminders.create', 0.50682), ('apple/reminders.delete', 0.50682), ('apple/reminders.get', 0.50682), ('apple/reminders.list', 0.50682), ('apple/reminders.lists', 0.50682), ('apple/reminders.update', 0.50682), ('apple/calendar.events.create', 0.44804), ('apple/calendar.events.delete', 0.44804), ('apple/calendar.events.get', 0.44804), ('apple/calendar.events.list', 0.44804), ('apple/calendar.events.update', 0.44804), ('apple/health.sleep.query', 0.44804), ('apple/health.workouts.get', 0.44804), ('apple/health.workouts.list', 0.44804), ('apple/photos.albums.list', 0.44804)]

H013 Create one Anan-owned alarm at this future absolute timestamp.

comparison · intent_transfer · target ['apple/alarms.create']

One future absolute AlarmKit alarm.

MethodFirst acceptable rankTop5
Keyword matching1apple/alarms.create → apple/alarms.list → apple/alarms.cancel → apple/calendar.events.create → apple/contacts.create
BM25 · name＋description1apple/alarms.create → apple/alarms.list → apple/alarms.cancel → apple/calendar.events.create → apple/reminders.create
BM25 · name + description + Schema + native aliases1apple/alarms.create → apple/alarms.list → apple/alarms.cancel → apple/calendar.events.create → apple/reminders.create
BM25 · name + description + Schema + extended aliases1apple/alarms.create → apple/alarms.list → apple/alarms.cancel → apple/calendar.events.create → apple/reminders.create
Full rankings, within-method scores

Keyword matching

[('apple/alarms.create', 14), ('apple/alarms.list', 8), ('apple/alarms.cancel', 7.5), ('apple/calendar.events.create', 6.5), ('apple/contacts.create', 5), ('apple/reminders.create', 5), ('builtin/Memory', 2.5), ('apple/contacts.delete', 2), ('apple/music.playback', 2), ('builtin/ScheduledTasks', 2), ('apple/calendar.events.update', 1.5), ('apple/calendar.events.delete', 1), ('apple/clipboard.read', 1), ('apple/clipboard.write', 1), ('apple/contacts.update', 1), ('apple/health.workouts.get', 1), ('apple/location.get', 1), ('apple/maps.route', 1), ('apple/music.status', 1), ('apple/photos.get', 1), ('apple/photos.search', 1), ('apple/reminders.delete', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1), ('builtin/EditFile', 1), ('builtin/ListFiles', 1), ('builtin/WriteFile', 1), ('apple/calendar.events.list', 0.5), ('apple/health.sleep.query', 0.5), ('apple/health.statistics', 0.5), ('apple/health.workouts.list', 0.5), ('apple/links.open', 0.5), ('builtin/ReadFile', 0.5)]

BM25 · name＋description

[('apple/alarms.create', 17.50567), ('apple/alarms.list', 13.74881), ('apple/alarms.cancel', 9.36395), ('apple/calendar.events.create', 5.94182), ('apple/reminders.create', 5.75219), ('apple/contacts.create', 4.70041), ('apple/contacts.delete', 3.76956), ('builtin/Memory', 3.279), ('builtin/WriteFile', 2.81316), ('apple/music.playback', 2.17336), ('builtin/ScheduledTasks', 2.0673), ('apple/music.status', 2.06537), ('apple/weather.current', 1.99252), ('apple/weather.hourly', 1.95475), ('apple/clipboard.read', 1.93144), ('apple/weather.daily', 1.817), ('apple/photos.search', 1.69739), ('builtin/ListFiles', 1.68221), ('builtin/EditFile', 1.5025), ('apple/contacts.update', 1.45931), ('apple/reminders.delete', 1.41606), ('apple/health.workouts.get', 1.39598), ('apple/calendar.events.update', 1.34012), ('apple/location.get', 1.19299), ('apple/calendar.events.delete', 1.12484), ('apple/clipboard.write', 1.09971), ('apple/maps.route', 1.05267), ('apple/photos.get', 0.98921)]

BM25 · name + description + Schema + native aliases

[('apple/alarms.create', 32.29932), ('apple/alarms.list', 19.46453), ('apple/alarms.cancel', 18.15041), ('apple/calendar.events.create', 11.73697), ('apple/reminders.create', 11.25903), ('apple/contacts.create', 10.10762), ('builtin/Memory', 4.1383), ('apple/contacts.delete', 3.74548), ('builtin/WriteFile', 2.8775), ('builtin/ScheduledTasks', 2.49562), ('apple/music.playback', 2.09235), ('apple/music.status', 2.06392), ('apple/calendar.events.update', 2.03119), ('apple/clipboard.read', 1.92215), ('builtin/ListFiles', 1.65142), ('builtin/EditFile', 1.45986), ('apple/contacts.update', 1.38404), ('apple/health.workouts.get', 1.36911), ('apple/reminders.delete', 1.36911), ('apple/weather.current', 1.20804), ('apple/weather.hourly', 1.18397), ('apple/location.get', 1.139), ('apple/weather.daily', 1.09655), ('apple/calendar.events.delete', 1.083), ('apple/clipboard.write', 1.04448), ('apple/photos.search', 1.02115), ('apple/maps.route', 0.99719), ('apple/photos.get', 0.93377), ('apple/health.sleep.query', 0.86419), ('apple/calendar.events.list', 0.82533), ('apple/health.statistics', 0.81615), ('apple/health.workouts.list', 0.7984), ('builtin/ReadFile', 0.51852), ('apple/links.open', 0.2902)]

BM25 · name + description + Schema + extended aliases

[('apple/alarms.create', 32.29932), ('apple/alarms.list', 19.46453), ('apple/alarms.cancel', 18.15041), ('apple/calendar.events.create', 11.73697), ('apple/reminders.create', 11.25903), ('apple/contacts.create', 10.10762), ('builtin/WriteFile', 5.71516), ('builtin/Memory', 4.1383), ('apple/contacts.delete', 3.74548), ('builtin/ScheduledTasks', 2.49562), ('apple/music.playback', 2.09235), ('apple/music.status', 2.06392), ('apple/calendar.events.update', 2.03119), ('apple/clipboard.read', 1.92215), ('builtin/ListFiles', 1.65142), ('builtin/EditFile', 1.45986), ('apple/contacts.update', 1.38404), ('apple/health.workouts.get', 1.36911), ('apple/reminders.delete', 1.36911), ('apple/weather.current', 1.20804), ('apple/weather.hourly', 1.18397), ('apple/location.get', 1.139), ('apple/weather.daily', 1.09655), ('apple/calendar.events.delete', 1.083), ('apple/clipboard.write', 1.04448), ('apple/photos.search', 1.02115), ('apple/maps.route', 0.99719), ('apple/photos.get', 0.93377), ('apple/health.sleep.query', 0.86419), ('apple/calendar.events.list', 0.82533), ('apple/health.statistics', 0.81615), ('apple/health.workouts.list', 0.7984), ('builtin/ReadFile', 0.51852), ('apple/links.open', 0.2902)]

H014 List the native alarms owned by Anan, not all Clock app alarms.

comparison · intent_transfer · target ['apple/alarms.list']

Scope is actual Anan-owned native alarms.

MethodFirst acceptable rankTop5
Keyword matching1apple/alarms.list → apple/alarms.create → apple/alarms.cancel → apple/photos.albums.list → apple/contacts.list
BM25 · name＋description2apple/alarms.create → apple/alarms.list → apple/alarms.cancel → apple/music.status → apple/reminders.create
BM25 · name + description + Schema + native aliases1apple/alarms.list → apple/alarms.create → apple/alarms.cancel → apple/music.status → apple/photos.albums.list
BM25 · name + description + Schema + extended aliases1apple/alarms.list → apple/alarms.create → apple/alarms.cancel → apple/music.status → apple/photos.albums.list
Full rankings, within-method scores

Keyword matching

[('apple/alarms.list', 13), ('apple/alarms.create', 11.5), ('apple/alarms.cancel', 10), ('apple/photos.albums.list', 6.5), ('apple/contacts.list', 5.5), ('apple/reminders.list', 5), ('apple/calendar.events.list', 4.5), ('apple/links.open', 4.5), ('builtin/ScheduledTasks', 4.5), ('apple/calendar.list', 4), ('apple/music.status', 4), ('apple/reminders.lists', 4), ('builtin/ListFiles', 4), ('apple/health.workouts.list', 3.5), ('apple/reminders.create', 3.5), ('apple/calendar.events.create', 2.5), ('apple/calendar.events.update', 2.5), ('apple/health.statistics', 2), ('apple/music.playback', 2), ('apple/music.search', 2), ('apple/shortcuts.run', 2), ('apple/weather.current', 2), ('apple/weather.daily', 2), ('builtin/Memory', 2), ('apple/calendar.events.delete', 1.5), ('apple/contacts.delete', 1.5), ('apple/health.sleep.query', 1.5), ('apple/maps.search', 1.5), ('apple/battery.get', 1), ('apple/calendar.events.get', 1), ('apple/clipboard.read', 1), ('apple/clipboard.write', 1), ('apple/location.geocode', 1), ('apple/location.get', 1), ('apple/maps.eta', 1), ('apple/maps.show', 1), ('apple/photos.get', 1), ('apple/photos.search', 1), ('apple/weather.hourly', 1), ('apple/contacts.get', 0.5), ('apple/contacts.update', 0.5), ('apple/health.workouts.get', 0.5)]

BM25 · name＋description

[('apple/alarms.create', 16.20664), ('apple/alarms.list', 15.4689), ('apple/alarms.cancel', 14.05692), ('apple/music.status', 7.05906), ('apple/reminders.create', 5.09401), ('apple/photos.albums.list', 4.61819), ('apple/calendar.events.create', 4.45927), ('apple/music.playback', 4.33653), ('apple/links.open', 4.30211), ('apple/calendar.events.update', 4.27892), ('builtin/ScheduledTasks', 3.81693), ('apple/weather.current', 3.34017), ('apple/shortcuts.run', 3.03135), ('apple/health.statistics', 2.94957), ('apple/music.search', 2.63006), ('apple/contacts.delete', 2.43055), ('apple/contacts.list', 2.42221), ('apple/reminders.lists', 2.1669), ('builtin/ListFiles', 1.99034), ('apple/weather.daily', 1.98903), ('apple/calendar.list', 1.97043), ('apple/clipboard.read', 1.93144), ('apple/weather.hourly', 1.92849), ('apple/calendar.events.list', 1.6149), ('apple/health.sleep.query', 1.56294), ('apple/reminders.list', 1.49249), ('apple/calendar.events.get', 1.458), ('apple/health.workouts.list', 1.33902), ('apple/clipboard.write', 1.30984), ('apple/photos.get', 1.21623), ('builtin/Memory', 1.16476), ('apple/calendar.events.delete', 1.12484), ('apple/location.geocode', 1.07002), ('apple/location.get', 1.03112), ('apple/battery.get', 1.01877), ('apple/maps.eta', 1.01877), ('apple/maps.show', 0.90984), ('apple/maps.search', 0.85499)]

BM25 · name + description + Schema + native aliases

[('apple/alarms.list', 23.98289), ('apple/alarms.create', 23.45666), ('apple/alarms.cancel', 23.20634), ('apple/music.status', 6.87895), ('apple/photos.albums.list', 6.46895), ('apple/links.open', 5.6753), ('apple/reminders.create', 5.1068), ('apple/contacts.list', 4.59464), ('apple/calendar.events.create', 4.54773), ('apple/reminders.list', 4.14438), ('apple/reminders.lists', 4.12289), ('apple/calendar.events.update', 4.02293), ('builtin/ScheduledTasks', 4.00926), ('apple/music.playback', 3.99917), ('builtin/ListFiles', 3.83921), ('apple/calendar.list', 3.8154), ('apple/calendar.events.list', 3.47608), ('apple/weather.current', 3.07067), ('apple/contacts.delete', 3.05744), ('apple/shortcuts.run', 2.75658), ('apple/health.statistics', 2.6901), ('apple/health.workouts.list', 2.54626), ('apple/music.search', 2.39427), ('apple/clipboard.read', 1.92215), ('apple/weather.hourly', 1.91167), ('builtin/Memory', 1.8696), ('apple/weather.daily', 1.82648), ('apple/health.sleep.query', 1.62443), ('apple/calendar.events.delete', 1.41756), ('apple/calendar.events.get', 1.12534), ('apple/maps.search', 1.09049), ('apple/clipboard.write', 1.07428), ('apple/photos.get', 0.99308), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/maps.show', 0.74361), ('apple/contacts.get', 0.60095), ('apple/health.workouts.get', 0.58448), ('apple/photos.search', 0.58223), ('apple/contacts.update', 0.39141)]

BM25 · name + description + Schema + extended aliases

[('apple/alarms.list', 23.74533), ('apple/alarms.create', 23.24107), ('apple/alarms.cancel', 22.97144), ('apple/music.status', 6.87895), ('apple/photos.albums.list', 6.46895), ('apple/links.open', 6.3359), ('apple/reminders.create', 5.1068), ('builtin/ListFiles', 5.03933), ('apple/contacts.list', 4.59464), ('apple/calendar.events.create', 4.54773), ('apple/reminders.list', 4.14438), ('apple/reminders.lists', 4.12289), ('apple/calendar.events.update', 4.02293), ('builtin/ScheduledTasks', 4.00926), ('apple/music.playback', 3.99917), ('apple/calendar.list', 3.8154), ('builtin/GetCurrentTime', 3.66024), ('apple/calendar.events.list', 3.47608), ('apple/weather.current', 3.07067), ('apple/contacts.delete', 3.05744), ('apple/shortcuts.run', 2.75658), ('apple/health.statistics', 2.6901), ('apple/health.workouts.list', 2.54626), ('apple/music.search', 2.39427), ('apple/clipboard.read', 1.92215), ('apple/weather.hourly', 1.91167), ('builtin/Memory', 1.8696), ('apple/weather.daily', 1.82648), ('apple/health.sleep.query', 1.62443), ('apple/calendar.events.delete', 1.41756), ('apple/calendar.events.get', 1.12534), ('apple/maps.search', 1.09049), ('apple/clipboard.write', 1.07428), ('apple/photos.get', 0.99308), ('apple/location.geocode', 0.88363), ('apple/location.get', 0.84936), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/maps.show', 0.74361), ('apple/contacts.get', 0.60095), ('apple/health.workouts.get', 0.58448), ('apple/photos.search', 0.58223), ('apple/contacts.update', 0.39141)]

H015 取消当前列表里这个UUID对应的Anan闹钟。

comparison · intent_transfer · target ['apple/alarms.cancel']

Cancel one current Anan-owned alarm UUID.

MethodFirst acceptable rankTop5
Keyword matching1apple/alarms.cancel → apple/alarms.list → apple/alarms.create → apple/health.workouts.get → apple/clipboard.read
BM25 · name＋description2apple/alarms.list → apple/alarms.cancel → apple/health.workouts.get → apple/contacts.delete → apple/music.status
BM25 · name + description + Schema + native aliases1apple/alarms.cancel → apple/alarms.list → apple/alarms.create → apple/health.workouts.get → apple/contacts.delete
BM25 · name + description + Schema + extended aliases1apple/alarms.cancel → apple/alarms.list → apple/alarms.create → apple/health.workouts.get → builtin/GetCurrentTime
Full rankings, within-method scores

Keyword matching

[('apple/alarms.cancel', 4), ('apple/alarms.list', 3.5), ('apple/alarms.create', 2.5), ('apple/health.workouts.get', 1.5), ('apple/clipboard.read', 1), ('apple/contacts.delete', 1), ('apple/music.playback', 1), ('apple/music.status', 1)]

BM25 · name＋description

[('apple/alarms.list', 4.54593), ('apple/alarms.cancel', 4.49783), ('apple/health.workouts.get', 3.49453), ('apple/contacts.delete', 2.43055), ('apple/music.status', 2.06537), ('apple/clipboard.read', 1.93144), ('apple/alarms.create', 1.74305), ('apple/music.playback', 1.40134)]

BM25 · name + description + Schema + native aliases

[('apple/alarms.cancel', 12.81164), ('apple/alarms.list', 9.02338), ('apple/alarms.create', 6.57651), ('apple/health.workouts.get', 5.5586), ('apple/contacts.delete', 2.45649), ('apple/music.status', 2.06392), ('apple/clipboard.read', 1.92215), ('apple/music.playback', 1.37228)]

BM25 · name + description + Schema + extended aliases

[('apple/alarms.cancel', 18.97399), ('apple/alarms.list', 8.32894), ('apple/alarms.create', 5.99344), ('apple/health.workouts.get', 5.5586), ('builtin/GetCurrentTime', 5.25514), ('builtin/ListFiles', 5.25514), ('apple/contacts.delete', 2.45649), ('apple/music.status', 2.06392), ('apple/clipboard.read', 1.92215), ('apple/music.playback', 1.37228)]

H016 What plain text is available on my clipboard?

comparison · intent_transfer · target ['apple/clipboard.read']

Read available foreground clipboard text.

MethodFirst acceptable rankTop5
Keyword matching1apple/clipboard.read → apple/clipboard.write → apple/shortcuts.run → apple/links.open → builtin/EditFile
BM25 · name＋description1apple/clipboard.read → apple/clipboard.write → apple/shortcuts.run → builtin/ReadFile → builtin/EditFile
BM25 · name + description + Schema + native aliases2apple/clipboard.write → apple/clipboard.read → apple/shortcuts.run → builtin/EditFile → builtin/ReadFile
BM25 · name + description + Schema + extended aliases2apple/clipboard.write → apple/clipboard.read → apple/shortcuts.run → builtin/EditFile → builtin/ReadFile
Full rankings, within-method scores

Keyword matching

[('apple/clipboard.read', 7), ('apple/clipboard.write', 6.5), ('apple/shortcuts.run', 3.5), ('apple/links.open', 1.5), ('builtin/EditFile', 1.5), ('apple/contacts.update', 1), ('apple/location.get', 1), ('apple/photos.albums.list', 1), ('apple/photos.get', 1), ('builtin/AskUserQuestions', 1), ('builtin/ReadFile', 1), ('builtin/Memory', 0.5)]

BM25 · name＋description

[('apple/clipboard.read', 13.18843), ('apple/clipboard.write', 10.91234), ('apple/shortcuts.run', 4.22937), ('builtin/ReadFile', 2.85815), ('builtin/EditFile', 2.72729), ('apple/location.get', 2.49955), ('builtin/AskUserQuestions', 2.16548), ('apple/photos.get', 2.0726), ('apple/photos.albums.list', 1.9364), ('apple/contacts.update', 1.89057), ('apple/links.open', 1.30948)]

BM25 · name + description + Schema + native aliases

[('apple/clipboard.write', 20.1766), ('apple/clipboard.read', 19.99408), ('apple/shortcuts.run', 7.65472), ('builtin/EditFile', 4.13067), ('builtin/ReadFile', 2.74541), ('apple/location.get', 2.50553), ('builtin/AskUserQuestions', 2.06218), ('apple/photos.get', 2.05407), ('apple/photos.albums.list', 1.93125), ('apple/contacts.update', 1.76213), ('apple/links.open', 1.51496), ('builtin/Memory', 1.35633)]

BM25 · name + description + Schema + extended aliases

[('apple/clipboard.write', 19.9075), ('apple/clipboard.read', 19.82341), ('apple/shortcuts.run', 7.47933), ('builtin/EditFile', 6.32338), ('builtin/ReadFile', 5.01984), ('apple/location.get', 2.50553), ('builtin/WriteFile', 2.50401), ('apple/photos.get', 2.05407), ('builtin/AskUserQuestions', 1.94052), ('apple/photos.albums.list', 1.93125), ('apple/contacts.update', 1.65817), ('apple/links.open', 1.51496), ('builtin/Memory', 1.27631)]

H017 Replace the system clipboard with this plain-text sentence.

comparison · intent_transfer · target ['apple/clipboard.write']

Write one local plain-text clipboard item.

MethodFirst acceptable rankTop5
Keyword matching1apple/clipboard.write → apple/clipboard.read → apple/shortcuts.run → builtin/EditFile → apple/contacts.update
BM25 · name＋description1apple/clipboard.write → apple/clipboard.read → builtin/EditFile → apple/shortcuts.run → apple/contacts.update
BM25 · name + description + Schema + native aliases1apple/clipboard.write → apple/clipboard.read → apple/shortcuts.run → builtin/EditFile → apple/contacts.update
BM25 · name + description + Schema + extended aliases1apple/clipboard.write → apple/clipboard.read → builtin/EditFile → apple/shortcuts.run → apple/contacts.update
Full rankings, within-method scores

Keyword matching

[('apple/clipboard.write', 7.5), ('apple/clipboard.read', 7), ('apple/shortcuts.run', 3.5), ('builtin/EditFile', 2.5), ('apple/contacts.update', 2), ('builtin/Memory', 1.5), ('apple/contacts.create', 1), ('apple/device.get', 1), ('apple/links.open', 1), ('apple/music.playback', 1), ('apple/music.status', 1), ('builtin/AskUserQuestions', 1), ('builtin/ReadFile', 1), ('builtin/ScheduledTasks', 1)]

BM25 · name＋description

[('apple/clipboard.write', 13.21645), ('apple/clipboard.read', 13.02795), ('builtin/EditFile', 5.87533), ('apple/shortcuts.run', 4.22937), ('apple/contacts.update', 4.0728), ('apple/device.get', 3.11442), ('builtin/ReadFile', 2.85815), ('apple/contacts.create', 2.46922), ('builtin/Memory', 2.44041), ('apple/music.status', 2.21238), ('builtin/AskUserQuestions', 2.16548), ('apple/music.playback', 1.61753), ('apple/links.open', 1.21521), ('builtin/ScheduledTasks', 1.07227)]

BM25 · name + description + Schema + native aliases

[('apple/clipboard.write', 22.4742), ('apple/clipboard.read', 19.83437), ('apple/shortcuts.run', 7.65472), ('builtin/EditFile', 7.34201), ('apple/contacts.update', 3.93115), ('builtin/Memory', 3.7683), ('apple/device.get', 3.11942), ('builtin/ReadFile', 2.74541), ('apple/contacts.create', 2.48585), ('apple/music.status', 2.21083), ('builtin/AskUserQuestions', 2.06218), ('apple/music.playback', 1.58398), ('apple/links.open', 1.18049), ('builtin/ScheduledTasks', 1.03748)]

BM25 · name + description + Schema + extended aliases

[('apple/clipboard.write', 21.89802), ('apple/clipboard.read', 19.66369), ('builtin/EditFile', 11.86656), ('apple/shortcuts.run', 7.47933), ('apple/contacts.update', 6.09147), ('builtin/ReadFile', 5.01984), ('builtin/Memory', 3.36592), ('apple/device.get', 3.11942), ('builtin/WriteFile', 2.50401), ('apple/calendar.events.update', 2.49193), ('apple/contacts.create', 2.48585), ('apple/reminders.update', 2.43265), ('apple/music.status', 2.21083), ('builtin/AskUserQuestions', 1.94052), ('apple/music.playback', 1.37228), ('apple/links.open', 1.18049), ('builtin/ScheduledTasks', 1.03748)]

H018 告诉我本机系统名称和版本，不需要用户自定义设备名。

comparison · intent_transfer · target ['apple/device.get']

Generic system metadata; user device name excluded.

MethodFirst acceptable rankTop5
Keyword matching1apple/device.get
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliases1apple/device.get
BM25 · name + description + Schema + extended aliases1apple/device.get → builtin/ListFiles
Full rankings, within-method scores

Keyword matching

[('apple/device.get', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/device.get', 5.87803)]

BM25 · name + description + Schema + extended aliases

[('apple/device.get', 5.25514), ('builtin/ListFiles', 5.25514)]

H019 Search the Apple Music catalog for a song title.

comparison · intent_transfer · target ['apple/music.search']

Catalog song search, not player status.

MethodFirst acceptable rankTop5
Keyword matching1apple/music.search → apple/music.playback → apple/links.open → apple/maps.search → apple/music.status
BM25 · name＋description1apple/music.search → apple/music.playback → apple/music.status → apple/links.open → apple/maps.search
BM25 · name + description + Schema + native aliases1apple/music.search → apple/music.playback → apple/music.status → apple/maps.search → apple/links.open
BM25 · name + description + Schema + extended aliases1apple/music.search → apple/music.playback → apple/music.status → apple/links.open → apple/maps.search
Full rankings, within-method scores

Keyword matching

[('apple/music.search', 16.5), ('apple/music.playback', 11), ('apple/links.open', 10.5), ('apple/maps.search', 8.5), ('apple/music.status', 8), ('apple/photos.search', 6), ('apple/alarms.create', 5.5), ('apple/alarms.list', 5), ('apple/shortcuts.run', 5), ('apple/alarms.cancel', 4), ('apple/maps.route', 4), ('apple/maps.show', 4), ('apple/reminders.list', 4), ('apple/weather.current', 4), ('apple/weather.daily', 4), ('apple/weather.hourly', 4), ('apple/calendar.events.create', 3.5), ('apple/calendar.events.update', 3.5), ('apple/photos.get', 3.5), ('apple/reminders.create', 3.5), ('apple/reminders.update', 3.5), ('apple/battery.get', 3), ('apple/calendar.events.delete', 3), ('apple/calendar.events.get', 3), ('apple/calendar.events.list', 3), ('apple/calendar.list', 3), ('apple/clipboard.read', 3), ('apple/clipboard.write', 3), ('apple/contacts.create', 3), ('apple/contacts.delete', 3), ('apple/contacts.get', 3), ('apple/contacts.list', 3), ('apple/contacts.update', 3), ('apple/device.get', 3), ('apple/health.sleep.query', 3), ('apple/health.statistics', 3), ('apple/health.workouts.get', 3), ('apple/health.workouts.list', 3), ('apple/location.geocode', 3), ('apple/location.get', 3), ('apple/location.reverseGeocode', 3), ('apple/maps.eta', 3), ('apple/photos.albums.list', 3), ('apple/reminders.complete', 3), ('apple/reminders.delete', 3), ('apple/reminders.get', 3), ('apple/reminders.lists', 3), ('builtin/ScheduledTasks', 3)]

BM25 · name＋description

[('apple/music.search', 18.19143), ('apple/music.playback', 11.5671), ('apple/music.status', 6.80246), ('apple/links.open', 6.42952), ('apple/maps.search', 5.14214), ('apple/alarms.create', 4.0239), ('apple/alarms.list', 3.71009), ('apple/shortcuts.run', 3.45174), ('builtin/ScheduledTasks', 2.31089), ('apple/reminders.list', 2.25376), ('apple/maps.show', 2.10128), ('apple/weather.current', 1.95529), ('apple/photos.search', 1.84417), ('apple/maps.route', 1.50453), ('apple/alarms.cancel', 1.48861), ('apple/weather.hourly', 1.33344), ('apple/weather.daily', 1.23947)]

BM25 · name + description + Schema + native aliases

[('apple/music.search', 36.12039), ('apple/music.playback', 22.83026), ('apple/music.status', 15.30661), ('apple/maps.search', 11.08037), ('apple/links.open', 8.70144), ('apple/photos.search', 7.13699), ('apple/alarms.create', 5.34724), ('builtin/ScheduledTasks', 2.93795), ('apple/shortcuts.run', 2.68354), ('apple/alarms.list', 2.33938), ('apple/reminders.list', 2.22261), ('apple/photos.get', 1.98883), ('apple/reminders.create', 1.58447), ('apple/reminders.update', 1.58447), ('apple/calendar.events.create', 1.43294), ('apple/calendar.events.update', 1.3923), ('apple/maps.show', 0.72451), ('apple/weather.current', 0.70854), ('apple/maps.route', 0.66243), ('apple/alarms.cancel', 0.66069), ('apple/weather.hourly', 0.64383), ('apple/weather.daily', 0.63371), ('apple/battery.get', 0.50682), ('apple/calendar.list', 0.50682), ('apple/clipboard.read', 0.50682), ('apple/clipboard.write', 0.50682), ('apple/contacts.create', 0.50682), ('apple/contacts.delete', 0.50682), ('apple/contacts.get', 0.50682), ('apple/contacts.list', 0.50682), ('apple/contacts.update', 0.50682), ('apple/device.get', 0.50682), ('apple/health.statistics', 0.50682), ('apple/location.geocode', 0.50682), ('apple/location.get', 0.50682), ('apple/maps.eta', 0.50682), ('apple/reminders.complete', 0.50682), ('apple/reminders.delete', 0.50682), ('apple/reminders.get', 0.50682), ('apple/reminders.lists', 0.50682), ('apple/calendar.events.delete', 0.44804), ('apple/calendar.events.get', 0.44804), ('apple/calendar.events.list', 0.44804), ('apple/health.sleep.query', 0.44804), ('apple/health.workouts.get', 0.44804), ('apple/health.workouts.list', 0.44804), ('apple/location.reverseGeocode', 0.44804), ('apple/photos.albums.list', 0.44804)]

BM25 · name + description + Schema + extended aliases

[('apple/music.search', 32.43078), ('apple/music.playback', 22.83026), ('apple/music.status', 15.30661), ('apple/links.open', 8.81545), ('apple/maps.search', 7.5413), ('apple/alarms.create', 5.34724), ('apple/photos.search', 4.8868), ('apple/alarms.list', 4.29319), ('apple/reminders.list', 3.98598), ('builtin/ScheduledTasks', 2.75831), ('apple/shortcuts.run', 2.68354), ('apple/contacts.list', 2.36054), ('apple/calendar.list', 2.31424), ('apple/photos.albums.list', 2.30176), ('apple/calendar.events.list', 2.25546), ('apple/health.workouts.list', 2.12947), ('apple/reminders.create', 1.58447), ('apple/reminders.update', 1.58447), ('apple/photos.get', 1.48585), ('apple/calendar.events.create', 1.43294), ('apple/calendar.events.update', 1.3923), ('apple/maps.show', 0.72451), ('apple/weather.current', 0.70854), ('apple/maps.route', 0.66243), ('apple/alarms.cancel', 0.66069), ('apple/weather.hourly', 0.64383), ('apple/weather.daily', 0.63371), ('apple/battery.get', 0.50682), ('apple/clipboard.read', 0.50682), ('apple/clipboard.write', 0.50682), ('apple/contacts.create', 0.50682), ('apple/contacts.delete', 0.50682), ('apple/contacts.get', 0.50682), ('apple/contacts.update', 0.50682), ('apple/device.get', 0.50682), ('apple/health.statistics', 0.50682), ('apple/location.geocode', 0.50682), ('apple/location.get', 0.50682), ('apple/maps.eta', 0.50682), ('apple/reminders.complete', 0.50682), ('apple/reminders.delete', 0.50682), ('apple/reminders.get', 0.50682), ('apple/reminders.lists', 0.50682), ('apple/calendar.events.delete', 0.44804), ('apple/calendar.events.get', 0.44804), ('apple/health.sleep.query', 0.44804), ('apple/health.workouts.get', 0.44804), ('apple/location.reverseGeocode', 0.44804)]

H020 Geocode this address without acquiring my current location.

comparison · intent_transfer · target ['apple/location.geocode']

Explicit address geocoding does not acquire current GPS.

MethodFirst acceptable rankTop5
Keyword matching1apple/location.geocode → apple/location.reverseGeocode → apple/weather.current → apple/location.get → builtin/GetCurrentTime
BM25 · name＋description1apple/location.geocode → apple/location.reverseGeocode → apple/maps.search → apple/weather.daily → apple/contacts.get
BM25 · name + description + Schema + native aliases1apple/location.geocode → apple/location.reverseGeocode → apple/weather.current → apple/location.get → apple/maps.search
BM25 · name + description + Schema + extended aliases1apple/location.geocode → apple/location.reverseGeocode → builtin/GetCurrentTime → apple/weather.current → apple/location.get
Full rankings, within-method scores

Keyword matching

[('apple/location.geocode', 11.5), ('apple/location.reverseGeocode', 10), ('apple/weather.current', 5), ('apple/location.get', 4), ('builtin/GetCurrentTime', 4), ('apple/maps.search', 3.5), ('apple/maps.eta', 2), ('apple/maps.route', 2), ('apple/maps.show', 2), ('apple/photos.search', 2), ('apple/weather.daily', 2), ('apple/weather.hourly', 2), ('apple/calendar.events.update', 1.5), ('builtin/EditFile', 1.5), ('apple/battery.get', 1), ('apple/contacts.get', 1), ('apple/contacts.list', 1), ('apple/health.statistics', 1), ('apple/music.playback', 1), ('apple/music.status', 1), ('apple/photos.get', 1), ('apple/calendar.events.create', 0.5), ('apple/contacts.create', 0.5), ('apple/contacts.update', 0.5), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('apple/location.geocode', 14.17809), ('apple/location.reverseGeocode', 11.74867), ('apple/maps.search', 5.08686), ('apple/weather.daily', 3.51274), ('apple/contacts.get', 3.44567), ('apple/weather.current', 3.42621), ('apple/maps.eta', 2.95199), ('apple/weather.hourly', 2.87133), ('apple/photos.search', 2.73799), ('apple/contacts.list', 2.69272), ('apple/maps.route', 2.63636), ('apple/maps.show', 2.63636), ('apple/battery.get', 2.46962), ('apple/location.get', 2.32271), ('builtin/GetCurrentTime', 2.20523), ('builtin/EditFile', 1.83299), ('apple/music.status', 1.38811), ('apple/photos.get', 1.27062), ('apple/calendar.events.update', 1.18713), ('apple/health.statistics', 0.96452), ('apple/music.playback', 0.94183)]

BM25 · name + description + Schema + native aliases

[('apple/location.geocode', 27.36201), ('apple/location.reverseGeocode', 22.60795), ('apple/weather.current', 6.70106), ('apple/location.get', 6.11719), ('apple/maps.search', 5.97584), ('builtin/GetCurrentTime', 5.21653), ('apple/weather.daily', 3.30351), ('apple/maps.eta', 2.80943), ('apple/contacts.get', 2.71066), ('apple/weather.hourly', 2.70073), ('apple/photos.search', 2.63552), ('apple/maps.route', 2.49143), ('apple/maps.show', 2.49143), ('apple/battery.get', 2.47355), ('builtin/EditFile', 2.354), ('apple/contacts.list', 2.08281), ('apple/calendar.events.update', 1.84369), ('apple/contacts.create', 1.52337), ('apple/contacts.update', 1.42565), ('apple/music.status', 1.31868), ('apple/photos.get', 1.19601), ('apple/health.statistics', 0.89881), ('apple/music.playback', 0.87677), ('apple/calendar.events.create', 0.75014), ('builtin/ScheduledTasks', 0.3157)]

BM25 · name + description + Schema + extended aliases

[('apple/location.geocode', 27.36201), ('apple/location.reverseGeocode', 22.60795), ('builtin/GetCurrentTime', 7.0813), ('apple/weather.current', 6.70106), ('apple/location.get', 6.11719), ('apple/maps.search', 5.97584), ('apple/weather.daily', 3.30351), ('apple/maps.eta', 2.80943), ('apple/contacts.get', 2.71066), ('apple/weather.hourly', 2.70073), ('apple/photos.search', 2.63552), ('apple/maps.route', 2.49143), ('apple/maps.show', 2.49143), ('apple/battery.get', 2.47355), ('builtin/EditFile', 2.354), ('apple/contacts.list', 2.08281), ('apple/calendar.events.update', 1.84369), ('apple/contacts.create', 1.52337), ('apple/contacts.update', 1.42565), ('apple/music.status', 1.31868), ('apple/photos.get', 1.19601), ('apple/health.statistics', 0.89881), ('apple/music.playback', 0.87677), ('apple/calendar.events.create', 0.75014), ('builtin/ScheduledTasks', 0.3157)]

H021 Reverse geocode these coordinates without reading GPS.

comparison · intent_transfer · target ['apple/location.reverseGeocode']

Explicit coordinates, no current location read.

MethodFirst acceptable rankTop5
Keyword matching1apple/location.reverseGeocode → apple/location.geocode → apple/battery.get → apple/health.statistics → apple/links.open
BM25 · name＋description1apple/location.reverseGeocode → apple/location.geocode → apple/health.statistics → apple/battery.get → apple/maps.eta
BM25 · name + description + Schema + native aliases1apple/location.reverseGeocode → apple/location.geocode → apple/health.statistics → apple/battery.get → apple/maps.eta
BM25 · name + description + Schema + extended aliases1apple/location.reverseGeocode → apple/location.geocode → apple/health.statistics → apple/battery.get → apple/maps.eta
Full rankings, within-method scores

Keyword matching

[('apple/location.reverseGeocode', 11), ('apple/location.geocode', 6), ('apple/battery.get', 1), ('apple/health.statistics', 1), ('apple/links.open', 1), ('apple/maps.eta', 1), ('apple/maps.search', 1), ('apple/maps.show', 1), ('apple/photos.search', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1)]

BM25 · name＋description

[('apple/location.reverseGeocode', 19.49478), ('apple/location.geocode', 10.67771), ('apple/health.statistics', 2.58396), ('apple/battery.get', 2.46962), ('apple/maps.eta', 2.13955), ('apple/maps.search', 2.0726), ('apple/maps.show', 1.91078), ('apple/weather.current', 1.72621), ('apple/photos.search', 1.69739), ('apple/weather.hourly', 1.6935), ('apple/weather.daily', 1.57416), ('apple/links.open', 1.13446)]

BM25 · name + description + Schema + native aliases

[('apple/location.reverseGeocode', 35.88352), ('apple/location.geocode', 19.16308), ('apple/health.statistics', 2.53295), ('apple/battery.get', 2.47355), ('apple/maps.eta', 2.14295), ('apple/maps.search', 2.05407), ('apple/maps.show', 1.90039), ('apple/weather.current', 1.70716), ('apple/weather.hourly', 1.67313), ('apple/photos.search', 1.66567), ('apple/weather.daily', 1.5496), ('apple/links.open', 1.10205)]

BM25 · name + description + Schema + extended aliases

[('apple/location.reverseGeocode', 35.88352), ('apple/location.geocode', 19.16308), ('apple/health.statistics', 2.53295), ('apple/battery.get', 2.47355), ('apple/maps.eta', 2.14295), ('apple/maps.search', 2.05407), ('apple/maps.show', 1.90039), ('apple/weather.current', 1.70716), ('apple/weather.hourly', 1.67313), ('apple/photos.search', 1.66567), ('apple/weather.daily', 1.5496), ('apple/links.open', 1.10205)]

H022 查找给定时区内、指定截止日期区间的未完成提醒。

comparison · intent_transfer · target ['apple/reminders.list']

Due range and timezone filters on reminder query.

MethodFirst acceptable rankTop5
Keyword matching1apple/reminders.list → apple/links.open → apple/alarms.list → apple/calendar.events.list → apple/calendar.list
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliases1apple/reminders.list → apple/reminders.complete → apple/reminders.lists → apple/reminders.delete → apple/reminders.update
BM25 · name + description + Schema + extended aliases3apple/reminders.complete → builtin/GetCurrentTime → apple/reminders.list → apple/reminders.lists → apple/reminders.delete
Full rankings, within-method scores

Keyword matching

[('apple/reminders.list', 3), ('apple/links.open', 2), ('apple/alarms.list', 1.5), ('apple/calendar.events.list', 1.5), ('apple/calendar.list', 1.5), ('apple/contacts.list', 1.5), ('apple/health.workouts.list', 1.5), ('apple/maps.search', 1.5), ('apple/music.search', 1.5), ('apple/photos.albums.list', 1.5), ('apple/photos.search', 1.5), ('apple/reminders.complete', 1.5), ('apple/reminders.create', 1.5), ('apple/reminders.delete', 1.5), ('apple/reminders.get', 1.5), ('apple/reminders.lists', 1.5), ('apple/reminders.update', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/reminders.list', 5.30356), ('apple/reminders.complete', 3.91446), ('apple/reminders.lists', 3.91446), ('apple/reminders.delete', 3.5842), ('apple/reminders.update', 3.5842), ('apple/reminders.create', 3.43912), ('apple/reminders.get', 3.43912), ('apple/alarms.list', 2.71823), ('apple/music.search', 2.53552), ('apple/photos.search', 2.53552), ('apple/contacts.list', 2.37582), ('apple/photos.albums.list', 2.37582), ('apple/calendar.events.list', 2.23504), ('apple/calendar.list', 2.23504), ('apple/maps.search', 2.23504), ('apple/health.workouts.list', 1.89771), ('apple/links.open', 1.37541)]

BM25 · name + description + Schema + extended aliases

[('apple/reminders.complete', 10.65561), ('builtin/GetCurrentTime', 10.51028), ('apple/reminders.list', 6.41416), ('apple/reminders.lists', 4.62689), ('apple/reminders.delete', 3.95844), ('apple/reminders.create', 3.65657), ('apple/reminders.get', 3.58816), ('apple/music.search', 3.51476), ('apple/photos.search', 3.51476), ('apple/maps.search', 3.37543), ('apple/reminders.update', 3.33835), ('apple/alarms.list', 3.24673), ('apple/contacts.list', 3.12748), ('apple/photos.albums.list', 3.12748), ('apple/calendar.events.list', 3.07108), ('apple/calendar.list', 3.07108), ('apple/health.workouts.list', 2.91346), ('apple/links.open', 2.21433)]

H023 Read the recurring event occurrence using eventId and occurrenceStart.

comparison · intent_transfer · target ['apple/calendar.events.get']

Get supports occurrenceStart, unlike mutation of recurrence.

MethodFirst acceptable rankTop5
Keyword matching1apple/calendar.events.get → apple/calendar.events.create → apple/calendar.events.update → apple/calendar.events.delete → apple/calendar.events.list
BM25 · name＋description1apple/calendar.events.get → apple/calendar.events.delete → apple/calendar.events.update → apple/calendar.events.create → apple/calendar.events.list
BM25 · name + description + Schema + native aliases1apple/calendar.events.get → apple/calendar.events.delete → apple/calendar.events.update → apple/calendar.events.create → apple/calendar.events.list
BM25 · name + description + Schema + extended aliases1apple/calendar.events.get → apple/calendar.events.delete → apple/calendar.events.update → apple/calendar.events.create → apple/calendar.events.list
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.get', 9), ('apple/calendar.events.create', 7.5), ('apple/calendar.events.update', 7.5), ('apple/calendar.events.delete', 7), ('apple/calendar.events.list', 6), ('apple/clipboard.read', 4), ('builtin/ReadFile', 4), ('builtin/ScheduledTasks', 3.5), ('apple/photos.search', 3), ('apple/photos.get', 2.5), ('apple/contacts.get', 1.5), ('apple/health.sleep.query', 1.5), ('apple/health.statistics', 1.5), ('apple/health.workouts.get', 1.5), ('apple/health.workouts.list', 1.5), ('apple/music.playback', 1.5), ('apple/reminders.complete', 1.5), ('apple/reminders.create', 1.5), ('apple/reminders.delete', 1.5), ('apple/reminders.get', 1.5), ('apple/reminders.list', 1.5), ('apple/reminders.update', 1.5), ('apple/alarms.list', 1), ('apple/battery.get', 1), ('apple/contacts.list', 1), ('apple/device.get', 1), ('apple/maps.eta', 1), ('apple/maps.show', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1), ('builtin/EditFile', 1), ('builtin/Memory', 1), ('apple/alarms.cancel', 0.5), ('apple/contacts.delete', 0.5), ('apple/contacts.update', 0.5), ('builtin/AskUserQuestions', 0.5)]

BM25 · name＋description

[('apple/calendar.events.get', 15.43314), ('apple/calendar.events.delete', 8.27174), ('apple/calendar.events.update', 7.21625), ('apple/calendar.events.create', 6.22021), ('apple/calendar.events.list', 4.47923), ('apple/photos.search', 3.779), ('apple/contacts.list', 3.51562), ('apple/photos.get', 3.15144), ('apple/reminders.complete', 2.76956), ('apple/reminders.get', 2.69446), ('apple/maps.show', 2.63514), ('apple/reminders.delete', 2.57038), ('builtin/ScheduledTasks', 2.47394), ('apple/reminders.update', 2.43055), ('apple/reminders.create', 2.06068), ('apple/music.playback', 1.8781), ('builtin/ReadFile', 1.73569), ('apple/reminders.list', 1.72296), ('builtin/Memory', 1.62085), ('apple/contacts.get', 1.59613), ('builtin/EditFile', 1.36324), ('apple/clipboard.read', 1.34398), ('apple/health.workouts.get', 1.26659), ('apple/weather.current', 1.24126), ('apple/health.workouts.list', 1.21491), ('apple/device.get', 1.08242), ('apple/health.sleep.query', 1.08242), ('apple/battery.get', 1.06946), ('apple/maps.eta', 1.06946), ('apple/alarms.list', 0.95511), ('apple/weather.hourly', 0.8465), ('apple/weather.daily', 0.78685), ('apple/health.statistics', 0.71734)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.get', 26.8431), ('apple/calendar.events.delete', 15.30712), ('apple/calendar.events.update', 13.96499), ('apple/calendar.events.create', 12.33642), ('apple/calendar.events.list', 10.43791), ('builtin/ReadFile', 4.55729), ('apple/clipboard.read', 4.05468), ('apple/photos.search', 3.80205), ('apple/reminders.complete', 3.68148), ('apple/contacts.list', 3.50098), ('apple/reminders.delete', 3.47663), ('apple/reminders.update', 3.038), ('builtin/ScheduledTasks', 3.00628), ('apple/photos.get', 2.90849), ('apple/reminders.get', 2.36601), ('apple/contacts.get', 2.30505), ('apple/health.workouts.get', 1.98104), ('apple/health.workouts.list', 1.91439), ('apple/health.sleep.query', 1.84357), ('apple/music.playback', 1.7244), ('apple/reminders.create', 1.70201), ('apple/maps.show', 1.67694), ('builtin/Memory', 1.61114), ('apple/health.statistics', 1.39155), ('builtin/EditFile', 1.39066), ('apple/reminders.list', 1.30883), ('apple/weather.current', 1.23137), ('apple/device.get', 1.08501), ('apple/battery.get', 1.07116), ('apple/maps.eta', 1.07116), ('apple/alarms.list', 0.94991), ('apple/weather.hourly', 0.83632), ('apple/weather.daily', 0.77457), ('apple/alarms.cancel', 0.69591), ('apple/contacts.delete', 0.69591), ('builtin/AskUserQuestions', 0.65878), ('apple/contacts.update', 0.45326)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.get', 27.6494), ('apple/calendar.events.delete', 15.1319), ('apple/calendar.events.update', 13.81385), ('apple/calendar.events.create', 12.22679), ('apple/calendar.events.list', 11.01582), ('builtin/ScheduledTasks', 5.25954), ('apple/contacts.list', 4.42762), ('builtin/ReadFile', 4.20178), ('apple/photos.get', 3.65225), ('apple/photos.search', 3.5912), ('apple/reminders.complete', 3.50536), ('apple/reminders.get', 3.31701), ('apple/reminders.delete', 3.31419), ('apple/reminders.update', 2.88507), ('apple/clipboard.read', 2.86941), ('apple/contacts.get', 2.83844), ('apple/health.workouts.get', 2.50329), ('apple/health.workouts.list', 2.39064), ('apple/reminders.list', 2.19031), ('apple/device.get', 1.80027), ('apple/battery.get', 1.79046), ('apple/music.playback', 1.7244), ('apple/reminders.create', 1.70201), ('apple/maps.show', 1.67694), ('apple/alarms.list', 1.64891), ('apple/health.sleep.query', 1.52224), ('apple/health.statistics', 1.186), ('builtin/Memory', 1.14017), ('builtin/EditFile', 0.98414), ('apple/location.get', 0.97668), ('apple/photos.albums.list', 0.92664), ('apple/calendar.list', 0.9035), ('apple/weather.current', 0.87142), ('apple/maps.eta', 0.75804), ('apple/alarms.cancel', 0.69591), ('apple/contacts.delete', 0.69591), ('builtin/AskUserQuestions', 0.65878), ('apple/weather.hourly', 0.59185), ('apple/weather.daily', 0.54815), ('apple/contacts.update', 0.45326)]

H024 Fetch the distance and energy for this one workout UUID, not a workout list.

comparison · intent_transfer · target ['apple/health.workouts.get']

One workout detail includes recorded distance/energy.

MethodFirst acceptable rankTop5
Keyword matching1apple/health.workouts.get → apple/health.workouts.list → apple/alarms.list → apple/contacts.list → apple/photos.albums.list
BM25 · name＋description1apple/health.workouts.get → apple/health.workouts.list → apple/alarms.list → apple/alarms.cancel → apple/maps.eta
BM25 · name + description + Schema + native aliases1apple/health.workouts.get → apple/health.workouts.list → apple/alarms.list → apple/alarms.cancel → builtin/ListFiles
BM25 · name + description + Schema + extended aliases1apple/health.workouts.get → apple/health.workouts.list → apple/alarms.list → apple/alarms.cancel → builtin/ListFiles
Full rankings, within-method scores

Keyword matching

[('apple/health.workouts.get', 9.5), ('apple/health.workouts.list', 7.5), ('apple/alarms.list', 6), ('apple/contacts.list', 5.5), ('apple/photos.albums.list', 5.5), ('apple/reminders.list', 5), ('builtin/ListFiles', 5), ('apple/calendar.events.list', 4.5), ('apple/alarms.cancel', 4), ('apple/calendar.list', 4), ('apple/reminders.lists', 4), ('apple/alarms.create', 3), ('builtin/ScheduledTasks', 3), ('apple/calendar.events.delete', 2.5), ('apple/calendar.events.update', 2.5), ('apple/health.statistics', 2.5), ('apple/reminders.create', 2.5), ('builtin/Memory', 2.5), ('apple/clipboard.write', 2), ('apple/location.get', 2), ('apple/maps.eta', 2), ('apple/maps.route', 2), ('apple/photos.get', 2), ('apple/calendar.events.create', 1.5), ('apple/contacts.delete', 1.5), ('apple/contacts.update', 1.5), ('apple/health.sleep.query', 1.5), ('apple/links.open', 1.5), ('apple/maps.search', 1.5), ('apple/battery.get', 1), ('apple/calendar.events.get', 1), ('apple/contacts.create', 1), ('apple/location.geocode', 1), ('apple/maps.show', 1), ('apple/music.playback', 1), ('apple/music.search', 1), ('apple/music.status', 1), ('apple/photos.search', 1), ('apple/reminders.delete', 1), ('apple/shortcuts.run', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('builtin/EditFile', 1), ('apple/contacts.get', 0.5), ('builtin/ReadFile', 0.5)]

BM25 · name＋description

[('apple/health.workouts.get', 17.28629), ('apple/health.workouts.list', 6.19047), ('apple/alarms.list', 5.37606), ('apple/alarms.cancel', 4.69034), ('apple/maps.eta', 3.9694), ('apple/maps.route', 3.68782), ('builtin/ListFiles', 3.67255), ('apple/health.statistics', 3.46256), ('apple/alarms.create', 3.16787), ('apple/photos.albums.list', 2.70283), ('apple/contacts.list', 2.42221), ('apple/clipboard.write', 2.40955), ('apple/reminders.create', 2.39849), ('builtin/Memory', 2.32952), ('apple/calendar.events.update', 2.26433), ('apple/calendar.events.delete', 2.24967), ('apple/location.get', 2.22411), ('apple/photos.get', 2.20544), ('apple/reminders.lists', 2.1669), ('apple/calendar.list', 1.97043), ('apple/contacts.create', 1.66967), ('apple/calendar.events.list', 1.6149), ('apple/health.sleep.query', 1.56294), ('builtin/EditFile', 1.5025), ('apple/reminders.list', 1.49249), ('apple/contacts.update', 1.45931), ('apple/calendar.events.get', 1.458), ('apple/reminders.delete', 1.41606), ('apple/weather.current', 1.38488), ('builtin/ScheduledTasks', 1.37604), ('apple/contacts.delete', 1.33902), ('apple/music.search', 1.21623), ('apple/location.geocode', 1.07002), ('apple/battery.get', 1.01877), ('apple/music.status', 0.98345), ('apple/shortcuts.run', 0.97937), ('apple/calendar.events.create', 0.96973), ('apple/maps.show', 0.90984), ('apple/links.open', 0.85983), ('apple/maps.search', 0.85499), ('apple/music.playback', 0.77202), ('apple/weather.daily', 0.74955)]

BM25 · name + description + Schema + native aliases

[('apple/health.workouts.get', 28.6764), ('apple/health.workouts.list', 13.5303), ('apple/alarms.list', 6.9876), ('apple/alarms.cancel', 6.96666), ('builtin/ListFiles', 5.49063), ('apple/contacts.list', 4.59464), ('apple/photos.albums.list', 4.55728), ('apple/health.statistics', 4.39122), ('apple/reminders.list', 4.14438), ('apple/reminders.lists', 4.12289), ('apple/calendar.list', 3.8154), ('apple/maps.eta', 3.79384), ('apple/maps.route', 3.618), ('apple/calendar.events.list', 3.47608), ('builtin/ScheduledTasks', 2.75404), ('apple/alarms.create', 2.6003), ('apple/calendar.events.delete', 2.50055), ('apple/reminders.create', 2.39057), ('apple/calendar.events.update', 2.25216), ('builtin/Memory', 2.20518), ('apple/clipboard.write', 2.11876), ('apple/location.get', 1.98836), ('apple/photos.get', 1.92685), ('apple/contacts.delete', 1.88994), ('apple/contacts.update', 1.77545), ('apple/health.sleep.query', 1.62443), ('apple/contacts.create', 1.59734), ('builtin/EditFile', 1.45986), ('apple/reminders.delete', 1.36911), ('apple/calendar.events.create', 1.2309), ('apple/weather.current', 1.13096), ('apple/calendar.events.get', 1.12534), ('apple/maps.search', 1.09049), ('apple/music.search', 0.99308), ('apple/location.geocode', 0.88363), ('apple/links.open', 0.84079), ('apple/battery.get', 0.83852), ('apple/music.status', 0.8076), ('apple/maps.show', 0.74361), ('apple/shortcuts.run', 0.72406), ('apple/music.playback', 0.72007), ('apple/weather.daily', 0.60635), ('apple/contacts.get', 0.60095), ('apple/photos.search', 0.58223), ('builtin/ReadFile', 0.51852)]

BM25 · name + description + Schema + extended aliases

[('apple/health.workouts.get', 31.07156), ('apple/health.workouts.list', 13.5303), ('apple/alarms.list', 6.9876), ('apple/alarms.cancel', 6.96666), ('builtin/ListFiles', 6.69075), ('apple/contacts.list', 4.59464), ('apple/photos.get', 4.58786), ('apple/location.get', 4.57752), ('apple/photos.albums.list', 4.55728), ('apple/health.statistics', 4.39122), ('apple/reminders.list', 4.14438), ('apple/reminders.lists', 4.12289), ('apple/calendar.list', 3.8154), ('apple/maps.eta', 3.79384), ('apple/calendar.events.get', 3.7145), ('apple/maps.route', 3.618), ('apple/battery.get', 3.57547), ('apple/calendar.events.list', 3.47608), ('apple/contacts.get', 3.26195), ('builtin/ScheduledTasks', 2.75404), ('apple/device.get', 2.73695), ('apple/alarms.create', 2.6003), ('apple/reminders.get', 2.52109), ('apple/calendar.events.delete', 2.50055), ('apple/reminders.create', 2.39057), ('apple/calendar.events.update', 2.25216), ('builtin/Memory', 2.20518), ('apple/clipboard.write', 2.11876), ('apple/contacts.delete', 1.88994), ('apple/contacts.update', 1.77545), ('apple/health.sleep.query', 1.62443), ('apple/contacts.create', 1.59734), ('builtin/EditFile', 1.45986), ('apple/reminders.delete', 1.36911), ('apple/calendar.events.create', 1.2309), ('apple/weather.current', 1.13096), ('apple/maps.search', 1.09049), ('apple/music.search', 0.99308), ('apple/location.geocode', 0.88363), ('apple/links.open', 0.84079), ('apple/music.status', 0.8076), ('apple/maps.show', 0.74361), ('apple/shortcuts.run', 0.72406), ('apple/music.playback', 0.72007), ('apple/weather.daily', 0.60635), ('apple/photos.search', 0.58223), ('builtin/ReadFile', 0.51852)]

H025 先拿到日历ID列表，后续查询事件要用。

comparison · intent_transfer · target ['apple/calendar.list']

Calendar identifier enumeration as a distinct preparation capability.

MethodFirst acceptable rankTop5
Keyword matching6apple/calendar.events.get → apple/calendar.events.list → apple/calendar.events.create → apple/calendar.events.delete → apple/calendar.events.update
BM25 · name＋descriptionNot retrievedapple/reminders.get → apple/photos.get → apple/reminders.create → apple/music.playback → apple/calendar.events.delete
BM25 · name + description + Schema + native aliases6apple/calendar.events.get → apple/calendar.events.delete → apple/calendar.events.list → apple/calendar.events.update → apple/calendar.events.create
BM25 · name + description + Schema + extended aliases5apple/calendar.events.get → builtin/ListFiles → apple/calendar.events.delete → apple/calendar.events.list → apple/calendar.list
Full rankings, within-method scores

Keyword matching

[('apple/calendar.events.get', 3.5), ('apple/calendar.events.list', 3.5), ('apple/calendar.events.create', 3), ('apple/calendar.events.delete', 3), ('apple/calendar.events.update', 3), ('apple/calendar.list', 3), ('apple/photos.get', 3), ('apple/reminders.get', 3), ('apple/reminders.list', 3), ('apple/contacts.get', 2), ('apple/health.workouts.get', 2), ('apple/links.open', 2), ('apple/photos.search', 2), ('apple/alarms.list', 1.5), ('apple/battery.get', 1.5), ('apple/contacts.list', 1.5), ('apple/device.get', 1.5), ('apple/health.statistics', 1.5), ('apple/health.workouts.list', 1.5), ('apple/location.get', 1.5), ('apple/maps.search', 1.5), ('apple/music.playback', 1.5), ('apple/music.search', 1.5), ('apple/photos.albums.list', 1.5), ('apple/reminders.create', 1.5), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.cancel', 0.5), ('apple/contacts.delete', 0.5), ('apple/contacts.update', 0.5), ('apple/reminders.complete', 0.5), ('apple/reminders.delete', 0.5), ('apple/reminders.update', 0.5), ('builtin/AskUserQuestions', 0.5)]

BM25 · name＋description

[('apple/reminders.get', 2.69446), ('apple/photos.get', 2.25391), ('apple/reminders.create', 2.06068), ('apple/music.playback', 1.8781), ('apple/calendar.events.delete', 1.8017), ('apple/reminders.list', 1.72296), ('apple/calendar.events.create', 1.55325), ('apple/calendar.events.update', 1.48035), ('builtin/ScheduledTasks', 1.44059)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.get', 5.13535), ('apple/calendar.events.delete', 4.83842), ('apple/calendar.events.list', 4.46993), ('apple/calendar.events.update', 4.35137), ('apple/calendar.events.create', 4.22328), ('apple/calendar.list', 4.09281), ('apple/reminders.get', 3.69574), ('apple/photos.get', 3.61686), ('apple/reminders.list', 2.56811), ('apple/contacts.get', 2.29378), ('apple/photos.search', 2.11814), ('apple/health.workouts.get', 1.93612), ('apple/music.playback', 1.7244), ('apple/alarms.list', 1.71303), ('apple/battery.get', 1.71303), ('apple/device.get', 1.71303), ('apple/reminders.create', 1.70201), ('apple/music.search', 1.59788), ('apple/contacts.list', 1.49723), ('apple/location.get', 1.49723), ('apple/photos.albums.list', 1.49723), ('apple/maps.search', 1.40852), ('apple/health.statistics', 1.32973), ('apple/health.workouts.list', 1.19593), ('builtin/ScheduledTasks', 0.99196), ('apple/reminders.delete', 0.86744), ('apple/reminders.complete', 0.85247), ('apple/links.open', 0.79971), ('apple/alarms.cancel', 0.69591), ('apple/contacts.delete', 0.69591), ('builtin/AskUserQuestions', 0.65878), ('apple/reminders.update', 0.58151), ('apple/contacts.update', 0.45326)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.get', 5.59189), ('builtin/ListFiles', 5.25514), ('apple/calendar.events.delete', 5.08805), ('apple/calendar.events.list', 4.86669), ('apple/calendar.list', 4.48956), ('apple/reminders.get', 4.37382), ('apple/calendar.events.create', 4.20758), ('apple/photos.get', 4.10483), ('apple/calendar.events.update', 3.79814), ('apple/reminders.list', 3.20993), ('apple/contacts.get', 2.78176), ('apple/photos.search', 2.73526), ('apple/health.workouts.get', 2.61222), ('apple/music.search', 2.215), ('apple/battery.get', 2.12719), ('apple/device.get', 2.12719), ('apple/maps.search', 2.12719), ('apple/alarms.list', 2.04608), ('apple/location.get', 2.04608), ('apple/contacts.list', 1.97093), ('apple/photos.albums.list', 1.97093), ('apple/health.statistics', 1.84794), ('apple/health.workouts.list', 1.83606), ('apple/music.playback', 1.7244), ('apple/reminders.create', 1.70201), ('apple/links.open', 1.37203), ('builtin/ScheduledTasks', 0.99196), ('apple/reminders.delete', 0.86744), ('apple/reminders.complete', 0.85247), ('apple/alarms.cancel', 0.69591), ('apple/contacts.delete', 0.69591), ('builtin/AskUserQuestions', 0.65878), ('apple/reminders.update', 0.58151), ('apple/contacts.update', 0.45326)]

H026 Find at most three map place candidates in an explicit bounding rectangle.

comparison · intent_transfer · target ['apple/maps.search']

Bounded place search, not route directions.

MethodFirst acceptable rankTop5
Keyword matching1apple/maps.search → apple/maps.route → apple/maps.show → apple/maps.eta → apple/links.open
BM25 · name＋description1apple/maps.search → apple/maps.show → apple/location.reverseGeocode → apple/location.geocode → apple/maps.route
BM25 · name + description + Schema + native aliases1apple/maps.search → apple/maps.show → apple/maps.route → apple/maps.eta → apple/location.reverseGeocode
BM25 · name + description + Schema + extended aliases1apple/maps.search → apple/maps.show → apple/maps.route → apple/maps.eta → apple/location.reverseGeocode
Full rankings, within-method scores

Keyword matching

[('apple/maps.search', 9), ('apple/maps.route', 5), ('apple/maps.show', 5), ('apple/maps.eta', 4), ('apple/links.open', 2), ('apple/location.geocode', 2), ('apple/location.reverseGeocode', 2), ('apple/calendar.events.create', 1), ('apple/calendar.events.update', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1), ('apple/alarms.create', 0.5), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('apple/maps.search', 14.08447), ('apple/maps.show', 5.67918), ('apple/location.reverseGeocode', 5.26573), ('apple/location.geocode', 4.86844), ('apple/maps.route', 4.85127), ('apple/maps.eta', 4.36784), ('apple/calendar.events.create', 1.38598), ('apple/weather.current', 1.3592), ('apple/weather.hourly', 1.33344), ('apple/calendar.events.update', 1.32092), ('apple/weather.daily', 1.23947)]

BM25 · name + description + Schema + native aliases

[('apple/maps.search', 22.10372), ('apple/maps.show', 11.33221), ('apple/maps.route', 10.08336), ('apple/maps.eta', 8.22863), ('apple/location.reverseGeocode', 5.29756), ('apple/location.geocode', 4.79878), ('apple/links.open', 1.86088), ('apple/calendar.events.create', 1.31346), ('apple/weather.current', 1.2734), ('apple/calendar.events.update', 1.24802), ('apple/weather.hourly', 1.24802), ('apple/weather.daily', 1.15587), ('builtin/ScheduledTasks', 1.09746), ('apple/alarms.create', 0.85051)]

BM25 · name + description + Schema + extended aliases

[('apple/maps.search', 23.71164), ('apple/maps.show', 11.33221), ('apple/maps.route', 10.08336), ('apple/maps.eta', 8.22863), ('apple/location.reverseGeocode', 5.29756), ('apple/location.geocode', 4.79878), ('apple/links.open', 2.81918), ('apple/music.search', 1.70526), ('apple/photos.search', 1.70526), ('apple/battery.get', 1.60792), ('apple/device.get', 1.60792), ('apple/contacts.get', 1.5633), ('apple/photos.get', 1.5633), ('apple/alarms.list', 1.5211), ('apple/calendar.events.get', 1.5211), ('apple/location.get', 1.5211), ('apple/reminders.get', 1.48111), ('apple/contacts.list', 1.44317), ('apple/photos.albums.list', 1.44317), ('apple/calendar.events.list', 1.40712), ('apple/calendar.list', 1.40712), ('apple/health.workouts.get', 1.40712), ('apple/reminders.list', 1.37284), ('apple/calendar.events.create', 1.31346), ('apple/health.workouts.list', 1.30904), ('apple/weather.current', 1.2734), ('apple/calendar.events.update', 1.24802), ('apple/weather.hourly', 1.24802), ('apple/weather.daily', 1.15587), ('builtin/ScheduledTasks', 1.09746), ('apple/alarms.create', 0.85051)]

H027 在剪贴板写入一个空字符串文本项，不承诺清除所有格式。

comparison · intent_transfer · target ['apple/clipboard.write']

Empty plain-text write is supported; complete clearing not claimed.

MethodFirst acceptable rankTop5
Keyword matching2apple/clipboard.read → apple/clipboard.write
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliases2apple/clipboard.read → apple/clipboard.write
BM25 · name + description + Schema + extended aliases2apple/clipboard.read → apple/clipboard.write → builtin/WriteFile → builtin/ReadFile
Full rankings, within-method scores

Keyword matching

[('apple/clipboard.read', 1.5), ('apple/clipboard.write', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/clipboard.read', 5.8987), ('apple/clipboard.write', 5.8987)]

BM25 · name + description + Schema + extended aliases

[('apple/clipboard.read', 6.86453), ('apple/clipboard.write', 6.86453), ('builtin/WriteFile', 4.38831), ('builtin/ReadFile', 4.26983)]

H028 Read a current foreground location sample including its accuracy and sample time.

comparison · intent_transfer · target ['apple/location.get']

Current sample with accuracy/time, approximate authorization accepted.

MethodFirst acceptable rankTop5
Keyword matching1apple/location.get → builtin/GetCurrentTime → apple/clipboard.read → apple/weather.current → apple/location.geocode
BM25 · name＋description1apple/location.get → apple/health.workouts.get → apple/weather.daily → apple/maps.eta → apple/battery.get
BM25 · name + description + Schema + native aliases1apple/location.get → builtin/GetCurrentTime → apple/health.workouts.get → apple/weather.current → apple/clipboard.read
BM25 · name + description + Schema + extended aliases1apple/location.get → builtin/GetCurrentTime → apple/health.workouts.get → apple/location.geocode → apple/weather.current
Full rankings, within-method scores

Keyword matching

[('apple/location.get', 8), ('builtin/GetCurrentTime', 8), ('apple/clipboard.read', 6), ('apple/weather.current', 6), ('apple/location.geocode', 5), ('apple/location.reverseGeocode', 5), ('apple/maps.eta', 4), ('apple/weather.daily', 4), ('apple/weather.hourly', 4), ('builtin/ReadFile', 4), ('apple/health.statistics', 3.5), ('apple/battery.get', 3), ('apple/health.workouts.get', 3), ('apple/maps.route', 3), ('apple/maps.search', 3), ('apple/maps.show', 3), ('apple/music.status', 3), ('apple/photos.get', 3), ('apple/photos.search', 3), ('apple/calendar.events.list', 2.5), ('builtin/EditFile', 2.5), ('builtin/ScheduledTasks', 2.5), ('apple/calendar.events.update', 2), ('apple/device.get', 2), ('apple/music.playback', 2), ('apple/alarms.create', 1.5), ('apple/health.sleep.query', 1.5), ('apple/health.workouts.list', 1.5), ('apple/reminders.list', 1.5), ('apple/alarms.cancel', 1), ('apple/alarms.list', 1), ('apple/calendar.events.create', 1), ('apple/calendar.events.get', 1), ('apple/contacts.get', 1), ('apple/links.open', 1), ('apple/music.search', 1), ('apple/photos.albums.list', 1), ('apple/shortcuts.run', 1), ('builtin/Memory', 1), ('apple/reminders.create', 0.5), ('apple/reminders.update', 0.5)]

BM25 · name＋description

[('apple/location.get', 13.52377), ('apple/health.workouts.get', 7.93889), ('apple/weather.daily', 6.47049), ('apple/maps.eta', 6.42279), ('apple/battery.get', 5.53411), ('apple/maps.search', 5.26609), ('builtin/GetCurrentTime', 5.19447), ('apple/clipboard.read', 5.13291), ('apple/weather.hourly', 5.1279), ('apple/weather.current', 4.66747), ('apple/music.status', 4.56934), ('apple/maps.route', 4.22735), ('apple/location.reverseGeocode', 4.09171), ('apple/maps.show', 3.98849), ('apple/health.statistics', 3.90117), ('apple/location.geocode', 3.87647), ('apple/photos.get', 3.43877), ('apple/calendar.events.get', 3.31895), ('builtin/EditFile', 3.19623), ('apple/calendar.events.list', 2.92087), ('apple/photos.search', 2.81624), ('apple/device.get', 2.61478), ('apple/music.playback', 1.93347), ('builtin/ReadFile', 1.73569), ('apple/reminders.list', 1.62576), ('builtin/Memory', 1.62085), ('apple/contacts.get', 1.59613), ('apple/alarms.cancel', 1.57416), ('apple/alarms.create', 1.45133), ('apple/music.search', 1.27062), ('apple/shortcuts.run', 1.25798), ('apple/health.workouts.list', 1.21491), ('apple/calendar.events.update', 1.18713), ('apple/photos.albums.list', 1.18713), ('apple/health.sleep.query', 1.08242), ('builtin/ScheduledTasks', 1.03334), ('apple/alarms.list', 0.95511), ('apple/links.open', 0.80278)]

BM25 · name + description + Schema + native aliases

[('apple/location.get', 16.52271), ('builtin/GetCurrentTime', 10.07522), ('apple/health.workouts.get', 8.17466), ('apple/weather.current', 7.93244), ('apple/clipboard.read', 7.82539), ('apple/location.geocode', 7.6123), ('apple/location.reverseGeocode', 7.41338), ('apple/weather.daily', 6.21511), ('apple/battery.get', 5.5429), ('apple/maps.eta', 5.47112), ('apple/maps.search', 5.07441), ('builtin/ReadFile', 4.55729), ('apple/weather.hourly', 4.4588), ('apple/health.statistics', 4.41289), ('apple/music.status', 3.91622), ('apple/maps.show', 3.83621), ('builtin/EditFile', 3.74466), ('apple/maps.route', 3.53838), ('apple/calendar.events.get', 3.43216), ('apple/photos.get', 3.34478), ('apple/calendar.events.list', 2.98393), ('apple/photos.search', 2.71232), ('apple/device.get', 2.62104), ('apple/calendar.events.update', 2.4332), ('apple/health.workouts.list', 1.86769), ('apple/music.playback', 1.84784), ('apple/health.sleep.query', 1.77202), ('builtin/ScheduledTasks', 1.73698), ('builtin/Memory', 1.61114), ('apple/contacts.get', 1.60914), ('apple/alarms.create', 1.57832), ('apple/reminders.list', 1.51429), ('apple/calendar.events.create', 1.36503), ('apple/music.search', 1.25926), ('apple/shortcuts.run', 1.24605), ('apple/photos.albums.list', 1.18397), ('apple/alarms.cancel', 1.03523), ('apple/alarms.list', 0.94991), ('apple/reminders.create', 0.86688), ('apple/reminders.update', 0.86688), ('apple/links.open', 0.77985)]

BM25 · name + description + Schema + extended aliases

[('apple/location.get', 17.49939), ('builtin/GetCurrentTime', 13.54792), ('apple/health.workouts.get', 8.69691), ('apple/location.geocode', 7.6123), ('apple/weather.current', 7.57248), ('apple/location.reverseGeocode', 7.41338), ('apple/clipboard.read', 6.64012), ('apple/battery.get', 6.26221), ('apple/weather.daily', 5.98869), ('apple/maps.eta', 5.158), ('apple/maps.search', 5.07441), ('apple/calendar.events.get', 4.40884), ('apple/weather.hourly', 4.21433), ('apple/health.statistics', 4.20734), ('builtin/ReadFile', 4.20178), ('apple/photos.get', 4.08853), ('apple/music.status', 3.91622), ('apple/maps.show', 3.83621), ('apple/calendar.events.list', 3.56184), ('apple/maps.route', 3.53838), ('builtin/EditFile', 3.33814), ('apple/device.get', 3.3363), ('apple/photos.search', 2.50146), ('apple/calendar.events.update', 2.4332), ('apple/reminders.list', 2.39578), ('apple/health.workouts.list', 2.34395), ('apple/contacts.get', 2.14254), ('apple/photos.albums.list', 2.11061), ('apple/music.playback', 1.84784), ('apple/alarms.list', 1.64891), ('apple/alarms.create', 1.57832), ('apple/health.sleep.query', 1.45069), ('builtin/ScheduledTasks', 1.40109), ('apple/calendar.events.create', 1.36503), ('apple/music.search', 1.25926), ('apple/shortcuts.run', 1.24605), ('builtin/Memory', 1.14017), ('apple/alarms.cancel', 1.03523), ('apple/reminders.get', 0.951), ('apple/contacts.list', 0.92664), ('apple/calendar.list', 0.9035), ('apple/reminders.create', 0.86688), ('apple/reminders.update', 0.86688), ('apple/links.open', 0.77985)]

H029 Find catalog songs for a nonblank term, returning only the first bounded slice.

comparison · intent_transfer · target ['apple/music.search']

First catalog slice, not subscription or playback operation.

MethodFirst acceptable rankTop5
Keyword matching1apple/music.search → apple/music.playback → apple/links.open → apple/photos.search → apple/alarms.create
BM25 · name＋description1apple/music.search → apple/music.playback → builtin/EditFile → apple/photos.search → apple/links.open
BM25 · name + description + Schema + native aliases1apple/music.search → apple/music.playback → builtin/EditFile → apple/photos.search → apple/links.open
BM25 · name + description + Schema + extended aliases1apple/music.search → apple/music.playback → apple/photos.search → builtin/Memory → builtin/EditFile
Full rankings, within-method scores

Keyword matching

[('apple/music.search', 8.5), ('apple/music.playback', 4.5), ('apple/links.open', 2), ('apple/photos.search', 2), ('apple/alarms.create', 1.5), ('apple/calendar.events.update', 1.5), ('apple/maps.show', 1.5), ('apple/shortcuts.run', 1.5), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.cancel', 1), ('apple/alarms.list', 1), ('apple/contacts.create', 1), ('apple/contacts.update', 1), ('apple/maps.search', 1), ('apple/music.status', 1), ('apple/photos.albums.list', 1), ('apple/photos.get', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1), ('builtin/EditFile', 1), ('apple/location.geocode', 0.5), ('apple/reminders.create', 0.5), ('apple/reminders.update', 0.5), ('builtin/Memory', 0.5), ('builtin/ReadFile', 0.5)]

BM25 · name＋description

[('apple/music.search', 19.78757), ('apple/music.playback', 7.85662), ('builtin/EditFile', 3.76118), ('apple/photos.search', 3.51144), ('apple/links.open', 3.07636), ('apple/contacts.create', 2.89084), ('apple/weather.hourly', 2.61891), ('apple/weather.daily', 2.43436), ('apple/contacts.update', 2.37094), ('apple/maps.search', 2.25182), ('apple/music.status', 1.8925), ('apple/shortcuts.run', 1.7045), ('apple/photos.albums.list', 1.63489), ('builtin/ScheduledTasks', 1.38941), ('apple/alarms.list', 1.28422), ('apple/maps.show', 1.28422), ('apple/alarms.cancel', 1.27063), ('apple/photos.get', 1.2068), ('apple/alarms.create', 1.17149), ('apple/calendar.events.update', 1.1275)]

BM25 · name + description + Schema + native aliases

[('apple/music.search', 25.76868), ('apple/music.playback', 12.40798), ('builtin/EditFile', 3.48903), ('apple/photos.search', 3.20006), ('apple/links.open', 2.92964), ('apple/weather.hourly', 2.58742), ('apple/weather.daily', 2.39637), ('apple/maps.show', 2.30357), ('apple/shortcuts.run', 2.25265), ('apple/alarms.create', 1.99973), ('apple/contacts.create', 1.82727), ('apple/music.status', 1.79823), ('apple/calendar.events.update', 1.62473), ('builtin/ScheduledTasks', 1.60674), ('apple/photos.albums.list', 1.5512), ('apple/contacts.update', 1.4796), ('apple/maps.search', 1.40119), ('builtin/ReadFile', 1.23924), ('apple/alarms.list', 1.21419), ('apple/alarms.cancel', 1.2006), ('apple/photos.get', 1.13698), ('apple/location.geocode', 0.93908), ('apple/reminders.create', 0.66419), ('apple/reminders.update', 0.66419), ('builtin/Memory', 0.38765)]

BM25 · name + description + Schema + extended aliases

[('apple/music.search', 26.58207), ('apple/music.playback', 12.40798), ('apple/photos.search', 4.90532), ('builtin/Memory', 4.43873), ('builtin/EditFile', 3.48903), ('apple/maps.search', 3.00911), ('apple/photos.albums.list', 2.99437), ('apple/links.open', 2.92964), ('apple/alarms.list', 2.73529), ('apple/photos.get', 2.70028), ('apple/weather.hourly', 2.58742), ('apple/weather.daily', 2.39637), ('apple/maps.show', 2.30357), ('apple/shortcuts.run', 2.25265), ('apple/alarms.create', 1.99973), ('apple/contacts.create', 1.82727), ('apple/music.status', 1.79823), ('apple/calendar.events.update', 1.62473), ('apple/battery.get', 1.60792), ('apple/device.get', 1.60792), ('builtin/ScheduledTasks', 1.60674), ('apple/contacts.get', 1.5633), ('apple/calendar.events.get', 1.5211), ('apple/location.get', 1.5211), ('apple/reminders.get', 1.48111), ('apple/contacts.update', 1.4796), ('apple/contacts.list', 1.44317), ('apple/calendar.events.list', 1.40712), ('apple/calendar.list', 1.40712), ('apple/health.workouts.get', 1.40712), ('apple/reminders.list', 1.37284), ('apple/health.workouts.list', 1.30904), ('builtin/ReadFile', 1.23924), ('apple/alarms.cancel', 1.2006), ('apple/location.geocode', 0.93908), ('apple/reminders.create', 0.66419), ('apple/reminders.update', 0.66419)]

H030 Cancel one alarm owned by Anan; do not cancel alarms from other apps.

comparison · intent_transfer · target ['apple/alarms.cancel']

Cancellation scope expressly limited to Anan-owned identifiers.

MethodFirst acceptable rankTop5
Keyword matching1apple/alarms.cancel → apple/alarms.create → apple/links.open → apple/alarms.list → apple/music.status
BM25 · name＋description1apple/alarms.cancel → apple/alarms.create → apple/alarms.list → apple/music.status → apple/links.open
BM25 · name + description + Schema + native aliases1apple/alarms.cancel → apple/alarms.create → apple/alarms.list → apple/links.open → apple/music.status
BM25 · name + description + Schema + extended aliases1apple/alarms.cancel → apple/alarms.create → apple/alarms.list → apple/links.open → apple/music.status
Full rankings, within-method scores

Keyword matching

[('apple/alarms.cancel', 11.5), ('apple/alarms.create', 9), ('apple/links.open', 7.5), ('apple/alarms.list', 7), ('apple/music.status', 4), ('apple/music.playback', 3), ('apple/health.sleep.query', 2.5), ('apple/photos.albums.list', 2.5), ('apple/calendar.events.create', 2), ('apple/clipboard.write', 2), ('apple/contacts.delete', 2), ('apple/health.workouts.list', 2), ('apple/location.get', 2), ('apple/photos.get', 2), ('apple/reminders.create', 2), ('builtin/ScheduledTasks', 2), ('apple/contacts.list', 1.5), ('apple/maps.search', 1.5), ('apple/battery.get', 1), ('apple/calendar.events.delete', 1), ('apple/calendar.events.update', 1), ('apple/clipboard.read', 1), ('apple/contacts.create', 1), ('apple/contacts.update', 1), ('apple/health.statistics', 1), ('apple/health.workouts.get', 1), ('apple/location.geocode', 1), ('apple/maps.eta', 1), ('apple/maps.route', 1), ('apple/maps.show', 1), ('apple/music.search', 1), ('apple/reminders.delete', 1), ('apple/shortcuts.run', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('builtin/AskUserQuestions', 1), ('builtin/EditFile', 1), ('builtin/ListFiles', 1), ('builtin/Memory', 1), ('apple/calendar.events.list', 0.5), ('apple/photos.search', 0.5), ('apple/reminders.list', 0.5), ('builtin/ReadFile', 0.5)]

BM25 · name＋description

[('apple/alarms.cancel', 14.86948), ('apple/alarms.create', 12.013), ('apple/alarms.list', 9.85903), ('apple/music.status', 7.81681), ('apple/links.open', 7.17612), ('apple/music.playback', 4.63006), ('apple/health.sleep.query', 4.54932), ('apple/reminders.create', 3.80749), ('apple/contacts.delete', 3.76956), ('apple/photos.albums.list', 3.67626), ('apple/calendar.events.create', 3.00149), ('apple/health.workouts.list', 2.80551), ('builtin/AskUserQuestions', 2.49955), ('apple/clipboard.write', 2.40955), ('apple/location.get', 2.22411), ('apple/photos.get', 2.20544), ('apple/shortcuts.run', 2.05198), ('apple/clipboard.read', 1.93144), ('builtin/ListFiles', 1.68221), ('apple/contacts.create', 1.66967), ('builtin/ScheduledTasks', 1.63209), ('builtin/EditFile', 1.5025), ('apple/contacts.update', 1.45931), ('apple/reminders.delete', 1.41606), ('apple/health.workouts.get', 1.39598), ('apple/weather.current', 1.38488), ('apple/calendar.events.update', 1.34012), ('apple/health.statistics', 1.24325), ('apple/music.search', 1.21623), ('builtin/Memory', 1.16476), ('apple/calendar.events.delete', 1.12484), ('apple/location.geocode', 1.07002), ('apple/maps.route', 1.05267), ('apple/battery.get', 1.01877), ('apple/maps.eta', 1.01877), ('apple/contacts.list', 0.92972), ('apple/maps.show', 0.90984), ('apple/maps.search', 0.85499), ('apple/weather.daily', 0.74955)]

BM25 · name + description + Schema + native aliases

[('apple/alarms.cancel', 34.04418), ('apple/alarms.create', 18.7069), ('apple/alarms.list', 16.5392), ('apple/links.open', 9.00029), ('apple/music.status', 7.63617), ('apple/health.sleep.query', 4.65717), ('apple/music.playback', 4.51016), ('apple/health.workouts.list', 4.04804), ('apple/photos.albums.list', 3.92678), ('apple/contacts.delete', 3.74548), ('apple/reminders.create', 3.63702), ('apple/calendar.events.create', 2.9565), ('builtin/AskUserQuestions', 2.53834), ('apple/clipboard.write', 2.11876), ('apple/shortcuts.run', 2.03253), ('apple/location.get', 1.98836), ('apple/photos.get', 1.92685), ('apple/clipboard.read', 1.92215), ('builtin/ListFiles', 1.65142), ('apple/contacts.create', 1.59734), ('builtin/ScheduledTasks', 1.49695), ('builtin/EditFile', 1.45986), ('apple/contacts.update', 1.38404), ('apple/health.workouts.get', 1.36911), ('apple/reminders.delete', 1.36911), ('apple/calendar.events.update', 1.27396), ('apple/contacts.list', 1.25), ('apple/weather.current', 1.13096), ('builtin/Memory', 1.09647), ('apple/maps.search', 1.09049), ('apple/calendar.events.delete', 1.083), ('apple/health.statistics', 1.00933), ('apple/maps.route', 0.99719), ('apple/music.search', 0.99308), ('apple/location.geocode', 0.88363), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/maps.show', 0.74361), ('apple/weather.daily', 0.60635), ('builtin/ReadFile', 0.51852), ('apple/reminders.list', 0.31471), ('apple/calendar.events.list', 0.30991), ('apple/photos.search', 0.2839)]

BM25 · name + description + Schema + extended aliases

[('apple/alarms.cancel', 41.11572), ('apple/alarms.create', 18.7069), ('apple/alarms.list', 16.5392), ('apple/links.open', 9.66089), ('apple/music.status', 7.63617), ('apple/health.sleep.query', 4.65717), ('apple/music.playback', 4.51016), ('apple/health.workouts.list', 4.04804), ('apple/photos.albums.list', 3.92678), ('apple/contacts.delete', 3.74548), ('apple/reminders.create', 3.63702), ('apple/calendar.events.create', 2.9565), ('builtin/AskUserQuestions', 2.53834), ('apple/clipboard.write', 2.11876), ('apple/shortcuts.run', 2.03253), ('apple/location.get', 1.98836), ('apple/photos.get', 1.92685), ('apple/clipboard.read', 1.92215), ('builtin/ListFiles', 1.65142), ('apple/contacts.create', 1.59734), ('builtin/ScheduledTasks', 1.49695), ('builtin/EditFile', 1.45986), ('apple/contacts.update', 1.38404), ('apple/health.workouts.get', 1.36911), ('apple/reminders.delete', 1.36911), ('apple/calendar.events.update', 1.27396), ('apple/contacts.list', 1.25), ('apple/weather.current', 1.13096), ('builtin/Memory', 1.09647), ('apple/maps.search', 1.09049), ('apple/calendar.events.delete', 1.083), ('apple/health.statistics', 1.00933), ('apple/maps.route', 0.99719), ('apple/music.search', 0.99308), ('apple/location.geocode', 0.88363), ('apple/battery.get', 0.83852), ('apple/maps.eta', 0.83852), ('apple/maps.show', 0.74361), ('apple/weather.daily', 0.60635), ('builtin/ReadFile', 0.51852), ('apple/reminders.list', 0.31471), ('apple/calendar.events.list', 0.30991), ('apple/photos.search', 0.2839)]

H031 请把说明文档第41行到第60行的文字取出来。

comparison · wording_transfer · target ['builtin/ReadFile']

UTF-8 line offset/limit reader.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliases1builtin/ReadFile → builtin/WriteFile → builtin/EditFile
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[('builtin/ReadFile', 8.77911), ('builtin/WriteFile', 3.91339), ('builtin/EditFile', 3.80773)]

H032 Peek inside plans/todo.md and show its text.

comparison · wording_transfer · target ['builtin/ReadFile']

Ordinary text contents, not just directory names.

MethodFirst acceptable rankTop5
Keyword matching12apple/maps.show → apple/clipboard.read → apple/clipboard.write → apple/shortcuts.run → builtin/EditFile
BM25 · name＋description6apple/clipboard.read → apple/maps.show → apple/calendar.events.get → apple/health.workouts.get → apple/clipboard.write
BM25 · name + description + Schema + native aliases8apple/maps.show → apple/clipboard.read → apple/clipboard.write → builtin/EditFile → apple/calendar.events.get
BM25 · name + description + Schema + extended aliases4apple/maps.show → apple/clipboard.read → builtin/EditFile → builtin/ReadFile → apple/clipboard.write
Full rankings, within-method scores

Keyword matching

[('apple/maps.show', 4), ('apple/clipboard.read', 3), ('apple/clipboard.write', 1.5), ('apple/shortcuts.run', 1.5), ('builtin/EditFile', 1.5), ('apple/calendar.events.get', 1), ('apple/contacts.update', 1), ('apple/health.workouts.get', 1), ('apple/maps.search', 1), ('builtin/AskUserQuestions', 1), ('builtin/Memory', 1), ('builtin/ReadFile', 1)]

BM25 · name＋description

[('apple/clipboard.read', 8.50205), ('apple/maps.show', 4.12697), ('apple/calendar.events.get', 3.31895), ('apple/health.workouts.get', 3.17777), ('apple/clipboard.write', 3.14747), ('builtin/ReadFile', 2.85815), ('builtin/EditFile', 2.72729), ('apple/maps.search', 2.25182), ('builtin/AskUserQuestions', 2.16548), ('apple/contacts.update', 1.89057), ('apple/shortcuts.run', 1.77773)]

BM25 · name + description + Schema + native aliases

[('apple/maps.show', 12.51078), ('apple/clipboard.read', 8.27602), ('apple/clipboard.write', 4.5613), ('builtin/EditFile', 4.13067), ('apple/calendar.events.get', 3.43216), ('apple/health.workouts.get', 3.27215), ('apple/shortcuts.run', 2.973), ('builtin/ReadFile', 2.74541), ('builtin/Memory', 2.44876), ('apple/maps.search', 2.23169), ('builtin/AskUserQuestions', 2.06218), ('apple/contacts.update', 1.76213)]

BM25 · name + description + Schema + extended aliases

[('apple/maps.show', 7.13877), ('apple/clipboard.read', 6.82898), ('builtin/EditFile', 6.32338), ('builtin/ReadFile', 5.01984), ('apple/clipboard.write', 4.2922), ('apple/calendar.events.get', 3.43216), ('apple/health.workouts.get', 3.27215), ('apple/shortcuts.run', 2.7976), ('builtin/WriteFile', 2.50401), ('apple/alarms.list', 2.43641), ('builtin/Memory', 2.36874), ('apple/contacts.list', 2.31158), ('apple/photos.albums.list', 2.31158), ('apple/calendar.events.list', 2.25385), ('apple/calendar.list', 2.25385), ('apple/maps.search', 2.23169), ('apple/reminders.list', 2.19893), ('apple/health.workouts.list', 2.09674), ('builtin/AskUserQuestions', 1.94052), ('apple/contacts.update', 1.65817)]

H033 Read just the opening lines of a workspace text document.

comparison · wording_transfer · target ['builtin/ReadFile']

ReadFile supports bounded lines.

MethodFirst acceptable rankTop5
Keyword matching1builtin/ReadFile → apple/clipboard.read → builtin/EditFile → apple/clipboard.write → apple/links.open
BM25 · name＋description1builtin/ReadFile → apple/clipboard.read → builtin/EditFile → apple/links.open → apple/clipboard.write
BM25 · name + description + Schema + native aliases1builtin/ReadFile → apple/clipboard.read → builtin/EditFile → apple/clipboard.write → apple/links.open
BM25 · name + description + Schema + extended aliases1builtin/ReadFile → builtin/EditFile → builtin/WriteFile → apple/clipboard.read → apple/links.open
Full rankings, within-method scores

Keyword matching

[('builtin/ReadFile', 7), ('apple/clipboard.read', 5), ('builtin/EditFile', 3), ('apple/clipboard.write', 1.5), ('apple/links.open', 1.5), ('apple/shortcuts.run', 1.5), ('builtin/Memory', 1.5), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.list', 1), ('apple/battery.get', 1), ('apple/calendar.events.list', 1), ('apple/contacts.get', 1), ('apple/contacts.update', 1), ('apple/device.get', 1), ('apple/health.sleep.query', 1), ('apple/health.statistics', 1), ('apple/health.workouts.get', 1), ('apple/health.workouts.list', 1), ('apple/maps.eta', 1), ('apple/photos.get', 1), ('apple/photos.search', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1), ('builtin/AskUserQuestions', 1), ('builtin/ListFiles', 0.5), ('builtin/WriteFile', 0.5)]

BM25 · name＋description

[('builtin/ReadFile', 9.74005), ('apple/clipboard.read', 4.43697), ('builtin/EditFile', 4.09054), ('apple/links.open', 4.05022), ('apple/clipboard.write', 3.14747), ('builtin/AskUserQuestions', 2.16548), ('apple/contacts.update', 1.89057), ('apple/shortcuts.run', 1.77773), ('builtin/Memory', 1.62085), ('apple/contacts.get', 1.59613), ('apple/health.workouts.get', 1.26659), ('apple/weather.current', 1.24126), ('apple/health.workouts.list', 1.21491), ('apple/calendar.events.list', 1.0957), ('apple/device.get', 1.08242), ('apple/health.sleep.query', 1.08242), ('apple/battery.get', 1.06946), ('apple/maps.eta', 1.06946), ('builtin/ScheduledTasks', 1.03334), ('apple/alarms.list', 0.95511), ('apple/photos.get', 0.89753), ('apple/weather.hourly', 0.8465), ('apple/weather.daily', 0.78685), ('apple/photos.search', 0.73505), ('apple/health.statistics', 0.71734)]

BM25 · name + description + Schema + native aliases

[('builtin/ReadFile', 16.27281), ('apple/clipboard.read', 6.94765), ('builtin/EditFile', 6.65279), ('apple/clipboard.write', 4.5613), ('apple/links.open', 4.35189), ('apple/shortcuts.run', 2.973), ('builtin/Memory', 2.96747), ('builtin/AskUserQuestions', 2.06218), ('apple/contacts.update', 1.76213), ('apple/contacts.get', 1.60914), ('builtin/ListFiles', 1.36971), ('apple/health.workouts.get', 1.30421), ('apple/health.workouts.list', 1.24611), ('apple/weather.current', 1.23137), ('builtin/WriteFile', 1.19376), ('builtin/ScheduledTasks', 1.14906), ('apple/calendar.events.list', 1.11381), ('apple/health.sleep.query', 1.09922), ('apple/device.get', 1.08501), ('apple/battery.get', 1.07116), ('apple/maps.eta', 1.07116), ('apple/alarms.list', 0.94991), ('apple/photos.get', 0.88951), ('apple/weather.hourly', 0.83632), ('apple/weather.daily', 0.77457), ('apple/photos.search', 0.72131), ('apple/health.statistics', 0.70318)]

BM25 · name + description + Schema + extended aliases

[('builtin/ReadFile', 26.97083), ('builtin/EditFile', 12.24672), ('builtin/WriteFile', 7.61116), ('apple/clipboard.read', 5.59171), ('apple/links.open', 4.35189), ('apple/clipboard.write', 4.2922), ('apple/shortcuts.run', 2.7976), ('builtin/Memory', 2.41648), ('apple/contacts.get', 2.14254), ('builtin/AskUserQuestions', 1.94052), ('apple/health.workouts.get', 1.82646), ('apple/device.get', 1.80027), ('apple/battery.get', 1.79046), ('apple/health.workouts.list', 1.72237), ('apple/calendar.events.list', 1.69172), ('apple/contacts.update', 1.65817), ('apple/alarms.list', 1.64891), ('apple/photos.get', 1.63326), ('builtin/ListFiles', 1.36971), ('apple/calendar.events.get', 0.97668), ('apple/location.get', 0.97668), ('apple/reminders.get', 0.951), ('apple/contacts.list', 0.92664), ('apple/photos.albums.list', 0.92664), ('apple/calendar.list', 0.9035), ('apple/reminders.list', 0.88148), ('apple/weather.current', 0.87142), ('builtin/ScheduledTasks', 0.81317), ('apple/health.sleep.query', 0.77789), ('apple/maps.eta', 0.75804), ('apple/weather.hourly', 0.59185), ('apple/weather.daily', 0.54815), ('apple/photos.search', 0.51046), ('apple/health.statistics', 0.49762)]

H034 把这份纪要另存为一个新的UTF-8文档。

comparison · wording_transfer · target ['builtin/WriteFile']

Create a new complete UTF-8 file.

MethodFirst acceptable rankTop5
Keyword matching2builtin/ReadFile → builtin/WriteFile
BM25 · name＋description2builtin/ReadFile → builtin/WriteFile
BM25 · name + description + Schema + native aliases2builtin/ReadFile → builtin/WriteFile
BM25 · name + description + Schema + extended aliases2builtin/ReadFile → builtin/WriteFile → builtin/EditFile
Full rankings, within-method scores

Keyword matching

[('builtin/ReadFile', 2), ('builtin/WriteFile', 2)]

BM25 · name＋description

[('builtin/ReadFile', 8.83998), ('builtin/WriteFile', 8.70082)]

BM25 · name + description + Schema + native aliases

[('builtin/ReadFile', 9.05502), ('builtin/WriteFile', 8.89982)]

BM25 · name + description + Schema + extended aliases

[('builtin/ReadFile', 12.86275), ('builtin/WriteFile', 12.81322), ('builtin/EditFile', 3.80773)]

H035 Persist the whole draft under a fresh filename.

comparison · wording_transfer · target ['builtin/WriteFile']

Whole new file, not one-segment patch.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedapple/links.open → builtin/ScheduledTasks → apple/calendar.events.delete → apple/contacts.list → apple/maps.search
BM25 · name＋descriptionNot retrievedapple/reminders.delete → apple/calendar.events.delete → apple/contacts.list → builtin/ScheduledTasks → apple/maps.search
BM25 · name + description + Schema + native aliasesNot retrievedapple/reminders.delete → apple/calendar.events.delete → apple/contacts.list → builtin/ScheduledTasks → apple/maps.search
BM25 · name + description + Schema + extended aliasesNot retrievedbuiltin/ListFiles → apple/reminders.delete → apple/calendar.events.delete → apple/contacts.list → builtin/ScheduledTasks
Full rankings, within-method scores

Keyword matching

[('apple/links.open', 2), ('builtin/ScheduledTasks', 1.5), ('apple/calendar.events.delete', 1), ('apple/contacts.list', 1), ('apple/maps.search', 1), ('apple/reminders.delete', 1)]

BM25 · name＋description

[('apple/reminders.delete', 3.97497), ('apple/calendar.events.delete', 3.1575), ('apple/contacts.list', 3.01951), ('builtin/ScheduledTasks', 2.93947), ('apple/maps.search', 2.77679)]

BM25 · name + description + Schema + native aliases

[('apple/reminders.delete', 4.03499), ('apple/calendar.events.delete', 3.19175), ('apple/contacts.list', 3.00693), ('builtin/ScheduledTasks', 2.8876), ('apple/maps.search', 2.75197), ('apple/links.open', 1.50559)]

BM25 · name + description + Schema + extended aliases

[('builtin/ListFiles', 5.25514), ('apple/reminders.delete', 4.03499), ('apple/calendar.events.delete', 3.19175), ('apple/contacts.list', 3.00693), ('builtin/ScheduledTasks', 2.8876), ('apple/maps.search', 2.75197), ('apple/links.open', 2.39064)]

H036 在phone工作区落一份完整文本，不覆盖已存在文件。

comparison · wording_transfer · target ['builtin/WriteFile']

New-file write with no overwrite.

MethodFirst acceptable rankTop5
Keyword matching7apple/contacts.update → apple/contacts.list → apple/contacts.create → builtin/EditFile → builtin/ListFiles
BM25 · name＋descriptionNot retrievedapple/contacts.list → apple/contacts.update
BM25 · name + description + Schema + native aliases6apple/contacts.update → apple/contacts.list → apple/contacts.create → builtin/ListFiles → builtin/ReadFile
BM25 · name + description + Schema + extended aliases1builtin/WriteFile → builtin/ReadFile → builtin/ListFiles → apple/contacts.update → apple/contacts.list
Full rankings, within-method scores

Keyword matching

[('apple/contacts.update', 1.5), ('apple/contacts.list', 1), ('apple/contacts.create', 0.5), ('builtin/EditFile', 0.5), ('builtin/ListFiles', 0.5), ('builtin/ReadFile', 0.5), ('builtin/WriteFile', 0.5)]

BM25 · name＋description

[('apple/contacts.list', 3.01951), ('apple/contacts.update', 2.92367)]

BM25 · name + description + Schema + native aliases

[('apple/contacts.update', 3.41372), ('apple/contacts.list', 1.94441), ('apple/contacts.create', 1.6142), ('builtin/ListFiles', 1.0922), ('builtin/ReadFile', 0.98816), ('builtin/WriteFile', 0.95189), ('builtin/EditFile', 0.90222)]

BM25 · name + description + Schema + extended aliases

[('builtin/WriteFile', 9.2536), ('builtin/ReadFile', 9.06573), ('builtin/ListFiles', 6.6829), ('apple/contacts.update', 3.41372), ('apple/contacts.list', 1.94441), ('apple/contacts.create', 1.6142), ('builtin/EditFile', 0.90222)]

H037 这个文件夹的第一层都放了啥？

comparison · wording_transfer · target ['builtin/ListFiles']

One-level listing.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliases1builtin/ListFiles
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[('builtin/ListFiles', 5.25514)]

H038 Enumerate entries immediately under archive/.

comparison · wording_transfer · target ['builtin/ListFiles']

Directory entries, not file content.

MethodFirst acceptable rankTop5
Keyword matching4apple/links.open → apple/music.playback → apple/music.status → builtin/ListFiles
BM25 · name＋descriptionNot retrievedapple/links.open → apple/music.status → apple/music.playback
BM25 · name + description + Schema + native aliases4apple/links.open → apple/music.status → apple/music.playback → builtin/ListFiles
BM25 · name + description + Schema + extended aliases1builtin/ListFiles → apple/links.open → apple/alarms.list → apple/contacts.list → apple/photos.albums.list
Full rankings, within-method scores

Keyword matching

[('apple/links.open', 1.5), ('apple/music.playback', 1), ('apple/music.status', 1), ('builtin/ListFiles', 0.5)]

BM25 · name＋description

[('apple/links.open', 3.1022), ('apple/music.status', 2.84833), ('apple/music.playback', 1.93258)]

BM25 · name + description + Schema + native aliases

[('apple/links.open', 3.03112), ('apple/music.status', 2.58833), ('apple/music.playback', 1.72095), ('builtin/ListFiles', 1.36971)]

BM25 · name + description + Schema + extended aliases

[('builtin/ListFiles', 5.02995), ('apple/links.open', 3.03112), ('apple/alarms.list', 2.76105), ('apple/contacts.list', 2.6196), ('apple/photos.albums.list', 2.6196), ('apple/music.status', 2.58833), ('apple/calendar.events.list', 2.55417), ('apple/calendar.list', 2.55417), ('apple/reminders.list', 2.49193), ('apple/health.workouts.list', 2.37613), ('apple/music.playback', 1.72095)]

H039 Show directory entries without walking subfolders.

comparison · wording_transfer · target ['builtin/ListFiles']

Listing is explicitly one level.

MethodFirst acceptable rankTop5
Keyword matching6apple/maps.show → apple/health.workouts.list → apple/links.open → apple/maps.eta → apple/maps.route
BM25 · name＋description1builtin/ListFiles → apple/maps.show → apple/health.workouts.list → builtin/Memory → apple/links.open
BM25 · name + description + Schema + native aliases2apple/maps.show → builtin/ListFiles → apple/maps.eta → apple/health.workouts.list → apple/maps.route
BM25 · name + description + Schema + extended aliases1builtin/ListFiles → apple/maps.show → apple/health.workouts.list → apple/maps.eta → apple/maps.route
Full rankings, within-method scores

Keyword matching

[('apple/maps.show', 4), ('apple/health.workouts.list', 1.5), ('apple/links.open', 1.5), ('apple/maps.eta', 1.5), ('apple/maps.route', 1.5), ('builtin/ListFiles', 1.5), ('apple/battery.get', 1), ('apple/clipboard.read', 1), ('apple/location.geocode', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.search', 1), ('apple/music.playback', 1), ('apple/music.status', 1), ('apple/photos.search', 1), ('builtin/Memory', 1)]

BM25 · name＋description

[('builtin/ListFiles', 4.72208), ('apple/maps.show', 4.12697), ('apple/health.workouts.list', 3.35194), ('builtin/Memory', 3.26957), ('apple/links.open', 3.1022), ('apple/clipboard.read', 2.98687), ('apple/maps.eta', 2.95063), ('apple/music.status', 2.84833), ('apple/location.reverseGeocode', 2.80551), ('apple/maps.route', 2.63514), ('apple/location.geocode', 2.59384), ('apple/battery.get', 2.46962), ('apple/maps.search', 2.0726), ('apple/music.playback', 1.93258), ('apple/photos.search', 1.69739)]

BM25 · name + description + Schema + native aliases

[('apple/maps.show', 12.51078), ('builtin/ListFiles', 6.23669), ('apple/maps.eta', 4.76669), ('apple/health.workouts.list', 4.48243), ('apple/maps.route', 4.43218), ('builtin/Memory', 3.23148), ('apple/links.open', 3.03112), ('apple/clipboard.read', 2.97251), ('apple/location.reverseGeocode', 2.87756), ('apple/location.geocode', 2.60663), ('apple/music.status', 2.58833), ('apple/battery.get', 2.47355), ('apple/maps.search', 2.05407), ('apple/music.playback', 1.72095), ('apple/photos.search', 1.66567)]

BM25 · name + description + Schema + extended aliases

[('builtin/ListFiles', 14.41048), ('apple/maps.show', 7.13877), ('apple/health.workouts.list', 6.57918), ('apple/maps.eta', 4.76669), ('apple/maps.route', 4.43218), ('builtin/Memory', 3.23148), ('apple/links.open', 3.03112), ('apple/location.reverseGeocode', 2.87756), ('apple/location.geocode', 2.60663), ('apple/music.status', 2.58833), ('apple/battery.get', 2.47355), ('apple/alarms.list', 2.43641), ('apple/contacts.list', 2.31158), ('apple/photos.albums.list', 2.31158), ('apple/calendar.events.list', 2.25385), ('apple/calendar.list', 2.25385), ('apple/reminders.list', 2.19893), ('apple/maps.search', 2.05407), ('apple/music.playback', 1.72095), ('apple/clipboard.read', 1.69614), ('apple/photos.search', 1.66567)]

H040 只将唯一的旧称呼换成新称呼，其他字保留。

comparison · wording_transfer · target ['builtin/EditFile']

Unique exact text replacement.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliasesNot retrieved无召回
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[]

H041 Patch the unique occurrence of alpha to beta inside draft.md.

comparison · wording_transfer · target ['builtin/EditFile']

Exact segment patch.

MethodFirst acceptable rankTop5
Keyword matching4apple/links.open → apple/calendar.events.get → builtin/ScheduledTasks → builtin/EditFile → builtin/Memory
BM25 · name＋descriptionNot retrievedapple/calendar.events.get → builtin/ScheduledTasks
BM25 · name + description + Schema + native aliases3apple/calendar.events.get → builtin/ScheduledTasks → builtin/EditFile → apple/links.open → builtin/Memory
BM25 · name + description + Schema + extended aliases4apple/calendar.events.get → builtin/ScheduledTasks → apple/links.open → builtin/EditFile → builtin/Memory
Full rankings, within-method scores

Keyword matching

[('apple/links.open', 2), ('apple/calendar.events.get', 1.5), ('builtin/ScheduledTasks', 1.5), ('builtin/EditFile', 0.5), ('builtin/Memory', 0.5)]

BM25 · name＋description

[('apple/calendar.events.get', 4.76515), ('builtin/ScheduledTasks', 2.93947)]

BM25 · name + description + Schema + native aliases

[('apple/calendar.events.get', 7.89511), ('builtin/ScheduledTasks', 2.8876), ('builtin/EditFile', 1.62448), ('apple/links.open', 1.50559), ('builtin/Memory', 1.09243)]

BM25 · name + description + Schema + extended aliases

[('apple/calendar.events.get', 7.89511), ('builtin/ScheduledTasks', 2.8876), ('apple/links.open', 2.39064), ('builtin/EditFile', 1.62448), ('builtin/Memory', 1.09243)]

H042 Correct one exact passage after checking the current document.

comparison · wording_transfer · target ['builtin/EditFile']

Single-match edit after read.

MethodFirst acceptable rankTop5
Keyword matching4apple/weather.current → builtin/GetCurrentTime → apple/music.playback → builtin/EditFile → apple/clipboard.write
BM25 · name＋description3builtin/Memory → apple/shortcuts.run → builtin/EditFile → apple/contacts.create → apple/reminders.delete
BM25 · name + description + Schema + native aliases3builtin/GetCurrentTime → apple/weather.current → builtin/EditFile → apple/clipboard.write → builtin/Memory
BM25 · name + description + Schema + extended aliases1builtin/EditFile → builtin/GetCurrentTime → apple/contacts.update → apple/weather.current → apple/reminders.update
Full rankings, within-method scores

Keyword matching

[('apple/weather.current', 4), ('builtin/GetCurrentTime', 4), ('apple/music.playback', 3), ('builtin/EditFile', 3), ('apple/clipboard.write', 2.5), ('apple/shortcuts.run', 2.5), ('builtin/Memory', 2.5), ('apple/alarms.create', 2), ('apple/calendar.events.create', 2), ('apple/contacts.create', 2), ('apple/contacts.delete', 2), ('apple/contacts.update', 2), ('apple/location.get', 2), ('apple/maps.route', 2), ('apple/maps.show', 2), ('apple/reminders.delete', 2), ('apple/links.open', 1.5), ('apple/maps.search', 1.5), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.cancel', 1), ('apple/calendar.events.delete', 1), ('apple/calendar.events.update', 1), ('apple/health.statistics', 1), ('apple/health.workouts.get', 1), ('apple/location.geocode', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.eta', 1), ('apple/music.status', 1), ('apple/photos.albums.list', 1), ('apple/photos.get', 1), ('apple/reminders.complete', 1), ('apple/reminders.create', 1), ('apple/reminders.update', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1), ('builtin/ListFiles', 1), ('builtin/ReadFile', 0.5)]

BM25 · name＋description

[('builtin/Memory', 3.8162), ('apple/shortcuts.run', 3.36524), ('builtin/EditFile', 3.33549), ('apple/contacts.create', 3.14247), ('apple/reminders.delete', 3.05832), ('apple/contacts.delete', 2.89194), ('apple/clipboard.write', 2.85726), ('apple/contacts.update', 2.66723), ('apple/music.playback', 2.60919), ('apple/calendar.events.create', 2.57868), ('apple/location.get', 2.57655), ('apple/alarms.create', 2.55796), ('apple/maps.show', 2.50505), ('apple/maps.route', 2.33689), ('builtin/GetCurrentTime', 2.20523), ('apple/photos.albums.list', 2.10385), ('apple/reminders.complete', 1.76952), ('builtin/ListFiles', 1.68221), ('apple/weather.current', 1.66897), ('apple/location.reverseGeocode', 1.63355), ('apple/weather.daily', 1.56055), ('apple/reminders.update', 1.55292), ('apple/location.geocode', 1.5103), ('apple/reminders.create', 1.49204), ('apple/maps.eta', 1.43797), ('apple/links.open', 1.42271), ('apple/health.workouts.get', 1.39598), ('apple/music.status', 1.38811), ('apple/calendar.events.update', 1.34012), ('apple/maps.search', 1.2068), ('apple/weather.hourly', 1.13818), ('apple/calendar.events.delete', 1.12484), ('builtin/ScheduledTasks', 1.04307), ('apple/alarms.cancel', 1.04154), ('apple/photos.get', 0.98921), ('apple/health.statistics', 0.96452)]

BM25 · name + description + Schema + native aliases

[('builtin/GetCurrentTime', 5.21653), ('apple/weather.current', 5.04538), ('builtin/EditFile', 4.71609), ('apple/clipboard.write', 3.99126), ('builtin/Memory', 3.79281), ('apple/shortcuts.run', 3.70289), ('apple/contacts.create', 3.08006), ('apple/reminders.delete', 3.03617), ('apple/contacts.delete', 2.85848), ('apple/contacts.update', 2.58464), ('apple/calendar.events.create', 2.53336), ('apple/location.get', 2.52587), ('apple/alarms.create', 2.49059), ('apple/music.playback', 2.47362), ('apple/maps.show', 2.42839), ('apple/maps.route', 2.21138), ('apple/maps.search', 2.14433), ('apple/reminders.complete', 1.80751), ('apple/photos.albums.list', 1.67313), ('builtin/ListFiles', 1.65142), ('apple/location.reverseGeocode', 1.5928), ('apple/reminders.update', 1.56949), ('apple/reminders.create', 1.5035), ('apple/weather.daily', 1.46638), ('apple/location.geocode', 1.44283), ('apple/maps.eta', 1.36917), ('apple/health.workouts.get', 1.36911), ('builtin/ScheduledTasks', 1.33101), ('apple/music.status', 1.31868), ('apple/links.open', 1.31248), ('apple/calendar.events.update', 1.27396), ('apple/calendar.events.delete', 1.083), ('apple/weather.hourly', 1.069), ('apple/alarms.cancel', 0.98603), ('apple/photos.get', 0.93377), ('apple/health.statistics', 0.89881), ('builtin/ReadFile', 0.51852)]

BM25 · name + description + Schema + extended aliases

[('builtin/EditFile', 13.49519), ('builtin/GetCurrentTime', 7.0813), ('apple/contacts.update', 6.10707), ('apple/weather.current', 5.04538), ('apple/reminders.update', 4.92434), ('apple/calendar.events.update', 4.71056), ('builtin/ReadFile', 4.32625), ('apple/clipboard.write', 3.99126), ('builtin/WriteFile', 3.91339), ('builtin/Memory', 3.79281), ('apple/shortcuts.run', 3.70289), ('apple/contacts.create', 3.08006), ('apple/reminders.delete', 3.03617), ('apple/contacts.delete', 2.85848), ('apple/calendar.events.create', 2.53336), ('apple/location.get', 2.52587), ('apple/alarms.create', 2.49059), ('apple/music.playback', 2.47362), ('apple/maps.show', 2.42839), ('apple/maps.route', 2.21138), ('apple/maps.search', 2.14433), ('apple/reminders.complete', 1.80751), ('apple/photos.albums.list', 1.67313), ('builtin/ListFiles', 1.65142), ('apple/location.reverseGeocode', 1.5928), ('apple/reminders.create', 1.5035), ('apple/weather.daily', 1.46638), ('apple/location.geocode', 1.44283), ('apple/maps.eta', 1.36917), ('apple/health.workouts.get', 1.36911), ('builtin/ScheduledTasks', 1.33101), ('apple/music.status', 1.31868), ('apple/links.open', 1.31248), ('apple/calendar.events.delete', 1.083), ('apple/weather.hourly', 1.069), ('apple/alarms.cancel', 0.98603), ('apple/photos.get', 0.93377), ('apple/health.statistics', 0.89881)]

H043 以后每次推荐餐馆都考虑我吃素。

comparison · wording_transfer · target ['builtin/Memory']

Persist a lasting user preference, not a one-off note.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliasesNot retrieved无召回
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[]

H044 What lasting preferences did you store about me?

comparison · wording_transfer · target ['builtin/Memory']

Inspect lasting user memory.

MethodFirst acceptable rankTop5
Keyword matching1builtin/Memory
BM25 · name＋description1builtin/Memory
BM25 · name + description + Schema + native aliases1builtin/Memory
BM25 · name + description + Schema + extended aliases1builtin/Memory
Full rankings, within-method scores

Keyword matching

[('builtin/Memory', 1)]

BM25 · name＋description

[('builtin/Memory', 3.80676)]

BM25 · name + description + Schema + native aliases

[('builtin/Memory', 3.76242)]

BM25 · name + description + Schema + extended aliases

[('builtin/Memory', 13.19578)]

H045 把已保存的偏好里那条旧习惯忘掉。

comparison · wording_transfer · target ['builtin/Memory']

Delete within lasting memory scope.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliases2builtin/WriteFile → builtin/Memory
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[('builtin/WriteFile', 5.10932), ('builtin/Memory', 4.71668)]

H046 Change my persistent profile from tea to coffee.

comparison · wording_transfer · target ['builtin/Memory']

Update lasting memory through exact edit.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedapple/alarms.cancel → apple/calendar.events.list → apple/calendar.events.update → apple/contacts.list → apple/maps.search
BM25 · name＋descriptionNot retrievedapple/reminders.delete → apple/reminders.complete → apple/maps.show → apple/alarms.cancel → apple/maps.search
BM25 · name + description + Schema + native aliasesNot retrievedapple/reminders.delete → apple/reminders.complete → apple/maps.show → apple/alarms.cancel → apple/maps.search
BM25 · name + description + Schema + extended aliases1builtin/Memory → apple/calendar.events.update → apple/reminders.update → apple/reminders.delete → apple/reminders.complete
Full rankings, within-method scores

Keyword matching

[('apple/alarms.cancel', 1), ('apple/calendar.events.list', 1), ('apple/calendar.events.update', 1), ('apple/contacts.list', 1), ('apple/maps.search', 1), ('apple/maps.show', 1), ('apple/music.playback', 1), ('apple/photos.albums.list', 1), ('apple/reminders.complete', 1), ('apple/reminders.delete', 1), ('apple/reminders.list', 1), ('apple/reminders.update', 1), ('builtin/ScheduledTasks', 1)]

BM25 · name＋description

[('apple/reminders.delete', 3.22348), ('apple/reminders.complete', 2.44391), ('apple/maps.show', 2.39629), ('apple/alarms.cancel', 2.37094), ('apple/maps.search', 2.25182), ('apple/calendar.events.update', 2.14653), ('apple/reminders.update', 2.14476), ('apple/calendar.events.list', 1.9343), ('apple/contacts.list', 1.72296), ('apple/reminders.list', 1.72296), ('apple/photos.albums.list', 1.48035), ('apple/music.playback', 1.23657), ('builtin/ScheduledTasks', 0.88332)]

BM25 · name + description + Schema + native aliases

[('apple/reminders.delete', 3.27215), ('apple/reminders.complete', 2.49638), ('apple/maps.show', 2.38325), ('apple/alarms.cancel', 2.35658), ('apple/maps.search', 2.23169), ('apple/reminders.update', 2.16765), ('apple/calendar.events.update', 2.14239), ('apple/calendar.events.list', 1.96627), ('apple/contacts.list', 1.71579), ('apple/reminders.list', 1.71579), ('apple/photos.albums.list', 1.47641), ('apple/music.playback', 1.21093), ('builtin/ScheduledTasks', 0.85466)]

BM25 · name + description + Schema + extended aliases

[('builtin/Memory', 7.74041), ('apple/calendar.events.update', 3.87378), ('apple/reminders.update', 3.84966), ('apple/reminders.delete', 3.01171), ('apple/reminders.complete', 2.22753), ('apple/maps.show', 2.19357), ('builtin/EditFile', 2.17402), ('apple/alarms.cancel', 2.16902), ('apple/maps.search', 2.05407), ('apple/contacts.update', 2.01112), ('apple/calendar.events.list', 1.75452), ('apple/contacts.list', 1.53101), ('apple/reminders.list', 1.53101), ('apple/photos.albums.list', 1.31741), ('apple/music.playback', 1.08052), ('builtin/ScheduledTasks', 0.76262)]

H047 列一下长期记忆文件的目录。

comparison · wording_transfer · target ['builtin/Memory']

Memory view enumerates its own namespace.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliases2builtin/ListFiles → builtin/Memory → builtin/WriteFile → builtin/ReadFile
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[('builtin/ListFiles', 10.84583), ('builtin/Memory', 9.43336), ('builtin/WriteFile', 3.91339), ('builtin/ReadFile', 3.80773)]

H048 本机现在的当地日期和UTC偏移各是什么？

comparison · wording_transfer · target ['builtin/GetCurrentTime']

Current device time metadata.

MethodFirst acceptable rankTop5
Keyword matching2apple/photos.search → builtin/GetCurrentTime → apple/alarms.create → apple/calendar.events.create → apple/calendar.events.list
BM25 · name＋description1builtin/GetCurrentTime → apple/photos.search
BM25 · name + description + Schema + native aliases1builtin/GetCurrentTime → apple/photos.search → apple/health.sleep.query → apple/calendar.events.list → apple/health.statistics
BM25 · name + description + Schema + extended aliases1builtin/GetCurrentTime → apple/photos.search → apple/health.sleep.query → apple/calendar.events.list → apple/health.statistics
Full rankings, within-method scores

Keyword matching

[('apple/photos.search', 1.5), ('builtin/GetCurrentTime', 1), ('apple/alarms.create', 0.5), ('apple/calendar.events.create', 0.5), ('apple/calendar.events.list', 0.5), ('apple/calendar.events.update', 0.5), ('apple/health.sleep.query', 0.5), ('apple/health.statistics', 0.5), ('apple/health.workouts.list', 0.5), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('builtin/GetCurrentTime', 4.03298), ('apple/photos.search', 2.2741)]

BM25 · name + description + Schema + native aliases

[('builtin/GetCurrentTime', 2.2422), ('apple/photos.search', 2.11129), ('apple/health.sleep.query', 1.01685), ('apple/calendar.events.list', 0.97113), ('apple/health.statistics', 0.96033), ('apple/alarms.create', 0.94939), ('apple/health.workouts.list', 0.93944), ('apple/calendar.events.create', 0.92933), ('apple/calendar.events.update', 0.89099), ('builtin/ScheduledTasks', 0.23159)]

BM25 · name + description + Schema + extended aliases

[('builtin/GetCurrentTime', 12.75248), ('apple/photos.search', 2.11129), ('apple/health.sleep.query', 1.01685), ('apple/calendar.events.list', 0.97113), ('apple/health.statistics', 0.96033), ('apple/alarms.create', 0.94939), ('apple/health.workouts.list', 0.93944), ('apple/calendar.events.create', 0.92933), ('apple/calendar.events.update', 0.89099), ('builtin/ScheduledTasks', 0.23159)]

H049 我还没定方案，你先让我选一个。

comparison · wording_transfer · target ['builtin/AskUserQuestions']

Collect missing user choice.

MethodFirst acceptable rankTop5
Keyword matchingNot retrieved无召回
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrieved无召回
BM25 · name + description + Schema + extended aliasesNot retrieved无召回
Full rankings, within-method scores

Keyword matching

[]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[]

BM25 · name + description + Schema + extended aliases

[]

H050 下周起每周五帮我查一次天气，允许延迟。

comparison · wording_transfer · target ['builtin/ScheduledTasks']

Best-effort weekly read-only task, not a calendar event.

MethodFirst acceptable rankTop5
Keyword matchingNot retrievedapple/weather.current → apple/weather.daily → apple/weather.hourly
BM25 · name＋descriptionNot retrieved无召回
BM25 · name + description + Schema + native aliasesNot retrievedapple/weather.current → apple/weather.daily → apple/weather.hourly
BM25 · name + description + Schema + extended aliasesNot retrievedapple/weather.current → apple/weather.daily → apple/weather.hourly
Full rankings, within-method scores

Keyword matching

[('apple/weather.current', 1.5), ('apple/weather.daily', 1.5), ('apple/weather.hourly', 1.5)]

BM25 · name＋description

[]

BM25 · name + description + Schema + native aliases

[('apple/weather.current', 5.26033), ('apple/weather.daily', 5.26033), ('apple/weather.hourly', 5.26033)]

BM25 · name + description + Schema + extended aliases

[('apple/weather.current', 6.12162), ('apple/weather.daily', 6.12162), ('apple/weather.hourly', 6.12162)]

H051 Send an email directly, without relying on a preconfigured shortcut.

comparison · unsupported · target []

No mail-send tool; URL handoff alone does not send mail.

MethodFirst acceptable rankTop5
Keyword matching—apple/shortcuts.run → apple/contacts.update → apple/battery.get → apple/contacts.list → apple/location.geocode
BM25 · name＋description—apple/shortcuts.run → builtin/GetCurrentTime → apple/contacts.list → apple/contacts.update → apple/location.reverseGeocode
BM25 · name + description + Schema + native aliases—apple/shortcuts.run → builtin/GetCurrentTime → apple/contacts.update → apple/location.reverseGeocode → apple/contacts.list
BM25 · name + description + Schema + extended aliases—apple/shortcuts.run → builtin/GetCurrentTime → apple/contacts.update → apple/location.reverseGeocode → apple/contacts.list
Full rankings, within-method scores

Keyword matching

[('apple/shortcuts.run', 5), ('apple/contacts.update', 1.5), ('apple/battery.get', 1), ('apple/contacts.list', 1), ('apple/location.geocode', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.search', 1), ('apple/photos.search', 1), ('builtin/GetCurrentTime', 1), ('apple/contacts.create', 0.5)]

BM25 · name＋description

[('apple/shortcuts.run', 5.80448), ('builtin/GetCurrentTime', 4.69561), ('apple/contacts.list', 3.01951), ('apple/contacts.update', 2.92367), ('apple/location.reverseGeocode', 2.80551), ('apple/location.geocode', 2.59384), ('apple/battery.get', 2.46962), ('apple/maps.search', 2.0726), ('apple/photos.search', 1.69739)]

BM25 · name + description + Schema + native aliases

[('apple/shortcuts.run', 20.05915), ('builtin/GetCurrentTime', 4.84864), ('apple/contacts.update', 4.70783), ('apple/location.reverseGeocode', 2.87756), ('apple/contacts.list', 2.68151), ('apple/location.geocode', 2.60663), ('apple/battery.get', 2.47355), ('apple/contacts.create', 2.22612), ('apple/maps.search', 2.05407), ('apple/photos.search', 1.66567)]

BM25 · name + description + Schema + extended aliases

[('apple/shortcuts.run', 20.05915), ('builtin/GetCurrentTime', 4.84864), ('apple/contacts.update', 4.70783), ('apple/location.reverseGeocode', 2.87756), ('apple/contacts.list', 2.68151), ('apple/location.geocode', 2.60663), ('apple/battery.get', 2.47355), ('apple/contacts.create', 2.22612), ('apple/maps.search', 2.05407), ('apple/photos.search', 1.66567)]

H052 Write the full body of a new Apple Notes note without a saved shortcut.

comparison · unsupported · target []

No Notes content-write capability.

MethodFirst acceptable rankTop5
Keyword matching—apple/shortcuts.run → apple/links.open → apple/clipboard.write → apple/calendar.events.update → apple/health.statistics
BM25 · name＋description—apple/shortcuts.run → apple/health.statistics → builtin/WriteFile → apple/clipboard.write → apple/calendar.events.update
BM25 · name + description + Schema + native aliases—apple/shortcuts.run → apple/clipboard.write → builtin/WriteFile → apple/health.statistics → apple/links.open
BM25 · name + description + Schema + extended aliases—apple/shortcuts.run → builtin/WriteFile → apple/clipboard.write → apple/links.open → apple/health.statistics
Full rankings, within-method scores

Keyword matching

[('apple/shortcuts.run', 11), ('apple/links.open', 10), ('apple/clipboard.write', 7.5), ('apple/calendar.events.update', 6), ('apple/health.statistics', 5.5), ('apple/maps.search', 5), ('apple/music.playback', 5), ('apple/calendar.events.create', 4.5), ('apple/reminders.update', 4.5), ('apple/alarms.cancel', 4), ('apple/alarms.create', 4), ('apple/alarms.list', 4), ('apple/battery.get', 4), ('apple/calendar.events.delete', 4), ('apple/contacts.delete', 4), ('apple/contacts.get', 4), ('apple/health.sleep.query', 4), ('apple/health.workouts.get', 4), ('apple/health.workouts.list', 4), ('apple/location.geocode', 4), ('apple/location.reverseGeocode', 4), ('apple/maps.route', 4), ('apple/maps.show', 4), ('apple/music.search', 4), ('apple/music.status', 4), ('apple/photos.search', 4), ('apple/reminders.list', 4), ('apple/weather.current', 4), ('apple/weather.daily', 4), ('apple/weather.hourly', 4), ('apple/reminders.create', 3.5), ('apple/calendar.events.get', 3), ('apple/calendar.events.list', 3), ('apple/calendar.list', 3), ('apple/clipboard.read', 3), ('apple/contacts.create', 3), ('apple/contacts.list', 3), ('apple/contacts.update', 3), ('apple/device.get', 3), ('apple/location.get', 3), ('apple/maps.eta', 3), ('apple/photos.albums.list', 3), ('apple/photos.get', 3), ('apple/reminders.complete', 3), ('apple/reminders.delete', 3), ('apple/reminders.get', 3), ('apple/reminders.lists', 3), ('builtin/WriteFile', 3), ('builtin/ScheduledTasks', 2), ('builtin/Memory', 1.5), ('builtin/EditFile', 0.5)]

BM25 · name＋description

[('apple/shortcuts.run', 10.73643), ('apple/health.statistics', 5.09681), ('builtin/WriteFile', 3.8796), ('apple/clipboard.write', 3.79364), ('apple/calendar.events.update', 3.50068), ('apple/maps.search', 3.48643), ('apple/links.open', 3.44228), ('apple/contacts.get', 3.13334), ('apple/reminders.update', 3.04811), ('apple/location.reverseGeocode', 2.80551), ('apple/location.geocode', 2.59384), ('builtin/ScheduledTasks', 2.53593), ('apple/battery.get', 2.46962), ('apple/reminders.list', 2.44865), ('apple/music.status', 2.384), ('apple/music.playback', 2.27021), ('apple/health.workouts.get', 2.10985), ('apple/maps.show', 2.10128), ('apple/contacts.delete', 2.02377), ('apple/health.workouts.list', 2.02377), ('apple/alarms.create', 2.01195), ('apple/weather.current', 1.95529), ('apple/health.sleep.query', 1.80306), ('apple/calendar.events.delete', 1.70006), ('apple/photos.search', 1.69739), ('apple/alarms.list', 1.50453), ('apple/maps.route', 1.50453), ('apple/alarms.cancel', 1.48861), ('apple/calendar.events.create', 1.46562), ('apple/music.search', 1.41383), ('apple/weather.hourly', 1.33344), ('apple/weather.daily', 1.23947)]

BM25 · name + description + Schema + native aliases

[('apple/shortcuts.run', 23.41812), ('apple/clipboard.write', 12.26108), ('builtin/WriteFile', 7.76229), ('apple/health.statistics', 6.9262), ('apple/links.open', 6.89973), ('apple/calendar.events.update', 6.36675), ('apple/reminders.update', 3.80685), ('apple/location.reverseGeocode', 3.3256), ('builtin/Memory', 3.1665), ('apple/location.geocode', 3.11345), ('apple/contacts.get', 3.03736), ('apple/calendar.events.create', 3.03034), ('apple/battery.get', 2.98037), ('apple/music.status', 2.88915), ('apple/maps.search', 2.70661), ('apple/health.workouts.get', 2.62055), ('apple/contacts.delete', 2.55219), ('apple/health.workouts.list', 2.52378), ('apple/alarms.create', 2.49759), ('apple/reminders.list', 2.45123), ('builtin/ScheduledTasks', 2.3326), ('apple/health.sleep.query', 2.27909), ('apple/photos.search', 2.17249), ('apple/calendar.events.delete', 2.16654), ('apple/music.playback', 1.7618), ('apple/reminders.create', 1.35036), ('builtin/EditFile', 1.24425), ('apple/maps.show', 0.72451), ('apple/weather.current', 0.70854), ('apple/alarms.list', 0.66243), ('apple/maps.route', 0.66243), ('apple/alarms.cancel', 0.66069), ('apple/music.search', 0.65254), ('apple/weather.hourly', 0.64383), ('apple/weather.daily', 0.63371), ('apple/calendar.list', 0.50682), ('apple/clipboard.read', 0.50682), ('apple/contacts.create', 0.50682), ('apple/contacts.list', 0.50682), ('apple/contacts.update', 0.50682), ('apple/device.get', 0.50682), ('apple/location.get', 0.50682), ('apple/maps.eta', 0.50682), ('apple/photos.get', 0.50682), ('apple/reminders.complete', 0.50682), ('apple/reminders.delete', 0.50682), ('apple/reminders.get', 0.50682), ('apple/reminders.lists', 0.50682), ('apple/calendar.events.get', 0.44804), ('apple/calendar.events.list', 0.44804), ('apple/photos.albums.list', 0.44804)]

BM25 · name + description + Schema + extended aliases

[('apple/shortcuts.run', 23.41812), ('builtin/WriteFile', 13.98196), ('apple/clipboard.write', 12.26108), ('apple/links.open', 8.50181), ('apple/health.statistics', 6.9262), ('apple/calendar.events.update', 5.79634), ('apple/calendar.events.create', 5.69134), ('apple/alarms.create', 5.40022), ('apple/reminders.create', 3.93951), ('apple/reminders.update', 3.80685), ('apple/location.reverseGeocode', 3.3256), ('apple/contacts.create', 3.24377), ('apple/location.geocode', 3.11345), ('apple/contacts.get', 3.03736), ('apple/battery.get', 2.98037), ('apple/music.status', 2.88915), ('builtin/Memory', 2.74625), ('apple/maps.search', 2.70661), ('apple/health.workouts.get', 2.62055), ('apple/contacts.delete', 2.55219), ('apple/health.workouts.list', 2.52378), ('apple/reminders.list', 2.45123), ('builtin/ScheduledTasks', 2.3326), ('apple/health.sleep.query', 2.27909), ('apple/photos.search', 2.17249), ('apple/calendar.events.delete', 2.16654), ('apple/music.playback', 1.7618), ('builtin/EditFile', 0.84605), ('apple/maps.show', 0.72451), ('apple/weather.current', 0.70854), ('apple/alarms.list', 0.66243), ('apple/maps.route', 0.66243), ('apple/alarms.cancel', 0.66069), ('apple/music.search', 0.65254), ('apple/weather.hourly', 0.64383), ('apple/weather.daily', 0.63371), ('apple/calendar.list', 0.50682), ('apple/clipboard.read', 0.50682), ('apple/contacts.list', 0.50682), ('apple/contacts.update', 0.50682), ('apple/device.get', 0.50682), ('apple/location.get', 0.50682), ('apple/maps.eta', 0.50682), ('apple/photos.get', 0.50682), ('apple/reminders.complete', 0.50682), ('apple/reminders.delete', 0.50682), ('apple/reminders.get', 0.50682), ('apple/reminders.lists', 0.50682), ('apple/calendar.events.get', 0.44804), ('apple/calendar.events.list', 0.44804), ('apple/photos.albums.list', 0.44804)]

H053 Crop the actual pixels of an image in my photo library.

comparison · unsupported · target []

Photo tools return metadata, not pixel editing.

MethodFirst acceptable rankTop5
Keyword matching—apple/photos.albums.list → apple/photos.get → apple/photos.search → apple/links.open → apple/alarms.list
BM25 · name＋description—apple/photos.albums.list → apple/photos.get → apple/photos.search → apple/reminders.complete → apple/contacts.get
BM25 · name + description + Schema + native aliases—apple/photos.get → apple/photos.albums.list → apple/photos.search → apple/reminders.complete → apple/reminders.update
BM25 · name + description + Schema + extended aliases—apple/photos.get → apple/photos.albums.list → apple/photos.search → apple/reminders.complete → apple/reminders.update
Full rankings, within-method scores

Keyword matching

[('apple/photos.albums.list', 6), ('apple/photos.get', 5.5), ('apple/photos.search', 5.5), ('apple/links.open', 2), ('apple/alarms.list', 1), ('apple/clipboard.read', 1), ('apple/contacts.get', 1), ('apple/location.get', 1), ('apple/music.search', 1), ('apple/reminders.complete', 1), ('apple/reminders.update', 1)]

BM25 · name＋description

[('apple/photos.albums.list', 8.9163), ('apple/photos.get', 6.93155), ('apple/photos.search', 4.61221), ('apple/reminders.complete', 3.47326), ('apple/contacts.get', 3.13334), ('apple/reminders.update', 3.04811), ('apple/music.search', 2.77679), ('apple/location.get', 2.71569), ('apple/clipboard.read', 2.42218), ('apple/alarms.list', 2.39629)]

BM25 · name + description + Schema + native aliases

[('apple/photos.get', 14.50536), ('apple/photos.albums.list', 13.6936), ('apple/photos.search', 12.62146), ('apple/reminders.complete', 3.54782), ('apple/reminders.update', 3.08065), ('apple/contacts.get', 2.92093), ('apple/location.get', 2.72219), ('apple/music.search', 2.45414), ('apple/clipboard.read', 2.41054), ('apple/alarms.list', 2.38325), ('apple/links.open', 1.12377)]

BM25 · name + description + Schema + extended aliases

[('apple/photos.get', 14.50536), ('apple/photos.albums.list', 13.6936), ('apple/photos.search', 12.62146), ('apple/reminders.complete', 3.54782), ('apple/reminders.update', 3.08065), ('apple/contacts.get', 2.92093), ('apple/location.get', 2.72219), ('apple/music.search', 2.45414), ('apple/clipboard.read', 2.41054), ('apple/alarms.list', 2.38325), ('apple/links.open', 1.78438)]

H054 Create a playlist in my Apple Music library.

comparison · unsupported · target []

Catalog/player tools do not create library playlists.

MethodFirst acceptable rankTop5
Keyword matching—apple/music.search → apple/alarms.create → apple/links.open → apple/music.playback → apple/music.status
BM25 · name＋description—apple/music.search → apple/music.status → apple/music.playback → apple/alarms.create → apple/photos.albums.list
BM25 · name + description + Schema + native aliases—apple/music.status → apple/music.search → apple/music.playback → apple/alarms.create → apple/reminders.create
BM25 · name + description + Schema + extended aliases—apple/music.status → apple/music.search → apple/music.playback → apple/alarms.create → apple/reminders.create
Full rankings, within-method scores

Keyword matching

[('apple/music.search', 9), ('apple/alarms.create', 8), ('apple/links.open', 8), ('apple/music.playback', 8), ('apple/music.status', 8), ('apple/calendar.events.create', 7), ('apple/contacts.create', 7), ('apple/reminders.create', 7), ('apple/shortcuts.run', 5), ('apple/alarms.cancel', 4), ('apple/alarms.list', 4), ('apple/maps.route', 4), ('apple/maps.search', 4), ('apple/maps.show', 4), ('apple/photos.albums.list', 4), ('apple/weather.current', 4), ('apple/weather.daily', 4), ('apple/weather.hourly', 4), ('apple/photos.search', 3.5), ('apple/battery.get', 3), ('apple/calendar.events.delete', 3), ('apple/calendar.events.get', 3), ('apple/calendar.events.list', 3), ('apple/calendar.events.update', 3), ('apple/calendar.list', 3), ('apple/clipboard.read', 3), ('apple/clipboard.write', 3), ('apple/contacts.delete', 3), ('apple/contacts.get', 3), ('apple/contacts.list', 3), ('apple/contacts.update', 3), ('apple/device.get', 3), ('apple/health.sleep.query', 3), ('apple/health.statistics', 3), ('apple/health.workouts.get', 3), ('apple/health.workouts.list', 3), ('apple/location.geocode', 3), ('apple/location.get', 3), ('apple/location.reverseGeocode', 3), ('apple/maps.eta', 3), ('apple/photos.get', 3), ('apple/reminders.complete', 3), ('apple/reminders.delete', 3), ('apple/reminders.get', 3), ('apple/reminders.list', 3), ('apple/reminders.lists', 3), ('apple/reminders.update', 3), ('builtin/ScheduledTasks', 2.5), ('builtin/Memory', 1.5), ('builtin/WriteFile', 1)]

BM25 · name＋description

[('apple/music.search', 8.29057), ('apple/music.status', 6.80246), ('apple/music.playback', 4.65231), ('apple/alarms.create', 4.51258), ('apple/photos.albums.list', 3.76182), ('apple/shortcuts.run', 3.45174), ('apple/links.open', 3.44228), ('builtin/ScheduledTasks', 3.22274), ('apple/reminders.create', 3.05666), ('apple/contacts.create', 3.03074), ('apple/calendar.events.create', 2.94033), ('builtin/WriteFile', 2.81316), ('builtin/Memory', 2.11424), ('apple/maps.show', 2.10128), ('apple/weather.current', 1.95529), ('apple/alarms.list', 1.50453), ('apple/maps.route', 1.50453), ('apple/alarms.cancel', 1.48861), ('apple/maps.search', 1.41383), ('apple/weather.hourly', 1.33344), ('apple/weather.daily', 1.23947)]

BM25 · name + description + Schema + native aliases

[('apple/music.status', 15.30661), ('apple/music.search', 15.14277), ('apple/music.playback', 12.04394), ('apple/alarms.create', 10.41189), ('apple/reminders.create', 9.04963), ('apple/contacts.create', 9.0171), ('apple/calendar.events.create', 8.4387), ('apple/links.open', 5.43301), ('apple/photos.albums.list', 3.79627), ('builtin/ScheduledTasks', 3.41676), ('builtin/Memory', 3.04182), ('builtin/WriteFile', 2.8775), ('apple/shortcuts.run', 2.68354), ('apple/photos.search', 1.50741), ('apple/maps.show', 0.72451), ('apple/weather.current', 0.70854), ('apple/alarms.list', 0.66243), ('apple/maps.route', 0.66243), ('apple/alarms.cancel', 0.66069), ('apple/maps.search', 0.65254), ('apple/weather.hourly', 0.64383), ('apple/weather.daily', 0.63371), ('apple/battery.get', 0.50682), ('apple/calendar.list', 0.50682), ('apple/clipboard.read', 0.50682), ('apple/clipboard.write', 0.50682), ('apple/contacts.delete', 0.50682), ('apple/contacts.get', 0.50682), ('apple/contacts.list', 0.50682), ('apple/contacts.update', 0.50682), ('apple/device.get', 0.50682), ('apple/health.statistics', 0.50682), ('apple/location.geocode', 0.50682), ('apple/location.get', 0.50682), ('apple/maps.eta', 0.50682), ('apple/photos.get', 0.50682), ('apple/reminders.complete', 0.50682), ('apple/reminders.delete', 0.50682), ('apple/reminders.get', 0.50682), ('apple/reminders.list', 0.50682), ('apple/reminders.lists', 0.50682), ('apple/reminders.update', 0.50682), ('apple/calendar.events.delete', 0.44804), ('apple/calendar.events.get', 0.44804), ('apple/calendar.events.list', 0.44804), ('apple/calendar.events.update', 0.44804), ('apple/health.sleep.query', 0.44804), ('apple/health.workouts.get', 0.44804), ('apple/health.workouts.list', 0.44804), ('apple/location.reverseGeocode', 0.44804)]

BM25 · name + description + Schema + extended aliases

[('apple/music.status', 15.30661), ('apple/music.search', 15.14277), ('apple/music.playback', 12.04394), ('apple/alarms.create', 10.41189), ('apple/reminders.create', 9.04963), ('apple/contacts.create', 9.0171), ('apple/calendar.events.create', 8.4387), ('apple/links.open', 6.14047), ('builtin/WriteFile', 5.71516), ('apple/photos.albums.list', 3.79627), ('builtin/ScheduledTasks', 3.41676), ('builtin/Memory', 3.04182), ('apple/shortcuts.run', 2.68354), ('apple/photos.search', 1.50741), ('apple/maps.show', 0.72451), ('apple/weather.current', 0.70854), ('apple/alarms.list', 0.66243), ('apple/maps.route', 0.66243), ('apple/alarms.cancel', 0.66069), ('apple/maps.search', 0.65254), ('apple/weather.hourly', 0.64383), ('apple/weather.daily', 0.63371), ('apple/battery.get', 0.50682), ('apple/calendar.list', 0.50682), ('apple/clipboard.read', 0.50682), ('apple/clipboard.write', 0.50682), ('apple/contacts.delete', 0.50682), ('apple/contacts.get', 0.50682), ('apple/contacts.list', 0.50682), ('apple/contacts.update', 0.50682), ('apple/device.get', 0.50682), ('apple/health.statistics', 0.50682), ('apple/location.geocode', 0.50682), ('apple/location.get', 0.50682), ('apple/maps.eta', 0.50682), ('apple/photos.get', 0.50682), ('apple/reminders.complete', 0.50682), ('apple/reminders.delete', 0.50682), ('apple/reminders.get', 0.50682), ('apple/reminders.list', 0.50682), ('apple/reminders.lists', 0.50682), ('apple/reminders.update', 0.50682), ('apple/calendar.events.delete', 0.44804), ('apple/calendar.events.get', 0.44804), ('apple/calendar.events.list', 0.44804), ('apple/calendar.events.update', 0.44804), ('apple/health.sleep.query', 0.44804), ('apple/health.workouts.get', 0.44804), ('apple/health.workouts.list', 0.44804), ('apple/location.reverseGeocode', 0.44804)]

H055 Merge duplicate contacts into one atomically.

comparison · unsupported · target []

No contact merge/transaction tool.

MethodFirst acceptable rankTop5
Keyword matching—apple/contacts.create → apple/contacts.delete → apple/contacts.update → apple/contacts.get → apple/contacts.list
BM25 · name＋description—apple/contacts.create → apple/contacts.delete → apple/contacts.update → apple/contacts.get → apple/calendar.events.create
BM25 · name + description + Schema + native aliases—apple/contacts.create → apple/contacts.delete → apple/contacts.get → apple/contacts.update → apple/contacts.list
BM25 · name + description + Schema + extended aliases—apple/contacts.create → apple/contacts.delete → apple/contacts.get → apple/contacts.update → apple/contacts.list
Full rankings, within-method scores

Keyword matching

[('apple/contacts.create', 7), ('apple/contacts.delete', 5.5), ('apple/contacts.update', 5.5), ('apple/contacts.get', 4.5), ('apple/contacts.list', 4), ('apple/calendar.events.create', 2), ('apple/alarms.cancel', 1), ('apple/alarms.create', 1), ('apple/calendar.events.delete', 1), ('apple/calendar.events.update', 1), ('apple/clipboard.write', 1), ('apple/health.workouts.get', 1), ('apple/location.get', 1), ('apple/maps.route', 1), ('apple/music.playback', 1), ('apple/photos.get', 1), ('apple/reminders.delete', 1), ('builtin/EditFile', 1), ('builtin/ListFiles', 1), ('builtin/Memory', 1), ('builtin/ReadFile', 0.5), ('builtin/ScheduledTasks', 0.5)]

BM25 · name＋description

[('apple/contacts.create', 13.29333), ('apple/contacts.delete', 4.96014), ('apple/contacts.update', 4.51685), ('apple/contacts.get', 4.06233), ('apple/calendar.events.create', 3.69182), ('apple/contacts.list', 3.59088), ('builtin/ListFiles', 1.68221), ('builtin/EditFile', 1.5025), ('apple/reminders.delete', 1.41606), ('apple/health.workouts.get', 1.39598), ('apple/calendar.events.update', 1.34012), ('apple/location.get', 1.19299), ('builtin/Memory', 1.16476), ('apple/calendar.events.delete', 1.12484), ('apple/clipboard.write', 1.09971), ('apple/maps.route', 1.05267), ('apple/alarms.cancel', 1.04154), ('apple/photos.get', 0.98921), ('apple/alarms.create', 0.96027), ('apple/music.playback', 0.77202)]

BM25 · name + description + Schema + native aliases

[('apple/contacts.create', 19.25559), ('apple/contacts.delete', 12.43568), ('apple/contacts.get', 11.96374), ('apple/contacts.update', 11.45274), ('apple/contacts.list', 9.52374), ('apple/calendar.events.create', 3.64708), ('builtin/ListFiles', 1.65142), ('builtin/EditFile', 1.45986), ('apple/health.workouts.get', 1.36911), ('apple/reminders.delete', 1.36911), ('apple/calendar.events.update', 1.27396), ('apple/location.get', 1.139), ('builtin/Memory', 1.09647), ('apple/calendar.events.delete', 1.083), ('apple/clipboard.write', 1.04448), ('apple/maps.route', 0.99719), ('apple/alarms.cancel', 0.98603), ('apple/photos.get', 0.93377), ('apple/alarms.create', 0.905), ('builtin/ScheduledTasks', 0.86526), ('apple/music.playback', 0.72007), ('builtin/ReadFile', 0.51852)]

BM25 · name + description + Schema + extended aliases

[('apple/contacts.create', 19.25559), ('apple/contacts.delete', 12.43568), ('apple/contacts.get', 11.96374), ('apple/contacts.update', 11.45274), ('apple/contacts.list', 9.52374), ('apple/calendar.events.create', 3.64708), ('builtin/ListFiles', 1.65142), ('builtin/EditFile', 1.45986), ('apple/health.workouts.get', 1.36911), ('apple/reminders.delete', 1.36911), ('apple/calendar.events.update', 1.27396), ('apple/location.get', 1.139), ('builtin/Memory', 1.09647), ('apple/calendar.events.delete', 1.083), ('apple/clipboard.write', 1.04448), ('apple/maps.route', 0.99719), ('apple/alarms.cancel', 0.98603), ('apple/photos.get', 0.93377), ('apple/alarms.create', 0.905), ('builtin/ScheduledTasks', 0.86526), ('apple/music.playback', 0.72007), ('builtin/ReadFile', 0.51852)]

H056 Delete recorded samples from HealthKit.

comparison · unsupported · target []

Health tools are read-only.

MethodFirst acceptable rankTop5
Keyword matching—apple/health.statistics → apple/health.workouts.get → apple/calendar.events.delete → apple/contacts.delete → apple/health.workouts.list
BM25 · name＋description—apple/health.workouts.get → apple/health.workouts.list → apple/health.statistics → apple/reminders.delete → apple/contacts.delete
BM25 · name + description + Schema + native aliases—apple/health.statistics → apple/health.workouts.get → apple/health.workouts.list → apple/reminders.delete → apple/contacts.delete
BM25 · name + description + Schema + extended aliases—apple/health.statistics → apple/health.workouts.get → apple/health.workouts.list → apple/reminders.delete → apple/contacts.delete
Full rankings, within-method scores

Keyword matching

[('apple/health.statistics', 6), ('apple/health.workouts.get', 4.5), ('apple/calendar.events.delete', 4), ('apple/contacts.delete', 4), ('apple/health.workouts.list', 4), ('apple/reminders.delete', 4), ('apple/health.sleep.query', 3), ('builtin/ScheduledTasks', 3), ('builtin/Memory', 1.5), ('apple/alarms.create', 1), ('apple/alarms.list', 1), ('apple/location.get', 1), ('apple/maps.search', 1), ('apple/photos.albums.list', 1), ('apple/photos.get', 1), ('apple/weather.current', 1)]

BM25 · name＋description

[('apple/health.workouts.get', 6.84346), ('apple/health.workouts.list', 6.56423), ('apple/health.statistics', 6.15581), ('apple/reminders.delete', 3.75288), ('apple/contacts.delete', 3.62112), ('apple/location.get', 3.3488), ('apple/calendar.events.delete', 3.22484), ('builtin/ScheduledTasks', 3.03985), ('apple/alarms.list', 2.66867), ('apple/health.sleep.query', 2.49955), ('builtin/Memory', 2.44041), ('apple/maps.search', 1.79559), ('apple/photos.get', 1.79559), ('apple/alarms.create', 1.74305), ('apple/weather.current', 1.72621), ('apple/photos.albums.list', 1.6776)]

BM25 · name + description + Schema + native aliases

[('apple/health.statistics', 12.35056), ('apple/health.workouts.get', 11.99158), ('apple/health.workouts.list', 10.17112), ('apple/reminders.delete', 10.1562), ('apple/contacts.delete', 9.97994), ('apple/calendar.events.delete', 8.6982), ('apple/health.sleep.query', 6.31587), ('builtin/ScheduledTasks', 4.63914), ('apple/location.get', 3.35681), ('builtin/Memory', 3.1123), ('apple/alarms.list', 2.65852), ('apple/maps.search', 1.77954), ('apple/photos.get', 1.77954), ('apple/alarms.create', 1.7247), ('apple/weather.current', 1.70716), ('apple/photos.albums.list', 1.67313)]

BM25 · name + description + Schema + extended aliases

[('apple/health.statistics', 12.35056), ('apple/health.workouts.get', 11.99158), ('apple/health.workouts.list', 10.17112), ('apple/reminders.delete', 10.1562), ('apple/contacts.delete', 9.97994), ('apple/calendar.events.delete', 8.6982), ('apple/health.sleep.query', 6.31587), ('builtin/ScheduledTasks', 4.63914), ('apple/location.get', 3.35681), ('builtin/Memory', 3.1123), ('apple/alarms.list', 2.65852), ('apple/maps.search', 1.77954), ('apple/photos.get', 1.77954), ('apple/alarms.create', 1.7247), ('apple/weather.current', 1.70716), ('apple/photos.albums.list', 1.67313)]

H057 Install an App Store application automatically.

comparison · unsupported · target []

No application-install tool.

MethodFirst acceptable rankTop5
Keyword matching—apple/links.open → apple/music.status → apple/alarms.create → apple/shortcuts.run → builtin/ScheduledTasks
BM25 · name＋description—apple/music.status → apple/links.open → apple/shortcuts.run → apple/alarms.create → builtin/ScheduledTasks
BM25 · name + description + Schema + native aliases—apple/links.open → apple/music.status → apple/shortcuts.run → apple/alarms.create → builtin/ScheduledTasks
BM25 · name + description + Schema + extended aliases—apple/links.open → apple/music.status → apple/shortcuts.run → apple/alarms.create → builtin/ScheduledTasks
Full rankings, within-method scores

Keyword matching

[('apple/links.open', 4), ('apple/music.status', 2), ('apple/alarms.create', 1), ('apple/shortcuts.run', 1), ('builtin/ScheduledTasks', 1)]

BM25 · name＋description

[('apple/music.status', 6.10277), ('apple/links.open', 5.48492), ('apple/shortcuts.run', 2.05198), ('apple/alarms.create', 2.01195), ('builtin/ScheduledTasks', 1.15544)]

BM25 · name + description + Schema + native aliases

[('apple/links.open', 6.81879), ('apple/music.status', 6.09849), ('apple/shortcuts.run', 2.03253), ('apple/alarms.create', 1.99077), ('builtin/ScheduledTasks', 1.11796)]

BM25 · name + description + Schema + extended aliases

[('apple/links.open', 7.47939), ('apple/music.status', 6.09849), ('apple/shortcuts.run', 2.03253), ('apple/alarms.create', 1.99077), ('builtin/ScheduledTasks', 1.11796)]

H058 Read my saved passwords from the system Keychain.

comparison · unsupported · target []

No password or Keychain-read capability.

MethodFirst acceptable rankTop5
Keyword matching—apple/clipboard.read → builtin/ReadFile → builtin/ScheduledTasks → apple/device.get → apple/health.sleep.query
BM25 · name＋description—apple/device.get → apple/clipboard.read → apple/health.workouts.get → apple/health.workouts.list → apple/health.sleep.query
BM25 · name + description + Schema + native aliases—apple/clipboard.read → builtin/ReadFile → apple/device.get → apple/health.workouts.get → apple/health.workouts.list
BM25 · name + description + Schema + extended aliases—apple/clipboard.read → apple/device.get → builtin/ReadFile → apple/health.workouts.get → apple/health.workouts.list
Full rankings, within-method scores

Keyword matching

[('apple/clipboard.read', 5), ('builtin/ReadFile', 4), ('builtin/ScheduledTasks', 2.5), ('apple/device.get', 2), ('apple/health.sleep.query', 2), ('apple/health.statistics', 2), ('apple/health.workouts.get', 2), ('apple/health.workouts.list', 2), ('apple/shortcuts.run', 1.5), ('apple/alarms.list', 1), ('apple/battery.get', 1), ('apple/calendar.events.create', 1), ('apple/calendar.events.delete', 1), ('apple/calendar.events.list', 1), ('apple/calendar.events.update', 1), ('apple/contacts.create', 1), ('apple/contacts.delete', 1), ('apple/contacts.get', 1), ('apple/links.open', 1), ('apple/maps.eta', 1), ('apple/music.playback', 1), ('apple/music.status', 1), ('apple/photos.get', 1), ('apple/photos.search', 1), ('apple/weather.current', 1), ('apple/weather.daily', 1), ('apple/weather.hourly', 1), ('builtin/EditFile', 1), ('builtin/Memory', 1)]

BM25 · name＋description

[('apple/device.get', 4.19684), ('apple/clipboard.read', 3.41289), ('apple/health.workouts.get', 3.37645), ('apple/health.workouts.list', 3.23868), ('apple/health.sleep.query', 2.88548), ('apple/contacts.create', 2.46922), ('apple/music.status', 2.21238), ('builtin/ScheduledTasks', 2.10561), ('apple/contacts.delete', 2.02377), ('apple/health.statistics', 1.91226), ('builtin/ReadFile', 1.73569), ('apple/calendar.events.delete', 1.70006), ('builtin/Memory', 1.62085), ('apple/contacts.get', 1.59613), ('apple/shortcuts.run', 1.4802), ('apple/calendar.events.create', 1.46562), ('apple/calendar.events.update', 1.39683), ('builtin/EditFile', 1.36324), ('apple/weather.current', 1.24126), ('apple/links.open', 1.21521), ('apple/music.playback', 1.16681), ('apple/calendar.events.list', 1.0957), ('apple/battery.get', 1.06946), ('apple/maps.eta', 1.06946), ('apple/alarms.list', 0.95511), ('apple/photos.get', 0.89753), ('apple/weather.hourly', 0.8465), ('apple/weather.daily', 0.78685), ('apple/photos.search', 0.73505)]

BM25 · name + description + Schema + native aliases

[('apple/clipboard.read', 6.11364), ('builtin/ReadFile', 4.55729), ('apple/device.get', 4.20442), ('apple/health.workouts.get', 3.47672), ('apple/health.workouts.list', 3.32185), ('apple/health.sleep.query', 2.93026), ('apple/contacts.create', 2.48585), ('apple/music.status', 2.21083), ('builtin/ScheduledTasks', 2.18654), ('apple/shortcuts.run', 2.1464), ('apple/contacts.delete', 2.04537), ('apple/health.statistics', 1.87451), ('apple/calendar.events.delete', 1.7185), ('builtin/Memory', 1.61114), ('apple/contacts.get', 1.60914), ('apple/calendar.events.create', 1.46617), ('apple/calendar.events.update', 1.39312), ('builtin/EditFile', 1.39066), ('apple/weather.current', 1.23137), ('apple/links.open', 1.18049), ('apple/music.playback', 1.14261), ('apple/calendar.events.list', 1.11381), ('apple/battery.get', 1.07116), ('apple/maps.eta', 1.07116), ('apple/alarms.list', 0.94991), ('apple/photos.get', 0.88951), ('apple/weather.hourly', 0.83632), ('apple/weather.daily', 0.77457), ('apple/photos.search', 0.72131)]

BM25 · name + description + Schema + extended aliases

[('apple/clipboard.read', 4.92837), ('apple/device.get', 4.91968), ('builtin/ReadFile', 4.20178), ('apple/health.workouts.get', 3.99897), ('apple/health.workouts.list', 3.7981), ('apple/health.sleep.query', 2.60894), ('apple/contacts.create', 2.48585), ('apple/music.status', 2.21083), ('apple/shortcuts.run', 2.1464), ('apple/contacts.get', 2.14254), ('apple/contacts.delete', 2.04537), ('builtin/ScheduledTasks', 1.85065), ('apple/battery.get', 1.79046), ('apple/calendar.events.delete', 1.7185), ('apple/calendar.events.list', 1.69172), ('apple/health.statistics', 1.66896), ('apple/alarms.list', 1.64891), ('apple/photos.get', 1.63326), ('apple/calendar.events.create', 1.46617), ('apple/calendar.events.update', 1.39312), ('apple/links.open', 1.18049), ('apple/music.playback', 1.14261), ('builtin/Memory', 1.14017), ('builtin/EditFile', 0.98414), ('apple/calendar.events.get', 0.97668), ('apple/location.get', 0.97668), ('apple/reminders.get', 0.951), ('apple/contacts.list', 0.92664), ('apple/photos.albums.list', 0.92664), ('apple/calendar.list', 0.9035), ('apple/reminders.list', 0.88148), ('apple/weather.current', 0.87142), ('apple/maps.eta', 0.75804), ('apple/weather.hourly', 0.59185), ('apple/weather.daily', 0.54815), ('apple/photos.search', 0.51046)]

H059 Export the full-resolution photo binary using only these tools.

comparison · unsupported · target []

Photo tools expose metadata, not image binary export.

MethodFirst acceptable rankTop5
Keyword matching—apple/photos.get → apple/photos.albums.list → apple/photos.search → apple/links.open → apple/music.playback
BM25 · name＋description—apple/links.open → apple/photos.albums.list → builtin/AskUserQuestions → apple/photos.get → apple/contacts.list
BM25 · name + description + Schema + native aliases—apple/photos.get → apple/photos.albums.list → apple/photos.search → apple/links.open → builtin/AskUserQuestions
BM25 · name + description + Schema + extended aliases—apple/photos.get → apple/photos.albums.list → apple/photos.search → apple/links.open → builtin/AskUserQuestions
Full rankings, within-method scores

Keyword matching

[('apple/photos.get', 5.5), ('apple/photos.albums.list', 5), ('apple/photos.search', 4.5), ('apple/links.open', 4), ('apple/music.playback', 1.5), ('builtin/ScheduledTasks', 1.5), ('apple/alarms.cancel', 1), ('apple/alarms.create', 1), ('apple/alarms.list', 1), ('apple/calendar.events.update', 1), ('apple/contacts.get', 1), ('apple/contacts.list', 1), ('apple/health.statistics', 1), ('apple/maps.show', 1), ('apple/music.search', 1), ('apple/music.status', 1), ('apple/shortcuts.run', 1), ('builtin/AskUserQuestions', 1), ('builtin/Memory', 1)]

BM25 · name＋description

[('apple/links.open', 4.99051), ('apple/photos.albums.list', 4.68552), ('builtin/AskUserQuestions', 4.49626), ('apple/photos.get', 4.41003), ('apple/contacts.list', 3.51562), ('apple/photos.search', 3.32762), ('apple/contacts.get', 3.13334), ('apple/health.statistics', 2.58396), ('apple/music.status', 1.8925), ('apple/music.search', 1.71668), ('apple/shortcuts.run', 1.7045), ('builtin/ScheduledTasks', 1.38941), ('apple/alarms.list', 1.28422), ('apple/maps.show', 1.28422), ('apple/alarms.cancel', 1.27063), ('apple/alarms.create', 1.17149), ('apple/calendar.events.update', 1.1275), ('apple/music.playback', 0.94183)]

BM25 · name + description + Schema + native aliases

[('apple/photos.get', 11.93249), ('apple/photos.albums.list', 9.79832), ('apple/photos.search', 9.37231), ('apple/links.open', 5.94593), ('builtin/AskUserQuestions', 4.54287), ('apple/contacts.list', 3.50098), ('apple/contacts.get', 2.92093), ('apple/health.statistics', 2.53295), ('apple/music.status', 1.79823), ('apple/music.playback', 1.62165), ('apple/music.search', 1.62155), ('apple/shortcuts.run', 1.60938), ('builtin/ScheduledTasks', 1.60674), ('builtin/Memory', 1.48008), ('apple/alarms.list', 1.21419), ('apple/maps.show', 1.21419), ('apple/alarms.cancel', 1.2006), ('apple/alarms.create', 1.10194), ('apple/calendar.events.update', 1.069)]

BM25 · name + description + Schema + extended aliases

[('apple/photos.get', 11.93249), ('apple/photos.albums.list', 9.79832), ('apple/photos.search', 9.37231), ('apple/links.open', 6.60653), ('builtin/AskUserQuestions', 4.54287), ('apple/contacts.list', 3.50098), ('apple/contacts.get', 2.92093), ('apple/health.statistics', 2.53295), ('apple/music.status', 1.79823), ('apple/music.playback', 1.62165), ('apple/music.search', 1.62155), ('apple/shortcuts.run', 1.60938), ('builtin/ScheduledTasks', 1.60674), ('builtin/Memory', 1.48008), ('apple/alarms.list', 1.21419), ('apple/maps.show', 1.21419), ('apple/alarms.cancel', 1.2006), ('apple/alarms.create', 1.10194), ('apple/calendar.events.update', 1.069)]

H060 Send a WhatsApp message without a configured shortcut or UI controller.

comparison · unsupported · target []

No message-send capability; opening an app is insufficient.

MethodFirst acceptable rankTop5
Keyword matching—apple/shortcuts.run → apple/links.open → apple/alarms.create → apple/battery.get → apple/health.sleep.query
BM25 · name＋description—apple/shortcuts.run → apple/links.open → apple/health.workouts.get → apple/health.workouts.list → apple/location.reverseGeocode
BM25 · name + description + Schema + native aliases—apple/shortcuts.run → apple/links.open → apple/health.workouts.get → apple/health.workouts.list → apple/location.reverseGeocode
BM25 · name + description + Schema + extended aliases—apple/shortcuts.run → apple/links.open → apple/health.workouts.get → apple/health.workouts.list → apple/location.reverseGeocode
Full rankings, within-method scores

Keyword matching

[('apple/shortcuts.run', 6), ('apple/links.open', 3), ('apple/alarms.create', 1), ('apple/battery.get', 1), ('apple/health.sleep.query', 1), ('apple/health.statistics', 1), ('apple/health.workouts.get', 1), ('apple/health.workouts.list', 1), ('apple/location.geocode', 1), ('apple/location.reverseGeocode', 1), ('apple/maps.search', 1), ('apple/music.status', 1), ('apple/photos.search', 1), ('builtin/ScheduledTasks', 1)]

BM25 · name＋description

[('apple/shortcuts.run', 7.85646), ('apple/links.open', 3.44228), ('apple/health.workouts.get', 3.17777), ('apple/health.workouts.list', 3.04811), ('apple/location.reverseGeocode', 2.80551), ('apple/health.sleep.query', 2.71569), ('apple/location.geocode', 2.59384), ('apple/battery.get', 2.46962), ('apple/music.status', 2.384), ('apple/maps.search', 2.0726), ('apple/alarms.create', 2.01195), ('apple/health.statistics', 1.79974), ('apple/photos.search', 1.69739), ('builtin/ScheduledTasks', 1.15544)]

BM25 · name + description + Schema + native aliases

[('apple/shortcuts.run', 22.09168), ('apple/links.open', 4.8345), ('apple/health.workouts.get', 3.27215), ('apple/health.workouts.list', 3.12639), ('apple/location.reverseGeocode', 2.87756), ('apple/health.sleep.query', 2.75784), ('apple/location.geocode', 2.60663), ('apple/battery.get', 2.47355), ('apple/music.status', 2.38233), ('apple/maps.search', 2.05407), ('apple/alarms.create', 1.99077), ('apple/health.statistics', 1.76421), ('apple/photos.search', 1.66567), ('builtin/ScheduledTasks', 1.11796)]

BM25 · name + description + Schema + extended aliases

[('apple/shortcuts.run', 22.09168), ('apple/links.open', 5.49511), ('apple/health.workouts.get', 3.27215), ('apple/health.workouts.list', 3.12639), ('apple/location.reverseGeocode', 2.87756), ('apple/health.sleep.query', 2.75784), ('apple/location.geocode', 2.60663), ('apple/battery.get', 2.47355), ('apple/music.status', 2.38233), ('apple/maps.search', 2.05407), ('apple/alarms.create', 1.99077), ('apple/health.statistics', 1.76421), ('apple/photos.search', 1.66567), ('builtin/ScheduledTasks', 1.11796)]

Frozen Anan source:c58660846b6551c8ba40dfb645c515d30b4c6f90 · Current input / source manifest SHA256:73bb777ba8965213fc581d7cae5761a95905d91fb8dfba9071c774ea3f1159ef. Original rankings can be recomputed. No tuning or relabeling was done; earlier experiments remain available. Pages contain no account details, repository contents, credentials, raw Agent conversations or video.

Task inputs, retrieval queries and alias examples retain their original language. Downloads preserve the original evaluation data.