Assistant Benchmark: consumer travel
Five services are observed through booking, seats, changes and cancellation. Miso includes a human travel team.
David Pawlan / Assistant Benchmark · Updated 2026-10-23 · reviewed 2026-10-29
Scope and method
Five services × eight scored dimensions; dimensions can share a trip. Independent trip/repeat count is not established. Protocol travel-v1 is Draft.
Anchored 1–10 reviewed scores: booking speed receives 40%, the other seven dimensions share 60%. Speed score starts at flight selection. Human assistance is recorded separately.
How PAEval uses it
A travel track of the existing AB family, not an additional independent evaluator. Miso remains a source-labelled travel service, not a new profiled core PA.
Evidence family: assistant-benchmark. Shared tasks and tracks are not independent evidence votes.
Published travel ratings /10
Publisher weighted overall copied unchanged; do not average it with the general AB score.
| Dimension | Miso | Instinct | Grok Bot | Dots | Muse |
|---|---|---|---|---|---|
| Overall | 9.3 | 8.0 | 7.9 | 7.1 | 6.9 |
| Onboarding | 10 | 7 | 7 | 7 | 7 |
| Booking | 7 | 7 | 10 | 7 | 10 |
| Seats | 10 | 10 | 9 | 10 | 3 |
| Traveler profile | 10 | 10 | 10 | 10 | 7 |
| Cancellation | 7 | 7 | 9 | 7 | 10 |
| Rebooking | 10 | 3 | 7 | 10 | 3 |
| Booking speed | 10 | 9 | 7 | 5 | 7 |
| Proactiveness | 8 | 7 | 7 | 8 | 8 |
Intervention and follow-through
Observed instances; not guarantees of current support or future fulfillment.
| Service | Changes | Cancellation / refund follow-up |
|---|---|---|
| Miso | Cancelled and rebooked; human backup | Human team confirmed seat refund |
| Instinct | Not completed; support needed | $35 seat-refund request failed |
| Grok Bot | User clicked Change flight | User completed steps including cancellation |
| Dots | Completed | $60.01 credit unresolved |
| Muse | Not completed | Full refund confirmed |
Limits and interpretation
- Different trips, fares, starting profiles and intervention levels; no controlled success-rate estimate.
- Cancellation does not establish refund settlement. General-board travel cells and this report are overlapping evidence.
- Miso had a saved traveler profile; its human backup is part of service delivery.
Sources and reuse
PAEval summarizes existing published evidence and runs no new tests. Public readability does not establish permission to redistribute full datasets, conversations or images.
Related product records
Instinct · Grok Bot · OpenAI dots · Meta Muse
All benchmark sources → · How to read outcomes and intervention →