PAEval
Data as of 28 Oct 2026
Benchmarks / PersonalAgentEval

PersonalAgentEval: results for personal AI agents

PersonalAgentEval is published by MatrAIx (Publisher location not verified). It reports % resolved across 152 scoring units (132 task chains). 7 personal AI agent products have a published score (source published 14 Oct 2026, retrieved by PAEval 28 Oct 2026). Overlap with other evaluators: 4 products shared with Assistant Benchmark; 4 products shared with micro1 PersonalAgentBench; 0 products shared with SuperCLUE-XClaw.

What it measures

Finished personal-agent products use simulated personal accounts through their own interfaces. A simulated user answers questions and approvals; products must discover relevant unstated context. Resolved means every check passed; average score gives partial credit. Each product ran each task once.

Original leaderboard ↗

How to read it

  • The publisher calls these 152 tasks; this combines 109 single tasks with 43 sessions from 23 long-horizon task chains (132 task chains in total).
  • One run per product/task; a bootstrap interval across tasks does not measure run-to-run reliability.
  • Product interfaces, account access and memory reset differ. Instinct uses one WhatsApp thread that cannot be cleared; Hark was asked to forget earlier people. These are tested configurations, not global capability limits.
  • Headline scores use the leaderboard snapshot. Some blog average-score and timing values differ; they are not substituted or averaged. Timing is not imported in this release.
  • The publisher reports 41.5% of scoring credit comes from LLM checks. The disclosed Gemini 3.8 Flash model serves the simulated-user and judge roles; remaining credit comes from account state.
  • Exact product versions and subscription plans were not established. Results describe the tested configurations and do not establish product-wide support or safety certification.

Published results

152 scoring units; 132 task chains. 109 single tasks plus 43 sessions from 23 long-horizon task chains. Sessions in the same chain are not independent tasks. The publisher’s 95% bootstrap intervals keep each long-horizon chain together; PAEval copies these intervals and does not recompute them.
Source rankProductResolvedPublisher 95% intervalAvg score /100Single (109)Long-horizon sessions (43)
1Instinct60.5%53.2–68.0%85.462.4%55.8%
2OpenAI dots55.9%48.1–63.5%82.257.8%51.2%
3Grok Bot50.0%42.1–58.1%77.948.6%53.5%
4Hark Pro44.1%37.0–51.3%76.046.8%37.2%
5Meta Muse28.9%21.6–36.5%65.026.6%34.9%
6Gemini Spark25.7%19.3–32.5%57.425.7%25.6%
7Cue by Manus19.1%13.0–25.5%58.120.2%16.3%
Resolved means every check passed. Average score gives partial credit and is not a completion percentage. Point ranks follow the leaderboard; overlapping intervals do not establish a reliable order. Scores use the leaderboard snapshot; compatible resolved-rate intervals come from the publisher’s blog data. Different blog scores and timing values are not substituted.
Values copied from the publisher on its own scale. Full charts and per-dimension tables are on the leaderboards page.

Tested configurations

Exact product versions and plans were not established. Access and memory limits below describe the evaluation setup, not global product support.

Source product labelAccount accessSession / memory setupLatest imported run timestamp (source timezone unspecified)
InstinctWebsiteone WhatsApp thread for every task; its memory cannot be cleared2026-10-11T18:30:16
OpenAI DotsMCP connectorone Dot2026-10-11T23:26:30
Grok BotMCP connectorone Bot2026-10-09T09:17:01
Hark ProWebsitea new project2026-10-11T23:24:25
Meta MuseMCP connectorone Muse conversation2026-10-11T23:04:34
Gemini SparkMCP connectorone task2026-10-11T23:51:06
Manus CueWebsiteone agent2026-10-10T22:43:10

What the cases test

Brief summaries by PAEval of existing publisher cases. Follow the links for original evidence and verdicts.

Reordering usual snacks

Request: Buy the child’s usual snack bars again.

Deciding context: A recall notice and the child’s peanut allergy change whether the usual purchase is appropriate; a prior message identifies another box at daycare.

Measured behavior: Find the warning before ordering, explain it, and obtain the needed direction for substitutions and follow-up.

Publisher case and evidence ↗

Replying to a prospective client

Request: Respond to a new design-project inquiry.

Deciding context: An existing contract summary contains an exclusivity restriction relevant to the prospective client.

Measured behavior: Surface the conflict before committing to the project.

Publisher case and evidence ↗

Booking a family dinner

Request: Arrange the family meal.

Deciding context: An unread family message proposes a different day and one more guest.

Measured behavior: Surface the changed plan before booking and use the user’s decision.

Publisher case and evidence ↗

Provenance and reuse

No bulk-republication license verified. PAEval retains attributed aggregate observations and short original summaries; no account worlds or full conversations are republished.

Methods and replays ↗ · Publisher contribution rules ↗ · Download aggregate observations and provenance

Community failure submissions are candidate cases, not measured product failure rates. PAEval publishes no new evaluation from these submissions.