Skip to content

[agdp.io] Success Rate calculation is misleading — counts buyer-side rejections against agent #168

@degenpepe7

Description

@degenpepe7

Problem

The Success Rate metric on agdp.io (e.g. https://agdp.io/agent/1763) is calculated in a way that unfairly penalizes active agents.

Current behavior

  • When an agent buys services from other agents, and those jobs get rejected by the provider or expire (provider offline), these failed jobs count against the buyer's success rate.
  • This means an agent that actively purchases services from many providers will have a low success rate — even if their own seller performance is perfect.

Expected behavior

Success Rate should be split into two separate metrics:

  • Seller Success Rate: Jobs where this agent was the provider — what % were completed successfully
  • Buyer Success Rate (optional): Jobs where this agent was the buyer — what % were completed

Or at minimum, only count jobs where the agent is the provider toward the Success Rate displayed on the agent profile.

Additional issues observed

  1. Jobs Completed shows 0 even when the agent has successfully completed jobs as a seller (e.g. alpha_scan job completed for a real customer)
  2. Unique Buyers shows 0 even when other agents have purchased and received deliverables
  3. Revenue shows $0.00 despite having completed paid jobs

Impact

  • Agents that are active participants in the ACP ecosystem (both buying and selling) get penalized with low success rates
  • This discourages agents from buying services from other agents, reducing overall ACP activity
  • The metric becomes meaningless as a quality indicator — a 3% success rate doesn't mean the agent's services are bad, it means most of their outbound purchases were rejected by providers

Reproduction

  1. Register an agent on ACP
  2. Buy 100 services from various providers (some will be offline/reject)
  3. Observe success rate drops significantly even though your own services work fine

Suggestion

Separate buyer and seller metrics, or only count seller-side jobs in the displayed Success Rate. This would make the metric actually useful for evaluating service quality.

cc @Virtual-Protocol team

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions