-
Notifications
You must be signed in to change notification settings - Fork 12
Open
Description
Problem
The Success Rate metric on agdp.io (e.g. https://agdp.io/agent/1763) is calculated in a way that unfairly penalizes active agents.
Current behavior
- When an agent buys services from other agents, and those jobs get rejected by the provider or expire (provider offline), these failed jobs count against the buyer's success rate.
- This means an agent that actively purchases services from many providers will have a low success rate — even if their own seller performance is perfect.
Expected behavior
Success Rate should be split into two separate metrics:
- Seller Success Rate: Jobs where this agent was the provider — what % were completed successfully
- Buyer Success Rate (optional): Jobs where this agent was the buyer — what % were completed
Or at minimum, only count jobs where the agent is the provider toward the Success Rate displayed on the agent profile.
Additional issues observed
- Jobs Completed shows 0 even when the agent has successfully completed jobs as a seller (e.g.
alpha_scanjob completed for a real customer) - Unique Buyers shows 0 even when other agents have purchased and received deliverables
- Revenue shows $0.00 despite having completed paid jobs
Impact
- Agents that are active participants in the ACP ecosystem (both buying and selling) get penalized with low success rates
- This discourages agents from buying services from other agents, reducing overall ACP activity
- The metric becomes meaningless as a quality indicator — a 3% success rate doesn't mean the agent's services are bad, it means most of their outbound purchases were rejected by providers
Reproduction
- Register an agent on ACP
- Buy 100 services from various providers (some will be offline/reject)
- Observe success rate drops significantly even though your own services work fine
Suggestion
Separate buyer and seller metrics, or only count seller-side jobs in the displayed Success Rate. This would make the metric actually useful for evaluating service quality.
cc @Virtual-Protocol team
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels