Skip to content

Do not use abandoned arms as pending points; add completed trials without data as pending#4955

Closed
sdaulton wants to merge 1 commit intofacebook:mainfrom
sdaulton:export-D94531161
Closed

Do not use abandoned arms as pending points; add completed trials without data as pending#4955
sdaulton wants to merge 1 commit intofacebook:mainfrom
sdaulton:export-D94531161

Conversation

@sdaulton
Copy link
Contributor

Summary:
Two changes to how pending observation features are computed:

  1. Do not use individually abandoned arms as pending points. Previously,
    arms that were individually abandoned within a BatchTrial (via
    mark_arm_abandoned) were explicitly added as pending observation features.
    This is removed — individually abandoned arms should not occupy the pending
    point space. Note: arms belonging to fully abandoned trials are still
    treated as pending, since the entire trial was abandoned and the model should
    avoid re-suggesting those configurations.

  2. Treat completed trials without data as pending. Previously, completed
    trials were never considered pending, even if they had no data for some
    metrics. Now, get_pending_observation_features checks completed trials for
    missing metric data and marks those arms as pending for the missing metrics,
    consistent with how deployed (STAGED/RUNNING) trials are handled.

Differential Revision: D94531161

…hout data as pending

Summary:
Two changes to how pending observation features are computed:

1. **Do not use individually abandoned arms as pending points.** Previously,
   arms that were individually abandoned within a BatchTrial (via
   `mark_arm_abandoned`) were explicitly added as pending observation features.
   This is removed — individually abandoned arms should not occupy the pending
   point space. Note: arms belonging to fully *abandoned trials* are still
   treated as pending, since the entire trial was abandoned and the model should
   avoid re-suggesting those configurations.

2. **Treat completed trials without data as pending.** Previously, completed
   trials were never considered pending, even if they had no data for some
   metrics. Now, `get_pending_observation_features` checks completed trials for
   missing metric data and marks those arms as pending for the missing metrics,
   consistent with how deployed (STAGED/RUNNING) trials are handled.

Differential Revision: D94531161
@meta-cla meta-cla bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label Feb 26, 2026
@meta-codesync
Copy link

meta-codesync bot commented Feb 26, 2026

@sdaulton has exported this pull request. If you are a Meta employee, you can view the originating Diff in D94531161.

@meta-codesync
Copy link

meta-codesync bot commented Feb 27, 2026

This pull request has been merged in 8215f32.

@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.83%. Comparing base (c2f9aec) to head (21758f6).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4955      +/-   ##
==========================================
- Coverage   96.83%   96.83%   -0.01%     
==========================================
  Files         596      596              
  Lines       63108    63103       -5     
==========================================
- Hits        61109    61103       -6     
- Misses       1999     2000       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed Do not delete this pull request or issue due to inactivity. fb-exported Merged meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants