CLDSRV-821: add backbeat calls to logs #6050

leif-scality · 2026-01-16T18:49:14Z

This PR makes cloudserver write backbeat requests to the server access logs file.

The loggingEnabled field is always set to false to prevent the log from entering the server-access clickhouse table and it being consumed by log courier.
The operation field is always set to REST.${req.method}.BACKBEAT
The action field is set to the backbeat route or BACKBEAT_INVALID if the route is not valid or does not exists.

bert-e · 2026-01-16T18:49:17Z

Hello leif-scality,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2026-01-16T18:49:23Z

Incorrect fix version

The Fix Version/s in issue CLDSRV-821 contains:

None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

9.2.15
9.3.0

Please check the Fix Version/s of CLDSRV-821, or the target
branch of this pull request.

codecov · 2026-01-16T18:54:45Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.38%. Comparing base (c022d57) to head (e145523).
✅ All tests successful. No failed tests found.

Additional details and impacted files

Files with missing lines	Coverage Δ
lib/routes/routeBackbeat.js	`76.84% <100.00%> (+0.51%)`	⬆️
lib/server.js	`79.41% <100.00%> (+0.20%)`	⬆️
lib/utilities/serverAccessLogger.js	`77.01% <100.00%> (+0.54%)`	⬆️

@@                 Coverage Diff                 @@
##           development/9.2    #6050      +/-   ##
===================================================
+ Coverage            84.35%   84.38%   +0.02%     
===================================================
  Files                  206      206              
  Lines                12997    13019      +22     
===================================================
+ Hits                 10964    10986      +22     
  Misses                2033     2033

Flag	Coverage Δ
file-ft-tests	`67.44% <96.00%> (+0.03%)`	⬆️
kmip-ft-tests	`28.10% <20.00%> (-0.02%)`	⬇️
mongo-v0-ft-tests	`68.66% <96.00%> (+0.01%)`	⬆️
mongo-v1-ft-tests	`68.68% <96.00%> (+0.01%)`	⬆️
multiple-backend	`35.40% <100.00%> (+0.11%)`	⬆️
sur-tests	`35.57% <20.00%> (-0.84%)`	⬇️
sur-tests-inflights	`37.38% <20.00%> (-0.07%)`	⬇️
unit	`69.89% <44.00%> (-0.06%)`	⬇️
utapi-v2-tests	`34.27% <20.00%> (-0.03%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…tion

francoisferrand · 2026-01-19T08:08:24Z

Should we log backbeat calls in Bucket Logging?
I am really just asking, because backbeat routes are often kind of "internal" calls (typically used for inter-site replication or lifecycle), and as such not sure what the expected semantics is: do we want the user to actually see everything that is happening on the platform, or only "user-facing" operations (like AWS I guess) ?

scality-gdoumergue · 2026-01-19T08:15:36Z

Hello @leif-scality, can you confirm that backbeat calls will still be seen in Grafana dashboards, for CS and customers to troubleshoot and measure backbeat-backed features performance? Thank you.

bert-e · 2026-01-19T08:15:45Z

Incorrect fix version

The Fix Version/s in issue CLDSRV-821 contains:

None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

9.2.16
9.3.0

Please check the Fix Version/s of CLDSRV-821, or the target
branch of this pull request.

dvasilas

Let's add tests for backbeat routes to tests/unit/utils/serverAccessLogger.js?

dvasilas · 2026-01-19T08:39:55Z

lib/server.js

-        if (isLoggingEnabled && !req.url.startsWith('/_/')) {
+        const isInternalRoute = req.url.startsWith('/_');
+        const isBackbeatRoute = req.url.startsWith('/_/backbeat/');
+        if (isLoggingEnabled && isInternalRoute ? isBackbeatRoute : true) {


nit: Could be refactored to

Suggested change

if (isLoggingEnabled && isInternalRoute ? isBackbeatRoute : true) {

if (isLoggingEnabled && (!isInternalRoute || isBackbeatRoute);) {

dvasilas · 2026-01-19T08:41:33Z

lib/routes/routeBackbeat.js

+        }
+
+        // eslint-disable-next-line no-param-reassign
+        request.serverAccessLog.analyticsAction = route.name;


Suggested change

request.serverAccessLog.analyticsAction = route.name;

request.serverAccessLog.analyticsAction = route?.name ?? 'BACKBEAT_INVALID';

Normally route is never undefined or null because it is checked above, but added the check just in case

nicolas2bert · 2026-01-19T09:10:05Z

@scality-gdoumergue about

Hello @leif-scality, can you confirm that backbeat calls will still be seen in Grafana dashboards, for CS and customers to troubleshoot and measure backbeat-backed features performance? Thank you.

yes, however this improvement https://scality.atlassian.net/browse/RD-1246 won't be there yet. We will be working on it after it is merged.

dvasilas · 2026-01-19T09:40:01Z

Should we log backbeat calls in Bucket Logging? I am really just asking, because backbeat routes are often kind of "internal" calls (typically used for inter-site replication or lifecycle), and as such not sure what the expected semantics is: do we want the user to actually see everything that is happening on the platform, or only "user-facing" operations (like AWS I guess) ?

@francoisferrand we will not log internal backbeat calls in bucket logging. We add them to the server-access.logs file so that they appear in S3 analytics Grafana dashboards, but loggingEnabled (controls bucket logging) will be set to false.

leif-scality force-pushed the improvement/CLDSRV-821-add-backbeat-calls-to-logs branch from b8f7e9c to e733e90 Compare January 16, 2026 18:53

leif-scality added 3 commits January 16, 2026 20:07

CLDSRV-821: record and write backbeat server access log data

6691967

CLDSRV-821: update server_access_log.schema.json action field descrip…

b62ecbc

…tion

CLDSRV-821: log backbeat routes

e145523

leif-scality force-pushed the improvement/CLDSRV-821-add-backbeat-calls-to-logs branch from e733e90 to e145523 Compare January 16, 2026 19:07

dvasilas reviewed Jan 19, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

CLDSRV-821: add backbeat calls to logs #6050

CLDSRV-821: add backbeat calls to logs #6050

leif-scality commented Jan 16, 2026

Uh oh!

bert-e commented Jan 16, 2026

Uh oh!

bert-e commented Jan 16, 2026

Uh oh!

codecov bot commented Jan 16, 2026 •

edited

Loading

Uh oh!

francoisferrand commented Jan 19, 2026

Uh oh!

scality-gdoumergue commented Jan 19, 2026

Uh oh!

bert-e commented Jan 19, 2026

Uh oh!

dvasilas left a comment

Uh oh!

dvasilas Jan 19, 2026

Uh oh!

dvasilas Jan 19, 2026

Uh oh!

leif-scality Jan 19, 2026

Uh oh!

nicolas2bert commented Jan 19, 2026

Uh oh!

dvasilas commented Jan 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

	if (isLoggingEnabled && isInternalRoute ? isBackbeatRoute : true) {
	if (isLoggingEnabled && (!isInternalRoute \|\| isBackbeatRoute);) {

	request.serverAccessLog.analyticsAction = route.name;
	request.serverAccessLog.analyticsAction = route?.name ?? 'BACKBEAT_INVALID';

CLDSRV-821: add backbeat calls to logs #6050

Are you sure you want to change the base?

CLDSRV-821: add backbeat calls to logs #6050

Conversation

leif-scality commented Jan 16, 2026

Uh oh!

bert-e commented Jan 16, 2026

Hello leif-scality,

Uh oh!

bert-e commented Jan 16, 2026

Incorrect fix version

Uh oh!

codecov bot commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

francoisferrand commented Jan 19, 2026

Uh oh!

scality-gdoumergue commented Jan 19, 2026

Uh oh!

bert-e commented Jan 19, 2026

Incorrect fix version

Uh oh!

dvasilas left a comment

Choose a reason for hiding this comment

Uh oh!

dvasilas Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

dvasilas Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

leif-scality Jan 19, 2026

Choose a reason for hiding this comment

Uh oh!

nicolas2bert commented Jan 19, 2026

Uh oh!

dvasilas commented Jan 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

codecov bot commented Jan 16, 2026 •

edited

Loading