Runner gets stuck in infinite loop of retries when registration is not found #4191
Description
Describe the bug
When a runner cannot locate the registration, it enters an infinite retry loop.
Expected behavior
Terminate after the set number of retries fail. Do not reset retries and keep going.
Runner Version and Platform
Version of your runner?
Current runner version: '2.330.0'
OS of the machine running the runner? OSX/Windows/Linux/...
Running on EKS 1.32 using bottlerocket-aws-k8s-1.32-x86_64-v1.42.0-5ed15786
What's not working?
Registration error:
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] Catch exception during request
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] GitHub.Services.OAuth.VssOAuthTokenRequestException: Registration <REDACTED> was not found.
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] at GitHub.Services.OAuth.VssOAuthTokenProvider.OnGetTokenAsync(IssuedToken failedToken, CancellationToken cancellationToken)
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] at GitHub.Services.Common.IssuedTokenProvider.GetTokenOperation.GetTokenAsync(VssTraceActivity traceActivity)
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] at GitHub.Services.Common.IssuedTokenProvider.GetTokenAsync(IssuedToken failedToken, CancellationToken cancellationToken)
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] at GitHub.Services.Common.RawHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] at GitHub.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] at Sdk.WebApi.WebApi.RawHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] at Sdk.WebApi.WebApi.RawHttpClientBase.SendAsync[T](HttpRequestMessage message, Boolean readErrorBody, Object userState, CancellationToken cancellationToken)
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] at Sdk.WebApi.WebApi.RawHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Uri requestUri, HttpContent content, IEnumerable`1 queryParameters, Boolean readErrorBody, Object userState, CancellationToken cancellationToken)
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] at GitHub.Actions.RunService.WebApi.BrokerHttpClient.GetRunnerMessageAsync(Nullable`1 sessionId, String runnerVersion, Nullable`1 status, String os, String architecture, Nullable`1 disableUpdate, CancellationToken cancellationToken)
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] at GitHub.Runner.Common.BrokerServer.<>c__DisplayClass7_0.<<GetRunnerMessageAsync>b__0>d.MoveNext()
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] --- End of stack trace from previous location ---
[RUNNER 2026-01-09 17:55:48Z ERR BrokerServer] at GitHub.Runner.Common.RunnerService.RetryRequest[T](Func`1 func, CancellationToken cancellationToken, Int32 maxAttempts, Func`2 shouldRetry)
[RUNNER 2026-01-09 17:55:48Z WARN BrokerServer] Back off 9.418 seconds before next retry. 4 attempt left.
It says it will only retry 4 times, but continues forever. This leaves the pod in the "Running" status so it is never terminated without manual intervention.
It goes from 4 attempt left -> 3 attempt left -> 2 attempt left -> 1 attempt left -> 4 attempt left and repeats forever until the pod is deleted.
Job Log Output
Jobs are not picked up. They remain queued until the runner pods are deleted.
This can impact one set of runners while others remain unaffected at the same time.