Failed Jobs

Enqueued 0 / 0 Scheduled 0 Processing 0 Succeeded 365,899 Failed 958 Deleted 1,524 Awaiting 0

Failed jobs do not become expired to allow you to re-queue them without any time pressure. You should re-queue or delete them manually, or apply AutomaticRetry(OnAttemptsExceeded = AttemptsExceededAction.Delete) attribute to delete them automatically.

Id	Job	Failed
#70826	DispatchAdminNotificationJobService.ExecuteAsync An exception occurred during performance of the job. Fewer details...	06/30/2026 07:50:19
#70826	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") ---> System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it. at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionWithTelemetryAsync(HttpRequestMessage request, HttpConnectionPool pool, Boolean async, CancellationToken requestCancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpMessageInvoker.<SendAsync>g__SendAsyncWithTelemetry\|6_0(HttpMessageHandler handler, HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcAdminNotificationService.ExecuteAdminDispatch(AdminGrpcRequest request) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\GrpcServices\Services\GrpcAdminNotificationService.cs:line 38 at DispatchAdminNotificationJobService.ExecuteAsync(Int32 pageSize) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\Jobs\Notifications\DispatchAdminNotificationJobService.cs:line 18 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#70822	DispatchAdminNotificationJobService.ExecuteAsync An exception occurred during performance of the job. More details...	06/30/2026 07:49:09
#70822	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") ---> System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it. at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionWithTelemetryAsync(HttpRequestMessage request, HttpConnectionPool pool, Boolean async, CancellationToken requestCancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpMessageInvoker.<SendAsync>g__SendAsyncWithTelemetry\|6_0(HttpMessageHandler handler, HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcAdminNotificationService.ExecuteAdminDispatch(AdminGrpcRequest request) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\GrpcServices\Services\GrpcAdminNotificationService.cs:line 38 at DispatchAdminNotificationJobService.ExecuteAsync(Int32 pageSize) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\Jobs\Notifications\DispatchAdminNotificationJobService.cs:line 18 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#70820	ScoringEngineQueueProcessingService.ExecuteAsync An exception occurred during performance of the job. More details...	06/30/2026 07:51:12
#70820	Hangfire.PostgreSql.PostgreSqlDistributedLockException Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'hangfire:ScoringEngineQueueProcessingService.ExecuteAsync' resource. Hangfire.PostgreSql.PostgreSqlDistributedLockException: Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'hangfire:ScoringEngineQueueProcessingService.ExecuteAsync' resource. at Hangfire.PostgreSql.PostgreSqlDistributedLock.LockHandler.Lock(String resource, TimeSpan timeout, IDbConnection connection, PostgreSqlStorageOptions options) at Hangfire.PostgreSql.PostgreSqlConnection.AcquireLock(String resource, TimeSpan timeout) at Hangfire.PostgreSql.PostgreSqlConnection.AcquireDistributedLock(String resource, TimeSpan timeout) at Hangfire.DisableConcurrentExecutionAttribute.OnPerforming(PerformingContext context) in C:\projects\hangfire-525\src\Hangfire.Core\DisableConcurrentExecutionAttribute.cs:line 51 at Hangfire.Server.BackgroundJobPerformer.InvokeOnPerforming(KeyValuePair`2 ctx) in C:\projects\hangfire-525\src\Hangfire.Core\Server\BackgroundJobPerformer.cs:line 198
#70819	DispatchAdminNotificationJobService.ExecuteAsync An exception occurred during performance of the job. More details...	06/30/2026 07:47:16
#70819	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") ---> System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it. at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionWithTelemetryAsync(HttpRequestMessage request, HttpConnectionPool pool, Boolean async, CancellationToken requestCancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpMessageInvoker.<SendAsync>g__SendAsyncWithTelemetry\|6_0(HttpMessageHandler handler, HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcAdminNotificationService.ExecuteAdminDispatch(AdminGrpcRequest request) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\GrpcServices\Services\GrpcAdminNotificationService.cs:line 38 at DispatchAdminNotificationJobService.ExecuteAsync(Int32 pageSize) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\Jobs\Notifications\DispatchAdminNotificationJobService.cs:line 18 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#70818	DispatchAdminNotificationJobService.ExecuteAsync An exception occurred during performance of the job. More details...	06/30/2026 07:46:11
#70818	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") ---> System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it. at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionWithTelemetryAsync(HttpRequestMessage request, HttpConnectionPool pool, Boolean async, CancellationToken requestCancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpMessageInvoker.<SendAsync>g__SendAsyncWithTelemetry\|6_0(HttpMessageHandler handler, HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcAdminNotificationService.ExecuteAdminDispatch(AdminGrpcRequest request) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\GrpcServices\Services\GrpcAdminNotificationService.cs:line 38 at DispatchAdminNotificationJobService.ExecuteAsync(Int32 pageSize) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\Jobs\Notifications\DispatchAdminNotificationJobService.cs:line 18 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#70813	DispatchAdminNotificationJobService.ExecuteAsync An exception occurred during performance of the job. More details...	06/30/2026 07:44:17
#70813	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") ---> System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it. at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionWithTelemetryAsync(HttpRequestMessage request, HttpConnectionPool pool, Boolean async, CancellationToken requestCancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpMessageInvoker.<SendAsync>g__SendAsyncWithTelemetry\|6_0(HttpMessageHandler handler, HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcAdminNotificationService.ExecuteAdminDispatch(AdminGrpcRequest request) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\GrpcServices\Services\GrpcAdminNotificationService.cs:line 38 at DispatchAdminNotificationJobService.ExecuteAsync(Int32 pageSize) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\Jobs\Notifications\DispatchAdminNotificationJobService.cs:line 18 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#70811	DispatchAdminNotificationJobService.ExecuteAsync An exception occurred during performance of the job. More details...	06/30/2026 07:43:09
#70811	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") ---> System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it. at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionWithTelemetryAsync(HttpRequestMessage request, HttpConnectionPool pool, Boolean async, CancellationToken requestCancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpMessageInvoker.<SendAsync>g__SendAsyncWithTelemetry\|6_0(HttpMessageHandler handler, HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcAdminNotificationService.ExecuteAdminDispatch(AdminGrpcRequest request) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\GrpcServices\Services\GrpcAdminNotificationService.cs:line 38 at DispatchAdminNotificationJobService.ExecuteAsync(Int32 pageSize) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\Jobs\Notifications\DispatchAdminNotificationJobService.cs:line 18 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#70800	IdentityJobsService.ExecuteAsync An exception occurred during performance of the job. More details...	06/30/2026 07:42:38
#70800	System.ArgumentNullException Value cannot be null. (Parameter 'uriString') System.ArgumentNullException: Value cannot be null. (Parameter 'uriString') at System.ArgumentNullException.Throw(String paramName) at System.ArgumentNullException.ThrowIfNull(Object argument, String paramName) at System.Uri..ctor(String uriString) at Grpc.Net.Client.GrpcChannel.ForAddress(String address, GrpcChannelOptions channelOptions) at Gateway.Services.GrpcServices.Services.GrpcMobileUserClientService..ctor(IConfiguration configuration, IHttpContextAccessor contextAccessor, ILogger`1 logger, ILoggerFactory loggerFactory) in /src/Hangfire/GrpcServices/Services/GrpcMobileUserClientService.cs:line 31 at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor) at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr) at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite callSite, RuntimeResolverContext context) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument) at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope) at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateServiceAccessor(ServiceIdentifier serviceIdentifier) at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory) at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType) at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType) at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider) at IdentityJobsService.ExecuteAsync(Int32 pageSize) in /src/Hangfire/Jobs/Identity/IdentityJobsService.cs:line 23 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#70789	DispatchAdminNotificationJobService.ExecuteAsync An exception occurred during performance of the job. More details...	06/30/2026 07:40:13
#70789	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) SocketException: No connection could be made because the target machine actively refused it.", DebugException="System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107)") ---> System.Net.Http.HttpRequestException: No connection could be made because the target machine actively refused it. (localhost:7107) ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it. at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionWithTelemetryAsync(HttpRequestMessage request, HttpConnectionPool pool, Boolean async, CancellationToken requestCancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpMessageInvoker.<SendAsync>g__SendAsyncWithTelemetry\|6_0(HttpMessageHandler handler, HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcAdminNotificationService.ExecuteAdminDispatch(AdminGrpcRequest request) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\GrpcServices\Services\GrpcAdminNotificationService.cs:line 38 at DispatchAdminNotificationJobService.ExecuteAsync(Int32 pageSize) in C:\Work\Madarek\Madark-HangFire\Madark-HangFire\Hangfire\Jobs\Notifications\DispatchAdminNotificationJobService.cs:line 18 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#70783	ScoringEngineQueueProcessingService.ExecuteAsync An exception occurred during performance of the job. More details...	06/30/2026 07:41:10
#70783	Hangfire.PostgreSql.PostgreSqlDistributedLockException Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'hangfire:ScoringEngineQueueProcessingService.ExecuteAsync' resource. Hangfire.PostgreSql.PostgreSqlDistributedLockException: Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'hangfire:ScoringEngineQueueProcessingService.ExecuteAsync' resource. at Hangfire.PostgreSql.PostgreSqlDistributedLock.LockHandler.Lock(String resource, TimeSpan timeout, IDbConnection connection, PostgreSqlStorageOptions options) at Hangfire.PostgreSql.PostgreSqlConnection.AcquireLock(String resource, TimeSpan timeout) at Hangfire.DisableConcurrentExecutionAttribute.OnPerforming(PerformingContext context) in C:\projects\hangfire-525\src\Hangfire.Core\DisableConcurrentExecutionAttribute.cs:line 48 at Hangfire.Server.BackgroundJobPerformer.InvokeOnPerforming(KeyValuePair`2 ctx) in C:\projects\hangfire-525\src\Hangfire.Core\Server\BackgroundJobPerformer.cs:line 198

Grpc.Core.RpcException

Grpc.Core.RpcException

Hangfire.PostgreSql.PostgreSqlDistributedLockException

Grpc.Core.RpcException

Grpc.Core.RpcException

Grpc.Core.RpcException

Grpc.Core.RpcException

System.ArgumentNullException

Grpc.Core.RpcException

Hangfire.PostgreSql.PostgreSqlDistributedLockException