Failed Jobs

Enqueued 0 / 0 Scheduled 0 Processing 0 Succeeded 65,640 Failed 309 Deleted 1,524 Awaiting 0

Failed jobs do not become expired to allow you to re-queue them without any time pressure. You should re-queue or delete them manually, or apply AutomaticRetry(OnAttemptsExceeded = AttemptsExceededAction.Delete) attribute to delete them automatically.

Id	Job	Failed
#58334	SimahInstantUpdateProcessingService.ExecuteAsync An exception occurred during performance of the job. Fewer details...	06/26/2026 08:25:12
#58334	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcIntegrationService.ProcessSimahInstantUpdate(GrpcRequest request) in /src/Hangfire/GrpcServices/Services/GrpcIntegrationService.cs:line 87 at SimahInstantUpdateProcessingService.ExecuteAsync(Int32 pageSize) in /src/Hangfire/Jobs/Integration/SimahInstantUpdateProcessingService.cs:line 19 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#58324	SimahInstantUpdateProcessingService.ExecuteAsync An exception occurred during performance of the job. More details...	06/26/2026 08:20:12
#58324	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcIntegrationService.ProcessSimahInstantUpdate(GrpcRequest request) in /src/Hangfire/GrpcServices/Services/GrpcIntegrationService.cs:line 87 at SimahInstantUpdateProcessingService.ExecuteAsync(Int32 pageSize) in /src/Hangfire/Jobs/Integration/SimahInstantUpdateProcessingService.cs:line 19 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#58315	SimahInstantUpdateProcessingService.ExecuteAsync An exception occurred during performance of the job. More details...	06/26/2026 08:15:12
#58315	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcIntegrationService.ProcessSimahInstantUpdate(GrpcRequest request) in /src/Hangfire/GrpcServices/Services/GrpcIntegrationService.cs:line 87 at SimahInstantUpdateProcessingService.ExecuteAsync(Int32 pageSize) in /src/Hangfire/Jobs/Integration/SimahInstantUpdateProcessingService.cs:line 19 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#58305	SimahInstantUpdateProcessingService.ExecuteAsync An exception occurred during performance of the job. More details...	06/26/2026 08:10:12
#58305	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcIntegrationService.ProcessSimahInstantUpdate(GrpcRequest request) in /src/Hangfire/GrpcServices/Services/GrpcIntegrationService.cs:line 87 at SimahInstantUpdateProcessingService.ExecuteAsync(Int32 pageSize) in /src/Hangfire/Jobs/Integration/SimahInstantUpdateProcessingService.cs:line 19 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#58297	SimahInstantUpdateProcessingService.ExecuteAsync An exception occurred during performance of the job. More details...	06/26/2026 08:05:11
#58297	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcIntegrationService.ProcessSimahInstantUpdate(GrpcRequest request) in /src/Hangfire/GrpcServices/Services/GrpcIntegrationService.cs:line 87 at SimahInstantUpdateProcessingService.ExecuteAsync(Int32 pageSize) in /src/Hangfire/Jobs/Integration/SimahInstantUpdateProcessingService.cs:line 19 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#58286	SimahInstantUpdateProcessingService.ExecuteAsync An exception occurred during performance of the job. More details...	06/26/2026 08:00:11
#58286	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcIntegrationService.ProcessSimahInstantUpdate(GrpcRequest request) in /src/Hangfire/GrpcServices/Services/GrpcIntegrationService.cs:line 87 at SimahInstantUpdateProcessingService.ExecuteAsync(Int32 pageSize) in /src/Hangfire/Jobs/Integration/SimahInstantUpdateProcessingService.cs:line 19 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#58285	BenchMatrixDataCollectionService.ExecuteAsync An exception occurred during performance of the job. More details...	06/26/2026 08:00:11
#58285	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcIntegrationService.ProcessBenchMatrixDataCollection(GrpcRequest request) in /src/Hangfire/GrpcServices/Services/GrpcIntegrationService.cs:line 72 at BenchMatrixDataCollectionService.ExecuteAsync(Int32 pageSize) in /src/Hangfire/Jobs/Integration/BenchMatrixDataCollectionService.cs:line 19 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#58272	SimahInstantUpdateProcessingService.ExecuteAsync An exception occurred during performance of the job. More details...	06/26/2026 07:55:11
#58272	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcIntegrationService.ProcessSimahInstantUpdate(GrpcRequest request) in /src/Hangfire/GrpcServices/Services/GrpcIntegrationService.cs:line 87 at SimahInstantUpdateProcessingService.ExecuteAsync(Int32 pageSize) in /src/Hangfire/Jobs/Integration/SimahInstantUpdateProcessingService.cs:line 19 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#58262	SimahInstantUpdateProcessingService.ExecuteAsync An exception occurred during performance of the job. More details...	06/26/2026 07:50:11
#58262	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcIntegrationService.ProcessSimahInstantUpdate(GrpcRequest request) in /src/Hangfire/GrpcServices/Services/GrpcIntegrationService.cs:line 87 at SimahInstantUpdateProcessingService.ExecuteAsync(Int32 pageSize) in /src/Hangfire/Jobs/Integration/SimahInstantUpdateProcessingService.cs:line 19 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
#58253	SimahInstantUpdateProcessingService.ExecuteAsync An exception occurred during performance of the job. More details...	06/26/2026 07:45:10
#58253	Grpc.Core.RpcException Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (integration:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (integration:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation\|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout) --- End of inner exception stack trace --- at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync() at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1 responseTask, String methodName, Stopwatch stopwatch) at Gateway.Services.GrpcServices.Services.GrpcIntegrationService.ProcessSimahInstantUpdate(GrpcRequest request) in /src/Hangfire/GrpcServices/Services/GrpcIntegrationService.cs:line 87 at SimahInstantUpdateProcessingService.ExecuteAsync(Int32 pageSize) in /src/Hangfire/Jobs/Integration/SimahInstantUpdateProcessingService.cs:line 19 at InvokeStub_TaskAwaiter.GetResult(Object, Object, IntPtr*) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

Grpc.Core.RpcException

Grpc.Core.RpcException

Grpc.Core.RpcException

Grpc.Core.RpcException

Grpc.Core.RpcException

Grpc.Core.RpcException

Grpc.Core.RpcException

Grpc.Core.RpcException

Grpc.Core.RpcException

Grpc.Core.RpcException