Failed jobs do not become expired to allow you to re-queue them without any
time pressure. You should re-queue or delete them manually, or apply AutomaticRetry(OnAttemptsExceeded = AttemptsExceededAction.Delete)
attribute to delete them automatically.
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (application:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (application:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (application:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Application.GrpcApplicationService.SweepNotificationOutbox(GrpcRequestrequest) in /src/Hangfire/GrpcServices/Services/Application/GrpcApplicationService.cs:line 206
at NotificationOutboxSweepJob.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Application/NotificationOutboxSweepJob.cs:line 22
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
An exception occurred during performance of the job. More details...
11/16/2026 14:28:41
Grpc.Core.RpcException
Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: An error occurred while sending the request. HttpIOException: The response ended prematurely. (ResponseEnded)", DebugException="System.Net.Http.HttpRequestException: An error occurred while sending the request.")
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: An error occurred while sending the request. HttpIOException: The response ended prematurely. (ResponseEnded)", DebugException="System.Net.Http.HttpRequestException: An error occurred while sending the request.") ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.HttpIOException: The response ended prematurely. (ResponseEnded)
at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Application.GrpcApplicationService.ProcessSimahDefaultedLoans(GrpcRequestrequest) in /src/Hangfire/GrpcServices/Services/Application/GrpcApplicationService.cs:line 226
at SimahDefaultedLoansService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Application/SimahDefaultedLoansService.cs:line 19
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
An exception occurred during performance of the job. More details...
11/16/2026 14:28:41
Grpc.Core.RpcException
Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: An error occurred while sending the request. HttpIOException: The response ended prematurely. (ResponseEnded)", DebugException="System.Net.Http.HttpRequestException: An error occurred while sending the request.")
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: An error occurred while sending the request. HttpIOException: The response ended prematurely. (ResponseEnded)", DebugException="System.Net.Http.HttpRequestException: An error occurred while sending the request.") ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.Http.HttpIOException: The response ended prematurely. (ResponseEnded)
at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Application.GrpcApplicationService.ProcessSimahMonthlyAccountUpdate(GrpcRequestrequest) in /src/Hangfire/GrpcServices/Services/Application/GrpcApplicationService.cs:line 221
at SimahMonthlyAccountUpdateService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Application/SimahMonthlyAccountUpdateService.cs:line 19
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
An exception occurred during performance of the job. More details...
11/16/2026 14:28:33
Grpc.Core.RpcException
Status(StatusCode="Unimplemented", Detail="Method is unimplemented.")
Grpc.Core.RpcException: Status(StatusCode="Unimplemented", Detail="Method is unimplemented.")
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Application.GrpcApplicationService.NotificationOutboxHousekeeping(GrpcRequestrequest) in /src/Hangfire/GrpcServices/Services/Application/GrpcApplicationService.cs:line 211
at NotificationOutboxHousekeepingJob.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Application/NotificationOutboxSweepJob.cs:line 36
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
An exception occurred during performance of the job. More details...
11/16/2026 14:28:33
Grpc.Core.RpcException
Status(StatusCode="Unimplemented", Detail="Method is unimplemented.")
Grpc.Core.RpcException: Status(StatusCode="Unimplemented", Detail="Method is unimplemented.")
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Application.GrpcApplicationService.SweepNotificationOutbox(GrpcRequestrequest) in /src/Hangfire/GrpcServices/Services/Application/GrpcApplicationService.cs:line 206
at NotificationOutboxSweepJob.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Application/NotificationOutboxSweepJob.cs:line 22
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (educational:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Educational.GrpcEducationalClientService.ProcessOutboxEvents(GrpcRequestinput) in /src/Hangfire/GrpcServices/Services/Educational/GrpcEducationalClientService.cs:line 94
at OutboxDispatchJobService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Educational/OutboxDispatchJobService.cs:line 18
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (educational:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Educational.GrpcEducationalClientService.ProcessOutboxEvents(GrpcRequestinput) in /src/Hangfire/GrpcServices/Services/Educational/GrpcEducationalClientService.cs:line 94
at OutboxDispatchJobService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Educational/OutboxDispatchJobService.cs:line 18
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Connection refused (madarak-identity-dev.madarak-identity-dev:80) SocketException: Connection refused", DebugException="System.Net.Http.HttpRequestException: Connection refused (madarak-identity-dev.madarak-identity-dev:80)") ---> System.Net.Http.HttpRequestException: Connection refused (madarak-identity-dev.madarak-identity-dev:80) ---> System.Net.Sockets.SocketException: Connection refused
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.GrpcMobileUserClientService.ProcessIdentityJobs(GrpcRequestrequest) in /src/Hangfire/GrpcServices/Services/GrpcMobileUserClientService.cs:line 267
at IdentityJobsService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Identity/IdentityJobsService.cs:line 24
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Connection refused (madarak-identity-dev.madarak-identity-dev:80) SocketException: Connection refused", DebugException="System.Net.Http.HttpRequestException: Connection refused (madarak-identity-dev.madarak-identity-dev:80)") ---> System.Net.Http.HttpRequestException: Connection refused (madarak-identity-dev.madarak-identity-dev:80) ---> System.Net.Sockets.SocketException: Connection refused
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.GrpcMobileUserClientService.ProcessIdentityJobs(GrpcRequestrequest) in /src/Hangfire/GrpcServices/Services/GrpcMobileUserClientService.cs:line 267
at IdentityJobsService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Identity/IdentityJobsService.cs:line 24
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (educational:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Educational.GrpcEducationalClientService.ProcessOutboxEvents(GrpcRequestinput) in /src/Hangfire/GrpcServices/Services/Educational/GrpcEducationalClientService.cs:line 94
at OutboxDispatchJobService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Educational/OutboxDispatchJobService.cs:line 18
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Connection refused (educational:80) SocketException: Connection refused", DebugException="System.Net.Http.HttpRequestException: Connection refused (educational:80)") ---> System.Net.Http.HttpRequestException: Connection refused (educational:80) ---> System.Net.Sockets.SocketException: Connection refused
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Educational.GrpcEducationalClientService.ProcessOutboxEvents(GrpcRequestinput) in /src/Hangfire/GrpcServices/Services/Educational/GrpcEducationalClientService.cs:line 94
at OutboxDispatchJobService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Educational/OutboxDispatchJobService.cs:line 18
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (educational:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Educational.GrpcEducationalClientService.ProcessOutboxEvents(GrpcRequestinput) in /src/Hangfire/GrpcServices/Services/Educational/GrpcEducationalClientService.cs:line 94
at OutboxDispatchJobService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Educational/OutboxDispatchJobService.cs:line 18
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (educational:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Educational.GrpcEducationalClientService.ProcessOutboxEvents(GrpcRequestinput) in /src/Hangfire/GrpcServices/Services/Educational/GrpcEducationalClientService.cs:line 94
at OutboxDispatchJobService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Educational/OutboxDispatchJobService.cs:line 18
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (educational:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Educational.GrpcEducationalClientService.ProcessOutboxEvents(GrpcRequestinput) in /src/Hangfire/GrpcServices/Services/Educational/GrpcEducationalClientService.cs:line 94
at OutboxDispatchJobService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Educational/OutboxDispatchJobService.cs:line 18
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (educational:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Educational.GrpcEducationalClientService.ProcessOutboxEvents(GrpcRequestinput) in /src/Hangfire/GrpcServices/Services/Educational/GrpcEducationalClientService.cs:line 94
at OutboxDispatchJobService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Educational/OutboxDispatchJobService.cs:line 18
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (educational:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Educational.GrpcEducationalClientService.ProcessOutboxEvents(GrpcRequestinput) in /src/Hangfire/GrpcServices/Services/Educational/GrpcEducationalClientService.cs:line 94
at OutboxDispatchJobService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Educational/OutboxDispatchJobService.cs:line 18
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (educational:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Educational.GrpcEducationalClientService.ProcessOutboxEvents(GrpcRequestinput) in /src/Hangfire/GrpcServices/Services/Educational/GrpcEducationalClientService.cs:line 94
at OutboxDispatchJobService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Educational/OutboxDispatchJobService.cs:line 18
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (educational:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Educational.GrpcEducationalClientService.ProcessOutboxEvents(GrpcRequestinput) in /src/Hangfire/GrpcServices/Services/Educational/GrpcEducationalClientService.cs:line 94
at OutboxDispatchJobService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Educational/OutboxDispatchJobService.cs:line 18
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (educational:80) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80)") ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (educational:80) ---> System.Net.Sockets.SocketException: Resource temporarily unavailable
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketErrorerror, CancellationTokencancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16token)
at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgssaea, ValueTaskconnectTask, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(Stringhost, Int32port, HttpRequestMessageinitialRequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItemqueueItem)
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationTokencancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessagerequest, Booleanasync, BooleandoRequestAuth, CancellationTokencancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessagerequest, Booleanasync, CancellationTokencancellationToken)
at Grpc.Net.Client.Web.GrpcWebHandler.SendAsyncCore(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Balancer.Internal.BalancerHttpHandler.SendAsync(HttpRequestMessagerequest, CancellationTokencancellationToken)
at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessagerequest, Nullable`1timeout)
--- End of inner exception stack trace ---
at Grpc.Net.Client.Internal.GrpcCall`2.GetResponseHeadersCoreAsync()
at Madarak.Configurations.Interceptor.GrpcClientInterceptor.LogResponseAsync[TResponse](Task`1responseTask, StringmethodName, Stopwatchstopwatch)
at Gateway.Services.GrpcServices.Services.Educational.GrpcEducationalClientService.ProcessOutboxEvents(GrpcRequestinput) in /src/Hangfire/GrpcServices/Services/Educational/GrpcEducationalClientService.cs:line 94
at OutboxDispatchJobService.ExecuteAsync(Int32pageSize) in /src/Hangfire/Jobs/Educational/OutboxDispatchJobService.cs:line 18
at InvokeStub_TaskAwaiter.GetResult(Object,Object, IntPtr*)
at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Objectobj, BindingFlagsinvokeAttr)
Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'hangfire:OutboxDispatchJobService.ExecuteAsync' resource.
Hangfire.PostgreSql.PostgreSqlDistributedLockException: Timeout expired. The timeout elapsed prior to obtaining a distributed lock on the 'hangfire:OutboxDispatchJobService.ExecuteAsync' resource.
at Hangfire.PostgreSql.PostgreSqlDistributedLock.LockHandler.Lock(Stringresource, TimeSpantimeout, IDbConnectionconnection, PostgreSqlStorageOptionsoptions)
at Hangfire.PostgreSql.PostgreSqlConnection.AcquireLock(Stringresource, TimeSpantimeout)
at Hangfire.DisableConcurrentExecutionAttribute.OnPerforming(PerformingContextcontext) in C:\projects\hangfire-525\src\Hangfire.Core\DisableConcurrentExecutionAttribute.cs:line 48
at Hangfire.Server.BackgroundJobPerformer.InvokeOnPerforming(KeyValuePair`2ctx) in C:\projects\hangfire-525\src\Hangfire.Core\Server\BackgroundJobPerformer.cs:line 198