| Gcs Debug state: | |
| GcsNodeManager: | |
| - RegisterNode request count: 1 | |
| - DrainNode request count: 0 | |
| - GetAllNodeInfo request count: 38 | |
| GcsActorManager: | |
| - RegisterActor request count: 0 | |
| - CreateActor request count: 0 | |
| - GetActorInfo request count: 0 | |
| - GetNamedActorInfo request count: 0 | |
| - GetAllActorInfo request count: 0 | |
| - KillActor request count: 0 | |
| - ListNamedActors request count: 0 | |
| - Registered actors count: 0 | |
| - Destroyed actors count: 0 | |
| - Named actors count: 0 | |
| - Unresolved actors count: 0 | |
| - Pending actors count: 0 | |
| - Created actors count: 0 | |
| - owners_: 0 | |
| - actor_to_register_callbacks_: 0 | |
| - actor_to_restart_callbacks_: 0 | |
| - actor_to_create_callbacks_: 0 | |
| - sorted_destroyed_actor_list_: 0 | |
| GcsResourceManager: | |
| - GetAllAvailableResources request count: 1 | |
| - GetAllTotalResources request count: 0 | |
| - GetAllResourceUsage request count: 2 | |
| GcsPlacementGroupManager: | |
| - CreatePlacementGroup request count: 0 | |
| - RemovePlacementGroup request count: 0 | |
| - GetPlacementGroup request count: 0 | |
| - GetAllPlacementGroup request count: 0 | |
| - WaitPlacementGroupUntilReady request count: 0 | |
| - GetNamedPlacementGroup request count: 0 | |
| - Scheduling pending placement group count: 0 | |
| - Registered placement groups count: 0 | |
| - Named placement group count: 0 | |
| - Pending placement groups count: 0 | |
| - Infeasible placement groups count: 0 | |
| Publisher: | |
| GCS_JOB_CHANNEL | |
| - cumulative published messages: 2 | |
| - cumulative published bytes: 493 | |
| - current buffered bytes: 251 | |
| GCS_NODE_INFO_CHANNEL | |
| - cumulative published messages: 1 | |
| - cumulative published bytes: 534 | |
| - current buffered bytes: 0 | |
| RAY_LOG_CHANNEL | |
| - cumulative published messages: 7 | |
| - cumulative published bytes: 4271 | |
| - current buffered bytes: 240 | |
| GCS_WORKER_DELTA_CHANNEL | |
| - cumulative published messages: 25 | |
| - cumulative published bytes: 2425 | |
| - current buffered bytes: 97 | |
| [runtime env manager] ID to URIs table: | |
| [runtime env manager] URIs reference table: | |
| GcsTaskManager: | |
| -Total num task events reported: 24 | |
| -Total num status task events dropped: 0 | |
| -Total num profile events dropped: 0 | |
| -Current num of task events stored: 7 | |
| -Total num of actor creation tasks: 0 | |
| -Total num of actor tasks: 0 | |
| -Total num of normal tasks: 6 | |
| -Total num of driver tasks: 1 | |
| GcsAutoscalerStateManager: | |
| - last_seen_autoscaler_state_version_: 0 | |
| - last_cluster_resource_state_version_: 0 | |
| - pending demands: | |
| {} * 0 | |
| Global stats: 704 total (5 active) | |
| Queueing time: mean = 3.717 ms, max = 285.263 ms, min = 945.000 ns, total = 2.617 s | |
| Execution time: mean = 689.729 us, total = 485.569 ms | |
| Event stats: | |
| GcsInMemoryStore.Get - 102 total (0 active), Execution time: mean = 67.246 us, total = 6.859 ms, Queueing time: mean = 29.316 us, max = 558.703 us, min = 4.757 us, total = 2.990 ms | |
| GcsInMemoryStore.Put - 93 total (0 active), Execution time: mean = 3.101 ms, total = 288.378 ms, Queueing time: mean = 21.388 ms, max = 284.301 ms, min = 2.666 us, total = 1.989 s | |
| InternalKVGcsService.grpc_server.InternalKVGet.HandleRequestImpl - 75 total (0 active), Execution time: mean = 48.247 us, total = 3.618 ms, Queueing time: mean = 111.912 us, max = 251.399 us, min = 23.546 us, total = 8.393 ms | |
| InternalKVGcsService.grpc_server.InternalKVGet - 75 total (0 active), Execution time: mean = 716.006 us, total = 53.700 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| event_loop_lag_probe - 42 total (0 active), Execution time: mean = 26.108 us, total = 1.097 ms, Queueing time: mean = 376.284 us, max = 15.021 ms, min = 5.193 us, total = 15.804 ms | |
| NodeInfoGcsService.grpc_server.GetAllNodeInfo - 38 total (0 active), Execution time: mean = 665.435 us, total = 25.287 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeInfoGcsService.grpc_server.GetAllNodeInfo.HandleRequestImpl - 38 total (0 active), Execution time: mean = 45.699 us, total = 1.737 ms, Queueing time: mean = 131.850 us, max = 491.439 us, min = 35.183 us, total = 5.010 ms | |
| WorkerInfoGcsService.grpc_server.ReportWorkerFailure.HandleRequestImpl - 25 total (0 active), Execution time: mean = 53.393 us, total = 1.335 ms, Queueing time: mean = 93.396 us, max = 271.599 us, min = 2.683 us, total = 2.335 ms | |
| WorkerInfoGcsService.grpc_server.AddWorkerInfo.HandleRequestImpl - 25 total (0 active), Execution time: mean = 35.513 us, total = 887.818 us, Queueing time: mean = 93.124 us, max = 176.607 us, min = 11.209 us, total = 2.328 ms | |
| WorkerInfoGcsService.grpc_server.AddWorkerInfo - 25 total (0 active), Execution time: mean = 654.894 us, total = 16.372 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| WorkerInfoGcsService.grpc_server.ReportWorkerFailure - 25 total (0 active), Execution time: mean = 912.980 us, total = 22.825 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVPut.HandleRequestImpl - 19 total (0 active), Execution time: mean = 46.205 us, total = 877.892 us, Queueing time: mean = 101.028 us, max = 136.535 us, min = 33.812 us, total = 1.920 ms | |
| InternalKVGcsService.grpc_server.InternalKVPut - 19 total (0 active), Execution time: mean = 689.366 us, total = 13.098 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| GcsResourceManager::Update - 12 total (0 active), Execution time: mean = 99.492 us, total = 1.194 ms, Queueing time: mean = 111.663 us, max = 141.090 us, min = 96.501 us, total = 1.340 ms | |
| RayletLoadPulled - 11 total (1 active), Execution time: mean = 275.340 us, total = 3.029 ms, Queueing time: mean = 185.120 us, max = 1.288 ms, min = 62.246 us, total = 2.036 ms | |
| NodeManagerService.grpc_client.GetResourceLoad.OnReplyReceived - 8 total (1 active), Execution time: mean = 76.219 us, total = 609.750 us, Queueing time: mean = 107.524 us, max = 172.050 us, min = 102.159 us, total = 860.194 us | |
| NodeManagerService.grpc_client.GetResourceLoad - 8 total (0 active), Execution time: mean = 2.099 ms, total = 16.795 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| GcsInMemoryStore.GetAll - 6 total (0 active), Execution time: mean = 18.848 us, total = 113.089 us, Queueing time: mean = 94.534 us, max = 126.060 us, min = 14.208 us, total = 567.206 us | |
| NodeInfoGcsService.grpc_server.GetClusterId.HandleRequestImpl - 5 total (0 active), Execution time: mean = 973.572 us, total = 4.868 ms, Queueing time: mean = 294.302 us, max = 1.059 ms, min = 57.917 us, total = 1.472 ms | |
| NodeInfoGcsService.grpc_server.GetClusterId - 5 total (0 active), Execution time: mean = 1.825 ms, total = 9.125 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| ClusterResourceManager.ResetRemoteNodeView - 4 total (1 active), Execution time: mean = 8.610 us, total = 34.441 us, Queueing time: mean = 70.337 us, max = 106.681 us, min = 85.902 us, total = 281.349 us | |
| NodeInfoGcsService.grpc_server.CheckAlive.HandleRequestImpl - 4 total (0 active), Execution time: mean = 36.677 us, total = 146.707 us, Queueing time: mean = 138.872 us, max = 170.873 us, min = 120.930 us, total = 555.489 us | |
| NodeInfoGcsService.grpc_server.CheckAlive - 4 total (0 active), Execution time: mean = 704.419 us, total = 2.818 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| PeriodicalRunner.RunFnPeriodically - 4 total (0 active), Execution time: mean = 235.645 us, total = 942.581 us, Queueing time: mean = 145.104 ms, max = 285.263 ms, min = 4.766 ms, total = 580.416 ms | |
| GcsInMemoryStore.Keys - 3 total (0 active), Execution time: mean = 4.984 us, total = 14.953 us, Queueing time: mean = 45.715 us, max = 49.943 us, min = 43.314 us, total = 137.145 us | |
| NodeResourceInfoGcsService.grpc_server.GetAllResourceUsage - 2 total (0 active), Execution time: mean = 773.647 us, total = 1.547 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeResourceInfoGcsService.grpc_server.GetAllResourceUsage.HandleRequestImpl - 2 total (0 active), Execution time: mean = 74.664 us, total = 149.328 us, Queueing time: mean = 117.308 us, max = 125.416 us, min = 109.200 us, total = 234.616 us | |
| GcsHealthCheckManager::AddNode - 1 total (0 active), Execution time: mean = 21.375 us, total = 21.375 us, Queueing time: mean = 945.000 ns, max = 945.000 ns, min = 945.000 ns, total = 945.000 ns | |
| HealthCheck - 1 total (0 active), Execution time: mean = 8.815 us, total = 8.815 us, Queueing time: mean = 126.867 us, max = 126.867 us, min = 126.867 us, total = 126.867 us | |
| GcsInMemoryStore.BatchDelete - 1 total (0 active), Execution time: mean = 1.817 us, total = 1.817 us, Queueing time: mean = 9.516 us, max = 9.516 us, min = 9.516 us, total = 9.516 us | |
| NodeInfoGcsService.grpc_server.RegisterNode.HandleRequestImpl - 1 total (0 active), Execution time: mean = 259.823 us, total = 259.823 us, Queueing time: mean = 184.665 us, max = 184.665 us, min = 184.665 us, total = 184.665 us | |
| JobInfoGcsService.grpc_server.GetAllJobInfo - 1 total (0 active), Execution time: mean = 564.092 us, total = 564.092 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeResourceInfoGcsService.grpc_server.GetAllAvailableResources - 1 total (0 active), Execution time: mean = 583.054 us, total = 583.054 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVDel.HandleRequestImpl - 1 total (0 active), Execution time: mean = 82.715 us, total = 82.715 us, Queueing time: mean = 111.694 us, max = 111.694 us, min = 111.694 us, total = 111.694 us | |
| InternalKVGcsService.grpc_server.InternalKVExists - 1 total (0 active), Execution time: mean = 604.558 us, total = 604.558 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.MarkJobFinished.HandleRequestImpl - 1 total (0 active), Execution time: mean = 13.401 us, total = 13.401 us, Queueing time: mean = 207.181 us, max = 207.181 us, min = 207.181 us, total = 207.181 us | |
| JobInfoGcsService.grpc_server.AddJob - 1 total (0 active), Execution time: mean = 959.532 us, total = 959.532 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVDel - 1 total (0 active), Execution time: mean = 924.167 us, total = 924.167 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| GcsInMemoryStore.Exists - 1 total (0 active), Execution time: mean = 15.537 us, total = 15.537 us, Queueing time: mean = 11.603 us, max = 11.603 us, min = 11.603 us, total = 11.603 us | |
| GCSServer.deadline_timer.debug_state_dump - 1 total (1 active, 1 running), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.GetInternalConfig - 1 total (0 active), Execution time: mean = 585.445 us, total = 585.445 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeResourceInfoGcsService.grpc_server.GetAllAvailableResources.HandleRequestImpl - 1 total (0 active), Execution time: mean = 47.214 us, total = 47.214 us, Queueing time: mean = 137.753 us, max = 137.753 us, min = 137.753 us, total = 137.753 us | |
| InternalKVGcsService.grpc_server.InternalKVExists.HandleRequestImpl - 1 total (0 active), Execution time: mean = 50.720 us, total = 50.720 us, Queueing time: mean = 106.615 us, max = 106.615 us, min = 106.615 us, total = 106.615 us | |
| JobInfoGcsService.grpc_server.GetNextJobID.HandleRequestImpl - 1 total (0 active), Execution time: mean = 29.307 us, total = 29.307 us, Queueing time: mean = 104.704 us, max = 104.704 us, min = 104.704 us, total = 104.704 us | |
| GCSServer.deadline_timer.debug_state_event_stats_print - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.AddJob.HandleRequestImpl - 1 total (0 active), Execution time: mean = 167.827 us, total = 167.827 us, Queueing time: mean = 133.055 us, max = 133.055 us, min = 133.055 us, total = 133.055 us | |
| InternalKVGcsService.grpc_server.GetInternalConfig.HandleRequestImpl - 1 total (0 active), Execution time: mean = 56.146 us, total = 56.146 us, Queueing time: mean = 112.695 us, max = 112.695 us, min = 112.695 us, total = 112.695 us | |
| JobInfoGcsService.grpc_server.MarkJobFinished - 1 total (0 active), Execution time: mean = 1.070 ms, total = 1.070 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.GetAllJobInfo.HandleRequestImpl - 1 total (0 active), Execution time: mean = 39.452 us, total = 39.452 us, Queueing time: mean = 44.555 us, max = 44.555 us, min = 44.555 us, total = 44.555 us | |
| NodeInfoGcsService.grpc_server.RegisterNode - 1 total (0 active), Execution time: mean = 1.439 ms, total = 1.439 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| GcsInMemoryStore.Delete - 1 total (0 active), Execution time: mean = 22.331 us, total = 22.331 us, Queueing time: mean = 12.840 us, max = 12.840 us, min = 12.840 us, total = 12.840 us | |
| JobInfoGcsService.grpc_server.GetNextJobID - 1 total (0 active), Execution time: mean = 564.252 us, total = 564.252 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s |