File size: 13,484 Bytes
c011401 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 |
Gcs Debug state:
GcsNodeManager:
- RegisterNode request count: 1
- DrainNode request count: 0
- GetAllNodeInfo request count: 38
GcsActorManager:
- RegisterActor request count: 0
- CreateActor request count: 0
- GetActorInfo request count: 0
- GetNamedActorInfo request count: 0
- GetAllActorInfo request count: 0
- KillActor request count: 0
- ListNamedActors request count: 0
- Registered actors count: 0
- Destroyed actors count: 0
- Named actors count: 0
- Unresolved actors count: 0
- Pending actors count: 0
- Created actors count: 0
- owners_: 0
- actor_to_register_callbacks_: 0
- actor_to_restart_callbacks_: 0
- actor_to_create_callbacks_: 0
- sorted_destroyed_actor_list_: 0
GcsResourceManager:
- GetAllAvailableResources request count: 1
- GetAllTotalResources request count: 0
- GetAllResourceUsage request count: 2
GcsPlacementGroupManager:
- CreatePlacementGroup request count: 0
- RemovePlacementGroup request count: 0
- GetPlacementGroup request count: 0
- GetAllPlacementGroup request count: 0
- WaitPlacementGroupUntilReady request count: 0
- GetNamedPlacementGroup request count: 0
- Scheduling pending placement group count: 0
- Registered placement groups count: 0
- Named placement group count: 0
- Pending placement groups count: 0
- Infeasible placement groups count: 0
Publisher:
GCS_JOB_CHANNEL
- cumulative published messages: 2
- cumulative published bytes: 493
- current buffered bytes: 251
GCS_NODE_INFO_CHANNEL
- cumulative published messages: 1
- cumulative published bytes: 534
- current buffered bytes: 0
RAY_LOG_CHANNEL
- cumulative published messages: 7
- cumulative published bytes: 4271
- current buffered bytes: 240
GCS_WORKER_DELTA_CHANNEL
- cumulative published messages: 25
- cumulative published bytes: 2425
- current buffered bytes: 97
[runtime env manager] ID to URIs table:
[runtime env manager] URIs reference table:
GcsTaskManager:
-Total num task events reported: 24
-Total num status task events dropped: 0
-Total num profile events dropped: 0
-Current num of task events stored: 7
-Total num of actor creation tasks: 0
-Total num of actor tasks: 0
-Total num of normal tasks: 6
-Total num of driver tasks: 1
GcsAutoscalerStateManager:
- last_seen_autoscaler_state_version_: 0
- last_cluster_resource_state_version_: 0
- pending demands:
{} * 0
Global stats: 704 total (5 active)
Queueing time: mean = 3.717 ms, max = 285.263 ms, min = 945.000 ns, total = 2.617 s
Execution time: mean = 689.729 us, total = 485.569 ms
Event stats:
GcsInMemoryStore.Get - 102 total (0 active), Execution time: mean = 67.246 us, total = 6.859 ms, Queueing time: mean = 29.316 us, max = 558.703 us, min = 4.757 us, total = 2.990 ms
GcsInMemoryStore.Put - 93 total (0 active), Execution time: mean = 3.101 ms, total = 288.378 ms, Queueing time: mean = 21.388 ms, max = 284.301 ms, min = 2.666 us, total = 1.989 s
InternalKVGcsService.grpc_server.InternalKVGet.HandleRequestImpl - 75 total (0 active), Execution time: mean = 48.247 us, total = 3.618 ms, Queueing time: mean = 111.912 us, max = 251.399 us, min = 23.546 us, total = 8.393 ms
InternalKVGcsService.grpc_server.InternalKVGet - 75 total (0 active), Execution time: mean = 716.006 us, total = 53.700 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
event_loop_lag_probe - 42 total (0 active), Execution time: mean = 26.108 us, total = 1.097 ms, Queueing time: mean = 376.284 us, max = 15.021 ms, min = 5.193 us, total = 15.804 ms
NodeInfoGcsService.grpc_server.GetAllNodeInfo - 38 total (0 active), Execution time: mean = 665.435 us, total = 25.287 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
NodeInfoGcsService.grpc_server.GetAllNodeInfo.HandleRequestImpl - 38 total (0 active), Execution time: mean = 45.699 us, total = 1.737 ms, Queueing time: mean = 131.850 us, max = 491.439 us, min = 35.183 us, total = 5.010 ms
WorkerInfoGcsService.grpc_server.ReportWorkerFailure.HandleRequestImpl - 25 total (0 active), Execution time: mean = 53.393 us, total = 1.335 ms, Queueing time: mean = 93.396 us, max = 271.599 us, min = 2.683 us, total = 2.335 ms
WorkerInfoGcsService.grpc_server.AddWorkerInfo.HandleRequestImpl - 25 total (0 active), Execution time: mean = 35.513 us, total = 887.818 us, Queueing time: mean = 93.124 us, max = 176.607 us, min = 11.209 us, total = 2.328 ms
WorkerInfoGcsService.grpc_server.AddWorkerInfo - 25 total (0 active), Execution time: mean = 654.894 us, total = 16.372 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
WorkerInfoGcsService.grpc_server.ReportWorkerFailure - 25 total (0 active), Execution time: mean = 912.980 us, total = 22.825 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
InternalKVGcsService.grpc_server.InternalKVPut.HandleRequestImpl - 19 total (0 active), Execution time: mean = 46.205 us, total = 877.892 us, Queueing time: mean = 101.028 us, max = 136.535 us, min = 33.812 us, total = 1.920 ms
InternalKVGcsService.grpc_server.InternalKVPut - 19 total (0 active), Execution time: mean = 689.366 us, total = 13.098 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
GcsResourceManager::Update - 12 total (0 active), Execution time: mean = 99.492 us, total = 1.194 ms, Queueing time: mean = 111.663 us, max = 141.090 us, min = 96.501 us, total = 1.340 ms
RayletLoadPulled - 11 total (1 active), Execution time: mean = 275.340 us, total = 3.029 ms, Queueing time: mean = 185.120 us, max = 1.288 ms, min = 62.246 us, total = 2.036 ms
NodeManagerService.grpc_client.GetResourceLoad.OnReplyReceived - 8 total (1 active), Execution time: mean = 76.219 us, total = 609.750 us, Queueing time: mean = 107.524 us, max = 172.050 us, min = 102.159 us, total = 860.194 us
NodeManagerService.grpc_client.GetResourceLoad - 8 total (0 active), Execution time: mean = 2.099 ms, total = 16.795 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
GcsInMemoryStore.GetAll - 6 total (0 active), Execution time: mean = 18.848 us, total = 113.089 us, Queueing time: mean = 94.534 us, max = 126.060 us, min = 14.208 us, total = 567.206 us
NodeInfoGcsService.grpc_server.GetClusterId.HandleRequestImpl - 5 total (0 active), Execution time: mean = 973.572 us, total = 4.868 ms, Queueing time: mean = 294.302 us, max = 1.059 ms, min = 57.917 us, total = 1.472 ms
NodeInfoGcsService.grpc_server.GetClusterId - 5 total (0 active), Execution time: mean = 1.825 ms, total = 9.125 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
ClusterResourceManager.ResetRemoteNodeView - 4 total (1 active), Execution time: mean = 8.610 us, total = 34.441 us, Queueing time: mean = 70.337 us, max = 106.681 us, min = 85.902 us, total = 281.349 us
NodeInfoGcsService.grpc_server.CheckAlive.HandleRequestImpl - 4 total (0 active), Execution time: mean = 36.677 us, total = 146.707 us, Queueing time: mean = 138.872 us, max = 170.873 us, min = 120.930 us, total = 555.489 us
NodeInfoGcsService.grpc_server.CheckAlive - 4 total (0 active), Execution time: mean = 704.419 us, total = 2.818 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
PeriodicalRunner.RunFnPeriodically - 4 total (0 active), Execution time: mean = 235.645 us, total = 942.581 us, Queueing time: mean = 145.104 ms, max = 285.263 ms, min = 4.766 ms, total = 580.416 ms
GcsInMemoryStore.Keys - 3 total (0 active), Execution time: mean = 4.984 us, total = 14.953 us, Queueing time: mean = 45.715 us, max = 49.943 us, min = 43.314 us, total = 137.145 us
NodeResourceInfoGcsService.grpc_server.GetAllResourceUsage - 2 total (0 active), Execution time: mean = 773.647 us, total = 1.547 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
NodeResourceInfoGcsService.grpc_server.GetAllResourceUsage.HandleRequestImpl - 2 total (0 active), Execution time: mean = 74.664 us, total = 149.328 us, Queueing time: mean = 117.308 us, max = 125.416 us, min = 109.200 us, total = 234.616 us
GcsHealthCheckManager::AddNode - 1 total (0 active), Execution time: mean = 21.375 us, total = 21.375 us, Queueing time: mean = 945.000 ns, max = 945.000 ns, min = 945.000 ns, total = 945.000 ns
HealthCheck - 1 total (0 active), Execution time: mean = 8.815 us, total = 8.815 us, Queueing time: mean = 126.867 us, max = 126.867 us, min = 126.867 us, total = 126.867 us
GcsInMemoryStore.BatchDelete - 1 total (0 active), Execution time: mean = 1.817 us, total = 1.817 us, Queueing time: mean = 9.516 us, max = 9.516 us, min = 9.516 us, total = 9.516 us
NodeInfoGcsService.grpc_server.RegisterNode.HandleRequestImpl - 1 total (0 active), Execution time: mean = 259.823 us, total = 259.823 us, Queueing time: mean = 184.665 us, max = 184.665 us, min = 184.665 us, total = 184.665 us
JobInfoGcsService.grpc_server.GetAllJobInfo - 1 total (0 active), Execution time: mean = 564.092 us, total = 564.092 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
NodeResourceInfoGcsService.grpc_server.GetAllAvailableResources - 1 total (0 active), Execution time: mean = 583.054 us, total = 583.054 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
InternalKVGcsService.grpc_server.InternalKVDel.HandleRequestImpl - 1 total (0 active), Execution time: mean = 82.715 us, total = 82.715 us, Queueing time: mean = 111.694 us, max = 111.694 us, min = 111.694 us, total = 111.694 us
InternalKVGcsService.grpc_server.InternalKVExists - 1 total (0 active), Execution time: mean = 604.558 us, total = 604.558 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
JobInfoGcsService.grpc_server.MarkJobFinished.HandleRequestImpl - 1 total (0 active), Execution time: mean = 13.401 us, total = 13.401 us, Queueing time: mean = 207.181 us, max = 207.181 us, min = 207.181 us, total = 207.181 us
JobInfoGcsService.grpc_server.AddJob - 1 total (0 active), Execution time: mean = 959.532 us, total = 959.532 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
InternalKVGcsService.grpc_server.InternalKVDel - 1 total (0 active), Execution time: mean = 924.167 us, total = 924.167 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
GcsInMemoryStore.Exists - 1 total (0 active), Execution time: mean = 15.537 us, total = 15.537 us, Queueing time: mean = 11.603 us, max = 11.603 us, min = 11.603 us, total = 11.603 us
GCSServer.deadline_timer.debug_state_dump - 1 total (1 active, 1 running), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
InternalKVGcsService.grpc_server.GetInternalConfig - 1 total (0 active), Execution time: mean = 585.445 us, total = 585.445 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
NodeResourceInfoGcsService.grpc_server.GetAllAvailableResources.HandleRequestImpl - 1 total (0 active), Execution time: mean = 47.214 us, total = 47.214 us, Queueing time: mean = 137.753 us, max = 137.753 us, min = 137.753 us, total = 137.753 us
InternalKVGcsService.grpc_server.InternalKVExists.HandleRequestImpl - 1 total (0 active), Execution time: mean = 50.720 us, total = 50.720 us, Queueing time: mean = 106.615 us, max = 106.615 us, min = 106.615 us, total = 106.615 us
JobInfoGcsService.grpc_server.GetNextJobID.HandleRequestImpl - 1 total (0 active), Execution time: mean = 29.307 us, total = 29.307 us, Queueing time: mean = 104.704 us, max = 104.704 us, min = 104.704 us, total = 104.704 us
GCSServer.deadline_timer.debug_state_event_stats_print - 1 total (1 active), Execution time: mean = 0.000 s, total = 0.000 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
JobInfoGcsService.grpc_server.AddJob.HandleRequestImpl - 1 total (0 active), Execution time: mean = 167.827 us, total = 167.827 us, Queueing time: mean = 133.055 us, max = 133.055 us, min = 133.055 us, total = 133.055 us
InternalKVGcsService.grpc_server.GetInternalConfig.HandleRequestImpl - 1 total (0 active), Execution time: mean = 56.146 us, total = 56.146 us, Queueing time: mean = 112.695 us, max = 112.695 us, min = 112.695 us, total = 112.695 us
JobInfoGcsService.grpc_server.MarkJobFinished - 1 total (0 active), Execution time: mean = 1.070 ms, total = 1.070 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
JobInfoGcsService.grpc_server.GetAllJobInfo.HandleRequestImpl - 1 total (0 active), Execution time: mean = 39.452 us, total = 39.452 us, Queueing time: mean = 44.555 us, max = 44.555 us, min = 44.555 us, total = 44.555 us
NodeInfoGcsService.grpc_server.RegisterNode - 1 total (0 active), Execution time: mean = 1.439 ms, total = 1.439 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s
GcsInMemoryStore.Delete - 1 total (0 active), Execution time: mean = 22.331 us, total = 22.331 us, Queueing time: mean = 12.840 us, max = 12.840 us, min = 12.840 us, total = 12.840 us
JobInfoGcsService.grpc_server.GetNextJobID - 1 total (0 active), Execution time: mean = 564.252 us, total = 564.252 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s |