How to use Cloudera Manager API to check a service role exited unexpectedly – Hadoop Troubleshooting Guide

This blog explains the steps to use Cloudera Manager API to check for a service role in CDH that was exited unexpectedly, so that proper action can be taken. To check a service role’s status via Cloudera Manager API, please follow the steps below (I am taking Impala as an example):

Determine the version of API you are using:
```
curl -u username:password http://:7180/api/version
```
in CDH5.7.x, it should return “v12”.

get the cluster name from the output of:

curl -u username:password http://:7180/api/v12/clusters

sample output:

{
  "items" : [ {
    "name" : "cluster",
    "displayName" : "Cluster 1",
    "version" : "CDH5",
    "fullVersion" : "5.7.0",
    "maintenanceMode" : false,
    "maintenanceOwners" : [ ],
    "clusterUrl" : "http://:7180/cmf/clusterRedirect/cluster",
    "hostsUrl" : "http://:7180/cmf/clusterRedirect/cluster/hosts",
    "entityStatus" : "GOOD_HEALTH"
  } ]
}

take note of the value for “name” attribute, in my case it is “cluster”.

get the services in the cluster:

curl -u username:password http://:7180/api/v12/clusters//services

substitute with our cluster’s name:

curl -u username:password http://:7180/api/v12/clusters/cluster/services

please locate the impala service and get its “name”, in my case it is “impala”:

{
    "name" : "impala",
    "type" : "IMPALA",
    "clusterRef" : {
      "clusterName" : "cluster"
    },
    ....
}

get the all the roles under impala service:

curl -u username:password http://:7180/api/v12/clusters//services//roles/

in my case should be:

curl -u username:password http://:7180/api/v12/clusters/cluster/services/impala/roles

locate the role that you want to monitor, I picked Statestore:

{
    "name" : "impala-STATESTORE-52cc0fbf54f5cc038b2b0a67634034fe",
    "type" : "STATESTORE",
    "serviceRef" : {
      "clusterName" : "cluster",
      "serviceName" : "impala"
    },
    ....
}

in my case it is “impala-STATESTORE-52cc0fbf54f5cc038b2b0a67634034fe”

get the status for the role you want to monitor:

curl -u username:password http://:7180/api/v12/clusters//services//roles/

after substitution, it should be:

curl -u username:password http://:7180/api/v12/clusters/cluster/services/impala/roles/impala-STATESTORE-52cc0fbf54f5cc038b2b0a67634034fe

this will give you full status output for this particular role:

{
  "name" : "impala-STATESTORE-52cc0fbf54f5cc038b2b0a67634034fe",
  "type" : "STATESTORE",
  "serviceRef" : {
    "clusterName" : "cluster",
    "serviceName" : "impala"
  },
  "hostRef" : {
    "hostId" : "eff96b49-739e-48d4-a19b-e5865a83b164"
  },
  .....
  "configStalenessStatus" : "FRESH",
  "maintenanceMode" : false,
  "maintenanceOwners" : [ ],
  "commissionState" : "COMMISSIONED",
  "roleConfigGroupRef" : {
    "roleConfigGroupName" : "impala-STATESTORE-BASE"
  },
  "entityStatus" : "GOOD_HEALTH"
}

look for the last attribute called “entityStatus”, it has the following possible values:

UNKNOWN	
NONE	
STOPPED	
DOWN	
UNKNOWN_HEALTH	
DISABLED_HEALTH	
CONCERNING_HEALTH	
BAD_HEALTH	
GOOD_HEALTH	
STARTING	
STOPPING	
HISTORY_NOT_AVAILABLE

in the case that it is exited unexpectedly, the value would be “DOWN”, so that we can programmatically decide whether we can just restart it or not.

Please note that if you have enabled SSL for Cloudera Manager, the URL should be changed to: https://:7183 instead of http://:7180 More information about the apiRole entity can be found here: apiRole

M	T	W	T	F	S	S
		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30

Other posts that you might also be interested:

Leave a Reply Cancel reply