Hive query failed with error: Killing the Job. mapResourceReqt: 1638 maxContainerCapability:1200

Hive query failed with error: Killing the Job. mapResourceReqt: 1638 maxContainerCapability:1200

When running a Hive query, get the following error in the jobhistory:
MAP capability required is more than the supported max container capability in the cluster. 
Killing the Job. mapResourceRequest:  maxContainerCapability:
hive-error This is caused by the following settings in YARN:
mapreduce.map.memory.mb => 1638
yarn.scheduler.maximum-allocation-mb => 1200
yarn.nodemanager.resource.memory-mb => 1300
The solution is to setup the settings mentioned above in the following way: mapreduce.map.memory.mb < yarn.nodemanager.resource.memory-mb < yarn.scheduler.maximum-allocation-mb Then the problem should be resolved.

Leave a Reply

Your email address will not be published.

My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!