This article explains some clean up tasks needed to be done after Kerberos is enabled in the cluster for Hive to continue functioning.
1) Cleanup YARN user cache directory at /yarn/nm/usercache/xxxxx. This needs to be run on all nodes in the cluster, and all the directories that are defined under config name “NodeManager Local Directory List” in the CM > YARN > Configuration page.
This is because when cluster was running under simple AUTH, the yarn jobs were created by normally yarn or nobody user, depending on setup, but after Kerberos AUTH is enabled, the yarn job will be run as the user who triggered the job, and because user changed, the new job will not be able to overwrite original directories or files.
To fix this:
a) Stop YARN service
b) Remove user cache directory by running “rm -fr /yarn/nm/usercache/*”, remember, this has to be done on all machines in the cluster
c) Restart YARN service
2) Need to sync all users between Kerberos, OS system user as well as HDFS user directory, again, on all machines in the cluster.
For example, if you have a kerberos principal for user “foo”, you will need to create a “foo” system user on all server nodes in the cluster, you will also need to create a HDFS directory for this user under “/user/foo” and owned by “foo:foo” so that it will have permission to write to this directory.
After this change, the Hive permission error should get fixed.
Related articles
