Sympton: After upgrading Cloudera Manager (CM) from CDH5.3 to CDH5.4, the requirement was to also upgrade the Hive Metastore Database Schema: However, it failed with the following message: It is not immediately apparent what needs to be done. Initially you might think that some of the indexes contained too many …
Apache Sqoop became the Top-Level Project in Apache in March 2012. Since then, Sqoop has developed a lot and become very popular amongst Hadoop ecosystem. In this post, I will cover the ways to specify database passwords to Sqoop in a secure way. The following ways are common to pass …
This article explains how to trouble shoot the issue when Hive METASTORE dies with no apparent error message in the log. Cloudera Manager (short for CM) has a mechanism to kill the processes that have reached its memory allocations. So when this scenario happens, it usually means that Hive METASTORE …
This article explains some clean up tasks needed to be done after Kerberos is enabled in the cluster for Hive to continue functioning. 1) Cleanup YARN user cache directory at /yarn/nm/usercache/xxxxx. This needs to be run on all nodes in the cluster, and all the directories that are defined under …
Last week I needed to do a quick compile of a sample Hive UDF for testing purpose. I didn’t want to install all Maven stuff and create a pom.xml file just for this simple task. Thanks to my colleague YiBing, who is an expert in Hive, I got it working …
My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!