As more and more companies store large amount of data, either privately or in the cloud, data governance has become more and more important, which can help companies to decide: Where is the source of data How data is transformed When data was modified How data is classified etc etc …
When you create Hive table, you can specify the LOCATION of the table, regardless if the table is managed or external table. The syntax is like below: If LOCATION clause is omitted, then Hive will use default location under /user/hive/warehouse and then use the DB and Table name specified in …
Yesterday i was working on a very weird issue in Hive that the SHOW CREATE TABLE for a VIEW returned partial output. This happened in Hue, Beeline as well as Hive CLI. Please see below test case: 1. From Hue interface: 2. From Beeline: 3. From Hive CLI: After some …
Recently I was dealing with a case where customer wanted to migrate from Cloudera Manager’s Embedded PostgreSQL to MySQL due to the fact that the Embedded PostgreSQL provided by Cloudera Manager is not recommended for production use, instead, using external databases like MySQL, PostgreSQL or Oracle etc is highly recommended. …
Recently when I was testing Sqoop1 command in my CDH cluster, I kept getting “java.io.IOException: No columns to generate for ClassWriter” error. The full command was like below: And full stacktrace: Although I do not have the RCA yet for this issue, I do have a workaround, which is adding …
My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!