Today I have discovered a bug in Hive, that when user submits an “INSERT OVERWRITE” query with dynamic partitioning, Hive does not lock the underlining table “exclusively”, rather it only applies “shared” lock. To confirm the problem, I created a simple table: Then I issued a dynamic partitioning query: And …
Beeline will return non-zero exit code on failure in the following CDH versions 5.2.2 onwards. The returned exit code simply means the number of errors occurred during beeline’s execution. So for example, the following command will result the exit code of 2 because the first two “show” command will fail …
When Hive tries to “INSERT OVERWRITE” to a partition of an external table under existing directory, depending on whether the partition definition already exists in the metastore or not, Hive will behave differently: 1) if partition definition does not exist, it will not try to guess where the target partition …
Reading JSON data in Hive is super easy, you can simply use the custom JSON SerDe library that has been created by someone else and load it into Hive and you are ready to go. No UDFs or UDAFs are required. Just follow the steps below: 1. Assuming you have …
When working with Hive, there are lots of scenarios that we need to move data from one cluster to another. For instance, sometimes we need to copy some production data into alpha/dev environment for testing purpose. Luckily that Hive provides two easy commands for us to do it. Since version …
My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!