When you run a really big job in Hive that failed with the following error: This indicated that the value for mapreduce.job.split.metainfo.maxsize is too small for your job (default value of 10000000). There are two options to fix this: 1. Set the value of mapreduce.job.split.metainfo.maxsize to be “-1” (unlimited) specifically …
I have seen lots of cases that Hive users were trying to give user permission to a certain directory using Sentry command to GRANT URI in the Sentry HDFS sync enabled cluster. This logic seems to be correct, however it will not work. Sentry HDFS sync will only sync Sentry …
This article explains the situation that will cause Hive to leave its staging directories that were created during processing and not doing clean up after job finished successfully. The issue happens when user runs Hive query through Hue’s Hive Editor, it does not apply to queries running through Beeline, Hive …
My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!