This article explains the workarounds to avoid the Hive query failure when processing snappy files generated by Hive and Flume under the same directory. The following are the steps to re-produce the issue: A Hive table (from_hive) with its data injected from Flume Create another table with same column structure (from_flume) Insert data …
Recently I discovered a bug in Sqoop that it wrongly detects a malformed query in options file, but in fact the query is totally correct. Example as below sqoop command: and in the /path/to/options-file.txt, it contains the following content: Sqoop will fail with the following error: This is caused by …
Currently Cloudera Manager does not support setting up multiple KDCs for the krb5.conf file natively, this article explains the workarounds we can have using the existing feature provided by Cloudera Manager. This article also assumes that you have krb5.conf file managed by Cloudera Manager. If you are using Cloudera Manager …
This article examples ONE of the possible causes for the issue that Impala query failed with IllegalStateException error. Recently I was dealing with an Impala issue that when runnnig a simple SELECT query against a table failed with IllegalStateException error: Checking the impala daemon log shows the following stacktrace: Based on …
Currently Sqoop leverages Apache Commons Net FTP library with some customisations to handle the way mainframe returns FTP listings. The current Apache Commons Net library does NOT support SFTP but does support FTPS. FTPS and SFTP are very different protocols so we would have to use a different underlying library to …
My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!