An Oozie Spark job failed with the following error: This is caused by long running Spark job in a kerberized environment the checkpointing fails as Token is not renewed properly. The workaround is to add “–conf spark.hadoop.fs.hdfs.impl.disable.cache=true” to Spark job command line parameters to disable the token cache from spark …
This article explains the steps needed to load Spark2 into Oozie under CDH5.9.x which comes with Spark1.6. Although this was tested under CDH5.9.0, it should be similar for earlier releases. Please follow the steps below: Locate the current shared-lib directory by running: you will get something like below: This tells …
In the last few weeks, I have been dealing the Teradata Support for a Sqoop issue that the value with Timestamp(6) data type in Teradata will lost last 3 digits of nano seconds after importing into HDFS using Sqoop command. The following test case validates the issue: Create a table …
My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!