Spark jobs failed with delegation token renewal error
An Oozie Spark job failed with the following error: This is caused by long running Spark job in a kerberized environment the checkpointing fails as Token is not renewed properly. The workaround is to add “–conf spark.hadoop.fs.hdfs.impl.disable.cache=true” to Spark job command line parameters to disable the token cache from spark …
