I have seen lots of Hadoop users are not using Big Data technology correctly. Hadoop is designed for Big Data, so it works well with large file sizes and this is why we have block size for HDFS as 256MB or 512MB depending on use cases. However, lots of users, …
This article explains what to do when you are unable to start up SparkHistory server which keeps crashing with OutOfMemory errors after using Spark for some time. To confirm that Spark History Server keeps failing with OutOfMemory error on start up, we can check the run time process directory for …
My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!