After Hadoop is install manually using binary package on CentOS, Snappy compression is not supported by default and there are extra steps required in order for Snappy to work in Hadoop. It is straightforward but might not be obvious if you don’t know what to do. Firstly, if you are …
Disk might be cheap, but when it comes to deal with TB of data, you might want to consider compression for your data storage. When you want to create a table with compression enabled, you will need to use “STORED AS SEQUENCEFILE” when create a table in Hive: However, you …
Snappy is a compression/decompression library developed by Google. It aims for very high speeds and reasonable compression ( might be bigger than other standard compression algorithms but faster speed ). Snappy is shipped with Hadoop, unlike LZO compression which is excluded due to licensing issues. To enable Snappy in your …
My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!