Does Sqoop Mainframe Import Support SFTP?

Does Sqoop Mainframe Import Support SFTP?

Currently Sqoop leverages Apache Commons Net FTP library with some customisations to handle the way mainframe returns FTP listings. The current Apache Commons Net library does NOT support SFTP but does support FTPS.

FTPS and SFTP are very different protocols so we would have to use a different underlying library to incorporate SFTP capability into the mainframe module:

  • FTPS: FTPS is FTP with SSL for security. It uses a control channel and opens new connections for the data transfer. As it uses SSL, it requires a certificate.
  • SFTP: SFTP (SSH File Transfer Protocol/Secure File Transfer Protocol) was designed as an extension of SSH to provide file transfer capability, so it usually uses only the SSH port for both data and control.

In most SSH server installations you will have SFTP support, but FTPS would need the additional configuration of a supported FTP server.

However, even though Apache Commons Net FTP supports FTPS, Sqoop currently only uses class, NOT which supports FTPS. Please check the source code here

So, the conclusion is that currently Sqoop does NOT support either FTPS or SFTP, and the only protocol it supports is FTP.


Leave a Reply

Your email address will not be published. Required fields are marked *

My new Snowflake Blog is now live. I will not be updating this blog anymore but will continue with new contents in the Snowflake world!