sqoop import --connect jdbc:mysql://And full stacktrace:/test --table test --username --password --target-dir sqoop_test -m 1
16/08/20 03:03:13 ERROR manager.SqlManager: Error reading from database: java.sql.SQLException: Streaming result set [email protected] is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries. java.sql.SQLException: Streaming result set [email protected] is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries. at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:934) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:931) at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:2735) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1899) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569) at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1524) at com.mysql.jdbc.ConnectionImpl.getMaxBytesPerChar(ConnectionImpl.java:3003) at com.mysql.jdbc.Field.getMaxBytesPerCharacter(Field.java:602) at com.mysql.jdbc.ResultSetMetaData.getPrecision(ResultSetMetaData.java:445) at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:305) at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260) at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:246) at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:327) at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1846) at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1646) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615) at org.apache.sqoop.Sqoop.run(Sqoop.java:143) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) at org.apache.sqoop.Sqoop.main(Sqoop.java:236) 16/08/20 03:03:13 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1652) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615) at org.apache.sqoop.Sqoop.run(Sqoop.java:143) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) at org.apache.sqoop.Sqoop.main(Sqoop.java:236)Although I do not have the RCA yet for this issue, I do have a workaround, which is adding “––driver com.mysql.jdbc.Driver” to the Sqoop parameters. So the full command becomes:
sqoop import --connect jdbc:mysql://Hopefully this can help with anyone who might have the same issue./test --table test --username --password --target-dir sqoop_test -m 1 --driver com.mysql.jdbc.Driver
Thanks a lot!!!
Thank you so much…i spent almost an hour on this issue.
Hi I am new in bigdata and i am using –>sqoop import –connect jdbc:mysql://xxx.xxx.xx.xx:3306/FBI_DB –table FBI_CRIME –hbase-table H_FBI_CRIME –column-family cd –hbase-row-key ID –m 1 –username root -P. But I am not able to import data. Once I had used (––driver com.mysql.jdbc.Driver) but still didn’t get success . Always showing error —->>>ERROR tool.ImportTool: Import failed: java.io.IOException: No columns to generate for ClassWriter.
Please help, What is wrong.
Hi Arun,
Thanks for visiting my blog and posting question. In order to find out why it is still not working for you, can you please share the full –verbose output of your Sqoop command output and email to my address eric at lin dot fit. I will need to check the full error details to be able to determine what else to look for.
Cheers
bro why iam still got same error , help me bro
Hi Christian,
Do you mind share the full error message here? It might help to see what’s going on.
Cheers
Eric
Hi Eric
my error still same “ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter”
and this is my syntax for import with sqoop “sqoop import –connect jdbc:mysql://localhost/movielens –driver com.mysql.jdbc.Driver –table movies -m 1
”
can you help me how to solve this error , thanks…
Hi Christian,
Can you try to below changes:
1. from “localhost” to IP or hostname?
2. you do not have username and password for the mysql connection, please add them
Please also share the errors before “No columns to generate for ClassWriter”, as there might be other useful info there as well.
Cheers
Eric
hi Eric
I am a newbie in sqoop, came across your blog, I am hoping you could help me resolve this issue.
I have a table in MySQL with the primary key as varchar which is carId, importing it to Hadoop with mapper = 5.
–command–
sqoop import –connect jdbc:mysql://sqoopdb.slbdh.cloudlabs.com:3306/ –username –password –table stg_Cars_Classified_Ads –target-dir /user//cars/mysql_hadoop -m 5
It gives error “Caused by: Generating splits for a textual index column allowed only in case of “-Dorg.apache.sqoop.splitter.allow_text_splitter=true” property passed as a parameter”.
I tried adding -Dorg.apache.sqoop.splitter.allow_text_splitter=true in the command , it gives another error of “Import failed: java.io.IOException: No columns to generate for ClassWriter”.
I believe it is because the carId is varchar and not int, how do you go about fixing it.
Hi Samira,
Thanks for visiting my blog and posting questions.
Can you add “–verbose” to the Sqoop command and see if you can get more details about the error? I realise “No columns to generate for ClassWriter” is quite generic and can happen by many different causes.
Cheers
Eric
Hello Eric,
It took me while to get back. The error says, connection refused.
I have tried researching a lot, but could not understand.
I have the error message below.
Thank you,
samira
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
21/02/17 04:42:47 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-cdh6.3.2
21/02/17 04:42:47 DEBUG tool.BaseSqoopTool: Enabled debug logging.
21/02/17 04:42:47 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
21/02/17 04:42:47 DEBUG tool.BaseSqoopTool: Parquet job configurator implementation set: HADOOP
21/02/17 04:42:47 DEBUG sqoop.ConnFactory: Loaded manager factory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
21/02/17 04:42:47 DEBUG sqoop.ConnFactory: Loaded manager factory: org.apache.sqoop.manager.DefaultManagerFactory
21/02/17 04:42:47 DEBUG sqoop.ConnFactory: Trying ManagerFactory: org.apache.sqoop.manager.oracle.OraOopManagerFactory
21/02/17 04:42:47 DEBUG oracle.OraOopUtilities: Enabled OraOop debug logging.
21/02/17 04:42:47 DEBUG oracle.OraOopManagerFactory: Data Connector for Oracle and Hadoop can be called by Sqoop!
21/02/17 04:42:47 DEBUG sqoop.ConnFactory: Trying ManagerFactory: org.apache.sqoop.manager.DefaultManagerFactory
21/02/17 04:42:47 DEBUG manager.DefaultManagerFactory: Trying with scheme: jdbc:mysql:
21/02/17 04:42:47 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
21/02/17 04:42:47 DEBUG sqoop.ConnFactory: Instantiated ConnManager [email protected]
21/02/17 04:42:47 INFO tool.CodeGenTool: Beginning code generation
21/02/17 04:42:47 DEBUG manager.SqlManager: Execute getColumnInfoRawQuery : SELECT t.* FROM `stg_Cars_Classified_Ads` AS t LIMIT 1
Loading class `com.mysql.jdbc.Driver’. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver’. The driver is automatically registered via the SPI a
nd manual loading of the driver class is generally unnecessary.
21/02/17 04:42:47 DEBUG manager.SqlManager: No connection paramenters specified. Using regular API for making connection.
21/02/17 04:42:47 ERROR manager.SqlManager: Error executing statement: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
I am really sorry Samira, I over looked your reply.
Based on the error “Communications link failure”, it looks like driver was not able to reach to the MySQL server, have you tried and tested that all hosts in the cluster can reach the MySQL server?
Cheers
Eric