Exception in thread "main" java.lang.IncompatibleClassChangeError:
Found class org.apache.hadoop.mapreduce.JobContext, but interface was
expected at org.apache.sqoop.config.ConfigurationHelper.getJobNumMaps(ConfigurationHelp
er.java:53)
-- Clean build fixes the issue.
Exception in thread "main" java.lang.NoSuchMethodError: org/apache/thrift/EncodingUtils.setBit(BIZ)B
-- Ensure libthrift.jar in HIVE, SQOOP, and HBASE are same
Exception in thread "main" java.lang.NoClassDefFoundError: com.sun.security.auth.UnixPrincipal
-- Need Oracle JDK as Hadoop is built using Oracle JDK
Command: $start-hbase.sh
java.net.ConnectException: Call to localhost/127.0.0.1:8020 failed on connection exception: java.net.ConnectException: Connection refused
-- HDFS listens to port 9000, hence the hbase should point to port 9000.
FATAL zookeeper.ZKConfig: The server in zoo.cfg cannot be set to localhost in a fully-distributed setup because it won't be reachable. See "Getting Started" for more information.
-- Its a warning.
org.apache.hadoop.hbase.util.FileSystemVersionException: File system needs to be upgraded. You have version null and I want version 7. Run the '${HBASE_HOME}/bin/hbase migrate' script.
at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:349)
at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:354)
at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:130)
at org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:115)
at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:513)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:376)
at java.lang.Thread.run(Thread.java:780)
2013-05-30 19:56:16,050 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
2013-05-30 19:56:16,050 DEBUG org.apache.hadoop.hbase.master.HMaster: Stopping service threads
2013-05-30 19:56:16,050 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60000
--Following command fixes the issue:
$ bin/hadoop fs -rmr /hbase
Deleted hdfs://localhost:9000/hbase
13/05/30 22:04:45 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x13ef63ab322000d, negotiated timeout = 40000
java.lang.IllegalArgumentException: Not a host:port pair:
-- This issue is caused by version mismatch of the jar files loaded in the client. The issue can be avoided by replacing all client jar with server specific jar files.
2013-05-31 09:33:23,583 WARN conf.HiveConf (HiveConf.java:(75)) - hive-site.xml not found on CLASSPATH
2013-05-31 09:33:24,273 ERROR tool.ImportTool (ImportTool.java:run(484)) - Encountered IOException running import job: java.io.IOException: Exception thrown in Hive
at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:335)
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:131)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:123)
at hadoop.sqoop.HiveSqooper.cleanUp(HiveSqooper.java:145)
at hadoop.sqoop.HiveSqooper.main(HiveSqooper.java:47)
ERROR tool.ImportTool (ImportTool.java:run(484)) - Encountered IOException running import job: java.io.IOException: Exception thrown in Hive
Caused by: java.lang.NoClassDefFoundError: org/antlr/runtime/tree/TreeAdaptor
Caused by: java.lang.ClassNotFoundException: org.antlr.runtime.tree.TreeAdaptor
------- hadoop-env.sh
export HIVE_CONF_DIR=/home/surajit/hadoop/hive-0.10.0/conf
# Extra Java CLASSPATH elements. Optional.
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HIVE_CONF_DIR
2013-05-31 12:16:01,938 ERROR ql.Driver (SessionState.java:printError(401)) - FAILED: SemanticException [Error 10028]: Line 2:17 Path is not legal ''file:/home/surajit/workspace/infolytics/bse30'': Move from: file:/home/surajit/workspace/infolytics/bse30 to: hdfs://localhost:9000/user/hive/warehouse/bse30 is not valid. Please check that values for params "default.fs.name" and "hive.metastore.warehouse.dir" do not conflict.
org.apache.hadoop.hive.ql.parse.SemanticException: Line 2:17 Path is not legal ''file:/home/surajit/workspace/infolytics/bse30'': Move from: file:/home/surajit/workspace/infolytics/bse30 to: hdfs://localhost:9000/user/hive/warehouse/bse30 is not valid. Please check that values for params "default.fs.name" and "hive.metastore.warehouse.dir" do not conflict.
SqoopOptions.setWarehouseDir("hdfs://localhost:9000/user/hive/warehouse/sensex.db"); --- fixes the issues.
PIG Exceptions:
-------------------------------------------------------------------------------------
ERROR 1070: Could not resolve HBaseStorage using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse:
-- Full path required - org.apache.pig.backend.hadoop.hbase.HBaseStorage
FMCG1 = LOAD 'hbase://bse30' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('firstdecade:*') as (id:bytearray, histdate:chararray, index_name:chararray, open:double, high:double, low:double, close:double);
ERROR 2998: Unhandled internal error. org.apache.zookeeper.ZooKeeper
java.lang.NoClassDefFoundError: org.apache.zookeeper.ZooKeeper
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.initialiseHBaseClassLoaderResources(HBaseStorage.java:712)
at org.apache.pig.backend.hadoop.hbase.HBaseStorage.setLocation(HBaseStorage.java:667)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:473)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:294)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:177)
at org.apache.pig.PigServer.launchPlan(PigServer.java:1264)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
at org.apache.pig.PigServer.storeEx(PigServer.java:931)
at org.apache.pig.PigServer.store(PigServer.java:898)
at org.apache.pig.PigServer.openIterator(PigServer.java:811)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:538)
at org.apache.pig.Main.main(Main.java:157)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.ZooKeeper
---- Copy hbase-*.jar and zookeeper*.jar into $PIG_HOME/lib folder
1 comment:
Thanks, the following solution worked
Exception in thread "main" java.lang.NoSuchMethodError: org/apache/thrift/EncodingUtils.setBit(BIZ)B
-- Ensure libthrift.jar in HIVE, SQOOP, and HBASE are same
Post a Comment