Making Pig 0.12 work with hadoop 2.2.0

Vote count:

0

I have hadoop 2.2.0 running on remote cluster and Pig 0.12 on a separate machine. I need to make Pig communicate with hadoop and the first steps seems to be to build Pig 0.12 with hadoop 2.2.0. Here is what I did:

1. In ivy/libraries.properties changed the hadoop-core.version, hadoop-common.version, hadoop-hdfs.version, hadoop-mapreduce.version to 2.2.0.


2. In ivy.xml replaced the hadoop-core dependency to hadoop-client dependency.


3. Build pig using "ant clean jar-all -Dhadoopversion=23"

When I run "pig" from command line I get the following error:

ERROR 2998: Unhandled internal error. org/apache/hadoop/hdfs/DistributedFileSystem

java.lang.NoClassDefFoundError: org/apache/hadoop/hdfs/DistributedFileSystem at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:173) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:122) at org.apache.pig.impl.PigContext.connect(PigContext.java:301) at org.apache.pig.PigServer.(PigServer.java:222) at org.apache.pig.PigServer.(PigServer.java:207) at org.apache.pig.tools.grunt.Grunt.(Grunt.java:47) at org.apache.pig.Main.run(Main.java:538) at org.apache.pig.Main.main(Main.java:156) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hdfs.DistributedFileSystem at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247)

asked 1 min ago

user2242284

6