Running MapReduce Programs from eclipse under Windows

Generally when we try to run a mapreduce program from eclipse under windows operating system it gives the fallowing error.

14/08/25 18:09:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

14/08/25 18:09:21 ERROR security.UserGroupInformation: PriviledgedActionException as:{username} cause:java.io.IOException: Failed to set permissions of path: \tmp\hadoop-{username}\mapred\staging\{username}1667268516\.staging to 0700

Exception in thread "main" java.io.IOException: Failed to set permissions of path: \tmp\hadoop-{username}\mapred\staging\{username}1667268516\.staging to 0700

at org.apache.hadoop.fs.FileUtil.checkReturnValue(FileUtil.java:691)

at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:664)

at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:514)

at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:349)

at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:193)

at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:126)

at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:942)

at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)

at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)

at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)

at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)

at wordcount.NewMapRed.main(NewMapRed.java:61)

after some workaround i found the solution for this issue, now i am going to share it with all...

Solution:

install jdk 1.7, set path to jdk1.7/bin
install eclipse(j2se) is fine
install maven plug for eclipse, google "maven plugin for eclipse update site"
download

fix the following error message:

java.io.IOException: Failed to set permissions of path: \tmp\hadoop-XY1267\mapred\staging\XY1267625012668\.staging to 0700

download patch-hadoop_7682-1.0.x-win from the following url

https://github.com/congainc/patch-hadoop_7682-1.0.x-win
add this jar file to eclipse java project classpath

in your program, add the following configuration:

job.getConfiguration().set("fs.file.impl", "com.conga.services.hadoop.patch.HADOOP_7682.WinLocalFileSystem");

now you are ready to run map reduce inside eclipse, on windows! you don't even need to install hadoop onto your pc!

***Thank you ***