Hadoop MAC OS installation woes

So I'm trying to install hadoop on MAC OS X Leopard following the steps in this note: Running Hadoop on a OS X Single Node Cluster.

I reached Step 4: Formatting and running Hadoop, where I entered the following:

hadoop-*/bin/hadoop namenode -format

This produced the following unpleasant output:

Macbook009:~ Hadoop$ hadoop-*/bin/hadoop namenode -format
    Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version number in .class file
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:676)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:317)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:280)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:375)
Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version number in .class file
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:676)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:317)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:280)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
    at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:375)

I did some googling, and learned that JAVA_HOME may be set incorrectly. I created a .bash_profile file like this:

export JAVA_HOME=/system/Library/Frameworks/JavaVM.framework/Versions/1.6/Home
export HADOOP_HOME=~/Users/Hadoop/hadoop-0.20.203.0

export PATH=$HADOOP_HOME/bin:$PATH

No go. Same freaking error. What am I doing wrong?

It may still be a problem with setting your JAVA_HOME as it could be different from examples on the web. Use this command in your terminal to find your JAVA_HOME directory /usr/libexec/java_home

I suspect that the JVM that's actually running Hadoop is not the expected one, but an older one (Java 5). Verify this by running ps (or any Mac equivalent) and examining the command line.

Try to set JAVA_HOME in $HADOOP_HOME/conf/hadoop-env.sh to the same path as you did in your .bash_profile.

This is caused when the jre you are running is older than the jre that compiled the class files. For example running 1.6 compiled java with the 1.5 jre. If the class was compiled just for 1.6 it will not work with 1.5.

Do a

  java -version

and see which jre you have. Most likely you have an old one and need to upgrade it.

Try typing jps and seeing how many nodes are actually running . There should be 6 of them . You shouldn't have a problem hopefully .

Following steps worked seamlessly for me:

http://ragrawal.wordpress.com/2012/04/28/installing-hadoop-on-mac-osx-lion

For wordcount example, you need to copy your file to hdfs, for which you can find the command here: (this is the only step where I struggled after following the above page).

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/#running-a-mapreduce-job

However for newer versions of OS X like 10.9 setting PATH described above is not working.

Found a way where you can add default PATH environment for MAC in file /etc/paths.

Open this file using Terminal in SUDO mode.

$ sudo nano /etc/paths (enter password when prompted).

Append the path in below format.

/users/hadoop/hadoop-1.2.1/bin

/users/hadoop/hadoop-1.2.1/sbin

save the file and restart the Machine. Next time no need to type whole command to run Hadoop script command from Script.