Running a mapreduce jar on Hadoop cluster

I'm trying to run the map reduce implementation of quadratic sieve algorithm on Hadoop. For this purpose I'm using karmasphere Hadoop community plugin with Netbeans. The program works fine using the plugin. But I'm unable to run it on actual cluster. I'm running this command

             bin/hadoop jar 开发者_如何学GoMRIF.jar 689 

Where MRIF.jar is the jar file made by building the netbeans project and 689 is number to be factored. The input and output directories are hard coded in program itself. When running on actual cluster, it appears that the inside java classes are not being processed as reduce completes to 100% before map being at 0% itself. And input and output files are created with no content.

But this works fine when running using Karmasphere plugin.

Try running it as bin/hadoop -jar MRIF.jar 689. The -jar forces it to run local and displays information to the console as well as logs to that machine. You can also check the Hadoop logs to see if they have any indicators of why it's not happening correctly.

When using -jar you can use System.out.println(...); to display information on the console, further helping to debug.

You can also use Hadoop Counters (link is random blog post I found) to assist in troubleshooting when running (psuedo-)distributed.

I admit this post isn't a 'solution' to the problem; Without more/further information about what is happening and where, there is a wide range of things that could be going on. If it is, as you mention, not processing the 'inside java classes' then it would likely be your implementation, of which we can't see to make suggestions, ect.

More data about the issue, such as logs, errors or output will likely assist in getting more solution-y responses instead of debugging tips. :)

EDIT: Thanks for the link to the files. I think your call is missing a component.

I looked in the run.sh and think this might get it to work for you:

bin/hadoop jar mrif.jar com.javiertordable.mrif.MapReduceQuadraticSieve 689