Neural Network Output Grouping 0.5?

I tried to write a Neural Network system, but even running through simple AND/OR/NOR type problems, the outputs seem to group around 0.5 (for a bias of -1) and 0.7 (for a bias of 1). It doesn't look exactly "wrong"... The 1,1 in the AND pattern does seem higher than the rest and the 0,0 in the OR looks lower, but they are still all grouped so it's 开发者_高级运维debatable.

I was wondering a) if there's some obvious mistake I've made or b) if there's any advice for debugging Neural Nets... seeing as you can't always track back exactly where an answer came from...

Thanks!

Mike

for a) it's impossible to tell without more information.

for b) the best way I have found is to have a pre-calculated set of vectors and weights for a small test version of my network architecture and to have some debug code that dumps the network state out to a csv file so I can look at it in Excel.

The forward (and backward) passes of your network should be completely calculable manually for a single pattern, and a known network initial state, so it's a good idea to have a candidate pattern, or a couple if you can bear doing it manually, fully enumerated for your tests. It's not actually as bad as it sounds.

Doing this has the added advantage that it reinforces your understanding of how the algorithm works and will cause you to check your implementation again from a different standpoint.

good luck.

P.S. shameless plug on backprop blog entry here...

I answered a similar question and I posted some values you can test against:

Artificial neural networks benchmark

Use those values in order to test your neural network. Your outputs should match the values VERY closely, even if you're using different languages... you should be able match at least 6 digits after the decimal without a problem.