When can Java produce a NaN?

I know what Java Double.NaN is. I have some Java code that produces NaN.

// calculate errors
delta = m1 + m2 - M;
eta = f1 + f2 - F;
f开发者_JAVA百科or (int i = 0; i < numChildren; i++) {
  epsilon[i] = p[i]*m1+(1-p[i])*m2+q[i]*f1+(1-q[i])*f2-C[i];
}

// use errors in gradient descent
// set aside differences for the p's and q's
float mDiff = m1 - m2;
float fDiff = f1 - f2;
// first update m's and f's
m1 -= rate*delta;
m2 -= rate*delta;
f1 -= rate*eta;
f2 -= rate*eta;
for (int i = 0; i < numChildren; i++) {
  m1 -= rate*epsilon[i]*p[i];
  m2 -= rate*epsilon[i]*(1-p[i]);
  f1 -= rate*epsilon[i]*q[i];
  f2 -= rate*epsilon[i]*(1-q[i]);
}
// now update the p's and q's
for (int i = 0; i < numChildren; i++) {
  p[i] -= rate*epsilon[i]*mDiff;
  q[i] -= rate*epsilon[i]*fDiff;  
}

Under what circumstances will Java produce a NaN value?

NaN is triggered by the following occurrences:

• results that are complex values
  • √x where x is negative
  • log(x) where x is negative
  • tan(x) where x mod 180 is 90
  • asin(x) or acos(x) where x is outside [-1..1]
• 0/0
• ∞/∞
• ∞/−∞
• −∞/∞
• −∞/−∞
• 0×∞
• 0×−∞
• 1^∞
• ∞ + (−∞)
• (−∞) + ∞

Sorry for such a general answer, but I hope that helped.

According to Wikipedia:

There are three kinds of operation which return NaN:

• Operations with a NaN as at least one operand
• Indeterminate forms
  • The divisions 0/0, ∞/∞, ∞/−∞, −∞/∞, and −∞/−∞
  • The multiplications 0×∞ and 0×−∞
  • The power 1^∞
  • The additions ∞ + (−∞), (−∞) + ∞ and equivalent subtractions.
• Real operations with complex results:
  • The square root of a negative number
  • The logarithm of a negative number
  • The tangent of an odd multiple of 90 degrees (or π/2 radians)
  • The inverse sine or cosine of a number which is less than −1 or greater than +1.

This Java snippet illustrates all of the above, except the tangent one (I suspect because of limited precision of double):

import java.util.*;
import static java.lang.Double.NaN;
import static java.lang.Double.POSITIVE_INFINITY;
import static java.lang.Double.NEGATIVE_INFINITY;

public class NaN {
    public static void main(String args[]) {
        double[] allNaNs = {
            0D/0D,
            POSITIVE_INFINITY / POSITIVE_INFINITY,
            POSITIVE_INFINITY / NEGATIVE_INFINITY,
            NEGATIVE_INFINITY / POSITIVE_INFINITY,
            NEGATIVE_INFINITY / NEGATIVE_INFINITY,
            0 * POSITIVE_INFINITY,
            0 * NEGATIVE_INFINITY,
            Math.pow(1, POSITIVE_INFINITY),
            POSITIVE_INFINITY + NEGATIVE_INFINITY,
            NEGATIVE_INFINITY + POSITIVE_INFINITY,
            POSITIVE_INFINITY - POSITIVE_INFINITY,
            NEGATIVE_INFINITY - NEGATIVE_INFINITY,
            Math.sqrt(-1),
            Math.log(-1),
            Math.asin(-2),
            Math.acos(+2),
        };
        System.out.println(Arrays.toString(allNaNs));
        // prints "[NaN, NaN...]"
        System.out.println(NaN == NaN); // prints "false"
        System.out.println(Double.isNaN(NaN)); // prints "true"
    }
}

References

• Wikipedia/NaN
• JLS 15.21.1 Numerical Equality Operators == and !=

  If either operand is NaN, then the result of == is false but the result of != is true. Indeed, the test x!=x is true if and only if the value of x is NaN. (The methods Float.isNaN and Double.isNaN may also be used to test whether a value is NaN.)

Given what I know about gradient descent, you are most probably jumping out to infinity because you do not have adaptive rate (i.e. your rate is too big).

Have you tried sprinkling your code with System.out.println statements to determine exactly where NaNs start occuring?