Is using double faster than float?

Double values store higher precision and are double the size of 开发者_运维问答a float, but are Intel CPUs optimized for floats?

That is, are double operations just as fast or faster than float operations for +, -, *, and /?

Does the answer change for 64-bit architectures?

There isn't a single "intel CPU", especially in terms of what operations are optimized with respect to others!, but most of them, at CPU level (specifically within the FPU), are such that the answer to your question:

are double operations just as fast or faster than float operations for +, -, *, and /?

is "yes" -- within the CPU, except for division and sqrt which are somewhat slower for double than for float. (Assuming your compiler uses SSE2 for scalar FP math, like all x86-64 compilers do, and some 32-bit compilers depending on options. Legacy x87 doesn't have different widths in registers, only in memory (it converts on load/store), so historically even sqrt and division were just as slow for double).

For example, Haswell has a divsd throughput of one per 8 to 14 cycles (data-dependent), but a divss (scalar single) throughput of one per 7 cycles. x87 fdiv is 8 to 18 cycle throughput. (Numbers from https://agner.org/optimize/. Latency correlates with throughput for division, but is higher than the throughput numbers.)

The float versions of many library functions like logf(float) and sinf(float) will also be faster than log(double) and sin(double), because they have many fewer bits of precision to get right. They can use polynomial approximations with fewer terms to get full precision for float vs. double

However, taking up twice the memory for each number clearly implies heavier load on the cache(s) and more memory bandwidth to fill and spill those cache lines from/to RAM; the time you care about performance of a floating-point operation is when you're doing a lot of such operations, so the memory and cache considerations are crucial.

@Richard's answer points out that there are also other ways to perform FP operations (the SSE / SSE2 instructions; good old MMX was integers-only), especially suitable for simple ops on lot of data ("SIMD", single instruction / multiple data) where each vector register can pack 4 single-precision floats or only 2 double-precision ones, so this effect will be even more marked.

In the end, you do have to benchmark, but my prediction is that for reasonable (i.e., large;-) benchmarks, you'll find advantage to sticking with single precision (assuming of course that you don't need the extra bits of precision!-).

If all floating-point calculations are performed within the FPU, then, no, there is no difference between a double calculation and a float calculation because the floating point operations are actually performed with 80 bits of precision in the FPU stack. Entries of the FPU stack are rounded as appropriate to convert the 80-bit floating point format to the double or float floating-point format. Moving sizeof(double) bytes to/from RAM versus sizeof(float) bytes is the only difference in speed.

If, however, you have a vectorizable computation, then you can use the SSE extensions to run four float calculations in the same time as two double calculations. Therefore, clever use of the SSE instructions and the XMM registers can allow higher throughput on calculations that only use floats.

Another point to consider is if you are using GPU(the graphics card). I work with a project that is numerically intensive, yet we do not need the percision that double offers. We use GPU cards to help further speed the processing. CUDA GPU's need a special package to support double, and the amount of local RAM on a GPU is quite fast, but quite scarce. As a result, using float also doubles the amount of data we can store on the GPU.

Yet another point is the memory. Floats take half as much RAM as doubles. If you are dealing with VERY large datasets, this can be a really important factor. If using double means you have to cache to disk vs pure ram, your difference will be huge.

So for the application I am working with, the difference is quite important.

I just want to add to the already existing great answers that the __m256? family of same-instruction-multiple-data (SIMD) C++ intrinsic functions operate on either 4 double s in parallel (e.g. _mm256_add_pd), or 8 floats in parallel (e.g. _mm256_add_ps).

I'm not sure if this can translate to an actual speed up, but it seems possible to process 2x as many floats per instruction when SIMD is used.

In experiments of adding 3.3 for 2000000000 times, results are:

Summation time in s: 2.82 summed value: 6.71089e+07 // float
Summation time in s: 2.78585 summed value: 6.6e+09 // double
Summation time in s: 2.76812 summed value: 6.6e+09 // long double

So double is faster and default in C and C++. It's more portable and the default across all C and C++ library functions. Alos double has significantly higher precision than float.

Even Stroustrup recommends double over float:

"The exact meaning of single-, double-, and extended-precision is implementation-defined. Choosing the right precision for a problem where the choice matters requires significant understanding of floating-point computation. If you don't have that understanding, get advice, take the time to learn, or use double and hope for the best."

Perhaps the only case where you should use float instead of double is on 64bit hardware with a modern gcc. Because float is smaller; double is 8 bytes and float is 4 bytes.

The only really useful answer is: only you can tell. You need to benchmark your scenarios. Small changes in instruction and memory patterns could have a significant impact.

It will certainly matter if you are using the FPU or SSE type hardware (former does all its work with 80bit extended precision, so double will be closer; later is natively 32bit, i.e. float).

Update: s/MMX/SSE/ as noted in another answer.

Alex Martelli's answer is good enough, but I want to mention a wrong but somewhat popular test method that may have misled some people:

#include <cstdio>
#include <ctime>
int main() {
  const auto start_clock = clock();
  float a = 0;
  for (int i = 0; i < 256000000; i++) {
    // bad latency benchmark that includes as much division as other operations
    a += 0.11;  // note the implicit conversions of a to double to match 0.11
    a -= 0.13;  // rather than 0.11f
    a *= 0.17;
    a /= 0.19;
  }
  printf("c++ float duration = %.3f\n", 
    (double)(clock() - start_clock) / CLOCKS_PER_SEC);
  printf("%.3f\n", a);
  return 0;
}

It's wrong! C++ default use double, if you replace += 0.11 by += 0.11f, float will usually be faster then double, on x86 CPU.

By the way, on modern SSE instruction set, both float and double have same speed except of division operation, in the CPU core itself. float being smaller may have fewer cache misses if you have arrays of them.

And if the compiler can auto-vectorize, float vectors work on twice as many elements per instruction as double.

Floating point is normally an extension to one's general purpose CPU. The speed will therefore be dependent on the hardware platform used. If the platform has floating point support, I will be surprised if there is any difference.

Previous answers missing a factor that may cause big diff(> 4 X) between float and double: denormal. Avoiding denormal values in C++ Since double have a much wider normal range, for a specific problem that contains many small values, There is much higher probability to fall into denormal range with float than with double, so float could be much slower than double in this case.

In addition some real data of a benchmark to get a glimpse:

For Intel 3770k, GCC 9.3.0 -O2 [3]
Run on (8 X 3503 MHz CPU s)
CPU Caches:
  L1 Data 32 KiB (x4)
  L1 Instruction 32 KiB (x4)
  L2 Unified 256 KiB (x4)
  L3 Unified 8192 KiB (x1)
--------------------------------------------------------------------
Benchmark                          Time             CPU   Iterations
--------------------------------------------------------------------
BM_FloatCreation               0.281 ns        0.281 ns   1000000000
BM_DoubleCreation              0.284 ns        0.281 ns   1000000000
BM_Vector3FCopy                0.558 ns        0.562 ns   1000000000
BM_Vector3DCopy                 5.61 ns         5.62 ns    100000000
BM_Vector3F_CopyDefault        0.560 ns        0.546 ns   1000000000
BM_Vector3D_CopyDefault         5.57 ns         5.56 ns    112178768
BM_Vector3F_Copy123            0.841 ns        0.817 ns    897430145
BM_Vector3D_Copy123             5.59 ns         5.42 ns    112178768
BM_Vector3F_Add                0.841 ns        0.834 ns    897430145
BM_Vector3D_Add                 5.59 ns         5.46 ns    100000000
BM_Vector3F_Mul                0.842 ns        0.782 ns    897430145
BM_Vector3D_Mul                 5.60 ns         5.56 ns    112178768
BM_Vector3F_Compare            0.840 ns        0.800 ns    897430145
BM_Vector3D_Compare             5.61 ns         5.62 ns    100000000
BM_Vector3F_ARRAY_ADD           3.25 ns         3.29 ns    213673844        
BM_Vector3D_ARRAY_ADD           3.13 ns         3.06 ns    224357536        

where operations on 3 float(F) or 3 double(D) are compared and - BM_Vector3XCopy is the pure copy of a (1,2,3) initialized vector not repeated before copy, - BM_Vector3X_CopyDefault with default initialization repeated every copy, - BM_Vector3X_Copy123 with repeated initialization of (1,2,3),

• Add/Mul Each initialize 3 vectors(1,2,3) and add/multiplicate the first and second into the third,

• Compare Checks for equality of two initialized vectors,

• ARRAY_ADD Sums up vector(1,2,3) + vector(3,4,5) + vector(6,7,8) via std::valarray what in my case leads to SSE instructions.

Remember that these are isolated tests and the results differ with compiler settings, from machine to machine or architecture to architecture. With caching (issues) and real world use-cases this may be completely different. So the theory can greatly differ from reality. The only way to find out is a practical test such as with google-benchmark[1] and checking the result of the compiler output for your particular problem solution[2].

1. https://github.com/google/benchmark
2. https://sourceware.org/binutils/docs/binutils/objdump.html -> objdump -S
3. https://github.com/Jedzia/oglTemplate/blob/dd812b72d846ae888238d6f726d503485b796b68/benchmark/Playground/BM_FloatingPoint.cpp