Can not get any information from gdb backtrace

I have a release version server process running under linux 64-bit systems. It got crashed and left a coredump file. I use gdb to debug it like this:

gdb svr coredump file

And got the following backtraces:

(gdb) where
#0  0x0000000000432691 in ?? ()
#1  0x00002b07655a50cc in ?? ()
#2  0x00002b07655a50c4 in ?? ()
#3  0x00007fff9fade920 in ?? ()
#4  0x0000000000439db3 in ?? ()
#5  0x00007fff9fade910 in ?? ()
#6  0x00007fff9fade910 in ?? ()
#7  0x00007fff9fade8e0 in ?? ()
#8  0x00000000004663e2 in ?? ()
#9  0x00007fff9fade910 in ?? ()
#10 0x00007fff9fade910 in ?? ()
#11 0x00007fff9fade930 in ?? ()
#12 0x0000000000605108 in ?? ()
#13 0x00002b07655a274c in ?? ()
#14 0x0000000000ebc678 in ?? ()
#15 0x169f49f100000001 in ?? ()
#16 0x00000000021dc6c0 in ?? ()
#17 0x00002b07655a284c in ?? ()
#18 0x00002b07655a27dc in ?? ()
#19 0x00007fff9fade960 in ?? ()
#20 0x000000000043a03b in ?? ()
#21 0x00007fff9fade960 in ?? ()
#22 0x0000000000994d02 in ?? ()
#23 0x00000000000ecd57 in ?? ()
#24 0x00002b07655a274c in ?? ()
#25 0x00002b07655a274c in ?? ()
#26 0x00002b07655a27dc in ?? ()
#27 0x00007fff9fade980 in ?? ()
#28 0x000000000060a5eb in ?? ()
#29 0x000000009fadeb50 in ?? ()
#30 0x00002b07655a27开发者_运维问答4c in ?? ()
#31 0x00007fff9fade9d0 in ?? ()
#32 0x000000000060a8f0 in ?? ()
#33 0x00007fff9fadeb50 in ?? ()
#34 0x00007fff9fadea10 in ?? ()
#35 0x00002b07655a274c in ?? ()
#36 0x00007fff9fadea10 in ?? ()
#37 0x000000009fade9d0 in ?? ()
#38 0x00007fff9fadeb58 in ?? ()
#39 0x0000000000000000 in ?? ()

The address info can not be analyzed by addr2line, what is the problem and how can I find what is the root cause of coredump?

Thanks!

Are you running GDB on the machine on which the core dump was produced?

For GDB to correctly reconstruct the crash stack trace, it must have access to exact binaries which were used at the time of the crash (or you get garbage).

The easiest way to achieve this is to analyze the core on the machine on which it was produced. If that's not feasible, you must copy all shared libraries to e.g. /tmp/solib-on-server/ (preserve the path, so e.g. /lib/libc.so.6 goes into /tmp/solib-on-server/lib/libc.so.6), then use GDB set solib-absolute-prefix /tmp/solib-on-server command before loading the core. E.g.

gdb -ex 'set solib-absolute-prefix /tmp/solib-on-server' \
    -ex 'core /path/to/core' /path/to/executable

Its very hard to debug programs without debug symbols. As you are using release version of the application the core dump will not contain any debug information.

I am not sure, but if you can correlate the stack trace with the ".map" file of your application you could be getting somewhere. If I was in your position I would have recompiled the app with debug symbols(-g compiler flag) and then proceed on debugging.

You can use below commands in gdb to set shared library path.

set solib-search-path [Directories]

[Directories] : paths separated by colon(Ex /usr/lib:/usr/lib64)

show solib-search-path

These two commands helped me to get some info.