hunting/doc/gdb_coredump_guide.md
2024-07-13 11:24:05 +08:00

9.7 KiB
Raw Blame History

1. gdb coredump分析

本文介绍ubuntu系统环境下coredump文件的分析方法。重要此文的场景在Ubuntu每次开机后需要重复设置coredump调试环境例如开启coredump和设置coredump文件生成路径。

一个coredump示例

$ ../output_files/test/bin/McuManagerTest --gtest_filter=McuManagerTest.RH_INTEGRATION_McuManager_AUTO_OtherSideSendIpcMission
Note: Google Test filter = McuManagerTest.RH_INTEGRATION_McuManager_AUTO_OtherSideSendIpcMission
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from McuManagerTest
NewLog succeed.
LogImplInit
[ RUN      ] McuManagerTest.RH_INTEGRATION_McuManager_AUTO_OtherSideSendIpcMission
2024-05-15 17:43:04,782 INFO [default] [McuManagerMakePtr.cpp:23]:CreateMcuManager is ok.

2024-05-15 17:43:04,783 INFO [default] [IMcuManager.cpp:72]:Instance changed succeed.

2024-05-15 17:43:04,783 INFO [default] [UartDeviceImpl.cpp:277]:Create the uart device object.

Can't Open Serial Port: No such file or directory
2024-05-15 17:43:04,784 ERROR [default] [McuDevice.cpp:51]:IUartOpen failed.

2024-05-15 17:43:04,787 INFO [default] [UartDeviceImpl.cpp:277]:Create the uart device object.

Can't Open Serial Port: No such file or directory
2024-05-15 17:43:04,787 ERROR [default] [McuDevice.cpp:51]:IUartOpen failed.

terminate called without an active exception
Aborted (core dumped)

1.1. coredump文件生成路径查询

$ cat /proc/sys/kernel/core_pattern
|/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E

修改coredump文件生成路径

$ sudo sh -c 'echo /home/xiaojiazhu/project/ipc-sdk/ipc/cmake-shell/core.%e.%p > /proc/sys/kernel/core_pattern'
$ cat /proc/sys/kernel/core_pattern
/home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/cmake-shell/core.%e.%p

命令: sudo sh -c 'echo /path/core.%e.%p > /proc/sys/kernel/core_pattern'

例如(根据自己的实际情况写路径参数)

sudo sh -c 'echo /home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/cmake-shell/core.%e.%p > /proc/sys/kernel/core_pattern'

发现路径下并未生成coredump文件

$ ulimit -a
core file size          (blocks, -c) 0  // 0表示终端未开启core dump
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 63499
max locked memory       (kbytes, -l) 65536
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 63499
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

解决:

$ ulimit -c unlimited

1.2. gdb查看堆栈信息

Ubuntu系统需要加sudo执行程序才会生成coredump文件。

1.2.1. 无法识别coredump文件

在gdb中无法识别coredump文件如下所示

$ sudo gdb ../output_files/test/bin/McuManagerTest core.smbd.3390383
[sudo] password for xiaojiazhu:
Sorry, try again.
[sudo] password for xiaojiazhu:
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ../output_files/test/bin/McuManagerTest...

warning: core file may not match specified executable file.  // 表示coredump文件与可执行文件不匹配
[New LWP 3390383]
Core was generated by `/usr/sbin/smbd --foreground --no-process-group'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007fd59718600b in ?? ()
(gdb) bt
#0  0x00007fd59718600b in ?? ()
#1  0x0000000000001c80 in ?? ()
#2  0x0000000000000000 in ?? ()

由于gdb和asan同时启用会冲突导致无法识别coredump文件。解决办法如下

修改://build/sdk_config.cmake

# Gdb debug
if(${TARGET_PLATFORM} MATCHES ${DEFINE_LINUX})
    message("---------------------------Debug mode.-----------------------")
    SET(CMAKE_BUILD_TYPE "Debug")
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -O0 -Wall -g -ggdb")
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -O0 -Wall -g -ggdb")
    SET(CMAKE_CXX_FLAGS_DEBUG "$ENV{CXXFLAGS} -O0 -Wall -g -ggdb")
    SET(CMAKE_CXX_FLAGS_RELEASE "$ENV{CXXFLAGS} -O3 -Wall")
    # asan
    # set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fsanitize=leak -fsanitize=address -fno-omit-frame-pointer -fsanitize=undefined")
    # set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsanitize=leak -fsanitize=address -fno-omit-frame-pointer -fsanitize=undefined")
else()
    message("---------------------------Release mode.-----------------------")
    SET(CMAKE_BUILD_TYPE "Release")
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Os")
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Os")
endif()

解决后:

$ sudo gdb ../output_files/test/bin/McuManagerTest core.McuManagerTest.3406751
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ../output_files/test/bin/McuManagerTest...
[New LWP 3406751]
[New LWP 3406753]
[New LWP 3406752]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `../output_files/test/bin/McuManagerTest --gtest_filter=McuManagerTest.RH_INTEGR'.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
[Current thread is 1 (Thread 0x7fd355968740 (LWP 3406751))]
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007fd35598f859 in __GI_abort () at abort.c:79
#2  0x00007fd355d678d1 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007fd355d7337c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007fd355d733e7 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x000055c247b3ae41 in std::thread::operator= (__t=..., this=0x55c248d19748) at /usr/include/c++/9/thread:152
#6  McuProtocol::Init (this=this@entry=0x55c248d196f8) at /home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/utils/McuProtocol/src/McuProtocol.cpp:58
#7  0x000055c247b03e57 in McuManagerImpl::Init (this=0x55c248d19680)
    at /home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/middleware/McuManager/src/McuManagerImpl.cpp:46
#8  0x000055c247acd3d4 in McuManagerTest::McuManagerTest_RH_INTEGRATION_McuManager_AUTO_OtherSideSendIpcMission_Test::TestBody (this=<optimized out>)
    at /usr/include/c++/9/bits/shared_ptr_base.h:1020
#9  0x000055c247b303fd in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) () at /home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/utils/McuProtocol/./include/McuProtocol.h:121
#10 0x000055c247b23343 in testing::Test::Run() () at /home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/utils/McuProtocol/./include/McuProtocol.h:121
#11 0x000055c247b2342b in testing::TestInfo::Run() ()
    at /home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/utils/McuProtocol/./include/McuProtocol.h:121
#12 0x000055c247b2376e in testing::TestSuite::Run() ()
    at /home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/utils/McuProtocol/./include/McuProtocol.h:121
#13 0x000055c247b2a0fa in testing::internal::UnitTestImpl::RunAllTests() ()
    at /home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/utils/McuProtocol/./include/McuProtocol.h:121
#14 0x000055c247b3085e in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) ()
    at /home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/utils/McuProtocol/./include/McuProtocol.h:121
#15 0x000055c247b23529 in testing::UnitTest::Run() ()
    at /home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/utils/McuProtocol/./include/McuProtocol.h:121
#16 0x000055c247ac9287 in RUN_ALL_TESTS ()
    at /home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/external/gtest/googletest-release-1.11.0/googletest/include/gtest/gtest.h:2490
#17 main (argc=<optimized out>, argv=<optimized out>)
    at /home/xiaojiazhu/project/rkipc/battery/ipc-rk1106/ipc-sdk/test/middleware/McuManager/mainTest.cpp:22
(gdb)