coredump是一个包含程序意外终止时的内存快照的文件。它可以用于事后调试，以了解崩溃发生的原因以及其中涉及的变量。通过GOTRACEBACK，Go提供了一个环境变量来控制程序崩溃时产生的输出。这个变量可以强制生成coredump，便于调试。

让golang程序生成core文件
a. ulimit -c unlimited 修改 core 文件的大小
b. 环境变量export GOTRACEBACK=crash 说明golang程序产生coredump

可以使用gdb对coredump进行查看，delve对golang的兼容更好

1. 编译delve

# git clone https://github.com/derekparker/delve.git
# cd delve/cmd/dlv/
# go build

2. 将 dlv 拷贝到线上有coredump的服务器。
分析coredump

./dlv core ./engine core.1871450_engine --check-go-version=false

3. 输入help可以查看Delve支持的命令

Type 'help' for list of commands.
(dlv) help
The following commands are available:

Running the program:
    call ------------------------ Resumes process, injecting a function call (EXPERIMENTAL!!!)
    continue (alias: c) --------- Run until breakpoint or program termination.
    next (alias: n) ------------- Step over to next source line.
    rebuild --------------------- Rebuild the target executable and restarts it. It does not work if the executable was not built by delve.
    restart (alias: r) ---------- Restart process.
    rev ------------------------- Reverses the execution of the target program for the command specified.
    rewind (alias: rw) ---------- Run backwards until breakpoint or program termination.
    step (alias: s) ------------- Single step through program.
    step-instruction (alias: si)  Single step a single cpu instruction.
    stepout (alias: so) --------- Step out of the current function.

Manipulating breakpoints:
    break (alias: b) ------- Sets a breakpoint.
    breakpoints (alias: bp)  Print out info for active breakpoints.
    clear ------------------ Deletes breakpoint.
    clearall --------------- Deletes multiple breakpoints.
    condition (alias: cond)  Set breakpoint condition.
    on --------------------- Executes a command when a breakpoint is hit.
    toggle ----------------- Toggles on or off a breakpoint.
    trace (alias: t) ------- Set tracepoint.
    watch ------------------ Set watchpoint.

Viewing program variables and memory:
    args ----------------- Print function arguments.
    display -------------- Print value of an expression every time the program stops.
    examinemem (alias: x)  Examine memory:
    locals --------------- Print local variables.
    print (alias: p) ----- Evaluate an expression.
    regs ----------------- Print contents of CPU registers.
    set ------------------ Changes the value of a variable.
    vars ----------------- Print package variables.
    whatis --------------- Prints type of an expression.

Listing and switching between threads and goroutines:
    goroutine (alias: gr) -- Shows or changes current goroutine
    goroutines (alias: grs)  List program goroutines.
    thread (alias: tr) ----- Switch to the specified thread.
    threads ---------------- Print out info for every traced thread.

Viewing the call stack and selecting frames:
    deferred --------- Executes command in the context of a deferred call.
    down ------------- Move the current frame down.
    frame ------------ Set the current frame, or execute command on a different frame.
    stack (alias: bt)  Print stack trace.
    up --------------- Move the current frame up.

Other commands:
    check (alias: checkpoint) ----------- Creates a checkpoint at the current position.
    checkpoints ------------------------- Print out info for existing checkpoints.
    clear-checkpoint (alias: clearcheck)  Deletes checkpoint.
    config ------------------------------ Changes configuration parameters.
    disassemble (alias: disass) --------- Disassembler.
    dump -------------------------------- Creates a core dump from the current process state
    edit (alias: ed) -------------------- Open where you are in $DELVE_EDITOR or $EDITOR
    exit (alias: quit | q) -------------- Exit the debugger.
    funcs ------------------------------- Print list of functions.
    help (alias: h) --------------------- Prints the help message.
    libraries --------------------------- List loaded dynamic libraries
    list (alias: ls | l) ---------------- Show source code.
    source ------------------------------ Executes a file containing a list of delve commands
    sources ----------------------------- Print list of source files.
    types ------------------------------- Print list of types

4. goroutine 显示或修改当前goroutine

(dlv) goroutine
Thread 1871459 at /usr/local/go/src/runtime/sys_linux_amd64.s:165
Goroutine 2072788:
        Runtime: /usr/local/go/src/runtime/sys_linux_amd64.s:165 runtime.raise (0x474461)
        User: /data/home/XXX/XXX/XXX.go:104 XXXXXX/XXX/XXX/XXX (0x14f454d)
        Go:/data/home/XXX/XXX/XXXXXXX/XXX/XXX/[email protected]/xxx.go:281 XXXXXX/XXX/XXX/XXX (0xc9c9e7)
        Start: /data/home/XXX/XXX/XXXXXXX/XXX/XXX/[email protected]/xxx.go:122 XXXXXX/XXX/XXX/XXX (0xc9be20)

可以看到core发生的位置