If you're a programmer and you want to put a certain functionality in your software, you start by thinking of ways to implement it—such as writing a method, defining a class, or creating new data types. Then you write the implementation in a language that the compiler or interpreter can understand. But what if the compiler or interpreter does not understand the instructions as you had them in mind, even though you're sure you did everything right? What if the software works fine most of the time but causes bugs in certain circumstances? In these cases, you have to know how to use a debugger correctly to find the source of your troubles.
The GNU Project Debugger ([GDB][2]) is a powerful tool for finding bugs in programs. It helps you uncover the reason for an error or crash by tracking what is going on inside the program during execution.
This article is a hands-on tutorial on basic GDB usage. To follow along with the examples, open the command line and clone this repository:
Every command in GDB can be shortened. For example, `info break`, which shows the set breakpoints, can be shortened to `i break`. You might see those abbreviations elsewhere, but in this article, I will write out the entire command so that it is clear which function is used.
You can attach GDB to every executable. Navigate to the repository you cloned, and compile it by running `make`. You should now have an executable called **coredump**. (See my article on [Creating and debugging Linux dump files][3] for more information..
To attach GDB to the executable, type: `gdb coredump`.
Your output should look like this:
![gdb coredump output][4]
It says no debugging symbols were found.
Debugging information is part of the object file (the executable) and includes data types, function signatures, and the relationship between the source code and the opcode. At this point, you have two options:
The `g` option tells the compiler to include the debug information. Run `make clean` followed by `make` and invoke GDB again. You should get this output and can start debugging the code:
The additional debugging information will increase the size of the executable. In this case, it increases the executable by 2.5 times (from 26,088 byte to 65,480 byte).
The command `info shared` prints a list of dynamic libraries with their addresses in the virtual address space that was loaded on startup so that the program will execute:
You may have noticed that you can start the program inside GDB with the `run` command. The `run` command accepts command-line arguments like you would use to start the program from the console. The `-c1` switch will cause the program to crash on stage 4. To run the program from the beginning, you don't have to quit GDB; simply use the `run` command again. Without the `-c1` switch, the program executes an infinite loop. You would have to stop it with **Ctrl+C**.
You can also execute a program step by step. In C/C++, the entry point is the `main` function. Use the command `list main` to open the part of the source code that shows the `main` function:
You are now in GDB's text user interface (TUI) mode. Use the Up and Down arrow keys to scroll through the source code.
GDB highlights the line to be executed. By typing `next` (n), you can execute the commands line by line. GBD executes the last command if you don't specify a new one. To step through the code, just hit the **Enter** key.
From time to time, you will notice that TUI's output gets a bit corrupted:
The heart of this example program consists of a state machine running in an infinite loop. The variable `n_state` is a simple enum that determines the current state:
You want to stop the program when `n_state` is set to the value `State_5`. To do so, stop the program at the `main` function and set a watchpoint for `n_state` :
As you can see, watchpoints are numbers. To delete a specific watchpoint, type `delete` followed by the number of the watchpoint. For example, my watchpoint has the number 2; to remove this watchpoint, enter `delete 2`.
To remove a single breakpoint, type `delete` followed by its number. Alternatively, you can remove a breakpoint by specifying its line number. For example, the command `clear 78` will remove breakpoint number 7, which is set on line 78.
#### Disable or enable breakpoints and watchpoints
Instead of removing a breakpoint or watchpoint, you can disable it by typing `disable` followed by its number. In the following, breakpoints 3 and 4 are disabled and are marked with a minus sign in the code window:
It is also possible to modify a range of breakpoints or watchpoints by typing something like `disable 2 - 4`. If you want to reactivate the points, type `enable` followed by their numbers.
### Conditional breakpoints
First, remove all breakpoints and watchpoints by typing `delete`. You still want the program to stop at the `main` function, but instead of specifying a line number, add a breakpoint by naming the function directly. Type `break main` to add a breakpoint at the `main` function.
Type `run` to start the execution from the beginning, and the program will stop at the `main` function.
The `main` function includes the variable `n_state_3_count`, which is incremented when the state machine hits state 3.
To add a conditional breakpoint based on the value of `n_state_3_count` type:
Continue the execution. The program will execute the state machine three times before it stops at line 54. To check the value of `n_state_3_count`, type:
It is also possible to make an existing breakpoint conditional. Remove the recently added breakpoint with `clear 54`, and add a simple breakpoint by typing `break 54`. You can make this breakpoint conditional by typing:
If you have a program that consists of several source files, you can set breakpoints by specifying the file name before the line number, e.g., `break main.cpp:54`.
#### Catchpoints
In addition to breakpoints and watchpoints, you can also set catchpoints. Catchpoints apply to program events like performing syscalls, loading shared libraries, or raising exceptions.
To catch the `write` syscall, which is used to write to STDOUT, enter:
Printing the values of variables is done with the `print` command. The general syntax is `print <expression> <value>`. The value of a variable can be modified by typing:
The command `gdb attach <process-id>` allows you to attach to an already running process by specifying the process ID (PID). Luckily, the `coredump` program prints its current PID to the screen, so you don't have to manually find it with [ps][33] or [top][34].
The output shows the process interrupted while executing the `std::this_thread::sleep_for<...>(...)` function that was called in line 92 of `main.cpp`.
As soon as you quit GDB, the process will continue running.
Usually, the compiler will create a subroutine for each function or method. Each subroutine has its own stack frame, so moving upwards in the stackframe means moving upwards in the callstack.
When attaching to an already running process, GDB will look for the source files in the current working directory. Alternatively, you can specify the source directories manually with the [directory command][41].
The output of `backtrace` shows that the crash happened five stack frames away from `main.cpp`. Enter to jump directly to the faulty line of code in `main.cpp` :
A look at the source code shows that the program tried to free a pointer that was not returned by a memory management function. This results in undefined behavior and caused the `SIGABRT`.
### Debug without symbols
If there are no sources available, things get very hard. I had my first experience with this when trying to solve reverse-engineering challenges. It is also useful to have some knowledge of [assembly language][47].
Check out how it works with this example.
Go to the source directory, open the **Makefile**, and edit line 9 like this:
To recompile the program, run `make clean`followed by `make` and start GDB. The program no longer has any debugging symbols to lead the way through the source code.
The entry point corresponds with the beginning of the `.text` area, which contains the actual opcode. To add a breakpoint at the entry point, type `break *0x401110` then start execution by typing `run` :
Before digging deeper into assembly, you can choose which [assembly flavor][51] to use. GDB's default is AT&T, but I prefer the Intel syntax. Change it with:
Now open the assembly and register the window by typing `layout asm` and `layout reg`. You should now see output like this:
![layout asm and layout reg output][53]
#### Save configuration files
Although you have already entered many commands, you haven't actually started debugging. If you are heavily debugging an application or trying to solve a reverse-engineering challenge, it can be useful to save your GDB-specific settings in a file.
The `set write on` command enables you to modify the binary during execution.
Quit GDB and reopen it with the configuration file:`gdb -x gdbinit coredump`.
#### Read instructions
With the `c2` switch applied, the program will crash. The program stops at the entry function, so you have to write `continue` to proceed with execution:
![continuing execution after crash][55]
The `idiv` instruction performs an integer division with the dividend in the `RAX` register and the divisor specified as an argument. The quotient is loaded into the `RAX` register, and the remainder is loaded into `RDX`.
To read raw memory content, you must specify a few more parameters than for reading symbols. When you scroll up a bit in the assembly output, you can see the division of the stack:
![stack division output][56]
You're most interested in the value of `rbp-0x4` because this is the position where the argument for `idiv` is stored. From the screenshot, you can see that the next variable is located at `rbp-0x8`, so the variable at `rbp-0x4` is 4 bytes wide.
Skip this subroutine completely. You can check the call stack with `backtrace`. You are only one stack frame ahead of your `main` function, so you can go back to `main` with a single `up` :
The subroutine `zeroDivide()` is entered only when `jump equal (je)` evaluates to `true`. You can easily replace this with a `jump-not-equal (jne)` instruction, which has the opcode `0x75` (provided you are on an x86/64 architecture; the opcodes are different on other architectures). Restart the program by typing `run`. When the program stops at the entry function, manipulate the opcode by typing:
Finally, type `continue`. The program will skip the subroutine `zeroDivide()` and won't crash anymore.
### Conclusion
You can find GDB working in the background in many integrated development environments (IDEs), including Qt Creator and the [Native Debug][62] extension for VSCodium.
![GDB in VSCodium][63]
It's useful to know how to leverage GDB's functionality. Usually, not all of GDB's functions can be used from the IDE, so you benefit from having experience using GDB from the command line.
