GDB Init Files Save Time At Startup

I can’t leave the subject of GDB alone for too long, so today I thought I’d talk about GDB init files.

Each time GDB is run, it checks first your home directory and then the current directory for the existence of a file called .gdbinit. If it finds this file, it reads the contents and runs any commands it finds there.

Note: as of version 7.5 GDB needs permission to load files from anywhere other than your home directory. See this update for further details.

This is extremely handy if you are repeatedly debugging an executable and don’t want to keep typing in the same old commands at startup.

For example, if you always put a breakpoint in a certain method because you like it as a starting point, you can specify this in the .gdbinit file, exactly as you would on the command line for GDB:

b main

And if you just can’t be bothered to keep typing ‘r’ to run GDB once it’s loaded your program, you can specify that too:

b main

You can also add any arguments that you repeatedly type in:

set args param1 param2
b main

And any configuration settings that you might want:

set prompt debug-->
set args param1 param2
b main

Now all you have to do is run GDB on your exe as usual, but now it will set up a custom prompt, pass in your arguments, set a breakpoint in main, and even start the program for you.

Next thing you know, you’ve got a GDB prompt at the top of the main function and you’re ready to go!

GDB Conditional Breakpoints

Last week we looked at setting breakpoints in GDB. All well and good.

But what if you have a large loop running and you only want to look at what’s happening as it nears the end? Do you really have to step through 99 iterations in a 100 item loop?

Of course you don’t. Step forward the conditional breakpoint.

Set a conditional breakpoint using a condition

In GDB you can specify a condition in the programming language you are debugging and apply it to any breakpoint. Let’s stop a loop at the 99th iteration (I’m debugging C/C++, so my conditions are written in C/C++):

(gdb) b Message.cpp:112 if i == 99

That’s all there is to it.

To ensure gdb stops execution, use the first line of code inside the loop as the stopping point, not the loop itself.

You can also specify a condition on an existing breakpoint by using the breakpoint number as a reference:

(gdb) cond 3 i == 99

And remove a condition from a breakpoint using:

(gdb) cond 3
Breakpoint 3 now unconditional.

That’s great! What else can you do?

Pretty much anything you like! Just write the condition exactly as if you were testing for it in your code, e.g.:

(gdb) cond 1 strcmp(message,"earthquake") == 0
//stop if the array message is equal to 'earthquake'
(gdb) cond 2 *p == 'r'
//stop if the char* pointer p points to the letter 'r'
(gdb) cond 3 num < 0.75
//stop while the float num is less than 0.75

Conditional breakpoints covered and your efficiency increased, all in less than 5 minutes!

GDB Breakpoints

Today we’re going to take a quick look at the humble breakpoint in GDB.

You can set a breakpoint:

  • before you run the program in GDB
  • if you interrupt GDB with CTRL-C

Positioning your breakpoints

Set a breakpoint using the handy shortcut ‘b’ followed by the location. There are lots of ways to specify the location, but the most common are:

(gdb) b MessageSender.cpp:118

Pattern: filename:linenumber. Nice and simple. Puts a breakpoint at line 118 in the file MessageSender.cpp

(gdb) b 267

Pattern: linenumber. Puts a breakpoint at line 267 in the current source file. If execution has not started, the breakpoint will be inserted in the main program file.

(gdb) b main

Pattern: functionname. Puts a breakpoint at the beginning of the named function. For C++  class methods, see below.

(gdb) b Message::sendMessage

Pattern: classname::method. Puts a breakpoint at the beginning of the named method. Use tab after the :: to display a list of available methods.

(gdb) b MessageParser.c:getLen

Pattern: filename:function. Puts a breakpoint at the beginning of the named function in the file specified (useful if there is any ambiguity caused by functions with the same name).

Viewing and deleting breakpoints

To see all your breakpoints in a numbered list, type:

(gdb) i b

Delete them all with:

(gdb) d

Or specifically with the breakpoint number from the list:

(gdb) d 4

Preserve your breakpoints

Don’t quit GDB. Just type CTRL-C and then ‘r’ (for run). This will restart your program from the beginning, leaving all your breakpoints intact. Lovely!


GDB Tutorial Command Line Walkthrough : Part 3

Infinite loop

Well, the segfault is gone, but now the code just runs in a continuous loop, which is arguably worse. Run the invader program from GDB and press Ctrl-C to halt execution. View a backtrace by typing bt to see where we are in the code (screenshot below). Note that in a larger, multi-threaded program, when you Ctrl-C into a loop, it may stop at different places each time, so it is often worth trying it more than once and examining the backtrace of each.

gdb backtrace

As you can see, GDB has listed an impressive selection of function calls.

This is the call stack, and it shows you how your program got to its current point. The top line is where the program stopped when you interrupted it, and the last line is the original function call.

This call stack actually contains almost all system library calls, which are not that interesting, as it is unlikely that the bug is in anyone’s code but ours. To examine the frame you are interested in, select it using the framecommand:

(gdb) f 9

GDB prints out the line about to be executed, which is the printf call. A look at the local variables with i lo doesn’t show anything untoward (you can also display the local variables for every stack frame in a backtrace with the command bt full), so what’s going on here?

At least some output is appearing on the screen when the program is run, so it must be the outer forloop (the one that contains the printf statement), that is running repeatedly. It is set to run no more than 36 times, so you can assume that count is never getting high enough to exit the loop. Put a watch on count again to see if’s behaving as expected i.e. incrementing by 2 each time:

(gdb) b 31
(gdb) c
(gdb) watch count
(gdb) c

As soon as GDB stops you can see what the problem is – count is being changed from 2 back to 0. The line reported is the for loop, but this is yet to be executed, so what was the previous statement that just changed count? That’s right – the last statement in the loop resets count to zero. The programmer resets the buffer after printing to the screen and has been over-zealous with their tidying up.

Exit GDB, remove the count = 0; statement at line 47, recompile the program and hurrah! Everything works as expected.

ascii space invader

Luckily this program only had a couple of errors in it, but the techniques that you’ve used to find them will enable you to track down all sorts of lovely bugs, from simple typos to subtle mistakes in pointer handling.

Debugging is very much like being a detective, and GDB is a great tool to aid you in confirming your suspicions about what might be going wrong.

Some other topics worth mentioning

1. Reversible debugging

One of the more exciting new features in GDB 7.0 onwards, is reversible debugging. This is still in its early stages of development, but you can see it at work as follows (type gdb –version to check if you have a recent enough version number). Start the invader program with GDB. Set a breakpoint at line 44. Type run and when the breakpoint is hit, type rec (or target record). Now use the ncommand twice and print out the buffer:

(gdb) p buffer
$1 = ' ' <repeats 45 times>

The buffer has been reset – oops! But we wanted to see what was in it – lets turn back time with rn(or reverse-next). The memset is ‘undone’, and now if we print out the buffer, we can see that it contains the characters that were printed to the screen. Amazing! Examples of other commands you can use to step backwards are rc (reverse-continue), and rs (reverse-step).

Reversible debugging is still in its infancy, and it does make your program run slower (because it is recording and storing so much extra information), but it has great potential and it is fun to see it at work.

2. Automate GDB startup

Each time GDB is run, it checks the local directory for a .gdbinit file and runs any commands it finds there. This is a really useful way to save configuration information that you don’t want to type in each time, such as any runtime arguments or GDB settings. For example, in Part 1 of the tutorial, you can see the following line is output by GDB in Figure 1:

Missing separate debuginfos, use: debuginfo-install glibc-2.11.1-6.i686

This is because more recent versions of GDB will warn you if you do not have the debug information installed for system libraries that are called by your application. For most of us, this is an unnecessary distraction. You can hide these reports by typing in

set build-id-verbose 0

when you start up GDB, or, you can add this line to your .gdbinit file. Just open any text editor, type in your commands (one per line) and save the file as .gdbinit. This file needs to be saved in the directory that you run GDB from, which is usually the one that contains your executable. Remember that files that start with a dot are hidden on Linux, so you won’t be able to see it in the graphical browser unless you enable viewing of hidden files, but running ls -a on the command line will display it. You can set up a separate .gdbnit file for each application you are debugging, as long as your applications don’t all live in the same folder!

3. Attach to a process

You don’t have to start your application with GDB – you can also use GDB to attach to a running program. In a new terminal window, look up the process id (pid) by running ps and note down the number. Then start GDB in a separate terminal window and type attach. GDB will halt the program’s execution and display the now familiar (gdb) prompt, allowing you to debug as if you had run the application directly. To let the program continue as normal, use the detach command to quietly slip away.

4. Core dumps

Traditionally, GDB has been used to examine core dump files. When a program crashes, the Linux kernel can create a “core dump” of data about what was happening in the program at that time. GDB can read this file to give you valuable information about a crash after the event. Many Linux distributions have this ability disabled by default – you need to type ulimit -c unlimited to allow the creation of the files. On Fedora you may also need to update the abrt package withyum update abrt, as it can interfere with core file creation. Once you’ve done that, when a crash occurs, a file called should be created in the same directory as the executable, where pid is the process id.

To examine a core file, just pass it to gdb with gdb (obviously, using your core file’s actual name). GDB will load up all the info and it will look as though you have just run the program and seen the error. You’ll be able to view a backtrace and a range of other information, but it will be a frozen “snapshot” of execution. The advantage of using a core dump is that you don’t have to wait around for the crash to happen, and users can send core files back to the programmer to illustrate a problem, but beyond that there is no substitute for debugging a program live if you are able to.

The end of the beginning

GDB has hundreds of configuration settings and many more commands, so your next stop is the GDB documentation. There is also a slightly out-of-date, but still useful, GDB quick ref PDF reference card floating around online, (just Google for GDB quick ref), which you can hide under some books on your desk, thus fooling people that you really have committed 500 slightly obscure commands to memory.

There should be nothing stopping you now, so enjoy using GDB and long may it aid you in your quest for wonderfully bug free code.

Appendix: GDB Tutorial Quick Ref

b <line num/function>      breakpoint
r <args>                   run/restart, with program arguments
n                          next : step over next line
s                          step : step into next line
finish                     step out of function
c                          continue
l			   list next 10 lines of source
l -		           list previous 10 lines of source
i lo			   info locals – display all variables in the current stack frame
i b 			   info breakpoints – display all watchpoints and breakpoints
watch <var>	           watchpoint
d <num> 		   delete breakpoint/watchpoint
d			   delete all breakpoints/watchpoints
attach <pid> 		   attach to running process
detach			   detach from running process
p			   print
bt			   backtrace
bt full			   backtrace and print all local variables for each frame
frame <num>		   select frame in stack
rec			   target record (for reversible debugging)
rs			   reverse step
rc			   reverse continue
rn			   reverse next