Debugging a SPU

Debugging a Chromium SPU can be difficult (especially on the application side). The normal approach of running an application with a debugger won't work since the crappfaker is needed in order to make the application use Chromium instead of the native OpenGL library. Debugging the crappfaker doesn't typically work since crappfaker forks/execs the application.

This section describes some techniques you can use to debug a SPU. 

Windows Technique 1: Write a batch file

This is by far the simplest thing you can do with Windows. Create a batch file called crdebug.bat, which contains the following commands:

cp %1.exe c:/work/cr/scratch
cp c:/work/cr/bin/WIN_NT/crfaker.dll c:/work/cr/scratch/opengl32.dll
"C:\VS60\Common\MSDev98\Bin\MSDEV.EXE" c:\work\cr\scratch\%1.exe

Using this batch file (typically put in bin/WIN_NT/), go to the directory containing the executable, and type:

crdebug <executable_name>

That's it.  But one will have to modify the paths to match where Chromium is installed on the system.

This uses a "scratch" directory made in c:\work\cr\scratch.  It basically copies the faker DLL to the scratch directory, along with the executable, and runs the debugger on the executable.  Windows will search the directory that contains the executable first, so this technique works (it's a lot like how the application faker works).

This technique has a drawback, however.  Since SPUs are loaded explicitly when an OpenGL context is created, you cannot set a breakpoint inside a SPU when the program is loaded.  You have to single-step through the program until the context is created, and then you may set breakpoints.  Every time you restart the Microsoft debugger, your breakpoints will all be temporarily disabled.  You can re-enable them all at once by using the "Edit->Breakpoints" menu item.  This is tedious, but it's the path of least resistance.

Windows Technique 2: Use a programmatic breakpoint

In Windows, you can add the statement:

DebugBreak();

anywhere in your code, and when that statement is executed, you will be prompted to load the debugger at that point.  The debugger will come up pointing at some assembly code, and by stepping forward (with the F10 key) twice, you'll be back at the place where DebugBreak appears.  The program is still running!

This is often a useful technique for setting conditional breakpoints deep within a library, since you don't have to worry about waiting for the context to be created, etc.  Also, this is by far the best way to enter the debugger in your SPUInit function.

Linux: Create a "debug" directory

To debug SPU's on Linux, create a directory called ldebug/, which is populated with symlinks to the OpenGL faker library.  Here is a listing of the contents of that directory:

chromite(cr)% ls -l ldebug
total 0
lrwxrwxrwx 1 humper graphics 41 Jul 3 10:24 libGL.so -> /u/humper/work/cr/lib/Linux/libcrfaker.so*
lrwxrwxrwx 1 humper graphics 8 Jul 3 10:25 libGL.so.1 -> libGL.so*
lrwxrwxrwx 1 humper graphics 8 Jul 3 10:25 libGL.so.1.0.1251 -> libGL.so*

If you have "." (the current directory) on your LD_LIBRARY_PATH, you can just change to this directory before invoking the debugger.  Otherwise, add this directory to the head of your LD_LIBRARY_PATH. Then, run gdb on the application:

% gdb atlantis

Set a breakpoint in main and run the program:

(gdb) break main
(gdb) run

When execution stops in main, set a breakpoint in glXMakeCurrent() then continue:

(gdb) break glXMakeCurrent
(gdb) c

At this point the SPUs will have been loaded and you can set breakpoints in SPU functions.

One disadvantage to this technique is that you won't have control over the application's working directory.  If you need data files (textures, usually), you can make symlinks to those too.

Windows/Unix: Attach a debugger to a running process

Many systems will allow you to attach a debugger to a running process without interrupting it.  In Windows, you can do this simply by right clicking on the process name in the Task Manager. With Linux start gdb with "gdb program processID".   This way, if you want to emulate the DebugBreak functionality without using it explicitly, you can do something like:

volatile int i = 10; // so the compiler doesn't warn or optimize it away
while (i == 10)
{
    // EMPTY BODY
}

Once your program starts to loop forever, attach the debugger, force the value of i to be something else, and you've got your homemade breakpoint.  This is pretty tedious, but handy sometimes when DebugBreak isn't available.

On Unix, use ps to find the process ID of the application. Then, start the debugger, specifying the program and process ID. On Linux this can done with:

gdb atlantis 12345

Where 12345 is the process ID.

Any OS: Debug SPUs on the crserver

Debugging the crserver and SPUs hosted on the crserver is usually easier than debugging SPUs on the application side because no library tricks are used, nor is fork/exec used.

Simply run the crserver with your debugger. Set a breakpoint in the crSPULoadChain. After that function has finished, you should be able to put breakpoints in any SPU function.

Any OS: Use the Print SPU liberally

The print SPU has made debugging of other SPU's much easier.  By placing it between the client and its SPU, and also in front of the render SPU on a server, you can see exactly what transformations are being made to the stream by comparing the two printouts.

Any OS: printf

If all else fails, the tried and true method of inserting printf()'s in your code is one way to determine what's going on.