Eigenmumbles

I spent an enormous fraction of this week helping students debug numerical code. Since I keep repeating the same things in these conversations, it probably makes sense to write a few of them down.

Start small

In general, it’s easier to debug a five line routine that does one basic task than to debug a thirty line routine that does six things. You’d really rather not be in the position of having to look at several hundred or thousand lines of code in order to diagnose a problem that some basic sanity checks would have caught when you were looking at a much smaller code segment.

Show pictures

One of the beautiful things about numerical code is that the results can often be interpreted in terms of pictures – and it’s often easier to see wrong behavior in a picture than in a table of numbers. You can get terrific insight from looking at convergence plots, visualizations of solutions, plots of functions that you’re supposedly minimizing, etc.

Use consistent notation

If you let n be a very small number, epsilon be a loop index, and i be a logical flag, you shouldn’t be surprised when you overlook a type error that should have been obvious. Even in very short code segments, notation matters.

Assert your invariants

It’s often easy to check that an output satisfies some basic properties (such as satisfying an equation or a range condition). If something is going wrong in a sequence of steps and you’re not sure where, it makes sense to explicitly test at each step that the output of one step (and input of the next) satisfies the properties you think it ought to have.

Write unit tests

It’s a useful skill to decide in advance what properties each function is supposed to have, and to write a little test script that makes sure your functions do indeed have the right properties. And if you’ve gone to the trouble of writing a test, it might be worth keeping it around for later.

Think about hard cases

If there is a singular case that could cause your code to face plant, you should make sure to check that your code handles that case (and nearby problems) with some modicum of grace.

Beware non-fatal errors

Watch out for code that “works” on the first try. Are you getting the speed and accuracy you expected? For example, Newton’s method converges (locally) quadratically to the solution to a nonlinear equation, but many almost-Newton methods converge linearly. If your code for Newton iteration converges, that doesn’t actually mean that you’ve computed all your derivatives correctly (for example) – it just means that you’re close enough to get something convergent. The difference will only show up if you look at details of the rate of convergence.

Beware error propogation

Watch out for ill-posed intermediate problems. Just because two codes are accurate on their own does not mean the composition of those codes will be accurate in arbitrary settings.

Not all errors are numerical

If Matlab tells you that you’re trying to solve a nearly-singular linear system, there may be some numerical issue in your problem formulation that you should worry about. Or you might have made an indexing mistake when you set up the matrix in the linear system. Think a little bit before ruling out the possibility that your bug might have nothing to do with numerical subtleties and everything to do with good old-fashioned off-by-one mistakes.