The Biggest Programming Mistake I Ever Made

Well folks, as mentioned in How To Learn From Your Programming Mistakes, here it is.

The story that still makes me shudder when I think about it.

When it comes to all time, super-duper cock-ups, this is one I will never forget.

Strictly speaking I wasn’t actually trying to program anything when it happened. I was trying to…

Oh let’s just start from the beginning shall we?

The danger of embedded systems

Not long out of university, I was working on an embedded linux platform. The embedded OS, or target, was accessed via telnet from Linux desktop machines. At any one time your terminal window could be logged into your desktop, or into the embedded Linux platform (can you see where this is going?).

I was asked to help out a colleague who was having trouble getting his target up and running.

He pointed me to his machine in the lab, and then disappeared, leaving me to it.

Straight away I could see that the embedded board he was using was in a mess. Files were missing, some were corrupt, and it was basically unrecoverable.

I had several terminal windows open at the same time, and I was switching rapidly between them to check build version, firmware version, and to access my own machine remotely. I downloaded the latest build from my machine and logged into the target to reinstall it.

At least, I thought I did.

I typed:

rm -rf /

to delete everything, so I could start the install from scratch[1].

But as I pressed the return key, I realised I was using the terminal window for his desktop machine.

And I was logged in as root.

If I had to describe that moment using a movie scene, it would look like this[2]:

 

 

I cancelled the process immediately, but it was too late. I had totally destroyed his filesystem. Now he had no target and no desktop computer either.

What did I do?

After I’d finished reeling from the massive mistake I’d just made, I gathered myself together, went to find my poor victim and confessed.

And apologized.

A lot.

He very graciously took it in his stride (since there was bugger all he could do about it, I guess), and seemed happy enough once I promised I would immediately reinstall his operating system AND fix the target, as I had originally intended.

Now, to save face (and enhance my career prospects), I suppose I could have done the reinstall in secret, and feigned ignorance if he had asked where his local files had gone.

But I owned up, and it wasn’t as bad as I thought it would be.

And now I can laugh about it (just).

And of course, I have NEVER made the same mistake again.

 

[1] The command I used, for those of you that aren’t familiar with Linux, is the equivalent of selecting your hard drive and pressing the delete key.

[2] Jaws contains a classic example of the Dolly Zoom: that wonderfully abused effect where the background moves away while the actor appears to loom larger, usually accompanied by some tense string music.

 

How to Learn From Your Programming Mistakes (in 5 Easy Steps)

Once upon a time, I made a very big mistake on a project at work. It was astonishingly stupid of me.

I’ll share that wonderful event with you tomorrow (see how I’m craftily tempting you to come back and visit me again?). In the meantime, here’s my take on how you can learn from those moments you’d rather forget…

1. Acknowledge what you did wrong

Even in this day and age of tools that record everything, automated comments, log files and digital history, there are still programmers out there who won’t admit when they have made a mistake. This varies from the vague ‘I don’t know how that happened,’ to the more outrageous ‘It wasn’t me!’.

Programmers, more than any other professionals I know, are very protective of their work and hate to look like they don’t know what they are doing. Compounding this is the culture of schadenfreude in finding fault with peers, and the retelling of major errors over and over again, especially on team events and nights out: “hey dude, at least I’ve never checked in code with half an email to my wife pasted in the comment at the top, ha ha ha!”

These stories, that provide so much amusement to everyone listening (except the protagonist), are popular because they make us all feel that we aren’t the only ones that make mistakes. And you know what? That’s actually a good thing.

No one gets it right all the time, so why try and hide your errors? As embarrassing as they are, covering them up and trying to forget about them is counter-productive. Not only are you trying to fool others that you never get it wrong, you are also fooling yourself that you don’t need to improve, or take more care, or understand a tool better, or learn something new.

So next time you check in a source file and find you’ve accidentally left memory leaking all over the show, stand up and be proud to admit it. In fact, get in first and tell people what you’ve done. Throw your hands in the air and shout about it. Apologise to your manager for causing 15 crashes in the overnight test suite.

Be human and accept that the only errors computers make are the ones we tell them to make.

2. Understand why it was wrong

In some cases this is easy, but in others you need to really look at why you made the mistake in the first place. If you don’t really, truly understand exactly what the error is that you made, and why you made it, you can’t be sure that you won’t ever do it again.

Was it a lack of understanding in a particular area? Maybe you don’t fully appreciate the workings of the library you are using, or your knowledge of pointers is less than perfect, or you’ve never really worked with this particular language/algorithm/operating system.

Whatever it is, understand why. It could be as simple as the fact that you were rushing and trying to do too many things at once (focus, people – attention to detail is one of your biggest assets). Or it could be that you were making modifications to code using intuition rather than knowledge. (Oops. Guilty.)

Once you know exactly why you made a mistake, you are a very, very long way ahead of the person next to you that just sweeps it under the carpet and tries to forget about it.

3. Commit to improving

Here’s the hard bit. This is the step that won’t happen instantly and that won’t happen by itself either. You HAVE to make the effort to improve in the area in which you made the mistake.

Created a mind-boggling memory leak? Read up on malloc and free (or new and delete; pick your weapon and don’t get them mixed up). Got your iterators all over the place? Go back to basics with STL and learn what you were trying to do. Deleted an afternoon’s worth of code by accident? Get the manual out, google a tutorial, or read the help files and understand the tool you are working with.

Make the commitment to NEVER make that mistake again. But if you do, go back to step 1 😉

4. Look back and laugh (eventually)

It is very important to maintain a sense of humour about your mistakes. Going over things you have done wrong can destroy your confidence and leave you sitting in a corner, unwilling to contribute, failing to suggest ideas, refusing to raise your hand, and staying out of the debate.

Never let a big programming mistake, especially a publicly humiliating one (my story tomorrow!), take away your confidence or your desire for learning. Don’t give up.

If you admit your mistake, understand it, and improve your knowledge, then you will be able to forgive yourself.

And you might even end up as the one saying, “Hey dude, do you remember that time I checked in code with half an email to my wife pasted in the comment at the top, ha ha ha!”.

5. Be gracious when someone else makes a mistake (very important)

I’m not saying you can’t laugh at someone else’s coding misfortunes, or that you have to cuddle your colleagues and tell them it’s OK to mess up the overnight test suite.

What I am saying is don’t go overboard on the humiliation. We left a voodoo doll on the desk of a colleague that messed up a release build one evening (OK, that’s actually quite horrible). BUT we were nice about it. We rallied around and talked about the disasters we had caused, we supported his mistake by talking about our own.

Remember that you’re all in it together. Your team, however diverse and intelligent/incompetent they are, will ALL benefit if everyone approaches mistakes with the right mindset.

Admit.

Understand.

Learn.

Laugh.

Join in.

Easy, eh?