Bioinformatics Zen

As I write my first paper, my biggest worry is that my results are wrong. In particular, that my code, which I think does one thing, has a bug and does something different. This, in turn, produces inaccurate results and leads me to incorrect conclusions. I then produce a paper where the story I am telling is wrong.

For example, in a recent story, five papers were retracted because an incorrect sign change lead to the wrong protein structure prediction. This is very unfortunate, and I think that every bioinformatician could empathise with this: how often can you be 100% sure that your scripts and programs are doing what you expect?

To contrast this with the wet lab, one of the the most important criteria is to be very meticulous about positive and negative controls. This makes me think, could I use positive and negative controls in my computational work to test my experiments? Whether this is testing what my program is doing, or whether the results I have produced are true; without this level of certainty, how can I be sure about my conclusions?

Good programming

This may sound rather like I am preaching, but I hope that by following good programming practices I can prevent many errors that could result from simple mistakes.

Behaviour, or unit driven code testing

If you’re unfamiliar, unit/behaviour testing involves writing short snippets of code to test your program function. The best way to do this, is to write all the possible tests before you begin writing your program, then as you are coding you can keep running the tests. This makes sure you the program is always doing what you expect.

I think regular testing is one of the most critical parts of code production, if not the most critical, and if you take anything away from reading this article, consider implementing tests in your work routine.

There are plenty of libraries to support code testing, so the process isn’t too arduous. My personal favourite is test/spec, which I use for behaviour driven code testing in Ruby, but there are libraries for every language.

For example, as in the laboratory, you could write tests to make sure that your program produces positive and negative results when you expect. A good habit to get into, is to keep adding more tests where you think of other possible errors that could be introduced. Then, when you start actually doing the analysis, you can be sure that your code is doing everything that you want, and nothing else.

Reusing code, and making it open source

Using existing libraries is a great habit to get into. Writing your own solution to a problem means more code that has to be tested, and debugged. On the other hand, if code has already been written, and in my experience 70% of the time it has, then you can just slot this in, and move on to the next step, saving you time.

If you have a problem where there isn’t already have an existing library for, then package up your solution and release it on a site like SourceForge.net or RubyForge.org. You’ll be helping other people in their work, and also if your problem is an important one, you will find that people contribute and improve the code you’ve written, which will directly feedback into your own work.

A bonus about creating packages, is that it keeps your code modular. For example, do you have a piece of code that you regularly use in your scripts? Something you keep pasting in when you need it? If you found an error in this code, you would have to go back to all the places where this code was used, and correct the problem.

On the other hand, if the code was part of a package that your script was calling, when you update the package to fix the problem, every corresponding script gets updated too. This is an important principle in software production, Don’t repeat yourself, and one I try to stick to as much as possible.

Biological understanding

Writing tests and and using open source software only takes a little extra time, so the next step is to produce results, and then interpret them.

Let negative results drive the research

Again, this is something that I think is really important. Before I start to find my important results, I will perform the simple analyses: the things I already know the answer to. Doing this will confirm that, again, all the code I’ve written is working, but also that in terms of biology I am seeing the types of results I expect. This point may seem obvious, but I am a bioinformatician, and after writing the code as an informatician, I have to put my biological hat on, and use my knowledge to verify what I am producing, the current literature predicts.

I hope, that by doing my research this way, I can creep slowly forward from the areas already understood to those less certain. This means I can be confident that each further, and more insecure, conclusion is built on the solid logical foundation of the previous result. Of course, this is easy to write in principle, and this is not possible in every situation, but I try where I can, and I believe it’s worth it.

Results match expectations

Finally, getting to the crux of my research, I hope I’ll be producing the results that will answer important questions and give me the material I need to write a paper. Here is where I may have an expectation of what the result is going to be, but a wildly different result is possible. This could indicate a breakthrough in understanding, or a mistake in my method. But hopefully, by being rigorous both in my practice of informatics and biology, I can attribute this result to a new insight, rather than worrying that there is still a bug lurking in my code.

Summary

The point I’ve tried to make here, is that it’s worth taking a little extra time to rigorous in code production. However, I always keep in mind that you can never be 100% sure that your code does what you think, so I also try to keep biological expectations in mind when I interpret results.

Related
Boscoh also wrote a similar article to this a few months ago at his blog, Trapped in the USA.