As stated, I’m currently working in academia and the salary in academia is lower than the private/industry - after all the motivation is discovery and not money. If you’re just starting (0-2yrs of experience) it’s unlikely you’ll get paid $70K (I’m not sure which job pays that high without even an undergrad degree). I’ll be also surprised to find any internship in Bioinformatics that pays >$40K. Case in point, most post docs (in academia) get paid around that amount and it’s unlikely an intern or a temp would get paid high. But again, this is just for Bioinformatics, if you’re looking for a job as a software engineer in Bioinformatics things will be different…

Mercurial is an excellent piece of software, no doubt, but in this case BioRuby really needs github since there’s so many different directions people want to take it. Git and github could really inject some life into it.

I’m “meh” on Lighthouse, but it would certainly be better than the present situation. svn isn’t even worth considering at this point - too little too late.

Video introductions to git:
http://www.youtube.com/watch?v=4XpnKHJAok8
http://video.google.com/videoplay?docid=-3999952944619245780

I will email the BioRuby mailing list to make the suggestion.

@ Charles
Thanks for the discussion of forking. I’ll be the first to admit I don’t completely understand git’s capabilities and but there are some interesting features that look like exploring in more detail. For example git rebase –interactive is looks interesting

@Jan
I saw your post on BioRuby participation and agree that widening participation can only benefit. In my case I want to contribute more, it’s just that it is difficult for me to find the time, think the more simpler it is to contribute the better and the more likely people will contribute

@Sebastian
Yes, Mercurial is also worth looking at. I wrote about git after seeing that Rails and many other Ruby based projects were moving over. That’s not to say that Mercurial wouldn’t be worth a try too, the more options the better.

During the last hackathon in Japan, we decided to - over time - move bioruby from CVS to SVN together with the move to open-bio and when Toshiaki would have some time to do that. Purely by coincidence, I mailed him not more than an hour ago about having a look at git instead of SVN. My reasons being exactly the ones pinpointed by you two here. It’d be helpful if you (both?) could start a discussing on the bioruby mailing list about that. I’ve posted a while back on the state of bioruby on my own blog arguing that it would gain immensely from a more active community. I believe git can make that work. Toshiaki could still maintain the “main” repository that everyone trusts and can pull from, but people can trust other developers as well who they can pull directly from.

I read pretty much all the posts, and probably two people posted the actual figures of how much a Bioinformatician could earn. Most shocking of all was the last post from “Ske”. Even though you have a MS in Bioinformatics you’d earn between $40K - $50K? That I reckon is really under paid. An intern completing his CPT/OPT in an IT industry could earn $40K/yr easily…correct me if I’m wrong. You don’t even need an undergraduate degree to be earning close to $70k, if you’re good at programming. So, why is the job market so cheap for Bioinformaticians?

I wonder if we could contact the bioruby people, and see if they would be amenable to a fork of BioRuby into github with the understanding that the release versions still go through CVS. It seems like it would open up the playing field for more patches, and then at a later date they could determine if they wanted to include them, but it would not preclude others from using those patches in the meantime.

re: transformation. As a side-note, if multiplying subsets of U and V, you should also multiply by the equivalent elements of D (the eigenvalues).

Nice work. I’ll have to keep this in mind as I’m interfacing a lot with external databases. As a matter of fact, I’ll have to think about this to incorporate in the Ensembl API: make it possible to use a local sqlite3 store or something.

BTW: We’re at the moment rewriting part of bioruby so that Bio::EMBL, Bio::GenBank and colleagues actually create Bio::Sequence objects (which makes sense, because EMBL and GenBank are nothing more than sequence _formats_). That’s all still in an SVN branch though and has not been merged into trunk.

As for R code for NMF, I haven’t found anything either, but did see this exchange on R-help:

http://www.mail-archive.com/r-help@stat.math.ethz.ch/msg88593.html

BTW, the original “normalize” line should work fine if you prepend the line “library(som)”.

my $seqio = Bio::SeqIO->new(’-fh’ => IO::String->new(get($url)), ‘-format’ => ‘fasta’);

One thing to watch out for: raw/plain text from URLs is sometimes wrapped in HTML ‘pre’ tags.

I’m not sure if you experience this, but in my experience, scripts intended as “one-offs” have a habit of staying around. Cutting and pasting between them creates crufty, brittle code. Each successive time I touch that “one-off” I make sure the code becomes more tidy when I leave it than when I open it. This way, it incrementally develops into a full-fledged module or program appropriately.

And, of course, if I plan to publish the result, making sure that script has unit/functional tests is a Really Good Idea.

I am looking for information regarding the biological meaning behind graph trends. Whenever I get a set of results, I am not sure which trend to choose to describe it. I think that the common idea of finding the best matching trend (with the highest r square value) might be a mistake, because there has to be a difference in the meaning of, let’s say, exponential trend vs. 4th degree polynomial trend. I would like to receive a link for information that might help me clarify this issue better.

Thank you

On “What language should I learn?”: Learning a very high level language like Python encourages learning to program well. As a multi-paradigm programming language, Python makes a good entry point for the beginning programmer. Python philosophy centers on clarity and readability in the code, as well, and the language makes writing obfuscated code difficult, whereas other languages posit a challenge in writing code clearly.

For these reasons, Prof. Greg Wilson selected Python as the programming language of instruction for Software Carpentry (http://swc.scipy.org/). On a related note, SWC does not teach Python, it teaches scientists and engineers good software development techniques. The meta-programming practices covered in its material provide more benefit for one’s future than learning any programming language. I would urge one to learn and exercise good habits like using revision control, debugging with symbolic debuggers instead of print statements, and implementing unit tests. These will carry you furthest.

You’re right about the Master’s as requirement. I’m on that. Doing a Masters and wondering if I should a PhD. And as Mike pointed, the thesis of the master will be a good tests set of what a PhD may represent.
And about the programming, I totally agree. It’s not a matter of the language but of the algorithmics, the how to design the solution of a problem. Then to start with, I’d recommend C, pure C. It’s where you learn how to manage your memory because you’re really aware of that all the time. That will save you troubles when using the easy going Perl and others. Then for scripting, now that I’m getting into Python, I like it for thr OO programming approach. But sometimes is so readable that it’s too opaque. For easy text parsing and simple sequence handling I’d totally be into Perl.

By the way! I’m actually looking for a lab -abroad, I’m from Barcelona- to do the project and I am in this situation of having to contact. But more of the very selective use of the contacting procedure. I’ll do it once, I’ll play it at one card :).

Cheers.

In bioinformatics, the job market is not as large and you really need to have a strong research background ie publications. So, make sure you publish or else your PhD is worthless.

Who cares about the languages so much. While they are important, I think the publications is more important. Learning the languages is easy.

Regarding programming languages, Java and Perl are good but instead of learning programming languages, instead it is better to learn how to program in general. I.e. a bioinformatics student should learn object-oriented programming, functional programming, understand when to use scripting languages like Perl/Python and when not to, etc. This is far better than learning individual programming languages. Say if you know about imperative programming, and know a few real programming languages like C++ and Java and have a proper background in c.s., you can pick up Python/Perl in no time. Or any other programming language your task requires. That’s far better for your career than saying “sorry I can’t do the project because I don’t to Prolog, I only do Java & Perl”.

Another way to get a ticket is to work on building social networks. This may be possible by going to regional conferences and taking advantage of social events, or poster sessions. Some universities will even pay for students to go to meetings to present. When networking the point is not to “get the job”, but to make contacts who might help you get your foot in the door. It would probably be worthwhile finding out if your campus has coaching for such things — might be in the business school.

I take your point about unsolicited telephone calls, senior researchers are very busy - emails are a distraction and telephone calls are worse. I see how it is rather irresponsible for me to advise prospective candidates to start cold calling everybody.

However I would like to say, from my perspective as someone looking for a PhD not too many years ago, that it is quite difficult. As you say, many people spam researchers, so it’s easy to get lost in this deluge. The spam is of course annoying for people when their inbox is filled up, but also rather frustrating for students who are genuinely interested in a particular research project but they aren’t getting any response.

I’ve updated the post, to try and be more balanced.

First of all I have to agree with the comment about spam. As someone who has just spent many hours wading through CVs for a number of posts it’s very easy to tell when a generic cover email/letter has been used. This is not a good strategy. Enquiries to anyone, whether for PhD or job should be personalised.

Secondly, email approaches will often get filed to /dev/null. I know it’s hard if you’re applying to somewhere in a foreign country but the slew of email requests for projects, placements, jobs that I get have only one place to go, and that is the bin. If there’s a job there, or a studentship it WILL be advertised - use the appropriate channels to find these and focus on tailoring your covering letter and CV to them.

However I would like to take offence at ‘give them a call”! Unsolicited emails are one thing, unsolicited phone calls something completely different. Please make a traditional approach (letter, email) first and politely request to place a call *at the PI’s convenience* if you think there are questions you feel would be answered (or you feel you can make a stronger case) over the phone.

As for furthering this genre, I think I might. This is the most popular post I’ve ever had. Weapons and Armour lists are good idea. As are the levels for character progression. Any other suggestions are welcome too.

I am definitely a LV with the ‘blogger’ trait!

and you better include it in Bio::Blogs

In other words, I could not stress enough how decoupling parsing into a more easily used data structure helps.

Compared to a straight-up load of a CSV or fixed-width file, XML is really quite sluggish. If particular algorithms could benefit from some re-arranging of the data from the original source, not only does this encourage the building of more flexible tools, but expedites repeated executions with tweakable probabilistic algorithms (read: machine learning).

Bioinformatics Zen Blog is very useful for me. so I translate this blog’s article to Korean(sure i’m Korean ^^;;). And don’t want to, I will delete translated article. thank you.

translate article here http://socmaster.homelinux.org/~hongiiv/299

Thanks for finding this bug. This script worked when I wrote it a few months back. I can’t find the normalize function now, which is really weird. I was in Spain when I wrote this, but doesn’t really explain much.

You can get a similar row-normalise functionality by transposing the data, normalising the data by columns then transposing back. This is quite awkward but is the first solution that came to mind.

crabs.x <- as.matrix(t(scale(t(crabs[,4:8]))))

> crabs.x <- as.matrix(normalize(crabs[,4:8]))
Error in as.matrix(normalize(crabs[, 4:8])) :
could not find function “normalize”

any hints? Are you assuming a standard package I don’t have?

I’ve been playing with it for some of my projects. Java based GUI. Open source. Tried is since my previous post above and thought it might be something to consider also.

Here’s a link to the product overview screencast. http://www.pentaho.com/products/demos/PDI_overview/PDI_overview.html

@Chris
I have thought about what you wrote about journals requiring authors to submit tests as part of papers. I don’t think it is overkill, wetlab biologists have to be very rigorous about what controls they use, so why shouldn’t bioinformatician’s have to do the sane? Most modern testing software produces readable results, that for example a reviewer could quickly cast their eye over.

@Paradoxus
I agree that going through your code can be repetitive and boring, so in no way do I advocate this. In fact, I think in programming if something is hard work there is usually already an existing solution to make it easier. For example unit testing is meant to run quickly and easily every time you want to make sure that your program is working. This is usually better tthat adding print statements through out your code. Worth a look perhaps?

@Andy
I agree that you should be prepared for unexpected results, and I hope the point I was trying to make was that if you’re sure that your code has no bugs in, then you can be more confident in the results you have produced. Especially in the case where the results challenge previous research. I’m not advocating routines that conform, rather that they do what you have written them to do.

@Pedro
I am exactly the same, most of the time I write posts of what I think I should be doing rather than what I am actually doing.

1) Testing is good. In fact, testing is essential. But - particularly in bioinformatics - it is equally important that you understand why your test inputs should produce your test outputs. Writing code to produce predictable results is required where the process is well understood and validated, but if there is the potential for uncertainty every concept needs to be challenged. One should never be afraid to come up with a result that differs from the expected. If you read the boscoh post fully, you can see that it took a long time for the results to be questioned because no-one believed in their results enough to challenge the previous ones. If we were all coding routines that had to conform with the conventional wisdom, we would never advance.

2) The really important thing is to be *auditable*. Version control is a non-negotiable. Accurate records of which version of the code was run on what date with what input is, likewise, something that is essential. Checksum everything. Record everything. Dare to challenge, but be prepared to be challenged is the essential maxim.

Another issue is the program you use which was written and published by someone else. If (just an imaginary scenario) a well known program is found to be faulty, what will the consequence be?

It also takes time to go through all of your code again and again. Debugging is never ending. For my analysis, first I make sure the core code (analysis/model) is correct, then I spent more time focusing on the parsing the correct format of the data.

At the moment I am using a standalone software called Memoranda (http://memoranda.sourceforge.net/) which is not a wiki. As the webpage states, it is a diary manager and a tool for scheduling personal projects. It comes with an agenda, events, tasks, notes, and resources. One thing I like is the timetracking of the software. I get notifications when a task of a project is due or an event occurs. The only downside is that it makes the impression of a beta version and doing backups of projects is rather cumbersome. But it is worth a look.

As for your post about organizing files, I found it always practical to name folders with a date first (20071121_projectname). That way they are ordered and if I am searching something and know at least somehow when it was it is easier to find. Another practise that I use is, that in each project folder I place a README. That contains short descriptions of the project.
I am working a lot on a shell with a lot of files, using several unix/linux programs (sort, uniq, egrep, awk, sed, cut, bash,…) to remodel files, produce new files, etc. One thing I do as well is after I make use of such a command line step, I normally pipe the command line into the README. This way I can see how different files evolved, when I do have a look in the README. That helped me quite often.

Best, Sebastian