Thinking Parallel

My regular readers know that I am doing a lot of parallel programming for work. I am writing my PhD-thesis on it. I am maintaining this blog on the very topic and have even stated in the past that I love parallel programming and why. Yet, there are still a few days when I wish things would just go a little smother. And on even fewer of these days, I hate parallel programming. This post lists and explains the top five reasons why this is the case sometimes. It is also my submission for Darren Rowses (problogger) new group writing project.

Here they are:

1. It’s the platform that matters, and parallel programming platforms are not on par with their sequential counterparts. This hits me every time I search for a good parallel profiler or debugger. Or when I am looking for a particular library only to realize that it is not thread-safe and may therefore not be usable for me. I feel the pain when I am looking at an IDE and it does not understand the parallel constructs I am working with every day. Thats when I know that parallel programming still has a long way to go before mainstream adaption.
2. It’s way too easy to make mistakes (I have a Top 10 List pending for that topic alone), therefore I will not elaborate it here.
3. The programming languages that many parallel programming systems are based on are either not powerful enough (C, C++, Fortran), or their performance is suboptimal (all the scripting languages – yes, you can do parallel programming in some of them – I have even tried it in the past). If you have programmed in a scripting language before, you will know what I mean (feature- and performance-wise). I don’t want to care about how many elements fit into my arrays, I have come to expect that the language takes care of that for me, just to name an example. I guess I am spoiled by programming for the web. *sigh*
4. Parallel Programming is still mostly a manual process – one needs to think about data- and task-distribution without much help from any tools – which makes the process error-prone and one needs a lot of experience to do a good job at it. And even with a lot of experience, some of it just boils down to trial-and-error. And I thought this was supposed to be an engineering process :sad:.
5. Compilers are buggier when writing parallel code than with sequential code – simply because fewer people write parallel code and therefore the compilers are less well tested in this regard. I have blogged about this before and unfortunately it is still true for many systems. I hope this changes soon as more people explore the realm of parallel programming.

There, now you have it – my personal hate-list about the very topic I usually love. I do realize that it is subjective. There are e.g. languages suitable for parallel programming that provide a very powerful platform – Java comes to my mind, which I am looking into right now and a lot has changed since the last time I have worked with it (especially with the concurrency features introduced in Java 1.5) – will post about this later. Whether or not its performance has improved since I last played with it, remains to be seen and is the topic of another post ;-). Anyways, feel free to add your very own reasons, what is it that bothers you most about parallel programming? I am very anxious to hear your comments!