Thinking Parallel

While not exactly related to parallel programming, I am so sick of hearing this: Moore’s Law no longer holds! Clock speeds are no longer increasing, therefore the world is going parallel! I have shortly commented on this a couple of weeks ago. You can sometimes read it in blogs or hear it in conversations. And while the second sentence is actually true, the first one is not and comes from a severe misunderstanding of what Moore actually said and meant. This article comes to the rescue of all the poor victims to this misunderstanding :-).

So let’s approach this the wrong way around and see what many people think Moore’s Law says:

The clock speed of chips doubles every 24 month.

This misunderstanding is so common, because that’s how the chips actually did behave for quite some time in the past. Those times are gone now, as heat problems have forced the chip makers to abandon the MHz-race. The sentence in itself builds on a severe misunderstanding – the one that clock speed equals computing power. As could be observed for the Pentium 4 architecture, this is not the case. Computing power is of course not merely dependent on how many clock cycles you manage to do each second, but also on how much work can be done in each clock cycle. Which heavily depends on your architecture – but since GHz are so much easier to work with for the marketing department, this part of the equation used to be dropped.

Anyways, this is the past and history. Many people know this by now. Another commonly heard version of Moore’s Law is this:

The computing power of chips doubles every 24 month.

Seems logical, right? Computing power is what counts and Mister Moore must have been talking about that. While this version is better than the first (for the reasons stated above), it is still not accurate. Does the computing power double when you have two cores on a chip? Depends on your problem, right? And for most problems the answer is unfortunately: no (read up on Amdahl’s Law if you don’t know from the top of your head why that’s the case). Therefore this interpretation of Moore’s Law does not hold up in court anymore as well.

Which needs not concern us, really, because it is not what Moore said, either :-). I am not going to hold you off (or bore you 🙂 ) any longer, but instead tell you what Moore’s Law really is about:

The density of transistors on chips doubles every 24 month.

Actually, thats the short version and easy to comprehend version, if you want to know what he has really said, you need to look here:

Reduced cost is one of the big attractions of integrated electronics, and the cost advantage continues to increase as the technology evolves toward the production of larger and larger circuit functions on a single semiconductor substrate. For simple circuits, the cost per component is nearly inversely proportional to the number of components, the result of the equivalent piece of semiconductor in the equivalent package containing more components. But as components are added, decreased yields more than compensate for the increased complexity, tending to raise the cost per component. Thus there is a minimum cost at any given time in the evolution of the technology. At present, it is reached when 50 components are used per circuit. But the minimum is rising rapidly while the entire cost curve is falling (see graph below). If we look ahead five years, a plot of costs suggests that the minimum cost per component might be expected in circuits with about 1,000 components per circuit (providing such circuit functions can be produced in moderate quantities.) In 1970, the manufacturing cost per component can be expected to be only a tenth of the present cost.

The complexity for minimum component costs has increased at a rate of roughly a factor of two per year (see graph on next page). Certainly over the short term this rate can be expected to continue, if not to increase. Over the longer term, the rate of increase is a bit more uncertain, although there is no reason to believe it will not remain nearly constant for at least 10 years. That means by 1975, the number of components per integrated circuit for minimum cost will be 65,000.

But since this is more difficult to work with, I am going to stick to the easy version above, which basically means the same thing. And it is still true today. All that has changed is the way these massive amounts of transistors are used. Now they are being invested into more cores on a single chip.

I have been told by people working for the chip makers that they don’t believe this law will last forever, but that they expect it to be true for a couple more years. And who knows what kinds of chips we will be having in a couple of years – think quantum computing or stuff like that.

In the end I am going to ask a little favor from you: the next time you hear or read somewhere that Moore’s Law is dying, please leave a comment with a link to this article. Or to the Wikipedia one. Or this FAQ on news.com. Or just kindly explain what I have just told you here, so we can finally get rid of this misconception. Thank You!