Wow this is brilliant!

I do remember you giving Swathi/me a link to the book you were working on (perfbook.git IIRC) when she wanted to use RCU for DNS related work! This is the same book, right ?!

Holy crap, this is great! Just skimming it now. I learned a lot from reading the linux kernel docs, but this book is way better. Hopefully more people take a crack at this stuff.

Wow... This is great. I guess this means you don't have any concrete plans for physical publication, right?

Great work Paul, and everyone else who put time into this wonderful book! Very much appreciated.

Paul,

I'm working on a flow-based programming system in which cooperating components are the "IC/Integrated Circuit" of it's parallel programming model. The components are modeled as cooperating co-routines/green threads/fibers which read and write to each others I/O channels.

In the simplest implementation there are no actual contention, locking or critical code sections because each component reads from it's inputs, performs some computation and writes to it's outputs. After the component is done it yields the CPU to the scheduler which transfers control to the next component.

In actual practice, the cooperative components are distributed between amongst 'n' available threads, with one thread per CPU core. Data transfer between components running concurrently on separate threads is is passed through non-locking, cache-coherent, FIFOs.

This system has the advantages of simplicity and re-usability of programming (much like UNIX pipes) with relative freedom from cache stalls and the thread rescheduling that happens when a locked critical section is encountered by a thread.

My system is, of course, nothing new. Systems like LINDA started flow-based programming a long time ago.

I was wondering if you were going to be covering flow-based systems within your analysis (Obvious MPI approaches this using message passing).

thx

eris

I also would like to see a section on Flow-Based Programming (FBP) in your book, or at least a link to my web site - http://www.jpaulmorrison.com/fbp . FBP started within IBM, and was used for the batch part of a major banking project in the '70s. IBM had a couple of FBP projects, but eventually dropped the last one around 1992. Since then, however, it has now spread world-wide, and IMHO a book on parallel programming is not really complete without a description of FBP. BTW the 2nd edition of my book on FBP came out in May of last year. Feel free to contact me if you need more info.
Regards, Paul Morrison

Thank you very much for this great book!

I'm reading the book now, and I feel the need to state that you rather thoroughly misapply Moore's law on page 15. Moore's Law has exactly nothing to do with either MIPS or clock speed except in the most roundabout way possible. It merely states that:

Transistor count per unit area per unit money will approximately double every 18 months.

Thus, your chart in no way substantiates the statement that Moore's Law has ceased to provide benefit. Pure MIPS might have been slightly more useful than the mixed MIPS/clock rate chart you present, seeing as the Core line of CPUs has lower clock speed than the Pentium 4 series almost across the board, but is far more powerful. Still, Moore's law is the very thing that has made multicore possible - aside from dedicating more transistors to a single core to make it faster, chipmakers have enough to dedicate them to an entire second, fourth, or sixth core.

Yes, that would help a great deal. On a different note, how would you prefer to be given spelling and grammar corrections? For instance, in the preface you say "You [sic] mission, if you choose to accept..." Not only should the 'You' be 'Your', but that idiom is typically phrased as "Your mission, should you choose to accept it...". Also, on page 14 you typo'd Steven Hawking's last name, putting a superfluous 's' at the end.

Any chance of also outputting an HTML version?

I'm not a fan of PDF; it's (generally) not reflowable.

It's now on my list of things to do…

A quick look brings up pandoc, which can convert a subset of LaTeX to HTML as well as ePub (useful for all those folks with electronic readers, e.g. Nook, iPad).

I created a HTML version by naively executing latex2html. It probably has some missing parts and it gave a number of warnings. I then used calibre to generate epub and mobi formats. I will read the mobi format on my Kindle.

I don't have the time to check that the formatting works, but here's the result for anyone else as impatient as I am :-)

http://www.bone.id.au/~paul/perfbook.epub
http://www.bone.id.au/~paul/perfbook.mobi

Thanks.

> The pthread_mutex_lock() primitive “acquires”
the specified lock, and the pthread_mutex_lock()
“releases” the specified lock.

lacks a "un" in the second pthread_mutex_lock()

Regards,
Etienne.

Someone beat you to the missing "un", but you are quite right that the footnote looks a bit like an exponent. I have moved it to the end of the sentence.

Thanks for putting this together!
Let me know if there's a way to donate $ for your efforts, as a thank you. =)

Thanks! This looks very interesting.

One typo -- on page 154, item 5 in the list should be the consumer picking up the producer's timestamp, yes?

Is this intended to be an introduction to parallel programming as a concept (using POSIX as an illustration), or more of an introduction to parallel programming as a practical endeavour?

If the former, there's really not a whole lot that needs to be added to the book. It's very clearly written and covers most of what anyone would need to understand parallelism. The only chapter I don't see is one on interconnects. This matters because interconnects define what will (and will not) work in the way of techniques. It's not that the interconnects are directly important, but they do place limits on those things that are.

On the other hand, if it's a book on practical parallel programming, you'd need a section covering the ideas in communication. "Conflicting Visions of the Future" is excellent as-is and might well be the correct place to discuss things like RDMA, but it doesn't feel right for discussing message passing versus networked inter-process communication.

I have absolutely no idea if you'd want to cover instruction-level parallelism (as per Silk and Silk++, but also an area a lot of early parallel research looked into). It's such a totally different beast than regular parallelism.

Bottom line, great book and I'd love to see if there's anything I could usefully contribute, but to avoid wasting your time or anyone else's, I'd want to know what would be interesting to you as the author-in-chief and editor-in-chief.

In terms of interconnects, I'm thinking busses (in the case of multicores and SMP) and ethernet (for budget message-passing clusters, MOSIX and Kerrighed users and remote calls - including RPC, CORBA and Linux' TIPC). I wish that I could include fast serial links, but the Transputer died in the 1990s and Intel's iWarp died soon after. Those are more Visions of the Past, which I consider a shame.

In other words, stuff that makes a system almost transparently scalable. Explicit high-level stuff like MPI-2 and Bulk Synchronous Processing get ugly, there's masses of tedious detail, and frankly most of the important stuff (avoiding deadlocks, avoiding simultaneous writes, etc) is stuff you already cover. Some aspects of parallelism are universal.

In the case of ethernet, for example, there's the question of how to pretend to have shared memory. (Distributed Shared Memory schemes tend to be rare and are often badly written.) There's also the question of whether you can use process migration systems like MOSIX or Kerrighed to do MIMD-style parallel processing more effectively than you could on a multi-core or SMP computer.

Also on Ethernet, you can very easily pass around a lot of data to a lot of machines simultaneously, using Scalable Reliable Multicast. Which, incidentally, MPI does not use. MPI rotates round the list of destinations in a collective call and sends messages on a sequential basis. Implementations of SRM and NORM (NACK-Oriented Reliable Multicast) are widely available for most platforms, Windows and Linux included, so it's vendor-independent. They are not, however, significantly used in the industry, although one would think that MISD-style parallelism would find the mechanism to be extremely valuable.

For many busses, the time it takes to switch a lane from one direction or target to another is absurdly high. The cost of working at such fine detail is also absurdly high, since a lot of parallel processing languages (UPC, Erlang, etc) are too high a level to let you do much tuning. The question is how to make the best use of time to get the best use out of the system. And that's a hard problem.

"What area am I expert in that a lot of people need to know or will soon need to know?" If I knew that, I'd be rich. :) I'm expert in plenty of areas, I frequently find use for that expertise, but anticipating which areas would be useful for others has always been a tough one for me.

.... here and so ... ..

When running "make", the generated DVI was sent directly to printer rather than into a .ps file. The patch at http://pastebin.com/tkA8wj6W fixed it for me.

Btw. do you know if there's a way to generate a Table of Contents in the PDF, which would be displayed in the side pane in many PDF viewers? That would be really useful for viewing such a large book.

Kudos for including a chapter on API ease of use in the book, but I find the content skeletal. What's the plan for that chapter?

Jin

Yes there should realize the reader to RSS my feed to RSS commentary, quite simply

I just added this web site to my feed reader, great stuff. Can not get enough!