Log in

No account? Create an account

Previous Entry | Next Entry

Parallel Programming: Announcement

As some of you know, I have been working on a book on parallel programming. My thought had been to complete it, then announce it. But I finally realized that it really never will be complete, at least not as long as people keep coming up with new parallel-programming ideas, which I most definitely hope will continue for a very long time.

So, here it is!


Page 1 of 2
<<[1] [2] >>
Gautham Shenoy
Jan. 3rd, 2011 09:37 am (UTC)
Wow this is brilliant!

I do remember you giving Swathi/me a link to the book you were working on (perfbook.git IIRC) when she wanted to use RCU for DNS related work! This is the same book, right ?!
Jan. 4th, 2011 06:53 am (UTC)
Yep, same book, later version. ;-)
Jan. 3rd, 2011 11:24 am (UTC)
wow - thanks!
Holy crap, this is great! Just skimming it now. I learned a lot from reading the linux kernel docs, but this book is way better. Hopefully more people take a crack at this stuff.
Jan. 3rd, 2011 11:48 am (UTC)
Wow... This is great. I guess this means you don't have any concrete plans for physical publication, right?
Jan. 3rd, 2011 11:54 am (UTC)
Because nobody has ever published a successful technical book and also made it available on the Internet? :)
Physical publication - (Anonymous) - Jan. 3rd, 2011 06:58 pm (UTC) - Expand
Jan. 3rd, 2011 01:44 pm (UTC)
Thanks Paul
Great work Paul, and everyone else who put time into this wonderful book! Very much appreciated.
Jan. 3rd, 2011 07:27 pm (UTC)
Flow-Based Programming

I'm working on a flow-based programming system in which cooperating components are the "IC/Integrated Circuit" of it's parallel programming model. The components are modeled as cooperating co-routines/green threads/fibers which read and write to each others I/O channels.

In the simplest implementation there are no actual contention, locking or critical code sections because each component reads from it's inputs, performs some computation and writes to it's outputs. After the component is done it yields the CPU to the scheduler which transfers control to the next component.

In actual practice, the cooperative components are distributed between amongst 'n' available threads, with one thread per CPU core. Data transfer between components running concurrently on separate threads is is passed through non-locking, cache-coherent, FIFOs.

This system has the advantages of simplicity and re-usability of programming (much like UNIX pipes) with relative freedom from cache stalls and the thread rescheduling that happens when a locked critical section is encountered by a thread.

My system is, of course, nothing new. Systems like LINDA started flow-based programming a long time ago.

I was wondering if you were going to be covering flow-based systems within your analysis (Obvious MPI approaches this using message passing).



Jan. 4th, 2011 08:52 pm (UTC)
Re: Flow-Based Programming
Hello, Eris,

The closest thing to this approach currently in the book is pipelining, which is a trivial variant of flow-based programming. I don't have any immediate plans to go beyond this at the moment,

Were you interested in writing up a survey of the various flow-based programming efforts, with emphasis on open-source software?

Thanx, Paul
Re: Flow-Based Programming - (Anonymous) - Jan. 10th, 2011 01:53 am (UTC) - Expand
Jan. 3rd, 2011 07:33 pm (UTC)
Thank you very much for this great book!
Jan. 4th, 2011 03:36 am (UTC)
On the subject of Moore's Law
I'm reading the book now, and I feel the need to state that you rather thoroughly misapply Moore's law on page 15. Moore's Law has exactly nothing to do with either MIPS or clock speed except in the most roundabout way possible. It merely states that:

Transistor count per unit area per unit money will approximately double every 18 months.

Thus, your chart in no way substantiates the statement that Moore's Law has ceased to provide benefit. Pure MIPS might have been slightly more useful than the mixed MIPS/clock rate chart you present, seeing as the Core line of CPUs has lower clock speed than the Pentium 4 series almost across the board, but is far more powerful. Still, Moore's law is the very thing that has made multicore possible - aside from dedicating more transistors to a single core to make it faster, chipmakers have enough to dedicate them to an entire second, fourth, or sixth core.
Jan. 4th, 2011 08:55 pm (UTC)
Re: On the subject of Moore's Law
The wording on page 15 is "ceased to provide its traditional benefit" rather than "ceased to provide benefit". For a long time, the benefits of Moore's Law included frequency increases due to transistor scaling, and those traditional frequency increases are no longer happening.

That said, I can see how you might read the current wording in that way. I am considering adding a Quick Quiz raising your point. Does that seem reasonable to you?
Jan. 4th, 2011 04:08 am (UTC)
HTML version?
Any chance of also outputting an HTML version?

I'm not a fan of PDF; it's (generally) not reflowable.
Jan. 4th, 2011 08:58 pm (UTC)
Re: HTML version?
No immediate plans. That said, I would welcome a good patch that did this.

The challenges will be handling the Quick Quizzes and the other script-extracted content. It might be possible to provide scripting that fixed things up for latex2html, but I have not put much thought into this.

Again, I would welcome a good patch that did this.
Re: HTML version? - tamasrepus - Jan. 4th, 2011 09:50 pm (UTC) - Expand
Re: HTML version? - paulmck - Jan. 5th, 2011 07:09 pm (UTC) - Expand
Re: HTML version? - paulmck - Jan. 6th, 2011 06:17 am (UTC) - Expand
Jan. 4th, 2011 01:12 pm (UTC)
typo in perfbook
> The pthread_mutex_lock() primitive “acquires”
the specified lock, and the pthread_mutex_lock()
“releases” the specified lock.

lacks a "un" in the second pthread_mutex_lock()

Jan. 4th, 2011 05:17 pm (UTC)
Re: typo in perfbook
And the footnote 4) in paragraph 5.3.1 seems misplaced.

Re: typo in perfbook - paulmck - Jan. 4th, 2011 09:04 pm (UTC) - Expand
Jan. 4th, 2011 04:18 pm (UTC)
Great book
Thanks for putting this together!
Let me know if there's a way to donate $ for your efforts, as a thank you. =)
Jan. 4th, 2011 09:05 pm (UTC)
Re: Great book
Thanks to all for the props!

No need for $$$, but thank you for asking.
Jan. 5th, 2011 03:15 am (UTC)
Thanks! This looks very interesting.

One typo -- on page 154, item 5 in the list should be the consumer picking up the producer's timestamp, yes?
Jan. 5th, 2011 07:20 pm (UTC)
Good catch!!! I have committed and pushed a fix into the git archive.

Thank you!
Jan. 5th, 2011 06:27 pm (UTC)
What, specifically, is the focus of the book?
Is this intended to be an introduction to parallel programming as a concept (using POSIX as an illustration), or more of an introduction to parallel programming as a practical endeavour?

If the former, there's really not a whole lot that needs to be added to the book. It's very clearly written and covers most of what anyone would need to understand parallelism. The only chapter I don't see is one on interconnects. This matters because interconnects define what will (and will not) work in the way of techniques. It's not that the interconnects are directly important, but they do place limits on those things that are.

On the other hand, if it's a book on practical parallel programming, you'd need a section covering the ideas in communication. "Conflicting Visions of the Future" is excellent as-is and might well be the correct place to discuss things like RDMA, but it doesn't feel right for discussing message passing versus networked inter-process communication.

I have absolutely no idea if you'd want to cover instruction-level parallelism (as per Silk and Silk++, but also an area a lot of early parallel research looked into). It's such a totally different beast than regular parallelism.

Bottom line, great book and I'd love to see if there's anything I could usefully contribute, but to avoid wasting your time or anyone else's, I'd want to know what would be interesting to you as the author-in-chief and editor-in-chief.
Jan. 5th, 2011 07:34 pm (UTC)
Re: What, specifically, is the focus of the book?
An introduction to parallel programming as a practical endeavor, with emphasis on helping people understand how to adjust the overall design of their software so as to get the most parallelism benefit for the least pain, time, and trouble.

The focus at the moment is primarily on shared memory because such systems are cheap, easy to set up, and readily available. Plus because my own experience has been primarily with shared memory, and additionally because the most likely audience are people using multi-core systems.

When you talk about interconnects, are you thinking in terms of the internal interconnects in shared-memory/multicore systems, or in terms of the message-passing interconnects used in large supercomputing clusters?

"Conflicting Visions of the Future" should be for topics where there is genuine uncertainty and conflict. Possible topics include limits to hardware and software scalability, multicore-computing fads from various groups, and what sorts of parallel-debug assists are required. That said, this book is not to be a marketing vehicle for either vendors or academia.

I of course welcome contributions! One question to ask yourself is "what area am I expert in that a lot of people need to know or will soon need to know?" Thoghts?
Re: What, specifically, is the focus of the book? - (Anonymous) - Jan. 5th, 2011 10:28 pm (UTC) - Expand
Jan. 9th, 2011 06:49 pm (UTC)
Paul E. McKenney&#39;s Journal - Post a comment
.... here and so ... ..
Jan. 13th, 2011 10:21 am (UTC)
[Patch] send dvips output to file
When running "make", the generated DVI was sent directly to printer rather than into a .ps file. The patch at http://pastebin.com/tkA8wj6W fixed it for me.

Btw. do you know if there's a way to generate a Table of Contents in the PDF, which would be displayed in the side pane in many PDF viewers? That would be really useful for viewing such a large book.
Jan. 16th, 2011 12:01 am (UTC)
Re: [Patch] send dvips output to file
I added the "-o" flags to dvips, thank you! It might be possible to generate the table of contents in the PDF by switching to pdflatex, which might be helpful for other reasons.
Re: [Patch] send dvips output to file - paulmck - Feb. 8th, 2011 08:45 pm (UTC) - Expand
Feb. 8th, 2011 02:21 pm (UTC)
API ease of use
Kudos for including a chapter on API ease of use in the book, but I find the content skeletal. What's the plan for that chapter?

Feb. 8th, 2011 08:42 pm (UTC)
Re: API ease of use
The plan for that chapter, as with the other chapters, is to fill it out over time. ;-)
Page 1 of 2
<<[1] [2] >>

Latest Month

April 2019
Powered by LiveJournal.com
Designed by Tiffany Chow