You are viewing paulmck

And it used to be so simple...

buggytux
It is something I have done many times under many different operating systems. You write something into the startup scripts, take the system down, then regain control at startup. So I planned to take this approach for my KVM-based rcutorture setup: Mount the image file, deposit a test script in the startup scripts, unmount the image file, start qemu, then analyze the results (either deposited in the filesystem or dumped to the console). What could be easier?

So I manually mounted the file I use as the qemu disk image, my plan being to manually create simple tests as a first step towards automation.

Hmmm... Everything looks very strange and unfamiliar. Ah, yes, this is upstart. No problem, the Internet is but a mouse-click away, right? But wait, the documentation says that upstart and the more-familiar (to old guys like myself, anyway) sysvinit can co-exist on the same system. And what about this new systemd thing that I keep hearing about?

So my tests are going to have to inspect the filesystem and figure out which of these three things is installed. And maybe someone installed one of them, then switched to the other, leaving traces of their earlier choice lying around. How is my script supposed to figure all that out? Of course, if it was just me running the tests, I could simply make sure that a pre-selected boot-up mechanism was installed. But that is silly — anyone anywhere should be able to test RCU. So I need to be able to deal with any of the three boot-up mechanisms. But wait, it is far worse than that: What is my script supposed to do when someone else comes up with a fourth, fifth, or sixth boot-up mechanism? Ouch!!!

It is clearly time to just say “no”! After all, my main test (rcutorture) is coded in the kernel, so it is clearly going to be far easier to translate my driver scripts to Linux kernel code than to code up a script that can outwit all present and future perpetrators of bootup mechanisms. In-kernel coding will allow me to fully control the test using kernel boot parameters, and thus happily ignore which of sysvinit, upstart, systemd, or whatever is installed. Some times you just have to get a horse!

Tags:

The -rcu tree is back on kernel.org

RCU
Thanks to the key-signing last week, -rcu is back on kernel.org, though at a slightly different address: git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git

Tags:

SequentialCaveman
The git archive for “Is Parallel Programming Hard, And, If So, What Can You Do About It?” is once again available from git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/perfbook.git. A big “thank you” to everyone who worked so hard to get kernel.org back!!!

In addition, Lu Yang, Xie Baoyou, and Yu Chen are working on another translation effort, with much of the book already translated to Chinese. A download is said to be available here, though my Chinese is not up to the task of figuring out exactly how to download it, even with the help of Google Translate.

I am hoping that this group and the group reported earlier will continue to make great progress on this!
SequentialCaveman
Cheedoong Drung has started a Chinese translation of “Is Parallel Programming Hard, And, If So, What Can You Do About It?”. He is making good progress, and would also be happy to have others join in. So if you have good Chinese and English skills (I guess I am one for two on that one), get in touch with Cheedoong. (His contact information is on his web site).

Linux Plumbers Conference 2011

inside
Now that Linux Plumbers Conference has come to a successful conclusion and I have had a weekend to catch up on sleep, I want to offer my heartfelt thanks to the members of the program committee, whose efforts were key to this conference's success, in email-address order: Allison Randal, Jon Corbet, James Bottomley, Jesse Barnes, Jes Sorenson, Jesse Barker, Joel Becker, James Morris, Julia Lawall, Kate Stewart, John Linville, Matthew Garret, Rafael Wysocki, Thomas Gleixner, and Tom Herbert. And of course to the speakers and attendees as well!

Almost all of the presentations are now available on the session pages linked from the schedule.

On the Development Tools Microconference: Julia Lawall noted that there were many different types of tools presented. As a result, she posted a survey to gather ideas for further directions on development tools: Please take a few minutes to fill out the survey.

On the Bufferbloat Microconference: Jim Gettys was happy to see that most people now agree that there is a serious buffer-management problem (as opposed to merely a specific active queue management (AQM) problem), he remains concerned that to few people understand how serious the problem really is. ;—)

On the Scaling Microconference: Good presentations by Mathieu Desnoyers on user-space scalability and by Andi Kleen on kernel scalability, but only a very few members of the audience were actively working on user-space scalability, despite a number of prominent user-space applications having single global locks that sharply limit scalability.

On the Tracing Microconference: There was some good discussion of coordinated tracing in virtualized environments between host and guest OSes, which should be helpful to the increasing numbers of people running such environments.

And much much more...

Parallel Programming: Temporary Home

inside
Due to some unscheduled downtime, perfbook's usual home is unavailable. I have therefore posted a recent PDF of perfbook here.

Parallel Programming: August 2011 Update

SequentialCaveman
This release of the parallel programming book features a new data-ownership chapter, the start of a new validation chapter, and lots of fixes, primarily from Elie De Brauwer.

I expect to put out the next release in December. I will continue on validation, and hopefully get a few other things accomplished.

As always, git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/perfbook.git will be updated in real time.

Stupid RCU Tricks: Bug Hunt Number 2

inside
Lai Jiangshan spotted another bug in the solution posted for bug hunt number 1. Here is the fixed (but still buggy) version:

  1 struct foo {
  2   struct list_head list;
  3   int key;
  4   int data;
  5 };
  6
  7 LIST_HEAD(mylist);
  8 DEFINE_SPINLOCK(mylock);
  9 struct foo *cache;
 10
 11 int search(int key, int *data)
 12 {
 13   struct foo *p;
 14
 15   rcu_read_lock();
 16   p = rcu_dereference(cache);
 17   if (p != NULL && p->key == key)
 18     goto found;
 19   list_for_each_entry_rcu(p, &mylist, list)
 20     if (p->key == key) {
 21       rcu_assign_pointer(cache, p);
 22       goto found;
 23     }
 24   rcu_read_unlock();
 25   return -ENOENT;
 26 found:
 27   *data = p->data;
 28   rcu_read_unlock();
 29   return 0;
 30 }
 31
 32 int insert(int key, int data)
 33 {
 34   struct foo *p = kmalloc(sizeof(*p), GFP_KERNEL);
 35
 36   if (p == NULL)
 37     return -ENOMEM;
 38   p->key = key;
 39   p->data = data;
 40   spin_lock(&mylock);
 41   list_add_rcu(&p->list, &mylist);
 42   spin_unlock(&mylock);
 43 }
 44
 45 int delete(int key)
 46 {
 47   struct foo *p;
 48
 49   spin_lock(&mylock);
 50   list_for_each_entry(p, &mylist, list)
 51     if (p->key == key) {
 52       list_del_rcu(&p->list);
 53       RCU_INIT_POINTER(cache, NULL);
 54       spin_unlock(&mylock);
 55       synchronize_rcu();
 56       free(p);
 57       return 0;
 58     }
 59   spin_unlock(&mylock);
 60   return -ENOENT;
 61 }


Can you spot the additional bug?

Stupid RCU Tricks: RCU-Protected Deletion

RCU
Consider the following code fragment:

 

 

  1 rcu_read_lock();
  2 p = rcu_dereference(gp);
  3 if (must_delete(p)) {
  4   spin_lock(&my_lock);
  5   if (must_delete(p)) {
  6     kfree_rcu(p);
  7     rcu_assign_pointer(gp, q);
  8     already_deleted(p);
  9   }
 10   spin_unlock(&my_lock);
 11 } else {
 12   do_something_with(p);
 13 }
 14 rcu_read_unlock();

Note that the whole fragment is enclosed in an RCU read-side critical section. Line 2 picks up a local pointer to the global RCU-protected pointer gp, which we assume is never NULL. Line 3 checks to see if it is necessary to delete the structure referenced by gp, and, if so, lines 4-10 do the deleting. Otherwise, line 12 does something with the just-accessed structure.

If the structure must be deleted, we first acquire a spinlock on line 4, and then line 5 re-checks whether it is still necessary to delete the structure, but this time under the lock. If the structure still needs to be deleted, then line 6 frees it (but only after an RCU grace period has elapsed) and line 7 makes gp point to some replacement structure q. Clearly this replacement structure must have been allocated somehow, but the details of exactly how that happens are not important to this example. Next, line 8 marks the old structure as having already been deleted, which resolves races when multiple CPUs concurrently notice that the structure needs deleting, and is also the reason that line 5 rechecks under the lock. Finally, line 10 releases the spinlock.

This code fragment is a bit unconventional: Normally lines 6 and 7 would be interchanged. But the RCU read-side critical section will prevent the structure from being freed until after the global pointer is updated, right? Why or why not?

Puzzle: How many unlock paths?

BookAndGlasses
I was recently at dinner with some Linux kernel hackers who were showing off their smartphones. These phones have an interesting unlock mechanism: the phone displays an 3-by-3 array of circles, and the user traces out a path among the circles. If the path is correct, the phone unlocks.

The paths are not arbitrary. From what I could see, here are the rules governing them:

  1. Paths are composed of a series of straight line segments.
  2. Each line segment connects a pair of circles.
  3. If the preceding line segment arrived at a given circle, then the next line segment must leave that same circle.
  4. A given circle may be visited only once.
  5. A line segment may pass over a given circle without visiting it only if that circle has already been visited. (Attempting to pass over a not-yet-visited circle will instead visit that circle.)

How many unique paths are there?

Tags: