Loading Documentation/IRQ-affinity.txt +28 −9 Original line number Original line Diff line number Diff line ChangeLog: Started by Ingo Molnar <mingo@redhat.com> Update by Max Krasnyansky <maxk@qualcomm.com> SMP IRQ affinity, started by Ingo Molnar <mingo@redhat.com> SMP IRQ affinity /proc/irq/IRQ#/smp_affinity specifies which target CPUs are permitted /proc/irq/IRQ#/smp_affinity specifies which target CPUs are permitted for a given IRQ source. It's a bitmask of allowed CPUs. It's not allowed for a given IRQ source. It's a bitmask of allowed CPUs. It's not allowed to turn off all CPUs, and if an IRQ controller does not support IRQ to turn off all CPUs, and if an IRQ controller does not support IRQ affinity then the value will not change from the default 0xffffffff. affinity then the value will not change from the default 0xffffffff. /proc/irq/default_smp_affinity specifies default affinity mask that applies to all non-active IRQs. Once IRQ is allocated/activated its affinity bitmask will be set to the default mask. It can then be changed as described above. Default mask is 0xffffffff. Here is an example of restricting IRQ44 (eth1) to CPU0-3 then restricting Here is an example of restricting IRQ44 (eth1) to CPU0-3 then restricting the IRQ to CPU4-7 (this is an 8-CPU SMP box): it to CPU4-7 (this is an 8-CPU SMP box): [root@moon 44]# cd /proc/irq/44 [root@moon 44]# cat smp_affinity [root@moon 44]# cat smp_affinity ffffffff ffffffff [root@moon 44]# echo 0f > smp_affinity [root@moon 44]# echo 0f > smp_affinity [root@moon 44]# cat smp_affinity [root@moon 44]# cat smp_affinity 0000000f 0000000f Loading @@ -21,17 +30,27 @@ PING hell (195.4.7.3): 56 data bytes --- hell ping statistics --- --- hell ping statistics --- 6029 packets transmitted, 6027 packets received, 0% packet loss 6029 packets transmitted, 6027 packets received, 0% packet loss round-trip min/avg/max = 0.1/0.1/0.4 ms round-trip min/avg/max = 0.1/0.1/0.4 ms [root@moon 44]# cat /proc/interrupts | grep 44: [root@moon 44]# cat /proc/interrupts | grep 'CPU\|44:' 44: 0 1785 1785 1783 1783 1 CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 1 0 IO-APIC-level eth1 44: 1068 1785 1785 1783 0 0 0 0 IO-APIC-level eth1 As can be seen from the line above IRQ44 was delivered only to the first four processors (0-3). Now lets restrict that IRQ to CPU(4-7). [root@moon 44]# echo f0 > smp_affinity [root@moon 44]# echo f0 > smp_affinity [root@moon 44]# cat smp_affinity 000000f0 [root@moon 44]# ping -f h [root@moon 44]# ping -f h PING hell (195.4.7.3): 56 data bytes PING hell (195.4.7.3): 56 data bytes .. .. --- hell ping statistics --- --- hell ping statistics --- 2779 packets transmitted, 2777 packets received, 0% packet loss 2779 packets transmitted, 2777 packets received, 0% packet loss round-trip min/avg/max = 0.1/0.5/585.4 ms round-trip min/avg/max = 0.1/0.5/585.4 ms [root@moon 44]# cat /proc/interrupts | grep 44: [root@moon 44]# cat /proc/interrupts | 'CPU\|44:' 44: 1068 1785 1785 1784 1784 1069 1070 1069 IO-APIC-level eth1 CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 [root@moon 44]# 44: 1068 1785 1785 1783 1784 1069 1070 1069 IO-APIC-level eth1 This time around IRQ44 was delivered only to the last four processors. i.e counters for the CPU0-3 did not change. Documentation/RCU/NMI-RCU.txt +3 −0 Original line number Original line Diff line number Diff line Loading @@ -93,6 +93,9 @@ Since NMI handlers disable preemption, synchronize_sched() is guaranteed not to return until all ongoing NMI handlers exit. It is therefore safe not to return until all ongoing NMI handlers exit. It is therefore safe to free up the handler's data as soon as synchronize_sched() returns. to free up the handler's data as soon as synchronize_sched() returns. Important note: for this to work, the architecture in question must invoke irq_enter() and irq_exit() on NMI entry and exit, respectively. Answer to Quick Quiz Answer to Quick Quiz Loading Documentation/RCU/RTFP.txt +108 −0 Original line number Original line Diff line number Diff line Loading @@ -52,6 +52,10 @@ of each iteration. Unfortunately, chaotic relaxation requires highly structured data, such as the matrices used in scientific programs, and structured data, such as the matrices used in scientific programs, and is thus inapplicable to most data structures in operating-system kernels. is thus inapplicable to most data structures in operating-system kernels. In 1992, Henry (now Alexia) Massalin completed a dissertation advising parallel programmers to defer processing when feasible to simplify synchronization. RCU makes extremely heavy use of this advice. In 1993, Jacobson [Jacobson93] verbally described what is perhaps the In 1993, Jacobson [Jacobson93] verbally described what is perhaps the simplest deferred-free technique: simply waiting a fixed amount of time simplest deferred-free technique: simply waiting a fixed amount of time before freeing blocks awaiting deferred free. Jacobson did not describe before freeing blocks awaiting deferred free. Jacobson did not describe Loading Loading @@ -138,6 +142,13 @@ blocking in read-side critical sections appeared [PaulEMcKenney2006c], Robert Olsson described an RCU-protected trie-hash combination Robert Olsson described an RCU-protected trie-hash combination [RobertOlsson2006a]. [RobertOlsson2006a]. 2007 saw the journal version of the award-winning RCU paper from 2006 [ThomasEHart2007a], as well as a paper demonstrating use of Promela and Spin to mechanically verify an optimization to Oleg Nesterov's QRCU [PaulEMcKenney2007QRCUspin], a design document describing preemptible RCU [PaulEMcKenney2007PreemptibleRCU], and the three-part LWN "What is RCU?" series [PaulEMcKenney2007WhatIsRCUFundamentally, PaulEMcKenney2008WhatIsRCUUsage, and PaulEMcKenney2008WhatIsRCUAPI]. Bibtex Entries Bibtex Entries Loading Loading @@ -202,6 +213,20 @@ Bibtex Entries ,Year="1991" ,Year="1991" } } @phdthesis{HMassalinPhD ,author="H. Massalin" ,title="Synthesis: An Efficient Implementation of Fundamental Operating System Services" ,school="Columbia University" ,address="New York, NY" ,year="1992" ,annotation=" Mondo optimizing compiler. Wait-free stuff. Good advice: defer work to avoid synchronization. " } @unpublished{Jacobson93 @unpublished{Jacobson93 ,author="Van Jacobson" ,author="Van Jacobson" ,title="Avoid Read-Side Locking Via Delayed Free" ,title="Avoid Read-Side Locking Via Delayed Free" Loading Loading @@ -635,3 +660,86 @@ Revised: " " } } @unpublished{PaulEMcKenney2007PreemptibleRCU ,Author="Paul E. McKenney" ,Title="The design of preemptible read-copy-update" ,month="October" ,day="8" ,year="2007" ,note="Available: \url{http://lwn.net/Articles/253651/} [Viewed October 25, 2007]" ,annotation=" LWN article describing the design of preemptible RCU. " } ######################################################################## # # "What is RCU?" LWN series. # @unpublished{PaulEMcKenney2007WhatIsRCUFundamentally ,Author="Paul E. McKenney and Jonathan Walpole" ,Title="What is {RCU}, Fundamentally?" ,month="December" ,day="17" ,year="2007" ,note="Available: \url{http://lwn.net/Articles/262464/} [Viewed December 27, 2007]" ,annotation=" Lays out the three basic components of RCU: (1) publish-subscribe, (2) wait for pre-existing readers to complete, and (2) maintain multiple versions. " } @unpublished{PaulEMcKenney2008WhatIsRCUUsage ,Author="Paul E. McKenney" ,Title="What is {RCU}? Part 2: Usage" ,month="January" ,day="4" ,year="2008" ,note="Available: \url{http://lwn.net/Articles/263130/} [Viewed January 4, 2008]" ,annotation=" Lays out six uses of RCU: 1. RCU is a Reader-Writer Lock Replacement 2. RCU is a Restricted Reference-Counting Mechanism 3. RCU is a Bulk Reference-Counting Mechanism 4. RCU is a Poor Man's Garbage Collector 5. RCU is a Way of Providing Existence Guarantees 6. RCU is a Way of Waiting for Things to Finish " } @unpublished{PaulEMcKenney2008WhatIsRCUAPI ,Author="Paul E. McKenney" ,Title="{RCU} part 3: the {RCU} {API}" ,month="January" ,day="17" ,year="2008" ,note="Available: \url{http://lwn.net/Articles/264090/} [Viewed January 10, 2008]" ,annotation=" Gives an overview of the Linux-kernel RCU API and a brief annotated RCU bibliography. " } @article{DinakarGuniguntala2008IBMSysJ ,author="D. Guniguntala and P. E. McKenney and J. Triplett and J. Walpole" ,title="The read-copy-update mechanism for supporting real-time applications on shared-memory multiprocessor systems with {Linux}" ,Year="2008" ,Month="April" ,journal="IBM Systems Journal" ,volume="47" ,number="2" ,pages="@@-@@" ,annotation=" RCU, realtime RCU, sleepable RCU, performance. " } Documentation/RCU/checklist.txt +60 −29 Original line number Original line Diff line number Diff line Loading @@ -13,10 +13,13 @@ over a rather long period of time, but improvements are always welcome! detailed performance measurements show that RCU is nonetheless detailed performance measurements show that RCU is nonetheless the right tool for the job. the right tool for the job. The other exception would be where performance is not an issue, Another exception is where performance is not an issue, and RCU and RCU provides a simpler implementation. An example of this provides a simpler implementation. An example of this situation situation is the dynamic NMI code in the Linux 2.6 kernel, is the dynamic NMI code in the Linux 2.6 kernel, at least on at least on architectures where NMIs are rare. architectures where NMIs are rare. Yet another exception is where the low real-time latency of RCU's read-side primitives is critically important. 1. Does the update code have proper mutual exclusion? 1. Does the update code have proper mutual exclusion? Loading @@ -39,9 +42,10 @@ over a rather long period of time, but improvements are always welcome! 2. Do the RCU read-side critical sections make proper use of 2. Do the RCU read-side critical sections make proper use of rcu_read_lock() and friends? These primitives are needed rcu_read_lock() and friends? These primitives are needed to suppress preemption (or bottom halves, in the case of to prevent grace periods from ending prematurely, which rcu_read_lock_bh()) in the read-side critical sections, could result in data being unceremoniously freed out from and are also an excellent aid to readability. under your read-side code, which can greatly increase the actuarial risk of your kernel. As a rough rule of thumb, any dereference of an RCU-protected As a rough rule of thumb, any dereference of an RCU-protected pointer must be covered by rcu_read_lock() or rcu_read_lock_bh() pointer must be covered by rcu_read_lock() or rcu_read_lock_bh() Loading @@ -54,15 +58,30 @@ over a rather long period of time, but improvements are always welcome! be running while updates are in progress. There are a number be running while updates are in progress. There are a number of ways to handle this concurrency, depending on the situation: of ways to handle this concurrency, depending on the situation: a. Make updates appear atomic to readers. For example, a. Use the RCU variants of the list and hlist update primitives to add, remove, and replace elements on an RCU-protected list. Alternatively, use the RCU-protected trees that have been added to the Linux kernel. This is almost always the best approach. b. Proceed as in (a) above, but also maintain per-element locks (that are acquired by both readers and writers) that guard per-element state. Of course, fields that the readers refrain from accessing can be guarded by the update-side lock. This works quite well, also. c. Make updates appear atomic to readers. For example, pointer updates to properly aligned fields will appear pointer updates to properly aligned fields will appear atomic, as will individual atomic primitives. Operations atomic, as will individual atomic primitives. Operations performed under a lock and sequences of multiple atomic performed under a lock and sequences of multiple atomic primitives will -not- appear to be atomic. primitives will -not- appear to be atomic. This is almost always the best approach. This can work, but is starting to get a bit tricky. b. Carefully order the updates and the reads so that d. Carefully order the updates and the reads so that readers see valid data at all phases of the update. readers see valid data at all phases of the update. This is often more difficult than it sounds, especially This is often more difficult than it sounds, especially given modern CPUs' tendency to reorder memory references. given modern CPUs' tendency to reorder memory references. Loading Loading @@ -123,18 +142,22 @@ over a rather long period of time, but improvements are always welcome! when publicizing a pointer to a structure that can when publicizing a pointer to a structure that can be traversed by an RCU read-side critical section. be traversed by an RCU read-side critical section. 5. If call_rcu(), or a related primitive such as call_rcu_bh(), 5. If call_rcu(), or a related primitive such as call_rcu_bh() or is used, the callback function must be written to be called call_rcu_sched(), is used, the callback function must be from softirq context. In particular, it cannot block. written to be called from softirq context. In particular, it cannot block. 6. Since synchronize_rcu() can block, it cannot be called from 6. Since synchronize_rcu() can block, it cannot be called from any sort of irq context. any sort of irq context. Ditto for synchronize_sched() and synchronize_srcu(). 7. If the updater uses call_rcu(), then the corresponding readers 7. If the updater uses call_rcu(), then the corresponding readers must use rcu_read_lock() and rcu_read_unlock(). If the updater must use rcu_read_lock() and rcu_read_unlock(). If the updater uses call_rcu_bh(), then the corresponding readers must use uses call_rcu_bh(), then the corresponding readers must use rcu_read_lock_bh() and rcu_read_unlock_bh(). Mixing things up rcu_read_lock_bh() and rcu_read_unlock_bh(). If the updater will result in confusion and broken kernels. uses call_rcu_sched(), then the corresponding readers must disable preemption. Mixing things up will result in confusion and broken kernels. One exception to this rule: rcu_read_lock() and rcu_read_unlock() One exception to this rule: rcu_read_lock() and rcu_read_unlock() may be substituted for rcu_read_lock_bh() and rcu_read_unlock_bh() may be substituted for rcu_read_lock_bh() and rcu_read_unlock_bh() Loading @@ -143,9 +166,9 @@ over a rather long period of time, but improvements are always welcome! such cases is a must, of course! And the jury is still out on such cases is a must, of course! And the jury is still out on whether the increased speed is worth it. whether the increased speed is worth it. 8. Although synchronize_rcu() is a bit slower than is call_rcu(), 8. Although synchronize_rcu() is slower than is call_rcu(), it it usually results in simpler code. So, unless update usually results in simpler code. So, unless update performance performance is critically important or the updaters cannot block, is critically important or the updaters cannot block, synchronize_rcu() should be used in preference to call_rcu(). synchronize_rcu() should be used in preference to call_rcu(). An especially important property of the synchronize_rcu() An especially important property of the synchronize_rcu() Loading Loading @@ -187,23 +210,23 @@ over a rather long period of time, but improvements are always welcome! number of updates per grace period. number of updates per grace period. 9. All RCU list-traversal primitives, which include 9. All RCU list-traversal primitives, which include list_for_each_rcu(), list_for_each_entry_rcu(), rcu_dereference(), list_for_each_rcu(), list_for_each_entry_rcu(), list_for_each_continue_rcu(), and list_for_each_safe_rcu(), list_for_each_continue_rcu(), and list_for_each_safe_rcu(), must be within an RCU read-side critical section. RCU must be either within an RCU read-side critical section or must be protected by appropriate update-side locks. RCU read-side critical sections are delimited by rcu_read_lock() read-side critical sections are delimited by rcu_read_lock() and rcu_read_unlock(), or by similar primitives such as and rcu_read_unlock(), or by similar primitives such as rcu_read_lock_bh() and rcu_read_unlock_bh(). rcu_read_lock_bh() and rcu_read_unlock_bh(). Use of the _rcu() list-traversal primitives outside of an The reason that it is permissible to use RCU list-traversal RCU read-side critical section causes no harm other than primitives when the update-side lock is held is that doing so a slight performance degradation on Alpha CPUs. It can can be quite helpful in reducing code bloat when common code is also be quite helpful in reducing code bloat when common shared between readers and updaters. code is shared between readers and updaters. 10. Conversely, if you are in an RCU read-side critical section, 10. Conversely, if you are in an RCU read-side critical section, you -must- use the "_rcu()" variants of the list macros. and you don't hold the appropriate update-side lock, you -must- Failing to do so will break Alpha and confuse people reading use the "_rcu()" variants of the list macros. Failing to do so your code. will break Alpha and confuse people reading your code. 11. Note that synchronize_rcu() -only- guarantees to wait until 11. Note that synchronize_rcu() -only- guarantees to wait until all currently executing rcu_read_lock()-protected RCU read-side all currently executing rcu_read_lock()-protected RCU read-side Loading @@ -230,6 +253,14 @@ over a rather long period of time, but improvements are always welcome! must use whatever locking or other synchronization is required must use whatever locking or other synchronization is required to safely access and/or modify that data structure. to safely access and/or modify that data structure. RCU callbacks are -usually- executed on the same CPU that executed the corresponding call_rcu(), call_rcu_bh(), or call_rcu_sched(), but are by -no- means guaranteed to be. For example, if a given CPU goes offline while having an RCU callback pending, then that RCU callback will execute on some surviving CPU. (If this was not the case, a self-spawning RCU callback would prevent the victim CPU from ever going offline.) 14. SRCU (srcu_read_lock(), srcu_read_unlock(), and synchronize_srcu()) 14. SRCU (srcu_read_lock(), srcu_read_unlock(), and synchronize_srcu()) may only be invoked from process context. Unlike other forms of may only be invoked from process context. Unlike other forms of RCU, it -is- permissible to block in an SRCU read-side critical RCU, it -is- permissible to block in an SRCU read-side critical Loading Documentation/RCU/torture.txt +33 −15 Original line number Original line Diff line number Diff line Loading @@ -10,23 +10,30 @@ status messages via printk(), which can be examined via the dmesg command (perhaps grepping for "torture"). The test is started command (perhaps grepping for "torture"). The test is started when the module is loaded, and stops when the module is unloaded. when the module is loaded, and stops when the module is unloaded. However, actually setting this config option to "y" results in the system CONFIG_RCU_TORTURE_TEST_RUNNABLE running the test immediately upon boot, and ending only when the system is taken down. Normally, one will instead want to build the system It is also possible to specify CONFIG_RCU_TORTURE_TEST=y, which will with CONFIG_RCU_TORTURE_TEST=m and to use modprobe and rmmod to control result in the tests being loaded into the base kernel. In this case, the test, perhaps using a script similar to the one shown at the end of the CONFIG_RCU_TORTURE_TEST_RUNNABLE config option is used to specify this document. Note that you will need CONFIG_MODULE_UNLOAD in order whether the RCU torture tests are to be started immediately during to be able to end the test. boot or whether the /proc/sys/kernel/rcutorture_runnable file is used to enable them. This /proc file can be used to repeatedly pause and restart the tests, regardless of the initial state specified by the CONFIG_RCU_TORTURE_TEST_RUNNABLE config option. You will normally -not- want to start the RCU torture tests during boot (and thus the default is CONFIG_RCU_TORTURE_TEST_RUNNABLE=n), but doing this can sometimes be useful in finding boot-time bugs. MODULE PARAMETERS MODULE PARAMETERS This module has the following parameters: This module has the following parameters: nreaders This is the number of RCU reading threads supported. irqreaders Says to invoke RCU readers from irq level. This is currently The default is twice the number of CPUs. Why twice? done via timers. Defaults to "1" for variants of RCU that To properly exercise RCU implementations with preemptible permit this. (Or, more accurately, variants of RCU that do read-side critical sections. -not- permit this know to ignore this variable.) nfakewriters This is the number of RCU fake writer threads to run. Fake nfakewriters This is the number of RCU fake writer threads to run. Fake writer threads repeatedly use the synchronous "wait for writer threads repeatedly use the synchronous "wait for Loading @@ -37,6 +44,16 @@ nfakewriters This is the number of RCU fake writer threads to run. Fake to trigger special cases caused by multiple writers, such as to trigger special cases caused by multiple writers, such as the synchronize_srcu() early return optimization. the synchronize_srcu() early return optimization. nreaders This is the number of RCU reading threads supported. The default is twice the number of CPUs. Why twice? To properly exercise RCU implementations with preemptible read-side critical sections. shuffle_interval The number of seconds to keep the test threads affinitied to a particular subset of the CPUs, defaults to 3 seconds. Used in conjunction with test_no_idle_hz. stat_interval The number of seconds between output of torture stat_interval The number of seconds between output of torture statistics (via printk()). Regardless of the interval, statistics (via printk()). Regardless of the interval, statistics are printed when the module is unloaded. statistics are printed when the module is unloaded. Loading @@ -44,10 +61,11 @@ stat_interval The number of seconds between output of torture be printed -only- when the module is unloaded, and this be printed -only- when the module is unloaded, and this is the default. is the default. shuffle_interval stutter The length of time to run the test before pausing for this The number of seconds to keep the test threads affinitied same period of time. Defaults to "stutter=5", so as to a particular subset of the CPUs, defaults to 5 seconds. to run and pause for (roughly) five-second intervals. Used in conjunction with test_no_idle_hz. Specifying "stutter=0" causes the test to run continuously without pausing, which is the old default behavior. test_no_idle_hz Whether or not to test the ability of RCU to operate in test_no_idle_hz Whether or not to test the ability of RCU to operate in a kernel that disables the scheduling-clock interrupt to a kernel that disables the scheduling-clock interrupt to Loading Loading
Documentation/IRQ-affinity.txt +28 −9 Original line number Original line Diff line number Diff line ChangeLog: Started by Ingo Molnar <mingo@redhat.com> Update by Max Krasnyansky <maxk@qualcomm.com> SMP IRQ affinity, started by Ingo Molnar <mingo@redhat.com> SMP IRQ affinity /proc/irq/IRQ#/smp_affinity specifies which target CPUs are permitted /proc/irq/IRQ#/smp_affinity specifies which target CPUs are permitted for a given IRQ source. It's a bitmask of allowed CPUs. It's not allowed for a given IRQ source. It's a bitmask of allowed CPUs. It's not allowed to turn off all CPUs, and if an IRQ controller does not support IRQ to turn off all CPUs, and if an IRQ controller does not support IRQ affinity then the value will not change from the default 0xffffffff. affinity then the value will not change from the default 0xffffffff. /proc/irq/default_smp_affinity specifies default affinity mask that applies to all non-active IRQs. Once IRQ is allocated/activated its affinity bitmask will be set to the default mask. It can then be changed as described above. Default mask is 0xffffffff. Here is an example of restricting IRQ44 (eth1) to CPU0-3 then restricting Here is an example of restricting IRQ44 (eth1) to CPU0-3 then restricting the IRQ to CPU4-7 (this is an 8-CPU SMP box): it to CPU4-7 (this is an 8-CPU SMP box): [root@moon 44]# cd /proc/irq/44 [root@moon 44]# cat smp_affinity [root@moon 44]# cat smp_affinity ffffffff ffffffff [root@moon 44]# echo 0f > smp_affinity [root@moon 44]# echo 0f > smp_affinity [root@moon 44]# cat smp_affinity [root@moon 44]# cat smp_affinity 0000000f 0000000f Loading @@ -21,17 +30,27 @@ PING hell (195.4.7.3): 56 data bytes --- hell ping statistics --- --- hell ping statistics --- 6029 packets transmitted, 6027 packets received, 0% packet loss 6029 packets transmitted, 6027 packets received, 0% packet loss round-trip min/avg/max = 0.1/0.1/0.4 ms round-trip min/avg/max = 0.1/0.1/0.4 ms [root@moon 44]# cat /proc/interrupts | grep 44: [root@moon 44]# cat /proc/interrupts | grep 'CPU\|44:' 44: 0 1785 1785 1783 1783 1 CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 1 0 IO-APIC-level eth1 44: 1068 1785 1785 1783 0 0 0 0 IO-APIC-level eth1 As can be seen from the line above IRQ44 was delivered only to the first four processors (0-3). Now lets restrict that IRQ to CPU(4-7). [root@moon 44]# echo f0 > smp_affinity [root@moon 44]# echo f0 > smp_affinity [root@moon 44]# cat smp_affinity 000000f0 [root@moon 44]# ping -f h [root@moon 44]# ping -f h PING hell (195.4.7.3): 56 data bytes PING hell (195.4.7.3): 56 data bytes .. .. --- hell ping statistics --- --- hell ping statistics --- 2779 packets transmitted, 2777 packets received, 0% packet loss 2779 packets transmitted, 2777 packets received, 0% packet loss round-trip min/avg/max = 0.1/0.5/585.4 ms round-trip min/avg/max = 0.1/0.5/585.4 ms [root@moon 44]# cat /proc/interrupts | grep 44: [root@moon 44]# cat /proc/interrupts | 'CPU\|44:' 44: 1068 1785 1785 1784 1784 1069 1070 1069 IO-APIC-level eth1 CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 [root@moon 44]# 44: 1068 1785 1785 1783 1784 1069 1070 1069 IO-APIC-level eth1 This time around IRQ44 was delivered only to the last four processors. i.e counters for the CPU0-3 did not change.
Documentation/RCU/NMI-RCU.txt +3 −0 Original line number Original line Diff line number Diff line Loading @@ -93,6 +93,9 @@ Since NMI handlers disable preemption, synchronize_sched() is guaranteed not to return until all ongoing NMI handlers exit. It is therefore safe not to return until all ongoing NMI handlers exit. It is therefore safe to free up the handler's data as soon as synchronize_sched() returns. to free up the handler's data as soon as synchronize_sched() returns. Important note: for this to work, the architecture in question must invoke irq_enter() and irq_exit() on NMI entry and exit, respectively. Answer to Quick Quiz Answer to Quick Quiz Loading
Documentation/RCU/RTFP.txt +108 −0 Original line number Original line Diff line number Diff line Loading @@ -52,6 +52,10 @@ of each iteration. Unfortunately, chaotic relaxation requires highly structured data, such as the matrices used in scientific programs, and structured data, such as the matrices used in scientific programs, and is thus inapplicable to most data structures in operating-system kernels. is thus inapplicable to most data structures in operating-system kernels. In 1992, Henry (now Alexia) Massalin completed a dissertation advising parallel programmers to defer processing when feasible to simplify synchronization. RCU makes extremely heavy use of this advice. In 1993, Jacobson [Jacobson93] verbally described what is perhaps the In 1993, Jacobson [Jacobson93] verbally described what is perhaps the simplest deferred-free technique: simply waiting a fixed amount of time simplest deferred-free technique: simply waiting a fixed amount of time before freeing blocks awaiting deferred free. Jacobson did not describe before freeing blocks awaiting deferred free. Jacobson did not describe Loading Loading @@ -138,6 +142,13 @@ blocking in read-side critical sections appeared [PaulEMcKenney2006c], Robert Olsson described an RCU-protected trie-hash combination Robert Olsson described an RCU-protected trie-hash combination [RobertOlsson2006a]. [RobertOlsson2006a]. 2007 saw the journal version of the award-winning RCU paper from 2006 [ThomasEHart2007a], as well as a paper demonstrating use of Promela and Spin to mechanically verify an optimization to Oleg Nesterov's QRCU [PaulEMcKenney2007QRCUspin], a design document describing preemptible RCU [PaulEMcKenney2007PreemptibleRCU], and the three-part LWN "What is RCU?" series [PaulEMcKenney2007WhatIsRCUFundamentally, PaulEMcKenney2008WhatIsRCUUsage, and PaulEMcKenney2008WhatIsRCUAPI]. Bibtex Entries Bibtex Entries Loading Loading @@ -202,6 +213,20 @@ Bibtex Entries ,Year="1991" ,Year="1991" } } @phdthesis{HMassalinPhD ,author="H. Massalin" ,title="Synthesis: An Efficient Implementation of Fundamental Operating System Services" ,school="Columbia University" ,address="New York, NY" ,year="1992" ,annotation=" Mondo optimizing compiler. Wait-free stuff. Good advice: defer work to avoid synchronization. " } @unpublished{Jacobson93 @unpublished{Jacobson93 ,author="Van Jacobson" ,author="Van Jacobson" ,title="Avoid Read-Side Locking Via Delayed Free" ,title="Avoid Read-Side Locking Via Delayed Free" Loading Loading @@ -635,3 +660,86 @@ Revised: " " } } @unpublished{PaulEMcKenney2007PreemptibleRCU ,Author="Paul E. McKenney" ,Title="The design of preemptible read-copy-update" ,month="October" ,day="8" ,year="2007" ,note="Available: \url{http://lwn.net/Articles/253651/} [Viewed October 25, 2007]" ,annotation=" LWN article describing the design of preemptible RCU. " } ######################################################################## # # "What is RCU?" LWN series. # @unpublished{PaulEMcKenney2007WhatIsRCUFundamentally ,Author="Paul E. McKenney and Jonathan Walpole" ,Title="What is {RCU}, Fundamentally?" ,month="December" ,day="17" ,year="2007" ,note="Available: \url{http://lwn.net/Articles/262464/} [Viewed December 27, 2007]" ,annotation=" Lays out the three basic components of RCU: (1) publish-subscribe, (2) wait for pre-existing readers to complete, and (2) maintain multiple versions. " } @unpublished{PaulEMcKenney2008WhatIsRCUUsage ,Author="Paul E. McKenney" ,Title="What is {RCU}? Part 2: Usage" ,month="January" ,day="4" ,year="2008" ,note="Available: \url{http://lwn.net/Articles/263130/} [Viewed January 4, 2008]" ,annotation=" Lays out six uses of RCU: 1. RCU is a Reader-Writer Lock Replacement 2. RCU is a Restricted Reference-Counting Mechanism 3. RCU is a Bulk Reference-Counting Mechanism 4. RCU is a Poor Man's Garbage Collector 5. RCU is a Way of Providing Existence Guarantees 6. RCU is a Way of Waiting for Things to Finish " } @unpublished{PaulEMcKenney2008WhatIsRCUAPI ,Author="Paul E. McKenney" ,Title="{RCU} part 3: the {RCU} {API}" ,month="January" ,day="17" ,year="2008" ,note="Available: \url{http://lwn.net/Articles/264090/} [Viewed January 10, 2008]" ,annotation=" Gives an overview of the Linux-kernel RCU API and a brief annotated RCU bibliography. " } @article{DinakarGuniguntala2008IBMSysJ ,author="D. Guniguntala and P. E. McKenney and J. Triplett and J. Walpole" ,title="The read-copy-update mechanism for supporting real-time applications on shared-memory multiprocessor systems with {Linux}" ,Year="2008" ,Month="April" ,journal="IBM Systems Journal" ,volume="47" ,number="2" ,pages="@@-@@" ,annotation=" RCU, realtime RCU, sleepable RCU, performance. " }
Documentation/RCU/checklist.txt +60 −29 Original line number Original line Diff line number Diff line Loading @@ -13,10 +13,13 @@ over a rather long period of time, but improvements are always welcome! detailed performance measurements show that RCU is nonetheless detailed performance measurements show that RCU is nonetheless the right tool for the job. the right tool for the job. The other exception would be where performance is not an issue, Another exception is where performance is not an issue, and RCU and RCU provides a simpler implementation. An example of this provides a simpler implementation. An example of this situation situation is the dynamic NMI code in the Linux 2.6 kernel, is the dynamic NMI code in the Linux 2.6 kernel, at least on at least on architectures where NMIs are rare. architectures where NMIs are rare. Yet another exception is where the low real-time latency of RCU's read-side primitives is critically important. 1. Does the update code have proper mutual exclusion? 1. Does the update code have proper mutual exclusion? Loading @@ -39,9 +42,10 @@ over a rather long period of time, but improvements are always welcome! 2. Do the RCU read-side critical sections make proper use of 2. Do the RCU read-side critical sections make proper use of rcu_read_lock() and friends? These primitives are needed rcu_read_lock() and friends? These primitives are needed to suppress preemption (or bottom halves, in the case of to prevent grace periods from ending prematurely, which rcu_read_lock_bh()) in the read-side critical sections, could result in data being unceremoniously freed out from and are also an excellent aid to readability. under your read-side code, which can greatly increase the actuarial risk of your kernel. As a rough rule of thumb, any dereference of an RCU-protected As a rough rule of thumb, any dereference of an RCU-protected pointer must be covered by rcu_read_lock() or rcu_read_lock_bh() pointer must be covered by rcu_read_lock() or rcu_read_lock_bh() Loading @@ -54,15 +58,30 @@ over a rather long period of time, but improvements are always welcome! be running while updates are in progress. There are a number be running while updates are in progress. There are a number of ways to handle this concurrency, depending on the situation: of ways to handle this concurrency, depending on the situation: a. Make updates appear atomic to readers. For example, a. Use the RCU variants of the list and hlist update primitives to add, remove, and replace elements on an RCU-protected list. Alternatively, use the RCU-protected trees that have been added to the Linux kernel. This is almost always the best approach. b. Proceed as in (a) above, but also maintain per-element locks (that are acquired by both readers and writers) that guard per-element state. Of course, fields that the readers refrain from accessing can be guarded by the update-side lock. This works quite well, also. c. Make updates appear atomic to readers. For example, pointer updates to properly aligned fields will appear pointer updates to properly aligned fields will appear atomic, as will individual atomic primitives. Operations atomic, as will individual atomic primitives. Operations performed under a lock and sequences of multiple atomic performed under a lock and sequences of multiple atomic primitives will -not- appear to be atomic. primitives will -not- appear to be atomic. This is almost always the best approach. This can work, but is starting to get a bit tricky. b. Carefully order the updates and the reads so that d. Carefully order the updates and the reads so that readers see valid data at all phases of the update. readers see valid data at all phases of the update. This is often more difficult than it sounds, especially This is often more difficult than it sounds, especially given modern CPUs' tendency to reorder memory references. given modern CPUs' tendency to reorder memory references. Loading Loading @@ -123,18 +142,22 @@ over a rather long period of time, but improvements are always welcome! when publicizing a pointer to a structure that can when publicizing a pointer to a structure that can be traversed by an RCU read-side critical section. be traversed by an RCU read-side critical section. 5. If call_rcu(), or a related primitive such as call_rcu_bh(), 5. If call_rcu(), or a related primitive such as call_rcu_bh() or is used, the callback function must be written to be called call_rcu_sched(), is used, the callback function must be from softirq context. In particular, it cannot block. written to be called from softirq context. In particular, it cannot block. 6. Since synchronize_rcu() can block, it cannot be called from 6. Since synchronize_rcu() can block, it cannot be called from any sort of irq context. any sort of irq context. Ditto for synchronize_sched() and synchronize_srcu(). 7. If the updater uses call_rcu(), then the corresponding readers 7. If the updater uses call_rcu(), then the corresponding readers must use rcu_read_lock() and rcu_read_unlock(). If the updater must use rcu_read_lock() and rcu_read_unlock(). If the updater uses call_rcu_bh(), then the corresponding readers must use uses call_rcu_bh(), then the corresponding readers must use rcu_read_lock_bh() and rcu_read_unlock_bh(). Mixing things up rcu_read_lock_bh() and rcu_read_unlock_bh(). If the updater will result in confusion and broken kernels. uses call_rcu_sched(), then the corresponding readers must disable preemption. Mixing things up will result in confusion and broken kernels. One exception to this rule: rcu_read_lock() and rcu_read_unlock() One exception to this rule: rcu_read_lock() and rcu_read_unlock() may be substituted for rcu_read_lock_bh() and rcu_read_unlock_bh() may be substituted for rcu_read_lock_bh() and rcu_read_unlock_bh() Loading @@ -143,9 +166,9 @@ over a rather long period of time, but improvements are always welcome! such cases is a must, of course! And the jury is still out on such cases is a must, of course! And the jury is still out on whether the increased speed is worth it. whether the increased speed is worth it. 8. Although synchronize_rcu() is a bit slower than is call_rcu(), 8. Although synchronize_rcu() is slower than is call_rcu(), it it usually results in simpler code. So, unless update usually results in simpler code. So, unless update performance performance is critically important or the updaters cannot block, is critically important or the updaters cannot block, synchronize_rcu() should be used in preference to call_rcu(). synchronize_rcu() should be used in preference to call_rcu(). An especially important property of the synchronize_rcu() An especially important property of the synchronize_rcu() Loading Loading @@ -187,23 +210,23 @@ over a rather long period of time, but improvements are always welcome! number of updates per grace period. number of updates per grace period. 9. All RCU list-traversal primitives, which include 9. All RCU list-traversal primitives, which include list_for_each_rcu(), list_for_each_entry_rcu(), rcu_dereference(), list_for_each_rcu(), list_for_each_entry_rcu(), list_for_each_continue_rcu(), and list_for_each_safe_rcu(), list_for_each_continue_rcu(), and list_for_each_safe_rcu(), must be within an RCU read-side critical section. RCU must be either within an RCU read-side critical section or must be protected by appropriate update-side locks. RCU read-side critical sections are delimited by rcu_read_lock() read-side critical sections are delimited by rcu_read_lock() and rcu_read_unlock(), or by similar primitives such as and rcu_read_unlock(), or by similar primitives such as rcu_read_lock_bh() and rcu_read_unlock_bh(). rcu_read_lock_bh() and rcu_read_unlock_bh(). Use of the _rcu() list-traversal primitives outside of an The reason that it is permissible to use RCU list-traversal RCU read-side critical section causes no harm other than primitives when the update-side lock is held is that doing so a slight performance degradation on Alpha CPUs. It can can be quite helpful in reducing code bloat when common code is also be quite helpful in reducing code bloat when common shared between readers and updaters. code is shared between readers and updaters. 10. Conversely, if you are in an RCU read-side critical section, 10. Conversely, if you are in an RCU read-side critical section, you -must- use the "_rcu()" variants of the list macros. and you don't hold the appropriate update-side lock, you -must- Failing to do so will break Alpha and confuse people reading use the "_rcu()" variants of the list macros. Failing to do so your code. will break Alpha and confuse people reading your code. 11. Note that synchronize_rcu() -only- guarantees to wait until 11. Note that synchronize_rcu() -only- guarantees to wait until all currently executing rcu_read_lock()-protected RCU read-side all currently executing rcu_read_lock()-protected RCU read-side Loading @@ -230,6 +253,14 @@ over a rather long period of time, but improvements are always welcome! must use whatever locking or other synchronization is required must use whatever locking or other synchronization is required to safely access and/or modify that data structure. to safely access and/or modify that data structure. RCU callbacks are -usually- executed on the same CPU that executed the corresponding call_rcu(), call_rcu_bh(), or call_rcu_sched(), but are by -no- means guaranteed to be. For example, if a given CPU goes offline while having an RCU callback pending, then that RCU callback will execute on some surviving CPU. (If this was not the case, a self-spawning RCU callback would prevent the victim CPU from ever going offline.) 14. SRCU (srcu_read_lock(), srcu_read_unlock(), and synchronize_srcu()) 14. SRCU (srcu_read_lock(), srcu_read_unlock(), and synchronize_srcu()) may only be invoked from process context. Unlike other forms of may only be invoked from process context. Unlike other forms of RCU, it -is- permissible to block in an SRCU read-side critical RCU, it -is- permissible to block in an SRCU read-side critical Loading
Documentation/RCU/torture.txt +33 −15 Original line number Original line Diff line number Diff line Loading @@ -10,23 +10,30 @@ status messages via printk(), which can be examined via the dmesg command (perhaps grepping for "torture"). The test is started command (perhaps grepping for "torture"). The test is started when the module is loaded, and stops when the module is unloaded. when the module is loaded, and stops when the module is unloaded. However, actually setting this config option to "y" results in the system CONFIG_RCU_TORTURE_TEST_RUNNABLE running the test immediately upon boot, and ending only when the system is taken down. Normally, one will instead want to build the system It is also possible to specify CONFIG_RCU_TORTURE_TEST=y, which will with CONFIG_RCU_TORTURE_TEST=m and to use modprobe and rmmod to control result in the tests being loaded into the base kernel. In this case, the test, perhaps using a script similar to the one shown at the end of the CONFIG_RCU_TORTURE_TEST_RUNNABLE config option is used to specify this document. Note that you will need CONFIG_MODULE_UNLOAD in order whether the RCU torture tests are to be started immediately during to be able to end the test. boot or whether the /proc/sys/kernel/rcutorture_runnable file is used to enable them. This /proc file can be used to repeatedly pause and restart the tests, regardless of the initial state specified by the CONFIG_RCU_TORTURE_TEST_RUNNABLE config option. You will normally -not- want to start the RCU torture tests during boot (and thus the default is CONFIG_RCU_TORTURE_TEST_RUNNABLE=n), but doing this can sometimes be useful in finding boot-time bugs. MODULE PARAMETERS MODULE PARAMETERS This module has the following parameters: This module has the following parameters: nreaders This is the number of RCU reading threads supported. irqreaders Says to invoke RCU readers from irq level. This is currently The default is twice the number of CPUs. Why twice? done via timers. Defaults to "1" for variants of RCU that To properly exercise RCU implementations with preemptible permit this. (Or, more accurately, variants of RCU that do read-side critical sections. -not- permit this know to ignore this variable.) nfakewriters This is the number of RCU fake writer threads to run. Fake nfakewriters This is the number of RCU fake writer threads to run. Fake writer threads repeatedly use the synchronous "wait for writer threads repeatedly use the synchronous "wait for Loading @@ -37,6 +44,16 @@ nfakewriters This is the number of RCU fake writer threads to run. Fake to trigger special cases caused by multiple writers, such as to trigger special cases caused by multiple writers, such as the synchronize_srcu() early return optimization. the synchronize_srcu() early return optimization. nreaders This is the number of RCU reading threads supported. The default is twice the number of CPUs. Why twice? To properly exercise RCU implementations with preemptible read-side critical sections. shuffle_interval The number of seconds to keep the test threads affinitied to a particular subset of the CPUs, defaults to 3 seconds. Used in conjunction with test_no_idle_hz. stat_interval The number of seconds between output of torture stat_interval The number of seconds between output of torture statistics (via printk()). Regardless of the interval, statistics (via printk()). Regardless of the interval, statistics are printed when the module is unloaded. statistics are printed when the module is unloaded. Loading @@ -44,10 +61,11 @@ stat_interval The number of seconds between output of torture be printed -only- when the module is unloaded, and this be printed -only- when the module is unloaded, and this is the default. is the default. shuffle_interval stutter The length of time to run the test before pausing for this The number of seconds to keep the test threads affinitied same period of time. Defaults to "stutter=5", so as to a particular subset of the CPUs, defaults to 5 seconds. to run and pause for (roughly) five-second intervals. Used in conjunction with test_no_idle_hz. Specifying "stutter=0" causes the test to run continuously without pausing, which is the old default behavior. test_no_idle_hz Whether or not to test the ability of RCU to operate in test_no_idle_hz Whether or not to test the ability of RCU to operate in a kernel that disables the scheduling-clock interrupt to a kernel that disables the scheduling-clock interrupt to Loading
