diff options
Diffstat (limited to 'Documentation/RCU')
| -rw-r--r-- | Documentation/RCU/00-INDEX | 4 | ||||
| -rw-r--r-- | Documentation/RCU/RTFP.txt | 2273 | ||||
| -rw-r--r-- | Documentation/RCU/checklist.txt | 138 | ||||
| -rw-r--r-- | Documentation/RCU/listRCU.txt | 2 | ||||
| -rw-r--r-- | Documentation/RCU/lockdep.txt | 5 | ||||
| -rw-r--r-- | Documentation/RCU/rcu_dereference.txt | 371 | ||||
| -rw-r--r-- | Documentation/RCU/rcubarrier.txt | 40 | ||||
| -rw-r--r-- | Documentation/RCU/rcuref.txt | 61 | ||||
| -rw-r--r-- | Documentation/RCU/stallwarn.txt | 116 | ||||
| -rw-r--r-- | Documentation/RCU/torture.txt | 61 | ||||
| -rw-r--r-- | Documentation/RCU/trace.txt | 515 | ||||
| -rw-r--r-- | Documentation/RCU/whatisRCU.txt | 101 |
12 files changed, 3113 insertions, 574 deletions
diff --git a/Documentation/RCU/00-INDEX b/Documentation/RCU/00-INDEX index 1d7a885761f..f773a264ae0 100644 --- a/Documentation/RCU/00-INDEX +++ b/Documentation/RCU/00-INDEX @@ -8,8 +8,12 @@ listRCU.txt - Using RCU to Protect Read-Mostly Linked Lists lockdep.txt - RCU and lockdep checking +lockdep-splat.txt + - RCU Lockdep splats explained. NMI-RCU.txt - Using RCU to Protect Dynamic NMI Handlers +rcu_dereference.txt + - Proper care and feeding of return values from rcu_dereference() rcubarrier.txt - RCU and Unloadable Modules rculist_nulls.txt diff --git a/Documentation/RCU/RTFP.txt b/Documentation/RCU/RTFP.txt index c43460dade0..2f0fcb2112d 100644 --- a/Documentation/RCU/RTFP.txt +++ b/Documentation/RCU/RTFP.txt @@ -1,9 +1,10 @@ -Read the F-ing Papers! +Read the Fscking Papers! This document describes RCU-related publications, and is followed by the corresponding bibtex entries. A number of the publications may -be found at http://www.rdrop.com/users/paulmck/RCU/. +be found at http://www.rdrop.com/users/paulmck/RCU/. For others, browsers +and search engines will usually find what you are looking for. The first thing resembling RCU was published in 1980, when Kung and Lehman [Kung80] recommended use of a garbage collector to defer destruction @@ -30,6 +31,14 @@ has lapsed, so this approach may be used in non-GPL software, if desired. (In contrast, implementation of RCU is permitted only in software licensed under either GPL or LGPL. Sorry!!!) +In 1987, Rashid et al. described lazy TLB-flush [RichardRashid87a]. +At first glance, this has nothing to do with RCU, but nevertheless +this paper helped inspire the update-side batching used in the later +RCU implementation in DYNIX/ptx. In 1988, Barbara Liskov published +a description of Argus that noted that use of out-of-date values can +be tolerated in some situations. Thus, this paper provides some early +theoretical justification for use of stale data. + In 1990, Pugh [Pugh90] noted that explicitly tracking which threads were reading a given data structure permitted deferred free to operate in the presence of non-terminating threads. However, this explicit @@ -38,13 +47,13 @@ in read-mostly situations. This algorithm does take pains to avoid write-side contention and parallelize the other write-side overheads by providing a fine-grained locking design, however, it would be interesting to see how much of the performance advantage reported in 1990 remains -in 2004. +today. -At about this same time, Adams [Adams91] described ``chaotic relaxation'', -where the normal barriers between successive iterations of convergent -numerical algorithms are relaxed, so that iteration $n$ might use -data from iteration $n-1$ or even $n-2$. This introduces error, -which typically slows convergence and thus increases the number of +At about this same time, Andrews [Andrews91textbook] described ``chaotic +relaxation'', where the normal barriers between successive iterations +of convergent numerical algorithms are relaxed, so that iteration $n$ +might use data from iteration $n-1$ or even $n-2$. This introduces +error, which typically slows convergence and thus increases the number of iterations required. However, this increase is sometimes more than made up for by a reduction in the number of expensive barrier operations, which are otherwise required to synchronize the threads at the end @@ -54,7 +63,8 @@ is thus inapplicable to most data structures in operating-system kernels. In 1992, Henry (now Alexia) Massalin completed a dissertation advising parallel programmers to defer processing when feasible to simplify -synchronization. RCU makes extremely heavy use of this advice. +synchronization [HMassalinPhD]. RCU makes extremely heavy use of +this advice. In 1993, Jacobson [Jacobson93] verbally described what is perhaps the simplest deferred-free technique: simply waiting a fixed amount of time @@ -85,28 +95,33 @@ DYNIX/ptx kernel. The corresponding conference paper appeared in 1998 [McKenney98]. In 1999, the Tornado and K42 groups described their "generations" -mechanism, which quite similar to RCU [Gamsa99]. These operating systems -made pervasive use of RCU in place of "existence locks", which greatly -simplifies locking hierarchies. +mechanism, which is quite similar to RCU [Gamsa99]. These operating +systems made pervasive use of RCU in place of "existence locks", which +greatly simplifies locking hierarchies and helps avoid deadlocks. -2001 saw the first RCU presentation involving Linux [McKenney01a] -at OLS. The resulting abundance of RCU patches was presented the -following year [McKenney02a], and use of RCU in dcache was first -described that same year [Linder02a]. +The year 2000 saw an email exchange that would likely have +led to yet another independent invention of something like RCU +[RustyRussell2000a,RustyRussell2000b]. Instead, 2001 saw the first +RCU presentation involving Linux [McKenney01a] at OLS. The resulting +abundance of RCU patches was presented the following year [McKenney02a], +and use of RCU in dcache was first described that same year [Linder02a]. Also in 2002, Michael [Michael02b,Michael02a] presented "hazard-pointer" techniques that defer the destruction of data structures to simplify non-blocking synchronization (wait-free synchronization, lock-free synchronization, and obstruction-free synchronization are all examples of -non-blocking synchronization). In particular, this technique eliminates -locking, reduces contention, reduces memory latency for readers, and -parallelizes pipeline stalls and memory latency for writers. However, -these techniques still impose significant read-side overhead in the -form of memory barriers. Researchers at Sun worked along similar lines -in the same timeframe [HerlihyLM02]. These techniques can be thought -of as inside-out reference counts, where the count is represented by the -number of hazard pointers referencing a given data structure (rather than -the more conventional counter field within the data structure itself). +non-blocking synchronization). The corresponding journal article appeared +in 2004 [MagedMichael04a]. This technique eliminates locking, reduces +contention, reduces memory latency for readers, and parallelizes pipeline +stalls and memory latency for writers. However, these techniques still +impose significant read-side overhead in the form of memory barriers. +Researchers at Sun worked along similar lines in the same timeframe +[HerlihyLM02]. These techniques can be thought of as inside-out reference +counts, where the count is represented by the number of hazard pointers +referencing a given data structure rather than the more conventional +counter field within the data structure itself. The key advantage +of inside-out reference counts is that they can be stored in immortal +variables, thus allowing races between access and deletion to be avoided. By the same token, RCU can be thought of as a "bulk reference count", where some form of reference counter covers all reference by a given CPU @@ -119,8 +134,10 @@ can be thought of in other terms as well. In 2003, the K42 group described how RCU could be used to create hot-pluggable implementations of operating-system functions [Appavoo03a]. -Later that year saw a paper describing an RCU implementation of System -V IPC [Arcangeli03], and an introduction to RCU in Linux Journal +Later that year saw a paper describing an RCU implementation +of System V IPC [Arcangeli03] (following up on a suggestion by +Hugh Dickins [Dickins02a] and an implementation by Mingming Cao +[MingmingCao2002IPCRCU]), and an introduction to RCU in Linux Journal [McKenney03a]. 2004 has seen a Linux-Journal article on use of RCU in dcache @@ -160,19 +177,63 @@ which Mathieu Desnoyers is now maintaining [MathieuDesnoyers2009URCU] [MathieuDesnoyersPhD]. TINY_RCU [PaulEMcKenney2009BloatWatchRCU] made its appearance, as did expedited RCU [PaulEMcKenney2009expeditedRCU]. The problem of resizeable RCU-protected hash tables may now be on a path -to a solution [JoshTriplett2009RPHash]. +to a solution [JoshTriplett2009RPHash]. A few academic researchers are now +using RCU to solve their parallel problems [HariKannan2009DynamicAnalysisRCU]. + +2010 produced a simpler preemptible-RCU implementation +based on TREE_RCU [PaulEMcKenney2010SimpleOptRCU], lockdep-RCU +[PaulEMcKenney2010LockdepRCU], another resizeable RCU-protected hash +table [HerbertXu2010RCUResizeHash] (this one consuming more memory, +but allowing arbitrary changes in hash function, as required for DoS +avoidance in the networking code), realization of the 2009 RCU-protected +hash table with atomic node move [JoshTriplett2010RPHash], an update on +the RCU API [PaulEMcKenney2010RCUAPI]. + +2011 marked the inclusion of Nick Piggin's fully lockless dentry search +[LinusTorvalds2011Linux2:6:38:rc1:NPigginVFS], an RCU-protected red-black +tree using software transactional memory to protect concurrent updates +(strange, but true!) [PhilHoward2011RCUTMRBTree], yet another variant of +RCU-protected resizeable hash tables [Triplett:2011:RPHash], the 3.0 RCU +trainwreck [PaulEMcKenney2011RCU3.0trainwreck], and Neil Brown's "Meet the +Lockers" LWN article [NeilBrown2011MeetTheLockers]. Some academic +work looked at debugging uses of RCU [Seyster:2011:RFA:2075416.2075425]. + +In 2012, Josh Triplett received his Ph.D. with his dissertation +covering RCU-protected resizable hash tables and the relationship +between memory barriers and read-side traversal order: If the updater +is making changes in the opposite direction from the read-side traveral +order, the updater need only execute a memory-barrier instruction, +but if in the same direction, the updater needs to wait for a grace +period between the individual updates [JoshTriplettPhD]. Also in 2012, +after seventeen years of attempts, an RCU paper made it into a top-flight +academic journal, IEEE Transactions on Parallel and Distributed Systems +[MathieuDesnoyers2012URCU]. A group of researchers in Spain applied +user-level RCU to crowd simulation [GuillermoVigueras2012RCUCrowd], and +another group of researchers in Europe produced a formal description of +RCU based on separation logic [AlexeyGotsman2012VerifyGraceExtended], +which was published in the 2013 European Symposium on Programming +[AlexeyGotsman2013ESOPRCU]. + + Bibtex Entries @article{Kung80 ,author="H. T. Kung and Q. Lehman" -,title="Concurrent Maintenance of Binary Search Trees" +,title="Concurrent Manipulation of Binary Search Trees" ,Year="1980" ,Month="September" ,journal="ACM Transactions on Database Systems" ,volume="5" ,number="3" ,pages="354-382" +,annotation={ + Use garbage collector to clean up data after everyone is done with it. + . + Oldest use of something vaguely resembling RCU that I have found. + http://portal.acm.org/citation.cfm?id=320619&dl=GUIDE, + [Viewed December 3, 2007] +} } @techreport{Manber82 @@ -184,6 +245,31 @@ Bibtex Entries ,number="82-01-01" ,month="January" ,pages="28" +,annotation={ + . + Superseded by Manber84. + . + Describes concurrent AVL tree implementation. Uses a + garbage-collection mechanism to handle concurrent use and deletion + of nodes in the tree, but lacks the summary-of-execution-history + concept of read-copy locking. + . + Keeps full list of processes that were active when a given + node was to be deleted, and waits until all such processes have + -terminated- before allowing this node to be reused. This is + not described in great detail -- one could imagine using process + IDs for this if the ID space was large enough that overlapping + never occurred. + . + This restriction makes this algorithm unsuitable for use in + systems comprised of long-lived processes. It also produces + completely unacceptable overhead in systems with large numbers + of processes. Finally, it is specific to AVL trees. + . + Cites Kung80, so not an independent invention, but the first + RCU-like usage that does not rely on an automatic garbage + collector. +} } @article{Manber84 @@ -195,6 +281,74 @@ Bibtex Entries ,volume="9" ,number="3" ,pages="439-455" +,annotation={ + Describes concurrent AVL tree implementation. Uses a + garbage-collection mechanism to handle concurrent use and deletion + of nodes in the tree, but lacks the summary-of-execution-history + concept of read-copy locking. + . + Keeps full list of processes that were active when a given + node was to be deleted, and waits until all such processes have + -terminated- before allowing this node to be reused. This is + not described in great detail -- one could imagine using process + IDs for this if the ID space was large enough that overlapping + never occurred. + . + This restriction makes this algorithm unsuitable for use in + systems comprised of long-lived processes. It also produces + completely unacceptable overhead in systems with large numbers + of processes. Finally, it is specific to AVL trees. +} +} + +@Conference{RichardRashid87a +,Author="Richard Rashid and Avadis Tevanian and Michael Young and +David Golub and Robert Baron and David Black and William Bolosky and +Jonathan Chew" +,Title="Machine-Independent Virtual Memory Management for Paged +Uniprocessor and Multiprocessor Architectures" +,Booktitle="{2\textsuperscript{nd} Symposium on Architectural Support +for Programming Languages and Operating Systems}" +,Publisher="Association for Computing Machinery" +,Month="October" +,Year="1987" +,pages="31-39" +,Address="Palo Alto, CA" +,note="Available: +\url{http://www.cse.ucsc.edu/~randal/221/rashid-machvm.pdf} +[Viewed February 17, 2005]" +,annotation={ + Describes lazy TLB flush, where one waits for each CPU to pass + through a scheduling-clock interrupt before reusing a given range + of virtual address. Does not describe how one determines that + all CPUs have in fact taken such an interrupt, though there are + no shortage of straightforward methods for accomplishing this. + . + Note that it does not make sense to just wait a fixed amount of + time, since a given CPU might have interrupts disabled for an + extended amount of time. +} +} + +@article{BarbaraLiskov1988ArgusCACM +,author = {Barbara Liskov} +,title = {Distributed programming in {Argus}} +,journal = {Commun. ACM} +,volume = {31} +,number = {3} +,year = {1988} +,issn = {0001-0782} +,pages = {300--312} +,doi = {http://doi.acm.org/10.1145/42392.42399} +,publisher = {ACM} +,address = {New York, NY, USA} +,annotation={ + At the top of page 307: "Conflicts with deposits and withdrawals + are necessary if the reported total is to be up to date. They + could be avoided by having total return a sum that is slightly + out of date." Relies on semantics -- approximate numerical + values sometimes OK. +} } @techreport{Hennessy89 @@ -216,13 +370,30 @@ Bibtex Entries ,year="1990" ,number="CS-TR-2222.1" ,month="June" +,annotation={ + Concurrent access to skip lists. Has both weak and strong search. + Uses concept of ``garbage queue'', but has no real way of cleaning + the garbage efficiently. + . + Appears to be an independent invention of an RCU-like mechanism. +} } -@Book{Adams91 -,Author="Gregory R. Adams" +# Was Adams91, see also syncrefs.bib. +@Book{Andrews91textbook +,Author="Gregory R. Andrews" ,title="Concurrent Programming, Principles, and Practices" ,Publisher="Benjamin Cummins" ,Year="1991" +,annotation={ + Has a few paragraphs describing ``chaotic relaxation'', a + numerical analysis technique that allows multiprocessors to + avoid synchronization overhead by using possibly-stale data. + . + Seems like this is descended from yet another independent + invention of RCU-like function -- but this is restricted + in that reclamation is not necessary. +} } @phdthesis{HMassalinPhD @@ -232,11 +403,12 @@ System Services" ,school="Columbia University" ,address="New York, NY" ,year="1992" -,annotation=" +,annotation={ Mondo optimizing compiler. Wait-free stuff. - Good advice: defer work to avoid synchronization. -" + Good advice: defer work to avoid synchronization. See page 90 + (PDF page 106), Section 5.4, fourth bullet point. +} } @unpublished{Jacobson93 @@ -244,7 +416,13 @@ System Services" ,title="Avoid Read-Side Locking Via Delayed Free" ,year="1993" ,month="September" -,note="Verbal discussion" +,note="private communication" +,annotation={ + Use fixed time delay to approximate grace period. Very simple, + but subject to random memory corruption under heavy load. + . + Independent invention of RCU-like mechanism. +} } @Conference{AjuJohn95 @@ -256,41 +434,52 @@ System Services" ,Year="1995" ,pages="11-23" ,Address="New Orleans, LA" +,note="Available: +\url{https://www.usenix.org/publications/library/proceedings/neworl/full_papers/john.a} +[Viewed October 1, 2010]" +,annotation={ + Age vnodes out of the cache, and have a fixed time set by a kernel + parameter. Not clear that all races were in fact correctly handled. + Used a 20-minute time by default, which would most definitely not + be suitable during DoS attacks or virus scans. + . + Apparently independent invention of RCU-like mechanism. +} } -@conference{Pu95a, -Author = "Calton Pu and Tito Autrey and Andrew Black and Charles Consel and +@conference{Pu95a +,Author = "Calton Pu and Tito Autrey and Andrew Black and Charles Consel and Crispin Cowan and Jon Inouye and Lakshmi Kethana and Jonathan Walpole and -Ke Zhang", -Title = "Optimistic Incremental Specialization: Streamlining a Commercial -Operating System", -Booktitle = "15\textsuperscript{th} ACM Symposium on -Operating Systems Principles (SOSP'95)", -address = "Copper Mountain, CO", -month="December", -year="1995", -pages="314-321", -annotation=" +Ke Zhang" +,Title = "Optimistic Incremental Specialization: Streamlining a Commercial +,Operating System" +,Booktitle = "15\textsuperscript{th} ACM Symposium on +,Operating Systems Principles (SOSP'95)" +,address = "Copper Mountain, CO" +,month="December" +,year="1995" +,pages="314-321" +,annotation={ Uses a replugger, but with a flag to signal when people are using the resource at hand. Only one reader at a time. -" -} - -@conference{Cowan96a, -Author = "Crispin Cowan and Tito Autrey and Charles Krasic and -Calton Pu and Jonathan Walpole", -Title = "Fast Concurrent Dynamic Linking for an Adaptive Operating System", -Booktitle = "International Conference on Configurable Distributed Systems -(ICCDS'96)", -address = "Annapolis, MD", -month="May", -year="1996", -pages="108", -isbn="0-8186-7395-8", -annotation=" +} +} + +@conference{Cowan96a +,Author = "Crispin Cowan and Tito Autrey and Charles Krasic and +,Calton Pu and Jonathan Walpole" +,Title = "Fast Concurrent Dynamic Linking for an Adaptive Operating System" +,Booktitle = "International Conference on Configurable Distributed Systems +(ICCDS'96)" +,address = "Annapolis, MD" +,month="May" +,year="1996" +,pages="108" +,isbn="0-8186-7395-8" +,annotation={ Uses a replugger, but with a counter to signal when people are using the resource at hand. Allows multiple readers. -" +} } @techreport{Slingwine95 @@ -301,31 +490,47 @@ Utilizing Execution History and Thread Monitoring" ,institution="US Patent and Trademark Office" ,address="Washington, DC" ,year="1995" -,number="US Patent 5,442,758 (contributed under GPL)" +,number="US Patent 5,442,758" ,month="August" +,annotation={ + Describes the parallel RCU infrastructure. Includes NUMA aspect + (structure of bitmap can reflect bus structure of computer system). + . + Another independent invention of an RCU-like mechanism, but the + "real" RCU this time! +} } @techreport{Slingwine97 ,author="John D. Slingwine and Paul E. McKenney" -,title="Method for maintaining data coherency using thread -activity summaries in a multicomputer system" +,title="Method for Maintaining Data Coherency Using Thread Activity +Summaries in a Multicomputer System" ,institution="US Patent and Trademark Office" ,address="Washington, DC" ,year="1997" -,number="US Patent 5,608,893 (contributed under GPL)" +,number="US Patent 5,608,893" ,month="March" +,pages="19" +,annotation={ + Describes use of RCU to synchronize data between a pair of + SMP/NUMA computer systems. +} } @techreport{Slingwine98 ,author="John D. Slingwine and Paul E. McKenney" -,title="Apparatus and method for achieving reduced overhead -mutual exclusion and maintaining coherency in a multiprocessor -system utilizing execution history and thread monitoring" +,title="Apparatus and Method for Achieving Reduced Overhead Mutual +Exclusion and Maintaining Coherency in a Multiprocessor System +Utilizing Execution History and Thread Monitoring" ,institution="US Patent and Trademark Office" ,address="Washington, DC" ,year="1998" -,number="US Patent 5,727,209 (contributed under GPL)" +,number="US Patent 5,727,209" ,month="March" +,annotation={ + Describes doing an atomic update by copying the data item and + then substituting it into the data structure. +} } @Conference{McKenney98 @@ -337,6 +542,14 @@ Problems" ,Year="1998" ,pages="509-518" ,Address="Las Vegas, NV" +,annotation={ + Describes and analyzes RCU mechanism in DYNIX/ptx. Describes + application to linked list update and log-buffer flushing. + Defines 'quiescent state'. Includes both measured and analytic + evaluation. + http://www.rdrop.com/users/paulmck/RCU/rclockpdcsproof.pdf + [Viewed December 3, 2007] +} } @Conference{Gamsa99 @@ -349,18 +562,75 @@ Operating System Design and Implementation}" ,Year="1999" ,pages="87-100" ,Address="New Orleans, LA" +,annotation={ + Use of RCU-like facility in K42/Tornado. Another independent + invention of RCU. + See especially pages 7-9 (Section 5). + http://www.usenix.org/events/osdi99/full_papers/gamsa/gamsa.pdf + [Viewed August 30, 2006] +} +} + +@unpublished{RustyRussell2000a +,Author="Rusty Russell" +,Title="Re: modular net drivers" +,month="June" +,year="2000" +,day="23" +,note="Available: +\url{http://oss.sgi.com/projects/netdev/archive/2000-06/msg00250.html} +[Viewed April 10, 2006]" +,annotation={ + Proto-RCU proposal from Phil Rumpf and Rusty Russell. + Yet another independent invention of RCU. + Outline of algorithm to unload modules... + . + Appeared on net-dev mailing list. +} +} + +@unpublished{RustyRussell2000b +,Author="Rusty Russell" +,Title="Re: modular net drivers" +,month="June" +,year="2000" +,day="24" +,note="Available: +\url{http://oss.sgi.com/projects/netdev/archive/2000-06/msg00254.html} +[Viewed April 10, 2006]" +,annotation={ + Proto-RCU proposal from Phil Rumpf and Rusty Russell. + . + Appeared on net-dev mailing list. +} +} + +@unpublished{McKenney01b +,Author="Paul E. McKenney and Dipankar Sarma" +,Title="Read-Copy Update Mutual Exclusion in {Linux}" +,month="February" +,year="2001" +,note="Available: +\url{http://lse.sourceforge.net/locking/rcu/rcupdate_doc.html} +[Viewed October 18, 2004]" +,annotation={ + Prototypical Linux documentation for RCU. +} } @techreport{Slingwine01 ,author="John D. Slingwine and Paul E. McKenney" -,title="Apparatus and method for achieving reduced overhead -mutual exclusion and maintaining coherency in a multiprocessor -system utilizing execution history and thread monitoring" +,title="Apparatus and Method for Achieving Reduced Overhead Mutual +Exclusion and Maintaining Coherency in a Multiprocessor System +Utilizing Execution History and Thread Monitoring" ,institution="US Patent and Trademark Office" ,address="Washington, DC" ,year="2001" -,number="US Patent 5,219,690 (contributed under GPL)" +,number="US Patent 6,219,690" ,month="April" +,annotation={ + 'Change in mode' aspect of RCU. Can be thought of as a lazy barrier. +} } @Conference{McKenney01a @@ -372,12 +642,75 @@ Orran Krieger and Rusty Russell and Dipankar Sarma and Maneesh Soni" ,Year="2001" ,note="Available: \url{http://www.linuxsymposium.org/2001/abstracts/readcopy.php} -\url{http://www.rdrop.com/users/paulmck/rclock/rclock_OLS.2001.05.01c.pdf} +\url{http://www.rdrop.com/users/paulmck/RCU/rclock_OLS.2001.05.01c.pdf} +[Viewed June 23, 2004]" +,annotation={ + Described RCU, and presented some patches implementing and using + it in the Linux kernel. +} +} + +@unpublished{McKenney01f +,Author="Paul E. McKenney" +,Title="{RFC:} patch to allow lock-free traversal of lists with insertion" +,month="October" +,year="2001" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=100259266316456&w=2} [Viewed June 23, 2004]" -annotation=" -Described RCU, and presented some patches implementing and using it in -the Linux kernel. -" +,annotation={ + Memory-barrier and Alpha thread. 100 messages, not too bad... +} +} + +@unpublished{Spraul01 +,Author="Manfred Spraul" +,Title="Re: {RFC:} patch to allow lock-free traversal of lists with insertion" +,month="October" +,year="2001" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=100264675012867&w=2} +[Viewed June 23, 2004]" +,annotation={ + Suggested burying memory barriers in Linux's list-manipulation + primitives. +} +} + +@unpublished{LinusTorvalds2001a +,Author="Linus Torvalds" +,Title="{Re:} {[Lse-tech]} {Re:} {RFC:} patch to allow lock-free traversal of lists with insertion" +,month="October" +,year="2001" +,note="Available: +\url{http://lkml.org/lkml/2001/10/13/105} +[Viewed August 21, 2004]" +,annotation={ +} +} + +@unpublished{Blanchard02a +,Author="Anton Blanchard" +,Title="some RCU dcache and ratcache results" +,month="March" +,year="2002" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=101637107412972&w=2} +[Viewed October 18, 2004]" +} + +@conference{Michael02b +,author="Maged M. Michael" +,title="High Performance Dynamic Lock-Free Hash Tables and List-Based Sets" +,Year="2002" +,Month="August" +,booktitle="{Proceedings of the 14\textsuperscript{th} Annual ACM +Symposium on Parallel +Algorithms and Architecture}" +,pages="73-82" +,annotation={ +Like the title says... +} } @Conference{Linder02a @@ -387,6 +720,10 @@ the Linux kernel. ,Month="June" ,Year="2002" ,pages="289-300" +,annotation={ + Measured scalability of Linux 2.4 kernel's directory-entry cache + (dcache), and measured some scalability enhancements. +} } @Conference{McKenney02a @@ -400,6 +737,36 @@ Andrea Arcangeli and Andi Kleen and Orran Krieger and Rusty Russell" ,note="Available: \url{http://www.linux.org.uk/~ajh/ols2002_proceedings.pdf.gz} [Viewed June 23, 2004]" +,annotation={ + Presented and compared a number of RCU implementations for the + Linux kernel. +} +} + +@unpublished{Sarma02a +,Author="Dipankar Sarma" +,Title="specweb99: dcache scalability results" +,month="July" +,year="2002" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=102645767914212&w=2} +[Viewed June 23, 2004]" +,annotation={ + Compare fastwalk and RCU for dcache. RCU won. +} +} + +@unpublished{Barbieri02 +,Author="Luca Barbieri" +,Title="Re: {[PATCH]} Initial support for struct {vfs\_cred}" +,month="August" +,year="2002" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=103082050621241&w=2} +[Viewed: June 23, 2004]" +,annotation={ + Suggested RCU for vfs\_shared\_cred. +} } @conference{Michael02a @@ -411,27 +778,21 @@ Reads and Writes" ,booktitle="{Proceedings of the 21\textsuperscript{st} Annual ACM Symposium on Principles of Distributed Computing}" ,pages="21-30" -,annotation=" +,annotation={ Each thread keeps an array of pointers to items that it is currently referencing. Sort of an inside-out garbage collection mechanism, but one that requires the accessing code to explicitly state its needs. Also requires read-side memory barriers on most architectures. -" +} } -@conference{Michael02b -,author="Maged M. Michael" -,title="High Performance Dynamic Lock-Free Hash Tables and List-Based Sets" -,Year="2002" -,Month="August" -,booktitle="{Proceedings of the 14\textsuperscript{th} Annual ACM -Symposium on Parallel -Algorithms and Architecture}" -,pages="73-82" -,annotation=" - Like the title says... -" +@unpublished{Dickins02a +,author="Hugh Dickins" +,title="Use RCU for System-V IPC" +,year="2002" +,month="October" +,note="private communication" } @InProceedings{HerlihyLM02 @@ -445,6 +806,51 @@ Symposium on Distributed Computing} ,pages="339-353" } +@unpublished{Sarma02b +,Author="Dipankar Sarma" +,Title="Some dcache\_rcu benchmark numbers" +,month="October" +,year="2002" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=103462075416638&w=2} +[Viewed June 23, 2004]" +,annotation={ + Performance of dcache RCU on kernbench for 16x NUMA-Q and 1x, + 2x, and 4x systems. RCU does no harm, and helps on 16x. +} +} + +@unpublished{MingmingCao2002IPCRCU +,Author="Mingming Cao" +,Title="[PATCH]updated ipc lock patch" +,month="October" +,year="2002" +,note="Available: +\url{https://lkml.org/lkml/2002/10/24/262} +[Viewed February 15, 2014]" +,annotation={ + Mingming Cao's patch to introduce RCU to SysV IPC. +} +} + +@unpublished{LinusTorvalds2003a +,Author="Linus Torvalds" +,Title="Re: {[PATCH]} small fixes in brlock.h" +,month="March" +,year="2003" +,note="Available: +\url{http://lkml.org/lkml/2003/3/9/205} +[Viewed March 13, 2006]" +,annotation={ + Linus suggests replacing brlock with RCU and/or seqlocks: + . + 'It's entirely possible that the current user could be replaced + by RCU and/or seqlocks, and we could get rid of brlocks entirely.' + . + Steve Hemminger responds by replacing them with RCU. +} +} + @article{Appavoo03a ,author="J. Appavoo and K. Hui and C. A. N. Soules and R. W. Wisniewski and D. M. {Da Silva} and O. Krieger and M. A. Auslander and D. J. Edelsohn and @@ -457,6 +863,20 @@ B. Rosenburg and M. Stumm and J. Xenidis" ,volume="42" ,number="1" ,pages="60-76" +,annotation={ + Use of RCU to enable hot-swapping for autonomic behavior in K42. +} +} + +@unpublished{Seigh03 +,author="Joseph W. {Seigh II}" +,title="Read Copy Update" +,Year="2003" +,Month="March" +,note="email correspondence" +,annotation={ + Described the relationship of the VM/XA passive serialization to RCU. +} } @Conference{Arcangeli03 @@ -470,6 +890,25 @@ Dipankar Sarma" ,year="2003" ,month="June" ,pages="297-310" +,annotation={ + Compared updated RCU implementations for the Linux kernel, and + described System V IPC use of RCU, including order-of-magnitude + performance improvements. + http://www.rdrop.com/users/paulmck/RCU/rcu.FREENIX.2003.06.14.pdf +} +} + +@Conference{Soules03a +,Author="Craig A. N. Soules and Jonathan Appavoo and Kevin Hui and +Dilma {Da Silva} and Gregory R. Ganger and Orran Krieger and +Michael Stumm and Robert W. Wisniewski and Marc Auslander and +Michal Ostrowski and Bryan Rosenburg and Jimi Xenidis" +,Title="System Support for Online Reconfiguration" +,Booktitle="Proceedings of the 2003 USENIX Annual Technical Conference" +,Publisher="USENIX Association" +,year="2003" +,month="June" +,pages="141-154" } @article{McKenney03a @@ -481,6 +920,24 @@ Dipankar Sarma" ,volume="1" ,number="114" ,pages="18-26" +,note="Available: +\url{http://www.linuxjournal.com/article/6993} +[Viewed November 14, 2007]" +,annotation={ + Reader-friendly intro to RCU, with the infamous old-man-and-brat + cartoon. +} +} + +@unpublished{Sarma03a +,Author="Dipankar Sarma" +,Title="RCU low latency patches" +,month="December" +,year="2003" +,note="Message ID: 20031222180114.GA2248@in.ibm.com" +,annotation={ + dipankar/ct.2004.03.27/RCUll.2003.12.22.patch +} } @techreport{Friedberg03a @@ -489,9 +946,14 @@ Dipankar Sarma" ,institution="US Patent and Trademark Office" ,address="Washington, DC" ,year="2003" -,number="US Patent 6,662,184 (contributed under GPL)" +,number="US Patent 6,662,184" ,month="December" ,pages="112" +,annotation={ + Applies RCU to a wildcard-search Patricia tree in order to permit + synchronization-free lookup. RCU is used to retain removed nodes + for a grace period before freeing them. +} } @article{McKenney04a @@ -503,6 +965,11 @@ Dipankar Sarma" ,volume="1" ,number="118" ,pages="38-46" +,annotation={ + Reader friendly intro to dcache and RCU. + http://www.linuxjournal.com/node/7124 + [Viewed December 26, 2010] +} } @Conference{McKenney04b @@ -514,8 +981,106 @@ Dipankar Sarma" ,Address="Adelaide, Australia" ,note="Available: \url{http://www.linux.org.au/conf/2004/abstracts.html#90} -\url{http://www.rdrop.com/users/paulmck/rclock/lockperf.2004.01.17a.pdf} +\url{http://www.rdrop.com/users/paulmck/RCU/lockperf.2004.01.17a.pdf} [Viewed June 23, 2004]" +,annotation={ + Compares performance of RCU to that of other locking primitives + over a number of CPUs (x86, Opteron, Itanium, and PPC). +} +} + +@unpublished{Sarma04a +,Author="Dipankar Sarma" +,Title="{[PATCH]} {RCU} for low latency (experimental)" +,month="March" +,year="2004" +,note="\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=108003746402892&w=2}" +,annotation={ + Head of thread: dipankar/2004.03.23/rcu-low-lat.1.patch +} +} + +@unpublished{Sarma04b +,Author="Dipankar Sarma" +,Title="Re: {[PATCH]} {RCU} for low latency (experimental)" +,month="March" +,year="2004" +,note="\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=108016474829546&w=2}" +,annotation={ + dipankar/rcuth.2004.03.24/rcu-throttle.patch +} +} + +@unpublished{Spraul04a +,Author="Manfred Spraul" +,Title="[RFC] 0/5 rcu lock update" +,month="May" +,year="2004" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=108546407726602&w=2} +[Viewed June 23, 2004]" +,annotation={ + Hierarchical-bitmap patch for RCU infrastructure. +} +} + +@unpublished{Steiner04a +,Author="Jack Steiner" +,Title="Re: [Lse-tech] [RFC, PATCH] 1/5 rcu lock update: +Add per-cpu batch counter" +,month="May" +,year="2004" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=108551764515332&w=2} +[Viewed June 23, 2004]" +,annotation={ + RCU runs reasonably on a 512-CPU SGI using Manfred Spraul's patches, + which may be found at: + https://lkml.org/lkml/2004/5/20/49 (split vars into cachelines) + https://lkml.org/lkml/2004/5/22/114 (cpu_quiet() patch) + https://lkml.org/lkml/2004/5/25/24 (0/5) + https://lkml.org/lkml/2004/5/25/23 (1/5) + https://lkml.org/lkml/2004/5/25/265 (works for Jack) + https://lkml.org/lkml/2004/5/25/20 (2/5) + https://lkml.org/lkml/2004/5/25/22 (3/5) + https://lkml.org/lkml/2004/5/25/19 (4/5) + https://lkml.org/lkml/2004/5/25/21 (5/5) +} +} + +@Conference{Sarma04c +,Author="Dipankar Sarma and Paul E. McKenney" +,Title="Making {RCU} Safe for Deep Sub-Millisecond Response +Realtime Applications" +,Booktitle="Proceedings of the 2004 USENIX Annual Technical Conference +(FREENIX Track)" +,Publisher="USENIX Association" +,year="2004" +,month="June" +,pages="182-191" +,annotation={ + Describes and compares a number of modifications to the Linux RCU + implementation that make it friendly to realtime applications. + https://www.usenix.org/conference/2004-usenix-annual-technical-conference/making-rcu-safe-deep-sub-millisecond-response + [Viewed July 26, 2012] +} +} + +@article{MagedMichael04a +,author="Maged M. Michael" +,title="Hazard Pointers: Safe Memory Reclamation for Lock-Free Objects" +,Year="2004" +,Month="June" +,journal="IEEE Transactions on Parallel and Distributed Systems" +,volume="15" +,number="6" +,pages="491-504" +,url="Available: +\url{http://www.research.ibm.com/people/m/michael/ieeetpds-2004.pdf} +[Viewed March 1, 2005]" +,annotation={ + New canonical hazard-pointer citation. +} } @phdthesis{PaulEdwardMcKenneyPhD @@ -526,20 +1091,120 @@ in Operating System Kernels" ,school="OGI School of Science and Engineering at Oregon Health and Sciences University" ,year="2004" +,annotation={ + Describes RCU implementations and presents design patterns + corresponding to common uses of RCU in several operating-system + kernels. + http://www.rdrop.com/users/paulmck/RCU/RCUdissertation.2004.07.14e1.pdf + [Viewed October 15, 2004] +} +} + +@unpublished{PaulEMcKenney2004rcu:dereference +,Author="Dipankar Sarma" +,Title="{Re: RCU : Abstracted RCU dereferencing [5/5]}" +,month="August" +,year="2004" +,note="Available: +\url{http://lkml.org/lkml/2004/8/6/237} +[Viewed June 8, 2010]" +,annotation={ + Introduce rcu_dereference(). +} +} + +@unpublished{JimHouston04a +,Author="Jim Houston" +,Title="{[RFC\&PATCH] Alternative {RCU} implementation}" +,month="August" +,year="2004" ,note="Available: -\url{http://www.rdrop.com/users/paulmck/RCU/RCUdissertation.2004.07.14e1.pdf} +\url{http://lkml.org/lkml/2004/8/30/87} +[Viewed February 17, 2005]" +,annotation={ + Uses active code in rcu_read_lock() and rcu_read_unlock() to + make RCU happen, allowing RCU to function on CPUs that do not + receive a scheduling-clock interrupt. +} +} + +@unpublished{TomHart04a +,Author="Thomas E. Hart" +,Title="Master's Thesis: Applying Lock-free Techniques to the {Linux} Kernel" +,month="October" +,year="2004" +,note="Available: +\url{http://www.cs.toronto.edu/~tomhart/masters_thesis.html} [Viewed October 15, 2004]" +,annotation={ + Proposes comparing RCU to lock-free methods for the Linux kernel. +} } -@Conference{Sarma04c -,Author="Dipankar Sarma and Paul E. McKenney" -,Title="Making RCU Safe for Deep Sub-Millisecond Response Realtime Applications" -,Booktitle="Proceedings of the 2004 USENIX Annual Technical Conference -(FREENIX Track)" -,Publisher="USENIX Association" +@unpublished{Vaddagiri04a +,Author="Srivatsa Vaddagiri" +,Title="Subject: [RFC] Use RCU for tcp\_ehash lookup" +,month="October" ,year="2004" -,month="June" -,pages="182-191" +,note="Available: +\url{http://marc.theaimsgroup.com/?t=109395731700004&r=1&w=2} +[Viewed October 18, 2004]" +,annotation={ + Srivatsa's RCU patch for tcp_ehash lookup. +} +} + +@unpublished{Thirumalai04a +,Author="Ravikiran Thirumalai" +,Title="Subject: [patchset] Lockfree fd lookup 0 of 5" +,month="October" +,year="2004" +,note="Available: +\url{http://marc.theaimsgroup.com/?t=109144217400003&r=1&w=2} +[Viewed October 18, 2004]" +,annotation={ + Ravikiran's lockfree FD patch. +} +} + +@unpublished{Thirumalai04b +,Author="Ravikiran Thirumalai" +,Title="Subject: Re: [patchset] Lockfree fd lookup 0 of 5" +,month="October" +,year="2004" +,note="Available: +\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=109152521410459&w=2} +[Viewed October 18, 2004]" +,annotation={ + Ravikiran's lockfree FD patch. +} +} + +@unpublished{PaulEMcKenney2004rcu:assign:pointer +,Author="Paul E. McKenney" +,Title="{[PATCH 1/3] RCU: \url{rcu_assign_pointer()} removal of memory barriers}" +,month="October" +,year="2004" +,note="Available: +\url{http://lkml.org/lkml/2004/10/23/241} +[Viewed June 8, 2010]" +,annotation={ + Introduce rcu_assign_pointer(). +} +} + +@unpublished{JamesMorris04a +,Author="James Morris" +,Title="{[PATCH 2/3] SELinux} scalability - convert {AVC} to {RCU}" +,day="15" +,month="November" +,year="2004" +,note="\url{http://marc.theaimsgroup.com/?l=linux-kernel&m=110054979416004&w=2}" +,annotation={ + James Morris posts Kaigai Kohei's patch to LKML. + [Viewed December 10, 2004] + Kaigai's patch is at https://lkml.org/lkml/2004/9/27/52 +} } @unpublished{JamesMorris04b @@ -550,6 +1215,85 @@ Oregon Health and Sciences University" ,note="Available: \url{http://www.livejournal.com/users/james_morris/2153.html} [Viewed December 10, 2004]" +,annotation={ + RCU helps SELinux performance. ;-) Made LWN. +} +} + +@unpublished{PaulMcKenney2005RCUSemantics +,Author="Paul E. McKenney and Jonathan Walpole" +,Title="{RCU} Semantics: A First Attempt" +,month="January" +,year="2005" +,day="30" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/rcu-semantics.2005.01.30a.pdf} +[Viewed December 6, 2009]" +,annotation={ + Early derivation of RCU semantics. +} +} + +@unpublished{PaulMcKenney2005e +,Author="Paul E. McKenney" +,Title="Real-Time Preemption and {RCU}" +,month="March" +,year="2005" +,day="17" +,note="Available: +\url{http://lkml.org/lkml/2005/3/17/199} +[Viewed September 5, 2005]" +,annotation={ + First posting showing how RCU can be safely adapted for + preemptable RCU read side critical sections. +} +} + +@unpublished{EsbenNeilsen2005a +,Author="Esben Neilsen" +,Title="Re: Real-Time Preemption and {RCU}" +,month="March" +,year="2005" +,day="18" +,note="Available: +\url{http://lkml.org/lkml/2005/3/18/122} +[Viewed March 30, 2006]" +,annotation={ + Esben Neilsen suggests read-side suppression of grace-period + processing for crude-but-workable realtime RCU. The downside + is indefinite grace periods... But this is OK for experimentation + and testing. +} +} + +@unpublished{TomHart05a +,Author="Thomas E. Hart and Paul E. McKenney and Angela Demke Brown" +,Title="Efficient Memory Reclamation is Necessary for Fast Lock-Free +Data Structures" +,month="March" +,year="2005" +,note="Available: +\url{ftp://ftp.cs.toronto.edu/csrg-technical-reports/515/} +[Viewed March 4, 2005]" +,annotation={ + Comparison of RCU, QBSR, and EBSR. RCU wins for read-mostly + workloads. ;-) +} +} + +@unpublished{JonCorbet2005DeprecateSyncKernel +,Author="Jonathan Corbet" +,Title="API change: synchronize_kernel() deprecated" +,month="May" +,day="3" +,year="2005" +,note="Available: +\url{http://lwn.net/Articles/134484/} +[Viewed May 3, 2005]" +,annotation={ + Jon Corbet describes deprecation of synchronize_kernel() + in favor of synchronize_rcu() and synchronize_sched(). +} } @unpublished{PaulMcKenney05a @@ -560,15 +1304,15 @@ Oregon Health and Sciences University" ,note="Available: \url{http://lkml.org/lkml/2005/5/9/185} [Viewed May 13, 2005]" -,annotation=" +,annotation={ First publication of working lock-based deferred free patches for the CONFIG_PREEMPT_RT environment. -" +} } @conference{PaulMcKenney05b ,Author="Paul E. McKenney and Dipankar Sarma" -,Title="Towards Hard Realtime Response from the Linux Kernel on SMP Hardware" +,Title="Towards Hard Realtime Response from the {Linux} Kernel on {SMP} Hardware" ,Booktitle="linux.conf.au 2005" ,month="April" ,year="2005" @@ -576,9 +1320,122 @@ Oregon Health and Sciences University" ,note="Available: \url{http://www.rdrop.com/users/paulmck/RCU/realtimeRCU.2005.04.23a.pdf} [Viewed May 13, 2005]" -,annotation=" +,annotation={ Realtime turns into making RCU yet more realtime friendly. -" + http://lca2005.linux.org.au/Papers/Paul%20McKenney/Towards%20Hard%20Realtime%20Response%20from%20the%20Linux%20Kernel/LKS.2005.04.22a.pdf +} +} + +@unpublished{PaulEMcKenneyHomePage +,Author="Paul E. McKenney" +,Title="{Paul} {E.} {McKenney}" +,month="May" +,year="2005" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/} +[Viewed May 25, 2005]" +,annotation={ + Paul McKenney's home page. +} +} + +@unpublished{PaulEMcKenneyRCUPage +,Author="Paul E. McKenney" +,Title="Read-Copy Update {(RCU)}" +,month="May" +,year="2005" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU} +[Viewed May 25, 2005]" +,annotation={ + Paul McKenney's RCU page. +} +} + +@unpublished{JosephSeigh2005a +,Author="Joseph Seigh" +,Title="{RCU}+{SMR} (hazard pointers)" +,month="July" +,year="2005" +,note="Personal communication" +,annotation={ + Joe Seigh announcing his atomic-ptr-plus project. + http://sourceforge.net/projects/atomic-ptr-plus/ +} +} + +@unpublished{JosephSeigh2005b +,Author="Joseph Seigh" +,Title="Lock-free synchronization primitives" +,month="July" +,day="6" +,year="2005" +,note="Available: +\url{http://sourceforge.net/projects/atomic-ptr-plus/} +[Viewed August 8, 2005]" +,annotation={ + Joe Seigh's atomic-ptr-plus project. +} +} + +@unpublished{PaulMcKenney2005c +,Author="Paul E.McKenney" +,Title="{[RFC,PATCH] RCU} and {CONFIG\_PREEMPT\_RT} sane patch" +,month="August" +,day="1" +,year="2005" +,note="Available: +\url{http://lkml.org/lkml/2005/8/1/155} +[Viewed March 14, 2006]" +,annotation={ + First operating counter-based realtime RCU patch posted to LKML. +} +} + +@unpublished{PaulMcKenney2005d +,Author="Paul E. McKenney" +,Title="Re: [Fwd: Re: [patch] Real-Time Preemption, -RT-2.6.13-rc4-V0.7.52-01]" +,month="August" +,day="8" +,year="2005" +,note="Available: +\url{http://lkml.org/lkml/2005/8/8/108} +[Viewed March 14, 2006]" +,annotation={ + First operating counter-based realtime RCU patch posted to LKML, + but fixed so that various unusual combinations of configuration + parameters all function properly. +} +} + +@unpublished{PaulMcKenney2005rcutorture +,Author="Paul E. McKenney" +,Title="{[PATCH]} {RCU} torture testing" +,month="October" +,day="1" +,year="2005" +,note="Available: +\url{http://lkml.org/lkml/2005/10/1/70} +[Viewed March 14, 2006]" +,annotation={ + First rcutorture patch. +} +} + +@unpublished{DavidSMiller2006HashedLocking +,Author="David S. Miller" +,Title="Re: [{PATCH}, {RFC}] {RCU} : {OOM} avoidance and lower latency" +,month="January" +,day="6" +,year="2006" +,note="Available: +\url{https://lkml.org/lkml/2006/1/7/22} +[Viewed February 29, 2012]" +,annotation={ + David Miller's view on hashed arrays of locks: used to really + like it, but time he saw an opportunity for this technique, + something else always proved superior. Partitioning or RCU. ;-) +} } @conference{ThomasEHart2006a @@ -591,27 +1448,74 @@ Distributed Processing Symposium" ,year="2006" ,day="25-29" ,address="Rhodes, Greece" -,annotation=" - Compares QSBR (AKA "classic RCU"), HPBR, EBR, and lock-free - reference counting. -" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/hart_ipdps06.pdf} +[Viewed April 28, 2008]" +,annotation={ + Compares QSBR, HPBR, EBR, and lock-free reference counting. + http://www.cs.toronto.edu/~tomhart/perflab/ipdps06.tgz +} +} + +@unpublished{NickPiggin2006radixtree +,Author="Nick Piggin" +,Title="[patch 3/3] radix-tree: {RCU} lockless readside" +,month="June" +,day="20" +,year="2006" +,note="Available: +\url{http://lkml.org/lkml/2006/6/20/238} +[Viewed March 25, 2008]" +,annotation={ + RCU-protected radix tree. +} } @Conference{PaulEMcKenney2006b ,Author="Paul E. McKenney and Dipankar Sarma and Ingo Molnar and Suparna Bhattacharya" -,Title="Extending RCU for Realtime and Embedded Workloads" +,Title="Extending {RCU} for Realtime and Embedded Workloads" ,Booktitle="{Ottawa Linux Symposium}" ,Month="July" ,Year="2006" ,pages="v2 123-138" ,note="Available: -\url{http://www.linuxsymposium.org/2006/index_2006.php} +\url{http://www.linuxsymposium.org/2006/view_abstract.php?content_key=184} \url{http://www.rdrop.com/users/paulmck/RCU/OLSrtRCU.2006.08.11a.pdf} [Viewed January 1, 2007]" -,annotation=" +,annotation={ Described how to improve the -rt implementation of realtime RCU. -" +} +} + +@unpublished{WikipediaRCU +,Author="Paul E. McKenney and Chris Purcell and Algae and Ben Schumin and +Gaius Cornelius and Qwertyus and Neil Conway and Sbw and Blainster and +Canis Rufus and Zoicon5 and Anome and Hal Eisen" +,Title="Read-Copy Update" +,month="July" +,day="8" +,year="2006" +,note="\url{http://en.wikipedia.org/wiki/Read-copy-update}" +,annotation={ + Wikipedia RCU page as of July 8 2006. + [Viewed August 21, 2006] +} +} + +@Conference{NickPiggin2006LocklessPageCache +,Author="Nick Piggin" +,Title="A Lockless Pagecache in Linux---Introduction, Progress, Performance" +,Booktitle="{Ottawa Linux Symposium}" +,Month="July" +,Year="2006" +,pages="v2 249-254" +,note="Available: +\url{http://www.linuxsymposium.org/2006/view_abstract.php?content_key=184} +[Viewed January 11, 2009]" +,annotation={ + Uses RCU-protected radix tree for a lockless page cache. +} } @unpublished{PaulEMcKenney2006c @@ -625,9 +1529,9 @@ Suparna Bhattacharya" Revised: \url{http://www.rdrop.com/users/paulmck/RCU/srcu.2007.01.14a.pdf} [Viewed August 21, 2006]" -,annotation=" +,annotation={ LWN article introducing SRCU. -" +} } @unpublished{RobertOlsson2006a @@ -636,40 +1540,357 @@ Revised: ,month="August" ,day="18" ,year="2006" +,note="\url{http://www.nada.kth.se/~snilsson/publications/TRASH/trash.pdf}" +,annotation={ + RCU-protected dynamic trie-hash combination. + [Viewed March 4, 2011] +} +} + +@unpublished{ChristophHellwig2006RCU2SRCU +,Author="Christoph Hellwig" +,Title="Re: {[-mm PATCH 1/4]} {RCU}: split classic rcu" +,month="September" +,day="28" +,year="2006" +,note="Available: +\url{http://lkml.org/lkml/2006/9/28/160} +[Viewed March 27, 2008]" +} + +@unpublished{PaulEMcKenneyRCUusagePage +,Author="Paul E. McKenney" +,Title="{RCU} {Linux} Usage" +,month="October" +,year="2006" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/linuxusage.html} +[Viewed January 14, 2007]" +,annotation={ + Paul McKenney's RCU page showing graphs plotting Linux-kernel + usage of RCU. +} +} + +@unpublished{PaulEMcKenneyRCUusageRawDataPage +,Author="Paul E. McKenney" +,Title="Read-Copy Update {(RCU)} Usage in {Linux} Kernel" +,month="October" +,year="2006" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/linuxusage/rculocktab.html} +[Viewed January 14, 2007]" +,annotation={ + Paul McKenney's RCU page showing Linux usage of RCU in tabular + form, with links to corresponding cscope databases. +} +} + +@unpublished{GauthamShenoy2006RCUrwlock +,Author="Gautham R. Shenoy" +,Title="[PATCH 4/5] lock\_cpu\_hotplug: Redesign - Lightweight implementation of lock\_cpu\_hotplug" +,month="October" +,year="2006" +,day=26 +,note="Available: +\url{http://lkml.org/lkml/2006/10/26/73} +[Viewed January 26, 2009]" +,annotation={ + RCU-based reader-writer lock that allows readers to proceed with + no memory barriers or atomic instruction in absence of writers. + If writer do show up, readers must of course wait as required by + the semantics of reader-writer locking. This is a recursive + lock. +} +} + +@unpublished{JensAxboe2006SlowSRCU +,Author="Jens Axboe" +,Title="Re: [patch] cpufreq: mark \url{cpufreq_tsc()} as +\url{core_initcall_sync}" +,month="November" +,year="2006" +,day=17 +,note="Available: +\url{http://lkml.org/lkml/2006/11/17/56} +[Viewed May 28, 2007]" +,annotation={ + SRCU's grace periods are too slow for Jens, even after a + factor-of-three speedup. + Sped-up version of SRCU at http://lkml.org/lkml/2006/11/17/359. +} +} + +@unpublished{OlegNesterov2006QRCU +,Author="Oleg Nesterov" +,Title="Re: [patch] cpufreq: mark {\tt cpufreq\_tsc()} as +{\tt core\_initcall\_sync}" +,month="November" +,year="2006" +,day=19 +,note="Available: +\url{http://lkml.org/lkml/2006/11/19/69} +[Viewed May 28, 2007]" +,annotation={ + First cut of QRCU. Expanded/corrected versions followed. + Used to be OlegNesterov2007QRCU, now time-corrected. +} +} + +@unpublished{OlegNesterov2006aQRCU +,Author="Oleg Nesterov" +,Title="Re: [RFC, PATCH 1/2] qrcu: {"quick"} srcu implementation" +,month="November" +,year="2006" +,day=30 +,note="Available: +\url{http://lkml.org/lkml/2006/11/29/330} +[Viewed November 26, 2008]" +,annotation={ + Expanded/corrected version of QRCU. + Used to be OlegNesterov2007aQRCU, now time-corrected. +} +} + +@unpublished{EvgeniyPolyakov2006RCUslowdown +,Author="Evgeniy Polyakov" +,Title="Badness in postponing work" +,month="December" +,year="2006" +,day=05 +,note="Available: +\url{http://www.ioremap.net/node/41} +[Viewed October 28, 2008]" +,annotation={ + Using RCU as a pure delay leads to a 2.5x slowdown in skbs in + the Linux kernel. +} +} + +@inproceedings{ChrisMatthews2006ClusteredObjectsRCU +,author = {Matthews, Chris and Coady, Yvonne and Appavoo, Jonathan} +,title = {Portability events: a programming model for scalable system infrastructures} +,booktitle = {PLOS '06: Proceedings of the 3rd workshop on Programming languages and operating systems} +,year = {2006} +,isbn = {1-59593-577-0} +,pages = {11} +,location = {San Jose, California} +,doi = {http://doi.acm.org/10.1145/1215995.1216006} +,publisher = {ACM} +,address = {New York, NY, USA} +,annotation={ + Uses K42's RCU-like functionality to manage clustered-object + lifetimes. +} +} + +@article{DilmaDaSilva2006K42 +,author = {Silva, Dilma Da and Krieger, Orran and Wisniewski, Robert W. and Waterland, Amos and Tam, David and Baumann, Andrew} +,title = {K42: an infrastructure for operating system research} +,journal = {SIGOPS Oper. Syst. Rev.} +,volume = {40} +,number = {2} +,year = {2006} +,issn = {0163-5980} +,pages = {34--42} +,doi = {http://doi.acm.org/10.1145/1131322.1131333} +,publisher = {ACM} +,address = {New York, NY, USA} +,annotation={ + Describes relationship of K42 generations to RCU. +} +} + +# CoreyMinyard2007list_splice_rcu +@unpublished{CoreyMinyard2007list:splice:rcu +,Author="Corey Minyard and Paul E. McKenney" +,Title="{[PATCH]} add an {RCU} version of list splicing" +,month="January" +,year="2007" +,day=3 ,note="Available: -\url{http://www.nada.kth.se/~snilsson/public/papers/trash/trash.pdf} -[Viewed February 24, 2007]" -,annotation=" +\url{http://lkml.org/lkml/2007/1/3/112} +[Viewed May 28, 2007]" +,annotation={ + Patch for list_splice_rcu(). +} +} + +@unpublished{PaulEMcKenney2007rcubarrier +,Author="Paul E. McKenney" +,Title="{RCU} and Unloadable Modules" +,month="January" +,day="14" +,year="2007" +,note="Available: +\url{http://lwn.net/Articles/217484/} +[Viewed November 22, 2007]" +,annotation={ + LWN article introducing the rcu_barrier() primitive. +} +} + +@unpublished{PeterZijlstra2007SyncBarrier +,Author="Peter Zijlstra and Ingo Molnar" +,Title="{[PATCH 3/7]} barrier: a scalable synchonisation barrier" +,month="January" +,year="2007" +,day=28 +,note="Available: +\url{http://lkml.org/lkml/2007/1/28/34} +[Viewed March 27, 2008]" +,annotation={ + RCU-like implementation for frequent updaters and rare readers(!). + Subsumed into QRCU. Maybe... +} +} + +@unpublished{PaulEMcKenney2007BoostRCU +,Author="Paul E. McKenney" +,Title="Priority-Boosting {RCU} Read-Side Critical Sections" +,month="February" +,day="5" +,year="2007" +,note="\url{http://lwn.net/Articles/220677/}" +,annotation={ + LWN article introducing RCU priority boosting. + Revised: + http://www.rdrop.com/users/paulmck/RCU/RCUbooststate.2007.04.16a.pdf + [Viewed September 7, 2007] +} +} + +@unpublished{PaulMcKenney2007QRCUpatch +,Author="Paul E. McKenney" +,Title="{[PATCH]} {QRCU} with lockless fastpath" +,month="February" +,year="2007" +,day=24 +,note="Available: +\url{http://lkml.org/lkml/2007/2/25/18} +[Viewed March 27, 2008]" +,annotation={ + Patch for QRCU supplying lock-free fast path. +} +} + +@article{JonathanAppavoo2007K42RCU +,author = {Appavoo, Jonathan and Silva, Dilma Da and Krieger, Orran and Auslander, Marc and Ostrowski, Michal and Rosenburg, Bryan and Waterland, Amos and Wisniewski, Robert W. and Xenidis, Jimi and Stumm, Michael and Soares, Livio} +,title = {Experience distributing objects in an SMMP OS} +,journal = {ACM Trans. Comput. Syst.} +,volume = {25} +,number = {3} +,year = {2007} +,issn = {0734-2071} +,pages = {6/1--6/52} +,doi = {http://doi.acm.org/10.1145/1275517.1275518} +,publisher = {ACM} +,address = {New York, NY, USA} +,annotation={ + Role of RCU in K42. +} +} + +@conference{RobertOlsson2007Trash +,Author="Robert Olsson and Stefan Nilsson" +,Title="{TRASH}: A dynamic {LC}-trie and hash data structure" +,booktitle="Workshop on High Performance Switching and Routing (HPSR'07)" +,month="May" +,year="2007" +,note="Available: +\url{http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=4281239} +[Viewed October 1, 2010]" +,annotation={ RCU-protected dynamic trie-hash combination. -" +} } -@unpublished{ThomasEHart2007a -,Author="Thomas E. Hart and Paul E. McKenney and Angela Demke Brown and Jonathan Walpole" -,Title="Performance of memory reclamation for lockless synchronization" -,journal="J. Parallel Distrib. Comput." +@conference{PeterZijlstra2007ConcurrentPagecacheRCU +,Author="Peter Zijlstra" +,Title="Concurrent Pagecache" +,Booktitle="Linux Symposium" +,month="June" +,year="2007" +,address="Ottawa, Canada" +,note="Available: +\url{http://ols.108.redhat.com/2007/Reprints/zijlstra-Reprint.pdf} +[Viewed April 14, 2008]" +,annotation={ + Page-cache modifications permitting RCU readers and concurrent + updates. +} +} + +@unpublished{PaulEMcKenney2007whatisRCU +,Author="Paul E. McKenney" +,Title="What is {RCU}?" ,year="2007" -,note="To appear in J. Parallel Distrib. Comput. - \url{doi=10.1016/j.jpdc.2007.04.010}" +,month="07" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/whatisRCU.html} +[Viewed July 6, 2007]" ,annotation={ - Compares QSBR (AKA "classic RCU"), HPBR, EBR, and lock-free - reference counting. Journal version of ThomasEHart2006a. + Describes RCU in Linux kernel. } } @unpublished{PaulEMcKenney2007QRCUspin ,Author="Paul E. McKenney" -,Title="Using Promela and Spin to verify parallel algorithms" +,Title="Using {Promela} and {Spin} to verify parallel algorithms" ,month="August" ,day="1" ,year="2007" ,note="Available: \url{http://lwn.net/Articles/243851/} [Viewed September 8, 2007]" -,annotation=" +,annotation={ LWN article describing Promela and spin, and also using Oleg Nesterov's QRCU as an example (with Paul McKenney's fastpath). -" + Merged patch at: http://lkml.org/lkml/2007/2/25/18 +} +} + +@unpublished{PaulEMcKenney2007WG21DDOatomics +,Author="Paul E. McKenney and Hans-J. Boehm and Lawrence Crowl" +,Title="C++ Data-Dependency Ordering: Atomics and Memory Model" +,month="August" +,day="3" +,year="2007" +,note="Available: +\url{http://open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2664.htm} +[Viewed December 7, 2009]" +,annotation={ + RCU for C++, parts 1 and 2. +} +} + +@unpublished{PaulEMcKenney2007WG21DDOannotation +,Author="Paul E. McKenney and Lawrence Crowl" +,Title="C++ Data-Dependency Ordering: Function Annotation" +,month="September" +,day="18" +,year="2008" +,note="Available: +\url{http://open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2782.htm} +[Viewed December 7, 2009]" +,annotation={ + RCU for C++, part 2, updated many times. +} +} + +@unpublished{PaulEMcKenney2007PreemptibleRCUPatch +,Author="Paul E. McKenney" +,Title="[PATCH RFC 0/9] {RCU}: Preemptible {RCU}" +,month="September" +,day="10" +,year="2007" +,note="Available: +\url{http://lkml.org/lkml/2007/9/10/213} +[Viewed October 25, 2007]" +,annotation={ + Final patch for preemptable RCU to -rt. (Later patches were + to mainline, eventually incorporated.) +} } @unpublished{PaulEMcKenney2007PreemptibleRCU @@ -681,15 +1902,52 @@ Revised: ,note="Available: \url{http://lwn.net/Articles/253651/} [Viewed October 25, 2007]" -,annotation=" +,annotation={ LWN article describing the design of preemptible RCU. -" } +} + +@article{ThomasEHart2007a +,Author="Thomas E. Hart and Paul E. McKenney and Angela Demke Brown and Jonathan Walpole" +,Title="Performance of memory reclamation for lockless synchronization" +,journal="J. Parallel Distrib. Comput." +,volume={67} +,number="12" +,year="2007" +,issn="0743-7315" +,pages="1270--1285" +,doi="http://dx.doi.org/10.1016/j.jpdc.2007.04.010" +,publisher="Academic Press, Inc." +,address="Orlando, FL, USA" +,annotation={ + Compares QSBR, HPBR, EBR, and lock-free reference counting. + Journal version of ThomasEHart2006a. +} +} + +# MathieuDesnoyers2007call_rcu_schedNeeded +@unpublished{MathieuDesnoyers2007call:rcu:schedNeeded +,Author="Mathieu Desnoyers" +,Title="Re: [patch 1/2] {Linux} Kernel Markers - Support Multiple Probes" +,month="December" +,day="20" +,year="2007" +,note="Available: +\url{http://lkml.org/lkml/2007/12/20/244} +[Viewed March 27, 2008]" +,annotation={ + Request for call_rcu_sched() and rcu_barrier_sched(). +} +} + ######################################################################## # # "What is RCU?" LWN series. # +# http://lwn.net/Articles/262464/ (What is RCU, Fundamentally?) +# http://lwn.net/Articles/263130/ (What is RCU's Usage?) +# http://lwn.net/Articles/264090/ (What is RCU's API?) @unpublished{PaulEMcKenney2007WhatIsRCUFundamentally ,Author="Paul E. McKenney and Jonathan Walpole" @@ -700,11 +1958,11 @@ Revised: ,note="Available: \url{http://lwn.net/Articles/262464/} [Viewed December 27, 2007]" -,annotation=" +,annotation={ Lays out the three basic components of RCU: (1) publish-subscribe, (2) wait for pre-existing readers to complete, and (2) maintain multiple versions. -" +} } @unpublished{PaulEMcKenney2008WhatIsRCUUsage @@ -716,15 +1974,15 @@ Revised: ,note="Available: \url{http://lwn.net/Articles/263130/} [Viewed January 4, 2008]" -,annotation=" +,annotation={ Lays out six uses of RCU: 1. RCU is a Reader-Writer Lock Replacement 2. RCU is a Restricted Reference-Counting Mechanism 3. RCU is a Bulk Reference-Counting Mechanism 4. RCU is a Poor Man's Garbage Collector 5. RCU is a Way of Providing Existence Guarantees - 6. RCU is a Way of Waiting for Things to Finish -" + 6. RCU is a Way of Waiting for Things to Finish +} } @unpublished{PaulEMcKenney2008WhatIsRCUAPI @@ -736,10 +1994,10 @@ Revised: ,note="Available: \url{http://lwn.net/Articles/264090/} [Viewed January 10, 2008]" -,annotation=" +,annotation={ Gives an overview of the Linux-kernel RCU API and a brief annotated RCU bibliography. -" +} } # @@ -747,18 +2005,96 @@ Revised: # ######################################################################## + +@unpublished{SteveRostedt2008dyntickRCUpatch +,Author="Steven Rostedt and Paul E. McKenney" +,Title="{[PATCH]} add support for dynamic ticks and preempt rcu" +,month="January" +,day="29" +,year="2008" +,note="Available: +\url{http://lkml.org/lkml/2008/1/29/208} +[Viewed March 27, 2008]" +,annotation={ + Patch that prevents preemptible RCU from unnecessarily waking + up dynticks-idle CPUs. +} +} + +@unpublished{PaulEMcKenney2008LKMLDependencyOrdering +,Author="Paul E. McKenney" +,Title="Re: [PATCH 02/22 -v7] Add basic support for gcc profiler instrumentation" +,month="February" +,day="1" +,year="2008" +,note="Available: +\url{http://lkml.org/lkml/2008/2/2/255} +[Viewed October 18, 2008]" +,annotation={ + Explanation of compilers violating dependency ordering. +} +} + +@Conference{PaulEMcKenney2008Beijing +,Author="Paul E. McKenney" +,Title="Introducing Technology Into {Linux} Or: +Introducing your technology Into {Linux} will require introducing a +lot of {Linux} into your technology!!!" +,Booktitle="2008 Linux Developer Symposium - China" +,Publisher="OSS China" +,Month="February" +,Year="2008" +,Address="Beijing, China" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/TechIntroLinux.2008.02.19a.pdf} +[Viewed August 12, 2008]" +} + +@unpublished{PaulEMcKenney2008dynticksRCU +,Author="Paul E. McKenney and Steven Rostedt" +,Title="Integrating and Validating dynticks and Preemptable RCU" +,month="April" +,day="24" +,year="2008" +,note="Available: +\url{http://lwn.net/Articles/279077/} +[Viewed April 24, 2008]" +,annotation={ + Describes use of Promela and Spin to validate (and fix!) the + dynticks/RCU interface. +} +} + @article{DinakarGuniguntala2008IBMSysJ ,author="D. Guniguntala and P. E. McKenney and J. Triplett and J. Walpole" ,title="The read-copy-update mechanism for supporting real-time applications on shared-memory multiprocessor systems with {Linux}" ,Year="2008" -,Month="April" +,Month="May" ,journal="IBM Systems Journal" ,volume="47" ,number="2" -,pages="@@-@@" -,annotation=" +,pages="221-236" +,annotation={ RCU, realtime RCU, sleepable RCU, performance. -" + http://www.research.ibm.com/journal/sj/472/guniguntala.pdf + [Viewed April 24, 2008] +} +} + +@unpublished{LaiJiangshan2008NewClassicAlgorithm +,Author="Lai Jiangshan" +,Title="[{RFC}][{PATCH}] rcu classic: new algorithm for callbacks-processing" +,month="June" +,day="3" +,year="2008" +,note="Available: +\url{http://lkml.org/lkml/2008/6/2/539} +[Viewed December 10, 2008]" +,annotation={ + Updated RCU classic algorithm. Introduced multi-tailed list + for RCU callbacks and also pulling common code into + __call_rcu(). +} } @article{PaulEMcKenney2008RCUOSR @@ -775,6 +2111,54 @@ Revised: ,address="New York, NY, USA" ,annotation={ Linux changed RCU to a far greater degree than RCU has changed Linux. + http://portal.acm.org/citation.cfm?doid=1400097.1400099 +} +} + +@unpublished{ManfredSpraul2008StateMachineRCU +,Author="Manfred Spraul" +,Title="[{RFC}, {PATCH}] state machine based rcu" +,month="August" +,day="21" +,year="2008" +,note="Available: +\url{http://lkml.org/lkml/2008/8/21/336} +[Viewed December 8, 2008]" +,annotation={ + State-based RCU. One key thing that this patch does is to + separate the dynticks handling of NMIs and IRQs. +} +} + +@unpublished{ManfredSpraul2008dyntickIRQNMI +,Author="Manfred Spraul" +,Title="Re: [{RFC}, {PATCH}] v4 scalable classic {RCU} implementation" +,month="September" +,day="6" +,year="2008" +,note="Available: +\url{http://lkml.org/lkml/2008/9/6/86} +[Viewed December 8, 2008]" +,annotation={ + Manfred notes a fix required to my attempt to separate irq + and NMI processing for hierarchical RCU's dynticks interface. +} +} + +# Was PaulEMcKenney2011cyclicRCU +@techreport{PaulEMcKenney2008cyclicRCU +,author="Paul E. McKenney" +,title="Efficient Support of Consistent Cyclic Search With Read-Copy Update" +,institution="US Patent and Trademark Office" +,address="Washington, DC" +,year="2008" +,number="US Patent 7,426,511" +,month="September" +,pages="23" +,annotation={ + Maintains an additional level of indirection to allow + readers to confine themselves to the desired snapshot of the + data structure. Only permits one update at a time. } } @@ -784,13 +2168,27 @@ Revised: ,month="November" ,day="3" ,year="2008" -,note="Available: -\url{http://lwn.net/Articles/305782/} -[Viewed November 6, 2008]" -,annotation=" +,note="\url{http://lwn.net/Articles/305782/}" +,annotation={ RCU with combining-tree-based grace-period detection, permitting it to handle thousands of CPUs. -" + [Viewed November 6, 2008] +} +} + +@unpublished{PaulEMcKenney2009BloatwatchRCU +,Author="Paul E. McKenney" +,Title="Re: [PATCH fyi] RCU: the bloatwatch edition" +,month="January" +,day="14" +,year="2009" +,note="Available: +\url{http://lkml.org/lkml/2009/1/14/449} +[Viewed January 15, 2009]" +,annotation={ + Small-footprint implementation of RCU for uniprocessor + embedded applications -- and also for exposition purposes. +} } @conference{PaulEMcKenney2009MaliciousURCU @@ -803,9 +2201,9 @@ Revised: ,note="Available: \url{http://www.rdrop.com/users/paulmck/RCU/urcutorture.2009.01.22a.pdf} [Viewed February 2, 2009]" -,annotation=" +,annotation={ Realtime RCU and torture-testing RCU uses. -" +} } @unpublished{MathieuDesnoyers2009URCU @@ -814,17 +2212,17 @@ Revised: ,month="February" ,day="5" ,year="2009" -,note="Available: -\url{http://lkml.org/lkml/2009/2/5/572} -\url{git://lttng.org/userspace-rcu.git} -[Viewed February 20, 2009]" -,annotation=" +,note="\url{http://lttng.org/urcu}" +,annotation={ Mathieu Desnoyers's user-space RCU implementation. git://lttng.org/userspace-rcu.git -" + http://lttng.org/cgi-bin/gitweb.cgi?p=userspace-rcu.git + http://lttng.org/urcu + http://lkml.org/lkml/2009/2/5/572 +} } -@unpublished{PaulEMcKenney2009BloatWatchRCU +@unpublished{PaulEMcKenney2009LWNBloatWatchRCU ,Author="Paul E. McKenney" ,Title="{RCU}: The {Bloatwatch} Edition" ,month="March" @@ -833,9 +2231,24 @@ Revised: ,note="Available: \url{http://lwn.net/Articles/323929/} [Viewed March 20, 2009]" -,annotation=" +,annotation={ Uniprocessor assumptions allow simplified RCU implementation. -" +} +} + +@unpublished{EvgeniyPolyakov2009EllipticsNetwork +,Author="Evgeniy Polyakov" +,Title="The Elliptics Network" +,month="April" +,day="17" +,year="2009" +,note="Available: +\url{http://www.ioremap.net/projects/elliptics} +[Viewed April 30, 2009]" +,annotation={ + Distributed hash table with transactions, using elliptic + hash functions to distribute data. +} } @unpublished{PaulEMcKenney2009expeditedRCU @@ -847,9 +2260,23 @@ Revised: ,note="Available: \url{http://lkml.org/lkml/2009/6/25/306} [Viewed August 16, 2009]" -,annotation=" +,annotation={ First posting of expedited RCU to be accepted into -tip. -" +} +} + +@unpublished{PaulEMcKenney2009fastRTRCU +,Author="Paul E. McKenney" +,Title="[{PATCH} {RFC} -tip 0/4] {RCU} cleanups and simplified preemptable {RCU}" +,month="July" +,day="23" +,year="2009" +,note="Available: +\url{http://lkml.org/lkml/2009/7/23/294} +[Viewed August 15, 2009]" +,annotation={ + First posting of simple and fast preemptable RCU. +} } @unpublished{JoshTriplett2009RPHash @@ -858,9 +2285,10 @@ Revised: ,month="September" ,year="2009" ,note="Linux Plumbers Conference presentation" -,annotation=" +,annotation={ RP fun with hash tables. -" + Superseded by JoshTriplett2010RPHash +} } @phdthesis{MathieuDesnoyersPhD @@ -872,4 +2300,513 @@ Revised: ,note="Available: \url{http://www.lttng.org/pub/thesis/desnoyers-dissertation-2009-12.pdf} [Viewed December 9, 2009]" +,annotation={ + Chapter 6 (page 97) covers user-level RCU. +} +} + +@unpublished{RelativisticProgrammingWiki +,Author="Josh Triplett and Paul E. McKenney and Jonathan Walpole" +,Title="Relativistic Programming" +,month="September" +,year="2009" +,note="Available: +\url{http://wiki.cs.pdx.edu/rp/} +[Viewed December 9, 2009]" +,annotation={ + Main Relativistic Programming Wiki. +} +} + +@conference{PaulEMcKenney2009DeterministicRCU +,Author="Paul E. McKenney" +,Title="Deterministic Synchronization in Multicore Systems: the Role of {RCU}" +,Booktitle="Eleventh Real Time Linux Workshop" +,month="September" +,year="2009" +,address="Dresden, Germany" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/realtime/paper/DetSyncRCU.2009.08.18a.pdf} +[Viewed January 14, 2009]" +} + +@unpublished{PaulEMcKenney2009HuntingHeisenbugs +,Author="Paul E. McKenney" +,Title="Hunting Heisenbugs" +,month="November" +,year="2009" +,day="1" +,note="Available: +\url{http://paulmck.livejournal.com/14639.html} +[Viewed June 4, 2010]" +,annotation={ + Day-one bug in Tree RCU that took forever to track down. +} +} + +@unpublished{MathieuDesnoyers2009defer:rcu +,Author="Mathieu Desnoyers" +,Title="Kernel RCU: shrink the size of the struct rcu\_head" +,month="December" +,year="2009" +,note="Available: +\url{http://lkml.org/lkml/2009/10/18/129} +[Viewed December 29, 2009]" +,annotation={ + Mathieu proposed defer_rcu() with fixed-size per-thread pool + of RCU callbacks. +} +} + +@unpublished{MathieuDesnoyers2009VerifPrePub +,Author="Mathieu Desnoyers and Paul E. McKenney and Michel R. Dagenais" +,Title="Multi-Core Systems Modeling for Formal Verification of Parallel Algorithms" +,month="December" +,year="2009" +,note="Submitted to IEEE TPDS" +,annotation={ + OOMem model for Mathieu's user-level RCU mechanical proof of + correctness. +} +} + +@unpublished{MathieuDesnoyers2009URCUPrePub +,Author="Mathieu Desnoyers and Paul E. McKenney and Alan Stern and Michel R. Dagenais and Jonathan Walpole" +,Title="User-Level Implementations of Read-Copy Update" +,month="December" +,year="2010" +,url={\url{http://www.computer.org/csdl/trans/td/2012/02/ttd2012020375-abs.html}} +,annotation={ + RCU overview, desiderata, semi-formal semantics, user-level RCU + usage scenarios, three classes of RCU implementation, wait-free + RCU updates, RCU grace-period batching, update overhead, + http://www.rdrop.com/users/paulmck/RCU/urcu-main-accepted.2011.08.30a.pdf + http://www.rdrop.com/users/paulmck/RCU/urcu-supp-accepted.2011.08.30a.pdf + Superseded by MathieuDesnoyers2012URCU. +} +} + +@inproceedings{HariKannan2009DynamicAnalysisRCU +,author = {Kannan, Hari} +,title = {Ordering decoupled metadata accesses in multiprocessors} +,booktitle = {MICRO 42: Proceedings of the 42nd Annual IEEE/ACM International Symposium on Microarchitecture} +,year = {2009} +,isbn = {978-1-60558-798-1} +,pages = {381--390} +,location = {New York, New York} +,doi = {http://doi.acm.org/10.1145/1669112.1669161} +,publisher = {ACM} +,address = {New York, NY, USA} +,annotation={ + Uses RCU to protect metadata used in dynamic analysis. +} +} + +@conference{PaulEMcKenney2010SimpleOptRCU +,Author="Paul E. McKenney" +,Title="Simplicity Through Optimization" +,Booktitle="linux.conf.au 2010" +,month="January" +,year="2010" +,address="Wellington, New Zealand" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/SimplicityThruOptimization.2010.01.21f.pdf} +[Viewed October 10, 2010]" +,annotation={ + TREE_PREEMPT_RCU optimizations greatly simplified the old + PREEMPT_RCU implementation. +} +} + +@unpublished{PaulEMcKenney2010LockdepRCU +,Author="Paul E. McKenney" +,Title="Lockdep-{RCU}" +,month="February" +,year="2010" +,day="1" +,note="\url{https://lwn.net/Articles/371986/}" +,annotation={ + CONFIG_PROVE_RCU, or at least an early version. + [Viewed June 4, 2010] +} +} + +@unpublished{AviKivity2010KVM2RCU +,Author="Avi Kivity" +,Title="[{PATCH} 37/40] {KVM}: Bump maximum vcpu count to 64" +,month="February" +,year="2010" +,note="Available: +\url{http://www.mail-archive.com/kvm@vger.kernel.org/msg28640.html} +[Viewed March 20, 2010]" +,annotation={ + Use of RCU permits KVM to increase the size of guest OSes from + 16 CPUs to 64 CPUs. +} +} + +@unpublished{HerbertXu2010RCUResizeHash +,Author="Herbert Xu" +,Title="bridge: Add core IGMP snooping support" +,month="February" +,year="2010" +,note="Available: +\url{http://kerneltrap.com/mailarchive/linux-netdev/2010/2/26/6270589} +[Viewed March 20, 2011]" +,annotation={ + Use a pair of list_head structures to support RCU-protected + resizable hash tables. +} +} + +@mastersthesis{AbhinavDuggal2010Masters +,author="Abhinav Duggal" +,title="Stopping Data Races Using Redflag" +,school="Stony Brook University" +,year="2010" +,annotation={ + Data-race detector incorporating RCU. + http://www.filesystems.org/docs/abhinav-thesis/abhinav_thesis.pdf +} +} + +@article{JoshTriplett2010RPHash +,author="Josh Triplett and Paul E. McKenney and Jonathan Walpole" +,title="Scalable Concurrent Hash Tables via Relativistic Programming" +,journal="ACM Operating Systems Review" +,year=2010 +,volume=44 +,number=3 +,month="July" +,annotation={ + RP fun with hash tables. + http://portal.acm.org/citation.cfm?id=1842733.1842750 +} +} + +@unpublished{PaulEMcKenney2010RCUAPI +,Author="Paul E. McKenney" +,Title="The {RCU} {API}, 2010 Edition" +,month="December" +,day="8" +,year="2010" +,note="\url{http://lwn.net/Articles/418853/}" +,annotation={ + Includes updated software-engineering features. + [Viewed December 8, 2010] +} +} + +@mastersthesis{AndrejPodzimek2010masters +,author="Andrej Podzimek" +,title="Read-Copy-Update for OpenSolaris" +,school="Charles University in Prague" +,year="2010" +,note="Available: +\url{https://andrej.podzimek.org/thesis.pdf} +[Viewed January 31, 2011]" +,annotation={ + Reviews RCU implementations and creates a few for OpenSolaris. + Drives quiescent-state detection from RCU read-side primitives, + in a manner roughly similar to that of Jim Houston. +} +} + +@unpublished{LinusTorvalds2011Linux2:6:38:rc1:NPigginVFS +,Author="Linus Torvalds" +,Title="Linux 2.6.38-rc1" +,month="January" +,year="2011" +,note="Available: +\url{https://lkml.org/lkml/2011/1/18/322} +[Viewed March 4, 2011]" +,annotation={ + "The RCU-based name lookup is at the other end of the spectrum - the + absolute anti-gimmick. It's some seriously good stuff, and gets rid of + the last main global lock that really tends to hurt some kernel loads. + The dentry lock is no longer a big serializing issue. What's really + nice about it is that it actually improves performance a lot even for + single-threaded loads (on an SMP kernel), because it gets rid of some + of the most expensive parts of path component lookup, which was the + d_lock on every component lookup. So I'm seeing improvements of 30-50% + on some seriously pathname-lookup intensive loads." +} +} + +@techreport{JoshTriplett2011RPScalableCorrectOrdering +,author = {Josh Triplett and Philip W. Howard and Paul E. McKenney and Jonathan Walpole} +,title = {Scalable Correct Memory Ordering via Relativistic Programming} +,year = {2011} +,number = {11-03} +,institution = {Portland State University} +,note = {\url{http://www.cs.pdx.edu/pdfs/tr1103.pdf}} +} + +@inproceedings{PhilHoward2011RCUTMRBTree +,author = {Philip W. Howard and Jonathan Walpole} +,title = {A Relativistic Enhancement to Software Transactional Memory} +,booktitle = {Proceedings of the 3rd USENIX conference on Hot topics in parallelism} +,series = {HotPar'11} +,year = {2011} +,location = {Berkeley, CA} +,pages = {1--6} +,numpages = {6} +,url = {http://www.usenix.org/event/hotpar11/tech/final_files/Howard.pdf} +,publisher = {USENIX Association} +,address = {Berkeley, CA, USA} +} + +@techreport{PaulEMcKenney2011cyclicparallelRCU +,author="Paul E. McKenney and Jonathan Walpole" +,title="Efficient Support of Consistent Cyclic Search With Read-Copy Update and Parallel Updates" +,institution="US Patent and Trademark Office" +,address="Washington, DC" +,year="2011" +,number="US Patent 7,953,778" +,month="May" +,pages="34" +,annotation={ + Maintains an array of generation numbers to track in-flight + updates and keeps an additional level of indirection to allow + readers to confine themselves to the desired snapshot of the + data structure. +} +} + +@inproceedings{Triplett:2011:RPHash +,author = {Triplett, Josh and McKenney, Paul E. and Walpole, Jonathan} +,title = {Resizable, Scalable, Concurrent Hash Tables via Relativistic Programming} +,booktitle = {Proceedings of the 2011 USENIX Annual Technical Conference} +,month = {June} +,year = {2011} +,pages = {145--158} +,numpages = {14} +,url={http://www.usenix.org/event/atc11/tech/final_files/Triplett.pdf} +,publisher = {The USENIX Association} +,address = {Portland, OR USA} +} + +@unpublished{PaulEMcKenney2011RCU3.0trainwreck +,Author="Paul E. McKenney" +,Title="3.0 and {RCU:} what went wrong" +,month="July" +,day="27" +,year="2011" +,note="\url{http://lwn.net/Articles/453002/}" +,annotation={ + Analysis of the RCU trainwreck in Linux kernel 3.0. + [Viewed July 27, 2011] +} +} + +@unpublished{NeilBrown2011MeetTheLockers +,Author="Neil Brown" +,Title="Meet the {Lockers}" +,month="August" +,day="3" +,year="2011" +,note="Available: +\url{http://lwn.net/Articles/453685/} +[Viewed September 2, 2011]" +,annotation={ + The Locker family as an analogy for locking, reference counting, + RCU, and seqlock. +} +} + +@inproceedings{Seyster:2011:RFA:2075416.2075425 +,author = {Seyster, Justin and Radhakrishnan, Prabakar and Katoch, Samriti and Duggal, Abhinav and Stoller, Scott D. and Zadok, Erez} +,title = {Redflag: a framework for analysis of Kernel-level concurrency} +,booktitle = {Proceedings of the 11th international conference on Algorithms and architectures for parallel processing - Volume Part I} +,series = {ICA3PP'11} +,year = {2011} +,isbn = {978-3-642-24649-4} +,location = {Melbourne, Australia} +,pages = {66--79} +,numpages = {14} +,url = {http://dl.acm.org/citation.cfm?id=2075416.2075425} +,acmid = {2075425} +,publisher = {Springer-Verlag} +,address = {Berlin, Heidelberg} +} + +@phdthesis{JoshTriplettPhD +,author="Josh Triplett" +,title="Relativistic Causal Ordering: A Memory Model for Scalable Concurrent Data Structures" +,school="Portland State University" +,year="2012" +,annotation={ + RCU-protected hash tables, barriers vs. read-side traversal order. + . + If the updater is making changes in the opposite direction from + the read-side traveral order, the updater need only execute a + memory-barrier instruction, but if in the same direction, the + updater needs to wait for a grace period between the individual + updates. +} +} + +@article{MathieuDesnoyers2012URCU +,Author="Mathieu Desnoyers and Paul E. McKenney and Alan Stern and Michel R. Dagenais and Jonathan Walpole" +,Title="User-Level Implementations of Read-Copy Update" +,journal="IEEE Transactions on Parallel and Distributed Systems" +,volume={23} +,year="2012" +,issn="1045-9219" +,pages="375-382" +,doi="http://doi.ieeecomputersociety.org/10.1109/TPDS.2011.159" +,publisher="IEEE Computer Society" +,address="Los Alamitos, CA, USA" +,annotation={ + RCU overview, desiderata, semi-formal semantics, user-level RCU + usage scenarios, three classes of RCU implementation, wait-free + RCU updates, RCU grace-period batching, update overhead, + http://www.rdrop.com/users/paulmck/RCU/urcu-main-accepted.2011.08.30a.pdf + http://www.rdrop.com/users/paulmck/RCU/urcu-supp-accepted.2011.08.30a.pdf + http://www.computer.org/cms/Computer.org/dl/trans/td/2012/02/extras/ttd2012020375s.pdf +} +} + +@inproceedings{AustinClements2012RCULinux:mmapsem +,author = {Austin Clements and Frans Kaashoek and Nickolai Zeldovich} +,title = {Scalable Address Spaces Using {RCU} Balanced Trees} +,booktitle = {Architectural Support for Programming Languages and Operating Systems (ASPLOS 2012)} +,month = {March} +,year = {2012} +,pages = {199--210} +,numpages = {12} +,publisher = {ACM} +,address = {London, UK} +,url="http://people.csail.mit.edu/nickolai/papers/clements-bonsai.pdf" +} + +@unpublished{PaulEMcKenney2012ELCbattery +,Author="Paul E. McKenney" +,Title="Making {RCU} Safe For Battery-Powered Devices" +,month="February" +,day="15" +,year="2012" +,note="Available: +\url{http://www.rdrop.com/users/paulmck/RCU/RCUdynticks.2012.02.15b.pdf} +[Viewed March 1, 2012]" +,annotation={ + RCU_FAST_NO_HZ, round 2. +} +} + +@article{GuillermoVigueras2012RCUCrowd +,author = {Vigueras, Guillermo and Ordu\~{n}a, Juan M. and Lozano, Miguel} +,day = {25} +,doi = {10.1007/s11227-012-0766-x} +,issn = {0920-8542} +,journal = {The Journal of Supercomputing} +,keywords = {linux, simulation} +,month = apr +,posted-at = {2012-05-03 09:12:04} +,priority = {2} +,title = {{A Read-Copy Update based parallel server for distributed crowd simulations}} +,url = {http://dx.doi.org/10.1007/s11227-012-0766-x} +,year = {2012} +} + + +@unpublished{JonCorbet2012ACCESS:ONCE +,Author="Jon Corbet" +,Title="{ACCESS\_ONCE()}" +,month="August" +,day="1" +,year="2012" +,note="\url{http://lwn.net/Articles/508991/}" +,annotation={ + A couple of simple specific compiler optimizations that motivate + ACCESS_ONCE(). +} +} + +@unpublished{AlexeyGotsman2012VerifyGraceExtended +,Author="Alexey Gotsman and Noam Rinetzky and Hongseok Yang" +,Title="Verifying Highly Concurrent Algorithms with Grace (extended version)" +,month="July" +,day="10" +,year="2012" +,note="\url{http://software.imdea.org/~gotsman/papers/recycling-esop13-ext.pdf}" +,annotation={ + Separation-logic formulation of RCU uses. +} +} + +@unpublished{PaulMcKenney2012RCUUsage +,Author="Paul E. McKenney and Silas Boyd-Wickizer and Jonathan Walpole" +,Title="{RCU} Usage In the Linux Kernel: One Decade Later" +,month="September" +,day="17" +,year="2012" +,url=http://rdrop.com/users/paulmck/techreports/survey.2012.09.17a.pdf +,note="Technical report paulmck.2012.09.17" +,annotation={ + Overview of the first variant of no-CBs CPUs for RCU. +} +} + +@unpublished{JonCorbet2012NOCB +,Author="Jon Corbet" +,Title="Relocating RCU callbacks" +,month="October" +,day="31" +,year="2012" +,note="\url{http://lwn.net/Articles/522262/}" +,annotation={ + Overview of the first variant of no-CBs CPUs for RCU. +} +} + +@phdthesis{JustinSeyster2012PhD +,author="Justin Seyster" +,title="Runtime Verification of Kernel-Level Concurrency Using Compiler-Based Instrumentation" +,school="Stony Brook University" +,year="2012" +,annotation={ + Looking for data races, including those involving RCU. + Proposal: + http://www.fsl.cs.sunysb.edu/docs/jseyster-proposal/redflag.pdf + Dissertation: + http://www.fsl.cs.sunysb.edu/docs/jseyster-dissertation/redflag.pdf +} +} + +@unpublished{PaulEMcKenney2013RCUUsage +,Author="Paul E. McKenney and Silas Boyd-Wickizer and Jonathan Walpole" +,Title="{RCU} Usage in the {Linux} Kernel: One Decade Later" +,month="February" +,day="24" +,year="2013" +,note="\url{http://rdrop.com/users/paulmck/techreports/RCUUsage.2013.02.24a.pdf}" +,annotation={ + Usage of RCU within the Linux kernel. +} +} + +@inproceedings{AlexeyGotsman2013ESOPRCU +,author = {Alexey Gotsman and Noam Rinetzky and Hongseok Yang} +,title = {Verifying concurrent memory reclamation algorithms with grace} +,booktitle = {ESOP'13: European Symposium on Programming} +,year = {2013} +,pages = {249--269} +,publisher = {Springer} +,address = {Rome, Italy} +,annotation={ + http://software.imdea.org/~gotsman/papers/recycling-esop13.pdf +} +} + +@unpublished{PaulEMcKenney2013NoTinyPreempt +,Author="Paul E. McKenney" +,Title="Simplifying RCU" +,month="March" +,day="6" +,year="2013" +,note="\url{http://lwn.net/Articles/541037/}" +,annotation={ + Getting rid of TINY_PREEMPT_RCU. +} } diff --git a/Documentation/RCU/checklist.txt b/Documentation/RCU/checklist.txt index bff2d8be1e1..877947130eb 100644 --- a/Documentation/RCU/checklist.txt +++ b/Documentation/RCU/checklist.txt @@ -114,12 +114,16 @@ over a rather long period of time, but improvements are always welcome! http://www.openvms.compaq.com/wizard/wiz_2637.html The rcu_dereference() primitive is also an excellent - documentation aid, letting the person reading the code - know exactly which pointers are protected by RCU. + documentation aid, letting the person reading the + code know exactly which pointers are protected by RCU. Please note that compilers can also reorder code, and they are becoming increasingly aggressive about doing - just that. The rcu_dereference() primitive therefore - also prevents destructive compiler optimizations. + just that. The rcu_dereference() primitive therefore also + prevents destructive compiler optimizations. However, + with a bit of devious creativity, it is possible to + mishandle the return value from rcu_dereference(). + Please see rcu_dereference.txt in this directory for + more information. The rcu_dereference() primitive is used by the various "_rcu()" list-traversal primitives, such @@ -162,9 +166,9 @@ over a rather long period of time, but improvements are always welcome! when publicizing a pointer to a structure that can be traversed by an RCU read-side critical section. -5. If call_rcu(), or a related primitive such as call_rcu_bh() or - call_rcu_sched(), is used, the callback function must be - written to be called from softirq context. In particular, +5. If call_rcu(), or a related primitive such as call_rcu_bh(), + call_rcu_sched(), or call_srcu() is used, the callback function + must be written to be called from softirq context. In particular, it cannot block. 6. Since synchronize_rcu() can block, it cannot be called from @@ -180,6 +184,20 @@ over a rather long period of time, but improvements are always welcome! operations that would not normally be undertaken while a real-time workload is running. + In particular, if you find yourself invoking one of the expedited + primitives repeatedly in a loop, please do everyone a favor: + Restructure your code so that it batches the updates, allowing + a single non-expedited primitive to cover the entire batch. + This will very likely be faster than the loop containing the + expedited primitive, and will be much much easier on the rest + of the system, especially to real-time workloads running on + the rest of the system. + + In addition, it is illegal to call the expedited forms from + a CPU-hotplug notifier, or while holding a lock that is acquired + by a CPU-hotplug notifier. Failing to observe this restriction + will result in deadlock. + 7. If the updater uses call_rcu() or synchronize_rcu(), then the corresponding readers must use rcu_read_lock() and rcu_read_unlock(). If the updater uses call_rcu_bh() or @@ -188,11 +206,12 @@ over a rather long period of time, but improvements are always welcome! updater uses call_rcu_sched() or synchronize_sched(), then the corresponding readers must disable preemption, possibly by calling rcu_read_lock_sched() and rcu_read_unlock_sched(). - If the updater uses synchronize_srcu(), the the corresponding - readers must use srcu_read_lock() and srcu_read_unlock(), - and with the same srcu_struct. The rules for the expedited - primitives are the same as for their non-expedited counterparts. - Mixing things up will result in confusion and broken kernels. + If the updater uses synchronize_srcu() or call_srcu(), then + the corresponding readers must use srcu_read_lock() and + srcu_read_unlock(), and with the same srcu_struct. The rules for + the expedited primitives are the same as for their non-expedited + counterparts. Mixing things up will result in confusion and + broken kernels. One exception to this rule: rcu_read_lock() and rcu_read_unlock() may be substituted for rcu_read_lock_bh() and rcu_read_unlock_bh() @@ -202,9 +221,14 @@ over a rather long period of time, but improvements are always welcome! whether the increased speed is worth it. 8. Although synchronize_rcu() is slower than is call_rcu(), it - usually results in simpler code. So, unless update performance - is critically important or the updaters cannot block, - synchronize_rcu() should be used in preference to call_rcu(). + usually results in simpler code. So, unless update performance is + critically important, the updaters cannot block, or the latency of + synchronize_rcu() is visible from userspace, synchronize_rcu() + should be used in preference to call_rcu(). Furthermore, + kfree_rcu() usually results in even simpler code than does + synchronize_rcu() without synchronize_rcu()'s multi-millisecond + latency. So please take advantage of kfree_rcu()'s "fire and + forget" memory-freeing capabilities where it applies. An especially important property of the synchronize_rcu() primitive is that it automatically self-limits: if grace periods @@ -236,10 +260,10 @@ over a rather long period of time, but improvements are always welcome! variations on this theme. b. Limiting update rate. For example, if updates occur only - once per hour, then no explicit rate limiting is required, - unless your system is already badly broken. The dcache - subsystem takes this approach -- updates are guarded - by a global lock, limiting their rate. + once per hour, then no explicit rate limiting is + required, unless your system is already badly broken. + Older versions of the dcache subsystem take this approach, + guarding updates with a global lock, limiting their rate. c. Trusted update -- if updates can only be done manually by superuser or some other trusted user, then it might not @@ -248,23 +272,31 @@ over a rather long period of time, but improvements are always welcome! the machine. d. Use call_rcu_bh() rather than call_rcu(), in order to take - advantage of call_rcu_bh()'s faster grace periods. + advantage of call_rcu_bh()'s faster grace periods. (This + is only a partial solution, though.) e. Periodically invoke synchronize_rcu(), permitting a limited number of updates per grace period. - The same cautions apply to call_rcu_bh() and call_rcu_sched(). + The same cautions apply to call_rcu_bh(), call_rcu_sched(), + call_srcu(), and kfree_rcu(). + + Note that although these primitives do take action to avoid memory + exhaustion when any given CPU has too many callbacks, a determined + user could still exhaust memory. This is especially the case + if a system with a large number of CPUs has been configured to + offload all of its RCU callbacks onto a single CPU, or if the + system has relatively little free memory. 9. All RCU list-traversal primitives, which include - rcu_dereference(), list_for_each_entry_rcu(), - list_for_each_continue_rcu(), and list_for_each_safe_rcu(), - must be either within an RCU read-side critical section or - must be protected by appropriate update-side locks. RCU - read-side critical sections are delimited by rcu_read_lock() - and rcu_read_unlock(), or by similar primitives such as - rcu_read_lock_bh() and rcu_read_unlock_bh(), in which case - the matching rcu_dereference() primitive must be used in order - to keep lockdep happy, in this case, rcu_dereference_bh(). + rcu_dereference(), list_for_each_entry_rcu(), and + list_for_each_safe_rcu(), must be either within an RCU read-side + critical section or must be protected by appropriate update-side + locks. RCU read-side critical sections are delimited by + rcu_read_lock() and rcu_read_unlock(), or by similar primitives + such as rcu_read_lock_bh() and rcu_read_unlock_bh(), in which + case the matching rcu_dereference() primitive must be used in + order to keep lockdep happy, in this case, rcu_dereference_bh(). The reason that it is permissible to use RCU list-traversal primitives when the update-side lock is held is that doing so @@ -282,9 +314,9 @@ over a rather long period of time, but improvements are always welcome! all currently executing rcu_read_lock()-protected RCU read-side critical sections complete. It does -not- necessarily guarantee that all currently running interrupts, NMIs, preempt_disable() - code, or idle loops will complete. Therefore, if you do not have - rcu_read_lock()-protected read-side critical sections, do -not- - use synchronize_rcu(). + code, or idle loops will complete. Therefore, if your + read-side critical sections are protected by something other + than rcu_read_lock(), do -not- use synchronize_rcu(). Similarly, disabling preemption is not an acceptable substitute for rcu_read_lock(). Code that attempts to use preemption @@ -295,6 +327,12 @@ over a rather long period of time, but improvements are always welcome! code under the influence of preempt_disable(), you instead need to use synchronize_irq() or synchronize_sched(). + This same limitation also applies to synchronize_rcu_bh() + and synchronize_srcu(), as well as to the asynchronous and + expedited forms of the three primitives, namely call_rcu(), + call_rcu_bh(), call_srcu(), synchronize_rcu_expedited(), + synchronize_rcu_bh_expedited(), and synchronize_srcu_expedited(). + 12. Any lock acquired by an RCU callback must be acquired elsewhere with softirq disabled, e.g., via spin_lock_irqsave(), spin_lock_bh(), etc. Failing to disable irq on a given @@ -319,28 +357,22 @@ over a rather long period of time, but improvements are always welcome! victim CPU from ever going offline.) 14. SRCU (srcu_read_lock(), srcu_read_unlock(), srcu_dereference(), - synchronize_srcu(), and synchronize_srcu_expedited()) may only - be invoked from process context. Unlike other forms of RCU, it - -is- permissible to block in an SRCU read-side critical section - (demarked by srcu_read_lock() and srcu_read_unlock()), hence the - "SRCU": "sleepable RCU". Please note that if you don't need - to sleep in read-side critical sections, you should be using - RCU rather than SRCU, because RCU is almost always faster and - easier to use than is SRCU. - - If you need to enter your read-side critical section in a - hardirq or exception handler, and then exit that same read-side - critical section in the task that was interrupted, then you need - to srcu_read_lock_raw() and srcu_read_unlock_raw(), which avoid - the lockdep checking that would otherwise this practice illegal. + synchronize_srcu(), synchronize_srcu_expedited(), and call_srcu()) + may only be invoked from process context. Unlike other forms of + RCU, it -is- permissible to block in an SRCU read-side critical + section (demarked by srcu_read_lock() and srcu_read_unlock()), + hence the "SRCU": "sleepable RCU". Please note that if you + don't need to sleep in read-side critical sections, you should be + using RCU rather than SRCU, because RCU is almost always faster + and easier to use than is SRCU. Also unlike other forms of RCU, explicit initialization and cleanup is required via init_srcu_struct() and cleanup_srcu_struct(). These are passed a "struct srcu_struct" that defines the scope of a given SRCU domain. Once initialized, the srcu_struct is passed to srcu_read_lock(), srcu_read_unlock() - synchronize_srcu(), and synchronize_srcu_expedited(). A given - synchronize_srcu() waits only for SRCU read-side critical + synchronize_srcu(), synchronize_srcu_expedited(), and call_srcu(). + A given synchronize_srcu() waits only for SRCU read-side critical sections governed by srcu_read_lock() and srcu_read_unlock() calls that have been passed the same srcu_struct. This property is what makes sleeping read-side critical sections tolerable -- @@ -360,7 +392,7 @@ over a rather long period of time, but improvements are always welcome! requiring SRCU's read-side deadlock immunity or low read-side realtime latency. - Note that, rcu_assign_pointer() relates to SRCU just as they do + Note that, rcu_assign_pointer() relates to SRCU just as it does to other forms of RCU. 15. The whole point of call_rcu(), synchronize_rcu(), and friends @@ -381,9 +413,9 @@ over a rather long period of time, but improvements are always welcome! read-side critical sections. It is the responsibility of the RCU update-side primitives to deal with this. -17. Use CONFIG_PROVE_RCU, CONFIG_DEBUG_OBJECTS_RCU_HEAD, and - the __rcu sparse checks to validate your RCU code. These - can help find problems as follows: +17. Use CONFIG_PROVE_RCU, CONFIG_DEBUG_OBJECTS_RCU_HEAD, and the + __rcu sparse checks (enabled by CONFIG_SPARSE_RCU_POINTER) to + validate your RCU code. These can help find problems as follows: CONFIG_PROVE_RCU: check that accesses to RCU-protected data structures are carried out under the proper RCU diff --git a/Documentation/RCU/listRCU.txt b/Documentation/RCU/listRCU.txt index 4349c1487e9..adb5a378284 100644 --- a/Documentation/RCU/listRCU.txt +++ b/Documentation/RCU/listRCU.txt @@ -205,7 +205,7 @@ RCU ("read-copy update") its name. The RCU code is as follows: audit_copy_rule(&ne->rule, &e->rule); ne->rule.action = newaction; ne->rule.file_count = newfield_count; - list_replace_rcu(e, ne); + list_replace_rcu(&e->list, &ne->list); call_rcu(&e->rcu, audit_free_rule); return 0; } diff --git a/Documentation/RCU/lockdep.txt b/Documentation/RCU/lockdep.txt index a102d4b3724..cd83d2348fe 100644 --- a/Documentation/RCU/lockdep.txt +++ b/Documentation/RCU/lockdep.txt @@ -64,6 +64,11 @@ checking of rcu_dereference() primitives: but retain the compiler constraints that prevent duplicating or coalescsing. This is useful when when testing the value of the pointer itself, for example, against NULL. + rcu_access_index(idx): + Return the value of the index and omit all barriers, but + retain the compiler constraints that prevent duplicating + or coalescsing. This is useful when when testing the + value of the index itself, for example, against -1. The rcu_dereference_check() check expression can be any boolean expression, but would normally include a lockdep expression. However, diff --git a/Documentation/RCU/rcu_dereference.txt b/Documentation/RCU/rcu_dereference.txt new file mode 100644 index 00000000000..ceb05da5a5a --- /dev/null +++ b/Documentation/RCU/rcu_dereference.txt @@ -0,0 +1,371 @@ +PROPER CARE AND FEEDING OF RETURN VALUES FROM rcu_dereference() + +Most of the time, you can use values from rcu_dereference() or one of +the similar primitives without worries. Dereferencing (prefix "*"), +field selection ("->"), assignment ("="), address-of ("&"), addition and +subtraction of constants, and casts all work quite naturally and safely. + +It is nevertheless possible to get into trouble with other operations. +Follow these rules to keep your RCU code working properly: + +o You must use one of the rcu_dereference() family of primitives + to load an RCU-protected pointer, otherwise CONFIG_PROVE_RCU + will complain. Worse yet, your code can see random memory-corruption + bugs due to games that compilers and DEC Alpha can play. + Without one of the rcu_dereference() primitives, compilers + can reload the value, and won't your code have fun with two + different values for a single pointer! Without rcu_dereference(), + DEC Alpha can load a pointer, dereference that pointer, and + return data preceding initialization that preceded the store of + the pointer. + + In addition, the volatile cast in rcu_dereference() prevents the + compiler from deducing the resulting pointer value. Please see + the section entitled "EXAMPLE WHERE THE COMPILER KNOWS TOO MUCH" + for an example where the compiler can in fact deduce the exact + value of the pointer, and thus cause misordering. + +o Do not use single-element RCU-protected arrays. The compiler + is within its right to assume that the value of an index into + such an array must necessarily evaluate to zero. The compiler + could then substitute the constant zero for the computation, so + that the array index no longer depended on the value returned + by rcu_dereference(). If the array index no longer depends + on rcu_dereference(), then both the compiler and the CPU + are within their rights to order the array access before the + rcu_dereference(), which can cause the array access to return + garbage. + +o Avoid cancellation when using the "+" and "-" infix arithmetic + operators. For example, for a given variable "x", avoid + "(x-x)". There are similar arithmetic pitfalls from other + arithmetic operatiors, such as "(x*0)", "(x/(x+1))" or "(x%1)". + The compiler is within its rights to substitute zero for all of + these expressions, so that subsequent accesses no longer depend + on the rcu_dereference(), again possibly resulting in bugs due + to misordering. + + Of course, if "p" is a pointer from rcu_dereference(), and "a" + and "b" are integers that happen to be equal, the expression + "p+a-b" is safe because its value still necessarily depends on + the rcu_dereference(), thus maintaining proper ordering. + +o Avoid all-zero operands to the bitwise "&" operator, and + similarly avoid all-ones operands to the bitwise "|" operator. + If the compiler is able to deduce the value of such operands, + it is within its rights to substitute the corresponding constant + for the bitwise operation. Once again, this causes subsequent + accesses to no longer depend on the rcu_dereference(), causing + bugs due to misordering. + + Please note that single-bit operands to bitwise "&" can also + be dangerous. At this point, the compiler knows that the + resulting value can only take on one of two possible values. + Therefore, a very small amount of additional information will + allow the compiler to deduce the exact value, which again can + result in misordering. + +o If you are using RCU to protect JITed functions, so that the + "()" function-invocation operator is applied to a value obtained + (directly or indirectly) from rcu_dereference(), you may need to + interact directly with the hardware to flush instruction caches. + This issue arises on some systems when a newly JITed function is + using the same memory that was used by an earlier JITed function. + +o Do not use the results from the boolean "&&" and "||" when + dereferencing. For example, the following (rather improbable) + code is buggy: + + int a[2]; + int index; + int force_zero_index = 1; + + ... + + r1 = rcu_dereference(i1) + r2 = a[r1 && force_zero_index]; /* BUGGY!!! */ + + The reason this is buggy is that "&&" and "||" are often compiled + using branches. While weak-memory machines such as ARM or PowerPC + do order stores after such branches, they can speculate loads, + which can result in misordering bugs. + +o Do not use the results from relational operators ("==", "!=", + ">", ">=", "<", or "<=") when dereferencing. For example, + the following (quite strange) code is buggy: + + int a[2]; + int index; + int flip_index = 0; + + ... + + r1 = rcu_dereference(i1) + r2 = a[r1 != flip_index]; /* BUGGY!!! */ + + As before, the reason this is buggy is that relational operators + are often compiled using branches. And as before, although + weak-memory machines such as ARM or PowerPC do order stores + after such branches, but can speculate loads, which can again + result in misordering bugs. + +o Be very careful about comparing pointers obtained from + rcu_dereference() against non-NULL values. As Linus Torvalds + explained, if the two pointers are equal, the compiler could + substitute the pointer you are comparing against for the pointer + obtained from rcu_dereference(). For example: + + p = rcu_dereference(gp); + if (p == &default_struct) + do_default(p->a); + + Because the compiler now knows that the value of "p" is exactly + the address of the variable "default_struct", it is free to + transform this code into the following: + + p = rcu_dereference(gp); + if (p == &default_struct) + do_default(default_struct.a); + + On ARM and Power hardware, the load from "default_struct.a" + can now be speculated, such that it might happen before the + rcu_dereference(). This could result in bugs due to misordering. + + However, comparisons are OK in the following cases: + + o The comparison was against the NULL pointer. If the + compiler knows that the pointer is NULL, you had better + not be dereferencing it anyway. If the comparison is + non-equal, the compiler is none the wiser. Therefore, + it is safe to compare pointers from rcu_dereference() + against NULL pointers. + + o The pointer is never dereferenced after being compared. + Since there are no subsequent dereferences, the compiler + cannot use anything it learned from the comparison + to reorder the non-existent subsequent dereferences. + This sort of comparison occurs frequently when scanning + RCU-protected circular linked lists. + + o The comparison is against a pointer that references memory + that was initialized "a long time ago." The reason + this is safe is that even if misordering occurs, the + misordering will not affect the accesses that follow + the comparison. So exactly how long ago is "a long + time ago"? Here are some possibilities: + + o Compile time. + + o Boot time. + + o Module-init time for module code. + + o Prior to kthread creation for kthread code. + + o During some prior acquisition of the lock that + we now hold. + + o Before mod_timer() time for a timer handler. + + There are many other possibilities involving the Linux + kernel's wide array of primitives that cause code to + be invoked at a later time. + + o The pointer being compared against also came from + rcu_dereference(). In this case, both pointers depend + on one rcu_dereference() or another, so you get proper + ordering either way. + + That said, this situation can make certain RCU usage + bugs more likely to happen. Which can be a good thing, + at least if they happen during testing. An example + of such an RCU usage bug is shown in the section titled + "EXAMPLE OF AMPLIFIED RCU-USAGE BUG". + + o All of the accesses following the comparison are stores, + so that a control dependency preserves the needed ordering. + That said, it is easy to get control dependencies wrong. + Please see the "CONTROL DEPENDENCIES" section of + Documentation/memory-barriers.txt for more details. + + o The pointers are not equal -and- the compiler does + not have enough information to deduce the value of the + pointer. Note that the volatile cast in rcu_dereference() + will normally prevent the compiler from knowing too much. + +o Disable any value-speculation optimizations that your compiler + might provide, especially if you are making use of feedback-based + optimizations that take data collected from prior runs. Such + value-speculation optimizations reorder operations by design. + + There is one exception to this rule: Value-speculation + optimizations that leverage the branch-prediction hardware are + safe on strongly ordered systems (such as x86), but not on weakly + ordered systems (such as ARM or Power). Choose your compiler + command-line options wisely! + + +EXAMPLE OF AMPLIFIED RCU-USAGE BUG + +Because updaters can run concurrently with RCU readers, RCU readers can +see stale and/or inconsistent values. If RCU readers need fresh or +consistent values, which they sometimes do, they need to take proper +precautions. To see this, consider the following code fragment: + + struct foo { + int a; + int b; + int c; + }; + struct foo *gp1; + struct foo *gp2; + + void updater(void) + { + struct foo *p; + + p = kmalloc(...); + if (p == NULL) + deal_with_it(); + p->a = 42; /* Each field in its own cache line. */ + p->b = 43; + p->c = 44; + rcu_assign_pointer(gp1, p); + p->b = 143; + p->c = 144; + rcu_assign_pointer(gp2, p); + } + + void reader(void) + { + struct foo *p; + struct foo *q; + int r1, r2; + + p = rcu_dereference(gp2); + if (p == NULL) + return; + r1 = p->b; /* Guaranteed to get 143. */ + q = rcu_dereference(gp1); /* Guaranteed non-NULL. */ + if (p == q) { + /* The compiler decides that q->c is same as p->c. */ + r2 = p->c; /* Could get 44 on weakly order system. */ + } + do_something_with(r1, r2); + } + +You might be surprised that the outcome (r1 == 143 && r2 == 44) is possible, +but you should not be. After all, the updater might have been invoked +a second time between the time reader() loaded into "r1" and the time +that it loaded into "r2". The fact that this same result can occur due +to some reordering from the compiler and CPUs is beside the point. + +But suppose that the reader needs a consistent view? + +Then one approach is to use locking, for example, as follows: + + struct foo { + int a; + int b; + int c; + spinlock_t lock; + }; + struct foo *gp1; + struct foo *gp2; + + void updater(void) + { + struct foo *p; + + p = kmalloc(...); + if (p == NULL) + deal_with_it(); + spin_lock(&p->lock); + p->a = 42; /* Each field in its own cache line. */ + p->b = 43; + p->c = 44; + spin_unlock(&p->lock); + rcu_assign_pointer(gp1, p); + spin_lock(&p->lock); + p->b = 143; + p->c = 144; + spin_unlock(&p->lock); + rcu_assign_pointer(gp2, p); + } + + void reader(void) + { + struct foo *p; + struct foo *q; + int r1, r2; + + p = rcu_dereference(gp2); + if (p == NULL) + return; + spin_lock(&p->lock); + r1 = p->b; /* Guaranteed to get 143. */ + q = rcu_dereference(gp1); /* Guaranteed non-NULL. */ + if (p == q) { + /* The compiler decides that q->c is same as p->c. */ + r2 = p->c; /* Locking guarantees r2 == 144. */ + } + spin_unlock(&p->lock); + do_something_with(r1, r2); + } + +As always, use the right tool for the job! + + +EXAMPLE WHERE THE COMPILER KNOWS TOO MUCH + +If a pointer obtained from rcu_dereference() compares not-equal to some +other pointer, the compiler normally has no clue what the value of the +first pointer might be. This lack of knowledge prevents the compiler +from carrying out optimizations that otherwise might destroy the ordering +guarantees that RCU depends on. And the volatile cast in rcu_dereference() +should prevent the compiler from guessing the value. + +But without rcu_dereference(), the compiler knows more than you might +expect. Consider the following code fragment: + + struct foo { + int a; + int b; + }; + static struct foo variable1; + static struct foo variable2; + static struct foo *gp = &variable1; + + void updater(void) + { + initialize_foo(&variable2); + rcu_assign_pointer(gp, &variable2); + /* + * The above is the only store to gp in this translation unit, + * and the address of gp is not exported in any way. + */ + } + + int reader(void) + { + struct foo *p; + + p = gp; + barrier(); + if (p == &variable1) + return p->a; /* Must be variable1.a. */ + else + return p->b; /* Must be variable2.b. */ + } + +Because the compiler can see all stores to "gp", it knows that the only +possible values of "gp" are "variable1" on the one hand and "variable2" +on the other. The comparison in reader() therefore tells the compiler +the exact value of "p" even in the not-equals case. This allows the +compiler to make the return values independent of the load from "gp", +in turn destroying the ordering between this load and the loads of the +return values. This can result in "p->b" returning pre-initialization +garbage values. + +In short, rcu_dereference() is -not- optional when you are going to +dereference the resulting pointer. diff --git a/Documentation/RCU/rcubarrier.txt b/Documentation/RCU/rcubarrier.txt index e439a0edee2..b10cfe711e6 100644 --- a/Documentation/RCU/rcubarrier.txt +++ b/Documentation/RCU/rcubarrier.txt @@ -70,18 +70,33 @@ in realtime kernels in order to avoid excessive scheduling latencies. rcu_barrier() -We instead need the rcu_barrier() primitive. This primitive is similar -to synchronize_rcu(), but instead of waiting solely for a grace -period to elapse, it also waits for all outstanding RCU callbacks to -complete. Pseudo-code using rcu_barrier() is as follows: +We instead need the rcu_barrier() primitive. Rather than waiting for +a grace period to elapse, rcu_barrier() waits for all outstanding RCU +callbacks to complete. Please note that rcu_barrier() does -not- imply +synchronize_rcu(), in particular, if there are no RCU callbacks queued +anywhere, rcu_barrier() is within its rights to return immediately, +without waiting for a grace period to elapse. + +Pseudo-code using rcu_barrier() is as follows: 1. Prevent any new RCU callbacks from being posted. 2. Execute rcu_barrier(). 3. Allow the module to be unloaded. -Quick Quiz #1: Why is there no srcu_barrier()? +There are also rcu_barrier_bh(), rcu_barrier_sched(), and srcu_barrier() +functions for the other flavors of RCU, and you of course must match +the flavor of rcu_barrier() with that of call_rcu(). If your module +uses multiple flavors of call_rcu(), then it must also use multiple +flavors of rcu_barrier() when unloading that module. For example, if +it uses call_rcu_bh(), call_srcu() on srcu_struct_1, and call_srcu() on +srcu_struct_2(), then the following three lines of code will be required +when unloading: + + 1 rcu_barrier_bh(); + 2 srcu_barrier(&srcu_struct_1); + 3 srcu_barrier(&srcu_struct_2); -The rcutorture module makes use of rcu_barrier in its exit function +The rcutorture module makes use of rcu_barrier() in its exit function as follows: 1 static void @@ -162,7 +177,7 @@ for any pre-existing callbacks to complete. Then lines 55-62 print status and do operation-specific cleanup, and then return, permitting the module-unload operation to be completed. -Quick Quiz #2: Is there any other situation where rcu_barrier() might +Quick Quiz #1: Is there any other situation where rcu_barrier() might be required? Your module might have additional complications. For example, if your @@ -242,7 +257,7 @@ reaches zero, as follows: 4 complete(&rcu_barrier_completion); 5 } -Quick Quiz #3: What happens if CPU 0's rcu_barrier_func() executes +Quick Quiz #2: What happens if CPU 0's rcu_barrier_func() executes immediately (thus incrementing rcu_barrier_cpu_count to the value one), but the other CPU's rcu_barrier_func() invocations are delayed for a full grace period? Couldn't this result in @@ -259,12 +274,7 @@ so that your module may be safely unloaded. Answers to Quick Quizzes -Quick Quiz #1: Why is there no srcu_barrier()? - -Answer: Since there is no call_srcu(), there can be no outstanding SRCU - callbacks. Therefore, there is no need to wait for them. - -Quick Quiz #2: Is there any other situation where rcu_barrier() might +Quick Quiz #1: Is there any other situation where rcu_barrier() might be required? Answer: Interestingly enough, rcu_barrier() was not originally @@ -278,7 +288,7 @@ Answer: Interestingly enough, rcu_barrier() was not originally implementing rcutorture, and found that rcu_barrier() solves this problem as well. -Quick Quiz #3: What happens if CPU 0's rcu_barrier_func() executes +Quick Quiz #2: What happens if CPU 0's rcu_barrier_func() executes immediately (thus incrementing rcu_barrier_cpu_count to the value one), but the other CPU's rcu_barrier_func() invocations are delayed for a full grace period? Couldn't this result in diff --git a/Documentation/RCU/rcuref.txt b/Documentation/RCU/rcuref.txt index 4202ad09313..141d531aa14 100644 --- a/Documentation/RCU/rcuref.txt +++ b/Documentation/RCU/rcuref.txt @@ -20,7 +20,7 @@ release_referenced() delete() { { ... write_lock(&list_lock); atomic_dec(&el->rc, relfunc) ... - ... delete_element + ... remove_element } write_unlock(&list_lock); ... if (atomic_dec_and_test(&el->rc)) @@ -52,7 +52,7 @@ release_referenced() delete() { { ... spin_lock(&list_lock); if (atomic_dec_and_test(&el->rc)) ... - call_rcu(&el->head, el_free); delete_element + call_rcu(&el->head, el_free); remove_element ... spin_unlock(&list_lock); } ... if (atomic_dec_and_test(&el->rc)) @@ -64,3 +64,60 @@ Sometimes, a reference to the element needs to be obtained in the update (write) stream. In such cases, atomic_inc_not_zero() might be overkill, since we hold the update-side spinlock. One might instead use atomic_inc() in such cases. + +It is not always convenient to deal with "FAIL" in the +search_and_reference() code path. In such cases, the +atomic_dec_and_test() may be moved from delete() to el_free() +as follows: + +1. 2. +add() search_and_reference() +{ { + alloc_object rcu_read_lock(); + ... search_for_element + atomic_set(&el->rc, 1); atomic_inc(&el->rc); + spin_lock(&list_lock); ... + + add_element rcu_read_unlock(); + ... } + spin_unlock(&list_lock); 4. +} delete() +3. { +release_referenced() spin_lock(&list_lock); +{ ... + ... remove_element + if (atomic_dec_and_test(&el->rc)) spin_unlock(&list_lock); + kfree(el); ... + ... call_rcu(&el->head, el_free); +} ... +5. } +void el_free(struct rcu_head *rhp) +{ + release_referenced(); +} + +The key point is that the initial reference added by add() is not removed +until after a grace period has elapsed following removal. This means that +search_and_reference() cannot find this element, which means that the value +of el->rc cannot increase. Thus, once it reaches zero, there are no +readers that can or ever will be able to reference the element. The +element can therefore safely be freed. This in turn guarantees that if +any reader finds the element, that reader may safely acquire a reference +without checking the value of the reference counter. + +In cases where delete() can sleep, synchronize_rcu() can be called from +delete(), so that el_free() can be subsumed into delete as follows: + +4. +delete() +{ + spin_lock(&list_lock); + ... + remove_element + spin_unlock(&list_lock); + ... + synchronize_rcu(); + if (atomic_dec_and_test(&el->rc)) + kfree(el); + ... +} diff --git a/Documentation/RCU/stallwarn.txt b/Documentation/RCU/stallwarn.txt index 083d88cbc08..68fe3ad2701 100644 --- a/Documentation/RCU/stallwarn.txt +++ b/Documentation/RCU/stallwarn.txt @@ -12,14 +12,39 @@ CONFIG_RCU_CPU_STALL_TIMEOUT This kernel configuration parameter defines the period of time that RCU will wait from the beginning of a grace period until it issues an RCU CPU stall warning. This time period is normally - ten seconds. + 21 seconds. -RCU_SECONDS_TILL_STALL_RECHECK + This configuration parameter may be changed at runtime via the + /sys/module/rcutree/parameters/rcu_cpu_stall_timeout, however + this parameter is checked only at the beginning of a cycle. + So if you are 10 seconds into a 40-second stall, setting this + sysfs parameter to (say) five will shorten the timeout for the + -next- stall, or the following warning for the current stall + (assuming the stall lasts long enough). It will not affect the + timing of the next warning for the current stall. - This macro defines the period of time that RCU will wait after - issuing a stall warning until it issues another stall warning - for the same stall. This time period is normally set to three - times the check interval plus thirty seconds. + Stall-warning messages may be enabled and disabled completely via + /sys/module/rcupdate/parameters/rcu_cpu_stall_suppress. + +CONFIG_RCU_CPU_STALL_VERBOSE + + This kernel configuration parameter causes the stall warning to + also dump the stacks of any tasks that are blocking the current + RCU-preempt grace period. + +CONFIG_RCU_CPU_STALL_INFO + + This kernel configuration parameter causes the stall warning to + print out additional per-CPU diagnostic information, including + information on scheduling-clock ticks and RCU's idle-CPU tracking. + +RCU_STALL_DELAY_DELTA + + Although the lockdep facility is extremely useful, it does add + some overhead. Therefore, under CONFIG_PROVE_RCU, the + RCU_STALL_DELAY_DELTA macro allows five extra seconds before + giving an RCU CPU stall warning message. (This is a cpp + macro, not a kernel configuration parameter.) RCU_STALL_RAT_DELAY @@ -28,7 +53,8 @@ RCU_STALL_RAT_DELAY However, if the offending CPU does not detect its own stall in the number of jiffies specified by RCU_STALL_RAT_DELAY, then some other CPU will complain. This delay is normally set to - two jiffies. + two jiffies. (This is a cpp macro, not a kernel configuration + parameter.) When a CPU detects that it is stalling, it will print a message similar to the following: @@ -62,7 +88,75 @@ printing, there will be a spurious stall-warning message: INFO: rcu_bh_state detected stalls on CPUs/tasks: { } (detected by 4, 2502 jiffies) -This is rare, but does happen from time to time in real life. +This is rare, but does happen from time to time in real life. It is also +possible for a zero-jiffy stall to be flagged in this case, depending +on how the stall warning and the grace-period initialization happen to +interact. Please note that it is not possible to entirely eliminate this +sort of false positive without resorting to things like stop_machine(), +which is overkill for this sort of problem. + +If the CONFIG_RCU_CPU_STALL_INFO kernel configuration parameter is set, +more information is printed with the stall-warning message, for example: + + INFO: rcu_preempt detected stall on CPU + 0: (63959 ticks this GP) idle=241/3fffffffffffffff/0 softirq=82/543 + (t=65000 jiffies) + +In kernels with CONFIG_RCU_FAST_NO_HZ, even more information is +printed: + + INFO: rcu_preempt detected stall on CPU + 0: (64628 ticks this GP) idle=dd5/3fffffffffffffff/0 softirq=82/543 last_accelerate: a345/d342 nonlazy_posted: 25 .D + (t=65000 jiffies) + +The "(64628 ticks this GP)" indicates that this CPU has taken more +than 64,000 scheduling-clock interrupts during the current stalled +grace period. If the CPU was not yet aware of the current grace +period (for example, if it was offline), then this part of the message +indicates how many grace periods behind the CPU is. + +The "idle=" portion of the message prints the dyntick-idle state. +The hex number before the first "/" is the low-order 12 bits of the +dynticks counter, which will have an even-numbered value if the CPU is +in dyntick-idle mode and an odd-numbered value otherwise. The hex +number between the two "/"s is the value of the nesting, which will +be a small positive number if in the idle loop and a very large positive +number (as shown above) otherwise. + +The "softirq=" portion of the message tracks the number of RCU softirq +handlers that the stalled CPU has executed. The number before the "/" +is the number that had executed since boot at the time that this CPU +last noted the beginning of a grace period, which might be the current +(stalled) grace period, or it might be some earlier grace period (for +example, if the CPU might have been in dyntick-idle mode for an extended +time period. The number after the "/" is the number that have executed +since boot until the current time. If this latter number stays constant +across repeated stall-warning messages, it is possible that RCU's softirq +handlers are no longer able to execute on this CPU. This can happen if +the stalled CPU is spinning with interrupts are disabled, or, in -rt +kernels, if a high-priority process is starving RCU's softirq handler. + +For CONFIG_RCU_FAST_NO_HZ kernels, the "last_accelerate:" prints the +low-order 16 bits (in hex) of the jiffies counter when this CPU last +invoked rcu_try_advance_all_cbs() from rcu_needs_cpu() or last invoked +rcu_accelerate_cbs() from rcu_prepare_for_idle(). The "nonlazy_posted:" +prints the number of non-lazy callbacks posted since the last call to +rcu_needs_cpu(). Finally, an "L" indicates that there are currently +no non-lazy callbacks ("." is printed otherwise, as shown above) and +"D" indicates that dyntick-idle processing is enabled ("." is printed +otherwise, for example, if disabled via the "nohz=" kernel boot parameter). + + +Multiple Warnings From One Stall + +If a stall lasts long enough, multiple stall-warning messages will be +printed for it. The second and subsequent messages are printed at +longer intervals, so that the time between (say) the first and second +message will be about three times the interval between the beginning +of the stall and the first message. + + +What Causes RCU CPU Stall Warnings? So your kernel printed an RCU CPU stall warning. The next question is "What caused it?" The following problems can result in RCU CPU stall @@ -104,7 +198,7 @@ o A CPU-bound real-time task in a CONFIG_PREEMPT_RT kernel that o A hardware or software issue shuts off the scheduler-clock interrupt on a CPU that is not in dyntick-idle mode. This problem really has happened, and seems to be most likely to - result in RCU CPU stall warnings for CONFIG_NO_HZ=n kernels. + result in RCU CPU stall warnings for CONFIG_NO_HZ_COMMON=n kernels. o A bug in the RCU implementation. @@ -128,4 +222,6 @@ is occurring, which will usually be in the function nearest the top of that portion of the stack which remains the same from trace to trace. If you can reliably trigger the stall, ftrace can be quite helpful. -RCU bugs can often be debugged with the help of CONFIG_RCU_TRACE. +RCU bugs can often be debugged with the help of CONFIG_RCU_TRACE +and with RCU's event tracing. For information on RCU's event tracing, +see include/trace/events/rcu.h. diff --git a/Documentation/RCU/torture.txt b/Documentation/RCU/torture.txt index d67068d0d2b..dac02a6219b 100644 --- a/Documentation/RCU/torture.txt +++ b/Documentation/RCU/torture.txt @@ -42,11 +42,31 @@ fqs_holdoff Holdoff time (in microseconds) between consecutive calls fqs_stutter Wait time (in seconds) between consecutive bursts of calls to force_quiescent_state(). +gp_normal Make the fake writers use normal synchronous grace-period + primitives. + +gp_exp Make the fake writers use expedited synchronous grace-period + primitives. If both gp_normal and gp_exp are set, or + if neither gp_normal nor gp_exp are set, then randomly + choose the primitive so that about 50% are normal and + 50% expedited. By default, neither are set, which + gives best overall test coverage. + irqreader Says to invoke RCU readers from irq level. This is currently done via timers. Defaults to "1" for variants of RCU that permit this. (Or, more accurately, variants of RCU that do -not- permit this know to ignore this variable.) +n_barrier_cbs If this is nonzero, RCU barrier testing will be conducted, + in which case n_barrier_cbs specifies the number of + RCU callbacks (and corresponding kthreads) to use for + this testing. The value cannot be negative. If you + specify this to be non-zero when torture_type indicates a + synchronous RCU implementation (one for which a member of + the synchronize_rcu() rather than the call_rcu() family is + used -- see the documentation for torture_type below), an + error will be reported and no testing will be carried out. + nfakewriters This is the number of RCU fake writer threads to run. Fake writer threads repeatedly use the synchronous "wait for current readers" function of the interface selected by @@ -69,6 +89,13 @@ onoff_interval CPU-hotplug operations regardless of what value is specified for onoff_interval. +onoff_holdoff The number of seconds to wait until starting CPU-hotplug + operations. This would normally only be used when + rcutorture was built into the kernel and started + automatically at boot time, in which case it is useful + in order to avoid confusing boot-time code with CPUs + coming and going. + shuffle_interval The number of seconds to keep the test threads affinitied to a particular subset of the CPUs, defaults to 3 seconds. @@ -79,6 +106,24 @@ shutdown_secs The number of seconds to run the test before terminating zero, which disables test termination and system shutdown. This capability is useful for automated testing. +stall_cpu The number of seconds that a CPU should be stalled while + within both an rcu_read_lock() and a preempt_disable(). + This stall happens only once per rcutorture run. + If you need multiple stalls, use modprobe and rmmod to + repeatedly run rcutorture. The default for stall_cpu + is zero, which prevents rcutorture from stalling a CPU. + + Note that attempts to rmmod rcutorture while the stall + is ongoing will hang, so be careful what value you + choose for this module parameter! In addition, too-large + values for stall_cpu might well induce failures and + warnings in other parts of the kernel. You have been + warned! + +stall_cpu_holdoff + The number of seconds to wait after rcutorture starts + before stalling a CPU. Defaults to 10 seconds. + stat_interval The number of seconds between output of torture statistics (via printk()). Regardless of the interval, statistics are printed when the module is unloaded. @@ -139,6 +184,9 @@ torture_type The type of RCU to test, with string values as follows: and synchronize_rcu_bh_expedited(). "srcu": srcu_read_lock(), srcu_read_unlock() and + call_srcu(). + + "srcu_sync": srcu_read_lock(), srcu_read_unlock() and synchronize_srcu(). "srcu_expedited": srcu_read_lock(), srcu_read_unlock() and @@ -163,7 +211,7 @@ OUTPUT The statistics output is as follows: rcu-torture:--- Start of test: nreaders=16 nfakewriters=4 stat_interval=30 verbose=0 test_no_idle_hz=1 shuffle_interval=3 stutter=5 irqreader=1 fqs_duration=0 fqs_holdoff=0 fqs_stutter=3 test_boost=1/0 test_boost_interval=7 test_boost_duration=4 - rcu-torture: rtc: (null) ver: 155441 tfle: 0 rta: 155441 rtaf: 8884 rtf: 155440 rtmbe: 0 rtbke: 0 rtbre: 0 rtbf: 0 rtb: 0 nt: 3055767 + rcu-torture: rtc: (null) ver: 155441 tfle: 0 rta: 155441 rtaf: 8884 rtf: 155440 rtmbe: 0 rtbe: 0 rtbke: 0 rtbre: 0 rtbf: 0 rtb: 0 nt: 3055767 rcu-torture: Reader Pipe: 727860534 34213 0 0 0 0 0 0 0 0 0 rcu-torture: Reader Batch: 727877838 17003 0 0 0 0 0 0 0 0 0 rcu-torture: Free-Block Circulation: 155440 155440 155440 155440 155440 155440 155440 155440 155440 155440 0 @@ -205,6 +253,9 @@ o "rtmbe": A non-zero value indicates that rcutorture believes that rcu_assign_pointer() and rcu_dereference() are not working correctly. This value should be zero. +o "rtbe": A non-zero value indicates that one of the rcu_barrier() + family of functions is not working correctly. + o "rtbke": rcutorture was unable to create the real-time kthreads used to force RCU priority inversion. This value should be zero. @@ -271,11 +322,13 @@ The following script may be used to torture RCU: #!/bin/sh modprobe rcutorture - sleep 100 + sleep 3600 rmmod rcutorture dmesg | grep torture: The output can be manually inspected for the error flag of "!!!". One could of course create a more elaborate script that automatically -checked for such errors. The "rmmod" command forces a "SUCCESS" or -"FAILURE" indication to be printk()ed. +checked for such errors. The "rmmod" command forces a "SUCCESS", +"FAILURE", or "RCU_HOTPLUG" indication to be printk()ed. The first +two are self-explanatory, while the last indicates that while there +were no RCU failures, CPU-hotplug problems were detected. diff --git a/Documentation/RCU/trace.txt b/Documentation/RCU/trace.txt index 49587abfc2f..910870b15ac 100644 --- a/Documentation/RCU/trace.txt +++ b/Documentation/RCU/trace.txt @@ -10,51 +10,63 @@ for rcutree and next for rcutiny. CONFIG_TREE_RCU and CONFIG_TREE_PREEMPT_RCU debugfs Files and Formats -These implementations of RCU provides several debugfs files under the +These implementations of RCU provide several debugfs directories under the top-level directory "rcu": -rcu/rcudata: +rcu/rcu_bh +rcu/rcu_preempt +rcu/rcu_sched + +Each directory contains files for the corresponding flavor of RCU. +Note that rcu/rcu_preempt is only present for CONFIG_TREE_PREEMPT_RCU. +For CONFIG_TREE_RCU, the RCU flavor maps onto the RCU-sched flavor, +so that activity for both appears in rcu/rcu_sched. + +In addition, the following file appears in the top-level directory: +rcu/rcutorture. This file displays rcutorture test progress. The output +of "cat rcu/rcutorture" looks as follows: + +rcutorture test sequence: 0 (test in progress) +rcutorture update version number: 615 + +The first line shows the number of rcutorture tests that have completed +since boot. If a test is currently running, the "(test in progress)" +string will appear as shown above. The second line shows the number of +update cycles that the current test has started, or zero if there is +no test in progress. + + +Within each flavor directory (rcu/rcu_bh, rcu/rcu_sched, and possibly +also rcu/rcu_preempt) the following files will be present: + +rcudata: Displays fields in struct rcu_data. -rcu/rcudata.csv: - Comma-separated values spreadsheet version of rcudata. -rcu/rcugp: +rcuexp: + Displays statistics for expedited grace periods. +rcugp: Displays grace-period counters. -rcu/rcuhier: +rcuhier: Displays the struct rcu_node hierarchy. -rcu/rcu_pending: +rcu_pending: Displays counts of the reasons rcu_pending() decided that RCU had work to do. -rcu/rcutorture: - Displays rcutorture test progress. -rcu/rcuboost: +rcuboost: Displays RCU boosting statistics. Only present if CONFIG_RCU_BOOST=y. -The output of "cat rcu/rcudata" looks as follows: - -rcu_sched: - 0 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=545/1/0 df=50 of=0 ri=0 ql=163 qs=NRW. kt=0/W/0 ktl=ebc3 b=10 ci=153737 co=0 ca=0 - 1 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=967/1/0 df=58 of=0 ri=0 ql=634 qs=NRW. kt=0/W/1 ktl=58c b=10 ci=191037 co=0 ca=0 - 2 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=1081/1/0 df=175 of=0 ri=0 ql=74 qs=N.W. kt=0/W/2 ktl=da94 b=10 ci=75991 co=0 ca=0 - 3 c=20942 g=20943 pq=1 pgp=20942 qp=1 dt=1846/0/0 df=404 of=0 ri=0 ql=0 qs=.... kt=0/W/3 ktl=d1cd b=10 ci=72261 co=0 ca=0 - 4 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=369/1/0 df=83 of=0 ri=0 ql=48 qs=N.W. kt=0/W/4 ktl=e0e7 b=10 ci=128365 co=0 ca=0 - 5 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=381/1/0 df=64 of=0 ri=0 ql=169 qs=NRW. kt=0/W/5 ktl=fb2f b=10 ci=164360 co=0 ca=0 - 6 c=20972 g=20973 pq=1 pgp=20973 qp=0 dt=1037/1/0 df=183 of=0 ri=0 ql=62 qs=N.W. kt=0/W/6 ktl=d2ad b=10 ci=65663 co=0 ca=0 - 7 c=20897 g=20897 pq=1 pgp=20896 qp=0 dt=1572/0/0 df=382 of=0 ri=0 ql=0 qs=.... kt=0/W/7 ktl=cf15 b=10 ci=75006 co=0 ca=0 -rcu_bh: - 0 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=545/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/0 ktl=ebc3 b=10 ci=0 co=0 ca=0 - 1 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=967/1/0 df=3 of=0 ri=1 ql=0 qs=.... kt=0/W/1 ktl=58c b=10 ci=151 co=0 ca=0 - 2 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=1081/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/2 ktl=da94 b=10 ci=0 co=0 ca=0 - 3 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=1846/0/0 df=8 of=0 ri=1 ql=0 qs=.... kt=0/W/3 ktl=d1cd b=10 ci=0 co=0 ca=0 - 4 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=369/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/4 ktl=e0e7 b=10 ci=0 co=0 ca=0 - 5 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=381/1/0 df=4 of=0 ri=1 ql=0 qs=.... kt=0/W/5 ktl=fb2f b=10 ci=0 co=0 ca=0 - 6 c=1480 g=1480 pq=1 pgp=1480 qp=0 dt=1037/1/0 df=6 of=0 ri=1 ql=0 qs=.... kt=0/W/6 ktl=d2ad b=10 ci=0 co=0 ca=0 - 7 c=1474 g=1474 pq=1 pgp=1473 qp=0 dt=1572/0/0 df=8 of=0 ri=1 ql=0 qs=.... kt=0/W/7 ktl=cf15 b=10 ci=0 co=0 ca=0 - -The first section lists the rcu_data structures for rcu_sched, the second -for rcu_bh. Note that CONFIG_TREE_PREEMPT_RCU kernels will have an -additional section for rcu_preempt. Each section has one line per CPU, -or eight for this 8-CPU system. The fields are as follows: +The output of "cat rcu/rcu_preempt/rcudata" looks as follows: + + 0!c=30455 g=30456 pq=1 qp=1 dt=126535/140000000000000/0 df=2002 of=4 ql=0/0 qs=N... b=10 ci=74572 nci=0 co=1131 ca=716 + 1!c=30719 g=30720 pq=1 qp=0 dt=132007/140000000000000/0 df=1874 of=10 ql=0/0 qs=N... b=10 ci=123209 nci=0 co=685 ca=982 + 2!c=30150 g=30151 pq=1 qp=1 dt=138537/140000000000000/0 df=1707 of=8 ql=0/0 qs=N... b=10 ci=80132 nci=0 co=1328 ca=1458 + 3 c=31249 g=31250 pq=1 qp=0 dt=107255/140000000000000/0 df=1749 of=6 ql=0/450 qs=NRW. b=10 ci=151700 nci=0 co=509 ca=622 + 4!c=29502 g=29503 pq=1 qp=1 dt=83647/140000000000000/0 df=965 of=5 ql=0/0 qs=N... b=10 ci=65643 nci=0 co=1373 ca=1521 + 5 c=31201 g=31202 pq=1 qp=1 dt=70422/0/0 df=535 of=7 ql=0/0 qs=.... b=10 ci=58500 nci=0 co=764 ca=698 + 6!c=30253 g=30254 pq=1 qp=1 dt=95363/140000000000000/0 df=780 of=5 ql=0/0 qs=N... b=10 ci=100607 nci=0 co=1414 ca=1353 + 7 c=31178 g=31178 pq=1 qp=0 dt=91536/0/0 df=547 of=4 ql=0/0 qs=.... b=10 ci=109819 nci=0 co=1115 ca=969 + +This file has one line per CPU, or eight for this 8-CPU system. +The fields are as follows: o The number at the beginning of each line is the CPU number. CPUs numbers followed by an exclamation mark are offline, @@ -64,11 +76,13 @@ o The number at the beginning of each line is the CPU number. substantially larger than the number of actual CPUs. o "c" is the count of grace periods that this CPU believes have - completed. Offlined CPUs and CPUs in dynticks idle mode may - lag quite a ways behind, for example, CPU 6 under "rcu_sched" - above, which has been offline through not quite 40,000 RCU grace - periods. It is not unusual to see CPUs lagging by thousands of - grace periods. + completed. Offlined CPUs and CPUs in dynticks idle mode may lag + quite a ways behind, for example, CPU 4 under "rcu_sched" above, + which has been offline through 16 RCU grace periods. It is not + unusual to see offline CPUs lagging by thousands of grace periods. + Note that although the grace-period number is an unsigned long, + it is printed out as a signed long to allow more human-friendly + representation near boot time. o "g" is the count of grace periods that this CPU believes have started. Again, offlined CPUs and CPUs in dynticks idle mode @@ -84,30 +98,25 @@ o "pq" indicates that this CPU has passed through a quiescent state CPU has not yet reported that fact, (2) some other CPU has not yet reported for this grace period, or (3) both. -o "pgp" indicates which grace period the last-observed quiescent - state for this CPU corresponds to. This is important for handling - the race between CPU 0 reporting an extended dynticks-idle - quiescent state for CPU 1 and CPU 1 suddenly waking up and - reporting its own quiescent state. If CPU 1 was the last CPU - for the current grace period, then the CPU that loses this race - will attempt to incorrectly mark CPU 1 as having checked in for - the next grace period! - o "qp" indicates that RCU still expects a quiescent state from this CPU. Offlined CPUs and CPUs in dyntick idle mode might well have qp=1, which is OK: RCU is still ignoring them. o "dt" is the current value of the dyntick counter that is incremented - when entering or leaving dynticks idle state, either by the - scheduler or by irq. This number is even if the CPU is in - dyntick idle mode and odd otherwise. The number after the first - "/" is the interrupt nesting depth when in dyntick-idle state, - or one greater than the interrupt-nesting depth otherwise. - The number after the second "/" is the NMI nesting depth. + when entering or leaving idle, either due to a context switch or + due to an interrupt. This number is even if the CPU is in idle + from RCU's viewpoint and odd otherwise. The number after the + first "/" is the interrupt nesting depth when in idle state, + or a large number added to the interrupt-nesting depth when + running a non-idle task. Some architectures do not accurately + count interrupt nesting when running in non-idle kernel context, + which can result in interesting anomalies such as negative + interrupt-nesting levels. The number after the second "/" + is the NMI nesting depth. o "df" is the number of times that some other CPU has forced a quiescent state on behalf of this CPU due to this CPU being in - dynticks-idle state. + idle state. o "of" is the number of times that some other CPU has forced a quiescent state on behalf of this CPU due to this CPU being @@ -119,14 +128,14 @@ o "of" is the number of times that some other CPU has forced a CPU is offline when it is really alive and kicking) is a fatal error, so it makes sense to err conservatively. -o "ri" is the number of times that RCU has seen fit to send a - reschedule IPI to this CPU in order to get it to report a - quiescent state. - o "ql" is the number of RCU callbacks currently residing on - this CPU. This is the total number of callbacks, regardless - of what state they are in (new, waiting for grace period to - start, waiting for grace period to end, ready to invoke). + this CPU. The first number is the number of "lazy" callbacks + that are known to RCU to only be freeing memory, and the number + after the "/" is the total number of callbacks, lazy or not. + These counters count callbacks regardless of what phase of + grace-period processing that they are in (new, waiting for + grace period to start, waiting for grace period to end, ready + to invoke). o "qs" gives an indication of the state of the callback queue with four characters: @@ -154,6 +163,43 @@ o "qs" gives an indication of the state of the callback queue If there are no callbacks in a given one of the above states, the corresponding character is replaced by ".". +o "b" is the batch limit for this CPU. If more than this number + of RCU callbacks is ready to invoke, then the remainder will + be deferred. + +o "ci" is the number of RCU callbacks that have been invoked for + this CPU. Note that ci+nci+ql is the number of callbacks that have + been registered in absence of CPU-hotplug activity. + +o "nci" is the number of RCU callbacks that have been offloaded from + this CPU. This will always be zero unless the kernel was built + with CONFIG_RCU_NOCB_CPU=y and the "rcu_nocbs=" kernel boot + parameter was specified. + +o "co" is the number of RCU callbacks that have been orphaned due to + this CPU going offline. These orphaned callbacks have been moved + to an arbitrarily chosen online CPU. + +o "ca" is the number of RCU callbacks that have been adopted by this + CPU due to other CPUs going offline. Note that ci+co-ca+ql is + the number of RCU callbacks registered on this CPU. + + +Kernels compiled with CONFIG_RCU_BOOST=y display the following from +/debug/rcu/rcu_preempt/rcudata: + + 0!c=12865 g=12866 pq=1 qp=1 dt=83113/140000000000000/0 df=288 of=11 ql=0/0 qs=N... kt=0/O ktl=944 b=10 ci=60709 nci=0 co=748 ca=871 + 1 c=14407 g=14408 pq=1 qp=0 dt=100679/140000000000000/0 df=378 of=7 ql=0/119 qs=NRW. kt=0/W ktl=9b6 b=10 ci=109740 nci=0 co=589 ca=485 + 2 c=14407 g=14408 pq=1 qp=0 dt=105486/0/0 df=90 of=9 ql=0/89 qs=NRW. kt=0/W ktl=c0c b=10 ci=83113 nci=0 co=533 ca=490 + 3 c=14407 g=14408 pq=1 qp=0 dt=107138/0/0 df=142 of=8 ql=0/188 qs=NRW. kt=0/W ktl=b96 b=10 ci=121114 nci=0 co=426 ca=290 + 4 c=14405 g=14406 pq=1 qp=1 dt=50238/0/0 df=706 of=7 ql=0/0 qs=.... kt=0/W ktl=812 b=10 ci=34929 nci=0 co=643 ca=114 + 5!c=14168 g=14169 pq=1 qp=0 dt=45465/140000000000000/0 df=161 of=11 ql=0/0 qs=N... kt=0/O ktl=b4d b=10 ci=47712 nci=0 co=677 ca=722 + 6 c=14404 g=14405 pq=1 qp=0 dt=59454/0/0 df=94 of=6 ql=0/0 qs=.... kt=0/W ktl=e57 b=10 ci=55597 nci=0 co=701 ca=811 + 7 c=14407 g=14408 pq=1 qp=1 dt=68850/0/0 df=31 of=8 ql=0/0 qs=.... kt=0/W ktl=14bd b=10 ci=77475 nci=0 co=508 ca=1042 + +This is similar to the output discussed above, but contains the following +additional fields: + o "kt" is the per-CPU kernel-thread state. The digit preceding the first slash is zero if there is no work pending and 1 otherwise. The character between the first pair of slashes is @@ -188,35 +234,51 @@ o "ktl" is the low-order 16 bits (in hexadecimal) of the count of This field is displayed only for CONFIG_RCU_BOOST kernels. -o "b" is the batch limit for this CPU. If more than this number - of RCU callbacks is ready to invoke, then the remainder will - be deferred. -o "ci" is the number of RCU callbacks that have been invoked for - this CPU. Note that ci+ql is the number of callbacks that have - been registered in absence of CPU-hotplug activity. +The output of "cat rcu/rcu_preempt/rcuexp" looks as follows: -o "co" is the number of RCU callbacks that have been orphaned due to - this CPU going offline. These orphaned callbacks have been moved - to an arbitrarily chosen online CPU. +s=21872 d=21872 w=0 tf=0 wd1=0 wd2=0 n=0 sc=21872 dt=21872 dl=0 dx=21872 -o "ca" is the number of RCU callbacks that have been adopted due to - other CPUs going offline. Note that ci+co-ca+ql is the number of - RCU callbacks registered on this CPU. +These fields are as follows: -There is also an rcu/rcudata.csv file with the same information in -comma-separated-variable spreadsheet format. +o "s" is the starting sequence number. +o "d" is the ending sequence number. When the starting and ending + numbers differ, there is an expedited grace period in progress. -The output of "cat rcu/rcugp" looks as follows: +o "w" is the number of times that the sequence numbers have been + in danger of wrapping. -rcu_sched: completed=33062 gpnum=33063 -rcu_bh: completed=464 gpnum=464 +o "tf" is the number of times that contention has resulted in a + failure to begin an expedited grace period. -Again, this output is for both "rcu_sched" and "rcu_bh". Note that -kernels built with CONFIG_TREE_PREEMPT_RCU will have an additional -"rcu_preempt" line. The fields are taken from the rcu_state structure, -and are as follows: +o "wd1" and "wd2" are the number of times that an attempt to + start an expedited grace period found that someone else had + completed an expedited grace period that satisfies the + attempted request. "Our work is done." + +o "n" is number of times that contention was so great that + the request was demoted from an expedited grace period to + a normal grace period. + +o "sc" is the number of times that the attempt to start a + new expedited grace period succeeded. + +o "dt" is the number of times that we attempted to update + the "d" counter. + +o "dl" is the number of times that we failed to update the "d" + counter. + +o "dx" is the number of times that we succeeded in updating + the "d" counter. + + +The output of "cat rcu/rcu_preempt/rcugp" looks as follows: + +completed=31249 gpnum=31250 age=1 max=18 + +These fields are taken from the rcu_state structure, and are as follows: o "completed" is the number of grace periods that have completed. It is comparable to the "c" field from rcu/rcudata in that a @@ -224,44 +286,42 @@ o "completed" is the number of grace periods that have completed. that the corresponding RCU grace period has completed. o "gpnum" is the number of grace periods that have started. It is - comparable to the "g" field from rcu/rcudata in that a CPU - whose "g" field matches the value of "gpnum" is aware that the - corresponding RCU grace period has started. + similarly comparable to the "g" field from rcu/rcudata in that + a CPU whose "g" field matches the value of "gpnum" is aware that + the corresponding RCU grace period has started. + + If these two fields are equal, then there is no grace period + in progress, in other words, RCU is idle. On the other hand, + if the two fields differ (as they are above), then an RCU grace + period is in progress. - If these two fields are equal (as they are for "rcu_bh" above), - then there is no grace period in progress, in other words, RCU - is idle. On the other hand, if the two fields differ (as they - do for "rcu_sched" above), then an RCU grace period is in progress. +o "age" is the number of jiffies that the current grace period + has extended for, or zero if there is no grace period currently + in effect. +o "max" is the age in jiffies of the longest-duration grace period + thus far. -The output of "cat rcu/rcuhier" looks as follows, with very long lines: +The output of "cat rcu/rcu_preempt/rcuhier" looks as follows: -c=6902 g=6903 s=2 jfq=3 j=72c7 nfqs=13142/nfqsng=0(13142) fqlh=6 -1/1 ..>. 0:127 ^0 -3/3 ..>. 0:35 ^0 0/0 ..>. 36:71 ^1 0/0 ..>. 72:107 ^2 0/0 ..>. 108:127 ^3 -3/3f ..>. 0:5 ^0 2/3 ..>. 6:11 ^1 0/0 ..>. 12:17 ^2 0/0 ..>. 18:23 ^3 0/0 ..>. 24:29 ^4 0/0 ..>. 30:35 ^5 0/0 ..>. 36:41 ^0 0/0 ..>. 42:47 ^1 0/0 ..>. 48:53 ^2 0/0 ..>. 54:59 ^3 0/0 ..>. 60:65 ^4 0/0 ..>. 66:71 ^5 0/0 ..>. 72:77 ^0 0/0 ..>. 78:83 ^1 0/0 ..>. 84:89 ^2 0/0 ..>. 90:95 ^3 0/0 ..>. 96:101 ^4 0/0 ..>. 102:107 ^5 0/0 ..>. 108:113 ^0 0/0 ..>. 114:119 ^1 0/0 ..>. 120:125 ^2 0/0 ..>. 126:127 ^3 -rcu_bh: -c=-226 g=-226 s=1 jfq=-5701 j=72c7 nfqs=88/nfqsng=0(88) fqlh=0 -0/1 ..>. 0:127 ^0 -0/3 ..>. 0:35 ^0 0/0 ..>. 36:71 ^1 0/0 ..>. 72:107 ^2 0/0 ..>. 108:127 ^3 -0/3f ..>. 0:5 ^0 0/3 ..>. 6:11 ^1 0/0 ..>. 12:17 ^2 0/0 ..>. 18:23 ^3 0/0 ..>. 24:29 ^4 0/0 ..>. 30:35 ^5 0/0 ..>. 36:41 ^0 0/0 ..>. 42:47 ^1 0/0 ..>. 48:53 ^2 0/0 ..>. 54:59 ^3 0/0 ..>. 60:65 ^4 0/0 ..>. 66:71 ^5 0/0 ..>. 72:77 ^0 0/0 ..>. 78:83 ^1 0/0 ..>. 84:89 ^2 0/0 ..>. 90:95 ^3 0/0 ..>. 96:101 ^4 0/0 ..>. 102:107 ^5 0/0 ..>. 108:113 ^0 0/0 ..>. 114:119 ^1 0/0 ..>. 120:125 ^2 0/0 ..>. 126:127 ^3 +c=14407 g=14408 s=0 jfq=2 j=c863 nfqs=12040/nfqsng=0(12040) fqlh=1051 oqlen=0/0 +3/3 ..>. 0:7 ^0 +e/e ..>. 0:3 ^0 d/d ..>. 4:7 ^1 -This is once again split into "rcu_sched" and "rcu_bh" portions, -and CONFIG_TREE_PREEMPT_RCU kernels will again have an additional -"rcu_preempt" section. The fields are as follows: +The fields are as follows: -o "c" is exactly the same as "completed" under rcu/rcugp. +o "c" is exactly the same as "completed" under rcu/rcu_preempt/rcugp. -o "g" is exactly the same as "gpnum" under rcu/rcugp. +o "g" is exactly the same as "gpnum" under rcu/rcu_preempt/rcugp. -o "s" is the "signaled" state that drives force_quiescent_state()'s +o "s" is the current state of the force_quiescent_state() state machine. o "jfq" is the number of jiffies remaining for this grace period before force_quiescent_state() is invoked to help push things - along. Note that CPUs in dyntick-idle mode throughout the grace - period will not report on their own, but rather must be check by - some other CPU via force_quiescent_state(). + along. Note that CPUs in idle mode throughout the grace period + will not report on their own, but rather must be check by some + other CPU via force_quiescent_state(). o "j" is the low-order four hex digits of the jiffies counter. Yes, Paul did run into a number of problems that turned out to @@ -272,7 +332,8 @@ o "nfqs" is the number of calls to force_quiescent_state() since o "nfqsng" is the number of useless calls to force_quiescent_state(), where there wasn't actually a grace period active. This can - happen due to races. The number in parentheses is the difference + no longer happen due to grace-period processing being pushed + into a kthread. The number in parentheses is the difference between "nfqs" and "nfqsng", or the number of times that force_quiescent_state() actually did some real work. @@ -280,28 +341,27 @@ o "fqlh" is the number of calls to force_quiescent_state() that exited immediately (without even being counted in nfqs above) due to contention on ->fqslock. -o Each element of the form "1/1 0:127 ^0" represents one struct - rcu_node. Each line represents one level of the hierarchy, from - root to leaves. It is best to think of the rcu_data structures - as forming yet another level after the leaves. Note that there - might be either one, two, or three levels of rcu_node structures, - depending on the relationship between CONFIG_RCU_FANOUT and - CONFIG_NR_CPUS. +o Each element of the form "3/3 ..>. 0:7 ^0" represents one rcu_node + structure. Each line represents one level of the hierarchy, + from root to leaves. It is best to think of the rcu_data + structures as forming yet another level after the leaves. + Note that there might be either one, two, three, or even four + levels of rcu_node structures, depending on the relationship + between CONFIG_RCU_FANOUT, CONFIG_RCU_FANOUT_LEAF (possibly + adjusted using the rcu_fanout_leaf kernel boot parameter), and + CONFIG_NR_CPUS (possibly adjusted using the nr_cpu_ids count of + possible CPUs for the booting hardware). o The numbers separated by the "/" are the qsmask followed by the qsmaskinit. The qsmask will have one bit - set for each entity in the next lower level that - has not yet checked in for the current grace period. + set for each entity in the next lower level that has + not yet checked in for the current grace period ("e" + indicating CPUs 5, 6, and 7 in the example above). The qsmaskinit will have one bit for each entity that is currently expected to check in during each grace period. The value of qsmaskinit is assigned to that of qsmask at the beginning of each grace period. - For example, for "rcu_sched", the qsmask of the first - entry of the lowest level is 0x14, meaning that we - are still waiting for CPUs 2 and 4 to check in for the - current grace period. - o The characters separated by the ">" indicate the state of the blocked-tasks lists. A "G" preceding the ">" indicates that at least one task blocked in an RCU @@ -316,48 +376,39 @@ o Each element of the form "1/1 0:127 ^0" represents one struct A "." character appears if the corresponding condition does not hold, so that "..>." indicates that no tasks are blocked. In contrast, "GE>T" indicates maximal - inconvenience from blocked tasks. + inconvenience from blocked tasks. CONFIG_TREE_RCU + builds of the kernel will always show "..>.". o The numbers separated by the ":" are the range of CPUs served by this struct rcu_node. This can be helpful in working out how the hierarchy is wired together. - For example, the first entry at the lowest level shows - "0:5", indicating that it covers CPUs 0 through 5. + For example, the example rcu_node structure shown above + has "0:7", indicating that it covers CPUs 0 through 7. o The number after the "^" indicates the bit in the - next higher level rcu_node structure that this - rcu_node structure corresponds to. - - For example, the first entry at the lowest level shows - "^0", indicating that it corresponds to bit zero in - the first entry at the middle level. - - -The output of "cat rcu/rcu_pending" looks as follows: - -rcu_sched: - 0 np=255892 qsp=53936 rpq=85 cbr=0 cng=14417 gpc=10033 gps=24320 nf=6445 nn=146741 - 1 np=261224 qsp=54638 rpq=33 cbr=0 cng=25723 gpc=16310 gps=2849 nf=5912 nn=155792 - 2 np=237496 qsp=49664 rpq=23 cbr=0 cng=2762 gpc=45478 gps=1762 nf=1201 nn=136629 - 3 np=236249 qsp=48766 rpq=98 cbr=0 cng=286 gpc=48049 gps=1218 nf=207 nn=137723 - 4 np=221310 qsp=46850 rpq=7 cbr=0 cng=26 gpc=43161 gps=4634 nf=3529 nn=123110 - 5 np=237332 qsp=48449 rpq=9 cbr=0 cng=54 gpc=47920 gps=3252 nf=201 nn=137456 - 6 np=219995 qsp=46718 rpq=12 cbr=0 cng=50 gpc=42098 gps=6093 nf=4202 nn=120834 - 7 np=249893 qsp=49390 rpq=42 cbr=0 cng=72 gpc=38400 gps=17102 nf=41 nn=144888 -rcu_bh: - 0 np=146741 qsp=1419 rpq=6 cbr=0 cng=6 gpc=0 gps=0 nf=2 nn=145314 - 1 np=155792 qsp=12597 rpq=3 cbr=0 cng=0 gpc=4 gps=8 nf=3 nn=143180 - 2 np=136629 qsp=18680 rpq=1 cbr=0 cng=0 gpc=7 gps=6 nf=0 nn=117936 - 3 np=137723 qsp=2843 rpq=0 cbr=0 cng=0 gpc=10 gps=7 nf=0 nn=134863 - 4 np=123110 qsp=12433 rpq=0 cbr=0 cng=0 gpc=4 gps=2 nf=0 nn=110671 - 5 np=137456 qsp=4210 rpq=1 cbr=0 cng=0 gpc=6 gps=5 nf=0 nn=133235 - 6 np=120834 qsp=9902 rpq=2 cbr=0 cng=0 gpc=6 gps=3 nf=2 nn=110921 - 7 np=144888 qsp=26336 rpq=0 cbr=0 cng=0 gpc=8 gps=2 nf=0 nn=118542 - -As always, this is once again split into "rcu_sched" and "rcu_bh" -portions, with CONFIG_TREE_PREEMPT_RCU kernels having an additional -"rcu_preempt" section. The fields are as follows: + next higher level rcu_node structure that this rcu_node + structure corresponds to. For example, the "d/d ..>. 4:7 + ^1" has a "1" in this position, indicating that it + corresponds to the "1" bit in the "3" shown in the + "3/3 ..>. 0:7 ^0" entry on the next level up. + + +The output of "cat rcu/rcu_sched/rcu_pending" looks as follows: + + 0!np=26111 qsp=29 rpq=5386 cbr=1 cng=570 gpc=3674 gps=577 nn=15903 ndw=0 + 1!np=28913 qsp=35 rpq=6097 cbr=1 cng=448 gpc=3700 gps=554 nn=18113 ndw=0 + 2!np=32740 qsp=37 rpq=6202 cbr=0 cng=476 gpc=4627 gps=546 nn=20889 ndw=0 + 3 np=23679 qsp=22 rpq=5044 cbr=1 cng=415 gpc=3403 gps=347 nn=14469 ndw=0 + 4!np=30714 qsp=4 rpq=5574 cbr=0 cng=528 gpc=3931 gps=639 nn=20042 ndw=0 + 5 np=28910 qsp=2 rpq=5246 cbr=0 cng=428 gpc=4105 gps=709 nn=18422 ndw=0 + 6!np=38648 qsp=5 rpq=7076 cbr=0 cng=840 gpc=4072 gps=961 nn=25699 ndw=0 + 7 np=37275 qsp=2 rpq=6873 cbr=0 cng=868 gpc=3416 gps=971 nn=25147 ndw=0 + +The fields are as follows: + +o The leading number is the CPU number, with "!" indicating + an offline CPU. o "np" is the number of times that __rcu_pending() has been invoked for the corresponding flavor of RCU. @@ -381,49 +432,27 @@ o "gpc" is the number of times that an old grace period had o "gps" is the number of times that a new grace period had started, but this CPU was not yet aware of it. -o "nf" is the number of times that this CPU suspected that the - current grace period had run for too long, and thus needed to - be forced. - - Please note that "forcing" consists of sending resched IPIs - to holdout CPUs. If that CPU really still is in an old RCU - read-side critical section, then we really do have to wait for it. - The assumption behing "forcing" is that the CPU is not still in - an old RCU read-side critical section, but has not yet responded - for some other reason. - -o "nn" is the number of times that this CPU needed nothing. Alert - readers will note that the rcu "nn" number for a given CPU very - closely matches the rcu_bh "np" number for that same CPU. This - is due to short-circuit evaluation in rcu_pending(). - - -The output of "cat rcu/rcutorture" looks as follows: - -rcutorture test sequence: 0 (test in progress) -rcutorture update version number: 615 +o "ndw" is the number of times that a wakeup of an rcuo + callback-offload kthread had to be deferred in order to avoid + deadlock. -The first line shows the number of rcutorture tests that have completed -since boot. If a test is currently running, the "(test in progress)" -string will appear as shown above. The second line shows the number of -update cycles that the current test has started, or zero if there is -no test in progress. +o "nn" is the number of times that this CPU needed nothing. The output of "cat rcu/rcuboost" looks as follows: -0:5 tasks=.... kt=W ntb=0 neb=0 nnb=0 j=2f95 bt=300f - balk: nt=0 egt=989 bt=0 nb=0 ny=0 nos=16 -6:7 tasks=.... kt=W ntb=0 neb=0 nnb=0 j=2f95 bt=300f - balk: nt=0 egt=225 bt=0 nb=0 ny=0 nos=6 +0:3 tasks=.... kt=W ntb=0 neb=0 nnb=0 j=c864 bt=c894 + balk: nt=0 egt=4695 bt=0 nb=0 ny=56 nos=0 +4:7 tasks=.... kt=W ntb=0 neb=0 nnb=0 j=c864 bt=c894 + balk: nt=0 egt=6541 bt=0 nb=0 ny=126 nos=0 This information is output only for rcu_preempt. Each two-line entry -corresponds to a leaf rcu_node strcuture. The fields are as follows: +corresponds to a leaf rcu_node structure. The fields are as follows: o "n:m" is the CPU-number range for the corresponding two-line entry. In the sample output above, the first entry covers - CPUs zero through five and the second entry covers CPUs 6 - and 7. + CPUs zero through three and the second entry covers CPUs four + through seven. o "tasks=TNEB" gives the state of the various segments of the rnp->blocked_tasks list: @@ -505,113 +534,21 @@ o "nos" counts the number of times we balked for other reasons, e.g., the grace period ended first. -CONFIG_TINY_RCU and CONFIG_TINY_PREEMPT_RCU debugfs Files and Formats +CONFIG_TINY_RCU debugfs Files and Formats These implementations of RCU provides a single debugfs file under the top-level directory RCU, namely rcu/rcudata, which displays fields in -rcu_bh_ctrlblk, rcu_sched_ctrlblk and, for CONFIG_TINY_PREEMPT_RCU, -rcu_preempt_ctrlblk. +rcu_bh_ctrlblk and rcu_sched_ctrlblk. The output of "cat rcu/rcudata" is as follows: -rcu_preempt: qlen=24 gp=1097669 g197/p197/c197 tasks=... - ttb=. btg=no ntb=184 neb=0 nnb=183 j=01f7 bt=0274 - normal balk: nt=1097669 gt=0 bt=371 b=0 ny=25073378 nos=0 - exp balk: bt=0 nos=0 rcu_sched: qlen: 0 rcu_bh: qlen: 0 -This is split into rcu_preempt, rcu_sched, and rcu_bh sections, with the -rcu_preempt section appearing only in CONFIG_TINY_PREEMPT_RCU builds. -The last three lines of the rcu_preempt section appear only in -CONFIG_RCU_BOOST kernel builds. The fields are as follows: +This is split into rcu_sched and rcu_bh sections. The field is as +follows: o "qlen" is the number of RCU callbacks currently waiting either for an RCU grace period or waiting to be invoked. This is the only field present for rcu_sched and rcu_bh, due to the short-circuiting of grace period in those two cases. - -o "gp" is the number of grace periods that have completed. - -o "g197/p197/c197" displays the grace-period state, with the - "g" number being the number of grace periods that have started - (mod 256), the "p" number being the number of grace periods - that the CPU has responded to (also mod 256), and the "c" - number being the number of grace periods that have completed - (once again mode 256). - - Why have both "gp" and "g"? Because the data flowing into - "gp" is only present in a CONFIG_RCU_TRACE kernel. - -o "tasks" is a set of bits. The first bit is "T" if there are - currently tasks that have recently blocked within an RCU - read-side critical section, the second bit is "N" if any of the - aforementioned tasks are blocking the current RCU grace period, - and the third bit is "E" if any of the aforementioned tasks are - blocking the current expedited grace period. Each bit is "." - if the corresponding condition does not hold. - -o "ttb" is a single bit. It is "B" if any of the blocked tasks - need to be priority boosted and "." otherwise. - -o "btg" indicates whether boosting has been carried out during - the current grace period, with "exp" indicating that boosting - is in progress for an expedited grace period, "no" indicating - that boosting has not yet started for a normal grace period, - "begun" indicating that boosting has bebug for a normal grace - period, and "done" indicating that boosting has completed for - a normal grace period. - -o "ntb" is the total number of tasks subjected to RCU priority boosting - periods since boot. - -o "neb" is the number of expedited grace periods that have had - to resort to RCU priority boosting since boot. - -o "nnb" is the number of normal grace periods that have had - to resort to RCU priority boosting since boot. - -o "j" is the low-order 16 bits of the jiffies counter in hexadecimal. - -o "bt" is the low-order 16 bits of the value that the jiffies counter - will have at the next time that boosting is scheduled to begin. - -o In the line beginning with "normal balk", the fields are as follows: - - o "nt" is the number of times that the system balked from - boosting because there were no blocked tasks to boost. - Note that the system will balk from boosting even if the - grace period is overdue when the currently running task - is looping within an RCU read-side critical section. - There is no point in boosting in this case, because - boosting a running task won't make it run any faster. - - o "gt" is the number of times that the system balked - from boosting because, although there were blocked tasks, - none of them were preventing the current grace period - from completing. - - o "bt" is the number of times that the system balked - from boosting because boosting was already in progress. - - o "b" is the number of times that the system balked from - boosting because boosting had already completed for - the grace period in question. - - o "ny" is the number of times that the system balked from - boosting because it was not yet time to start boosting - the grace period in question. - - o "nos" is the number of times that the system balked from - boosting for inexplicable ("not otherwise specified") - reasons. This can actually happen due to races involving - increments of the jiffies counter. - -o In the line beginning with "exp balk", the fields are as follows: - - o "bt" is the number of times that the system balked from - boosting because there were no blocked tasks to boost. - - o "nos" is the number of times that the system balked from - boosting for inexplicable ("not otherwise specified") - reasons. diff --git a/Documentation/RCU/whatisRCU.txt b/Documentation/RCU/whatisRCU.txt index 6bbe8dcdc3d..49b8551a3b6 100644 --- a/Documentation/RCU/whatisRCU.txt +++ b/Documentation/RCU/whatisRCU.txt @@ -265,9 +265,9 @@ rcu_dereference() rcu_read_lock(); p = rcu_dereference(head.next); rcu_read_unlock(); - x = p->address; + x = p->address; /* BUG!!! */ rcu_read_lock(); - y = p->data; + y = p->data; /* BUG!!! */ rcu_read_unlock(); Holding a reference from one RCU read-side critical section @@ -326,11 +326,11 @@ used as follows: a. synchronize_rcu() rcu_read_lock() / rcu_read_unlock() call_rcu() rcu_dereference() -b. call_rcu_bh() rcu_read_lock_bh() / rcu_read_unlock_bh() - rcu_dereference_bh() +b. synchronize_rcu_bh() rcu_read_lock_bh() / rcu_read_unlock_bh() + call_rcu_bh() rcu_dereference_bh() c. synchronize_sched() rcu_read_lock_sched() / rcu_read_unlock_sched() - preempt_disable() / preempt_enable() + call_rcu_sched() preempt_disable() / preempt_enable() local_irq_save() / local_irq_restore() hardirq enter / hardirq exit NMI enter / NMI exit @@ -499,6 +499,8 @@ The foo_reclaim() function might appear as follows: { struct foo *fp = container_of(rp, struct foo, rcu); + foo_cleanup(fp->a); + kfree(fp); } @@ -521,6 +523,12 @@ o Use call_rcu() -after- removing a data element from an read-side critical sections that might be referencing that data item. +If the callback for call_rcu() is not doing anything more than calling +kfree() on the structure, you can use kfree_rcu() instead of call_rcu() +to avoid having to write your own callback: + + kfree_rcu(old_fp, rcu); + Again, see checklist.txt for additional rules governing the use of RCU. @@ -773,8 +781,8 @@ a single atomic update, converting to RCU will require special care. Also, the presence of synchronize_rcu() means that the RCU version of delete() can now block. If this is a problem, there is a callback-based -mechanism that never blocks, namely call_rcu(), that can be used in -place of synchronize_rcu(). +mechanism that never blocks, namely call_rcu() or kfree_rcu(), that can +be used in place of synchronize_rcu(). 7. FULL LIST OF RCU APIs @@ -786,12 +794,22 @@ in docbook. Here is the list, by category. RCU list traversal: + list_entry_rcu + list_first_entry_rcu + list_next_rcu list_for_each_entry_rcu + list_for_each_entry_continue_rcu + hlist_first_rcu + hlist_next_rcu + hlist_pprev_rcu hlist_for_each_entry_rcu + hlist_for_each_entry_rcu_bh + hlist_for_each_entry_continue_rcu + hlist_for_each_entry_continue_rcu_bh + hlist_nulls_first_rcu hlist_nulls_for_each_entry_rcu - - list_for_each_continue_rcu (to be deprecated in favor of new - list_for_each_entry_continue_rcu) + hlist_bl_first_rcu + hlist_bl_for_each_entry_rcu RCU pointer/list update: @@ -800,27 +818,38 @@ RCU pointer/list update: list_add_tail_rcu list_del_rcu list_replace_rcu - hlist_del_rcu hlist_add_after_rcu hlist_add_before_rcu hlist_add_head_rcu + hlist_del_rcu + hlist_del_init_rcu hlist_replace_rcu list_splice_init_rcu() + hlist_nulls_del_init_rcu + hlist_nulls_del_rcu + hlist_nulls_add_head_rcu + hlist_bl_add_head_rcu + hlist_bl_del_init_rcu + hlist_bl_del_rcu + hlist_bl_set_first_rcu RCU: Critical sections Grace period Barrier rcu_read_lock synchronize_net rcu_barrier rcu_read_unlock synchronize_rcu rcu_dereference synchronize_rcu_expedited - call_rcu - + rcu_read_lock_held call_rcu + rcu_dereference_check kfree_rcu + rcu_dereference_protected bh: Critical sections Grace period Barrier rcu_read_lock_bh call_rcu_bh rcu_barrier_bh rcu_read_unlock_bh synchronize_rcu_bh rcu_dereference_bh synchronize_rcu_bh_expedited - + rcu_dereference_bh_check + rcu_dereference_bh_protected + rcu_read_lock_bh_held sched: Critical sections Grace period Barrier @@ -828,16 +857,21 @@ sched: Critical sections Grace period Barrier rcu_read_unlock_sched call_rcu_sched [preempt_disable] synchronize_sched_expedited [and friends] + rcu_read_lock_sched_notrace + rcu_read_unlock_sched_notrace rcu_dereference_sched + rcu_dereference_sched_check + rcu_dereference_sched_protected + rcu_read_lock_sched_held SRCU: Critical sections Grace period Barrier - srcu_read_lock synchronize_srcu N/A - srcu_read_unlock synchronize_srcu_expedited - srcu_read_lock_raw - srcu_read_unlock_raw - srcu_dereference + srcu_read_lock synchronize_srcu srcu_barrier + srcu_read_unlock call_srcu + srcu_dereference synchronize_srcu_expedited + srcu_dereference_check + srcu_read_lock_held SRCU: Initialization/cleanup init_srcu_struct @@ -845,9 +879,13 @@ SRCU: Initialization/cleanup All: lockdep-checked RCU-protected pointer access - rcu_dereference_check - rcu_dereference_protected + rcu_access_index rcu_access_pointer + rcu_dereference_index_check + rcu_dereference_raw + rcu_lockdep_assert + rcu_sleep_check + RCU_NONIDLE See the comment headers in the source code (or the docbook generated from them) for more information. @@ -858,32 +896,31 @@ list can be helpful: a. Will readers need to block? If so, you need SRCU. -b. Is it necessary to start a read-side critical section in a - hardirq handler or exception handler, and then to complete - this read-side critical section in the task that was - interrupted? If so, you need SRCU's srcu_read_lock_raw() and - srcu_read_unlock_raw() primitives. - -c. What about the -rt patchset? If readers would need to block +b. What about the -rt patchset? If readers would need to block in an non-rt kernel, you need SRCU. If readers would block in a -rt kernel, but not in a non-rt kernel, SRCU is not necessary. -d. Do you need to treat NMI handlers, hardirq handlers, +c. Do you need to treat NMI handlers, hardirq handlers, and code segments with preemption disabled (whether via preempt_disable(), local_irq_save(), local_bh_disable(), or some other mechanism) as if they were explicit RCU readers? - If so, you need RCU-sched. + If so, RCU-sched is the only choice that will work for you. -e. Do you need RCU grace periods to complete even in the face +d. Do you need RCU grace periods to complete even in the face of softirq monopolization of one or more of the CPUs? For example, is your code subject to network-based denial-of-service attacks? If so, you need RCU-bh. -f. Is your workload too update-intensive for normal use of +e. Is your workload too update-intensive for normal use of RCU, but inappropriate for other synchronization mechanisms? If so, consider SLAB_DESTROY_BY_RCU. But please be careful! +f. Do you need read-side critical sections that are respected + even though they are in the middle of the idle loop, during + user-mode execution, or on an offlined CPU? If so, SRCU is the + only choice that will work for you. + g. Otherwise, use RCU. Of course, this all assumes that you have determined that RCU is in fact |