diff options
| author | Paul E. McKenney <paulmck@linux.vnet.ibm.com> | 2009-11-02 13:52:28 -0800 |
|---|---|---|
| committer | Greg Kroah-Hartman <gregkh@suse.de> | 2009-12-18 14:03:00 -0800 |
| commit | 1a0d7df3352d775ae3e034b1f2658ea493055a30 (patch) | |
| tree | 32a4ce75cdfa07a7f2308b660507d2f21e98e9de /kernel/rcutree.h | |
| parent | 1aab3863fee66343b420cb1e5d268c3ee8da27a6 (diff) | |
rcu: Fix synchronization for rcu_process_gp_end() uses of ->completed counter
commit d09b62dfa336447c52a5ec9bb88adbc479b0f3b8 upstream.
Impose a clear locking design on the rcu_process_gp_end()
function's use of the ->completed counter. This is done by
creating a ->completed field in the rcu_node structure, which
can safely be accessed under the protection of that structure's
lock. Performance and scalability are maintained by using a
form of double-checked locking, so that rcu_process_gp_end()
only acquires the leaf rcu_node structure's ->lock if a grace
period has recently ended.
This fix reduces rcutorture failure rate by at least two orders
of magnitude under heavy stress with force_quiescent_state()
being invoked artificially often. Without this fix,
unsynchronized access to the ->completed field can cause
rcu_process_gp_end() to advance callbacks whose grace period has
not yet expired. (Bad idea!)
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: laijs@cn.fujitsu.com
Cc: dipankar@in.ibm.com
Cc: mathieu.desnoyers@polymtl.ca
Cc: josh@joshtriplett.org
Cc: dvhltc@us.ibm.com
Cc: niv@us.ibm.com
Cc: peterz@infradead.org
Cc: rostedt@goodmis.org
Cc: Valdis.Kletnieks@vt.edu
Cc: dhowells@redhat.com
LKML-Reference: <12571987494069-git-send-email->
Signed-off-by: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Diffstat (limited to 'kernel/rcutree.h')
| -rw-r--r-- | kernel/rcutree.h | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/kernel/rcutree.h b/kernel/rcutree.h index 8a4c1650ad8..c1891c3cae6 100644 --- a/kernel/rcutree.h +++ b/kernel/rcutree.h @@ -84,6 +84,9 @@ struct rcu_node { long gpnum; /* Current grace period for this node. */ /* This will either be equal to or one */ /* behind the root rcu_node's gpnum. */ + long completed; /* Last grace period completed for this node. */ + /* This will either be equal to or one */ + /* behind the root rcu_node's gpnum. */ unsigned long qsmask; /* CPUs or groups that need to switch in */ /* order for current grace period to proceed.*/ /* In leaf rcu_node, each bit corresponds to */ |