aboutsummaryrefslogtreecommitdiff
path: root/arch/x86
AgeCommit message (Collapse)Author
2010-08-02KVM: MMU: invalidate and flush on spte small->large page size changeMarcelo Tosatti
Always invalidate spte and flush TLBs when changing page size, to make sure different sized translations for the same address are never cached in a CPU's TLB. Currently the only case where this occurs is when a non-leaf spte pointer is overwritten by a leaf, large spte entry. This can happen after dirty logging is disabled on a memslot, for example. Noticed by Andrea. KVM-Stable-Tag Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Avi Kivity <avi@redhat.com> (cherry picked from commit 3be2264be3c00865116f997dc53ebcc90fe7fc4b)
2010-08-02KVM: SVM: Implement workaround for Erratum 383Joerg Roedel
This patch implements a workaround for AMD erratum 383 into KVM. Without this erratum fix it is possible for a guest to kill the host machine. This patch implements the suggested workaround for hypervisors which will be published by the next revision guide update. [jan: fix overflow warning on i386] [xiao: fix unused variable warning] Cc: stable@kernel.org Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com> Signed-off-by: Avi Kivity <avi@redhat.com> (cherry picked from commit 67ec66077799f2fef84b21a643912b179c422281)
2010-08-02KVM: SVM: Handle MCEs early in the vmexit processJoerg Roedel
This patch moves handling of the MC vmexits to an earlier point in the vmexit. The handle_exit function is too late because the vcpu might alreadry have changed its physical cpu. Cc: stable@kernel.org Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> (cherry picked from commit fe5913e4e1700cbfc337f4b1da9ddb26f6a55586)
2010-08-02KVM: MMU: Remove user access when allowing kernel access to gpte.w=0 pageAvi Kivity
If cr0.wp=0, we have to allow the guest kernel access to a page with pte.w=0. We do that by setting spte.w=1, since the host cr0.wp must remain set so the host can write protect pages. Once we allow write access, we must remove user access otherwise we mistakenly allow the user to write the page. Reviewed-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com> Signed-off-by: Avi Kivity <avi@redhat.com> (cherry picked from commit 69325a122580d3a7b26589e8efdd6663001c3297)
2010-08-02x86: Do not try to disable hpet if it hasn't been initialized beforeStefano Stabellini
commit ff4878089e1eaeac79d57878ad4ea32910fb4037 upstream. hpet_disable is called unconditionally on machine reboot if hpet support is compiled in the kernel. hpet_disable only checks if the machine is hpet capable but doesn't make sure that hpet has been initialized. [ tglx: Made it a one liner and removed the redundant hpet_address check ] Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> Acked-by: Venkatesh Pallipadi <venki@google.com> LKML-Reference: <alpine.DEB.2.00.1007211726240.22235@kaball-desktop> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-08-02x86: Fix x2apic preenabled system with kexecYinghai Lu
commit fd19dce7ac07973f700b0f13fb7f94b951414a4c upstream. Found one x2apic system kexec loop test failed when CONFIG_NMI_WATCHDOG=y (old) or CONFIG_LOCKUP_DETECTOR=y (current tip) first kernel can kexec second kernel, but second kernel can not kexec third one. it can be duplicated on another system with BIOS preenabled x2apic. First kernel can not kexec second kernel. It turns out, when kernel boot with pre-enabled x2apic, it will not execute disable_local_APIC on shutdown path. when init_apic_mappings() is called in setup_arch, it will skip setting of apic_phys when x2apic_mode is set. ( x2apic_mode is much early check_x2apic()) Then later, disable_local_APIC() will bail out early because !apic_phys. So check !x2apic_mode in x2apic_mode in disable_local_APIC with !apic_phys. another solution could be updating init_apic_mappings() to set apic_phys even for preenabled x2apic system. Actually even for x2apic system, that lapic address is mapped already in early stage. BTW: is there any x2apic preenabled system with apicid of boot cpu > 255? Signed-off-by: Yinghai Lu <yinghai@kernel.org> LKML-Reference: <4C3EB22B.3000701@kernel.org> Acked-by: Suresh Siddha <suresh.b.siddha@intel.com> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-08-02x86, Calgary: Limit the max PHB number to 256Darrick J. Wong
commit d596043d71ff0d7b3d0bead19b1d68c55f003093 upstream. The x3950 family can have as many as 256 PCI buses in a single system, so change the limits to the maximum. Since there can only be 256 PCI buses in one domain, we no longer need the BUG_ON check. Signed-off-by: Darrick J. Wong <djwong@us.ibm.com> LKML-Reference: <20100701004519.GQ15515@tux1.beaverton.ibm.com> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-08-02x86, Calgary: Increase max PHB numberDarrick J. Wong
commit 499a00e92dd9a75395081f595e681629eb1eebad upstream. Newer systems (x3950M2) can have 48 PHBs per chassis and 8 chassis, so bump the limits up and provide an explanation of the requirements for each class. Signed-off-by: Darrick J. Wong <djwong@us.ibm.com> Acked-by: Muli Ben-Yehuda <muli@il.ibm.com> Cc: Corinna Schultz <cschultz@linux.vnet.ibm.com> LKML-Reference: <20100624212647.GI15515@tux1.beaverton.ibm.com> [ v2: Fixed build bug, added back PHBS_PER_CALGARY == 4 ] Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-08-02x86: Fix vsyscall on gcc 4.5 with -OsAndi Kleen
commit 124482935fb7fb9303c8a8ab930149c6a93d9910 upstream. This fixes the -Os breaks with gcc 4.5 bug. rdtsc_barrier needs to be force inlined, otherwise user space will jump into kernel space and kill init. This also addresses http://gcc.gnu.org/bugzilla/show_bug.cgi?id=44129 I believe. Signed-off-by: Andi Kleen <ak@linux.intel.com> LKML-Reference: <20100618210859.GA10913@basil.fritz.box> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-07-05KVM: x86: Inject #GP with the right rip on efer writesRoedel, Joerg
This patch fixes a bug in the KVM efer-msr write path. If a guest writes to a reserved efer bit the set_efer function injects the #GP directly. The architecture dependent wrmsr function does not see this, assumes success and advances the rip. This results in a #GP in the guest with the wrong rip. This patch fixes this by reporting efer write errors back to the architectural wrmsr function. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit b69e8caef5b190af48c525f6d715e7b7728a77f6)
2010-07-05KVM: x86: Add missing locking to arch specific vcpu ioctlsAvi Kivity
Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 8fbf065d625617bbbf6b72d5f78f84ad13c8b547)
2010-07-05KVM: Fix wallclock version writing raceAvi Kivity
Wallclock writing uses an unprotected global variable to hold the version; this can cause one guest to interfere with another if both write their wallclock at the same time. Acked-by: Glauber Costa <glommer@redhat.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 9ed3c444ab8987c7b219173a2f7807e3f71e234e)
2010-07-05KVM: MMU: Don't read pdptrs with mmu spinlock held in mmu_alloc_rootsAvi Kivity
On svm, kvm_read_pdptr() may require reading guest memory, which can sleep. Push the spinlock into mmu_alloc_roots(), and only take it after we've read the pdptr. Tested-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 8facbbff071ff2b19268d3732e31badc60471e21)
2010-07-05KVM: VMX: enable VMXON check with SMX enabled (Intel TXT)Shane Wang
Per document, for feature control MSR: Bit 1 enables VMXON in SMX operation. If the bit is clear, execution of VMXON in SMX operation causes a general-protection exception. Bit 2 enables VMXON outside SMX operation. If the bit is clear, execution of VMXON outside SMX operation causes a general-protection exception. This patch is to enable this kind of check with SMX for VMXON in KVM. Signed-off-by: Shane Wang <shane.wang@intel.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit cafd66595d92591e4bd25c3904e004fc6f897e2d)
2010-07-05KVM: MMU: Segregate shadow pages with different cr0.wpAvi Kivity
When cr0.wp=0, we may shadow a gpte having u/s=1 and r/w=0 with an spte having u/s=0 and r/w=1. This allows excessive access if the guest sets cr0.wp=1 and accesses through this spte. Fix by making cr0.wp part of the base role; we'll have different sptes for the two cases and the problem disappears. Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 3dbe141595faa48a067add3e47bba3205b79d33c)
2010-07-05KVM: x86: Check LMA bit before set_eferSheng Yang
kvm_x86_ops->set_efer() would execute vcpu->arch.efer = efer, so the checking of LMA bit didn't work. Signed-off-by: Sheng Yang <sheng@linux.intel.com> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit a3d204e28579427609c3d15d2310127ebaa47d94)
2010-07-05KVM: Don't allow lmsw to clear cr0.peAvi Kivity
The current lmsw implementation allows the guest to clear cr0.pe, contrary to the manual, which breaks EMM386.EXE. Fix by ORing the old cr0.pe with lmsw's operand. Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit f78e917688edbf1f14c318d2e50dc8e7dad20445)
2010-07-05x86, paravirt: Add a global synchronization point for pvclockGlauber Costa
In recent stress tests, it was found that pvclock-based systems could seriously warp in smp systems. Using ingo's time-warp-test.c, I could trigger a scenario as bad as 1.5mi warps a minute in some systems. (to be fair, it wasn't that bad in most of them). Investigating further, I found out that such warps were caused by the very offset-based calculation pvclock is based on. This happens even on some machines that report constant_tsc in its tsc flags, specially on multi-socket ones. Two reads of the same kernel timestamp at approx the same time, will likely have tsc timestamped in different occasions too. This means the delta we calculate is unpredictable at best, and can probably be smaller in a cpu that is legitimately reading clock in a forward ocasion. Some adjustments on the host could make this window less likely to happen, but still, it pretty much poses as an intrinsic problem of the mechanism. A while ago, I though about using a shared variable anyway, to hold clock last state, but gave up due to the high contention locking was likely to introduce, possibly rendering the thing useless on big machines. I argue, however, that locking is not necessary. We do a read-and-return sequence in pvclock, and between read and return, the global value can have changed. However, it can only have changed by means of an addition of a positive value. So if we detected that our clock timestamp is less than the current global, we know that we need to return a higher one, even though it is not exactly the one we compared to. OTOH, if we detect we're greater than the current time source, we atomically replace the value with our new readings. This do causes contention on big boxes (but big here means *BIG*), but it seems like a good trade off, since it provide us with a time source guaranteed to be stable wrt time warps. After this patch is applied, I don't see a single warp in time during 5 days of execution, in any of the machines I saw them before. Signed-off-by: Glauber Costa <glommer@redhat.com> Acked-by: Zachary Amsden <zamsden@redhat.com> CC: Jeremy Fitzhardinge <jeremy@goop.org> CC: Avi Kivity <avi@redhat.com> CC: Marcelo Tosatti <mtosatti@redhat.com> CC: Zachary Amsden <zamsden@redhat.com> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de> (cherry picked from commit 489fb490dbf8dab0249ad82b56688ae3842a79e8)
2010-07-05perf_events: Fix resource leak in x86 __hw_perf_event_init()Stephane Eranian
commit 4b24a88b35e15e04bd8f2c5dda65b5dc8ebca05f upstream. If reserve_pmc_hardware() succeeds but reserve_ds_buffers() fails, then we need to release_pmc_hardware. It won't be done by the destroy() callback because we return before setting it in case of error. Signed-off-by: Stephane Eranian <eranian@google.com> Cc: peterz@infradead.org Cc: paulus@samba.org Cc: davem@davemloft.net Cc: fweisbec@gmail.com Cc: robert.richter@amd.com Cc: perfmon2-devel@lists.sf.net LKML-Reference: <4ba1568b.15185e0a.182a.7802@mx.google.com> Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-07-05xen: ensure timer tick is resumed even on CPU driving the resumeIan Campbell
commit cd52e17ea8278f8449b6174a8e5ed439a2e44ffb upstream. The core suspend/resume code is run from stop_machine on CPU0 but parts of the suspend/resume machinery (including xen_arch_resume) are run on whichever CPU happened to schedule the xenwatch kernel thread. As part of the non-core resume code xen_arch_resume is called in order to restart the timer tick on non-boot processors. The boot processor itself is taken care of by core timekeeping code. xen_arch_resume uses smp_call_function which does not call the given function on the current processor. This means that we can end up with one CPU not receiving timer ticks if the xenwatch thread happened to be scheduled on CPU > 0. Use on_each_cpu instead of smp_call_function to ensure the timer tick is resumed everywhere. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> Acked-by: Jeremy Fitzhardinge <jeremy@goop.org> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-07-05x86, setup: Phoenix BIOS fixup is needed on Dell Inspiron Mini 1012Gabor Gombas
commit 3d6e77a3ddb8e4156b89f4273ff8c7d37abaf781 upstream. The low-memory corruption checker triggers during suspend/resume, so we need to reserve the low 64k. Don't be fooled that the BIOS identifies itself as "Dell Inc.", it's still Phoenix BIOS. [ hpa: I think we blacklist almost every BIOS in existence. We should either change this to a whitelist or just make it unconditional. ] Signed-off-by: Gabor Gombas <gombasg@digikabel.hu> LKML-Reference: <201005241913.o4OJDIMM010877@imap1.linux-foundation.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-07-05x86/amd-iommu: Fix suspend/resume with IOMMUJoerg Roedel
This is a suspend resume fix for 2.6.32-stable inclusion. The problem with this patch is that it is not upstream because the code changed with 2.6.33 and the function where this bug is in was removed. So this fix does not make sense anymore for anything later than 2.6.32. The patch was tested by multiple partys and is confirmed to fix the broken suspend/resume issue with the 2.6.32 kernel. This patch fixes suspend/resume with AMD IOMMU enabled. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-07-05oprofile/x86: fix uninitialized counter usage during cpu hotplugRobert Richter
commit 2623a1d55a6260c855e1f6d1895900b50b40a896 upstream. This fixes a NULL pointer dereference that is triggered when taking a cpu offline after oprofile was initialized, e.g.: $ opcontrol --init $ opcontrol --start-daemon $ opcontrol --shutdown $ opcontrol --deinit $ echo 0 > /sys/devices/system/cpu/cpu1/online See the crash dump below. Though the counter has been disabled the cpu notifier is still active and trying to use already freed counter data. This fix is for linux-stable. To proper fix this, the hotplug code must be rewritten. Thus I will leave a WARN_ON_ONCE() message with this patch. BUG: unable to handle kernel NULL pointer dereference at (null) IP: [<ffffffff8132ad57>] op_amd_stop+0x2d/0x8e PGD 0 Oops: 0000 [#1] SMP last sysfs file: /sys/devices/system/cpu/cpu1/online CPU 1 Modules linked in: Pid: 0, comm: swapper Not tainted 2.6.34-rc5-oprofile-x86_64-standard-00210-g8c00f06 #16 Anaheim/Anaheim RIP: 0010:[<ffffffff8132ad57>] [<ffffffff8132ad57>] op_amd_stop+0x2d/0x8e RSP: 0018:ffff880001843f28 EFLAGS: 00010006 RAX: 0000000000000000 RBX: 0000000000000000 RCX: dead000000200200 RDX: ffff880001843f68 RSI: dead000000100100 RDI: 0000000000000000 RBP: ffff880001843f48 R08: 0000000000000000 R09: ffff880001843f08 R10: ffffffff8102c9a5 R11: ffff88000184ea80 R12: 0000000000000000 R13: ffff88000184f6c0 R14: 0000000000000000 R15: 0000000000000000 FS: 00007fec6a92e6f0(0000) GS:ffff880001840000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000000 CR3: 000000000163b000 CR4: 00000000000006e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper (pid: 0, threadinfo ffff88042fcd8000, task ffff88042fcd51d0) Stack: ffff880001843f48 0000000000000001 ffff88042e9f7d38 ffff880001843f68 <0> ffff880001843f58 ffffffff8132a602 ffff880001843f98 ffffffff810521b3 <0> ffff880001843f68 ffff880001843f68 ffff880001843f88 ffff88042fcd9fd8 Call Trace: <IRQ> [<ffffffff8132a602>] nmi_cpu_stop+0x21/0x23 [<ffffffff810521b3>] generic_smp_call_function_single_interrupt+0xdf/0x11b [<ffffffff8101804f>] smp_call_function_single_interrupt+0x22/0x31 [<ffffffff810029f3>] call_function_single_interrupt+0x13/0x20 <EOI> [<ffffffff8102c9a5>] ? wake_up_process+0x10/0x12 [<ffffffff81008701>] ? default_idle+0x22/0x37 [<ffffffff8100896d>] c1e_idle+0xdf/0xe6 [<ffffffff813f1170>] ? atomic_notifier_call_chain+0x13/0x15 [<ffffffff810012fb>] cpu_idle+0x4b/0x7e [<ffffffff813e8a4e>] start_secondary+0x1ae/0x1b2 Code: 89 e5 41 55 49 89 fd 41 54 45 31 e4 53 31 db 48 83 ec 08 89 df e8 be f8 ff ff 48 98 48 83 3c c5 10 67 7a 81 00 74 1f 49 8b 45 08 <42> 8b 0c 20 0f 32 48 c1 e2 20 25 ff ff bf ff 48 09 d0 48 89 c2 RIP [<ffffffff8132ad57>] op_amd_stop+0x2d/0x8e RSP <ffff880001843f28> CR2: 0000000000000000 ---[ end trace 679ac372d674b757 ]--- Kernel panic - not syncing: Fatal exception in interrupt Pid: 0, comm: swapper Tainted: G D 2.6.34-rc5-oprofile-x86_64-standard-00210-g8c00f06 #16 Call Trace: <IRQ> [<ffffffff813ebd6a>] panic+0x9e/0x10c [<ffffffff810474b0>] ? up+0x34/0x39 [<ffffffff81031ccc>] ? kmsg_dump+0x112/0x12c [<ffffffff813eeff1>] oops_end+0x81/0x8e [<ffffffff8101efee>] no_context+0x1f3/0x202 [<ffffffff8101f1b7>] __bad_area_nosemaphore+0x1ba/0x1e0 [<ffffffff81028d24>] ? enqueue_task_fair+0x16d/0x17a [<ffffffff810264dc>] ? activate_task+0x42/0x53 [<ffffffff8102c967>] ? try_to_wake_up+0x272/0x284 [<ffffffff8101f1eb>] bad_area_nosemaphore+0xe/0x10 [<ffffffff813f0f3f>] do_page_fault+0x1c8/0x37c [<ffffffff81028d24>] ? enqueue_task_fair+0x16d/0x17a [<ffffffff813ee55f>] page_fault+0x1f/0x30 [<ffffffff8102c9a5>] ? wake_up_process+0x10/0x12 [<ffffffff8132ad57>] ? op_amd_stop+0x2d/0x8e [<ffffffff8132ad46>] ? op_amd_stop+0x1c/0x8e [<ffffffff8132a602>] nmi_cpu_stop+0x21/0x23 [<ffffffff810521b3>] generic_smp_call_function_single_interrupt+0xdf/0x11b [<ffffffff8101804f>] smp_call_function_single_interrupt+0x22/0x31 [<ffffffff810029f3>] call_function_single_interrupt+0x13/0x20 <EOI> [<ffffffff8102c9a5>] ? wake_up_process+0x10/0x12 [<ffffffff81008701>] ? default_idle+0x22/0x37 [<ffffffff8100896d>] c1e_idle+0xdf/0xe6 [<ffffffff813f1170>] ? atomic_notifier_call_chain+0x13/0x15 [<ffffffff810012fb>] cpu_idle+0x4b/0x7e [<ffffffff813e8a4e>] start_secondary+0x1ae/0x1b2 ------------[ cut here ]------------ WARNING: at /local/rrichter/.source/linux/arch/x86/kernel/smp.c:118 native_smp_send_reschedule+0x27/0x53() Hardware name: Anaheim Modules linked in: Pid: 0, comm: swapper Tainted: G D 2.6.34-rc5-oprofile-x86_64-standard-00210-g8c00f06 #16 Call Trace: <IRQ> [<ffffffff81017f32>] ? native_smp_send_reschedule+0x27/0x53 [<ffffffff81030ee2>] warn_slowpath_common+0x77/0xa4 [<ffffffff81030f1e>] warn_slowpath_null+0xf/0x11 [<ffffffff81017f32>] native_smp_send_reschedule+0x27/0x53 [<ffffffff8102634b>] resched_task+0x60/0x62 [<ffffffff8102653a>] check_preempt_curr_idle+0x10/0x12 [<ffffffff8102c8ea>] try_to_wake_up+0x1f5/0x284 [<ffffffff8102c986>] default_wake_function+0xd/0xf [<ffffffff810a110d>] pollwake+0x57/0x5a [<ffffffff8102c979>] ? default_wake_function+0x0/0xf [<ffffffff81026be5>] __wake_up_common+0x46/0x75 [<ffffffff81026ed0>] __wake_up+0x38/0x50 [<ffffffff81031694>] printk_tick+0x39/0x3b [<ffffffff8103ac37>] update_process_times+0x3f/0x5c [<ffffffff8104dc63>] tick_periodic+0x5d/0x69 [<ffffffff8104dc90>] tick_handle_periodic+0x21/0x71 [<ffffffff81018fd0>] smp_apic_timer_interrupt+0x82/0x95 [<ffffffff81002853>] apic_timer_interrupt+0x13/0x20 [<ffffffff81030cb5>] ? panic_blink_one_second+0x0/0x7b [<ffffffff813ebdd6>] ? panic+0x10a/0x10c [<ffffffff810474b0>] ? up+0x34/0x39 [<ffffffff81031ccc>] ? kmsg_dump+0x112/0x12c [<ffffffff813eeff1>] ? oops_end+0x81/0x8e [<ffffffff8101efee>] ? no_context+0x1f3/0x202 [<ffffffff8101f1b7>] ? __bad_area_nosemaphore+0x1ba/0x1e0 [<ffffffff81028d24>] ? enqueue_task_fair+0x16d/0x17a [<ffffffff810264dc>] ? activate_task+0x42/0x53 [<ffffffff8102c967>] ? try_to_wake_up+0x272/0x284 [<ffffffff8101f1eb>] ? bad_area_nosemaphore+0xe/0x10 [<ffffffff813f0f3f>] ? do_page_fault+0x1c8/0x37c [<ffffffff81028d24>] ? enqueue_task_fair+0x16d/0x17a [<ffffffff813ee55f>] ? page_fault+0x1f/0x30 [<ffffffff8102c9a5>] ? wake_up_process+0x10/0x12 [<ffffffff8132ad57>] ? op_amd_stop+0x2d/0x8e [<ffffffff8132ad46>] ? op_amd_stop+0x1c/0x8e [<ffffffff8132a602>] ? nmi_cpu_stop+0x21/0x23 [<ffffffff810521b3>] ? generic_smp_call_function_single_interrupt+0xdf/0x11b [<ffffffff8101804f>] ? smp_call_function_single_interrupt+0x22/0x31 [<ffffffff810029f3>] ? call_function_single_interrupt+0x13/0x20 <EOI> [<ffffffff8102c9a5>] ? wake_up_process+0x10/0x12 [<ffffffff81008701>] ? default_idle+0x22/0x37 [<ffffffff8100896d>] ? c1e_idle+0xdf/0xe6 [<ffffffff813f1170>] ? atomic_notifier_call_chain+0x13/0x15 [<ffffffff810012fb>] ? cpu_idle+0x4b/0x7e [<ffffffff813e8a4e>] ? start_secondary+0x1ae/0x1b2 ---[ end trace 679ac372d674b758 ]--- Cc: Andi Kleen <andi@firstfloor.org> Signed-off-by: Robert Richter <robert.richter@amd.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-05-26x86, amd: Check X86_FEATURE_OSVW bit before accessing OSVW MSRsAndreas Herrmann
commit f01487119dda3d9f58c9729c7361ecc50a61c188 upstream. If host CPU is exposed to a guest the OSVW MSRs are not guaranteed to be present and a GP fault occurs. Thus checking the feature flag is essential. Signed-off-by: Andreas Herrmann <andreas.herrmann3@amd.com> LKML-Reference: <20100427101348.GC4489@alberich.amd.com> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-05-26x86, cacheinfo: Turn off L3 cache index disable feature in virtualized ↵Frank Arnold
environments commit 7f284d3cc96e02468a42e045f77af11e5ff8b095 upstream. When running a quest kernel on xen we get: BUG: unable to handle kernel NULL pointer dereference at 0000000000000038 IP: [<ffffffff8142f2fb>] cpuid4_cache_lookup_regs+0x2ca/0x3df PGD 0 Oops: 0000 [#1] SMP last sysfs file: CPU 0 Modules linked in: Pid: 0, comm: swapper Tainted: G W 2.6.34-rc3 #1 /HVM domU RIP: 0010:[<ffffffff8142f2fb>] [<ffffffff8142f2fb>] cpuid4_cache_lookup_regs+0x 2ca/0x3df RSP: 0018:ffff880002203e08 EFLAGS: 00010046 RAX: 0000000000000000 RBX: 0000000000000003 RCX: 0000000000000060 RDX: 0000000000000000 RSI: 0000000000000040 RDI: 0000000000000000 RBP: ffff880002203ed8 R08: 00000000000017c0 R09: ffff880002203e38 R10: ffff8800023d5d40 R11: ffffffff81a01e28 R12: ffff880187e6f5c0 R13: ffff880002203e34 R14: ffff880002203e58 R15: ffff880002203e68 FS: 0000000000000000(0000) GS:ffff880002200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000000000000038 CR3: 0000000001a3c000 CR4: 00000000000006f0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process swapper (pid: 0, threadinfo ffffffff81a00000, task ffffffff81a44020) Stack: ffffffff810d7ecb ffff880002203e20 ffffffff81059140 ffff880002203e30 <0> ffffffff810d7ec9 0000000002203e40 000000000050d140 ffff880002203e70 <0> 0000000002008140 0000000000000086 ffff880040020140 ffffffff81068b8b Call Trace: <IRQ> [<ffffffff810d7ecb>] ? sync_supers_timer_fn+0x0/0x1c [<ffffffff81059140>] ? mod_timer+0x23/0x25 [<ffffffff810d7ec9>] ? arm_supers_timer+0x34/0x36 [<ffffffff81068b8b>] ? hrtimer_get_next_event+0xa7/0xc3 [<ffffffff81058e85>] ? get_next_timer_interrupt+0x19a/0x20d [<ffffffff8142fa23>] get_cpu_leaves+0x5c/0x232 [<ffffffff8106a7b1>] ? sched_clock_local+0x1c/0x82 [<ffffffff8106a9a0>] ? sched_clock_tick+0x75/0x7a [<ffffffff8107748c>] generic_smp_call_function_single_interrupt+0xae/0xd0 [<ffffffff8101f6ef>] smp_call_function_single_interrupt+0x18/0x27 [<ffffffff8100a773>] call_function_single_interrupt+0x13/0x20 <EOI> [<ffffffff8143c468>] ? notifier_call_chain+0x14/0x63 [<ffffffff810295c6>] ? native_safe_halt+0xc/0xd [<ffffffff810114eb>] ? default_idle+0x36/0x53 [<ffffffff81008c22>] cpu_idle+0xaa/0xe4 [<ffffffff81423a9a>] rest_init+0x7e/0x80 [<ffffffff81b10dd2>] start_kernel+0x40e/0x419 [<ffffffff81b102c8>] x86_64_start_reservations+0xb3/0xb7 [<ffffffff81b103c4>] x86_64_start_kernel+0xf8/0x107 Code: 14 d5 40 ff ae 81 8b 14 02 31 c0 3b 15 47 1c 8b 00 7d 0e 48 8b 05 36 1c 8b 00 48 63 d2 48 8b 04 d0 c7 85 5c ff ff ff 00 00 00 00 <8b> 70 38 48 8d 8d 5c ff ff ff 48 8b 78 10 ba c4 01 00 00 e8 eb RIP [<ffffffff8142f2fb>] cpuid4_cache_lookup_regs+0x2ca/0x3df RSP <ffff880002203e08> CR2: 0000000000000038 ---[ end trace a7919e7f17c0a726 ]--- The L3 cache index disable feature of AMD CPUs has to be disabled if the kernel is running as guest on top of a hypervisor because northbridge devices are not available to the guest. Currently, this fixes a boot crash on top of Xen. In the future this will become an issue on KVM as well. Check if northbridge devices are present and do not enable the feature if there are none. [ hpa: backported to 2.6.34 ] Signed-off-by: Frank Arnold <frank.arnold@amd.com> LKML-Reference: <1271945222-5283-3-git-send-email-bp@amd64.org> Acked-by: Borislav Petkov <borislav.petkov@amd.com> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-05-26x86, k8: Fix build error when K8_NB is disabledBorislav Petkov
commit ade029e2aaacc8965a548b0b0f80c5bee97ffc68 upstream. K8_NB depends on PCI and when the last is disabled (allnoconfig) we fail at the final linking stage due to missing exported num_k8_northbridges. Add a header stub for that. Signed-off-by: Borislav Petkov <borislav.petkov@amd.com> LKML-Reference: <20100503183036.GJ26107@aftab> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-05-12KVM: remove unused load_segment_descriptor_to_kvm_desctMarcelo Tosatti
Commit 78ce64a384 in v2.6.32.12 introduced a warning due to unused load_segment_descriptor_to_kvm_desct helper, which has been opencoded by this commit. On upstream, the helper was removed as part of a different commit. Remove the now unused function. Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-05-12ACPI: introduce kernel parameter acpi_sleep=sci_force_enableZhang Rui
commit d7f0eea9e431e1b8b0742a74db1a9490730b2a25 upstream. Introduce kernel parameter acpi_sleep=sci_force_enable some laptop requires SCI_EN being set directly on resume, or else they hung somewhere in the resume code path. We already have a blacklist for these laptops but we still need this option, especially when debugging some suspend/resume problems, in case there are systems that need this workaround and are not yet in the blacklist. Signed-off-by: Zhang Rui <rui.zhang@intel.com> Acked-by: Rafael J. Wysocki <rjw@sisk.pl> Signed-off-by: Len Brown <len.brown@intel.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-05-12x86, AMD: Fix stale cpuid4_info shared_map data in shared_cpu_map cpumasksPrarit Bhargava
commit ebb682f522411abbe358059a256a8672ec0bd55b upstream. The per_cpu cpuid4_info shared_map can contain stale data when CPUs are added and removed. The stale data can lead to a NULL pointer derefernce panic on a remove of a CPU that has had siblings previously removed. This patch resolves the panic by verifying a cpu is actually online before adding it to the shared_cpu_map, only examining cpus that are part of the same lower level cache, and by updating other siblings lowest level cache maps when a cpu is added. Signed-off-by: Prarit Bhargava <prarit@redhat.com> LKML-Reference: <20091209183336.17855.98708.sendpatchset@prarit.bos.redhat.com> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-05-12x86, k8 nb: Fix boot crash: enable k8_northbridges unconditionally on AMD ↵Borislav Petkov
systems commit 0e152cd7c16832bd5cadee0c2e41d9959bc9b6f9 upstream. de957628ce7c84764ff41331111036b3ae5bad0f changed setting of the x86_init.iommu.iommu_init function ptr only when GART IOMMU is found. One side effect of it is that num_k8_northbridges is not initialized anymore if not explicitly called. This resulted in uninitialized pointers in <arch/x86/kernel/cpu/intel_cacheinfo.c:amd_calc_l3_indices()>, for example, which uses the num_k8_northbridges thing through node_to_k8_nb_misc(). Fix that through an initcall that runs right after the PCI subsystem and does all the scanning. Then, remove initialization in gart_iommu_init() which is a rootfs_initcall and we're running before that. What is more, since num_k8_northbridges is being used in other places beside GART IOMMU, include it whenever we add AMD CPU support. The previous dependency chain in kconfig contained K8_NB depends on AGP_AMD64|GART_IOMMU which was clearly incorrect. The more natural way in terms of hardware dependency should be AGP_AMD64|GART_IOMMU depends on K8_NB depends on CPU_SUP_AMD && PCI. Make it so Number One! Signed-off-by: Borislav Petkov <borislav.petkov@amd.com> Cc: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp> Cc: Joerg Roedel <joerg.roedel@amd.com> LKML-Reference: <20100312144303.GA29262@aftab> Signed-off-by: Ingo Molnar <mingo@elte.hu> Tested-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-05-12x86: Disable large pages on CPUs with Atom erratum AAE44H. Peter Anvin
commit 7a0fc404ae663776e96db43879a0fa24fec1fa3a upstream. Atom erratum AAE44/AAF40/AAG38/AAH41: "If software clears the PS (page size) bit in a present PDE (page directory entry), that will cause linear addresses mapped through this PDE to use 4-KByte pages instead of using a large page after old TLB entries are invalidated. Due to this erratum, if a code fetch uses this PDE before the TLB entry for the large page is invalidated then it may fetch from a different physical address than specified by either the old large page translation or the new 4-KByte page translation. This erratum may also cause speculative code fetches from incorrect addresses." [http://download.intel.com/design/processor/specupdt/319536.pdf] Where as commit 211b3d03c7400f48a781977a50104c9d12f4e229 seems to workaround errata AAH41 (mixed 4K TLBs) it reduces the window of opportunity for the bug to occur and does not totally remove it. This patch disables mixed 4K/4MB page tables totally avoiding the page splitting and not tripping this processor issue. This is based on an original patch by Colin King. Originally-by: Colin Ian King <colin.king@canonical.com> Cc: Colin Ian King <colin.king@canonical.com> Cc: Ingo Molnar <mingo@elte.hu> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> LKML-Reference: <1269271251-19775-1-git-send-email-colin.king@canonical.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-05-12x86-64: Clear a 64-bit FS/GS base on fork if selector is nonzeroH. Peter Anvin
commit 7ce5a2b9bb2e92902230e3121d8c3047fab9cb47 upstream. When we do a thread switch, we clear the outgoing FS/GS base if the corresponding selector is nonzero. This is taken by __switch_to() as an entry invariant; it does not verify that it is true on entry. However, copy_thread() doesn't enforce this constraint, which can result in inconsistent results after fork(). Make copy_thread() match the behavior of __switch_to(). Reported-and-tested-by: Samuel Thibault <samuel.thibault@inria.fr> Signed-off-by: H. Peter Anvin <hpa@zytor.com> LKML-Reference: <4BD1E061.8030605@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-05-12powernow-k8: Fix frequency reportingMark Langsdorf
commit b810e94c9d8e3fff6741b66cd5a6f099a7887871 upstream. With F10, model 10, all valid frequencies are in the ACPI _PST table. Signed-off-by: Mark Langsdorf <mark.langsdorf@amd.com> LKML-Reference: <1270065406-1814-6-git-send-email-bp@amd64.org> Signed-off-by: Borislav Petkov <borislav.petkov@amd.com> Reviewed-by: Thomas Renninger <trenn@suse.de> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-05-12core, x86: make LIST_POISON less deadlyAvi Kivity
commit a29815a333c6c6e677294bbe5958e771d0aad3fd upstream. The list macros use LIST_POISON1 and LIST_POISON2 as undereferencable pointers in order to trap erronous use of freed list_heads. Unfortunately userspace can arrange for those pointers to actually be dereferencable, potentially turning an oops to an expolit. To avoid this allow architectures (currently x86_64 only) to override the default values for these pointers with truly-undereferencable values. This is easy on x86_64 as the virtual address space is large and contains areas that cannot be mapped. Other 64-bit architectures will likely find similar unmapped ranges. [ingo: switch to 0xdead000000000000 as the unmapped area] [ingo: add comments, cleanup] [jaswinder: eliminate sparse warnings] Acked-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Jaswinder Singh Rajput <jaswinderrajput@gmail.com> Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26x86/gart: Disable GART explicitly before initializationJoerg Roedel
commit 4b83873d3da0704987cb116833818ed96214ee29 upstream. If we boot into a crash-kernel the gart might still be enabled and its caches might be dirty. This can result in undefined behavior later. Fix it by explicitly disabling the gart hardware before initialization and flushing the caches after enablement. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26KVM: x86: Fix TSS size check for 16-bit tasksJan Kiszka
(Cherry-picked from commit e8861cfe2c75bdce36655b64d7ce02c2b31b604d) A 16-bit TSS is only 44 bytes long. So make sure to test for the correct size on task switch. Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26KVM: fix the handling of dirty bitmaps to avoid overflowsTakuya Yoshikawa
(Cherry-picked from commit 87bf6e7de1134f48681fd2ce4b7c1ec45458cb6d) Int is not long enough to store the size of a dirty bitmap. This patch fixes this problem with the introduction of a wrapper function to calculate the sizes of dirty bitmaps. Note: in mark_page_dirty(), we have to consider the fact that __set_bit() takes the offset as int, not long. Signed-off-by: Takuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26KVM: MMU: fix kvm_mmu_zap_page() and its calling pathXiao Guangrong
(Cherry-picked from commit 77662e0028c7c63e34257fda03ff9625c59d939d) This patch fix: - calculate zapped page number properly in mmu_zap_unsync_children() - calculate freeed page number properly kvm_mmu_change_mmu_pages() - if zapped children page it shoud restart hlist walking KVM-Stable-Tag. Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26KVM: VMX: Save/restore rflags.vm correctly in real modeAvi Kivity
(Cherry-picked from commit 78ac8b47c566dd6177a3b9b291b756ccb70670b7) Currently we set eflags.vm unconditionally when entering real mode emulation through virtual-8086 mode, and clear it unconditionally when we enter protected mode. The means that the following sequence KVM_SET_REGS (rflags.vm=1) KVM_SET_SREGS (cr0.pe=1) Ends up with rflags.vm clear due to KVM_SET_SREGS triggering enter_pmode(). Fix by shadowing rflags.vm (and rflags.iopl) correctly while in real mode: reads and writes to those bits access a shadow register instead of the actual register. Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26KVM: allow bit 10 to be cleared in MSR_IA32_MC4_CTLAndre Przywara
(Cherry-picked from commit 114be429c8cd44e57f312af2bbd6734e5a185b0d) There is a quirk for AMD K8 CPUs in many Linux kernels (see arch/x86/kernel/cpu/mcheck/mce.c:__mcheck_cpu_apply_quirks()) that clears bit 10 in that MCE related MSR. KVM can only cope with all zeros or all ones, so it will inject a #GP into the guest, which will let it panic. So lets add a quirk to the quirk and ignore this single cleared bit. This fixes -cpu kvm64 on all machines and -cpu host on K8 machines with some guest Linux kernels. Signed-off-by: Andre Przywara <andre.przywara@amd.com> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26KVM: Don't spam kernel log when injecting exceptions due to bad cr writesAvi Kivity
(Cherry-picked from commit d6a23895aa82353788a1cc5a1d9a1c963465463e) These are guest-triggerable. Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26KVM: SVM: Fix memory leaks that happen when svm_create_vcpu() failsTakuya Yoshikawa
(Cherry-picked from commit b7af40433870aa0636932ad39b0c48a0cb319057) svm_create_vcpu() does not free the pages allocated during the creation when it fails to complete the allocations. This patch fixes it. Signed-off-by: Takuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp> Signed-off-by: Avi Kivity <avi@redhat.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26perf_events, x86: Implement Intel Westmere/Nehalem-EX supportPeter Zijlstra
original patch commit ids: 452a339a976e7f782c786eb3f73080401e2fa3a6 and 134fbadf028a5977a1b06b0253d3ee33e6f0c642 perf_events, x86: Implement Intel Westmere support The new Intel documentation includes Westmere arch specific event maps that are significantly different from the Nehalem ones. Add support for this generation. Found the CPUID model numbers on wikipedia. Also ammend some Nehalem constraints, spotted those when looking for the differences between Nehalem and Westmere. Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Stephane Eranian <eranian@google.com> LKML-Reference: <20100127221122.151865645@chello.nl> Signed-off-by: Ingo Molnar <mingo@elte.hu> perf, x86: Enable Nehalem-EX support According to Intel Software Devel Manual Volume 3B, the Nehalem-EX PMU is just like regular Nehalem (except for the uncore support, which is completely different). Signed-off-by: Vince Weaver <vweaver1@eecs.utk.edu> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Paul Mackerras <paulus@samba.org> Cc: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Arjan van de Ven <arjan@linux.intel.com> Cc: Lin Ming <ming.m.lin@intel.com> LKML-Reference: <alpine.DEB.2.00.1004060956580.1417@cl320.eecs.utk.edu> Signed-off-by: Ingo Molnar <mingo@elte.hu> Cc: Youquan Song <youquan.song@linux.intel.com>
2010-04-26x86/PCI: irq and pci_ids patch for Intel Cougar Point DeviceIDsSeth Heasley
commit 93da6202264ce1256b04db8008a43882ae62d060 upstream. This patch adds the Intel Cougar Point (PCH) LPC and SMBus Controller DeviceIDs. Signed-off-by: Seth Heasley <seth.heasley@intel.com> Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org> Cc: maximilian attems <max@stro.at> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26x86-64, rwsem: Avoid store forwarding hazard in __downgrade_writeAvi Kivity
commit 0d1622d7f526311d87d7da2ee7dd14b73e45d3fc upstream. The Intel Architecture Optimization Reference Manual states that a short load that follows a long store to the same object will suffer a store forwading penalty, particularly if the two accesses use different addresses. Trivially, a long load that follows a short store will also suffer a penalty. __downgrade_write() in rwsem incurs both penalties: the increment operation will not be able to reuse a recently-loaded rwsem value, and its result will not be reused by any recently-following rwsem operation. A comment in the code states that this is because 64-bit immediates are special and expensive; but while they are slightly special (only a single instruction allows them), they aren't expensive: a test shows that two loops, one loading a 32-bit immediate and one loading a 64-bit immediate, both take 1.5 cycles per iteration. Fix this by changing __downgrade_write to use the same add instruction on i386 and on x86_64, so that it uses the same operand size as all the other rwsem functions. Signed-off-by: Avi Kivity <avi@redhat.com> LKML-Reference: <1266049992-17419-1-git-send-email-avi@redhat.com> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26x86-64: support native xadd rwsem implementationLinus Torvalds
commit bafaecd11df15ad5b1e598adc7736afcd38ee13d upstream. This one is much faster than the spinlock based fallback rwsem code, with certain artifical benchmarks having shown 300%+ improvement on threaded page faults etc. Again, note the 32767-thread limit here. So this really does need that whole "make rwsem_count_t be 64-bit and fix the BIAS values to match" extension on top of it, but that is conceptually a totally independent issue. NOT TESTED! The original patch that this all was based on were tested by KAMEZAWA Hiroyuki, but maybe I screwed up something when I created the cleaned-up series, so caveat emptor.. Also note that it _may_ be a good idea to mark some more registers clobbered on x86-64 in the inline asms instead of saving/restoring them. They are inline functions, but they are only used in places where there are not a lot of live registers _anyway_, so doing for example the clobbers of %r8-%r11 in the asm wouldn't make the fast-path code any worse, and would make the slow-path code smaller. (Not that the slow-path really matters to that degree. Saving a few unnecessary registers is the _least_ of our problems when we hit the slow path. The instruction/cycle counting really only matters in the fast path). Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> LKML-Reference: <alpine.LFD.2.00.1001121810410.17145@localhost.localdomain> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26x86-64, rwsem: 64-bit xadd rwsem implementationH. Peter Anvin
commit 1838ef1d782f7527e6defe87e180598622d2d071 upstream. For x86-64, 32767 threads really is not enough. Change rwsem_count_t to a signed long, so that it is 64 bits on x86-64. This required the following changes to the assembly code: a) %z0 doesn't work on all versions of gcc! At least gcc 4.4.2 as shipped with Fedora 12 emits "ll" not "q" for 64 bits, even for integer operands. Newer gccs apparently do this correctly, but avoid this problem by using the _ASM_ macros instead of %z. b) 64 bits immediates are only allowed in "movq $imm,%reg" constructs... no others. Change some of the constraints to "e", and fix the one case where we would have had to use an invalid immediate -- in that case, we only care about the upper half anyway, so just access the upper half. Signed-off-by: H. Peter Anvin <hpa@zytor.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> LKML-Reference: <tip-bafaecd11df15ad5b1e598adc7736afcd38ee13d@git.kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26x86: clean up rwsem type systemLinus Torvalds
commit 5d0b7235d83eefdafda300656e97d368afcafc9a upstream. The fast version of the rwsems (the code that uses xadd) has traditionally only worked on x86-32, and as a result it mixes different kinds of types wildly - they just all happen to be 32-bit. We have "long", we have "__s32", and we have "int". To make it work on x86-64, the types suddenly matter a lot more. It can be either a 32-bit or 64-bit signed type, and both work (with the caveat that a 32-bit counter will only have 15 bits of effective write counters, so it's limited to 32767 users). But whatever type you choose, it needs to be used consistently. This makes a new 'rwsem_counter_t', that is a 32-bit signed type. For a 64-bit type, you'd need to also update the BIAS values. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> LKML-Reference: <alpine.LFD.2.00.1001121755220.17145@localhost.localdomain> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26x86-32: clean up rwsem inline asm statementsLinus Torvalds
commit 59c33fa7791e9948ba467c2b83e307a0d087ab49 upstream. This makes gcc use the right register names and instruction operand sizes automatically for the rwsem inline asm statements. So instead of using "(%%eax)" to specify the memory address that is the semaphore, we use "(%1)" or similar. And instead of forcing the operation to always be 32-bit, we use "%z0", taking the size from the actual semaphore data structure itself. This doesn't actually matter on x86-32, but if we want to use the same inline asm for x86-64, we'll need to have the compiler generate the proper 64-bit names for the registers (%rax instead of %eax), and if we want to use a 64-bit counter too (in order to avoid the 15-bit limit on the write counter that limits concurrent users to 32767 threads), we'll need to be able to generate instructions with "q" accesses rather than "l". Since this header currently isn't enabled on x86-64, none of that matters, but we do want to use the xadd version of the semaphores rather than have to take spinlocks to do a rwsem. The mm->mmap_sem can be heavily contended when you have lots of threads all taking page faults, and the fallback rwsem code that uses a spinlock performs abysmally badly in that case. [ hpa: modified the patch to skip size suffixes entirely when they are redundant due to register operands. ] Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> LKML-Reference: <alpine.LFD.2.00.1001121613560.17145@localhost.localdomain> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2010-04-26x86, cacheinfo: Enable L3 CID only on AMDBorislav Petkov
commit cb19060abfdecac0d1eb2d2f0e7d6b7a3f8bc4f4 upstream. Final stage linking can fail with arch/x86/built-in.o: In function `store_cache_disable': intel_cacheinfo.c:(.text+0xc509): undefined reference to `amd_get_nb_id' arch/x86/built-in.o: In function `show_cache_disable': intel_cacheinfo.c:(.text+0xc7d3): undefined reference to `amd_get_nb_id' when CONFIG_CPU_SUP_AMD is not enabled because the amd_get_nb_id helper is defined in AMD-specific code but also used in generic code (intel_cacheinfo.c). Reorganize the L3 cache index disable code under CONFIG_CPU_SUP_AMD since it is AMD-only anyway. Signed-off-by: Borislav Petkov <borislav.petkov@amd.com> LKML-Reference: <20100218184210.GF20473@aftab> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>