diff options
Diffstat (limited to 'Documentation/edac.txt')
| -rw-r--r-- | Documentation/edac.txt | 283 |
1 files changed, 169 insertions, 114 deletions
diff --git a/Documentation/edac.txt b/Documentation/edac.txt index 8eda3fb6641..73fff13e848 100644 --- a/Documentation/edac.txt +++ b/Documentation/edac.txt @@ -6,6 +6,8 @@ Written by Doug Thompson <dougthompson@xmission.com> 7 Dec 2005 17 Jul 2007 Updated +(c) Mauro Carvalho Chehab +05 Aug 2009 Nehalem interface EDAC is maintained and written by: @@ -23,8 +25,8 @@ first time, it was renamed to 'EDAC'. The bluesmoke project at sourceforge.net is now utilized as a 'staging area' for EDAC development, before it is sent upstream to kernel.org -At the bluesmoke/EDAC project site, is a series of quilt patches against -recent kernels, stored in a SVN respository. For easier downloading, there +At the bluesmoke/EDAC project site is a series of quilt patches against +recent kernels, stored in a SVN repository. For easier downloading, there is also a tarball snapshot available. ============================================================================ @@ -73,14 +75,14 @@ the vendor should tie the parity status bits to 0 if they do not intend to generate parity. Some vendors do not do this, and thus the parity bit can "float" giving false positives. -In the kernel there is a pci device attribute located in sysfs that is +In the kernel there is a PCI device attribute located in sysfs that is checked by the EDAC PCI scanning code. If that attribute is set, -PCI parity/error scannining is skipped for that device. The attribute +PCI parity/error scanning is skipped for that device. The attribute is: broken_parity_status -as is located in /sys/devices/pci<XXX>/0000:XX:YY.Z directorys for +as is located in /sys/devices/pci<XXX>/0000:XX:YY.Z directories for PCI devices. FUTURE HARDWARE SCANNING @@ -194,7 +196,7 @@ csrow3. The representation of the above is reflected in the directory tree in EDAC's sysfs interface. Starting in directory /sys/devices/system/edac/mc each memory controller will be represented -by its own 'mcX' directory, where 'X" is the index of the MC. +by its own 'mcX' directory, where 'X' is the index of the MC. ..../edac/mc/ @@ -205,7 +207,7 @@ by its own 'mcX' directory, where 'X" is the index of the MC. .... Under each 'mcX' directory each 'csrowX' is again represented by a -'csrowX', where 'X" is the csrow index: +'csrowX', where 'X' is the csrow index: .../mc/mc0/ @@ -230,119 +232,22 @@ EDAC control and attribute files. In 'mcX' directories are EDAC control and attribute files for -this 'X" instance of the memory controllers: - - -Counter reset control file: - - 'reset_counters' - - This write-only control file will zero all the statistical counters - for UE and CE errors. Zeroing the counters will also reset the timer - indicating how long since the last counter zero. This is useful - for computing errors/time. Since the counters are always reset at - driver initialization time, no module/kernel parameter is available. - - RUN TIME: echo "anything" >/sys/devices/system/edac/mc/mc0/counter_reset - - This resets the counters on memory controller 0 - - -Seconds since last counter reset control file: - - 'seconds_since_reset' - - This attribute file displays how many seconds have elapsed since the - last counter reset. This can be used with the error counters to - measure error rates. - - - -Memory Controller name attribute file: - - 'mc_name' - - This attribute file displays the type of memory controller - that is being utilized. - - -Total memory managed by this memory controller attribute file: - - 'size_mb' - - This attribute file displays, in count of megabytes, of memory - that this instance of memory controller manages. - - -Total Uncorrectable Errors count attribute file: - - 'ue_count' - - This attribute file displays the total count of uncorrectable - errors that have occurred on this memory controller. If panic_on_ue - is set this counter will not have a chance to increment, - since EDAC will panic the system. - - -Total UE count that had no information attribute fileY: - - 'ue_noinfo_count' - - This attribute file displays the number of UEs that - have occurred have occurred with no informations as to which DIMM - slot is having errors. - - -Total Correctable Errors count attribute file: - - 'ce_count' - - This attribute file displays the total count of correctable - errors that have occurred on this memory controller. This - count is very important to examine. CEs provide early - indications that a DIMM is beginning to fail. This count - field should be monitored for non-zero values and report - such information to the system administrator. - - -Total Correctable Errors count attribute file: - - 'ce_noinfo_count' - - This attribute file displays the number of CEs that - have occurred wherewith no informations as to which DIMM slot - is having errors. Memory is handicapped, but operational, - yet no information is available to indicate which slot - the failing memory is in. This count field should be also - be monitored for non-zero values. - -Device Symlink: - - 'device' - - Symlink to the memory controller device. - -Sdram memory scrubbing rate: - - 'sdram_scrub_rate' - - Read/Write attribute file that controls memory scrubbing. The scrubbing - rate is set by writing a minimum bandwidth in bytes/sec to the attribute - file. The rate will be translated to an internal value that gives at - least the specified rate. - - Reading the file will return the actual scrubbing rate employed. - - If configuration fails or memory scrubbing is not implemented, the value - of the attribute file will be -1. +this 'X' instance of the memory controllers. +For a description of the sysfs API, please see: + Documentation/ABI/testing/sysfs/devices-edac ============================================================================ 'csrowX' DIRECTORIES +When CONFIG_EDAC_LEGACY_SYSFS is enabled, the sysfs will contain the +csrowX directories. As this API doesn't work properly for Rambus, FB-DIMMs +and modern Intel Memory Controllers, this is being deprecated in favor +of dimmX directories. + In the 'csrowX' directories are EDAC control and attribute files for -this 'X" instance of csrow: +this 'X' instance of csrow: Total Uncorrectable Errors count attribute file: @@ -645,7 +550,7 @@ installs itself as: /sys/devices/systm/edac/test-instance in this directory are various controls, a symlink and one or more 'instance' -directorys. +directories. The standard default controls are: @@ -718,3 +623,153 @@ unique drivers for their hardware systems. The 'test_device_edac' sample driver is located at the bluesmoke.sourceforge.net project site for EDAC. +======================================================================= +NEHALEM USAGE OF EDAC APIs + +This chapter documents some EXPERIMENTAL mappings for EDAC API to handle +Nehalem EDAC driver. They will likely be changed on future versions +of the driver. + +Due to the way Nehalem exports Memory Controller data, some adjustments +were done at i7core_edac driver. This chapter will cover those differences + +1) On Nehalem, there are one Memory Controller per Quick Patch Interconnect + (QPI). At the driver, the term "socket" means one QPI. This is + associated with a physical CPU socket. + + Each MC have 3 physical read channels, 3 physical write channels and + 3 logic channels. The driver currently sees it as just 3 channels. + Each channel can have up to 3 DIMMs. + + The minimum known unity is DIMMs. There are no information about csrows. + As EDAC API maps the minimum unity is csrows, the driver sequencially + maps channel/dimm into different csrows. + + For example, supposing the following layout: + Ch0 phy rd0, wr0 (0x063f4031): 2 ranks, UDIMMs + dimm 0 1024 Mb offset: 0, bank: 8, rank: 1, row: 0x4000, col: 0x400 + dimm 1 1024 Mb offset: 4, bank: 8, rank: 1, row: 0x4000, col: 0x400 + Ch1 phy rd1, wr1 (0x063f4031): 2 ranks, UDIMMs + dimm 0 1024 Mb offset: 0, bank: 8, rank: 1, row: 0x4000, col: 0x400 + Ch2 phy rd3, wr3 (0x063f4031): 2 ranks, UDIMMs + dimm 0 1024 Mb offset: 0, bank: 8, rank: 1, row: 0x4000, col: 0x400 + The driver will map it as: + csrow0: channel 0, dimm0 + csrow1: channel 0, dimm1 + csrow2: channel 1, dimm0 + csrow3: channel 2, dimm0 + +exports one + DIMM per csrow. + + Each QPI is exported as a different memory controller. + +2) Nehalem MC has the hability to generate errors. The driver implements this + functionality via some error injection nodes: + + For injecting a memory error, there are some sysfs nodes, under + /sys/devices/system/edac/mc/mc?/: + + inject_addrmatch/*: + Controls the error injection mask register. It is possible to specify + several characteristics of the address to match an error code: + dimm = the affected dimm. Numbers are relative to a channel; + rank = the memory rank; + channel = the channel that will generate an error; + bank = the affected bank; + page = the page address; + column (or col) = the address column. + each of the above values can be set to "any" to match any valid value. + + At driver init, all values are set to any. + + For example, to generate an error at rank 1 of dimm 2, for any channel, + any bank, any page, any column: + echo 2 >/sys/devices/system/edac/mc/mc0/inject_addrmatch/dimm + echo 1 >/sys/devices/system/edac/mc/mc0/inject_addrmatch/rank + + To return to the default behaviour of matching any, you can do: + echo any >/sys/devices/system/edac/mc/mc0/inject_addrmatch/dimm + echo any >/sys/devices/system/edac/mc/mc0/inject_addrmatch/rank + + inject_eccmask: + specifies what bits will have troubles, + + inject_section: + specifies what ECC cache section will get the error: + 3 for both + 2 for the highest + 1 for the lowest + + inject_type: + specifies the type of error, being a combination of the following bits: + bit 0 - repeat + bit 1 - ecc + bit 2 - parity + + inject_enable starts the error generation when something different + than 0 is written. + + All inject vars can be read. root permission is needed for write. + + Datasheet states that the error will only be generated after a write on an + address that matches inject_addrmatch. It seems, however, that reading will + also produce an error. + + For example, the following code will generate an error for any write access + at socket 0, on any DIMM/address on channel 2: + + echo 2 >/sys/devices/system/edac/mc/mc0/inject_addrmatch/channel + echo 2 >/sys/devices/system/edac/mc/mc0/inject_type + echo 64 >/sys/devices/system/edac/mc/mc0/inject_eccmask + echo 3 >/sys/devices/system/edac/mc/mc0/inject_section + echo 1 >/sys/devices/system/edac/mc/mc0/inject_enable + dd if=/dev/mem of=/dev/null seek=16k bs=4k count=1 >& /dev/null + + For socket 1, it is needed to replace "mc0" by "mc1" at the above + commands. + + The generated error message will look like: + + EDAC MC0: UE row 0, channel-a= 0 channel-b= 0 labels "-": NON_FATAL (addr = 0x0075b980, socket=0, Dimm=0, Channel=2, syndrome=0x00000040, count=1, Err=8c0000400001009f:4000080482 (read error: read ECC error)) + +3) Nehalem specific Corrected Error memory counters + + Nehalem have some registers to count memory errors. The driver uses those + registers to report Corrected Errors on devices with Registered Dimms. + + However, those counters don't work with Unregistered Dimms. As the chipset + offers some counters that also work with UDIMMS (but with a worse level of + granularity than the default ones), the driver exposes those registers for + UDIMM memories. + + They can be read by looking at the contents of all_channel_counts/ + + $ for i in /sys/devices/system/edac/mc/mc0/all_channel_counts/*; do echo $i; cat $i; done + /sys/devices/system/edac/mc/mc0/all_channel_counts/udimm0 + 0 + /sys/devices/system/edac/mc/mc0/all_channel_counts/udimm1 + 0 + /sys/devices/system/edac/mc/mc0/all_channel_counts/udimm2 + 0 + + What happens here is that errors on different csrows, but at the same + dimm number will increment the same counter. + So, in this memory mapping: + csrow0: channel 0, dimm0 + csrow1: channel 0, dimm1 + csrow2: channel 1, dimm0 + csrow3: channel 2, dimm0 + The hardware will increment udimm0 for an error at the first dimm at either + csrow0, csrow2 or csrow3; + The hardware will increment udimm1 for an error at the second dimm at either + csrow0, csrow2 or csrow3; + The hardware will increment udimm2 for an error at the third dimm at either + csrow0, csrow2 or csrow3; + +4) Standard error counters + + The standard error counters are generated when an mcelog error is received + by the driver. Since, with udimm, this is counted by software, it is + possible that some errors could be lost. With rdimm's, they displays the + contents of the registers |