Loading CREDITS +1 −1 Original line number Original line Diff line number Diff line Loading @@ -2800,7 +2800,7 @@ D: Starter of Linux1394 effort S: ask per mail for current address S: ask per mail for current address N: Nicolas Pitre N: Nicolas Pitre E: nico@cam.org E: nico@fluxnic.net D: StrongARM SA1100 support integrator & hacker D: StrongARM SA1100 support integrator & hacker D: Xscale PXA architecture D: Xscale PXA architecture D: unified SMC 91C9x/91C11x ethernet driver (smc91x) D: unified SMC 91C9x/91C11x ethernet driver (smc91x) Loading Documentation/00-INDEX +2 −0 Original line number Original line Diff line number Diff line Loading @@ -82,6 +82,8 @@ block/ - info on the Block I/O (BIO) layer. - info on the Block I/O (BIO) layer. blockdev/ blockdev/ - info on block devices & drivers - info on block devices & drivers btmrvl.txt - info on Marvell Bluetooth driver usage. cachetlb.txt cachetlb.txt - describes the cache/TLB flushing interfaces Linux uses. - describes the cache/TLB flushing interfaces Linux uses. cdrom/ cdrom/ Loading Documentation/ABI/testing/sysfs-bus-pci +10 −0 Original line number Original line Diff line number Diff line Loading @@ -84,6 +84,16 @@ Description: from this part of the device tree. from this part of the device tree. Depends on CONFIG_HOTPLUG. Depends on CONFIG_HOTPLUG. What: /sys/bus/pci/devices/.../reset Date: July 2009 Contact: Michael S. Tsirkin <mst@redhat.com> Description: Some devices allow an individual function to be reset without affecting other functions in the same device. For devices that have this support, a file named reset will be present in sysfs. Writing 1 to this file will perform reset. What: /sys/bus/pci/devices/.../vpd What: /sys/bus/pci/devices/.../vpd Date: February 2008 Date: February 2008 Contact: Ben Hutchings <bhutchings@solarflare.com> Contact: Ben Hutchings <bhutchings@solarflare.com> Loading Documentation/DocBook/uio-howto.tmpl +163 −0 Original line number Original line Diff line number Diff line Loading @@ -25,6 +25,10 @@ <year>2006-2008</year> <year>2006-2008</year> <holder>Hans-Jürgen Koch.</holder> <holder>Hans-Jürgen Koch.</holder> </copyright> </copyright> <copyright> <year>2009</year> <holder>Red Hat Inc, Michael S. Tsirkin (mst@redhat.com)</holder> </copyright> <legalnotice> <legalnotice> <para> <para> Loading @@ -41,6 +45,13 @@ GPL version 2. </abstract> </abstract> <revhistory> <revhistory> <revision> <revnumber>0.9</revnumber> <date>2009-07-16</date> <authorinitials>mst</authorinitials> <revremark>Added generic pci driver </revremark> </revision> <revision> <revision> <revnumber>0.8</revnumber> <revnumber>0.8</revnumber> <date>2008-12-24</date> <date>2008-12-24</date> Loading Loading @@ -809,6 +820,158 @@ framework to set up sysfs files for this region. Simply leave it alone. </chapter> </chapter> <chapter id="uio_pci_generic" xreflabel="Using Generic driver for PCI cards"> <?dbhtml filename="uio_pci_generic.html"?> <title>Generic PCI UIO driver</title> <para> The generic driver is a kernel module named uio_pci_generic. It can work with any device compliant to PCI 2.3 (circa 2002) and any compliant PCI Express device. Using this, you only need to write the userspace driver, removing the need to write a hardware-specific kernel module. </para> <sect1 id="uio_pci_generic_binding"> <title>Making the driver recognize the device</title> <para> Since the driver does not declare any device ids, it will not get loaded automatically and will not automatically bind to any devices, you must load it and allocate id to the driver yourself. For example: <programlisting> modprobe uio_pci_generic echo "8086 10f5" > /sys/bus/pci/drivers/uio_pci_generic/new_id </programlisting> </para> <para> If there already is a hardware specific kernel driver for your device, the generic driver still won't bind to it, in this case if you want to use the generic driver (why would you?) you'll have to manually unbind the hardware specific driver and bind the generic driver, like this: <programlisting> echo -n 0000:00:19.0 > /sys/bus/pci/drivers/e1000e/unbind echo -n 0000:00:19.0 > /sys/bus/pci/drivers/uio_pci_generic/bind </programlisting> </para> <para> You can verify that the device has been bound to the driver by looking for it in sysfs, for example like the following: <programlisting> ls -l /sys/bus/pci/devices/0000:00:19.0/driver </programlisting> Which if successful should print <programlisting> .../0000:00:19.0/driver -> ../../../bus/pci/drivers/uio_pci_generic </programlisting> Note that the generic driver will not bind to old PCI 2.2 devices. If binding the device failed, run the following command: <programlisting> dmesg </programlisting> and look in the output for failure reasons </para> </sect1> <sect1 id="uio_pci_generic_internals"> <title>Things to know about uio_pci_generic</title> <para> Interrupts are handled using the Interrupt Disable bit in the PCI command register and Interrupt Status bit in the PCI status register. All devices compliant to PCI 2.3 (circa 2002) and all compliant PCI Express devices should support these bits. uio_pci_generic detects this support, and won't bind to devices which do not support the Interrupt Disable Bit in the command register. </para> <para> On each interrupt, uio_pci_generic sets the Interrupt Disable bit. This prevents the device from generating further interrupts until the bit is cleared. The userspace driver should clear this bit before blocking and waiting for more interrupts. </para> </sect1> <sect1 id="uio_pci_generic_userspace"> <title>Writing userspace driver using uio_pci_generic</title> <para> Userspace driver can use pci sysfs interface, or the libpci libray that wraps it, to talk to the device and to re-enable interrupts by writing to the command register. </para> </sect1> <sect1 id="uio_pci_generic_example"> <title>Example code using uio_pci_generic</title> <para> Here is some sample userspace driver code using uio_pci_generic: <programlisting> #include <stdlib.h> #include <stdio.h> #include <unistd.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <errno.h> int main() { int uiofd; int configfd; int err; int i; unsigned icount; unsigned char command_high; uiofd = open("/dev/uio0", O_RDONLY); if (uiofd < 0) { perror("uio open:"); return errno; } configfd = open("/sys/class/uio/uio0/device/config", O_RDWR); if (uiofd < 0) { perror("config open:"); return errno; } /* Read and cache command value */ err = pread(configfd, &command_high, 1, 5); if (err != 1) { perror("command config read:"); return errno; } command_high &= ~0x4; for(i = 0;; ++i) { /* Print out a message, for debugging. */ if (i == 0) fprintf(stderr, "Started uio test driver.\n"); else fprintf(stderr, "Interrupts: %d\n", icount); /****************************************/ /* Here we got an interrupt from the device. Do something to it. */ /****************************************/ /* Re-enable interrupts. */ err = pwrite(configfd, &command_high, 1, 5); if (err != 1) { perror("config write:"); break; } /* Wait for next interrupt. */ err = read(uiofd, &icount, 4); if (err != 4) { perror("uio read:"); break; } } return errno; } </programlisting> </para> </sect1> </chapter> <appendix id="app1"> <appendix id="app1"> <title>Further information</title> <title>Further information</title> <itemizedlist> <itemizedlist> Loading Documentation/PCI/pci-error-recovery.txt +77 −42 Original line number Original line Diff line number Diff line Loading @@ -4,15 +4,17 @@ February 2, 2006 February 2, 2006 Current document maintainer: Current document maintainer: Linas Vepstas <linas@austin.ibm.com> Linas Vepstas <linasvepstas@gmail.com> updated by Richard Lary <rlary@us.ibm.com> and Mike Mason <mmlnx@us.ibm.com> on 27-Jul-2009 Many PCI bus controllers are able to detect a variety of hardware Many PCI bus controllers are able to detect a variety of hardware PCI errors on the bus, such as parity errors on the data and address PCI errors on the bus, such as parity errors on the data and address busses, as well as SERR and PERR errors. Some of the more advanced busses, as well as SERR and PERR errors. Some of the more advanced chipsets are able to deal with these errors; these include PCI-E chipsets, chipsets are able to deal with these errors; these include PCI-E chipsets, and the PCI-host bridges found on IBM Power4 and Power5-based pSeries and the PCI-host bridges found on IBM Power4, Power5 and Power6-based boxes. A typical action taken is to disconnect the affected device, pSeries boxes. A typical action taken is to disconnect the affected device, halting all I/O to it. The goal of a disconnection is to avoid system halting all I/O to it. The goal of a disconnection is to avoid system corruption; for example, to halt system memory corruption due to DMA's corruption; for example, to halt system memory corruption due to DMA's to "wild" addresses. Typically, a reconnection mechanism is also to "wild" addresses. Typically, a reconnection mechanism is also Loading @@ -37,10 +39,11 @@ is forced by the need to handle multi-function devices, that is, devices that have multiple device drivers associated with them. devices that have multiple device drivers associated with them. In the first stage, each driver is allowed to indicate what type In the first stage, each driver is allowed to indicate what type of reset it desires, the choices being a simple re-enabling of I/O of reset it desires, the choices being a simple re-enabling of I/O or requesting a hard reset (a full electrical #RST of the PCI card). or requesting a slot reset. If any driver requests a full reset, that is what will be done. After a full reset and/or a re-enabling of I/O, all drivers are If any driver requests a slot reset, that is what will be done. After a reset and/or a re-enabling of I/O, all drivers are again notified, so that they may then perform any device setup/config again notified, so that they may then perform any device setup/config that may be required. After these have all completed, a final that may be required. After these have all completed, a final "resume normal operations" event is sent out. "resume normal operations" event is sent out. Loading Loading @@ -101,7 +104,7 @@ if it implements any, it must implement error_detected(). If a callback is not implemented, the corresponding feature is considered unsupported. is not implemented, the corresponding feature is considered unsupported. For example, if mmio_enabled() and resume() aren't there, then it For example, if mmio_enabled() and resume() aren't there, then it is assumed that the driver is not doing any direct recovery and requires is assumed that the driver is not doing any direct recovery and requires a reset. If link_reset() is not implemented, the card is assumed as a slot reset. If link_reset() is not implemented, the card is assumed to not care about link resets. Typically a driver will want to know about not care about link resets. Typically a driver will want to know about a slot_reset(). a slot_reset(). Loading @@ -111,7 +114,7 @@ sequence described below. STEP 0: Error Event STEP 0: Error Event ------------------- ------------------- PCI bus error is detect by the PCI hardware. On powerpc, the slot A PCI bus error is detected by the PCI hardware. On powerpc, the slot is isolated, in that all I/O is blocked: all reads return 0xffffffff, is isolated, in that all I/O is blocked: all reads return 0xffffffff, all writes are ignored. all writes are ignored. Loading Loading @@ -139,7 +142,7 @@ The driver must return one of the following result codes: a chance to extract some diagnostic information (see a chance to extract some diagnostic information (see mmio_enable, below). mmio_enable, below). - PCI_ERS_RESULT_NEED_RESET: - PCI_ERS_RESULT_NEED_RESET: Driver returns this if it can't recover without a hard Driver returns this if it can't recover without a slot reset. slot reset. - PCI_ERS_RESULT_DISCONNECT: - PCI_ERS_RESULT_DISCONNECT: Driver returns this if it doesn't want to recover at all. Driver returns this if it doesn't want to recover at all. Loading Loading @@ -169,11 +172,11 @@ is STEP 6 (Permanent Failure). >>> The current powerpc implementation doesn't much care if the device >>> The current powerpc implementation doesn't much care if the device >>> attempts I/O at this point, or not. I/O's will fail, returning >>> attempts I/O at this point, or not. I/O's will fail, returning >>> a value of 0xff on read, and writes will be dropped. If the device >>> a value of 0xff on read, and writes will be dropped. If more than >>> driver attempts more than 10K I/O's to a frozen adapter, it will >>> EEH_MAX_FAILS I/O's are attempted to a frozen adapter, EEH >>> assume that the device driver has gone into an infinite loop, and >>> assumes that the device driver has gone into an infinite loop >>> it will panic the kernel. There doesn't seem to be any other >>> and prints an error to syslog. A reboot is then required to >>> way of stopping a device driver that insists on spinning on I/O. >>> get the device working again. STEP 2: MMIO Enabled STEP 2: MMIO Enabled ------------------- ------------------- Loading @@ -182,15 +185,14 @@ DMA), and then calls the mmio_enabled() callback on all affected device drivers. device drivers. This is the "early recovery" call. IOs are allowed again, but DMA is This is the "early recovery" call. IOs are allowed again, but DMA is not (hrm... to be discussed, I prefer not), with some restrictions. This not, with some restrictions. This is NOT a callback for the driver to is NOT a callback for the driver to start operations again, only to start operations again, only to peek/poke at the device, extract diagnostic peek/poke at the device, extract diagnostic information, if any, and information, if any, and eventually do things like trigger a device local eventually do things like trigger a device local reset or some such, reset or some such, but not restart operations. This callback is made if but not restart operations. This is callback is made if all drivers on all drivers on a segment agree that they can try to recover and if no automatic a segment agree that they can try to recover and if no automatic link reset link reset was performed by the HW. If the platform can't just re-enable IOs was performed by the HW. If the platform can't just re-enable IOs without without a slot reset or a link reset, it will not call this callback, and a slot reset or a link reset, it wont call this callback, and instead instead will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset) will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset) >>> The following is proposed; no platform implements this yet: >>> The following is proposed; no platform implements this yet: >>> Proposal: All I/O's should be done _synchronously_ from within >>> Proposal: All I/O's should be done _synchronously_ from within Loading Loading @@ -228,9 +230,6 @@ proceeds to either STEP3 (Link Reset) or to STEP 5 (Resume Operations). If any driver returned PCI_ERS_RESULT_NEED_RESET, then the platform If any driver returned PCI_ERS_RESULT_NEED_RESET, then the platform proceeds to STEP 4 (Slot Reset) proceeds to STEP 4 (Slot Reset) >>> The current powerpc implementation does not implement this callback. STEP 3: Link Reset STEP 3: Link Reset ------------------ ------------------ The platform resets the link, and then calls the link_reset() callback The platform resets the link, and then calls the link_reset() callback Loading @@ -253,16 +252,33 @@ The platform then proceeds to either STEP 4 (Slot Reset) or STEP 5 >>> The current powerpc implementation does not implement this callback. >>> The current powerpc implementation does not implement this callback. STEP 4: Slot Reset STEP 4: Slot Reset ------------------ ------------------ The platform performs a soft or hard reset of the device, and then calls the slot_reset() callback. A soft reset consists of asserting the adapter #RST line and then In response to a return value of PCI_ERS_RESULT_NEED_RESET, the the platform will peform a slot reset on the requesting PCI device(s). The actual steps taken by a platform to perform a slot reset will be platform-dependent. Upon completion of slot reset, the platform will call the device slot_reset() callback. Powerpc platforms implement two levels of slot reset: soft reset(default) and fundamental(optional) reset. Powerpc soft reset consists of asserting the adapter #RST line and then restoring the PCI BAR's and PCI configuration header to a state restoring the PCI BAR's and PCI configuration header to a state that is equivalent to what it would be after a fresh system that is equivalent to what it would be after a fresh system power-on followed by power-on BIOS/system firmware initialization. power-on followed by power-on BIOS/system firmware initialization. Soft reset is also known as hot-reset. Powerpc fundamental reset is supported by PCI Express cards only and results in device's state machines, hardware logic, port states and configuration registers to initialize to their default conditions. For most PCI devices, a soft reset will be sufficient for recovery. Optional fundamental reset is provided to support a limited number of PCI Express PCI devices for which a soft reset is not sufficient for recovery. If the platform supports PCI hotplug, then the reset might be If the platform supports PCI hotplug, then the reset might be performed by toggling the slot electrical power off/on. performed by toggling the slot electrical power off/on. Loading @@ -274,10 +290,12 @@ may result in hung devices, kernel panics, or silent data corruption. This call gives drivers the chance to re-initialize the hardware This call gives drivers the chance to re-initialize the hardware (re-download firmware, etc.). At this point, the driver may assume (re-download firmware, etc.). At this point, the driver may assume that he card is in a fresh state and is fully functional. In that the card is in a fresh state and is fully functional. The slot particular, interrupt generation should work normally. is unfrozen and the driver has full access to PCI config space, memory mapped I/O space and DMA. Interrupts (Legacy, MSI, or MSI-X) will also be available. Drivers should not yet restart normal I/O processing operations Drivers should not restart normal I/O processing operations at this point. If all device drivers report success on this at this point. If all device drivers report success on this callback, the platform will call resume() to complete the sequence, callback, the platform will call resume() to complete the sequence, and let the driver restart normal I/O processing. and let the driver restart normal I/O processing. Loading @@ -302,11 +320,21 @@ driver performs device init only from PCI function 0: - PCI_ERS_RESULT_DISCONNECT - PCI_ERS_RESULT_DISCONNECT Same as above. Same as above. Drivers for PCI Express cards that require a fundamental reset must set the needs_freset bit in the pci_dev structure in their probe function. For example, the QLogic qla2xxx driver sets the needs_freset bit for certain PCI card types: + /* Set EEH reset type to fundamental if required by hba */ + if (IS_QLA24XX(ha) || IS_QLA25XX(ha) || IS_QLA81XX(ha)) + pdev->needs_freset = 1; + Platform proceeds either to STEP 5 (Resume Operations) or STEP 6 (Permanent Platform proceeds either to STEP 5 (Resume Operations) or STEP 6 (Permanent Failure). Failure). >>> The current powerpc implementation does not currently try a >>> The current powerpc implementation does not try a power-cycle >>> power-cycle reset if the driver returned PCI_ERS_RESULT_DISCONNECT. >>> reset if the driver returned PCI_ERS_RESULT_DISCONNECT. >>> However, it probably should. >>> However, it probably should. Loading Loading @@ -348,7 +376,7 @@ software errors. Conclusion; General Remarks Conclusion; General Remarks --------------------------- --------------------------- The way those callbacks are called is platform policy. A platform with The way the callbacks are called is platform policy. A platform with no slot reset capability may want to just "ignore" drivers that can't no slot reset capability may want to just "ignore" drivers that can't recover (disconnect them) and try to let other cards on the same segment recover (disconnect them) and try to let other cards on the same segment recover. Keep in mind that in most real life cases, though, there will recover. Keep in mind that in most real life cases, though, there will Loading @@ -361,8 +389,8 @@ That is, the recovery API only requires that: - There is no guarantee that interrupt delivery can proceed from any - There is no guarantee that interrupt delivery can proceed from any device on the segment starting from the error detection and until the device on the segment starting from the error detection and until the resume callback is sent, at which point interrupts are expected to be slot_reset callback is called, at which point interrupts are expected fully operational. to be fully operational. - There is no guarantee that interrupt delivery is stopped, that is, - There is no guarantee that interrupt delivery is stopped, that is, a driver that gets an interrupt after detecting an error, or that detects a driver that gets an interrupt after detecting an error, or that detects Loading @@ -381,16 +409,23 @@ anyway :) >>> Implementation details for the powerpc platform are discussed in >>> Implementation details for the powerpc platform are discussed in >>> the file Documentation/powerpc/eeh-pci-error-recovery.txt >>> the file Documentation/powerpc/eeh-pci-error-recovery.txt >>> As of this writing, there are six device drivers with patches >>> As of this writing, there is a growing list of device drivers with >>> implementing error recovery. Not all of these patches are in >>> patches implementing error recovery. Not all of these patches are in >>> mainline yet. These may be used as "examples": >>> mainline yet. These may be used as "examples": >>> >>> >>> drivers/scsi/ipr.c >>> drivers/scsi/ipr >>> drivers/scsi/sym53cxx_2 >>> drivers/scsi/sym53c8xx_2 >>> drivers/scsi/qla2xxx >>> drivers/scsi/lpfc >>> drivers/next/bnx2.c >>> drivers/next/e100.c >>> drivers/next/e100.c >>> drivers/net/e1000 >>> drivers/net/e1000 >>> drivers/net/e1000e >>> drivers/net/ixgb >>> drivers/net/ixgb >>> drivers/net/ixgbe >>> drivers/net/cxgb3 >>> drivers/net/s2io.c >>> drivers/net/s2io.c >>> drivers/net/qlge The End The End ------- ------- Loading
CREDITS +1 −1 Original line number Original line Diff line number Diff line Loading @@ -2800,7 +2800,7 @@ D: Starter of Linux1394 effort S: ask per mail for current address S: ask per mail for current address N: Nicolas Pitre N: Nicolas Pitre E: nico@cam.org E: nico@fluxnic.net D: StrongARM SA1100 support integrator & hacker D: StrongARM SA1100 support integrator & hacker D: Xscale PXA architecture D: Xscale PXA architecture D: unified SMC 91C9x/91C11x ethernet driver (smc91x) D: unified SMC 91C9x/91C11x ethernet driver (smc91x) Loading
Documentation/00-INDEX +2 −0 Original line number Original line Diff line number Diff line Loading @@ -82,6 +82,8 @@ block/ - info on the Block I/O (BIO) layer. - info on the Block I/O (BIO) layer. blockdev/ blockdev/ - info on block devices & drivers - info on block devices & drivers btmrvl.txt - info on Marvell Bluetooth driver usage. cachetlb.txt cachetlb.txt - describes the cache/TLB flushing interfaces Linux uses. - describes the cache/TLB flushing interfaces Linux uses. cdrom/ cdrom/ Loading
Documentation/ABI/testing/sysfs-bus-pci +10 −0 Original line number Original line Diff line number Diff line Loading @@ -84,6 +84,16 @@ Description: from this part of the device tree. from this part of the device tree. Depends on CONFIG_HOTPLUG. Depends on CONFIG_HOTPLUG. What: /sys/bus/pci/devices/.../reset Date: July 2009 Contact: Michael S. Tsirkin <mst@redhat.com> Description: Some devices allow an individual function to be reset without affecting other functions in the same device. For devices that have this support, a file named reset will be present in sysfs. Writing 1 to this file will perform reset. What: /sys/bus/pci/devices/.../vpd What: /sys/bus/pci/devices/.../vpd Date: February 2008 Date: February 2008 Contact: Ben Hutchings <bhutchings@solarflare.com> Contact: Ben Hutchings <bhutchings@solarflare.com> Loading
Documentation/DocBook/uio-howto.tmpl +163 −0 Original line number Original line Diff line number Diff line Loading @@ -25,6 +25,10 @@ <year>2006-2008</year> <year>2006-2008</year> <holder>Hans-Jürgen Koch.</holder> <holder>Hans-Jürgen Koch.</holder> </copyright> </copyright> <copyright> <year>2009</year> <holder>Red Hat Inc, Michael S. Tsirkin (mst@redhat.com)</holder> </copyright> <legalnotice> <legalnotice> <para> <para> Loading @@ -41,6 +45,13 @@ GPL version 2. </abstract> </abstract> <revhistory> <revhistory> <revision> <revnumber>0.9</revnumber> <date>2009-07-16</date> <authorinitials>mst</authorinitials> <revremark>Added generic pci driver </revremark> </revision> <revision> <revision> <revnumber>0.8</revnumber> <revnumber>0.8</revnumber> <date>2008-12-24</date> <date>2008-12-24</date> Loading Loading @@ -809,6 +820,158 @@ framework to set up sysfs files for this region. Simply leave it alone. </chapter> </chapter> <chapter id="uio_pci_generic" xreflabel="Using Generic driver for PCI cards"> <?dbhtml filename="uio_pci_generic.html"?> <title>Generic PCI UIO driver</title> <para> The generic driver is a kernel module named uio_pci_generic. It can work with any device compliant to PCI 2.3 (circa 2002) and any compliant PCI Express device. Using this, you only need to write the userspace driver, removing the need to write a hardware-specific kernel module. </para> <sect1 id="uio_pci_generic_binding"> <title>Making the driver recognize the device</title> <para> Since the driver does not declare any device ids, it will not get loaded automatically and will not automatically bind to any devices, you must load it and allocate id to the driver yourself. For example: <programlisting> modprobe uio_pci_generic echo "8086 10f5" > /sys/bus/pci/drivers/uio_pci_generic/new_id </programlisting> </para> <para> If there already is a hardware specific kernel driver for your device, the generic driver still won't bind to it, in this case if you want to use the generic driver (why would you?) you'll have to manually unbind the hardware specific driver and bind the generic driver, like this: <programlisting> echo -n 0000:00:19.0 > /sys/bus/pci/drivers/e1000e/unbind echo -n 0000:00:19.0 > /sys/bus/pci/drivers/uio_pci_generic/bind </programlisting> </para> <para> You can verify that the device has been bound to the driver by looking for it in sysfs, for example like the following: <programlisting> ls -l /sys/bus/pci/devices/0000:00:19.0/driver </programlisting> Which if successful should print <programlisting> .../0000:00:19.0/driver -> ../../../bus/pci/drivers/uio_pci_generic </programlisting> Note that the generic driver will not bind to old PCI 2.2 devices. If binding the device failed, run the following command: <programlisting> dmesg </programlisting> and look in the output for failure reasons </para> </sect1> <sect1 id="uio_pci_generic_internals"> <title>Things to know about uio_pci_generic</title> <para> Interrupts are handled using the Interrupt Disable bit in the PCI command register and Interrupt Status bit in the PCI status register. All devices compliant to PCI 2.3 (circa 2002) and all compliant PCI Express devices should support these bits. uio_pci_generic detects this support, and won't bind to devices which do not support the Interrupt Disable Bit in the command register. </para> <para> On each interrupt, uio_pci_generic sets the Interrupt Disable bit. This prevents the device from generating further interrupts until the bit is cleared. The userspace driver should clear this bit before blocking and waiting for more interrupts. </para> </sect1> <sect1 id="uio_pci_generic_userspace"> <title>Writing userspace driver using uio_pci_generic</title> <para> Userspace driver can use pci sysfs interface, or the libpci libray that wraps it, to talk to the device and to re-enable interrupts by writing to the command register. </para> </sect1> <sect1 id="uio_pci_generic_example"> <title>Example code using uio_pci_generic</title> <para> Here is some sample userspace driver code using uio_pci_generic: <programlisting> #include <stdlib.h> #include <stdio.h> #include <unistd.h> #include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> #include <errno.h> int main() { int uiofd; int configfd; int err; int i; unsigned icount; unsigned char command_high; uiofd = open("/dev/uio0", O_RDONLY); if (uiofd < 0) { perror("uio open:"); return errno; } configfd = open("/sys/class/uio/uio0/device/config", O_RDWR); if (uiofd < 0) { perror("config open:"); return errno; } /* Read and cache command value */ err = pread(configfd, &command_high, 1, 5); if (err != 1) { perror("command config read:"); return errno; } command_high &= ~0x4; for(i = 0;; ++i) { /* Print out a message, for debugging. */ if (i == 0) fprintf(stderr, "Started uio test driver.\n"); else fprintf(stderr, "Interrupts: %d\n", icount); /****************************************/ /* Here we got an interrupt from the device. Do something to it. */ /****************************************/ /* Re-enable interrupts. */ err = pwrite(configfd, &command_high, 1, 5); if (err != 1) { perror("config write:"); break; } /* Wait for next interrupt. */ err = read(uiofd, &icount, 4); if (err != 4) { perror("uio read:"); break; } } return errno; } </programlisting> </para> </sect1> </chapter> <appendix id="app1"> <appendix id="app1"> <title>Further information</title> <title>Further information</title> <itemizedlist> <itemizedlist> Loading
Documentation/PCI/pci-error-recovery.txt +77 −42 Original line number Original line Diff line number Diff line Loading @@ -4,15 +4,17 @@ February 2, 2006 February 2, 2006 Current document maintainer: Current document maintainer: Linas Vepstas <linas@austin.ibm.com> Linas Vepstas <linasvepstas@gmail.com> updated by Richard Lary <rlary@us.ibm.com> and Mike Mason <mmlnx@us.ibm.com> on 27-Jul-2009 Many PCI bus controllers are able to detect a variety of hardware Many PCI bus controllers are able to detect a variety of hardware PCI errors on the bus, such as parity errors on the data and address PCI errors on the bus, such as parity errors on the data and address busses, as well as SERR and PERR errors. Some of the more advanced busses, as well as SERR and PERR errors. Some of the more advanced chipsets are able to deal with these errors; these include PCI-E chipsets, chipsets are able to deal with these errors; these include PCI-E chipsets, and the PCI-host bridges found on IBM Power4 and Power5-based pSeries and the PCI-host bridges found on IBM Power4, Power5 and Power6-based boxes. A typical action taken is to disconnect the affected device, pSeries boxes. A typical action taken is to disconnect the affected device, halting all I/O to it. The goal of a disconnection is to avoid system halting all I/O to it. The goal of a disconnection is to avoid system corruption; for example, to halt system memory corruption due to DMA's corruption; for example, to halt system memory corruption due to DMA's to "wild" addresses. Typically, a reconnection mechanism is also to "wild" addresses. Typically, a reconnection mechanism is also Loading @@ -37,10 +39,11 @@ is forced by the need to handle multi-function devices, that is, devices that have multiple device drivers associated with them. devices that have multiple device drivers associated with them. In the first stage, each driver is allowed to indicate what type In the first stage, each driver is allowed to indicate what type of reset it desires, the choices being a simple re-enabling of I/O of reset it desires, the choices being a simple re-enabling of I/O or requesting a hard reset (a full electrical #RST of the PCI card). or requesting a slot reset. If any driver requests a full reset, that is what will be done. After a full reset and/or a re-enabling of I/O, all drivers are If any driver requests a slot reset, that is what will be done. After a reset and/or a re-enabling of I/O, all drivers are again notified, so that they may then perform any device setup/config again notified, so that they may then perform any device setup/config that may be required. After these have all completed, a final that may be required. After these have all completed, a final "resume normal operations" event is sent out. "resume normal operations" event is sent out. Loading Loading @@ -101,7 +104,7 @@ if it implements any, it must implement error_detected(). If a callback is not implemented, the corresponding feature is considered unsupported. is not implemented, the corresponding feature is considered unsupported. For example, if mmio_enabled() and resume() aren't there, then it For example, if mmio_enabled() and resume() aren't there, then it is assumed that the driver is not doing any direct recovery and requires is assumed that the driver is not doing any direct recovery and requires a reset. If link_reset() is not implemented, the card is assumed as a slot reset. If link_reset() is not implemented, the card is assumed to not care about link resets. Typically a driver will want to know about not care about link resets. Typically a driver will want to know about a slot_reset(). a slot_reset(). Loading @@ -111,7 +114,7 @@ sequence described below. STEP 0: Error Event STEP 0: Error Event ------------------- ------------------- PCI bus error is detect by the PCI hardware. On powerpc, the slot A PCI bus error is detected by the PCI hardware. On powerpc, the slot is isolated, in that all I/O is blocked: all reads return 0xffffffff, is isolated, in that all I/O is blocked: all reads return 0xffffffff, all writes are ignored. all writes are ignored. Loading Loading @@ -139,7 +142,7 @@ The driver must return one of the following result codes: a chance to extract some diagnostic information (see a chance to extract some diagnostic information (see mmio_enable, below). mmio_enable, below). - PCI_ERS_RESULT_NEED_RESET: - PCI_ERS_RESULT_NEED_RESET: Driver returns this if it can't recover without a hard Driver returns this if it can't recover without a slot reset. slot reset. - PCI_ERS_RESULT_DISCONNECT: - PCI_ERS_RESULT_DISCONNECT: Driver returns this if it doesn't want to recover at all. Driver returns this if it doesn't want to recover at all. Loading Loading @@ -169,11 +172,11 @@ is STEP 6 (Permanent Failure). >>> The current powerpc implementation doesn't much care if the device >>> The current powerpc implementation doesn't much care if the device >>> attempts I/O at this point, or not. I/O's will fail, returning >>> attempts I/O at this point, or not. I/O's will fail, returning >>> a value of 0xff on read, and writes will be dropped. If the device >>> a value of 0xff on read, and writes will be dropped. If more than >>> driver attempts more than 10K I/O's to a frozen adapter, it will >>> EEH_MAX_FAILS I/O's are attempted to a frozen adapter, EEH >>> assume that the device driver has gone into an infinite loop, and >>> assumes that the device driver has gone into an infinite loop >>> it will panic the kernel. There doesn't seem to be any other >>> and prints an error to syslog. A reboot is then required to >>> way of stopping a device driver that insists on spinning on I/O. >>> get the device working again. STEP 2: MMIO Enabled STEP 2: MMIO Enabled ------------------- ------------------- Loading @@ -182,15 +185,14 @@ DMA), and then calls the mmio_enabled() callback on all affected device drivers. device drivers. This is the "early recovery" call. IOs are allowed again, but DMA is This is the "early recovery" call. IOs are allowed again, but DMA is not (hrm... to be discussed, I prefer not), with some restrictions. This not, with some restrictions. This is NOT a callback for the driver to is NOT a callback for the driver to start operations again, only to start operations again, only to peek/poke at the device, extract diagnostic peek/poke at the device, extract diagnostic information, if any, and information, if any, and eventually do things like trigger a device local eventually do things like trigger a device local reset or some such, reset or some such, but not restart operations. This callback is made if but not restart operations. This is callback is made if all drivers on all drivers on a segment agree that they can try to recover and if no automatic a segment agree that they can try to recover and if no automatic link reset link reset was performed by the HW. If the platform can't just re-enable IOs was performed by the HW. If the platform can't just re-enable IOs without without a slot reset or a link reset, it will not call this callback, and a slot reset or a link reset, it wont call this callback, and instead instead will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset) will have gone directly to STEP 3 (Link Reset) or STEP 4 (Slot Reset) >>> The following is proposed; no platform implements this yet: >>> The following is proposed; no platform implements this yet: >>> Proposal: All I/O's should be done _synchronously_ from within >>> Proposal: All I/O's should be done _synchronously_ from within Loading Loading @@ -228,9 +230,6 @@ proceeds to either STEP3 (Link Reset) or to STEP 5 (Resume Operations). If any driver returned PCI_ERS_RESULT_NEED_RESET, then the platform If any driver returned PCI_ERS_RESULT_NEED_RESET, then the platform proceeds to STEP 4 (Slot Reset) proceeds to STEP 4 (Slot Reset) >>> The current powerpc implementation does not implement this callback. STEP 3: Link Reset STEP 3: Link Reset ------------------ ------------------ The platform resets the link, and then calls the link_reset() callback The platform resets the link, and then calls the link_reset() callback Loading @@ -253,16 +252,33 @@ The platform then proceeds to either STEP 4 (Slot Reset) or STEP 5 >>> The current powerpc implementation does not implement this callback. >>> The current powerpc implementation does not implement this callback. STEP 4: Slot Reset STEP 4: Slot Reset ------------------ ------------------ The platform performs a soft or hard reset of the device, and then calls the slot_reset() callback. A soft reset consists of asserting the adapter #RST line and then In response to a return value of PCI_ERS_RESULT_NEED_RESET, the the platform will peform a slot reset on the requesting PCI device(s). The actual steps taken by a platform to perform a slot reset will be platform-dependent. Upon completion of slot reset, the platform will call the device slot_reset() callback. Powerpc platforms implement two levels of slot reset: soft reset(default) and fundamental(optional) reset. Powerpc soft reset consists of asserting the adapter #RST line and then restoring the PCI BAR's and PCI configuration header to a state restoring the PCI BAR's and PCI configuration header to a state that is equivalent to what it would be after a fresh system that is equivalent to what it would be after a fresh system power-on followed by power-on BIOS/system firmware initialization. power-on followed by power-on BIOS/system firmware initialization. Soft reset is also known as hot-reset. Powerpc fundamental reset is supported by PCI Express cards only and results in device's state machines, hardware logic, port states and configuration registers to initialize to their default conditions. For most PCI devices, a soft reset will be sufficient for recovery. Optional fundamental reset is provided to support a limited number of PCI Express PCI devices for which a soft reset is not sufficient for recovery. If the platform supports PCI hotplug, then the reset might be If the platform supports PCI hotplug, then the reset might be performed by toggling the slot electrical power off/on. performed by toggling the slot electrical power off/on. Loading @@ -274,10 +290,12 @@ may result in hung devices, kernel panics, or silent data corruption. This call gives drivers the chance to re-initialize the hardware This call gives drivers the chance to re-initialize the hardware (re-download firmware, etc.). At this point, the driver may assume (re-download firmware, etc.). At this point, the driver may assume that he card is in a fresh state and is fully functional. In that the card is in a fresh state and is fully functional. The slot particular, interrupt generation should work normally. is unfrozen and the driver has full access to PCI config space, memory mapped I/O space and DMA. Interrupts (Legacy, MSI, or MSI-X) will also be available. Drivers should not yet restart normal I/O processing operations Drivers should not restart normal I/O processing operations at this point. If all device drivers report success on this at this point. If all device drivers report success on this callback, the platform will call resume() to complete the sequence, callback, the platform will call resume() to complete the sequence, and let the driver restart normal I/O processing. and let the driver restart normal I/O processing. Loading @@ -302,11 +320,21 @@ driver performs device init only from PCI function 0: - PCI_ERS_RESULT_DISCONNECT - PCI_ERS_RESULT_DISCONNECT Same as above. Same as above. Drivers for PCI Express cards that require a fundamental reset must set the needs_freset bit in the pci_dev structure in their probe function. For example, the QLogic qla2xxx driver sets the needs_freset bit for certain PCI card types: + /* Set EEH reset type to fundamental if required by hba */ + if (IS_QLA24XX(ha) || IS_QLA25XX(ha) || IS_QLA81XX(ha)) + pdev->needs_freset = 1; + Platform proceeds either to STEP 5 (Resume Operations) or STEP 6 (Permanent Platform proceeds either to STEP 5 (Resume Operations) or STEP 6 (Permanent Failure). Failure). >>> The current powerpc implementation does not currently try a >>> The current powerpc implementation does not try a power-cycle >>> power-cycle reset if the driver returned PCI_ERS_RESULT_DISCONNECT. >>> reset if the driver returned PCI_ERS_RESULT_DISCONNECT. >>> However, it probably should. >>> However, it probably should. Loading Loading @@ -348,7 +376,7 @@ software errors. Conclusion; General Remarks Conclusion; General Remarks --------------------------- --------------------------- The way those callbacks are called is platform policy. A platform with The way the callbacks are called is platform policy. A platform with no slot reset capability may want to just "ignore" drivers that can't no slot reset capability may want to just "ignore" drivers that can't recover (disconnect them) and try to let other cards on the same segment recover (disconnect them) and try to let other cards on the same segment recover. Keep in mind that in most real life cases, though, there will recover. Keep in mind that in most real life cases, though, there will Loading @@ -361,8 +389,8 @@ That is, the recovery API only requires that: - There is no guarantee that interrupt delivery can proceed from any - There is no guarantee that interrupt delivery can proceed from any device on the segment starting from the error detection and until the device on the segment starting from the error detection and until the resume callback is sent, at which point interrupts are expected to be slot_reset callback is called, at which point interrupts are expected fully operational. to be fully operational. - There is no guarantee that interrupt delivery is stopped, that is, - There is no guarantee that interrupt delivery is stopped, that is, a driver that gets an interrupt after detecting an error, or that detects a driver that gets an interrupt after detecting an error, or that detects Loading @@ -381,16 +409,23 @@ anyway :) >>> Implementation details for the powerpc platform are discussed in >>> Implementation details for the powerpc platform are discussed in >>> the file Documentation/powerpc/eeh-pci-error-recovery.txt >>> the file Documentation/powerpc/eeh-pci-error-recovery.txt >>> As of this writing, there are six device drivers with patches >>> As of this writing, there is a growing list of device drivers with >>> implementing error recovery. Not all of these patches are in >>> patches implementing error recovery. Not all of these patches are in >>> mainline yet. These may be used as "examples": >>> mainline yet. These may be used as "examples": >>> >>> >>> drivers/scsi/ipr.c >>> drivers/scsi/ipr >>> drivers/scsi/sym53cxx_2 >>> drivers/scsi/sym53c8xx_2 >>> drivers/scsi/qla2xxx >>> drivers/scsi/lpfc >>> drivers/next/bnx2.c >>> drivers/next/e100.c >>> drivers/next/e100.c >>> drivers/net/e1000 >>> drivers/net/e1000 >>> drivers/net/e1000e >>> drivers/net/ixgb >>> drivers/net/ixgb >>> drivers/net/ixgbe >>> drivers/net/cxgb3 >>> drivers/net/s2io.c >>> drivers/net/s2io.c >>> drivers/net/qlge The End The End ------- -------