aboutsummaryrefslogtreecommitdiff
path: root/drivers/infiniband/ulp/srp/ib_srp.h
AgeCommit message (Collapse)Author
2011-03-15IB/srp: try to use larger FMR sizes to cover our mappingsDavid Dillow
Now that we can get larger SG lists, we can take advantage of HCAs that allow us to use larger FMR sizes. In many cases, we can use up to 512 entries, so start there and work our way down. Signed-off-by: David Dillow <dillowda@ornl.gov>
2011-03-15IB/srp: add support for indirect tables that don't fit in SRP_CMDDavid Dillow
This allows us to guarantee the ability to submit up to 8 MB requests based on the current value of SCSI_MAX_SG_CHAIN_SEGMENTS. While FMR will usually condense the requests into 8 SG entries, it is imperative that the target support external tables in case the FMR mapping fails or is not supported. We add a safety valve to allow targets without the needed support to reap the benefits of the large tables, but fail in a manner that lets the user know that the data didn't make it to the device. The user must add "allow_ext_sg=1" to the target parameters to indicate that the target has the needed support. If indirect_sg_entries is not specified in the modules options, then the sg_tablesize for the target will default to cmd_sg_entries unless overridden by the target options. Signed-off-by: David Dillow <dillowda@ornl.gov>
2011-03-15IB/srp: rework mapping engine to use multiple FMR entriesDavid Dillow
Instead of forcing all of the S/G entries to fit in one FMR, and falling back to indirect descriptors if that fails, allow the use of as many FMRs as needed to map the request. This lays the groundwork for allowing indirect descriptor tables that are larger than can fit in the command IU, but should marginally improve performance now by reducing the number of indirect descriptors needed. We increase the minimum page size for the FMR pool to 4K, as larger pages help increase the coverage of each FMR, and it is rare that the kernel would send down a request with scattered 512 byte fragments. This patch also move some of the target initialization code afte the parsing of options, to keep it together with the new code that needs to allocate memory based on the options given. Signed-off-by: David Dillow <dillowda@ornl.gov>
2011-03-15IB/srp: allow sg_tablesize to be set for each targetDavid Dillow
Different configurations of target software allow differing max sizes of the command IU. Allowing this to be changed per-target allows all targets on an initiator to get an optimal setting. We deprecate srp_sg_tablesize and replace it with cmd_sg_entries in preparation for allowing more indirect descriptors than can fit in the IU. Signed-off-by: David Dillow <dillowda@ornl.gov>
2011-01-10IB/srp: consolidate hot-path variables into cache linesDavid Dillow
Put the variables accessed together in the hot-path into common cachelines, and separate them by RW vs RO to avoid false dirtying. We keep a local copy of the lkey and rkey in the target to avoid traversing pointers (and associated cache lines) to find them. Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: David Dillow <dillowda@ornl.gov>
2011-01-10IB/srp: stop sharing the host lock with SCSIBart Van Assche
We don't need protection against the SCSI stack, so use our own lock to allow parallel progress on separate CPUs. Signed-off-by: Bart Van Assche <bvanassche@acm.org> [ broken out and small cleanups by David Dillow ] Signed-off-by: David Dillow <dillowda@ornl.gov>
2011-01-10IB/srp: reduce local coverage for command submission and EHBart Van Assche
We only need locks to protect our lists and number of credits available. By pre-consuming the credit for the request, we can reduce our lock coverage to just those areas. If we don't actually send the request, we'll need to put the credit back into the pool. Signed-off-by: Bart Van Assche <bvanassche@acm.org> [ broken out and small cleanups by David Dillow ] Signed-off-by: David Dillow <dillowda@ornl.gov>
2011-01-10IB/srp: don't move active requests to their own listBart Van Assche
We use req->scmnd != NULL to indicate an active request, so there's no need to keep a separate list for them. We can afford the array iteration during error handling, and dropping it gives us one less item that needs lock protection. Signed-off-by: Bart Van Assche <bvanassche@acm.org> [ broken out and small cleanups by David Dillow ] Signed-off-by: David Dillow <dillowda@ornl.gov>
2011-01-05IB/srp: allow lockless work postingBart Van Assche
Only one CPU at a time will own an RX IU, so using the address of the IU as the work request cookie allows us to avoid taking a lock. We can similarly prepare the TX path for lockless posting by moving the free TX IUs to a list. This also removes the requirement that the queue sizes be a power of 2. Signed-off-by: Bart Van Assche <bvanassche@acm.org> [ broken out, small cleanups, and modified to avoid needing an extra field in the IU by David Dillow] Signed-off-by: David Dillow <dillowda@ornl.gov>
2011-01-05IB/srp: allow task management without a previous requestDavid Dillow
We can only have one task management comment outstanding, so move the completion and status to the target port. This allows us to handle resets of a LUN without a corresponding request having been sent. Meanwhile, we don't need to play games with host_scribble, just use it as the pointer it is. This fixes a crash when we issue a bus reset using sg_reset. Fixes: https://bugzilla.kernel.org/show_bug.cgi?id=13893 Reported-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: David Dillow <dillowda@ornl.gov>
2010-10-22IB/srp: Implement SRP_CRED_REQ and SRP_AER_REQDavid Dillow
This patch adds support for SRP_CRED_REQ to avoid a lockup by targets that use that mechanism to return credits to the initiator. This prevents a lockup observed in the field where we would never add the credits from the SRP_CRED_REQ to our current count, and would therefore never send another command to the target. Minimal support for SRP_AER_REQ is also added, as these messages can also be used to convey additional credits to the initiator. Based upon extensive debugging and code by Bart Van Assche and a bug report by Chris Worley. Signed-off-by: David Dillow <dillowda@ornl.gov> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2010-10-22IB/srp: Preparation for transmit ring response allocationBart Van Assche
The transmit ring in ib_srp (srp_target.tx_ring) is currently only used for allocating requests sent by the initiator to the target. This patch prepares using that ring for allocation of both requests and responses. Also, this patch differentiates the uses of SRP_SQ_SIZE, increases the size of the IB send completion queue by one element and reserves one transmit ring slot for SRP_TSK_MGMT requests. Signed-off-by: Bart Van Assche <bvanassche@acm.org> Signed-off-by: David Dillow <dillowda@ornl.gov> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2010-03-01IB/srp: Split send and recieve CQs to reduce number of interruptsBart Van Assche
We can reduce the number of IB interrupts from two interrupts per srp_queuecommand() call to one by using separate CQs for send and receive completions and processing send completions by polling every time a TX IU is allocated. Receive completion events still trigger an interrupt. Signed-off-by: Bart Van Assche <bart.vanassche@gmail.com> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2008-07-14RDMA: Remove subversion $Id tagsRoland Dreier
They don't get updated by git and so they're worse than useless. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2008-04-19SCSI: convert struct class_device to struct deviceTony Jones
It's big, but there doesn't seem to be a way to split it up smaller... Signed-off-by: Tony Jones <tonyj@suse.de> Signed-off-by: Kay Sievers <kay.sievers@vrfy.org> Cc: Roland Dreier <rolandd@cisco.com> Cc: Sean Hefty <sean.hefty@intel.com> Cc: Hal Rosenstock <hal.rosenstock@gmail.com> Cc: James Bottomley <James.Bottomley@HansenPartnership.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2008-04-19IB: rename "dev" to "srp_dev" in srp_host structureGreg Kroah-Hartman
This sets us up to be able to convert the srp_host to use a struct device instead of a class_device. Based on a original patch from Tony Jones, but split up into this piece by Greg. Signed-off-by: Tony Jones <tonyj@suse.de> Signed-off-by: Kay Sievers <kay.sievers@vrfy.org> Reviewed-by: Roland Dreier <rolandd@cisco.com> Cc: Sean Hefty <sean.hefty@intel.com> Cc: Hal Rosenstock <hal.rosenstock@gmail.com> Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
2008-02-04IB/srp: Retry stale connectionsDavid Dillow
When a host just goes away (crash, power loss, etc.) without tearing down its IB connections, it can get stale connection errors when it tries to reconnect to targets upon rebooting. Retrying the connection a few times will prevent sysadmins from playing the "which disk(s) went missing?" game. This would have made things slightly quicker when tracking down some of the recent bugs, but it also helps quite a bit when you've got a large number of targets hanging off a wedged server. Signed-off-by: David Dillow <dillowda@ornl.gov> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2008-01-25IB/srp: Respect target credit limitDavid Dillow
The current SRP initiator will send requests even if it has no credits available. The results of sending extra requests are vendor specific, but on some devices, overrunning credits will cost 85% of peak performance -- e.g. 100 MB/s vs 720 MB/s. Other devices may just drop the requests. This patch will tell the SCSI midlayer to queue requests if there are fewer than two credits remaining, and will not issue a task management request if there are no credits remaining. The mid-layer will retry the queued command once an outstanding command completes. The patch also removes the unlikely() in __srp_get_tx_iu(), as it is not at all unlikely to hit this limit under heavy load. Signed-off-by: David Dillow <dillowda@ornl.gov> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-06-07[SCSI] ib_srp: convert to use the data buffer accessorsFUJITA Tomonori
- remove the unnecessary map_single path. - convert to use the new accessors for the sg lists and the parameters. Jens Axboe <jens.axboe@oracle.com> did the for_each_sg cleanup. Signed-off-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp> Acked-by: Roland Dreier <rdreier@cisco.com> Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com>
2007-05-06IB/srp: Add orig_dgid sysfs attribute to scsi_hostIshai Rabinovitz
Add an orig_dgid attribute in sysfs for SRP scsi_hosts, so that userspace can tell what the original dgid value written to the add_target file was, even if the connection is redirected to a different port while connecting. Signed-off-by: Ishai Rabinovitz <ishai@mellanox.co.il> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2007-02-04IB/srp: Don't wait for response when QP is in error state.Ishai Rabinovitz
When there is a call to send_tsk_mgmt SRP posts a send and waits for 5 seconds to get a response. When the QP is in the error state it is obvious that there will be no response so it is quite useless to wait. In fact, the timeout causes SRP to wait a long time to reconnect when a QP error occurs. (Each abort and each reset_device calls send_tsk_mgmt, which waits for the timeout). The following patch solves this problem by identifying the failure and returning an immediate error code. Signed-off-by: Ishai Rabinovitz <ishai@mellanox.co.il> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-12-15IB/srp: Fix FMR mapping for 32-bit kernels and addresses above 4GRoland Dreier
struct srp_device.fmr_page_mask was unsigned long, which means that the top part of addresses above 4G was being chopped off on 32-bit architectures. Of course nothing good happens when data from SRP targets is DMAed to the wrong place. Fix this by changing fmr_page_mask to u64, to match the addresses actually used by IB devices. Thanks to Brian Cain <Brian.Cain@ge.com> and David McMillen <davem@systemfabricworks.com> for help diagnosing the bug and testing the fix. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-12-12IB/srp: Use new verbs IB DMA mapping functionsRalph Campbell
Convert SRP to use the new verbs DMA mapping functions for kernel verbs consumers. Signed-off-by: Ralph Campbell <ralph.campbell@qlogic.com> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-10-10IB/srp: Enable multiple connections to the same targetIshai Rabinovitz
Enable multiple concurrent connections to the same SRP target: 1) Use port GUID instead of node GUID in the initiator port identifier. This allows connections to be made from multiple HCA ports at the same time. 2) Let the user specify the identifier extention when adding the device. This allows userspace to make multiple connections even from the same port, if it wants too. Without this, only one connection can be made from any given HCA, even if it has multiple ports, because we don't use multi-channel mode, so targets will only allow one connection from a given initiator port ID. Signed-off-by: Ishai Rabinovitz <ishai@mellanox.co.il> Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-06-17IB/srp: Support SRP rev. 10 targetsRamachandra K
There has been a change in the format of port identifiers between revision 10 of the SRP specification and the current revision 16A. Revision 10 specifies port identifier format as lower 8 bytes : GUID upper 8 bytes : Extension Whereas revision 16A specifies it as lower 8 bytes : Extension upper 8 bytes : GUID There are older targets (e.g. SilverStorm Virtual Fibre Channel Bridge) which conform to revision 10 of the SRP specification. The I/O class of revision 10 is 0xFF00 and the I/O class of revision 16A is 0x0100. For supporting older targets, this patch: 1) Adds a new optional target creation parameter "io_class". Default value of io_class is 0x0100 (i.e. revision 16A) 2) Uses the correct port identifier format for targets with IO class of 0xFF00 (i.e. conforming to revision 10) Signed-off-by: Ramachandra K <rkuchimanchi@silverstorm.com> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-06-17IB/srp: Get rid of "Target has req_lim 0" messagesRoland Dreier
It's perfectly valid for a connection to an SRP target to have a request limit of 0, so get rid of the message about it, which can spam kernel logs even with printk_ratelimit(). Keep a count of such events in a "zero_req_lim" SCSI host attribute instead, so someone who cares can look at the statistics. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-06-17IB/srp: Allow sg_tablesize to be adjustedVu Pham
Make the sg_tablesize used by SRP adjustable at module load time via a module parameter. Calculate the corresponding IU length required to support this. Signed-off-by: Vu Pham <vu@mellanox.com> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-06-17IB/srp: Change target_mutex to a spinlockMatthew Wilcox
The SRP driver never sleeps while holding target_mutex, and it's just used to protect some simple list operations, so hold times will be short. So just convert it to a spinlock, which is smaller and faster. Signed-off-by: Matthew Wilcox <matthew@wil.cx> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-06-17IB/srp: Use FMRs to map gather/scatter listsRoland Dreier
Create an SRP FMR pool on HCAs that support FMRs, and use FMRs to map gather/scatter lists that have more than one entry into a single memory region that appears virtually contiguous to the SRP target (which is the RDMA initiator). This patch bails out on FMR mapping for SCSI commands where the gather/scatter list cannot be mapped into a single FMR because there are sub-page-sized entries in middle of the list. An unaligned start or end of the list is OK. Based on a patch by Vu Pham <vuhuong@mellanox.com>. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-05-09IB/srp: Fix tracking of pending requests during error handlingRoland Dreier
If a SCSI abort completes, or the command completes successfully, then the driver must remove the command from its queue of pending commands. Similarly, if a device reset succeeds, then all commands queued for the given device must be removed from the queue. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-03-24IB/srp: Use a fake scatterlist for non-SG SCSI commandsRoland Dreier
Since the SCSI midlayer is moving towards entirely getting rid of commands with use_sg == 0, we should treat this case as an exception. Therefore, change the IB SRP initiator to create a fake scatterlist for these commands with sg_init_one(). This simplifies the flow of DMA mapping and unmapping, since SRP can just use dma_map_sg() and dma_unmap_sg() unconditionally, rather than having to choose between the dma_{map,unmap}_sg() and dma_{map,unmap}_single() variants. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2006-01-30IB/srp: Semaphore to mutex conversionIngo Molnar
Convert srp_host->target_mutex from a semaphore to a mutex. Signed-off-by: Ingo Molnar <mingo@elte.hu> Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-11-11[IB] srp: increase max_lunsRoland Dreier
Increase SRP max_luns to 512 to match the kernel's default, since SRP storage targets can have lots of LUNs and the SRP initiator itself doesn't have any particular limit. Signed-off-by: Roland Dreier <rolandd@cisco.com>
2005-11-02IB: Add SCSI RDMA Protocol (SRP) initiatorRoland Dreier
Add an InfiniBand SCSI RDMA Protocol (SRP) initiator. This driver is used to talk talk to InfiniBand SRP targets (storage devices). Signed-off-by: Roland Dreier <rolandd@cisco.com>