Understanding SRP Addressing and Discovery

Identifying Available SCSI Targets using IB Device Management Services

SRP initiator systems can discover the SCSI resources on SRP target systems using IB Device Management (DM) services. SRP initiators use the query subset of the full IB DM protocol.

Understanding the SRP session model and the IB target addressing model in more detail will help you configure other SRP initiators to access COMSTAR SRP SCSI Targets.

SRP SCSI Session - the I-T Nexus

Each SRP SCSI Session is identified by a 128-bit initiator identifier and a 128-bit target identifier. This pair of identifiers uniquely identifies an initiator-target nexus. Only a single SRP SCSI session can be active at one time between a particular initiator identifier and target identifier pair.

The initiator identifier consists of a 64-bit initiator GUID and a 64-bit initiator extension. The initiator system administrator can set the initiator extension to be different for different connections. Thus, the initiator extension enables the initiator system to maintain multiple independent connections to the same target at the same time. This feature can be useful when enabling multipath I/O, for example. Both halves of the initiator identifier are reflected in the initiator alias that is reported by stmfadm list-target -v. Specifically, the initiator alias is set to initiator_extension.initiator_GUID.

The SRP target identifier consists of the SCSI target ID followed by the I/O controller GUID. For the COMSTAR SRP target, which uses the "one target per HCA" model, both of these values will be set to the target HCA GUID.

InfiniBand Subnet Addressing - Channel Endpoints

At the IB level, when an initiator wishes to create a channel to communicate with a target service, it specifies both the source and destination endpoints of the IB channel. The endpoints of the channel are two physical ports: one port on a Host Channel Adapter of the initiator and one port on a Target Channel Adapter of the target. The IB subnet manager assigns to each port a local identifier (LID) and and a global identifier (GID). The GID consists of the port's 64-bit GUID, which is assigned at the manufacturer, together with a 64-bit subnet prefix, assigned by the subnet manager, that identifies the IB subnet on which that port can be found.

InfiniBand Target Service Addressing

In the InfiniBand architecture, target systems contain Target Channel Adapters (TCAs) that give access to I/O units that give access to I/O controllers that give access to specific target-side services. An IB target-side application can register to receive incoming connection requests on a particular IB service ID, and can specify the particular IB ports on which to listen. For a general SRP target, all of these different entities may be independent from one another and may have their own IDs. For the COMSTAR SRP target, they are close to identical.

Summary of Target Service Parameters

When queried by IB DM, each SRP service is reported as two related fields in a single unit called a service entry. The first half of the entry identifies the InfiniBand service ID the target is listening on. The second half of the entry identifies the SCSI target ID, reported in the standard form SRP.T10:Target ID. For the COMSTAR SRP target, both values are always identical because they both match the GUID of the target HCA.

Linux SRP Discovery Example

For example, when using the Linux Open Fabrics Enterprise Distribution (OFED) SRP initiator, the srp_daemon -n -o command uses IB DM services to display all the available SRP I/O controllers and associated SCSI targets that are available through a particular IB port.

Example on Red Hat Linux Initiator

# srp_daemon -a -o -i mthca0 -p 1
IO Unit Info:
    port LID:        0027
    port GID:        fe800000000000000003ba0001002e49
    change ID:       0004
    max controllers: 0x10

    controller[  1]
        GUID:      0003ba0001002e48
        vendor ID: 0015b3
        device ID: 006278
        IO class : 0100
        ID:        Solaris SRP Target 0.9a
        service entries: 1
            service[  0]: 0003ba0001002e48 / SRP.T10:0003ba0001002e48

The srp_daemon output shows the destination GID (fe800000000000000003ba0001002e49) of the physical port that is providing access to the SRP service. The GID consists of the default subnet prefix fe80000000000000 followed by the GUID of the port 0003ba0001002e49. The GUID of an HCA's physical port is equal to the HCA GUID plus the port number. This output shows a target that is available through port 1 on that HCA. The output also shows the SRP-capable I/O controller, with its I/O controller GUID set to the HCA GUID. That I/O controller exports a single service entry that contains both a service ID and a SCSI target identifier, both of which are also set to the HCA GUID. The I/O unit GUID is not displayed in srp_daemon output.