iscsi Provider

{anchor:top}

h1. {anchor:CHP-iscsi} {{iscsi}} Provider

The {{iscsi}} provider provides probes for tracing iSCSI target activity.

h3.

This is a kernel provider built into the COMSTAR iSCSI target port provider. The COMSTAR iSCSI target and the user-land iSCSI target (/usr/sbin/iscsitgtd) are mutually exclusive. Only one of the targets can be enabled at a time. The COMSTAR iSCSI target DTrace provider provides all the probes that are provided by the user-land iSCSI provider, so that any DTrace script written for the userland provider built into the iSCSI target daemon (iscsitgtd) will work with the COMSTAR iSCSI target port provider as well without any modification. Since this provider instruments the iSCSI target activity, DTrace commands and scripts must be run on the iSCSI target server to observe these probes.

h2. {anchor:CHP-iscsi-Probes} Probes

|| SCSI Event || Probes ||
| SCSI command/response | {{iscsi:::scsi-command{}}}{{{}iscsi:::scsi-response}} |
| Data out/in/request (rtt) | {{iscsi:::data-send{}}}{{{}iscsi:::data-receive{}}}{{{}iscsi:::data-request}} |
| Login and logout command/response | {{iscsi:::login-command{}}}{{{}iscsi:::login-response{}}}{{{}iscsi:::logout-command{}}}{{{}iscsi:::logout-response}} |
| NOP out/in (pings) | {{iscsi:::nop-receive{}}}{{{}iscsi:::nop-send}} |
| Text and task command/response | {{iscsi:::task-command{}}}{{{}iscsi:::task-response{}}}{{{}iscsi:::text-command{}}}{{{}iscsi:::text-response}} |
| Asynchronous message from target | {{iscsi:::async-send}} |
| Buffer dispatch and completion (not available with the USDT provider) | {{iscsi:::xfer-start{}}}{{{}iscsi:::xfer-done}} |

[Top|#top]

h2. {anchor:CHP-iscsi-Arguments} Arguments

|| Probes || Variable || Type || Description ||
| \* | {{args\[0\]}} | {{conninfo_t *}} | connection info |
| \* | {{args\[1\]}} | {{iscsiinfo_t *}} | common iSCSI properties |
| {{scsi-command}} | {{args\[2\]}} | {{scsicmd_t *}} | SCSI command block (cdb) |
| {{xfer-start{}}}{{{}xfer-done}} | {{args\[2\]}} | {{xferinfo_t *}} | Buffer info |


[Top|#top]

h2. {anchor:CHP-iscsi-Types} Types

All COMSTAR iSCSI target probes have the first and second argument in common:

args\[0\] conninfo_t * connection information

{{conninfo_t}}
{noformat}
typedef struct conninfo {
string ci_local; /* local host IP address */
string ci_remote; /* remote host IP address */
string ci_protocol; /* protocol ("ipv4", "ipv6") */
} conninfo_t;
The conninfo_t structure is used by NFSv4 provider, Fibre Channel provider and is intended for use by all
application protocol providers as the first argument to indicate some basic information about the connection.
{noformat}

args\[1\] iscsiinfo_t * common iSCSI properties

{{iscsiinfo_t}}
{noformat}
typedef struct iscsiinfo {
string ii_target; /* target iqn */
string ii_initiator; /* initiator iqn */
string ii_isid; /* initiator session identifier */
string ii_tsih; /* target session identifying handle */
string ii_transport; /* transport type ("iser-ib", "sockets") */

uint64_t ii_lun; /* target logical unit number */

uint32_t ii_itt; /* initiator task tag */
uint32_t ii_ttt; /* target transfer tag */

uint32_t ii_cmdsn; /* command sequence number */
uint32_t ii_statsn; /* status sequence number */
uint32_t ii_datasn; /* data sequence number */

uint32_t ii_datalen; /* length of data payload */
uint32_t ii_flags; /* probe-specific flags */
} iscsiinfo_t;
The iscsiinfo_t structure is used to provide identifying information about the target and the initiator and
also some PDU level information such as lun, data length and sequence numbers.
{noformat}
The third argument is only used for the SCSI command probe or the data transfer probe

args\[2\] scsicmd_t * SCSI command block (cdb)

{{scsicmd_t}}
{noformat}
typedef struct scsicmd {
uint64_t ic_len; /* CDB length */
uint8_t *ic_cdb; /* CDB data */
} scsicmd_t;
The scsicmd_t structure is used by the SCSI command probe and it contains information about the SCSI command

{{xferinfo_t}}
{noformat}
typedef struct xferinfo {
uintptr_t xfer_laddr; /* local buffer address */
uint32_t xfer_loffset; /* offset within the local buffer */
uint32_t xfer_lkey; /* access control to local memory */
uintptr_t xfer_raddr; /* remote virtual address */
uint32_t xfer_roffset; /* offset from the remote address */
uint32_t xfer_rkey; /* access control to remote virtual address */
uint32_t xfer_len; /* transfer length */
uint32_t xfer_type; /* Read or Write */
} xferinfo_t;
The xferinfo_t structure is used by the xfer-start and the xfer-done probes and contain information about the
data transfer. When the transport type is iSER, the remote buffer information is given by the xfer_raddr,
xfer_rkey and xfer_roffset fields. It is set to 0 when the transport type is sockets.
{noformat}
[Top|#top]

h2. {anchor:CHP-iscsi-Examples} Examples


h6. One-liners

Frequency of iSCSI command types:
{noformat}
# dtrace -n 'iscsi*::: { @[probename] = count(); }'
{noformat}

Frequency of iSCSI client IP addresses:
{noformat}
# dtrace -n 'iscsi*::: { @[args[0]->ci_remote] = count(); }'
{noformat}

Payload bytes by iSCSI command type:
{noformat}
# dtrace -n 'iscsi*::: { @[probename] = sum(args[1]->ii_datalen); }'
{noformat}

Payload byte distribution by iSCSI command type:
{noformat}
# dtrace -n 'iscsi*::: { @[probename] = quantize(args[1]->ii_datalen); }'
{noformat}

h6. iscsiwho.d

This is a simple script to produce a report of the remote IP addresses and a count of iSCSI events. This is intended to provide a quick summary of iSCSI activity when run on the iSCSI target server:
{noformat}
#!/usr/sbin/dtrace -s

#pragma D option quiet

dtrace:::BEGIN
{
printf("Tracing... Hit Ctrl-C to end.\n");
}

iscsi*:::
{
@events[args[0]->ci_remote, probename] = count();
}

dtrace:::END
{
printf(" %-26s %14s %8s\n", "REMOTE IP", "iSCSI EVENT", "COUNT");
printa(" %-26s %14s %@8d\n", @events);
}
{noformat}
This output shows the host and the number of iSCSI operations:
{noformat}
# ./iscsiwho.d
Tracing... Hit Ctrl-C to end.
^C
REMOTE IP iSCSI EVENT COUNT
192.168.1.109 nop-receive 1
192.168.1.109 nop-send 1
192.168.1.109 scsi-response 126
192.168.1.109 scsi-command 208
192.168.1.109 data-request 1207
192.168.1.109 data-receive 1207
{noformat}
The fields are:
|| field || description ||
| {{REMOTE IP}} | IP address of the client |
| {{iSCSI EVENT}} | iSCSI event type |
| {{COUNT}} | Number of events traced |
The example output shows normal traffic. For this simple script, these event names are not translated beyond their iSCSI provider probe names, and require some thought to comprehend (they are from the perspective of the iSCSI target server).

h6. iscsixfer.d

Although the transport layer is transparent to the user, the COMSTAR iSCSI target also supports iSCSI over Remote DMA (RDMA), also known as iSER. An iSER initiator should be able to read and write data from an iSER target at high data rates with relatively low CPU utilization compared to iSCSI using TCP/IP. In order to see the transport layer in use, display the ii_transport field from the iscsiinfo_t structure.

Since the data transfer phases are mapped to Remote DMA (RDMA) operations in iSER, the data-send, data-receive and data-request probes cannot be used with iSER. Instead here is a simple script to print an aggregation of all the data transferred between two points using the xfer-start probe. This can be used for iSCSI using TCP/IP and iSCSI over Remote DMA.

The data-receive, data-request and data-send probes can be used when a user wants to track the SCSI Data-IN and Data-OUT PDUs specifically (e.g. if the PDUs are received out of order, one might want to trace the ii_ttt, ii_datasn, ii_statsn etc.). To just get a trace of IO activity, the xfer-start/xfer-done probes should suffice.

{noformat}
#!/usr/sbin/dtrace -s

#pragma D option quiet

iscsi:::xfer-start
{
@[args[0]->ci_remote, args[2]->xfer_type] = sum(args[2]->xfer_len);
}

END
{
printf("%26s %10s %8s\n", "REMOTE IP", "READ/WRITE", "BYTES");
printa("%26s %10s %15@d\n", @);
}
{noformat}
This output shows the transfer of bytes:
{noformat}
# ./iscsixfer.d
Tracing... Hit Ctrl-C to end.
^C

REMOTE IP READ/WRITE BYTES
192.168.1.109 write 464
192.168.1.109 read 1024
{noformat}
The fields are:
|| field || description ||
| {{REMOTE IP}} | IP address of the client |
| {{READ/WRITE}} | read or write |
| {{BYTES}} | Number of bytes transferred |

Now if a user is interested in just seeing the data move (read or write) as it happens, one could use this script
{noformat}
#!/usr/sbin/dtrace -s

#pragma D option quiet

BEGIN
{
printf(" %-26s %8s %10s\n", "REMOTE IP", "BYTES", "READ/WRITE");

}

iscsi:::xfer-start
{
printf("%26s %%8d %108s\n", args[0]->ci_remote, args[2]->xfer_len, args[2]->xfer_type);
}
{noformat}

An interpretation for some of these events are:

|| iSCSI event || Interpretation ||
| {{scsi-command}} | A SCSI command was issued, such as a read or a write. Use other scripts for a breakdown on the SCSI command type. |
| {{data-send}} | Data was sent from the iSCSI target server to the client; the client is performing a read. |
| {{data-receive}} | Data was received on the iSCSI target server from the client. The client is performing a write. |