View Source

h1. GLDv3 Porting Guide

This guide describes how to port drivers to from GLDv2 architecture to GLDv3 architecture. For a complete guide to writing Solaris drivers, see the [_Writing Device Drivers_|http://docs.sun.com/app/docs/doc/819-3196] book. For information on writing network drivers for the Solaris OS, see [Chapter 19, "Drivers for Network Devices."|http://docs.sun.com/app/docs/doc/819-3196/gld-1?a=view]

h2. 1 Overview

The explanations in this document assume that you have a GLDv2 driver that has already been written and tested, and you simply want to convert that GLDv2 driver to be a GLDv3 driver. No prior knowledge of GLDv2 or GLDv3 is assumed. The conversion process primarily involves declaring and initializing new data structures that are used when registering with the framework.

h3. 1.1 Advantages of GLDv3 Over GLDv2

The majority of code changes necessary to port GLDv2 drivers to GLDv3 simply involve translating the data
structures used and renaming some functions. GLDv3 provides several major advantages over GLDv2.
* Interrupt blanking
* Hardware checksum
* Link aggregation (802.3ad)
* VLANs (802.1q)

h3. 1.2 Porting Guidelines

This section provides some suggestions on how to port to GLDv3 most efficiently.

h4. 1.2.1 Overview of Essential Porting Steps

The following steps for porting from GLDv2 to GLDv3 must be done in order to get basic send and receive functionality to work with the driver.

{panel:bgColor=#FFFFCE}No {{mac_info_t}}?
No {{mac_t}}? Instead, static {{mac_callbacks_t}} passed to {{mac_register()}} to get opaque {{mac_handle_t}}.
No {{unregister()}}?
Ethernet: VLANs are used, full frame size support _required_.{panel}

# Change the {{attach()}} entry point to use the {{mac_info_t}} structure instead of the {{gld_mac_info_t}} structure. Translate the structure fields.
# Set up entry points in {{mac_t}}. Modify the functions that the entry points are set to so that the arguments are in the correct order.
{panel:bgColor=#FFFFCE}No {{mac_t}}{panel}
# Modify the calls to {{unregister()}} to use the GLDv3 version.
{panel:bgColor=#FFFFCE}ethernet: VLANs are used - full frame size support is required{panel}
# Update header files to include {{mac.h}} and {{mac_ether.h}} instead of {{gld.h}}.
# Update the makefiles.

h4. 1.2.2 Overview of Additional Porting Steps

The following steps allow you to utilize the extended functionality in GLDv3. You should follow these steps as well as possible to allow for optimal driver performance under the GLDv3 framework.
# Enable hardware blanking by setting up an additional entry point and providing a blanking function based on instructions from the NIC vendor.
# Set up the {{checksum}} field of the {{mac_info_t}} data structure to enable hardware checksum.

h4. 1.2.3 Recommended Development Strategy

The most efficient strategy to follow for porting GLDv2 drivers to GLDv3 drivers includes the following steps:
# Starting with the existing GLDv2 driver code, port the essential driver functions to GLDv3.
# Make sure the updated driver compiles successfully using the {{mac.h}} header
file and the {{misc/mac}} library.
# Make sure the updated driver loads.
# Check for link up.
# Test receive and transmit.
{panel:bgColor=#FFFFCE}Add VLAN support (ethernet only){panel}
# Port the extended features of the GLDv3 drivers.
# Test the extended features, such as interrupt blanking and hardware checksum,
in the hardware test environment.
# Perform stress and performance testing of the driver.

h3. 1.3 Code Examples

This guide shows code samples that you can reference. In addition, you should review drivers that have already been ported. GLDv3 drivers are available for viewing on [the OpenSolaris project|http://www.opensolaris.org/os/]. A good driver to review is the {{bge}} driver. Take the following steps to view the {{bge}} driver:
# On the OpenSolaris project site, click the Source Browser button to go to [http://cvs.opensolaris.org/source/|http://cvs.opensolaris.org/source/].
# In the File Path text field, enter {{bge}}.
# Make sure the {{onnv}} Project is selected.
# Click the Search button.

h3. 1.4 List of Driver Functions to Modify

The following list shows the functions that must be modified during a port to GLDv3 for drivers that are structured similar to {{bge}}.

{panel:bgColor=#FFFFCE}Use {{afe}}. It is much easier to understand.{panel}

||Generic Function||{{bge}} Function||{{bge}} Source File||Reference Section||
|{{attach()}}|{{bge_attach()}}|{{bge_main2.c}}|{{attach()}} Changes|
|{{detach()}}|{{bge_detach()}}|{{bge_main2.c}}|{{detach()}} Changes|
|{{reschedule()}}|{{bge_reschedule()}}|{{bge_send.c ??}}|{{mac_tx_update()}}|
|{{receive()}}|{{bge_receive()}}|{{bge_recv2.c}}|{{mac_rx()}}|
|{{chip_factotum()}}|{{bge_chip_factotum()}}|{{bge_chip2.c}}|{{mac_link_update()}}|
|{{m_resources()}}|{{bge_m_resources()}}|{{bge_main2.c}}|Interrupt Blanking|

h2. 2 Driver Conversion Tasks

This section details the changes that must be made to the {{attach()}} and {{detach()}} entry points, as well as specifics of implementing other GLDv3 driver entry points.

h3. 2.1 {{attach()}} Changes

The {{attach()}} entry point attaches a device to the system.

{panel:bgColor=#FFFFCE}# Allocates registration structure using {{mac_alloc()}}
and {{mac_register_t}}
# Initialize hardware. Important note: Set up RX(?) filer.
# {{mac_register_t}} is populated with pointers to callback structure, mac address, etc.
# {{mac_register()}}
# {{mac_free()}}{panel}

h4. 2.1.1 Set Up State Information Data Structures

The global header file might have a data structure associated with the card for
per-card state information. If you find a declaration of a {{gld_mac_info_t}} pointer, change it to a {{mac_t}} pointer.

{panel:bgColor=#FFFFCE}No {{mac_t}}{panel}

h4. 2.1.2 Set Up {{mac_info_t *}}

In GLDv2, the code uses a {{gld_mac_info_t}} pointer. In GLDv3, this needs to be removed and replaced with a {{mac_info_t}} pointer.

GLDv3 must allocation a {{mac_t}} structure, and then set the {{mac_info_t}} {{m_info}} pointer to {{mac_t}} {{*macp}}. The {{mac_info_t}} pointer can be used in the next step when setting up device specific functions.

||GLDv2||GLDv3||
|{code} bge_attach(dev_info_t *devinfo,
ddi_attach_cmd_t cmd)
gld_mac_info_t *macinfo; /* GLD */
bge_t *bgep; /* Our private data */
macinfo = gld_mac_alloc(devinfo);
ddi_set_driver_private(devinfo, macinfo);
bgep = kmem_zalloc(sizeof (*bgep), KM_SLEEP);
bgep->bge_guard = BGE_GUARD;
bgep->devinfo = devinfo;
bgep->macinfo = macinfo;
macinfo->gldm_private = (caddr_t)bgep;
...
macinfo->gldm_ident = bge_ident;
macinfo->gldm_type = DL_ETHER;
macinfo->gldm_minpkt = 0;
macinfo->gldm_maxpkt = ETHERMTU;
macinfo->gldm_addrlen = ETHERADDRL;
macinfo->gldm_saplen= -2;
macinfo->gldm_broadcast_addr=bge_broadcast_addr;
macinfo->gldm_vendor_addr=cidp->vendor_addr.addr;
macinfo->gldm_ppa= instance;
macinfo->gldm_devinfo= devinfo;
macinfo->gldm_cookie=(ddi_iblock_cookie_t)
(uint64_t bgep->intr_pri;{code}| |

h4. 2.1.3 Set Up {{mac_info_t}} Fields

Pointers to device specific functions that will be used by the generic layer must be initized. This is done by setting fields in the {{mac_info_t}} data structure, which is contained in the {{mac_t}} structure. This structure is later passed to {{mac_register()}}.

The fields of the {{mac_info_t}} structure must be set. See also Section 2.5, "Converting MAC Structures."

h4. 2.1.4 Set Up Entry Points in {{mac_t}}

The fields of the {{mac_t}} structure need to be set to point to the corresponding entry points in the driver. See also Section 2.6, "Converting MAC Structure Entry Points."

h4. 2.1.5 Register With MAC Layer

At this point, the {{mac_t}} data structure is set up and can be registered. GLDv2 uses {{gld_register()}}. In GLDv3, this changes to {{mac_register()}}.

h3. 2.2 {{detach()}} Changes

The {{detach()}} entry point detaches a device from the system.

{panel:bgColor=#FFFFCE}# {{mac_unregister()}}
# teardown hardware{panel}

h4. 2.2.1 Unregister

In GLDv2, get a pointer to the {{gld_mac_info_t}} structure from {{ddi_get_driver_private()}} and pass the pointer to {{gld_unregister()}}.

In GLDv3, get a pointer to the {{bge_t}} structure from {{ddi_get_driver_private()}} and pass the pointer to {{mac_unregister()}}.

h3. 2.3 {{_init()}} Changes

{panel:bgColor=#FFFFCE}Register type ops{panel}

h3. 2.4 {{_fini()}} Changes

{panel:bgColor=#FFFFCE}Unregister type ops{panel}

h3. 2.5 Converting MAC Structures

The GLDv3 {{mac_info_t}} structure is defined in {{sys/mac.h}}.

{code}/*
* Immutable information. (This may not be modified after registration).
*/
typedef struct mac_info_s {
uint_t mi_media;
uint_t mi_nativemedia;
uint_t mi_addr_length;
uint8_t *mi_unicst_addr;
uint8_t *mi_brdcst_addr;
} mac_info_t;{code}

||GLDv2 gld_mac_info_t||GLDv3 mac_info_t||Usage||Value||
|{{gldm_type}}|{{mi_media}}|Media type of device|Media type. See {{/usr/src/uts/common/sysdlpi.h}}|
|{{gldm_minpkt}}|{{mi_sdu_min}}|Minimum payload size|Typically 0|
|{{gldm_maxpkt}}|{{mi_sdu_max}}|Maximum payload size for medium|For ethernet, this value is 1500|
|{{gldm_capabilities}}|{{mi_cksum}}|Checksum capabilities of the device|Use a combination of the flags shown in "Checksum Flags" below.|
|_new in GLDv3_|{{mi_poll}}|Polling capability|DL_CAPAB_POLL|
|{{gldm_addrlen}}|{{mi_addr_length}}|Length of address used by media|ETHERADDRL|
|{{gldm_broadcast_addr}}|{{mi_brdcstaddr}}|Broadcast address of the media. Use {{bcopy()}} to set the address.^1^|Broadcast address of length {{mi_addr_length}}|
|gldm_vendor_addr}}|{{mi_unicst_addr}}|bcopy to the unicast address of the media. Use {{bcopy()}} to set the address.^2^|Unicast address of length {{mi_addr_length}}|
|_new in GLDv3_|{{mi_stat [xx_statistic_name]}}]|Set unsupported statistics in the {{mip->mi_stat}} array to B_FALSE where necessary.|B_FALSE|

^1^ {{bcopy(ether_brdcst, mip->mi_brdcst_addr, ETHERADDRL);}}

^2^ {{bcopy(bgep>curr_addr.addr, mip->mi_unicst_addr, ETHERADDRL);}}

||Checksum Flags|| ||
||GLDv2|| ||
|GLD_CAP_CKSUM_IPHDR|IP checksum offload|
|GLD_CAP_CKSUM_PARTIAL|TCP/UDP partial|
|GLD_CAP_CKSUM_FULL_V4|TCP/UDP full|
|GLD_CAP_CKSUM_ANY|Any or all of the above|
|GLD_CAP_ZEROCOPY|Zerocopy|
||GLDv3|| ||
|HCKSUM_ENABLE|Enable hardware checksum capability|
|HCKSUM_INET_PARTIAL|Partial 1's complement checksum capability|
|HCKSUM_INET_FULL_V4|Full 1's complement checksum capability for IPv4 packets|
|HCKSUM_IPHDRCKSUM|IPv4 header checksum offload capability|

h3. 2.6 Converting MAC Structure Entry Points

h3. 2.7 Header Changes

In GLDv2, include {{gld.h}}. In GLDv3, include {{mac.h}} and {{mac_ether.h}}.

h3. 2.8 Makefile Changes

In GLDv2, link with {{misc/gld}}. In GLDv3, link with {{misc/mac}}.

h3. 2.9 Additional Required GLDv3 Functions

This section describes other GLDv3 functions and where these functions need to be called. These functions do not directly correspond to GLDv2 functions.

h4. 2.9.1 {{mac_resource_add()}}

{panel:bgColor=#FFFFCE}Only needed for interrupt blanking.{panel}

{code}extern mac_resource_handle_t
mac_resource_add(mac_handle_t, mac_resource_t *);{code}

The {{mac_resource_add()}} function should be called from the {{m_resources()}} entry ;point to register individual receive resources (commonly ring buffers of DMA _descriptors_) with the MAC module.

The returned {{mac_resource_handle_t}} value should then be suplied in calls to {{mac_rx()}}.

The second argument to {{mac_resource_add()}} specifies the resource to be added. Resources are specified by the {{mac_resource_t}} structure. Currently, only resources of type {{MAC_RX_FIFO}} are supported. {{MAC_RX_FIFO}} resources are described by the {{mac_rx_fifo_t}} structure.

{code}typedef struct mac_rx_fifo_s {
mac_resource_type_t mrf_type; /* MAC_RX_FIFO */
mac_blank_t mrf_blank;
void *mrf_arg;
time_t mrf_normal_blank_time;
uint_t mrf_normal_pkt_count;
} mac_rx_fifo_t;{code}

The {{mrf_blank}} field of the {{mac_rx_fifo_t}} structure contains a {{mac_blank_t}} function. This function is meant to be used by the upper layers to control the interrupt rate of the device. The {{mrf_arg}} field is the device context that is meant to be used as the first argument to {{poll_blank()}}.

The {{mrf_normal_blank_time}} field specifies the default interrupt interval, and the {{mrf_normal_pkt_count}} field specifies the packet count threshold. These parameters can be used as the second and third arguments to {{poll_blank}} when you need to revert to the default interrupt rate. You can also increase or decrease the interrupt rate by passing a multiple of these values as the last two arguments of {{poll_blank()}}.

h4. 2.9.2 {{mac_tx_update()}}

{code}extern void mac_tx_update(mac_handle_t){code}

The {{mac_tx_update()}} function is called by a driver to indicate that packet transmission resources are now available. The MAC module then sends a notification.

{code}/*
* Link state data (protected by genlock)
* Defined in sys/mac.h
*/
typedef enum {
LINK_STATE_UNKNOWN = -1,
LINK_STATE_DOWN,
LINK_STATE_UP
} link_state_t;{code}

The {{attach()}} function should call {{mac_tx_update()}} to indicate that packet transmission resources are now available.

{panel:bgColor=#FFFFCE}The {{attach()}} function should register a softint for reschedule by means of {{ddi_add_softinptr()}}. -- Not necessarily?{panel}

h4. 2.9.3 {{mac_rx()}}

{code}extern void mac_rx(mac_handle_t, mac_resource_handle_t, mblk_t *);{code}

Use the {{mac_rx()}} function to submit a chain of packets for reception. The chain of packets is contained in {{mblk_t}} structures. Fragments of the same packet should be linked together using the {{b_cont}} field. Separate packets should be linked using the {{b_next}} field of the leading fragment.

{code}/*
* Message block descriptor
* Defined in sys/stream.h
*/
typedef struct msgb {
struct msgb *b_next;
struct msgb *b_prev;
struct msgb *b_cont;
unsigned char *b_rptr;
unsigned char *b_wptr;
struct datab *b_datap;
unsigned char b_band;
unsigned char b_tag;
unsigned short b_flag;
queue_t *b_queue; /* for sync queues */
} mblk_t;{code}

If the packet chain was received by a registered resource, then the appropriate {{mac_resource_handle_t}} value should be supplied as the second argument to the function. The protocol stack uses this value as a hint when trying to load-spread across multiple CPUs. It is assumed that packets that belong to the same flow will always be received by the same resource.

If the resource is unknown or is unregistered, then NULL should be passed as the second argument.

h4. 2.9.4 {{mac_link_update()}}

{code}extern void mac_link_update(mac_handle_t, link_state_t);{code}

The {{mac_link_update()}} function should be called whenever the state of the media link changes. The second argument should be a value in the {{link_state_t}} enumeration shown above. The MAC module saves this value and sends a notification.

Calls to {{gld_linkstate()}} need to be replaced with calls to {{mac_link_update()}} to propagate {{MAC_NOTE_LINK}} notifications.

{panel:bgColor=#FFFFCE}Calls to log link state change (for example, via {{cmn_err()}}) should be removed, ?? does it for driver.{panel}

h3. 2.10 Enabling Additional GLDv3 Features

This section describes the procedure for enabling GLDv3 features. Note that zero copy and link aggregation (802.3ad) are supported without any additional code modifications.

{panel:bgColor=#FFFFCE}VLANs (802.1q) require full ?? frame size changes.{panel}

h4. 2.10.1 Interrupt Blanking

In the {{m_resources()}} entry point, set up the {{mac_rx_fifo_t}} data structure to have {{mac_rx_fifo_t.mrf_blank}} set to point to _xx_{{_chip_blank()}}, where _xx_{{_chip_blank()}} makes the appropriate calls to {{ddi_put32()}}.

h4. 2.10.2 Hardware Checksum

Set the {{mac_info_t.mi_cksum}} field to enable hardware checksum.

h2. 3 Adding Debug Statements

Do not use debug constructs such as the following:

{code}#if_def xx_DEBUG
...debug code...
#endif{code}

or

{code}xxdebug_printf(...);{code}

The above debug techniques are of limited value after initial development and are not consistent with coding standards. Be sure to use DTrace probes instead.

A great amount of documentation of DTrace exists, and already-instrumented code can be used for an example. use {{cscope}} or another tool to look around in the kernel code for calls to {{DTRACE_PROBE()}}, {{DTRACE_PROBE2()}}, and other similar calls.

The individuals who post here are part of the extended Sun Microsystems community and they might not be employed or in any way formally affiliated with Sun Microsystems. The opinions expressed here are their own, are not necessarily reviewed in advance by anyone but the individual authors, and neither Sun nor any other party necessarily agrees with them.

Copyright 1994-2009 Sun Microsystems, Inc.
Powered by Atlassian Confluence
Sun Guidelines on Public Discourse Privacy Policy Terms of Use Trademarks Site Map Employment Investor Relations Contact