Serial Attached SCSI (SAS) chipsets — Linux support status
Revised: Fri Jun 9 13:24:56 PDT 2006
- Linux on Serial Attached SCSI
Linux on Serial Attached SCSI
Serial Attached SCSI (SAS) is a architectural standard for high-speed mass storage, proposed by the SCSI Trade Association in 2003 and approved by the ANSI INCITS organisation's T10 Committee in January 2004 as a replacement for Ultra320 (U320) SCSI. Unlike its U320 predecessor and like its Serial ATA cousin, it relies on a serial-type signaling standard, using two pairs of differential cables supporting point-to-point connections. (Thus, each connected storage device "node" has a unique, predefined address, unlike the case with FiberChannel, which uses loop addressing rather than point-to-point addressing.) A SAS host-bus adapter (HBA), whose circuit design is very similar to that of 10 gigabit Ethernet switches, defaults to four high-speed "channels" that can either connect to storage nodes or to "expander" circuitry to increase the number of possible nodes. Using one level of "expanders", dubbed "fanout expanders", permits addressing of 128 nodes. Adding subsidiary "edge expanders" to the first set increases maximum node count to 16,256. SAS-compliant hardware first appeared in mid-2005, and it should rapidly and completely replace U320 during 2006 as midmarket storage (between FibreChannel on the top end and pure Serial ATA at the bottom).
Implementing the same ideas tested with the Serial ATA (SATA) standard, SAS permits SCSI devices and cabling to simultaneously be significantly cheaper, faster (initial 3GB/sec bus speed, soon going to 20GB/sec and up), with longer cable runs (6 metres), with less cooling impairment (thin cables), running at lower voltage (hence cooler), and with much greater scalability as mentioned above. Although the the U320 spec's extrapolation to a faster U640 parallel-SCSI standard did reach a completed draft stage, SAS has turned out to have such compelling advantages that very likely no U640 hardware will ever come to market.
Because SATA and SAS share a cabling/signaling standard, any SAS HBA may serve simultaneously both SAS and SATA nodes (hard drives, and in theory similar devices). Both types of devices will enjoy the common high bus bandwidth. However, only SAS nodes benefit from SAS device-arbitration abilities and the full SCSI command set. (Those drives are managed using an implementation of SCSI called Serial SCSI Protocol, or SSP.) SATA devices monopolise the bus segments they live on, and use the simpler ATA command set. (Those drives are managed using Serial ATA Tunneling Protocol, or STP. Expander circuitry, if present, is managed using Serial Attached SCSI Management Protocol, or SMP.)
Known SAS HBAs and Their Linux Drivers
Known HBAs are listed below, with our current information on suitable Linux drivers.
- AIC-9410/9410W, adp84xx driver
- AIC-9405/9405W, adp84xx driver
- AIC-44300 (PCI-X), adp84xx driver
- AIC-48300 (PCI-X), adp84xx driver
- AIC-58300, (PCI-X), adp84xx driver
- ASC/ASR-4800SAS (PCI-X), aacraid driver
- ASC/ASR-4805SAS (PCI-Express), aacraid driver
- ASR-4005SAS, aacraid driver
- ASR-4000SAS, aacraid driver
- ASR-4810SAS, aacraid driver
ATTO Technology, Inc.:
- ExpressSAS RAID Adapter (PCI-Express): driver so far unclear (card announced April 24, 2006)
- RAIDcore BC5000 (Broadcom 8C5500 family chips, PCI Express, software RAID), bcraid driver
- RAIDcore BC5500 (Broadcom 8C5500 family chips, PCI Express, hardware RAID), bcraid driver
- PERC 5/i: This is a rebranded LSI Logic MegaRAID card, megaraid_sas driver (N.B.: All of Dell's PERC = PowerEdge RAID Controller series are rebrandings of various OEM models.)
- HP Smart Array 6x and 5x Series plus P600, P400, P400i, E200, E200i, 6400, 6400EM (and no doubt other) HBAs: Compaq cciss driver. Note that some other (non-SAS?) Smart Array cards use the mptlinux driver set, being based on MPT Fusion chipsets; there may be some SAS variants based on that chip family.
ICP (Intelligent Computer Peripherals, aka ICP Vortex Computersysteme GmbH, an Adaptec property):
- ICP9085LI, aacraid driver
- ICP5085BR, aacraid driver
- MegaRAID SAS 8300XLP, PCI-X, zero-channel RAID, megaraid_sas driver
- MegaRAID SAS 8308ELP, 4-port PCI-Express, Intel IOP333 I/O processor, megaraid_sas driver
- MegaRAID SAS 8344ELP, 4-port PCI-Express, Intel IOP333 I/O processor, megaraid_sas driver
- MegaRAID SAS 8408E, 8-port PCI-Express, Intel IOP333 I/O processor, battery-backed 256MB SDRAM cache module megaraid_sas driver. Correspondent Berkley Shands advises: 1. Disable write caching, to get 170MB/sec throughput without, on 4-drive RAID, versus 100MB/sec with. 2. "echo 8 > /sys/block/sdb/queue/nr_requests" or the units fail completely and require a system hard reset. LSI says there is a fix for this, but it isn't ready yet. 3. Unit yields 1.2GB/Sec sustained reads off 4 X 4 RAID0. 4. RAID5 writes are very slow: 30MB/sec per 4-drive set. Reads, on the other hand, were outstanding at 870MB/sec, the limit of the PCI-X to PCI-e bridge chipset in the controller. 6. Much lower CPU load than HighPoint RocketRAID 2320 (SATA-II-only), with a better user interface.
- MegaRAID SAS 8408SE, megaraid_sas driver
- MegaRAID SAS 8480SE, megaraid_sas driver
- MetaRAID SAS 8480E, 8-port PCI-Express, megaraid_sas driver
- MegaRAID SAS 3800X, 8 external ports, PCI-X, mptsas driver (mptlinux driver set for MPT Fusion chipsets). Uses an LSI SAS1068(E) chip.
- MegaRAID SAS 3041E, 4-port PCI-Express, mptsas driver (mptlinux driver set for MPT Fusion chipsets). Uses an LSI SAS1064(E) chip.
- MegaRAID SAS 3041X, 4-port PCI-X, mptsas driver (mptlinux driver set for MPT Fusion chipsets). Uses an LSI SAS1064(E) chip.
- MegaRAID SAS 3080X, PCI-X, mptsas driver (mptlinux driver set for MPT Fusion chipsets). Uses an LSI SAS1068(E) chip.
- MegaRAID SAS 3442X 8-port (half external, half internal) PCI-X, mptsas driver (mptlinux driver set for MPT Fusion chipsets)
- MegaRAID SAS 3442E 8-port (half external, half internal) PCI-Express, mptsas driver (mptlinux driver set for MPT Fusion chipsets). Uses an LSI SAS1068(E) chip.
- MegaRAID SAS 3443X-R, adds RAID, PCI-X, mptsas driver (mptlinux driver set for MPT Fusion chipsets)
Any other cards based on LSI SAS1064 / SAS1064a / SAS1064E / SAS1066 / SAS1066E / SAS1068 / SAS1068E / SAS1078 chips can be counted on being compatible with the mptsas driver.
- TARO M9000 SODIMM card: This is an OEMed Adaptec AIC-9410 chip on a Tyan-specific card (for some of that manufacturer's motherboards), and thus uses the adp84xx driver
Linux Distributions' Support Status
The Linux kernel community have not achieved consensus on the SAS driver model, in part because of some contentious history: An LSI megaraid_sas driver had existed in the mainline kernel for a while, when Adaptec employee Luben Tuikov released a radically different "adp94xx" driver and proposed it for adoption by mainline. That work was then harshly criticised by Andrew Patterson (of Hewlett-Packard), Christoph Hellwig, and other principal SCSI kernel programmers, as being gratuitously Adaptec-specific, and was not accepted. No unifying overall framework has yet been worked out, so support has been largely so far (a/o June 2006) relegated to vendor kernels and third-party patches.
SLES9 Service Pack 3:
- aacraid driver
- adp84xx driver
- megaraid_sas driver
- mptsas driver
- possibly others (this page is unfinished)
RHEL4 Update 3 (w/kernel 2.6.9):
- aic94xx (appears to be Red Hat driver author Jeff Garzik's alternative to Adaptec's adp84xx driver)
RHEL3 Update 6 (w/kernel 2.4.21-37):
- probably others (this list is probably incomplete)
Notable Server Hardware and Drivers
- HP ProLiant BL, DL, and ML models (BL10e model G1, BL20p models G1/G2/G3, BL30p all models, BL40p all models, DL580 models G1/G2/G3, DL380 models G1/G2/G3/G4/G4 Packaged Cluster with MSA500 G2, DL360 models G1/G2/G3/G4, DL320 models G1/G2/G3, ML570 models G1/G2/G3, ML330 models G1/G2/G3, ML370 models G1/G2/G3/G4, ML350 models G1/G2/G3/G4/G4p, ML310 models G1/G2): Compaq cciss driver
- IBM eServer xSeries 366 and xSeries 460: adp84xx driver
- Sun Fire series: seems to uniformly use LSI-SAS1064 and similar MPT Fusion chips: mptsas driver
I've created this document from public sources in order to learn about the subject, but cannot claim any expertise. Accordingly, it may contain significant errors — and corrections are greatly appreciated. Please send all such to email@example.com.