Big Red RAID

Dave Schuller with three of the four Big Red RAID servers. The fourth has already been assigned for duty on the MacCHESS Linux Beowulf cluster Sirius .

MacCHESS staff recently (February 2002) completed assembly of 4 two terabyte Linux RAID disk servers in the Big Red RAID project.

Back to Schuller staff page

© Last updated March 2002
Webmeister: David J. Schuller



Design Goals
The point was to build a lot of RAID capacity for not a lot of money. Performance was a secondary consideration. Therefore we went with ATA disks instead of SCSI, and did not invest in hot swap drives or power supplies.

The Name
The enclosures are beige. Big Red is the unofficial nickname of Cornell University athletics.

Hardware & Construction

Parts, per server:

Construction
Some of the interesting points:

Cabinet views

Front view, doors removed
The 7 lowest drives on the right hand side have the custom sleds. The root disk is located behind the power switch panel at the top left.

Front view, with doors

Inside view
Look at that mess of cables! We moved the motherboard to the right side of the chassis to make more room for the cables coming off the PCI cards. The special 460 W NMB power supply is just above the motherboard, it runs the motherboard, fans, root disk, CD-ROM and right-side RAID disks. The additional 300 W ATX power supply (bottom foreground) runs the left-side RAID disks.

Software & Configuration

Capacity

Each server contains 24x 100 GB disks for a total raw RAID space of 2.4 TB. After RAID configuration, formatting and assignment of spares, the usable space is 1.95 TB. Current Linux kernels have a hard limit of 2 TB per filesystem, so there was no point in trying to squeeze any extra space out of it.

Benchmarks

I found a Linux RPM version of Bonnie, here are the numbers with some other systems as well for comparison:

              ---Sequential Output (nosync)--- ---Sequential Input-- --Rnd Seek-
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k (03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
alpha1 1*4096 19303 49.2 31346 17.9 17858 12.1 33897 78.9 48043 18.2 241.2 2.5
alpha2 1*4096 26601 67.4 65757 38.6 41512 29.5 37615 86.5106638 40.7 514.1 6.1
bigrd1 1*2047 17510 100 32501 41.9 10908 8.0 10430 63.5 27525 14.3 116.6 1.0
bigrd2 1*2047 14475 87.9 27616 20.9 21327 12.4 14566 90.1125586 57.3 332.8 3.2
bigrd3 1*2047 15737 97.9103108 93.2 81002 67.4 15455 97.6151118 84.7 353.1 6.3 ****

Commentary
Why we did what we did, and what we would do differently:

Personnel

Planning and construction were carried out by Dave Schuller and Bill Miller of MacCHESS.

  • Back to Top
    Back to Schuller staff page
    MacCHESS home page