Please consider a donation to the Higher Intellect project. See https://preterhuman.net/donate.php or the Donate to Higher Intellect page for more info. |
SGI Altix
Altix is the name used for a broad line of servers and supercomputers produced by Silicon Graphics (International) using Intel processors. It succeeded the MIPS/IRIX-based Origin 3000 servers.
The line was first announced on January 7, 2003, with the Altix 3000 series, based on Intel Itanium 2 processors and SGI's NUMAlink processor interconnect. At product introduction, the system supported up to 64 processors running Linux as a single system image and shipped with a Linux distribution called SGI Advanced Linux Environment, which was compatible with Red Hat Advanced Server. By August 2003, many SGI Altix customers were running Linux on 128p and even 256p SGI Altix systems, but SGI officially announced 256-processor support within a single system image of Linux on March 10, 2004 using an 2.4-based kernel. The SGI Advanced Linux Environment was eventually dropped after support using a standard, unmodified SUSE Linux Enterprise Server (SLES) distribution for SGI Altix was provided with SLES 8 and SLES 9. Later, SGI Altix 512-processor systems were officially supported using unmodified, standard Linux distribution with the launch of SLES 9 SP1. Besides full support of SGI Altix on SUSE Linux Enterprise Server, a standard and unmodified Red Hat Enterprise Linux was also fully supported starting with SGI Altix 3700 Bx2 with RHEL 4 and RHEL 5 with system processor limits defined by Red Hat for those releases.
On 14 November 2005, SGI introduced the Altix 4000 series based on the Itanium 2. SGI later officially supported 1024-processors systems on an unmodified, standard Linux distribution with the launch of SLES 10 in July 2006. SGI Altix 4700 was also officially supported by Red Hat with RHEL 4 and RHEL 5—maximum processor limits were as defined by Red Hat for its RHEL releases.
The Altix brand has been also used for a variety of systems based on multi-core Intel Xeon processors. These includes the Altix XE rackmount servers, Altix ICE blade servers and Altix UV supercomputers.
NASA's Columbia supercomputer, installed in 2004, is a 10,240-microprocessor cluster of twenty Altix 3000 systems, each with 512 microprocessors, interconnected with InfiniBand.
Altix 3000[edit]
The Altix 3000 was the first generation of Altix systems. At the middle- and high-end it largely took the existing SGI Origin 3000 packaging and interconnect, and substituted Intel Itanium processors for the traditional MIPS CPUs. As a result many components can be swapped between the two families, such as routers, NUMAlink cables, etc.
It was succeeded by the Altix 4000 in 2004, and the last model was discontinued on 31 December 2006.
Altix 330[edit]
The Altix 330 is an entry-level server, with a target of under US$7,000 for a basic single module system. Like higher-end models, the Altix 330 can be built up to its maximum configuration by connecting multiple modules through the NUMAlink interconnect. A single system image may contain 1 to 16 Itanium 2 processors and 2 to 128 GB of memory.
Altix 350[edit]
The Altix 350 is a mid-range "technical server" which supports up to 32 Itanium 2 processors and up to 384GB of RAM.
Altix 1330[edit]
The Altix 1330 is a cluster of Altix 330 systems. The systems are networked with gigabit Ethernet or 4X Infiniband.
Altix 1350[edit]
The Altix 1350 is a cluster of Altix 350 systems.
Altix 3300[edit]
The Altix 3300 is a mid-range model supporting 4 to 12 processors and 2 to 48 GB of memory. It is packaged in a short (17U) rack.
Altix 3700[edit]
The Altix 3700 is a high-end model supporting 16 to 512 processors and 8 GB to 2 TB of memory. It requires one or multiple tall (39U) rack(s). A variant of the Altix 3000 with graphics capability is known as the Prism.
It is based on the third generation NUMAflex distributed shared memory architecture and it uses the NUMAlink 3 interconnection fabric. The Altix 3000 supports a single system image of 64 processors. If there are more than 64 processors in a system, then the system must be partitioned.
The basic building block is the C-brick. Each C-brick contains two nodes. A C-brick is a 4U high rackmount unit. Each node contains two Intel Itanium 2 processors which connects to the Super-Bedrock ASIC through a single front side bus. The Super-Bedrock ASIC is a crossbar for the processors, the local RAM, the network interface and the I/O interface. The two Super-Bedrock ASICs in each brick are connected internally by a single 6.4 GB/s NUMAlink 4 channel. A processor node also contains 16 DIMM slots that accept standard DDR1 DIMMs with capacities of 4 to 16 GB.
Altix 3700 Bx2[edit]
The Altix 3700 Bx2 is a high-end model supporting 16 to 2,048 Itanium 2 processors and 12 GB to 24 TB of memory. It requires one or multiple tall (40U) rack(s).
Altix 4000[edit]
Codenamed "Tollhouse," the Altix 4000 series was Silicon Graphics' second and last Itanium-based product family. It was composed of two models: the Altix 450, a mid-range server generally limited to no more than a single full-sized rack; and the Altix 4700, a high-end system for the High Performance Computing and data-intensive workload markets.
The Altix 4000 family used a different approach to packaging components. Whereas the SGI Origin 3000 and Altix 3000 used "bricks" of 2U/3U compute packages, the Altix 4000 would be built up from blade-based Individual Rack Units or IRUs. Each IRU can be configured with a number of "blades," where each blade can offer CPU, memory, I/O expansion, or new FPGA-based Reconfigurable Application Specific Computing (RASC) resources. Each IRU incorporates two NUMAlink 4 routers to handle interconnecting multiple IRUs into a larger configuration.
Operating system choices included RedHat Enterprise Linux Advanced Server and SuSE Linux Enterprise Server, with SGI's ProPack adding various tools and management utilities.
Altix 4700[edit]
The Altix 4700 was announced in November 2005 as the first member of the Altix 4000 family. It was introduced with blades using the single-core Itanium2 "Madison" CPUs, with promises that it would support the dual-core "Montecito" and "Montvale" processors that Intel would deliver at a later date. Maximum configuration would initially be 512 sockets per single system image, with some sources reporting 2,048 sockets in later shipments and a theoretical maximum of 8,192. Maximum memory was 128TB, ultimately expandable to 192TB using 16GB DIMMs in single-socket compute/memory blades. Peak performance in a single 42U rack was expected to approach 1 teraflop using dual-core CPUs.
Altix 450[edit]
On June 26, 2006 SGI announced the Altix 450. Featuring the dual core Itanium2 "Montecito" processors, the Altix 450 can deliver up to 2.5 times the performance of the previous generation Altix 350 system. Up to 4 IRUs can be linked through their built-in NUMAlink routers, yielding a single system image with a maximum 38 sockets (76 cores) and 608GB of memory. Two of these configurations can be packaged in one full-height 42U cabinet, and could be clustered together.
Altix XE[edit]
The Altix XE servers are based on Intel Xeon x86-64-architecture processors. Models include the:
- Altix XE210 server supported up to two Dual or Quad-Core Intel Xeon processors, 5100 Series or 5300 series, 32GB DDR2 667 MHz FBDIMM memory, 1 x PCIe x8 (low profile) and 1 x PCI-X 133 MHz (full height) PCI slots, and Three SATA/SAS drive bays. It uses the Intel S5000PAL motherboard.
- Altix XE240 server supported up to two Dual or Quad-Core Intel Xeon processors, 5100 Series or 5300 series, 32GB DDR2 667 MHz FBDIMM memory, two PCI slots configuration options (option 1: 2 x PCIe x4 (low profile), 2 x PCIe x4 (full height), 1 x PCEe x8 (full height); or option 2: 2 x PCIe x4 (low profile), 3 x PCI-X 133 MHz (full height), 1 x PCI-X 133 MHz (full height), and Five SATA/SAS drive bays. As the XE210, it uses the Intel S5000PAL motherboard.
- Altix XE250 server which uses the Super Micro X7DWN+ motherboard.
- Altix XE270 server is a 2U configuration with Intel Xeon processor 5500 series, Choice of up to 18 DDR3 DIMMs (2GB, 4GB, or 8GB DIMMs), 2 x PCIe x8 gen 2 (low profile), 1 x PCIe x4 gen 1 (low profile), 2 x PCI-x 133/100 (low profile) PCI slots, Eight SATA or SAS drive bays with optional hardware RAID (0, 1, 5, 6, 10). Uses the Super Micro X8DTH-6F motherboard.
- Altix XE310 server was introduced January 8, 2007 and contained two "Atoka" nodes per XE310 (probably Super Micro X7DBT (w/o InfiniBand) and X7DBT-INF), up to four Dual or Quad-Core Intel Itanium Xeon processors, 5100 Series or 5300 series (two per node), 64GB DDR2 667 MHz or 800 MHz FBDIMM memory (32GB per node), 2 x PCIe x8 (1 per node) PCI slots, and Four SATA/SAS drive bays (two per node).
- Altix XE320 server was introduced November 12, 2007 and contained two Super Micro X7DWT-INF nodes. It contained up to four Xeon 5100, 5200, 5300 or 5400 CPUs and 64 GB memory.
- Altix XE340 server contains 2 compute nodes within a 1U configuration, Intel Xeon processor 5500 series, Choice of up to 12 DDR3 DIMMs per node (2GB, 4GB, or 8GB DIMMs), 2 x PCIe x16 (1 per node) - low profile PCI slot, and Four SATA drive bays (2 per node) with optional SAS and hardware RAID 0, 1. It uses the Super Micro X8DTT-IBQF (with InfiniBand) or X8DTT-F (without InfiniBand) nodeboards.
- Altix XE500 server is a 3U configuration with Intel Xeon processor 5500 series, Choice of up to 18 DDR3 DIMMs (2GB, 4GB, or 8GB DIMMs), 2 x PCIe x16 gen2 (full height) and 4 x PCIe x8 gen2 (full height) PCI slots, and Eight SATA or SAS drives with optional hardware RAID (0, 1, 5, 6, 10). It uses the Super Micro X8DAH+ motherboard.
- Altix XE1200 cluster
- Altix XE1300 combines multiple XE310 compute nodes with an Altix XE240 head node
- Altix XE1300 with Integrated Visualization combines multiple XE310 compute nodes with Virtu VN200 nodes and an Altix XE250 head node
All Altix XE systems supported Novell SUSE Linux Enterprise Server, Red Hat Enterprise Linux, and Microsoft Windows. Later, VMware support was added across the Altix XE product line.
Altix ICE[edit]
The Altix ICE blade platform is an Intel Xeon-based system featuring diskless compute blades and a Hierarchical Management Framework (HMF) for scalability, performance, and resiliency. While the earlier Itanium-based Altix systems ran a single-system image (SSI) Linux kernel on 1024 processors or more using a standard SuSE Linux Enterprise Server (SLES) distribution, the Altix ICE's clustering capabilities uses standard SLES or Red Hat Enterprise Linux distributions. NASA's Pleiades is currently the largest deployed Altix ICE consisting of 168 ICE 8200EX and ICE 8400EX racks with a total of 100,352 cores.
Altix ICE 8200[edit]
The system consists of 42U racks which hold up to 4 IRUs (individual rack units) which support up to 16 single-wide blades.
Two IRU variants are available:
- ICE 8200 - Dual plane, two DDR InfiniBand switch blades.
- ICE 8200LX - Single plane, one (later two) DDR (20 Gb/s) or QDR (40Gb/s) InfiniBand switch blades.
- ICE 8200EX - Dual plane, four DDR (20 Gb/s) or QDR (40Gb/s) InfiniBand switch blades.
Following blade variants are available:
- IP83 (Atoka-P) - single-wide, up to 2 Intel Xeon 5100/5300 or 5200/5400 CPUs, 8 FB-DIMM slots.
- IP85 - double-wide, up to 2 Intel Xeon 5100/5300 or 5200/5400 CPUs, 16 FB-DIMM slots.
- IP95 - single-wide, up to 2 Intel Xeon 5500/5600 CPUs, 12 DDR3 DIMM slots, 1 dual-port InfiniBand HCA.
Other rack components:
- 1U Rack leader controller - required in each system rack, distributes jobs to the compute nodes, holds blade boot images.
- 1U Administrative server with PCIe/PCI-X expansion - supports an optional console, administrative software and three PCI Express option cards.
Optional service nodes, which can be added to a system rack and used specifically as an optional login, batch, or fabric functional node:
- 2U Service node - Altix XE250 or Altix XE270.
- 3U Service node - Altix XE500.
Altix ICE 8400[edit]
The system consists of 42U racks which hold up to 4 IRUs (individual rack units) which support up to 16 blades.
Two IRU variants are available:
- ICE 8400LX - Single plane, two 40Gb/s InfiniBand switch blades.
- ICE 8400EX - Dual plane, four 40Gb/s InfiniBand switch blades.
Following blade variants are available (these blades are not compatible with the earlier Altix ICE 8200):
- IP101 - up to 2 Intel Xeon 5500/5600 CPUs, 12 DDR3 DIMM slots, uses a single-port InfiniBand host controller adapter (for 8400LX).
- IP103 - up to 2 Intel Xeon 5500/5600 CPUs, 12 DDR3 DIMM slots, uses 1 dual-port InfiniBand HCA (for 8400EX).
- IP105 - up to 2 Intel Xeon 5500/5600 CPUs, 12 DDR3 DIMM slots, uses 2 single-port InfiniBand HCAs (for 8400EX).
- IP106 - up to 2 AMD Opteron 6100, 16 DDR3 DIMM slots, uses 2 single-port InfiniBand HCAs (for 8400EX).
- IP110 - up to 2 AMD Opteron 6100, 16 DDR3 DIMM slots, uses 1 dual-port InfiniBand HCA (for 8400EX).
Other rack components:
- 1U Rack leader controller - required in each system rack, distributes jobs to the compute nodes, holds blade boot images.
- 2U Administrative server with PCIe expansion - supports an optional console, administrative software and PCI Express option cards. The administrative server is installed in the primary rack in the system. It is an SGI Rackable C2108-TY10, previously offered as the SGI Altix XE270.
Optional service nodes, which can be added to a system rack and used specifically as an optional login, batch, or fabric functional node:
- 1U Service node - ?.
- 2U Service node - SGI Rackable H2106-G7.
- 3U Service node - SGI Rackable C3108-TY11, previously marketed as the Altix XE500.
- 4U Service node - SGI Altix UV 10.
Altix UV 10[edit]
The Altix UV 10 is a rebadged QSSC-S4R server. Being a regular PC server, its only connection to the bigger Altix UVs is its marketing name.
Altix UV 100/1000[edit]
The Altix UV supercomputer architecture was announced in November 2009. Codenamed Ultraviolet during development, the Altix UV combines a development of the NUMAlink interconnect used in the Altix 4000 (NUMAlink 5) with quad-, six- or eight-core "Nehalem-EX" Intel Xeon 7500 processors or ten and eight-core "Westmere-EX" Xeon E7 processors. Altix UV systems run either SuSE Linux Enterprise Server or Red Hat Enterprise Linux, and scale from 32 to 2,048 cores with support for up to 16 Terabytes (TB) of shared memory in a single system image.
The main component of the Altix UV 100/1000 servers is the IP93 nodeborad. One nodeboard contains 2 Intel Xeon 7500 series (Beckton) processors with four, six or eight cores or E7 series (Westmere-EX) with eight or ten cores, the Intel 7500 (Boxboro) chipset and the UV Hub which works with the CPU as a node controller.
UV Hub provides:
- GRU (Global Reference Unit) - which provides cache coherence, extends the address space from (44-bits of Nehalem Xeon to 53-bits), block copy operations (used to accelerate sending MPI messages), scatter/gather operations.
- AMU (Active Memory Unit) - which provides atomic operations, update multicasting and message queues.
Altix UV 1000[edit]
The system consists of 42U racks which hold up to 2 IRUs (individual rack units) which support up to 16 blades. Following blade variants are available:
- Compute blade - provides 2 LGA 1567 sockets and up to 16 DIMMs.
- BaseIO enabled compute blade - A compute blade with an I/O riser card.
- Dual disk enabled compute blade.
- Two-Slot Internal PCIe enabled compute blade - provides 2 PCIe slots (up to x16).
- External PCIe enabled compute blade - used to connect an external PCIe expansion enclosure (x16, x4, x8 and x4).
The system scales as an SSI up to 256 CPUs (this is a Xeon limitation). Such configuration requires 4 racks and provides up to 2048 cores, 4096 threads (2560 cores /5120 threads with Westmere-EX) and 2048 DIMMs (up to 16 TB addressable by the Nehalem-EX). With the use of the Jolt technology 16384 nodeboards (32768 CPUs, 512 racks) can be connected together (NUMAlink node id limitation) to form a 262144 core machine with up to 8 PB memory (53-bit GRU limit).
Supported operating systems and limitations:
- Windows Server 2008 / 2008 R2: 256 cores / 2 TB
- Red Hat Enterprise Linux 6: 128 cores / 2 TB
- SUSE Linux Enterprise Server 11: 16 TB
UV 2000[edit]
Hardware wise, the biggest change is the use of the lower end Xeon E5 4600 (Sandy Bridge-EP) or Xeon E5 4600v2 (Ivy Bridge-EP) CPUs (both Socket LGA2011), which acquired some E7 properties - they are scalable to more than 2 sockets. The maximum core number per SSI for the Sandy Bridge variant is lower than the UV 1000 - 2048 instead of 2560 (the biggest E5 is eight-core, the biggest E7 ten-core, they still don't support more than 256 sockets). There are sources reporting the use of Xeon E5-4650v2 CPUs with 10 cores per CPU. Supported memory has quadrupled to 64TB (Sandby Bridge 46-bit addressing vs 44-bit addressing of Beckton/Westmere-EX). A new NUMAlink generation is used - NUMAlink 6, up to 6.7 Gb/s. Cluster limitations are the same as for UV 1000 - 16384 (NUMAlink node id limitation) nodeboards (32768 CPUs), can be connected together to form a 262144 core machine with up to 8 PB memory (53-bit GRU limit).
Two types of blades are available:
- 2 socket with Xeon E5-4600 family
- 1 socket with Xeon E5-4600 family + 1 accelerator: Xeon Phi or nVidia Quadro or nVidia Tesla K20
UV 300H[edit]
The UV 300H is the UV for SAP HANA. This server uses 4 socket nodes connected via NUMAlink 7. Sockets are populated with Xeon E7 v2 (Ivy Bridge-EX, Socket LGA2011-1) CPUs with up to 15 cores each. The 300H is designed to scale to 32 sockets (480 cores) and 24 TB of RAM.
External links[edit]
- SGI's webpage for Altix
- Linux Journal article regarding scaling of Linux on Altix
- New Altix Software Allows 256-Processor Linux System
- Linux Magazine about scaling Altix to 512p
- HPCwire interview about scaling Altix to 1024p
- SGI Altix Again Crushes World Record for Memory Bandwidth
- SGI Altix Servers Attain Common Criteria Security Certification
- Article on SGI ProPack, Real-time, and Cluster Support for Altix
- Linux Magazine article on Altix one year after the announcement of the 3000
- Hardware Today: SGI Server Snapshot at ServerWatch - An overview of the Altix product line as of Fall 2005.
Manuals[edit]
- 007-4326-003 - SGI Altix Systems Dual-Port Gigabit Ethernet Board User’s Guide
- 007-4417-002 - SGI Altix 350 Quick Start Guide
- 007-4579-001 - SGI Altix 3000 User’s Guide
- 007-4660-002 - SGI Altix 350 System User’s Guide
- 007-4805-001 - Connecting a System Console to an SGI Altix 330 System
- 007-4823-003 - SGI Altix 4700 System User’s Guide
- 007-4857-003 - SGI Altix 450 System User’s Guide
- 007-4870-002 - SGI Altix XE210 User’s Guide
- 007-4873-003 - SGI Altix XE240 User’s Guide
- 007-4960-005 - SGI Altix XE310 System User’s Guide
- 007-4979-004 - SGI Altix XE1300 Cluster Quick Reference Guide
- 007-4986-005 - SGI Altix ICE 8200 Series System Hardware User’s Guide
- 007-5466-003 - SGI Altix XE320 System User’s Guide
- 007-5467-004 - SGI Altix XE250 System User’s Guide
- 007-5515-002 - SGI Altix Systems Quad-Port Gigabit Ethernet PCI Express Low Profile Card User’s Guide
- 007-5635-002 - SGI Altix ICE 8400 Series System Hardware User’s Guide
- 007-5662-003 - SGI Altix UV 100 System User’s Guide
- 007-5663-003 - SGI Altix UV 1000 System User’s Guide
- 007-5762-003 - Windows on SGI Altix UV Software Install Guide