QsNet

Introduction
HPC Architecture
  1. Shared-memory SIMD machines
  2. Distributed-memory SIMD machines
  3. Shared-memory MIMD machines
  4. Distributed-memory MIMD machines
  5. ccNUMA machines
  6. Clusters
  7. Processors
    1. AMD Opteron
    2. IBM POWER6
    3. IBM PowerPC 970
    4. IBM BlueGene processors
    5. Intel Itanium 2
    6. Intel Xeon
    7. The MIPS processor
    8. The SPARC processors
  8. Accelerators
    1. GPU accelerators
    2. General accelerators
    3. FPGA accelerators
  9. Networks
    1. Infiniband
    2. InfiniPath
    3. Myrinet
    4. QsNet
Available systems
  1. The Bull NovaScale
  2. The C-DAC PARAM Padma
  3. The Cray XT3
  4. The Cray XT4
  5. The Cray XT5h
  6. The Cray XMT
  7. The Fujitsu/Siemens M9000
  8. The Fujitsu/Siemens PRIMEQUEST
  9. The Hitachi BladeSymphony
  10. The Hitachi SR11000
  11. The HP Integrity Superdome
  12. The IBM BlueGene/L&P
  13. The IBM eServer p575
  14. The IBM System Cluster 1350
  15. The Liquid Computing LiquidIQ
  16. The NEC Express5800/1000
  17. The NEC SX-9
  18. The SGI Altix 4000
  19. The SiCortex SC series
  20. The Sun M9000
Systems disappeared from the list
Systems under development
Glossary
Acknowledgments
References

QsNet is the main product of Quadrics, a company that initially started in the 1990s as the British firm Meiko that made parallel systems called the Computer Surface 1 and 2. In the CS-2 a very good logarithmic network was included that was made into a separate product after Meiko closed down and the network part was taken over by the the Italian company Alenia and put in the independent company Quadrics. The success came when Digital/Compaq chose QsNet as the network for its large high-performance computer products, the AlphaServer SC line. Some very large configurations of these machines were sold, e.g., at the Pittsburgh Supercomputing Centre and the French CAE. QsNet proved to be a very fast and reliable network and since about three years QsNet is also offered for cluster systems.

Like Infiniband and Myrinet the network has effectively two parts: the ELAN interface cards, comparable to Infiniband Host Bus Adaptors or Myrinet's Lanai interface cards, and the Elite switch, comparable to an Infiniband switch/router or a Myrinet switch. The topology that is used is a (quaternary) fat tree like in most Infiniband switches, see Figure 5b for an example. The ELAN card interfaces with the PCI-X port of a host computer.

Ready-made Elite switches for clusters come in two sizes: with 8, 32 and 128 ports but nothing in between 32 and 128 ports. Of course one can put together networks for other sizes but this goes at the cost of compactness and speed. A difference with the other switches lies in the providing two virtual bi-directional channels per link. Since the end of 2003 Quadrics sells its second generation QsNet, QsNetII. The structure, protocols, etc., are very similar to those of the former QsNet but much faster: where in [38] a speed of 300 MB/s for an MPI Ping-Pong experiment was measured while QsNetII has a link speed of 1.3 GB/s. In this case the PCI-X bus speed is a limiting factor: it allows for somewhat more than about 900 MB/s. However, with the advent of PCI Express this limitation may be over and can go up to just over 1 GB/s. Also the latency for short messages has improved from ≅ 5 µs to close to 1 µs.
Furthermore, the switch supports two priority levels which greatly helps in a fair distribution of message packets. This is more than Myrinet provides but less than the 16 priority levels of Infiniband.

Like Infiniband, QsNetII has RDMA capabilities that allows to write/read from/to remote memory regions on the ELAN cards. This can however be extended to memory regions of the host processor itself. So, in principle, one would be able to view a QsNet-connected system as a virtual shared memory system. As yet, this has not been realised nor on integrated systems, nor on clusters. Nevertheless, it could be an attractive alternative for the much slower memory paged based virtual shared memory systems like ThreadMarks [1]. A Cray-style shmem library is offered that enables one-sided communication via put and get operations as well as an MPI-2 implementation that supports one-sided communication calls.

Since early 2006 Quadrics also offers 10 Gbit Ethernet cards and switches under the name QSTenG. These products capitalise on the experience obtained in the development of the two generations of QsNet but they are not meant for inter-processor communication. As yet Quadrics does not seem to consider developing multi-protocol products like Myricom's Myri-10G and Infiniband.