There is a bluish glow in the corridors of the new HoreKa supercomputer at the Karlsruhe Institute of Technology (KIT). Walls full of technology tower up to the left and right, cables spread like an octopus. The system has been working on for a long time. Among other things, astrophysicists from the CERN particle accelerator have tested the system with calculations, said Jennifer Buchmüller, head of the High Performance Computing (HPC) department at the Steinbuch Center for Computing of the KIT to dpa. HoreKa will be officially inaugurated on Friday.
Scientists from all over Germany should use the supercomputer. According to the information, it can handle around 17 quadrillion arithmetic operations per second. In other words: 17 PetaFlops with double precision (FP64). This makes HoreKa one of the 15 fastest computers in Europe. It is intended to help answer questions from the earth system and material sciences, energy and mobility research in engineering, and particle and astroparticle physics.
Intel-CPUs und Nvidia-GPUs
Intel’s first generation of server processors with structure widths of 10 nanometers, Ice Lake-SP, paired with Nvidia’s A100 GPU accelerators is used. The Xeon Platinum 8368 used each have 38 CPU cores and achieve clock frequencies between 2.4 and 3.4 GHz. HoreKa is already included in the Top500 list of the world’s fastest supercomputers. Measured against the Linpack benchmark, the system achieved 15 PetaFlops, which is enough for 52nd place. Depending on the application, the measured computing power is higher or lower.
The A100 accelerator is also used by the fastest European supercomputer, the Jewels Booster Module, which is also located in Germany, more precisely at the Jülich Research Center. Instead of Intel CPUs, AMD’s Epyc 7002, alias Rome, is used. The combination of processors and A100 accelerators creates 44.1 PetaFlops.
High efficiency with hot water cooling
The KIT promotes HoreKa with a green topic: On the international Green500 list of the most energy-efficient supercomputers worldwide, HoreKa lands in 13th place. With one watt of electrical energy, the system manages around 22.4 billion operations per second (22.4 GigaFlops / watt ). The mentioned Jewels Booster Module ranks 8th on the Green500 list with 25 GigaFlops per watt.
In Karlsruhe, the hot water cooling of the computer building is used to cool the system all year round with minimal use of energy, as Buchmüller explained. “In the colder seasons, the offices can also be heated with the waste heat.” According to KIT, up to 90,000 liters of cooling water flow through the pipes per hour.
Lenovo’s server platform ThinkSystem SD650 v2, on which the KIT relies on the HoreKa supercomputer.
(Image: Lenovo)
According to Nick Kriegeskotte, Head of Infrastructure and Regulation at the Bitkom industry association, hot water cooling and waste heat utilization are more widespread in the HPC area than elsewhere. “Although a temporarily higher energy consumption should be allowed for research purposes than in standard applications, high-performance computing systems in particular are often significantly more efficient than commercial systems,” he said.
The infrastructure for hot water cooling comes in its own cabinet, including the pumps, for example.
(Image: KIT)
Another approach: better utilizing the capacities of the servers with the help of virtualization. “Here, a physical server is divided into several virtual environments in which different tasks are processed, all of which use the computing power of one server”, explained Kriegeskotte. Chirag Dekate from the analysis company Gartner also gave the example “to deliver extreme performance with the smallest computing requirement”.
Booking system for use
According to Kriegeskotte, high-performance computers, in contrast to commercial data centers, also allow computing power to be planned more easily, “so that the systems can be optimally used”. This is also where Buchmüller and her team at KIT want to start: Researchers have to register their project, are allocated a quota and are placed on a waiting list.
In addition, the HoreKa experts want to promote programs that conserve resources. “How to program in an optimized way is a fine art,” says Buchmüller. For this purpose, extra courses are offered, but if necessary the individual code can also be examined and improved. “No physicist needs to become an expert in programming,” says the specialist.
The subject of energy efficiency has long since arrived in the world of supercomputers. The energy consumption has been reduced enormously in the past few years, said Kriegeskotte. Especially since there is general consensus that an expansion of the digital infrastructure is absolutely necessary – among other things, in order to be able to achieve the climate targets.
Analyst Dekate also warns against playing off sustainability and performance against each other in supercomputing, “because it implies the availability of non-existent alternatives”. The developers of HPC architectures do not consider energy efficiency and power consumption afterwards, but rather important design criteria. The cost of not investing in solving challenges like carbon sequestration or introducing myopic delays “would be immeasurably worse”.
(mmma)
