Intel Xeon Phi to Power New Petascale Supercomputer at National Renewable Energy Laboratory

Intel Xeon Phi to Power World's Most Power Efficient 1PFLOPS HPC System

by Anton Shilov
09/05/2012 | 09:59 PM

Intel Corp. on Wednesday said that it would work with HP to design and provide the U.S. department of energy's national renewable energy laboratory (NREL) with high-performance computing (HPC) system that will drive research across a number of energy-related initiatives. The HPC system will be based on Intel Xeon processors and Intel Xeon Phi co-processors to deliver 1PFLOPS of performance with a power usage effectiveness (PUE) rating of 1.06 or better.

Another HPC with Intel MIC


The $10 million HPC system will reside at the Energy Systems Integration Facility (ESIF), under construction on the Golden, Colorado, campus. The new system will greatly expand NREL’s modeling and simulation capabilities, including advancing materials research and developing a deeper understanding of biological and chemical processes. It will also support research into fully integrated energy systems that would otherwise be too expensive, or even impossible, to study directly.

The system is scheduled to deliver full compute capacity in the summer of 2013 and will feature approximately 3200 Intel Xeon processors including current-generation Intel Xeon processor E5-2670, future 22nm Ivy Bridge based processors and approximately 600 new Intel Xeon Phi co-processors. The total peak performance of the system is expected to exceed 1PFLOPS (equivalent to a thousand trillion floating point operations per second) and it will be the largest supercomputer dedicated solely to renewable energy and energy efficiency research.

The HPC system will be deployed in two phases that will include scalable HP ProLiant SL230s and SL250s eight-generation servers based on eight-core Intel Xeon E5-2670 processors as well as the next generation of servers featuring future 22nm Ivy Bridge architecture based Xeon processors and Intel Xeon Phi co-processors. The first phase of the HPC installation will begin in November 2012, reaching petascale capacity in the summer of 2013.

"The heart of NREL is based on a powerful combination of the Intel Xeon processor E5 product family, which leads the data center industry in performance per watt, and Intel Xeon Phi co-processors which are setting new records for energy efficiency. We are proud that the very best energy-efficient processing technology in computing is the foundation for the supercomputer that will drive the research for renewable energy and energy-efficient technologies," said Raj Hazra, vice president and general manager of Intel technical computing group.

Maximum Efficiency

Leading energy-efficient capabilities of Intel Xeon processors and Intel Xeon Phi co-processors combined with the new HP warm water cooling solution and innovative data center design will result in this facility likely being the world's most efficient data center with a power usage effectiveness (PUE) rating of 1.06 or better.

The data center industry uses the PUE rating as the standard measurement to describe the power usage effectiveness and measure the efficiency of any given site. The ideal PUE rating would be 1.0, and PUE of 2.0 means that for every watt of power used for IT equipment, an additional watt is used in overhead like cooling systems. According to the EPA's Energy Star program, a typical PUE is 1.92. With a PUE rating for this system expected to be 1.06 or better, NREL's installation will be nearly twice efficient as average.

"At NREL, we have taken a holistic approach to sustainable computing. This new system will allow NREL to increase our computational capabilities while being mindful of energy and water used. We will take advantage of both the bytes of information produced and the BTUs produced. The new HPC system will dramatically improve our modeling and simulations capabilities used to advance energy efficiency and renewable energy technologies as well as energy system integration. At the same time, NREL's partnership with Intel and HP will demonstrate best-in-class for energy-efficient computing and data center design and will provide a laboratory for collaborative research to address future energy challenges in HPC systems and data centers," said Steve Hammond, NREL computational science director.

Intel and HP experts worked with data center designers at NREL to leverage innovative warm water liquid cooling technology to maximize the reuse of heat. There will not be any traditional mechanical or compressor-based cooling systems; the "waste heat" from the computer system will be used as the primary heat source in the adjacent office and lab space. Excess heat can also be exported to other areas of the NREL campus. The data center design is compact, resulting in short runs for both electrical and plumbing components.