Talk Abstracts
Monday, May 3rd
Acceptance and Testing
Acceptance Testing the Chicoma HPE Cray EX Supercomputer
Contributors:
Kody Everson, Paul Ferrell, Jennifer Green, Francine Lapid, Daniel Magee, Jordan Ogas, Calvin Seamons, Nicholas Sly
Description:
Since the installation of MANIAC I in 1952, Los Alamos National Laboratory (LANL) has been at the forefront of addressing global crises using state-of-the-art computational resources to accelerate scientific innovation and discovery. This generation faces a new crisis in the global COVID-19 pandemic that continues to damage economies, health, and wellbeing; LANL is supplying high-performance computing (HPC) resources to contribute to the recovery from the impacts of this virus. Every system that LANL's HPC Division installs requires an understanding of the intended workloads for the system, the specifications and expectations of performance and reliability for supporting the science, and a testing plan to ensure that the final installation has met those requirements. Chicoma, named for a mountain peak close to Los Alamos, NM, USA, is a new HPC system at LANL purchased for supporting the Department of Energy's Office of Science Advanced Scientific Computing Research (ASCR) program. It is intended to serve as a platform to supply molecular dynamics simulation computing cycles for epidemiological modeling, bioinformatics, and chromosome/RNA simulations as part of the 2020 Coronavirus Aid, Relief, and Economic Security (CARES) Act. Chicoma is among the earliest installations of the HPE Cray EX supercomputer running the HPE-Cray Programming Environment (PE) for Shasta Architecture. This paper documents the Chicoma acceptance test-suite, configurations and tuning of the tests under Pavilion, presents the results of acceptance testing, and concludes with a discussion of the outcomes of the acceptance testing effort on Chicoma and future work.
Contributors:
Veronica G. Vergara Larrea, Reuben Budiardja, Paul Peltz, Jeffery Niles, Christopher Zimmer, Daniel Dietz, Christopher Fuson, Hong Liu, Paul Newman, James Simmons
Description:
In this paper, we summarize the deployment of the Air Force Weather (AFW) HPC11 system at Oak Ridge National Laboratory (ORNL) including the process followed to successfully complete acceptance testing of the system. HPC11 is the first HPE/Cray EX 3000 system that has been successfully deployed to production in a federal facility. HPC11 consists of two identical 800-node supercomputers, Fawbush and Miller, with access to two independent and identical Lustre parallel file systems. HPC11 is equipped with Slingshot 10 interconnect technology and relies on the HPE Performance Cluster Manager (HPCM) software for system configuration. ORNL has a clearly defined acceptance testing process used to ensure that every new system deployed can provide the necessary capabilities to support user workloads. We worked closely with HPE and AFW to develop a set of tests for the United Kingdom’s Meteorological Office’s Unified Model (UM) and 4DVAR. We also included benchmarks and applications from the Oak Ridge Leadership Computing Facility (OLCF) portfolio to fully exercise the HPE/Cray programming environment and evaluate the functionality and performance of the system. Acceptance testing of HPC11 required parallel execution of each element on Fawbush and Miller. In addition, careful coordination was needed to ensure successful acceptance of the newly deployed Lustre file systems alongside the compute resources. In this work, we present test results from specific system components and provide an overview of the issues identified, challenges encountered, and the lessons learned along the way.
Storage and I/O 1
New data path solutions from HPE for HPC simulation, AI, and high performance workloads
Contributors:
Lance Evans, Marc Roskow
Description:
HPE is extending its HPC storage portfolio to include an IBM Spectrum Scale based solution. The HPE solution will leverage HPE servers and the robustness of Spectrum scale to address the increasing demand for “enterprise” HPC systems. IBM Spectrum Scale is an enterprise-grade parallel file system that provides superior resiliency, scalability and control. IBM Spectrum Scale delivers scalable capacity and performance to handle demanding data analytics, content repositories and technical computing workloads. Storage administrators can combine flash, disk, cloud, and tape storage into a unified system with higher performance and lower cost than traditional approaches. Leveraging HPE Proliant servers, we will deliver a range of storage (NSD), protocol, and data mover servers with a granularity that addresses small AI systems to large HPC scratch spaces with exceptional cost and flexibility. HPE is working with an emerging HPC data path software entrant known as DAOS. Designed from the ground up through a collaboration between Argonne National Labs and Intel, DAOS reduces I/O friction arising from increasingly complex and competing HPC IO workloads, and exposes the full performance potential of next-generation fabrics, persistent memory, and flash media. DAOS improves both latency and concurrency via an efficient scale-out software-defined storage architecture, without relying on centralized metadata or locking services. Applications can interact with DAOS either through its native object interface or application-specific middleware keeping core functionality lean. HPE’s initial solution bundle combines DAOS with Proliant servers, Infiniband or Slingshot networking, and HPE system management software to deploy DAOS for maximum productivity.
Contributors:
Mark Wiertalla
Description:
Lustre and Spectrum Scale: Simplify parallel file system workflows with HPE Data Management Framework The expanded use of Lustre and Spectrum Scale across supercomputing environments is creating a new set of data management challenges at scale – indexing, finding, moving and protecting data. Additionally, the tool kits have evolved independently around each of them – marginally supported opensource tools for Lustre, as well as licensed products & services from IBM for Spectrum Scale. Cray users who have been using RobinHood, HPSS, and other tools to supplement their chosen PFS – and sometimes both – will want to attend this session to learn how HPE Data Management Framework can solve these old and new problems using a single, scalable software stack. In this session, co-hosted by a hands-on development director and a seasoned solution architect, we will explore the revolutionary architecture behind HPE Data Management Framework that enables Exascale data management solutions for high-performance computing.
Storage and I/O 2
h5bench: HDF5 I/O Kernel Suite for Exercising HPC I/O Patterns
Contributors:
Tonglin Li, Houjun Tang, Qiao Kang, John Ravi, Quincey Koziol, Suren Byna
Description:
Parallel I/O is a critical technique for moving data between compute and storage subsystems of supercomputing systems. With massive amounts of data being produced or consumed by compute nodes, efficient parallel I/O is essential. I/O benchmarks play an important role in this process, however, there is a scarcity of good I/O benchmarks that are representative of current workloads on HPC systems. Towards creating representative I/O kernels from real world applications, we have created h5bench, a set of I/O kernels that exercise HDF5 I/O on parallel file systems in numerous dimensions. These include I/O operations (read, write, metadata), data locality (contiguous or strided in memory or in storage), array dimensionality (1D arrays, 2D meshes, 3D cubes), I/O modes (synchronous and asynchronous), and processor type (CPUs and GPUs). In this paper, we present the observed performance of h5bench executed along all these dimensions on two Cray systems: Cori at NERSC using both the DataWarp burst buffer and a Lustre file system, and Theta at Argonne Leadership Computing Facility (ALCF) using a Lustre file system. These performance measurements help to find performance bottlenecks, identify root causes of any poor performance, and optimize I/O performance. As the I/O patterns of h5bench are diverse and capture the I/O behaviors of various HPC applications, this study will be helpful not only to the CUG community but also to the broader supercomputing community.
Architecture and Performance of Perlmutter's 35 PB ClusterStor E1000 All-Flash File System
Contributors:
Glenn K. Lockwood, Nicholas J. Wright
Description:
NERSC's newest system, Perlmutter, features a 35 PB all-NVMe Lustre file system build on HPE Cray ClusterStor E1000. In this paper, we will present the architecture of the Perlmutter file system starting with its node-level design that balances SSD, PCIe, and Slingshot performance, then discussing the high-level network integration. We also demonstrate early Lustre performance measurements on ClusterStor E1000 for both traditional dimensions of I/O performance (peak bulk-synchronous bandwidth and metadata rates) and non-optimal workloads endemic to production HPC (low-concurrency, misaligned, and incoherent I/O). These results are compared to the performance of the disk-based Lustre and NVMe burst buffer of NERSC's previous-generation system, Cori, to illustrate where all-NVMe provides unique new capabilities for parallel I/O.