Since 1986 - Covering the Fastest Computers in the World and the People Who Run Them

Language Flags
February 19, 2014

Scheduling HPC as a Service

Carlo del Mundo
smnetwork

HPC has always been the go-to field for solving large-scale scientific and engineering problems. However, running applications on HPC systems requires significant technical know-how of the underlying systems software to effectively run applications. To address the tedium of setting up HPC environments, HPC as a Service (HPCaaS) has recently been proposed to move HPC into the cloud.

Borrowing from the success of Software as a Service (SaaS), HPCaaS purports to do the same — simplifying and commoditizing HPC to the masses via an automated cloud delivery system. The emphasis is making the process of scheduling jobs on HPC resources as transparent as possible. A user need not know how many processors to use but that a job gets executed with a specified amount of parallelism.

So, what’s limiting the adoption of HPC as a service? For one, typical HPC jobs are rigid in nature; they must execute under a finite set of resources. For instance, a user must explicitly specify the exact number of processors before submitting a parallel job. If there is enough processors, the job is run. If not, the system stalls until there is enough processors to fulfill the task. This rigid way of scheduling works well when there are enough resources to fulfill the job.  However, in cases where HPC is seen as a service, resource contention causes stalls resulting in poor utilization.

To alleviate the burden of poor utilization when there aren’t enough resources available, Kuo-Chan Huang, associate professor in the Department of Computer Science from National Taichung University applies the concept of moldable jobs (borrowed from MPI) to HPC.  He notes that, “a moldable job approach can automatically select a most appropriate amount of processors for a job’s execution based on application speedup models and workload conditions at the moment.”

Then, the workload management system adapts to the needs of the application — throttling down as resources become scarce. Such moldable properties for HPC jobs allows the job management and scheduling system to allocate resources based on the needs of the job. This flexibility is critical in executing jobs at an efficient rate.

Huang’s team proposes two new moldable scheduling techniques achieving up to 78% and 89% performance improvement in terms of average turnaround time.

 

Tags: ,

SC14 Virtual Booth Tours

AMD SC14 video AMD Virtual Booth Tour @ SC14
Click to Play Video
Cray SC14 video Cray Virtual Booth Tour @ SC14
Click to Play Video
Datasite SC14 video DataSite and RedLine @ SC14
Click to Play Video
HP SC14 video HP Virtual Booth Tour @ SC14
Click to Play Video
IBM DCS3860 and Elastic Storage @ SC14 video IBM DCS3860 and Elastic Storage @ SC14
Click to Play Video
IBM Flash Storage
@ SC14 video IBM Flash Storage @ SC14  
Click to Play Video
IBM Platform @ SC14 video IBM Platform @ SC14
Click to Play Video
IBM Power Big Data SC14 video IBM Power Big Data @ SC14
Click to Play Video
Intel SC14 video Intel Virtual Booth Tour @ SC14
Click to Play Video
Lenovo SC14 video Lenovo Virtual Booth Tour @ SC14
Click to Play Video
Mellanox SC14 video Mellanox Virtual Booth Tour @ SC14
Click to Play Video
Panasas SC14 video Panasas Virtual Booth Tour @ SC14
Click to Play Video
Quanta SC14 video Quanta Virtual Booth Tour @ SC14
Click to Play Video
Seagate SC14 video Seagate Virtual Booth Tour @ SC14
Click to Play Video
Supermicro SC14 video Supermicro Virtual Booth Tour @ SC14
Click to Play Video