LEAD real-time urgent weather runs using SPRUCE
This Spring 07, LEAD portal for weather forecasting and analysis, the SPRUCE Urgent Computing gateway and the UC/ANL TeraGrid resource providers have joined forces to perform real time, on-demand severe weather runs.
Additionally, the UC/ANL IA64 machine currently
supports preemption for the highest priority urgent jobs. As an
incentive to use the platform even though jobs may be killed, users
are charged at a rate of 90 percent of the standard CPU service unit
billing. Deciding which jobs are preempted is determined by an
internal scheduler algorithm that considers several aspects, such as
the elapsed time for the existing job, number of nodes and jobs per user.
The complete policy mapping on the IA64 UC/ANL resource is as
- yellow - elevated priority
- orange - next-to-run status
- red - preemptive access
This collaboration clearly demonstrates the capability SPRUCE can provide to existing workflows. LEAD was given a limited number of tokens for use throughout the tornado season. Urgent runs are triggered automatically by parsing the RSS feed of advisories published by National Weather Service for possible warning flags. A right-of-way token at the required urgency level is activated based on the warning, and a list of users is added onto the token automatically via the Web-services interface to the SPRUCE server. Alternatively, community users can activate tokens given to them from the LEAD portal directly, without logging into the SPRUCE portal as seen in the above figure.
For more technical details of how this was done, or if you want to collaborate with SPRUCE for your resource or application, please Contact Us.
Currently, SPRUCE is deployed at the University of Chicago/Argonne National Laboratory (UC/ANL), Purdue University, Texas Advanced Computing Center (TACC), San Diego Supercomputer Center (SDSC), and the National Center for Supercomputing Applications (NCSA) and is being deployed at Indiana University. Virginia Tech University is one of our early non-TeraGrid adopters planning to use this system for simulating epidemiological spread patterns as a part of EPISIMS. Louisiana State University is also considering installing SPRUCE on it's local cluster for the SCOOP project.