urgent computing topline

 

SPRUCE Advisor prototype


This document briefly describes the SPRUCE Advisor.

Table of Contents

Overview

The Advisor is a prototype tool to calculate the probability that an urgent computation will meet its deadline. A tpyical urgent computing workflow consist of four phases: input file staging, resource allocation (e.g., batch queue), execution delay, and output file staging delay. Together, these four phases make up the total turnaround time. The inspiration of this work was the Queue Bounds Estimation from Time Series (QBETS) tool which generates a delay based upon a specified quantile for batch queue delay. For example, if you were predicted the upper 0.95 quantile to be 2 hours, then you could say that there is a 95% chance that your job will wait less than 2 hours in the queue. These probabilistic upper bounds are intriguing because they are more useful than a single-point value or summary statistics. The advisor generates these probabilistic bounds for each of the four phases using different methods for each phase. Then, it creates a compsoite bound by combining the individual phase bounds. For each phase, it queries a small subset of predictions (e.g., 5). It then iterates through all possible composite bounds and finds the highest composite probability with a bound less than the user-specified deadline. The user then can submit their job with greater confidence that their deadline will be met. For more information, please Nick Trebon's master thesis.

[TOC]

Code

The code for the Advisor may be found on the software page. This archive includes the Advisor code (which is wrapped up in an Axis2 web service), the client code (web service) and a website front-end. It is important to note that none of these will work out of the box because they depend on database access or inactive third party services. The code is merely provided for instructional purposes. AdvisorSite contains the website front-end. It is a php page that makes use of Pear/SOAP to invoke the web-service. It also assumes access to a database on localhost in order to help generate the input parameters. The username and passwords have been x'd out. The AdvisorClientCode is a simple Axis2/Java client. I believe it was built using Axis2-1.0. The AdvisorCode contains the all of the code used for the Advisor, wrapped up in an Axis2-1.0 web service. Again, this code will *not* work out of the box. It relies on, among other things, a working NWS installation that is monitoring the bandwidth between targeted endhosts, access to a database on localhost, the authorization to query the TeraGrid MDS service and the QBETS predictor hosted at UCSB. The code comes in both a threaded and non-threaded mode.

[TOC]