iSGTW - International Science Grid This Week
iSGTW - International Science Grid This Week

Home > iSGTW 19 September 2007 > iSGTW Feature - Pegasus invites new communities to saddle up


Feature - Pegasus invites new communities to saddle up

This mosaic image of the Orion Nebula was created by Montage using Pegasus and DAGMan tools. Workflow softare Pegasus chains dependent tasks together, so that multiple tasks can be automatically completed in the correct order.  Image courtesy of Pegasus

Grid workflow system Pegasus recently earned new wings with a share in a US$1.7 million grant from the U.S. Office of CyberInfrastructure, awarded over three years to make further improvements to Pegasus’ usability and breadth of support.

Pegasus’ principal investigator, Ewa Deelman from the University of Southern California’s Information Sciences Institute, says the grant has come at the perfect time.

“We’re now in a position to work with more communities and more users, including those using Open Science Grid and TeraGrid, to bring Pegasus technologies to their computing work.”

Pegasus allows researchers to translate complex computational tasks into workflows that link and manage ensembles of dependent tasks and related data files. 

“Pegasus automatically chains dependent tasks together, so that a single scientist can complete complex computations that once required many different people,” explains Deelman. “It compiles not only software and data, but also the expertise of the people who developed the software.”

Deelman says the OCI grant will allow her team to introduce Pegasus with a host of new capacities.

“We’re planning to improve ease of deployment, introduce new debug capabilities, support richer workflow representations and enhance monitoring. We really want to work with new communities; we’re keen to get them started and help them benefit from this technology.”

Comparison of actual and simulated earthquake waves used by the Southern California Earthquake Center CyberShake workflows
Image courtesy of Robert Graves, SCEC

Efficient, hands-free workflow management

Pegasus takes a modular approach to workflow systems, working in partnership with workflow execution engine DAGMan, which was developed by the Condor team at the University of Wisconsin and shares in this month’s OCI grant.

The Pegasus/DAGman combination is already used in a variety of scientific applications, including gravitational-wave physics project LIGO, astronomy software Montage and applications at the Southern California Earthquake Center, and can scale to manage large numbers of tasks.

“Some workflows consume terabytes of data and can also produce terabytes of data,” says Deelman. “Some of the SCEC workflows have in the order of 100,000 tasks.”

Another bonus is that Pegasus cleans as it works.

“It minimizes its workflow storage requirements by cleaning up the data produced by the workflow as the computation progresses; it removes data that has already been processed to keep the workflow footprint as small as possible, which helps improve efficiency.”

“Pegasus can also cluster tasks in the workflow,” she adds. “This reduces waiting times, which can add up when you need to manage lots of smaller tasks. This can significantly improve efficiency when you’re dealing with workflows with hundreds of thousands of nodes.”

Deelman says she hopes to continue to leverage collaborations with science communities to advance the state of the art in workflow technology. 

Pegasus and DAGMan were first developed under the National Science Foundation GriPhyN project.

- Cristy Burne, iSGTW


 iSGTW 22 December 2010

Feature – Army of Women allies with CaBIG for online longitudinal studies

Special Announcement - iSGTW on Holiday

Video of the Week - Learn about LiDAR


NeHC launches social media

PRACE announces third Tier-0 machine

iRODS 2011 User Group Meeting

Jobs in distributed computing


Enter your email address to subscribe to iSGTW.


 iSGTW Blog Watch

Keep up with the grid’s blogosphere

 Mark your calendar

December 2010

13-18, AGU Fall Meeting

14-16, UCC 2010

17, ICETI 2011 and ICSIT 2011

24, Abstract Submission deadline, EGI User Forum


January 2011

11, HPCS 2011 Submission Deadline

11, SPCloud 2011

22, ALENEX11

30 Jan – 3 Feb, ESCC/Internet2


February 2011

1 - 4, GlobusWorld '11

2, Lift 11

15 - 16, Cloudscape III

More calendar items . . .


FooterINFSOMEuropean CommissionDepartment of EnergyNationalĀ ScienceĀ Foundation RSSHeadlines | Site Map