iSGTW - International Science Grid This Week
iSGTW - International Science Grid This Week
Null

Home > iSGTW - 21 January 2009 > iSGTW Opinion - Perspectives on advancements in distributed computing

Opinion - Perspectives on advancements in distributed computing


At an informal SC08 discussion titled "Traditional & Distributed HPC–What has changed? What remains the same?" participants — both users and developers — shared experiences and insights on the advancement of grid-based High Performance Computing. Led by Gabrielle Allen and Daniel S. Katz of the LSU Center for Computation & Technology, and Gary Crane, SURA Director of IT Initiatives (including SURAgrid), the group identified several areas in which it believes grid technology needs to advance in order to deliver cost-effective service.

Management of distributed data

Many research domains require data from diverse sources, and output from one application is frequently used as input to another. Data definitions that cross application boundaries are rare, making sharing between applications difficult or impossible. Participants noted the progress in relevant standards development by the Common Component Architecture group. They also cited caBIG as an exemplary data-driven infrastructure and recognized optical networks as positive underpinning for effective management and use of distributed data.

Standards for abstraction of middleware layers

Abstraction refers to making the workings of these layers transparent to users so that the grid is easy to use. Standards are needed to support both automatic discovery of resources for job execution and intelligent data transport. Along with easier — even automatic — job submission, these standards are critical for wider adoption. The group believes that the grid needs coordinated direction for development of sophisticated, distributed file systems and schedulers to manage the dynamism of multiple jobs, users, resources and administrative domains. Virtual Machine technology can provide some of this through its dynamic, load-based resource sharing.

The SURA Coastal Ocean Observing and Prediction (SCOOP ) Program is integrating distributed data and computation to improve storm surge forecasting and model visualizations on www.OpenIOOS.org

Image courtesy of Joanne Bintz, SURA

Measuring cost-effectiveness

Researchers weigh the costs of using distributed versus local resources. They'll tailor their application to use remote resources when the lower wait and execution times compensate for the extra work. However, the decreasing cost of HPC hardware is making local resources easier to acquire (more local resources mean less wait time), and MPI jobs (jobs that need to communicate with each other during execution) still work best locally.

In traditional (non-distributed) HPC, it's easy to measure cost-effectiveness using system performance metrics. Similarly useful metrics for distributed computing don’t exist yet. Funding is needed to develop these performance metrics.  

Industry still hasn't found the economic driver for distributed computing, and is not investing in it. This may be changing, given recent developments from Google, Amazon, IBM and Microsoft. However, other funding is needed until there is more widespread support, particularly for scientific computing.

M.F. Yafchak, SURA

Tags:



Null
 iSGTW 22 December 2010

Feature – Army of Women allies with CaBIG for online longitudinal studies

Special Announcement - iSGTW on Holiday

Video of the Week - Learn about LiDAR

 Announcements

NeHC launches social media

PRACE announces third Tier-0 machine

iRODS 2011 User Group Meeting

Jobs in distributed computing

 Subscribe

Enter your email address to subscribe to iSGTW.

Unsubscribe

 iSGTW Blog Watch

Keep up with the grid’s blogosphere

 Mark your calendar

December 2010

13-18, AGU Fall Meeting

14-16, UCC 2010

17, ICETI 2011 and ICSIT 2011

24, Abstract Submission deadline, EGI User Forum

 

January 2011

11, HPCS 2011 Submission Deadline

11, SPCloud 2011

22, ALENEX11

30 Jan – 3 Feb, ESCC/Internet2

 

February 2011

1 - 4, GlobusWorld '11

2, Lift 11

15 - 16, Cloudscape III


More calendar items . . .

 

FooterINFSOMEuropean CommissionDepartment of EnergyNational¬†Science¬†Foundation RSSHeadlines | Site Map