iSGTW - International Science Grid This Week
iSGTW - International Science Grid This Week

Home > iSGTW 5 August 2009 > Feature - GridFTP accelerates traffic on the World Wide Grid

Feature – GridFTP accelerates traffic on the World Wide Grid

Image courtesy of the Washington State Department of Transportation.

The standard system for transferring data online can’t handle the data-intensive needs of today’s larger scientific collaborations, resulting in delays and data loss. However, the GridFTP data transfer system is changing all that.

“There is a great need for bulk data movement in almost all areas of science,” said Raj Kettimuthu, technology coordinator of the GridFTP project at Argonne National Laboratory. “Collaborative science experiments have large volumes of data in their repositories, but the data must be distributed among researchers all across the world for analysis. With GridFTP, researchers can move the bulk data quickly and save a lot of time.”

GridFTP, which builds on the widely used File Transfer Protocol, transfers data 20 to 30 times faster than FTP, with speeds of up to 200 megabytes per second. It achieves the speedup by using parallel data movement, wherein large datasets can be broken into several smaller pieces that transfer simultaneously, speeding up the transfer even further. The system has the ability to utilize multiple computers at each end of the transfer, providing multiple levels of parallelism.

Another useful feature not available with FTP is that users who only need to transfer a small subset of a large file can select individual parts to send instead of the entire thing.

And any interrupted transfers can be restarted from that point instead of from the beginning.

Speed is not GridFTP’s only advantage; Kettimuthu and his team have also improved security and made the system more user-friendly. For example, GridFTP now supports security protocol Secure Shell (SSH) in addition to Grid Security Infrastructure (GSI). Obtaining certificates and setting up GSI proved difficult for the average user, and SSH is far simpler, said Kettimuthu.

Today, GridFTP has already been widely adopted in the scientific community. The number of transfers the program handles has doubled since last year, coming in at five million transfers a day and growing.

Developers are currently working on several upgrades to draw in even more users, including a user-friendly graphical interface. This new release of GridFTP, which is expected to come online later this year, will self-install the software so that users no longer have to figure out how to install it themselves. It will also make initiating transfers easier by allowing researchers to simply drag and drop files from a remote host to their local machine or between two remote hosts.

GridFTP developers are also working to create a hosted data transfer service so that in the future users won’t have to install, run, and monitor anything on their own systems. Users would access the hosted service through a web site and specify the files they need transferred, and the service would monitor the transfer send the user an email notification when it is complete.

“Now the focus is on making GridFTP really simple for the average user,” Kettimuthu said. “The hosted service would allow the transfer to run without the user having to babysit it, making things much easier.”


—Amelia Williamson, for iSGTW


 iSGTW 22 December 2010

Feature – Army of Women allies with CaBIG for online longitudinal studies

Special Announcement - iSGTW on Holiday

Video of the Week - Learn about LiDAR


NeHC launches social media

PRACE announces third Tier-0 machine

iRODS 2011 User Group Meeting

Jobs in distributed computing


Enter your email address to subscribe to iSGTW.


 iSGTW Blog Watch

Keep up with the grid’s blogosphere

 Mark your calendar

December 2010

13-18, AGU Fall Meeting

14-16, UCC 2010

17, ICETI 2011 and ICSIT 2011

24, Abstract Submission deadline, EGI User Forum


January 2011

11, HPCS 2011 Submission Deadline

11, SPCloud 2011

22, ALENEX11

30 Jan – 3 Feb, ESCC/Internet2


February 2011

1 - 4, GlobusWorld '11

2, Lift 11

15 - 16, Cloudscape III

More calendar items . . .


FooterINFSOMEuropean CommissionDepartment of EnergyNational¬†Science¬†Foundation RSSHeadlines | Site Map