SCHEDULE: NOV 15-20, 2015

When viewing the Technical Program schedule, on the far righthand side is a column labeled "PLANNER." Use this planner to build your own schedule. Once you select an event and want to add it to your personal schedule, just click on the calendar icon of your choice (outlook calendar, ical calendar or google calendar) and that event will be stored there. As you select events in this manner, you will have your own schedule to guide you through the week.

Rapid Replication of Multi-Petabyte File Systems

SESSION: Regular & ACM Student Research Competition Poster Reception

EVENT TYPE: Posters, Receptions, ACM Student Research Competition

EVENT TAG(S): HPC Beginner Friendly, ACM Student Research Competition Poster

TIME: 5:15PM - 7:00PM

SESSION CHAIR(S): Michela Becchi, Manish Parashar, Dorian C. Arnold

AUTHOR(S):Justin G. Sybrandt

ROOM:Level 4 - Lobby

ABSTRACT:

As file systems grow larger, tools which were once industry standard become unsustainable at scale. Today, large data sets containing hundreds of millions of files often take longer to traverse than to copy. The time needed to replicate a file system has grown from hours to weeks, an unrealistic wait for a backup. Distsync is our new utility that can quickly update an out-of-date file system replica. By utilizing General Parallel File System (GPFS) policy scans, distsync finds changed files without navigating between directories. It can then parallelize work across multiple nodes, maximizing the performance of a GPFS. NERSC is currently using distsync to replicate file systems of over 100 million inodes and over 4 petabytes.

Chair/Author Details:

Michela Becchi, Manish Parashar, Dorian C. Arnold (Chair) - University of Missouri|Rutgers University|University of New Mexico|

Justin G. Sybrandt - National Energy Research Scientific Computing Center

  Click here to download .ics calendar file

  Click here to download .vcs calendar file

  Click here to add event to your Google Calendar