Title: STXXL - Standard Template Library for XXL Data sets
Speaker: Roman Dementiev, University of Karlsruhe, Germany
Date: Tuesday, March 19, 2007 12:00 - 1:30 pm
Location: DyDan Center, CoRE Bldg, Room 431, Rutgers University, Busch Campus, Piscataway, NJ
Abstract:
We present the software library Stxxl (stxxl.sourceforge.net) that is an implementation of the C++ standard template library STL for processing huge data sets that can fit only on hard disks. It supports parallel disks, overlapping between disk I/O and computation and it is the first I/O-efficient algorithm library that supports the pipelining technique that can save more than half of the I/Os. Stxxl has been applied both in academical and industrial environments for a wide range of problems including text processing, graph algorithms, computational geometry, gaussian elimination, visualization, and analysis of microscopic images, differential cryptographic analysis, etc. The performance of Stxxl and its applications is evaluated on many synthetic and real-world inputs. We present the design of the library, how its performance features are supported, and demonstrate how the library integrates with STL. As part of "DyDan Universal Information Graphs project" lead by James Abello we have built a platform (ema.rutgers.edu) that includes Stxxl. A short term goal is to have a library of external memory graph algorithms that can be used to build and explore very large graphs of interest to DyDan's researchers.