Hierarchical Data Format 1.8.0 Alpha 5 review

Hierarchical Data Format is a general purpose library and file format for storing scientific data. HDF5 can store two primary obje

License: BSD License File size: 5479K Developer: NCSA

0

Hierarchical Data Format is a general purpose library and file format for storing scientific data.

HDF5 can store two primary objects: datasets and groups. A dataset is essentially a multidimensional array of data elements, and a group is a structure for organizing objects in an HDF5 file. Using these two basic objects, one can create and store almost any kind of scientific data structure, such as images, arrays of vectors, and structured and unstructured grids. You can also mix and match them in HDF5 files according to your needs.

Efficient storage and I/O.

HDF5 was created to address the data management needs of scientists and engineers working in high performance, data intensive computing environments. As a result, the HDF5 library and format emphasize storage and I/O efficiency. For instance, the HDF5 format can accommodate data in a variety of ways, such as compressed or chunked. And the library is tuned and adapted to read and write data efficiently on parallel computing systems.

Software.

NCSA maintains a suite of free, open source software, including the HDF5 I/O library and several utilities. The HDF5 user community also develops and contributes software, much of it freely available. Unlike HDF4, there is little commercial support for HDF5 at this time, but we are successfully working with vendors to change this.

Emphasis on standards.

Data can be stored in HDF5 in an endless variety of ways, so it is important for communities of users to standardize on how their data is to be organized in HDF5. This makes it possible to share data easily, and also to build and share tools for accessing and analyzing data stored in HDF5. The NCSA HDF team works with users to encourage them to organize HDF5 files in standard ways.

Large and varied user community.

HDF5 users range across a variety of engineering and scientific fields, and even some non-technical fields. Data stored in HDF5 is used for a wide range of applications, from computational fluid dynamics to film making.

Here are some key features of "Hierarchical Data Format":
Parallel HDF5 - Information on installing and using Parallel HDF5
SZIP Compression - Information about SZIP Compression in HDF5
Thread Safe HDF5 - Information on thread-safe capabilities of HDF5 and how to install
The High Level HDF5 APIs, previously distributed separately, are now distributed as part of the main HDF5 Library:
High Level HDF5 APIs - Information on installing and using the High Level HDF5 APIs

Applications:

HDF Java Products - HDF4/HDF5 Java interfaces and viewer, HDFView.
HDF Web-browser Plug-in - The HDF Web-browser plug-in is a windowed browser plug-in that is launched from a web browser to display HDF4 and HDF5 files.
netCDF-4 - The NCSA and NetCDF groups are collaborating on a version of NetCDF built on top of HDF5.
HDF5 XML Information Page - DTD and tools for using HDF5 with XML
HDF5 WRF I/O Module - I/O module that reads HDF5 datasets for the Weather Research and Forecasting Model
HDF5 Mesh API (prototype) - API for storing and retrieving structured and unstructured mesh data

What's New in This Release:
Improved support for 32-bit and 64-bit HPUX.
Link APIs for hard and soft links have been added to data objects within an HDF5 file.
Links may by local or span across HDF5 files.
Operations have been added for opening and closing objects of an unknown type, and opening objects by address within a file.
Additional minor API updates and minor bugfixes.