NOAA IOOS^® Data Integration Framework

 

The NOAA IOOS program initiated development of a Data Integration Framework (DIF) to improve management and delivery of an initial subset of ocean observations. The DIF will establish the technical infrastructure, standards, and protocols needed to improve delivery of atleast six of 20 IOOS core oceanographic variables defined in the U.S. IOOS Development Plan, as well as winds. Read More

 

Data Access

The following services are the first to be established by the NOAA IOOS program and its partners to provide access to data. These services are now undergoing beta-test and should still be considered experimental. The services will be modified and enhanced during the course of the DIF project.
• NDBC Sensor Observation Service (SOS) This server provides in-situ temperature, salinity, currents, water level, waves and winds data from National Data Buoy Center (NDBC) moorings, IOOS Regional Coastal Ocean Observing Systems, Deep-ocean Assessment and Reporting of Tsunamis (DART) buoys, and Tropical Atmosphere Ocean (TAO) buoys. The server is operated by the NOAA National Weather Service (NWS) NDBC. SOS is an Open Geospatial Consortium (OGC) standard.
• CO-OPS SOS This server provides in-situ temperature, conductivity, currents, water level, and waves data from the National Water Level Observing Network (NWLON) and the Physical Oceanographic Real-Time System (PORTS). The server is operated by the NOAA National Ocean Service (NOS) Center for Operational Oceanographic Products and Services (CO-OPS).
• NDBC THREDDS Data Server (TDS)  This server provides gridded surface currents derived from high-frequency radar (HFR) installations along the coasts. The server supports Web Coverage Service (WCS) and Open-source Project for a Network Data Access Protocol (OpenDAP) and is operated by NDBC. WSC is an OGC standard.
• SECOORA SOS This server provides a variety of in-situ parameters from a collection of data providers.  Server is operated by SECOORA (Southeast Coastal Ocean Observing Regional Association). Software documentation on creating this service from the supporting database is at http://code.google.com/p/xenia/wiki/XeniaSOS
• CoastWatch TDS (coming soon!) This server will provide access to chlorophyll concentrations derived from satellite ocean color observations. (Expected November 2008). This server is operated by NOAA National Environmental Satellite, Data, and Information Service (NESDIS) CoastWatch program.

 

Technical Documentation

• IOOS DIF Draft Architecture & Recommendations This set of slides includes a partial set of architecture diagrams and provides an overview of the data access services used by the DIF.
• IOOS DIF XML Schema beta v0.6.1 - This Geography Markup Language (GML) application schema defines the XML encoding used by the NDBC and CO-OPS SOS to serve in situ data. This encoding is designed to be complete and machine-readable, and may be transformed to simplified representations using code such as the Extensible Styleheet Language Transformation (XSLT) example below.
• DIF Design Document:(pdf, 1,039kb) This document defines the high-level DIF architecture, standards, conventions, and services to be employed, intersections with NOAA Enterprise Architecture, and risks and mitigation strategies.

 

Software

The following software is provided on an as-is basis with no support or warranty. Read the full disclaimer before downloading.



Server Code

• NDBC SOS software beta version 0.6.1(zip, 88kb)This PHP script and related files are used by the NDBC to provide a SOS interface to their existing MySQL database back-end. This beta software is still under active development and test.
• GCOOS SOS software beta version 0.6.1(zip, 88kb)This PHP script was modified from the NDBC version to support SQLite database engine. This beta software is still under active development and test.
• THREDDS Data Server (TDS) is open-source software from Unidata that provides OpenDAP and WCS functionality. It is used by the NDBC and CoastWatch.

Format Converters

• IOOS SOS to CSV v0.6.1This XSLT file converts the IOOS SOS v0.6.1 observation values from GML to comma-separated value (CSV) format suitable for use by Excel and other simple applications.
• netCDF to BUFR converter beta v0.1 This sample application was developed at AOML. It converts data from a netCDF format accepted by DIF to the Binary Universal Form for the Representation of meteorological data (BUFR) format maintained by the World Meteorological Organization (WMO). The archive contains C source code, descriptor file, and brief help on compilation and execution.

Other Tools

• Tcl/Tk Example These Tcl/Tk routines are used by the Coastal Inundation customer for retrieving water level observations from the CO-OPS SOS.

 

Core DIF Standards

These are some of the basic standards and specifications adopted by the NOAA IOOS Data Integration Framework.
• OGC Sensor Observation Service (SOS) specification NOAA IOOS uses this service type to provide access to in-situ oceanographic data in an XML encoding defined by the GML application schema referenced above.
• OGC Web Coverage Service (WCS) specification NOAA IOOS uses this service type to provide access to gridded remotely sensed data in binary formats such as NetCDF and GeoTIFF.
• OPeNDAP information This service type is used by NOAA IOOS to provide access to gridded remotely sensed data in binary formats such as NetCDF and GeoTIFF.
• OGC Web Map Service (WMS) specification This service type will be used to provide georeferenced images of data.

 

Project Documents

• IOOS REGIONAL DATA INTEGRATION FRAMEWORK IMPLEMENTATION WORKSHOP:(pdf, 13mb)The Integrated Ocean Observing System (IOOS) Regional Data Integration Framework (DIF) Implementation Workshop was organized with the goal of improving the overall production and execution of IOOS data management across the eleven IOOS Regions. To accomplish the goal of DIF implementation, a set of objectives were developed in the workshop planning process.
• The NOAA IOOS Data Integration Framework: Initial Implementation Report:Slides (pdf, 315kb) and associated paper (pdf, 125kb) presented at MTS/IEEE Oceans 2008 conference in Quebec, Canada 2008-09-17.
• Project Plan:(pdf, 168kb)The Project Plan records planning ideas and agreement details, approved scope, cost, and schedule baselines.
• Functional Requirements:(pdf, 333kb)This document defines the functional requirements of the DIF and is a guide to further system documentation and design.
• Concept of Operations:(pdf, 410kb)The Concept of Operations (ConOps) document is the translation of the DIF functional requirements into operational scenarios. The ConOps provides important input into the high-level system design.
• DIF Customer Implementation Project Summary and Performance Assessment Plan:(pdf, 457kb) This document describes the 4 customer implementation projects used to assess the value of DIF's integrated data. Provides overview of benchmarks, metrics, and performance assessment methodology.

 

Data Management and Communications Documents

• IOOS High Level Functional Requirements:(pdf, 168kb)As a part of the IOOS implementation planning effort, the NOAA IOOS Program collected functional requirements from a variety of existing sources and compiled them into this single document. This IOOS High-Level Functional Requirements Document will be used to assist developing more detailed documents as the program focuses its efforts in developing and deploying a National DMAC to serve all IOOS stakeholders.
• IOOS DMAC Concept of Operations:(pdf, 168kb)This document describes the initial high-level concept of operations (ConOps) for the DMAC subsystem; it is not intended to be an implementation or transition plan. The focus of the document is to define the functions and services that IOOS stakeholders desire the DMAC to perform. It does not address the technology or architecture of how it will perform those functions and services. NOAA is currently running a pilot project, the Data Integration Framework (DIF), to address some of the DMAC functionality and some of the technology issues.

 

(top)