Data Management Plan
Purposes and Scope of the JCOMM Data Management Plan
- The Plan can be downloaded from HERE
JCOMM was established by the World Meteorological Organization (WMO) and UNESCO’s Intergovernmental Oceanographic Commission (IOC) in 1999 to be their major advisory body on all technical aspects of operational marine meteorology and oceanography. Prior to this, marine meteorological and oceanographic observations, data management and service provision programmes were internationally coordinated by two separate bodies: IOC working jointly with WMO through the Committee for the Integrated Global Ocean Services System (IGOSS), and WMO through its Commission for Marine Meteorology (CMM). The IGOSS dealt with real-time oceanographic data (defined as data collected within the last 30 days) and managed physical oceanographic variables only – hence the perception on the oceanographic side that JCOMM deals in real-time data only. On the meteorological side, CMM’s work covered the complete time frame from real-time to delayed-mode (data not distributed in real-time and usually of higher resolution and quality) and built and maintained archives of marine data. This current JCOMM Data Management Plan (DMPlan) must address issues relevant to both real-time data handling as well as managing delayed-mode data in archives.
The present structure of JCOMM (in 2009) has three Programme Areas (PAs), one for Observations (OPA), one for Services and Forecasting Systems (SFSPA) and one for Data Management (DMPA). The groups in the OPA and SFSPA are focused on activities supporting a type of data (or sometimes an observing technique), or on a service such as safety or emergency support. Each of these has its own history of managing data and information prior to its incorporation into JCOMM. When JCOMM was formed, a decision had to be made about how to organize the cross-cutting activities of data management. The choice was made to place data management in a separate PA to recognize that managing the data and information of JCOMM is an important activity equal to acquiring and delivering data and services. The potential weakness is that the activities of the DMPA may not be strongly linked to the day-to-day data management activities in the various groups of the other PAs. The challenge of the DMPA is to work within the requirements of the activities within the OPA and SFSPA and still achieve the broad goals of JCOMM. This plan will adopt an approach that looks for commonalities across all of these systems and exploits these to improve interoperability.
A main goal of this DMPlan, therefore, must be to explain how data management can be conducted under the present structure to promote the long-term objectives of JCOMM.
JCOMM deals in a variety of data within the broad domains of oceanography and marine meteorology. Both meteorology and physical oceanography have a strong history of data exchange, and it is these types of data that are normally considered part of JCOMM activities. Biological or chemical variables have a history of data exchange within oceanography but only in delayed-mode, and only for a limited number of variables. Only recently have these kinds of data been exchanged in real-time, such as in the International Ocean Carbon Coordination Project.
The coastal module of GOOS has defined common variables to be exchanged and more than half of these are outside of the physical oceanographic domain. In this context, JCOMM must position itself to handle this broader range of variables.
New observing technologies are being developed at a fast pace. In addition to the improvements in measuring traditional physical oceanographic variables, such as temperature and salinity, there are sensors being built that can provide immediate and reliable measurements of chemical and biological components in the ocean. These data can be transmitted instantly through satellite systems. New offshore cabled networks allow for the streaming of data of all kinds, from television images, acoustics or more conventional oceanographic and meteorological observations with 2-way communication to the sensors. Open ocean moorings with both meteorological and oceanographic measurements available immediately are being deployed. All of these data will be challenging for JCOMM to both coordinate and manage.
Computer modeling of the atmosphere has been an important activity for many years.
In the last few years, modeling of the ocean has increased substantially. Now, ocean and atmospheric models are starting to be fully coupled. Ice modeling is being injected into these coupled models and there are some developments to include biological and chemical components in the ocean as well.
Model results are valuable for forecasting and in hindcast studies as well. Models permit us to identify data gaps, propose strategies to fill in those gaps, and to predict conditions where data are sparse. When these results are reformulated as products, they can become particularly useful in decision making, disaster mitigation and a host of other uses.
Models can be operated in research, operational or reanalysis-mode. Models run for research purposes are constantly being checked, results verified against observations, model characteristics altered and so on until such time as the newer version is determined to be an improvement on an existing operational model. Results from research model runs are of use to the research community primarily.
In contrast, the characteristics of operational, and to an even larger extent, reanalysis models are held stable. Results from these models are the basis for many products because they use the physics in the models to fill in information in oceanic and atmospheric areas where observations are not available. These results are valuable to save and can be used in a similar way as are historical observations.
Satellite observations are also of interest to JCOMM. Satellites provide the synoptic and broad scale views that are unattainable from in situ observing systems. They are a complement to the in situ systems in that they provide surface conditions on broad spatial scales at an instant of time. There is already a well developed international system for managing satellite data (see http://www.ceos.org/, the home page of the Committee on Earth Observation Satellites, CEOS). The DMPlan needs to take into consideration the level of interoperability that is required with CEOS and how this can be attained.
Metadata must also be considered as an important component of JCOMM. Metadata is a term used to cover a wide range of information. It may be information that describes the contents of archives (such as what data they contain, over what time and space scales) down to detailed information about characteristics of the instrumentation, placement of sensors, or characteristics of the models. No common terminology has developed to talk about these different kinds of metadata, so this document will provide examples of the kind of information considered whenever this term is used.
Metadata are important for a number of uses. Just as the scope of metadata is wide, so its uses are broad. For example, information about the contents of archives is used in cataloguing systems so that potential users can locate data of interest. Information about instrument characteristics or sampling schemes is important in comparing measurements from different instruments to ensure that systematic differences are taken into consideration.
The DMPA is not alone in addressing issues of managing oceanographic and meteorological data in the international arena. On the oceanographic side, the Intergovernmental Data and Information Exchange (IODE, http://www.iode.org/) committee of IOC has operated for many years managing many different kinds of data including types common with JCOMM.
The difference has been that IODE has mostly concentrated on data that arrive in the data system with significant time delays, some that may be up to years while the management of the real-time data was left to IGOSS. The IODE is a close partner in managing the oceanographic data and is a co-sponsor of many of the data management activities of relevance to JCOMM.
A more recent initiative of WMO, thus far advanced largely through its Commission for Basic Systems (CBS), is its WMO Information System, WIS (see web link for further details: http://www.wmo.ch/pages/prog/www/WIS-Web/home.html). This is an overarching approach and a single coordinated global infrastructure for the collection, distribution, retrieval of, and access to data and information of all WMO and related programmes. JCOMM, as a technical commission being co-sponsored by WMO, is a contributor to the WMO Information System (WIS).
Both oceanographic and meteorological data contribute to the holdings within the World Data Center (WDC) system (see http://www.ngdc.noaa.gov/wdc/wdcmain.html ). It is expected that the WDC system will ultimately archive all of the data collected and managed by JCOMM. Therefore, the JCOMM data management activities need to provide data and information to the WDCs and work with them to build a complete, global data system.
The creation of the Global Climate Observing System (GCOS) and all of its components views JCOMM as an important contributor. The GCOS Implementation Plan (GCOS-92) (see http://www.wmo.int/pages/prog/gcos/Publications/gcos-92_GIP.pdf) cites 21 specific actions requiring the support of JCOMM as an ‘Agent for Implementation’. Of these actions, some relate directly to the data systems. The DMPlan must provide the direction that will ensure these expectations are met.
Details on how JCOMM should link into these various programmes are provided later in this document. The initial sections of this plan discuss what activities JCOMM should undertake to ensure that the data collected under its Programme Areas are well coordinated and managed.
Finally, it is obvious that the data management component of JCOMM is broad and has to make many connections both within and outside of the two parent organizations of WMO and IOC. This DMPlan provides the broad outlines and recommendations by which the DMPA will help attain the vision of JCOMM. This plan will not provide the details of how the recommendations will be met; rather, this is the subject of an implementation plan that must be built from the agreed plan. As technology advances, and as the implementation develops, there will be changes in emphasis or new capabilities not anticipated by this DMPlan.
The Data Management Plan was approved by the Fourth Session of JCOMM (Yeos, Rep. of Korea, May 2012).