Frequently Asked Questions
- Where to go for further information?
- Acknowledgement of the Ocean Colour CCI dataset
- Reading the products
- Reduced quality of MODIS-Aqua data from 2014 onwards
- Depth of the satellite Chlorophyll-a concentration data
- Downloading and extracting data
- Problems reading the files in Linux ncdump and Matlab ncload
- Ocean Colour CCI chlorophyll product compared to CMEMS and GlobColour
- Sinusoidal versus geographic products
- Error characterisation
- Data processing in sunglint conditions
For a detailed description of the OC_CCI products please consult the Product User Guide (PUG) http://www.esa-oceancolour-cci.org/index.php?q=webfm_send/622
There's also been an article published in Ocean Challenge, see the PDF below.
If you encounter any difficulties and would like support then please email: email@example.com
When referencing the Ocean Colour CCI dataset within peer-reviewed papers or any other publications we suggest the following citation in the acknowledgements, alongside any description within the methodology:
Ocean Colour Climate Change Initiative dataset, Version Version Number, European Space Agency, available online at http://www.esa-oceancolour-cci.org/
We would also like to be notified so that we can list publications within the project website at http://www.esa-oceancolour-cci.org/?q=publications
The data can be directly read into BEAM (http://earth.esa.int/beam) or SeaDAS (http://seadas.gsfc.nasa.gov/) using the NetCDF-CF import option. Currently (April 2015) BEAM 5.0 will requires an update of the “Level-3 Binning Processor” modules before it is capable of reading the v2.0 sinusoidal product format.
You can also read the products into your favourite language, for example:
- Python using “netCDF4” that’s available from http://code.google.com/p/netcdf4-python/ or using “pip install netCDF4”
- R using “ncdf4”, which can be added to your R build using install.packages(‘ncdf4’) and added to your session using library(‘ncdf4’)
- IDL using the NetCDF library, see the PUG (http://www.esa-oceancolour-cci.org/index.php?q=webfm_send/622) for an example
The version 2 dataset extended the time series to the end of 2013 using just MODIS-Aqua from the end of MERIS onwards. However, the project decided to omit the 2014 MODIS-Aqua data because of concerns over its quality, and hence ability to consider it within a climate quality dataset. A presented figure (Bryan Franz, NASA) showed a sharp decline in the Chlorophyll-a values related to 2014 radiometry shifting (up in blue, down in green). The aim was to wait for NASA to undertake a MODIS reprocessing, which would correct for the decay.
For our version 3 dataset we've been able to extend the time series to the end of 2015 by bringing in VIIRS, with reprocessed R2014 MODIS-Aqua extended to the end of 2015. NASA have since issued reprocessed R2014.0.1 MODIS-Aqua data, and so a version 3.0.1 will be issued. However, the quality of the MODIS-Aqua 2014-15 data remains a concern because of the recurrent problems that NASA has been facing.
For satellite derived chlorophyll the depth it pertains to varies with the turbidity and hence is related to the concentration itself; the signal may return from depths as deep as 20+ m in the open ocean to as shallow as 1-2 m in highly productive waters. Also, the signal from the water closer to the surface will have a stronger influence than the signal from greater depths.
It is possible to use the diffuse attenuation coefficient to understand the penetration of light, which is often used when converting in-situ chlorophyll profiles into datasets suitable for satellite algorithm development and validation; e.g. see Sections 2.2.2 and 2.3.2 of http://oceancolor.gsfc.nasa.gov/MEETINGS/OCBAM/docs/werdell_and_bailey_2...
Access to several options, including FTP and the Composite Browser, is available via http://www.oceancolour.org/
The data can be directly downloaded from: ftp://oc-cci-data:ELaiWai8ae@oceancolour.org/
FTP server: oceancolour.org
Support is available from firstname.lastname@example.org
Some people have reported that the Mac FTP client has trouble with files bigger than 4GB. If you look at the monthly composites directory and the files there appear to be around 500-600MB rather than ~4.5GB, you're affected by this problem. If you attempt to download, you'll get ~500MB corrupt files. So you'll need to use an alternative FTP client or the terminal.
If you’re interested in downloading a subset of the dataset then THREDDS provides several options. As an example, the NetCDF subset service allows for a spatial and / or temporal subset to be extracted.
From https://rsg.pml.ac.uk/thredds/catalog-cci.html first choose the temporal composites required (daily, 8day or monthly) and then access Option 5, which is NetcdfSubset. Now choose the products, Latitude / Longitude subset and time subset options. Then press Submit at the bottom of the page.
If you receive an error message then it’s likely you’re trying to download too much data in one block – files cannot be larger than 4GB. The Stride parameter can be set to a value greater than 1 if you only want every nth value.
The output is a single NetCDF file and so this method will become unwieldy for large areas / time periods.If you’d prefer binary or ASCII data then choose Option 1, which is OPeNDAP.
To browse the data in an interactive viewer then the Web GIS Portal (http://www.oceancolour.org/portal/) is an option that can produce plots / graphics of the data as well as also exporting layers. First select the product(s) that you’re interested in, then select ‘Analysis’ that should be in the top right hand corner and choose what you’d like to do.
this could be because the OC Climate Change Initiative files are in a compressed version of NetCDF 4. Try reading the 3 examples given in the zip file below, which contains:
- NetCDF version 3
- NetCDF version 4
- NetCDF version 4 with compression (the closest to our dataset)
If you can read the NetCDF 3, but not NetCDF 4 files then the ideal option is to have your machine updated to use the latest NetCDF libraries. If this isn’t possible then an alternative option is to download and build nccopy that can then be used to convert the files from NetCDF 4 to NetCDF 3; they will significantly increase in size as the internal NetCDF compression will no longer be available.
The OC-CCI chlorophyll product (units of mg/m^3) is derived from merged MODIS-Aqua, SeaWiFS, VIIRS and MERIS data. The SeaWiFS and VIIRS data are processed from Level 1 to 2 within the project using the NASA standard processing (SeaDAS 7.0), and POLYMER is used for the MERIS atmospheric correction. In v2 MODIS was processed using SeaDAS while in V3.0 POLYMER is used. Then the MODIS, VIIRS and MERIS data are band shifted to the six main SeaWiFS bands and a bias correction applied before merging, after which the chlorophyll algorithm is applied. The OC-CCI products have used different chlorophyll algorithms for each version:
- v2.0: OC4v6; generated in 2015 with the current input dataset versions, VIIRS not included at this stage.
- v3.0: an Ocean Colour Algorithm Blending (OCAB) approach has been used to combine two chlorophyll-a algorithms to obtain optimal overall results: OC5 algorithm (Gohin et al., 2002 IJRS, 2005 RSE) and OC4v6-OCI algorithm (Hu et al., 2012 JGR) as it performed better in the clearest oceanic waters (optical water classes 1 to 10). See the OCAB ATBD for more details on the blending algorithm http://www.esa-oceancolour-cci.org/?q=webfm_send/587
The Copernicus Marine Environment Monitoring Service (CMEMS) products use the OC-CCI-v2.0 4km bias-corrected/merged remote-sensing reflectance (Rrs, surface ratio of upwelling radiance emerging from seawater to downwelling radiative flux in air) for the global products, with 1km data processed for the regional datasets. Then, either the OC4 or OC5CI chlorophyll algorithms are applied alongside Rrs being made available. See MyOcean Product User Manual http://marine.copernicus.eu/documents/PUM/CMEMS-OC-PUM-009-ALL.pdf
The GlobColour RAN (reanalysis) products are produced by ACRI-ST using the GlobColour processor that merges MODIS-Aqua, SeaWiFS and MERIS; the dataset was originally generated in 2008 and has since been reprocessed (2nd reprocessing). There are several chlorophyll products available:
- CHL1 merged products that include:
- GSM derived product based on merged Level 2 data, following the approach proposed by Maritorena and Siegel (2005, RSE);
- OC4Me algorithm (an inter-calibration is applied to the MODIS-Aqua and SeaWiFS Level 2 chlorophyll products to get MERIS algorithm compatible inputs) with a weighted averaging merging (AVW) method then applied to merge the adjusted MODIS and SeaWiFS products with MERIS data – there is also a simple average (AV) single sensor OC4Me product;
- CHL2 that’s the MERIS C2R Neural Network algorithm (only available for input MERIS data).
See GlobColour Product User Guide http://www.globcolour.info/CDR_Docs/GlobCOLOUR_PUG.pdf
The sinusoidal products are produced according to the NASA SeaWiFS binning algorithm (“Level-3 SeaWiFS Data Products: Spatial and Temporal Binning Algorithms" J.W. Campbell, J.M. Blasidell, and M. Darzi, NASA Technical Memorandum 104566, Vol.32.); unlike the original NASA format, all grid cells are retained and those with no valid data contain NetCDF fill values. These products are transformed into regular grid geographic products (8640 by 4320 pixels in size as a Plate Carrée projection), but this can result in Moiré effects i.e. pixels with missing data where the input and output spatial resolutions are similar.
The sinusoidal products should be used in preference, but it’s acknowledged that many software packages cannot deal with this format and need to resample the data to a geographic projection in order to display it.
The PUG provides detailed information on how to read the products and there is also the freely available ESA BEAM VISAT package http://www.brockmann-consult.de/cms/web/beam/
The user consultation carried out at the beginning of the Ocean Colour CCI project indicated an overwhelming preference for uncertainty characterisation based on comparison with in situ data. Each of the products, generated from a merged time series of ocean-colour data, has uncertainties (bias and Root Mean Square Difference) assigned to pixels based on the validation of each of the products against corresponding in situ observations. For further details please see the Uncertainty Characterisation Document (UCD) http://www.esa-oceancolour-cci.org/?q=webfm_send/321
In contrast, GlobColour products have pixel-by-pixel error bars based on a theoretical computation using a Look-Up-Table that uses the value of the variable and observation conditions or the output of the merging model. For further details consult the GlobColour Product User Guide at http://www.globcolour.info/CDR_Docs/GlobCOLOUR_PUG.pdf
Some OC_CCI observations don’t have a corresponding uncertainty.
This is because some pixels do not fit the currently pre-defined water classes, and thus OC-CCI-v1.0 and v2.0 don't currently have a computed uncertainty for them value; despite a product being produced. It’s expected this will be solved in the next data release.
What are the chlor_a_log10_rmsd and chlor_a_log10_bias values?
The Chlorophyll-a data were log transformed prior to the estimation of the uncertainty characteristics, using match-up in situ data, which means that the RMSD and bias are provided for log10(Chlorophyll).
Elliptical data gaps within the daily SeaWiFS data.
OC-CCI-v1.0 and v2.0 include elliptical data gaps (also in the NASA daily SeaWiFS Level-3 products) because a HIGLINT flag is applied as a mask at Level-3, which was adopted because the POLYMER atmospheric correction is not currently used for SeaWiFS. The HIGLINT flag started to be used as a mask during the NASA Ocean Color Reprocessing 2009 so that SeaWiFS and MODIS Level-3 data are consistent. Therefore, the original GlobColour (http://www.globcolour.info/) dataset often does not have these gaps because at the time it was produced this flag was not applied as a mask at Level-3; the GlobColour GLOB_4KM/RAN dataset was originally generated in 2008. It has since been reprocessed (2nd reprocessing) and so now also has these gaps.
Processing under sunglint conditions.
POLYMER is capable of retrieving data under sun glint conditions, which substantially increases the spatial coverage of the MERIS retrieved data. However, in OC-CCI-v1.0 and v2.0, POLYMER was not used for the MODIS or SeaWiFS atmospheric correction, and so data gaps due to glint do occur for these sensors. POLYMER is described in the following document http://www.esa-oceancolour-cci.org/?q=webfm_send/179