How weather data generator MarkSim navigates worldwide data dilemmas

The Crop Modelling Community of Practice provided a mini-grant to Peter Jones of Waen Associates in order to solve MarkSim data dilemmas. Read about the journey to the solution: MetGrid files.

Crop modelling has the potential to significantly contribute to improved global food and nutrition security because it allows for foresight analysis of agricultural systems. As the global climate changes, it is pertinent that crop researchers leverage crop modelling’s ability to consider global change scenarios and predict the potential consequences of food system shocks and climate variability.

Crop simulations rely on daily weather data to produce high quality, accurate simulations, but coordinating global, daily weather data is no small feat. Daily weather data including rainfall, temperatures, and wind run have been recorded in some regions of the world for centuries, but recent technological advances have rapidly increased the scope and speed of weather data collection. Today, digital, global climate data are integrated with geographic information system (GIS) applications and environmental indices can be calculated from monthly climate data (Hijmans et. al, 2005).

However, the amount of weather data required to run crop simulation models is too voluminous to store in high-resolution GIS global coverage. Moreover, the availability and quality of historical weather data, particularly in developing countries, is often limited. Furthermore, in crop modelling analysis, the datasets need to include realistic variation, both short term and annual.

To solve the data storage and availability problem, many researchers turn to weather generators, as they require significantly less data storage space, while providing weather data where the availability and quality of historical weather data is limited.

MarkSim is one such weather generator tool that provides crop modellers with simulated daily weather data: rainfall, maximum and minimum temperatures, and solar radiation. It provides files of daily weather data that are specifically designed for crop modelling systems, thus helping agronomists assessing climate risks. The MarkSim tool was developed by Waen Associates in collaboration with the International Centre for Tropical Agriculture (CIAT) and the International Livestock Research Institute (ILRI) in the late 1990s. It builds on decades of work undertaken at CIAT (Jones & Thornton, 1993, 1997, 1999, 2000, Jones et. al., 2002), including a calibration dataset of about 10,000 stations worldwide with an average of 15-20 years of historical daily data.

Recently, a capability to simulate future weather sequences from a suite of 17 Global Circulation Models (GCMs) has been added to the application (Jones & Thornton, 2013). MarkSim GCM can be used, with confidence, to estimate uncertainties in rainfall scenarios as a stochastic downscaling method. Using the convenient MarkSim GCM map interface, users can click on any point in the world map and get a series of simulated daily data for any year from 2012 to 2100. Moreover, one to 99 replicates of that year can be generated which alleviates the problem of taking a running mean of 20 or 30 years of real, or GCM, data to approximate the weather at the target year. Additionally, the program also now allows users to download the most probable soils profiles for a specific site. MarkSim GCM outputs are specifically tailored to directly feed into the Decision Support System for Agrotechnology Transfer (DSSAT) system.

A second version of the program, MarkSim Standalone, was designed to run on Windows computers for users that need to process a large amount of data. MarkSim Standalone allows users to map climate and weather responses over geographic areas. Users can download the EXE files and run as many examples as needed, in replicates, on a grid, or both. The program was designed to be used with DSSAT and so it uses DSSAT CLI files as the basis of the input. These CLI files are available from the CCAFS webpage as panes of files compressed with WinRAR. The panes are always 120 pixels on a side irrespective of resolution, so they are always a maximum of 20 Mb for ease of downloading over slow lines. They may be much smaller since only land areas are present. They are basic CLI files and contain none of the weatherman coefficients for internal DSSAT weather generators, but they can be read easily and viewed in an ASCII editor. This is designed so that that MarkSim Standalone can be used seamlessly with these and other DSSAT applications.

Data dilemmas lead to new MarkSim structures: MetGrid files

In its first years, MarkSim relied on the CIAT climate database, which became the basis for WorldClim 1.0. Now, the new version WorldClim 2.0 released contains another generation of baseline data with ten more years of research, which are provided in a raster GIS format of monthly mean data with one coverage per month for each variable. However, these data sources pose a problem for MarkSim, which requires all of the data for a specific pixel in one record.

In order to solve this issue, increase the functionality of WorldClim 2.0, and update the MarkSim GCM software (both the web-based and the standalone versions) to use Worldclim2.0, the Crop Modelling Community of Practice provided a mini-grant to Peter Jones, from Waen Associates.

To increase WorldClim 2.0 functionality, Jones’s work consisted in developing direct access or grid files that include variables by location, as well as providing the basis for deriving a key variable not currently included, number of rain days per month.

MetGrid files are compressed climate data files that have a dedicated MarkSim direct access format. They have a complex format and they cannot be read without the correct routines. Therefore, a Fortran module, MetGrid_Handler, was developed and is available from the CCAFS webpage, which allows seamless access to the MetGrid files at four resolutions: 10 arc minutes, 5 arc minutes, 2.5 arc minutes and 30 arc seconds. The module allows access to the compressed data and to various transformations that the user might need. MetGrid files are geographically indexed so a single record can be referenced by its latitude and longitude directly in a single read. This is the first time that MetGrid files have been made public, and they now allow for use of the weather generator MarkSim with the last WorldClim update.

 

Illustration of the different data structures of MetGrid and WorldClim.

References

Fick S.E. and Hijmans R.J. 2017 WorldClim 2: new 1‐km spatial resolution climate surfaces for global land areas. International Journal of Climatology, 2017 – Wiley Online Library.

Hijmans, R.J., Cameron, S., Parra, J., Jones, P. G. and Jarvis, A. (2005) WorldClim, a very high resolution global terrestrial climate surfaces for monthly temperature and precipitation.  International Journal of Climatology 25: 1965–1978.

Jones, P. G. and Thornton, P. K.  1993.  A rainfall generator for agricultural applications in the tropics.  Agric. For. Meteor. 63: 1-19.

Jones, P. G. and Thornton, P. K.  1997.  Spatial and temporal variability of rainfall related to a third order Markov model.  Agric. Forest Meteor.  86:127-138.

Jones, P. G. and Thornton, P. K.  1999.  Fitting a third order Markov rainfall model to interpolated climatic surfaces.  Agric. Forest Met. 97: 213-231.

Jones P. G. and Thornton P. K. 2000. MarkSim: Software to generate daily weather data for Latin America and Africa. Agron. J 93:445-453.

Jones, P. G., Thornton, P. K., Diaz, W. and Wilkens, P. W. (2002) MarkSim, Version 1. A computer tool that generates simulated weather data for crop modelling and risk assessment. CIAT CD-ROM series, Cali, Colombia. Centro Internacional de Agricultura tropical. CD-ROM + Guide (87p.).

Jones P. G. and Thornton P. K. 2013. Generating downscaled weather data from a suite of climate models for agricultural modelling applications. Agricultural Systems 114 (2013) 1–5, Elsevier.

Jones P. G & Thornton P. K. (2015) Representative soil profiles for the Harmonized World Soil Database at different spatial resolutions for agricultural modelling applications. Agricultural Systems Volume 139 :  93-99.

Köppen, W., (1884) Die wärmezonen der erde, nach der dauer der heissen, gemässigten und kalten zeit undnach der wirkung der wärme auf die organische welt betrachtet. – Meteorol. Z. 1: 215–226.

January 10, 2020

P. G. Jones


Waen Associates
Dolgellau, Wales

  •  
  •  
  •  
  •  
  •  

Latest news

  •  
  •  
  •  
  •  
  •