Task Group 1.2: Automatic QC / QA of impact model output

Objectives

The ultimate goal would be to establish a QC/ QA workflow including the tools to firstly check plausibility of the model output and proper data format (QC) and afterwards, with ensured plausible data and a similar file structure, to assess key variables with benchmark data (QA).

This should serve both, model development, and to avoid publishing incorrect data at the ISIMIP repository. Whereas the QC tool is already existing and in use for some sectors, the QA tool is not existing. Potentially, one sector could act as pilot sector for developing such an workflow, including a QA tool.

In particular, the goals of this TG are:

Quality control / quality check (QC) tool: further developing the automatic quality control / plausibility check (e.g. 120 m limit for tree height) of model output, in particular

  • for all sectors
  • for all variables
  • robust

Quality assessment (QA) tool: develop a quality assessment (evaluation tool) of model output with benchmark data (QA).

  • assess existing evaluation frameworks (e.g. iLAMB, ESMValTool),
  • identification of sector-specific key variables, benchmark data, evaluation metrics
    if possible, adapt existing tools to ingest ISIMIP data, otherwise develop workflow and tool itself. Among this, clarify, where benchmark data are stored, where scripts are executed etc.


Task Group Leader

Hannes Müller Schmied

Task Group Contributors

Matthias Büchner, Jochen Klar, Iliusi Vega del Valle, Christopher Reyer, Aristeidis Koutroulis, Simon Gosling, ISIMIP sectoral coordinators in general