Objectives

The ultimate goal would be to establish a QC/ QA workflow including the tools to firstly check plausibility of the model output and proper data format (QC) and afterwards, with ensured plausible data and a similar file structure, to assess key variables with benchmark data (QA).

This should serve both, model development, and to avoid publishing incorrect data at the ISIMIP repository. Whereas the QC tool is already existing and in use for some sectors, the QA tool is not existing. Potentially, one sector could act as pilot sector for developing such an workflow, including a QA tool.

In particular, the goals of this TG are:

Quality control / quality check (QC) tool: further developing the automatic quality control / plausibility check (e.g. 120 m limit for tree height) of model output, in particular

• for all sectors
• for all variables
• robust

Quality assessment (QA) tool: develop a quality assessment (evaluation tool) of model output with benchmark data (QA).

• assess existing evaluation frameworks (e.g. iLAMB, ESMValTool),
• identification of sector-specific key variables, benchmark data, evaluation metrics
  if possible, adapt existing tools to ingest ISIMIP data, otherwise develop workflow and tool itself. Among this, clarify, where benchmark data are stored, where scripts are executed etc.

Task Group Leader

Hannes Müller Schmied

Output

Müller Schmied, H. & Schiebener, L.: Assessing the suitability of streamflow station observations for consistent evaluation of simulated river discharge data of the ISIMIP global water sector (Report)

Nyenah, E.: Using ISIMIP model output with ILAMB and ESMVal Tool (Presentation)

Nyenah, E. et al.: Using ISIMIP model output with ILAMB and ESMVal Tool (Report)

Nyenah, E.: Reviewing model evaluation tools and their potential use for assessing climate impact model outputs (Virtual Mobility Report)

Koutroulis, A. & Müller Schmied, H.: QA tools of impact model output. An assessment of existing evaluation frameworks. (Review)

Code for impact model quality control and evaluation (ISIMIP Data Quality Assessment)

Code for impact model quality control and evaluation (ISIMIP Data Quality Control)