Source and FEWS (Together At Last?)

Delft-FEWS is a great system for bringing together different simulation models in an operational context. FEWS has been widely applied in Europe, Asia and North America and, increasingly, in Australia. eWater Source is a flexible simulation model for water resource systems, with the ability to model a wide range of runoff, supply, allocation and demand situations. Source has a significant installed user base in Australia and a growing user group in Asia and elsewhere.

I was first exposed to FEWS in late-2005 when I was working on the, then nascent, Source project. I was struck then by the apparent complementary nature of the two systems and the idea of a Source model running in an operational context inside FEWS was very appealing. In a case of better late than never, I’ve had several recent opportunities to integrate Source and FEWS.

Motivation

I see two main motivations to link Source and FEWS into a system with the simulation capabilities of Source and the operational capabilities of FEWS. The first, and perhaps most obvious reason is for organisations that are building an operational forecasting system and want to leverage an existing investment in Source models (or, perhaps, commence a Source modelling project to populate the operational system). A second reason is for users undertaking large Source modelling projects, with a need to automate tasks such as data preprocessing, or reporting.

FEWS is designed to run an operational forecasting system, but it has no built in forecasting models. Rather, FEWS is a shell that can have other models plugged in, using custom adapters. This is a great strength, allowing new FEWS installations to take advantage of existing modelling investments in an organisation. FEWS has a great many appealing features for operational systems, including client-server operation, sophisticated data imports, data transformations and gap-filling and standardised reporting and archiving: all wrapped up in a highly configurable workflow-based system for automation.

Source, on the other hand, has a substantial suite of modelling capabilities, particularly for runoff generation in unregulated catchments and for modelling the management and transport processes through regulated systems.¹ Source does have some operational capabilities itself, such as results visualisation and the ability to dynamically reload updated data sources, but the power and flexibility of FEWS includes much that is not readily available in Source.

The Source-FEWS Adapter

As part of several recent FEWS projects, I’ve built a FEWS adapter for eWater Source. Using the adapter, Source models can be incorporated into operational FEWS platforms, with FEWS providing updated data feeds and Source generating forecasts that can then be imported into FEWS for visualisation, reporting or for use as inputs to subsequent models, such as receiving waters models.

The adapter is bundled as a ‘Community Plugin’ with the eWater Source release from the September 2013 release of Source onwards. The adapter is also available in source code form from Github, where you can also find documentation on configuring the adapter in FEWS. The SourceFEWSAdapter is free to use and to modify as per the terms of the LGPLv3 open-source license. (However, I would very much like to hear from you if you find it useful!)

Applying the Adapter

Source models can exhibit a wide range of characteristics, in terms of node-link network structure, input time series, parameters and patterns of usage. In particular, Catchment style models tend to be constructed in a very different fashion to River Management style models. While these models are treated consistently by the Adapter, FEWS can be configured to utilise these models in quite different ways. Typically, an unregulated catchment runoff model will be used as a forecast model, where updated input time series are generated from a range of recent climate inputs². The catchment model then runs with the new inputs and returns a streamflow forecast to FEWS. A model of a highly regulated river could potentially run in much the same way, but it is also possible to configure the model for interactive use, with operators making manual changes to inputs to account for missing data feeds or to test scenarios. The user interface of FEWS is highly configurable, and it’s possible to create custom ‘Operations Views’, such as the spreadsheet-like views commonly used in river operations:

Automate, Automate, Automate

For me, the automation features of FEWS are appealing in their own right, independent of running an operational system.

A good example is importing gridded rainfall data into a Source model. Source already includes a preprocessor for importing gridded climate data (such as AWAP) and generating the catchment average inputs required by the rainfall runoff components. The trouble is, when the gridded inputs change, such as extending the series with recent data, Source users must manually go back into the tool and repeat the process of importing the grids and computing the catchment averages.

FEWS also has the ability to generate catchment average inputs from grids, but in the case of FEWS, you would configure this as a workflow that gets run incrementally on new grids as they are received³. This avoids the manual step and ensures that the catchment average inputs are always ready for Source.

We can take this further with FEWS and construct a workflow that combines multiple data sources, while still producing the catchment average inputs required by Source. For example, the gridded AWAP data can be augmented with point observations to capture the hours between the most recent AWAP release and the current time. If appropriate, the time series can be further extended with forecasts from numerical weather prediction (NWP) models:

FEWS: Keep Your Source Model On Its Toes

Without automation, very few people would consider updating their Source model weekly (let alone daily). The data processing capabilities of FEWS, coupled with the workflow engine, make it feasible to keep the Source model current at all times.

I expect most organisations will continue to adopt and use FEWS for its operational capabilities, and in that sense, being able to incorporate existing Source models will be a key requirement for some. However, I also suspect that, for some Source users, the need to keep a Source model current, and to automate complex input processing, output processing and reporting will warrant investigation of FEWS for its workflow automation capabilities alone.

---