Evaluation

TCRM provides functionality to evaluate the performance of the track model over the model domain through the Evaluate module. By running a number of simulations, each generating a similar number of years of events to the input track database, we can compare the synthetic events with the observations. Metrics include track density, genesis distribution, minimum pressure distributions, landfall rates and longitude crossing rates.

Setup

An example configuration for executing an evaluation. Other sections of the configuration file should remain unchanged.

[Actions]
DataProcess=True
ExecuteStat=True
ExecuteTrackGenerator=True
ExecuteWindfield=False
ExecuteHazard=False
PlotHazard=False
PlotData=False
ExecuteEvaluate=True
DownloadData=True

[Input]
LandMask=input/landmask.nc
MSLPFile=MSLP/slp.day.ltm.nc
CoastlineGates=input/gates.txt

[TrackGenerator]
NumSimulations=500
YearsPerSimulation=30
NumTimeSteps=360
TimeStep=1.0
SeasonSeed=1
TrackSeed=1

Note that the YearsPerSimulation option is now set to 30, so TCRM will generate 500 event sets, each with the equivalent of 30 years of TCs. The annual frequency of events is based on a Poisson distribution around the mean annual frequency, which is determined from the input track database.

A csv-format file containing a set of gates around the coastline is required for Evaluate.landfallRates. Gates are defined as a series of points around the coastline in number, longitude, latitude format. Below are the first 15 points in the sample gates provided in the code base.

0.00, 114.78, -34.70
1.00, 115.20, -32.95
2.00, 114.90, -31.17
3.00, 114.49, -29.42
4.00, 113.66, -27.83
5.00, 112.73, -26.28
6.00, 112.96, -24.50
7.00, 113.22, -22.72
8.00, 114.47, -21.43
9.00, 115.98, -20.44
10.00, 117.74, -20.08
11.00, 119.46, -19.55
12.00, 121.12, -18.84
13.00, 121.74, -17.15
14.00, 123.04, -15.91
15.00, 124.49, -14.85

Output

The Evaluate module generates a set of figures that compare the synthetic event set to the input track database. For the synthetic events, the module calculates a mean value and upper and lower percentiles of the distribution (commonly the 5th and 95th percentile values).

Some of the results are also saved to netCDF files (track density, pressure distributions and longitude crossing rates) for users to import into other graphics packages.