Evaluation¶
TCRM provides functionality to evaluate the performance of the track
model over the model domain through the Evaluate
module. By
running a number of simulations, each generating a similar number of
years of events to the input track database, we can compare the
synthetic events with the observations. Metrics include track density,
genesis distribution, minimum pressure distributions, landfall rates
and longitude crossing rates.
Setup¶
An example configuration for executing an evaluation. Other sections of the configuration file should remain unchanged.
[Actions]
DataProcess=True
ExecuteStat=True
ExecuteTrackGenerator=True
ExecuteWindfield=False
ExecuteHazard=False
PlotHazard=False
PlotData=False
ExecuteEvaluate=True
DownloadData=True
[Input]
LandMask=input/landmask.nc
MSLPFile=MSLP/slp.day.ltm.nc
CoastlineGates=input/gates.txt
[TrackGenerator]
NumSimulations=500
YearsPerSimulation=30
NumTimeSteps=360
TimeStep=1.0
SeasonSeed=1
TrackSeed=1
Note that the YearsPerSimulation
option is now set to 30, so TCRM
will generate 500 event sets, each with the equivalent of 30 years of
TCs. The annual frequency of events is based on a Poisson distribution
around the mean annual frequency, which is determined from the input
track database.
A csv-format file containing a set of gates around the coastline is
required for Evaluate.landfallRates
. Gates are defined as a
series of points around the coastline in number, longitude, latitude
format. Below are the first 15 points in the sample gates provided in
the code base.
0.00, 114.78, -34.70
1.00, 115.20, -32.95
2.00, 114.90, -31.17
3.00, 114.49, -29.42
4.00, 113.66, -27.83
5.00, 112.73, -26.28
6.00, 112.96, -24.50
7.00, 113.22, -22.72
8.00, 114.47, -21.43
9.00, 115.98, -20.44
10.00, 117.74, -20.08
11.00, 119.46, -19.55
12.00, 121.12, -18.84
13.00, 121.74, -17.15
14.00, 123.04, -15.91
15.00, 124.49, -14.85
Output¶
The Evaluate
module generates a set of figures that compare the
synthetic event set to the input track database. For the synthetic
events, the module calculates a mean value and upper and lower
percentiles of the distribution (commonly the 5th and 95th percentile
values).
Some of the results are also saved to netCDF files (track density, pressure distributions and longitude crossing rates) for users to import into other graphics packages.