Utilities package¶
Module contents¶
Submodules¶
Utilities.Intersections module¶
Intersections
– determine line intersections for polygons and lines¶
-
class
Crossings
¶ Bases:
object
Determine if a a line intersects some other geometric feature (another line, a circle, a polygon).
-
CircleLine
(c, r, a1, a2)¶ Determine the intersection points of a circle and a line segment.
- Parameters
- Returns
Intersection
object with theIntersection.status
updated. “Inside” means the line segment is wholly contained within the circle; “Outside” means the line segment is wholly outside the circle; “Intersection” means the line intersects the circle. In this final case, thepoints
attribute of theIntersection
object is populated with aPoint
object of the location of the intersection.
-
CirclePolygon
(c, r, points)¶ Determine the intersection points of a circle and a polygon. Uses
Crossings.CircleLine()
on each line segment in the polygon to determine if the two features intersect.- Parameters
c –
Point
representing the centre of the circle.r – Radius of the circle in map units.
- Paran points
Array of
Point
objects representing the polygon.- Returns
Intersection
object with updated status and points attributes.
-
LineLine
(a1, a2, b1, b2)¶ Determine if two line segments intersect.
- Parameters
- Returns
Intersection
object withstatus
set toIntersection
if the lines intersect,Coincident
if the lines overlap,Parallel
if the lines are parallel orNo Intersection
if the lines do not intersect. If the lines intersect, then thepoints
attribute is set to the location of the intersection.
-
LinePolygon
(a1, a2, points)¶ Determine if a line intersects a polygon.
-
-
class
Intersection
(state=None)¶ Bases:
object
An Intersection object.
- Parameters
state (str) – Initial state of the
Intersection
(defaultNone
).
Parameters: None Members: status (string) and points (list) Methods: None Internal Methods: None
-
class
Point
(x, y)¶ Bases:
object
Class representation for a Point Contains x and y members Created by Geoff Xu, 2006
-
getX
()¶
-
getY
()¶
-
Utilities.columns module¶
columns
– load delimited data files¶
-
colReadCSV
(configFile, dataFile, source)¶ Loads a csv file containing ‘column’ data into a record (numpy) array with columns labelled by ‘fields’. There must be a section in the
configFile
namedsource
that sets out the format of the data file.- Parameters
configFile (str) – Path to a configuration file that holds details of the input data.
dataFile (str) – Path to the input file to load.
source (str) – Name of the source format. There must be a corresponding section in the
configFile
.
- Returns
A
numpy.ndarray
that contains the input data.
Utilities.config module¶
config
– reading configuration files¶
-
ConfigParser
()¶
-
cnfGetIniValue
(configFile, section, option, default=None)¶ Helper function to interface with code that uses the old config parser.
- Parameters
configFile (str) – path to the configuration file to read.
section (str) – Section name to read.
option (str) – Option name to read.
default – Optional default value to use if the section/option pair is not present in the file.
- Returns
Value recorded in the section/option of the config file, if present; the default value otherwise (if defined).
- Raises
NoSectionError, NoOptionError – if the section/option is not defined, and no default value given.
-
formatList
(lst)¶ Convert a list into a comma-joined string.
- Parameters
lst (list) – Input list to join.
- Returns
A string comprised of the list elements joined by commas.
- Return type
str
-
parseBool
(txt)¶ Parser for boolean options
- Parameters
txt (str) – String from config file to parse.
- Returns
True
if the string is ‘True’,False
otherwise.- Return type
boolean
-
parseList
(txt)¶ Parse a comma-separated line into a list.
- Parameters
txt (str) – String from config file to parse.
- Returns
List, based on the input string.
- Return type
list
-
reset
()¶ Re-instantiate ConfigParser (only for use in tests)
Utilities.confjson module¶
-
class
StrictConfigParser
(defaults=None, dict_type=<class 'collections.OrderedDict'>, allow_no_value=False, *, delimiters=('=', ':'), comment_prefixes=('#', ';'), inline_comment_prefixes=None, strict=True, empty_lines_in_values=True, default_section='DEFAULT', interpolation=<object object>, converters=<object object>)¶ Bases:
configparser.ConfigParser
-
dget
(section, option, default=None, type=<class 'str'>)¶
-
-
flatten
(config)¶
Utilities.convolve module¶
convolve
– convolve two 2-d fields¶
-
convolve
(im, direction, mtype='terrain', res=25.0, height=5.0)¶ Smooth a 2D array im by convolving with a kernel of size n.
- Parameters
im (
numpy.ndarray
) – 2-d array of values to be smoothed.dir (str) – One of ‘N’,’NE’,’E’,’SE’,’S’,’SW’,’W’,’NW’ to define the direction of the site multiplier to evaluate.
mtype (str) – Model type = either “terrain” or “shield”.
res (float) – Resolution of the input grid dataset.
height (float) – nominal height of the buildings to be used in evaluating the shielding multiplier.
- Returns
2-d array of convolved data (convolved with the appropriate kernel). The output array is the same size as the input array, with boundary values set to be filled to a value of 1.0
- Return type
numpy.ndarray
-
getKernel
(d, mtype, res=25.0, height=5.0)¶ Define an appropriate kernel for smoothing the data
- Parameters
d (str) – Wind direction - one of [‘N’,’NE’,’E’,’SE’,’S’,’SW’,’W’,’NW’]
m (str) – Multiplier type - either ‘terrain’ or ‘shield’
res (float) – Resolution of the dataset (in metres) (default is 25)
height (float) – Nominal height of buildings, for use in evaluating the shielding multiplier (default is 5)
- Returns
kernel to apply to the raw multiplier values to generate directional multiplier values.
- Return type
2-d
numpy.ndarray
Utilities.datasets module¶
datasets
– manage downloaded datasets¶
-
class
DataSet
(name, url, path, filename=None)¶ Bases:
object
Download a dataset from a url, store in a specified path with a given filename.
- Parameters
name (str) – Name of the dataset to be downloaded.
url (str) – URL of the dataset.
path (str) – path name for the storage location of the dataset
filename (str or None) – name of the file to be saved (can be different from the name of the dataset).
-
download
(callback=None)¶ Execute the download of the dataset. If the dataset has already been downloaded (the
isDownloaded
will be set to True), then don’t try to download.- Parameters
callback (function) – Callback function (for reporting status to STDOUT).
- Raises
IOError – Unable to download the dataset.
-
isDownloaded
()¶ Determine if a file has already been downloaded
- Returns
True if the file exists, False otherwise.
-
checkAndDownload
(callback=None)¶ Check the
DATASETS
list and download each, if it has not previously been downoladed.- Parameters
callback (function) – Callback function (for reporting status to STDOUT).
-
loadDatasets
(configFile)¶ Load the details of the datasets to be downloaded from the configuration settings. This updates the
DATASETS
list.
Utilities.files module¶
-
exception
WindowsError
¶ Bases:
OSError
-
flConfigFile
(extension='.ini', prefix='', level=None)¶ Build a configuration filename (default extension .ini) based on the name and path of the function/module calling this function. Can also be useful for setting log file names automatically. If prefix is passed, this is preprended to the filename.
- Parameters
extension (str) – file extension to use (default ‘.ini’). The period (‘.’) must be included.
prefix (str) – Optional prefix to the filename (default ‘’).
level – Optional level in the stack of the main script (default = maximum level in the stack).
- Returns
Full path of calling function/module, with the source file’s extension replaced with extension, and optionally prefix inserted after the last path separator.
Example: configFile = flConfigFile(‘.ini’) Calling flConfigFile from /foo/bar/baz.py should return /foo/bar/baz.ini
-
flGetStat
(filename, CHUNK=65536)¶ Get basic statistics of filename - namely directory, name (excluding base path), md5sum and the last modified date. Useful for checking if a file has previously been processed.
- Parameters
filename (str) – Filename to check.
CHUNK (int) – (optional) chunk size (for md5sum calculation).
- Returns
path, name, md5sum, modification date for the file.
- Raises
TypeError – if the input file is not a string.
IOError – if the file is not a valid file, or if the file cannot be opened.
Example: dir, name, md5sum, moddate = flGetStat(filename)
-
flLoadFile
(filename, comments='%', delimiter=',', skiprows=0)¶ Load a delimited text file – uses
numpy.genfromtxt()
- Parameters
filename (file or str) – File, filename, or generator to read
comments (str, optional) – (default ‘%’) indicator
delimiter (str, int or sequence, optional) – The string used to separate values.
-
flLogFatalError
(tblines)¶ Log the error messages normally reported in a traceback so that all error messages can be caught, then exit. The input ‘tblines’ is created by calling
traceback.format_exc().splitlines()
.- Parameters
tblines (list) – List of lines from the traceback.
-
flModDate
(filename, dateformat='%Y-%m-%d %H:%M:%S')¶ Return the last modified date of the input file
- Parameters
filename (str) – file name (full path).
dateformat (str) – Format string for the date (default ‘%Y-%m-%d %H:%M:%S’)
- Returns
File modification date/time as a string
- Return type
str
Example: modDate = flModDate( ‘C:/foo/bar.csv’ , dateformat=’%Y-%m-%dT%H:%M:%S’ )
-
flModuleName
(level=1)¶ Get the name of the module <level> levels above this function
- Parameters
level (int) – Level in the stack of the module calling this function (default = 1, function calling
flModuleName
)- Returns
Module name.
- Return type
str
Example: mymodule = flModuleName( ) Calling flModuleName() from “/foo/bar/baz.py” returns “baz”
-
flModulePath
(level=1)¶ Get the path of the module <level> levels above this function
- Parameters
level (int) – level in the stack of the module calling this function (default = 1, function calling
flModulePath
)- Returns
path, basename and extension of the file containing the module
Example: path, base, ext = flModulePath( ) Calling flModulePath() from “/foo/bar/baz.py” produces the result “/foo/bar”, “baz”, “.py”
-
flProgramVersion
(level=None)¶ Return the __version__ string from the top-level program, where defined.
If it is not defined, return an empty string.
- Parameters
level (int) – level in the stack of the main script (default = maximum level in the stack)
- Returns
version string (defined as the
__version__
global variable)
-
flSaveFile
(filename, data, header='', delimiter=',', fmt='%.18e')¶ Save data to a file.
Does some basic checks to ensure the path exists before attempting to write the file. Uses
numpy.savetxt
to save the data.- Parameters
filename (str) – Path to the destination file.
data – Array data to be written to file.
header (str) – Column headers (optional).
delimiter (str) – Field delimiter (default ‘,’).
fmt (str) – Format statement for writing the data.
-
flSize
(filename)¶ Return the size of the input file in bytes
- Parameters
filename (str) – Full path to the file.
- Returns
File size in bytes.
- Return type
int
Example: file_size = flSize( ‘C:/foo/bar.csv’ )
-
flStartLog
(logFile, logLevel, verbose=False, datestamp=False, newlog=True)¶ Start logging to logFile all messages of logLevel and higher. Setting
verbose=True
will report all messages to STDOUT as well.- Parameters
logFile (str) – Full path to log file.
logLevel (str) – String specifiying one of the standard Python logging levels (‘NOTSET’,’DEBUG’,’INFO’,’WARNING’,’ERROR’, ‘CRITICAL’)
verbose (boolean) –
True
will echo all logging calls to STDOUTdatestamp (boolean) –
True
will include a timestamp of the creation time in the filename.newlog (boolean) –
True
will create a new log file each time this function is called.False
will append to the existing file.
- Returns
logging.logger
object.
Example: flStartLog(‘/home/user/log/app.log’, ‘INFO’, verbose=True)
Utilities.grid module¶
grid
– read, write and sample ascii grid files¶
Provide functions to read, write and sample ascii grid files. The ascii files are assumend to have and ArcGIS GIRD format:
ncols nx
nrows ny
xllcorner xll
yllcorner yll
cellsize dx
NODATA_value -9999
data data data ...
: : :
: : :
-
class
SampleGrid
(filename)¶ Bases:
object
Sample data from a gridded data file. The class is instantiated with a gridded data file (either an ascii file or a netcdf file with single 2-d variable), and the
SampleGrid.sampleGrid()
method returns the value of the grid point closest to the given longitude and latitude.- Parameters
filename (str) – Path to a file containing gridded data.
Example:
>>> grid = SampleGrid( '/foo/bar/grid.nc' ) >>> value = grid.sampleGrid( 100., -25. )
-
sampleGrid
(lon, lat)¶ Sample a value from the grid at the given cLon, cLat At this time, does not interplolate from the input grid to the given location.
- Parameters
lon (float) – Longitude of the point to sample.
lat (float) – Latitude of the point to sample.
- Returns
Value of the nearest grid point to the given lon/lat point.
-
grdRead
(filename, delimiter=None)¶ Read formatted data from an ascii grid format file. Returns the longitude and latitude of the grid and the data values
- Parameters
filename (str) – Path to an ascii grid format or netcdf file.
delimiter – Delimiter for the ascii format file (optional).
- Returns
longitude, latitude, data
- Return type
numpy.ndarray
Usage: longitude, latitude, data = grdRead(filename, [delimiter])
-
grdReadFromNetcdf
(filename)¶ Read formatted data from a netcdf file. Returns the longitude and latitude of the grid and the data values. Assumes that there is only one (non-coordinate) variable in the file, which is only 2 dimensional.
- Parameters
filename (str) – Path to a netcdf file to read.
- Returns
longitude, latitude and grid data.
Usage: longitude, latitude, data = grdReadFromNetcdf(filename)
-
grdSave
(filename, data, lon, lat, delta, delimiter=' ', nodata=- 9999, fmt='%.10e', coords='latlon')¶ Save formatted data to an ascii grid format file. The files have 6 header lines describing the data, followed by the data in a gridded format.
- Parameters
filename (str) – Path to the file to be written.
data – 2-d array of data values to store.
lon – Array of longitudes corresponding to data points.
lat – Array of latitudes corresponding to data points.
delta (float) – Spacing between grid points.
delimiter (str) – Delimiter to put between data points (default ‘ ‘).
nodata (float) – Value to indicate missing values (default -9999).
fmt (str) – String format statement.
coords (str) – Optionally store the data in UTM coordinates. Default is to store the data in geographic coordinates (
coords='latlon'
). Ifcoords='UTM'
, then the latitude & longitudes are converted to the local UTM coordinate system.
- Raises
ValueError – If the
filename
is not a string of file handle.
Usage:
>>> grdSave(filename, data, lon, lat, delta, delimiter=' ', nodata=-9999, fmt='%.10e, coords='latlon')
Utilities.interp3d module¶
interp3d
– interpolate to a set of points in 3-dimensional space¶
Use scipy.ndimage.interpolation.map-coordinates()
to interpolate
data in three dimensions.
-
interp3d
(input_array, coords, scale=[360.0, 180.0, 365.0], offset=[0.0, - 90.0, 0.0], prefilter=True)¶ Wrapper to
scipy.ndimage.interpolation.map_coordinates()
, which converts coordinates of points to indices that correspond to the array. We assume that one is working with lon, lat, day data (i.e. initially designed to work with daily long term mean sea level pressure)- Parameters
input_array – A 3-d array of data at regular intervals, representing the data to be evaluated.
coords – A 3-by-n array of coordinates at which the data in
input_array
will be interpolated to.scale (list) – A (list of) scale factor(s) that reduces the
coords
values to the range of indices ininput_array
.offset (list) – A (list of) offset factor(s) that is subtracted from the
coords
values before adjusting thescale
(above).prefilter (boolean) – If
True
(default), then apply a spline filter before interpolation (necessary for spline interpolation of order > 1). IfFalse
, it is assumed that the input is already filtered. Default isTrue
.
- Returns
1-d array of values corresponding to the interpolated values at the points given in
coords
.
Example:
>>> vals = interp3d(data, coords, scale=[360., 180., 365.], ... offset=[0., -90., 0.])
Utilities.interpTrack module¶
interpTrack
– interpolate TC track to a shorter time interval¶
-
ShowSyntax
(exit_code=0)¶ Print basic help information on using this module as a script
- Parameters
exit_code (int) – Exit code.
-
interpolateTrack
(configFile, trackFile, source, delta=0.1, interpolation_type=None)¶ Interpolate the data in a track file to the time interval delta hours.
- Parameters
configFile (str) – Configuration file that contains information on the source format of the track file.
trackFile (str) – Path to csv format track file.
source (str) – Name of the data source. There must be a corresponding section in the configuration file that contains the description of the data.
delta (float) – Time interval in hours to interpolate to. Default is 0.1 hours
interpolation_type (str) – Optionally use Akima or linear interpolation for the track positions. Default is linear 1-dimensional spline interpolation.
- Returns
10 arrays (id, time, date, lon, lat, bearing, forward speed, central pressure, environmental pressure and radius to maximum wind) that describe the track at
delta
hours intervals.
-
main
(argv)¶ Main part of the program
- Parameters
argv (list) – List of command line arguments.
Utilities.lat_long_UTM_conversion module¶
lat_long_UTM_conversion
– convert between geographic & UTM coordinates¶
Usage:
>>> zone, easting, northing = LLToUTM(latitude, longitude,
... ReferenceEllipsoid=23)
>>> latitude, longitude = UTMtoLL(northing, easting, zone,
... isSouthernHemisphere=True,
... ReferenceEllipsoid=23)
-
LLtoUTM
(lat, long, reference_ellipsoid=23, ZoneNumber=None)¶ Converts lat/long to UTM coords. Equations from USGS Bulletin 1532
Lat and long are in decimal degrees Written by Chuck Gantz- chuck.gantz@globalstar.com
- Parameters
Lat (float) – Latitude. North latitudes are positive, south latitudes are negative.
Lon (float) – longitude. East longitudes are positive, west longitudes are negative.
reference_ellipsoid (int) – Reference ellipsoid for the coordinate system (default=23 – WGS84).
ZoneNumber (str) – UTM Zone for coordinates.
- Returns
Tuple of (ZoneNumber, UTMEasting, UTMNorthing)
-
UTMtoLL
(northing, easting, zone, isSouthernHemisphere=True, reference_ellipsoid=23)¶ Converts UTM coords to lat/long. Equations from USGS Bulletin 1532 East longitudes are positive, West longitudes are negative. North latitudes are positive, South latitudes are negative lat and long are in decimal degrees. Written by Chuck Gantz- chuck.gantz@globalstar.com Converted to Python by Russ Nelson <nelson@crynwr.com>
FIXME: This is set up to work for the Southern Hemisphere.
Using http://www.ga.gov.au/geodesy/datums/redfearn_geo_to_grid.jsp
Site Name: GDA-MGA: (UTM with GRS80 ellipsoid) Zone: 36 Easting: 511669.521 Northing: 19328195.112 Latitude: 84 0 ' 0.00000 '' Longitude: 34 0 ' 0.00000 '' Grid Convergence: 0 -59 ' 40.28 '' Point Scale: 0.99960166 ____________ Site Name: GDA-MGA: (UTM with GRS80 ellipsoid) Zone: 36 Easting: 519384.803 Northing: 1118247.585 Latitude: -80 0 ' 0.00000 '' Longitude: 34 0 ' 0.00000 '' Grid Convergence: 0 59 ' 5.32 '' Point Scale: 0.99960459 ____________ Site Name: GDA-MGA: (UTM with GRS80 ellipsoid) Zone: 36 Easting: 611263.812 Northing: 10110547.106 Latitude: 1 0 ' 0.00000 '' Longitude: 34 0 ' 0.00000 '' Grid Convergence: 0 -1 ' 2.84 '' Point Scale: 0.99975325 ____________ Site Name: GDA-MGA: (UTM with GRS80 ellipsoid) Zone: 36 Easting: 611263.812 Northing: 9889452.894 Latitude: -1 0 ' 0.00000 '' Longitude: 34 0 ' 0.00000 '' Grid Convergence: 0 1 ' 2.84 '' Point Scale: 0.99975325
So this uses a false northing of 10000000 in the both hemispheres. ArcGIS used a false northing of 0 in the northern hem though. Therefore it is difficult to actually know what hemisphere you are in.
- Parameters
northing (float) – Northing coordinate in UTM coordinates.
easting (float) – Easting coordinate in UTM coordinates
zone (int) – UTM zone number.
isSouthernHemisphere (boolean) –
True
if the location is in the southern hemisphere (default).reference_ellipsoid (int) – Reference ellipsoid for the coordinate system (default=23 – WGS84).
- Returns
Tuple pair of latitude, longitude for the point.
Utilities.lmomentFit module¶
lmomentFit
– fit GEV functions¶
Two functions {pelgev, samlmu} from the LMOMENTS Fortran package ported to Python to fit a Generalised Extreme Value Distribution function. Original code developed by: J. R. M. HOSKING, IBM RESEARCH DIVISION, T. J. WATSON RESEARCH CENTER, YORKTOWN HEIGHTS, NEW YORK 10598, U.S.A.
Note
Permission to use, copy, modify and distribute this software for any purpose and without fee is hereby granted, provided that this copyright and permission notice appear on all copies of the software. The name of the IBM Corporation may not be used in any advertising or publicity pertaining to the use of the software. IBM makes no warranty or representations about the suitability of the software for any purpose. It is provided “AS IS” without any express or implied warranty, including the implied warranties of merchantability, fitness for a particular purpose and non-infringement. IBM shall not be liable for any direct, indirect, special or consequential damages resulting from the loss of use, data or projects, whether in an action of contract or tort, arising out of or in connection with the use or performance of this software.
-
pelgev
(xmom)¶ Parameter estimation via L-moments for the Generalised Extreme Value Distribution. For -0.8 <= TAU3 < 1., K is approximated by rational functions as in Donaldson (1996, Commun. Statist. Simul. Comput.). If TAU3 is outside this range, Newton-Raphson iteration is used.
- Parameters
xmom (List or
numpy.ndarray
) – Array of length 3, containing the L-moments Lambda-1, Lambda-2 and TAU3.- Returns
Location, scale and shape parameters of the GEV distribution.
- Return type
numpy.ndarray
-
pelgpa
(xmom)¶ Parameter estimation via L-moments for the Generalised Pareto Distribution.
-
samlmu
(data, nmom)¶ Sample L-moments for a data array.
- Parameters
data (
numpy.ndarray
) – Array of length N, containing the data in ascending order.nmom – Number of L-moments to be found (maximum 100).
- Returns
The sample L-moments.
- Return type
numpy.ndarray
-
samlmu3
(data)¶ Functional equivalent to lmoments.samlmu(data, 3). Vectorised for speed but still about half speed of original fortran package.
- Parameters
data – Array of length N, containing the data in ascending order.
- Returns
First 3 L-moments.
- Return type
numpy.ndarray
Utilities.loadData module¶
loadData
- load TC track data from formatted files¶
Load a formatted csv file that contains tropical cyclone track
information and return a collection of Track
objects.
The format of the files is defined in a named section of the input configuration file. See the Source Formats section of the User Guide for details on specifying the format.
TODO:
Modify the source to be a dict containing the kwargs to
numpy.genfromtxt
to make reading additional formats possible
(e.g. include converters for some data fields, as in B-deck format
track files.
-
date2ymdh
(dates, datefmt='%Y-%m-%d %H:%M:%S')¶ Convert date strings to arrays for date components.
- Parameters
dates (
numpy.ndarray
) – Array of str objects that describe a date.datefmt (str) – Format string of the date values in dates, default=’%Y-%m-%d %H:%M:%S’
- Returns
numpy.ndarray
arrays containing the year, month, day, hour and minute of the dates contained in the input array dates.
-
filterPressure
(pressure, inputPressureUnits='hPa', missingValue=9223372036854775807)¶ Filter pressure values to remove any non-physical values.
- Parameters
pressure (
numpy.ndarray
) – input pressure values to check.inputPressureUnits (str) – The units of the pressure values. Can be one of
hPa
,Pa
,kPa
,Pascals
ormmHg
.missingValue (int or float (default
sys.maxsize
)) – replace all null values in the input data with this value.
- Returns
numpy.ndarray
with only valid pressure values.
-
getInitialPositions
(data)¶ Get the array indices corresponding to the initial position of TCs in the input dataset. This is done through examining the data for a number of specific fields to see when they change, or if the data has a field that indicates such an instance.
- Parameters
data (dict) –
dict
of arrays that contains the data loaded from the input file- Returns
numpy.ndarray
of indices that can be used to slice the observations and return those corresponding to an initial TC position.
Note
Using only the ‘num’ field will result in different results than using ‘num’ and ‘season’. From experience, the ‘num’ field in best-track datasets refers to the sequential number of the storm for that season - i.e. it starts at 1 for each season and increments for each new storm. The use of ‘num’ only should be reserved for those situations where the dataset is known to have unique numbers for each storm (e.g in simulated data).
-
getMaxWind
(track, missingValue=9223372036854775807)¶ Determine the maximum wind speed of a
Track
instance- Parameters
track – A
Track
instancemissingValue (int or float (default
sys.maxsize
)) – replace all null values in the input data with this value.
- Returns
Track.trackMaxWind
attribute updated with calculated wind speed updated.
-
getMinPressure
(track, missingValue=9223372036854775807)¶ Determine the minimum pressure of a
Track
instance- Parameters
track – A
Track
instancemissingValue – Replace missing values with this value (default
sys.maxsize
).
- Returns
Track.trackMinPressure
attribute updated
-
getPoci
(penv, pcentre, lat, jdays, eps, coeffs=[2324.1564738613392, - 0.6539853183796136, - 1.3984456535888878, 0.0007407292800881893, 0.004446923142934609, - 1.4337623534206905], missingValue=9223372036854775807)¶ Calculate a modified pressure for the outermost closed isobar, based on a model of daily long-term mean SLP values, central pressure, latitude and day of year.
- Parameters
penv – environmental pressure estimate (from long term mean pressure dataset, hPa).
pcentre – Central pressure of storm (hPa).
lat – Latitude of storm (degrees).
jdays – Julian day (day of year).
eps – random variate. Retained as a constant for a single storm.
coeffs (list) – Coefficients of the model. Defaults based on Southern Hemisphere data (IBTrACS v03r06, 1981-2014).
- Returns
Revised estimate for the pressure of outermost closed isobar.
-
getSpeedBearing
(index, lon, lat, deltatime, ieast=1, missingValue=9223372036854775807)¶ Calculate the speed and bearing of a TC.
- Parameters
index (
numpy.ndarray
) – Array of 0/1 indicating start of new TC (1)lon (
numpy.ndarray
) – Longitudes of TC positions.lat (
numpy.ndarray
) – Latitudes of TC positions.deltatime (
numpy.ndarray
) – Time difference (hours) between consecutive TC observations.ieast (int) – Indicate which direction has positive longitude. 1 = positive longitude eastwards -1 = positive longiture westwards.
missingValue (int or float, default = sys.maxsize) – Replace questionable values with missingValue.
- Returns
speed and bearing :
numpy.ndarray
Example:
>>> speed, bearing = getSpeedBearing(index, lon, lat, deltatime)
-
getTime
(year, month, day, hour, minute)¶ Calculate the number of days since 0001-01-01 00:00:00 UTC + 1
- Parameters
year (
numpy.ndarray
or list) – Year values.month (
numpy.ndarray
or list) – Month values.day (
numpy.ndarray
or list) – Day values.hour (
numpy.ndarray
or list) – Hour values.minute (
numpy.ndarray
or list) – Minute values.
- Returns
numpy.ndarray
of days since 0001-01-01 00:00:00 UTC + 1
-
getTimeDelta
(year, month, day, hour, minute)¶ Calculate the time difference between consecutive observations.
- Parameters
year –
numpy.ndarray
of the year of all observations.month – As for year, but for the month of observation.
day – As for year, but for the day of observation.
hour – As for year, but for the hour of the observation.
minutes – As for year, but for the hour of the observation.
seconds – As for year, but for the hour of the observation.
- Returns
numpy.ndarray
of time difference between observations in hours.
-
getTimeElapsed
(indicator, year, month, day, hour, minute)¶ Calculate the age in hours of each event.
- Parameters
indicator (
numpy.ndarray
) – Array (values of 1 or 0) indicating the beginning of a new TC in the input dataset.year –
numpy.ndarray
of the year of all observations.month – As for year, but for the month of observation.
day – As for year, but for the day of observation.
hour – As for year, but for the hour of the observation.
minutes – As for year, but for the hour of the observation.
- Returns
numpy.ndarray
of time since the initial observation for each TC (in hours).
-
julianDays
(year, month, day, hour, minute)¶ Calculate the julian day (day of year) based on the known date/time information.
- Parameters
year –
numpy.ndarray
of the year of all observations.month – As for year, but for the month of observation.
day – As for year, but for the day of observation.
hour – As for year, but for the hour of the observation.
minute – As for year, but for the hour of the observation.
- Returns
numpy.ndarray
of julian day values for each observation.
-
loadTrackFile
(configFile, trackFile, source, missingValue=0, calculateWindSpeed=True)¶ Load TC track data from the given input file, from a specified source. The configFile is a configuration file that contains a section called ‘source’ that describes the data. This returns a collection of
Track
objects that contains the details of the TC tracks in the input file.- Parameters
configFile (str) – Configuration file with a section
source
.trackFile (str) – Path to a csv-formatted file containing TC data.
missingValue – Replace all null values in the input data with this value (default=0).
calculateWindSpeed (boolean) – Calculate maximum wind speed using a pressure-wind relation described in
maxWindSpeed()
- Pararm str source
Name of the source format of the TC data. There must be a section in
configFile
matching this string, containing the details of the format of the data.- Returns
A collection of
Track
objects. If any of the variables are not present in the input dataset, they are (where possible) calculated (date/time/windspeed), sampled from default datasets (e.g. environmental pressure) or set to the missing value.
Example:
>>> tracks = loadTrackFile('tcrm.ini', 'IBTRaCS.csv', 'IBTrACS' )
-
ltmPressure
(jdays, time, lon, lat, ncfile, ncvar='slp')¶ Extract pressure value from a daily long-term mean SLP dataset at the given day of year and lon,lat position To use this function (and hence some form of daily LTM SLP data) requires knowledge of the day of year.
- Parameters
jdays (
numpy.ndarray
) – Julian day (day of year) values.time (
numpy.ndarray
) – Time of day for each observation (fraction of a day).lon (
numpy.ndarray
) – Longitude of TC position.lat (
numpy.ndarray
) – Latitude of TC position.ncfile (str) – Path to netCDF file containing daily long-term mean sea level pressure data.
ncvar (str) – Name of the netcdf variable that holds the SLP data
- Returns
numpy.ndarray
of long-term mean sea level pressure values at the day of year and positions given.
-
maxWindSpeed
(index, deltatime, lon, lat, pressure, penv, gustfactor=0.9524)¶ Calculate the 10-minute-mean maximum wind speed from the central pressure deficit, using the method described in Holland et al. (2010).
- Parameters
indicator (
numpy.ndarray
) – Array (values of 1 or 0) indicating the beginning of a new TC in the input dataset.deltatime (
numpy.ndarray
) – Time difference (in hours) between each point in the record.lon (
numpy.ndarray
) – Longitudes of TC postions.lat (
numpy.ndarray
) – Latitudes of TC positions.pressure (
numpy.ndarray
) – Central pressure estimate of TCs (hPa).penv (
numpy.ndarray
) – Environmental pressure estimates for each TC postion (hPa).gf (float) – Gust factor - default value represents converting from a 1-minute sustained wind speed to a 10-minute mean wind speed. Based on Harper et al. 2010, WMO-TD1555.
- Returns
numpy.ndarray
of estimated wind speed based on central pressure deficit.
Example:
>>> v = maxWindSpeed(indicator, dt, lon, lat, pressure, penv)
-
parseAge
(data, indicator)¶ Parse the TC age information to get a proxy date record. Assumes every TC starts at 2000-01-01 00:00 and calculates year, month, day, hour and minute values based on the age field.
Assumes the age is given in hours since the initial timestep.
- Parameters
data (dict) –
dict
of arrays that contains the data loaded from the input file.indicator (
numpy.ndarray
) – Array (values of 1 or 0) indicating the beginning of a new TC in the input dataset.
- Returns
numpy.ndarray`s of year, month, day, hour, minute and :class:`datetime.datetime
objects.
-
parseDates
(data, indicator, datefmt='%Y-%m-%d %H:%M:%S')¶ Parse the date/time information to extract year, month, day, hour and minute details for the input dataset.
- Parameters
data (dict) –
dict
of arrays that contains the data loaded from the input file.indicator (
numpy.ndarray
) – Array (values of 1 or 0) indicating the beginning of a new TC in the input dataset.datefmt (str) – Format string of the date values in dates, default=’%Y-%m-%d %H:%M:%S’
- Returns
numpy.ndarray`s of year, month, day, hour, minute and :class:`datetime.datetime
objects.
Utilities.maputils module¶
maputils
– mapping functions¶
Contains mapping functions that operate on arrays. Supercedes some of the functions lying around in some unusual places (like DataProcess).
-
bear2LatLon
(bearing, distance, oLon, oLat)¶ Calculate the longitude and latitude of a new point from an origin point given a distance and bearing.
- Parameters
bearing – Direction to new position (degrees, +ve clockwise from north).
distance – Distance to new position (km).
oLon – Initial longitude.
oLat – Initial latitude.
- Returns
new longitude and latitude (in degrees)
-
bearing2theta
(bearing)¶ Converts bearing in azimuth coordinate system into theta in cartesian coordinate system. Assumes -2*pi <= bearing <= 2*pi
- Parameters
bearing – Bearing to convert (in radians) (+ve clockwise from north).
- Returns
Angle in cartesian coordinate system (+ve anticlockwise from east).
-
coriolis
(lat)¶ Calculate the Coriolis factor (f) for a given latitude (degrees). If a list is passed, return a list, else return a single value.
- Parameters
lat (Array-like.) – Latitude (degrees).
- Returns
Coriolis factor.
-
dist2GC
(cLon1, cLat1, cLon2, cLat2, lonArray, latArray, units='km')¶ Calculate the distance between an array of points and the great circle joining two (other) points. All input values are in degrees. By default returns distance in km, other units specified by the ‘units’ kwarg.
Based on a cross-track error formulation from: http://williams.best.vwh.net/avform.htm#XTE
- Parameters
cLon1 (float) – Longitude of first point.
cLat1 (float) – Latitude of first point.
cLon2 (float) – Longitude of second point.
cLat2 (float) – Latitude of second point.
lonArray –
numpy.ndarray
of longitudes for which the distance to the line joining the two points will be calculated.latArray –
numpy.ndarray
of latitudes for which the distance to the line joining the two points will be calculated.
- Returns
2-d array of distances between the array points and the line joining two points.
- Return type
numpy.ndarray
-
distGC
(lat, lon)¶ Distance based on the great circle navigation between pairs of points.
- Parameters
lat – A pair of latitude values for the two points.
lon – A pair of longitude values for the two points.
- Returns
Distance (in kilometres) between the two points, based on great circle navigation.
Example:
>>> dist = distGC([-20, -40],[120,190]) 6914.42
-
find_index
(array, value)¶ Find the index of ‘array’ with a value closest to ‘value’
- Parameters
array (
numpy.ndarray
or list.) – array of data values.value – a value to search array for (or find the index of the nearest value to value).
idx (int) – index of array that most closely matches value
- Raises
ValueError if value is a
numpy.ndarray
or a list
Example:
>>> find_index(np.arange(0., 100., 0.5), 15.25) 30
-
find_nearest
(array, value)¶ Find the closest value in ‘array’ to ‘value’
- Parameters
array (
numpy.ndarray
) – array of data values.value (int or float) – a value to search the array for (or find the index of the nearest value to ‘value’
- Returns
array[idx], where idx is the index of array that corresponds to the value closest to ‘value’.
- Raises
ValueError – If value is a
numpy.ndarray
or a list.IndexError – If the value cannot be found in the array
Example:
>>> n = find_nearest( np.arange(0,100.,0.5), 15.25 ) 15.0
-
gridLatLonBear
(cLon, cLat, lonArray, latArray)¶ Generate a grid containing the bearing of the points defined by (lonArray,latArray) from the point defined by (cLon,cLat). (lonArray,latArray) and (cLon,cLat) are in degrees. Returns bearing in radians.
- Parameters
cLon (float) – Longitude of the point to measure the distance from.
cLat (float) – Latitude of the point to measure the distance from.
lonArray – 1-d array of longitude values that will define the grid over which distances will be calculated.
latArray – 1-d array of latitude values that will define the grid over which distances will be calculated.
- Returns
2-d array containing the bearing (direction) of the points defined in
lonArray
andlatArray
from the point (cLon
,cLat
)
Example:
>>> from maputils import gridLatLonBear >>> import numpy as np >>> lonArray = np.arange(90.,100.,0.1) >>> latArray = np.arange(-20.,-10.,0.1) >>> gridLatLonBear( 105., -15., lonArray, latArray) array([[-1.94475949, -1.94659552, -1.94845671, ..., -2.36416927, -2.37344337, -2.38290081], [-1.93835542, -1.94015859, -1.94198663, ..., -2.35390045, -2.36317282, -2.37263233], [-1.93192776, -1.93369762, -1.93549204, ..., -2.34343069, -2.35269718, -2.36215458], ..., [-1.29066433, -1.28850464, -1.28632113, ..., -0.84374983, -0.83405688, -0.82416555], [-1.28446304, -1.28227062, -1.28005406, ..., -0.83332654, -0.82361918, -0.813717 ], [-1.27828819, -1.27606348, -1.27381433, ..., -0.82310335, -0.81338586, -0.80347714]])
-
gridLatLonDist
(cLon, cLat, lonArray, latArray, units=None)¶ Generate a grid containing the spherical earth distance of the points defined by (lonarray, latarray) from the point defined by (clon, clat). (lonarray,latarray) and (clon,clat) are in degrees. Returns distances in km by default, other units specified by the ‘units’ kwarg.
Based on m_lldist.m by Rich Pawlowicz (rich@ocgy.ubc.ca) Modified by Craig Arthur 2006-11-13
- Parameters
cLon (float) – Longitude of the point to measure the distance from.
cLat (float) – Latitude of the point to measure the distance from.
lonArray – 1-d array of longitude values that will define the grid over which distances will be calculated.
latArray – 1-d array of latitude values that will define the grid over which distances will be calculated.
units (str) – Units of distance to be returned (default is kilometre)
- Returns
2-d array containing the distance of the points defined in
lonArray
andlatArray
from the point (cLon
,cLat
).
Example:
>>> lonArray = np.arange(90.,100.,0.1) >>> latArray = np.arange(-20.,-10.,0.1) >>> dist = gridLatLonDist( 105., -15., lonArray, latArray, 'km')
-
latLon2Azi
(lat, lon, ieast=1, azimuth=0, wantdeg=True)¶ Returns the bearing and distance (in km) between consecutive members of the array pair (lat,lon).
- Parameters
lat – Latitudes of positions.
lon – Longitudes of positions.
ieast (int) – 1 for longitudes increasing towards East, -1 for longitudes increasing towards West (default 1).
azimuth (float) – Local coordinate system constructed with origin at latr,lonr, X axis (‘North’) in direction of azimuth, and Y axis such that X x Y = Z(down) when going from (lat,lon) to (x,y) (default 0).
wantdeg (boolean) – If
True
return bearings as degrees, not radians.
- Returns
azimuth (+ve clockwise from north) and distance (in km).
-
latLon2XY
(xr, yr, lat, lon, ieast=1, azimuth=0)¶ Calculate the cartesian distance between consecutive lat,lon points. Will bomb at North and South Poles. Assumes geographical coordinates and azimuth in decimal degrees, local Cartesian coordinates in km.
- Parameters
xr – Reference longitude, normally 0.
yr – Reference latitude, normally 0.
lat – Array of latitudes.
lon – Array of longitudes.
ieast (int) – 1 if longitude increases toward the East (normal case), -1 if longitude increases toward the West.
azimuth (int) – local coordinate system constructed with origin at latr,lonr, X axis (‘North’) in direction of azimuth, and Y axis such that X x Y = Z(down) when going from (lat,lon) to (x,y) scalar or array.
- Returns
Array of northward and eastward distances between consecutive points. use
xy2r()
to convert to a distance between consecutive points.
-
makeGrid
(cLon, cLat, margin=2, resolution=0.01, minLon=None, maxLon=None, minLat=None, maxLat=None)¶ Generate a grid of the distance and angle of a grid of points surrounding a storm centre given the location of the storm. The grid margin and grid size can be set in configuration files. xMargin, yMargin and gridSize are in degrees.
- Parameters
cLon (float) – Reference longitude.
cLat (float) – Reference latitude.
margin (float) – Distance (in degrees) around the centre to fit the grid.
resolution (float) – Resolution of the grid (in degrees).
minLon (float) – Minimum longitude of points to include in the grid.
maxLon (float) – Maximum longitude of points to include in the grid.
minLat (float) – Minimum latitude of points to include in the grid.
maxLat (float) – Maximum latitude of points to include in the grid.
- Returns
2 2-d arrays containing the distance (km) and bearing (azimuthal) of all points in a grid from the
cLon
,cLat
.
-
makeGridDomain
(cLon, cLat, minLon, maxLon, minLat, maxLat, margin=2, resolution=0.01)¶ Generate a grid of the distance and angle of a grid of points across a complete model domain, given the location of the storm.
- Parameters
cLon (float) – Reference longitude.
cLat (float) – Reference latitude.
minLon (float) – Minimum longitude of points to include in the grid.
maxLon (float) – Maximum longitude of points to include in the grid.
minLat (float) – Minimum latitude of points to include in the grid.
maxLat (float) – Maximum latitude of points to include in the grid.
margin (float) – Distance (in degrees) around the centre to fit the grid.
resolution (float) – Resolution of the grid (in degrees).
- Returns
2 2-d arrays containing the distance (km) and bearing (azimuthal) of all points in a grid from the
cLon
,cLat
, spanning the complete region.
-
meshLatLon
(cLon, cLat, margin=2, resolution=0.01)¶ Create a meshgrid of the longitudes and latitudes of a grid.
- Parameters
cLon (float) – Longitude of centre of grid.
cLat (float) – Latitude of centre of grid.
margin (float) – Distance (in degrees) around the centre to build the grid.
resolution (float) – Resolution of the grid (degrees).
- Returns
Coordinate matrices for the longitude and latitude vectors, covering the region within
margin
degrees of (cLon
,cLat
).
-
meshLatLonDomain
(minLon, maxLon, minLat, maxLat, margin=2, resolution=0.01)¶ Create a meshgrid of the lon/lat grid across th full model domain.
- Parameters
minLon (float) – Minimum longitude of the domain.
maxLon (float) – Maximum longitude of the domain.
minLat (float) – Minimum latitude of the domain.
maxLat (float) – Maximum latitude of the domain.
margin (float) – Distance (in degrees) around the centre to build the grid.
resolution (float) – Resolution of the grid (degrees).
- Returns
Coordinate matrices for the longitude and latitude vectors, covering the full domain, plus an additional margin of
margin
degrees.
-
theta2bearing
(theta)¶ Converts a cartesian angle (in radians) to an azimuthal bearing (in radians). Assumes -2*pi <= theta <= 2*pi
- Parameters
theta – Angle in cartesian coordinate system (+ve anticlockwise from east).
- Returns
Bearing in azimuth coordinate system (+ve clockwise from north).
-
xy2r
(x, y)¶ Given x and y arrays, returns the distance between consecutive elements.
- Parameters
x (
numpy.ndarray
) – x-coordinate of points.y (
numpy.ndarray
) – y-coordinate of points.
- Returns
Distance (in native units) between consecutive points.
- Return type
numpy.ndarray
Utilities.metutils module¶
metutils
– perform basic meteorological calculations¶
-
convert
(value, inunits, outunits)¶ Convert value from input units to output units.
- Parameters
value – Value to be converted
inunits (str) – Input units.
outunits (str) – Output units.
- Returns
Value converted to
outunits
units.
-
coriolis
(lat)¶ Calculate the Coriolis factor (f) for a given latitude (degrees).
- Parameters
lat (
numpy.ndarray
or scalar float) – Latitude (degrees).- Returns
Coriolis factor
- Return type
numpy.ndarray
or scalar float
-
dewPointToRH
(t_dry, t_dew)¶ Calculate relative humidity from dry bulb and dew point (in degrees Celsius)
- Parameters
t_dry (float) – Dry bulb temperature (degrees Celsius).
t_dew (float) – Dew point temperature (degrees Celsius).
- Returns
Relative humidity (%)
- Return type
float
-
dewPointToVapPr
(t_dp, units_vp='hPa')¶ Calculate vapour pressure from dew point temperature.
- Parameters
t_dp (float) – Dew point temperature (degrees Celsius)
units_vp (str) – Output units (default
gPressureUnits
).
- Returns
Vapour pressure of water content.
- Return type
float
-
dewPointToWetBulb
(T, Td, pressure)¶ Calculate wet bulb temperature from dry bulb temperature, dew point temperature and air pressure.
Based on the code underpinning the form at: http://www.srh.noaa.gov/epz/?n=wxcalc_dewpoint
- Parameters
T (float) – Dry bulb temperature (degrees Celcius).
Td (float) – Dew point temperature (degrees Celcius).
pressure (float) – Air pressure (hPa).
- Returns
Wet bulb temperature (degrees Celcius).
- Return type
float
TODO: Add unit tests.
-
elevToAirPr
(elev, units_ap='hPa')¶ Approximate air pressure in hectopascals (mb) at a given elevation.
- Parameters
elev (float) – Elevation above sea level (metres).
units_ap (str) – Units of air pressure (default
"hPa"
).
- Returns
Approximate air pressure at the given elevation in the specified or default units.
- Return type
float
-
genesisPotential
(zeta, rh, vmax, shear)¶ Calculate genesis potential index
-
mixRatToVapPr
(rat, prs)¶ Calculate vapour pressure from mixing ratio.
- Parameters
rat (float) – Mixing ratio.
prs (float) – Air pressure.
- Returns
Vapour pressure.
- Return type
float
-
rHToDewPoint
(rh, t_dry)¶ Calculate dew point from relative humidity and dry bulb (in degrees Celsius).
- Parameters
rh (float) – Relative humidity (%).
t_dry (float) – Dry bulb temperature (degrees Celsius).
- Returns
Dew point temperture (degrees Celsius).
- Return type
float
-
rHToMixRat
(rh, tmp, prs, tmp_units='C')¶ Calculate mixing ratio from relative humidity, temperature and pressure.
- Parameters
rh (float) – Relative humidity (%).
tmp (float) – Temperature (any units, default degrees Celsius).
prs (float) – Air pressure (hPa).
tmp_units (str) – Air temperature units (default degrees Celsius).
- Returns
Mixing ratio (g/kg).
- Return type
float
-
satVapPr
(temp, units_vp='hPa')¶ Saturation vapour pressure from temperature in degrees celsius.
- Parameters
temp (float) – Temperature (degrees celsius).
units_vp (str) – Units of the vapour pressure to return. Default is
gPressureUnits
.
- Returns
saturation vapour pressure in the specified or default units.
Note
Calculation is in kPa with a conversion at the end to the required units.
Example:
>>> from metutils import satVapPr >>> satVapPr(25.) 31.697124349060619
-
spHumToMixRat
(q, units='gkg')¶ Calculate mixing ratio from specific humidity. Assumes the input specific humidity variable is in units of g/kg.
- Parameters
q (float) – Specific humidity (any units, assumed g/kg).
units (str) – Units of specific humidity, default g/kg.
- Returns
Mixing ratio (kg/kg).
- Return type
float
-
spHumToRH
(q, tmp, prs)¶ Calculate relative humidity from specific humidity, temperature and pressure.
- Parameters
q (float) – Specific humidity (g/kg).
tmp (float) – Temperature (degrees Celsius).
prs (float) – Air pressure (hPa).
- Returns
Relative humidity (%)
- Return type
float
-
vapPrToDewPoint
(vp, units_vp='hPa')¶ Calculate dew point from vapour pressure (in kPa)
- Parameters
vp (float) – Input vapour pressure.
units_vp (str) – Input units (default
gPressureUnits
)
- Returns
dew point temperature (in degrees Kelvin)
- Return type
float
-
vapPrToMixRat
(es, prs)¶ Calculate mixing ratio from vapour pressure In this function, we (mis)use the symbol es for vapour pressure, when it correctly represents saturation vapour pressure. The function can be used for both.
- Parameters
es (float) – Vapour pressure.
prs (float) – Air pressure.
- Returns
Mixing ratio.
- Return type
float
-
vapPrToRH
(vp, sat_vp)¶ Calculate relative humidity from vapour pressure and saturated vapour pressure.
- Parameters
vp (float) – Vapour pressure (hPa)
sat_vp (float) – Saturation vapour pressure (hPa)
- Returns
Relative humidity (%)
Example:
>>> from metutils import vapPrToRH >>> vapPrToRH(10., 30.) 33.33333333333
-
vapPrToSpHum
(es, prs)¶ Convert vapour pressure to specific humidity.
- Parameters
es (float) – Vapour pressure.
prs (float) – Air pressure.
- Returs
Specific humidity.
- Return type
float
-
vapour
(temp)¶ Determine saturation vapour pressure, given temperature).
- Parameters
temp (float) – Air temperature (degrees Celsius)
- Returns
Saturation vapour pressure (kPa).
- Return type
float
-
wetBulbGlobeTemp
(t_dp, temp)¶ Calculate Wet Bulb Globe Temperature from Dew Point Temperature and Dry Bulb Temperature (same as air temperature).
- Parameters
t_dp (float) – Dew point temperature (degees Celsius).
temp (float) – Air temperature (degrees Celsius).
- Returns
Wet bulb globe temperature (degrees Celsius).
- Return type
float
-
wetBulbToDewPoint
(db, wb, elev=0)¶ Calculate Dew Point from dry bulb and wet bulb temperatures.
- Parameters
db (float) – Dry bulb temperature (degrees Celsius).
wb (float) – Wet bulb temperature (degrees Celsius).
elev (float) – Optional elevation of the observation (metres).
- Returns
Dew point temperature (degrees Kelvin).
- Return type
float
-
wetBulbToRH
(t_db, t_wb, elev)¶ Calculate relative humidity from dry bulb and wet bulb temperatures, and optional elevation.
- Parameters
t_db (float) – Dry bulb temperature (degrees Celsius).
t_wb (float) – Wet bulb temperature (degrees Celsius).
elev (float) – Elevation (metres).
- Returns
Relative humidity (%).
Example:
>>> from metutils import wetBulbToRH >>> wetBulbToRH(25., 10., 0) 6.747686 >>> wetBulbToRH(25., 20., 0) 63.01954
-
wetBulbToVapPr
(db, wb, elev, units_vp='hPa')¶ Calculate vapour pressure from dry bulb and wet bulb temperatures, and optional elevation.
- Parameters
db (float) – Dry bulb temperature (degrees Celsius).
wb (float) – Wet bulb temperature (degrees Celsius).
elev (float) – Elevation (metres).
units_vp (str) – Output units (default
gPressureUnits
)
- Returns
Vapour pressure.
- Return type
float
Utilities.nctools module¶
nctools
– NetCDF utility functions¶
-
ncCreateDim
(ncobj, name, values, dtype, atts=None)¶ Create a dimension instance in a
netcdf4.Dataset
ornetcdf4.Group
instance.- Parameters
ncobj –
netCDF4.Dataset
ornetCDF4.Group
instance.name (str) – Name of the dimension.
values (numpy.ndarray) – Dimension values.
dtype (numpy.dtype) – Data type of the dimension.
atts (dict or None) – Attributes to assign to the dimension instance
-
ncCreateVar
(ncobj, name, dimensions, dtype, data=None, atts=None, **kwargs)¶ Create a Variable instance in a
netCDF4.Dataset
ornetCDF4.Group
instance.- Parameters
ncobj (
netCDF4.Dataset
ornetCDF4.Group
) –netCDF4.Dataset
ornetCDF4.Group
instance where the variable will be stored.name (str) – Name of the variable to be created.
dimensions (tuple) – dimension names that define the structure of the variable.
dtype (
numpy.dtype
) –numpy.dtype
data type.data (
numpy.ndarray
or None.) –numpy.ndarray
Array holding the data to be stored.atts (dict) – Dict of attributes to assign to the variable.
kwargs – additional keyword args passed directly to the
netCDF4.Variable
constructor
- Returns
netCDF4.Variable
instance- Return type
netCDF4.Variable
-
ncFileInfo
(filename, group=None, variable=None, dimension=None)¶ Print summary information about a netCDF file.
Based on ncinfo (https://code.google.com/p/netcdf4-python/source/browse/trunk/utils/ncinfo)
- Parameters
filename (str) – Path to valid netCDF file.
group (str) – Name of a netCDF4.Group instance to describe.
variable (str) – Name of a netCDF4.Variable instance to describe.
dimension (str) – Name of a netCDF4.Dimension instance to describe.
-
ncGetData
(ncobj, var)¶ Extract data values from a variable in a netCDF file.
Note that the variable object is a better way to manipulate the variables, as the object includes all attributes (e.g. units, range, long_name, etc) - use ncGetVar for that purpose.
- Parameters
ncobj (
NetCDF4.Dataset
) –NetCDF4.Dataset
object.var (str) – Name of the variable in the dataset to extract.
- Returns
numpy.masked_array containing the data of the variable, with missing values masked.
- Return type
numpy.ndarray
- Raises
KeyError – If variable does not exist in the given file.
-
ncGetDims
(ncobj, dim, dtype=<class 'float'>)¶ Extract the value of a dimension from a netCDF file. This function assumes the file is written following the CF convention, with the values of the dimension stored in a 1-d variable with the same name.
- Parameters
ncobj (
netCDF4.Dataset
) –netCDF4.Dataset
objectdim (str) – Name of the desired dimension.
dtype (
numpy.dtype
) – Data type of the dimension
- Returns
numpy.ndarray
of the requested dimension.
-
ncGetTimes
(ncobj, name='time')¶ Get the time data from a netcdf file.
- Parameters
ncobj –
netCDF4.Dataset
ornetCDF4.Group
instance.name (str) – Name of the time variable.
- Return times
Array of time dimension values as true Python
datetime
objects.- Return type
numpy.ndarray
ofdatetime
objects
-
ncGetVar
(ncobj, name)¶ Return a netCDF4.variable object.
- Parameters
ncobj (
netCDF.Group
ornetCDF.Dataset
) –netCDF.Group
ornetCDF.Dataset
instance.name (str) – Name of the desired variable.
- Return varobj
netCDF.Variable
instance- Return type
netCDF.Variable
-
ncLoadFile
(filename)¶ Load a netCDF file and return a
netCDF4.Dataset
object.- Parameters
filename (str) – Path to the netCDF file to open.
- Returns
netCDF4.Dataset
object- Return type
netCDF4.Dataset
-
ncSaveGrid
(filename, dimensions, variables, nodata=- 9999, datatitle=None, gatts={}, writedata=True, keepfileopen=False, zlib=True, complevel=4, lsd=None)¶ Save a gridded dataset to a netCDF file using NetCDF4.
- Parameters
filename (str) – Full path to the file to write to.
dimensions –
dict
The input dict ‘dimensions’ has a strict structure, to permit insertion of multiple dimensions. The dimensions should be keyed with the slowest varying dimension as dimension 0.dimesions = {0:{'name': 'values': 'dtype': 'atts':{'long_name': 'units': ...} }, 1:{'name': 'values': 'type': 'atts':{'long_name': 'units': ...} }, ...}
variables –
dict
The input dict ‘variables’ similarly requires a strict structure:variables = {0:{'name': 'dims': 'values': 'dtype': 'atts':{'long_name': 'units': ...} }, 1:{'name': 'dims': 'values': 'dtype': 'atts':{'long_name': 'units': ...} }, ...}
The value for the ‘dims’ key must be a tuple that is a subset of the dimensions specified above.
nodata (float) – Value to assign to missing data, default is -9999.
datatitle (str) – Optional title to give the stored dataset.
gatts (dict or None) – Optional dictionary of global attributes to include in the file.
dtype (
numpy.dtype
) – The data type of the missing value. If not given, infer from other input arguments.writedata (bool) – If true, then the function will write the provided data (passed in via the variables dict) to the file. Otherwise, no data is written.
keepfileopen (bool) – If True, return a netcdf object and keep the file open, so that data can be written by the calling program. Otherwise, flush data to disk and close the file.
zlib (bool) – If true, compresses data in variables using gzip compression.
complevel (integer) – Value between 1 and 9, describing level of compression desired. Ignored if zlib=False.
lsd (integer) – Variable data will be truncated to this number of significant digits.
- Returns
netCDF4.Dataset object (if keepfileopen=True)
- Return type
netCDF4.Dataset
- Raises
KeyError – If input dimension or variable dicts do not have required keys.
IOError – If output file cannot be created.
ValueError – if there is a mismatch between dimensions and shape of values to write.
Utilities.parallel module¶
parallel
– base parallel processing functions¶
-
class
DummyCommWorld
¶ Bases:
object
A dummy COMM_WORLD class that provides the bare essential methods for running the code. This is used for basic parallelisation (task distribution).
This is returned only if mpi4py raises an ImportError or ModuleNotFoundError.
-
Get_rank
()¶
-
Get_size
()¶
-
barrier
()¶
-
finalize
()¶
-
property
name
¶
-
property
rank
¶
-
property
size
¶
-
-
class
DummyStatus
¶ Bases:
object
A dummy Status class that provides a placeholder for the methods that are used to control processing in parallel implementation
-
attemptParallel
()¶ Attempt to load mpi4py.MPI globally as MPI. If mpi4py loads successfully, then a call to MPI.Finalize is registered to be called at exit of the Python interpreter. This is to ensure that MPI exits cleanly.
If mpi4py.MPI cannot be loaded then a dummy mpi4py.MPI is created.
- Returns
An mpi4py.MPI object - either a dummy or the real thing
-
disableOnWorkers
(f)¶ Decorator to disable function f calculation on workers. The function will only be evaluated on the master thread.
- Parameters
f (function) – Function to be wrapped
Utilities.pathLocator module¶
pathLocator
– determine directory one level above the Utilites folder¶
-
getRootDirectory
()¶ Return the name of the path one level above the directory of this current file.
- Returns
Path name one level above this.
- Return type
str
-
is_frozen
()¶ Determine if modules have been built into the interpreter, e.g. by py2exe.
- Returns
True if the modules are frozen, False otherwise.
Utilities.process module¶
process
– control processing of files¶
-
pAlreadyProcessed
(directory, filename, attribute, value)¶ Determine if a file has already been processed (i.e. it is stored in GLOBAL_PROCFILES)
- Parameters
directory (str) – Base path of the file being checked.
filename (str) – Name of the file.
attribute (str) – Attribute name to be checked.
value (str) – Value of the attribute to be tested.
- Returns
True if the value matches that stored in GLOBAL_PROCFILES, False otherwise.
- Return type
boolean
-
pArchiveDateFormat
(date_format=None)¶ Set or get archive date format. Archived files can optionally have a date string inserted into the filename to retain all files with the same name, but different timestamps.
- Parameters
date_format (str) – archive date format (if setting it)
- Returns
archive date format
- Return type
str
-
pArchiveDir
(archive_dir=None)¶ Set or get the archive directory. If setting the directory, its existence is checked and the directory is created.
- Parameters
archive_dir (str) – Archive directory (if setting it).
- Returns
The archive directory.
- Return type
str
- Raises
OSError – if the directory cannot be created
-
pArchiveFile
(filename)¶ Move the file to the archive directory (if specified), inserting a timestamp in the name.
- Parameters
filename (str) – Full path of the file to be archived.
- Returns
True if the file is successfully moved, False otherwise.
- Return type
bool
-
pArchiveTimestamp
(timestamp=False)¶ Set or get archive timstamp flag. If the flag is True, then files that are to be archived will have a timestamp inserted into the file name.
- Parameters
timestamp (bool) – True or False (if setting it)
- Returns
The value of GLOBAL_TIMESTAMP
- Return type
bool
-
pDeleteDatFile
()¶ Delete the existing data file - defined in the GLOBAL_DATFILE variable (list of previously-processed files).
- Returns
True if existing dat file successfully deleted, False otherwise
- Return type
bool
-
pGetProcessedEntry
(directory, filename, attribute)¶ Retrieve the value of an attribute for a file from the GLOBAL_PROCFILES dictionary.
- Parameters
directory (str) – Path name of the file.
filename (str) – File name to retrieve the details of.
attribute (str) – Attribute to retrieve.
- Returns
Attribute value
-
pGetProcessedFiles
(datFileName=None)¶ Retrieve a list of processed files from a dat file. This will also set the global GLOBAL_DATFILE.
- Parameters
datFileName (str) – Name of a data file to read from.
- Returns
True if successfully read the data file, False otherwise.
- Return type
bool
-
pMoveFile
(origin, destination)¶ Move a single file to an archive directory.
- Parameters
origin (str) – Full path of the file to be moved.
destination (str) – Full path of the file destination.
- Returns
True if the file is successfully moved, False otherwise.
- Return type
bool
-
pSetProcessedEntry
(directory, filename, attribute, value)¶ Update the attribute of the given file with the given value.
- Parameters
directory (str) – Base directory of the file.
filename (str) – File name.
attribute (str) – Name of the file attribute to be updated.
value (str) – Attribute value.
-
pWriteProcessedFile
(filename)¶ Write the various attributes of the given file to GLOBAL_DATFILE
- Parameters
filename (str) – Name of file that has been processed.
- Returns
True if the attributes of the file are successfully stored in GLOBAL_PROCFILES and written to GLOBAL_DATFILE, False otherwise.
- Return type
bool
Utilities.progressbar module¶
progressbar
– display progress on STDOUT¶
-
class
SimpleProgressBar
(modname, showbar=True)¶ Bases:
Utilities.progressbar.ProgressBar
-
update
(progress, startPos=0, endPos=1, incr=5.0)¶
-
Utilities.shptools module¶
shptools
- helper functions for manipulating shape files¶
-
parseData
(data)¶ Parse a dict of dicts to generate a list of lists describing the fields, and an array of the corresponding data records
- Parameters
data (dict) – a dict of dicts with field names as keys, and each sub-dict containing keys of ‘Type’, ‘Length’, ‘Precision’ and ‘Data’.
- Returns
fields, records
- Return type
list
-
shpCreateFile
(outputFile, shapes, data, shpType)¶ Create a shapefile of the give type, containing the given fields.
- Parameters
outputFile (str) – full path (excluding extension!) to the shapefile to create.
shapes (
shapefile._Shape
) – Collection of shape objects representing the geometry of features.shptype (int) –
shapefile
object type (these are integer values, but you can also use the shapelib.SHPT value).fields (dict) – a dictionary of dictionaries with field names as keys, and each sub-dictionary containing keys of ‘Type’, ‘Length’,’Precision’ and ‘Data’: ‘Type’ must be one of the following integer values: 0 - strings 1 - integers 2 - doubles 4 - Invalid
- Raises
shapefile.ShapefileException
if unable to write the file.
-
shpGetField
(shape_file, field_name, dtype=<class 'float'>)¶ Extract from the records the value of the field corresponding to fieldname.
- Parameters
shpFile (str) – path to a valid shape file
fieldname (str) – name of a field in the attribute table of the shape file (.dbf)
dtype (dtype) – type of values in the requested field. Default is float
- Returns
the value of the given field for each feature
- Return type
array or list (if dtype is a string)
-
shpGetVertices
(shape_file, key_name=None)¶ Returns a dictionary of arrays containing coordinate pairs representing vertices contained in the shapefile.
Dictionary keys can either be a field contained within the corresponding dbf file (through the optional kwarg keyName), or if no key name is provided, the object id number (i.e. the record number) is used.
WARNING: If any records share the same value for the chosen key, then only one record will be retained in the returned values.
Input: :type shape_file: str :param shape_file: path to a shape file, excluding the extension
- Parameters
key_name (optional str) – name of a field in the shape file that acts as a key for the dictionary of returned vertices
- Returns
dict keyed by object id number or optionally a field name, with values being arrays of vertices of the corresponding shape object.
- Return type
dict
Example: vertices = shpGetVertices(‘/foo/bar/baz/shp’, ‘FIELD1’)
This function is retained for backwards compatibility. We recommend using the shapefile interface directly for extracting shapes and records.
-
shpReadShapeFile
(shape_file)¶ Return the vertices and records for the given shape file
- Parameters
shape_file (str) – path of input shape file.
- Returns
vertices
- Return type
dict
- Returns
records
- Return type
dict
-
shpSaveTrackFile
(outputFile, tracks, fmt='point')¶ Save track data to shapefile. The fields are sorted using the same function as in shpCreateFile, so the fields should be in the correct order.
- Parameters
outputFile (str) – name for the output shapefile, excluding extension.
tracks (
Track
object) – collection of track features.format – Type of features to save. “point” will save each record as a single point. “lines” will save each individual TC track as a single (POLYLINE) feature. “segments” will save each segment of a track to a line feature between consecutive observations.
-
shpWriteShapeFile
(outputFile, shpType, fields, shapes, records)¶ Save data to a shapefile. The fields are sorted using the same function as in shpCreateFile, so the fields should be in the correct order.
- Parameters
outputFile – A dbf file object created by shpCreateFile
shpType – The type of features to be created.
data (dict) – A dictionary of dictionaries with field names as keys, and each sub-dictionary containing keys of ‘Type’, ‘Length’,’Precision’ and ‘Data’ ‘Type’ must be one of the following integer values: 0 - strings 1 - integers 2 - doubles 4 - Invalid
- Raises
shapefile.ShapefileException
if unable to write the file.
-
tracks2line
(tracks, outputFile, dissolve=False)¶ Writes tracks to a shapefile as a collection of line features
If dissolve==True, then each track feature is written as a single polyline feature, otherwise each track segment is stored as a separate feature.
- Parameters
tracks (list of
Track
objects) –Track
features to store in a shape fileoutputFile (str) – Path to output file destination
dissolve (boolean) – Store track features or track segments.
-
tracks2point
(tracks, outputFile)¶ Writes tracks to a shapefile as a collection of point features
- Parameters
tracks (list of
Track
objects) –Track
features to store in a shape fileoutputFile (str) – Path to output file destination
Utilities.smooth module¶
smooth
– smooth a 2D field using a Gaussian filter¶
-
gaussKern
(size)¶ Calculate a normalised Gaussian kernel to apply as a smoothing function.
- Parameters
size (int) – the size of the kernel to use (how many points will be used in the smoothing operation).
- Returns
numpy.ndarray
normalised 2D kernel array for use in convolutions
-
smooth
(im, n=15)¶ Smooth a 2D array im by convolving with a Gaussian kernel of size n.
- Parameters
im (
numpy.ndarray
) – Array of values to be smoothedn (int) – Number of points to include in the smoothing.
- Returns
smoothed array (same dimensions as the input array)
Utilities.stats module¶
stats
– helper functions for statistical methods¶
-
bandwidth
(data)¶ Calculate the bandwidth for a kernel density estimation, using the normal reference method.
- Parameters
data –
numpy.ndarray
of float values- Returns
Float value of the “optimum” bandwidth for the kernel density estimate
-
between
(value, minval, maxval, fuzz=2, inclusive=True)¶ Test whether a value is within some range with some fuzziness at the edges to allow for floating point noise.
The fuzziness is implemented by expanding the range at each end fuzz steps using the numpy.nextafter function. For example, with the inputs minval = 1, maxval = 2, and fuzz = 2; the range would be expanded to minval = 0.99999999999999978 and maxval = 2.0000000000000009 before doing comparisons.
- Parameters
val (float) – Value being tested.
minval (float) – Lower bound of range. Must be lower than maxval.
maxval (float) – Upper bound of range. Must be higher than minval.
fuzz (int) – Number of times to expand bounds using numpy.nextafter.
inclusive (boolean) – Set whether endpoints are within the range.
- Returns
True if val is between minval and maxval, false otherwise.
-
cdf
(x, y)¶ Cumulative Density Function extracted from cdf_lin.m
-
cdf2d
(x, y, z)¶ 2D Cumulative Density Function extracted from cdf2d.m Assumes the grid is uniformly defined, i.e. dx and dy are constant.
-
circmean
(samples, high=6.283185307179586, low=0)¶ Compute the circular mean for samples assumed to be in the range [low to high]
-
circstd
(samples, high=6.283185307179586, low=0)¶ Compute the circular standard deviation for samples assumed to be in the range [low to high]
-
circvar
(samples, high=6.283185307179586, low=0)¶ Compute the circular variance for samples assumed to be in the range [low to high]
-
getCellLonLat
(cellNum, gridLimit, gridSpace)¶ Return the lon/lat of a given cell, based on gridLimit and gridSpace
-
getCellNum
(lon, lat, gridLimit, gridSpace)¶ Return the cell number given longitude and latititude
-
getOccurence
(occurList, indList)¶ Returns an array of indices corresponding to cyclone observations that
-
logger
= <RootLogger root (WARNING)>¶ Functions:
- cdf(x,y)1D array of float
Cumulative Density Function extracted from cdf_lin.m
- cdf2d(x,y,z): 2D array of float
2D Cumulative Density Function extracted from cdf2d.m
- getCellNum(lon, lat, gridLimit, gridSpace): int
Determine the cell number based on the lat/lon, the grid bounds and the grid spacing.
- getCellLonLat(cellNum, gridLimit, gridSpace): 2D float
Determine the lat/lon of the northwestern corner of cellNum
- validCellNum(cellNum, gridLimit, gridSpace): boolean
Determine whether the given cell number can exist within the bounds of the region defined by gridSpace and gridLimit
- maxCellNum(gridLimit, gridSpace): int
Determine maximum cell number based on grid limits and spacing.
-
maxCellNum
(gridLimit, gridSpace)¶ Get maximum cell number based on grid and grid space
-
probability
(return_period)¶ Return an annual probability given a return period
-
rMaxDist
(mean, sig, maxrad=200.0)¶ Based on the logarithmic distribution reported by Willoughby & Rahn (2004)
-
statCellFraction
(gridLimit, gridSpace, valueFile)¶ Calculate the fractional value of each grid cell, based on the values stored in valueFile. :param dict gridLimit: Dictionary of bounds of the grid. :param dict gridSpace: Resolution of the grid to calculate values. :param str valueFile: Path to the ascii grid file containing values to sample.
- Returns
numpy.ndarray
of fractional values, with length equal to the number of cells
Notes: Still need to include bounds checking to ensure the valueFile data actually covers the gridLimits.
-
statMaxRange
(minval, maxval, step)¶ Returns the maximum value in a range used for arranging the cells to be evenly spaced
-
statMinRange
(minval, maxval, step)¶ Returns the minimum value in a range Used for arranging the cells to be evenly spaced
-
statRemoveNum
(a, Num=9223372036854775807)¶ Remove all elements in an array for which value is Num
-
validCellNum
(cellNum, gridLimit, gridSpace)¶ Checks whether the given cell number is valid
Utilities.tcrandom module¶
tcrandom
– extended version of Python’s random
library¶
-
class
Random
(seed, stream)¶ Bases:
object
Pseudorandom number generator.
Expect each simulation to instantiate this class with the same seed integer, and with a unique stream integer (drawn from e.g. an enumeration of all the simulations)
-
logisticvariate
(mu, sigma)¶ Random variate from the logistic distribution.
- Parameters
mu (float) – Location parameter (μ real).
sigma (float) – Scale parameter (σ > 0).
- Returns
A random variate from the logistic distribution.
-
lognormvariate
(xi, mu=0.0, sigma=1.0)¶ Random variate from the lognormal distribution.
- Parameters
xi (float) – Shape parameter
mu (float) – Location parameter
sigma (float) – Scale paramter (σ > 0)
- Returns
A random variate from the lognormal distribution
-
normalvariate
(loc=0, scale=1, shape=None)¶
-
random
()¶
-
uniform
(low=0, high=1, shape=None)¶
-
Utilities.template module¶
template
– replace strings with another value¶
-
replace
(infile, outfile, replacements)¶ Replace all instances of the keys with values in infile and w write to outfile.
In the input file, keywords to be replaced should be written as ‘{keyword}’.
e.g. if infile has a line containing a formatted keyword:
Input = {PATH}
and
replacements = dict('PATH': '/foo/baz')
, then the output file will be written as:Input = /foo/baz
- Parameters
infile (str) – Path to an input file.
outfile (str) – Destination file to be written.
replacements (dict) – A set of key-value pairs that dictate the replacements to occur in the input file.
Utilities.timeseries module¶
timeseries
- Extract timeseries from each timestep of a simulation¶
Extract station timeseries from each timestep of a simulation. This samples the regional wind speed, not the site-specific wind speed. To include site-specific effects, you will first need to include the multiplier values for each site in the station file, then run tsmultipliers.py to apply said multipliers to the output.
-
class
Station
(stationid, longitude, latitude)¶ Bases:
object
Station:
- Description: An object to represent a location for which time series
data will be extracted
Members: id: Unique id string for the station lon: Longitude of the station (geographic coordinates) lat: Latitude of the station (geographic coordinates) data: A DynamicRecArray to hold the time series data
Methods: insideGrid: Determine if the station is inside the simulation domain.
-
insideGrid
(gridx, gridy)¶ Determine if a point is within the defined grid
-
class
Timeseries
(configFile)¶ Bases:
object
Timeseries:
Description: Extract data at a set of :class:`Station`s
Parameters:
- Parameters
configFile (str) – Path to a TCRM configuration file
Members: meta: Boolean whether additional metadata is attached to the Station`s `outputPath: Directory where extracted data will be stored in csv-format files minfile: Name of the file where minima for all `Station`s will be stored.
This will be the outputPath folder
maxfile: As above, but for maxima (e.g. maximum wind speeds) stations: A list of Station objects, read from a file containing details of the stations
Methods:
Internal methods:
-
extract
(dt, spd, uu, vv, prs, gridx, gridy)¶ Extract data from the grid at the given locations. Data is stored in a dictionary, with keys as the station id’s.
- Parameters
tstep (float) – time step being evaluated, as a float (output from matplotlib.num2date)
spd –
numpy.ndarray
of speed values.uu –
numpy.ndarray
of eastward wind speed values.vv –
numpy.ndarray
of northward wind speed values.prs –
numpy.ndarray
of pressure values.gridx –
numpy.ndarray
of grid longitudes.gridy –
numpy.ndarray
of grid latitudes.
-
sample
(lon, lat, spd, uu, vv, prs, gridx, gridy)¶ Extract values from 2-dimensional grids at the given lat/lon.
- Parameters
lon (float) – Longitude of the point to extract.
lat (float) – Latitude of the point to extract.
spd –
numpy.ndarray
of speed values.uu –
numpy.ndarray
of eastward wind speed values.vv –
numpy.ndarray
of northward wind speed values.prs –
numpy.ndarray
of pressure values.gridx –
numpy.ndarray
of grid longitudes.gridy –
numpy.ndarray
of grid latitudes.
- Returns
speed, esatward and northward wind components, and pressure values at the given location
- Return type
tuple
-
shutdown
()¶ Write the data to file, each station to a separate file.
Utilities.track module¶
track
- track-related attributes and functions¶
-
class
Track
(data)¶ Bases:
object
A single tropical cyclone track.
The object exposes the track data through the object attributes. For example, If data contains the tropical cyclone track data (numpy.array) loaded with the
readTrackData()
function, then the central pressure column can be printed out with the code:t = Track(data) print(t.CentralPressure)
- Parameters
data (numpy.ndarray) – the tropical cyclone track data.
-
inRegion
(gridLimit)¶ Check if the tropical cyclone track starts within a region.
- Parameters
gridLimit (
dict
) – the region to check. Thedict
should contain the keysxMin
,xMax
,yMin
andyMax
. The x variable bounds the latitude and the y variable bounds the longitude.
-
minimumDistance
(points)¶ Calculate the minimum distance between a track and a collection of
shapely.geometry.Point
points. Assumes the points and theLongitude
andLatitude
attributes share the same coordinate system (presumed to be geographic coordinates).- Parameters
points – sequence of
shapely.geometry.Point
objects.- Returns
numpy.ndarray
of minimum distances between the set of points and the line features (in km).
-
exception
WindowsError
¶ Bases:
OSError
-
loadTracks
(trackfile)¶ Read tracks from a track .nc file and return a list of
Track
objects.This calls the function ncReadTrackData to parse the track .nc file.
- Parameters
trackfile (str) – the track data filename.
- Returns
list of
Track
objects.
-
loadTracksFromFiles
(trackfiles)¶ Generator that yields
Track
objects from a list of track filenames.When run in parallel, the list trackfiles is distributed across the MPI processors using the balanced function. Track files are loaded in a lazy fashion to reduce memory consumption. The generator returns individual tracks (recall: a trackfile can contain multiple tracks) and only moves on to the next file once all the tracks from the current file have been returned.
- Parameters
trackfiles (list of strings) – list of track filenames. The filenames must include the path to the file.
- Raises
TypeError if input argument is not a list.
-
loadTracksFromPath
(path)¶ Helper function to obtain a generator that yields
Track
objects from a directory containing track .csv files.This function calls loadTracksFromFiles to obtain the generator and track filenames are processed in alphabetical order.
- Parameters
path (str) – the directory path.
- Raises
IOError if the path does not exist.
-
ncReadTrackData
(trackfile)¶ Read a netcdf-format track file into a collection of
Track
objects. The returnedTrack
objects must have all attributes accessed by the __getattr__ method.- Parameters
trackfile (str) – track data filename (netCDF4 format).
- Returns
track data
- Return type
list of
Track
objects
-
ncSaveTracks
(trackfile, tracks, timeunits='hours since 1900-01-01 00:00', calendar='standard', attributes={})¶ Save a collection of
Track
objects to a netCDF file. This makes use of netCDF4 compound data types to store the data as a structure akin to anumpy.recarray
. Each track in the collection is stored as a separatenetCDF4.Group
instance in the output file.The
Track
objects hold datetime information as an array ofdatetime.datetime
objects - there is no equivalent data type in netCDF4, so the datetime information is converted to floats using thenetCDF4.date2num
function.- Parameters
trackfile (str) – Path to the file to save data to.
tracks (list) – Collection of
Track
objects.timeunits (str) – A string of the form ‘time units since reference time’ describing the time units. Default is ‘hours since 1900-01-01 00:00’.
calendar (str) – Calendar used for time calculations. Valid calendars are ‘standard’, ‘gregorian’, ‘proleptic_gregorian’, ‘noleap’, ‘365_day’, ‘360_day’, ‘julian’, ‘all_leap’, ‘366_day’. Default is ‘standard’, which is a mixed Julian/Gregorian calendar.
attributes (dict) – Global attributes to add to the file.
-
readMultipleTrackData
(trackfile)¶ Reads all the track datas from a .csv file into a list of numpy.ndarrays. The tracks are seperated based in their cyclone id. This function calls readTrackData to read the data from the file.
- Parameters
trackfile (str) – the track data filename.
- Returns
a collection of
Track
objects
-
readTrackData
(trackfile)¶ Read a track .csv file into a numpy.ndarray.
The track format and converters are specified with the global variables
TRACKFILE_COLS – The column names TRACKFILE_FMTS – The entry formats TRACKFILE_CNVT – The column converters
- Parameters
trackfile (str) – the track data filename.
- Returns
track data
- Return type
numpy.ndarray
Utilities.tracks2shp module¶
tracks2shp
– save a track instance as a shape file¶
-
add_category
(tracks)¶ Add a category field (for central pressure) to the tracks.
-
add_field
(a, descr)¶ Add a field to the description of a track.
-
recdropfields
(rec, names)¶ Return a new numpy record array with fields in names dropped. From http://matplotlib.org/api/mlab_api.html#matplotlib.mlab.rec_drop_fields
- Parameters
rec –
numpy.recarray
containing a number of fields.names (list) – List of names to drop from the record array.
- Returns
A new
numpy.recarray
with the fields in names dropped.
-
tracks2line
(tracks, outputFile, dissolve=False, netcdf_format=False)¶ Writes tracks to a shapefile as a collection of line features
If dissolve==True, then each track feature is written as a single polyline feature, otherwise each track segment is stored as a separate feature.
- Parameters
tracks (list of
Track
objects) –Track
features to store in a shape fileoutputFile (str) – Path to output file destination
dissolve (boolean) – Store track features or track segments.
netcdf_format (bool) – Whether tracks are in TCRM format
- Raises
shapefile.ShapefileException
if there is an error when attempting to save the file.
-
tracks2point
(tracks, outputFile, netcdf_format=False)¶ Writes tracks to a shapefile as a collection of point features.
- Parameters
tracks (list of
Track
objects) –Track
features to store in a shape fileoutputFile (str) – Path to output file destination
netcdf_format (bool) – Whether tracks are in TCRM format
- Raises
shapefile.ShapefileException
if there is an error when attempting to save the file.
Utilities.tsmultipliers module¶
tsmultipliers
– apply site-exposure multipliers to time series output¶
-
process_timeseries
(config_file)¶ Process a set of timeseries files to include the multiplier values.
The combined multiplier values are stored in a shape file as fields, and records are keyed by the same code that is used to select stations for sampling.
- Parameters
config_file (str) – Path to a configuration file.
-
startup
()¶ Parse command line arguments and call the
main()
function.
-
tsmultiply
(inputFile, multipliers, outputFile)¶ Apply multipliers to a single file. Values are combined then written back to the source file.
- Parameters
inputFile (str) – Path to the input timeseries file. This will need to contain the values of the three multipliers (topography, terrain and shielding) for each of eight directions.
multipliers (tuple) – The eight combined multiplier values for the location.
outputFile (str) – Destination for the processed file.
- Returns
The records corresponding to the maximum (localised) wind speed and minimum pressure.
Utilities.version module¶
version
– provide details of software version¶
-
git
(command)¶ Execute the given command with git
- Parameters
command (str) – A valid git command.
- Returns
Output from the given command.
- Return type
str
-
status
()¶ Check status TCRM of code.
- Returns
A message containing a listing of recent changes to the model code.
- Return type
str
-
version
()¶ Check version of TCRM code.
- Returns
Current git commit hash and the date/time of the commit.
- Return type
str
- Raises
subprocess.CalledProcessError
if the git command fails.
Note
This requires
git
to be installed to execute.
Utilities.vorticity module¶
-
absolute
(u, v, x, y)¶
-
can_use_sphere
(longitude, latitude)¶ Test if can use sphere package.
- Calling Sequence:
Result = can_use_sphere(longitude, latitude)
Note that Result is a 3-element tuple, not a scalar. See the description below for details.
Test if the longitude-latitude domain and Python package configur- ation allows use of the NCAR SPHEREPACK 3.0 package. Specifically the function tests:
Can you import sphere?
Is the longitude vector evenly spaced?
Does the longitude vector entirely span the globe, but doesn’t repeat?
Is the latitude vector gaussian? If not, is it evenly spaced and includes the poles?
If the answer to (1) is no, then there’s no use to go through the other tests and the function return tuple element [0] is 0. If (2) or (3) is no, return tuple element [0] is 0. If (4) is both not gaussian and not evenly spaced with the poles, return tuple element [0] is 0. Otherwise, return tuple element [0] is 1. Note that (4) uses the latitude checker in the sphere package.
Method Arguments: * longitude: Vector of longitudes of the domain [deg]. Can be
in any order, and a Numeric array or regular list/tuple.
latitude: Vector of latitudes of the domain [deg]. Can be in any order, and a Numeric array or regular list/tuple.
Output Result: * A 3-element tuple:
- [0]: 1 if the longitude-latitude domain and Python package
configuration passes tests to allow use of the NCAR SPHEREPACK 3.0 package. 0 if does not pass those tests.
- [1]: String containing messages written to stdout by calls to
module sphere used for checking latitude (e.g. if it is gaussian, etc.). If there are no messages or there were no calls needed to sphere for checking latitude, value is empty string.
[2]: Same as [1] except contains stderr messages.
Examples: >>> from can_use_sphere import can_use_sphere >>> import numpy as np >>> lon = np.arange(36)*10 >>> lat = [-90, -60, -30, 0, 30, 60, 90] >>> can_use_sphere(lon, lat)[0] 1 >>> lat = [-90, -60, -30, 0, 30, 60, 87] >>> can_use_sphere(lon, lat)[0] 0 >>> can_use_sphere(lon, lat)[1].splitlines()[1] ‘CANNOT PROCESS THE DATA - Latitude values are incorrect’
-
curl_2d
(x, y, Fx, Fy, missing=1e+20, algorithm='default', R_sphere=6371220.0)¶ Curl of a vector F on a 2-D “rectangular” grid.
The 2-D grid F is defined on is rectangular, meaning that while the grid spacings do not have to be even, each grid box is rectangular and grid boundaries are rectilinear and parallel to each other. If the 2-D grid is on a sphere, we assume that lines of constant longitude and latitude form grids that are rectangular (though in reality this is not strictly true).
Since F does not have a z-component, the curl of F is positive pointing as a normal out of the plane defined by the basis for x and y.
Positional Input Arguments: * x: x-coordinate of each F. 1-D Numeric array with number of
elements equal to the number of columns in array F. Typically x is in km, or on a sphere the longitude in deg. Floating or integer type.
y: y-coordinate of each F. 1-D Numeric array with number of elements equal to the number of rows in array F. Typically y is in km, or on a sphere the latitude in deg. Floating or integer type.
Fx: x-component of vector quantity F. 2-D Numeric array of shape (len(y), len(x)). Floating or integer type.
Fy: y-component of vector quantity F. 2-D Numeric array of same shape and size as Fx. Floating or integer type.
Keyword Input Arguments: * missing: If Fx and/or Fy has missing values, this is the
missing value value. Scalar. Default is 1e+20.
algorithm: Name of the algorithm to use. String scalar. Default is ‘default’. Possible values include:
‘default’: Default method. Algorithm ‘order1_cartesian’ is used.
‘default_spherical’: If ‘spherepack’ can be used, chooses that method. Otherwise ‘order1_spherical’ is used. Either way this option assumes the domain is on a sphere of radius equal to keyword R_sphere.
‘order1_cartesian’: First-order finite-differencing (back- ward and forward differencing used at the endpoints, and centered differencing used everywhere else). If abscissa intervals are irregular, differencing will be correspon- dingly asymmetric. Grid is assumed Cartesian. The curl returned is in units of F divided by the units of x and y (x and y must be in the same units).
‘order1_spherical’: First-order finite-differencing (back- ward and forward differencing used at the endpoints, and centered differencing used everywhere else). If abscissa intervals are irregular, differencing will be correspon- dingly asymmetric. Grid is assumed spherical and x and y are longitude and latitude (in deg). The curl returned is in units of F divided by units of R_sphere. Note because of singularities in the algorithm near the poles, the curl for all points north of 88N or south of 88S latitude are set to missing.
‘spherepack’: The NCAR package SPHEREPACK 3.0 is used for calculating the curl. The spatial domain is the global sphere; x and y are longitude and latitude (in deg), respectively; x must be evenly spaced and y must be gauss- ian or evenly spaced. The domain must cover the entire sphere and x and y must be longitude and latitude, respec- tively, in deg. There can be no missing values. The algorithm also assumes that x and y are monotonically in- creasing from index 0. The curl returned is in units of F divided by m. Thus, if F is velocity in m/s, the curl of F is in units 1/s (and is the vorticity). Note that in this function’s implementation, the final curl output using algorithm ‘spherepack’ is adjusted for R_sphere not equal to the default mean earth radius, if appropriate, so this algorithm can be used on other planets.
Note that SPHEREPACK operates on single precision floating point (Float32) values. This function silently converts input to Float32 for the computations, as needed (the input parameters are not altered, however).
R_sphere: If the grid is the surface of a sphere, this is the radius of the sphere. This keyword is only used when the algorithm is ‘order1_spherical’ or ‘spherepack’ (either chosen explicitly by the algorithm keyword or automatically when ‘default_spherical’ is used). Default value is the “mean” radius of the Earth, in meters. “Mean” is in quotes because this value changes depending on what you consider the mean; the default in this function is the same Earth radius used in module sphere. Note that if the level is a distance z above the “mean” radius of the earth, R_sphere should be set to the “mean” radius plus z.
Value can be a scalar or a Numeric or MA array of the same size and shape as Fx. If keyword is an array, the value of R_sphere for each element corresponds to the same location as in the Fx and Fy arrays; there should also be no missing values in R_sphere (function doesn’t check for this, however).
Output: * Curl of F. Units depend on algorithm used to calculate curl
(see above discussion of keyword algorithm). Numeric array of same shape as Fx and Fy inputs.
If there are missing values, those elements in the output that used missing values in the calculation of the curl are set to the value in keyword missing. If there are missing values in the output due to math errors and keyword missing is set to None, output will fill those missing values with the MA default value of 1e+20.
References: * Glickman, T. S. (Ed.) (2000): “Curl,” Glossary of Meteorology,
Boston, MA: American Meteorological Society, ISBN 1-878220- 34-9.
Haltiner, G. J, and R. T. Williams (1980): Numerical Prediction and Dynamic Meteorology, New York: John Wiley & Sons, pp. 8-9.
Holton, J. R. (1992): An Introduction to Dynamic Meteorology, San Diego, CA: Academic Press, p. 482.
Kauffman, B. G., and W. G. Large (2002): The CCSM Coupler, Version 5.0, User’s Guide, Source Code Reference, and Scientific Description. Boulder, CO: NCAR. Value for earth radius is given at URL: http://www.ccsm.ucar.edu/models/ccsm2.0/cpl5/users_guide/node10.html
Overview of the CDAT Interface to the NCAR SPHEREPACK 3.0. SPHEREPACK is by J. C. Adams and P. N. Swarztrauber. More information on SPHEREPACK can be found at:
- while information on the CDAT implementation used here is at:
Example of a synthetic dataset of winds. The curl is the vorti- city. Images of the wind field and the vorticity are online at http://www.johnny-lin.com/py_pkgs/gemath/doc/test_curl_2d_add.html. Velocity at y greater than 60 and less than -60 are set to 0, to prevent spectral algorithms from giving errors at the poles. Tiny values less than 1e-10 are also set to 0:
Import statements and create data:
>>> import Numeric as N >>> from curl_2d import curl_2d >>> nx = 181 >>> ny = 91 >>> x = N.arange(nx) * 2.0 - 180.0 >>> y = N.arange(ny) * 2.0 - 90.0 >>> y_2d = N.reshape( N.repeat(y, nx), (ny, nx) ) >>> x_2d = N.reshape( N.repeat(N.reshape(x,(1,nx)), ny), (ny, nx) ) >>> u = N.sin(x_2d*N.pi/180.)*N.sin((y_2d-90.)*N.pi/30.) >>> v = N.sin((x_2d-30.)*N.pi/180.)*N.sin((y_2d-90.)*N.pi/30.) >>> N.putmask( u, N.where(N.absolute(u) < 1e-10, 1, 0), 0. ) >>> N.putmask( v, N.where(N.absolute(v) < 1e-10, 1, 0), 0. ) >>> N.putmask( u, N.where(N.absolute(y_2d) > 60., 1, 0), 0. ) >>> N.putmask( v, N.where(N.absolute(y_2d) > 60., 1, 0), 0. )
Use order 1 Cartesian algorithm: If x and y are in meters, and u and v are in m/s, the curl is in 1/s:
>>> curl = curl_2d(x, y, u, v, algorithm='order1_cartesian') >>> ['%.7g' % curl[45,i] for i in range(88,92)] ['-0.007251593', '-0.003628007', '0', '0.003628007'] >>> ['%.7g' % curl[0,i] for i in range(8,12)] ['0', '0', '0', '0']
Use spectral method from SPHEREPACK (need to first remove the repeating dateline point from the domain). Here we take x and y in deg, while u and v are in m/s. If so the curl is in 1/s. These results as much less than for example (b) above since the domain is much larger:
>>> x = x[0:-1] >>> u = u[:,0:-1] >>> v = v[:,0:-1] >>> curl = curl_2d(x, y, u, v, algorithm='spherepack') >>> ['%.7g' % curl[45,i] for i in range(88,92)] ['-6.436402e-08', '-3.220152e-08', '1.33852e-13', '3.220165e-08'] >>> ['%.7g' % curl[0,i] for i in range(8,12)] ['-2.299736e-20', '-4.806512e-21', '-9.283145e-22', '-1.368445e-20']
Use order 1 spherical algorithm: x and y are in deg and u and v are in m/s. The curl is in 1/s. Note that these results are nearly identical to those from SPHEREPACK, except near the poles the values are set to missing:
>>> curl = curl_2d(x, y, u, v, algorithm='order1_spherical') >>> ['%.7g' % curl[45,i] for i in range(88,92)] ['-6.517317e-08', '-3.260645e-08', '0', '3.260645e-08'] >>> ['%.7g' % curl[0,i] for i in range(8,12)] ['1e+20', '1e+20', '1e+20', '1e+20']
Use “default” algorithm, which for this domain chooses the order 1 cartesian algorithm:
>>> curl = curl_2d(x, y, u, v, algorithm='default') >>> ['%.7g' % curl[45,i] for i in range(88,92)] ['-0.007251593', '-0.003628007', '0', '0.003628007']
Use “default_spherical” algorithm with missing data (equal to 1e+20), which because of the missing data will choose the order 1 spherical algorithm. We also do the calculation on Mars, and illustrate passing in the radius as a constant as well as an array of the same size and shape as u and v:
>>> u[30:46,90:114] = 1e+20 >>> curl = curl_2d( x, y, u, v, algorithm='default_spherical' , R_sphere=3390e+3) >>> ['%.7g' % curl[45,i] for i in range(88,92)] ['-1.224875e-07', '-6.128107e-08', '1e+20', '1e+20']
>>> R_mars = N.zeros(u.shape, typecode=N.Float) + 3390e+3 >>> curl = curl_2d( x, y, u, v, algorithm='default_spherical' , R_sphere=R_mars ) >>> ['%.7g' % curl[45,i] for i in range(88,92)] ['-1.224875e-07', '-6.128107e-08', '1e+20', '1e+20']
-
deriv
(*args, **kwargs)¶ Calculate the derivative along a single dimension.
- Calling Sequence:
Result = deriv([x_in,] y_in, missing=1e+20, algorithm=’default’)
Positional Input Arguments: * x_in: Abscissa values of y_in to take with respect to. If
not set, the derivative of y_in is take with respect to unit abscissa intervals. Numeric array of same shape and size as y_in. Must be monotonic and with no duplicate values. Optional. First positional argument out of two, if present.
y_in: Ordinate values, to take the derivative with respect to. Numeric array vector of rank 1. Required. Second posi- tional argument, if x_in is present; only positional argument if x_in is absent.
Keyword Input Arguments: * missing: If y_in and/or x_in has missing values, this is the
missing value value. Scalar. Default is 1e+20.
algorithm: Name of the algorithm to use. String scalar. Default is ‘default’. Possible values include: + ‘default’: Default method (currently set to ‘order1’). + ‘order1’: First-order finite-differencing (backward and
forward differencing used at the endpoints, and centered differencing used everywhere else). If abscissa intervals are irregular, differencing will be correspondingly asym- metric.
Output Result: * Derivative of y_in with respect to x_in (or unit interval
abscissa, if x_in is not given). Numeric array of same shape and size as y_in. If there are missing values, those elements in the output are set to the value in |missing|. For instance, if y_in is only one element, a one-element vector is returned as the derivative with the value of |missing|. If there are missing values in the output due to math errors and |missing| is set to None, output will fill those missing values with the MA default value of 1e+20.
References: * Press, W. H., et al. (1992): Numerical Recipes in Fortran
77: The Art of Scientific Computing. New York, NY: Cambridge University Press, pp. 180-184.
Wang, Y. (1999): “Numerical Differentiation,” Introduction to MHD Numerical Simulation in Space, ESS265: Instrumentation, Data Processing and Data Analysis in Space Physics (UCLA). URL: http://www-ssc.igpp.ucla.edu/personnel/russell/ESS265/ Ch10/ylwang/node21.html.
Example with one argument, no missing values, using the default method: >>> from deriv import deriv >>> import Numeric as N >>> y = N.sin(N.arange(8)) >>> dydx = deriv(y) >>> [‘%.7g’ % dydx[i] for i in range(4)] [‘0.841471’, ‘0.4546487’, ‘-0.3501755’, ‘-0.83305’] >>> true = N.cos(N.arange(8)) #- Compare with exact solution >>> [‘%.7g’ % true[i] for i in range(4)] [‘1’, ‘0.5403023’, ‘-0.4161468’, ‘-0.9899925’]
Example with two arguments with missing values, using first- order differencing: >>> x = N.arange(8)/(2.*N.pi) >>> y = N.sin(x) >>> y[3] = 1e20 #- Set an element to missing value >>> dydx = deriv(x, y, missing=1e20, algorithm=’order1’) >>> [‘%.7g’ % dydx[i] for i in range(5)] [‘0.9957836’, ‘0.9831985’, ‘1e+20’, ‘0.8844179’, ‘1e+20’] >>> true = N.cos(x) #- Compare with exact solution >>> [‘%.7g’ % true[i] for i in range(5)] [‘1’, ‘0.9873616’, ‘0.9497657’, ‘0.8881628’, ‘0.8041098’]
-
has_close
(data, value, rtol=1e-05, atol=1e-08)¶ Test if data has any values “equal” to value.
Returns 1 if any of the elements of argument data has a value “equal” to argument value; returns 0 otherwise. If data or value is floating point, “equal” means where abs(data-value) <= atol + rtol * abs(value). This is essentially the same algorithm used in the Numeric function allclose. If data and value are integer, “equal” means strict equality.
Positional Input Arguments: * data: Data. Scalar or Numeric array, Python list/tuple of
any size and shape. Floating or integer type.
- value: Test value. Scalar or 1 element Numeric array, list,
or tuple. Floating or integer type.
Keyword Input Arguments: * rtol: “Relative” tolerance. Default is 1.e-5. Used in the
comparison between data and value only if the two are floating point. Floating or integer type.
- atol: “Absolute” tolerance. Default is 1.e-8. Used in the
comparison between data and value only if the two are floating point. Floating or integer type.
Examples: >>> from has_close import has_close >>> data = [20., -32., -1., 2., 5., 29.] >>> has_close(data, -1.) 1 >>> has_close(data, 10.) 0
-
relative
(u, v, x, y)¶