Loading a TimeSeries from a text file

The scikits.timeseries package provides a convenient way to create a TimeSeries object from the information stored in an ASCII file.

tsfromtxt(fname, dtype=None, freq='U', comments='#', delimiter=None, skip_header=0, skip_footer=0, skiprows=0, converters=None, dateconverter=None, missing='', missing_values=None, filling_values=None, usecols=None, datecols=None, names=None, excludelist=None, deletechars=None, autostrip=True, case_sensitive=True, defaultfmt='f%i', unpack=None, loose=True, asrecarray=False, invalid_raise=True)

Load a TimeSeries from a text file.

Each line of the input after the first skiprows ones is split at delimiter. Characters occuring after comments are discarded.

If a column is named 'dates' (case insensitive), it is used to define the dates. The freq parameter should be set to the expected frequency of the output series. If the date information spans several columns (for example, year in col #1, month in col #2...), a specific conversion function must be defined with the dateconverter parameter. This function should accept as many inputs as date columns, and return a valid Date object.

Parameters:

fname : file or string

File or filename to read. If the file extension is .gz or .bz2, the file is first decompressed.

dtype : data-type, optional

Data type of the resulting array. If it is a structured data-type, the resulting array is 1-dimensional, and each row is interpreted as an element of the array. In this case, the number of columns used must match the number of fields in the dtype and the names of each field are set by the corresponding name of the dtype. If None, the dtypes will be determined by the contents of each column, individually.

comments : {string}, optional

The character used to indicate the start of a comment. All the characters occurring on a line after a comment are discarded.

delimiter : {string}, optional

The string used to separate values. By default, any consecutive whitespace act as delimiter.

skip_header : int, optional

The numbers of lines to skip at the beginning of the file.

skip_footer : int, optional

The numbers of lines to skip at the end of the file

converters : variable or None, optional

The set of functions that convert the data of a column to a value. The converters can also be used to provide a default value for missing data: converters = {3: lambda s: float(s or 0)}.

dateconverter : {function}, optional

The function to convert the date information to a Date object. This function requires as many parameters as number of datecols. This parameter is mandatory if dtype=None.

missing_values : variable or None, optional

The set of strings corresponding to missing data.

filling_values : variable or None, optional

The set of values to be used as default when the data are missing.

usecols : sequence or None, optional

Which columns to read, with 0 being the first. For example, usecols = (1, 4, 5) will extract the 2nd, 5th and 6th columns.

datecols : {None, int, sequence}, optional

Which columns store the date information.

names : {None, True, str, sequence}, optional

If names is True, the field names are read from the first valid line after the first skiprows lines. If names is a sequence or a single-string of comma-separated names, the names will be used to define the field names in a structured dtype. If names is None, the names of the dtype fields will be used, if any.

excludelist : sequence, optional

A list of names to exclude. This list is appended to the default list [‘return’,’file’,’print’]. Excluded names are appended an underscore: for example, file would become file_.

deletechars : str, optional

A string combining invalid characters that must be deleted from the names.

defaultfmt : str, optional

A format used to define default field names, such as “f%i” or “f_%02i”.

autostrip : bool, optional

Whether to automatically strip white spaces from the variables.

case_sensitive : {True, False, ‘upper’, ‘lower’}, optional

If True, field names are case sensitive. If False or ‘upper’, field names are converted to upper case. If ‘lower’, field names are converted to lower case.

unpack : bool, optional

If True, the returned array is transposed, so that arguments may be unpacked using x, y, z = loadtxt(...)

usemask : bool, optional

If True, return a masked array. If False, return a regular array.

asrecarray : {False, True}, optional

Whether to return a TimeSeriesRecords or a series with a structured dtype.

invalid_raise : bool, optional

If True, an exception is raised if an inconsistency is detected in the number of columns. If False, a warning is emitted and the offending lines are skipped.

Returns:

out : MaskedArray

Data read from the text file.

See also

numpy.lib.io.genfromtxt

Equivalent function for standard arrays

Notes

• When spaces are used as delimiters, or when no delimiter has been given as input, there should not be any missing data between two fields.
• When the variable are named (either by a flexible dtype or with names, there must not be any header in the file (else a ValueError exception is raised).
• If names is True or a sequence of strings, these names overwrite the fields names of a structured array.
• The sequence of names must NOT take the date columns into account.
• If the datatype is not given explicitly (dtype=None), a dateconverter must be given explicitly.
• If the dtype is given explicitly, it must NOT refer to the date columns.

Examples

>>> data = "year, month, a, b\n 2001, 01, 0.0, 10.\n 2001, 02, 1.1, 11."
>>> dateconverter = lambda y, m: Date('M', year=int(y), month=int(m))
>>> series = tsfromtxt(StringIO.StringIO(data), delimiter=',', names=True,
...                    datecols=(0,1), dateconverter=dateconverter,)
>>> series
timeseries([(0.0, 10.0) (1.1, 11.0)],
   dtype = [('a', '<f8'), ('b', '<f8')],
   dates = [Jan-2001 Feb-2001],
   freq  = M)
>>> series = tsfromtxt(StringIO.StringIO(data), delimiter=",",
...                    datecols=(0, 1), dateconverter=dateconverter,
...                    names="A, B", skip_header=1)
timeseries([(0.0, 10.0) (1.1000000000000001, 11.0)],
   dtype = [('A', '<f8'), ('B', '<f8')],
   dates = [Jan-2001 Feb-2001],
   freq  = M)

Saving a TimeSeries to a text file

Several options are available to store a TimeSeries object to an ASCII file.

A first possibility is to use the numpy.savetxt function on a structured array, after having converted the object with the toflex method.

Another possibility is to use the Report class, described in the Reports section.