phildb package

Submodules

phildb.console module

phildb.constants module

phildb.create module

phildb.create.create(tsdb_path)[source]
phildb.create.main()[source]

phildb.database module

class phildb.database.PhilDB(tsdb_path)[source]

Bases: object

add_attribute(attribute_id, description)[source]

Define an attribute.

Parameters:
  • attribute_id (string) – Identifier of the attribute.
  • description (string) – Description of the attribute.
add_attribute_value(attribute_id, value)[source]

Store an attribute value.

Parameters:
  • attribute_id (string) – Identifier of the attribute.
  • value (string) – The attribute value to store.
add_measurand(measurand_short_id, measurand_long_id, description)[source]

Create a measurand entry.

Measurand being a measurable timeseries type. e.g. Streamflow, Temperature, Rainfall, etc.

Parameters:
  • measurand_short_id (string) – Short identifier of the measurand.
  • measurand_long_id (string) – Long identifier of the measurand.
  • description (string) – Description of the measurand.
add_source(source, description)[source]

Define a source.

Source being the origin of the data. For example the source used in the examples/hrs example is BOM_HRS. Indicated the origin of the data was the Bureau of Metorology Hydrologic Reference Stations project.

Parameters:
  • source (string) – Identifier of the source.
  • description (string) – Description of the source.
add_timeseries(identifier)[source]

Create a timeseries entry to be identified by the supplied ID.

Parameters:identifier (string) – Identifier of the timeseries.
add_timeseries_instance(identifier, freq, initial_metadata, **kwargs)[source]

Define an instance of a timeseries.

A timeseries instance is a combination of a timeseries, frequency and attributes.

Parameters:
  • identifier (string) – Identifier of the timeseries.
  • freq (string) – Data frequency (e.g. ‘D’ for day, as supported by pandas.)
  • initial_metadata (string) – Store some metadata about this series. Potentially freeform header from a source file about to be loaded.
  • **kwargs (kwargs) – Any additional attributes to attach to the timeseries instance.
get_file_path(identifier, freq, ftype='tsdb', **kwargs)[source]

Get a path to a file for a given timeseries instance.

Parameters:
  • identifier (string) – Identifier of the timeseries.
  • ftype (string) – File extension to use (i.e. the type of file). (Default=’tsdb’)
Returns:

string – Path to file for a timeseries instance identified by the given arguments.

help()[source]

List methods of the PhilDB class with the first line of their docstring.

list_ids()[source]

Returns list of timeseries IDs for all timeseries records.

Returns:list(string) – Sorted list of timeseries identifiers.
list_measurands()[source]

Returns list of measurand short IDs for all measurand records.

Returns:list(string) – Sorted list of timeseries identifiers.
list_sources()[source]

Returns list of source IDs for all sources.

Returns:list(string) – Sorted list of source identifiers.
list_timeseries_instances(**kwargs)[source]

Returns list of timeseries instances for all instance records.

Can filter by using keyword arguments.

Returns:list(string) – Sorted list of timeseries instances.
read(identifier, freq, **kwargs)[source]

Read the entire timeseries record for the requested timeseries instance.

Parameters:
  • identifier (string) – Identifier of the timeseries.
  • freq (string) – Timeseries data frequency.
  • kwargs (kwargs) – Attributes to match against timeseries instances (e.g. source, measurand).
Returns:

pandas.DataFrame – Timeseries data.

read_all(freq, excludes=None, **kwargs)[source]

Read the entire timeseries record for all matching timeseries instances. Optionally exclude timeseries from the final DataFrame by specifying IDs in the exclude argument.

Parameters:
  • identifier (string) – Identifier of the timeseries.
  • freq (string) – Timeseries data frequency.
  • excludes (array[string]) – IDs of timeseries to exclude from final DataFrame.
  • kwargs (kwargs) – Attributes to match against timeseries instances (e.g. source, measurand).
Returns:

pandas.DataFrame – Timeseries data.

read_dataframe(identifiers, freq, **kwargs)[source]

Read the entire timeseries record for the requested timeseries instances.

Parameters:
  • identifiers (array[string]) – Identifiers of the timeseries to read into a DataFrame.
  • freq (string) – Timeseries data frequency.
  • kwargs (kwargs) – Attributes to match against timeseries instances (e.g. source, measurand).
Returns:

pandas.DataFrame – Timeseries data.

read_log(identifier, freq, as_at_datetime, **kwargs)[source]

Read timeseries record for the requested timeseries instance as it was at specified datetime in the log.

Parameters:
  • identifier (string) – Identifier of the timeseries.
  • freq (string) – Timeseries data frequency.
  • as_at_datetime (datetime) – Filter to a timeseries, as available at this specified datetime, from the log.
  • kwargs (kwargs) – Attributes to match against timeseries instances (e.g. source, measurand).
Returns:

pandas.DataFrame – Timeseries data.

read_metadata(ts_id, freq, **kwargs)[source]

Returns the metadata that was associated with an initial TimeseriesInstance.

Parameters:identifier (string) – Identifier of the timeseries.
Returns:string – The initial metadata that was recorded on instance creation.
ts_list(**kwargs)[source]

Returns list of primary ID for all timeseries records.

Parameters:kwargs (kwargs) – Restrict to records associated with this the kwargs attributes supplied. (Optional).
Returns:list(string) – Sorted list of timeseries identifiers.
version()[source]

Returns the version number of the database schema.

Returns:string – Schema version.
write(identifier, freq, ts, **kwargs)[source]

Write/update timeseries data for existing timeseries.

Parameters:
  • identifier (string) – Identifier of the timeseries.
  • freq (string) – Data frequency (e.g. ‘D’ for day, as supported by pandas.)
  • ts (pd.Series) – Timeseries data to write into the database.

phildb.exceptions module

exception phildb.exceptions.AlreadyExistsError[source]

Bases: exceptions.Exception

exception phildb.exceptions.DataError[source]

Bases: exceptions.Exception

exception phildb.exceptions.DuplicateError[source]

Bases: exceptions.Exception

exception phildb.exceptions.MissingAttributeError[source]

Bases: exceptions.Exception

exception phildb.exceptions.MissingDataError[source]

Bases: exceptions.Exception

phildb.log_handler module

class phildb.log_handler.LogHandler(filename, mode)[source]
FILTERS = <Mock name='mock.Filters()' id='140001761522064'>
create_skeleton()[source]

Create the skeleton of the log self.hdf5.

read(as_at_datetime)[source]
write(log_entries, operation_datetime)[source]

phildb.reader module

phildb.reader.read(filename)[source]
phildb.reader.read_log(log_file, as_at_datetime)[source]

phildb.writer module

phildb.writer.write(tsdb_file, ts, freq)[source]

Smart write.

Will only update existing values where they have changed. Changed existing values are returned in a list.

Parameters:
  • tsdb_file (string) – File to write timeseries data into.
  • ts (pd.Series) – Timeseries data to write.
  • freq (string) – Frequency of the data. (e.g. ‘D’ for daily, ‘1Min’ for minutely). Accepts any string that pandas.TimeSeries.asfreq does or ‘IRR’ for irregular data.
phildb.writer.write_irregular_data(tsdb_file, series)[source]

Smart write of irregular data.

Will only update existing values where they have changed. Changed existing values are returned in a list.

Parameters:
  • tsdb_file (string) – File to write timeseries data into.
  • series (pandas.Series) – Pandas Series of irregular data to write.
phildb.writer.write_log(log_file, modified, replacement_datetime)[source]
phildb.writer.write_regular_data(tsdb_file, series)[source]

Smart write. Expects continuous time series.

Will only update existing values where they have changed. Changed existing values are returned in a list.

Parameters:
  • tsdb_file (string) – File to write timeseries data into.
  • series (pandas.Series) – Pandas Series of regular data to write.

Module contents