asi_core.utils.daily_h5¶

Classes¶

`DailyH5`	Base class for DailyH5 file manipulation
`DailyH5Writer`	Base class for DailyH5 file manipulation
`DailyH5Reader`	Base class for DailyH5 file manipulation

Module Contents¶

class asi_core.utils.daily_h5.DailyH5(products_path, meta_infos={})¶

Base class for DailyH5 file manipulation

products_path¶

meta_infos¶

daily_h5¶

get_file(timestamp)¶

Get the path to the current daily h5 file. Initialize if not done yet.

Parameters:: timestamp – Timestamp of the data to be stored
Returns:: Path to the current daily h5 file

init_h5file(timestamp)¶

Initialize daily h5 file for reading or writing.

Parameters:: timestamp – Timestamp of the current data to be stored

process_entry(timestamp, mode, data=None, timestamp_forecasted=None)¶

Stores a dataset of one timestamp to the daily h5 file

Parameters:

timestamp – Timestamp based on which dataset was created
mode – character, r/w/a, i.e. read, write or append
data – Dataset to be saved either dataset which can be stored by h5py or dict of such datasets
timestamp_forecasted – (Optional) timestamp forecasted by the dataset

process_sub_entry(label, data=None)¶

Defines the read/ write operation to be applied recursively

Parameters:

label – Label of the current data to be stored/ read
data – Data to be processed

class asi_core.utils.daily_h5.DailyH5Writer(products_path, meta_infos={})¶

Bases: DailyH5

Base class for DailyH5 file manipulation

store_entry(timestamp, data, timestamp_forecasted=None)¶

Stores a dataset of one timestamp to the daily h5 file

Parameters:

timestamp – Timestamp based on which dataset was created
timestamp_forecasted – (Optional) timestamp forecasted by the dataset
data – Dataset to be saved either dataset which can be stored by h5py or dict of such datasets

init_h5file(timestamp, do_not_overwrite=True)¶

Initialize daily h5 file, create folders if required, store meta infos to a new h5 file.

Parameters:

timestamp – Timestamp of the current data to be stored
do_not_overwrite – If called and a daily file already exists, create additional file instead of overwriting the previous one.

process_sub_entry(handle, label, data)¶

Recursively store all datasets in data

Parameters:

handle – Handle to an h5file or a group in an h5 file
label – Label under which data will be stored
data – dataset or dict of datasets

class asi_core.utils.daily_h5.DailyH5Reader(products_path, meta_infos={})¶

Bases: DailyH5

Base class for DailyH5 file manipulation

static list_entries(h5_path)¶

Generate a dataframe of the keys and corresponding timestamps and forecasted timestamps in the h5 file.

Parameters:: h5_path – Path of the h5 file, the ke
Returns:: Dataframe with columns key, timestamp, forecasted_timestamp

static init_from_path(timestamp, h5_path)¶

Create a DailyH5Reader instance and initializes it from a specific h5 file

Parameters:

timestamp – Timestamp based on which dataset was created
h5_path – Path of the h5 file to be read (naming can deviate from convention of DailyH5Writer)

Returns:

DailyH5Reader instance

get_entry(timestamp, timestamp_forecasted=None)¶

Stores a dataset of one timestamp to the daily h5 file

Parameters:

timestamp – Timestamp based on which dataset was created
timestamp_forecasted – (Optional) timestamp forecasted by the dataset

Returns:

Dataset to be saved either dataset which can be stored by h5py or dict of such datasets

process_sub_entry(handle, label, data=None)¶

Recursively get all datasets in handle

Parameters:

handle – Handle to an h5file or a group in an h5 file
label – Label under which data will be stored
data – dataset or dict of datasets

init_h5file(timestamp, file_counter=0)¶

Initialize daily h5 file, load meta data.

Parameters:

timestamp – Timestamp of the current data to be stored
file_counter – Appends counter suffix to file name. May be useful if multiple files created for a day.