asi_core.basics¶

This module provides basic helper functions.

Classes¶

`DailyH5`	Base class for DailyH5 file manipulation
`DailyH5Writer`	Base class for DailyH5 file manipulation
`DailyH5Reader`	Base class for DailyH5 file manipulation

Functions¶

`ifnone`(var, otherwise)	If a variable's original value is None, replace it with an alternative value
`get_absolute_path`(filepath[, root, as_string])	Combine root and relative path and resolve to absolute path.
`replace_double_backslashes_with_slashes_in_path`(str_path)	Replace double backslashes from windows paths with slashes.
`_get_files`(p, fs[, extensions, substring])	Get all files in path with 'extensions' and a name containing a 'substring'.
`get_files`(path[, extensions, substring, recursive, ...])	Get all files in path with optional extensions or substring, optionally recursive, only in folders,
`get_image_files`(path[, recursive, folders, ...])	Get image files in path recursively, only in folders and with substring, if specified.
`copy_file`(src_file[, tgt_file, tgt_dir, create_parents])	Copies a file to a specified target file or directory.
`copy_file_relative_to_directories`(rel_filepath, ...)	Copies a file from the source directory to the target directory while preserving its relative path.
`parallel_copy_files`(filepaths, src_dir, tgt_dir[, ...])	Copies multiple files in parallel, optionally preserving directory structure.
`get_number_of_nans`(df)	Count number of nans in a dataframe columns-wise and total number of rows with a NaN value in at least one column.
`get_ETC_GMT_timezone`([desired_timezone])	Return a pytz timezone object corresponding to the desired Etc/GMT timezone.
`parse_datetime`(dt_string[, datetime_format])	Extracts timestamp from filename.
`get_temporal_resolution_from_timeseries`(data)	Get temporal resolution from a pandas DataFrame with a DatetimeIndex.
`assemble_path`(path_structure, camera_name, timestamp)	Assemble path to images or other files and replace timestamp and camera name 'tags' with actual values
`fstring_to_re`(string)	Convert from f-string syntax to regular expression syntax
`adjust_gamma`(image[, gamma])	Only for improved visibility, for radiometric evaluations reconsider

Module Contents¶

asi_core.basics.ifnone(var, otherwise)¶: If a variable’s original value is None, replace it with an alternative value

asi_core.basics.get_absolute_path(filepath, root=None, as_string=False)¶: Combine root and relative path and resolve to absolute path.

asi_core.basics.replace_double_backslashes_with_slashes_in_path(str_path, root_dir=None)¶: Replace double backslashes from windows paths with slashes.

asi_core.basics._get_files(p, fs, extensions=None, substring=None)¶: Get all files in path with ‘extensions’ and a name containing a ‘substring’.

asi_core.basics.get_files(path, extensions=None, substring=None, recursive=True, folders=None, followlinks=True)¶: Get all files in path with optional extensions or substring, optionally recursive, only in folders, if specified.

asi_core.basics.get_image_files(path, recursive=True, folders=None, extensions=IMAGE_EXTENSIONS, substring=None)¶: Get image files in path recursively, only in folders and with substring, if specified.

asi_core.basics.copy_file(src_file, tgt_file=None, tgt_dir=None, create_parents=False)¶

Copies a file to a specified target file or directory.

Parameters:

src_file – Path to the source file.
tgt_file – Path to the target file (optional if tgt_dir is provided).
tgt_dir – Path to the target directory (optional if tgt_file is provided).
create_parents – Whether to create parent directories if they do not exist (default: False).

asi_core.basics.copy_file_relative_to_directories(rel_filepath, src_dir, tgt_dir)¶

Copies a file from the source directory to the target directory while preserving its relative path.

Parameters:

rel_filepath – File path relative to the source directory.
src_dir – Root source directory.
tgt_dir – Root target directory.

asi_core.basics.parallel_copy_files(filepaths, src_dir, tgt_dir, keep_dir_structure=False, num_workers=0)¶

Copies multiple files in parallel, optionally preserving directory structure.

Parameters:

filepaths – List of file paths to copy.
src_dir – Source directory (used to determine relative paths).
tgt_dir – Target directory where files will be copied.
keep_dir_structure – Whether to maintain the original directory structure (default: False).
num_workers – Number of parallel workers for file copying (default: 0, meaning sequential execution).

asi_core.basics.get_number_of_nans(df)¶: Count number of nans in a dataframe columns-wise and total number of rows with a NaN value in at least one column.

asi_core.basics.get_ETC_GMT_timezone(desired_timezone='GMT')¶

Return a pytz timezone object corresponding to the desired Etc/GMT timezone.

Example: UTC+2: get_ETC_GMT_timezone(‘GMT+2’) returns pytz timezone object for “Etc/GMT-2”

Parameters:: desired_timezone – (str) A string specifying the desired GMT timezone. It should be in the format ‘GMT+1’ (in the case of UTC+1). If a ‘+’ sign is used, it will be replaced with ‘-’ to follow the Etc/GMT convention (https://en.wikipedia.org/wiki/Tz_database#Area).
Returns:: pytz.timezone, Etc/GMT timezone

asi_core.basics.parse_datetime(dt_string, datetime_format='%Y%m%d%H%M%S')¶: Extracts timestamp from filename.

asi_core.basics.get_temporal_resolution_from_timeseries(data)¶: Get temporal resolution from a pandas DataFrame with a DatetimeIndex.

asi_core.basics.assemble_path(path_structure, camera_name, timestamp, set_subday_to_wildcard=False, exposure_time=None)¶

Assemble path to images or other files and replace timestamp and camera name ‘tags’ with actual values

Parameters:

path_structure – (str) path to each image, containing {camera_name} where the camera name should be inserted and {timestamp:…} (e.g. {timestamp:%Y%m%d%H%M%S%f}) where the evaluated timestamp should be inserted
camera_name – (str) Name of the camera as specified in config file and used in (image) folder structure
timestamp – (datetime, tz-aware) Timestamp for which an (image) file is requested
set_subday_to_wildcard – (bool) If True, replace formatters indicating hours, minutes etc. with wildcards
exposure_time – (int) exposure time of images has to be set if set_subday_to_wildcard is False

Return assemble_path:

(str) assembled path

asi_core.basics.fstring_to_re(string)¶

Convert from f-string syntax to regular expression syntax

Only a limited set of formatters supported so far, FSTRING_RE should be extended as needed.

class asi_core.basics.DailyH5(products_path, meta_infos={})¶

Base class for DailyH5 file manipulation

products_path¶

meta_infos¶

daily_h5¶

get_file(timestamp)¶

Get the path to the current daily h5 file. Initialize if not done yet.

Parameters:: timestamp – Timestamp of the data to be stored
Returns:: Path to the current daily h5 file

init_h5file(timestamp)¶

Initialize daily h5 file for reading or writing.

Parameters:: timestamp – Timestamp of the current data to be stored

process_entry(timestamp, mode, data=None, timestamp_forecasted=None)¶

Stores a dataset of one timestamp to the daily h5 file

Parameters:

timestamp – Timestamp based on which dataset was created
mode – character, r/w/a, i.e. read, write or append
data – Dataset to be saved either dataset which can be stored by h5py or dict of such datasets
timestamp_forecasted – (Optional) timestamp forecasted by the dataset

process_sub_entry(label, data=None)¶

Defines the read/ write operation to be applied recursively

Parameters:

label – Label of the current data to be stored/ read
data – Data to be processed

class asi_core.basics.DailyH5Writer(products_path, meta_infos={})¶

Bases: DailyH5

Base class for DailyH5 file manipulation

store_entry(timestamp, data, timestamp_forecasted=None)¶

Stores a dataset of one timestamp to the daily h5 file

Parameters:

timestamp – Timestamp based on which dataset was created
timestamp_forecasted – (Optional) timestamp forecasted by the dataset
data – Dataset to be saved either dataset which can be stored by h5py or dict of such datasets

init_h5file(timestamp, do_not_overwrite=True)¶

Initialize daily h5 file, create folders if required, store meta infos to a new h5 file.

Parameters:

timestamp – Timestamp of the current data to be stored
do_not_overwrite – If called and a daily file already exists, create additional file instead of overwriting the previous one.

process_sub_entry(handle, label, data)¶

Recursively store all datasets in data

Parameters:

handle – Handle to an h5file or a group in an h5 file
label – Label under which data will be stored
data – dataset or dict of datasets

class asi_core.basics.DailyH5Reader(products_path, meta_infos={})¶

Bases: DailyH5

Base class for DailyH5 file manipulation

static list_entries(h5_path)¶

Generate a dataframe of the keys and corresponding timestamps and forecasted timestamps in the h5 file.

Parameters:: h5_path – Path of the h5 file, the ke
Returns:: Dataframe with columns key, timestamp, forecasted_timestamp

static init_from_path(timestamp, h5_path)¶

Create a DailyH5Reader instance and initializes it from a specific h5 file

Parameters:

timestamp – Timestamp based on which dataset was created
h5_path – Path of the h5 file to be read (naming can deviate from convention of DailyH5Writer)

Returns:

DailyH5Reader instance

get_entry(timestamp, timestamp_forecasted=None)¶

Stores a dataset of one timestamp to the daily h5 file

Parameters:

timestamp – Timestamp based on which dataset was created
timestamp_forecasted – (Optional) timestamp forecasted by the dataset

Returns:

Dataset to be saved either dataset which can be stored by h5py or dict of such datasets

process_sub_entry(handle, label, data=None)¶

Recursively get all datasets in handle

Parameters:

handle – Handle to an h5file or a group in an h5 file
label – Label under which data will be stored
data – dataset or dict of datasets

init_h5file(timestamp, file_counter=0)¶

Initialize daily h5 file, load meta data.

Parameters:

timestamp – Timestamp of the current data to be stored
file_counter – Appends counter suffix to file name. May be useful if multiple files created for a day.

asi_core.basics.adjust_gamma(image, gamma=1.0)¶

Only for improved visibility, for radiometric evaluations reconsider

Taken from https://pyimagesearch.com/2015/10/05/opencv-gamma-correction/

Build a lookup table mapping the pixel values [0, 255] to their adjusted gamma values and apply it.

Parameters:

image – Input RGB image
gamma – Gamma scalar parameter

Returns:

Gamma-corrected image

asi_core.basics¶

Classes¶

Functions¶

Module Contents¶

Table of Contents

Previous topic

Next topic

This Page