asi_core.meteo.meteo

This module is for handling and organizing meteorological measurement data.

Classes

MeteoData	Class for meteo time series data.
MesorMat	Class for creating MeteoData objects from Mesor files.
CSOnline	Class for retrieving real time meteo data from CS logger

Functions

_ineichen_linke_turbidity_from_dni(dni, altitude, ...)	Computes the Linke turbidity factor using the Ineichen equation for clear-sky DNI and DNI measurements.
linke_turbidity_ineichen_dlr(dni, altitude, sun_el, ...)	Computes the Linke turbidity factor using the Ineichen DLR method.
compute_dni_variability_classes(dni, clear_sky_dni[, ...])	DNI variability procedure for time series in 1 min resolution according to: Schroedter-Homscheidt et al.,
aggregate_temporal_resolution(meteo_data, new_freq, ...)	Averages meteo data to lower time resolution rejecting all low-resolution periods with incomplete data.
compute_earth_sun_distance_correction_factor(day_of_year)	Calculate distance between sun and earth acoording to Liou, An Introduction to Atmospheric Radiation,
time_dependent_envelope_curve(dni_series)	Get upper and lower DNI envelope curves according toSchroedter-Homscheidt et al. 2018
find_valid_extrema(dni_series, local_extrema_series[, ...])	Function to detect valid maxima or minima within a lower and upper time limit for a time window. This is according
trapezoidal_integral_over_time_window(window)	Function to compute the trapezoidal integral over a given time window. The time window must have
variability_index(k_dni_diff, time_period)	Calculates the Variability Index (V) over a specified time period, according to Coimbra et al. (2013)
variability_index_indicator(dni_diff, ...)	Calculates the Variability Index Indicator (VI) over a specified time period, according to Stein et al. (2012)
plot_series(series)	Plot a timeseries with a datetime index. It generates a simple line plot of the series (helpful for debugging).
check_daily_index_continuity(ts[, temporal_res])	Check if index (pd.DatetimeIndex) has gaps.
read_mesor_txt_file(file_path)	Read meteo data from mesor txt file.
convert_invalid_values_to_nan(data, invalid_value[, ...])	Converts invalid values in dataframe to nans.
get_missing_timestamps(dt, temporal_resolution, tz[, ...])	Identifies missing timestamps within a given datetime range based on a specified temporal resolution.
get_data_availability_stats(ts_missing, ts_available)	Returns a DataFrame containing statistics about data availability, comparing missing and available timestamps.
get_number_of_nans(df)	Count number of nans in a dataframe columns-wise and total number of rows with a NaN value in at least one column.

Module Contents

class asi_core.meteo.meteo.MeteoData(name=None, latitude=None, longitude=None, altitude=None, tz=None, data=None, temporal_resolution=None)

Bases: object

Class for meteo time series data.

columns = ['ghi', 'dni', 'dhi', 't_amb', 'p_amb', 'rel_humid', 'wind_dir', 'wind_speed', 'sun_el',...

units

data

name = None

latitude = None

longitude = None

altitude = None

tz = None

temporal_resolution = None

get_p_amb(use_default_for_nans=True)

Gets ambient pressure from self.data

get_solar_pos(recompute=False)

Gets solar position from self.data

get_airmass(absolute=True, recompute=False)

Gets airmass from self.data

get_linke_turbidity(recompute=False, method_linke_turbidity='mcclear')

Gets linke turbidity from self.data

get_clear_sky_irradiance(recompute=False, method_clear_sky_irradiance='ineichen', method_linke_turbidity='mcclear')

Gets clear sky irradiance from self.data.

get_dni_variability_classes()

Gets dni variability classes from self.data.

remove_duplicates(keep='first', inplace=False)

Removes rows in dataframes with duplicated indexes.

insert_missing_timestamps(inplace=False, min_ele=None, method_solar_pos='nrel_numpy')

Inserts missing timestamps

compute()

compute_solar_position(method='nrel_numpy', apparent_elevation=True)

Computes solar position using pvlib.

compute_airmass(method_airmass='kastenyoung1989', fill_default_pressure=True)

Computes airmass using pvlib.

compute_linke_turbidity(method_linke_turbidity='ineichen_dlr', solar_constant=1361.1, limit_sun_elevation=10, limit_dni=300, variability_tl_lim=0.01, tl_cloud_limit=8)

Computes linke turbidity using pvlib.

compute_clear_sky_irradiance(method_clear_sky_irradiance='ineichen', method_linke_turbidity='ineichen_dlr', solar_constant=1364)

Computes the clear sky irradiance using pvlib.

This method calculates the clear sky Global Horizontal Irradiance (GHI), Direct Normal Irradiance (DNI), and Diffuse Horizontal Irradiance (DHI) based on solar position, airmass, and Linke turbidity factors. It uses the specified clear sky irradiance model (currently supports the ‘ineichen’ model) and Linke turbidity method.

The computed values are stored in the self.data DataFrame under the columns: - ‘clear_sky_ghi’: Clear sky GHI (W/m²) - ‘clear_sky_dni’: Clear sky DNI (W/m²) - ‘clear_sky_dhi’: Clear sky DHI (W/m²)

Additionally, the method applies masking rules to adjust clear sky estimates: - If measured GHI exceeds 1.4 times the computed clear sky GHI, it replaces the clear sky GHI with measured GHI. - If measured DNI exceeds the computed clear sky DNI, it replaces the clear sky DNI with measured DNI. - Clear sky DHI is recomputed as the difference between GHI and DNI, adjusted by the solar elevation angle.

Parameters:

• method_clear_sky_irradiance (str) – Clear sky model to use (currently only ‘ineichen’ is supported).

• method_linke_turbidity (str) – Method for calculating Linke turbidity (‘ineichen_dlr’ is recommended).

• solar_constant (float) – Value of the solar constant (W/m²), default is 1364 W/m².

Returns:

DataFrame with columns ‘clear_sky_ghi’, ‘clear_sky_dni’, ‘clear_sky_dhi’.

Return type:

pandas.DataFrame

Raises:

NotImplementedError – If an unsupported clear sky model is requested.

compute_dni_var_class(time_period='15T')

DNI variability procedure for time series in 1 min resolution according to: Schroedter-Homscheidt et al., Classifying ground-measured 1 minute temporal variability within hourly intervals for direct normal irradiances, Meteorologische Zeitschrift, (2018) :param time_period: used for classification :return: self.data[‘dni_var_class’]: (pd series)

interpolate(limit, columns=None, method='index', limit_area='inside', limit_direction='both', inplace=False)

Interpolates missing data in dataframe.

filter_by_sun_el(min_sun_el, inplace=False)

Filters data by sun elevation.

get_temporal_resolution()

Gets temporal resolution of dataframe

get_number_of_nans()

plot_distributions(columns=None, n_rows=2, df_ref=None, figsize=(20, 12), **kwargs)

reduce_temporal_resolution(new_freq, min_fraction=60 / 60)

Averages meteo data to lower time resolution rejecting all low-resolution periods with incomplete data.

Parameters:

• new_freq – (str, datetime.timedelta, or DateOffset), new temporal resolution, to be understood as “freq” by pandas

• min_fraction – (float) Minimum fraction of valid readings required to retain average of a period.

check_daily_index_continuity()

classmethod from_csv(csv_file, delimiter=',', header='infer', rename_columns={}, **kwargs)

Initialize MeteoData object from csv file with measurements.

Parameters:

• csv_file (str) – csv file from which csv should be initialized

• delimiter (str) – argument as used by pd.read_csv

• header (int or str or None) – argument as used by pd.read_csv

• rename_columns (dict) – falsy or mapping from column names in csv file to column names expected by cls

Params kwargs:

keyword arguments accepted by the cls init method

Returns:

cls instance

classmethod from_pangaea_tab(tab_file, delimiter=',', header='infer', rename_columns={}, **kwargs)

Initialize MeteoData object from csv file with measurements downloaded from Pangea.

Tested so far with this dataset: Fabel, Yann; Blum, Niklas; Nouri, Bijan; Wilbert, Stefan; Gonzalez Rodriguez, Sergio; Zarzalejo, Luis (2025): SolarVision Almería: Meteorological measurements [dataset]. PANGAEA, https://doi.org/10.1594/PANGAEA.980085

Parameters:

• tab_file (str) – tab-separated table file from which csv should be initialized

• delimiter (str) – argument as used by pd.read_csv

• header (int or str or None) – argument as used by pd.read_csv

• rename_columns (dict) – falsy or mapping from column names in csv file to column names expected by cls

Params kwargs:

keyword arguments accepted by the cls init method

Returns:

cls instance

classmethod from_mesor_txt(txt_file)

Initialize MeteoData object from mesor txt file.

class asi_core.meteo.meteo.MesorMat(mesor_files: list, remove_duplicates=True, insert_missing_timestamps=True, min_sun_el=0, interpolate=True, interpolate_limit=10, temporal_resolution=None, mesor_nan=-9999, compute_solar_position=True, compute_clear_sky_irradiance=True, compute_dni_var_class=False, time_period_dni_var_class='15T', tl_files=None, dni_var_class_files=None, method_clear_sky_irradiance='ineichen', method_linke_turbidity='mcclear', file_type_dni_var_class='.mat')

Bases: MeteoData

Class for creating MeteoData objects from Mesor files.

mesor_files

add_linke_turbidity(tl_files)

Adds linke turbidity from mat files to data.

add_dni_variability_classes(dnivc_files, file_type_dni_var_class='.mat')

Adds dni variability classes from mat files to data.

class asi_core.meteo.meteo.CSOnline(url_cs_logger_table: str, latitude, longitude, altitude, timezone, name_desired_columns_cs_table=None)

Bases: MeteoData

Class for retrieving real time meteo data from CS logger

url_cs_logger_table

latitude

longitude

altitude

tz

name_desired_columns_cs_table = None

update_meteo_data_real_time(compute_solar_position=True, compute_clear_sky_irradiance=True, method_clear_sky_irradiance='ineichen', method_linke_turbidity='ineichen_dlr', solar_constant=1364)

This method updates the meteorological data stored in the object in real-time by fetching data from a CS logger table and optionally computing solar position and clear sky irradiance.

Parameters:

• compute_solar_position – (bool, optional) Whether to compute solar position data (default is True)

• compute_clear_sky_irradiance – (bool, optional) Whether to compute clear sky irradiance (default is True)

• method_clear_sky_irradiance (str) – Clear sky model to use (currently only ‘ineichen’ is supported).

• method_linke_turbidity (str) – Method for calculating Linke turbidity (‘ineichen_dlr’ is recommended).

• solar_constant (float) – Value of the solar constant (W/m²), default is 1364 W/m².

Returns:

asi_core.meteo.meteo._ineichen_linke_turbidity_from_dni(dni, altitude, airmass_abs, solar_constant=1361.1)

Computes the Linke turbidity factor using the Ineichen equation for clear-sky DNI and DNI measurements. Refer to https://doi.org/10.1016/S0038-092x(02)00045-2 for details on the equations.

Parameters:

• dni – (pd.Series) Measured DNI in W/m²

• altitude – (float) Altitude of the location in meters above sea level.

• airmass_abs – (pd.Series) Absolute airmass values, indexed by datetime.

• solar_constant – (float) Solar constant in W/m^2

Returns:

linke_turbidity: (pd.Series) Linke turbidity factors soley based on DNI measurements.

asi_core.meteo.meteo.linke_turbidity_ineichen_dlr(dni, altitude, sun_el, airmass_abs, solar_constant=1361.1, limit_sun_elevation=20, limit_dni=300, variability_tl_lim=0.01, tl_cloud_limit=8)

Computes the Linke turbidity factor using the Ineichen DLR method. This method estimates the Linke turbidity, which describes atmospheric turbidity based on DNI and atmospheric conditions, and applies various filters and adjustments to derive a daily mean turbidity.

Parameters:

• dni – (pd.Series) Direct normal irradiance (DNI) values, indexed by datetime.

• altitude – (float) Altitude of the location in meters above sea level.

• sun_el – (pd.Series) Solar elevation angle in degrees, indexed by datetime.

• airmass_abs – (pd.Series) Absolute airmass values, indexed by datetime.

• solar_constant – (float) Solar constant in W/m^2

• limit_sun_elevation – (float) Minimum sun elevation angle in degrees for filtering

• limit_dni – (float) Minimum DNI threshold for filtering

• variability_tl_lim – (float) Maximum allowed variability in turbidity factor

• tl_cloud_limit – (float) Maximum Linke turbidity threshold for clear-sky filtering

Returns:

linke_turbidity: (pd.Series) Processed and filtered Linke turbidity, indexed by datetime.

asi_core.meteo.meteo.compute_dni_variability_classes(dni, clear_sky_dni, time_period='15T')

DNI variability procedure for time series in 1 min resolution according to: Schroedter-Homscheidt et al., Classifying ground-measured 1 minute temporal variability within hourly intervals for direct normal irradiances, Meteorologische Zeitschrift, (2018) :param dni: (pd.Series) Series of dni values with pd.DatetimeIndex. :param clear_sky_dni: (pd.Series) Series of clear-sky dni values with pd.DatetimeIndex. :param time_period: (str) Time period used for classification. :return: dni_var_class: (pd series) Series of dni variability classes with pd.DatetimeIndex

asi_core.meteo.meteo.aggregate_temporal_resolution(meteo_data, new_freq, old_freq, min_fraction=60 / 60)

Averages meteo data to lower time resolution rejecting all low-resolution periods with incomplete data.

Based on pyranocam/validation_thesis – data_post_preprocessing.generate_dif_time_res

Parameters:

• meteo_data – (pd DataFrame) define the dataframe, which contains the data in a specific time frequency (defined by ‘new_freq’-argument).

• new_freq – (str, datetime.timedelta, or DateOffset), new temporal resolution, to be understood as “freq” by pandas

• old_freq – (str, datetime.timedelta, or DateOffset), old temporal resolution, to be understood as “freq” by pandas

• min_fraction – (float) Minimum fraction of valid readings required to retain average of a period.

asi_core.meteo.meteo.compute_earth_sun_distance_correction_factor(day_of_year)

Calculate distance between sun and earth acoording to Liou, An Introduction to Atmospheric Radiation, 2002, p.49 :param day_of_year: (int, np.ndarray) (vector of) day(s) of year as Integer [1,366] :return: List with the following entries

• earth_sun_distance: Earth to Sun distance in meters

• earth_sun_corr_factor: Correction factor for Earth to Sun distance depending on the day of the year.

• solar_disk_radius: Angular radius of solar disk

asi_core.meteo.meteo.time_dependent_envelope_curve(dni_series)

Get upper and lower DNI envelope curves according toSchroedter-Homscheidt et al. 2018 :param dni_series: (pd series) dni with datetimeindex in 1 min resolution :return: upper_envelope: (pd series) upper envelope curve fitting to dni_series :return: upper_envelope: (pd series) lower envelope curve fitting to dni_series

asi_core.meteo.meteo.find_valid_extrema(dni_series, local_extrema_series, extrema_type='maxima', lower_time_limit=4, upper_time_limit=10)

Function to detect valid maxima or minima within a lower and upper time limit for a time window. This is according to the procedure described in Schroedter-Homscheidt et al. 2018 :param dni_series: (pd series) dni with datetimeindex in 1 min resolution :param local_extrema_series: (pd series) The series with local extrema (maxima or minima) filled. :param extrema_type: (string) either ‘maxima’ or ‘minima’ to specify the type of extrema to detect. :return: extrema_series (pd series) valid extrema

asi_core.meteo.meteo.trapezoidal_integral_over_time_window(window)

Function to compute the trapezoidal integral over a given time window. The time window must have a datetime index with 1-minute resolution.

Parameters:

window – (pd.Series) A time series with a datetime index and corresponding values for integration.

Returns:

(float) The computed integral value over the time window in hours, or NaN .

asi_core.meteo.meteo.variability_index(k_dni_diff, time_period)

Calculates the Variability Index (V) over a specified time period, according to Coimbra et al. (2013) This index provides insights on the variability of direct normal irradiance (DNI) compared to clear sky conditions.

Parameters:

• k_dni_diff – (pd.Series) The difference in clearness index (K) values over time.

• time_period – (str) A time-based window string (e.g., ‘15T’) used for rolling window calculations.

Returns:

v (pd.Series) variability index V: according to Coimbra et al. (2013)

asi_core.meteo.meteo.variability_index_indicator(dni_diff, clear_sky_dni_diff, time_period)

Calculates the Variability Index Indicator (VI) over a specified time period, according to Stein et al. (2012) This index provides insights on the variability of direct normal irradiance (DNI) compared to clear sky conditions.

Parameters:

• dni_diff – (pd.Series) The difference in DNI values over time.

• clear_sky_dni_diff – (pd.Series) The difference in clear-sky DNI values over time.

• time_period – (str) A time-based window string (e.g., ‘15T’) used for rolling window calculations.

Returns:

vi (pd.Series) variability index indicator VI: according to Stein et al. (2012)

asi_core.meteo.meteo.plot_series(series)

Plot a timeseries with a datetime index. It generates a simple line plot of the series (helpful for debugging).

Parameters:

series – (pd.Series) A time series with a datetime index.

asi_core.meteo.meteo.check_daily_index_continuity(ts, temporal_res=None)

Check if index (pd.DatetimeIndex) has gaps.

asi_core.meteo.meteo.read_mesor_txt_file(file_path)

Read meteo data from mesor txt file.

asi_core.meteo.meteo.convert_invalid_values_to_nan(data, invalid_value, condition='equals', columns=None, inplace=False)

Converts invalid values in dataframe to nans.

asi_core.meteo.meteo.get_missing_timestamps(dt, temporal_resolution, tz, min_ele=None, latitude=None, longitude=None, altitude=None, method_solar_pos='nrel_numpy', bounds_from_dt=False)

Identifies missing timestamps within a given datetime range based on a specified temporal resolution.

Parameters:

• dt – Array-like of existing timestamps (pandas DatetimeIndex or list of timestamps).

• temporal_resolution – String representing the desired time frequency (e.g., ‘30S’ for 30 seconds).

• tz – Time zone information for the generated timestamps.

• min_ele – (Optional) Minimum solar elevation angle to filter timestamps. Default is None.

• latitude – (Optional) Latitude of the location for solar position calculations. Required if min_ele is set.

• longitude – (Optional) Longitude of the location for solar position calculations. Required if min_ele is set.

• altitude – (Optional) Altitude of the location for solar position calculations. Default is None.

• method_solar_pos – (Optional) Method used for solar position calculations (default: ‘nrel_numpy’).

• bounds_from_dt – (Optional) If True, uses the first and last timestamps from dt as the range bounds. If False, expands the range to include the entire day (00:00:00 to 23:59:59).

Returns:

• dt_missing (pandas DatetimeIndex): Timestamps that are missing from dt within the expected range.

• dr (pandas DatetimeIndex): The full expected timestamp range.

asi_core.meteo.meteo.get_data_availability_stats(ts_missing, ts_available, idx_name='Data')

Returns a DataFrame containing statistics about data availability, comparing missing and available timestamps.

This function computes various metrics, including the total and fraction of missing data, number of days with complete or incomplete data, and daily missing data statistics (median, mean, and standard deviation).

Parameters:

• ts_missing (pandas.DatetimeIndex) – Timestamps corresponding to missing data points.

• ts_available (pandas.DatetimeIndex) – Timestamps corresponding to available data points.

• idx_name (str) – Name for the index of the output DataFrame (default is ‘Data’).

Returns:

A pandas DataFrame with the following columns: - ‘Total data’: Number of available data points. - ‘Total missing’: Number of missing data points. - ‘Fraction missing’: Fraction of missing data points relative to total. - ‘Total days’: Total number of unique days in the data. - ‘Num available days’: Number of days with at least one available data point. - ‘Num incomplete days’: Number of days with at least one missing data point. - ‘Fraction incomplete days’: Fraction of days with missing data. - ‘Median missing per day’: Median count of missing data points per day. - ‘Mean missing per day’: Mean count of missing data points per day. - ‘Std missing per day’: Standard deviation of missing data points per day.

Return type:

pandas.DataFrame

Raises:

AssertionError – If ts_missing and ts_available contain overlapping timestamps.

asi_core.meteo.meteo.get_number_of_nans(df)

Count number of nans in a dataframe columns-wise and total number of rows with a NaN value in at least one column.