hydropandas.io package

hydropandas.io.bro module

https://www.bro-productomgeving.nl/bpo/latest/informatie-voor-softwareleveranciers/url-s-publieke-rest-services

function levels: 1. get_obs_list_from_gmn: list of observations from grondwatermonitoring net

get_bro_groundwater: single observation

measurements_from_gld:

get_metadata_from_gmw:

get_gld_ids_from_gmw:

exception hydropandas.io.bro.BroDataParseError(message: str)[source]

Bases: Exception

Exception raised when Bro data cannot be parsed.

message -- explanation of the error

hydropandas.io.bro.get_bro_groundwater(bro_id, tube_nr=None, only_metadata=False, crs=28992, engine='hydropandas', **kwargs)[source]

get bro groundwater measurement from a GLD id or a GMW id with a filter number.

Parameters:

bro_id (str) – starts with ‘GLD’ or ‘GMW’ e.g. ‘GLD000000012893’.
tube_nr (int or None, optional) – tube number, required if bro_id starts with ‘GMW’. The default is None.
only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.
crs (str, int, pyproj.CRS or None) – The desired coordinate reference system of the observations, if it differs from the crs in BRO the coordinates are transformed, by default EPSG: 28992.
engine (str, optional) – Select how data from the bro-database is obtained, options are ‘hydropandas’ or ‘brodata’ The default is ‘hydropandas’.
**kwargs – passes to measurements_from_gld.

Raises:

ValueError – DESCRIPTION.

Returns:

dataframe – measurements.
dictionary – metadata.

hydropandas.io.bro.get_full_metadata_from_gmw(bro_id, tube_nr, engine='hydropandas')[source]

get metadata for a groundwater monitoring well.

Parameters:

bro_id (str) – bro id of groundwater monitoring well e.g. ‘GMW000000036287’.
tube_nr (int) – filter number you want metadata for.
engine (str, optional) – Select how data from the bro-database is obtained, options are ‘hydropandas’ or ‘brodata’ The default is ‘hydropandas’.

Raises:

ValueError – if bro_id is invalid.

Returns:

meta – dictionary with metadata.

Return type:

dict

hydropandas.io.bro.get_full_metadata_from_gmw_hpd(bro_id, tube_nr)[source]

get metadata for a groundwater monitoring well using the hydropandas engine.

Parameters:

bro_id (str) – bro id of groundwater monitoring well e.g. ‘GMW000000036287’.
tube_nr (int) – filter number you want metadata for.

Raises:

ValueError – if bro_id is invalid.

Returns:

meta – dictionary with metadata.

Return type:

dict

hydropandas.io.bro.get_gld_ids_from_gmw(bro_id, tube_nr)[source]

get bro_ids of multiple grondwaterstandendossier (gld) from a bro_id of a grondwatermonitoringsput (gmw).

Parameters:

bro_id (str) – starts with ‘GLD’ or ‘GMW’ e.g. ‘GMW000000036287’.
tube_nr (int) – tube number.

Raises:

ValueError – DESCRIPTION.
RuntimeError – DESCRIPTION.

Returns:

bro_ids of a grondwaterstandonderzoek (gld).

Return type:

list of str or None

hydropandas.io.bro.get_metadata_from_gmw(bro_id, tube_nr, crs, engine='hydropandas')[source]

get selection of metadata for a groundwater monitoring well. coordinates, ground_level, tube_top and tube screen

Parameters:

bro_id (str) – bro id of groundwater monitoring well e.g. ‘GMW000000036287’.
tube_nr (int) – tube number you want metadata for.
crs (pyproj.CRS) – coordinate reference system to which the coordinates should be transformed.
engine (str, optional) – Select how data from the bro-database is obtained, options are ‘hydropandas’ or ‘brodata’ The default is ‘hydropandas’.

Raises:

ValueError – if bro_id is invalid.
TypeError – if tube_nr is not an int

Returns:

meta – dictionary with metadata.

Return type:

dict

hydropandas.io.bro.get_metadata_from_gmw_hpd(bro_id, tube_nr, crs)[source]

get selection of metadata for a groundwater monitoring well using the hydropandas engine.

Parameters:

bro_id (str) – bro id of groundwater monitoring well e.g. ‘GMW000000036287’.
tube_nr (int) – tube number you want metadata for.
crs (pyproj.CRS) – coordinate reference system to which the coordinates should be transformed.

Raises:

ValueError – if bro_id is invalid.
TypeError – if tube_nr is not an int

Returns:

meta – dictionary with metadata.

Return type:

dict

hydropandas.io.bro.get_obs_list_from_extent(extent, ObsClass, tmin=None, tmax=None, only_metadata=False, keep_all_obs=True, crs=28992, ignore_max_obs=False, engine='hydropandas')[source]

get a list of gmw observations within an extent.

Parameters:

extent (list, tuple, numpy-array or None, optional) – get groundwater monitoring wells within this extent [xmin, xmax, ymin, ymax]
ObsClass (type) – class of the observations, e.g. GroundwaterObs or WaterlvlObs
tmin (str or None, optional) – start time of observations. The default is None.
tmax (str or None, optional) – end time of observations. The default is None.
only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.
crs (str, int, pyproj.CRS or None, optional) – The coordinate reference system of the extent and the observations, if it differs from the crs in BRO the coordinates are transformed, by default EPSG: 28992.
ignore_max_obs (bool, optional) – by default you get a prompt if you want to download over a 1000 observations at once. if ignore_max_obs is True you won’t get the prompt. The default is False
engine (str, optional) – Select how data from the bro-database is obtained, options are ‘hydropandas’, ‘brodata’ or ‘brodata_gm’. When engine=’brodata_gm’ use the dataset Grondwatermonitoring (GM) in samenhang - karakteristieken, hosted by PDOK. This up-to-date dataset combines well- and tube-properties. So users do not have to download each individual Groundwater Monitoring Well (GMW), which speeds up the request. The gm-dataset does not contain the attributes tube_top and ground_level, so you need to use engine=’brodata’ or ‘hydropandas’ if you need those. The Groundwater Level Dossiers (GLD) are still downloaded individually. The default is ‘hydropandas’.

Raises:

DESCRIPTION. –

Returns:

obs_list – DESCRIPTION.

Return type:

TYPE

hydropandas.io.bro.get_obs_list_from_gmn(bro_id, ObsClass, only_metadata=False, keep_all_obs=True, crs=28992, engine='hydropandas')[source]

get a list of observation from a groundwater monitoring network.

Parameters:

bro_id (str) – starts with ‘GMN’ e.g. ‘GMN000000000163’.
ObsClass (type) – class of the observations, so far only GroundwaterObs is supported
only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.
keep_all_obs (boolean, optional) – add all observation points to the collection, even without measurements
crs (str, int, pyproj.CRS or None, optional) – The desired coordinate reference system of the observations, if it differs from the crs in BRO the coordinates are transformed, by default EPSG: 28992.
engine (str, optional) – Select how data from the bro-database is obtained, options are ‘hydropandas’ or ‘brodata’ The default is ‘hydropandas’.

Raises:

ValueError – DESCRIPTION.

Returns:

obs_list (list) – list with observation objects.
meta (dict) – metadata of the groundwater monitoring net.

hydropandas.io.bro.get_obs_list_from_gmn_hpd(bro_id, ObsClass, only_metadata=False, keep_all_obs=True, crs=28992)[source]

get a list of observation from a groundwater monitoring network using the hydropandas engine.

Parameters:

bro_id (str) – starts with ‘GMN’ e.g. ‘GMN000000000163’.
ObsClass (type) – class of the observations, so far only GroundwaterObs is supported
only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.
keep_all_obs (boolean, optional) – add all observation points to the collection, even without measurements
crs (str, int, pyproj.CRS or None, optional) – The desired coordinate reference system of the observations, if it differs from the crs in BRO the coordinates are transformed, by default EPSG: 28992.

Raises:

ValueError – DESCRIPTION.

Returns:

obs_list (list) – list with observation objects.
meta (dict) – metadata of the groundwater monitoring net.

hydropandas.io.bro.get_tube_nrs_from_gmw(bro_id)[source]

returns all tube numbers from a groundwater monitoring well (gmw)

Parameters:: bro_id (str) – bro id of groundwater monitoring well e.g. ‘GMW000000036287’.
Returns:: tube numbers
Return type:: list of int

hydropandas.io.bro.measurements_from_gld(bro_id, tmin=None, tmax=None, to_wintertime=True, crs=28992, drop_duplicate_times=True)[source]

get measurements and metadata from a grondwaterstandonderzoek (gld) bro_id

Parameters:

bro_id (str) – e.g. ‘GLD000000012893’.
tmin (str or None, optional) – start date in format YYYY-MM-DD
tmax (str or None, optional) – end date in format YYYY-MM-DD
to_wintertime (bool, optional) – if True the time index is converted to Dutch winter time. The default is True.
crs (str, int, pyproj.CRS or None, optional) – The desired coordinate reference system of the observations, if it differs from the crs in BRO the coordinates are transformed, by default EPSG: 28992.
drop_duplicate_times (bool, optional) – if True rows with a duplicate time stamp are removed keeping only the first row. The default is True.
add_registration_history (bool, optional) – if True the registration history is added to the metadata. The defualt is True.

Raises:

ValueError – if bro_id is invalid.

Returns:

df (pd.DataFrame) – measurements.
meta (dict) – metadata.

hydropandas.io.dino module

hydropandas.io.dino.read_artdino_dir(dirname, ObsClass=None, subdir='csv', suffix='1.csv', unpackdir=None, force_unpack=False, preserve_datetime=False, keep_all_obs=True, **kwargs)[source]

Read Dino directory with point observations.

TODO: - Evt. nog verbeteren door meteen Dataframe te vullen op het moment dat een observatie wordt ingelezen. Nu wordt eerst alles ingelezen in een lijst en daar een dataframe van gemaakt.

Parameters:

dirname (str or pathlib.Path) – directory name, can be a .zip file or the parent directory of subdir
ObsClass (type) – class of the observations, e.g. GroundwaterObs or WaterlvlObs
subdir (str) – subdirectory of dirname with data files
suffix (str) – suffix of files in subdir that will be read
unpackdir (str or pathlib.Path) – destination directory of the unzipped file
force_unpack (boolean, optional) – force unpack if dst already exists
preserve_datetime (boolean, optional) – use date of the zipfile for the destination file
keep_all_obs (boolean, optional) – add all observation points to the collection, even without data or metadata
**kwargs (dict, optional) – Extra arguments are passed to ObsClass.from_artdino_file()

Returns:

obs_df – collection of multiple point observations

Return type:

pd.DataFrame

hydropandas.io.dino.read_artdino_groundwater_csv(path, to_mnap=True, read_series=True)[source]

Read dino groundwater quantity data from a CSV file as exported by ArtDiver.

Parameters:

path (str or pathlib.Path) – path to csv file
to_mnap (boolean, optional) – if True a column with ‘stand_m_tov_nap’ is added to the dataframe
read_series (boolean, optional) – if False only metadata is read, default is True

Returns:

measurements (pd.DataFrame)
meta (dict) – dictionary with metadata

hydropandas.io.dino.read_dino_dir(path: str | Path, ObsClass, subdir: str = 'DINO_Grondwaterstanden', suffix: str | None = None, keep_all_obs: bool = True, **kwargs: dict)[source]

Read Dino directory with point observations.

TODO: - Evt. nog verbeteren door meteen Dataframe te vullen op het moment dat een observatie wordt ingelezen. Nu wordt eerst alles ingelezen in een lijst en daar een dataframe van gemaakt. - aparte unzip functie maken en toch de juiste tijdelijke directory krijgen.

Parameters:

path (str | Path) – directory name, can be a .zip file or the parent directory of subdir
ObsClass (type) – class of the observations, e.g. GroundwaterObs or WaterlvlObs
subdir (str) – subdirectory of dirname with data files. For old school dino zip files this should be “Grondwaterstanden_Put”. For new style the default value ‘DINO_Grondwaterstanden’ is sufficient. The default is ‘DINO_Grondwaterstanden’.
suffix (str or None, optional) – suffix of files in subdir that will be read. For old school dino zip files this should be ‘1.csv’. For new style the default value None is sufficient. The default is None.
keep_all_obs (boolean, optional) – add all observation points to the collection, even without data or metadata
**kwargs (dict, optional) – Extra arguments are passed to ObsClass.from_dino_file()

Returns:

obs_df – collection of multiple point observations

Return type:

pd.DataFrame

hydropandas.io.dino.read_dino_groundwater_csv(f: str | Path | FileIO, to_mnap: bool = True, read_series: bool = True, remove_duplicates: bool = False, keep_dup: str = 'last')[source]

Read dino groundwater quantity data from a dinoloket csv file.

Parameters:

f (Union[str, Path, TextIOWrapper]) – path to csv file
to_mnap (boolean, optional) – if True a column with ‘stand_m_tov_nap’ is added to the dataframe
read_series (boolean, optional) – if False only metadata is read, default is True
remove_duplicates (boolean, optional) – if True duplicate indices are removed. Default is False.
keep_dup (str, optional) – indicate which duplicate indices should be kept, only used when remove_duplicates is True. Default is ‘last’

Returns:

measurements (pd.DataFrame)
meta (dict) – dictionary with metadata

hydropandas.io.dino.read_dino_groundwater_quality_txt(f: str | Path | FileIO)[source]

Read dino groundwater quality (grondwatersamenstelling) from a dinoloket txt file.

Notes

this function has not been tested thoroughly

Parameters:

filepath_or_buffer (str) – path to txt file

Returns:

measurements (pd.DataFrame)
meta (dict) – dictionary with metadata

hydropandas.io.dino.read_dino_waterlvl_csv(f: str | Path | FileIO, to_mnap: bool = True, read_series: bool = True)[source]

Read dino waterlevel data from a dinoloket csv file.

Parameters:

f (str, Path, FileIO) – path to dino water level csv file
to_mnap (boolean, optional) – if True a column with ‘stand_m_tov_nap’ is added to the dataframe
read_series (boolean, optional) – if False only metadata is read, default is True

hydropandas.io.fews module

hydropandas.io.fews.get_fews_pid(name: str) → dict[str, Obs][source]

Get matching ParameterId’s and HydroPandas Observation Classes

Parameters:: name (str) – Waterboard name
Returns:: Dictonary with ParameterId and the resulting Observation Class
Return type:: Dict[str, Obs]

hydropandas.io.fews.iterparse_pi_xml(fname: str, ObsClass: Obs | dict[str, Obs], translate_dic: dict[str, str] | None = None, filterdict: dict[str, list[str]] | None = None, locationIds: list[str] | None = None, return_events: bool = True, keep_flags: tuple[int] = (0, 1), return_df: bool = False, tags: tuple[str] = ('series', 'header', 'event'), crs: str | int | CRS | None = None)[source]

Read a FEWS XML-file with measurements, memory efficient.

Parameters:

fname (str or pathlib.Path) – full path to file
ObsClass (Union[Obs, Dict[str, Obs]],) – class of the observations, e.g. GroundwaterObs or WaterlvlObs
translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘location’}.
locationIds (tuple or list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read.
filterdict (dict, optional) – dictionary with tag name to apply filter to as keys, and list of accepted names as dictionary values to keep in final result, i.e. {“locationId”: [“B001”, “B002”]}
return_events (bool, optional) – return all event-information in a DataFrame per location, instead of just a Series (defaults to False). Overrules keep_flags kwarg.
keep_flags (list of ints, optional) – keep the values with these flags (defaults to 0 and 1). Only used when return_events is False.
tags (list of strings, optional) – Select the tags to be parsed. Defaults to series, header and event
crs (str, int, pyproj.CRS or None, optional) – The coordinate reference system of the observations. There is no check if the coordinates in the xml are actually this crs. This crs is only used to set the crs attribute of the observations, by default None
return_df (bool, optional) – return a DataFame with the data, instead of two lists (default is False)

Returns:

df (pandas.DataFrame) – a DataFrame containing the metadata and the series if ‘return_df’ is True
obs_list (list of pandas Series) – list of timeseries if ‘return_df’ is False

Read a list of xml files into a list of observation objects.

Parameters:

fnames (TYPE) – DESCRIPTION.
ObsClass (Union[Obs, Dict[str, Obs]]) – class of the observations, e.g. GroundwaterObs or WaterlvlObs
directory (TYPE, optional) – DESCRIPTION. The default is None.
locations (tuple or list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read.
translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘location’}.
filterdict (dict, optional) – dictionary with tag name to apply filter to as keys, and list of accepted names as dictionary values to keep in final result, i.e. {“locationId”: [“B001”, “B002”]}
remove_nan (boolean, optional) – remove nan values from measurements, flag information about the nan values is also lost, only used if low_memory=False
low_memory (bool, optional) – whether to use xml-parsing method with lower memory footprint, default is True
crs (str, int, pyproj.CRS or None, optional) – The coordinate reference system of the observations. There is no check if the coordinates in the xml are actually this crs. This crs is only used to set the crs attribute of the observations, by default None

Returns:

list of timeseries stored in ObsClass objects

Return type:

list of ObsClass objects

hydropandas.io.fews.read_xml_fname(fname: str, ObsClass: Obs | dict[str, Obs], translate_dic: dict[str, str] | None = None, low_memory: bool = True, locationIds: list[str] | None = None, filterdict: dict[str, list[str]] | None = None, return_events: bool = True, keep_flags: tuple[int] = (0, 1), return_df: bool = False, tags: tuple[str] = ('series', 'header', 'event'), remove_nan: bool = False, crs: str | int | CRS | None = None, **kwargs: dict)[source]

Read an xml filename into a list of observations objects.

Parameters:

fname (str or pathlib.Path) – full path to file
ObsClass (Union[Obs, Dict[str, Obs]]) – class of the observations, e.g. GroundwaterObs or WaterlvlObs
translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘location’}.
low_memory (bool, optional) – whether to use xml-parsing method with lower memory footprint, default is True
locationIds (tuple or list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read.
filterdict (dict, optional) – dictionary with tag name to apply filter to as keys, and list of accepted names as dictionary values to keep in final result, i.e. {“locationId”: [“B001”, “B002”]}
return_events (bool, optional) – return all event-information in a DataFrame per location, instead of just a Series (defaults to False). Overrules keep_flags kwarg.
keep_flags (list of ints, optional) – keep the values with these flags (defaults to 0 and 1). Only used when return_events is False.
tags (list of strings, optional) – Select the tags to be parsed. Defaults to series, header and event
return_df (bool, optional) – return a DataFame with the data, instead of two lists (default is False)
remove_nan (boolean, optional) – remove nan values from measurements, flag information about the nan values is also lost, only used if low_memory=False
crs (str, int, pyproj.CRS or None, optional) – The coordinate reference system of the observations. There is no check if the coordinates in the xml are actually this crs. This crs is only used to set the crs attribute of the observations, by default None

Returns:

list of timeseries stored in ObsClass objects

Return type:

list of ObsClass objects

Read a FEWS XML-file with measurements, return list of ObsClass objects.

Parameters:

root (xml.etree.ElementTree.Element) – root element of a fews xml
ObsClass (Union[Obs, Dict[str, Obs]],) – class of the observations, e.g. GroundwaterObs or WaterlvlObs
translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘location’}.
locationIds (tuple or list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read.
remove_nan (boolean, optional) – remove nan values from measurements, flag information about the nan values is also lost
crs (str, int, pyproj.CRS or None, optional) – The coordinate reference system of the observations. There is no check if the coordinates in the xml are actually this crs. This crs is only used to set the crs attribute of the observations, by default None

Returns:

list of timeseries stored in ObsClass objects

Return type:

list of ObsClass objects

Read xmlstring into an list of Obs objects. Xmlstrings are usually obtained using a fews api.

Parameters:

xmlstring (str) – xml string to be parsed. Typically from a fews api.
ObsClass (Union[Obs, Dict[str, Obs]]) – class of the observations, e.g. GroundwaterObs or WaterlvlObs
translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘location’}.
locationIds (tuple or list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read.
low_memory (bool, optional) – whether to use xml-parsing method with lower memory footprint, default is True
remove_nan (boolean, optional) – remove nan values from measurements, flag information about the nan values is also lost, only used if low_memory=False
crs (str, int, pyproj.CRS or None, optional) – The coordinate reference system of the observations. There is no check if the coordinates in the xml are actually this crs. This crs is only used to set the crs attribute of the observations, by default None

Returns:

list of timeseries stored in ObsClass objects

Return type:

list of ObsClass objects

hydropandas.io.fews.write_pi_xml(obs_coll, fname, timezone: float = 1.0, version: str = '1.24')[source]

Write TimeSeries object to PI-XML file.

Parameters:: fname (str or pathlib.Path) – path to XML file

hydropandas.io.knmi module

Module with functions to read or download time series with observations from knmi.

function levels: 1. get_knmi_obs_list: list of observations

get_knmi_obs: single observation

get_timeseries_stn, get_timeseries_from_file: get time series from station or file

fill_missing_measurements, get_evaporation: aggregate time series data

download_knmi_data: download a single timeseries

6a. get_hourly_meteo_api, get_daily_meteo_api, get_daily_rainfall_api,
get_daily_meteo_url, get_daily_rainfall_url 7a. request_api, request_url 7b. parse_data

6b. interpret_knmi_file

For knmi climate scenarios: 1. get_knmi_scenarios_obs_list: get knmi climate scenarios for a station

get_knmi_scenarios_data: get knmi climate scenarios for a station

hydropandas.io.knmi.download_knmi_data(stn: int, meteo_var: str, start: Timestamp, end: Timestamp, settings: dict[str, Any], stn_name: str | None = None) → tuple[DataFrame, dict[str, Any], DataFrame][source]

download knmi data of a measurements station for certain observation type.

Parameters:

stn (int) – measurement station.
meteo_var (str) – observation type.
start (pd.Timestamp) – start date of observations.
end (pd.Timestamp) – end date of observations.
settings (dict) – settings for obtaining data
stn_name (str, optional) – station name. If None the name is obtained form te stn number. The default is None.

Raises:

ValueError – if the data from knmi cannot not be read a ValueError is raised. Unless raise_exceptions is False

Returns:

ts_df (pandas DataFrame) – data from one station from one type of observation
variables (dictionary) – information about the observed variables
stations (pandas DataFrame) – information about the measurement station

hydropandas.io.knmi.fill_missing_measurements(stn: int, meteo_var: str, start: Timestamp, end: Timestamp, settings: dict[str, Any], stn_name: str | None = None) → tuple[DataFrame, dict[str, Any]][source]

fill missing measurements in knmi data.

Parameters:

stn (int) – measurement station.
meteo_var (str) – observation type.
start (pd.Timestamp) – start date of observations.
end (pd.Timestamp) – end date of observations.
settings (dict) – settings for obtaining data.
stn_name (str, optional) – station name. If None the name is obtained form te stn number. The default is None.

Returns:

knmi_df (pandas DataFrame) – data from one station from one type of observation, with additional column to see which station is used to fill the value
meta (dictionary) – metadata from the originally requested station even if this station has no data

hydropandas.io.knmi.get_daily_meteo_api(stn, start=None, end=None, meteo_var: str | None = None) → tuple[DataFrame, dict[str, Any]][source]

download and read knmi daily meteo data.

Parameters:

stn (int) – station number.
meteo_var (str) – e.g. ‘EV24’.
start (pd.Timestamp or None) – start time of observations.
end (pd.Timestamp or None) – end time of observations.

Returns:

pandas DataFrame – measurements.
variables (dictionary) – additional information about the variables
stations (pandas DataFrame) – additional data about the measurement station

hydropandas.io.knmi.get_daily_meteo_url(stn: int) → tuple[DataFrame, dict[str, Any]][source]

download and read knmi daily meteo data.

Parameters:

stn (int) – station number.

Returns:

pandas DataFrame – measurements.
meta (dictionary) – additional information about the variables

hydropandas.io.knmi.get_daily_rainfall_api(stn: int, start: Timestamp | None = None, end: Timestamp | None = None) → tuple[DataFrame, dict[str, Any]][source]

download and read knmi daily rainfall.

Parameters:

stn (int) – station number.
start (pd.Timestamp or None) – start time of observations.
end (pd.Timestamp or None) – end time of observations.

Raises:

ValueError – if there is no data for the provided stn an error is raised.

Returns:

pandas DataFrame – measurements.
variables (dictionary) – additional information about the variables

hydropandas.io.knmi.get_daily_rainfall_url(stn: int, stn_name: str) → tuple[DataFrame, dict[str, Any]][source]

download and read knmi daily rainfall.

Parameters:

stn (int) – station number.
stn_name (str) – the name of the station in capital letters, can be tricky

Raises:

ValueError – if there is no data for the provided stn an error is raised.

Returns:

pandas DataFrame – measurements.
variables (dictionary) – additional information about the variables

hydropandas.io.knmi.get_evaporation(meteo_var: str, stn: int = 260, start: Timestamp | None = None, end: Timestamp | None = None, settings: dict[str, Any] | None = None) → DataFrame[source]

Collect different types of (reference) evaporation from KNMI weather stations

Parameters:

meteo_var (str) – Choice between ‘penman’, ‘makkink’ or ‘hargraves’.
stn (str) – station number, defaults to 260 De Bilt
start (pd.Timestamp) – start time of observations.
end (pd.Timestamp) – end time of observations.
settings (dict or None, optional) – settings for the time series

Return type:

pandas.DataFrame

hydropandas.io.knmi.get_hourly_meteo_api(stn: int, start: Timestamp, end: Timestamp, meteo_var: str | None = None) → tuple[DataFrame, dict[str, Any]][source]

Retrieve hourly meteorological data from the KNMI API.

Parameters:

stn (int) – The station number.
start (pd.Timestamp or None) – The start date and time of the data.
end (pd.Timestamp or None) – The end date and time of the data.

Returns:

pandas.DataFrame – A DataFrame containing the requested meteorological variable data.
dict – A dictionary containing information about the variables in the DataFrame.

Raises:

requests.ConnectionError – If there is a connection error while accessing the KNMI API.
RuntimeError – If the KNMI API is down or returns unexpected data.

hydropandas.io.knmi.get_knmi_daily_meteo_api(stn, start=None, end=None, meteo_var: str | None = None) → tuple[DataFrame, dict[str, Any]][source]

download and read knmi daily meteo data.

Parameters:

stn (int) – station number.
meteo_var (str) – e.g. ‘EV24’.
start (pd.Timestamp or None) – start time of observations.
end (pd.Timestamp or None) – end time of observations.

Returns:

pandas DataFrame – measurements.
variables (dictionary) – additional information about the variables
stations (pandas DataFrame) – additional data about the measurement station

hydropandas.io.knmi.get_knmi_daily_meteo_url(stn: int) → tuple[DataFrame, dict[str, Any]][source]

download and read knmi daily meteo data.

Deprecated since version 0.13.3: get_knmi_daily_meteo_url will be removed in hydropandas 1.0.0, it is replaced by get_daily_meteo_url.

Parameters:

stn (int) – station number.

Returns:

pandas DataFrame – measurements.
meta (dictionary) – additional information about the variables

hydropandas.io.knmi.get_knmi_daily_rainfall_api(stn: int, start: Timestamp | None = None, end: Timestamp | None = None) → tuple[DataFrame, dict[str, Any]][source]

download and read knmi daily rainfall.

Deprecated since version 0.13.3: get_knmi_daily_rainfall_api will be removed in hydropandas 1.0.0, it is replaced by get_daily_rainfall_api.

Parameters:

stn (int) – station number.
start (pd.Timestamp or None) – start time of observations.
end (pd.Timestamp or None) – end time of observations.

Raises:

ValueError – if there is no data for the provided stn an error is raised.

Returns:

pandas DataFrame – measurements.
variables (dictionary) – additional information about the variables

hydropandas.io.knmi.get_knmi_daily_rainfall_url(stn: int, stn_name: str) → tuple[DataFrame, dict[str, Any]][source]

download and read knmi daily rainfall.

Deprecated since version 0.13.3: get_knmi_daily_rainfall_url will be removed in hydropandas 1.0.0, it is replaced by get_daily_rainfall_url.

Parameters:

stn (int) – station number.
stn_name (str) – the name of the station in capital letters, can be tricky

Raises:

ValueError – if there is no data for the provided stn an error is raised.

Returns:

pandas DataFrame – measurements.
variables (dictionary) – additional information about the variables

hydropandas.io.knmi.get_knmi_hourly_meteo_api(stn: int, start: Timestamp, end: Timestamp, meteo_var: str | None = None) → tuple[DataFrame, dict[str, Any]][source]

Retrieve hourly meteorological data from the KNMI API.

Deprecated since version 0.13.3: get_knmi_hourly_meteo_api will be removed in hydropandas 1.0.0, it is replaced by get_hourly_meteo_api.

Parameters:

stn (int) – The station number.
start (pd.Timestamp or None) – The start date and time of the data.
end (pd.Timestamp or None) – The end date and time of the data.

Returns:

pandas.DataFrame – A DataFrame containing the requested meteorological variable data.
dict – A dictionary containing information about the variables in the DataFrame.

Raises:

requests.ConnectionError – If there is a connection error while accessing the KNMI API.
RuntimeError – If the KNMI API is down or returns unexpected data.

get knmi observation from stn, fname or nearest xy coordinates.

Parameters:

stn (int, str or None, optional) – measurement station e.g. 829. The default is None.
fname (str, pathlib.Path, file-like object or None, optional) – filename of a knmi file. The default is None.
xy (list, tuple or None, optional) – RD coördinates of a location in the Netherlands. The station nearest to this location used. The Default is None.
meteo_var (str or None, optional) – meteo variable e.g. “RH” or “EV24”. See list with all options in the hydropandas documentation.
start (str, datetime or None, optional) – start date of observations. The default is None.
end (str, datetime or None, optional) – end date of observations. The default is None.
**kwargs –

fill_missing_obsbool, optional
if True nan values in time series are filled with nearby time series. The default is False. Note: if the given stn has no data between start and end the data from nearby stations is used. In this case the metadata of the Observation is the metadata from the nearest station that has any measurement in the given period.

fill_missing_obs_with_factorbool, optional
if True, donor-station values are scaled with an overlap-based factor before filling missing values. This automatically enables fill_missing_obs. The default is False.

intervalstr, optional
desired time interval for observations. Options are ‘daily’ and ‘hourly’. The default is ‘daily’.

use_apibool, optional

if True the api is used to obtain the data, API documentation is here:
https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

if False a text file is downloaded into a temporary folder and the data is read from there. Default is True since the api is back online (July 2021).

raise_exceptionsbool, optional
if True you get errors when no data is returned. The default is False.

Returns:

pd.DataFrame, measurements
dict, metadata

Raises:

ValueError – if no meteo_var is given or stn, fname and xy are all None.

Get a list of observations of knmi stations. Either specify a list of knmi stations (stns) or a dataframe with x, y coordinates (locations).

Parameters:

locations (pandas DataFrame or None) – dataframe with x and y coordinates. The default is None
stns (list of int or None) – list of knmi stations. The default is None
xy (list or numpy array, optional) – xy coordinates of the locations. e.g. [[10,25], [5,25]]
meteo_vars (list or tuple of str) – meteo variables e.g. [“RH”, “EV24”]. The default is (“RH”)
starts (None, str, datetime or list, optional) – start date of observations per meteo variable. The start date is included in the time series. If start is None the start date will be January 1st of the previous year. if start is str it will be converted to datetime if start is a list it should be the same length as meteo_vars and the start time for each variable. The default is None
ends (list of str, datetime or None) – end date of observations per meteo variable. The end date is included in the time series. If end is None the start date will be January 1st of the previous year. if end is a str it will be converted to datetime if end is a list it should be the same length as meteo_vars and the end time for each meteo variable. The default is None
ObsClasses (list of type or None) – class of the observations, can be PrecipitationObs or EvaporationObs. The default is None.
progress_callback (callable or None, optional) – callback function that is called with (i, total) for each station processed, where i is the zero-based index and total is the total number of stations. The default is None.
**kwargs –

fill_missing_obsbool, optional
if True nan values in time series are filled with nearby time series. The default is False. Note: if the given stn has no data between start and end the data from nearby stations is used. In this case the metadata of the Observation is the metadata from the nearest station that has any measurement in the given period.

fill_missing_obs_with_factorbool, optional
if True, donor-station values are scaled with an overlap-based factor before filling missing values. This automatically enables fill_missing_obs. The default is False.

intervalstr, optional
desired time interval for observations. Options are ‘daily’ and ‘hourly’. The default is ‘daily’.

use_apibool, optional

if True the api is used to obtain the data, API documentation is here:
https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

if False a text file is downloaded into a temporary folder and the data is read from there. Default is True since the api is back online (July 2021).

raise_exceptionsbool, optional
if True you get errors when no data is returned. The default is False.

Returns:

obs_list – collection of multiple point observations

Return type:

list of observation objects

hydropandas.io.knmi.get_knmi_scenarios_data(stn: int | str, years: Iterable[Literal['2033', '2050', '2100', '2150']] = ('2033', '2050', '2100', '2150'), scenarios: Iterable[Literal['Ld', 'Ln', 'Md', 'Mn', 'Hd', 'Hn']] = ('Ld', 'Ln', 'Md', 'Mn', 'Hd', 'Hn'), evap: Literal['EV24', 'makkink', 'penman', 'hargreaves'] = 'EV24') → dict[str, DataFrame][source]

Fetch and process KNMI climate scenario data for a station.

The station argument is accepted as an integer or string for convenience. Internally it is converted to a string when interacting with the KNMI API.

Retrieves climate scenario data from KNMI and returns a dictionary of processed DataFrames with temperature, precipitation, and evaporation data.

Parameters:

stn (int or str) – Station number (e.g., 550 or “550”).
years (tuple, optional) – Years of climate scenario. The default is (‘2033’,’2050’,’2100’,’2150’).
scenarios (tuple, optional) – Names of climate scenario. The default is (‘Ld’,’Ln’,’Md’,’Mn’,’Hd’,’Hn’). This includes all scenarios including the original measurements.
evap (str, optional) – Method for calculating evaporation. Options are ‘EV24’, ‘makkink’, ‘penman’, or ‘hargreaves’. The default is ‘EV24’.

Returns:

Dictionary mapping scenario names to pandas DataFrames with processed climate data. Each DataFrame has a datetime index and columns: TG (temperature), RH (precipitation), Q (radiation), TX, TN, UG, FG, and EV24 (evaporation).

Return type:

dict

Raises:

RuntimeError – If the API request fails or data cannot be retrieved.

hydropandas.io.knmi.get_knmi_scenarios_obs_list(stn: int | str, years: Iterable[Literal['2033', '2050', '2100', '2150']] = ('2033', '2050', '2100', '2150'), scenarios: Iterable[Literal['Ld', 'Ln', 'Md', 'Mn', 'Hd', 'Hn']] = ('Ld', 'Ln', 'Md', 'Mn', 'Hd', 'Hn'), evap: Literal['EV24', 'makkink', 'penman', 'hargreaves'] = 'EV24', meteo_vars: Iterable[Literal['TG', 'RD', 'Q', 'TX', 'TN', 'UG', 'FG', 'EV24']] | None = None, ObsClass: dict[str, Any] | None = None) → list[Any][source]

Convert climate scenario dataframes into observation objects.

Parameters:

stn (int or str) – Station number (e.g., 550 or “550”).
years (tuple, optional) – Years of climate scenario. The default is (‘2033’,’2050’,’2100’,’2150’).
scenarios (tuple, optional) – Names of climate scenario. The default is (‘Ld’,’Ln’,’Md’,’Mn’,’Hd’,’Hn’). This includes all scenarios including the original measurements.
evap (Literal["EV24", "makkink", "penman", "hargreaves"], optional) – Method for calculating evaporation. Options are ‘EV24’, ‘makkink’, ‘penman’, or ‘hargreaves’. The default is ‘EV24’.
meteo_vars (iterable of str or None, optional) – Meteorological variables to include in the ObsCollection. Possible variables include ‘TG’, ‘RD’, ‘Q’, ‘TX’, ‘TN’, ‘UG’, ‘FG’, and ‘EV24’. If None (default), all available variables are included.
ObsClass (dict[str, PrecipitationObs | EvaporationObs | MeteoObs]) – Dictionary mapping variable names to observation classes. The function will use these classes to instantiate the observations.

Returns:

List of instantiated observation objects. Each object has station and meteo_var attributes set in addition to the usual metadata.

Return type:

list

hydropandas.io.knmi.get_knmi_timeseries_fname(fname: str, meteo_var: str, settings: dict[str, Any], start: Timestamp, end: Timestamp) → tuple[DataFrame, dict[str, Any]][source]

Get a knmi time series and metadata from a file.

Deprecated since version 0.13.3: get_knmi_timeseries_fname will be removed in hydropandas 1.0.0, it is replaced by get_timeseries_from_file.

Parameters:

fname (str or pathlib.Path) – filename of the knmi file.
meteo_var (str) – observation type e.g. “RH” or “EV24”. See list with all options in the hpd.read_knmi function.
settings (dict) – settings for obtaining the right time series, see _get_default_settings for more information
start (pd.Timestamp) – start date of observations.
end (pd.Timestamp) – end date of observations.

Returns:

ts_df (pandas DataFrame) – time series with measurements.
meta (dictionary) – metadata from the measurement station.

hydropandas.io.knmi.get_knmi_timeseries_stn(stn: int, meteo_var: str, settings: dict[str, Any], start: Timestamp | None = None, end: Timestamp | None = None) → tuple[DataFrame, dict[str, Any]][source]

Get a knmi time series and metadata.

Deprecated since version 0.13.3: get_knmi_timeseries_stn will be removed in hydropandas 1.0.0, it is replaced by get_timeseries_stn.

Parameters:

stn (int) – measurement station e.g. 829.
meteo_var (str) – observation type e.g. “RH” or “EV24”. See list with all options in the hpd.read_knmi function.
settings (dict) – settings for obtaining the right time series, see _get_default_settings for more information
start (pd.Timestamp or None, optional) – start date of observations. The default is None.
end (pd.Timestamp or None, optional) – end date of observations. The default is None.

Returns:

ts_df (pandas DataFrame) – time series with measurements.
meta (dictionary) – metadata from the measurement station.

Find the N nearest KNMI stations that measure variable ‘meteo_var’ to the x, y coordinates.

Parameters:

xy (list, tuple or numpy.array of int or float) – single pair of xy coordinates. e.g. (150_000., 400_000.)
meteo_var (str) – measurement variable e.g. ‘RH’ or ‘EV24’
start (str, datetime or None, optional) – start date of observations. The default is None.
end (str, datetime or None, optional) – end date of observations. The default is None.
n (int, optional) – number of stations you want to return. The default is 1.
stations (pandas DataFrame, optional) – if None stations will be obtained using the get_stations function. The default is None.
ignore (list, optional) – list of stations to ignore. The default is None.

Returns:

station numbers.

Return type:

list

Find the KNMI stations that measure ‘meteo_var’ closest to the coordinates in ‘locations’.

Parameters:

locations (pandas.DataFrame) – DataFrame containing x and y coordinates
xcol (str) – name of the column in the locations dataframe with the x values
ycol (str) – name of the column in the locations dataframe with the y values
stations (pandas DataFrame, optional) – if None stations will be obtained using the get_stations function. The default is None.
meteo_var (str) – measurement variable e.g. ‘RH’ or ‘EV24’
start (str, datetime or None, optional) – start date of observations. The default is None.
end (str, datetime or None, optional) – end date of observations. The default is None.
ignore (list, optional) – list of stations to ignore. The default is None.

Returns:

stns – station numbers.

Return type:

list

hydropandas.io.knmi.get_nearest_station_xy(xy: list[list[float]], stations: DataFrame | None = None, meteo_var: str = 'RH', ignore: list[str] | None = None) → list[int][source]

find the KNMI stations that measure ‘meteo_var’ closest to the given x and y coordinates.

Parameters:

xy (list or numpy array, optional) – xy coordinates of the locations. e.g. [[10,25], [5,25]]
stations (pandas DataFrame, optional) – if None stations will be obtained using the get_stations function. The default is None.
meteo_var (str) – measurement variable e.g. ‘RH’ or ‘EV24’
ignore (list, optional) – list of stations to ignore. The default is None.

Returns:

stns – station numbers.

Return type:

list

Notes

assumes you have a structured rectangular grid.

hydropandas.io.knmi.get_station_name(stn: int, stations: DataFrame | None = None) → str[source]

Returns the station name from a KNMI station.

Modifies the station name in such a way that a valid url can be obtained.

Parameters:

stn (int) – station number.
stations (pandas DataFrame or None) – DataFrame with the station metadata.

Returns:

Name of the station or None if station is not found.

Return type:

str or None

hydropandas.io.knmi.get_stations(meteo_var: str, start: Timestamp | str | None = None, end: Timestamp | str | None = None) → DataFrame[source]

get knmi stations from json files according to variable.

Parameters:

meteo_var (str, optional) – type of meteodata, by default ‘RH’
start (str, datetime or None, optional) – start date of observations. The default is None.
end (str, datetime or None, optional) – end date of observations. The default is None.

Return type:

pandas DataFrame with stations, names and coordinates (Lat/Lon & RD)

hydropandas.io.knmi.get_stations_scenarios() → DataFrame[source]: Get KNMI station information for climate scenarios.

hydropandas.io.knmi.get_timeseries_from_file(fname: str, meteo_var: str, settings: dict[str, Any], start: Timestamp, end: Timestamp) → tuple[DataFrame, dict[str, Any]][source]

Get a knmi time series and metadata from a file.

Parameters:

fname (str or pathlib.Path) – filename of the knmi file.
meteo_var (str) – observation type e.g. “RH” or “EV24”. See list with all options in the hpd.read_knmi function.
settings (dict) – settings for obtaining the right time series, see _get_default_settings for more information
start (pd.Timestamp) – start date of observations.
end (pd.Timestamp) – end date of observations.

Returns:

ts_df (pandas DataFrame) – time series with measurements.
meta (dictionary) – metadata from the measurement station.

hydropandas.io.knmi.get_timeseries_stn(stn: int, meteo_var: str, settings: dict[str, Any], start: Timestamp | None = None, end: Timestamp | None = None) → tuple[DataFrame, dict[str, Any]][source]

Get a knmi time series and metadata.

Parameters:

stn (int) – measurement station e.g. 829.
meteo_var (str) – observation type e.g. “RH” or “EV24”. See list with all options in the hpd.read_knmi function.
settings (dict) – settings for obtaining the right time series, see _get_default_settings for more information
start (pd.Timestamp or None, optional) – start date of observations. The default is None.
end (pd.Timestamp or None, optional) – end date of observations. The default is None.

Returns:

ts_df (pandas DataFrame) – time series with measurements.
meta (dictionary) – metadata from the measurement station.

hydropandas.io.knmi.hargreaves(tmean: Series, tmin: Series, tmax: Series, dates: Series, lat: float = 52.1, x: list[float] | None = None) → Series[source]

Estimate of Hargraves potential evaporation according to Allen et al. 1990.

Parameters:

tmean (pandas.Series) – Daily mean temperature
tmin (pandas.Series) – Daily minimum temperature
tmax (pandas.Series) – Daily maximum temperature
dates (pandas.Series.index) – Dates
lat (float, optional) – Latitude of station, by default 52.1
x (_type_, optional) – Optional parameter to scale evaporation estimate, by default None

Return type:

pandas.Series

hydropandas.io.knmi.interpret_knmi_file(df: DataFrame, meta: dict[str, Any], meteo_var: str, start: Timestamp | None = None, end: Timestamp | None = None, add_day: bool = False, add_hour: bool = True) → tuple[DataFrame, dict[str, Any]][source]

interpret data from knmi by selecting meteo_var data and meta and transforming the variables

Parameters:

df (DataFrame) – dataframe with meteo_var as column
meta (dictionary) – dictionary with meteo_var as key
meteo_var (str) – e.g. ‘EV24’.
start (pd.Timestamp or None) – start time of observations.
end (pd.Timestamp or None) – end time of observations.
add_day (boolean, optional) – add 1 day so that the timestamp is at the end of the period the data describes, default is False, and has to be set per type of file.
add_hour (boolean, optional) – add 1 hour to convert from UT to UT+1 (standard-time in the Netherlands), default is True as this is usually the case.

Returns:

pandas DataFrame – measurements.
variables (dictionary) – additional information about the variables
station (int) – meteostation

hydropandas.io.knmi.makkink(tmean: Series, K: Series) → Series[source]

Estimate of Makkink reference evaporation according to KNMI.

Parameters:

tmean (tmean : pandas.Series) – Daily mean temperature in Celsius
K (pandas.Series) – Global radiation estimate in J/cm2

Return type:

pandas.Series

hydropandas.io.knmi.parse_data(path: str | Path | StringIO) → tuple[DataFrame, dict[str, Any]][source]

read knmi daily meteo data from a file

Parameters:

path (str, pathlib.Path or StringIO) – file path of .txt file or str representation.

Returns:

pandas DataFrame – measurements.
meta (dictionary) – additional information about the variables

hydropandas.io.knmi.penman(tmean: Series, tmin: Series, tmax: Series, K: Series, wind: Series, rh: Series, dates: Series, z: float = 1.0, lat: float = 52.1, G: float = 0.0, wh: float = 10.0, tdew: Series | None = None) → Series[source]

Estimate of Penman reference evaporation according to Allen et al 1990.

Parameters:

tmean (pandas.Series) – Daily mean temperature in Celsius
tmin (pandas.Series) – Daily minimum temperature in Celsius
tmax (pandas.Series) – Daily maximum temperature in Celsius
K (pandas.Series) – Global radiation estimate in J/cm2
wind (pandas.Series) – Daily mean wind speed in m/s
rh (pandas.Series) – Relative humidity in %
dates (pandas.Series) – Dates
z (float, optional) – Elevation of station in m, by default 1.0
lat (float, optional) – Latitude of station, by default 52.1
G (float, optional) – Ground flux in MJ/m2, by default 0.0
wh (float, optional) – Height of wind measurement in m, by default 10.0
tdew (pandas.Series) – Dew point temperature in C, by default None

Return type:

pandas.Series

hydropandas.io.knmi.request_api(url: str, params: dict[str, str], fname=None) → StringIO[source]

Download KNMI data from the API

Parameters:

url (str) – URL to parse the request to
params (Dict[str, str]) – Dictionary with parameters that are parsed to the request get
fname (str, pathlib.Path or None, optional) – filename to save the data to only used if not None, by default None

Return type:

StringIO

hydropandas.io.knmi.request_url(url: str, fname=None) → StringIO[source]

download and read knmi daily rainfall data.

Parameters:

stn (int) – station number.
fname (str, pathlib.Path or None, optional) – filename to save the data to, only used if not None. The default is None.

Return type:

StringIO

hydropandas.io.lizard module

Module with functions to read or download time series with observations from knmi.

function levels: 1. get_obs_list_from_extent: list of observations from extent

get_obs_list_from_codes: list of observations from codes

get_lizard_groundwater: get single obs object

get_timeseries_tube: get timeseries for a tube

get_timeseries_uuid: get timeseries for a uuid

hydropandas.io.lizard.check_status_obs(metadata, timeseries)[source]

Checks if a monitoring tube is still active.

If there are no measurements in the last 180 days, the monitoring tube is considered inactive.

Parameters:

metadata (dict) – metadata of the monitoring tube
timeseries (pandas DataFrame) – timeseries of the monitoring well

Return type:

metadata DataFrame including the status of the monitoring well

hydropandas.io.lizard.extent_to_wgs84_polygon(extent, crs=28992)[source]

Translates an extent (xmin, xmax, ymin, ymax) to a polygon with coordinate system WGS84.

Parameters:

extent (list or tuple) – extent in epsg 28992 within which the observations are collected.
crs (str, int or pyproj.CRS) – The coordinate reference system of the extent if it is not wgs84 the coordinates are transformed, by default EPSG: 28992.

Return type:

polygon of the extent with coordinate system WGS84

hydropandas.io.lizard.get_lizard_groundwater(code, tube_nr=None, tmin=None, tmax=None, type_timeseries=None, which_timeseries=('hand', 'diver'), datafilters=None, combine_method='merge', only_metadata=False, organisation='vitens', auth=None, crs=28992)[source]

Extracts the metadata and timeseries of an observation well from a LIZARD-API based on the code of a monitoring well.

Parameters:

code (str) – code of the measuring well, e.g. ‘27B-0444’
tube_nr (int, optional) – select specific tube top Default selects tube_nr = 1
tmin (str YYYY-m-d, optional) – start of the observations, by default the entire serie is returned
tmax (str YYYY-m-d, optional) – end of the observations, by default the entire serie is returned
type_timeseries (str, optional (deprecated)) – Old keyword, use which_timeseries instead.
which_timeseries (tuple of str, optional) – Which timeseries to retrieve. Options: “hand”, “diver”, “diver_validated”. Defaults to (“hand”, “diver”) (which should be correct for Vitens).
datafilters (list of strings, optional) – Methods to filter the timeseries data. If None (default), all measurements will be shown. Currently implemented datafilter methods: “remove_unvalidated_diver_values_when_validated_available”: Removes diver values before last date with validated diver. “remove_hand_during_diver_period”: Removes hand measurements during periods where diver or diver_validated measurements are available.
combine_method (str, optional) – “merge” (vertical stack with ‘origin’ column) or “columns” (side-by-side columns). If None, defaults to “merge”.
only_metadata (bool, optional) – if True only metadata is returned and no time series data. The default is False.
organisation (str, optional) – organisation as used by Lizard, currently only “vitens” is officially supported.
auth (tuple, optional) – authentication credentials for the API request, e.g.: (“__key__”, your_api_key)
crs (str, int or pyproj.CRS, optional) – The coordinate reference system of the extent and the observations, if it differs from the crs in Lizard the coordinates are transformed, by default EPSG: 28992.

Returns:

measurements (pd.DataFrame) – returns a DataFrame with metadata and timeseries
tube_metadata (dict) – dictionary containing metadata

hydropandas.io.lizard.get_metadata_mw_from_code(code, organisation='vitens', auth=None)[source]

Extracts the Groundwater Station parameters from a monitoring well based on the code of the monitoring well.

Parameters:

code (str) – code of the monitoring well
organisation (str) – organisation indicating URL endpoint, currently only “vitens” is officially supported.
auth (tuple, optional) – authentication credentials for the API request, e.g.: (“__key__”, your_api_key)

Raises:

ValueError – if code of the monitoring well is not known

Returns:

groundwaterstation_metadata – dictionary with all available metadata of the monitoring well and its filters

Return type:

dict

hydropandas.io.lizard.get_metadata_tube(metadata_mw, tube_nr, auth=None, crs=28992)[source]

Extract the metadata for a specific tube from the monitoring well metadata.

Parameters:

metadata_mw (dict) – dictionary with all available metadata of the monitoring well and all its filters
tube_nr (int or None) – select metadata from a specific tube number
auth (tuple, optional) – authentication credentials for the API request, e.g.: (“__key__”, your_api_key)
crs (str, int or pyproj.CRS, optional) – The coordinate reference system of the observation, if it differs from the crs in Lizard the coordinates are transformed, by default EPSG: 28992.

Raises:

ValueError – if code of the monitoring well is invalid.

Returns:

dictionary with metadata of a specific tube
.. warning:: – This function assumes that there is only one ‘hand’ timeseries and one ‘diver’ timeseries for each tube. This seems to comply with the Vitens use of Lizard.

hydropandas.io.lizard.get_obs_list_from_codes(codes, ObsClass, tube_nr='all', tmin=None, tmax=None, type_timeseries=None, which_timeseries=('hand', 'diver'), datafilters=None, combine_method='merge', only_metadata=False, organisation='vitens', auth=None, crs=28992)[source]

Get all observations from a list of codes of the monitoring wells and a list of tube numbers.

Parameters:

codes (list of str or str) – codes of the monitoring wells
ObsClass (type) – class of the observations, e.g. GroundwaterObs
tube_nr (lst of str) – list of tube numbers of the monitoring wells that should be selected. By default ‘all’ available tubes are selected.
tmin (str YYYY-m-d, optional) – start of the observations, by default the entire serie is returned
tmax (str YYYY-m-d, optional) – end of the observations, by default the entire serie is returned
type_timeseries (str, optional (deprecated)) – Old keyword, use which_timeseries instead.
which_timeseries (tuple of str, optional) – Which timeseries to retrieve. Options: “hand”, “diver”, “diver_validated”. Defaults to (“hand”, “diver”) (which should be correct for Vitens).
datafilters (list of strings, optional) – Methods to filter the timeseries data. If None (default), all measurements will be shown. Currently implemented datafilter methods: “remove_unvalidated_diver_values_when_validated_available”: Removes diver values before last date with validated diver. “remove_hand_during_diver_period”: Removes hand measurements during periods where diver or diver_validated measurements are available.
combine_method (str, optional) – “merge” (vertical stack with ‘origin’ column) or “columns” (side-by-side columns). If None, defaults to “merge”.
only_metadata (bool, optional) – if True only metadata is returned and no time series data. The default is False.
organisation (str, optional) – organisation as used by Lizard, currently only “vitens” is officially supported.
auth (tuple, optional) – authentication credentials for the API request, e.g.: (“__key__”, your_api_key)
crs (str, int or pyproj.CRS, optional) – The coordinate reference system of the extent and the observations, if it differs from the crs in Lizard the coordinates are transformed, by default EPSG: 28992.

Returns:

list of observations

Return type:

obs_list

hydropandas.io.lizard.get_obs_list_from_extent(extent, ObsClass, tube_nr='all', tmin=None, tmax=None, type_timeseries=None, which_timeseries=('hand', 'diver'), datafilters=None, combine_method='merge', only_metadata=False, page_size=100, nr_threads=10, organisation='vitens', auth=None, crs=28992)[source]

Get all observations within a specified extent.

Parameters:

extent (list or shapefile) – get groundwater monitoring wells within this extent [xmin, xmax, ymin, ymax] or within a predefined Polygon from a shapefile
ObsClass (type) – class of the observations, e.g. GroundwaterObs
tube_nr (lst of str) – list of tube numbers of the monitoring wells that should be selected. By default ‘all’ available tubes are selected.
tmin (str, optional) – start of the observations (format YYYY-m-d), by default the entire series is returned
tmax (str, optional) – end of the observations (format YYYY-m-d), by default the entire series is returned
type_timeseries (str, optional (deprecated)) – Old keyword, use which_timeseries instead.
which_timeseries (tuple of str, optional) – Which timeseries to retrieve. Options: “hand”, “diver”, “diver_validated”. Defaults to (“hand”, “diver”) (which should be correct for Vitens).
datafilters (list of strings, optional) – Methods to filter the timeseries data. If None (default), all measurements will be shown. Currently implemented datafilter methods: “remove_unvalidated_diver_values_when_validated_available”: Removes diver values before last date with validated diver. “remove_hand_during_diver_period”: Removes hand measurements during periods where diver or diver_validated measurements are available.
combine_method (str, optional) – “merge” (vertical stack with ‘origin’ column) or “columns” (side-by-side columns). If None, defaults to “merge”.
only_metadata (bool, optional) – if True only metadata is returned and no time series data. The default is False.
organisation (str) – organisation as used by Lizard, currently only “vitens” is officially supported.
auth (tuple, optional) – authentication credentials for the API request, e.g.: (“__key__”, your_api_key)
page_size (int, optional) – number of records to retrieve per page, default is 100
nr_threads (int, optional) – number of threads to use for the API requests, default is 10
crs (str, int, pyproj.CRS or None, optional) – The coordinate reference system of the extent (input) and the observations (output), by default EPSG: 28992.

Returns:

obs_col – ObsCollection DataFrame with the ‘obs’ column

Return type:

TYPE

hydropandas.io.lizard.get_timeseries_tube(tube_metadata, tmin, tmax, type_timeseries=None, which_timeseries=('hand', 'diver'), datafilters=None, combine_method='merge', organisation='vitens', auth=None)[source]

Extracts specified timeseries for a tube and combines them as requested.

Parameters:

tube_metadata (dict) – metadata of a tube
tmin (str YYYY-m-d, optional) – start of the observations
tmax (str YYYY-m-d, optional) – end of the observations
type_timeseries (str, optional (deprecated)) – deprecated, use ‘which_timeseries’ and ‘combine_method’ instead.
which_timeseries (tuple of str, optional) – Which timeseries to retrieve. Options: “hand”, “diver”, “diver_validated”. Defaults to (“hand”, “diver”) (which should be correct for Vitens).
datafilters (list of strings, optional) – Methods to filter the timeseries data. If None (default), all measurements will be shown. Currently implemented datafilter methods: “remove_unvalidated_diver_values_when_validated_available”: Removes diver values before last date with validated diver. “remove_hand_during_diver_period”: Removes hand measurements during periods where diver or diver_validated measurements are available.
combine_method (str, optional) – “merge” (vertical stack with ‘origin’ column) or “columns” (side-by-side columns). If None, defaults to “merge”.
organisation (str, optional) – organisation as used by Lizard.
auth (tuple, optional) – authentication credentials for the API request, e.g.: (“__key__”, your_api_key)

Returns:

measurements (pandas DataFrame) – timeseries of the monitoring well
metadata_df (dict) – metadata of the monitoring well

hydropandas.io.lizard.get_timeseries_uuid(uuid, tmin, tmax, page_size=100000, nr_threads=10, organisation='vitens', auth=None)[source]

Get the time series (hand or diver) using the uuid. Handles pagination to retrieve all results across multiple pages.

Parameters:

uuid (str) – Universally Unique Identifier of the tube and type of time series.
tmin (str YYYY-m-d) – start of the observations, by default the entire serie is returned
tmax (str YYYY-m-d) – end of the observations, by default the entire serie is returned
page_size (int, optional) – Query parameter which can extend the response size. The default is 100000.
nr_threads (int, optional) – number of threads to use for the API requests, default is 10
organisation (str, optional) – organisation as used by Lizard, currently only “vitens” is officially supported.
auth (tuple, optional) – authentication credentials for the API request, e.g.: (“__key__”, your_api_key)

Returns:

pandas DataFrame with the timeseries of the monitoring well

Return type:

pd.DataFrame

hydropandas.io.lizard.translate_flag(timeseries)[source]

Translates Vitens Lizard flags from integer to text.

Parameters:: timeseries (pandas.DataFrame) – timeseries of a monitoring well with flags
Returns:: timeseries – timeseries with translated quality flags
Return type:: pandas.DataFrame

hydropandas.io.menyanthes module

hydropandas.io.menyanthes.matlab2datetime(tindex)[source]

Transform a MATLAB serial date number to a Python datetime object, rounded to seconds.

Parameters:: tindex (float) – The MATLAB serial date number to convert.
Returns:: datetime – The equivalent datetime object in Python.
Return type:: datetime.datetime

Notes

MATLAB serial date numbers represent the number of days elapsed since January 1, 0000 (the proleptic Gregorian calendar), with January 1, 0000 as day 1. Fractions of a day can be represented as a decimal.

The returned datetime object is rounded to the nearest second.

Examples

>>> matlab2datetime(719529.496527778)
datetime.datetime(2019, 1, 1, 11, 55, 2)

hydropandas.io.menyanthes.read_file(path, ObsClass, load_oseries=True, load_stresses=True, crs=28992)[source]

Read data from a Menyanthes file and create observation objects.

Parameters:

path (str or pathlib.Path) – Full path of the Menyanthes file (.men) to read.
ObsClass (GroundwaterObs or WaterlvlObs) – Class of observation object to create.
load_oseries (bool, optional) – Flag indicating whether to load observation series or not, by default True.
load_stresses (bool, optional) – Flag indicating whether to load stresses or not, by default True.
crs (str, int or pyproj.CRS, optional) – The coordinate reference system of the observations. By default, EPSG:28992.

Returns:

obs_list – List of observation objects created from the Menyanthes file.

Return type:

list

hydropandas.io.menyanthes.read_oseries(mat)[source]

Read the oseries from a mat file from menyanthes.

Parameters:: mat (dict) – A dictionary object containing the Menyanthes file data.
Returns:: A dictionary containing oseries data, with oseries names as keys and their corresponding metadata and values as values.
Return type:: dict

Notes

This function reads the oseries data from a Menyanthes file in .mat format and returns it in a dictionary format. The oseries data contains the following metadata:

name: The name of the oseries.

x: The x-coordinate of the oseries location.

y: The y-coordinate of the oseries location.

source: The data source.

unit: The unit of measurement.

In addition to the metadata, the oseries data also contains a pandas Series object named ‘values’, which contains the time series data for the oseries.

Examples

>>> mat = loadmat('menyanthes_file.mat')
>>> d_h = read_oseries(mat)

hydropandas.io.menyanthes.read_stresses(mat)[source]

Reads the stresses from a mat file from menyanthes.

Parameters:: mat (dict) – A dictionary object containing the mat file.
Returns:: A dictionary object containing the stresses data.
Return type:: dict

hydropandas.io.modflow module

hydropandas.io.modflow.interp_weights(xy, uv, d=2)[source]

Calculate interpolation weights [1].

Parameters:

xy (np.array) – array containing x-coordinates in first column and y-coordinates in second column
uv (np.array) – array containing coordinates at which interpolation weights should be calculated, x-data in first column and y-data in second column
d (int, optional) – dimension of data? (the default is 2, which works for 2D data)

Returns:

vertices (np.array) – array containing interpolation vertices
weights (np.array) – array containing interpolation weights per point

References

speedup-scipy-griddata-for-multiple-interpolations-between-two-irregular-grids

hydropandas.io.modflow.interpolate(values, vtx, wts, fill_value=nan)[source]

Interpolate values at locations defined by vertices and points [2], as calculated by interp_weights function.

Parameters:

values (np.array) – array containing values to interpolate
vtx (np.array) – array containing interpolation vertices, see interp_weights()
wts (np.array) – array containing interpolation weights, see interp_weights()
fill_value (float) – fill value for points that have to be extrapolated (e.g. at or beyond edges of the known points)

Returns:

arr – array containing interpolated values at locations as given by vtx and wts

Return type:

np.array

References

speedup-scipy-griddata-for-multiple-interpolations-between-two-irregular-grids

hydropandas.io.modflow.read_imod_results(obs_collection, ml, runfile, mtime, model_ws, modelname='', nlay=None, exclude_layers=0)[source]

Read imod model results at point locations.

Parameters:

obs_collection (ObsCollection) – collection of observations at which points imod results will be read
ml (flopy.modflow.mf.model) – modflow model
runfile (Runfile) – imod runfile object
mtime (list of datetimes) – datetimes corresponding to the model periods
model_ws (str or pathlib.Path) – model workspace with imod model
nlay (int, optional) – number of layers if None the number of layers from ml is used.
modelname (str) – modelname
exclude_layers (int) – exclude modellayers from being read from imod

hydropandas.io.modflow.read_modflow_results(obs_collection, ml, hds_arr, mtime, modelname='', nlay=None, exclude_layers=None, method='linear')[source]

Read modflow groundwater heads at points in obs_collection.

Parameters:

obs_collection (ObsCollection) – locations of model observation
ml (flopy.modflow.mf.model) – modflow model
hds_arr (numpy array) – heads with shape (ntimesteps, nlayers, nrow, ncol)
mtime (list of datetimes) – dates for each model timestep
modelname (str, optional) – modelname
nlay (int, optional) – number of layers if None the number of layers from ml is used.
exclude_layers (list of int, optional) – exclude the observations in these modellayers
method (str, optional) – interpolation method, either ‘linear’ or ‘nearest’, default is linear.

hydropandas.io.pastas module

Created on Wed Sep 12 12:15:42 2018.

@author: Artesia

hydropandas.io.pastas.create_pastastore(oc, pstore, pstore_name='', conn=None, add_metadata=True, col=None, kind='oseries', overwrite=False)[source]

add observations to a new or existing pastastore.

Parameters:

oc (observation.ObsCollection) – collection of observations
pstore (pastastore.PastaStore, optional) – Existing pastastore, if None a new pastastore is created
pstore_name (str, optional) – Name of the pastastore only used if pstore is None
conn (pastastore.connectors) – connector for database
col (str or None, optional) – the column of the obs dataframe to use in pastas. The first numeric column is used if col is None, by default None.
kind (str, optional) – The kind of series that is added to the pastastore
add_metadata (boolean, optional) – If True metadata from the observations added to the pastastore
overwrite (boolean, optional) – if True, overwrite existing series in pastastore, default is False

Returns:

pstore – the pastastore with the series from the ObsCollection

Return type:

pastastore.PastaStore

hydropandas.io.pastas.read_pastastore_item(pstore, libname, name)[source]

Read item from pastastore library.

Parameters:

pstore (pastastore.PastaStore) – pastastore object
libname (str) – name of library containing item
name (str) – name of item

Returns:

series (pd.Series) – time series for item
meta (dict) – dictionary containing metadata

Raises:

ValueError – if library is not oseries or stresses

hydropandas.io.pastas.read_pastastore_library(pstore, libname, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, metadata_mapping=None)[source]

Read pastastore library.

Parameters:

pstore (pastastore.PastaStore) – pastastore object
libname (str) – name of library to read
ObsClass (Obs, optional) – type of Obs to read data as, by default GroundwaterObs
metadata_mapping (dict, optional) – dictionary containing map between metadata field names in pastastore and metadata field names expected by hydropandas, by default None.

Returns:

obs_list – list of Obs containing data

Return type:

list of Obs

hydropandas.io.solinst module

hydropandas.io.solinst.read_solinst_file(path, transform_coords=True)[source]

Read Solinst logger file (XLE)

Parameters:

path (str or pathlib.Path) – path to Solinst file (.xle)
transform_coords (boolean) – convert coordinates from WGS84 to RD

Returns:

df (pandas.DataFrame) – DataFrame containing file content
meta (dict, optional) – dict containing meta

hydropandas.io.waterinfo module

hydropandas.io.waterinfo.get_locations_gdf(crs=28992)[source]

Get locations from ddlpy and return as geodataframe

Parameters:: crs (str, int or pyproj.CRS) – coordinate reference system for the returned geodataframe. The default is 28992 (RD).
Returns:: gdf – geodataframe with locations. This dataframe is needed to obtain measurements using ddlpy.
Return type:: geopandas.GeoDataFrame

hydropandas.io.waterinfo.get_locations_within_extent(gdf, extent=(482.06, 306602.42, 284182.97, 637049.52))[source]

Get locations from ddlpy and return as geodataframe. Both gdf and extent should be in the same crs.

Parameters:

gdf (geopandas.GeoDataFrame) – geodataframe with locations. This dataframe is needed to obtain measurements using ddlpy
extent (tuple, optional) – extent of the locations. The default is the extent of the Netherlands (RD).

Returns:

gdf – geodataframe with locations. This dataframe is needed to obtain measurements using ddlpy

Return type:

geopandas.GeoDataFrame

hydropandas.io.waterinfo.get_measurements_ddlpy(location_gdf=None, locatie=None, grootheid_code=None, groepering_code=None, parameter_code=None, proces_type=None, tmin=None, tmax=None, crs=28992)[source]

Get measurements from ddlpy for a specific location and grootheid_code

Parameters:

location_gdf (geopandas.GeoDataFrame, optional) – geodataframe with one or more locations, default is None
locatie (str or list of str, optional) – name(s) of the location
grootheid_code (str or list of str, optional) – code(s) of the grootheid
groepering_code (str or list of str, optional) – code(s) of the groepering
parameter_code (str or list of str, optional) – code(s) of the parameter
proces_type (str or list of str, optional) – code(s) of the proces type, e.g. ‘meting’
tmin (pd.Timestamp, optional) – start date of the measurements, default is 2025-01-01
tmax (pd.Timestamp, optional) – end date of the measurements, default is now
crs (str, int or pyproj.CRS, optional) – desired coordinate reference system of the observation, if it differs from 25831 the coordinates are transformed, default is 28992

Returns:

df (pandas.DataFrame) – DataFrame with measurements
meta (dict) – dict with metadata

hydropandas.io.waterinfo.get_obs_list_from_extent(extent, ObsClass, locatie=None, grootheid_code=None, groepering_code=None, parameter_code=None, proces_type=None, tmin=None, tmax=None, only_metadata=False, keep_all_obs=False, crs=28992, location_gdf=None)[source]

Get observations within a specific extent and optionally for a specific location and grootheid_code.

Parameters:

extent (list, tuple, numpy-array or None, optional) – get waterinfo measurements within this extent [xmin, xmax, ymin, ymax]
ObsClass (type) – class of the observations, e.g. WaterlvlObs
locatie (str or list of str, optional) – select only measurement with this location(s), e.g. ‘schoonhoven’, default is None
grootheid_code (str or list of str, optional) – select only measurement with this grootheid_code, e.g. ‘WATHTE’, default is None
groepering_code (str or list of str, optional) – select only measurement with this groepering_code, e.g. ‘GETETBRKD2’, default is None
parameter_code (str or list of str, optional) – select only measurement with this parameter_code, e.g. ‘Cl’, default is None
proces_type (str or list of str, optional) – select only measurement with this proces type, e.g. ‘meting’, default is None
tmin (str or None, optional) – start time of observations. The default is None.
tmax (str or None, optional) – end time of observations. The default is None.
only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.
keep_all_obs (bool, optional) – if False, only observations with measurements are kept. The default is True.
crs (str, int or pyproj.CRS, optional) – coordinate reference system of the extent. The default is 28992 (RD).
location_gdf (GeoDataFrame, optional) – geodataframe with the locations of the measurements you want to include. If location_gdf is provided the provided extent and epgs will be ignored.

Returns:

obs_list – list with Obs objects

Return type:

list

hydropandas.io.waterinfo.get_waterinfo_obs(path=None, location_gdf=None, locatie=None, grootheid_code=None, groepering_code=None, parameter_code=None, proces_type=None, tmin=None, tmax=None, crs=28992, **kwargs)[source]

Get waterinfo observations from a file or ddlpy

Parameters:

path (str or pathlib.Path, optional) – path to waterinfo file (.zip or .csv), default is None
location_gdf (geopandas.GeoDataFrame, optional) – geodataframe with locations, default is None
locatie (str or list of str, optional) – name of the location, e.g. ‘schoonhoven’, default is None
grootheid_code (str or list of str, optional) – code(s) of the grootheid, e.g. ‘WATHTE’, default is None
groepering_code (str or list of str, optional) – code(s) of the groepering, e.g. ‘GETETBRKD2’, default is None
parameter_code (str or list of str, optional) – code(s) of the parameter, e.g. ‘Cl’, default is None
proces_type (str or list of str, optional) – code(s) of the proces type, e.g. ‘meting’, default is None
tmin (pd.Timestamp, optional) – start date of the measurements, default is None
tmax (pd.Timestamp, optional) – end date of the measurements, default is None
crs (str, int or pyproj.CRS, optional) – desired coordinate reference system of the observation, if it differs from 25831 the coordinates are transformed, default is 28992 (RD)

Returns:

df (pandas.DataFrame) – DataFrame with measurements
meta (dict) – dict with metadata

hydropandas.io.waterinfo.read_waterinfo_file(path, index_cols=None, return_metadata=True, value_col=None, location_col=None, xcol=None, ycol=None, crs=28992)[source]

Read waterinfo file (CSV or zip)

Parameters:

path (str or pathlib.Path) – path to waterinfo file (.zip or .csv)
index_cols (list of str, optional) – columns to use as index, default is [“WAARNEMINGDATUM”, “WAARNEMINGTIJD (MET/CET)”]
return_metadata (bool, optional) – if True return metadata, default is True
value_col (str, optional) – name of the column containing the measurement values, default is “NUMERIEKEWAARDE”
location_col (str, optional) – name of the column containing the location identifiers, default is “MEETPUNT_IDENTIFICATIE”
xcol (str, optional) – name of the column containing the x coordinates, default is “X”
ycol (str, optional) – name of the column containing the y coordinates, default is “Y”
crs (str, int or pyproj.CRS, optional) – desired coordinate reference system of the observation, if it differs from 25831 the coordinates are transformed, default is 28992 (RD).

Returns:

df (pandas.DataFrame) – DataFrame containing file content
metadata (dict, optional) – dict containing metadata, returned if return_metadata is True, default is False

hydropandas.io.waterinfo.read_waterinfo_obs(file_or_dir, ObsClass, progressbar=False, crs=28992, **kwargs)[source]

Read waterinfo file or directory and extract locations and observations.

Parameters:

file_or_dir (str or pathlib.Path) – path to file or directory
ObsClass (Obs type) – type of Obs to store data in
progressbar (bool, optional) – show progressbar if True, default is False
crs (str, int or pyproj.CRS, optional) – coordinate reference system of the observations. The default is 28992 (RD).

Returns:

obs_collection – list of Obs objects

Return type:

list

hydropandas.io.wiski module

hydropandas.io.wiski.read_wiski_dir(dirname, ObsClass=None, suffix='.csv', unpackdir=None, force_unpack=False, preserve_datetime=False, keep_all_obs=True, **kwargs)[source]

Reads WISKI CSV files from a directory and returns a list of observation objects.

Parameters:

dirname (str or pathlib.Path) – The path of the directory containing the WISKI CSV files.
ObsClass (object, optional) – The observation class to use for creating observation objects. Default is None.
suffix (str, optional) – The file extension of the WISKI CSV files. Default is “.csv”.
unpackdir (str or pathlib.Path, optional) – The directory to which the files should be unpacked. Default is None.
force_unpack (bool, optional) – If True, forces the files to be unpacked even if they are already in the target directory. Default is False.
preserve_datetime (bool, optional) – If True, preserves the original modification times of the files when unpacking them. Default is False.
keep_all_obs (bool, optional) – If True, keeps all observation objects even if they have no measurements available. Default is True.
**kwargs – Additional keyword arguments to pass to the from_wiski method of the ObsClass object.

Returns:

A list of observation objects created from the WISKI CSV files in the directory.

Return type:

list

Raises:

FileNotFoundError – If no WISKI CSV files are found in the directory.

hydropandas.io.wiski.read_wiski_file(path, sep=';', header_sep=None, header_identifier='#', read_series=True, translate_dic=None, tz_localize=True, unit='', parse_dates=None, dayfirst=False, **kwargs)[source]

Read data from a WISKI file.

Parameters:

pathstr or pathlib.Path: The path of the file to be read.
sepstr, optional (default=”;”): The delimiter used to separate fields in the file.
header_sepstr, optional (default=None): The delimiter used to separate fields in the header. If None, the function will try to automatically detect the separator.
header_identifierstr, optional (default=”#”): The character used to identify header lines.
read_seriesbool, optional (default=True): Whether to read the time series data from the file.
translate_dicdict, optional (default=None): A dictionary mapping header field names to the desired output names.
parse_dateslist of int, optional (default=None): A list of column indices to parse as dates. If None, no columns will be parsed as dates.
dayfirstbool, optional (default=False): Whether to interpret the first value in a date as the day (True) or the month (False).
tz_localizebool, optional (default=True): Whether to localize the datetime index to the machine’s timezone.
unitstr, optional (default=””): The unit of measurement of the data.
**kwargskeyword arguments: Additional arguments to pass to the pandas read_csv function.

Returns:

datapandas.DataFrame or None: A dataframe containing the time series data from the file. Returns None if read_series is False.
metadatadict: A dictionary containing metadata about the data in the file.

hydropandas.io.wow module

hydropandas.io.wow.get_nearest_station_df(locations: DataFrame, stations: DataFrame, xcol: str = 'x', ycol: str = 'y', ignore: list[str] | None = None) → list[str][source]

Find the nearest stations that measure ‘meteo_var’ closest to the coordinates in ‘locations’.

Parameters:

locations (pandas.DataFrame) – DataFrame containing x and y coordinates
stations (pandas DataFrame, optional) – if None stations will be obtained using the get_stations function. The default is None.
xcol (str) – The name of the column in the locations dataframe with the x values, by default ‘x’. If lon in xcol, longitude is assumed.
ycol (str) – The name of the column in the locations dataframe with the y values, by default ‘y’. If lat in ycol, latitude is assumed.
ignore (list, optional) – A list of station IDs to ignore, by default None.

Returns:

stns – A list of station IDs that are closest to each location.

Return type:

list

hydropandas.io.wow.get_nearest_station_xy(xy: list[list[float]], stations: DataFrame, ignore: list[str] | None = None) → list[str][source]

find the stations that measure closest to the given longitude and latitude coordinates.

Parameters:

xy (list or numpy array, optional) – longitude, latitude coordinates of the locations. e.g. [[10,25], [5,25]]
stations (pandas DataFrame, optional) – if None stations will be obtained using the get_stations function. The default is None.
ignore (list, optional) – list of stations to ignore. The default is None.

Returns:

stns – station numbers.

Return type:

list

Notes

assumes you have a structured rectangular grid.

hydropandas.io.wow.get_wow(meteo_var: str, stn: str, xy: list[float] | None = None, start: Timestamp = None, end: Timestamp = None) → tuple[DataFrame, dict][source]

Get weather observations and metadata from a WOW-KNMI station based on the station name or lat, lon location

Parameters:

meteo_var (str) – The meteorological variable to query, one of ‘rain_rate’, ‘temperature’, ‘wind’, ‘humidity’, ‘pressure_msl’.
stn (str) – The station ID of the WOW-KNMI station.
xy (List[float], optional) – The coordinates of the location to query in the format [lon, lat], by default None. If specified, the nearest station within 1 degree will be used.
start (pd.Timestamp, optional) – The start date and time of the observations, by default None. If None, the januari first of the year before will be used.
end (pd.Timestamp, optional) – The end date and time of the observations, by default None. If None, the yesterday will be used.

Returns:

A tuple of a dataframe with the observations and a dictionary with the station metadata.

Return type:

Tuple[pd.DataFrame, dict]

Raises:

ValueError – If both stn and xy are None or if meteo_var is not valid.

hydropandas.io.wow.get_wow_measurements(stn: str, meteo_var: str, start: Timestamp = None, end: Timestamp = None) → DataFrame[source]

Get measurements from a WOW-KNMI station.

Parameters:

stn (str) – The station ID of the WOW-KNMI station.
meteo_var (str) – The meteorological variable to query, one of ‘rain_rate’, ‘temperature’, ‘wind’, ‘humidity’, ‘pressure_msl’.
start (pd.Timestamp, optional) – The start date and time of the observations, by default None. If None, the januari first of the year before will be used.
end (pd.Timestamp, optional) – The end date and time of the observations, by default None. If None, the yesterday will be used.

Returns:

A dataframe with the measurements.

Return type:

pd.DataFrame

Raises:

ValueError – If meteo_var is not valid.
requests.HTTPError – If the request to the WOW-KNMI API fails.

hydropandas.io.wow.get_wow_metadata(stn: str) → dict[source]: Get metadata from a WOW-KNMI station.

hydropandas.io.wow.get_wow_stations(meteo_var: str = 'rain_rate', date: Timestamp | None = None, bbox: list[float] | None = None, obs_filter: str | None = None) → DataFrame[source]

Get a DataFrame with the observation stations from WOW-KNMI.

Parameters:

meteo_var (str, optional) – The meteorological variable to query, one of ‘rain_rate’, ‘temperature’, ‘wind’, ‘humidity’, ‘pressure_msl’ by default ‘rain_rate’.
date (Optional[pd.Timestamp], optional) – The date and time of the observations, by default the latest available (10 minutes ago).
bbox (Optional[List[float]], optional) – The bounding box of the spatial query in the format [min_lon, min_lat, max_lon, max_lat], by default the extent of the Netherlands.
obs_filter (Optional[str], optional) – The filter to apply to the observations, one of ‘wow_observations’, ‘official_observations’, by default None which selects all stations.

Returns:

A dataframe with the station information and observations.

Return type:

pd.DataFrame

Raises:

ValueError – If meteo_var or obs_filter are not valid choices.
requests.HTTPError – If the request to the WOW-KNMI API fails.