hydropandas

Subpackages

hydropandas.obs_collection module

Module with ObsCollection class for a collection of observations.

The ObsCollection class is a subclass of a pandas DataFrame with additional attributes and methods.

More information about subclassing pandas DataFrames can be found here: http://pandas.pydata.org/pandas-docs/stable/development/extending.html#extending-subclassing-pandas

class hydropandas.obs_collection.ObsCollection(*args, **kwargs)[source]

Bases: DataFrame

Class for a collection of point observations.

An ObsCollection object is a subclass of a pandas.DataFrame and allows for additional attributes and methods. Additional attributes are defined in the ‘_metadata’ attribute.

Parameters

  • observations (*args) –

  • DataFrame (list of observations or a pandas) –

:param : :param **kwargs can be one of these:

namestr

name of the observation collection

metadic

metadata of the observation collection

add_meta_to_df(key='all')[source]

Get the values from the meta dictionary of each observation object and add these to the ObsCollection as a column.

to the ObsCollection

Parameters

key (str, int, tuple, list, set or None, optional) – key in meta dictionary of observation object. If key is ‘all’, all keys are added. The default is ‘all’.

Returns

out – ObsCollection with extra columns with metadata

Return type

ObsCollection

add_obs_collection(obs_collection, check_consistency=True, inplace=False, **kwargs)[source]

Add one observation collection to another observation collection. See add_observation method for more details.

Parameters

  • obs_collection (hpd.ObsCollection) – ObsCollection object.

  • check_consistency (bool, optional) – If True the consistency of both collections is first checked. The default is True.

  • inplace (bool, optional) – If True, modifies the ObsCollection in place (do not create a new object). The default is False.

  • Obs.merge_observation (**kwargs passed to) –

    merge_metadatabool, optional

    If True and observations are merged the metadata of the two objects are merged. If there are any differences the overlap parameter is used to determine which metadata is used. If merge_metadata is False, the metadata of the original observation is always used for the merged observation. The default is True.

    overlapstr, optional

    How to deal with overlapping timeseries with different values. Options are: - error : Raise a ValueError - use_left : use the overlapping part from the existing observations - use_right : use the overlapping part from the new observation Default is ‘error’.

Raises

RuntimeError – when the observation collection is inconsistent.

Returns

merged ObsCollection if inplace=True.

Return type

ObsCollection or None

add_observation(o, check_consistency=True, inplace=False, **kwargs)[source]

Add an observation to an existing observation collection. If the observation exists the two observations are merged.

Parameters

  • o (hpd.observation.Obs) – Observation object.

  • check_consistency (bool, optional) – If True the consistency of the collection is first checked. The default is True.

  • inplace (bool, optional) – If True, modifies the ObsCollection in place (do not create a new object). The default is False.

  • Obs.merge_observation (**kwargs passed to) –

    merge_metadatabool, optional

    If True and observations are merged the metadata of the two objects are merged. If there are any differences the overlap parameter is used to determine which metadata is used. If merge_metadata is False, the metadata of the original observation is always used for the merged observation. The default is True.

    overlapstr, optional

    How to deal with overlapping timeseries with different values. Options are: - error : Raise a ValueError - use_left : use the overlapping part from the existing observations - use_right : use the overlapping part from the new observation Default is ‘error’.

Raises

  • RuntimeError – when the observation collection is inconsistent.

  • TypeError – when the observation type is wrong.

Return type

None.

copy(deep=False)[source]

Make a copy of this object’s indices and data.

Parameters

deep (bool, default True) – Make a deep copy, including a deep copy of the observation objects. With deep=False neither the indices nor the data are copied.

Return type

ObsCollection

classmethod from_artdino_dir(dirname=None, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, subdir='csv', suffix='.csv', unpackdir=None, force_unpack=False, preserve_datetime=False, keep_all_obs=True, name=None, **kwargs)[source]

Read a dino directory.

Parameters

  • extent (list, optional) – get dinodata online within this extent [xmin, xmax, ymin, ymax]

  • dirname (str, optional) – directory name, can be a .zip file or the parent directory of subdir

  • ObsClass (type) – class of the observations, e.g. GroundwaterObs or WaterlvlObs

  • subdir (str) – subdirectory of dirname with data files

  • suffix (str) – suffix of files in subdir that will be read

  • unpackdir (str) – destination directory of the unzipped file

  • force_unpack (boolean, optional) – force unpack if dst already exists

  • preserve_datetime (boolean, optional) – use date of the zipfile for the destination file

  • keep_all_obs (boolean, optional) – add all observation points to the collection, even without data or metadata

  • name (str, optional) – the name of the observation collection

  • kwargs – kwargs are passed to the hydropandas.io.dino.read_dino_dir() function

Returns

cls(obs_df) – collection of multiple point observations

Return type

ObsCollection

classmethod from_bro(extent=None, bro_id=None, name='', tmin=None, tmax=None, only_metadata=False, keep_all_obs=True, epsg=28992, ignore_max_obs=False, engine='hydropandas')[source]

Get all the observations within an extent or within a groundwatermonitoring net.

Parameters

  • extent (list, tuple, numpy-array or None, optional) – get groundwater monitoring wells within this extent [xmin, xmax, ymin, ymax]

  • bro_id (str or None, optional) – starts with ‘GMN’.

  • name (str, optional) – name of the observation collection

  • tmin (str or None, optional) – start time of observations. The default is None.

  • tmax (str or None, optional) – end time of observations. The default is None.

  • only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.

  • keep_all_obs (boolean, optional) – add all observation points to the collection, even without measurements

  • epsg (int, optional) – epsg code of the extent. The default is 28992 (RD).

  • ignore_max_obs (bool, optional) – by default you get a prompt if you want to download over a 1000 observations at once. if ignore_max_obs is True you won’t get the prompt. The default is False

  • engine (str, optional) – Select how data from the bro-database is obtained, options are ‘hydropandas’, ‘brodata’ or ‘brodata_gm’. ‘brodata_gm’ is only available when extent is not None. When engine=’brodata_gm’ use the dataset Grondwatermonitoring (GM) in samenhang - karakteristieken, hosted by PDOK. This up-to-date dataset combines well- and tube-properties. So users do not have to download each individual Groundwater Monitoring Well (GMW), which speeds up the request. The gm-dataset does not contain the attributes tube_top and ground_level, so you need to use engine=’brodata’ or ‘hydropandas’ if you need those. The Groundwater Level Dossiers (GLD) are still downloaded individually. The default is ‘hydropandas’.

Returns

ObsCollection DataFrame with the ‘obs’ column

Return type

ObsCollection

classmethod from_bronhouderportaal_bro(dirname, full_meta=False)[source]

Get all the metadata from dirname.

Parameters

  • dirname (str) – name of dirname that holds XML files

  • full_meta (bool , optional) – process all metadata. The default is False.

Returns

ObsCollection DataFrame without the ‘obs’ column

Return type

ObsCollection

classmethod from_dataframe(df, obs_list=None, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>)[source]

Create an observation collection from a DataFrame and optionally a list of observations. If no list of observations is given empty observations are added of the type specified by ObsClass.

Parameters

  • df (pandas DataFrame) – input dataframe. If this dataframe has a column named ‘obs’ the column is replaced with new observation objects.

  • obs_list (list of observation.Obs, optional) – list of observations. Default is None

  • ObsClass (class, optional) – observation class used to create empty obs object, by default GroundwaterObs

Returns

ObsCollection DataFrame with the ‘obs’ column

Return type

ObsCollection

classmethod from_dict(d, **kwargs)[source]

Create an observation collection from a dictionary. The dictionary should have the same format as dictionaries created with the to_dict method of an ObsCollection.

Parameters

d (dict) – dictionary with data

Return type

ObsCollection

classmethod from_dino(dirname=None, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, subdir='DINO_Grondwaterstanden', suffix=None, keep_all_obs=True, name=None, **kwargs)[source]

Read dino data within an extent from the server or from a directory with downloaded files.

Parameters

  • dirname (str, optional) – directory name, can be a .zip file or the parent directory of subdir

  • ObsClass (type) – class of the observations, so far only GroundwaterObs is supported

  • subdir (str) – subdirectory of dirname with data files. For old school dino zip files this should be “Grondwaterstanden_Put”. For new style the default value DINO_Grondwaterstanden is sufficient. The default is DINO_Grondwaterstanden.

  • suffix (str or None, optional) – suffix of files in subdir that will be read. For old school dino zip files this should be ‘1.csv’. For new style the default value None is sufficient. The default is None.

  • keep_all_obs (boolean, optional) – add all observation points to the collection, even the points without measurements or metadata

  • name (str, optional) – the name of the observation collection

  • kwargs – kwargs are passed to the hydropandas.io.dino.read_dino_dir() function

Returns

cls(obs_df) – collection of multiple point observations

Return type

ObsCollection

classmethod from_excel(path, meta_sheet_name='metadata')[source]

Create an observation collection from an excel file. The excel file should have the same format as excel files created with the to_excel method of an ObsCollection.

Parameters

  • path (str) – full file path (including extension) of the excel file.

  • meta_sheet_name (str, optional) – sheetname with metadata. The default is “metadata”.

Return type

ObsCollection

Notes

if you write an excel file using the ‘to_excel’ method and read an excel with the ‘read_excel’ method you lose this information: - The ‘name’ and ‘meta’ attributes of the ObsCollection - metadata of each Observation stored in the ‘meta’ attribute

If you don’t want to lose this data consider using the to_json and read_json function.

classmethod from_fews_xml(file_or_dir=None, xmlstring=None, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, name='fews', translate_dic=None, filterdict=None, locations=None, remove_nan=True, low_memory=True, unpackdir=None, force_unpack=False, preserve_datetime=False, **kwargs)[source]

Read one or several FEWS PI-XML files.

Parameters

  • file_or_dir (str) – zip, xml or directory with zips or xml files to read

  • xmlstring (str or None) – string with xml data, only used if file_or_dir is None. Default is None

  • ObsClass (type or dict) – class of the observations, e.g. GroundwaterObs or WaterlvlObs or a dictionary with a class for each locationId

  • name (str, optional) – name of the observation collection, ‘fews’ by default

  • translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘locatie’}.

  • filterdict (dict, optional) – dictionary with tag name to apply filter to as keys, and list of accepted names as dictionary values to keep in final result, i.e. {“locationId”: [“B001”, “B002”]}

  • locations (list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read. Only supported by low_memory=True method!

  • low_memory (bool, optional) – whether to use xml-parsing method with lower memory footprint, default is True

  • remove_nan (boolean, optional) – remove nan values from measurements, flag information about the nan values is also lost, only used if low_memory=False

  • unpackdir (str) – destination directory to unzip file if path is a .zip

  • force_unpack (boolean, optional) – force unpack if dst already exists

  • preserve_datetime (boolean, optional) – whether to preserve datetime from zip archive

Returns

cls(obs_df) – collection of multiple point observations

Return type

ObsCollection

classmethod from_imod(obs_collection, ml, runfile, mtime, model_ws, modelname='', nlay=None, exclude_layers=0)[source]

Read imod model results at point locations.

Parameters

  • obs_collection (ObsCollection) – collection of observations at which points imod results will be read

  • ml (flopy.modflow.mf.model) – modflow model

  • runfile (Runfile) – imod runfile object

  • mtime (list of datetimes) – datetimes corresponding to the model periods

  • model_ws (str) – model workspace with imod model

  • nlay (int, optional) – number of layers if None the number of layers from ml is used.

  • modelname (str) – modelname

  • exclude_layers (int) – exclude modellayers from being read from imod

classmethod from_json(path, **kwargs)[source]

Create an observation collection from a json file. The json file should have the same format as json files created with the to_json method of an ObsCollection.

Parameters

path (str) – full file path (including extension) of the json file.

Return type

ObsCollection

classmethod from_knmi(locations=None, stns=None, xy=None, meteo_vars=('RH',), name='', starts=None, ends=None, ObsClasses=None, fill_missing_obs=False, interval='daily', use_api=True, raise_exceptions=True, progress_callback=None, fill_missing_obs_with_factor=False)[source]

Get knmi observations from a list of locations or a list of stations.

Parameters

  • locations (pandas DataFrame or None) – dataframe with columns ‘x’ and ‘y’ as coordinates. The default is None

  • stns (list of str or None) – list of knmi stations. The default is None

  • xy (list or numpy array, optional) – xy coordinates of the locations. e.g. [[10,25], [5,25]]

  • meteo_vars (list or tuple of str) – meteo variables e.g. [“RH”, “EV24”]. The default is (“RH”). See list of all possible variables in the hpd.read_knmi docstring.

  • name (str, optional) – name of the obscollection. The default is ‘’

  • starts (None, str, datetime or list, optional) – start date of observations per meteo variable. The start date is included in the time series. If start is None the start date will be January 1st of the previous year. If start is str it will be converted to datetime. If start is a list it should be the same length as meteo_vars and the start time for each variable. The default is None

  • ends (list of str, datetime or None) – end date of observations per meteo variable. The end date is included in the time series. If end is None the start date will be January 1st of the previous year. If end is a str it will be converted to datetime. If end is a list it should be the same length as meteo_vars and the end time for each meteo variable. The default is None

  • ObsClasses (list of type or None) – class of the observations, can be PrecipitationObs, EvaporationObs or MeteoObs. If None the type of observations is derived from the meteo_vars.

  • fill_missing_obs (bool, optional) – if True nan values in time series are filled with nearby time series. The default is False.

  • interval (str, optional) – desired time interval for observations. Options are ‘daily’ and ‘hourly’. The default is ‘daily’.

  • use_api (bool, optional) –

    if True the api is used to obtain the data, API documentation is here:

    https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

    if False a text file is downloaded into a temporary folder and the data is read from there. Default is True since the api is back online (July 2021).

  • raise_exceptions (bool, optional) – if True you get errors when no data is returned. The default is False.

  • progress_callback (callable or None, optional) – callback function called with (i, total) for each station processed. The default is None.

  • fill_missing_obs_with_factor (bool, optional) – if True, donor-station values are scaled with an overlap-based factor before filling missing values. This automatically enables fill_missing_obs. The default is False.

  • **kwargs – kwargs are passed to the hydropandas.io.knmi.get_knmi_obslist function

classmethod from_knmi_scenarios(stn: int | str, years: Iterable[Literal['2033', '2050', '2100', '2150']] = ('2033', '2050', '2100', '2150'), scenarios: Iterable[Literal['Ld', 'Ln', 'Md', 'Mn', 'Hd', 'Hn']] = ('Ld', 'Ln', 'Md', 'Mn', 'Hd', 'Hn'), evap: Literal['EV24', 'makkink', 'penman', 'hargreaves'] = 'EV24', meteo_vars: Optional[Iterable[Literal['TG', 'RD', 'Q', 'TX', 'TN', 'UG', 'FG', 'EV24']]] = None, name: str = '')[source]

Create ObsCollection from KNMI climate scenario data.

The stn argument may be provided as an integer or a string. The data are downloaded once and converted into individual observations. By default every variable present in the returned dataset is turned into an Obs; a user can restrict the output by specifying meteo_vars.

Parameters

  • stn (int or str) – Station number (e.g., 550 or “550”).

  • years (tuple, optional) – Years of climate scenario. The default is (‘2033’,’2050’,’2100’,’2150’).

  • scenarios (tuple, optional) – Names of climate scenario. The default is (‘Ld’,’Ln’,’Md’,’Mn’,’Hd’,’Hn’). This includes all scenarios including the original measurements.

  • evap (str, optional) – Method for calculating evaporation. Options are ‘EV24’, ‘makkink’, ‘penman’, or ‘hargreaves’. The default is ‘EV24’.

  • meteo_vars (iterable of str or None, optional) – Meteorological variables to include in the ObsCollection. Possible variables include ‘TG’, ‘RD’, ‘Q’, ‘TX’, ‘TN’, ‘UG’, ‘FG’, and ‘EV24’. If None (default), all available variables are included.

  • name (str, optional) – Name of the observation collection. The default is “”.

Returns

Collection with climate scenario observations.

Return type

ObsCollection

classmethod from_list(obs_list, name='')[source]

Read observations from a list of obs objects.

Parameters

  • obs_list (list of observation.Obs) – list of observations

  • name (str, optional) – name of the observation collection

classmethod from_lizard(extent=None, codes=None, name='', tube_nr='all', tmin=None, tmax=None, type_timeseries=None, which_timeseries=('hand', 'diver'), datafilters=None, combine_method='merge', only_metadata=False, organisation='vitens', auth=None)[source]

Get all observations within a specified extent.

Parameters

  • extent (list, shapefile path or None) – get groundwater monitoring wells wihtin this extent [xmin, xmax, ymin, ymax] or within a predefined Polygon from a shapefile

  • codes (lst of str or None) – codes of the monitoring wells

  • tube_nr (lst of str) – list of tube numbers of the monitoring wells that should be selected. By default ‘all’ available tubes are selected.

  • tmin (str YYYY-m-d, optional) – start of the observations, by default the entire serie is returned

  • tmax (Ttr YYYY-m-d, optional) – end of the observations, by default the entire serie is returned

  • type_timeseries (str, optional (deprecated)) – Old keyword, use which_timeseries instead.

  • which_timeseries (tuple of str, optional) – Which timeseries to retrieve. Options: “hand”, “diver”, “diver_validated”. Defaults to (“hand”, “diver”) (which should be correct for Vitens).

  • datafilters (list of strings, optional) – Methods to filter the timeseries data. If None (default), all measurements will be shown. Currently implemented datafilter methods: “remove_unvalidated_diver_values_when_validated_available”: Removes diver values before last date with validated diver. “remove_hand_during_diver_period”: Removes hand measurements during periods where diver or diver_validated measurements are available.

  • combine_method (str, optional) – “merge” (vertical stack with ‘origin’ column) or “columns” (side-by-side columns). If None, defaults to “merge”.

  • only_metadata (bool, optional) – if True only metadata is returned and no time series data. The default is False.

  • organisation (str, optional) – organisation of the data. The default is “vitens”.

  • auth (tuple, optional) – authentication credentials for the API request, e.g.: (“__key__”, your_api_key)

Returns

ObsCollection DataFrame with the ‘obs’ column

Return type

ObsCollection

classmethod from_matroos(extent=None, name='', ObsClass=<class 'hydropandas.observation.WaterlvlObs'>, locations=None, units=None, sources=None, tmin=None, tmax=None, only_metadata=False, keep_all_obs=False, **kwargs)[source]

Read measurement using the Matroos API within an extent.

Parameters

  • extent (list, tuple, numpy-array or None, optional) – get measurements within this extent [xmin, xmax, ymin, ymax]

  • name (str, optional) – name of the collection, by default “”

  • ObsClass (type) – class of the observations, e.g. WaterlvlObs

  • locations (list, tuple or None, optional) – locations to select, if None all locations are selected, by default None

  • units (list, tuple or None, optional) – units to select, if None all units are selected, by default None

  • sources (list, tuple or None, optional) – sources to select, if None all sources are selected, by default None

  • tmin (pd.Timestamp, str or None, optional) – start time of observations. The default is None.

  • tmax (pd.Timestamp, str or None, optional) – end time of observations. The default is None.

  • only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.

  • keep_all_obs (bool, optional) – if False, only observations with measurements are kept. The default is True.

  • **kwargs – additional keyword arguments are passed to the ObsClass.from_matroos() method

Returns

ObsCollection containing data

Return type

ObsCollection

classmethod from_menyanthes(path, name='', ObsClass=<class 'hydropandas.observation.Obs'>, load_oseries=True, load_stresses=True)[source]
classmethod from_modflow(obs_collection, ml, hds_arr, mtime, modelname='', nlay=None, exclude_layers=None, method='linear')[source]

Read modflow groundwater heads at points in obs_collection.

Parameters

  • obs_collection (ObsCollection) – locations of model observation

  • ml (flopy.modflow.mf.model) – modflow model

  • hds_arr (numpy array) – heads with shape (ntimesteps, nlayers, nrow, ncol)

  • mtime (list of datetimes) – dates for each model timestep

  • modelname (str, optional) – modelname

  • nlay (int, optional) – number of layers if None the number of layers from ml is used.

  • exclude_layers (list of int, optional) – exclude the observations in these model layers

  • method (str, optional) – interpolation method, either ‘linear’ or ‘nearest’, default is linear

classmethod from_pastastore(pstore, libname, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, metadata_mapping=None)[source]

Read pastastore library.

Parameters

  • pstore (pastastore.PastaStore) – PastaStore object

  • libname (str) – name of library (e.g. oseries or stresses)

  • ObsClass (Obs, optional) – type of Obs to read data as, by default GroundwaterObs

  • metadata_mapping (dict, optional) – dictionary containing map between metadata field names in pastastore and metadata field names expected by hydropandas, by default None.

Returns

ObsCollection containing data

Return type

ObsCollection

classmethod from_waterconnect(extent=None, name='', ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, tmin=None, tmax=None, only_metadata=False, keep_all_obs=False, location_gdf=None, update=False, **kwargs)[source]

Read waterconnect measurement within an extent or from a file or directory.

Parameters

  • extent (list, tuple, numpy-array or None, optional) – get water connect measurements within this extent [xmin, xmax, ymin, ymax]

  • name (str, optional) – name of the collection, by default “”

  • ObsClass (type) – class of the observations, e.g. GroundwaterObs

  • tmin (str or None, optional) – start time of observations. The default is None.

  • tmax (str or None, optional) – end time of observations. The default is None.

  • only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.

  • keep_all_obs (bool, optional) – if False, only observations with measurements are kept. The default is True.

  • location_gdf (GeoDataFrame, optional) – geodataframe with the locations of the water drill holes you want to include.

  • update (bool, optional) – if True new locations are downloaded and stored locally (slow) otherwise a cached version of the locations is used. By default False

  • **kwargs – additional keyword arguments are passed to the ObsClass.from_waterconnect() method

Returns

ObsCollection containing data

Return type

ObsCollection

classmethod from_waterinfo(file_or_dir=None, extent=None, name='', ObsClass=<class 'hydropandas.observation.WaterlvlObs'>, locatie=None, grootheid_code=None, groepering_code=None, parameter_code=None, proces_type=None, tmin=None, tmax=None, only_metadata=False, keep_all_obs=False, epsg=28992, progressbar=True, location_gdf=None, **kwargs)[source]

Read waterinfo measurement within an extent or from a file or directory.

Parameters

  • file_or_dir (str) – path to file or directory. Files can be .csv or .zip

  • extent (list, tuple, numpy-array or None, optional) – get waterinfo measurements within this extent [xmin, xmax, ymin, ymax]

  • name (str, optional) – name of the collection, by default “”

  • ObsClass (Obs, optional) – type of Obs to read data as, by default WaterlvlObs

  • locatie (str or list of str, optional) – select only measurement with this location(s), e.g. ‘schoonhoven’, default is None

  • grootheid_code (str or list of str, optional) – select only measurement with this grootheid_code, e.g. ‘WATHTE’, default is None

  • groepering_code (str or list of str, optional) – select only measurement with this groepering_code, e.g. ‘GETETBRKD2’, default is None

  • parameter_code (str or list of str, optional) – select only measurement with this parameter_code, e.g. ‘Cl’, default is None

  • proces_type (str or list of str, optional) – select only measurement with this proces_type, e.g. ‘meting’, default is None

  • tmin (pd.Timestamp, str or None, optional) – start time of observations. The default is None.

  • tmax (pd.Timestamp, str or None, optional) – end time of observations. The default is None.

  • only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.

  • keep_all_obs (bool, optional) – if False, only observations with measurements are kept. The default is True.

  • epsg (int, optional) – epsg code of the extent. The default is 28992 (RD).

  • progressbar (bool, optional) – show progressbar, by default True

  • location_gdf (GeoDataFrame, optional) – geodataframe with the locations of the measurements you want to include. If location_gdf is provided the provided extent and epgs will be ignored.

Returns

ObsCollection containing data

Return type

ObsCollection

classmethod from_wiski(dirname, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, suffix='.csv', unpackdir=None, force_unpack=False, preserve_datetime=False, keep_all_obs=True, **kwargs)[source]
geo

alias of GeoAccessor

get_obs(name=None, **kwargs)[source]

get an observation object from a collection

Parameters

  • name (str or None, optional) – name of the observation you want to select, by default None

  • **kwargs (any metadata, value pair e.g. for a collection of GroundwaterObs:) – tube_nr = 1 or source = ‘BRO’

Returns

Observation object from the collection.

Return type

hpd.Obs

Raises

  • ValueError – If multiple observations in the collection match the given attribute values.

  • ValueError – If no observation in the collection match the given attribute values.

get_series(tmin=None, tmax=None, col=None)[source]
Parameters

  • tmin (datetime, optional) – start time for series. The default is None.

  • tmax (datetime, optional) – end time for series. The default is None.

  • col (str or None, optional) – the column of the obs dataframe to get measurements from. The first numeric column is used if col is None, by default None.

Returns

series of a series of observations within a time frame.

Return type

series of Series

gwobs

alias of GwObsAccessor

interpolate(xy: list[list[float]], kernel: str = 'thin_plate_spline', kernel2: str = 'linear', epsilon: int | None = None, col: str | None = None)[source]

Interpolation method for ObsCollections using the Scipy radial basis function (RBF)

Parameters

  • xy (List[List[float]]) – xy coordinates of locations of interest e.g. [[10,25], [5,25]]

  • kernel (str, optional) – Type of radial basis funtion, by default thin_plate_spline. Other options are linear, gaussian, inverse_quadratic, multiquadric, inverse_multiquadric, cubic or quintic.

  • kernel2 (str, optional) – Kernel in case there are not enough observations (3 or 6) for time step, by default linear. Other options are gaussian, inverse_quadratic, multiquadric, or inverse_multiquadric.

  • epsilon (float, optional) – Shape parameter that scales the input to the RBF. If kernel is linear, thin_plate_spline, cubic, or quintic, this defaults to 1. Otherwise this must be specified.

  • col (str, optional) – Name of the column in the Obs dataframe to be used. If None the first numeric column in the Obs Dataframe is used.

Return type

ObsCollection

plots

alias of CollectionPlots

set_metadata_value(oname, att_name, value, add_to_meta=False)[source]
Set a value on two levels at once:

  1. the value in an ObsCollection DataFrame

2. the attribute of the observation Optionally set the value in the meta dictionary of an observation

Parameters

  • oname (str, int, float, ...) – observation name. Must be same type as self.index. e.g. B52D0111_3

  • att_name (str, int, float, ...) – name of the column in self.columns and attribute of the observation. e.g. ‘x’

  • value (str, int, float, ...) – value of the the att_name. e.g. 116234

  • add_to_meta (bool, optional) – if True the att_name, value pair is added to the meta dictionary of an observation. The default is False.

Raises

ValueError – if the oname is not in self.index the value cannot be set.

Return type

None.

stats

alias of StatsAccessor

to_csv(path, check_consistency=True, **kwargs)[source]

Write all observations in the ObsCollection to csv files.

Parameters

  • path (str) – directory to which the csv files will be written.

  • check_consistency (bool, optional) – If True the consistency of the collection is checked. If set to False the csv file(s) may be unreadable by hydropandas. The default is True.

  • **kwargs (keyword arguments) – kwargs are passed to the to_csv method of each observation.

Notes

if you write a csv file using the ‘to_csv’ method and read a csv with the ‘read_csv’ method you lose this information: - The ‘name’ and ‘meta’ attributes of the ObsCollection - metadata of each Observation stored in the ‘meta’ attribute - integer dtypes may become floats

If you don’t want to lose this data consider using the to_json and read_json function.

If you want to write the metadata to a single csv file consider using: ‘pd.DataFrame(oc).to_csv()’.

to_dict()[source]

Convert ObsCollection to dictionary.

Returns

dictionary with metadata and observations

Return type

dict

to_excel(path, meta_sheet_name='metadata', check_consistency=True)[source]

Write an ObsCollection to an excel, the first sheet in the excel contains the metadata, the other tabs are the timeseries of each observation.

The excel can be read using the read_excel function of hydropandas.

Parameters

  • path (str) – full path of xlsx file.

  • meta_sheet_name (str, optional) – sheetname with metadata. The default is “metadata”.

  • check_consistency (bool, optional) – If True the consistency of the collection is checked. If set to False the excel file may be unreadable by hydropandas. The default is True.

Raises

RuntimeError – If the ObsCollection is inconsistent.

Return type

None.

Notes

The following data is NOT written to the excel file: - The ‘name’ and ‘meta’ attributes of the ObsCollection - metadata of each Observation stored in the ‘meta’ dictionary

If you don’t want this consider using the to_json method.

to_gdf(xcol='x', ycol='y', crs=28992, drop_obs=True)[source]

Convert ObsCollection to GeoDataFrame.

Parameters

  • xcol (str) – column name with x values

  • ycol (str) – column name with y values

  • crs (int, optional) – coordinate reference system, by default 28992 (RD new).

  • drop_obs (bool, optional) – drop the column with observations. Useful for basic geodataframe manipulations that require JSON serializable columns. The default is True.

Returns

gdf

Return type

geopandas.GeoDataFrame

to_json(path=None, cls=<class 'hydropandas.serialization.HydropandasEncoder'>, **kwargs)[source]

Write ObsCollection to a JSON file.

Parameters

  • path_or_buf (str, path object, file-like object, or None, default None) – String, path object (implementing os.PathLike[str]), or file-like object implementing a write() function. If None, the result is returned as a string.

  • **kwargs – Additional keyword arguments passed to json.dump or json.dumps.

Return type

None

to_pastastore(pstore=None, pstore_name='', col=None, kind='oseries', add_metadata=True, conn=None, overwrite=False)[source]

Add observations to a new or existing pastastore.

Parameters

  • pstore (pastastore.PastaStore, optional) – Existing pastastore, if None a new pastastore is created

  • pstore_name (str, optional) – Name of the pastastore only used if pstore is None

  • col (str, optional) – Name of the column in the Obs dataframe to be used. If None the first numeric column in the Obs Dataframe is used.

  • kind (str, optional) – The kind of series that is added to the pastastore. Use ‘oseries’ for observations and anything else for stresses.

  • add_metadata (boolean, optional) – If True metadata from the observations added to the pastastore

  • conn (pastastore.connectors or None, optional) – type of connector, if None the DictConnector is used. Default is None.

  • overwrite (boolean, optional) – if True, overwrite existing series in pastastore, default is False

Returns

pstore – the pastastore with the series from the ObsCollection

Return type

pastastore.PastaStore

to_pi_xml(fname, timezone='', version='1.24')[source]
to_shapefile(path, xcol='x', ycol='y')[source]

Save ObsCollection as shapefile.

Parameters

  • path (str) – filename of shapefile (.shp) or geopackage (.gpkg). A geopackage has the advantage that column names will not be truncated.

  • xcol (str) – column name with x values

  • ycol (str) – column name with y values

hydropandas.obs_collection.read_bro(extent=None, bro_id=None, name='', tmin=None, tmax=None, only_metadata=False, keep_all_obs=True, epsg=28992, ignore_max_obs=False, engine='hydropandas')[source]

Get all the observations within an extent or within a groundwatermonitoring net.

Parameters

  • extent (list, tuple, numpy-array or None, optional) – get groundwater monitoring wells within this extent [xmin, xmax, ymin, ymax]

  • bro_id (str or None, optional) – starts with ‘GMN’.

  • name (str, optional) – name of the observation collection

  • tmin (str or None, optional) – start time of observations. The default is None.

  • tmax (str or None, optional) – end time of observations. The default is None.

  • only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.

  • keep_all_obs (boolean, optional) – add all observation points to the collection, even without measurements

  • epsg (int, optional) – epsg code of the extent. The default is 28992 (RD).

  • ignore_max_obs (bool, optional) – by default you get a prompt if you want to download over a 1000 observations at once. if ignore_max_obs is True you won’t get the prompt. The default is False

  • engine (str, optional) – Select how data from the bro-database is obtained, options are ‘hydropandas’ or ‘brodata’ The default is ‘hydropandas’.

Returns

ObsCollection DataFrame with the ‘obs’ column

Return type

ObsCollection

hydropandas.obs_collection.read_bronhouderportaal_bro(dirname, full_meta=False, add_to_df=False)[source]

get all the metadata from files in a directory. Files are GMW files of well construction, and are subbmitted to https://www.bronhouderportaal-bro.nl .

Parameters

  • dirname (str) – name of directory that holds XML files

  • full_meta (bool, optional) – process not only the standard metadata to ObsCollection

  • add_to_df (bool, optional) – add all the metadata to the ObsCollection DataFrame

Returns

ObsCollection DataFrame without the ‘obs’ column

Return type

ObsCollection

hydropandas.obs_collection.read_csv(path, parse_dates=True, index_col=0, **kwargs)[source]

Create an observation collection from one or more csv files. The csv file(s) should have the same format as csv files created with the to_csv method of an ObsCollection.

Parameters

  • path (str) – directory with csv files, a .zip file with csv files or a single csv file.

  • parse_dates (bool, optional) – whether to parse the dates when reading the csv files. The default is True.

  • index_col (int, optional) – column to use as index, by default 0

  • kwargs – kwargs are passed to the pandas.read_csv function

Return type

ObsCollection

Notes

if you write a csv file using the ‘to_csv’ method and read a csv with the ‘read_csv’ method you lose this information: - The ‘name’ and ‘meta’ attributes of the ObsCollection - metadata of each Observation stored in the ‘meta’ attribute - integer dtypes may become floats

If you don’t want to lose this data consider using the to_json and read_json function.

hydropandas.obs_collection.read_dino(dirname=None, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, subdir='DINO_Grondwaterstanden', suffix=None, keep_all_obs=True, name=None, **kwargs)[source]

Read dino observations from a directory with downloaded files.

Parameters

  • dirname (str, optional) – directory name, can be a .zip file or the parent directory of subdir

  • ObsClass (type) – class of the observations, so far only obs.GroundwaterObs is supported

  • subdir (str) – subdirectory of dirname with data files. For old school dino zip files this should be “Grondwaterstanden_Put”. For new style the default value DINO_Grondwaterstanden is sufficient. The default is DINO_Grondwaterstanden.

  • suffix (str or None, optional) – suffix of files in subdir that will be read. For old school dino zip files this should be ‘1.csv’. For new style the default value None is sufficient. The default is None

  • keep_all_obs (boolean, optional) – add all observation points to the collection, even the points without measurements or metadata

  • name (str, optional) – the name of the observation collection

  • kwargs – kwargs are passed to the hydropandas.io.dino.read_dino_dir() function

Returns

collection of multiple point observations

Return type

ObsCollection

hydropandas.obs_collection.read_excel(path, meta_sheet_name='metadata')[source]

Create an observation collection from an excel file. The excel file should have the same format as excel files created with the to_excel method of an ObsCollection.

Parameters

  • path (str) – full file path (including extension) of the excel file.

  • meta_sheet_name (str, optional) – sheetname with metadata. The default is “metadata”.

Return type

ObsCollection

Notes

if you write an excel file using the ‘to_excel’ method and read an excel with the ‘read_excel’ method you lose this information: - The ‘name’ and ‘meta’ attributes of the ObsCollection - metadata of each Observation stored in the ‘meta’ attribute

If you don’t want to lose this data consider using the to_json and read_json function.

hydropandas.obs_collection.read_fews(file_or_dir=None, xmlstring=None, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, name='fews', translate_dic=None, filterdict=None, locations=None, remove_nan=True, low_memory=True, unpackdir=None, force_unpack=False, preserve_datetime=False, **kwargs)[source]

Read one or several FEWS PI-XML files.

Parameters

  • file_or_dir (str) – zip, xml or directory with zips or xml files to read

  • xmlstring (str or None) – string with xml data, only used if file_or_dir is None. Default is None

  • ObsClass (type) – class of the observations, e.g. GroundwaterObs or WaterlvlObs

  • name (str, optional) – name of the observation collection, ‘fews’ by default

  • translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘locatie’}.

  • filterdict (dict, optional) – dictionary with tag name to apply filter to as keys, and list of accepted names as dictionary values to keep in final result, i.e. {“locationId”: [“B001”, “B002”]}

  • locations (list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read. Only supported by low_memory=True method!

  • low_memory (bool, optional) – whether to use xml-parsing method with lower memory footprint, default is True

  • remove_nan (boolean, optional) – remove nan values from measurements, flag information about the nan values is also lost, only used if low_memory=False

  • unpackdir (str) – destination directory to unzip file if file_or_dir is a .zip

  • force_unpack (boolean, optional) – force unpack if dst already exists

  • preserve_datetime (boolean, optional) – whether to preserve datetime from zip archive

Returns

collection of multiple point observations

Return type

ObsCollection

hydropandas.obs_collection.read_imod(obs_collection, ml, runfile, mtime, model_ws, modelname='', nlay=None, exclude_layers=0)[source]

Read imod model results at point locations.

Parameters

  • obs_collection (ObsCollection) – collection of observations at which points imod results will be read

  • ml (flopy.modflow.mf.model) – modflow model

  • runfile (Runfile) – imod runfile object

  • mtime (list of datetimes) – datetimes corresponding to the model periods

  • model_ws (str) – model workspace with imod model

  • nlay (int, optional) – number of layers if None the number of layers from ml is used.

  • modelname (str) – modelname

  • exclude_layers (int) – exclude modellayers from being read from imod

Returns

collection of multiple point observations

Return type

ObsCollection

hydropandas.obs_collection.read_json(path, **kwargs)[source]

Read an observation collection or an observation from a json file.

Parameters

  • path (str) – full file path (including extension) of the json file.

  • kwargs – kwargs are passed to the ObsCollection.from_json function

Return type

ObsCollection

hydropandas.obs_collection.read_knmi(locations=None, stns=None, xy=None, meteo_vars=('RH',), name='', starts=None, ends=None, ObsClasses=None, fill_missing_obs=False, interval='daily', use_api=True, raise_exceptions=True, progress_callback=None, fill_missing_obs_with_factor=False)[source]

Get knmi observations from a list of locations or a list of stations.

Parameters

  • locations (pandas DataFrame or None) – dataframe with columns ‘x’ and ‘y’ as coordinates. The default is None

  • stns (list of str or None) – list of knmi stations. The default is None

  • xy (list or numpy array, optional) – xy coordinates of the locations. e.g. [[10,25], [5,25]]

  • meteo_vars (list or tuple of str) – meteo variables e.g. [“RH”, “EV24”]. The default is (“RH”). See list of all possible variables below

  • name (str, optional) – name of the obscollection. The default is ‘’

  • starts (None, str, datetime or list, optional) – start date of observations per meteo variable. The start date is included in the time series. If start is None the start date will be January 1st of the previous year. If start is str it will be converted to datetime. If start is a list it should be the same length as meteo_vars and the start time for each variable. The default is None

  • ends (list of str, datetime or None) – end date of observations per meteo variable. The end date is included in the time series. If end is None the start date will be January 1st of the previous year. If end is a str it will be converted to datetime. If end is a list it should be the same length as meteo_vars and the end time for each meteo variable. The default is None

  • ObsClasses (list of type or None) – class of the observations, can be PrecipitationObs, EvaporationObs or MeteoObs. If None the type of observations is derived from the meteo_vars.

  • fill_missing_obs (bool, optional) – if True nan values in time series are filled with nearby time series. The default is False.

  • progress_callback (callable or None, optional) – callback function called with (i, total) for each station processed. The default is None.

  • fill_missing_obs_with_factor (bool, optional) – if True, donor-station values are scaled with an overlap-based factor before filling missing values. This automatically enables fill_missing_obs. The default is False.

  • **kwargs – kwargs are passed to the hydropandas.io.knmi.get_knmi_obslist function

  • variables (List of possible) – neerslagstations: RD = de 24-uurs neerslagsom, gemeten van 0800 utc op de voorafgaande dag tot 0800 utc op de vermelde datum meteostations: DDVEC = Vectorgemiddelde windrichting in graden (360=noord, 90=oost, 180=zuid, 270=west, 0=windstil/variabel). Zie http://www.knmi.nl/kennis-en-datacentrum/achtergrond/klimatologische-brochures-en-boeken / Vector mean wind direction in degrees (360=north, 90=east, 180=south, 270=west, 0=calm/variable) FHVEC = Vectorgemiddelde windsnelheid (in 0.1 m/s). Zie http://www.knmi.nl/kennis-en-datacentrum/achtergrond/klimatologische-brochures-en-boeken / Vector mean windspeed (in 0.1 m/s) FG = Etmaalgemiddelde windsnelheid (in 0.1 m/s) / Daily mean windspeed (in 0.1 m/s) FHX = Hoogste uurgemiddelde windsnelheid (in 0.1 m/s) / Maximum hourly mean windspeed (in 0.1 m/s) FHXH = Uurvak waarin FHX is gemeten / Hourly division in which FHX was measured FHN = Laagste uurgemiddelde windsnelheid (in 0.1 m/s) / Minimum hourly mean windspeed (in 0.1 m/s) FHNH = Uurvak waarin FHN is gemeten / Hourly division in which FHN was measured FXX = Hoogste windstoot (in 0.1 m/s) / Maximum wind gust (in 0.1 m/s) FXXH = Uurvak waarin FXX is gemeten / Hourly division in which FXX was measured TG = Etmaalgemiddelde temperatuur (in 0.1 graden Celsius) / Daily mean temperature in (0.1 degrees Celsius) TN = Minimum temperatuur (in 0.1 graden Celsius) / Minimum temperature (in 0.1 degrees Celsius) TNH = Uurvak waarin TN is gemeten / Hourly division in which TN was measured TX = Maximum temperatuur (in 0.1 graden Celsius) / Maximum temperature (in 0.1 degrees Celsius) TXH = Uurvak waarin TX is gemeten / Hourly division in which TX was measured T10N = Minimum temperatuur op 10 cm hoogte (in 0.1 graden Celsius) / Minimum temperature at 10 cm above surface (in 0.1 degrees Celsius) T10NH = 6-uurs tijdvak waarin T10N is gemeten / 6-hourly division in which T10N was measured; 6=0-6 UT, 12=6-12 UT, 18=12-18 UT, 24=18-24 UT SQ = Zonneschijnduur (in 0.1 uur) berekend uit de globale straling (-1 voor <0.05 uur) / Sunshine duration (in 0.1 hour) calculated from global radiation (-1 for <0.05 hour) SP = Percentage van de langst mogelijke zonneschijnduur / Percentage of maximum potential sunshine duration Q = Globale straling (in J/cm2) / Global radiation (in J/cm2) DR = Duur van de neerslag (in 0.1 uur) / Precipitation duration (in 0.1 hour) RH = Etmaalsom van de neerslag (in 0.1 mm) (-1 voor <0.05 mm) / Daily precipitation amount (in 0.1 mm) (-1 for <0.05 mm) RHX = Hoogste uursom van de neerslag (in 0.1 mm) (-1 voor <0.05 mm) / Maximum hourly precipitation amount (in 0.1 mm) (-1 for <0.05 mm) RHXH = Uurvak waarin RHX is gemeten / Hourly division in which RHX was measured PG = Etmaalgemiddelde luchtdruk herleid tot zeeniveau (in 0.1 hPa) berekend uit 24 uurwaarden / Daily mean sea level pressure (in 0.1 hPa) calculated from 24 hourly values PX = Hoogste uurwaarde van de luchtdruk herleid tot zeeniveau (in 0.1 hPa) / Maximum hourly sea level pressure (in 0.1 hPa) PXH = Uurvak waarin PX is gemeten / Hourly division in which PX was measured PN = Laagste uurwaarde van de luchtdruk herleid tot zeeniveau (in 0.1 hPa) / Minimum hourly sea level pressure (in 0.1 hPa) PNH = Uurvak waarin PN is gemeten / Hourly division in which PN was measured P = Luchtdruk (in 0.1 hPa) herleid tot zeeniveau, op het moment van meten / Air pressure (in 0.1 hPa) reduced to mean sea level, at the time of observation VVN = Minimum opgetreden zicht / Minimum visibility; 0: <100 m, 1:100-200 m, 2:200-300 m,…, 49:4900-5000 m, 50:5-6 km, 56:6-7 km, 57:7-8 km,…, 79:29-30 km, 80:30-35 km, 81:35-40 km, …, 89: >70 km) VVNH = Uurvak waarin VVN is gemeten / Hourly division in which VVN was measured VVX = Maximum opgetreden zicht / Maximum visibility; 0: <100 m, 1:100-200 m, 2:200-300 m,…, 49:4900-5000 m, 50:5-6 km, 56:6-7 km, 57:7-8 km,…, 79:29-30 km, 80:30-35 km, 81:35-40 km, …, 89: >70 km) VVXH = Uurvak waarin VVX is gemeten / Hourly division in which VVX was measured NG = Etmaalgemiddelde bewolking (bedekkingsgraad van de bovenlucht in achtsten, 9=bovenlucht onzichtbaar) / Mean daily cloud cover (in octants, 9=sky invisible) UG = Etmaalgemiddelde relatieve vochtigheid (in procenten) / Daily mean relative atmospheric humidity (in percents) UX = Maximale relatieve vochtigheid (in procenten) / Maximum relative atmospheric humidity (in percents) UXH = Uurvak waarin UX is gemeten / Hourly division in which UX was measured UN = Minimale relatieve vochtigheid (in procenten) / Minimum relative atmospheric humidity (in percents) UNH = Uurvak waarin UN is gemeten / Hourly division in which UN was measured EV24 = Referentiegewasverdamping (Makkink) (in 0.1 mm) / Potential evapotranspiration (Makkink) (in 0.1 mm)

Returns

collection of multiple point observations

Return type

ObsCollection

hydropandas.obs_collection.read_knmi_scenarios(stn: int | str, years: Iterable[Literal['2033', '2050', '2100', '2150']] = ('2033', '2050', '2100', '2150'), scenarios: Iterable[Literal['Ld', 'Ln', 'Md', 'Mn', 'Hd', 'Hn']] = ('Ld', 'Ln', 'Md', 'Mn', 'Hd', 'Hn'), evap: Literal['EV24', 'makkink', 'penman', 'hargreaves'] = 'EV24', meteo_vars: Optional[Iterable[Literal['TG', 'RD', 'Q', 'TX', 'TN', 'UG', 'FG', 'EV24']]] = None, name: str = '')[source]

Get KNMI climate scenario observations for a station.

Retrieves climate scenario data from KNMI and returns an ObsCollection with temperature, precipitation, and evaporation observations for different climate scenarios.

Parameters

  • stn (int or str) – Station number (e.g., 550 or “550”).

  • years (tuple, optional) – Years of climate scenario. The default is (‘2033’,’2050’,’2100’,’2150’).

  • scenarios (tuple, optional) – Names of climate scenario. The default is (‘Ld’,’Ln’,’Md’,’Mn’,’Hd’,’Hn’). This includes all scenarios including the original measurements.

  • evap (str, optional) – Method for calculating evaporation. Options are ‘EV24’, ‘makkink’, ‘penman’, or ‘hargreaves’. The default is ‘EV24’.

  • meteo_vars (iterable of str or None, optional) – Meteorological variables to include in the ObsCollection. Possible variables include ‘TG’, ‘RD’, ‘Q’, ‘TX’, ‘TN’, ‘UG’, ‘FG’, and ‘EV24’. If None (default), all available variables are included.

  • name (str, optional) – Name of the observation collection. The default is “”.

Returns

Collection of climate scenario observations with temperature, precipitation, and evaporation data for different scenarios.

Return type

ObsCollection

Examples

>>> oc = hpd.read_knmi_scenarios("550")
>>> oc = hpd.read_knmi_scenarios(
...     "550",
...     years=["2050", "2100"],
...     scenarios=["Md", "Hd"],
...     evap="Makkink"
... )
hydropandas.obs_collection.read_lizard(extent=None, codes=None, name='', tube_nr='all', tmin=None, tmax=None, type_timeseries=None, which_timeseries=('hand', 'diver'), datafilters=None, combine_method='merge', only_metadata=False, organisation='vitens', auth=None)[source]

Get all observations from a list of codes of the monitoring wells and a list of tube numbers.

Parameters

  • extent (list, shapefile path or None) – get groundwater monitoring wells within this extent [xmin, xmax, ymin, ymax] or within a predefined Polygon from a shapefile

  • codes (lst of str or None) – codes of the monitoring wells

  • tube_nr (lst of str) – list of tube numbers of the monitoring wells that should be selected. By default ‘all’ available tubes are selected.

  • tmin (str YYYY-m-d, optional) – start of the observations, by default the entire time series is returned

  • tmax (str YYYY-m-d, optional) – end of the observations, by default the entire time series is returned

  • type_timeseries (str, optional (deprecated)) – Old keyword, use which_timeseries instead.

  • which_timeseries (tuple of str, optional) – Which timeseries to retrieve. Options: “hand”, “diver”, “diver_validated”. Defaults to (“hand”, “diver”) (which should be correct for Vitens).

  • datafilters (list of strings, optional) – Methods to filter the timeseries data. If None (default), all measurements will be shown. Currently implemented datafilter methods: “remove_unvalidated_diver_values_when_validated_available”: Removes diver values before last date with validated diver. “remove_hand_during_diver_period”: Removes hand measurements during periods where diver or diver_validated measurements are available.

  • combine_method (str, optional) – “merge” (vertical stack with ‘origin’ column) or “columns” (side-by-side columns). If None, defaults to “merge”.

  • only_metadata (bool, optional) – if True only metadata is returned and no time series data. The default is False.

  • organisation (str, optional) – organisation of the data, by default “vitens”.

  • auth (tuple, optional) – authentication credentials for the API request, e.g.: (“__key__”, your_api_key)

Returns

ObsCollection DataFrame with the ‘obs’ column

Return type

ObsCollection

hydropandas.obs_collection.read_matroos(extent=None, name='', ObsClass=<class 'hydropandas.observation.WaterlvlObs'>, locations=None, units=None, sources=None, tmin=None, tmax=None, only_metadata=False, keep_all_obs=False, **kwargs)[source]

Read measurement using the Matroos API within an extent.

Parameters

  • extent (list, tuple, numpy-array or None, optional) – get measurements within this extent [xmin, xmax, ymin, ymax]

  • name (str, optional) – name of the collection, by default “”

  • ObsClass (type) – class of the observations, e.g. WaterlvlObs

  • locations (list, tuple or None, optional) – locations to select, if None all locations are selected, by default None

  • units (list, tuple or None, optional) – units to select, if None all units are selected, by default None

  • sources (list, tuple or None, optional) – sources to select, if None all sources are selected, by default None

  • tmin (pd.Timestamp, str or None, optional) – start time of observations. The default is None.

  • tmax (pd.Timestamp, str or None, optional) – end time of observations. The default is None.

  • only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.

  • keep_all_obs (bool, optional) – if False, only observations with measurements are kept. The default is True.

  • **kwargs – additional keyword arguments are passed to the ObsClass.from_matroos() method

Returns

ObsCollection containing data

Return type

ObsCollection

hydropandas.obs_collection.read_menyanthes(path, name='', ObsClass=<class 'hydropandas.observation.Obs'>, load_oseries=True, load_stresses=True)[source]

Read a Menyanthes file.

Parameters

  • path (str) – full path of the .men file.

  • name (str, optional) – name of the observation collection. The default is “”.

  • ObsClass (type, optional) – class of the observations, e.g. GroundwaterObs. The default is Obs.

  • load_oseries (bool, optional) – if True the observations are read. The default is True.

  • load_stresses (bool, optional) – if True the stresses are read. The default is True.

Returns

collection of multiple point observations

Return type

ObsCollection

hydropandas.obs_collection.read_modflow(obs_collection, ml, hds_arr, mtime, modelname='', nlay=None, exclude_layers=None, method='linear')[source]

Read modflow groundwater heads at locations in obs_collection.

Parameters

  • obs_collection (ObsCollection) – locations of model observation

  • ml (flopy.modflow.mf.model) – modflow model

  • hds_arr (numpy array) – heads with shape (ntimesteps, nlayers, nrow, ncol)

  • mtime (list of datetimes) – dates for each model timestep

  • modelname (str, optional) – modelname

  • nlay (int, optional) – number of layers if None the number of layers from ml is used.

  • exclude_layers (list of int, optional) – exclude the observations in these model layers

  • method (str, optional) – interpolation method, either ‘linear’ or ‘nearest’, default is linear

Returns

collection of multiple point observations

Return type

ObsCollection

hydropandas.obs_collection.read_pastastore(pstore, libname, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, metadata_mapping=None)[source]

Read pastastore library.

Parameters

  • pstore (pastastore.PastaStore) – PastaStore object

  • libname (str) – name of library (e.g. oseries or stresses)

  • ObsClass (Obs, optional) – type of Obs to read data as, by default GroundwaterObs

  • metadata_mapping (dict, optional) – dictionary containing map between metadata field names in pastastore and metadata field names expected by hydropandas, by default None.

Returns

ObsCollection containing data

Return type

ObsCollection

hydropandas.obs_collection.read_pickle(filepath_or_buffer, compression='infer', storage_options=None)[source]

Wrapper around pd.read_pickle.

Parameters

  • filepath_or_buffer (str, path object, or file-like object) –

    String, path object (implementing os.PathLike[str]), or file-like object implementing a binary readlines() function.

    Changed in version 1.0.0.

    Accept URL. URL is not limited to S3 and GCS.

  • compression (str or dict, default 'infer') –

    For on-the-fly decompression of on-disk data. If ‘infer’ and ‘filepath_or_buffer’ is path-like, then detect compression from the following extensions: ‘.gz’,’.bz2’, ‘.zip’, ‘.xz’, or ‘.zst’ (otherwise no compression). If using ‘zip’, the ZIP file must contain only one data file to be read in. Set to None for no decompression. Can also be a dict with key 'method' set to one of {'zip', 'gzip', 'bz2', 'zstd'} and other key-value pairs are forwarded to zipfile.ZipFile, gzip.GzipFile, bz2.BZ2File, or zstandard.ZstdDecompressor, respectively. As an example, the following could be passed for Zstandard decompression using a custom compression dictionary: compression={'method': 'zstd', 'dict_data': my_compression_dict}.

    Changed in version 1.4.0: Zstandard support.

  • storage_options (dict, optional) –

    Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.

    New in version 1.2.0.

Returns

ObsCollection

Return type

same type as object stored in file

hydropandas.obs_collection.read_waterconnect(extent=None, name='', ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, tmin=None, tmax=None, only_metadata=False, keep_all_obs=False, location_gdf=None, update=False, **kwargs)[source]

Read waterconnect measurement within an extent

Parameters

  • extent (list, tuple, numpy-array or None, optional) – get water connect measurements within this extent [xmin, xmax, ymin, ymax], coordinates are in lat (y) lon (x).

  • name (str, optional) – name of the collection, by default “”

  • ObsClass (type) – class of the observations, e.g. GroundwaterObs

  • tmin (str or None, optional) – start time of observations. The default is None.

  • tmax (str or None, optional) – end time of observations. The default is None.

  • only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.

  • keep_all_obs (bool, optional) – if False, only observations with measurements are kept. The default is True.

  • location_gdf (GeoDataFrame, optional) – geodataframe with the locations of the water drill holes you want to include.

  • update (bool, optional) – if True new locations are downloaded and stored locally (slow) otherwise a cached version of the locations is used. By default False

  • **kwargs – additional keyword arguments are passed to the ObsClass.from_waterconnect() method

Returns

ObsCollection containing data

Return type

ObsCollection

hydropandas.obs_collection.read_waterinfo(file_or_dir=None, extent=None, name='', ObsClass=<class 'hydropandas.observation.WaterlvlObs'>, locatie=None, grootheid_code=None, groepering_code=None, parameter_code=None, proces_type=None, tmin=None, tmax=None, only_metadata=False, keep_all_obs=False, epsg=28992, progressbar=True, location_gdf=None, **kwargs)[source]

Read waterinfo measurement within an extent or from a file or directory

Parameters

  • file_or_dir (str or None, optional) – path to file or directory. Files can be .csv or .zip

  • extent (list, tuple, numpy-array or None, optional) – get waterinfo measurements within this extent [xmin, xmax, ymin, ymax]

  • name (str, optional) – name of the collection, by default “”

  • ObsClass (Obs, optional) – type of Obs to read data as, by default WaterlvlObs

  • locatie (str or list of str, optional) – select only measurement with this location(s), e.g. ‘schoonhoven’, default is None

  • grootheid_code (str or list of str, optional) – select only measurement with this grootheid_code, e.g. ‘WATHTE’, default is None

  • groepering_code (str or list of str, optional) – select only measurement with this groepering_code, e.g. ‘GETETBRKD2’, default is None

  • parameter_code (str or list of str, optional) – select only measurement with this parameter_code, e.g. ‘Cl’, default is None

  • proces_type (str or list of str, optional) – select only measurement with this proces_type, e.g. ‘meting’, default is None

  • tmin (pd.Timestamp, str or None, optional) – start time of observations. The default is None.

  • tmax (pd.Timestamp, str or None, optional) – end time of observations. The default is None.

  • only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.

  • keep_all_obs (bool, optional) – if False, only observations with measurements are kept. The default is True.

  • epsg (int, optional) – epsg code of the extent. The default is 28992 (RD).

  • progressbar (bool, optional) – show progressbar, by default True

  • location_gdf (GeoDataFrame, optional) – geodataframe with the locations of the measurements you want to include. If location_gdf is provided the provided extent and epgs will be ignored.

Returns

ObsCollection containing data

Return type

ObsCollection

hydropandas.obs_collection.read_wiski(dirname, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, suffix='.csv', unpackdir=None, force_unpack=False, preserve_datetime=False, keep_all_obs=True, **kwargs)[source]
Parameters

  • dirname (str) – path of the zipfile with wiski data.

  • ObsClass (type, optional) – type of Obs. The default is GroundwaterObs.

  • suffix (str, optional) – extension of filenames to read. The default is “.csv”.

  • unpackdir (str or None, optional) – directory to unpack zipped directory. The default is None.

  • force_unpack (bool, optional) – force unzip, by default False.

  • preserve_datetime (bool, optional) – preserve datetime of unzipped files, by default False (useful for checking whether data has changed)

  • keep_all_obs (bool, optional) – If True keep all observations even those without metadata. The default is True.

  • **kwargs –

Returns

ObsCollection containing observation data

Return type

ObsCollection

hydropandas.observation module

Module with observation classes.

The Obs class is a subclass of a pandas DataFrame with additional attributes and methods. The specific classes (GroundwaterObs, WaterlvlObs, …) are subclasses of the Obs class.

The subclasses of a dataframe can have additional attributes and methods. Additional attributes have to be defined in the ‘_metadata’ attribute. In order to keep the subclass methods and attributes when selecting or slicing an object you need the ‘_constructor’ method.

More information about subclassing pandas DataFrames can be found here: http://pandas.pydata.org/pandas-docs/stable/development/extending.html#extending-subclassing-pandas

class hydropandas.observation.EvaporationObs(*args, **kwargs)[source]

Bases: MeteoObs

Class for evaporation timeseries.

Subclass of the MeteoObs class

classmethod from_knmi(meteo_var='EV24', stn=None, fname=None, xy=None, start=None, end=None, fill_missing_obs=False, interval='daily', use_api=True, raise_exceptions=True, startdate=None, enddate=None, fill_missing_obs_with_factor=False)[source]

Get an EvaporationObs timeseries from the KNMI evaporation in m.

Parameters

  • meteo_var (str, optional) – meteo variable should be “EV24”.

  • stn (int, str or None, optional) – measurement station e.g. 829. The default is None.

  • fname (str, path object, file-like object or None, optional) – filename of a knmi file. The default is None.

  • xy (list, tuple or None, optional) – RD coördinates of a location in the Netherlands. The station nearest to this location used. The Default is None.

  • start (str, datetime or None, optional) – start date of observations. The default is None.

  • end (str, datetime or None, optional) – end date of observations. The default is None.

  • fill_missing_obs (bool, optional) – if True nan values in time series are filled with nearby time series. The default is False.

  • interval (str, optional) – desired time interval for observations. Options are ‘daily’ and ‘hourly’. The default is ‘daily’.

  • inseason (boolean, optional) – flag to obtain inseason data. The default is False

  • raise_exceptions (bool, optional) – if True you get errors when no data is returned. The default is False.

  • use_api (bool, optional) –

    if True the api is used to obtain the data, API documentation is here:

    https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

    if False a text file is downloaded into a temporary directory and the data is read from there. Default is True since the api is back online (July 2021).

  • fill_missing_obs_with_factor (bool, optional) – if True, donor-station values are scaled with an overlap-based factor before filling missing values. This automatically enables fill_missing_obs. The default is False.

Return type

EvaporationObs object with an evaporation time series and attributes

class hydropandas.observation.GroundwaterObs(*args, **kwargs)[source]

Bases: Obs

Class for groundwater quantity observations.

Subclass of the Obs class. Has the following attributes: - tube_nr: 2 tubes at one piezometer should have a different ‘tube_nr’. - screen_top: top op the filter in m above date (NAP) - screen_bottom: bottom of the filter in m above date (NAP) - ground_level: surface level in m above date (NAP) (maaiveld in Dutch) - tube_top: top of the tube in m above date (NAP) - metadata_available: boolean indicating if metadata is available for

the measurement point.

Note

In hydropandas version 0.13.0 the ‘monitoring_well’ attribute was removed and replaced by the ‘location’ attribute

classmethod from_artdino_file(path=None, **kwargs)[source]

Read a dino csv file (artdiver style).

Parameters

  • path (str, optional) – path of dino csv filename

  • kwargs (key-word arguments) – these arguments are passed to hydropandas.io._dino.read_dino_groundwater_csv

classmethod from_bro(bro_id, tube_nr=None, tmin='1900-01-01', tmax='2040-01-01', to_wintertime=True, drop_duplicate_times=True, only_metadata=False, engine='hydropandas')[source]

Download BRO groundwater observations from the server.

Parameters

  • bro_id (str) – can be a GLD id or GMW id. If a GMW id is given a tube number is required as well. e.g. ‘GLD000000012893’.

  • tube_nr (str or None, optional) – if the bro_id is a GMW object the tube number should be given.

  • tmin (str or None, optional) – start date in format YYYY-MM-DD

  • tmax (str or None, optional) – end date in format YYYY-MM-DD

  • to_wintertime (bool, optional) – if True the time index is converted to Dutch winter time. The default is True.

  • drop_duplicate_times (bool, optional) – if True rows with a duplicate time stamp are removed keeping only the first row. The default is True.

  • only_metadata (bool, optional) – if True only metadata is returned and no time series data. The default is False

  • engine (str, optional) – Select how data from the bro-database is obtained, options are ‘hydropandas’ or ‘brodata’ The default is ‘hydropandas’.

Returns

DESCRIPTION.

Return type

TYPE

classmethod from_bronhouderportaal_bro(path, tube_nr, full_meta=False)[source]

Load BRO groundwater metadata from XML file. Mind that bro_id is applicable, because file is not yet imported in BRO.

Parameters

  • path (str) – filepath of XML file.

  • tube_nr (int) – tube number.

  • full_meta (bool) – process not only the standard metadata to ObsCollection.

Returns

ObsCollection containing observations from XML file.

Return type

ObsCollection

classmethod from_dino(path=None, **kwargs)[source]

Download dino data from the server.

Parameters

  • path (str, optional) – path of dino csv file

  • kwargs (key-word arguments) – these arguments are passed to hydropandas.io.dino.read_dino_groundwater_csv if path is not None and otherwise to hydropandas.io.dino.findMeetreeks

classmethod from_lizard(code, tube_nr=None, tmin=None, tmax=None, type_timeseries=None, which_timeseries=('hand', 'diver'), datafilters=None, combine_method='merge', only_metadata=False, organisation='vitens', auth=None)[source]

Extracts the metadata and timeseries of a observation well from a LIZARD-API based on the code of a monitoring well.

Parameters

  • code (str) – code of the measuring well

  • tube_nr (int, optional) – select specific tube top Default selects tube_nr = 1

  • tmin (str YYYY-m-d, optional) – start of the observations, by default the entire serie is returned

  • tmax (str YYYY-m-d, optional) – end of the observations, by default the entire serie is returned

  • type_timeseries (str, optional (deprecated)) – Old keyword, use which_timeseries instead.

  • which_timeseries (tuple of str, optional) – Which timeseries to retrieve. Options: “hand”, “diver”, “diver_validated”. Defaults to (“hand”, “diver”) (which should be correct for Vitens).

  • datafilters (list of strings, optional) – Methods to filter the timeseries data. If None (default), all measurements will be shown. Currently implemented datafilter methods: “remove_unvalidated_diver_values_when_validated_available”: Removes diver values before last date with validated diver. “remove_hand_during_diver_period”: Removes hand measurements during periods where diver or diver_validated measurements are available.

  • combine_method (str, optional) – “merge” (vertical stack with ‘origin’ column) or “columns” (side-by-side columns). If None, defaults to “merge”.

  • only_metadata (bool, optional) – if True only metadata is returned and no time series data. The default is False.

  • organisation (str, optional) – organisation of the data. Currently only ‘vitens’ is officially supported.

  • auth (tuple, optional) – authentication credentials for the API request, e.g.: (“__key__”, your_api_key)

Returns

Returns a DataFrame with metadata and timeseries

Return type

ObsCollection

classmethod from_pastastore(pstore, libname, name, metadata_mapping=None)[source]

Read item from pastastore library.

Parameters

  • pstore (pastastore.PastaStore) – pastastore object

  • libname (str) – name of library containinig item

  • name (str) – name of item

  • metadata_mapping (dict, optional) – dictionary containing map between metadata field names in pastastore (keys) and metadata field names expected by hydropandas (values), by default None.

classmethod from_solinst(path, transform_coords=True, screen_bottom=None, screen_top=None, ground_level=None, tube_nr=None, tube_top=None)[source]

Read data from Solinst xle file.

Parameters

path (str) – path to file (file can zip or xle)

classmethod from_waterconnect(dh_no, meta_series=None, tmin=None, tmax=None, only_metadata=False, verify=True, pumping=True, anomalous=True, **kwargs)[source]

Read data from water connect api.

Parameters

  • dh_no (int or str) – drill hole number

  • meta_series (pd.Series, optional) – series with metadata. Typically a row of the locations gdf.

  • tmin (str or None, optional) – start time of observations. The default is None.

  • tmax (str or None, optional) – end time of observations. The default is None.

  • only_metadata (bool, optional) – if True only metadata and no measurements are returned. BY default False

  • verify (bool, optional) – use verification to get a secure connection

  • pumping (bool, optional) – return observations from pumping wells

  • anomalous (bool, optional) – return anomalous observations

  • **kwargs – kwargs are passed to ‘get_waterconnect_obs’

Returns

GroundwaterObs object

Return type

GroundwaterObs

Raises

ValueError – if file contains data for more than one location

classmethod from_wiski(path, **kwargs)[source]

Read data from a WISKI file.

Parameters:

pathstr

The path of the file to be read.

sepstr, optional (default=”;”)

The delimiter used to separate fields in the file.

header_sepstr, optional (default=None)

The delimiter used to separate fields in the header. If None, the function will try to automatically detect the separator.

header_identifierstr, optional (default=”#”)

The character used to identify header lines.

read_seriesbool, optional (default=True)

Whether to read the time series data from the file.

translate_dicdict, optional (default=None)

A dictionary mapping header field names to the desired output names.

tz_localizebool, optional (default=True)

Whether to localize the datetime index to the machine’s timezone.

unitstr, optional (default=””)

The unit of measurement of the data.

**kwargskeyword arguments

Additional arguments to pass to the pandas read_csv function.

property monitoring_well
class hydropandas.observation.MeteoObs(*args, **kwargs)[source]

Bases: Obs

Class for meteorological timeseries.

Subclass of the Obs class

classmethod from_knmi(meteo_var, stn=None, fname=None, xy=None, start=None, end=None, fill_missing_obs=False, interval='daily', use_api=True, raise_exceptions=True, startdate=None, enddate=None, fill_missing_obs_with_factor=False)[source]

Get a MeteoObs timeseries from the KNMI meteo data.

Parameters

  • meteo_var (str) – meteo variable e.g. “RH” or “EV24”. For a list of possible variables see the hydropandas.read_knmi function.

  • stn (int, str or None, optional) – measurement station e.g. 829. The default is None.

  • fname (str, path object, file-like object or None, optional) – filename of a knmi file. The default is None.

  • xy (list, tuple or None, optional) – RD coördinates of a location in the Netherlands. The station nearest to this location used. The Default is None.

  • start (str, datetime or None, optional) – start date of observations. The default is None.

  • end (str, datetime or None, optional) – end date of observations. The default is None.

  • fill_missing_obs (bool, optional) – if True nan values in time series are filled with nearby time series. The default is False. Note: if the given stn has no data between start and end the data from nearby stations is used. In this case the metadata of the Observation is the metadata from the nearest station that has any measurement in the given period.

  • interval (str, optional) – desired time interval for observations. Options are ‘daily’ and ‘hourly’. The default is ‘daily’.

  • use_api (bool, optional) –

    if True the api is used to obtain the data, API documentation is here:

    https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

    if False a text file is downloaded into a temporary directory and the data is read from there. Default is True since the api is back online (July 2021).

  • raise_exceptions (bool, optional) – if True you get errors when no data is returned. The default is False.

  • fill_missing_obs_with_factor (bool, optional) – if True, donor-station values are scaled with an overlap-based factor before filling missing values. This automatically enables fill_missing_obs. The default is False.

Return type

MeteoObs object with meteorological observations

classmethod from_wow(meteo_var: str, stn: str | None = None, xy: list[float] | None = None, start: pandas._libs.tslibs.timestamps.Timestamp | None = None, end: pandas._libs.tslibs.timestamps.Timestamp | None = None)[source]

Get a MeteoObs timeseries from a wow.knmi.nl station.

Parameters

  • meteo_var (str) – wow meteo variable

  • stn (Optional[int, str], optional) – station name

  • xy (Optinal[List[float]], optinial) – longitude latitude of location [lon, lat] eg: [4.85, 51.95]

  • start (Optional[pd.Timestamp], optional) – start date of observations, by default None

  • end (Optional[pd.Timestamp], optional) – start date of observations, by default None

Return type

MeteoObs

class hydropandas.observation.ModelObs(*args, **kwargs)[source]

Bases: Obs

Class for model point results.

Subclass of the Obs class

class hydropandas.observation.Obs(*args, **kwargs)[source]

Bases: DataFrame

Generic class for a time series with measurements at a certain location.

Unless specified explicitly the first numeric column in the observation is used for analysis and plotting.

Parameters

  • name (str) – name

  • x (int or float) – x coordinate of observation point

  • y (int or float) – y coordinate of observation point

  • location (str) – label for the location of the observation. Observations on the same location but with a different ObsType or tube number should have the same label.

  • meta (dictionary) – metadata

  • filename (str) – filename with data of observation point

  • source (str) – source of the observation e.g. BRO or KNMI

  • unit (str) – unit of the first numerical column in the observation

copy(deep=True)[source]

Create a copy of the observation.

When deep=True (default), a new object will be created with a copy of the calling object’s data and indices. Modifications to the data or indices of the copy will not be reflected in the original object (see notes below).

When deep=False, a new object will be created without copying the calling object’s data or index (only references to the data and index are copied). Any changes to the data of the original will be reflected in the shallow copy (and vice versa).

Parameters

deep (bool, default True) – Make a deep copy, including a copy of the data and the indices. With deep=False neither the indices nor the data are copied.

Returns

o – copy of the observation.

Return type

hydropandas.Obs

classmethod from_csv(path, parse_dates=True, index_col=0, **kwargs)[source]

Read a csv file and return an Obs object.

Parameters

  • path (str) – path of the csv file

  • parse_dates (bool, optional) – whether to parse the dates when reading the csv file. The default is True.

  • index_col (int, optional) – column to use as index, by default 0

  • **kwargs – keyword arguments passed to pd.read_csv for reading the timeseries

Returns

Obs object with metadata and observations from the csv file.

Return type

Obs

classmethod from_dict(d, **kwargs)[source]

Create an Obs object from a dictionary.

The dictionary should contain the metadata and the observations.

Parameters

  • d (dict) – dictionary with metadata and observations.

  • **kwargs – keyword arguments passed to pd.DataFrame for creating the timeseries

Returns

Obs object with metadata and observations from the dictionary.

Return type

Obs

classmethod from_json(path, **kwargs)[source]

Read a JSON file and return an Obs object.

Parameters

  • path (str) – path of the JSON file

  • **kwargs – keyword arguments passed to pd.read_json for reading the timeseries

Returns

Obs object with metadata and observations from the JSON file.

Return type

Obs

geo

alias of GeoAccessorObs

gwobs

alias of GeoAccessorObs

merge_metadata(right, overlap='error')[source]

Merge the metadata of an Obs object with metadata from another Obs object.

Parameters

  • right (dict) – dictionary with the metadata of an Obs object.

  • overlap (str, optional) –

    How to deal with overlapping metadata with different values. Options are:

    error : Raise a ValueError use_left : Use the metadata from self use_right : Use the given metadata

    Default is ‘error’.

Raises

ValueError – if the metadata differs and overlap=’error’.

Returns

new_metadata – metadata after merge.

Return type

dict

merge_observation(right, overlap='error', merge_metadata=True)[source]

Merge with another observation of the same type.

Parameters

  • right (hpd.observation.Obs) – Observation object.

  • overlap (str, optional) –

    How to deal with overlapping timeseries or metadata with different values. Options are:

    error : Raise a ValueError use_left : use the part of the overlapping timeseries from self use_right : use the part of the overlapping timeseries from right

    Default is ‘error’.

  • merge_metadata (bool, optional) – If True the metadata of the two objects are merged. If there are any differences the overlap parameter is used to determine which metadata is used. If merge_metadata is False, the metadata of self is always used for the merged observation. The default is True.

Raises

  • TypeError – when the observation types are not the same.

  • ValueError – when the time series have different values on the same date or different values for the same metadata.

Return type

Observation object.

plots

alias of ObsPlots

stats

alias of StatsAccessorObs

to_collection_dict(include_meta=False)[source]

Get dictionary with registered attributes and their values of an Obs object.

This method can be used to create a dataframe from a collection of Obs objects.

Parameters

include_meta (boolean, optional) – include the meta dictionary in the collection dictionary, default is false

Returns

d – dictionary with Obs information

Return type

dictionary

to_csv(path, **kwargs)[source]

Write Obs object to a comma-separated values (csv) file.

Parameters

  • path (str or path object) – String, path object (implementing os.PathLike[str]), or file-like object implementing a write() function.

  • **kwargs – Additional keyword arguments passed to pandas.DataFrame.to_csv

Return type

None

to_dict()[source]

Convert the Obs object to a dictionary.

The dictionary contains the metadata and the observations.

Returns

d – dictionary with metadata and observations.

Return type

dict

to_json(path=None, cls=<class 'hydropandas.serialization.HydropandasEncoder'>, **kwargs)[source]

Write Obs object to a JSON file.

Parameters

  • path_or_buf (str, path object, file-like object, or None, default None) – String, path object (implementing os.PathLike[str]), or file-like object implementing a write() function. If None, the result is returned as a string.

  • **kwargs – Additional keyword arguments passed to json.dump or json.dumps.

Return type

None

class hydropandas.observation.PrecipitationObs(*args, **kwargs)[source]

Bases: MeteoObs

Class for precipitation timeseries.

Subclass of the MeteoObs class

classmethod from_knmi(meteo_var='RH', stn=None, fname=None, xy=None, start=None, end=None, fill_missing_obs=False, interval='daily', use_api=True, raise_exceptions=True, startdate=None, enddate=None, fill_missing_obs_with_factor=False)[source]

Get a PrecipitationObs timeseries from the KNMI precipitation. The precipitation is the Daily precipitation amount (in 0.1 mm) (-1 for.

<0.05 mm).

There are 3 different ways to obtain precipitation data from the knmi:

  1. Daily data from precipitation (neerslag) stations

  2. Daily data from meteo stations

  3. Hourly data from meteo stations

1. For daily data from a precipitation station (neerslagstation) meteo_var should be ‘RD’. 2. For daily data from a meteo station meteo_var should be ‘RH’ and interval should be ‘daily’ (default). 3. For hourly data from a meteo station meteo_var should be ‘RH’ and interval should be ‘hourly’.

More information about the differences between neerslag and meteo stations can be found in the hydropandas documentation -> 02_knmi_observations notebook.

Parameters

  • meteo_var (str, optional) – meteo variable can be “RH” or “RD”. “RD” if you want data from a precipitation station (neerslagstation). “RH” if you want data from a meteo station. The default is “RH”.

  • stn (int, str or None, optional) – measurement station e.g. 829. The default is None.

  • fname (str, path object, file-like object or None, optional) – filename of a knmi file. The default is None.

  • xy (list, tuple or None, optional) – RD coördinates of a location in the Netherlands. The station nearest to this location used. The Default is None.

  • start (str, datetime or None, optional) – start date of observations. The default is None.

  • end (str, datetime or None, optional) – end date of observations. The default is None.

  • fill_missing_obs (bool, optional) – if True nan values in time series are filled with nearby time series. The default is False.

  • interval (str, optional) – desired time interval for observations. Options are ‘daily’ and ‘hourly’. The default is ‘daily’.

  • use_api (bool, optional) –

    if True the api is used to obtain the data, API documentation is here:

    https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

    if False a text file is downloaded into a temporary folder and the data is read from there. Default is True since the api is back online (July 2021).

  • raise_exceptions (bool, optional) – if True you get errors when no data is returned. The default is False.

  • fill_missing_obs_with_factor (bool, optional) – if True, donor-station values are scaled with an overlap-based factor before filling missing values. This automatically enables fill_missing_obs. The default is False.

Return type

PrecipitationObs object with a precipitation time series and attributes

classmethod from_wow(stn: str | None = None, xy: list[float] | None = None, start: pandas._libs.tslibs.timestamps.Timestamp | None = None, end: pandas._libs.tslibs.timestamps.Timestamp | None = None)[source]

Get a PrecipitationObs timeseries from a wow.knmi.nl station.

Parameters

  • stn (Optional[int, str], optional) – station name

  • xy (Optinal[List[float]], optinial) – longitude latitude of location [lon, lat] eg: [4.85, 51.95]

  • start (Optional[pd.Timestamp], optional) – start date of observations, by default None

  • end (Optional[pd.Timestamp], optional) – start date of observations, by default None

Return type

PrecipitationObs

class hydropandas.observation.WaterQualityObs(*args, **kwargs)[source]

Bases: Obs

Class for water quality ((grond)watersamenstelling) point observations.

Subclass of the Obs class

classmethod from_dino(path, **kwargs)[source]

Read dino file with groundwater quality data.

Parameters

  • path (str) – path of dino txt filename

  • kwargs (key-word arguments) – these arguments are passed to hydropandas.io.dino.read_dino_groundwater_quality_txt

classmethod from_waterinfo(path=None, location_gdf=None, locatie=None, grootheid_code=None, groepering_code=None, parameter_code=None, proces_type=None, tmin=None, tmax=None, **kwargs)[source]

Read data from waterinfo csv, zip or using the API.

Parameters

  • path (str, optional) – path to file (file can zip or csv)

  • location_gdf (geopandas.GeoDataFrame, optional) – geodataframe with locations, only used if path is None, default is None

  • locatie (str or list of str, optional) – select only measurement with this location(s), e.g. ‘schoonhoven’, default is None

  • grootheid_code (str or list of str, optional) – select only measurement with this grootheid_code, e.g. ‘WATHTE’, default is None

  • groepering_code (str or list of str, optional) – select only measurement with this groepering_code, e.g. ‘GETETBRKD2’, default is None

  • parameter_code (str or list of str, optional) – select only measurement with this parameter_code, e.g. ‘Cl’, default is None

  • proces_type (str or list of str, optional) – select only measurement with this proces_type, e.g. ‘meting’, default is None

  • tmin (pd.Timestamp or str, optional) – start date of the measurements, only used if path is None, default is None

  • tmax (pd.Timestamp or str, optional) – end date of the measurements, only used if path is None, default is None

Returns

WaterlvlObs object

Return type

WaterlvlObs

Raises

ValueError – if file contains data for more than one location

property monitoring_well
class hydropandas.observation.WaterlvlObs(*args, **kwargs)[source]

Bases: Obs

Class for water level point observations.

Subclass of the Obs class

classmethod from_dino(path, **kwargs)[source]

Read a dino file with waterlvl data.

Parameters

  • path (str) – path of dino csv filename

  • kwargs (key-word arguments) – these arguments are passed to hydropandas.io.dino.read_dino_waterlvl_csv

classmethod from_matroos(location, source, unit, tmin=None, tmax=None, only_metadata=False, **kwargs)[source]

Read data using the Matroos API

Parameters

  • location (str) – location e.g. ‘krimpen a/d lek’

  • unit (str) – unit e.g. ‘waterlevel’

  • source (str) – source e.g. ‘observed’

  • tmin (pd.Timestamp, str or None, optional) – start of time series if None tmin is 10 days ago, by default None

  • tmax (pd.Timestamp, str or None, optional) – start of time series if None tmin is today, by default None

  • only_metadata (bool, optional) – if True download only metadata, slightly faster. The default is False.

Returns

df – WaterlvlObs object

Return type

WaterlvlObs

Raises

ValueError – if file contains data for more than one location

classmethod from_waterinfo(path=None, location_gdf=None, locatie=None, grootheid_code=None, groepering_code=None, parameter_code=None, proces_type=None, tmin=None, tmax=None, **kwargs)[source]

Read data from waterinfo csv-file, zip or using the API.

Parameters

  • path (str, optional) – path to file (file can zip or csv)

  • location_gdf (geopandas.GeoDataFrame, optional) – geodataframe with locations, only used if path is None, default is None

  • locatie (str or list of str, optional) – select only measurement with this location(s), e.g. ‘schoonhoven’, default is None

  • grootheid_code (str or list of str, optional) – select only measurement with this grootheid_code, e.g. ‘WATHTE’, default is None

  • groepering_code (str or list of str, optional) – select only measurement with this groepering_code, e.g. ‘GETETBRKD2’, default is None

  • parameter_code (str or list of str, optional) – select only measurement with this parameter_code, e.g. ‘Cl’, default is None

  • proces_type (str or list of str, optional) – select only measurement with this proces_type, e.g. ‘meting’, default is None

  • tmin (pd.Timestamp or str, optional) – start date of the measurements, only used if path is None, default is None

  • tmax (pd.Timestamp or str, optional) – end date of the measurements, only used if path is None, default is None

Returns

df – WaterlvlObs object

Return type

WaterlvlObs

Raises

ValueError – if file contains data for more than one location

property monitoring_well
hydropandas.observation.read_csv_obs(path, parse_dates=True, index_col=0, **kwargs)[source]

Read a csv file and return an Obs object.

Parameters

  • path (str) – path of the csv file

  • parse_dates (bool, optional) – whether to parse the dates when reading the csv file. The default is True.

  • index_col (int, optional) – column to use as index, by default 0

  • **kwargs – keyword arguments passed to pd.read_csv for reading the timeseries

Returns

Obs object with metadata and observations from the csv file.

Return type

Obs

hydropandas.util module

Created on Wed Sep 12 12:15:42 2018.

@author: Artesia

class hydropandas.util.ColoredFormatter(*args, colors: dict[str, str] | None = None, **kwargs)[source]

Bases: Formatter

Colored log formatter.

Taken from https://gist.github.com/joshbode/58fac7ababc700f51e2a9ecdebe563ad

format(record) → str[source]

Format the specified record as text.

hydropandas.util.df2gdf(df, xcol='x', ycol='y', crs=28992)[source]

Create a GeoDataFrame from a DataFrame with xy points.

Parameters

  • df (pd.DataFrame) – input dataframe

  • xcol (str, optional) – column name with x values. The default is ‘x’.

  • ycol (str, optional) – column name with y values. The default is ‘x’.

  • crs (int, optional) – coordinate reference system, by default 28992 (RD new).

Returns

geodataframe

Return type

geopandas GeoDataFrame

hydropandas.util.get_color_logger(level='INFO', logger_name=None)[source]

Get a logger with colored output.

Parameters

level (str, optional) – The logging level to set for the logger. Default is “INFO”.

Returns

logger – The configured logger object.

Return type

logging.Logger

hydropandas.util.get_files(file_or_dir, ext, unpackdir=None, force_unpack=False, preserve_datetime=False)[source]

Internal method to get list of files with specific extension from dirname.

Parameters

  • file_or_dir (str) – file or path to data.

  • ext (str) – extension of filenames to store in list.

  • unpackdir (str) – directory to story unpacked zip file, only used in case of a zipfile.

  • force_unpack (bool, optional) – force unzip, by default False.

  • preserve_datetime (bool, optional) – preserve datetime of unzipped files, by default False. Used for checking whether data has changed.

hydropandas.util.interpolate(xy: list[list[float]], obsdf: DataFrame, obsloc: DataFrame, kernel: str = 'thin_plate_spline', kernel2: str = 'linear', epsilon: int | None = None) → DataFrame[source]

Interpolation method using the Scipy radial basis function (RBF)

Parameters

  • xy (List[List[float]]) – xy coordinates of locations of interest e.g. [[10,25], [5,25]]

  • obsdf (DataFrame) – Dataframe containing the observation locations as columns and the observations at a measurement time in each row.

  • obsloc (DataFrame) – Dataframe containing the observation locations coordinates with observation locations as index and columns [“x”, “y”]

  • kernel (str, optional) – Type of radial basis funtion, by default thin_plate_spline. Other options are linear, gaussian, inverse_quadratic, multiquadric, inverse_multiquadric, cubic or quintic.

  • kernel2 (str, optional) – Kernel in case there are not enough observations (3 or 6) for time step, by default linear. Other options are gaussian, inverse_quadratic, multiquadric, or inverse_multiquadric.

  • epsilon (Optional[int], optional) – Shape parameter that scales the input to the RBF. If kernel is linear, thin_plate_spline, cubic, or quintic, this defaults to 1. Otherwise this must be specified.

Returns

DataFrame with locations of interest as columns and interpolated values at a measurement time in each row.

Return type

DataFrame

hydropandas.util.oc_to_df(oc, col: str | None = None) → DataFrame[source]

Convert an observation collection to a DataFrame where every column has one observation.

Parameters

  • oc (hydropandas ObsCollection) – observation collection

  • col (Optional[str], optional) – Name of a column in hte observation collection, by default None

Returns

_description_

Return type

DataFrame

hydropandas.util.unzip_file(src, dst, force=False, preserve_datetime=False)[source]

Unzip file.

Parameters

  • src (str) – source zip file

  • dst (str) – destination directory

  • force (boolean, optional) – force unpack if dst already exists

  • preserve_datetime (boolean, optional) – use date of the zipfile for the destination file

Returns

1 of True

Return type

int