hydropandas

Subpackages

hydropandas.obs_collection module

Module with ObsCollection class for a collection of observations.

The ObsCollection class is a subclass of a pandas DataFrame with additional attributes and methods.

More information about subclassing pandas DataFrames can be found here: http://pandas.pydata.org/pandas-docs/stable/development/extending.html#extending-subclassing-pandas

class hydropandas.obs_collection.ObsCollection(*args, **kwargs)[source]

Bases: DataFrame

Class for a collection of point observations.

An ObsCollection object is a subclass of a pandas.DataFrame and allows for additional attributes and methods. Additional attributes are defined in the ‘_metadata’ attribute.

Parameters

  • observations (*args) –

  • DataFrame (list of observations or a pandas) –

:param : :param **kwargs can be one of these:

namestr

name of the observation collection

metadic

metadata of the observation collection

add_meta_to_df(key='all')[source]

Get the values from the meta dictionary of each observation object and add these to the ObsCollection as a column.

to the ObsCollection

Parameters

key (str, int, tuple, list, set or None, optional) – key in meta dictionary of observation object. If key is ‘all’, all keys are added. The default is ‘all’.

add_obs_collection(obs_collection, check_consistency=True, inplace=False, **kwargs)[source]

Add one observation collection to another observation collection. See add_observation method for more details.

Parameters

  • obs_collection (hpd.ObsCollection) – ObsCollection object.

  • check_consistency (bool, optional) – If True the consistency of both collections is first checked. The default is True.

  • inplace (bool, optional) – If True, modifies the ObsCollection in place (do not create a new object). The default is False.

  • Obs.merge_observation (**kwargs passed to) –

    merge_metadatabool, optional

    If True and observations are merged the metadata of the two objects are merged. If there are any differences the overlap parameter is used to determine which metadata is used. If merge_metadata is False, the metadata of the original observation is always used for the merged observation. The default is True.

    overlapstr, optional

    How to deal with overlapping timeseries with different values. Options are: - error : Raise a ValueError - use_left : use the overlapping part from the existing observations - use_right : use the overlapping part from the new observation Default is ‘error’.

Raises

RuntimeError – when the observation collection is inconsistent.

Returns

merged ObsCollection if inplace=True.

Return type

ObsCollection or None

add_observation(o, check_consistency=True, **kwargs)[source]

Add an observation to an existing observation collection. If the observation exists the two observations are merged.

Parameters

  • o (hpd.observation.Obs) – Observation object.

  • check_consistency (bool, optional) – If True the consistency of the collection is first checked. The default is True.

  • Obs.merge_observation (**kwargs passed to) –

    merge_metadatabool, optional

    If True and observations are merged the metadata of the two objects are merged. If there are any differences the overlap parameter is used to determine which metadata is used. If merge_metadata is False, the metadata of the original observation is always used for the merged observation. The default is True.

    overlapstr, optional

    How to deal with overlapping timeseries with different values. Options are: - error : Raise a ValueError - use_left : use the overlapping part from the existing observations - use_right : use the overlapping part from the new observation Default is ‘error’.

Raises

  • RuntimeError – when the observation collection is inconsistent.

  • TypeError – when the observation type is wrong.

Return type

None.

copy(deep=False)[source]

Make a copy of this object’s indices and data.

Parameters

deep (bool, default True) – Make a deep copy, including a deep copy of the observation objects. With deep=False neither the indices nor the data are copied.

Return type

ObsCollection

classmethod from_artdino_dir(dirname=None, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, subdir='csv', suffix='.csv', unpackdir=None, force_unpack=False, preserve_datetime=False, keep_all_obs=True, name=None, **kwargs)[source]

Read a dino directory.

Parameters

  • extent (list, optional) – get dinodata online within this extent [xmin, xmax, ymin, ymax]

  • dirname (str, optional) – directory name, can be a .zip file or the parent directory of subdir

  • ObsClass (type) – class of the observations, e.g. GroundwaterObs or WaterlvlObs

  • subdir (str) – subdirectory of dirname with data files

  • suffix (str) – suffix of files in subdir that will be read

  • unpackdir (str) – destination directory of the unzipped file

  • force_unpack (boolean, optional) – force unpack if dst already exists

  • preserve_datetime (boolean, optional) – use date of the zipfile for the destination file

  • keep_all_obs (boolean, optional) – add all observation points to the collection, even without data or metadata

  • name (str, optional) – the name of the observation collection

  • kwargs – kwargs are passed to the hydropandas.io.dino.read_dino_dir() function

Returns

cls(obs_df) – collection of multiple point observations

Return type

ObsCollection

classmethod from_bro(extent=None, bro_id=None, name='', tmin=None, tmax=None, only_metadata=False, keep_all_obs=True, epsg=28992, ignore_max_obs=False)[source]

Get all the observations within an extent or within a groundwatermonitoring net.

Parameters

  • extent (list, tuple, numpy-array or None, optional) – get groundwater monitoring wells within this extent [xmin, xmax, ymin, ymax]

  • bro_id (str or None, optional) – starts with ‘GMN’.

  • name (str, optional) – name of the observation collection

  • tmin (str or None, optional) – start time of observations. The default is None.

  • tmax (str or None, optional) – end time of observations. The default is None.

  • only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.

  • keep_all_obs (boolean, optional) – add all observation points to the collection, even without measurements

  • epsg (int, optional) – epsg code of the extent. The default is 28992 (RD).

  • ignore_max_obs (bool, optional) – by default you get a prompt if you want to download over a 1000 observations at once. if ignore_max_obs is True you won’t get the prompt. The default is False

Returns

ObsCollection DataFrame with the ‘obs’ column

Return type

ObsCollection

classmethod from_bronhouderportaal_bro(dirname, full_meta=False)[source]

Get all the metadata from dirname.

Parameters

  • dirname (str) – name of dirname that holds XML files

  • full_meta (bool , optional) – process all metadata. The default is False.

Returns

ObsCollection DataFrame without the ‘obs’ column

Return type

ObsCollection

classmethod from_dataframe(df, obs_list=None, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>)[source]

Create an observation collection from a DataFrame by adding a column with empty observations.

Parameters

  • df (pandas DataFrame) – input dataframe. If this dataframe has a column named ‘obs’ the column is replaced with empty observation objects.

  • obs_list (list of observation.Obs, optional) – list of observations. Default is None

  • ObsClass (class, optional) – observation class used to create empty obs object, by default obs.GroundwaterObs

Returns

ObsCollection DataFrame with the ‘obs’ column

Return type

ObsCollection

classmethod from_dino(dirname=None, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, subdir='Grondwaterstanden_Put', suffix='1.csv', keep_all_obs=True, name=None, **kwargs)[source]

Read dino data within an extent from the server or from a directory with downloaded files.

Parameters

  • dirname (str, optional) – directory name, can be a .zip file or the parent directory of subdir

  • ObsClass (type) – class of the observations, so far only GroundwaterObs is supported

  • subdir (str) – subdirectory of dirname with data files

  • suffix (str) – suffix of files in subdir that will be read

  • keep_all_obs (boolean, optional) – add all observation points to the collection, even the points without measurements or metadata

  • name (str, optional) – the name of the observation collection

  • kwargs – kwargs are passed to the hydropandas.io.dino.read_dino_dir() function

Returns

cls(obs_df) – collection of multiple point observations

Return type

ObsCollection

classmethod from_excel(path, meta_sheet_name='metadata')[source]

Create an observation collection from an excel file. The excel file should have the same format as excel files created with the to_excel method of an ObsCollection.

Parameters

  • path (str) – full file path (including extension) of the excel file.

  • meta_sheet_name (str, optional) – sheetname with metadata. The default is “metadata”.

Return type

ObsCollection

Notes

if you write an excel file using the ‘to_excel’ method and read an excel with the ‘read_excel’ method you lose this information: - The ‘name’ and ‘meta’ attributes of the ObsCollection - metadata of each Observation stored in the ‘meta’ attribute

If you don’t want to lose this data consider using the to_pickle and read_pickle function.

classmethod from_fews_xml(file_or_dir=None, xmlstring=None, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, name='fews', translate_dic=None, filterdict=None, locations=None, remove_nan=True, low_memory=True, unpackdir=None, force_unpack=False, preserve_datetime=False, **kwargs)[source]

Read one or several FEWS PI-XML files.

Parameters

  • file_or_dir (str) – zip, xml or directory with zips or xml files to read

  • xmlstring (str or None) – string with xml data, only used if file_or_dir is None. Default is None

  • ObsClass (type) – class of the observations, e.g. GroundwaterObs or WaterlvlObs

  • name (str, optional) – name of the observation collection, ‘fews’ by default

  • translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘locatie’}.

  • filterdict (dict, optional) – dictionary with tag name to apply filter to as keys, and list of accepted names as dictionary values to keep in final result, i.e. {“locationId”: [“B001”, “B002”]}

  • locations (list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read. Only supported by low_memory=True method!

  • low_memory (bool, optional) – whether to use xml-parsing method with lower memory footprint, default is True

  • remove_nan (boolean, optional) – remove nan values from measurements, flag information about the nan values is also lost, only used if low_memory=False

  • unpackdir (str) – destination directory to unzip file if path is a .zip

  • force_unpack (boolean, optional) – force unpack if dst already exists

  • preserve_datetime (boolean, optional) – whether to preserve datetime from zip archive

Returns

cls(obs_df) – collection of multiple point observations

Return type

ObsCollection

classmethod from_imod(obs_collection, ml, runfile, mtime, model_ws, modelname='', nlay=None, exclude_layers=0)[source]

Read imod model results at point locations.

Parameters

  • obs_collection (ObsCollection) – collection of observations at which points imod results will be read

  • ml (flopy.modflow.mf.model) – modflow model

  • runfile (Runfile) – imod runfile object

  • mtime (list of datetimes) – datetimes corresponding to the model periods

  • model_ws (str) – model workspace with imod model

  • nlay (int, optional) – number of layers if None the number of layers from ml is used.

  • modelname (str) – modelname

  • exclude_layers (int) – exclude modellayers from being read from imod

classmethod from_knmi(locations=None, stns=None, xy=None, meteo_vars=('RH',), name='', starts=None, ends=None, ObsClasses=None, fill_missing_obs=False, interval='daily', use_api=True, raise_exceptions=True)[source]

Get knmi observations from a list of locations or a list of stations.

Parameters

  • locations (pandas DataFrame or None) – dataframe with columns ‘x’ and ‘y’ as coordinates. The default is None

  • stns (list of str or None) – list of knmi stations. The default is None

  • xy (list or numpy array, optional) – xy coordinates of the locations. e.g. [[10,25], [5,25]]

  • meteo_vars (list or tuple of str) – meteo variables e.g. [“RH”, “EV24”]. The default is (“RH”). See list of all possible variables in the hpd.read_knmi docstring.

  • name (str, optional) – name of the obscollection. The default is ‘’

  • starts (None, str, datetime or list, optional) – start date of observations per meteo variable. The start date is included in the time series. If start is None the start date will be January 1st of the previous year. If start is str it will be converted to datetime. If start is a list it should be the same length as meteo_vars and the start time for each variable. The default is None

  • ends (list of str, datetime or None) – end date of observations per meteo variable. The end date is included in the time series. If end is None the start date will be January 1st of the previous year. If end is a str it will be converted to datetime. If end is a list it should be the same length as meteo_vars and the end time for each meteo variable. The default is None

  • ObsClasses (list of type or None) – class of the observations, can be PrecipitationObs, EvaporationObs or MeteoObs. If None the type of observations is derived from the meteo_vars.

  • fill_missing_obs (bool, optional) – if True nan values in time series are filled with nearby time series. The default is False.

  • interval (str, optional) – desired time interval for observations. Options are ‘daily’ and ‘hourly’. The default is ‘daily’.

  • use_api (bool, optional) –

    if True the api is used to obtain the data, API documentation is here:

    https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

    if False a text file is downloaded into a temporary folder and the data is read from there. Default is True since the api is back online (July 2021).

  • raise_exceptions (bool, optional) – if True you get errors when no data is returned. The default is False.

  • **kwargs – kwargs are passed to the hydropandas.io.knmi.get_knmi_obslist function

classmethod from_list(obs_list, name='')[source]

Read observations from a list of obs objects.

Parameters

  • obs_list (list of observation.Obs) – list of observations

  • name (str, optional) – name of the observation collection

classmethod from_lizard(extent=None, codes=None, name='', tube_nr='all', tmin=None, tmax=None, type_timeseries='merge', only_metadata=False)[source]

Get all observations within a specified extent.

Parameters

  • extent (list, shapefile path or None) – get groundwater monitoring wells wihtin this extent [xmin, ymin, xmax, ymax] or within a predefined Polygon from a shapefile

  • codes (lst of str or None) – codes of the monitoring wells

  • tube_nr (lst of str) – list of tube numbers of the monitoring wells that should be selected. By default ‘all’ available tubes are selected.

  • tmin (str YYYY-m-d, optional) – start of the observations, by default the entire serie is returned

  • tmax (Ttr YYYY-m-d, optional) – end of the observations, by default the entire serie is returned

  • type_timeseries (str, optional) – hand: returns only hand measurements diver: returns only diver measurements merge: the hand and diver measurements into one time series (default) combine: keeps hand and diver measurements separeted The default is merge.

  • only_metadata (bool, optional) – if True only metadata is returned and no time series data. The default is False.

Returns

ObsCollection DataFrame with the ‘obs’ column

Return type

ObsCollection

classmethod from_menyanthes(path, name='', ObsClass=<class 'hydropandas.observation.Obs'>, load_oseries=True, load_stresses=True)[source]
classmethod from_modflow(obs_collection, ml, hds_arr, mtime, modelname='', nlay=None, exclude_layers=None, method='linear')[source]

Read modflow groundwater heads at points in obs_collection.

Parameters

  • obs_collection (ObsCollection) – locations of model observation

  • ml (flopy.modflow.mf.model) – modflow model

  • hds_arr (numpy array) – heads with shape (ntimesteps, nlayers, nrow, ncol)

  • mtime (list of datetimes) – dates for each model timestep

  • modelname (str, optional) – modelname

  • nlay (int, optional) – number of layers if None the number of layers from ml is used.

  • exclude_layers (list of int, optional) – exclude the observations in these model layers

  • method (str, optional) – interpolation method, either ‘linear’ or ‘nearest’, default is linear

classmethod from_pastastore(pstore, libname, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, metadata_mapping=None)[source]

Read pastastore library.

Parameters

  • pstore (pastastore.PastaStore) – PastaStore object

  • libname (str) – name of library (e.g. oseries or stresses)

  • ObsClass (Obs, optional) – type of Obs to read data as, by default obs.GroundwaterObs

  • metadata_mapping (dict, optional) – dictionary containing map between metadata field names in pastastore and metadata field names expected by hydropandas, by default None.

Returns

ObsCollection containing data

Return type

ObsCollection

classmethod from_waterinfo(file_or_dir, name='', ObsClass=<class 'hydropandas.observation.WaterlvlObs'>, progressbar=True, **kwargs)[source]

Read waterinfo file or directory.

Parameters

  • file_or_dir (str) – path to file or directory. Files can be .csv or .zip

  • name (str, optional) – name of the collection, by default “”

  • ObsClass (Obs, optional) – type of Obs to read data as, by default obs.WaterlvlObs

  • progressbar (bool, optional) – show progressbar, by default True

Returns

ObsCollection containing data

Return type

ObsCollection

classmethod from_wiski(dirname, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, suffix='.csv', unpackdir=None, force_unpack=False, preserve_datetime=False, keep_all_obs=True, **kwargs)[source]
geo

alias of GeoAccessor

get_obs(name=None, **kwargs)[source]

get an observation object from a collection

Parameters

  • name (str or None, optional) – name of the observation you want to select, by default None

  • **kwargs (any metadata, value pair e.g. for a collection of GroundwaterObs:) – tube_nr = 1 or source = ‘BRO’

Returns

Observation object from the collection.

Return type

hpd.Obs

Raises

  • ValueError – If multiple observations in the collection match the given attribute values.

  • ValueError – If no observation in the collection match the given attribute values.

get_series(tmin=None, tmax=None, col=None)[source]
Parameters

  • tmin (datetime, optional) – start time for series. The default is None.

  • tmax (datetime, optional) – end time for series. The default is None.

  • col (str or None, optional) – the column of the obs dataframe to get measurements from. The first numeric column is used if col is None, by default None.

Returns

series of a series of observations within a time frame.

Return type

series of Series

gwobs

alias of GwObsAccessor

interpolate(xy: List[List[float]], kernel: str = 'thin_plate_spline', kernel2: str = 'linear', epsilon: Optional[int] = None, col: Optional[str] = None)[source]

Interpolation method for ObsCollections using the Scipy radial basis function (RBF)

Parameters

  • xy (List[List[float]]) – xy coordinates of locations of interest e.g. [[10,25], [5,25]]

  • kernel (str, optional) – Type of radial basis funtion, by default thin_plate_spline. Other options are linear, gaussian, inverse_quadratic, multiquadric, inverse_multiquadric, cubic or quintic.

  • kernel2 (str, optional) – Kernel in case there are not enough observations (3 or 6) for time step, by default linear. Other options are gaussian, inverse_quadratic, multiquadric, or inverse_multiquadric.

  • epsilon (float, optional) – Shape parameter that scales the input to the RBF. If kernel is linear, thin_plate_spline, cubic, or quintic, this defaults to 1. Otherwise this must be specified.

  • col (str, optional) – Name of the column in the Obs dataframe to be used. If None the first numeric column in the Obs Dataframe is used.

Return type

ObsCollection

plots

alias of CollectionPlots

stats

alias of StatsAccessor

to_excel(path, meta_sheet_name='metadata')[source]

Write an ObsCollection to an excel, the first sheet in the excel contains the metadata, the other tabs are the timeseries of each observation.

The excel can be read using the read_excel function of hydropandas.

Parameters

  • path (str) – full path of xlsx file.

  • meta_sheet_name (str, optional) – sheetname with metadata. The default is “metadata”.

Raises

RuntimeError – If the ObsCollection is inconsistent.

Return type

None.

Notes

The following data is NOT written to the excel file: - The ‘name’ and ‘meta’ attributes of the ObsCollection - metadata of each Observation stored in the ‘meta’ dictionary

If you don’t want this consider using the to_pickle method.

to_gdf(xcol='x', ycol='y', crs=28992, drop_obs=True)[source]

Convert ObsCollection to GeoDataFrame.

Parameters

  • xcol (str) – column name with x values

  • ycol (str) – column name with y values

  • crs (int, optional) – coordinate reference system, by default 28992 (RD new).

  • drop_obs (bool, optional) – drop the column with observations. Useful for basic geodataframe manipulations that require JSON serializable columns. The default is True.

Returns

gdf

Return type

geopandas.GeoDataFrame

to_pastastore(pstore=None, pstore_name='', col=None, kind='oseries', add_metadata=True, conn=None, overwrite=False)[source]

Add observations to a new or existing pastastore.

Parameters

  • pstore (pastastore.PastaStore, optional) – Existing pastastore, if None a new pastastore is created

  • pstore_name (str, optional) – Name of the pastastore only used if pstore is None

  • col (str, optional) – Name of the column in the Obs dataframe to be used. If None the first numeric column in the Obs Dataframe is used.

  • kind (str, optional) – The kind of series that is added to the pastastore. Use ‘oseries’ for observations and anything else for stresses.

  • add_metadata (boolean, optional) – If True metadata from the observations added to the pastastore

  • conn (pastastore.connectors or None, optional) – type of connector, if None the DictConnector is used. Default is None.

  • overwrite (boolean, optional) – if True, overwrite existing series in pastastore, default is False

Returns

pstore – the pastastore with the series from the ObsCollection

Return type

pastastore.PastaStore

to_pi_xml(fname, timezone='', version='1.24')[source]
to_shapefile(path, xcol='x', ycol='y')[source]

Save ObsCollection as shapefile.

Parameters

  • path (str) – filename of shapefile (.shp) or geopackage (.gpkg). A geopackage has the advantage that column names will not be truncated.

  • xcol (str) – column name with x values

  • ycol (str) – column name with y values

hydropandas.obs_collection.read_bro(extent=None, bro_id=None, name='', tmin=None, tmax=None, only_metadata=False, keep_all_obs=True, epsg=28992, ignore_max_obs=False)[source]

Get all the observations within an extent or within a groundwatermonitoring net.

Parameters

  • extent (list, tuple, numpy-array or None, optional) – get groundwater monitoring wells within this extent [xmin, xmax, ymin, ymax]

  • bro_id (str or None, optional) – starts with ‘GMN’.

  • name (str, optional) – name of the observation collection

  • tmin (str or None, optional) – start time of observations. The default is None.

  • tmax (str or None, optional) – end time of observations. The default is None.

  • only_metadata (bool, optional) – if True download only metadata, significantly faster. The default is False.

  • keep_all_obs (boolean, optional) – add all observation points to the collection, even without measurements

  • epsg (int, optional) – epsg code of the extent. The default is 28992 (RD).

  • ignore_max_obs (bool, optional) – by default you get a prompt if you want to download over a 1000 observations at once. if ignore_max_obs is True you won’t get the prompt. The default is False

Returns

ObsCollection DataFrame with the ‘obs’ column

Return type

ObsCollection

hydropandas.obs_collection.read_bronhouderportaal_bro(dirname, full_meta=False, add_to_df=False)[source]

get all the metadata from files in a directory. Files are GMW files of well construction, and are subbmitted to https://www.bronhouderportaal-bro.nl .

Parameters

  • dirname (str) – name of directory that holds XML files

  • full_meta (bool, optional) – process not only the standard metadata to ObsCollection

  • add_to_df (bool, optional) – add all the metadata to the ObsCollection DataFrame

Returns

ObsCollection DataFrame without the ‘obs’ column

Return type

ObsCollection

hydropandas.obs_collection.read_dino(dirname=None, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, subdir='Grondwaterstanden_Put', suffix='1.csv', keep_all_obs=True, name=None, **kwargs)[source]

Read dino observations within an extent from the server or from a directory with downloaded files.

Parameters

  • dirname (str, optional) – directory name, can be a .zip file or the parent directory of subdir

  • ObsClass (type) – class of the observations, so far only GroundwaterObs is supported

  • subdir (str) – subdirectory of dirname with data files

  • suffix (str) – suffix of files in subdir that will be read

  • keep_all_obs (boolean, optional) – add all observation points to the collection, even the points without measurements or metadata

  • name (str, optional) – the name of the observation collection

  • kwargs – kwargs are passed to the hydropandas.io.dino.read_dino_dir() function

Returns

collection of multiple point observations

Return type

ObsCollection

hydropandas.obs_collection.read_excel(path, meta_sheet_name='metadata')[source]

Create an observation collection from an excel file. The excel file should have the same format as excel files created with the to_excel method of an ObsCollection.

Parameters

  • path (str) – full file path (including extension) of the excel file.

  • meta_sheet_name (str, optional) – sheetname with metadata. The default is “metadata”.

Return type

ObsCollection

Notes

if you write an excel file using the ‘to_excel’ method and read an excel with the ‘read_excel’ method you lose this information: - The ‘name’ and ‘meta’ attributes of the ObsCollection - metadata of each Observation stored in the ‘meta’ attribute

If you don’t want to lose this data consider using the to_pickle and read_pickle function.

hydropandas.obs_collection.read_fews(file_or_dir=None, xmlstring=None, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, name='fews', translate_dic=None, filterdict=None, locations=None, remove_nan=True, low_memory=True, unpackdir=None, force_unpack=False, preserve_datetime=False, **kwargs)[source]

Read one or several FEWS PI-XML files.

Parameters

  • file_or_dir (str) – zip, xml or directory with zips or xml files to read

  • xmlstring (str or None) – string with xml data, only used if file_or_dir is None. Default is None

  • ObsClass (type) – class of the observations, e.g. GroundwaterObs or WaterlvlObs

  • name (str, optional) – name of the observation collection, ‘fews’ by default

  • translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘locatie’}.

  • filterdict (dict, optional) – dictionary with tag name to apply filter to as keys, and list of accepted names as dictionary values to keep in final result, i.e. {“locationId”: [“B001”, “B002”]}

  • locations (list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read. Only supported by low_memory=True method!

  • low_memory (bool, optional) – whether to use xml-parsing method with lower memory footprint, default is True

  • remove_nan (boolean, optional) – remove nan values from measurements, flag information about the nan values is also lost, only used if low_memory=False

  • unpackdir (str) – destination directory to unzip file if file_or_dir is a .zip

  • force_unpack (boolean, optional) – force unpack if dst already exists

  • preserve_datetime (boolean, optional) – whether to preserve datetime from zip archive

Returns

collection of multiple point observations

Return type

ObsCollection

hydropandas.obs_collection.read_imod(obs_collection, ml, runfile, mtime, model_ws, modelname='', nlay=None, exclude_layers=0)[source]

Read imod model results at point locations.

Parameters

  • obs_collection (ObsCollection) – collection of observations at which points imod results will be read

  • ml (flopy.modflow.mf.model) – modflow model

  • runfile (Runfile) – imod runfile object

  • mtime (list of datetimes) – datetimes corresponding to the model periods

  • model_ws (str) – model workspace with imod model

  • nlay (int, optional) – number of layers if None the number of layers from ml is used.

  • modelname (str) – modelname

  • exclude_layers (int) – exclude modellayers from being read from imod

Returns

collection of multiple point observations

Return type

ObsCollection

hydropandas.obs_collection.read_knmi(locations=None, stns=None, xy=None, meteo_vars=('RH',), name='', starts=None, ends=None, ObsClasses=None, fill_missing_obs=False, interval='daily', use_api=True, raise_exceptions=True)[source]

Get knmi observations from a list of locations or a list of stations.

Parameters

  • locations (pandas DataFrame or None) – dataframe with columns ‘x’ and ‘y’ as coordinates. The default is None

  • stns (list of str or None) – list of knmi stations. The default is None

  • xy (list or numpy array, optional) – xy coordinates of the locations. e.g. [[10,25], [5,25]]

  • meteo_vars (list or tuple of str) – meteo variables e.g. [“RH”, “EV24”]. The default is (“RH”). See list of all possible variables below

  • name (str, optional) – name of the obscollection. The default is ‘’

  • starts (None, str, datetime or list, optional) – start date of observations per meteo variable. The start date is included in the time series. If start is None the start date will be January 1st of the previous year. If start is str it will be converted to datetime. If start is a list it should be the same length as meteo_vars and the start time for each variable. The default is None

  • ends (list of str, datetime or None) – end date of observations per meteo variable. The end date is included in the time series. If end is None the start date will be January 1st of the previous year. If end is a str it will be converted to datetime. If end is a list it should be the same length as meteo_vars and the end time for each meteo variable. The default is None

  • ObsClasses (list of type or None) – class of the observations, can be PrecipitationObs, EvaporationObs or MeteoObs. If None the type of observations is derived from the meteo_vars.

  • **kwargs – kwargs are passed to the hydropandas.io.knmi.get_knmi_obslist function

  • variables (List of possible) – neerslagstations: RD = de 24-uurs neerslagsom, gemeten van 0800 utc op de voorafgaande dag tot 0800 utc op de vermelde datum meteostations: DDVEC = Vectorgemiddelde windrichting in graden (360=noord, 90=oost, 180=zuid, 270=west, 0=windstil/variabel). Zie http://www.knmi.nl/kennis-en-datacentrum/achtergrond/klimatologische-brochures-en-boeken / Vector mean wind direction in degrees (360=north, 90=east, 180=south, 270=west, 0=calm/variable) FHVEC = Vectorgemiddelde windsnelheid (in 0.1 m/s). Zie http://www.knmi.nl/kennis-en-datacentrum/achtergrond/klimatologische-brochures-en-boeken / Vector mean windspeed (in 0.1 m/s) FG = Etmaalgemiddelde windsnelheid (in 0.1 m/s) / Daily mean windspeed (in 0.1 m/s) FHX = Hoogste uurgemiddelde windsnelheid (in 0.1 m/s) / Maximum hourly mean windspeed (in 0.1 m/s) FHXH = Uurvak waarin FHX is gemeten / Hourly division in which FHX was measured FHN = Laagste uurgemiddelde windsnelheid (in 0.1 m/s) / Minimum hourly mean windspeed (in 0.1 m/s) FHNH = Uurvak waarin FHN is gemeten / Hourly division in which FHN was measured FXX = Hoogste windstoot (in 0.1 m/s) / Maximum wind gust (in 0.1 m/s) FXXH = Uurvak waarin FXX is gemeten / Hourly division in which FXX was measured TG = Etmaalgemiddelde temperatuur (in 0.1 graden Celsius) / Daily mean temperature in (0.1 degrees Celsius) TN = Minimum temperatuur (in 0.1 graden Celsius) / Minimum temperature (in 0.1 degrees Celsius) TNH = Uurvak waarin TN is gemeten / Hourly division in which TN was measured TX = Maximum temperatuur (in 0.1 graden Celsius) / Maximum temperature (in 0.1 degrees Celsius) TXH = Uurvak waarin TX is gemeten / Hourly division in which TX was measured T10N = Minimum temperatuur op 10 cm hoogte (in 0.1 graden Celsius) / Minimum temperature at 10 cm above surface (in 0.1 degrees Celsius) T10NH = 6-uurs tijdvak waarin T10N is gemeten / 6-hourly division in which T10N was measured; 6=0-6 UT, 12=6-12 UT, 18=12-18 UT, 24=18-24 UT SQ = Zonneschijnduur (in 0.1 uur) berekend uit de globale straling (-1 voor <0.05 uur) / Sunshine duration (in 0.1 hour) calculated from global radiation (-1 for <0.05 hour) SP = Percentage van de langst mogelijke zonneschijnduur / Percentage of maximum potential sunshine duration Q = Globale straling (in J/cm2) / Global radiation (in J/cm2) DR = Duur van de neerslag (in 0.1 uur) / Precipitation duration (in 0.1 hour) RH = Etmaalsom van de neerslag (in 0.1 mm) (-1 voor <0.05 mm) / Daily precipitation amount (in 0.1 mm) (-1 for <0.05 mm) RHX = Hoogste uursom van de neerslag (in 0.1 mm) (-1 voor <0.05 mm) / Maximum hourly precipitation amount (in 0.1 mm) (-1 for <0.05 mm) RHXH = Uurvak waarin RHX is gemeten / Hourly division in which RHX was measured PG = Etmaalgemiddelde luchtdruk herleid tot zeeniveau (in 0.1 hPa) berekend uit 24 uurwaarden / Daily mean sea level pressure (in 0.1 hPa) calculated from 24 hourly values PX = Hoogste uurwaarde van de luchtdruk herleid tot zeeniveau (in 0.1 hPa) / Maximum hourly sea level pressure (in 0.1 hPa) PXH = Uurvak waarin PX is gemeten / Hourly division in which PX was measured PN = Laagste uurwaarde van de luchtdruk herleid tot zeeniveau (in 0.1 hPa) / Minimum hourly sea level pressure (in 0.1 hPa) PNH = Uurvak waarin PN is gemeten / Hourly division in which PN was measured P = Luchtdruk (in 0.1 hPa) herleid tot zeeniveau, op het moment van meten / Air pressure (in 0.1 hPa) reduced to mean sea level, at the time of observation VVN = Minimum opgetreden zicht / Minimum visibility; 0: <100 m, 1:100-200 m, 2:200-300 m,…, 49:4900-5000 m, 50:5-6 km, 56:6-7 km, 57:7-8 km,…, 79:29-30 km, 80:30-35 km, 81:35-40 km, …, 89: >70 km) VVNH = Uurvak waarin VVN is gemeten / Hourly division in which VVN was measured VVX = Maximum opgetreden zicht / Maximum visibility; 0: <100 m, 1:100-200 m, 2:200-300 m,…, 49:4900-5000 m, 50:5-6 km, 56:6-7 km, 57:7-8 km,…, 79:29-30 km, 80:30-35 km, 81:35-40 km, …, 89: >70 km) VVXH = Uurvak waarin VVX is gemeten / Hourly division in which VVX was measured NG = Etmaalgemiddelde bewolking (bedekkingsgraad van de bovenlucht in achtsten, 9=bovenlucht onzichtbaar) / Mean daily cloud cover (in octants, 9=sky invisible) UG = Etmaalgemiddelde relatieve vochtigheid (in procenten) / Daily mean relative atmospheric humidity (in percents) UX = Maximale relatieve vochtigheid (in procenten) / Maximum relative atmospheric humidity (in percents) UXH = Uurvak waarin UX is gemeten / Hourly division in which UX was measured UN = Minimale relatieve vochtigheid (in procenten) / Minimum relative atmospheric humidity (in percents) UNH = Uurvak waarin UN is gemeten / Hourly division in which UN was measured EV24 = Referentiegewasverdamping (Makkink) (in 0.1 mm) / Potential evapotranspiration (Makkink) (in 0.1 mm)

Returns

collection of multiple point observations

Return type

ObsCollection

hydropandas.obs_collection.read_lizard(extent=None, codes=None, name='', tube_nr='all', tmin=None, tmax=None, type_timeseries='merge', only_metadata=False)[source]

Get all observations from a list of codes of the monitoring wells and a list of tube numbers.

Parameters

  • extent (list, shapefile path or None) – get groundwater monitoring wells within this extent [xmin, ymin, xmax, ymax] or within a predefined Polygon from a shapefile

  • codes (lst of str or None) – codes of the monitoring wells

  • tube_nr (lst of str) – list of tube numbers of the monitoring wells that should be selected. By default ‘all’ available tubes are selected.

  • tmin (str YYYY-m-d, optional) – start of the observations, by default the entire time series is returned

  • tmax (Ttr YYYY-m-d, optional) – end of the observations, by default the entire time series is returned

  • type_timeseries (str, optional) – hand: returns only hand measurements diver: returns only diver measurements merge: the hand and diver measurements into one time series (default) combine: keeps hand and diver measurements separeted

  • only_metadata (bool, optional) – if True only metadata is returned and no time series data. The default is False.

Returns

ObsCollection DataFrame with the ‘obs’ column

Return type

ObsCollection

hydropandas.obs_collection.read_menyanthes(path, name='', ObsClass=<class 'hydropandas.observation.Obs'>, load_oseries=True, load_stresses=True)[source]

Read a Menyanthes file.

Parameters

  • path (str) – full path of the .men file.

  • name (str, optional) – name of the observation collection. The default is “”.

  • ObsClass (type, optional) – class of the observations, e.g. GroundwaterObs. The default is obs.Obs.

  • load_oseries (bool, optional) – if True the observations are read. The default is True.

  • load_stresses (bool, optional) – if True the stresses are read. The default is True.

Returns

collection of multiple point observations

Return type

ObsCollection

hydropandas.obs_collection.read_modflow(obs_collection, ml, hds_arr, mtime, modelname='', nlay=None, exclude_layers=None, method='linear')[source]

Read modflow groundwater heads at locations in obs_collection.

Parameters

  • obs_collection (ObsCollection) – locations of model observation

  • ml (flopy.modflow.mf.model) – modflow model

  • hds_arr (numpy array) – heads with shape (ntimesteps, nlayers, nrow, ncol)

  • mtime (list of datetimes) – dates for each model timestep

  • modelname (str, optional) – modelname

  • nlay (int, optional) – number of layers if None the number of layers from ml is used.

  • exclude_layers (list of int, optional) – exclude the observations in these model layers

  • method (str, optional) – interpolation method, either ‘linear’ or ‘nearest’, default is linear

Returns

collection of multiple point observations

Return type

ObsCollection

hydropandas.obs_collection.read_pastastore(pstore, libname, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, metadata_mapping=None)[source]

Read pastastore library.

Parameters

  • pstore (pastastore.PastaStore) – PastaStore object

  • libname (str) – name of library (e.g. oseries or stresses)

  • ObsClass (Obs, optional) – type of Obs to read data as, by default obs.GroundwaterObs

  • metadata_mapping (dict, optional) – dictionary containing map between metadata field names in pastastore and metadata field names expected by hydropandas, by default None.

Returns

ObsCollection containing data

Return type

ObsCollection

hydropandas.obs_collection.read_pickle(filepath_or_buffer, compression='infer', storage_options=None)[source]

Wrapper around pd.read_pickle.

Parameters

  • filepath_or_buffer (str, path object, or file-like object) –

    String, path object (implementing os.PathLike[str]), or file-like object implementing a binary readlines() function.

    Changed in version 1.0.0.

    Accept URL. URL is not limited to S3 and GCS.

  • compression (str or dict, default 'infer') –

    For on-the-fly decompression of on-disk data. If ‘infer’ and ‘filepath_or_buffer’ is path-like, then detect compression from the following extensions: ‘.gz’,’.bz2’, ‘.zip’, ‘.xz’, or ‘.zst’ (otherwise no compression). If using ‘zip’, the ZIP file must contain only one data file to be read in. Set to None for no decompression. Can also be a dict with key 'method' set to one of {'zip', 'gzip', 'bz2', 'zstd'} and other key-value pairs are forwarded to zipfile.ZipFile, gzip.GzipFile, bz2.BZ2File, or zstandard.ZstdDecompressor, respectively. As an example, the following could be passed for Zstandard decompression using a custom compression dictionary: compression={'method': 'zstd', 'dict_data': my_compression_dict}.

    Changed in version 1.4.0: Zstandard support.

  • storage_options (dict, optional) –

    Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with “s3://”, and “gcs://”) the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details.

    New in version 1.2.0.

Returns

ObsCollection

Return type

same type as object stored in file

hydropandas.obs_collection.read_waterinfo(file_or_dir, name='', ObsClass=<class 'hydropandas.observation.WaterlvlObs'>, progressbar=True, **kwargs)[source]

Read waterinfo file or directory.

Parameters

  • file_or_dir (str) – path to file or directory. Files can be .csv or .zip

  • name (str, optional) – name of the collection, by default “”

  • ObsClass (Obs, optional) – type of Obs to read data as, by default obs.WaterlvlObs

  • progressbar (bool, optional) – show progressbar, by default True

Returns

ObsCollection containing data

Return type

ObsCollection

hydropandas.obs_collection.read_wiski(dirname, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, suffix='.csv', unpackdir=None, force_unpack=False, preserve_datetime=False, keep_all_obs=True, **kwargs)[source]
Parameters

  • dirname (str) – path of the zipfile with wiski data.

  • ObsClass (type, optional) – type of Obs. The default is obs.GroundwaterObs.

  • suffix (str, optional) – extension of filenames to read. The default is “.csv”.

  • unpackdir (str or None, optional) – directory to unpack zipped directory. The default is None.

  • force_unpack (bool, optional) – force unzip, by default False.

  • preserve_datetime (bool, optional) – preserve datetime of unzipped files, by default False (useful for checking whether data has changed)

  • keep_all_obs (bool, optional) – If True keep all observations even those without metadata. The default is True.

  • **kwargs –

Returns

ObsCollection containing observation data

Return type

ObsCollection

hydropandas.observation module

Module with observation classes.

The Obs class is a subclass of a pandas DataFrame with additional attributes and methods. The specific classes (GroundwaterObs, WaterlvlObs, …) are subclasses of the Obs class.

The subclasses of a dataframe can have additional attributes and methods. Additional attributes have to be defined in the ‘_metadata’ attribute. In order to keep the subclass methods and attributes when selecting or slicing an object you need the ‘_constructor’ method.

More information about subclassing pandas DataFrames can be found here: http://pandas.pydata.org/pandas-docs/stable/development/extending.html#extending-subclassing-pandas

class hydropandas.observation.EvaporationObs(*args, **kwargs)[source]

Bases: MeteoObs

Class for evaporation timeseries.

Subclass of the MeteoObs class

classmethod from_knmi(meteo_var='EV24', stn=None, fname=None, xy=None, start=None, end=None, fill_missing_obs=False, interval='daily', use_api=True, raise_exceptions=True, startdate=None, enddate=None)[source]

Get an EvaporationObs timeseries from the KNMI evaporation in m.

Parameters

  • meteo_var (str, optional) – meteo variable should be “EV24”.

  • stn (int, str or None, optional) – measurement station e.g. 829. The default is None.

  • fname (str, path object, file-like object or None, optional) – filename of a knmi file. The default is None.

  • xy (list, tuple or None, optional) – RD coördinates of a location in the Netherlands. The station nearest to this location used. The Default is None.

  • start (str, datetime or None, optional) – start date of observations. The default is None.

  • end (str, datetime or None, optional) – end date of observations. The default is None.

  • fill_missing_obs (bool, optional) – if True nan values in time series are filled with nearby time series. The default is False.

  • interval (str, optional) – desired time interval for observations. Options are ‘daily’ and ‘hourly’. The default is ‘daily’.

  • inseason (boolean, optional) – flag to obtain inseason data. The default is False

  • raise_exceptions (bool, optional) – if True you get errors when no data is returned. The default is False.

  • use_api (bool, optional) –

    if True the api is used to obtain the data, API documentation is here:

    https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

    if False a text file is downloaded into a temporary directory and the data is read from there. Default is True since the api is back online (July 2021).

Return type

EvaporationObs object with an evaporation time series and attributes

class hydropandas.observation.GroundwaterObs(*args, **kwargs)[source]

Bases: Obs

Class for groundwater quantity observations.

Subclass of the Obs class. Has the following attributes:

  • monitoring_well: 2 tubes at one piezometer should have the same ‘monitoring_well’

  • tube_nr: 2 tubes at one piezometer should have a different ‘tube_nr’.

  • screen_top: top op the filter in m above date (NAP)

  • screen_bottom: bottom of the filter in m above date (NAP)

  • ground_level: surface level in m above date (NAP) (maaiveld in Dutch)

  • tube_top: top of the tube in m above date (NAP)

  • metadata_available: boolean indicating if metadata is available for the measurement point.

classmethod from_artdino_file(path=None, **kwargs)[source]

Read a dino csv file (artdiver style).

Parameters

  • path (str, optional) – path of dino csv filename

  • kwargs (key-word arguments) – these arguments are passed to hydropandas.io._dino.read_dino_groundwater_csv

classmethod from_bro(bro_id, tube_nr=None, tmin='1900-01-01', tmax='2040-01-01', to_wintertime=True, drop_duplicate_times=True, only_metadata=False)[source]

Download BRO groundwater observations from the server.

Parameters

  • bro_id (str) – can be a GLD id or GMW id. If a GMW id is given a tube number is required as well. e.g. ‘GLD000000012893’.

  • tube_nr (str or None, optional) – if the bro_id is a GMW object the tube number should be given.

  • tmin (str or None, optional) – start date in format YYYY-MM-DD

  • tmax (str or None, optional) – end date in format YYYY-MM-DD

  • to_wintertime (bool, optional) – if True the time index is converted to Dutch winter time. The default is True.

  • drop_duplicate_times (bool, optional) – if True rows with a duplicate time stamp are removed keeping only the first row. The default is True.

  • only_metadata (bool, optional) – if True only metadata is returned and no time series data. The default is False

Returns

DESCRIPTION.

Return type

TYPE

classmethod from_bronhouderportaal_bro(path, tube_nr, full_meta=False)[source]

Load BRO groundwater metadata from XML file. Mind that bro_id is applicable, because file is not yet imported in BRO.

Parameters

  • path (str) – filepath of XML file.

  • tube_nr (int) – tube number.

  • full_meta (bool) – process not only the standard metadata to ObsCollection.

Returns

ObsCollection containing observations from XML file.

Return type

ObsCollection

classmethod from_dino(path=None, **kwargs)[source]

Download dino data from the server.

Parameters

  • path (str, optional) – path of dino csv file

  • kwargs (key-word arguments) – these arguments are passed to hydropandas.io.dino.read_dino_groundwater_csv if path is not None and otherwise to hydropandas.io.dino.findMeetreeks

classmethod from_lizard(code, tube_nr=None, tmin=None, tmax=None, type_timeseries='merge', only_metadata=False)[source]

Extracts the metadata and timeseries of a observation well from a LIZARD-API based on the code of a monitoring well.

Parameters

  • code (str) – code of the measuring well

  • tube_nr (int, optional) – select specific tube top Default selects tube_nr = 1

  • tmin (str YYYY-m-d, optional) – start of the observations, by default the entire serie is returned

  • tmax (Ttr YYYY-m-d, optional) – end of the observations, by default the entire serie is returned

  • type_timeseries (str, optional) – hand: returns only hand measurements diver: returns only diver measurements merge: the hand and diver measurements into one time series (default) combine: keeps hand and diver measurements separated

  • only_metadata (bool, optional) – if True only metadata is returned and no time series data. The default is False.

Returns

Returns a DataFrame with metadata and timeseries

Return type

ObsCollection

classmethod from_pastastore(pstore, libname, name, metadata_mapping=None)[source]

Read item from pastastore library.

Parameters

  • pstore (pastastore.PastaStore) – pastastore object

  • libname (str) – name of library containinig item

  • name (str) – name of item

  • metadata_mapping (dict, optional) – dictionary containing map between metadata field names in pastastore (keys) and metadata field names expected by hydropandas (values), by default None.

classmethod from_solinst(path, transform_coords=True, screen_bottom=None, screen_top=None, ground_level=None, tube_nr=None, tube_top=None)[source]

Read data from Solinst xle file.

Parameters

path (str) – path to file (file can zip or xle)

classmethod from_wiski(path, **kwargs)[source]

Read data from a WISKI file.

Parameters:

pathstr

The path of the file to be read.

sepstr, optional (default=”;”)

The delimiter used to separate fields in the file.

header_sepstr, optional (default=None)

The delimiter used to separate fields in the header. If None, the function will try to automatically detect the separator.

header_identifierstr, optional (default=”#”)

The character used to identify header lines.

read_seriesbool, optional (default=True)

Whether to read the time series data from the file.

translate_dicdict, optional (default=None)

A dictionary mapping header field names to the desired output names.

tz_localizebool, optional (default=True)

Whether to localize the datetime index to the machine’s timezone.

unitstr, optional (default=””)

The unit of measurement of the data.

**kwargskeyword arguments

Additional arguments to pass to the pandas read_csv function.

class hydropandas.observation.MeteoObs(*args, **kwargs)[source]

Bases: Obs

Class for meteorological timeseries.

Subclass of the Obs class

classmethod from_knmi(meteo_var, stn=None, fname=None, xy=None, start=None, end=None, fill_missing_obs=False, interval='daily', use_api=True, raise_exceptions=True, startdate=None, enddate=None)[source]

Get a MeteoObs timeseries from the KNMI meteo data.

Parameters

  • meteo_var (str) – meteo variable e.g. “RH” or “EV24”. For a list of possible variables see the hydropandas.read_knmi function.

  • stn (int, str or None, optional) – measurement station e.g. 829. The default is None.

  • fname (str, path object, file-like object or None, optional) – filename of a knmi file. The default is None.

  • xy (list, tuple or None, optional) – RD coördinates of a location in the Netherlands. The station nearest to this location used. The Default is None.

  • start (str, datetime or None, optional) – start date of observations. The default is None.

  • end (str, datetime or None, optional) – end date of observations. The default is None.

  • fill_missing_obs (bool, optional) – if True nan values in time series are filled with nearby time series. The default is False.

  • interval (str, optional) – desired time interval for observations. Options are ‘daily’ and ‘hourly’. The default is ‘daily’.

  • use_api (bool, optional) –

    if True the api is used to obtain the data, API documentation is here:

    https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

    if False a text file is downloaded into a temporary directory and the data is read from there. Default is True since the api is back online (July 2021).

  • raise_exceptions (bool, optional) – if True you get errors when no data is returned. The default is False.

Return type

MeteoObs object with meteorological observations

classmethod from_wow(meteo_var: str, stn: str = None, xy: List[float] = None, start: Optional[Timestamp] = None, end: Optional[Timestamp] = None)[source]

Get a MeteoObs timeseries from a wow.knmi.nl station.

Parameters

  • meteo_var (str) – wow meteo variable

  • stn (Optional[int, str], optional) – station name

  • xy (Optinal[List[float]], optinial) – longitude latitude of location [lon, lat] eg: [4.85, 51.95]

  • start (Optional[pd.Timestamp], optional) – start date of observations, by default None

  • end (Optional[pd.Timestamp], optional) – start date of observations, by default None

Return type

MeteoObs

class hydropandas.observation.ModelObs(*args, **kwargs)[source]

Bases: Obs

Class for model point results.

Subclass of the Obs class

class hydropandas.observation.Obs(*args, **kwargs)[source]

Bases: DataFrame

Generic class for a time series with measurements at a certain location.

Unless specified explicitly the first numeric column in the observation is used for analysis and plotting.

Parameters

  • name (str) – name

  • x (int or float) – x coordinate of observation point

  • y (int or float) – y coordinate of observation point

  • meta (dictionary) – metadata

  • filename (str) – filename with data of observation point

  • source (str) – source of the observation e.g. BRO or KNMI

  • unit (str) – unit of the first numerical column in the observation

copy(deep=True)[source]

Create a copy of the observation.

When deep=True (default), a new object will be created with a copy of the calling object’s data and indices. Modifications to the data or indices of the copy will not be reflected in the original object (see notes below).

When deep=False, a new object will be created without copying the calling object’s data or index (only references to the data and index are copied). Any changes to the data of the original will be reflected in the shallow copy (and vice versa).

Parameters

deep (bool, default True) – Make a deep copy, including a copy of the data and the indices. With deep=False neither the indices nor the data are copied.

Returns

o – DESCRIPTION.

Return type

TYPE

geo

alias of GeoAccessorObs

gwobs

alias of GeoAccessorObs

merge_metadata(right, overlap='error')[source]

Merge the metadata of an Obs object with metadata from another Obs object.

Parameters

  • right (dict) – dictionary with the metadata of an Obs object.

  • overlap (str, optional) –

    How to deal with overlapping metadata with different values. Options are:

    error : Raise a ValueError use_left : Use the metadata from self use_right : Use the given metadata

    Default is ‘error’.

Raises

ValueError – if the metadata differs and overlap=’error’.

Returns

new_metadata – metadata after merge.

Return type

dict

merge_observation(right, overlap='error', merge_metadata=True)[source]

Merge with another observation of the same type.

Parameters

  • right (hpd.observation.Obs) – Observation object.

  • overlap (str, optional) –

    How to deal with overlapping timeseries or metadata with different values. Options are:

    error : Raise a ValueError use_left : use the part of the overlapping timeseries from self use_right : use the part of the overlapping timeseries from right

    Default is ‘error’.

  • merge_metadata (bool, optional) – If True the metadata of the two objects are merged. If there are any differences the overlap parameter is used to determine which metadata is used. If merge_metadata is False, the metadata of self is always used for the merged observation. The default is True.

Raises

  • TypeError – when the observation types are not the same.

  • ValueError – when the time series have different values on the same date or different values for the same metadata.

Return type

Observation object.

plots

alias of ObsPlots

stats

alias of StatsAccessorObs

to_collection_dict(include_meta=False)[source]

Get dictionary with registered attributes and their values of an Obs object.

This method can be used to create a dataframe from a collection of Obs objects.

Parameters

include_meta (boolean, optional) – include the meta dictionary in the collection dictionary, default is false

Returns

d – dictionary with Obs information

Return type

dictionary

class hydropandas.observation.PrecipitationObs(*args, **kwargs)[source]

Bases: MeteoObs

Class for precipitation timeseries.

Subclass of the MeteoObs class

classmethod from_knmi(meteo_var='RH', stn=None, fname=None, xy=None, start=None, end=None, fill_missing_obs=False, interval='daily', use_api=True, raise_exceptions=True, startdate=None, enddate=None)[source]

Get a PrecipitationObs timeseries from the KNMI precipitation. The precipitation is the Daily precipitation amount (in 0.1 mm) (-1 for.

<0.05 mm).

There are 3 different ways to obtain precipitation data from the knmi:

  1. Daily data from precipitation (neerslag) stations

  2. Daily data from meteo stations

  3. Hourly data from meteo stations

1. For daily data from a precipitation station (neerslagstation) meteo_var should be ‘RD’. 2. For daily data from a meteo station meteo_var should be ‘RH’ and interval should be ‘daily’ (default). 3. For hourly data from a meteo station meteo_var should be ‘RH’ and interval should be ‘hourly’.

More information about the differences between neerslag and meteo stations can be found in the hydropandas documentation -> 02_knmi_observations notebook.

Parameters

  • meteo_var (str, optional) – meteo variable can be “RH” or “RD”. “RD” if you want data from a precipitation station (neerslagstation). “RH” if you want data from a meteo station. The default is “RH”.

  • stn (int, str or None, optional) – measurement station e.g. 829. The default is None.

  • fname (str, path object, file-like object or None, optional) – filename of a knmi file. The default is None.

  • xy (list, tuple or None, optional) – RD coördinates of a location in the Netherlands. The station nearest to this location used. The Default is None.

  • start (str, datetime or None, optional) – start date of observations. The default is None.

  • end (str, datetime or None, optional) – end date of observations. The default is None.

  • fill_missing_obs (bool, optional) – if True nan values in time series are filled with nearby time series. The default is False.

  • interval (str, optional) – desired time interval for observations. Options are ‘daily’ and ‘hourly’. The default is ‘daily’.

  • use_api (bool, optional) –

    if True the api is used to obtain the data, API documentation is here:

    https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

    if False a text file is downloaded into a temporary folder and the data is read from there. Default is True since the api is back online (July 2021).

  • raise_exceptions (bool, optional) – if True you get errors when no data is returned. The default is False.

Return type

PrecipitationObs object with a precipitation time series and attributes

classmethod from_wow(stn: str = None, xy: List[float] = None, start: Optional[Timestamp] = None, end: Optional[Timestamp] = None)[source]

Get a PrecipitationObs timeseries from a wow.knmi.nl station.

Parameters

  • stn (Optional[int, str], optional) – station name

  • xy (Optinal[List[float]], optinial) – longitude latitude of location [lon, lat] eg: [4.85, 51.95]

  • start (Optional[pd.Timestamp], optional) – start date of observations, by default None

  • end (Optional[pd.Timestamp], optional) – start date of observations, by default None

Return type

PrecipitationObs

class hydropandas.observation.WaterQualityObs(*args, **kwargs)[source]

Bases: Obs

Class for water quality ((grond)watersamenstelling) point observations.

Subclass of the Obs class

classmethod from_dino(path, **kwargs)[source]

Read dino file with groundwater quality data.

Parameters

  • path (str) – path of dino txt filename

  • kwargs (key-word arguments) – these arguments are passed to hydropandas.io.dino.read_dino_groundwater_quality_txt

class hydropandas.observation.WaterlvlObs(*args, **kwargs)[source]

Bases: Obs

Class for water level point observations.

Subclass of the Obs class

classmethod from_dino(path, **kwargs)[source]

Read a dino file with waterlvl data.

Parameters

  • path (str) – path of dino csv filename

  • kwargs (key-word arguments) – these arguments are passed to hydropandas.io.dino.read_dino_waterlvl_csv

classmethod from_waterinfo(path, **kwargs)[source]

Read data from waterinfo csv-file or zip.

Parameters

path (str) – path to file (file can zip or csv)

Returns

df – WaterlvlObs object

Return type

WaterlvlObs

Raises

ValueError – if file contains data for more than one location

hydropandas.util module

Created on Wed Sep 12 12:15:42 2018.

@author: Artesia

class hydropandas.util.ColoredFormatter(*args, colors: Optional[Dict[str, str]] = None, **kwargs)[source]

Bases: Formatter

Colored log formatter.

Taken from https://gist.github.com/joshbode/58fac7ababc700f51e2a9ecdebe563ad

format(record) → str[source]

Format the specified record as text.

hydropandas.util.df2gdf(df, xcol='x', ycol='y', crs=28992)[source]

Create a GeoDataFrame from a DataFrame with xy points.

Parameters

  • df (pd.DataFrame) – input dataframe

  • xcol (str, optional) – column name with x values. The default is ‘x’.

  • ycol (str, optional) – column name with y values. The default is ‘x’.

  • crs (int, optional) – coordinate reference system, by default 28992 (RD new).

Returns

geodataframe

Return type

geopandas GeoDataFrame

hydropandas.util.get_color_logger(level='INFO')[source]
hydropandas.util.get_files(file_or_dir, ext, unpackdir=None, force_unpack=False, preserve_datetime=False)[source]

Internal method to get list of files with specific extension from dirname.

Parameters

  • file_or_dir (str) – file or path to data.

  • ext (str) – extension of filenames to store in list.

  • unpackdir (str) – directory to story unpacked zip file, only used in case of a zipfile.

  • force_unpack (bool, optional) – force unzip, by default False.

  • preserve_datetime (bool, optional) – preserve datetime of unzipped files, by default False. Used for checking whether data has changed.

hydropandas.util.interpolate(xy: List[List[float]], obsdf: DataFrame, obsloc: DataFrame, kernel: str = 'thin_plate_spline', kernel2: str = 'linear', epsilon: Optional[int] = None) → DataFrame[source]

Interpolation method using the Scipy radial basis function (RBF)

Parameters

  • xy (List[List[float]]) – xy coordinates of locations of interest e.g. [[10,25], [5,25]]

  • obsdf (DataFrame) – Dataframe containing the observation locations as columns and the observations at a measurement time in each row.

  • obsloc (DataFrame) – Dataframe containing the observation locations coordinates with observation locations as index and columns [“x”, “y”]

  • kernel (str, optional) – Type of radial basis funtion, by default thin_plate_spline. Other options are linear, gaussian, inverse_quadratic, multiquadric, inverse_multiquadric, cubic or quintic.

  • kernel2 (str, optional) – Kernel in case there are not enough observations (3 or 6) for time step, by default linear. Other options are gaussian, inverse_quadratic, multiquadric, or inverse_multiquadric.

  • epsilon (Optional[int], optional) – Shape parameter that scales the input to the RBF. If kernel is linear, thin_plate_spline, cubic, or quintic, this defaults to 1. Otherwise this must be specified.

Returns

DataFrame with locations of interest as columns and interpolated values at a measurement time in each row.

Return type

DataFrame

hydropandas.util.oc_to_df(oc, col: Optional[str] = None) → DataFrame[source]

Convert an observation collection to a DataFrame where every column has one observation.

Parameters

  • oc (hydropandas ObsCollection) – observation collection

  • col (Optional[str], optional) – Name of a column in hte observation collection, by default None

Returns

_description_

Return type

DataFrame

hydropandas.util.show_versions()[source]

Method to print the version of dependencies.

hydropandas.util.unzip_file(src, dst, force=False, preserve_datetime=False)[source]

Unzip file.

Parameters

  • src (str) – source zip file

  • dst (str) – destination directory

  • force (boolean, optional) – force unpack if dst already exists

  • preserve_datetime (boolean, optional) – use date of the zipfile for the destination file

Returns

1 of True

Return type

int