hydropandas.io package

hydropandas.io.arctic module

hydropandas.io.dino module

hydropandas.io.dino.read_artdino_dir(dirname, ObsClass=None, subdir='csv', suffix='1.csv', unpackdir=None, force_unpack=False, preserve_datetime=False, keep_all_obs=True, **kwargs)

Read Dino directory with point observations.

TODO: - Evt. nog verbeteren door meteen Dataframe te vullen op het moment dat een observatie wordt ingelezen. Nu wordt eerst alles ingelezen in een lijst en daar een dataframe van gemaakt.

Parameters

• dirname (str) – directory name, can be a .zip file or the parent directory of subdir

• ObsClass (type) – class of the observations, e.g. GroundwaterObs or WaterlvlObs

• subdir (str) – subdirectory of dirname with data files

• suffix (str) – suffix of files in subdir that will be read

• unpackdir (str) – destination directory of the unzipped file

• force_unpack (boolean, optional) – force unpack if dst already exists

• preserve_datetime (boolean, optional) – use date of the zipfile for the destination file

• keep_all_obs (boolean, optional) – add all observation points to the collection, even without data or metadata

• **kwargs (dict, optional) – Extra arguments are passed to ObsClass.from_artdino_file()

Returns

obs_df – collection of multiple point observations

Return type

pd.DataFrame

hydropandas.io.dino.read_artdino_groundwater_csv(path, to_mnap=True, read_series=True)

Read dino groundwater quantity data from a CSV file as exported by ArtDiver.

Parameters

• path (str) – path to csv file

• to_mnap (boolean, optional) – if True a column with ‘stand_m_tov_nap’ is added to the dataframe

• read_series (boolean, optional) – if False only metadata is read, default is True

Returns

• measurements (pd.DataFrame)

• meta (dict) – dictionary with metadata

hydropandas.io.dino.read_dino_dir(path: Union[str, Path], ObsClass, subdir: str = 'Grondwaterstanden_Put', suffix: str = '1.csv', keep_all_obs: bool = True, **kwargs: dict)

Read Dino directory with point observations.

TODO: - Evt. nog verbeteren door meteen Dataframe te vullen op het moment dat een observatie wordt ingelezen. Nu wordt eerst alles ingelezen in een lijst en daar een dataframe van gemaakt. - aparte unzip functie maken en toch de juiste tijdelijke directory krijgen.

Parameters

• path (str | Path) – directory name, can be a .zip file or the parent directory of subdir

• ObsClass (type) – class of the observations, e.g. GroundwaterObs or WaterlvlObs

• subdir (str) – subdirectory of dirname with data files

• suffix (str) – suffix of files in subdir that will be read

• keep_all_obs (boolean, optional) – add all observation points to the collection, even without data or metadata

• **kwargs (dict, optional) – Extra arguments are passed to ObsClass.from_dino_file()

Returns

obs_df – collection of multiple point observations

Return type

pd.DataFrame

hydropandas.io.dino.read_dino_groundwater_csv(f: Union[str, Path, FileIO], to_mnap: bool = True, read_series: bool = True, remove_duplicates: bool = False, keep_dup: str = 'last')

Read dino groundwater quantity data from a dinoloket csv file.

Parameters

• f (Union[str, Path, TextIOWrapper]) – path to csv file

• to_mnap (boolean, optional) – if True a column with ‘stand_m_tov_nap’ is added to the dataframe

• read_series (boolean, optional) – if False only metadata is read, default is True

• remove_duplicates (boolean, optional) – if True duplicate indices are removed. Default is False.

• keep_dup (str, optional) – indicate which duplicate indices should be kept, only used when remove_duplicates is True. Default is ‘last’

Returns

• measurements (pd.DataFrame)

• meta (dict) – dictionary with metadata

hydropandas.io.dino.read_dino_groundwater_quality_txt(f: Union[str, Path, FileIO])

Read dino groundwater quality (grondwatersamenstelling) from a dinoloket txt file.

Notes

this function has not been tested thoroughly

Parameters

filepath_or_buffer (str) – path to txt file

Returns

• measurements (pd.DataFrame)

• meta (dict) – dictionary with metadata

hydropandas.io.dino.read_dino_waterlvl_csv(f: Union[str, Path, FileIO], to_mnap: bool = True, read_series: bool = True)

Read dino waterlevel data from a dinoloket csv file.

Parameters

• f (str, Path, FileIO) – path to dino water level csv file

• to_mnap (boolean, optional) – if True a column with ‘stand_m_tov_nap’ is added to the dataframe

• read_series (boolean, optional) – if False only metadata is read, default is True

hydropandas.io.fieldlogger module

hydropandas.io.knmi module

Module with functions to read or download time series with observations from knmi.

The main function to download time series are:

• get_knmi_timeseries_xy: obtain a knmi station based on the xy location

• get_knmi_timeseries_stn: obtain a knmi station based on the station number

The main function to read time series txt files is:

• read_knmi_timeseries_file: read a knmi txt file

hydropandas.io.knmi.download_knmi_data(stn, meteo_var, start, end, settings, stn_name=None)

download knmi data of a measurements station for certain observation type.

Parameters

• stn (int) – measurement station.

• meteo_var (str) – observation type.

• start (pd.TimeStamp or None) – start date of observations.

• end (pd.TimeStamp or None) – end date of observations.

• settings (dict) – settings for obtaining data

• stn_name (str, optional) – station name. If None the name is obtained form te stn number. The default is None.

Raises

ValueError – if the data from knmi cannot not be read a ValueError is raised. Unless raise_exceptions is False

Returns

• knmi_df (pandas DataFrame) – data from one station from one type of observation

• variables (dictionary) – information about the observerd variables

• stations (pandas DataFrame) – information about the measurement station.

hydropandas.io.knmi.fill_missing_measurements(stn, meteo_var, start, end, settings, stn_name=None)

fill missing measurements in knmi data.

Parameters

• stn (int) – measurement station.

• meteo_var (str) – observation type.

• start (pd.TimeStamp) – start date of observations.

• end (pd.TimeStamp) – end date of observations.

• settings (dict) – settings for obtaining data.

• stn_name (str, optional) – station name. If None the name is obtained form te stn number. The default is None.

Returns

• knmi_df (pandas DataFrame) – data from one station from one type of observation, with additional column to see which station is used to fill the value

• variables (dictionary) – information about the observerd variables

• stations (pandas DataFrame) – information about the measurement station.

hydropandas.io.knmi.get_evaporation(meteo_var, stn=260, start=None, end=None, settings=None)

Collect different types of (reference) evaporation from KNMI weather stations

Parameters

• meteo_var (str) – Choice between ‘penman’, ‘makkink’ or ‘hargraves’.

• stn (str) – station number, defaults to 260 De Bilt

• start (pd.TimeStamp) – start time of observations.

• end (pd.TimeStamp) – end time of observations.

• settings (dict or None, optional) – settings for the time series

Return type

pandas.DataFrame

hydropandas.io.knmi.get_knmi_daily_meteo_api(stn, meteo_var, start=None, end=None)

download and read knmi daily meteo data.

Parameters

• stn (int) – station number.

• meteo_var (str) – e.g. ‘EV24’.

• start (pd.TimeStamp or None) – start time of observations.

• end (pd.TimeStamp or None) – end time of observations.

Returns

• pandas DataFrame – measurements.

• variables (dictionary) – additional information about the variables

• stations (pandas DataFrame) – additional data about the measurement station

hydropandas.io.knmi.get_knmi_daily_meteo_url(stn, meteo_var, start=None, end=None)

download and read knmi daily meteo data.

Parameters

• stn (str) – station number.

• meteo_var (str) – e.g. ‘EV24’.

• start (pd.TimeStamp or None) – start time of observations.

• end (pd.TimeStamp or None) – end time of observations.

Returns

• pandas DataFrame – measurements.

• variables (dictionary) – additional information about the variables

• stations (pandas DataFrame) – additional data about the measurement station

hydropandas.io.knmi.get_knmi_daily_rainfall_api(stn, start=None, end=None)

download and read knmi daily rainfall.

Parameters

• stn (int) – station number.

• start (pd.TimeStamp or None) – start time of observations.

• end (pd.TimeStamp or None) – end time of observations.

Raises

ValueError – if there is no data for the provided stn an error is raised.

Returns

• pandas DataFrame – measurements.

• variables (dictionary) – additional information about the variables

hydropandas.io.knmi.get_knmi_daily_rainfall_url(stn, stn_name, start=None, end=None)

download and read knmi daily rainfall.

Parameters

• stn (int) – station number.

• stn_name (str) – the name of the station in capital letters, can be tricky

• start (pd.TimeStamp or None) – start time of observations.

• end (pd.TimeStamp or None) – end time of observations.

Raises

ValueError – if there is no data for the provided stn an error is raised.

Returns

• pandas DataFrame – measurements.

• variables (dictionary) – additional information about the variables

hydropandas.io.knmi.get_knmi_hourly_api(stn, meteo_var, start, end)

Retrieve hourly meteorological data from the KNMI API.

Parameters

• stn (int) – The station number.

• meteo_var (str) – The meteorological variable to retrieve.

• start (pd.Timestamp or None) – The start date and time of the data.

• end (pd.Timestamp or None) – The end date and time of the data.

Returns

• pandas.DataFrame – A DataFrame containing the requested meteorological variable data.

• dict – A dictionary containing information about the variables in the DataFrame.

Raises

• requests.ConnectionError – If there is a connection error while accessing the KNMI API.

• RuntimeError – If the KNMI API is down or returns unexpected data.

hydropandas.io.knmi.get_knmi_obs(stn=None, fname=None, xy=None, meteo_var=None, start=None, end=None, **kwargs)

get knmi observation from stn, fname or nearest xy coordinates.

Parameters

• stn (int, str or None, optional) – measurement station e.g. 829. The default is None.

• fname (str, path object, file-like object or None, optional) – filename of a knmi file. The default is None.

• xy (list, tuple or None, optional) – RD coördinates of a location in the Netherlands. The station nearest to this location used. The Default is None.

• meteo_var (str or None, optional) – meteo variable e.g. “RH” or “EV24”. See list with all options in the hydropandas documentation.

• start (str, datetime or None, optional) – start date of observations. The default is None.

• end (str, datetime or None, optional) – end date of observations. The default is None.

• **kwargs –

  fill_missing_obsbool, optional

  if True nan values in time series are filled with nearby time series. The default is False.

  intervalstr, optional

  desired time interval for observations. Options are ‘daily’ and ‘hourly’. The default is ‘daily’.

  use_apibool, optional

  if True the api is used to obtain the data, API documentation is here:

  https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

  if False a text file is downloaded into a temporary folder and the data is read from there. Default is True since the api is back online (July 2021).

  raise_exceptionsbool, optional

  if True you get errors when no data is returned. The default is False.

Returns

• pd.DataFrame, measurements

• dict, metadata

Raises

ValueError – if no meteo_var is given or stn, fname and xy are all None.

hydropandas.io.knmi.get_knmi_obslist(locations=None, stns=None, xy=None, meteo_vars=('RH',), starts=None, ends=None, ObsClasses=None, **kwargs)

Get a list of observations of knmi stations. Either specify a list of knmi stations (stns) or a dataframe with x, y coordinates (locations).

Parameters

• locations (pandas DataFrame or None) – dataframe with x and y coordinates. The default is None

• stns (list of str or None) – list of knmi stations. The default is None

• xy (list or numpy array, optional) – xy coordinates of the locations. e.g. [[10,25], [5,25]]

• meteo_vars (list or tuple of str) – meteo variables e.g. [“RH”, “EV24”]. The default is (“RH”)

• starts (None, str, datetime or list, optional) – start date of observations per meteo variable. The start date is included in the time series. If start is None the start date will be January 1st of the previous year. if start is str it will be converted to datetime if start is a list it should be the same length as meteo_vars and the start time for each variable. The default is None

• ends (list of str, datetime or None) – end date of observations per meteo variable. The end date is included in the time series. If end is None the start date will be January 1st of the previous year. if end is a str it will be converted to datetime if end is a list it should be the same length as meteo_vars and the end time for each meteo variable. The default is None

• ObsClasses (list of type or None) – class of the observations, can be PrecipitationObs or EvaporationObs. The default is None.

• **kwargs –

  fill_missing_obsbool, optional

  if True nan values in time series are filled with nearby time series. The default is False.

  intervalstr, optional

  desired time interval for observations. Options are ‘daily’ and ‘hourly’. The default is ‘daily’.

  use_apibool, optional

  if True the api is used to obtain the data, API documentation is here:

  https://www.knmi.nl/kennis-en-datacentrum/achtergrond/data-ophalen-vanuit-een-script

  if False a text file is downloaded into a temporary folder and the data is read from there. Default is True since the api is back online (July 2021).

  raise_exceptionsbool, optional

  if True you get errors when no data is returned. The default is False.

Returns

obs_list – collection of multiple point observations

Return type

list of obsevation objects

hydropandas.io.knmi.get_knmi_timeseries_fname(fname, meteo_var, settings, start, end)

hydropandas.io.knmi.get_knmi_timeseries_stn(stn, meteo_var, settings, start=None, end=None)

Get a knmi time series and metadata.

Parameters

• stn (int) – measurement station e.g. 829.

• meteo_var (str) – observation type e.g. “RH” or “EV24”. See list with all options in the hpd.read_knmi function.

• settings (dict) – settings for obtaining the right time series, see _get_default_settings for more information

• start (pd.TimeStamp or None, optional) – start date of observations. The default is None.

• end (pd.TimeStamp or None, optional) – end date of observations. The default is None.

Returns

• knmi_df (pandas DataFrame) – time series with measurements.

• meta (dictionary) – metadata from the measurement station.

hydropandas.io.knmi.get_n_nearest_stations_xy(xy, meteo_var, n=1, stations=None, ignore=None)

Find the N nearest KNMI stations that measure variable ‘meteo_var’ to the x, y coordinates.

Parameters

• xy (list, tuple or numpy.array of int or float) – sinlge pair of xy coordinates. e.g. (150_000., 400_000.)

• meteo_var (str) – measurement variable e.g. ‘RH’ or ‘EV24’

• n (int, optional) – number of stations you want to return. The default is 1.

• stations (pandas DataFrame, optional) – if None stations will be obtained using the get_stations function. The default is None.

• ignore (list, optional) – list of stations to ignore. The default is None.

Returns

station numbers.

Return type

list

hydropandas.io.knmi.get_nearest_station_df(locations, xcol='x', ycol='y', stations=None, meteo_var='RH', ignore=None)

Find the KNMI stations that measure ‘meteo_var’ closest to the coordinates in ‘locations’.

Parameters

• locations (pandas.DataFrame) – DataFrame containing x and y coordinates

• xcol (str) – name of the column in the locations dataframe with the x values

• ycol (str) – name of the column in the locations dataframe with the y values

• stations (pandas DataFrame, optional) – if None stations will be obtained using the get_stations function. The default is None.

• meteo_var (str) – measurement variable e.g. ‘RH’ or ‘EV24’

• ignore (list, optional) – list of stations to ignore. The default is None.

Returns

stns – station numbers.

Return type

list

hydropandas.io.knmi.get_nearest_station_xy(xy, stations=None, meteo_var='RH', ignore=None)

find the KNMI stations that measure ‘meteo_var’ closest to the given x and y coordinates.

Parameters

• xy (list or numpy array, optional) – xy coordinates of the locations. e.g. [[10,25], [5,25]]

• stations (pandas DataFrame, optional) – if None stations will be obtained using the get_stations function. The default is None.

• meteo_var (str) – measurement variable e.g. ‘RH’ or ‘EV24’

• ignore (list, optional) – list of stations to ignore. The default is None.

Returns

stns – station numbers.

Return type

list

Notes

assumes you have a structured rectangular grid.

hydropandas.io.knmi.get_station_name(stn, stations=None)

Returns the station name from a KNMI station.

Modifies the station name in such a way that a valid url can be obtained.

Parameters

• stn (int) – station number.

• stations (pandas DataFrame) – DataFrame with the station metadata.

Returns

Name of the station or None if station is not found.

Return type

str or None

hydropandas.io.knmi.get_stations(meteo_var)

get knmi stations from json files according to variable.

Parameters

meteo_var (str, optional) – type of meteodata, by default ‘RH’

Return type

pandas DataFrame with stations, names and coordinates (Lat/Lon & RD)

hydropandas.io.knmi.hargreaves(tmean, tmin, tmax, dates, lat=52.1, x=None)

Estimate of Hargraves potential evaporation according to Allen et al. 1990.

Parameters

• tmean (pandas.Series) – Daily mean temperature

• tmin (pandas.Series) – Daily minimum temperature

• tmax (pandas.Series) – Daily maximum temperature

• dates (pandas.Series.index) – Dates

• lat (float, optional) – Latitude of station, by default 52.1

• x (_type_, optional) – Optional parameter to scale evaporation estimate, by default None

Return type

pandas.Series

hydropandas.io.knmi.makkink(tmean, K)

Estimate of Makkink reference evaporation according to KNMI.

Parameters

• tmean (tmean : pandas.Series) – Daily mean temperature in Celsius

• K (pandas.Series) – Global radiation estimate in J/cm2

Return type

pandas.Series

hydropandas.io.knmi.penman(tmean, tmin, tmax, K, wind, rh, dates, z=1.0, lat=52.1, G=0.0, wh=10.0, tdew=None)

Estimate of Penman reference evaporation according to Allen et al 1990.

Parameters

• tmean (pandas.Series) – Daily mean temperature in Celsius

• tmin (pandas.Series) – Daily minimum temperature in Celsius

• tmax (pandas.Series) – Daily maximum temperature in Celsius

• K (pandas.Series) – Global radiation estimate in J/cm2

• wind (pandas.Series) – Daily mean wind speed in m/s

• rh (pandas.Series) – Relative humidity in %

• dates (pandas.Series) – Dates

• z (float, optional) – Elevation of station in m, by default 1.0

• lat (float, optional) – Latitude of station, by default 52.1

• G (float, optional) – Ground flux in MJ/m2, by default 0.0

• wh (float, optional) – Height of wind measurement in m, by default 10.0

• tdew (pandas.Series) – Dew point temperature in C, by default None

Return type

pandas.Series

hydropandas.io.knmi.read_knmi_daily_meteo(f, meteo_var)

hydropandas.io.knmi.read_knmi_daily_meteo_file(path, meteo_var, start=None, end=None)

read knmi daily meteo data from a file

Parameters

• path (str) – file path of .txt file.

• meteo_var (str) – e.g. ‘EV24’.

• start (pd.TimeStamp or None) – start time of observations.

• end (pd.TimeStamp or None) – end time of observations.

Raises

ValueError – If the meteo var is not in the file.

Returns

• pandas DataFrame – measurements.

• variables (dictionary) – additional information about the variables

hydropandas.io.knmi.read_knmi_daily_rainfall(f, meteo_var)

Read daily rainfall data from a KNMI file.

Parameters

• f (file-like object) – The file object containing the KNMI data.

• meteo_var (str) – The meteorological variable to extract.

Returns

• pandas.DataFrame – The DataFrame containing the extracted daily rainfall data.

• dict – A dictionary containing information about the variables in the DataFrame.

Notes

This function assumes that the file object f is already open and positioned at the start of the data. The file is expected to have a header with variable names and a corresponding data table.

The DataFrame returned by this function has the following modifications: - The index is set to the datetime values derived from the ‘YYYYMMDD’ column. - The ‘YYYYMMDD’ column is dropped. - Duplicate indices are removed, keeping the first occurrence. - If the last row has missing values, it is removed. - The ‘meteo_var’ column is cast to float data type. - Variables are transformed using an internal function _transform_variables. - The unit of measurement for the ‘meteo_var’ variable is set to ‘m’.

hydropandas.io.knmi.read_knmi_daily_rainfall_file(fname_txt, start=None, end=None)

read a knmi file with daily rainfall data.

Parameters

• path (str) – file path of a knmi .txt file.

• start (pd.TimeStamp or None) – start time of observations.

• end (pd.TimeStamp or None) – end time of observations.

Returns

• pandas DataFrame – measurements.

• variables (dictionary) – additional information about the variables

hydropandas.io.knmi.read_knmi_hourly(f, meteo_var, start=None, end=None)

Read hourly KNMI file.

Parameters

f (str or filelike object) – path to file or filelike object

Returns

• df (pd.DataFrame) – DataFrame containing data

• variables (dict) – dictionary containing metadata about the variables

hydropandas.io.menyanthes module

hydropandas.io.menyanthes.matlab2datetime(tindex)

Transform a MATLAB serial date number to a Python datetime object, rounded to seconds.

Parameters

tindex (float) – The MATLAB serial date number to convert.

Returns

datetime – The equivalent datetime object in Python.

Return type

datetime.datetime

Notes

MATLAB serial date numbers represent the number of days elapsed since January 1, 0000 (the proleptic Gregorian calendar), with January 1, 0000 as day 1. Fractions of a day can be represented as a decimal.

The returned datetime object is rounded to the nearest second.

Examples

>>> matlab2datetime(719529.496527778)
datetime.datetime(2019, 1, 1, 11, 55, 2)

hydropandas.io.menyanthes.read_file(path, ObsClass, load_oseries=True, load_stresses=True)

Read data from a Menyanthes file and create observation objects.

Parameters

• path (str) – Full path of the Menyanthes file (.men) to read.

• ObsClass (GroundwaterObs or WaterlvlObs) – Class of observation object to create.

• load_oseries (bool, optional) – Flag indicating whether to load observation series or not, by default True.

• load_stresses (bool, optional) – Flag indicating whether to load stresses or not, by default True.

Returns

obs_list – List of observation objects created from the Menyanthes file.

Return type

list

hydropandas.io.menyanthes.read_oseries(mat)

Read the oseries from a mat file from menyanthes.

Parameters

mat (dict) – A dictionary object containing the Menyanthes file data.

Returns

A dictionary containing oseries data, with oseries names as keys and their corresponding metadata and values as values.

Return type

dict

Notes

This function reads the oseries data from a Menyanthes file in .mat format and returns it in a dictionary format. The oseries data contains the following metadata:

• name: The name of the oseries.

• x: The x-coordinate of the oseries location.

• y: The y-coordinate of the oseries location.

• source: The data source.

• unit: The unit of measurement.

In addition to the metadata, the oseries data also contains a pandas Series object named ‘values’, which contains the time series data for the oseries.

Examples

>>> mat = loadmat('menyanthes_file.mat')
>>> d_h = read_oseries(mat)

hydropandas.io.menyanthes.read_stresses(mat)

Reads the stresses from a mat file from menyanthes.

Parameters

mat (dict) – A dictionary object containing the mat file.

Returns

A dictionary object containing the stresses data.

Return type

dict

hydropandas.io.modflow module

hydropandas.io.modflow.interp_weights(xy, uv, d=2)

Calculate interpolation weights 1.

Parameters

• xy (np.array) – array containing x-coordinates in first column and y-coordinates in second column

• uv (np.array) – array containing coordinates at which interpolation weights should be calculated, x-data in first column and y-data in second column

• d (int, optional) – dimension of data? (the default is 2, which works for 2D data)

Returns

• vertices (np.array) – array containing interpolation vertices

• weights (np.array) – array containing interpolation weights per point

References

1

https://stackoverflow.com/questions/20915502/

speedup-scipy-griddata-for-multiple-interpolations-between-two-irregular-grids

hydropandas.io.modflow.interpolate(values, vtx, wts, fill_value=nan)

Interpolate values at locations defined by vertices and points 2, as calculated by interp_weights function.

Parameters

• values (np.array) – array containing values to interpolate

• vtx (np.array) – array containing interpolation vertices, see interp_weights()

• wts (np.array) – array containing interpolation weights, see interp_weights()

• fill_value (float) – fill value for points that have to be extrapolated (e.g. at or beyond edges of the known points)

Returns

arr – array containing interpolated values at locations as given by vtx and wts

Return type

np.array

References

2

https://stackoverflow.com/questions/20915502/

speedup-scipy-griddata-for-multiple-interpolations-between-two-irregular-grids

hydropandas.io.modflow.read_imod_results(obs_collection, ml, runfile, mtime, model_ws, modelname='', nlay=None, exclude_layers=0)

Read imod model results at point locations.

Parameters

• obs_collection (ObsCollection) – collection of observations at which points imod results will be read

• ml (flopy.modflow.mf.model) – modflow model

• runfile (Runfile) – imod runfile object

• mtime (list of datetimes) – datetimes corresponding to the model periods

• model_ws (str) – model workspace with imod model

• nlay (int, optional) – number of layers if None the number of layers from ml is used.

• modelname (str) – modelname

• exclude_layers (int) – exclude modellayers from being read from imod

hydropandas.io.modflow.read_modflow_results(obs_collection, ml, hds_arr, mtime, modelname='', nlay=None, exclude_layers=None, method='linear')

Read modflow groundwater heads at points in obs_collection.

Parameters

• obs_collection (ObsCollection) – locations of model observation

• ml (flopy.modflow.mf.model) – modflow model

• hds_arr (numpy array) – heads with shape (ntimesteps, nlayers, nrow, ncol)

• mtime (list of datetimes) – dates for each model timestep

• modelname (str, optional) – modelname

• nlay (int, optional) – number of layers if None the number of layers from ml is used.

• exclude_layers (list of int, optional) – exclude the observations in these modellayers

• method (str, optional) – interpolation method, either ‘linear’ or ‘nearest’, default is linear.

hydropandas.io.pastas module

Created on Wed Sep 12 12:15:42 2018.

@author: Artesia

hydropandas.io.pastas.create_pastastore(oc, pstore, pstore_name='', conn=None, add_metadata=True, col=None, kind='oseries', overwrite=False)

add observations to a new or existing pastastore.

Parameters

• oc (observation.ObsCollection) – collection of observations

• pstore (pastastore.PastaStore, optional) – Existing pastastore, if None a new pastastore is created

• pstore_name (str, optional) – Name of the pastastore only used if pstore is None

• conn (pastastore.connectors) – connector for database

• col (str or None, optional) – the column of the obs dataframe to use in pastas. The first numeric column is used if col is None, by default None.

• kind (str, optional) – The kind of series that is added to the pastastore

• add_metadata (boolean, optional) – If True metadata from the observations added to the pastastore

• overwrite (boolean, optional) – if True, overwrite existing series in pastastore, default is False

Returns

pstore – the pastastore with the series from the ObsCollection

Return type

pastastore.PastaStore

hydropandas.io.pastas.read_pastastore_item(pstore, libname, name)

Read item from pastastore library.

Parameters

• pstore (pastastore.PastaStore) – pastastore object

• libname (str) – name of library containing item

• name (str) – name of item

Returns

• series (pd.Series) – time series for item

• meta (dict) – dictionary containing metadata

Raises

ValueError – if library is not oseries or stresses

hydropandas.io.pastas.read_pastastore_library(pstore, libname, ObsClass=<class 'hydropandas.observation.GroundwaterObs'>, metadata_mapping=None)

Read pastastore library.

Parameters

• pstore (pastastore.PastaStore) – pastastore object

• libname (str) – name of library to read

• ObsClass (Obs, optional) – type of Obs to read data as, by default GroundwaterObs

• metadata_mapping (dict, optional) – dictionary containing map between metadata field names in pastastore and metadata field names expected by hydropandas, by default None.

Returns

obs_list – list of Obs containing data

Return type

list of Obs

hydropandas.io.pystore module

hydropandas.io.waterinfo module

hydropandas.io.waterinfo.read_waterinfo_file(path, index_cols=None, return_metadata=False, value_col=None, location_col=None, xcol=None, ycol=None, transform_coords=True)

Read waterinfo file (CSV or zip)

Parameters

path (str) – path to waterinfo file (.zip or .csv)

Returns

• df (pandas.DataFrame) – DataFrame containing file content

• metadata (dict, optional) – dict containing metadata, returned if return_metadata is True, default is False

hydropandas.io.waterinfo.read_waterinfo_obs(file_or_dir, ObsClass, progressbar=False, **kwargs)

Read waterinfo file or directory and extract locations and observations.

Parameters

• file_or_dir (str) – path to file or directory

• ObsClass (Obs type) – type of Obs to store data in

• progressbar (bool, optional) – show progressbar if True, default is False

Returns

obs_collection – list of Obs objects

Return type

list

hydropandas.io.wiski module

hydropandas.io.wiski.read_wiski_dir(dirname, ObsClass=None, suffix='.csv', unpackdir=None, force_unpack=False, preserve_datetime=False, keep_all_obs=True, **kwargs)

Reads WISKI CSV files from a directory and returns a list of observation objects.

Parameters

• dirname (str) – The path of the directory containing the WISKI CSV files.

• ObsClass (object, optional) – The observation class to use for creating observation objects. Default is None.

• suffix (str, optional) – The file extension of the WISKI CSV files. Default is “.csv”.

• unpackdir (str, optional) – The directory to which the files should be unpacked. Default is None.

• force_unpack (bool, optional) – If True, forces the files to be unpacked even if they are already in the target directory. Default is False.

• preserve_datetime (bool, optional) – If True, preserves the original modification times of the files when unpacking them. Default is False.

• keep_all_obs (bool, optional) – If True, keeps all observation objects even if they have no metadata available. Default is True.

• **kwargs – Additional keyword arguments to pass to the from_wiski method of the ObsClass object.

Returns

A list of observation objects created from the WISKI CSV files in the directory.

Return type

list

Raises

FileNotFoundError – If no WISKI CSV files are found in the directory.

hydropandas.io.wiski.read_wiski_file(path, sep=';', header_sep=None, header_identifier='#', read_series=True, translate_dic=None, tz_localize=True, unit='', **kwargs)

Read data from a WISKI file.

Parameters:

pathstr

The path of the file to be read.

sepstr, optional (default=”;”)

The delimiter used to separate fields in the file.

header_sepstr, optional (default=None)

The delimiter used to separate fields in the header. If None, the function will try to automatically detect the separator.

header_identifierstr, optional (default=”#”)

The character used to identify header lines.

read_seriesbool, optional (default=True)

Whether to read the time series data from the file.

translate_dicdict, optional (default=None)

A dictionary mapping header field names to the desired output names.

tz_localizebool, optional (default=True)

Whether to localize the datetime index to the machine’s timezone.

unitstr, optional (default=””)

The unit of measurement of the data.

**kwargskeyword arguments

Additional arguments to pass to the pandas read_csv function.

Returns:

datapandas.DataFrame or None

A dataframe containing the time series data from the file. Returns None if read_series is False.

metadatadict

A dictionary containing metadata about the data in the file.

hydropandas.io.fews module

hydropandas.io.fews.get_fews_pid(name: str) → Dict[str, Obs]

Get matching ParameterId’s and HydroPandas Observation Classes

Parameters

name (str) – Waterboard name

Returns

Dictonary with ParameterId and the resulting Observation Class

Return type

Dict[str, Obs]

hydropandas.io.fews.iterparse_pi_xml(fname: str, ObsClass: Union[Obs, Dict[str, Obs]], translate_dic: Optional[Dict[str, str]] = None, filterdict: Optional[Dict[str, List[str]]] = None, locationIds: Optional[List[str]] = None, return_events: bool = True, keep_flags: Tuple[int] = (0, 1), return_df: bool = False, tags: Tuple[str] = ('series', 'header', 'event'))

Read a FEWS XML-file with measurements, memory efficient.

Parameters

• fname (str) – full path to file

• ObsClass (Union[Obs, Dict[str, Obs]],) – class of the observations, e.g. GroundwaterObs or WaterlvlObs

• translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘monitoring_well’}.

• locationIds (tuple or list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read.

• filterdict (dict, optional) – dictionary with tag name to apply filter to as keys, and list of accepted names as dictionary values to keep in final result, i.e. {“locationId”: [“B001”, “B002”]}

• return_events (bool, optional) – return all event-information in a DataFrame per location, instead of just a Series (defaults to False). Overrules keep_flags kwarg.

• keep_flags (list of ints, optional) – keep the values with these flags (defaults to 0 and 1). Only used when return_events is False.

• tags (list of strings, optional) – Select the tags to be parsed. Defaults to series, header and event

• return_df (bool, optional) – return a DataFame with the data, instead of two lists (default is False)

Returns

• df (pandas.DataFrame) – a DataFrame containing the metadata and the series if ‘return_df’ is True

• obs_list (list of pandas Series) – list of timeseries if ‘return_df’ is False

hydropandas.io.fews.read_xml_filelist(fnames: List[str], ObsClass: Union[Obs, Dict[str, Obs]], directory: Optional[str] = None, locations: Optional[List[str]] = None, translate_dic: Optional[Dict[str, str]] = None, filterdict: Optional[Dict[str, List[str]]] = None, remove_nan: bool = False, low_memory: bool = True, **kwargs: dict)

Read a list of xml files into a list of observation objects.

Parameters

• fnames (TYPE) – DESCRIPTION.

• ObsClass (Union[Obs, Dict[str, Obs]]) – class of the observations, e.g. GroundwaterObs or WaterlvlObs

• directory (TYPE, optional) – DESCRIPTION. The default is None.

• locations (tuple or list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read.

• translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘monitoring_well’}.

• filterdict (dict, optional) – dictionary with tag name to apply filter to as keys, and list of accepted names as dictionary values to keep in final result, i.e. {“locationId”: [“B001”, “B002”]}

• remove_nan (boolean, optional) – remove nan values from measurements, flag information about the nan values is also lost, only used if low_memory=False

• low_memory (bool, optional) – whether to use xml-parsing method with lower memory footprint, default is True

Returns

list of timeseries stored in ObsClass objects

Return type

list of ObsClass objects

hydropandas.io.fews.read_xml_fname(fname: str, ObsClass: Union[Obs, Dict[str, Obs]], translate_dic: Optional[Dict[str, str]] = None, low_memory: bool = True, locationIds: Optional[List[str]] = None, filterdict: Optional[Dict[str, List[str]]] = None, return_events: bool = True, keep_flags: Tuple[int] = (0, 1), return_df: bool = False, tags: Tuple[str] = ('series', 'header', 'event'), remove_nan: bool = False, **kwargs: dict)

Read an xml filename into a list of observations objects.

Parameters

• fname (str) – full path to file

• ObsClass (Union[Obs, Dict[str, Obs]]) – class of the observations, e.g. GroundwaterObs or WaterlvlObs

• translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘monitoring_well’}.

• low_memory (bool, optional) – whether to use xml-parsing method with lower memory footprint, default is True

• locationIds (tuple or list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read.

• filterdict (dict, optional) – dictionary with tag name to apply filter to as keys, and list of accepted names as dictionary values to keep in final result, i.e. {“locationId”: [“B001”, “B002”]}

• return_events (bool, optional) – return all event-information in a DataFrame per location, instead of just a Series (defaults to False). Overrules keep_flags kwarg.

• keep_flags (list of ints, optional) – keep the values with these flags (defaults to 0 and 1). Only used when return_events is False.

• tags (list of strings, optional) – Select the tags to be parsed. Defaults to series, header and event

• return_df (bool, optional) – return a DataFame with the data, instead of two lists (default is False)

• remove_nan (boolean, optional) – remove nan values from measurements, flag information about the nan values is also lost, only used if low_memory=False

Returns

list of timeseries stored in ObsClass objects

Return type

list of ObsClass objects

hydropandas.io.fews.read_xml_root(root: Element, ObsClass: Union[Obs, Dict[str, Obs]], translate_dic: Dict[str, str] = None, locationIds: List[str] = None, remove_nan: bool = False)

Read a FEWS XML-file with measurements, return list of ObsClass objects.

Parameters

• root (xml.etree.ElementTree.Element) – root element of a fews xml

• ObsClass (Union[Obs, Dict[str, Obs]],) – class of the observations, e.g. GroundwaterObs or WaterlvlObs

• translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘monitoring_well’}.

• locationIds (tuple or list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read.

• remove_nan (boolean, optional) – remove nan values from measurements, flag information about the nan values is also lost

Returns

list of timeseries stored in ObsClass objects

Return type

list of ObsClass objects

hydropandas.io.fews.read_xmlstring(xmlstring: str, ObsClass: Union[Obs, Dict[str, Obs]], translate_dic: Optional[Dict[str, str]] = None, filterdict: Optional[Dict[str, List[str]]] = None, locationIds: Optional[List[str]] = None, low_memory: bool = True, remove_nan: bool = False)

Read xmlstring into an list of Obs objects. Xmlstrings are usually obtained using a fews api.

Parameters

• xmlstring (str) – xml string to be parsed. Typically from a fews api.

• ObsClass (Union[Obs, Dict[str, Obs]]) – class of the observations, e.g. GroundwaterObs or WaterlvlObs

• translate_dic (dic or None, optional) – translate names from fews. If None this default dictionary is used: {‘locationId’: ‘monitoring_well’}.

• locationIds (tuple or list of str, optional) – list of locationId’s to read from XML file, others are skipped. If None (default) all locations are read.

• low_memory (bool, optional) – whether to use xml-parsing method with lower memory footprint, default is True

• remove_nan (boolean, optional) – remove nan values from measurements, flag information about the nan values is also lost, only used if low_memory=False

Returns

list of timeseries stored in ObsClass objects

Return type

list of ObsClass objects

hydropandas.io.fews.write_pi_xml(obs_coll, fname: str, timezone: float = 1.0, version: str = '1.24')

Write TimeSeries object to PI-XML file.

Parameters

fname (path) – path to XML file