Exploring groundwater data from Lizard
In this notebook, you will experiment how to use the hydropandas package to access, visualize and explore meta data from Lizard, a cloud datawarehouse that can be used to store groundwater observations. Data sources:
Vitens (default, public access): Vitens is the largest drinking water company in the Netherlands, and it has more than 10.000 groundwater wells and more than 50.000 timeseries in its datawarehouse. The data spans from the 1930’s to the present, and it is constantly updated with new observations. Vitens also validates the data using ArtDiver and provides quality flags and comments for each observation. The data is open to the public and you can find more information at https://vitens.lizard.net.
Others: If you have access to other Lizard data sources, you can specify the
organisationparameter in thelizardfunctions to access those data sources. Currently, only “vitens” is officially supported. Note that you then may need to specify authentication as well by means of a personal API key. This has been implemented for the municipality of Rotterdam.
Feel free to customize and expand upon this introduction as needed.
Notebook contents
Read groundwater timeseries
Analyse groundwater observations
Build a Pastas model
[1]:
import pastas as ps
import hydropandas as hpd
/home/docs/checkouts/readthedocs.org/user_builds/hydropandas/envs/stable/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
[2]:
hpd.util.get_color_logger("INFO")
[2]:
<RootLogger root (INFO)>
1. Read groundwater timeseries
1.1 Authentication
The vitens data source is used by default, and does not require authentication. If you have access to other Lizard data sources, you can specify the organisation parameter in the read_lizard function to access those data sources. Currently, only “vitens” is officially supported.
For others than Vitens, you may need authentication. You can use the auth parameter to pass your credentials. According to the Lizard documentation, you should create a personal API key at https://{organisation}.lizard.net/management/ and enter that in the cell below. If you do not have access to other data sources, you can use auth=None to skip authentication.
[3]:
## Lizard uses the organisation parameter to specify the data source.
organisation = "vitens" # or: "rotterdam"
auth = None
## UNCOMMENT AND SET YOUR API KEY IF YOU NEED TO SPECIFY CREDENTIALS
# your_api_key = "your_api_key_here"
# auth = ("__key__", your_api_key)
Which timeseries to include? (which_timeseries)
In the Lizard database, the follow WNS codes are used:
WNS9040.hand: Hand measurements (in Hydropandas: “hand”)
WNS9040: Diver measurements (in Hydropandas: “diver”)
WNS9040.val: Validated diver measurements (in Hydropandas: “diver_validated”)
Use of WNS codes per organisation:
Vitens uses only categories 1 and 2. So for consistency with previous versions, Hydropandas will use “hand” and “diver” measurements when
which_timeseriesis not specified.Rotterdam uses all three categories. So for Rotterdam you can use
which_timeseries=["hand", "diver", "diver_validated"]to get both the validated and unvalidated diver measurements.
1.1 Get observations from extent
Use read_lizard to find monitoring wells by specifying a geographical extent in Rijksdriehoeks coordinates.
[4]:
my_extent = (137_000, 138_000, 458_000, 459_000)
oc = hpd.read_lizard(
extent=my_extent,
which_timeseries=["hand", "diver", "diver_validated"],
datafilters=None,
combine_method="merge",
organisation=organisation,
auth=auth,
)
INFO:hydropandas.io.lizard.get_obs_list_from_extent:Number of monitoring wells: 1
INFO:hydropandas.io.lizard.get_obs_list_from_extent:Number of pages: 1
Page: 100%|██████████| 1/1 [00:00<00:00, 1.92it/s]
monitoring well: 0%| | 0/1 [00:00<?, ?it/s]
INFO:hydropandas.io.lizard.get_timeseries_uuid:Successfully retrieved 849 timeseries events for UUID cd5bb67c-4b74-4c8a-8a59-511d2e89a333
INFO:hydropandas.io.lizard.get_timeseries_uuid:Successfully retrieved 35889 timeseries events for UUID 51219e93-8fc7-4bdd-8d1b-bbbd6ef4b18b
INFO:hydropandas.io.lizard.get_timeseries_uuid:Successfully retrieved 679 timeseries events for UUID d9441bc1-8909-48b6-bf5f-94706535a321
INFO:hydropandas.io.lizard.get_timeseries_uuid:Successfully retrieved 679 timeseries events for UUID 55e5394d-41f2-48d8-b139-a24887b9e370
monitoring well: 100%|██████████| 1/1 [00:09<00:00, 9.12s/it]
Visualize all groundwater wells inside the extent on a map (visualize the ObsCollection). The markers are clickable to show a preview of the availables observations.
[5]:
oc.plots.interactive_map(
color="red", zoom_start=15, tiles="Esri.WorldImagery", popup_width=350
)
[5]:
Print all the retrieved groundwater wells and tubes, and make a plot of the observations.
[6]:
oc
[6]:
| location | x | tube_nr | screen_bottom | ground_level | filename | tube_top | source | unit | y | screen_top | obs | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| name | ||||||||||||
| UPWP016001 | B31H0580 | 137401.640072 | 1 | -24.43 | 1.58 | 2.198 | lizard | m NAP | 458893.680615 | -22.43 | GroundwaterObs UPWP016001 -----metadata------ ... | |
| UPWP016003 | B31H0580 | 137401.640072 | 3 | -67.43 | 1.58 | 2.141 | lizard | m NAP | 458893.680615 | -65.43 | GroundwaterObs UPWP016003 -----metadata------ ... | |
| UPWP016002 | B31H0580 | 137401.640072 | 2 | -55.93 | 1.58 | 2.178 | lizard | m NAP | 458893.680615 | -53.93 | GroundwaterObs UPWP016002 -----metadata------ ... |
[7]:
oc.obs.values[0]
[7]:
hydropandas.GroundwaterObs
| UPWP016001 | |
|---|---|
| location | B31H0580 |
| x | 137401.640072 |
| tube_nr | 1 |
| screen_bottom | -24.43 |
| crs | EPSG:28992 |
| ground_level | 1.58 |
| filename | |
| tube_top | 2.198 |
| source | lizard |
| unit | m NAP |
| y | 458893.680615 |
| screen_top | -22.43 |
| value | flag | comment | origin | |
|---|---|---|---|---|
| peil_datum_tijd | ||||
| 1976-04-15 12:00:00 | -0.300 | 0 | hand | |
| 1976-04-28 12:00:00 | -0.260 | 0 | hand | |
| 1976-05-18 12:00:00 | -0.330 | 0 | hand | |
| 1976-06-02 12:00:00 | -0.230 | 0 | hand | |
| 1976-06-17 12:00:00 | -0.300 | 0 | hand | |
| ... | ... | ... | ... | ... |
| 2026-07-20 03:00:00 | -0.177 | 0 | diver | |
| 2026-07-20 06:00:00 | -0.178 | 0 | diver | |
| 2026-07-20 09:00:00 | -0.179 | 0 | diver | |
| 2026-07-20 12:00:00 | -0.183 | 0 | diver | |
| 2026-07-20 12:12:02 | -0.192 | 0 | hand |
36738 rows × 4 columns
[8]:
oc.plots.section_plot(plot_obs=True)
INFO:hydropandas.extensions.plots.section_plot:created sectionplot -> UPWP016001
INFO:hydropandas.extensions.plots.section_plot:created sectionplot -> UPWP016003
INFO:hydropandas.extensions.plots.section_plot:created sectionplot -> UPWP016002
[8]:
(<Figure size 1500x500 with 2 Axes>,
[<Axes: ylabel='m NAP'>, <Axes: ylabel='m NAP'>])
2. Analyse Groundwater observations
Now lets download the groundwater level observation using the from_lizard function of a GroundwaterObs object. The code below reads the groundwater level timeseries for the well UPWP016 from Lizard and makes a plot.
[9]:
gw_lizard = hpd.GroundwaterObs.from_lizard(
"UPWP016",
tmin="1900-01-01",
tmax="2030-01-01",
organisation="vitens",
auth=auth,
)
print(gw_lizard)
ax = gw_lizard["value"].plot(
figsize=(12, 5),
marker=".",
grid=True,
label=gw_lizard.name,
legend=True,
xlabel="Date",
ylabel="m NAP",
title="Groundwater observations for " + gw_lizard.name,
)
INFO:hydropandas.io.lizard.get_timeseries_uuid:Successfully retrieved 849 timeseries events for UUID cd5bb67c-4b74-4c8a-8a59-511d2e89a333
INFO:hydropandas.io.lizard.get_timeseries_uuid:Successfully retrieved 35889 timeseries events for UUID 51219e93-8fc7-4bdd-8d1b-bbbd6ef4b18b
GroundwaterObs UPWP016001
-----metadata------
location : B31H0580
x : 137401.64007157044
tube_nr : 1
screen_bottom : -24.43
crs : EPSG:28992
ground_level : 1.58
filename :
tube_top : 2.198
source : lizard
name : UPWP016001
unit : m NAP
y : 458893.68061490165
screen_top : -22.43
-----time series------
value flag comment origin
peil_datum_tijd
1976-04-15 12:00:00 -0.300 0 hand
1976-04-28 12:00:00 -0.260 0 hand
1976-05-18 12:00:00 -0.330 0 hand
1976-06-02 12:00:00 -0.230 0 hand
1976-06-17 12:00:00 -0.300 0 hand
... ... ... ... ...
2026-07-20 03:00:00 -0.177 0 diver
2026-07-20 06:00:00 -0.178 0 diver
2026-07-20 09:00:00 -0.179 0 diver
2026-07-20 12:00:00 -0.183 0 diver
2026-07-20 12:12:02 -0.192 0 hand
[36738 rows x 4 columns]
The groundwater observations contain a validation flag per timestamp. These can ‘betrouwbaar’ (reliable), ‘onbetrouwbaar’ (unreliable) en ‘onbeslist’ (unvalidated). Below flags of the timeseries are shown as a percentage, and the unreliable timestamps are printed.
[10]:
print(gw_lizard["flag"].value_counts(normalize=True) * 100)
gw_lizard[gw_lizard["flag"] == "onbetrouwbaar"]
flag
0 99.961892
1 0.016332
7 0.013610
6 0.005444
3 0.002722
Name: proportion, dtype: float64
[10]:
hydropandas.GroundwaterObs
| UPWP016001 | |
|---|---|
| location | B31H0580 |
| x | 137401.640072 |
| tube_nr | 1 |
| screen_bottom | -24.43 |
| crs | EPSG:28992 |
| ground_level | 1.58 |
| filename | |
| tube_top | 2.198 |
| source | lizard |
| unit | m NAP |
| y | 458893.680615 |
| screen_top | -22.43 |
| value | flag | comment | origin | |
|---|---|---|---|---|
| peil_datum_tijd |
3. Create a Pastas model
Lets make a Pastas model for this groundwater well (starting from 2015) and use the nearest KNMI station for meteorological data
[11]:
# Get the precipitation and evaporation data from the KNMI
precipitation = hpd.PrecipitationObs.from_knmi(
xy=(gw_lizard.x, gw_lizard.y),
start=gw_lizard.index[0],
end=gw_lizard.index[-1],
fill_missing_obs=True,
)
evaporation = hpd.EvaporationObs.from_knmi(
xy=(gw_lizard.x, gw_lizard.y),
meteo_var="EV24",
start=gw_lizard.index[0],
end=gw_lizard.index[-1],
fill_missing_obs=True,
)
# Create a Pastas Model
ml = ps.Model(
gw_lizard.loc[gw_lizard["origin"] == "diver", "value"], name=gw_lizard.name
)
# Add the recharge data as explanatory variable
ts1 = ps.RechargeModel(
precipitation["RH"].resample("1D").first(),
evaporation["EV24"].resample("1D").first(),
ps.Gamma(),
name="rainevap",
settings=("prec", "evap"),
)
# Add the stressmodel to the model and solve for period after 2015
ml.add_stressmodel(ts1)
ml.solve(tmin="2015")
ml.plots.results(figsize=(10, 6))
INFO:hydropandas.io.knmi.get_knmi_obs:get KNMI data from station nearest to coordinates (137401.64007157044, 458893.68061490165) and meteovariable RH
INFO:hydropandas.io.knmi.fill_missing_measurements:knmi station De Bilt has no measurements for RH after 2026-07-20 01:00:00 and an end date of 2026-07-20 12:12:02 was requested. Changing end to 2026-07-20 01:00:00
INFO:hydropandas.io.knmi._add_missing_indices:station 260 has no measurements after 2026-07-20 01:00:00
INFO:hydropandas.io.knmi.get_knmi_obs:get KNMI data from station nearest to coordinates (137401.64007157044, 458893.68061490165) and meteovariable EV24
INFO:hydropandas.io.knmi.fill_missing_measurements:knmi station De Bilt has no measurements for EV24 after 2026-07-20 01:00:00 and an end date of 2026-07-20 12:12:02 was requested. Changing end to 2026-07-20 01:00:00
INFO:hydropandas.io.knmi._add_missing_indices:station 260 has no measurements after 2026-07-20 01:00:00
WARNING:pastas.timeseries._validate_series:The series 'value' has nan-values. Pastas will use the `fill_nan` from the StressModel's settings (ps.timeseries.settings) parsed to the TimeSeries settings to fill up the nan-values.
INFO:pastas.timeseries._fill_nan:Time Series 'value': 6 nan-value(s) was/were found and filled with: drop.
WARNING:py.warnings._showwarnmsg:/home/docs/checkouts/readthedocs.org/user_builds/hydropandas/envs/stable/lib/python3.12/site-packages/pastas/decorators.py:278: FutureWarning: From Pastas 2.4, the first argument of RechargeModel needs to be a Pastas Model. Please provide the model as the first argument: RechargeModel(model=ml, ...).
warn(message=msg % (self._name, self._name), category=FutureWarning)
WARNING:py.warnings._showwarnmsg:/home/docs/checkouts/readthedocs.org/user_builds/hydropandas/envs/stable/lib/python3.12/site-packages/pastas/decorators.py:71: FutureWarning: add_stressmodel is deprecated and will not be available from Pastas version >= 2.4.0. Stressmodels are now added by adding the Pastas Model as the first argument during stressmodel initialization (i.e., ps.StressModel(model=ml, *args))
warn(message=msg, category=FutureWarning)
INFO:pastas.timeseries._fill_nan:Time Series 'value': 4 nan-value(s) was/were found and filled with: drop.
INFO:pastas.timeseries._fill_after:Time Series 'RH' was extended in the future to 2026-07-20 00:00:00 with the mean value (0.0024) of the time series.
INFO:pastas.timeseries._fill_after:Time Series 'EV24' was extended in the future to 2026-07-20 00:00:00 with the mean value (0.0017) of the time series.
Fit report UPWP016001 Fit Statistics
===================================================
nfev 12 EVP 61.09
nobs 4219 R2 0.61
noise False RMSE 0.03
tmin 2015-01-01 00:00:00 AICc -29126.55
tmax 2026-07-20 12:00:00 BIC -29094.82
freq D Obj 2.11
freq_obs None ___
warmup 3650 days 00:00:00 Interp. No
Parameters (5 optimized)
===================================================
optimal initial vary
rainevap_A 2.835916e+01 198.480743 True
rainevap_n 9.545693e-01 1.000000 True
rainevap_a 1.264038e+01 10.000000 True
rainevap_f -4.898852e-16 -1.000000 True
constant_d -2.016581e-01 -0.130120 True
Warnings! (1)
===================================================
Parameter 'rainevap_f' on upper bound: 0.00e+00
[11]:
[<Axes: label='sim', ylabel='Head'>,
<Axes: label='tab', title={'left': 'Model parameters ($N_c$=5)'}>,
<Axes: label='res', ylabel='Error'>,
<Axes: label='con_rainevap', title={'right': 'Stresses: [RH, EV24]'}, ylabel='Rise'>,
<Axes: label='rf_rainevap', ylabel='[unit head]/\n[unit stress]'>]
