Devereux Slough: breach and slough-rise regressions

Which inputs predict a breach, and how the slough responds to rain

Published

November 20, 2026

Code
import sys, itertools, warnings, json
from pathlib import Path
import numpy as np, pandas as pd
import matplotlib.pyplot as plt
import statsmodels.api as sm
import statsmodels.formula.api as smf
from statsmodels.stats.outliers_influence import variance_inflation_factor
from statsmodels.stats.diagnostic import het_breuschpagan, het_white, linear_reset, acorr_ljungbox
from statsmodels.stats.stattools import durbin_watson
from scipy import stats
from scipy.special import expit
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score, average_precision_score, brier_score_loss, log_loss, roc_curve
from IPython.display import Markdown, display

ROOT = Path.cwd().parent if Path.cwd().name == "analysis" else Path.cwd()
sys.path.insert(0, str(ROOT / "src"))
warnings.filterwarnings("ignore")
from model import load, WaterBalance
from unified import stage_from_readings, stage_from_measured

plt.rcParams.update({"axes.spines.top": False, "axes.spines.right": False, "figure.dpi": 110,
                     "axes.prop_cycle": plt.cycler(color=["#2D5A7B", "#D3A43B", "#5E6E3A", "#A65A3C", "#6B4A73", "#7FA3BD"])})
md = lambda s: display(Markdown(s))

d = load(ROOT / "data" / "processed" / "daily.csv").loc["2018-10-01":"2026-09-05"]
wb = WaterBalance(np.load(ROOT / "models" / "water_balance.npy"))
d["stage_est"], _ = stage_from_readings(d, wb)
d["stage_est"] = stage_from_measured(d, wb).combine_first(d.stage_est)

Data

The unit of analysis for breaches is a closed-mouth day, WY2019 to WY2026: every day the mouth was closed at the start of the day, ending with the breach day itself. The outcome is whether the berm breached that day. All inputs are known by the end of that day, as they would be in a forecast. Breach days are dated by when the slough began to drain in the sub-daily water-level record, not the day CCBER logged the open mouth (src/breach_timing.py): three breaches that drained overnight after a rain day, or with no logger record and rain only the day before, count on that rain day.

Input Meaning
rain_in rain that day (in, Coal Oil Point gauge scale)
rain_3d rain over the last 3 days
api antecedent precipitation index (daily decay 0.9): how wet the watershed already is
rain_wy rain since October 1
log_dc log of days since the mouth closed (berm age)
stage slough level (ft NAVD88): yesterday’s measured level rolled forward with today’s rain
tide_max highest predicted tide that day (ft NAVD88)
surge largest observed-minus-predicted ocean level that day (ft): storm surge
hs_max largest significant wave height at the Harvest buoy (m)
wave30 30-day mean wave power (H²T)
Code
risk = d[d.at_risk & d.dc.notna()].copy()
risk["rain_in"] = risk.rain_in.fillna(0)
risk["log_dc"] = np.log(risk.dc)
risk = risk.rename(columns={"stage_est": "stage", "tide_pred_max_ft": "tide_max", "tide_resid_max_ft": "surge",
                            "hs_harvest_max_m": "hs_max", "wavepow_harvest_30d": "wave30"})
CANDS = ["rain_in", "rain_3d", "api", "rain_wy", "log_dc", "stage", "tide_max", "surge", "hs_max", "wave30"]
for c in CANDS:  # a few wave/tide gaps: fill with the median
    risk[c] = risk[c].fillna(risk[c].median())
y = risk.breach.values
n_ev, n_days = int(y.sum()), len(risk)
md(f"**{n_days:,} closed-mouth days and {n_ev} breaches** (base rate {n_ev / n_days:.2%} per day) "
   f"across {risk.water_year.nunique()} water years.")
risk.groupby("breach")[CANDS].mean().T.rename(columns={0: "no breach (mean)", 1: "breach day (mean)"}).round(2)

2,554 closed-mouth days and 17 breaches (base rate 0.67% per day) across 8 water years.

breach no breach (mean) breach day (mean)
rain_in 0.03 1.55
rain_3d 0.07 2.52
api 0.26 3.10
rain_wy 12.04 13.25
log_dc 4.60 3.85
stage 5.62 8.03
tide_max 5.26 5.02
surge 0.38 0.76
hs_max 2.61 3.30
wave30 60.79 99.56

Part 1. Which inputs predict a breach?

Before fitting: sample size, collinearity, separation

Events per variable. With only 14 breaches, the usual guidance for logistic regression (5 to 10 events per estimated coefficient) supports about 2 to 3 predictors. Larger models overfit. So the search below caps models at 4 predictors and ranks them by leave-one-water-year-out cross-validation rather than in-sample fit.

Code
X = risk[CANDS]
corr = X.corr()
fig, ax = plt.subplots(figsize=(7, 5.5))
im = ax.imshow(corr, cmap="RdBu_r", vmin=-1, vmax=1)
ax.set_xticks(range(len(CANDS)), CANDS, rotation=45, ha="right"); ax.set_yticks(range(len(CANDS)), CANDS)
for i in range(len(CANDS)):
    for j in range(len(CANDS)):
        ax.text(j, i, f"{corr.iloc[i, j]:.2f}", ha="center", va="center", fontsize=7)
fig.colorbar(im, ax=ax, shrink=0.8); ax.set_title("Correlation between candidate inputs")
plt.tight_layout(); plt.show()

Xc = sm.add_constant((X - X.mean()) / X.std())
vif = pd.Series([variance_inflation_factor(Xc.values, i) for i in range(1, Xc.shape[1])], index=CANDS, name="VIF")
vif.round(2).to_frame().T

rain_in rain_3d api rain_wy log_dc stage tide_max surge hs_max wave30
VIF 2.7 3.86 2.84 1.65 1.5 1.79 1.07 1.16 1.36 2.07
Code
sep = []
for c in CANDS:
    lo_b, hi_b = risk.loc[y == 1, c].min(), risk.loc[y == 1, c].max()
    sep.append(dict(input=c, breach_min=lo_b, breach_max=hi_b,
                    nonbreach_days_in_breach_range=int(((risk[c] >= lo_b) & (risk[c] <= hi_b) & (y == 0)).sum())))
sep = pd.DataFrame(sep).round(2)
md("**Separation.** No single input splits breach days from the rest: each breach-day range overlaps "
   "many non-breach days (table), so maximum-likelihood estimates exist. "
   f"**Collinearity.** The highest VIF across all ten inputs is {vif.max():.1f} (`{vif.idxmax()}`); "
   "VIFs above about 5 flag redundancy, so pairs of strongly correlated inputs (the rain measures) "
   "share information and should not both be read as independent effects.")
sep

Separation. No single input splits breach days from the rest: each breach-day range overlaps many non-breach days (table), so maximum-likelihood estimates exist. Collinearity. The highest VIF across all ten inputs is 3.9 (rain_3d); VIFs above about 5 flag redundancy, so pairs of strongly correlated inputs (the rain measures) share information and should not both be read as independent effects.

input breach_min breach_max nonbreach_days_in_breach_range
0 rain_in 0.13 3.38 100
1 rain_3d 0.75 4.96 80
2 api 1.57 4.75 98
3 rain_wy 5.78 25.17 1417
4 log_dc 1.10 5.89 2520
5 stage 5.28 10.30 1467
6 tide_max 4.01 5.96 2059
7 surge 0.40 1.37 1024
8 hs_max 1.85 6.12 1951
9 wave30 44.00 181.08 1639

One input at a time

Code
def fit_logit(df, cols):
    Xm = sm.add_constant((df[cols] - df[cols].mean()) / df[cols].std())
    try:
        return sm.Logit(df.breach.values, Xm).fit(disp=0, maxiter=200)
    except np.linalg.LinAlgError:  # near-separable: a vanishingly small ridge keeps it finite
        return sm.Logit(df.breach.values, Xm).fit_regularized(alpha=1e-4, disp=0, maxiter=500)

null = sm.Logit(y, np.ones((n_days, 1))).fit(disp=0)
rows = []
for c in CANDS:
    m = fit_logit(risk, [c])
    lr = 2 * (m.llf - null.llf)
    rows.append(dict(input=c, coef_per_SD=m.params[c], odds_ratio_per_SD=np.exp(m.params[c]),
                     LR_chi2=lr, p_value=stats.chi2.sf(lr, 1), AUC=roc_auc_score(y, m.predict())))
uni = pd.DataFrame(rows).sort_values("LR_chi2", ascending=False)
uni.round({"coef_per_SD": 2, "odds_ratio_per_SD": 2, "LR_chi2": 1, "p_value": 4, "AUC": 3})
input coef_per_SD odds_ratio_per_SD LR_chi2 p_value AUC
1 rain_3d 0.90 2.46 116.3 0.0000 0.994
2 api 1.13 3.10 112.0 0.0000 0.990
0 rain_in 0.71 2.04 104.2 0.0000 0.992
5 stage 1.12 3.06 42.3 0.0000 0.894
7 surge 1.11 3.04 40.6 0.0000 0.894
9 wave30 0.92 2.51 22.1 0.0000 0.801
8 hs_max 0.50 1.65 6.6 0.0102 0.690
4 log_dc -0.51 0.60 6.5 0.0106 0.557
6 tide_max -0.36 0.70 2.1 0.1493 0.602
3 rain_wy 0.12 1.13 0.3 0.6081 0.534

Every combination of up to four inputs

Each model is fit eight times, leaving out one water year each time and predicting it. Scores use only those held-out predictions. A breach counts as caught when the predicted chance reaches 50% on the breach day or the day before; a false-alarm day is a day at 50% or more that is more than 2 days from any breach.

Code
best_k = res.loc[res.groupby("k").cv_logloss.idxmin()].sort_values("k")
md("**Best model of each size.** Gains should flatten once the events-per-variable limit is reached.")
best_k.round({"AIC": 1, "BIC": 1, "cv_logloss": 4, "cv_brier": 5, "cv_AUC": 3, "cv_PR_AUC": 3})

Best model of each size. Gains should flatten once the events-per-variable limit is reached.

inputs k AIC BIC cv_logloss cv_brier cv_AUC cv_PR_AUC caught_p50 false_alarm_days_p50
124 rain_3d 1 92.0 103.7 0.0187 0.00492 0.994 0.531 5 3
22 rain_3d + log_dc 2 82.4 100.0 0.0165 0.00432 0.993 0.603 9 4
3 rain_in + log_dc + stage 3 70.8 94.2 0.0145 0.00314 0.985 0.657 12 2
0 rain_in + rain_3d + log_dc + stage 4 64.6 93.9 0.0141 0.00350 0.990 0.668 10 4
Code
top25 = res.head(25)
freq = pd.Series({c: top25.inputs.str.split(" \\+ ").apply(lambda L: c in L).mean() for c in CANDS}).sort_values(ascending=False)
md("**How often each input appears among the 25 best models.** A robust input shows up regardless of "
   "what else is in the model.")
freq.round(2).to_frame("share of top 25").T

How often each input appears among the 25 best models. A robust input shows up regardless of what else is in the model.

stage rain_in log_dc rain_3d api tide_max hs_max rain_wy surge wave30
share of top 25 0.8 0.64 0.64 0.56 0.2 0.2 0.2 0.16 0.16 0.12
Code
U = pd.read_csv(ROOT / "data" / "processed" / "predictions_unified_measured.csv", parse_dates=["date"]).set_index("date")
pu = U.p_unified_fb.reindex(risk.index).fillna(U.p_rain.reindex(risk.index)).values
top = res.iloc[0]
cmp = pd.DataFrame([
    dict(model="best linear subset: " + top.inputs, **score(cv_predict(top.inputs.split(" + ")))),
    dict(model="website model: crest curve + rain (nonlinear)", **score(pu)),
    dict(model="rain only: rain_in + rain_3d + api + log_dc + rain_wy", **score(cv_predict(["rain_in", "rain_3d", "api", "log_dc", "rain_wy"]))),
    dict(model="constant rate", **clim)])
md("**Against the crest-curve model the website used at this stage** (crest curve fit jointly with rain, `src/unified.py`; the model in use now is in Part 7), on the same held-out years and the same level estimate:")
cmp.round({"cv_logloss": 4, "cv_brier": 5, "cv_AUC": 3, "cv_PR_AUC": 3})

Against the crest-curve model the website used at this stage (crest curve fit jointly with rain, src/unified.py; the model in use now is in Part 7), on the same held-out years and the same level estimate:

model cv_logloss cv_brier cv_AUC cv_PR_AUC caught_p50 false_alarm_days_p50
0 best linear subset: rain_in + rain_3d + log_dc... 0.0141 0.00350 0.990 0.668 10 4
1 website model: crest curve + rain (nonlinear) 0.0233 0.00434 0.983 0.552 12 8
2 rain only: rain_in + rain_3d + api + log_dc + ... 0.0215 0.00609 0.993 0.505 7 10
3 constant rate 0.0400 0.00661 0.500 0.007 0 0

The selected linear model, examined

Code
SEL = top.inputs.split(" + ")
msel = fit_logit(risk, SEL)
Xsel = pd.DataFrame(msel.model.exog, columns=msel.model.exog_names)
mclu = sm.Logit(y, Xsel).fit(disp=0, cov_type="cluster", cov_kwds={"groups": risk.water_year.values})

def firth(Xm, yv, iters=200):
    """Firth's penalized likelihood: reduces small-sample bias when events are few."""
    b = np.zeros(Xm.shape[1])
    for _ in range(iters):
        p = expit(Xm @ b); W = p * (1 - p)
        inv = np.linalg.inv(Xm.T @ (Xm * W[:, None]))
        Xw = Xm * np.sqrt(W)[:, None]
        h = np.einsum("ij,jk,ik->i", Xw, inv, Xw)
        step = inv @ (Xm.T @ (yv - p + h * (0.5 - p)))
        b += step
        if np.abs(step).max() < 1e-9: break
    return b

bf = pd.Series(firth(msel.model.exog, y), index=msel.model.exog_names)
tab = pd.DataFrame({"coef (per SD)": msel.params, "SE": msel.bse, "SE clustered by water year": mclu.bse,
                    "p (clustered)": mclu.pvalues, "odds ratio": np.exp(msel.params),
                    "OR 95% CI low": np.exp(mclu.conf_int()[0]), "OR 95% CI high": np.exp(mclu.conf_int()[1]),
                    "Firth coef": bf})
md(f"**{top.inputs}.** Coefficients are per standard deviation of each input. Standard errors are "
   "also shown clustered by water year, because days in the same season are not independent; the "
   "Firth (bias-reduced) estimates check for small-sample bias. Close Firth and ordinary estimates "
   "mean the 14 events are enough for this model.")
tab.round(3)

rain_in + rain_3d + log_dc + stage. Coefficients are per standard deviation of each input. Standard errors are also shown clustered by water year, because days in the same season are not independent; the Firth (bias-reduced) estimates check for small-sample bias. Close Firth and ordinary estimates mean the 14 events are enough for this model.

coef (per SD) SE SE clustered by water year p (clustered) odds ratio OR 95% CI low OR 95% CI high Firth coef
const -9.459 1.295 0.997 0.000 0.000 0.000 0.001 -8.714
rain_in 0.299 0.134 0.068 0.000 1.348 1.179 1.542 0.266
rain_3d 0.507 0.167 0.102 0.000 1.660 1.359 2.028 0.482
log_dc -1.450 0.369 0.503 0.004 0.235 0.087 0.630 -1.301
stage 1.805 0.494 0.387 0.000 6.079 2.845 12.991 1.639
Code
rows = []
for c in SEL:
    rest = [x for x in SEL if x != c]
    m0 = fit_logit(risk, rest) if rest else null
    lr = 2 * (msel.llf - m0.llf)
    rows.append(dict(dropped=c, LR_chi2=lr, df=1, p_value=stats.chi2.sf(lr, 1), AIC_increase=m0.aic - msel.aic))
md("**Likelihood-ratio tests:** does each input earn its place?")
pd.DataFrame(rows).round(4)

Likelihood-ratio tests: does each input earn its place?

dropped LR_chi2 df p_value AIC_increase
0 rain_in 5.6267 1 0.0177 3.6267
1 rain_3d 8.2012 1 0.0042 6.2012
2 log_dc 21.2885 1 0.0000 19.2885
3 stage 17.8703 1 0.0000 15.8703

Linearity in the logit (Box–Tidwell)

Code
rows = []
for c in SEL:
    v = risk[c] - risk[c].min() + 1
    df2 = risk.copy(); df2[c + "_xlogx"] = v * np.log(v)
    m2 = fit_logit(df2, SEL + [c + "_xlogx"])
    lr = 2 * (m2.llf - msel.llf)
    rows.append(dict(input=c, LR_chi2=lr, p_value=stats.chi2.sf(lr, 1)))
md("A small p-value means the input's effect on the log-odds is not linear and needs a transform.")
pd.DataFrame(rows).round(4)

A small p-value means the input’s effect on the log-odds is not linear and needs a transform.

input LR_chi2 p_value
0 rain_in 2.8411 0.0919
1 rain_3d 4.4941 0.0340
2 log_dc 2.1393 0.1436
3 stage 2.6439 0.1040

Calibration and discrimination (held-out predictions)

Code
psel = cv_predict(SEL)
def hosmer_lemeshow(yv, p, g=10):
    q = pd.qcut(p, g, duplicates="drop")
    t = pd.DataFrame({"y": yv, "p": p, "g": q}).groupby("g", observed=True).agg(o=("y", "sum"), e=("p", "sum"), n=("y", "size"))
    chi = (((t.o - t.e) ** 2) / (t.e * (1 - t.e / t.n) + 1e-12)).sum()
    return chi, stats.chi2.sf(chi, len(t) - 2)

hl_sel, hlp_sel = hosmer_lemeshow(y, psel)
hl_u, hlp_u = hosmer_lemeshow(y, pu)
fig, axs = plt.subplots(1, 2, figsize=(9.5, 3.8))
bins = np.r_[0, 0.01, 0.05, 0.2, 0.5, 1.0001]
for p, lab in [(psel, "selected linear"), (pu, "website model")]:
    b = np.digitize(p, bins) - 1
    obs = [y[b == i].mean() if (b == i).any() else np.nan for i in range(len(bins) - 1)]
    pred = [p[b == i].mean() if (b == i).any() else np.nan for i in range(len(bins) - 1)]
    axs[0].plot(pred, obs, "o-", label=lab)
    f, t, _ = roc_curve(y, p); axs[1].plot(f, t, label=f"{lab} (AUC {roc_auc_score(y, p):.3f})")
axs[0].plot([0, 1], [0, 1], ":", color="gray"); axs[0].set_xscale("symlog", linthresh=0.01); axs[0].set_yscale("symlog", linthresh=0.01)
axs[0].set_xlabel("predicted daily chance"); axs[0].set_ylabel("observed breach rate"); axs[0].legend(fontsize=8); axs[0].set_title("Calibration")
axs[1].plot([0, 1], [0, 1], ":", color="gray"); axs[1].set_xlabel("false positive rate"); axs[1].set_ylabel("true positive rate")
axs[1].legend(fontsize=8); axs[1].set_title("ROC (held-out)")
plt.tight_layout(); plt.show()
md(f"Hosmer–Lemeshow on held-out predictions: selected model χ² = {hl_sel:.1f} (p = {hlp_sel:.3f}); "
   f"website model χ² = {hl_u:.1f} (p = {hlp_u:.3f}). With {n_ev} events this test has little power, "
   "so the calibration plot is the more useful check.")

Hosmer–Lemeshow on held-out predictions: selected model χ² = 2.4 (p = 0.964); website model χ² = 11.1 (p = 0.198). With 17 events this test has little power, so the calibration plot is the more useful check.

Influence, residual dependence, and stability

Code
glm = sm.GLM(y, msel.model.exog, family=sm.families.Binomial()).fit()
cook = pd.Series(glm.get_influence().cooks_distance[0], index=risk.index)
top_inf = risk.assign(cooks_d=cook, p_fit=glm.fittedvalues).sort_values("cooks_d", ascending=False).head(6)
lb = acorr_ljungbox(glm.resid_pearson, lags=[1, 7], return_df=True)
md(f"**Most influential days** (Cook's distance). Ljung–Box test on Pearson residuals in date order: "
   f"lag 1 p = {lb.lb_pvalue.iloc[0]:.3f}, lag 7 p = {lb.lb_pvalue.iloc[1]:.3f}. Where residuals are "
   "correlated in time, the water-year-clustered standard errors above are the ones to use.")
top_inf[["breach", "p_fit", "cooks_d"] + SEL].round(3)

Most influential days (Cook’s distance). Ljung–Box test on Pearson residuals in date order: lag 1 p = 0.836, lag 7 p = 1.000. Where residuals are correlated in time, the water-year-clustered standard errors above are the ones to use.

breach p_fit cooks_d rain_in rain_3d log_dc stage
date
2023-03-22 0 0.725 0.178 0.19 2.14 1.946 8.138
2024-03-06 1 0.004 0.165 0.75 0.75 2.773 5.280
2023-03-21 0 0.852 0.142 1.59 1.99 1.792 7.361
2023-12-21 0 0.465 0.120 2.36 3.92 5.638 7.018
2023-12-22 0 0.314 0.104 0.24 3.77 5.642 8.698
2020-03-16 1 0.198 0.097 1.16 1.46 5.892 9.813
Code
rng = np.random.default_rng(1)
wys = np.unique(years); boots = []
for _ in range(500):
    pick = rng.choice(wys, len(wys), replace=True)
    idx = np.concatenate([np.where(years == w)[0] for w in pick])
    if y[idx].sum() < 3: continue
    try:
        bb = sm.Logit(y[idx], msel.model.exog[idx]).fit(disp=0, maxiter=200).params
        if np.all(np.isfinite(bb)) and np.abs(bb).max() < 50: boots.append(bb)
    except Exception:
        pass
boots = np.array(boots)
md(f"**Water-year block bootstrap** ({len(boots)} resamples of whole seasons). An input whose sign "
   "flips in many resamples is not reliably estimated from this record.")
pd.DataFrame({"coef": msel.params, "boot 2.5%": np.percentile(boots, 2.5, 0), "boot 97.5%": np.percentile(boots, 97.5, 0),
              "same sign in resamples": (np.sign(boots) == np.sign(msel.params.values)).mean(0)}).round(3)

Water-year block bootstrap (477 resamples of whole seasons). An input whose sign flips in many resamples is not reliably estimated from this record.

coef boot 2.5% boot 97.5% same sign in resamples
const -9.459 -20.774 -8.452 1.000
rain_in 0.299 0.010 0.487 0.977
rain_3d 0.507 0.401 1.776 1.000
log_dc -1.450 -4.433 -0.669 1.000
stage 1.805 1.355 4.987 1.000

Part 2. How the slough responds to rain

Storm events

A storm starts on a day with at least 0.1 in of rain after two dry days and continues while rain keeps falling (one-day gaps allowed). Storms of at least 0.25 in qualify when the mouth stayed closed from the day before through three days after, and a logger or the live sensor measured the level on at least 80% of those days. The response is the rise: the highest daily-mean level from the start of the storm to three days after it ends, minus the level the day before.

Code
D = d.copy()
meas = D.stage_mean_ft.where(D.stage_src.fillna("").str.endswith(("_logger", "_sensor")))
r = D.rain_in.fillna(0); idx = D.index; n = len(D)
rows, t = [], 0
while t < n:
    if r.iloc[t] >= 0.1 and r.iloc[max(0, t - 2):t].sum() < 0.05:
        s = e = t
        while e + 1 < n and (r.iloc[e + 1] >= 0.05 or (e + 2 < n and r.iloc[e + 2] >= 0.05)): e += 1
        w0, w1 = idx[s] - pd.Timedelta(days=1), idx[e] + pd.Timedelta(days=3)
        win = D.loc[w0:w1]; tot = r.iloc[s:e + 1].sum()
        if tot >= 0.25 and (win.mouth == "closed").all() and meas.loc[w0:w1].notna().mean() >= 0.8 and pd.notna(meas.get(w0)):
            rows.append(dict(start=idx[s], total=tot, max_day=r.iloc[s:e + 1].max(), duration=e - s + 1,
                             api0=D.api.get(w0), rain_wy0=D.rain_wy.get(w0), stage0=meas[w0],
                             log_dc=np.log(D.days_closed.get(w0)), rise=meas.loc[idx[s]:w1].max() - meas[w0]))
        t = e + 1
    else:
        t += 1
E = pd.DataFrame(rows).set_index("start")
E["total2"] = E.total ** 2
md(f"**{len(E)} storms qualify** ({E.total.min():.2f} to {E.total.max():.2f} in). With {len(E)} events, "
   "models are limited to about three predictors.")
fig, ax = plt.subplots(figsize=(6.5, 4))
sc = ax.scatter(E.total, E.rise, c=E.api0, cmap="viridis"); fig.colorbar(sc, label="antecedent index (in)")
ax.set_xlabel("storm rain (in)"); ax.set_ylabel("slough rise (ft)"); ax.set_title("Rise vs. storm rain, colored by watershed wetness")
plt.tight_layout(); plt.show()
E.describe().T[["mean", "std", "min", "max"]].round(2)

34 storms qualify (0.25 to 4.26 in). With 34 events, models are limited to about three predictors.

mean std min max
total 1.15 0.91 0.25 4.26
max_day 0.79 0.55 0.23 2.47
duration 1.97 1.22 1.00 5.00
api0 0.39 0.58 0.00 2.45
rain_wy0 7.53 8.87 0.00 27.98
stage0 6.00 1.55 3.86 9.56
log_dc 4.38 1.61 0.69 5.87
rise 0.97 0.89 0.06 4.26
total2 2.13 3.46 0.06 18.15

Candidate models

Code
FORMS = {
    "R0 intercept only": "rise ~ 1",
    "R1 total": "rise ~ total",
    "R1o total, through origin": "rise ~ total - 1",
    "R2 total + total²": "rise ~ total + total2",
    "R3 total + max_day": "rise ~ total + max_day",
    "R4 total + api0": "rise ~ total + api0",
    "R5 total + stage0": "rise ~ total + stage0",
    "R6 total + log_dc": "rise ~ total + log_dc",
    "R7 total + duration": "rise ~ total + duration",
    "R8 total + rain_wy0": "rise ~ total + rain_wy0",
    "R9 total + api0 + stage0": "rise ~ total + api0 + stage0",
    "R10 total + total² + api0": "rise ~ total + total2 + api0",
    "R11 power law: log(rise) ~ log(total)": "np.log(rise + 0.05) ~ np.log(total)",
}
def loo_rmse(formula, df):
    errs = []
    for i in range(len(df)):
        m = smf.ols(formula, df.drop(df.index[i])).fit()
        pred = m.predict(df.iloc[[i]]).iloc[0]
        if formula.startswith("np.log"): pred = np.exp(pred) - 0.05
        errs.append(df.rise.iloc[i] - pred)
    return float(np.sqrt(np.mean(np.square(errs))))
rows = []
for name, f in FORMS.items():
    m = smf.ols(f, E).fit()
    log_y = f.startswith("np.log")
    rows.append(dict(model=name, k=int(m.df_model), R2=m.rsquared, adj_R2=m.rsquared_adj,
                     AIC=np.nan if log_y else m.aic, BIC=np.nan if log_y else m.bic, LOO_RMSE_ft=loo_rmse(f, E)))
RM = pd.DataFrame(rows).sort_values("LOO_RMSE_ft").reset_index(drop=True)
md("AIC and BIC are not comparable for the log-response model, so they are left blank; leave-one-storm-out error compares all of them.")
RM.round(3)

AIC and BIC are not comparable for the log-response model, so they are left blank; leave-one-storm-out error compares all of them.

model k R2 adj_R2 AIC BIC LOO_RMSE_ft
0 R5 total + stage0 2 0.887 0.879 19.306 23.885 0.332
1 R4 total + api0 2 0.863 0.855 25.612 30.191 0.361
2 R1o total, through origin 1 0.927 0.925 27.687 29.213 0.373
3 R11 power law: log(rise) ~ log(total) 1 0.756 0.749 NaN NaN 0.375
4 R1 total 1 0.838 0.833 29.399 32.452 0.384
5 R8 total + rain_wy0 2 0.847 0.837 29.442 34.022 0.390
6 R7 total + duration 2 0.841 0.830 30.835 35.414 0.393
7 R6 total + log_dc 2 0.839 0.829 31.111 35.690 0.394
8 R9 total + api0 + stage0 3 0.892 0.881 19.564 25.670 0.405
9 R3 total + max_day 2 0.838 0.828 31.317 35.896 0.407
10 R2 total + total² 2 0.848 0.838 29.289 33.868 0.449
11 R10 total + total² + api0 3 0.871 0.858 25.726 31.831 0.452
12 R0 intercept only 0 0.000 0.000 89.301 90.827 0.900
Code
PRED = ["total", "total2", "max_day", "duration", "api0", "rain_wy0", "stage0", "log_dc"]
rows = []
for k in range(1, 4):
    for cols in itertools.combinations(PRED, k):
        if "total2" in cols and "total" not in cols: continue
        f = "rise ~ " + " + ".join(cols)
        m = smf.ols(f, E).fit()
        rows.append(dict(inputs=" + ".join(cols), k=k, adj_R2=m.rsquared_adj, AIC=m.aic, LOO_RMSE_ft=loo_rmse(f, E)))
RS = pd.DataFrame(rows).sort_values("LOO_RMSE_ft").reset_index(drop=True)
md(f"**All {len(RS)} subsets of up to three inputs**, ranked by leave-one-storm-out error:")
RS.head(12).round(3)

All 70 subsets of up to three inputs, ranked by leave-one-storm-out error:

inputs k adj_R2 AIC LOO_RMSE_ft
0 total + stage0 2 0.879 19.306 0.332
1 total + duration + stage0 3 0.880 20.032 0.341
2 total + rain_wy0 + stage0 3 0.884 18.954 0.342
3 total + total2 + stage0 3 0.887 17.919 0.344
4 total + stage0 + log_dc 3 0.879 20.327 0.345
5 total + max_day + stage0 3 0.875 21.289 0.356
6 total + api0 2 0.855 25.612 0.361
7 total + api0 + rain_wy0 3 0.853 26.908 0.365
8 total + api0 + log_dc 3 0.851 27.252 0.371
9 total + duration + api0 3 0.851 27.411 0.371
10 total + max_day + api0 3 0.850 27.514 0.379
11 total 1 0.833 29.399 0.384

The selected rise model, examined

Code
RSEL = RS.inputs.iloc[0]
fr = "rise ~ " + RSEL
mr = smf.ols(fr, E).fit()
mr_hc3 = smf.ols(fr, E).fit(cov_type="HC3")
Xr = mr.model.exog
bp = het_breuschpagan(mr.resid, Xr)
try:
    wt = het_white(mr.resid, Xr)
except Exception:
    wt = (np.nan, np.nan)
sw = stats.shapiro(mr.resid)
reset = linear_reset(mr, power=2, use_f=True)
vif_r = {c: variance_inflation_factor(Xr, i) for i, c in enumerate(mr.model.exog_names) if c != "Intercept"}
md(f"**{fr}**, ordinary least squares with HC3 (heteroscedasticity-robust) standard errors. VIF: "
   + ", ".join(f"{k} {v:.2f}" for k, v in vif_r.items()) + ".")
display(pd.DataFrame({"coef": mr.params, "SE (HC3)": mr_hc3.bse, "p (HC3)": mr_hc3.pvalues,
                      "95% CI low": mr_hc3.conf_int()[0], "95% CI high": mr_hc3.conf_int()[1]}).round(3))
pd.DataFrame([
    ("Shapiro–Wilk (residuals normal)", sw.statistic, sw.pvalue),
    ("Breusch–Pagan (constant variance)", bp[0], bp[1]),
    ("White (constant variance, general)", wt[0], wt[1]),
    ("Ramsey RESET (functional form)", float(reset.fvalue), float(reset.pvalue)),
    ("Durbin–Watson (independence; about 2 is good)", durbin_watson(mr.resid), np.nan),
], columns=["test", "statistic", "p_value"]).round(4)

rise ~ total + stage0, ordinary least squares with HC3 (heteroscedasticity-robust) standard errors. VIF: total 1.02, stage0 1.02.

coef SE (HC3) p (HC3) 95% CI low 95% CI high
Intercept 0.747 0.237 0.002 0.283 1.211
total 0.858 0.094 0.000 0.673 1.043
stage0 -0.127 0.035 0.000 -0.196 -0.058
test statistic p_value
0 Shapiro–Wilk (residuals normal) 0.9703 0.4694
1 Breusch–Pagan (constant variance) 3.7835 0.1508
2 White (constant variance, general) 7.7394 0.1712
3 Ramsey RESET (functional form) 4.0862 0.0522
4 Durbin–Watson (independence; about 2 is good) 2.0905 NaN
Code
infl = mr.get_influence()
fig, axs = plt.subplots(1, 3, figsize=(11, 3.4))
axs[0].scatter(mr.fittedvalues, mr.resid); axs[0].axhline(0, color="gray", ls=":")
axs[0].set_xlabel("fitted rise (ft)"); axs[0].set_ylabel("residual (ft)"); axs[0].set_title("Residuals vs fitted")
sm.qqplot(mr.resid, line="s", ax=axs[1]); axs[1].set_title("Normal Q–Q")
axs[2].stem(infl.cooks_distance[0]); axs[2].axhline(4 / len(E), color="gray", ls=":")
axs[2].set_xlabel("storm"); axs[2].set_ylabel("Cook's distance"); axs[2].set_title("Influence (line = 4/n)")
plt.tight_layout(); plt.show()
md("**Most influential storms:**")
E.assign(cooks_d=infl.cooks_distance[0], resid=mr.resid).sort_values("cooks_d", ascending=False).head(4)[
    ["total", "api0", "stage0", "rise", "resid", "cooks_d"]].round(3)

Most influential storms:

total api0 stage0 rise resid cooks_d
start
2023-12-18 4.26 0.018 4.810 4.260 0.470 0.786
2023-03-28 1.28 2.446 4.692 0.396 -0.853 0.144
2020-12-27 1.73 0.008 4.130 2.270 0.564 0.105
2024-01-03 0.30 1.619 9.558 0.073 0.285 0.089

Daily distributed-lag model

The storm model rests on fewer than 30 events. Every closed-mouth day with a measured level adds far more data: the day’s change in level, regressed on that day’s and earlier days’ rain, rain above a threshold (bigger storms shed proportionally more runoff), and a seasonal evaporation term. Newey–West standard errors allow for day-to-day correlation.

Code
L = D.copy()
L["meas"] = meas
L["dS"] = L.meas.diff()
L["closed2"] = (L.mouth == "closed") & (L.mouth.shift(1) == "closed")
for k in range(0, 5):
    L[f"P{k}"] = r.shift(k)
L["hinge"] = np.maximum(r - 0.5, 0)
L["season"] = np.cos(2 * np.pi * (L.index.dayofyear - 200) / 365.25)
Ld = L[L.closed2 & L.dS.notna() & L.P4.notna() & (L.dS.abs() < 3)]  # > 3 ft/day while closed = logger glitch
rows, fits = [], {}
for nl in range(1, 6):
    f = "dS ~ " + " + ".join(f"P{k}" for k in range(nl)) + " + hinge + season"
    m = smf.ols(f, Ld).fit(cov_type="HAC", cov_kwds={"maxlags": 7})
    fits[nl] = m
    rows.append(dict(lags=f"0 to {nl - 1}", n_days=int(m.nobs), adj_R2=m.rsquared_adj, AIC=m.aic, BIC=m.bic,
                     ft_per_inch_lag_sum=sum(m.params[f"P{k}"] for k in range(nl)), hinge_ft_per_inch=m.params["hinge"]))
LG = pd.DataFrame(rows)
best_nl = int(LG.BIC.idxmin()) + 1
mb = fits[best_nl]
wald = mb.t_test(" + ".join(f"P{k}" for k in range(best_nl)) + " = 0")
ci = np.asarray(wald.conf_int())[0]
md(f"BIC selects rain lags 0 to {best_nl - 1}. Combined rise per inch of rain (sum of the lag coefficients): "
   f"**{float(np.asarray(wald.effect)[0]):.2f} ft per inch** (95% CI {ci[0]:.2f} to {ci[1]:.2f}), plus "
   f"**{mb.params['hinge']:.2f} ft** for each inch of a single day's rain above 0.5 in.")
display(LG.round(4))
pd.DataFrame({"coef": mb.params, "SE (Newey–West)": mb.bse, "p": mb.pvalues}).round(4)

BIC selects rain lags 0 to 1. Combined rise per inch of rain (sum of the lag coefficients): 0.66 ft per inch (95% CI 0.55 to 0.76), plus 0.36 ft for each inch of a single day’s rain above 0.5 in.

lags n_days adj_R2 AIC BIC ft_per_inch_lag_sum hinge_ft_per_inch
0 0 to 0 2365 0.4354 -4584.6167 -4561.5426 0.3703 0.1966
1 0 to 1 2365 0.6878 -5984.9152 -5956.0725 0.6563 0.3623
2 0 to 2 2365 0.6878 -5983.8034 -5949.1922 0.6481 0.3622
3 0 to 3 2365 0.6883 -5986.5055 -5946.1257 0.6614 0.3645
4 0 to 4 2365 0.6882 -5984.7528 -5938.6046 0.6658 0.3642
coef SE (Newey–West) p
Intercept -0.0051 0.0016 0.0014
P0 0.1751 0.0577 0.0024
P1 0.4812 0.0389 0.0000
hinge 0.3623 0.1229 0.0032
season -0.0085 0.0020 0.0000

Part 3. First breach of the season versus later ones

The held-out predictions in Part 1 split cleanly by event type. The first breach of a water year comes after months behind a tall, wave-built berm; any later breach that season comes days or weeks after the mouth closed again, behind a thin, young berm. If those behave differently, a model that knows which kind of berm it is facing should do better. Here later = 1 on closed-mouth days after the season’s first breach.

Code
from scipy.special import expit
from scipy.optimize import minimize
from unified import Unified

wy_of = lambda t: t.year + (t.month >= 10)
bd_all = d.index[d.breach == 1]
bd_wy = np.array([wy_of(b) for b in bd_all])
risk["later"] = [int(((bd_all < t) & (bd_wy == wy_of(t))).any()) for t in risk.index]
risk["later_stage"] = risk.later * risk.stage
# crest floor, as in src/unified.py (highest level in the first 3 days after closing, known from then on)
closed = d.mouth.eq("closed")
spell = (closed & ~closed.shift(1, fill_value=False)).cumsum().where(closed)
early = d.stage_est.where(closed & (d.days_closed <= 3))
risk["crest_floor"] = early.groupby(spell).cummax().groupby(spell).ffill().shift(1).reindex(risk.index)
ev_dates = risk.index[y == 1]
ev_later = risk.later.values[y == 1]
evt = risk.loc[ev_dates, ["stage", "dc", "rain_in", "rain_3d", "rain_wy"]].assign(kind=np.where(ev_later, "later", "first"))
mw = stats.mannwhitneyu(evt.stage[evt.kind == "first"], evt.stage[evt.kind == "later"])
md(f"**{(ev_later == 0).sum()} first breaches and {(ev_later == 1).sum()} later ones.** Later breaches happen "
   f"{abs(evt.groupby('kind').stage.mean().diff().iloc[-1]):.1f} ft lower on average (Mann–Whitney p = {mw.pvalue:.4f}), "
   f"after a median of {evt[evt.kind == 'later'].dc.median():.0f} days closed versus {evt[evt.kind == 'first'].dc.median():.0f}. "
   f"`later` and log berm age correlate at r = {np.corrcoef(risk.later, risk.log_dc)[0, 1]:.2f} across all closed days.")
evt.groupby("kind").agg(["mean", "min", "max"]).round(2)

8 first breaches and 9 later ones. Later breaches happen 2.8 ft lower on average (Mann–Whitney p = 0.0001), after a median of 12 days closed versus 312. later and log berm age correlate at r = -0.49 across all closed days.

stage dc rain_in rain_3d rain_wy
mean min max mean min max mean min max mean min max mean min max
kind
first 9.50 8.91 10.30 285.88 99.0 362.0 2.00 1.16 3.38 3.15 1.46 4.96 7.51 5.78 9.71
later 6.72 5.28 8.15 11.78 3.0 22.0 1.15 0.13 2.54 1.96 0.75 3.06 18.35 9.33 25.17

Candidate models

Same held-out scoring as Part 1, with catches split by event type. The website model is scored three ways: as shipped, refit here, and refit with one extra parameter that lowers (or raises) the crest for later berms on top of what berm age already does.

Code
class UnifiedFL(Unified):
    names = Unified.names + ["d_later"]
    x0 = np.r_[Unified.x0, -1.5]
    def z(self, X, p=None):
        p = self.p if p is None else p
        crest = np.maximum(self.crest(X.dc, p), X.crest_floor.fillna(-np.inf)) + p[8] * X.later
        return p[3] + p[4] * (X.stage - crest) + p[5] * X.rain_in + p[6] * X.rain_3d + p[7] * X.api
    def fit(self, X, yv):
        yv = np.asarray(yv)
        def nll(p):
            pr = np.clip(expit(self.z(X, p)), 1e-9, 1 - 1e-9)
            prior = ((p[0] - 9.8) / 1.0) ** 2 / 2 + ((p[1] - 6.5) / 1.5) ** 2 / 2 + ((p[2] - np.log(70)) / 1.0) ** 2 / 2 \
                + 0.05 * np.sum(p[4:8] ** 2) + (p[8] / 1.5) ** 2 / 2
            return -np.sum(yv * np.log(pr) + (1 - yv) * np.log(1 - pr)) + prior
        best = None
        for start in (self.x0, self.x0 * [1, 1, 1, 1.5, 0.5, 2, 1, 1, 0.5]):
            r = minimize(nll, start, method="Nelder-Mead", options=dict(maxiter=20000, maxfev=20000, xatol=1e-5, fatol=1e-7))
            r = minimize(nll, r.x, method="BFGS")
            if best is None or r.fun < best.fun:
                best = r
        self.p = best.x
        return self

def cv_unified(cls):
    p = np.zeros(n_days)
    for wy in np.unique(years):
        tr, te = years != wy, years == wy
        p[te] = cls().fit(risk[tr], y[tr]).prob(risk[te])
    return p

def score_fl(p):
    out = score(p)
    hit = np.array([p[ix].max() >= 0.5 for ix in ev_idx])
    hl, hlp = hosmer_lemeshow(y, p)
    out.update(first_caught=f"{hit[ev_later == 0].sum()}/{(ev_later == 0).sum()}",
               later_caught=f"{hit[ev_later == 1].sum()}/{(ev_later == 1).sum()}", HL_p=hlp)
    return out

B3, B4 = ["rain_in", "rain_wy", "stage"], ["rain_in", "rain_3d", "rain_wy", "stage"]
FLP = {
    "website model, as shipped": pu,
    "website model, refit": cv_unified(Unified),
    "website model + later-berm crest shift": cv_unified(UnifiedFL),
    "3 inputs (rain_in + rain_wy + stage)": cv_predict(B3),
    "3 inputs + later": cv_predict(B3 + ["later"]),
    "3 inputs + later + later x stage": cv_predict(B3 + ["later", "later_stage"]),
    "best linear (rain_in + rain_3d + rain_wy + stage)": cv_predict(B4),
    "best linear + later": cv_predict(B4 + ["later"]),
    "best linear + later + later x stage": cv_predict(B4 + ["later", "later_stage"]),
}
FL = pd.DataFrame([dict(model=k, **score_fl(v)) for k, v in FLP.items()])
FL[["model", "cv_logloss", "cv_brier", "cv_AUC", "caught_p50", "first_caught", "later_caught", "false_alarm_days_p50", "HL_p"]].round(
    {"cv_logloss": 4, "cv_brier": 5, "cv_AUC": 3, "HL_p": 3})
model cv_logloss cv_brier cv_AUC caught_p50 first_caught later_caught false_alarm_days_p50 HL_p
0 website model, as shipped 0.0233 0.00434 0.983 12 7/8 5/9 8 0.198
1 website model, refit 0.0233 0.00434 0.983 12 7/8 5/9 8 0.195
2 website model + later-berm crest shift 0.0254 0.00445 0.974 12 7/8 5/9 8 0.000
3 3 inputs (rain_in + rain_wy + stage) 0.0168 0.00341 0.946 10 7/8 3/9 1 0.052
4 3 inputs + later 0.0170 0.00293 0.955 11 6/8 5/9 1 0.000
5 3 inputs + later + later x stage 0.0171 0.00296 0.952 12 7/8 5/9 1 0.000
6 best linear (rain_in + rain_3d + rain_wy + stage) 0.0153 0.00362 0.970 11 7/8 4/9 4 0.767
7 best linear + later 0.0173 0.00380 0.958 11 6/8 5/9 5 0.000
8 best linear + later + later x stage 0.0239 0.00435 0.963 12 7/8 5/9 6 0.000

Is the term real?

Code
rows = []
for name, cols in (("3 inputs", B3), ("best linear", B4)):
    m0, m1, m2 = fit_logit(risk, cols), fit_logit(risk, cols + ["later"]), fit_logit(risk, cols + ["later", "later_stage"])
    rows.append(dict(base=name, later_coef_per_SD=m1.params["later"], LR_p_later=stats.chi2.sf(2 * (m1.llf - m0.llf), 1),
                     LR_p_interaction=stats.chi2.sf(2 * (m2.llf - m1.llf), 1)))
u0 = Unified().fit(risk, y); u1 = UnifiedFL().fit(risk, y)
nll_ = lambda m: -np.sum(y * np.log(np.clip(m.prob(risk), 1e-9, 1)) + (1 - y) * np.log(np.clip(1 - m.prob(risk), 1e-9, 1)))
p_shift = stats.chi2.sf(2 * (nll_(u0) - nll_(u1)), 1)
# season-block bootstrap of the later coefficient in the 3-input model
rng = np.random.default_rng(3); wys = np.unique(years); bl = []
for _ in range(400):
    idx = np.concatenate([np.where(years == w)[0] for w in rng.choice(wys, len(wys), replace=True)])
    sub = risk.iloc[idx]
    if sub.breach.sum() < 3 or sub.later.nunique() < 2 or sub.breach[sub.later == 1].sum() == 0:
        continue
    try:
        bb = fit_logit(sub, B3 + ["later"]).params["later"]
        if np.isfinite(bb) and abs(bb) < 50: bl.append(bb)
    except Exception:
        pass
bl = np.array(bl)
md(f"In the crest-curve model the extra shift fits at **{u1.p[8]:+.2f} ft** (likelihood-ratio p = {p_shift:.2f}): once berm age "
   f"is in the crest curve, knowing the berm is a later one adds nothing. In the linear models, which have no berm age, "
   f"`later` is strongly significant, and its sign is positive in {(bl > 0).mean():.0%} of {len(bl)} season-block bootstrap resamples.")
pd.DataFrame(rows).round(4)

In the crest-curve model the extra shift fits at -0.33 ft (likelihood-ratio p = 0.75): once berm age is in the crest curve, knowing the berm is a later one adds nothing. In the linear models, which have no berm age, later is strongly significant, and its sign is positive in 100% of 399 season-block bootstrap resamples.

base later_coef_per_SD LR_p_later LR_p_interaction
0 3 inputs 3.6512 0.0001 0.2025
1 best linear 3.6380 0.0002 0.0002

At the alert threshold

The site alerted at 30% at this stage. Catches and false-alarm days at that line, on held-out seasons:

Code
def at(p, thr):
    hit = np.array([p[ix].max() >= thr for ix in ev_idx])
    miss = [str(b.date()) for b, h in zip(ev_dates, hit) if not h]
    return dict(caught=f"{hit.sum()}/{len(hit)}", missed=", ".join(miss), false_alarm_days=int(((p >= thr) & ~near).sum()))
AT = pd.DataFrame([dict(model=k, threshold=f"{t:.0%}", **at(FLP[k], t)) for k in
                   ("website model, as shipped", "3 inputs + later") for t in (0.3, 0.5)])
AT
model threshold caught missed false_alarm_days
0 website model, as shipped 30% 13/17 2020-03-16, 2020-04-06, 2024-03-06, 2025-03-13 8
1 website model, as shipped 50% 12/17 2020-03-16, 2020-04-06, 2024-03-06, 2024-03-31... 8
2 3 inputs + later 30% 13/17 2020-04-06, 2024-03-06, 2024-03-31, 2025-03-13 3
3 3 inputs + later 50% 11/17 2019-01-07, 2020-03-16, 2020-04-06, 2024-03-06... 1

What it means

Code
f = FL.set_index("model")
a30 = AT.set_index(["model", "threshold"])
md(f"""
- **Later breaches really are different:** they happen about {abs(evt.groupby('kind').stage.mean().diff().iloc[-1]):.1f} ft lower, behind berms
  only days or weeks old.
- **The crest-curve model already knows this,** through berm age: a separate later-berm shift fits at {u1.p[8]:+.2f} ft and is not
  significant (p = {p_shift:.2f}). With breaches dated by when the slough drained, it has the best held-out log loss
  ({f.loc['website model, refit', 'cv_logloss']:.4f}) and passes the calibration test.
- **At the alert line the 3-input + later model has fewer false alarms:** at 30% it flags {a30.loc[('3 inputs + later', '30%'), 'caught']} breaches
  with {a30.loc[('3 inputs + later', '30%'), 'false_alarm_days']} false-alarm day(s), versus {a30.loc[('website model, as shipped', '30%'), 'caught']}
  with {a30.loc[('website model, as shipped', '30%'), 'false_alarm_days']} for the crest curve. Both miss the same two later breaches
  ({a30.loc[('3 inputs + later', '30%'), 'missed']}), young berms that broke at low levels after moderate rain.
- **Its percentages are less trustworthy as odds** (it fails the calibration test), but with 14 breaches that test has little to
  stand on, and the site used the chance as a ranking with a single alert line.
- **Next:** Part 4 refits on days with a measured level, which is how the site runs, and settles the inputs.
""")
  • Later breaches really are different: they happen about 2.8 ft lower, behind berms only days or weeks old.
  • The crest-curve model already knows this, through berm age: a separate later-berm shift fits at -0.33 ft and is not significant (p = 0.75). With breaches dated by when the slough drained, it has the best held-out log loss (0.0233) and passes the calibration test.
  • At the alert line the 3-input + later model has fewer false alarms: at 30% it flags 13/17 breaches with 3 false-alarm day(s), versus 13/17 with 8 for the crest curve. Both miss the same two later breaches (2020-04-06, 2024-03-06, 2024-03-31, 2025-03-13), young berms that broke at low levels after moderate rain.
  • Its percentages are less trustworthy as odds (it fails the calibration test), but with 14 breaches that test has little to stand on, and the site used the chance as a ranking with a single alert line.
  • Next: Part 4 refits on days with a measured level, which is how the site runs, and settles the inputs.

Part 4. Days with a measured level

The site always starts from a measured level: yesterday’s mean from the live sensor. Earlier seasons sometimes had no logger running, and on those days the level above is a staff-gauge reading rolled forward with rain, or a filled-in typical value. Here the models are refit and scored only on closed-mouth days whose level was measured the day before. Breaches on days without one are left out rather than counted as misses. Season-to-date rain is dropped (its coefficient was not significant), and rain the day before is tested as a lag term.

Code
meas = d.stage_mean_ft.where(d.stage_src.fillna("").str.endswith(("_logger", "_sensor")))
risk["measured"] = meas.shift(1).reindex(risk.index).notna()
risk["rain_prev"] = d.rain_in.fillna(0).shift(1).reindex(risk.index)
M = risk[risk.measured].copy()
yM = M.breach.values
excluded = [str(b.date()) for b in risk.index[(risk.breach == 1) & ~risk.measured]]
evM = M.index[yM == 1]
ixM = [np.where((M.index >= b - pd.Timedelta(days=1)) & (M.index <= b))[0] for b in evM]
nearM = near[risk.measured.values]

def cvM(cols):
    p = np.zeros(len(M)); Xa = M[cols].values.astype(float); yrs = M.water_year.values
    for wy in np.unique(yrs):
        tr, te = yrs != wy, yrs == wy
        mu, sd = Xa[tr].mean(0), Xa[tr].std(0) + 1e-12
        m = LogisticRegression(penalty=None, max_iter=10000).fit((Xa[tr] - mu) / sd, yM[tr])
        p[te] = m.predict_proba((Xa[te] - mu) / sd)[:, 1]
    return p

def atM(p, thr):
    hit = np.array([p[ix].max() >= thr for ix in ixM])
    return dict(caught=f"{hit.sum()}/{len(hit)}", missed=", ".join(str(b.date()) for b, h in zip(evM, hit) if not h),
                false_alarm_days=int(((p >= thr) & ~nearM).sum()))

FINAL = ["rain_in", "stage", "later"]
V = {"rain that day + level + later (in use)": FINAL,
     "+ rain the day before": ["rain_in", "rain_prev", "stage", "later"],
     "+ rain since Oct 1": ["rain_in", "rain_wy", "stage", "later"],
     "+ both": ["rain_in", "rain_prev", "rain_wy", "stage", "later"]}
PM = {k: cvM(v) for k, v in V.items()}
PM["crest curve + rain (previous site model)"] = pu[risk.measured.values]
T4 = pd.DataFrame([dict(model=k, cv_logloss=log_loss(yM, np.clip(p, 1e-6, 1 - 1e-6)), cv_AUC=roc_auc_score(yM, p),
                        **{f"{k2}@30": v2 for k2, v2 in atM(p, 0.3).items()}, **{"caught@50": atM(p, 0.5)["caught"]})
                   for k, p in PM.items()])
md(f"**{len(M):,} days with a measured level, {int(yM.sum())} breaches.** Left out for lack of a measured level: "
   + ", ".join(excluded) + ".")
T4.round({"cv_logloss": 4, "cv_AUC": 3})

2,404 days with a measured level, 17 breaches. Left out for lack of a measured level: .

model cv_logloss cv_AUC caught@30 missed@30 false_alarm_days@30 caught@50
0 rain that day + level + later (in use) 0.0134 0.978 13/17 2020-04-06, 2024-03-06, 2024-03-31, 2025-03-13 0 12/17
1 + rain the day before 0.0119 0.980 13/17 2020-03-16, 2020-04-06, 2024-03-06, 2025-03-13 2 13/17
2 + rain since Oct 1 0.0150 0.964 13/17 2020-04-06, 2024-03-06, 2024-03-31, 2025-03-13 0 12/17
3 + both 0.0130 0.966 13/17 2020-03-16, 2020-04-06, 2024-03-06, 2025-03-13 2 13/17
4 crest curve + rain (previous site model) 0.0125 0.984 13/17 2020-03-16, 2020-04-06, 2024-03-06, 2025-03-13 4 12/17
Code
mF = sm.Logit(yM, sm.add_constant(M[FINAL].astype(float))).fit(disp=0, maxiter=500)
mP = sm.Logit(yM, sm.add_constant(M[V["+ rain the day before"]].astype(float))).fit(disp=0, maxiter=500)
t4 = T4.set_index("model")
md(f"Adding rain the day before is significant (p = {mP.pvalues['rain_prev']:.3f}) and improves the fit overall "
   f"(AIC {mP.aic:.1f} vs {mF.aic:.1f}), but at the 30% alert line it flags {t4.loc['+ rain the day before', 'caught@30']} "
   f"breaches against {t4.loc['rain that day + level + later (in use)', 'caught@30']}. The level input already carries "
   "yesterday's rain, because the water balance adds runoff over the two days after a storm, so the two overlap. "
   "The best of these simple models, for comparison with Part 5:")
pd.DataFrame({"coef": mF.params, "SE": mF.bse, "95% CI low": mF.conf_int()[0], "95% CI high": mF.conf_int()[1],
              "odds ratio": np.exp(mF.params), "p": mF.pvalues}).round(4)

Adding rain the day before is significant (p = 0.000) and improves the fit overall (AIC 54.4 vs 65.2), but at the 30% alert line it flags 13/17 breaches against 13/17. The level input already carries yesterday’s rain, because the water balance adds runoff over the two days after a storm, so the two overlap. The best of these simple models, for comparison with Part 5:

coef SE 95% CI low 95% CI high odds ratio p
const -29.4093 5.4757 -40.1416 -18.6771 0.0000 0.0
rain_in 4.0029 0.6792 2.6717 5.3341 54.7551 0.0
stage 2.5454 0.5685 1.4312 3.6595 12.7481 0.0
later 7.3361 1.8016 3.8050 10.8672 1534.7189 0.0

Part 5. Catching every breach

The goal for alerts is to flag every breach on the measured-level record with as few false alarms as possible; there is no fixed cutoff. For each model the cutoff is set as low as the weakest breach requires (its highest score on the breach day or the day before), and false alarms are counted at that cutoff, as days and as episodes (one alert per episode). That cutoff is tuned on the same predictions it is scored on, so each model is also checked with each season held out (cross-validated): every season’s cutoff comes only from the other seasons’ breaches. Every combination of up to five rain, level and berm-age inputs is searched; tide, surge and wave inputs are left out (they do not drive breaches here), and the crest-curve model is scored alongside.

Code
C5 = ["rain_in", "rain_prev", "rain_3d", "api", "rain_wy", "log_dc", "stage", "later"]
X5 = M[C5].astype(float).fillna(M[C5].median()).values
yrsM = M.water_year.values
evyr = np.array([yrsM[ix[-1]] for ix in ixM])
def n_eps(idx): return int((np.diff(np.r_[-99, idx]) > 3).sum()) if len(idx) else 0
def catch_all(p, name):
    peaks = np.array([p[ix].max() for ix in ixM]); thr = peaks.min()
    fa = np.where((p >= thr) & ~nearM)[0]
    c, nfa, miss = 0, [], []
    for w in np.unique(yrsM):
        t = peaks[evyr != w].min()
        hit = peaks[evyr == w] >= t
        c += int(hit.sum()); miss += [str(evM[i].date()) for i in np.where(evyr == w)[0][~hit]]
        nfa += list(np.where((p >= t) & ~nearM & (yrsM == w))[0])
    return dict(model=name, cutoff=thr, fa_days=len(fa), fa_episodes=n_eps(fa), honest_caught=f"{c}/{len(peaks)}",
                honest_fa_days=len(nfa), honest_fa_episodes=n_eps(np.sort(nfa)), honest_missed=", ".join(miss),
                caught_50=int((peaks >= 0.5).sum()), fa_days_50=int(((p >= 0.5) & ~nearM).sum()))
rows5 = []
for k in range(1, 6):
    for cols in itertools.combinations(range(len(C5)), k):
        Xa = X5[:, cols]; p = np.zeros(len(M))
        for w in np.unique(yrsM):
            tr, te = yrsM != w, yrsM == w
            mu, sd = Xa[tr].mean(0), Xa[tr].std(0) + 1e-12
            p[te] = LogisticRegression(penalty=None, max_iter=5000).fit((Xa[tr] - mu) / sd, yM[tr]).predict_proba((Xa[te] - mu) / sd)[:, 1]
        rows5.append(catch_all(p, " + ".join(C5[c] for c in cols)))
pcrest = pu[risk.measured.values]
rows5.append(catch_all(pcrest, "crest curve (level vs rebuilding berm + rain_in + rain_3d + api)"))
R5 = pd.DataFrame(rows5)
md(f"**{len(R5)} models.** Best ten by the honest check (breaches caught, then false-alarm episodes, which is what alerts send):")
R5.sort_values(["honest_caught", "honest_fa_episodes", "honest_fa_days"], ascending=[False, True, True]).head(10).round({"cutoff": 4})

219 models. Best ten by the season-held-out check (breaches caught, then false-alarm episodes, which is what alerts send):

model cutoff fa_days fa_episodes honest_caught honest_fa_days honest_fa_episodes honest_missed caught_50 fa_days_50
48 rain_in + api + log_dc 0.0417 30 9 16/17 28 8 2020-03-16 8 7
117 rain_in + api + rain_wy + log_dc 0.0471 29 8 16/17 29 8 2020-03-16 8 8
122 rain_in + api + stage + later 0.0156 44 9 16/17 30 8 2024-03-06 13 2
195 rain_in + api + log_dc + stage + later 0.0079 67 9 16/17 35 8 2024-03-06 13 2
87 api + stage + later 0.0075 69 11 16/17 52 8 2024-03-06 11 5
11 rain_in + rain_wy 0.0427 11 9 16/17 10 9 2024-03-06 8 2
53 rain_in + rain_wy + later 0.0425 11 9 16/17 10 9 2024-03-06 8 2
50 rain_in + api + later 0.0291 34 10 16/17 32 9 2020-03-16 9 7
121 rain_in + api + log_dc + later 0.0284 34 10 16/17 32 9 2020-03-16 8 7
14 rain_in + later 0.0322 12 10 16/17 12 10 2024-03-06 7 2
Code
cr = R5[R5.model.str.startswith("crest")].iloc[0]
md(f"""
No model catches all {len(evM)} breaches when the cutoff is set without seeing the season: with ten events
the weakest breach can't be anticipated from the others. The crest-curve model catches {cr.honest_caught}
that way with {cr.honest_fa_episodes} false-alarm episodes, the fewest of the leaders, and it is the only one
whose upper range means much: {cr.caught_50} of {len(evM)} breaches reached 0.5, against 4 to 5 for the rain-only
models that score as well at the low end. Tuned on all seasons, a cutoff of {cr.cutoff:.3f} catches every breach
with {cr.fa_days} false-alarm days in {cr.fa_episodes} episodes. The site then adopted the crest-curve model,
reporting levels rather than percentages with alerts at the lowest one. That is the earlier model: with the
reviewed labels of Part 6, the site moved to rain_in + rain_prev + margin + margin_max (**Part 7**), which reports
No Chance, Possible (every recorded breach reached it), Likely (0.2) and Imminent (0.5), with alerts at Possible.

""")

No model catches all 17 breaches when the cutoff is set without seeing the season: with ten events the weakest breach can’t be anticipated from the others. The crest-curve model catches 16/17 that way with 25 false-alarm episodes, the fewest of the leaders, and it is the only one whose upper range means much: 12 of 17 breaches reached 0.5, against 4 to 5 for the rain-only models that score as well at the low end. In-sample, with one cutoff tuned on all seasons, a cutoff of 0.000 catches every breach with 557 false-alarm days in 29 episodes. The model the site runs is in Part 7; it reports No Chance, Possible (every recorded breach reached it), Likely (0.2) and Imminent (0.5), with alerts at Possible.

Part 6. Breach dates, berm height and rain windows

Three questions from the Part 5 search. Are the breach dates right? Does an estimate of the berm height (the crest curve, or the highest level the berm has held since it closed) help when combined with rain? And would rain over the prior 24 or 48 hours, rather than same-day rain, catch the weakest breach, 2025-03-13? The scripts are in analysis/berm/, the case notes in analysis/berm/README.md and the level-gap audit in analysis/berm/data_gaps.md; this part reads their saved tables (the searches take a few minutes each). Scoring is as in Part 5: measured-level days, leave-one-season-out fits, the catch-all cutoff at the weakest breach, and the season-held-out check with each season’s cutoff from the other seasons. Tide, surge and wave inputs are left out. Levels in the figures are hourly logger readings put on the daily composite’s NAVD88 datum (analysis/berm/hourly_level.py).

Code
sys.path.insert(0, str(ROOT / "analysis" / "berm"))
import plots as BP
BR = ROOT / "analysis" / "berm" / "results"
rd = lambda f, **k: pd.read_csv(BR / f, **k)
SHOW = ["model", "h_caught", "h_fa_days", "h_fa_ep", "h_missed", "fa_days", "fa_ep"]
NICE = dict(model="Model", h_caught="Honest caught", h_fa_days="Honest false-alarm days",
            h_fa_ep="Honest episodes", h_missed="Honest miss", fa_days="Catch-all false-alarm days",
            fa_ep="Catch-all episodes")
def tbl(df, n=None):
    df = df[~df.degenerate] if "degenerate" in df else df
    return (df.head(n) if n else df)[SHOW].rename(columns=NICE).fillna("")
SN, SA = rd("search_nest_top300.csv"), rd("search_all_top300.csv")
F, S3 = rd("fixes.csv"), rd("select_nested_k3.csv")
OLD = lambda f: rd(f"old_labels/{f}")
nb = int(SN.h_caught.max()) + 1  # breaches on measured-level days (the honest check misses the weakest)

Breach dates

The model’s breach dates are CCBER’s logged mouth openings (build_dataset.mouth_state), each checked against the sub-daily level loggers (breach_timing.py), which would move a breach to the day the slough began to drain if that came earlier. Rebuilt from the level record alone (the highest level before the largest fall within 48 hours, infer_breaches.py), every logged breach with a logger record drained on its logged day (Figure 1).

Code
B6 = rd("inferred_breaches.csv", parse_dates=["apex48"])
B6["inferred apex (local)"] = B6.apex48.dt.strftime("%Y-%m-%d %H:%M").fillna("no logger")
B6[["ccber", "inferred apex (local)", "inferred_day", "diff_days_vs_ccber", "basis"]].rename(
    columns={"ccber": "CCBER date", "inferred_day": "Breach day", "diff_days_vs_ccber": "Days from CCBER", "basis": "Basis"})
CCBER date inferred apex (local) Breach day Days from CCBER Basis
0 2019-01-07 2019-01-07 03:00 2019-01-07 0 Veneco_Bridge_PTdata_2018-2023wy
1 2020-03-16 2020-03-16 16:00 2020-03-16 0 Veneco_Bridge_PTdata_2018-2023wy
2 2021-01-28 2021-01-28 15:30 2021-01-28 0 Veneco_Bridge_PTdata_2018-2023wy
3 2021-12-23 2021-12-23 16:30 2021-12-23 0 Veneco_Bridge_PTdata_2018-2023wy
4 2022-12-31 2022-12-31 16:38 2022-12-31 0 Veneco_Bridge_PTdata_2018-2023wy
5 2023-02-25 2023-02-25 01:30 2023-02-25 0 Veneco_Bridge_PTdata_2018-2023wy
6 2023-03-10 2023-03-10 14:15 2023-03-10 0 Veneco_Bridge_PTdata_2018-2023wy
7 2024-01-22 2024-01-22 08:39 2024-01-22 0 PIER_10.10.23_uncomp_ksba
8 2024-02-01 2024-02-01 05:24 2024-02-01 0 PIER_10.10.23_uncomp_ksba
9 2024-03-07 2024-03-07 09:09 2024-03-07 0 PIER_10.10.23_uncomp_ksba
10 2025-02-13 2025-02-13 15:39 2025-02-13 0 PIER_03.19.24_05.10.25_Compensated
11 2025-03-13 2025-03-13 02:24 2025-03-13 0 PIER_03.19.24_05.10.25_Compensated
12 2025-11-15 2025-11-15 17:00 2025-11-15 0 licor
13 2026-02-19 2026-02-19 09:00 2026-02-19 0 licor
Code
BP.breaches_grid();
Figure 1: Every breach with a level logger, hour by hour. Slough level (blue line, ft NAVD88), daily rain (gold bars) and predicted tide at Santa Barbara (thin gray line, unscaled, shown only to mark tidal exchange). Red lines mark when the slough began to drain (breach_timing.py); dashed red marks a breach day without a logged time.

The February 2026 breach drained on 2/19, not 2/18: the live sensor rose through 2/18 and peaked at 8.12 ft at 09:00 on 2/19 before falling 3.9 ft by evening. A plot of daily means peaks on 2/18 only because the 2/19 mean includes the drain. Dating it 2/18 also scored worse (with the old labels, rain-only models went from 8 to 23 false-alarm days).

Label review

Three stretches of the level record disagree with the mouth log; the days involved were false alarms in every model. They are corrected in data/mouth_corrections.csv, which mouth_state() applies after the log, and the whole pipeline was refit (train.py, unified.py measured, watch_model.py, levels.py). The 2020 and 2024 corrections are provisional, pending Garrett’s review of the figures below; the 2023 correction is his own read of the record.

  • March to April 2020, closed earlier and an unlogged breach (provisional). The log has the mouth open from the 3/16 breach until it closed on 4/5. The level stops following the tide about 3/21, drains a rain pulse back by 3/24 and rises steadily without rain from 3/26, with small steps at high tide on 4/1 to 4/4 as waves topped the berm: closed from 3/26. Then 1.31 in of rain on 4/5 lifted the slough to 6.79 ft at 02:00 on 4/6, it fell 1.3 ft in three hours, and it swung with the tides through 4/12 (Figure 3): an unlogged breach on 4/6, open to 4/12.
  • 2024-03-31, unlogged breach (provisional). After 1.8 in of rain the slough peaked at 6.5 ft on 3/30 and again about 03:00 on 3/31, then fell 1.8 ft by 4/3, and jumped back up on 4/5 as the berm re-closed (Figure 2). Mouth marked open 3/31 to 4/4.
  • March 2023, reviewed by Garrett. The log has the mouth open from the 3/10 breach until 3/30. The breach drain ends on 3/11 and the berm is closed from 3/12. On 3/14, 2.5 in of rain took the slough to 7.3 ft at 14:00 and it lost about 2 ft within a day: a brief opening, now a breach, closed again from 3/16. After that the level holds except for steps at high tide (3/17, 3/19, 3/22 to 24) that recede slowly: high tides topping a berm closed to real outflow, through about 4/14 (Figure 6).
  • 2024-03-23 to 03-24, left as closed. The slough rose 0.45 ft with 0.64 in of rain, then fell 0.8 ft over about 30 hours to below its pre-storm level, with no tidal signal. A brief, shallow overwash or seep that resealed is possible; it is not labeled a breach.
Code
BP.label_review_one(0);
Figure 2: March 20 to April 8, 2024 (provisional). Hourly slough level with daily rain and predicted tide. The 3/23 rise and slow fall is left as closed; the 3/31 drain (red line) and the 4/5 re-closure are now labeled a breach, pending review.
Code
BP.label_review_one(1);
Figure 3: March 14 to April 16, 2020 (provisional). After the 3/16 breach the tidal swing fades by 3/21 and the slough fills steadily from 3/26 (closed), with small high-tide steps over the berm on 4/1 to 4/4. Rain on 4/5 lifts it to 6.8 ft and it breaches at about 02:00 on 4/6 (red line), tidal to 4/12. The log had it open to 4/5 and closed after.
Code
BP.label_review_one(2);
Figure 4: March 1 to 16, 2025. The berm closed on 3/3 at 5.7 ft, rose to 6.0 ft with rain on 3/11 to 3/12, and failed at about 02:30 on 3/13 at 6.8 ft (red line).
Code
BP.label_review_one(3);
Figure 5: February 14 to March 10, 2024. The berm re-closes on 2/20 after the 2/1 breach; king tides swing the slough 0.6 to 1 ft a day on 2/21 to 2/24 without a breach; the breach drains from about 18:00 on 3/6 (red line), a day before CCBER’s 3/7 (reviewed). Daily rain in gold.
Code
BP.label_review_one(4);
Figure 6: March 10 to April 15, 2023 (reviewed). The 3/10 breach drains through 3/11 and the berm is closed from 3/12. A brief opening at about 14:00 on 3/14 (red line) drains 2 ft; closed again from 3/16, with high tides topping the berm and seeping back slowly through about 4/14. The log had it open until 3/30.

Berm height and rain windows

New inputs: margin, the slough level minus the crest curve’s estimate of the berm top (with the crest floor from the first three days after closure, as in src/unified.py); maxlvl, the highest measured level since the berm closed, through the day before, and margin_max, the level minus it; rain_2d, rain that day plus the day before; and from KSBA hourly rain, trail24 and trail48 (the 24 and 48 hours ending at midnight at the end of the day) and peak24 (the wettest 24 hours ending at any hour of the day, so a storm that straddles midnight counts in full).

The stored crest curve (models/unified_measured.npy) is fit on every season, so in a held-out test it leaks the held-out season into margin. Here the crest curve is refit with each season held out. With the old labels the leak made margin models look far better than they were; with the corrected labels it no longer helps:

Code
SAo, SNo = OLD("search_all_top300.csv").set_index("model"), OLD("search_nest_top300.csv").set_index("model")
keys = ["rain_in + margin", "margin + peak24", "rain_in + rain_prev + margin", "margin + trail24"]
f = lambda T, k: f"{T.loc[k].h_fa_days} / {T.loc[k].h_fa_ep}" if k in T.index else ""
pd.DataFrame({"Model": keys,
              "Old labels, leaky crest": [f(SAo, k) for k in keys], "Old labels, fair crest": [f(SNo, k) for k in keys],
              "Corrected, leaky crest": [f(SA.set_index("model"), k) for k in keys],
              "Corrected, fair crest": [f(SN.set_index("model"), k) for k in keys]}).set_index("Model")
Old labels, leaky crest Old labels, fair crest Corrected, leaky crest Corrected, fair crest
Model
rain_in + margin 0 / 0 15 / 6 153 / 16 153 / 18
margin + peak24 3 / 2 6 / 4 182 / 19 429 / 28
rain_in + rain_prev + margin 3 / 2 9 / 6 215 / 22 234 / 28
margin + trail24 2 / 2 15 / 6 173 / 17 210 / 21

The table entries are held-out false-alarm days / episodes. Every combination of up to five of the 16 inputs (6,886 models) was searched with the fair crest and the corrected labels. The best fifteen, and the reference models:

Code
REF = ["CREST-CURVE MODEL (stored, site)", "CREST-CURVE MODEL (refit, measured days)", "rain_in + rain_wy",
       "rain_in + later", "crest + maxlvl + rain_2d", "later + maxlvl + rain_2d", "margin + peak24", "margin + trail48", "rain_in + margin",
       "rain_in + rain_prev + margin", "margin + rain_2d", "rain_wy + rain_2d", "margin + trail24", "rain_2d", "trail24", "trail48", "peak24"]
tbl(SN[SN.model.isin(REF)])
Model Held out: caught Held out: false-alarm days Held out: episodes Held out: missed Catch-all false-alarm days Catch-all episodes
25 later + maxlvl + rain_2d 16 10 5 2020-04-06 11 6
237 rain_in + rain_wy 16 10 9 2024-03-06 11 9
301 crest + maxlvl + rain_2d 16 25 9 2022-12-31 26 10
310 rain_in + later 16 12 10 2024-03-06 12 10
350 rain_wy + rain_2d 16 21 12 2024-03-06 21 12
405 CREST-CURVE MODEL (refit, measured days) 16 131 22 2024-03-06 185 26
406 margin + rain_2d 16 219 25 2024-03-06 280 28
409 margin + trail48 16 273 27 2024-03-06 335 30
411 rain_in + rain_prev + margin 16 234 28 2024-03-06 294 31
413 margin + peak24 16 429 28 2024-03-06 480 33
427 rain_in + margin 15 153 18 2024-03-06;2024-03-31 214 21
428 margin + trail24 15 210 21 2024-03-06;2024-03-31 271 23
Code
SNi = SN.set_index("model")
crest = SNi.loc["CREST-CURVE MODEL (stored, site)"]
rw = SNi.loc["rain_in + rain_wy"]
rl = SNi.loc["rain_in + later"]
mp = SNi.loc["margin + peak24"]
old = SNi.loc["rain_in + rain_prev + margin + margin_max"]
best = SN[~SN.degenerate].iloc[0]
b3 = SN[(~SN.degenerate) & (SN.k <= 3)].iloc[0]
pk = rd("crest_variants.csv").set_index("variant").iloc[0]
md(f"""
These results use the corrected margin_max (the breach-day leak described in Part 7 is fixed).
**2024-03-06 is the weakest breach** for the crest-curve model and most others: a berm about two
weeks old that drained from about 6.0 ft after 0.75 in of rain. **2025-03-13 is caught** on the
honest check by the crest-curve model (peak score {pk.peak_20250313:.2f}) and by most models.

**Best models.** The leaders all use **maxlvl**, the highest level this berm has already held:
**{best.model}** catches {best.h_caught}/{nb} with {best.h_fa_days} false-alarm days in {best.h_fa_ep}
episodes, missing {best.h_missed}. The best of up to three inputs is **{b3.model}**: {b3.h_caught}/{nb},
{b3.h_fa_days} days in {b3.h_fa_ep} episodes, missing {b3.h_missed}. The earlier live model,
rain_in + rain_prev + margin + margin_max, falls to {old.h_fa_days} days in {old.h_fa_ep} episodes once
margin_max no longer leaks. The earlier crest-curve model, refit, catches {crest.h_caught}/{nb} with
{crest.h_fa_days} days in {crest.h_fa_ep} episodes; rain_in + later does {rl.h_fa_days} / {rl.h_fa_ep},
rain_in + rain_wy {rw.h_fa_days} / {rw.h_fa_ep}, and margin + peak24 {mp.h_fa_days} / {mp.h_fa_ep}.

**Overfitting check.** Repeating the whole search with each season held out (the best model of up
to three inputs chosen from the other seasons, alerts at its training cutoff) catches
**{S3.caught.sum()} of {S3.breaches.sum()}** with **{S3.fa_days.sum()} false-alarm days in {S3.fa_ep.sum()}
episodes**, missing {", ".join(m for m in S3.missed.dropna())}. It picks maxlvl or margin_max with a
rain total in every season, though which exact inputs win changes from season to season.
""")

2024-03-06 is the weakest breach for the crest-curve model and most others: a berm about two weeks old that drained from about 6.0 ft after 0.75 in of rain. 2025-03-13 is caught on the season-held-out check by the crest-curve model (peak score 0.12) and by most models.

Best models. The leaders all use maxlvl, the highest level this berm has already held: rain_in + rain_wy + log_dc + maxlvl catches 16/17 with 4 false-alarm days in 4 episodes, missing 2020-04-06. The best of up to three inputs is rain_in + later + maxlvl: 16/17, 5 days in 5 episodes, missing 2024-03-31. rain_in + rain_prev + margin + margin_max has 164 days in 20 episodes. The crest-curve model, refit, catches 16/17 with 474 days in 26 episodes; rain_in + later does 12 / 10, rain_in + rain_wy 10 / 9, and margin + peak24 429 / 28.

Overfitting check. Repeating the whole search with each season held out (the best model of up to three inputs chosen from the other seasons, alerts at its training cutoff) catches 15 of 17 with 13 false-alarm days in 7 episodes, missing 2022-12-31, 2024-03-06. It picks maxlvl or margin_max with a rain total in every season, though which exact inputs win changes from season to season.

Every bad result

Each missed breach and false-alarm episode of the crest-curve model, margin + peak24, rain_in + rain_prev + margin and rain_in + rain_wy was reviewed (analysis/berm/README.md, Figure 7). With all the corrections they fall into five causes:

Cause Cases Fix Fixed?
Wrong labels (unlogged breaches, closure dates) 2020-04-06, 2024-03-31, 2023-03-14, 2023-03-12 to 03-29 corrected labels (provisional) yes
Young, low berm 2024-03-07 (miss), 2024-02-21 to 22, 2026-02-09 to 16, 2019-03-22 the highest level held (maxlvl) with rain partly
Old tall berm near-miss in a big storm Dec 2019 (held at 9.7 ft), Dec 21 to 24, 2023 (2.4 in, held at 9.6 ft) none in the data no
Early warning counted as false 2025-02-06, 2020-03-11 report as early warnings n/a
Big rain far below the berm 2019-11-27, 2020-12-28, 2021-12-14, 2022-03-28, 2025-01-26 a berm term yes

2024-02-21 to 02-22 may be one more label error: the log has the berm closing on 2/20, but the level swung 0.7 to 1 ft a day, the logger skipped 2/22, and by 2/23 the slough was 2 ft lower and still swinging until 2/25. It is left as logged, pending review.

Code
CS = rd("cases.csv")
CS = CS[CS.check == "honest"]
CS[["model", "kind", "start", "end", "score", "cutoff", "stage", "crest", "margin", "dc", "rain_in", "trail24", "trail48", "rain_3d", "api", "later"]].rename(
    columns={"model": "Model", "kind": "Result", "start": "From", "end": "To", "score": "Score", "cutoff": "Cutoff",
             "stage": "Level", "crest": "Crest", "margin": "Gap", "dc": "Days closed", "rain_in": "Rain",
             "trail24": "24 h", "trail48": "48 h", "rain_3d": "3 day", "api": "API", "later": "Later"})
Model Result From To Score Cutoff Level Crest Gap Days closed Rain 24 h 48 h 3 day API Later
0 CREST-CURVE MODEL (stored, site) missed breach 2024-03-05 2024-03-06 0.0004 0.1027 5.28 7.41 -2.13 16.0 0.75 0.56 0.56 0.75 2.44 1
32 CREST-CURVE MODEL (stored, site) false alarm 2018-10-03 2018-10-09 0.0015 0.0004 4.70 6.43 -1.73 5.0 0.00 0.00 0.01 0.52 0.42 0
33 CREST-CURVE MODEL (stored, site) false alarm 2018-11-28 2019-01-02 0.0095 0.0004 6.12 8.62 -2.50 60.0 1.06 0.46 0.49 1.49 1.75 0
34 CREST-CURVE MODEL (stored, site) false alarm 2019-03-22 2019-05-02 0.0069 0.0004 5.96 6.31 -0.35 3.0 0.00 0.00 0.00 0.19 0.87 1
35 CREST-CURVE MODEL (stored, site) false alarm 2019-05-16 2019-06-10 0.0060 0.0004 6.71 8.62 -1.91 60.0 0.68 0.83 0.92 0.92 1.49 1
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
197 rain_in + rain_wy false alarm 2021-12-14 2021-12-14 0.0739 0.0427 6.10 10.27 -4.17 303.0 1.27 1.46 2.34 2.05 1.98 0
198 rain_in + rain_wy false alarm 2022-03-28 2022-03-28 0.0634 0.0427 6.22 8.93 -2.72 72.0 1.08 1.27 1.32 1.10 1.11 1
199 rain_in + rain_wy false alarm 2023-03-29 2023-03-29 0.0955 0.0427 5.10 6.91 -1.81 14.0 0.68 0.94 1.13 1.09 3.03 1
200 rain_in + rain_wy false alarm 2023-12-21 2023-12-21 0.7959 0.0588 7.02 10.01 -2.99 281.0 2.36 2.60 3.73 3.92 3.81 0
201 rain_in + rain_wy false alarm 2025-01-26 2025-01-26 0.8882 0.0427 6.50 10.16 -3.66 297.0 2.47 0.57 0.57 2.47 2.47 0

95 rows × 16 columns

Code
BP.false_alarm_grid(BP.FALSE_ALARMS);
Figure 7: The false alarms that remain. Hourly level, daily rain and tide (observed, same axis) around three false-alarm episodes (shaded). December 2019 and December 2023: old berms held the slough at 9.6 to 9.7 ft through big storms. February 2026: a young berm held 6.8 ft for a week, three days before it breached at 7.9 ft (red line).

Fixes

Each fix was added to the simple models and judged on the season-held-out check (corrected labels; the old-label results are in analysis/berm/results/old_labels/):

Code
keep = ["margin + peak24", "margin_mx + peak24", "margin + peak24 + rise_pos", "margin + peak24 + rain_x_gap",
        "margin + peak24 + later", "stage + log_dc + maxlvl + peak24", "rain_in + margin", "rain_in + rain_prev + margin",
        "CREST-CURVE MODEL (stored, site)", "CREST-CURVE MODEL (refit, measured days)", "rain_in + rain_wy"]
FF = F.set_index("model").loc[keep]
pd.DataFrame({"Fix": FF.fix.values,
              "Caught": [f"{r.h_caught}/{nb}" for r in FF.itertuples()],
              "False-alarm days / episodes": [f"{r.h_fa_days} / {r.h_fa_ep}" for r in FF.itertuples()],
              "Missed": FF.h_missed.fillna("").values}, index=pd.Index(keep, name="Model"))
Fix Caught False-alarm days / episodes Missed
Model
margin + peak24 base 16/17 429 / 28 2024-03-06
margin_mx + peak24 crest floor = max level 16/17 418 / 28 2024-03-06
margin + peak24 + rise_pos + rise_pos 16/17 379 / 30 2024-03-06
margin + peak24 + rain_x_gap + rain x gap 16/17 370 / 29 2024-03-06
margin + peak24 + later + later 16/17 635 / 28 2024-03-06
stage + log_dc + maxlvl + peak24 berm from data: stage + log_dc + maxlvl for ma... 16/17 55 / 12 2024-03-06
rain_in + margin base 15/17 153 / 18 2024-03-06;2024-03-31
rain_in + rain_prev + margin base 16/17 234 / 28 2024-03-06
CREST-CURVE MODEL (stored, site) reference 16/17 474 / 26 2024-03-06
CREST-CURVE MODEL (refit, measured days) reference 16/17 131 / 22 2024-03-06
rain_in + rain_wy reference (old best) 16/17 10 / 9 2024-03-06

With the 2024-03-07 breach in the record, margin-based fixes do not recover; the inputs that hold up are the highest level held since closure and a berm rebuilt from data (stage, berm age and that level), which are in the best search models above. Yesterday’s rise still adds false alarms, and the already-breached flag (later) helps only alongside the highest level held.

The crest curve and 2025-03-13

The crest curve assumes every berm starts at the same height and rebuilds at the same rate. Four low-parameter changes were refit inside the crest-curve model, each with seasons held out (crest_variants.py): (a) a starting crest that grows with how long the mouth was open before it closed (a mouth that re-closes soon after a breach builds a thin, low berm); (b) a starting crest set by each closure’s own level plus a fitted offset; (c) a slower rebuild for berms that close in February to April; (d) an uncertain crest whose spread shrinks as the berm ages, folded into the hazard. No waves, surge or tides.

Code
CV, CP = rd("crest_variants.csv"), rd("crest_variants_params.csv")
out = CV[["variant", "h_caught", "h_missed", "h_fa_days", "h_fa_ep", "peak_20250313", "crest_20250312"]].merge(
    CP.drop(columns=["b0", "b_rain", "b_rain3", "b_api"]), on="variant").round(3)
out.rename(columns={"variant": "Crest curve", "h_caught": "Caught", "h_missed": "Missed", "h_fa_days": "False-alarm days",
                    "h_fa_ep": "Episodes", "peak_20250313": "3/13/25 peak score", "crest_20250312": "Crest on 3/12/25 (ft)"}).fillna("").set_index("Crest curve")
Caught Missed False-alarm days Episodes 3/13/25 peak score Crest on 3/12/25 (ft) c_inf c0 tau_d b_margin k_open offset g_late s_young
Crest curve
base: global c0 (site) 16 2024-03-06 474 26 0.116 6.73 10.007 6.173 64.128 1.682
a: c0 + k*log(1 + days open before closure) 16 2024-03-06 395 28 0.112 6.71 10.046 6.817 58.805 1.618 -0.248
b: c0 = level at closure + offset 16 2024-03-06 1011 25 0.095 6.73 9.937 6.500 56.634 1.558 0.363
c: slower rebuild if closed Feb-Apr (tau * exp(g)) 16 2024-03-06 481 26 0.116 6.74 10.021 6.204 59.131 1.683 0.118
d: uncertain crest, sd = s*exp(-d/tau) 16 2024-03-06 156 25 0.112 7.18 9.659 6.818 74.239 3.385 1.066
Code
c = CV.set_index("variant")
base, late = c.iloc[0], c.iloc[3]
md(f"""
With all the corrections every version catches 2025-03-13 on the honest check and misses
2024-03-07 instead, which scores near zero on any crest curve and drags the cutoff down. The
uncertain crest (d) helps most ({c.iloc[4].h_fa_days} false-alarm days against {base.h_fa_days}),
the open-duration start (a) a little ({c.iloc[1].h_fa_days}), the late-season rebuild (c) not at all,
and closure-level starts (b) make it worse ({c.iloc[2].h_fa_days}). None comes close to the
berm-history models above. The fitted base curve now starts at {CP.iloc[0].c0:.2f} ft and rises
toward {CP.iloc[0].c_inf:.2f} ft with a {CP.iloc[0].tau_d:.0f}-day time constant (5.76 ft, 9.76 ft and 60
days in the committed model). The crest's shape is weakly pinned down by about 16 breaches; a
measured berm top (CCBER's 2024 COPR berm surveys, `data/berm_profiles` in their repository)
would test it.
""")

With all the corrections every version catches 2025-03-13 on the season-held-out check and misses 2024-03-07 instead, which scores near zero on any crest curve and drags the cutoff down. The uncertain crest (d) helps most (156 false-alarm days against 474), the open-duration start (a) a little (395), the late-season rebuild (c) not at all, and closure-level starts (b) make it worse (1011). None comes close to the berm-history models above. The fitted base curve now starts at 6.17 ft and rises toward 10.01 ft with a 64-day time constant (5.76 ft, 9.76 ft and 60 days in the committed model). The crest’s shape is weakly pinned down by about 16 breaches; a measured berm top (CCBER’s 2024 COPR berm surveys, data/berm_profiles in their repository) would test it.

A second rain gauge: CoCoRaHS CA-SB-1

CoCoRaHS station CA-SB-1 (Goleta 3.7 WNW) is about 2.5 km northwest of the slough, closer than the COP gauge the model uses. Its daily reports cover 24 hours ending about 07:00 and are credited to the report date, so sb1_rain.py spreads each report over the calendar days its window covers in proportion to KSBA’s hourly rain. The station’s rows are saved in analysis/berm/cocorahs_ca_sb_1.csv. It enters the search as its own inputs (sb1_rain_in, sb1_rain_2d) and as the larger of the two gauges (gmax_rain_in, gmax_rain_2d).

Code
SBc = rd("sb1_coverage.csv", index_col=0).value
SBs = rd("sb1_search_k3.csv").set_index("model")
pair = ["margin + margin_max + rain_2d", "margin + margin_max + sb1_rain_2d", "margin + margin_max + gmax_rain_2d",
        "margin + margin_max + trail24", "rain_in + later", "later + sb1_rain_in", "rain_in + rain_wy", "rain_wy + sb1_rain_in"]
md(f"""Since WY2019 the station is missing {SBc.days_missing} of {SBc.period_days} days ({SBc.missing_wet_season_days} in
November to April). On the {SBc.storm_days} storm days with at least 0.5 in at any gauge it caught
{float(SBc.ratio_sb1_cop):.2f} times the COP gauge's rain and {float(SBc.ratio_sb1_ksba):.2f} times KSBA's (correlation
{SBc.corr_sb1_cop} with COP). The same three-input models with each gauge (honest check):""")
SBs.loc[[p for p in pair if p in SBs.index], ["h_caught", "h_fa_days", "h_fa_ep", "h_missed"]].rename(columns=NICE)

Since WY2019 the station is missing 23 of 2971 days (7 in November to April). On the 115 storm days with at least 0.5 in at any gauge it caught 1.22 times the COP gauge’s rain and 1.33 times KSBA’s (correlation 0.901 with COP). The same three-input models with each gauge (season-held-out check):

Held out: caught Held out: false-alarm days Held out: episodes Held out: missed
model
margin + margin_max + rain_2d 16 123 16 2024-03-06
margin + margin_max + sb1_rain_2d 16 144 19 2024-03-06
margin + margin_max + gmax_rain_2d 16 53 21 2024-03-06
margin + margin_max + trail24 16 1707 24 2024-03-06
rain_in + later 16 12 10 2024-03-06
later + sb1_rain_in 16 14 12 2019-01-07
rain_in + rain_wy 16 10 9 2024-03-06
rain_wy + sb1_rain_in 16 18 15 2025-03-13
Code
SB3 = rd("select_nested_k3_sb1.csv")
a, b = SBs.loc["margin + margin_max + sb1_rain_2d"], SBs.loc["margin + margin_max + rain_2d"]
md(f"""With two days of CA-SB-1 rain, margin + margin_max catches {a.h_caught}/{nb} with {a.h_fa_days} false-alarm
days in {a.h_fa_ep} episodes (missing {a.h_missed}); with the COP gauge it catches {b.h_caught}/{nb} with {b.h_fa_days} /
{b.h_fa_ep} (missing {b.h_missed}). CA-SB-1 reports once a day at 07:00, so it cannot feed a live forecast. With the CA-SB-1 inputs among the candidates, the search held out
by season catches **{SB3.caught.sum()} of {SB3.breaches.sum()}** with **{SB3.fa_days.sum()} false-alarm days in
{SB3.fa_ep.sum()} episodes** (without them, {S3.caught.sum()} with {S3.fa_days.sum()} / {S3.fa_ep.sum()}).""")

With two days of CA-SB-1 rain, margin + margin_max catches 16/17 with 144 false-alarm days in 19 episodes (missing 2024-03-06); with the COP gauge it catches 16/17 with 123 / 16 (missing 2024-03-06). CA-SB-1 reports once a day at 07:00, so it cannot feed a live forecast. With the CA-SB-1 inputs among the candidates, the search held out by season catches 14 of 17 with 8 false-alarm days in 5 episodes (without them, 15 with 13 / 7).

When in the day: a breach-hour backtest

Once a breach is expected, the hour matters for anyone heading out to see it. For each of the 17 breaches with a logged apex, the breach time is the logged maximum before the drop (PST). The rain is Coal Oil Point’s hourly record (USCRN, PST), the site’s gauge, standing in for a perfect forecast. The storm is the run of wet hours (gaps of up to 6 h) the breach falls in or follows; from its first wet hour it runs forward on its own, so its end never depends on when the breach came. breach_hour.py tests simple rules, each fit with the breach’s season held out.

Code
BHs = rd("breach_hour_storms.csv")
BHp = rd("breach_hour_summary.csv").set_index("model")
md(f"""Breaches came a median {BHs.lag_onset_h.median():.0f} hours after the storm began and
{BHs.lag_peak3_h.median():.0f} hours after its wettest three hours, after a median {BHs.rain_to_breach_in.median():.2f} in of
storm rain. The rise per inch from the storm's start to the breach was {BHs.rise_per_in[BHs.start_level_ft < 7].median():.2f} ft
for slough levels under 7 ft and {BHs.rise_per_in[BHs.start_level_ft >= 7].median():.2f} ft above, in line with the larger
area at higher levels.""")
BHp.rename(columns={"n": "Breaches", "median_abs_error_h": "Median error (h)", "hit_3h": "Within 1.5 h",
                    "hit_4h": "Within 2 h", "hit_m1p3": "1 h before to 3 h after", "hit_pm4": "Within 4 h",
                    "hit_pm6": "Within 6 h", "worst_h": "Worst miss (h)"}).round(2)

Breaches came a median 17 hours after the storm began and 3 hours after its wettest three hours, after a median 1.93 in of storm rain. The rise per inch from the storm’s start to the breach was 0.74 ft for slough levels under 7 ft and 0.56 ft above, in line with the larger area at higher levels.

Breaches Median error (h) Within 1.5 h Within 2 h hit_pm3 1 h before to 3 h after Within 4 h Within 6 h Worst miss (h)
model
peak1 17 2.00 0.35 0.59 0.82 0.59 0.88 0.94 35.00
peak3 17 2.00 0.35 0.53 0.65 0.41 0.76 0.82 36.00
rainsum_shift 17 2.25 0.29 0.35 0.76 0.53 0.76 0.76 36.34
centroid 17 3.01 0.24 0.29 0.47 0.29 0.71 0.94 20.22
rainsum 17 3.41 0.29 0.41 0.47 0.47 0.65 0.82 38.00
Code
BP.breach_hour_fig();
Figure 8: Breach timing from the storm’s rain. Left: cumulative Coal Oil Point storm rain against hours to the breach for every breach with a logged apex (dashed line: the median storm rain at the breach). Right: errors of the site’s rule, the storm’s wettest hour plus the other seasons’ median lag, held out by season (misses beyond 12 h shown at the edge); dotted and dashed lines mark windows of 2 and 4 hours either side.
Code
import json as _json
BHJ = _json.loads((ROOT / "models" / "breach_hour.json").read_text())
p1 = BHp.loc["peak1"]
W, RW = BHJ["heldout"]["windows"], BHJ["heldout"]["recommended_window"]
md(f"""The site's rule takes the storm's wettest hour on Coal Oil Point's gauge and adds the typical lag
from the wettest hour to the breach. A storm with more than one burst uses its single wettest hour, as
long as the bursts are no more than 6 hours apart; a later burst after a longer dry spell is a new storm.
Held out by season (the lag from the other seasons' breaches), it lands within 2 hours of the breach for
{W['hit_4h']['hits']} of {W['hit_4h']['n']} breaches, within 3 hours for {W['hit_pm3']['hits']}, within 4 hours for
{W['hit_pm4']['hits']}, and from 1 hour before to 3 after for {W['hit_m1p3']['hits']}, with a median error of
{p1.median_abs_error_h:.1f} hours. Fit on all breaches the lag is **{BHJ['lag_h']:.0f} hours** (`models/breach_hour.json`).
The narrowest window that held at least three breaches in four runs from {abs(RW['from_h']):.0f} hours before to
{RW['to_h']:.0f} hours after the predicted time ({RW['hits']} of {RW['n']}). The misses are storms that kept raining
hard for a day or more, where the wettest hour came long after the breach.""")

The site’s rule takes the storm’s wettest hour on Coal Oil Point’s gauge and adds the typical lag from the wettest hour to the breach. A storm with more than one burst uses its single wettest hour, as long as the bursts are no more than 6 hours apart; a later burst after a longer dry spell is a new storm. Held out by season (the lag from the other seasons’ breaches), it lands within 2 hours of the breach for 10 of 17 breaches, within 3 hours for 14, within 4 hours for 15, and from 1 hour before to 3 after for 10, with a median error of 2.0 hours. Fit on all breaches the lag is 3 hours (models/breach_hour.json). The narrowest window that held at least three breaches in four runs from 2 hours before to 3 hours after the predicted time (13 of 17). The misses are storms that kept raining hard for a day or more, where the wettest hour came long after the breach.

Breach hour, head to head

A test of two ways to call the hour, on every breach with a logged apex (17), with the storm’s rain as a perfect forecast and nothing else from after the storm began (analysis/berm/breach_hour_compare.py). Every constant is fit with the breach’s water year held out. Rain comes from two gauges, both in PST: KSBA (the airport) and Coal Oil Point (USCRN Santa Barbara 11 W, hourly), the site’s rain source.

  • A, rain total: the hour the storm’s rain since onset reaches the median rain-to-breach of the other seasons, plus their median lead. The storm runs forward from its onset (wet hours, gaps up to 6 h) and does not know when the breach came.
  • B, crossing: the hourly level simulated from the last logged level before the storm, with the response fit on other seasons’ storms; the berm fails at the highest level it has held plus a delta fit on other seasons’ breaches. The hour is where the chance of having failed reaches 50%, on the median path or (Monte Carlo) on the spread of crossing times.
  • A+B: the average of the two.
Code
BC = rd("breach_hour_compare.csv", parse_dates=["breach", "predicted", "actual"])
BCs = rd("breach_hour_compare_summary.csv")
show = BCs.assign(**{"Within 3 h": BCs.hit3.astype(int).astype(str) + "/" + BCs.n.astype(str),
                     "Within 4 h": BCs.hit4.astype(int).astype(str) + "/" + BCs.n.astype(str)})
show = show.rename(columns={"gauge": "Rain gauge", "method": "Method", "median_abs_error_h": "Median error (h)", "worst_h": "Worst (h)"})
show[["Rain gauge", "Method", "Median error (h)", "Within 3 h", "Within 4 h", "Worst (h)"]].round(1)
Rain gauge Method Median error (h) Within 3 h Within 4 h Worst (h)
0 KSBA A rain total 2.9 4/17 4/17 9.3
1 KSBA B crossing 11.0 3/17 3/17 63.3
2 KSBA B crossing (MC median) 2.4 4/17 6/17 14.5
3 KSBA A+B average 7.4 2/17 2/17 34.6
4 KSBA A+B (MC) average 3.1 3/17 3/17 7.4
5 COP A rain total 2.2 5/17 6/17 36.3
6 COP B crossing 10.3 4/17 4/17 32.0
7 COP B crossing (MC median) 2.7 5/17 6/17 32.0
8 COP A+B average 6.1 3/17 5/17 34.2
9 COP A+B (MC) average 2.2 4/17 8/17 34.2
Code
BCst = rd("breach_hour_compare_storms.csv", parse_dates=["breach", "onset"])
on = BCst.set_index(["gauge", "breach"]).onset
meths = ["A rain total", "B crossing (MC median)", "A+B (MC) average", "B crossing"]
fig, axs = plt.subplots(2, 2, figsize=(8.5, 7.5), sharex=True, sharey=True)
for ax, m in zip(axs.flat, meths):
    for g, c in (("KSBA", "#2D5A7B"), ("COP", "#D3A43B")):
        x = BC[(BC.method == m) & (BC.gauge == g)].dropna(subset=["predicted"])
        o = on.reindex(list(zip(x.gauge, x.breach))).values
        xa = (x.actual.values - o) / pd.Timedelta(hours=1)
        ya = (x.predicted.values - o) / pd.Timedelta(hours=1)
        ax.scatter(xa, ya, s=22, color=c, label=g, alpha=0.85)
    ax.plot([0, 56], [0, 56], "k--", lw=0.8)
    ax.fill_between([0, 56], [-2, 54], [2, 58], color="0.85", zorder=0)
    ax.set_xlim(0, 56); ax.set_ylim(0, 60)
    ax.set_title(m, fontsize=9)
axs[1, 0].set_xlabel("actual: hours after onset"); axs[1, 1].set_xlabel("actual: hours after onset")
axs[0, 0].set_ylabel("predicted: hours after onset"); axs[1, 0].set_ylabel("predicted: hours after onset")
axs[0, 0].legend(fontsize=8)
plt.tight_layout(); plt.show()
Figure 9: Predicted vs actual breach hour. Hours from the storm’s onset to the logged apex (x) and to each method’s call (y), every constant fit with the breach’s season held out. Dashed: perfect; band: within 2 h (a 4 h window). Blue: KSBA rain; gold: Coal Oil Point rain. Bottom right: the crossing model’s median path, which often never crosses (then its highest point stands in).
Code
k = BCs.set_index(["gauge", "method"])
def line(g, m):
    r = k.loc[(g, m)]
    return f"{int(r.hit3)}/{int(r.n)} within 3 h, {int(r.hit4)}/{int(r.n)} within 4 h, median error {r.median_abs_error_h:.1f} h"
md(f"""
With each season held out, neither method calls the hour well. On Coal Oil Point rain, the site's source,
the rain total gets {line("COP", "A rain total")}; the Monte Carlo crossing gets
{line("COP", "B crossing (MC median)")}; their average {line("COP", "A+B (MC) average")}. On KSBA:
rain total {line("KSBA", "A rain total")}; crossing {line("KSBA", "B crossing (MC median)")}.
The crossing model's median path is worse ({line("KSBA", "B crossing")} on KSBA): for most breaches
the simulated level never quite reaches the fitted failure height, so it has no crossing to call.
Neither lands within 2 hours for more than about half the breaches. The site times
the breach from the storm's wettest hour (previous section), which does better held out.
""")

With each season held out, neither method calls the hour well. On Coal Oil Point rain, the site’s source, the rain total gets 5/17 within 3 h, 6/17 within 4 h, median error 2.2 h; the Monte Carlo crossing gets 5/17 within 3 h, 6/17 within 4 h, median error 2.7 h; their average 4/17 within 3 h, 8/17 within 4 h, median error 2.2 h. On KSBA: rain total 4/17 within 3 h, 4/17 within 4 h, median error 2.9 h; crossing 4/17 within 3 h, 6/17 within 4 h, median error 2.4 h. The crossing model’s median path is worse (3/17 within 3 h, 3/17 within 4 h, median error 11.0 h on KSBA): for most breaches the simulated level never quite reaches the fitted failure height, so it has no crossing to call. Neither lands within 2 hours for more than about half the breaches. The site times the breach from the storm’s wettest hour (previous section), which does better held out.

Which day gets the call

Garrett’s rule of thumb: if all the rain falls in the morning and the next day is dry, the breach comes the day it rains. The best model weighs yesterday’s rain (4.36 per inch) a little more than today’s (3.78), so its score the day after a storm stays close to the storm day’s. rain_timing.py and rain_timing2.py rebuild the daily rain from KSBA hourly rain in local standard time, in forms an hourly forecast can also give: 24-hour windows ending at 12:00 to 24:00 and the window before, rain before 16:00 plus the previous evening’s, and 6-hour blocks. Each is searched with the berm terms on the season-held-out check. Every breach’s apex falls on its labeled day in local standard time.

Code
RD = rd("rain_timing_day_after.csv")
md(f"""The last two columns use the {int(RD.n_pairs.iloc[0])} closed-mouth days with at least 0.5 in of rain followed by a
dry day: how often the rain day scores at least as high as the next day, and the median ratio of
the next day's score to the rain day's.""")
RD[["model", "h_caught", "h_fa_days", "h_fa_ep", "h_missed", "breach_peak_on_day", "rain_day_beats_next", "median_next_over_rain"]].rename(
    columns={**NICE, "breach_peak_on_day": "Breaches peaking on the day", "rain_day_beats_next": "Rain day scores higher",
             "median_next_over_rain": "Next day / rain day"}).set_index("Model")

The last two columns use the 17 closed-mouth days with at least 0.5 in of rain followed by a dry day: how often the rain day scores at least as high as the next day, and the median ratio of the next day’s score to the rain day’s.

Held out: caught Held out: false-alarm days Held out: episodes Held out: missed Breaches peaking on the day Rain day scores higher Next day / rain day
Model
rain_in + rain_wy + log_dc + maxlvl 16 4 4 2020-04-06 12 1.00 0.06
crest + maxlvl + rain_2d 16 25 9 2022-12-31 17 0.65 1.00
rain_in + crest + maxlvl + trail24 16 6 5 2024-03-06 13 1.00 0.02
rain_in + rain_prev + margin + margin_max 16 164 20 2024-03-06 15 0.65 0.65
rain_in + margin + margin_max 16 1868 31 2024-03-06 14 0.82 0.32
rain_in + prev_late16 + margin + margin_max 16 269 28 2024-03-06 16 0.82 0.04
rain_in + prev_late12 + margin + margin_max 16 240 25 2024-03-06 15 0.88 0.08
win24 + prev_late16 + margin + margin_max 16 166 17 2024-03-06 16 0.76 0.13
win18 + margin + margin_max + later 16 366 30 2024-03-06 16 0.82 0.07
win24 + margin + margin_max 16 1808 29 2024-03-06 14 0.53 0.45
eff + margin + margin_max 16 882 30 2024-03-06 17 0.65 0.10
win21 + margin + margin_max 16 876 24 2024-03-06 14 0.88 0.16
win18 + win18_prev + margin + margin_max 16 242 23 2024-03-06 17 0.65 0.30
rain_in + carry + margin + margin_max 16 1763 25 2024-03-06 14 0.88 0.45
eff + carry + margin + margin_max 16 373 35 2024-03-06 17 0.71 0.07
Code
RDi = RD.set_index("model")
bd = RDi.loc["rain_in + rain_wy + log_dc + maxlvl"]
om = RDi.loc["rain_in + rain_prev + margin + margin_max"]
md(f"""With margin_max corrected, the best model (rain_in + rain_wy + log_dc + maxlvl) uses only the day's
own rain, so it scores the rain day above the next day in {bd.rain_day_beats_next:.0%} of the wet-then-dry
storms; the earlier live model, which also weighs yesterday's rain, did so in {om.rain_day_beats_next:.0%}.
The best model's score peaks on the labeled breach day for {bd.breach_peak_on_day} of {nb} breaches; the
others drained in the early morning after a rain day (e.g. 2025-03-13 at 02:24), where it peaks on the
rain day itself. The hourly windows add nothing once the daily rain is the day's own.""")

The best model (rain_in + rain_wy + log_dc + maxlvl) uses only the day’s own rain, so it scores the rain day above the next day in 100% of the wet-then-dry storms; a model that also weighs yesterday’s rain (rain_in + rain_prev + margin + margin_max) does so in 65%. The best model’s score peaks on the labeled breach day for 12 of 17 breaches; the others drained in the early morning after a rain day (e.g. 2025-03-13 at 02:24), where it peaks on the rain day itself. The hourly windows add nothing once the daily rain is the day’s own.

Breach times and antecedent moisture

Breach times follow Garrett’s rule: the logged maximum level just before the breach drop, from the raw logger timestamps in local standard time (breach_apex.py). No breach day changes. Seven times move: 2021-01-28 from 10:00 to 15:30 (the old time came from a logger glitch at 10:00), 2024-03-31 from 03:00 to 04:24, 2024-03-06 from 18:00 to 18:54, 2023-03-14 from 14:00 to 14:45, 2023-03-10 about half an hour earlier, and 2020-04-06 from 02:00 to 02:15. 2022-12-31 moved to 23:45 in the logger QA below (its old 16:38 was a second logger failing). The event catalog and the breach-hour backtest use the corrected times.

Garrett’s read of March 2020: about two months without rain left the watershed dry, so the 3/10 to 3/14 storms barely raised the slough while the berm sat near its highest level. antecedent.py adds days since the last 0.25 in day, 30- and 60-day rain, the existing API, and rain terms scaled by 60-day rain or by wetness exp(-days/k). A term is kept only if it removes false-alarm days without adding any or losing a breach.

Code
AN = rd("antecedent.csv")
AN[["base", "added", "h_caught", "h_fa_days", "h_fa_ep", "h_missed", "removed", "new"]].rename(
    columns={"base": "Model", "added": "Added", "h_caught": "Honest caught", "h_fa_days": "False-alarm days",
             "h_fa_ep": "Episodes", "h_missed": "Missed", "removed": "False-alarm days removed", "new": "New false-alarm days"})
Model Added Held out: caught False-alarm days Episodes Missed False-alarm days removed New false-alarm days
0 best (none) 16 4 4 2020-04-06 0 0
1 best dsr 16 64 19 2020-04-06 3 63
2 best r30 16 16 8 2025-03-13 0 12
3 best r60 16 85 17 2020-04-06 1 82
4 best api 16 55 14 2025-03-13 0 51
5 best wet10 16 56 21 2020-04-06 2 54
6 best wet30 16 49 17 2020-04-06 3 48
7 best rain_in_x_r60 16 4 4 2020-04-06 2 2
8 best rain_in_x_wet10 16 66 17 2020-04-06 1 63
9 best rain_in_x_wet30 16 43 15 2020-04-06 3 42
10 win18 (none) 16 366 30 2024-03-06 0 0
11 win18 dsr 16 633 27 2024-03-06 33 300
12 win18 r30 15 656 29 2024-03-06;2024-03-31 0 290
13 win18 r60 15 2059 25 2024-03-06;2024-03-31 0 1693
14 win18 api 16 37 10 2024-03-06 330 1
15 win18 wet10 16 2059 24 2024-03-06 1 1694
16 win18 wet30 16 2058 24 2024-03-06 1 1693
17 win18 win18_x_r60 15 2058 24 2024-03-06;2024-03-31 1 1693
18 win18 win18_x_wet30 16 2058 24 2024-03-06 1 1693
Code
ANk = AN[AN.keep]
md("None passes." if ANk.empty else "Kept: " + ", ".join(ANk.base + " + " + ANk.added) + ".")
md("""Terms on the dry spell lower the March 2020 scores but cost a breach elsewhere or add false alarms
in other seasons (60-day rain, which the wet 2023 and 2024 seasons push high).""")

None passes.

Terms on the dry spell lower the March 2020 scores but cost a breach elsewhere or add false alarms in other seasons (60-day rain, which the wet 2023 and 2024 seasons push high).

An hourly threshold-crossing model

Garrett’s suggestion: model when the slough will pass the height the berm fails at, with the uncertainty turned into a chance for each day. crossing_model.py fits an hourly response to KSBA rain on the 45 closed-mouth storms (a gamma lag kernel and a rise per inch that shrinks with the lidar area curve), a breach threshold fit to the breach apex levels (the median, the crest curve plus an offset, the highest level held plus a margin, or a blend), and simulates 48 hours from the start of each day with draws over rain amount (lognormal, sd 0.4), rain timing (up to 3 hours either way) and the response. Each season’s response and threshold are fit on the others.

Code
CP6 = json.load(open(BR / "crossing_params.json"))
md(f"""The fitted response peaks {CP6['kernel_mode_h']:.1f} hours after the rain and rises
{CP6['r6_ft_per_in']:.2f} ft per inch at 6 ft, shrinking with area to the power {CP6['area_exponent']:.2f}
(the same damping the daily water balance found). The tightest threshold is the crest curve
(apex minus crest: offset {CP6['thresholds']['crest']['off']:+.2f} ft, spread {CP6['thresholds']['crest']['sd']:.2f} ft).""")

The fitted response peaks 1.7 hours after the rain and rises 0.96 ft per inch at 6 ft, shrinking with area to the power 0.25 (the same damping the daily water balance found). The tightest threshold is the crest curve (apex minus crest: offset +0.03 ft, spread 0.57 ft).

Garrett’s rule for every backtest: only the rain is known in advance, as a perfect forecast; everything else comes from data observed by the forecast time. The main crossing run uses the observed rain exactly (with the crest centre refit without the held-out season); the run with rain noise and timing jitter is kept as a sensitivity check. Combined with the daily model:

Code
CBp, CBn = rd("combine_backtest.csv"), rd("combine_backtest_noisy.csv")
CBp["rain"], CBn["rain"] = "perfect", "noisy"
CBx = pd.concat([CBp, CBn[CBn.form == "crossing only"]])
CBx[["form", "rain", "h_caught", "h_fa_days", "h_fa_ep", "h_missed", "breach_peak_on_day", "rain_day_beats_next"]].rename(
    columns={"form": "Form", "rain": "Rain", "h_caught": "Honest caught", "h_fa_days": "False-alarm days", "h_fa_ep": "Episodes",
             "h_missed": "Missed", "breach_peak_on_day": "Breaches peaking on the day", "rain_day_beats_next": "Rain day scores higher"})
Form Rain Held out: caught False-alarm days Episodes Missed Breaches peaking on the day Rain day scores higher
0 daily only perfect 16 4 4 2020-04-06 12 1.00
1 crossing only perfect 16 2062 27 2024-03-06 15 0.82
2 share perfect 16 9 8 2020-04-06 16 0.88
3 reweight perfect 16 16 12 2025-03-13 13 0.94
4 geomean perfect 15 2057 23 2024-03-06;2024-03-31 12 0.88
1 crossing only noisy 16 26 20 2024-03-06 16 0.82
Code
md(f"""The daily model alone (best_daily) already puts the call on the rain day in every wet-then-dry
storm and has the fewest false alarms; every combination with the crossing model adds false alarms.
The crossing model's hours are also no better than the wettest-hour rule ({CBp.windows.iloc[0]}). In the
1.2-inch morning scenario for 11/25/2026 the best daily model puts the call on 11/25 (0.0013 against
0.000008 on 11/26); with the NWS forecast (about 0.2 in) every version is near zero.""")

The daily model alone (best_daily) already puts the call on the rain day in every wet-then-dry storm and has the fewest false alarms; every combination with the crossing model adds false alarms. The crossing model’s hours are also no better than the wettest-hour rule (crossing hours: 3 h 3/17, 4 h 6/17; wettest-hour rule: 3 h 35%, 4 h 59%). In the 1.2-inch morning scenario for 11/25/2026 the best daily model puts the call on 11/25 (0.0013 against 0.000008 on 11/26); with the NWS forecast (about 0.2 in) every version is near zero.

The 2023-24 pier record

analysis/berm/data_gaps.md lists every gap of three days or more in the measured level record since WY2018 and what could fill it. One is now filled. CCBER’s repository has a pier logger record every 15 minutes from 2023-10-10 that was never downloaded because it is not compensated for air pressure. fetch_external.py now fetches it with KSBA hourly altimeter pressure (Iowa Environmental Mesonet, the source already used for KSBA rain), and loggers.py subtracts the pressure as meters of water and ties the result to CCBER’s compensated pier record over their first 60 days of overlap.

Code
import loggers
_, diag = loggers.compensate_uncomp(loggers.RAW / "PIER_10.10.23_08.29.25_Uncompensated.csv")
md(f"""The fitted offset is {diag['offset_m']*1000:.1f} mm and the slope against the compensated record
{diag['slope']:.4f}; over {diag['n']:,} matched 15-minute readings the two differ by a median of
{diag['resid_median_abs_ft']:.3f} ft (95th percentile {diag['resid_p95_abs_ft']:.3f} ft). The compensated
record stays the source from 2024-03-19 on; before that the new record is re-leveled to the
staff-gauge readings like every other logger segment. It adds WY2024's three logged breaches to the
measured-level days (@fig-pier-2024): the level-record apexes fall on CCBER's dates, at 08:40 on
1/22, 05:25 on 2/1 and 09:10 on 3/7. The 3/7 drain may have begun the evening of 3/6 (the level
peaks near 6.0 ft about 18:00 and falls from there, with tides from 3/7); CCBER's date is kept.""")

The fitted offset is 9.7 mm and the slope against the compensated record 0.9997; over 5,665 matched 15-minute readings the two differ by a median of 0.013 ft (95th percentile 0.036 ft). The compensated record stays the source from 2024-03-19 on; before that the new record is re-leveled to the staff-gauge readings like every other logger segment. It adds WY2024’s three logged breaches to the measured-level days (Figure 10): the level-record apexes fall on CCBER’s dates, at 08:40 on 1/22, 05:25 on 2/1 and 09:10 on 3/7. The 3/7 drain may have begun the evening of 3/6 (the level peaks near 6.0 ft about 18:00 and falls from there, with tides from 3/7); CCBER’s date is kept.

Code
BP.pier_2024_grid();
Figure 10: The three WY2024 breaches in the compensated pier record. Hourly level (ft NAVD88), daily rain and predicted tide around the 2024-01-22, 2024-02-01 and 2024-03-07 breaches (red lines mark the drain start; dashed where no time was dated).

The other gaps have nothing usable: WY2018 predates the loggers, spring 2021 has only the creek loggers upstream, and December 2022 has Venoco readings that were masked for swinging 4 ft a day.

Slough area from lidar

analysis/berm/dem_hypsometry.py builds the slough’s area and volume by level from the USGS 3DEP 1 m DEM (2018 Southern California wildfire lidar, flown May 27 to October 12, 2018; public, no sign-in), as the cells below each level connected to the main channel. The DEM is hydro-flattened: open water at flight time is one surface at 4.53 ft, so storage below that is missing. The basin it traces covers the lower slough and west arm; the north arm through NCOS is cut off at the road crossing in a bare-earth DEM, so its area is not included.

Code
BP.hypsometry_fig();
Figure 11: Slough area and volume by level, and what the loggers show. Left: area and volume from the 2018 lidar DEM (dotted line: the flattened water surface at 4.53 ft). Right: rise per inch of rain in closed-mouth storms by starting level, against pure volume over area (red) and the fitted damped form, area ratio to the 0.25 power (gold).
Code
HO = rd("dem_waterbalance_heldout.csv", index_col=[0, 1])
md(f"""Area grows about sevenfold from 5 to 9 ft, but the loggers' rise per inch falls only about
threefold, so the water balance with a pure volume-over-area term overcorrects. With the area ratio
raised to a fitted 0.25 power, storm peaks predicted with each season held out improve from
{HO.loc[('all', 'base'), 'rmse']:.2f} to {HO.loc[('all', 'dem_gamma'), 'rmse']:.2f} ft overall and from
{HO.loc[('start>=7ft', 'base'), 'rmse']:.2f} to {HO.loc[('start>=7ft', 'dem_gamma'), 'rmse']:.2f} ft for storms
starting above 7 ft (bias {HO.loc[('start>=7ft', 'base'), 'bias']:+.2f} to {HO.loc[('start>=7ft', 'dem_gamma'), 'bias']:+.2f}
ft; only {int(HO.loc[('start>=7ft', 'base'), 'n'])} such storms). It does not change the breach search
(in a side-by-side run the crest-curve model's false alarms changed by under 5% and the held-out
search did slightly worse), so the current form stays the default and the DEM form is optional: `WB_FORM=1 python
src/train.py` fits it (`models/hypsometry.csv`). The site's water balance would need the area curve
before it could use it.""")

Area grows about sevenfold from 5 to 9 ft, but the loggers’ rise per inch falls only about threefold, so the water balance with a pure volume-over-area term overcorrects. With the area ratio raised to a fitted 0.25 power, storm peaks predicted with each season held out improve from 0.50 to 0.46 ft overall and from 0.35 to 0.26 ft for storms starting above 7 ft (bias +0.19 to +0.00 ft; only 6 such storms). It does not change the breach search (in a side-by-side run the crest-curve model’s false alarms changed by under 5% and the held-out search did slightly worse), so the current form stays the default and the DEM form is optional: WB_FORM=1 python src/train.py fits it (models/hypsometry.csv). The site’s water balance would need the area curve before it could use it.

Overtopping and the event catalog

analysis/berm/events.py builds a catalog of every breach, closure and overtopping event from the hourly level record with the corrections applied (results/mouth_events.json and .csv, for the site). Overtopping is a closed-mouth step up in level at high tide with no rain: a rise of at least 0.05 ft over three hours ending within two hours of an observed high tide, still held six hours later (receding by no more than 70%), with no rain in the 12 hours before or that day, kept when one step is at least 0.15 ft or two or more tides add 0.15 ft (“step”). A rise of 0.3 ft or more at high tide that drains back within hours is kept too (“pulse”): water pushed over or through a berm closed to real outflow, as on 3/17/2023. The 3/22 to 24/2023 pulses came with rain and are left out by the no-rain rule. Tide is NOAA Santa Barbara (9411340) observed water level in ft NAVD88, drawn on the same axis as the slough (results/tide_hourly.csv); surge is observed minus predicted. Tide and surge describe events only and are never model inputs.

Code
import json
EV = pd.DataFrame(json.load(open(BR / "mouth_events.json")))
OV = EV[EV.type == "overtopping"].copy()
OV["year"] = OV.start_utc.str[:4]
BB = EV[EV.type == "breach"]
md(f"""The catalog holds {len(BB)} breaches ({(~BB.logged).sum()} not in the CCBER log: {int(BB.provisional.sum())} provisional,
{int(BB.reviewed.sum())} reviewed), {(EV.type == 'closure').sum()} closures and {len(OV)} overtopping events, {int(OV.summer.sum())} of them in
May to October. Summer overtopping by year: {", ".join(f"{y}: {n}" for y, n in OV[OV.summer].groupby("year").size().items())}.""")
OV[OV.summer].sort_values("total_rise_ft", ascending=False).head(10)[["start_utc", "total_rise_ft", "max_step_ft", "peak_level_ft", "tide_high_ft", "surge_ft", "sea_level_anomaly_ft"]].rename(
    columns={"start_utc": "Start (UTC)", "total_rise_ft": "Added (ft)", "max_step_ft": "Largest step (ft)", "peak_level_ft": "Slough (ft)",
             "tide_high_ft": "High tide, observed (ft)", "surge_ft": "Surge (ft)", "sea_level_anomaly_ft": "Month's sea-level anomaly (ft)"})

The catalog holds 20 breaches (4 not in the CCBER log: 0 provisional, 4 reviewed), 17 closures and 40 overtopping events, 18 of them in May to October. Summer overtopping by year: 2019: 1, 2020: 1, 2022: 1, 2023: 1, 2024: 1, 2025: 11, 2026: 2.

Start (UTC) Added (ft) Largest step (ft) Slough (ft) High tide, observed (ft) Surge (ft) Month's sea-level anomaly (ft)
56 2025-08-12T07:00Z 1.08 0.195 5.26 5.38 0.10 -0.06
14 2020-05-05T05:00Z 1.03 0.370 6.32 6.45 0.34 -0.00
66 2025-10-15T15:00Z 0.86 0.163 5.95 5.15 0.31 -0.08
65 2025-10-10T09:00Z 0.81 0.207 4.46 3.81 0.03 -0.08
58 2025-08-26T18:00Z 0.52 0.499 5.88 4.89 0.16 -0.06
75 2026-05-16T04:00Z 0.50 0.181 6.01 7.19 0.53 0.15
37 2023-05-05T05:00Z 0.41 0.079 6.47 6.06 0.30 0.14
61 2025-09-13T11:00Z 0.34 0.261 5.14 3.78 0.23 -0.04
59 2025-08-30T10:00Z 0.32 0.371 5.03 2.88 0.20 -0.06
57 2025-08-21T02:00Z 0.23 0.281 5.92 4.77 0.30 -0.06

Summer 2026 stands out for sea level rather than for the number of events. Tides at Santa Barbara ran above prediction all summer, by a monthly average of 0.15 ft in May rising to 0.5 ft in September and 0.94 ft in October, the highest of the record and consistent with El Nino-like warm water. The slough took 0.4 to 0.5 ft from 7.2 ft tides on 5/16 to 5/18, and on 10/11 a 7.3 ft tide, 1.5 ft above prediction, added about 0.2 ft. Summer 2025 had more events (August to October, steps up to 0.5 ft) on ordinary tides, which points to waves running up a lower berm.

Code
BP.overtopping_grid();
Figure 12: Tides topping a closed berm. Hourly slough level (blue) and observed tide (gray), both ft NAVD88 on the same axis, with daily rain. April 2020: the slough steps up at high tides before the 4/6 breach. August 2025: a summer step of 0.5 ft. May 2026: 7.2 ft tides running 0.5 ft above prediction add 0.4 to 0.5 ft over three days.
Code
md(f"""
**Summary.** The breach dates hold up against the hourly record. Garrett reviewed every label
correction: an unlogged breach on 2020-04-06 with the berm closed from 3/26; an earlier 2023
closure (3/12) with a brief opening on 3/14; the 2024 breach moved to the evening of 3/6; an
unlogged breach on 2024-03-31; and king-tide days on 2024-02-21 to 24 labeled leaky (not scored,
same berm). The 2023-24 pier record adds three logged breaches. With {nb} measured-level breaches
and the margin_max leak fixed, **{best.model}** catches {best.h_caught}/{nb} on the honest check with
{best.h_fa_days} false-alarm days in {best.h_fa_ep} episodes, against {crest.h_fa_days} days in
{crest.h_fa_ep} episodes for the refit crest-curve model; the search held out by season does
{S3.caught.sum()}/{S3.breaches.sum()} with {S3.fa_days.sum()} days in {S3.fa_ep.sum()} episodes. How high the
berm has already held the slough is the input that matters most.
""")

Summary. The breach dates hold up against the hourly record. Garrett reviewed every label correction: an unlogged breach on 2020-04-06 with the berm closed from 3/26; an earlier 2023 closure (3/12) with a brief opening on 3/14; the 2024 breach moved to the evening of 3/6; an unlogged breach on 2024-03-31; and king-tide days on 2024-02-21 to 24 labeled leaky (not scored, same berm). The 2023-24 pier record adds three logged breaches. With 17 measured-level breaches, rain_in + rain_wy + log_dc + maxlvl catches 16/17 on the season-held-out check with 4 false-alarm days in 4 episodes, against 474 days in 26 episodes for the refit crest-curve model; the search held out by season does 15/17 with 13 days in 7 episodes. How high the berm has already held the slough is the input that matters most.

Logger QA

Two stretches of the level record were logger faults, not water (data_gaps.md, “Logger QA”; plots results/gap_dec2022.png and results/qa_aug2020.png; reviewed by Garrett).

  • Dec 2022. The Venoco file interleaves two loggers from 12/15 to 1/12: the deployed one, out of the water at its 2.84 ft floor, and a second one that fails at 16:53 on 12/31. Their mixed daily mean swung about 4 ft, so the period had been masked, and the second logger’s failure had set the 12/31 breach time. No record shows tide before the breach, so the mouth was closed as logged. East Bridge recorded throughout but creeps up 0.073 ft/day on calm days; minus that creep and tied to the 12/9 staff reading (6.70 ft), it fills 2022-12-01 to 2023-01-08 (other staff readings within 0.14 to 0.35 ft). The 2022-12-31 breach now has a measured level and is scored (17 breaches, from 16), and its breach time is 23:45 (East Bridge apex), not 16:38.
  • Aug to Sep 2020. The Venoco logger slipped about 1.3 ft on 8/14 to 8/15 and read 2 ft above the staff gauge until it was serviced on 9/17. It is dropped there; East Bridge, rehung on 8/12, covers 8/12 to 9/1, and the staff readings cover the rest. The five summer 2020 overtopping events (8/16, 8/18, 8/29, 9/4, 9/13) were artifacts of the slip and are gone from the catalog.

Part 7. The model in use

The website runs the leader of the Part 6 search, a logistic model with four inputs (analysis/berm/export_best.py best_daily, models/best_daily.json).

Input Meaning
rain_in rain that day (in, Coal Oil Point gauge scale), the only input taken from the future, as a perfect forecast
rain_wy rain since October 1 of the water year, through that day (in)
log_dc natural log of the berm’s age in days since the mouth closed
maxlvl the highest measured daily mean level since the mouth closed, through the day before (ft NAVD88)

There is no same-day level term: the score rests on how high this berm has already held the slough, how old it is, how wet the season has been, and the day’s rain. Tide, surge and waves are not inputs. Everything below uses the reviewed labels and the measured-level closed-mouth days of Parts 4 to 6.

Code
import json
import engine as BE
LIVE = ["rain_in", "rain_wy", "log_dc", "maxlvl"]
LJ = json.loads((ROOT / "models" / "best_daily.json").read_text())
FA = BE.with_crest(BE.load(), "all")
FN = BE.with_crest(BE.load(), "nest")
sc7 = BE.Scorer(FN)
p7 = sc7.loyo(FN[LIVE].values.astype(float))
H7 = json.loads((BR / "heldout_best.json").read_text())
assert np.abs(np.array([v for _, v in H7["heldout"]]) - p7).max() < 1e-4
peaks7 = pd.Series({b: p7[ix].max() for b, ix in zip(sc7.breaches, sc7.peak_pos)})
n7, nb7 = len(FN), int(FN.breach.sum())
md(f"**{n7:,} measured-level closed-mouth days and {nb7} breaches**, WY{FN.water_year.min()} to WY{FN.water_year.max()}.")

2,404 measured-level closed-mouth days and 17 breaches, WY2019 to WY2026.

The fit on all measured days

The model is fit by maximum likelihood on every measured-level day. The 95% intervals are Wald intervals; odds ratios are per unit of each input.

Code
X7 = sm.add_constant(FA[LIVE].astype(float))
f7 = sm.Logit(FA.breach.values, X7).fit(disp=0, maxiter=500)
ci7 = f7.conf_int()
site7 = pd.Series({"const": LJ["fit"]["intercept"], **LJ["fit"]["coef"]})
T7 = pd.DataFrame({"Coefficient": f7.params, "SE": f7.bse, "95% CI low": ci7[0], "95% CI high": ci7[1],
                   "Odds ratio": np.exp(f7.params), "OR 95% low": np.exp(ci7[0]), "OR 95% high": np.exp(ci7[1]),
                   "p": f7.pvalues, "Site coefficient": site7}).rename(index={"const": "intercept"})
T7.round(4)
Coefficient SE 95% CI low 95% CI high Odds ratio OR 95% low OR 95% high p Site coefficient
intercept -14.8991 3.3459 -21.4571 -8.3412 0.0000 0.0000 0.0002 0.0000 -15.0101
rain_in 4.3366 0.6688 3.0258 5.6473 76.4437 20.6099 283.5349 0.0000 4.3462
rain_wy 0.0501 0.0686 -0.0843 0.1846 1.0514 0.9191 1.2027 0.4648 0.0548
log_dc -1.8700 0.4999 -2.8498 -0.8903 0.1541 0.0579 0.4105 0.0002 -1.8586
maxlvl 2.0994 0.5224 1.0756 3.1232 8.1612 2.9317 22.7191 0.0001 2.1000
Code
md(f"""
A tenth of an inch of rain multiplies the odds by about {np.exp(f7.params['rain_in'] / 10):.2f}, and a tenth
of a foot more held by the berm by {np.exp(f7.params['maxlvl'] / 10):.2f}. An older berm scores lower at the
same held level (log_dc {f7.params['log_dc']:.2f}), and a wetter season a little higher (rain_wy
{f7.params['rain_wy']:.3f} per inch). The intervals are wide because there are only {nb7} breaches
({nb7 / 4:.0f} per input). The site's coefficients come from the same unpenalized fit run with scikit-learn.
""")

A tenth of an inch of rain multiplies the odds by about 1.54, and a tenth of a foot more held by the berm by 1.23. An older berm scores lower at the same held level (log_dc -1.87), and a wetter season a little higher (rain_wy 0.050 per inch). The intervals are wide because there are only 17 breaches (4 per input). The site’s coefficients come from the same unpenalized fit run with scikit-learn.

Held-out score through time

Each day’s score comes from a fit that did not see its season. This is the series the Model tab shows (analysis/berm/results/heldout_best.json, from export_heldout.py).

Code
LV7 = [(l["key"].capitalize(), l["min"]) for l in LJ["levels"] if l["min"] > 0]
wys7 = sorted(FN.water_year.unique())
fig, axs = plt.subplots(len(wys7) // 2 + len(wys7) % 2, 2, figsize=(10, 9), sharey=True)
ps = pd.Series(p7, index=FN.index)
for ax, wy in zip(axs.flat, wys7):
    s = ps[FN.water_year.values == wy].asfreq("D")
    ax.plot(s.index, s.clip(lower=1e-5).values, lw=0.9, color="#2D5A7B")
    for name, lo in LV7:
        ax.axhline(lo, ls="--", lw=0.6, color="#A65A3C" if lo == LV7[0][1] else "#888")
    bb = [b for b in sc7.breaches if FN.water_year[b] == wy]
    ax.scatter(bb, [max(ps[b], 1e-5) for b in bb], s=36, facecolors="none", edgecolors="#C0392B", lw=1.4, zorder=3)
    ax.set_yscale("log"); ax.set_ylim(1e-5, 1.5)
    ax.set_xlim(pd.Timestamp(f"{wy - 1}-10-01"), pd.Timestamp(f"{wy}-09-30"))
    ax.set_title(f"WY{wy}", fontsize=9, loc="left")
    ax.xaxis.set_major_formatter(plt.matplotlib.dates.DateFormatter("%b"))
    ax.tick_params(labelsize=7)
for ax in list(axs.flat)[len(wys7):]:
    ax.axis("off")
fig.tight_layout();
Figure 13: Held-out daily score, by water year. Each point is a measured-level closed-mouth day scored by the model fit without that season (log scale). Red circles mark breach days; dashed lines mark the alert levels. Gaps are open-mouth days and days without a measured level.

Each breach

The held-out peak is the highest held-out score on the breach day or the day before.

Code
pd.DataFrame({"Breach": [str(b.date()) for b in peaks7.index], "Held-out peak": peaks7.values.round(4),
              "Rain (in)": FN.rain_in[peaks7.index].values, "Season rain (in)": FN.rain_wy[peaks7.index].values.round(2),
              "Days closed": FN.dc[peaks7.index].values.astype(int), "Highest held (ft)": FN.maxlvl[peaks7.index].values.round(2),
              "Level (ft)": FN.stage[peaks7.index].values.round(2)}).set_index("Breach")
Held-out peak Rain (in) Season rain (in) Days closed Highest held (ft) Level (ft)
Breach
2019-01-07 0.6215 1.61 7.04 99 8.02 9.00
2020-03-16 0.7103 1.16 9.71 362 9.73 9.81
2020-04-06 0.1322 0.13 12.12 12 5.76 6.40
2021-01-28 0.8456 2.81 6.89 291 7.04 9.12
2021-12-23 0.9899 3.38 7.26 312 7.30 8.91
2022-12-31 0.7065 1.96 7.50 350 8.47 9.22
2023-02-25 0.9279 0.44 18.31 22 7.18 8.15
2023-03-10 0.9846 2.06 21.50 7 5.77 6.64
2023-03-14 0.9993 2.54 24.09 3 5.37 6.39
2024-01-22 0.8668 1.62 8.29 313 9.67 10.30
2024-02-01 0.9755 2.52 11.36 4 4.81 6.08
2024-03-06 0.2009 0.75 22.26 16 6.96 5.28
2024-03-31 0.1386 0.71 25.17 17 6.32 7.05
2025-02-13 0.9378 1.75 7.60 315 9.45 10.25
2025-03-13 0.1359 0.20 9.33 11 6.03 6.58
2025-11-15 0.2647 1.74 5.78 245 7.44 9.35
2026-02-19 0.8088 1.00 21.00 14 7.56 7.93

Levels

The lowest level starts at the lowest held-out breach peak, rounded down to the nearest 0.5%; the others sit at 0.2 and 0.5. False-alarm days are days at or above the level more than two days from any breach; days separated by gaps of three days or less form one episode.

Code
rows = []
for name, lo in LV7:
    fa = np.where((p7 >= lo) & ~sc7.near)[0]
    rows.append({"Level": name, "Score at least": lo, "Breaches reaching it": f"{int((peaks7 >= lo).sum())}/{nb7}",
                 "False-alarm days": len(fa), "Episodes": len(sc7.episodes(fa))})
ep7 = sc7.episodes(np.where((p7 >= LV7[0][1]) & ~sc7.near)[0])
md(f"**False-alarm episodes at {LV7[0][0]}:** " + ("; ".join(f"{a.date()} to {b.date()} ({n} days)" for a, b, n in ep7) or "none"))
pd.DataFrame(rows).set_index("Level")

False-alarm episodes at Watch: 2019-12-22 to 2019-12-22 (1 days); 2020-01-16 to 2020-01-16 (1 days); 2020-03-10 to 2020-03-10 (1 days); 2023-12-21 to 2023-12-21 (1 days); 2025-01-26 to 2025-01-26 (1 days); 2026-02-16 to 2026-02-16 (1 days)

Score at least Breaches reaching it False-alarm days Episodes
Level
Watch 0.13 17/17 6 6
Warning 0.20 14/17 3 3
Likely 0.50 12/17 1 1

Against the other models

The search covered 6,886 models (every logistic model of one to five of the 16 candidate inputs, plus the crest-curve model, refit). The best ten, then two models for comparison: rain_in + rain_prev + margin + margin_max and the crest-curve model:

The season-held-out (cross-validated) check

The lowest level was set from all eight seasons’ breaches, so it has seen the season it scores. The season-held-out (cross-validated) check sets each season’s line only from the other seasons’ breaches.

Code
r7 = sc7.score(p7, detail=True)
hrows = []
for wy in sc7.seasons:
    c = r7["h_cut"][int(wy)]
    ks = np.where(sc7.bwy == wy)[0]
    fa = np.where((sc7.wys == wy) & (p7 >= c) & ~sc7.near)[0]
    hrows.append({"Season": f"WY{wy}", "Line (from other seasons)": round(c, 4), "Breaches": len(ks),
                  "Caught": int((peaks7.values[ks] >= c).sum()), "False-alarm days": len(fa),
                  "Episodes": len(sc7.episodes(fa))})
HT = pd.DataFrame(hrows).set_index("Season")
HT.loc["All"] = ["", HT.Breaches.sum(), HT.Caught.sum(), HT["False-alarm days"].sum(), HT.Episodes.sum()]
md(f"**Honest check: {r7['h_caught']} of {nb7} caught, {r7['h_fa_days']} false-alarm days in {r7['h_fa_ep']} episodes**, "
   f"missing {r7['h_missed']}.")
HT

Season-held-out check: 16 of 17 caught, 4 false-alarm days in 4 episodes, missing 2020-04-06.

Line (from other seasons) Breaches Caught False-alarm days Episodes
Season
WY2019 0.1322 1 1 0 0
WY2020 0.1359 2 1 2 2
WY2021 0.1322 1 1 0 0
WY2022 0.1322 1 1 0 0
WY2023 0.1322 4 4 0 0
WY2024 0.1322 4 4 1 1
WY2025 0.1322 2 2 1 1
WY2026 0.1322 2 2 0 0
All 17 16 4 4

Inputs known in advance

Only the day’s rain is taken from the future (a perfect forecast). Level, berm age, highest level held, the already-breached flag and every simulated level come from data through the day (or hour) before. leakage_test.py rebuilds every input with all breach labels moved a day earlier and later: no input on the breach days changes. The audit, input by input, is in analysis/berm/leakage_audit.md.

Summary

Code
su = score(pu)
md(f"""
**Model in use (Part 7).** The website runs **rain_in + rain_wy + log_dc + maxlvl**
(`models/best_daily.json`), fit on {n7:,} measured-level closed-mouth days with {nb7} breaches. With each
season's line set only from the other seasons it catches {r7['h_caught']} of {nb7} ({r7['h_missed']} missed) with
{r7['h_fa_days']} false-alarm days in {r7['h_fa_ep']} episodes. It replaces rain_in + rain_prev + margin +
margin_max, whose margin_max leaked the breach label (fixed; Part 7).
The crest-curve model below is the earlier model.

**Breach inputs (Part 1).** Among {len(res)} logistic models with up to four inputs, the best held-out performance came
from **{top.inputs}**: held-out log loss {top.cv_logloss:.4f} versus {clim['cv_logloss']:.4f} for a constant rate,
with {int(top.caught_p50)} of {n_ev} breaches caught at 50% and {int(top.false_alarm_days_p50)} false-alarm days.
The crest-curve model (nonlinear crest curve plus rain) scores {su['cv_logloss']:.4f} and catches {su['caught_p50']}
of {n_ev} with {su['false_alarm_days_p50']} false-alarm days. Tide height, surge and waves
{'appear' if any(c in top.inputs for c in ['tide_max', 'surge', 'hs_max', 'wave30']) else 'do not appear'} in the best model.
With {n_ev} events, differences of a few thousandths in log loss are within noise; the events-per-variable limit
and the bootstrap sign stability are the guardrails.

**Slough rise.** Across {len(E)} storms the best leave-one-storm-out model was **rise ~ {RSEL}**
(error {RS.LOO_RMSE_ft.iloc[0]:.2f} ft, versus {RM.set_index('model').loc['R0 intercept only', 'LOO_RMSE_ft']:.2f} ft
for the mean alone). The daily distributed-lag model, using {int(mb.nobs):,} closed-mouth days, puts the response at
{float(np.asarray(wald.effect)[0]):.2f} ft per inch of rain spread over {best_nl} day(s), plus {mb.params['hinge']:.2f} ft per
inch of a single day's rain above 0.5 in.
""")

Model in use (Part 7). The website runs rain_in + rain_wy + log_dc + maxlvl (models/best_daily.json), fit on 2,404 measured-level closed-mouth days with 17 breaches. With each season’s line set only from the other seasons it catches 16 of 17 (2020-04-06 missed) with 4 false-alarm days in 4 episodes.

Breach inputs (Part 1). Among 385 logistic models with up to four inputs, the best held-out performance came from rain_in + rain_3d + log_dc + stage: held-out log loss 0.0141 versus 0.0400 for a constant rate, with 10 of 17 breaches caught at 50% and 4 false-alarm days. The crest-curve model (nonlinear crest curve plus rain) scores 0.0233 and catches 12 of 17 with 8 false-alarm days. Tide height, surge and waves do not appear in the best model. With 17 events, differences of a few thousandths in log loss are within noise; the events-per-variable limit and the bootstrap sign stability are the guardrails.

Slough rise. Across 34 storms the best leave-one-storm-out model was rise ~ total + stage0 (error 0.33 ft, versus 0.90 ft for the mean alone). The daily distributed-lag model, using 2,365 closed-mouth days, puts the response at 0.66 ft per inch of rain spread over 2 day(s), plus 0.36 ft per inch of a single day’s rain above 0.5 in.