The unit of analysis for breaches is a closed-mouth day, WY2019 to WY2026: every day the mouth was closed at the start of the day, ending with the breach day itself. The outcome is whether the berm breached that day. All inputs are known by the end of that day, as they would be in a forecast. Breach days are dated by when the slough began to drain in the sub-daily water-level record, not the day CCBER logged the open mouth (src/breach_timing.py): three breaches that drained overnight after a rain day, or with no logger record and rain only the day before, count on that rain day.
Input
Meaning
rain_in
rain that day (in, Coal Oil Point gauge scale)
rain_3d
rain over the last 3 days
api
antecedent precipitation index (daily decay 0.9): how wet the watershed already is
largest observed-minus-predicted ocean level that day (ft): storm surge
hs_max
largest significant wave height at the Harvest buoy (m)
wave30
30-day mean wave power (H²T)
Code
risk = d[d.at_risk & d.dc.notna()].copy()risk["rain_in"] = risk.rain_in.fillna(0)risk["log_dc"] = np.log(risk.dc)risk = risk.rename(columns={"stage_est": "stage", "tide_pred_max_ft": "tide_max", "tide_resid_max_ft": "surge","hs_harvest_max_m": "hs_max", "wavepow_harvest_30d": "wave30"})CANDS = ["rain_in", "rain_3d", "api", "rain_wy", "log_dc", "stage", "tide_max", "surge", "hs_max", "wave30"]for c in CANDS: # a few wave/tide gaps: fill with the median risk[c] = risk[c].fillna(risk[c].median())y = risk.breach.valuesn_ev, n_days =int(y.sum()), len(risk)md(f"**{n_days:,} closed-mouth days and {n_ev} breaches** (base rate {n_ev / n_days:.2%} per day) "f"across {risk.water_year.nunique()} water years.")risk.groupby("breach")[CANDS].mean().T.rename(columns={0: "no breach (mean)", 1: "breach day (mean)"}).round(2)
2,554 closed-mouth days and 17 breaches (base rate 0.67% per day) across 8 water years.
breach
no breach (mean)
breach day (mean)
rain_in
0.03
1.55
rain_3d
0.07
2.52
api
0.26
3.10
rain_wy
12.04
13.25
log_dc
4.60
3.85
stage
5.62
8.03
tide_max
5.26
5.02
surge
0.38
0.76
hs_max
2.61
3.30
wave30
60.79
99.56
Part 1. Which inputs predict a breach?
Before fitting: sample size, collinearity, separation
Events per variable. With only 14 breaches, the usual guidance for logistic regression (5 to 10 events per estimated coefficient) supports about 2 to 3 predictors. Larger models overfit. So the search below caps models at 4 predictors and ranks them by leave-one-water-year-out cross-validation rather than in-sample fit.
Code
X = risk[CANDS]corr = X.corr()fig, ax = plt.subplots(figsize=(7, 5.5))im = ax.imshow(corr, cmap="RdBu_r", vmin=-1, vmax=1)ax.set_xticks(range(len(CANDS)), CANDS, rotation=45, ha="right"); ax.set_yticks(range(len(CANDS)), CANDS)for i inrange(len(CANDS)):for j inrange(len(CANDS)): ax.text(j, i, f"{corr.iloc[i, j]:.2f}", ha="center", va="center", fontsize=7)fig.colorbar(im, ax=ax, shrink=0.8); ax.set_title("Correlation between candidate inputs")plt.tight_layout(); plt.show()Xc = sm.add_constant((X - X.mean()) / X.std())vif = pd.Series([variance_inflation_factor(Xc.values, i) for i inrange(1, Xc.shape[1])], index=CANDS, name="VIF")vif.round(2).to_frame().T
rain_in
rain_3d
api
rain_wy
log_dc
stage
tide_max
surge
hs_max
wave30
VIF
2.7
3.86
2.84
1.65
1.5
1.79
1.07
1.16
1.36
2.07
Code
sep = []for c in CANDS: lo_b, hi_b = risk.loc[y ==1, c].min(), risk.loc[y ==1, c].max() sep.append(dict(input=c, breach_min=lo_b, breach_max=hi_b, nonbreach_days_in_breach_range=int(((risk[c] >= lo_b) & (risk[c] <= hi_b) & (y ==0)).sum())))sep = pd.DataFrame(sep).round(2)md("**Separation.** No single input splits breach days from the rest: each breach-day range overlaps ""many non-breach days (table), so maximum-likelihood estimates exist. "f"**Collinearity.** The highest VIF across all ten inputs is {vif.max():.1f} (`{vif.idxmax()}`); ""VIFs above about 5 flag redundancy, so pairs of strongly correlated inputs (the rain measures) ""share information and should not both be read as independent effects.")sep
Separation. No single input splits breach days from the rest: each breach-day range overlaps many non-breach days (table), so maximum-likelihood estimates exist. Collinearity. The highest VIF across all ten inputs is 3.9 (rain_3d); VIFs above about 5 flag redundancy, so pairs of strongly correlated inputs (the rain measures) share information and should not both be read as independent effects.
input
breach_min
breach_max
nonbreach_days_in_breach_range
0
rain_in
0.13
3.38
100
1
rain_3d
0.75
4.96
80
2
api
1.57
4.75
98
3
rain_wy
5.78
25.17
1417
4
log_dc
1.10
5.89
2520
5
stage
5.28
10.30
1467
6
tide_max
4.01
5.96
2059
7
surge
0.40
1.37
1024
8
hs_max
1.85
6.12
1951
9
wave30
44.00
181.08
1639
One input at a time
Code
def fit_logit(df, cols): Xm = sm.add_constant((df[cols] - df[cols].mean()) / df[cols].std())try:return sm.Logit(df.breach.values, Xm).fit(disp=0, maxiter=200)except np.linalg.LinAlgError: # near-separable: a vanishingly small ridge keeps it finitereturn sm.Logit(df.breach.values, Xm).fit_regularized(alpha=1e-4, disp=0, maxiter=500)null = sm.Logit(y, np.ones((n_days, 1))).fit(disp=0)rows = []for c in CANDS: m = fit_logit(risk, [c]) lr =2* (m.llf - null.llf) rows.append(dict(input=c, coef_per_SD=m.params[c], odds_ratio_per_SD=np.exp(m.params[c]), LR_chi2=lr, p_value=stats.chi2.sf(lr, 1), AUC=roc_auc_score(y, m.predict())))uni = pd.DataFrame(rows).sort_values("LR_chi2", ascending=False)uni.round({"coef_per_SD": 2, "odds_ratio_per_SD": 2, "LR_chi2": 1, "p_value": 4, "AUC": 3})
input
coef_per_SD
odds_ratio_per_SD
LR_chi2
p_value
AUC
1
rain_3d
0.90
2.46
116.3
0.0000
0.994
2
api
1.13
3.10
112.0
0.0000
0.990
0
rain_in
0.71
2.04
104.2
0.0000
0.992
5
stage
1.12
3.06
42.3
0.0000
0.894
7
surge
1.11
3.04
40.6
0.0000
0.894
9
wave30
0.92
2.51
22.1
0.0000
0.801
8
hs_max
0.50
1.65
6.6
0.0102
0.690
4
log_dc
-0.51
0.60
6.5
0.0106
0.557
6
tide_max
-0.36
0.70
2.1
0.1493
0.602
3
rain_wy
0.12
1.13
0.3
0.6081
0.534
Every combination of up to four inputs
Each model is fit eight times, leaving out one water year each time and predicting it. Scores use only those held-out predictions. A breach counts as caught when the predicted chance reaches 50% on the breach day or the day before; a false-alarm day is a day at 50% or more that is more than 2 days from any breach.
Code
years = risk.water_year.valuesbreach_days = d.index[d.breach ==1]near = np.zeros(n_days, bool)for b in breach_days: near |= (risk.index >= b - pd.Timedelta(days=2)) & (risk.index <= b + pd.Timedelta(days=2))ev_idx = [np.where((risk.index >= b - pd.Timedelta(days=1)) & (risk.index <= b))[0] for b in risk.index[y ==1]]def cv_predict(cols): p = np.zeros(n_days) Xa = risk[cols].valuesfor wy in np.unique(years): tr, te = years != wy, years == wy mu, sd = Xa[tr].mean(0), Xa[tr].std(0) +1e-12 m = LogisticRegression(penalty=None, max_iter=5000).fit((Xa[tr] - mu) / sd, y[tr]) p[te] = m.predict_proba((Xa[te] - mu) / sd)[:, 1]return pdef score(p): hits =sum(p[ix].max() >=0.5for ix in ev_idx)returndict(cv_logloss=log_loss(y, np.clip(p, 1e-6, 1-1e-6)), cv_brier=brier_score_loss(y, p), cv_AUC=roc_auc_score(y, p), cv_PR_AUC=average_precision_score(y, p), caught_p50=int(hits), false_alarm_days_p50=int(((p >=0.5) &~near).sum()))res = []for k inrange(1, 5):for cols in itertools.combinations(CANDS, k): cols =list(cols) m = fit_logit(risk, cols) res.append(dict(inputs=" + ".join(cols), k=k, AIC=m.aic, BIC=m.bic, **score(cv_predict(cols))))res = pd.DataFrame(res).sort_values("cv_logloss").reset_index(drop=True)clim = score(np.full(n_days, y.mean()))md(f"**{len(res)} models.** Baseline (a constant daily rate): held-out log loss {clim['cv_logloss']:.4f}, "f"Brier {clim['cv_brier']:.5f}. Top 15 by held-out log loss:")res.head(15).round({"AIC": 1, "BIC": 1, "cv_logloss": 4, "cv_brier": 5, "cv_AUC": 3, "cv_PR_AUC": 3})
385 models. Baseline (a constant daily rate): held-out log loss 0.0400, Brier 0.00661. Top 15 by held-out log loss:
inputs
k
AIC
BIC
cv_logloss
cv_brier
cv_AUC
cv_PR_AUC
caught_p50
false_alarm_days_p50
0
rain_in + rain_3d + log_dc + stage
4
64.6
93.9
0.0141
0.00350
0.990
0.668
10
4
1
rain_3d + log_dc + stage + tide_max
4
66.6
95.9
0.0144
0.00351
0.984
0.658
11
4
2
rain_in + log_dc + stage + hs_max
4
70.9
100.1
0.0144
0.00307
0.985
0.677
12
1
3
rain_in + log_dc + stage
3
70.8
94.2
0.0145
0.00314
0.985
0.657
12
2
4
rain_3d + log_dc + stage + hs_max
4
66.9
96.1
0.0145
0.00378
0.993
0.662
10
5
5
rain_in + log_dc + stage + tide_max
4
71.9
101.1
0.0146
0.00306
0.984
0.678
13
2
6
rain_3d + log_dc + stage + surge
4
70.3
99.5
0.0147
0.00384
0.990
0.631
9
5
7
rain_3d + log_dc + stage
3
68.3
91.7
0.0147
0.00382
0.989
0.633
9
5
8
rain_in + log_dc + stage + surge
4
72.7
102.0
0.0148
0.00314
0.985
0.637
12
2
9
rain_in + log_dc + stage + wave30
4
72.7
101.9
0.0150
0.00319
0.983
0.650
12
2
10
rain_3d + rain_wy + stage + tide_max
4
76.0
105.2
0.0153
0.00348
0.978
0.644
10
4
11
rain_in + rain_3d + rain_wy + stage
4
75.0
104.2
0.0153
0.00362
0.970
0.651
11
4
12
rain_in + api + stage + tide_max
4
81.1
110.3
0.0156
0.00374
0.987
0.634
9
3
13
rain_3d + log_dc + stage + wave30
4
69.9
99.1
0.0156
0.00397
0.983
0.585
9
5
14
rain_in + rain_3d + log_dc + hs_max
4
79.1
108.4
0.0158
0.00438
0.995
0.639
7
4
Code
best_k = res.loc[res.groupby("k").cv_logloss.idxmin()].sort_values("k")md("**Best model of each size.** Gains should flatten once the events-per-variable limit is reached.")best_k.round({"AIC": 1, "BIC": 1, "cv_logloss": 4, "cv_brier": 5, "cv_AUC": 3, "cv_PR_AUC": 3})
Best model of each size. Gains should flatten once the events-per-variable limit is reached.
inputs
k
AIC
BIC
cv_logloss
cv_brier
cv_AUC
cv_PR_AUC
caught_p50
false_alarm_days_p50
124
rain_3d
1
92.0
103.7
0.0187
0.00492
0.994
0.531
5
3
22
rain_3d + log_dc
2
82.4
100.0
0.0165
0.00432
0.993
0.603
9
4
3
rain_in + log_dc + stage
3
70.8
94.2
0.0145
0.00314
0.985
0.657
12
2
0
rain_in + rain_3d + log_dc + stage
4
64.6
93.9
0.0141
0.00350
0.990
0.668
10
4
Code
top25 = res.head(25)freq = pd.Series({c: top25.inputs.str.split(" \\+ ").apply(lambda L: c in L).mean() for c in CANDS}).sort_values(ascending=False)md("**How often each input appears among the 25 best models.** A robust input shows up regardless of ""what else is in the model.")freq.round(2).to_frame("share of top 25").T
How often each input appears among the 25 best models. A robust input shows up regardless of what else is in the model.
stage
rain_in
log_dc
rain_3d
api
tide_max
hs_max
rain_wy
surge
wave30
share of top 25
0.8
0.64
0.64
0.56
0.2
0.2
0.2
0.16
0.16
0.12
Code
U = pd.read_csv(ROOT /"data"/"processed"/"predictions_unified_measured.csv", parse_dates=["date"]).set_index("date")pu = U.p_unified_fb.reindex(risk.index).fillna(U.p_rain.reindex(risk.index)).valuestop = res.iloc[0]cmp= pd.DataFrame([dict(model="best linear subset: "+ top.inputs, **score(cv_predict(top.inputs.split(" + ")))),dict(model="website model: crest curve + rain (nonlinear)", **score(pu)),dict(model="rain only: rain_in + rain_3d + api + log_dc + rain_wy", **score(cv_predict(["rain_in", "rain_3d", "api", "log_dc", "rain_wy"]))),dict(model="constant rate", **clim)])md("**Against the crest-curve model the website used at this stage** (crest curve fit jointly with rain, `src/unified.py`; the model in use now is in Part 7), on the same held-out years and the same level estimate:")cmp.round({"cv_logloss": 4, "cv_brier": 5, "cv_AUC": 3, "cv_PR_AUC": 3})
Against the crest-curve model the website used at this stage (crest curve fit jointly with rain, src/unified.py; the model in use now is in Part 7), on the same held-out years and the same level estimate:
model
cv_logloss
cv_brier
cv_AUC
cv_PR_AUC
caught_p50
false_alarm_days_p50
0
best linear subset: rain_in + rain_3d + log_dc...
0.0141
0.00350
0.990
0.668
10
4
1
website model: crest curve + rain (nonlinear)
0.0233
0.00434
0.983
0.552
12
8
2
rain only: rain_in + rain_3d + api + log_dc + ...
0.0215
0.00609
0.993
0.505
7
10
3
constant rate
0.0400
0.00661
0.500
0.007
0
0
The selected linear model, examined
Code
SEL = top.inputs.split(" + ")msel = fit_logit(risk, SEL)Xsel = pd.DataFrame(msel.model.exog, columns=msel.model.exog_names)mclu = sm.Logit(y, Xsel).fit(disp=0, cov_type="cluster", cov_kwds={"groups": risk.water_year.values})def firth(Xm, yv, iters=200):"""Firth's penalized likelihood: reduces small-sample bias when events are few.""" b = np.zeros(Xm.shape[1])for _ inrange(iters): p = expit(Xm @ b); W = p * (1- p) inv = np.linalg.inv(Xm.T @ (Xm * W[:, None])) Xw = Xm * np.sqrt(W)[:, None] h = np.einsum("ij,jk,ik->i", Xw, inv, Xw) step = inv @ (Xm.T @ (yv - p + h * (0.5- p))) b += stepif np.abs(step).max() <1e-9: breakreturn bbf = pd.Series(firth(msel.model.exog, y), index=msel.model.exog_names)tab = pd.DataFrame({"coef (per SD)": msel.params, "SE": msel.bse, "SE clustered by water year": mclu.bse,"p (clustered)": mclu.pvalues, "odds ratio": np.exp(msel.params),"OR 95% CI low": np.exp(mclu.conf_int()[0]), "OR 95% CI high": np.exp(mclu.conf_int()[1]),"Firth coef": bf})md(f"**{top.inputs}.** Coefficients are per standard deviation of each input. Standard errors are ""also shown clustered by water year, because days in the same season are not independent; the ""Firth (bias-reduced) estimates check for small-sample bias. Close Firth and ordinary estimates ""mean the 14 events are enough for this model.")tab.round(3)
rain_in + rain_3d + log_dc + stage. Coefficients are per standard deviation of each input. Standard errors are also shown clustered by water year, because days in the same season are not independent; the Firth (bias-reduced) estimates check for small-sample bias. Close Firth and ordinary estimates mean the 14 events are enough for this model.
coef (per SD)
SE
SE clustered by water year
p (clustered)
odds ratio
OR 95% CI low
OR 95% CI high
Firth coef
const
-9.459
1.295
0.997
0.000
0.000
0.000
0.001
-8.714
rain_in
0.299
0.134
0.068
0.000
1.348
1.179
1.542
0.266
rain_3d
0.507
0.167
0.102
0.000
1.660
1.359
2.028
0.482
log_dc
-1.450
0.369
0.503
0.004
0.235
0.087
0.630
-1.301
stage
1.805
0.494
0.387
0.000
6.079
2.845
12.991
1.639
Code
rows = []for c in SEL: rest = [x for x in SEL if x != c] m0 = fit_logit(risk, rest) if rest else null lr =2* (msel.llf - m0.llf) rows.append(dict(dropped=c, LR_chi2=lr, df=1, p_value=stats.chi2.sf(lr, 1), AIC_increase=m0.aic - msel.aic))md("**Likelihood-ratio tests:** does each input earn its place?")pd.DataFrame(rows).round(4)
Likelihood-ratio tests: does each input earn its place?
dropped
LR_chi2
df
p_value
AIC_increase
0
rain_in
5.6267
1
0.0177
3.6267
1
rain_3d
8.2012
1
0.0042
6.2012
2
log_dc
21.2885
1
0.0000
19.2885
3
stage
17.8703
1
0.0000
15.8703
Linearity in the logit (Box–Tidwell)
Code
rows = []for c in SEL: v = risk[c] - risk[c].min() +1 df2 = risk.copy(); df2[c +"_xlogx"] = v * np.log(v) m2 = fit_logit(df2, SEL + [c +"_xlogx"]) lr =2* (m2.llf - msel.llf) rows.append(dict(input=c, LR_chi2=lr, p_value=stats.chi2.sf(lr, 1)))md("A small p-value means the input's effect on the log-odds is not linear and needs a transform.")pd.DataFrame(rows).round(4)
A small p-value means the input’s effect on the log-odds is not linear and needs a transform.
input
LR_chi2
p_value
0
rain_in
2.8411
0.0919
1
rain_3d
4.4941
0.0340
2
log_dc
2.1393
0.1436
3
stage
2.6439
0.1040
Calibration and discrimination (held-out predictions)
Code
psel = cv_predict(SEL)def hosmer_lemeshow(yv, p, g=10): q = pd.qcut(p, g, duplicates="drop") t = pd.DataFrame({"y": yv, "p": p, "g": q}).groupby("g", observed=True).agg(o=("y", "sum"), e=("p", "sum"), n=("y", "size")) chi = (((t.o - t.e) **2) / (t.e * (1- t.e / t.n) +1e-12)).sum()return chi, stats.chi2.sf(chi, len(t) -2)hl_sel, hlp_sel = hosmer_lemeshow(y, psel)hl_u, hlp_u = hosmer_lemeshow(y, pu)fig, axs = plt.subplots(1, 2, figsize=(9.5, 3.8))bins = np.r_[0, 0.01, 0.05, 0.2, 0.5, 1.0001]for p, lab in [(psel, "selected linear"), (pu, "website model")]: b = np.digitize(p, bins) -1 obs = [y[b == i].mean() if (b == i).any() else np.nan for i inrange(len(bins) -1)] pred = [p[b == i].mean() if (b == i).any() else np.nan for i inrange(len(bins) -1)] axs[0].plot(pred, obs, "o-", label=lab) f, t, _ = roc_curve(y, p); axs[1].plot(f, t, label=f"{lab} (AUC {roc_auc_score(y, p):.3f})")axs[0].plot([0, 1], [0, 1], ":", color="gray"); axs[0].set_xscale("symlog", linthresh=0.01); axs[0].set_yscale("symlog", linthresh=0.01)axs[0].set_xlabel("predicted daily chance"); axs[0].set_ylabel("observed breach rate"); axs[0].legend(fontsize=8); axs[0].set_title("Calibration")axs[1].plot([0, 1], [0, 1], ":", color="gray"); axs[1].set_xlabel("false positive rate"); axs[1].set_ylabel("true positive rate")axs[1].legend(fontsize=8); axs[1].set_title("ROC (held-out)")plt.tight_layout(); plt.show()md(f"Hosmer–Lemeshow on held-out predictions: selected model χ² = {hl_sel:.1f} (p = {hlp_sel:.3f}); "f"website model χ² = {hl_u:.1f} (p = {hlp_u:.3f}). With {n_ev} events this test has little power, ""so the calibration plot is the more useful check.")
Hosmer–Lemeshow on held-out predictions: selected model χ² = 2.4 (p = 0.964); website model χ² = 11.1 (p = 0.198). With 17 events this test has little power, so the calibration plot is the more useful check.
Influence, residual dependence, and stability
Code
glm = sm.GLM(y, msel.model.exog, family=sm.families.Binomial()).fit()cook = pd.Series(glm.get_influence().cooks_distance[0], index=risk.index)top_inf = risk.assign(cooks_d=cook, p_fit=glm.fittedvalues).sort_values("cooks_d", ascending=False).head(6)lb = acorr_ljungbox(glm.resid_pearson, lags=[1, 7], return_df=True)md(f"**Most influential days** (Cook's distance). Ljung–Box test on Pearson residuals in date order: "f"lag 1 p = {lb.lb_pvalue.iloc[0]:.3f}, lag 7 p = {lb.lb_pvalue.iloc[1]:.3f}. Where residuals are ""correlated in time, the water-year-clustered standard errors above are the ones to use.")top_inf[["breach", "p_fit", "cooks_d"] + SEL].round(3)
Most influential days (Cook’s distance). Ljung–Box test on Pearson residuals in date order: lag 1 p = 0.836, lag 7 p = 1.000. Where residuals are correlated in time, the water-year-clustered standard errors above are the ones to use.
breach
p_fit
cooks_d
rain_in
rain_3d
log_dc
stage
date
2023-03-22
0
0.725
0.178
0.19
2.14
1.946
8.138
2024-03-06
1
0.004
0.165
0.75
0.75
2.773
5.280
2023-03-21
0
0.852
0.142
1.59
1.99
1.792
7.361
2023-12-21
0
0.465
0.120
2.36
3.92
5.638
7.018
2023-12-22
0
0.314
0.104
0.24
3.77
5.642
8.698
2020-03-16
1
0.198
0.097
1.16
1.46
5.892
9.813
Code
rng = np.random.default_rng(1)wys = np.unique(years); boots = []for _ inrange(500): pick = rng.choice(wys, len(wys), replace=True) idx = np.concatenate([np.where(years == w)[0] for w in pick])if y[idx].sum() <3: continuetry: bb = sm.Logit(y[idx], msel.model.exog[idx]).fit(disp=0, maxiter=200).paramsif np.all(np.isfinite(bb)) and np.abs(bb).max() <50: boots.append(bb)exceptException:passboots = np.array(boots)md(f"**Water-year block bootstrap** ({len(boots)} resamples of whole seasons). An input whose sign ""flips in many resamples is not reliably estimated from this record.")pd.DataFrame({"coef": msel.params, "boot 2.5%": np.percentile(boots, 2.5, 0), "boot 97.5%": np.percentile(boots, 97.5, 0),"same sign in resamples": (np.sign(boots) == np.sign(msel.params.values)).mean(0)}).round(3)
Water-year block bootstrap (477 resamples of whole seasons). An input whose sign flips in many resamples is not reliably estimated from this record.
coef
boot 2.5%
boot 97.5%
same sign in resamples
const
-9.459
-20.774
-8.452
1.000
rain_in
0.299
0.010
0.487
0.977
rain_3d
0.507
0.401
1.776
1.000
log_dc
-1.450
-4.433
-0.669
1.000
stage
1.805
1.355
4.987
1.000
Part 2. How the slough responds to rain
Storm events
A storm starts on a day with at least 0.1 in of rain after two dry days and continues while rain keeps falling (one-day gaps allowed). Storms of at least 0.25 in qualify when the mouth stayed closed from the day before through three days after, and a logger or the live sensor measured the level on at least 80% of those days. The response is the rise: the highest daily-mean level from the start of the storm to three days after it ends, minus the level the day before.
Code
D = d.copy()meas = D.stage_mean_ft.where(D.stage_src.fillna("").str.endswith(("_logger", "_sensor")))r = D.rain_in.fillna(0); idx = D.index; n =len(D)rows, t = [], 0while t < n:if r.iloc[t] >=0.1and r.iloc[max(0, t -2):t].sum() <0.05: s = e = twhile e +1< n and (r.iloc[e +1] >=0.05or (e +2< n and r.iloc[e +2] >=0.05)): e +=1 w0, w1 = idx[s] - pd.Timedelta(days=1), idx[e] + pd.Timedelta(days=3) win = D.loc[w0:w1]; tot = r.iloc[s:e +1].sum()if tot >=0.25and (win.mouth =="closed").all() and meas.loc[w0:w1].notna().mean() >=0.8and pd.notna(meas.get(w0)): rows.append(dict(start=idx[s], total=tot, max_day=r.iloc[s:e +1].max(), duration=e - s +1, api0=D.api.get(w0), rain_wy0=D.rain_wy.get(w0), stage0=meas[w0], log_dc=np.log(D.days_closed.get(w0)), rise=meas.loc[idx[s]:w1].max() - meas[w0])) t = e +1else: t +=1E = pd.DataFrame(rows).set_index("start")E["total2"] = E.total **2md(f"**{len(E)} storms qualify** ({E.total.min():.2f} to {E.total.max():.2f} in). With {len(E)} events, ""models are limited to about three predictors.")fig, ax = plt.subplots(figsize=(6.5, 4))sc = ax.scatter(E.total, E.rise, c=E.api0, cmap="viridis"); fig.colorbar(sc, label="antecedent index (in)")ax.set_xlabel("storm rain (in)"); ax.set_ylabel("slough rise (ft)"); ax.set_title("Rise vs. storm rain, colored by watershed wetness")plt.tight_layout(); plt.show()E.describe().T[["mean", "std", "min", "max"]].round(2)
34 storms qualify (0.25 to 4.26 in). With 34 events, models are limited to about three predictors.
mean
std
min
max
total
1.15
0.91
0.25
4.26
max_day
0.79
0.55
0.23
2.47
duration
1.97
1.22
1.00
5.00
api0
0.39
0.58
0.00
2.45
rain_wy0
7.53
8.87
0.00
27.98
stage0
6.00
1.55
3.86
9.56
log_dc
4.38
1.61
0.69
5.87
rise
0.97
0.89
0.06
4.26
total2
2.13
3.46
0.06
18.15
Candidate models
Code
FORMS = {"R0 intercept only": "rise ~ 1","R1 total": "rise ~ total","R1o total, through origin": "rise ~ total - 1","R2 total + total²": "rise ~ total + total2","R3 total + max_day": "rise ~ total + max_day","R4 total + api0": "rise ~ total + api0","R5 total + stage0": "rise ~ total + stage0","R6 total + log_dc": "rise ~ total + log_dc","R7 total + duration": "rise ~ total + duration","R8 total + rain_wy0": "rise ~ total + rain_wy0","R9 total + api0 + stage0": "rise ~ total + api0 + stage0","R10 total + total² + api0": "rise ~ total + total2 + api0","R11 power law: log(rise) ~ log(total)": "np.log(rise + 0.05) ~ np.log(total)",}def loo_rmse(formula, df): errs = []for i inrange(len(df)): m = smf.ols(formula, df.drop(df.index[i])).fit() pred = m.predict(df.iloc[[i]]).iloc[0]if formula.startswith("np.log"): pred = np.exp(pred) -0.05 errs.append(df.rise.iloc[i] - pred)returnfloat(np.sqrt(np.mean(np.square(errs))))rows = []for name, f in FORMS.items(): m = smf.ols(f, E).fit() log_y = f.startswith("np.log") rows.append(dict(model=name, k=int(m.df_model), R2=m.rsquared, adj_R2=m.rsquared_adj, AIC=np.nan if log_y else m.aic, BIC=np.nan if log_y else m.bic, LOO_RMSE_ft=loo_rmse(f, E)))RM = pd.DataFrame(rows).sort_values("LOO_RMSE_ft").reset_index(drop=True)md("AIC and BIC are not comparable for the log-response model, so they are left blank; leave-one-storm-out error compares all of them.")RM.round(3)
AIC and BIC are not comparable for the log-response model, so they are left blank; leave-one-storm-out error compares all of them.
model
k
R2
adj_R2
AIC
BIC
LOO_RMSE_ft
0
R5 total + stage0
2
0.887
0.879
19.306
23.885
0.332
1
R4 total + api0
2
0.863
0.855
25.612
30.191
0.361
2
R1o total, through origin
1
0.927
0.925
27.687
29.213
0.373
3
R11 power law: log(rise) ~ log(total)
1
0.756
0.749
NaN
NaN
0.375
4
R1 total
1
0.838
0.833
29.399
32.452
0.384
5
R8 total + rain_wy0
2
0.847
0.837
29.442
34.022
0.390
6
R7 total + duration
2
0.841
0.830
30.835
35.414
0.393
7
R6 total + log_dc
2
0.839
0.829
31.111
35.690
0.394
8
R9 total + api0 + stage0
3
0.892
0.881
19.564
25.670
0.405
9
R3 total + max_day
2
0.838
0.828
31.317
35.896
0.407
10
R2 total + total²
2
0.848
0.838
29.289
33.868
0.449
11
R10 total + total² + api0
3
0.871
0.858
25.726
31.831
0.452
12
R0 intercept only
0
0.000
0.000
89.301
90.827
0.900
Code
PRED = ["total", "total2", "max_day", "duration", "api0", "rain_wy0", "stage0", "log_dc"]rows = []for k inrange(1, 4):for cols in itertools.combinations(PRED, k):if"total2"in cols and"total"notin cols: continue f ="rise ~ "+" + ".join(cols) m = smf.ols(f, E).fit() rows.append(dict(inputs=" + ".join(cols), k=k, adj_R2=m.rsquared_adj, AIC=m.aic, LOO_RMSE_ft=loo_rmse(f, E)))RS = pd.DataFrame(rows).sort_values("LOO_RMSE_ft").reset_index(drop=True)md(f"**All {len(RS)} subsets of up to three inputs**, ranked by leave-one-storm-out error:")RS.head(12).round(3)
All 70 subsets of up to three inputs, ranked by leave-one-storm-out error:
inputs
k
adj_R2
AIC
LOO_RMSE_ft
0
total + stage0
2
0.879
19.306
0.332
1
total + duration + stage0
3
0.880
20.032
0.341
2
total + rain_wy0 + stage0
3
0.884
18.954
0.342
3
total + total2 + stage0
3
0.887
17.919
0.344
4
total + stage0 + log_dc
3
0.879
20.327
0.345
5
total + max_day + stage0
3
0.875
21.289
0.356
6
total + api0
2
0.855
25.612
0.361
7
total + api0 + rain_wy0
3
0.853
26.908
0.365
8
total + api0 + log_dc
3
0.851
27.252
0.371
9
total + duration + api0
3
0.851
27.411
0.371
10
total + max_day + api0
3
0.850
27.514
0.379
11
total
1
0.833
29.399
0.384
The selected rise model, examined
Code
RSEL = RS.inputs.iloc[0]fr ="rise ~ "+ RSELmr = smf.ols(fr, E).fit()mr_hc3 = smf.ols(fr, E).fit(cov_type="HC3")Xr = mr.model.exogbp = het_breuschpagan(mr.resid, Xr)try: wt = het_white(mr.resid, Xr)exceptException: wt = (np.nan, np.nan)sw = stats.shapiro(mr.resid)reset = linear_reset(mr, power=2, use_f=True)vif_r = {c: variance_inflation_factor(Xr, i) for i, c inenumerate(mr.model.exog_names) if c !="Intercept"}md(f"**{fr}**, ordinary least squares with HC3 (heteroscedasticity-robust) standard errors. VIF: "+", ".join(f"{k}{v:.2f}"for k, v in vif_r.items()) +".")display(pd.DataFrame({"coef": mr.params, "SE (HC3)": mr_hc3.bse, "p (HC3)": mr_hc3.pvalues,"95% CI low": mr_hc3.conf_int()[0], "95% CI high": mr_hc3.conf_int()[1]}).round(3))pd.DataFrame([ ("Shapiro–Wilk (residuals normal)", sw.statistic, sw.pvalue), ("Breusch–Pagan (constant variance)", bp[0], bp[1]), ("White (constant variance, general)", wt[0], wt[1]), ("Ramsey RESET (functional form)", float(reset.fvalue), float(reset.pvalue)), ("Durbin–Watson (independence; about 2 is good)", durbin_watson(mr.resid), np.nan),], columns=["test", "statistic", "p_value"]).round(4)
rise ~ total + stage0, ordinary least squares with HC3 (heteroscedasticity-robust) standard errors. VIF: total 1.02, stage0 1.02.
The storm model rests on fewer than 30 events. Every closed-mouth day with a measured level adds far more data: the day’s change in level, regressed on that day’s and earlier days’ rain, rain above a threshold (bigger storms shed proportionally more runoff), and a seasonal evaporation term. Newey–West standard errors allow for day-to-day correlation.
Code
L = D.copy()L["meas"] = measL["dS"] = L.meas.diff()L["closed2"] = (L.mouth =="closed") & (L.mouth.shift(1) =="closed")for k inrange(0, 5): L[f"P{k}"] = r.shift(k)L["hinge"] = np.maximum(r -0.5, 0)L["season"] = np.cos(2* np.pi * (L.index.dayofyear -200) /365.25)Ld = L[L.closed2 & L.dS.notna() & L.P4.notna() & (L.dS.abs() <3)] # > 3 ft/day while closed = logger glitchrows, fits = [], {}for nl inrange(1, 6): f ="dS ~ "+" + ".join(f"P{k}"for k inrange(nl)) +" + hinge + season" m = smf.ols(f, Ld).fit(cov_type="HAC", cov_kwds={"maxlags": 7}) fits[nl] = m rows.append(dict(lags=f"0 to {nl -1}", n_days=int(m.nobs), adj_R2=m.rsquared_adj, AIC=m.aic, BIC=m.bic, ft_per_inch_lag_sum=sum(m.params[f"P{k}"] for k inrange(nl)), hinge_ft_per_inch=m.params["hinge"]))LG = pd.DataFrame(rows)best_nl =int(LG.BIC.idxmin()) +1mb = fits[best_nl]wald = mb.t_test(" + ".join(f"P{k}"for k inrange(best_nl)) +" = 0")ci = np.asarray(wald.conf_int())[0]md(f"BIC selects rain lags 0 to {best_nl -1}. Combined rise per inch of rain (sum of the lag coefficients): "f"**{float(np.asarray(wald.effect)[0]):.2f} ft per inch** (95% CI {ci[0]:.2f} to {ci[1]:.2f}), plus "f"**{mb.params['hinge']:.2f} ft** for each inch of a single day's rain above 0.5 in.")display(LG.round(4))pd.DataFrame({"coef": mb.params, "SE (Newey–West)": mb.bse, "p": mb.pvalues}).round(4)
BIC selects rain lags 0 to 1. Combined rise per inch of rain (sum of the lag coefficients): 0.66 ft per inch (95% CI 0.55 to 0.76), plus 0.36 ft for each inch of a single day’s rain above 0.5 in.
lags
n_days
adj_R2
AIC
BIC
ft_per_inch_lag_sum
hinge_ft_per_inch
0
0 to 0
2365
0.4354
-4584.6167
-4561.5426
0.3703
0.1966
1
0 to 1
2365
0.6878
-5984.9152
-5956.0725
0.6563
0.3623
2
0 to 2
2365
0.6878
-5983.8034
-5949.1922
0.6481
0.3622
3
0 to 3
2365
0.6883
-5986.5055
-5946.1257
0.6614
0.3645
4
0 to 4
2365
0.6882
-5984.7528
-5938.6046
0.6658
0.3642
coef
SE (Newey–West)
p
Intercept
-0.0051
0.0016
0.0014
P0
0.1751
0.0577
0.0024
P1
0.4812
0.0389
0.0000
hinge
0.3623
0.1229
0.0032
season
-0.0085
0.0020
0.0000
Part 3. First breach of the season versus later ones
The held-out predictions in Part 1 split cleanly by event type. The first breach of a water year comes after months behind a tall, wave-built berm; any later breach that season comes days or weeks after the mouth closed again, behind a thin, young berm. If those behave differently, a model that knows which kind of berm it is facing should do better. Here later = 1 on closed-mouth days after the season’s first breach.
Code
from scipy.special import expitfrom scipy.optimize import minimizefrom unified import Unifiedwy_of =lambda t: t.year + (t.month >=10)bd_all = d.index[d.breach ==1]bd_wy = np.array([wy_of(b) for b in bd_all])risk["later"] = [int(((bd_all < t) & (bd_wy == wy_of(t))).any()) for t in risk.index]risk["later_stage"] = risk.later * risk.stage# crest floor, as in src/unified.py (highest level in the first 3 days after closing, known from then on)closed = d.mouth.eq("closed")spell = (closed &~closed.shift(1, fill_value=False)).cumsum().where(closed)early = d.stage_est.where(closed & (d.days_closed <=3))risk["crest_floor"] = early.groupby(spell).cummax().groupby(spell).ffill().shift(1).reindex(risk.index)ev_dates = risk.index[y ==1]ev_later = risk.later.values[y ==1]evt = risk.loc[ev_dates, ["stage", "dc", "rain_in", "rain_3d", "rain_wy"]].assign(kind=np.where(ev_later, "later", "first"))mw = stats.mannwhitneyu(evt.stage[evt.kind =="first"], evt.stage[evt.kind =="later"])md(f"**{(ev_later ==0).sum()} first breaches and {(ev_later ==1).sum()} later ones.** Later breaches happen "f"{abs(evt.groupby('kind').stage.mean().diff().iloc[-1]):.1f} ft lower on average (Mann–Whitney p = {mw.pvalue:.4f}), "f"after a median of {evt[evt.kind =='later'].dc.median():.0f} days closed versus {evt[evt.kind =='first'].dc.median():.0f}. "f"`later` and log berm age correlate at r = {np.corrcoef(risk.later, risk.log_dc)[0, 1]:.2f} across all closed days.")evt.groupby("kind").agg(["mean", "min", "max"]).round(2)
8 first breaches and 9 later ones. Later breaches happen 2.8 ft lower on average (Mann–Whitney p = 0.0001), after a median of 12 days closed versus 312. later and log berm age correlate at r = -0.49 across all closed days.
stage
dc
rain_in
rain_3d
rain_wy
mean
min
max
mean
min
max
mean
min
max
mean
min
max
mean
min
max
kind
first
9.50
8.91
10.30
285.88
99.0
362.0
2.00
1.16
3.38
3.15
1.46
4.96
7.51
5.78
9.71
later
6.72
5.28
8.15
11.78
3.0
22.0
1.15
0.13
2.54
1.96
0.75
3.06
18.35
9.33
25.17
Candidate models
Same held-out scoring as Part 1, with catches split by event type. The website model is scored three ways: as shipped, refit here, and refit with one extra parameter that lowers (or raises) the crest for later berms on top of what berm age already does.
Code
class UnifiedFL(Unified): names = Unified.names + ["d_later"] x0 = np.r_[Unified.x0, -1.5]def z(self, X, p=None): p =self.p if p isNoneelse p crest = np.maximum(self.crest(X.dc, p), X.crest_floor.fillna(-np.inf)) + p[8] * X.laterreturn p[3] + p[4] * (X.stage - crest) + p[5] * X.rain_in + p[6] * X.rain_3d + p[7] * X.apidef fit(self, X, yv): yv = np.asarray(yv)def nll(p): pr = np.clip(expit(self.z(X, p)), 1e-9, 1-1e-9) prior = ((p[0] -9.8) /1.0) **2/2+ ((p[1] -6.5) /1.5) **2/2+ ((p[2] - np.log(70)) /1.0) **2/2\+0.05* np.sum(p[4:8] **2) + (p[8] /1.5) **2/2return-np.sum(yv * np.log(pr) + (1- yv) * np.log(1- pr)) + prior best =Nonefor start in (self.x0, self.x0 * [1, 1, 1, 1.5, 0.5, 2, 1, 1, 0.5]): r = minimize(nll, start, method="Nelder-Mead", options=dict(maxiter=20000, maxfev=20000, xatol=1e-5, fatol=1e-7)) r = minimize(nll, r.x, method="BFGS")if best isNoneor r.fun < best.fun: best = rself.p = best.xreturnselfdef cv_unified(cls): p = np.zeros(n_days)for wy in np.unique(years): tr, te = years != wy, years == wy p[te] = cls().fit(risk[tr], y[tr]).prob(risk[te])return pdef score_fl(p): out = score(p) hit = np.array([p[ix].max() >=0.5for ix in ev_idx]) hl, hlp = hosmer_lemeshow(y, p) out.update(first_caught=f"{hit[ev_later ==0].sum()}/{(ev_later ==0).sum()}", later_caught=f"{hit[ev_later ==1].sum()}/{(ev_later ==1).sum()}", HL_p=hlp)return outB3, B4 = ["rain_in", "rain_wy", "stage"], ["rain_in", "rain_3d", "rain_wy", "stage"]FLP = {"website model, as shipped": pu,"website model, refit": cv_unified(Unified),"website model + later-berm crest shift": cv_unified(UnifiedFL),"3 inputs (rain_in + rain_wy + stage)": cv_predict(B3),"3 inputs + later": cv_predict(B3 + ["later"]),"3 inputs + later + later x stage": cv_predict(B3 + ["later", "later_stage"]),"best linear (rain_in + rain_3d + rain_wy + stage)": cv_predict(B4),"best linear + later": cv_predict(B4 + ["later"]),"best linear + later + later x stage": cv_predict(B4 + ["later", "later_stage"]),}FL = pd.DataFrame([dict(model=k, **score_fl(v)) for k, v in FLP.items()])FL[["model", "cv_logloss", "cv_brier", "cv_AUC", "caught_p50", "first_caught", "later_caught", "false_alarm_days_p50", "HL_p"]].round( {"cv_logloss": 4, "cv_brier": 5, "cv_AUC": 3, "HL_p": 3})
model
cv_logloss
cv_brier
cv_AUC
caught_p50
first_caught
later_caught
false_alarm_days_p50
HL_p
0
website model, as shipped
0.0233
0.00434
0.983
12
7/8
5/9
8
0.198
1
website model, refit
0.0233
0.00434
0.983
12
7/8
5/9
8
0.195
2
website model + later-berm crest shift
0.0254
0.00445
0.974
12
7/8
5/9
8
0.000
3
3 inputs (rain_in + rain_wy + stage)
0.0168
0.00341
0.946
10
7/8
3/9
1
0.052
4
3 inputs + later
0.0170
0.00293
0.955
11
6/8
5/9
1
0.000
5
3 inputs + later + later x stage
0.0171
0.00296
0.952
12
7/8
5/9
1
0.000
6
best linear (rain_in + rain_3d + rain_wy + stage)
0.0153
0.00362
0.970
11
7/8
4/9
4
0.767
7
best linear + later
0.0173
0.00380
0.958
11
6/8
5/9
5
0.000
8
best linear + later + later x stage
0.0239
0.00435
0.963
12
7/8
5/9
6
0.000
Is the term real?
Code
rows = []for name, cols in (("3 inputs", B3), ("best linear", B4)): m0, m1, m2 = fit_logit(risk, cols), fit_logit(risk, cols + ["later"]), fit_logit(risk, cols + ["later", "later_stage"]) rows.append(dict(base=name, later_coef_per_SD=m1.params["later"], LR_p_later=stats.chi2.sf(2* (m1.llf - m0.llf), 1), LR_p_interaction=stats.chi2.sf(2* (m2.llf - m1.llf), 1)))u0 = Unified().fit(risk, y); u1 = UnifiedFL().fit(risk, y)nll_ =lambda m: -np.sum(y * np.log(np.clip(m.prob(risk), 1e-9, 1)) + (1- y) * np.log(np.clip(1- m.prob(risk), 1e-9, 1)))p_shift = stats.chi2.sf(2* (nll_(u0) - nll_(u1)), 1)# season-block bootstrap of the later coefficient in the 3-input modelrng = np.random.default_rng(3); wys = np.unique(years); bl = []for _ inrange(400): idx = np.concatenate([np.where(years == w)[0] for w in rng.choice(wys, len(wys), replace=True)]) sub = risk.iloc[idx]if sub.breach.sum() <3or sub.later.nunique() <2or sub.breach[sub.later ==1].sum() ==0:continuetry: bb = fit_logit(sub, B3 + ["later"]).params["later"]if np.isfinite(bb) andabs(bb) <50: bl.append(bb)exceptException:passbl = np.array(bl)md(f"In the crest-curve model the extra shift fits at **{u1.p[8]:+.2f} ft** (likelihood-ratio p = {p_shift:.2f}): once berm age "f"is in the crest curve, knowing the berm is a later one adds nothing. In the linear models, which have no berm age, "f"`later` is strongly significant, and its sign is positive in {(bl >0).mean():.0%} of {len(bl)} season-block bootstrap resamples.")pd.DataFrame(rows).round(4)
In the crest-curve model the extra shift fits at -0.33 ft (likelihood-ratio p = 0.75): once berm age is in the crest curve, knowing the berm is a later one adds nothing. In the linear models, which have no berm age, later is strongly significant, and its sign is positive in 100% of 399 season-block bootstrap resamples.
base
later_coef_per_SD
LR_p_later
LR_p_interaction
0
3 inputs
3.6512
0.0001
0.2025
1
best linear
3.6380
0.0002
0.0002
At the alert threshold
The site alerted at 30% at this stage. Catches and false-alarm days at that line, on held-out seasons:
Code
def at(p, thr): hit = np.array([p[ix].max() >= thr for ix in ev_idx]) miss = [str(b.date()) for b, h inzip(ev_dates, hit) ifnot h]returndict(caught=f"{hit.sum()}/{len(hit)}", missed=", ".join(miss), false_alarm_days=int(((p >= thr) &~near).sum()))AT = pd.DataFrame([dict(model=k, threshold=f"{t:.0%}", **at(FLP[k], t)) for k in ("website model, as shipped", "3 inputs + later") for t in (0.3, 0.5)])AT
model
threshold
caught
missed
false_alarm_days
0
website model, as shipped
30%
13/17
2020-03-16, 2020-04-06, 2024-03-06, 2025-03-13
8
1
website model, as shipped
50%
12/17
2020-03-16, 2020-04-06, 2024-03-06, 2024-03-31...
8
2
3 inputs + later
30%
13/17
2020-04-06, 2024-03-06, 2024-03-31, 2025-03-13
3
3
3 inputs + later
50%
11/17
2019-01-07, 2020-03-16, 2020-04-06, 2024-03-06...
1
What it means
Code
f = FL.set_index("model")a30 = AT.set_index(["model", "threshold"])md(f"""- **Later breaches really are different:** they happen about {abs(evt.groupby('kind').stage.mean().diff().iloc[-1]):.1f} ft lower, behind berms only days or weeks old.- **The crest-curve model already knows this,** through berm age: a separate later-berm shift fits at {u1.p[8]:+.2f} ft and is not significant (p = {p_shift:.2f}). With breaches dated by when the slough drained, it has the best held-out log loss ({f.loc['website model, refit', 'cv_logloss']:.4f}) and passes the calibration test.- **At the alert line the 3-input + later model has fewer false alarms:** at 30% it flags {a30.loc[('3 inputs + later', '30%'), 'caught']} breaches with {a30.loc[('3 inputs + later', '30%'), 'false_alarm_days']} false-alarm day(s), versus {a30.loc[('website model, as shipped', '30%'), 'caught']} with {a30.loc[('website model, as shipped', '30%'), 'false_alarm_days']} for the crest curve. Both miss the same two later breaches ({a30.loc[('3 inputs + later', '30%'), 'missed']}), young berms that broke at low levels after moderate rain.- **Its percentages are less trustworthy as odds** (it fails the calibration test), but with 14 breaches that test has little to stand on, and the site used the chance as a ranking with a single alert line.- **Next:** Part 4 refits on days with a measured level, which is how the site runs, and settles the inputs.""")
Later breaches really are different: they happen about 2.8 ft lower, behind berms only days or weeks old.
The crest-curve model already knows this, through berm age: a separate later-berm shift fits at -0.33 ft and is not significant (p = 0.75). With breaches dated by when the slough drained, it has the best held-out log loss (0.0233) and passes the calibration test.
At the alert line the 3-input + later model has fewer false alarms: at 30% it flags 13/17 breaches with 3 false-alarm day(s), versus 13/17 with 8 for the crest curve. Both miss the same two later breaches (2020-04-06, 2024-03-06, 2024-03-31, 2025-03-13), young berms that broke at low levels after moderate rain.
Its percentages are less trustworthy as odds (it fails the calibration test), but with 14 breaches that test has little to stand on, and the site used the chance as a ranking with a single alert line.
Next: Part 4 refits on days with a measured level, which is how the site runs, and settles the inputs.
Part 4. Days with a measured level
The site always starts from a measured level: yesterday’s mean from the live sensor. Earlier seasons sometimes had no logger running, and on those days the level above is a staff-gauge reading rolled forward with rain, or a filled-in typical value. Here the models are refit and scored only on closed-mouth days whose level was measured the day before. Breaches on days without one are left out rather than counted as misses. Season-to-date rain is dropped (its coefficient was not significant), and rain the day before is tested as a lag term.
Code
meas = d.stage_mean_ft.where(d.stage_src.fillna("").str.endswith(("_logger", "_sensor")))risk["measured"] = meas.shift(1).reindex(risk.index).notna()risk["rain_prev"] = d.rain_in.fillna(0).shift(1).reindex(risk.index)M = risk[risk.measured].copy()yM = M.breach.valuesexcluded = [str(b.date()) for b in risk.index[(risk.breach ==1) &~risk.measured]]evM = M.index[yM ==1]ixM = [np.where((M.index >= b - pd.Timedelta(days=1)) & (M.index <= b))[0] for b in evM]nearM = near[risk.measured.values]def cvM(cols): p = np.zeros(len(M)); Xa = M[cols].values.astype(float); yrs = M.water_year.valuesfor wy in np.unique(yrs): tr, te = yrs != wy, yrs == wy mu, sd = Xa[tr].mean(0), Xa[tr].std(0) +1e-12 m = LogisticRegression(penalty=None, max_iter=10000).fit((Xa[tr] - mu) / sd, yM[tr]) p[te] = m.predict_proba((Xa[te] - mu) / sd)[:, 1]return pdef atM(p, thr): hit = np.array([p[ix].max() >= thr for ix in ixM])returndict(caught=f"{hit.sum()}/{len(hit)}", missed=", ".join(str(b.date()) for b, h inzip(evM, hit) ifnot h), false_alarm_days=int(((p >= thr) &~nearM).sum()))FINAL = ["rain_in", "stage", "later"]V = {"rain that day + level + later (in use)": FINAL,"+ rain the day before": ["rain_in", "rain_prev", "stage", "later"],"+ rain since Oct 1": ["rain_in", "rain_wy", "stage", "later"],"+ both": ["rain_in", "rain_prev", "rain_wy", "stage", "later"]}PM = {k: cvM(v) for k, v in V.items()}PM["crest curve + rain (previous site model)"] = pu[risk.measured.values]T4 = pd.DataFrame([dict(model=k, cv_logloss=log_loss(yM, np.clip(p, 1e-6, 1-1e-6)), cv_AUC=roc_auc_score(yM, p),**{f"{k2}@30": v2 for k2, v2 in atM(p, 0.3).items()}, **{"caught@50": atM(p, 0.5)["caught"]})for k, p in PM.items()])md(f"**{len(M):,} days with a measured level, {int(yM.sum())} breaches.** Left out for lack of a measured level: "+", ".join(excluded) +".")T4.round({"cv_logloss": 4, "cv_AUC": 3})
2,404 days with a measured level, 17 breaches. Left out for lack of a measured level: .
model
cv_logloss
cv_AUC
caught@30
missed@30
false_alarm_days@30
caught@50
0
rain that day + level + later (in use)
0.0134
0.978
13/17
2020-04-06, 2024-03-06, 2024-03-31, 2025-03-13
0
12/17
1
+ rain the day before
0.0119
0.980
13/17
2020-03-16, 2020-04-06, 2024-03-06, 2025-03-13
2
13/17
2
+ rain since Oct 1
0.0150
0.964
13/17
2020-04-06, 2024-03-06, 2024-03-31, 2025-03-13
0
12/17
3
+ both
0.0130
0.966
13/17
2020-03-16, 2020-04-06, 2024-03-06, 2025-03-13
2
13/17
4
crest curve + rain (previous site model)
0.0125
0.984
13/17
2020-03-16, 2020-04-06, 2024-03-06, 2025-03-13
4
12/17
Code
mF = sm.Logit(yM, sm.add_constant(M[FINAL].astype(float))).fit(disp=0, maxiter=500)mP = sm.Logit(yM, sm.add_constant(M[V["+ rain the day before"]].astype(float))).fit(disp=0, maxiter=500)t4 = T4.set_index("model")md(f"Adding rain the day before is significant (p = {mP.pvalues['rain_prev']:.3f}) and improves the fit overall "f"(AIC {mP.aic:.1f} vs {mF.aic:.1f}), but at the 30% alert line it flags {t4.loc['+ rain the day before', 'caught@30']} "f"breaches against {t4.loc['rain that day + level + later (in use)', 'caught@30']}. The level input already carries ""yesterday's rain, because the water balance adds runoff over the two days after a storm, so the two overlap. ""The best of these simple models, for comparison with Part 5:")pd.DataFrame({"coef": mF.params, "SE": mF.bse, "95% CI low": mF.conf_int()[0], "95% CI high": mF.conf_int()[1],"odds ratio": np.exp(mF.params), "p": mF.pvalues}).round(4)
Adding rain the day before is significant (p = 0.000) and improves the fit overall (AIC 54.4 vs 65.2), but at the 30% alert line it flags 13/17 breaches against 13/17. The level input already carries yesterday’s rain, because the water balance adds runoff over the two days after a storm, so the two overlap. The best of these simple models, for comparison with Part 5:
coef
SE
95% CI low
95% CI high
odds ratio
p
const
-29.4093
5.4757
-40.1416
-18.6771
0.0000
0.0
rain_in
4.0029
0.6792
2.6717
5.3341
54.7551
0.0
stage
2.5454
0.5685
1.4312
3.6595
12.7481
0.0
later
7.3361
1.8016
3.8050
10.8672
1534.7189
0.0
Part 5. Catching every breach
The goal for alerts is to flag every breach on the measured-level record with as few false alarms as possible; there is no fixed cutoff. For each model the cutoff is set as low as the weakest breach requires (its highest score on the breach day or the day before), and false alarms are counted at that cutoff, as days and as episodes (one alert per episode). That cutoff is tuned on the same predictions it is scored on, so each model is also checked with each season held out (cross-validated): every season’s cutoff comes only from the other seasons’ breaches. Every combination of up to five rain, level and berm-age inputs is searched; tide, surge and wave inputs are left out (they do not drive breaches here), and the crest-curve model is scored alongside.
Code
C5 = ["rain_in", "rain_prev", "rain_3d", "api", "rain_wy", "log_dc", "stage", "later"]X5 = M[C5].astype(float).fillna(M[C5].median()).valuesyrsM = M.water_year.valuesevyr = np.array([yrsM[ix[-1]] for ix in ixM])def n_eps(idx): returnint((np.diff(np.r_[-99, idx]) >3).sum()) iflen(idx) else0def catch_all(p, name): peaks = np.array([p[ix].max() for ix in ixM]); thr = peaks.min() fa = np.where((p >= thr) &~nearM)[0] c, nfa, miss =0, [], []for w in np.unique(yrsM): t = peaks[evyr != w].min() hit = peaks[evyr == w] >= t c +=int(hit.sum()); miss += [str(evM[i].date()) for i in np.where(evyr == w)[0][~hit]] nfa +=list(np.where((p >= t) &~nearM & (yrsM == w))[0])returndict(model=name, cutoff=thr, fa_days=len(fa), fa_episodes=n_eps(fa), honest_caught=f"{c}/{len(peaks)}", honest_fa_days=len(nfa), honest_fa_episodes=n_eps(np.sort(nfa)), honest_missed=", ".join(miss), caught_50=int((peaks >=0.5).sum()), fa_days_50=int(((p >=0.5) &~nearM).sum()))rows5 = []for k inrange(1, 6):for cols in itertools.combinations(range(len(C5)), k): Xa = X5[:, cols]; p = np.zeros(len(M))for w in np.unique(yrsM): tr, te = yrsM != w, yrsM == w mu, sd = Xa[tr].mean(0), Xa[tr].std(0) +1e-12 p[te] = LogisticRegression(penalty=None, max_iter=5000).fit((Xa[tr] - mu) / sd, yM[tr]).predict_proba((Xa[te] - mu) / sd)[:, 1] rows5.append(catch_all(p, " + ".join(C5[c] for c in cols)))pcrest = pu[risk.measured.values]rows5.append(catch_all(pcrest, "crest curve (level vs rebuilding berm + rain_in + rain_3d + api)"))R5 = pd.DataFrame(rows5)md(f"**{len(R5)} models.** Best ten by the honest check (breaches caught, then false-alarm episodes, which is what alerts send):")R5.sort_values(["honest_caught", "honest_fa_episodes", "honest_fa_days"], ascending=[False, True, True]).head(10).round({"cutoff": 4})
219 models. Best ten by the season-held-out check (breaches caught, then false-alarm episodes, which is what alerts send):
model
cutoff
fa_days
fa_episodes
honest_caught
honest_fa_days
honest_fa_episodes
honest_missed
caught_50
fa_days_50
48
rain_in + api + log_dc
0.0417
30
9
16/17
28
8
2020-03-16
8
7
117
rain_in + api + rain_wy + log_dc
0.0471
29
8
16/17
29
8
2020-03-16
8
8
122
rain_in + api + stage + later
0.0156
44
9
16/17
30
8
2024-03-06
13
2
195
rain_in + api + log_dc + stage + later
0.0079
67
9
16/17
35
8
2024-03-06
13
2
87
api + stage + later
0.0075
69
11
16/17
52
8
2024-03-06
11
5
11
rain_in + rain_wy
0.0427
11
9
16/17
10
9
2024-03-06
8
2
53
rain_in + rain_wy + later
0.0425
11
9
16/17
10
9
2024-03-06
8
2
50
rain_in + api + later
0.0291
34
10
16/17
32
9
2020-03-16
9
7
121
rain_in + api + log_dc + later
0.0284
34
10
16/17
32
9
2020-03-16
8
7
14
rain_in + later
0.0322
12
10
16/17
12
10
2024-03-06
7
2
Code
cr = R5[R5.model.str.startswith("crest")].iloc[0]md(f"""No model catches all {len(evM)} breaches when the cutoff is set without seeing the season: with ten eventsthe weakest breach can't be anticipated from the others. The crest-curve model catches {cr.honest_caught}that way with {cr.honest_fa_episodes} false-alarm episodes, the fewest of the leaders, and it is the only onewhose upper range means much: {cr.caught_50} of {len(evM)} breaches reached 0.5, against 4 to 5 for the rain-onlymodels that score as well at the low end. Tuned on all seasons, a cutoff of {cr.cutoff:.3f} catches every breachwith {cr.fa_days} false-alarm days in {cr.fa_episodes} episodes. The site then adopted the crest-curve model,reporting levels rather than percentages with alerts at the lowest one. That is the earlier model: with thereviewed labels of Part 6, the site moved to rain_in + rain_prev + margin + margin_max (**Part 7**), which reportsNo Chance, Possible (every recorded breach reached it), Likely (0.2) and Imminent (0.5), with alerts at Possible.""")
No model catches all 17 breaches when the cutoff is set without seeing the season: with ten events the weakest breach can’t be anticipated from the others. The crest-curve model catches 16/17 that way with 25 false-alarm episodes, the fewest of the leaders, and it is the only one whose upper range means much: 12 of 17 breaches reached 0.5, against 4 to 5 for the rain-only models that score as well at the low end. In-sample, with one cutoff tuned on all seasons, a cutoff of 0.000 catches every breach with 557 false-alarm days in 29 episodes. The model the site runs is in Part 7; it reports No Chance, Possible (every recorded breach reached it), Likely (0.2) and Imminent (0.5), with alerts at Possible.
Part 6. Breach dates, berm height and rain windows
Three questions from the Part 5 search. Are the breach dates right? Does an estimate of the berm height (the crest curve, or the highest level the berm has held since it closed) help when combined with rain? And would rain over the prior 24 or 48 hours, rather than same-day rain, catch the weakest breach, 2025-03-13? The scripts are in analysis/berm/, the case notes in analysis/berm/README.md and the level-gap audit in analysis/berm/data_gaps.md; this part reads their saved tables (the searches take a few minutes each). Scoring is as in Part 5: measured-level days, leave-one-season-out fits, the catch-all cutoff at the weakest breach, and the season-held-out check with each season’s cutoff from the other seasons. Tide, surge and wave inputs are left out. Levels in the figures are hourly logger readings put on the daily composite’s NAVD88 datum (analysis/berm/hourly_level.py).
Code
sys.path.insert(0, str(ROOT /"analysis"/"berm"))import plots as BPBR = ROOT /"analysis"/"berm"/"results"rd =lambda f, **k: pd.read_csv(BR / f, **k)SHOW = ["model", "h_caught", "h_fa_days", "h_fa_ep", "h_missed", "fa_days", "fa_ep"]NICE =dict(model="Model", h_caught="Honest caught", h_fa_days="Honest false-alarm days", h_fa_ep="Honest episodes", h_missed="Honest miss", fa_days="Catch-all false-alarm days", fa_ep="Catch-all episodes")def tbl(df, n=None): df = df[~df.degenerate] if"degenerate"in df else dfreturn (df.head(n) if n else df)[SHOW].rename(columns=NICE).fillna("")SN, SA = rd("search_nest_top300.csv"), rd("search_all_top300.csv")F, S3 = rd("fixes.csv"), rd("select_nested_k3.csv")OLD =lambda f: rd(f"old_labels/{f}")nb =int(SN.h_caught.max()) +1# breaches on measured-level days (the honest check misses the weakest)
Breach dates
The model’s breach dates are CCBER’s logged mouth openings (build_dataset.mouth_state), each checked against the sub-daily level loggers (breach_timing.py), which would move a breach to the day the slough began to drain if that came earlier. Rebuilt from the level record alone (the highest level before the largest fall within 48 hours, infer_breaches.py), every logged breach with a logger record drained on its logged day (Figure 1).
Figure 1: Every breach with a level logger, hour by hour. Slough level (blue line, ft NAVD88), daily rain (gold bars) and predicted tide at Santa Barbara (thin gray line, unscaled, shown only to mark tidal exchange). Red lines mark when the slough began to drain (breach_timing.py); dashed red marks a breach day without a logged time.
The February 2026 breach drained on 2/19, not 2/18: the live sensor rose through 2/18 and peaked at 8.12 ft at 09:00 on 2/19 before falling 3.9 ft by evening. A plot of daily means peaks on 2/18 only because the 2/19 mean includes the drain. Dating it 2/18 also scored worse (with the old labels, rain-only models went from 8 to 23 false-alarm days).
Label review
Three stretches of the level record disagree with the mouth log; the days involved were false alarms in every model. They are corrected in data/mouth_corrections.csv, which mouth_state() applies after the log, and the whole pipeline was refit (train.py, unified.py measured, watch_model.py, levels.py). The 2020 and 2024 corrections are provisional, pending Garrett’s review of the figures below; the 2023 correction is his own read of the record.
March to April 2020, closed earlier and an unlogged breach (provisional). The log has the mouth open from the 3/16 breach until it closed on 4/5. The level stops following the tide about 3/21, drains a rain pulse back by 3/24 and rises steadily without rain from 3/26, with small steps at high tide on 4/1 to 4/4 as waves topped the berm: closed from 3/26. Then 1.31 in of rain on 4/5 lifted the slough to 6.79 ft at 02:00 on 4/6, it fell 1.3 ft in three hours, and it swung with the tides through 4/12 (Figure 3): an unlogged breach on 4/6, open to 4/12.
2024-03-31, unlogged breach (provisional). After 1.8 in of rain the slough peaked at 6.5 ft on 3/30 and again about 03:00 on 3/31, then fell 1.8 ft by 4/3, and jumped back up on 4/5 as the berm re-closed (Figure 2). Mouth marked open 3/31 to 4/4.
March 2023, reviewed by Garrett. The log has the mouth open from the 3/10 breach until 3/30. The breach drain ends on 3/11 and the berm is closed from 3/12. On 3/14, 2.5 in of rain took the slough to 7.3 ft at 14:00 and it lost about 2 ft within a day: a brief opening, now a breach, closed again from 3/16. After that the level holds except for steps at high tide (3/17, 3/19, 3/22 to 24) that recede slowly: high tides topping a berm closed to real outflow, through about 4/14 (Figure 6).
2024-03-23 to 03-24, left as closed. The slough rose 0.45 ft with 0.64 in of rain, then fell 0.8 ft over about 30 hours to below its pre-storm level, with no tidal signal. A brief, shallow overwash or seep that resealed is possible; it is not labeled a breach.
Code
BP.label_review_one(0);
Figure 2: March 20 to April 8, 2024 (provisional). Hourly slough level with daily rain and predicted tide. The 3/23 rise and slow fall is left as closed; the 3/31 drain (red line) and the 4/5 re-closure are now labeled a breach, pending review.
Code
BP.label_review_one(1);
Figure 3: March 14 to April 16, 2020 (provisional). After the 3/16 breach the tidal swing fades by 3/21 and the slough fills steadily from 3/26 (closed), with small high-tide steps over the berm on 4/1 to 4/4. Rain on 4/5 lifts it to 6.8 ft and it breaches at about 02:00 on 4/6 (red line), tidal to 4/12. The log had it open to 4/5 and closed after.
Code
BP.label_review_one(2);
Figure 4: March 1 to 16, 2025. The berm closed on 3/3 at 5.7 ft, rose to 6.0 ft with rain on 3/11 to 3/12, and failed at about 02:30 on 3/13 at 6.8 ft (red line).
Code
BP.label_review_one(3);
Figure 5: February 14 to March 10, 2024. The berm re-closes on 2/20 after the 2/1 breach; king tides swing the slough 0.6 to 1 ft a day on 2/21 to 2/24 without a breach; the breach drains from about 18:00 on 3/6 (red line), a day before CCBER’s 3/7 (reviewed). Daily rain in gold.
Code
BP.label_review_one(4);
Figure 6: March 10 to April 15, 2023 (reviewed). The 3/10 breach drains through 3/11 and the berm is closed from 3/12. A brief opening at about 14:00 on 3/14 (red line) drains 2 ft; closed again from 3/16, with high tides topping the berm and seeping back slowly through about 4/14. The log had it open until 3/30.
Berm height and rain windows
New inputs: margin, the slough level minus the crest curve’s estimate of the berm top (with the crest floor from the first three days after closure, as in src/unified.py); maxlvl, the highest measured level since the berm closed, through the day before, and margin_max, the level minus it; rain_2d, rain that day plus the day before; and from KSBA hourly rain, trail24 and trail48 (the 24 and 48 hours ending at midnight at the end of the day) and peak24 (the wettest 24 hours ending at any hour of the day, so a storm that straddles midnight counts in full).
The stored crest curve (models/unified_measured.npy) is fit on every season, so in a held-out test it leaks the held-out season into margin. Here the crest curve is refit with each season held out. With the old labels the leak made margin models look far better than they were; with the corrected labels it no longer helps:
Code
SAo, SNo = OLD("search_all_top300.csv").set_index("model"), OLD("search_nest_top300.csv").set_index("model")keys = ["rain_in + margin", "margin + peak24", "rain_in + rain_prev + margin", "margin + trail24"]f =lambda T, k: f"{T.loc[k].h_fa_days} / {T.loc[k].h_fa_ep}"if k in T.index else""pd.DataFrame({"Model": keys,"Old labels, leaky crest": [f(SAo, k) for k in keys], "Old labels, fair crest": [f(SNo, k) for k in keys],"Corrected, leaky crest": [f(SA.set_index("model"), k) for k in keys],"Corrected, fair crest": [f(SN.set_index("model"), k) for k in keys]}).set_index("Model")
Old labels, leaky crest
Old labels, fair crest
Corrected, leaky crest
Corrected, fair crest
Model
rain_in + margin
0 / 0
15 / 6
153 / 16
153 / 18
margin + peak24
3 / 2
6 / 4
182 / 19
429 / 28
rain_in + rain_prev + margin
3 / 2
9 / 6
215 / 22
234 / 28
margin + trail24
2 / 2
15 / 6
173 / 17
210 / 21
The table entries are held-out false-alarm days / episodes. Every combination of up to five of the 16 inputs (6,886 models) was searched with the fair crest and the corrected labels. The best fifteen, and the reference models:
SNi = SN.set_index("model")crest = SNi.loc["CREST-CURVE MODEL (stored, site)"]rw = SNi.loc["rain_in + rain_wy"]rl = SNi.loc["rain_in + later"]mp = SNi.loc["margin + peak24"]old = SNi.loc["rain_in + rain_prev + margin + margin_max"]best = SN[~SN.degenerate].iloc[0]b3 = SN[(~SN.degenerate) & (SN.k <=3)].iloc[0]pk = rd("crest_variants.csv").set_index("variant").iloc[0]md(f"""These results use the corrected margin_max (the breach-day leak described in Part 7 is fixed).**2024-03-06 is the weakest breach** for the crest-curve model and most others: a berm about twoweeks old that drained from about 6.0 ft after 0.75 in of rain. **2025-03-13 is caught** on thehonest check by the crest-curve model (peak score {pk.peak_20250313:.2f}) and by most models.**Best models.** The leaders all use **maxlvl**, the highest level this berm has already held:**{best.model}** catches {best.h_caught}/{nb} with {best.h_fa_days} false-alarm days in {best.h_fa_ep}episodes, missing {best.h_missed}. The best of up to three inputs is **{b3.model}**: {b3.h_caught}/{nb},{b3.h_fa_days} days in {b3.h_fa_ep} episodes, missing {b3.h_missed}. The earlier live model,rain_in + rain_prev + margin + margin_max, falls to {old.h_fa_days} days in {old.h_fa_ep} episodes oncemargin_max no longer leaks. The earlier crest-curve model, refit, catches {crest.h_caught}/{nb} with{crest.h_fa_days} days in {crest.h_fa_ep} episodes; rain_in + later does {rl.h_fa_days} / {rl.h_fa_ep},rain_in + rain_wy {rw.h_fa_days} / {rw.h_fa_ep}, and margin + peak24 {mp.h_fa_days} / {mp.h_fa_ep}.**Overfitting check.** Repeating the whole search with each season held out (the best model of upto three inputs chosen from the other seasons, alerts at its training cutoff) catches**{S3.caught.sum()} of {S3.breaches.sum()}** with **{S3.fa_days.sum()} false-alarm days in {S3.fa_ep.sum()}episodes**, missing {", ".join(m for m in S3.missed.dropna())}. It picks maxlvl or margin_max with arain total in every season, though which exact inputs win changes from season to season.""")
2024-03-06 is the weakest breach for the crest-curve model and most others: a berm about two weeks old that drained from about 6.0 ft after 0.75 in of rain. 2025-03-13 is caught on the season-held-out check by the crest-curve model (peak score 0.12) and by most models.
Best models. The leaders all use maxlvl, the highest level this berm has already held: rain_in + rain_wy + log_dc + maxlvl catches 16/17 with 4 false-alarm days in 4 episodes, missing 2020-04-06. The best of up to three inputs is rain_in + later + maxlvl: 16/17, 5 days in 5 episodes, missing 2024-03-31. rain_in + rain_prev + margin + margin_max has 164 days in 20 episodes. The crest-curve model, refit, catches 16/17 with 474 days in 26 episodes; rain_in + later does 12 / 10, rain_in + rain_wy 10 / 9, and margin + peak24 429 / 28.
Overfitting check. Repeating the whole search with each season held out (the best model of up to three inputs chosen from the other seasons, alerts at its training cutoff) catches 15 of 17 with 13 false-alarm days in 7 episodes, missing 2022-12-31, 2024-03-06. It picks maxlvl or margin_max with a rain total in every season, though which exact inputs win changes from season to season.
Every bad result
Each missed breach and false-alarm episode of the crest-curve model, margin + peak24, rain_in + rain_prev + margin and rain_in + rain_wy was reviewed (analysis/berm/README.md, Figure 7). With all the corrections they fall into five causes:
Cause
Cases
Fix
Fixed?
Wrong labels (unlogged breaches, closure dates)
2020-04-06, 2024-03-31, 2023-03-14, 2023-03-12 to 03-29
corrected labels (provisional)
yes
Young, low berm
2024-03-07 (miss), 2024-02-21 to 22, 2026-02-09 to 16, 2019-03-22
the highest level held (maxlvl) with rain
partly
Old tall berm near-miss in a big storm
Dec 2019 (held at 9.7 ft), Dec 21 to 24, 2023 (2.4 in, held at 9.6 ft)
2024-02-21 to 02-22 may be one more label error: the log has the berm closing on 2/20, but the level swung 0.7 to 1 ft a day, the logger skipped 2/22, and by 2/23 the slough was 2 ft lower and still swinging until 2/25. It is left as logged, pending review.
Figure 7: The false alarms that remain. Hourly level, daily rain and tide (observed, same axis) around three false-alarm episodes (shaded). December 2019 and December 2023: old berms held the slough at 9.6 to 9.7 ft through big storms. February 2026: a young berm held 6.8 ft for a week, three days before it breached at 7.9 ft (red line).
Fixes
Each fix was added to the simple models and judged on the season-held-out check (corrected labels; the old-label results are in analysis/berm/results/old_labels/):
Code
keep = ["margin + peak24", "margin_mx + peak24", "margin + peak24 + rise_pos", "margin + peak24 + rain_x_gap","margin + peak24 + later", "stage + log_dc + maxlvl + peak24", "rain_in + margin", "rain_in + rain_prev + margin","CREST-CURVE MODEL (stored, site)", "CREST-CURVE MODEL (refit, measured days)", "rain_in + rain_wy"]FF = F.set_index("model").loc[keep]pd.DataFrame({"Fix": FF.fix.values,"Caught": [f"{r.h_caught}/{nb}"for r in FF.itertuples()],"False-alarm days / episodes": [f"{r.h_fa_days} / {r.h_fa_ep}"for r in FF.itertuples()],"Missed": FF.h_missed.fillna("").values}, index=pd.Index(keep, name="Model"))
Fix
Caught
False-alarm days / episodes
Missed
Model
margin + peak24
base
16/17
429 / 28
2024-03-06
margin_mx + peak24
crest floor = max level
16/17
418 / 28
2024-03-06
margin + peak24 + rise_pos
+ rise_pos
16/17
379 / 30
2024-03-06
margin + peak24 + rain_x_gap
+ rain x gap
16/17
370 / 29
2024-03-06
margin + peak24 + later
+ later
16/17
635 / 28
2024-03-06
stage + log_dc + maxlvl + peak24
berm from data: stage + log_dc + maxlvl for ma...
16/17
55 / 12
2024-03-06
rain_in + margin
base
15/17
153 / 18
2024-03-06;2024-03-31
rain_in + rain_prev + margin
base
16/17
234 / 28
2024-03-06
CREST-CURVE MODEL (stored, site)
reference
16/17
474 / 26
2024-03-06
CREST-CURVE MODEL (refit, measured days)
reference
16/17
131 / 22
2024-03-06
rain_in + rain_wy
reference (old best)
16/17
10 / 9
2024-03-06
With the 2024-03-07 breach in the record, margin-based fixes do not recover; the inputs that hold up are the highest level held since closure and a berm rebuilt from data (stage, berm age and that level), which are in the best search models above. Yesterday’s rise still adds false alarms, and the already-breached flag (later) helps only alongside the highest level held.
The crest curve and 2025-03-13
The crest curve assumes every berm starts at the same height and rebuilds at the same rate. Four low-parameter changes were refit inside the crest-curve model, each with seasons held out (crest_variants.py): (a) a starting crest that grows with how long the mouth was open before it closed (a mouth that re-closes soon after a breach builds a thin, low berm); (b) a starting crest set by each closure’s own level plus a fitted offset; (c) a slower rebuild for berms that close in February to April; (d) an uncertain crest whose spread shrinks as the berm ages, folded into the hazard. No waves, surge or tides.
c: slower rebuild if closed Feb-Apr (tau * exp(g))
16
2024-03-06
481
26
0.116
6.74
10.021
6.204
59.131
1.683
0.118
d: uncertain crest, sd = s*exp(-d/tau)
16
2024-03-06
156
25
0.112
7.18
9.659
6.818
74.239
3.385
1.066
Code
c = CV.set_index("variant")base, late = c.iloc[0], c.iloc[3]md(f"""With all the corrections every version catches 2025-03-13 on the honest check and misses2024-03-07 instead, which scores near zero on any crest curve and drags the cutoff down. Theuncertain crest (d) helps most ({c.iloc[4].h_fa_days} false-alarm days against {base.h_fa_days}),the open-duration start (a) a little ({c.iloc[1].h_fa_days}), the late-season rebuild (c) not at all,and closure-level starts (b) make it worse ({c.iloc[2].h_fa_days}). None comes close to theberm-history models above. The fitted base curve now starts at {CP.iloc[0].c0:.2f} ft and risestoward {CP.iloc[0].c_inf:.2f} ft with a {CP.iloc[0].tau_d:.0f}-day time constant (5.76 ft, 9.76 ft and 60days in the committed model). The crest's shape is weakly pinned down by about 16 breaches; ameasured berm top (CCBER's 2024 COPR berm surveys, `data/berm_profiles` in their repository)would test it.""")
With all the corrections every version catches 2025-03-13 on the season-held-out check and misses 2024-03-07 instead, which scores near zero on any crest curve and drags the cutoff down. The uncertain crest (d) helps most (156 false-alarm days against 474), the open-duration start (a) a little (395), the late-season rebuild (c) not at all, and closure-level starts (b) make it worse (1011). None comes close to the berm-history models above. The fitted base curve now starts at 6.17 ft and rises toward 10.01 ft with a 64-day time constant (5.76 ft, 9.76 ft and 60 days in the committed model). The crest’s shape is weakly pinned down by about 16 breaches; a measured berm top (CCBER’s 2024 COPR berm surveys, data/berm_profiles in their repository) would test it.
A second rain gauge: CoCoRaHS CA-SB-1
CoCoRaHS station CA-SB-1 (Goleta 3.7 WNW) is about 2.5 km northwest of the slough, closer than the COP gauge the model uses. Its daily reports cover 24 hours ending about 07:00 and are credited to the report date, so sb1_rain.py spreads each report over the calendar days its window covers in proportion to KSBA’s hourly rain. The station’s rows are saved in analysis/berm/cocorahs_ca_sb_1.csv. It enters the search as its own inputs (sb1_rain_in, sb1_rain_2d) and as the larger of the two gauges (gmax_rain_in, gmax_rain_2d).
Code
SBc = rd("sb1_coverage.csv", index_col=0).valueSBs = rd("sb1_search_k3.csv").set_index("model")pair = ["margin + margin_max + rain_2d", "margin + margin_max + sb1_rain_2d", "margin + margin_max + gmax_rain_2d","margin + margin_max + trail24", "rain_in + later", "later + sb1_rain_in", "rain_in + rain_wy", "rain_wy + sb1_rain_in"]md(f"""Since WY2019 the station is missing {SBc.days_missing} of {SBc.period_days} days ({SBc.missing_wet_season_days} inNovember to April). On the {SBc.storm_days} storm days with at least 0.5 in at any gauge it caught{float(SBc.ratio_sb1_cop):.2f} times the COP gauge's rain and {float(SBc.ratio_sb1_ksba):.2f} times KSBA's (correlation{SBc.corr_sb1_cop} with COP). The same three-input models with each gauge (honest check):""")SBs.loc[[p for p in pair if p in SBs.index], ["h_caught", "h_fa_days", "h_fa_ep", "h_missed"]].rename(columns=NICE)
Since WY2019 the station is missing 23 of 2971 days (7 in November to April). On the 115 storm days with at least 0.5 in at any gauge it caught 1.22 times the COP gauge’s rain and 1.33 times KSBA’s (correlation 0.901 with COP). The same three-input models with each gauge (season-held-out check):
Held out: caught
Held out: false-alarm days
Held out: episodes
Held out: missed
model
margin + margin_max + rain_2d
16
123
16
2024-03-06
margin + margin_max + sb1_rain_2d
16
144
19
2024-03-06
margin + margin_max + gmax_rain_2d
16
53
21
2024-03-06
margin + margin_max + trail24
16
1707
24
2024-03-06
rain_in + later
16
12
10
2024-03-06
later + sb1_rain_in
16
14
12
2019-01-07
rain_in + rain_wy
16
10
9
2024-03-06
rain_wy + sb1_rain_in
16
18
15
2025-03-13
Code
SB3 = rd("select_nested_k3_sb1.csv")a, b = SBs.loc["margin + margin_max + sb1_rain_2d"], SBs.loc["margin + margin_max + rain_2d"]md(f"""With two days of CA-SB-1 rain, margin + margin_max catches {a.h_caught}/{nb} with {a.h_fa_days} false-alarmdays in {a.h_fa_ep} episodes (missing {a.h_missed}); with the COP gauge it catches {b.h_caught}/{nb} with {b.h_fa_days} /{b.h_fa_ep} (missing {b.h_missed}). CA-SB-1 reports once a day at 07:00, so it cannot feed a live forecast. With the CA-SB-1 inputs among the candidates, the search held outby season catches **{SB3.caught.sum()} of {SB3.breaches.sum()}** with **{SB3.fa_days.sum()} false-alarm days in{SB3.fa_ep.sum()} episodes** (without them, {S3.caught.sum()} with {S3.fa_days.sum()} / {S3.fa_ep.sum()}).""")
With two days of CA-SB-1 rain, margin + margin_max catches 16/17 with 144 false-alarm days in 19 episodes (missing 2024-03-06); with the COP gauge it catches 16/17 with 123 / 16 (missing 2024-03-06). CA-SB-1 reports once a day at 07:00, so it cannot feed a live forecast. With the CA-SB-1 inputs among the candidates, the search held out by season catches 14 of 17 with 8 false-alarm days in 5 episodes (without them, 15 with 13 / 7).
When in the day: a breach-hour backtest
Once a breach is expected, the hour matters for anyone heading out to see it. For each of the 17 breaches with a logged apex, the breach time is the logged maximum before the drop (PST). The rain is Coal Oil Point’s hourly record (USCRN, PST), the site’s gauge, standing in for a perfect forecast. The storm is the run of wet hours (gaps of up to 6 h) the breach falls in or follows; from its first wet hour it runs forward on its own, so its end never depends on when the breach came. breach_hour.py tests simple rules, each fit with the breach’s season held out.
Code
BHs = rd("breach_hour_storms.csv")BHp = rd("breach_hour_summary.csv").set_index("model")md(f"""Breaches came a median {BHs.lag_onset_h.median():.0f} hours after the storm began and{BHs.lag_peak3_h.median():.0f} hours after its wettest three hours, after a median {BHs.rain_to_breach_in.median():.2f} in ofstorm rain. The rise per inch from the storm's start to the breach was {BHs.rise_per_in[BHs.start_level_ft <7].median():.2f} ftfor slough levels under 7 ft and {BHs.rise_per_in[BHs.start_level_ft >=7].median():.2f} ft above, in line with the largerarea at higher levels.""")BHp.rename(columns={"n": "Breaches", "median_abs_error_h": "Median error (h)", "hit_3h": "Within 1.5 h","hit_4h": "Within 2 h", "hit_m1p3": "1 h before to 3 h after", "hit_pm4": "Within 4 h","hit_pm6": "Within 6 h", "worst_h": "Worst miss (h)"}).round(2)
Breaches came a median 17 hours after the storm began and 3 hours after its wettest three hours, after a median 1.93 in of storm rain. The rise per inch from the storm’s start to the breach was 0.74 ft for slough levels under 7 ft and 0.56 ft above, in line with the larger area at higher levels.
Breaches
Median error (h)
Within 1.5 h
Within 2 h
hit_pm3
1 h before to 3 h after
Within 4 h
Within 6 h
Worst miss (h)
model
peak1
17
2.00
0.35
0.59
0.82
0.59
0.88
0.94
35.00
peak3
17
2.00
0.35
0.53
0.65
0.41
0.76
0.82
36.00
rainsum_shift
17
2.25
0.29
0.35
0.76
0.53
0.76
0.76
36.34
centroid
17
3.01
0.24
0.29
0.47
0.29
0.71
0.94
20.22
rainsum
17
3.41
0.29
0.41
0.47
0.47
0.65
0.82
38.00
Code
BP.breach_hour_fig();
Figure 8: Breach timing from the storm’s rain. Left: cumulative Coal Oil Point storm rain against hours to the breach for every breach with a logged apex (dashed line: the median storm rain at the breach). Right: errors of the site’s rule, the storm’s wettest hour plus the other seasons’ median lag, held out by season (misses beyond 12 h shown at the edge); dotted and dashed lines mark windows of 2 and 4 hours either side.
Code
import json as _jsonBHJ = _json.loads((ROOT /"models"/"breach_hour.json").read_text())p1 = BHp.loc["peak1"]W, RW = BHJ["heldout"]["windows"], BHJ["heldout"]["recommended_window"]md(f"""The site's rule takes the storm's wettest hour on Coal Oil Point's gauge and adds the typical lagfrom the wettest hour to the breach. A storm with more than one burst uses its single wettest hour, aslong as the bursts are no more than 6 hours apart; a later burst after a longer dry spell is a new storm.Held out by season (the lag from the other seasons' breaches), it lands within 2 hours of the breach for{W['hit_4h']['hits']} of {W['hit_4h']['n']} breaches, within 3 hours for {W['hit_pm3']['hits']}, within 4 hours for{W['hit_pm4']['hits']}, and from 1 hour before to 3 after for {W['hit_m1p3']['hits']}, with a median error of{p1.median_abs_error_h:.1f} hours. Fit on all breaches the lag is **{BHJ['lag_h']:.0f} hours** (`models/breach_hour.json`).The narrowest window that held at least three breaches in four runs from {abs(RW['from_h']):.0f} hours before to{RW['to_h']:.0f} hours after the predicted time ({RW['hits']} of {RW['n']}). The misses are storms that kept raininghard for a day or more, where the wettest hour came long after the breach.""")
The site’s rule takes the storm’s wettest hour on Coal Oil Point’s gauge and adds the typical lag from the wettest hour to the breach. A storm with more than one burst uses its single wettest hour, as long as the bursts are no more than 6 hours apart; a later burst after a longer dry spell is a new storm. Held out by season (the lag from the other seasons’ breaches), it lands within 2 hours of the breach for 10 of 17 breaches, within 3 hours for 14, within 4 hours for 15, and from 1 hour before to 3 after for 10, with a median error of 2.0 hours. Fit on all breaches the lag is 3 hours (models/breach_hour.json). The narrowest window that held at least three breaches in four runs from 2 hours before to 3 hours after the predicted time (13 of 17). The misses are storms that kept raining hard for a day or more, where the wettest hour came long after the breach.
Breach hour, head to head
A test of two ways to call the hour, on every breach with a logged apex (17), with the storm’s rain as a perfect forecast and nothing else from after the storm began (analysis/berm/breach_hour_compare.py). Every constant is fit with the breach’s water year held out. Rain comes from two gauges, both in PST: KSBA (the airport) and Coal Oil Point (USCRN Santa Barbara 11 W, hourly), the site’s rain source.
A, rain total: the hour the storm’s rain since onset reaches the median rain-to-breach of the other seasons, plus their median lead. The storm runs forward from its onset (wet hours, gaps up to 6 h) and does not know when the breach came.
B, crossing: the hourly level simulated from the last logged level before the storm, with the response fit on other seasons’ storms; the berm fails at the highest level it has held plus a delta fit on other seasons’ breaches. The hour is where the chance of having failed reaches 50%, on the median path or (Monte Carlo) on the spread of crossing times.
BCst = rd("breach_hour_compare_storms.csv", parse_dates=["breach", "onset"])on = BCst.set_index(["gauge", "breach"]).onsetmeths = ["A rain total", "B crossing (MC median)", "A+B (MC) average", "B crossing"]fig, axs = plt.subplots(2, 2, figsize=(8.5, 7.5), sharex=True, sharey=True)for ax, m inzip(axs.flat, meths):for g, c in (("KSBA", "#2D5A7B"), ("COP", "#D3A43B")): x = BC[(BC.method == m) & (BC.gauge == g)].dropna(subset=["predicted"]) o = on.reindex(list(zip(x.gauge, x.breach))).values xa = (x.actual.values - o) / pd.Timedelta(hours=1) ya = (x.predicted.values - o) / pd.Timedelta(hours=1) ax.scatter(xa, ya, s=22, color=c, label=g, alpha=0.85) ax.plot([0, 56], [0, 56], "k--", lw=0.8) ax.fill_between([0, 56], [-2, 54], [2, 58], color="0.85", zorder=0) ax.set_xlim(0, 56); ax.set_ylim(0, 60) ax.set_title(m, fontsize=9)axs[1, 0].set_xlabel("actual: hours after onset"); axs[1, 1].set_xlabel("actual: hours after onset")axs[0, 0].set_ylabel("predicted: hours after onset"); axs[1, 0].set_ylabel("predicted: hours after onset")axs[0, 0].legend(fontsize=8)plt.tight_layout(); plt.show()
Figure 9: Predicted vs actual breach hour. Hours from the storm’s onset to the logged apex (x) and to each method’s call (y), every constant fit with the breach’s season held out. Dashed: perfect; band: within 2 h (a 4 h window). Blue: KSBA rain; gold: Coal Oil Point rain. Bottom right: the crossing model’s median path, which often never crosses (then its highest point stands in).
Code
k = BCs.set_index(["gauge", "method"])def line(g, m): r = k.loc[(g, m)]returnf"{int(r.hit3)}/{int(r.n)} within 3 h, {int(r.hit4)}/{int(r.n)} within 4 h, median error {r.median_abs_error_h:.1f} h"md(f"""With each season held out, neither method calls the hour well. On Coal Oil Point rain, the site's source,the rain total gets {line("COP", "A rain total")}; the Monte Carlo crossing gets{line("COP", "B crossing (MC median)")}; their average {line("COP", "A+B (MC) average")}. On KSBA:rain total {line("KSBA", "A rain total")}; crossing {line("KSBA", "B crossing (MC median)")}.The crossing model's median path is worse ({line("KSBA", "B crossing")} on KSBA): for most breachesthe simulated level never quite reaches the fitted failure height, so it has no crossing to call.Neither lands within 2 hours for more than about half the breaches. The site timesthe breach from the storm's wettest hour (previous section), which does better held out.""")
With each season held out, neither method calls the hour well. On Coal Oil Point rain, the site’s source, the rain total gets 5/17 within 3 h, 6/17 within 4 h, median error 2.2 h; the Monte Carlo crossing gets 5/17 within 3 h, 6/17 within 4 h, median error 2.7 h; their average 4/17 within 3 h, 8/17 within 4 h, median error 2.2 h. On KSBA: rain total 4/17 within 3 h, 4/17 within 4 h, median error 2.9 h; crossing 4/17 within 3 h, 6/17 within 4 h, median error 2.4 h. The crossing model’s median path is worse (3/17 within 3 h, 3/17 within 4 h, median error 11.0 h on KSBA): for most breaches the simulated level never quite reaches the fitted failure height, so it has no crossing to call. Neither lands within 2 hours for more than about half the breaches. The site times the breach from the storm’s wettest hour (previous section), which does better held out.
Which day gets the call
Garrett’s rule of thumb: if all the rain falls in the morning and the next day is dry, the breach comes the day it rains. The best model weighs yesterday’s rain (4.36 per inch) a little more than today’s (3.78), so its score the day after a storm stays close to the storm day’s. rain_timing.py and rain_timing2.py rebuild the daily rain from KSBA hourly rain in local standard time, in forms an hourly forecast can also give: 24-hour windows ending at 12:00 to 24:00 and the window before, rain before 16:00 plus the previous evening’s, and 6-hour blocks. Each is searched with the berm terms on the season-held-out check. Every breach’s apex falls on its labeled day in local standard time.
Code
RD = rd("rain_timing_day_after.csv")md(f"""The last two columns use the {int(RD.n_pairs.iloc[0])} closed-mouth days with at least 0.5 in of rain followed by adry day: how often the rain day scores at least as high as the next day, and the median ratio ofthe next day's score to the rain day's.""")RD[["model", "h_caught", "h_fa_days", "h_fa_ep", "h_missed", "breach_peak_on_day", "rain_day_beats_next", "median_next_over_rain"]].rename( columns={**NICE, "breach_peak_on_day": "Breaches peaking on the day", "rain_day_beats_next": "Rain day scores higher","median_next_over_rain": "Next day / rain day"}).set_index("Model")
The last two columns use the 17 closed-mouth days with at least 0.5 in of rain followed by a dry day: how often the rain day scores at least as high as the next day, and the median ratio of the next day’s score to the rain day’s.
Held out: caught
Held out: false-alarm days
Held out: episodes
Held out: missed
Breaches peaking on the day
Rain day scores higher
Next day / rain day
Model
rain_in + rain_wy + log_dc + maxlvl
16
4
4
2020-04-06
12
1.00
0.06
crest + maxlvl + rain_2d
16
25
9
2022-12-31
17
0.65
1.00
rain_in + crest + maxlvl + trail24
16
6
5
2024-03-06
13
1.00
0.02
rain_in + rain_prev + margin + margin_max
16
164
20
2024-03-06
15
0.65
0.65
rain_in + margin + margin_max
16
1868
31
2024-03-06
14
0.82
0.32
rain_in + prev_late16 + margin + margin_max
16
269
28
2024-03-06
16
0.82
0.04
rain_in + prev_late12 + margin + margin_max
16
240
25
2024-03-06
15
0.88
0.08
win24 + prev_late16 + margin + margin_max
16
166
17
2024-03-06
16
0.76
0.13
win18 + margin + margin_max + later
16
366
30
2024-03-06
16
0.82
0.07
win24 + margin + margin_max
16
1808
29
2024-03-06
14
0.53
0.45
eff + margin + margin_max
16
882
30
2024-03-06
17
0.65
0.10
win21 + margin + margin_max
16
876
24
2024-03-06
14
0.88
0.16
win18 + win18_prev + margin + margin_max
16
242
23
2024-03-06
17
0.65
0.30
rain_in + carry + margin + margin_max
16
1763
25
2024-03-06
14
0.88
0.45
eff + carry + margin + margin_max
16
373
35
2024-03-06
17
0.71
0.07
Code
RDi = RD.set_index("model")bd = RDi.loc["rain_in + rain_wy + log_dc + maxlvl"]om = RDi.loc["rain_in + rain_prev + margin + margin_max"]md(f"""With margin_max corrected, the best model (rain_in + rain_wy + log_dc + maxlvl) uses only the day'sown rain, so it scores the rain day above the next day in {bd.rain_day_beats_next:.0%} of the wet-then-drystorms; the earlier live model, which also weighs yesterday's rain, did so in {om.rain_day_beats_next:.0%}.The best model's score peaks on the labeled breach day for {bd.breach_peak_on_day} of {nb} breaches; theothers drained in the early morning after a rain day (e.g. 2025-03-13 at 02:24), where it peaks on therain day itself. The hourly windows add nothing once the daily rain is the day's own.""")
The best model (rain_in + rain_wy + log_dc + maxlvl) uses only the day’s own rain, so it scores the rain day above the next day in 100% of the wet-then-dry storms; a model that also weighs yesterday’s rain (rain_in + rain_prev + margin + margin_max) does so in 65%. The best model’s score peaks on the labeled breach day for 12 of 17 breaches; the others drained in the early morning after a rain day (e.g. 2025-03-13 at 02:24), where it peaks on the rain day itself. The hourly windows add nothing once the daily rain is the day’s own.
Breach times and antecedent moisture
Breach times follow Garrett’s rule: the logged maximum level just before the breach drop, from the raw logger timestamps in local standard time (breach_apex.py). No breach day changes. Seven times move: 2021-01-28 from 10:00 to 15:30 (the old time came from a logger glitch at 10:00), 2024-03-31 from 03:00 to 04:24, 2024-03-06 from 18:00 to 18:54, 2023-03-14 from 14:00 to 14:45, 2023-03-10 about half an hour earlier, and 2020-04-06 from 02:00 to 02:15. 2022-12-31 moved to 23:45 in the logger QA below (its old 16:38 was a second logger failing). The event catalog and the breach-hour backtest use the corrected times.
Garrett’s read of March 2020: about two months without rain left the watershed dry, so the 3/10 to 3/14 storms barely raised the slough while the berm sat near its highest level. antecedent.py adds days since the last 0.25 in day, 30- and 60-day rain, the existing API, and rain terms scaled by 60-day rain or by wetness exp(-days/k). A term is kept only if it removes false-alarm days without adding any or losing a breach.
ANk = AN[AN.keep]md("None passes."if ANk.empty else"Kept: "+", ".join(ANk.base +" + "+ ANk.added) +".")md("""Terms on the dry spell lower the March 2020 scores but cost a breach elsewhere or add false alarmsin other seasons (60-day rain, which the wet 2023 and 2024 seasons push high).""")
None passes.
Terms on the dry spell lower the March 2020 scores but cost a breach elsewhere or add false alarms in other seasons (60-day rain, which the wet 2023 and 2024 seasons push high).
An hourly threshold-crossing model
Garrett’s suggestion: model when the slough will pass the height the berm fails at, with the uncertainty turned into a chance for each day. crossing_model.py fits an hourly response to KSBA rain on the 45 closed-mouth storms (a gamma lag kernel and a rise per inch that shrinks with the lidar area curve), a breach threshold fit to the breach apex levels (the median, the crest curve plus an offset, the highest level held plus a margin, or a blend), and simulates 48 hours from the start of each day with draws over rain amount (lognormal, sd 0.4), rain timing (up to 3 hours either way) and the response. Each season’s response and threshold are fit on the others.
Code
CP6 = json.load(open(BR /"crossing_params.json"))md(f"""The fitted response peaks {CP6['kernel_mode_h']:.1f} hours after the rain and rises{CP6['r6_ft_per_in']:.2f} ft per inch at 6 ft, shrinking with area to the power {CP6['area_exponent']:.2f}(the same damping the daily water balance found). The tightest threshold is the crest curve(apex minus crest: offset {CP6['thresholds']['crest']['off']:+.2f} ft, spread {CP6['thresholds']['crest']['sd']:.2f} ft).""")
The fitted response peaks 1.7 hours after the rain and rises 0.96 ft per inch at 6 ft, shrinking with area to the power 0.25 (the same damping the daily water balance found). The tightest threshold is the crest curve (apex minus crest: offset +0.03 ft, spread 0.57 ft).
Garrett’s rule for every backtest: only the rain is known in advance, as a perfect forecast; everything else comes from data observed by the forecast time. The main crossing run uses the observed rain exactly (with the crest centre refit without the held-out season); the run with rain noise and timing jitter is kept as a sensitivity check. Combined with the daily model:
md(f"""The daily model alone (best_daily) already puts the call on the rain day in every wet-then-drystorm and has the fewest false alarms; every combination with the crossing model adds false alarms.The crossing model's hours are also no better than the wettest-hour rule ({CBp.windows.iloc[0]}). In the1.2-inch morning scenario for 11/25/2026 the best daily model puts the call on 11/25 (0.0013 against0.000008 on 11/26); with the NWS forecast (about 0.2 in) every version is near zero.""")
The daily model alone (best_daily) already puts the call on the rain day in every wet-then-dry storm and has the fewest false alarms; every combination with the crossing model adds false alarms. The crossing model’s hours are also no better than the wettest-hour rule (crossing hours: 3 h 3/17, 4 h 6/17; wettest-hour rule: 3 h 35%, 4 h 59%). In the 1.2-inch morning scenario for 11/25/2026 the best daily model puts the call on 11/25 (0.0013 against 0.000008 on 11/26); with the NWS forecast (about 0.2 in) every version is near zero.
The 2023-24 pier record
analysis/berm/data_gaps.md lists every gap of three days or more in the measured level record since WY2018 and what could fill it. One is now filled. CCBER’s repository has a pier logger record every 15 minutes from 2023-10-10 that was never downloaded because it is not compensated for air pressure. fetch_external.py now fetches it with KSBA hourly altimeter pressure (Iowa Environmental Mesonet, the source already used for KSBA rain), and loggers.py subtracts the pressure as meters of water and ties the result to CCBER’s compensated pier record over their first 60 days of overlap.
Code
import loggers_, diag = loggers.compensate_uncomp(loggers.RAW /"PIER_10.10.23_08.29.25_Uncompensated.csv")md(f"""The fitted offset is {diag['offset_m']*1000:.1f} mm and the slope against the compensated record{diag['slope']:.4f}; over {diag['n']:,} matched 15-minute readings the two differ by a median of{diag['resid_median_abs_ft']:.3f} ft (95th percentile {diag['resid_p95_abs_ft']:.3f} ft). The compensatedrecord stays the source from 2024-03-19 on; before that the new record is re-leveled to thestaff-gauge readings like every other logger segment. It adds WY2024's three logged breaches to themeasured-level days (@fig-pier-2024): the level-record apexes fall on CCBER's dates, at 08:40 on1/22, 05:25 on 2/1 and 09:10 on 3/7. The 3/7 drain may have begun the evening of 3/6 (the levelpeaks near 6.0 ft about 18:00 and falls from there, with tides from 3/7); CCBER's date is kept.""")
The fitted offset is 9.7 mm and the slope against the compensated record 0.9997; over 5,665 matched 15-minute readings the two differ by a median of 0.013 ft (95th percentile 0.036 ft). The compensated record stays the source from 2024-03-19 on; before that the new record is re-leveled to the staff-gauge readings like every other logger segment. It adds WY2024’s three logged breaches to the measured-level days (Figure 10): the level-record apexes fall on CCBER’s dates, at 08:40 on 1/22, 05:25 on 2/1 and 09:10 on 3/7. The 3/7 drain may have begun the evening of 3/6 (the level peaks near 6.0 ft about 18:00 and falls from there, with tides from 3/7); CCBER’s date is kept.
Code
BP.pier_2024_grid();
Figure 10: The three WY2024 breaches in the compensated pier record. Hourly level (ft NAVD88), daily rain and predicted tide around the 2024-01-22, 2024-02-01 and 2024-03-07 breaches (red lines mark the drain start; dashed where no time was dated).
The other gaps have nothing usable: WY2018 predates the loggers, spring 2021 has only the creek loggers upstream, and December 2022 has Venoco readings that were masked for swinging 4 ft a day.
Slough area from lidar
analysis/berm/dem_hypsometry.py builds the slough’s area and volume by level from the USGS 3DEP 1 m DEM (2018 Southern California wildfire lidar, flown May 27 to October 12, 2018; public, no sign-in), as the cells below each level connected to the main channel. The DEM is hydro-flattened: open water at flight time is one surface at 4.53 ft, so storage below that is missing. The basin it traces covers the lower slough and west arm; the north arm through NCOS is cut off at the road crossing in a bare-earth DEM, so its area is not included.
Code
BP.hypsometry_fig();
Figure 11: Slough area and volume by level, and what the loggers show. Left: area and volume from the 2018 lidar DEM (dotted line: the flattened water surface at 4.53 ft). Right: rise per inch of rain in closed-mouth storms by starting level, against pure volume over area (red) and the fitted damped form, area ratio to the 0.25 power (gold).
Code
HO = rd("dem_waterbalance_heldout.csv", index_col=[0, 1])md(f"""Area grows about sevenfold from 5 to 9 ft, but the loggers' rise per inch falls only aboutthreefold, so the water balance with a pure volume-over-area term overcorrects. With the area ratioraised to a fitted 0.25 power, storm peaks predicted with each season held out improve from{HO.loc[('all', 'base'), 'rmse']:.2f} to {HO.loc[('all', 'dem_gamma'), 'rmse']:.2f} ft overall and from{HO.loc[('start>=7ft', 'base'), 'rmse']:.2f} to {HO.loc[('start>=7ft', 'dem_gamma'), 'rmse']:.2f} ft for stormsstarting above 7 ft (bias {HO.loc[('start>=7ft', 'base'), 'bias']:+.2f} to {HO.loc[('start>=7ft', 'dem_gamma'), 'bias']:+.2f}ft; only {int(HO.loc[('start>=7ft', 'base'), 'n'])} such storms). It does not change the breach search(in a side-by-side run the crest-curve model's false alarms changed by under 5% and the held-outsearch did slightly worse), so the current form stays the default and the DEM form is optional: `WB_FORM=1 pythonsrc/train.py` fits it (`models/hypsometry.csv`). The site's water balance would need the area curvebefore it could use it.""")
Area grows about sevenfold from 5 to 9 ft, but the loggers’ rise per inch falls only about threefold, so the water balance with a pure volume-over-area term overcorrects. With the area ratio raised to a fitted 0.25 power, storm peaks predicted with each season held out improve from 0.50 to 0.46 ft overall and from 0.35 to 0.26 ft for storms starting above 7 ft (bias +0.19 to +0.00 ft; only 6 such storms). It does not change the breach search (in a side-by-side run the crest-curve model’s false alarms changed by under 5% and the held-out search did slightly worse), so the current form stays the default and the DEM form is optional: WB_FORM=1 python src/train.py fits it (models/hypsometry.csv). The site’s water balance would need the area curve before it could use it.
Overtopping and the event catalog
analysis/berm/events.py builds a catalog of every breach, closure and overtopping event from the hourly level record with the corrections applied (results/mouth_events.json and .csv, for the site). Overtopping is a closed-mouth step up in level at high tide with no rain: a rise of at least 0.05 ft over three hours ending within two hours of an observed high tide, still held six hours later (receding by no more than 70%), with no rain in the 12 hours before or that day, kept when one step is at least 0.15 ft or two or more tides add 0.15 ft (“step”). A rise of 0.3 ft or more at high tide that drains back within hours is kept too (“pulse”): water pushed over or through a berm closed to real outflow, as on 3/17/2023. The 3/22 to 24/2023 pulses came with rain and are left out by the no-rain rule. Tide is NOAA Santa Barbara (9411340) observed water level in ft NAVD88, drawn on the same axis as the slough (results/tide_hourly.csv); surge is observed minus predicted. Tide and surge describe events only and are never model inputs.
Code
import jsonEV = pd.DataFrame(json.load(open(BR /"mouth_events.json")))OV = EV[EV.type=="overtopping"].copy()OV["year"] = OV.start_utc.str[:4]BB = EV[EV.type=="breach"]md(f"""The catalog holds {len(BB)} breaches ({(~BB.logged).sum()} not in the CCBER log: {int(BB.provisional.sum())} provisional,{int(BB.reviewed.sum())} reviewed), {(EV.type=='closure').sum()} closures and {len(OV)} overtopping events, {int(OV.summer.sum())} of them inMay to October. Summer overtopping by year: {", ".join(f"{y}: {n}"for y, n in OV[OV.summer].groupby("year").size().items())}.""")OV[OV.summer].sort_values("total_rise_ft", ascending=False).head(10)[["start_utc", "total_rise_ft", "max_step_ft", "peak_level_ft", "tide_high_ft", "surge_ft", "sea_level_anomaly_ft"]].rename( columns={"start_utc": "Start (UTC)", "total_rise_ft": "Added (ft)", "max_step_ft": "Largest step (ft)", "peak_level_ft": "Slough (ft)","tide_high_ft": "High tide, observed (ft)", "surge_ft": "Surge (ft)", "sea_level_anomaly_ft": "Month's sea-level anomaly (ft)"})
The catalog holds 20 breaches (4 not in the CCBER log: 0 provisional, 4 reviewed), 17 closures and 40 overtopping events, 18 of them in May to October. Summer overtopping by year: 2019: 1, 2020: 1, 2022: 1, 2023: 1, 2024: 1, 2025: 11, 2026: 2.
Start (UTC)
Added (ft)
Largest step (ft)
Slough (ft)
High tide, observed (ft)
Surge (ft)
Month's sea-level anomaly (ft)
56
2025-08-12T07:00Z
1.08
0.195
5.26
5.38
0.10
-0.06
14
2020-05-05T05:00Z
1.03
0.370
6.32
6.45
0.34
-0.00
66
2025-10-15T15:00Z
0.86
0.163
5.95
5.15
0.31
-0.08
65
2025-10-10T09:00Z
0.81
0.207
4.46
3.81
0.03
-0.08
58
2025-08-26T18:00Z
0.52
0.499
5.88
4.89
0.16
-0.06
75
2026-05-16T04:00Z
0.50
0.181
6.01
7.19
0.53
0.15
37
2023-05-05T05:00Z
0.41
0.079
6.47
6.06
0.30
0.14
61
2025-09-13T11:00Z
0.34
0.261
5.14
3.78
0.23
-0.04
59
2025-08-30T10:00Z
0.32
0.371
5.03
2.88
0.20
-0.06
57
2025-08-21T02:00Z
0.23
0.281
5.92
4.77
0.30
-0.06
Summer 2026 stands out for sea level rather than for the number of events. Tides at Santa Barbara ran above prediction all summer, by a monthly average of 0.15 ft in May rising to 0.5 ft in September and 0.94 ft in October, the highest of the record and consistent with El Nino-like warm water. The slough took 0.4 to 0.5 ft from 7.2 ft tides on 5/16 to 5/18, and on 10/11 a 7.3 ft tide, 1.5 ft above prediction, added about 0.2 ft. Summer 2025 had more events (August to October, steps up to 0.5 ft) on ordinary tides, which points to waves running up a lower berm.
Code
BP.overtopping_grid();
Figure 12: Tides topping a closed berm. Hourly slough level (blue) and observed tide (gray), both ft NAVD88 on the same axis, with daily rain. April 2020: the slough steps up at high tides before the 4/6 breach. August 2025: a summer step of 0.5 ft. May 2026: 7.2 ft tides running 0.5 ft above prediction add 0.4 to 0.5 ft over three days.
Code
md(f"""**Summary.** The breach dates hold up against the hourly record. Garrett reviewed every labelcorrection: an unlogged breach on 2020-04-06 with the berm closed from 3/26; an earlier 2023closure (3/12) with a brief opening on 3/14; the 2024 breach moved to the evening of 3/6; anunlogged breach on 2024-03-31; and king-tide days on 2024-02-21 to 24 labeled leaky (not scored,same berm). The 2023-24 pier record adds three logged breaches. With {nb} measured-level breachesand the margin_max leak fixed, **{best.model}** catches {best.h_caught}/{nb} on the honest check with{best.h_fa_days} false-alarm days in {best.h_fa_ep} episodes, against {crest.h_fa_days} days in{crest.h_fa_ep} episodes for the refit crest-curve model; the search held out by season does{S3.caught.sum()}/{S3.breaches.sum()} with {S3.fa_days.sum()} days in {S3.fa_ep.sum()} episodes. How high theberm has already held the slough is the input that matters most.""")
Summary. The breach dates hold up against the hourly record. Garrett reviewed every label correction: an unlogged breach on 2020-04-06 with the berm closed from 3/26; an earlier 2023 closure (3/12) with a brief opening on 3/14; the 2024 breach moved to the evening of 3/6; an unlogged breach on 2024-03-31; and king-tide days on 2024-02-21 to 24 labeled leaky (not scored, same berm). The 2023-24 pier record adds three logged breaches. With 17 measured-level breaches, rain_in + rain_wy + log_dc + maxlvl catches 16/17 on the season-held-out check with 4 false-alarm days in 4 episodes, against 474 days in 26 episodes for the refit crest-curve model; the search held out by season does 15/17 with 13 days in 7 episodes. How high the berm has already held the slough is the input that matters most.
Logger QA
Two stretches of the level record were logger faults, not water (data_gaps.md, “Logger QA”; plots results/gap_dec2022.png and results/qa_aug2020.png; reviewed by Garrett).
Dec 2022. The Venoco file interleaves two loggers from 12/15 to 1/12: the deployed one, out of the water at its 2.84 ft floor, and a second one that fails at 16:53 on 12/31. Their mixed daily mean swung about 4 ft, so the period had been masked, and the second logger’s failure had set the 12/31 breach time. No record shows tide before the breach, so the mouth was closed as logged. East Bridge recorded throughout but creeps up 0.073 ft/day on calm days; minus that creep and tied to the 12/9 staff reading (6.70 ft), it fills 2022-12-01 to 2023-01-08 (other staff readings within 0.14 to 0.35 ft). The 2022-12-31 breach now has a measured level and is scored (17 breaches, from 16), and its breach time is 23:45 (East Bridge apex), not 16:38.
Aug to Sep 2020. The Venoco logger slipped about 1.3 ft on 8/14 to 8/15 and read 2 ft above the staff gauge until it was serviced on 9/17. It is dropped there; East Bridge, rehung on 8/12, covers 8/12 to 9/1, and the staff readings cover the rest. The five summer 2020 overtopping events (8/16, 8/18, 8/29, 9/4, 9/13) were artifacts of the slip and are gone from the catalog.
Part 7. The model in use
The website runs the leader of the Part 6 search, a logistic model with four inputs (analysis/berm/export_best.py best_daily, models/best_daily.json).
Input
Meaning
rain_in
rain that day (in, Coal Oil Point gauge scale), the only input taken from the future, as a perfect forecast
rain_wy
rain since October 1 of the water year, through that day (in)
log_dc
natural log of the berm’s age in days since the mouth closed
maxlvl
the highest measured daily mean level since the mouth closed, through the day before (ft NAVD88)
There is no same-day level term: the score rests on how high this berm has already held the slough, how old it is, how wet the season has been, and the day’s rain. Tide, surge and waves are not inputs. Everything below uses the reviewed labels and the measured-level closed-mouth days of Parts 4 to 6.
Code
import jsonimport engine as BELIVE = ["rain_in", "rain_wy", "log_dc", "maxlvl"]LJ = json.loads((ROOT /"models"/"best_daily.json").read_text())FA = BE.with_crest(BE.load(), "all")FN = BE.with_crest(BE.load(), "nest")sc7 = BE.Scorer(FN)p7 = sc7.loyo(FN[LIVE].values.astype(float))H7 = json.loads((BR /"heldout_best.json").read_text())assert np.abs(np.array([v for _, v in H7["heldout"]]) - p7).max() <1e-4peaks7 = pd.Series({b: p7[ix].max() for b, ix inzip(sc7.breaches, sc7.peak_pos)})n7, nb7 =len(FN), int(FN.breach.sum())md(f"**{n7:,} measured-level closed-mouth days and {nb7} breaches**, WY{FN.water_year.min()} to WY{FN.water_year.max()}.")
2,404 measured-level closed-mouth days and 17 breaches, WY2019 to WY2026.
The fit on all measured days
The model is fit by maximum likelihood on every measured-level day. The 95% intervals are Wald intervals; odds ratios are per unit of each input.
md(f"""A tenth of an inch of rain multiplies the odds by about {np.exp(f7.params['rain_in'] /10):.2f}, and a tenthof a foot more held by the berm by {np.exp(f7.params['maxlvl'] /10):.2f}. An older berm scores lower at thesame held level (log_dc {f7.params['log_dc']:.2f}), and a wetter season a little higher (rain_wy{f7.params['rain_wy']:.3f} per inch). The intervals are wide because there are only {nb7} breaches({nb7 /4:.0f} per input). The site's coefficients come from the same unpenalized fit run with scikit-learn.""")
A tenth of an inch of rain multiplies the odds by about 1.54, and a tenth of a foot more held by the berm by 1.23. An older berm scores lower at the same held level (log_dc -1.87), and a wetter season a little higher (rain_wy 0.050 per inch). The intervals are wide because there are only 17 breaches (4 per input). The site’s coefficients come from the same unpenalized fit run with scikit-learn.
Held-out score through time
Each day’s score comes from a fit that did not see its season. This is the series the Model tab shows (analysis/berm/results/heldout_best.json, from export_heldout.py).
Code
LV7 = [(l["key"].capitalize(), l["min"]) for l in LJ["levels"] if l["min"] >0]wys7 =sorted(FN.water_year.unique())fig, axs = plt.subplots(len(wys7) //2+len(wys7) %2, 2, figsize=(10, 9), sharey=True)ps = pd.Series(p7, index=FN.index)for ax, wy inzip(axs.flat, wys7): s = ps[FN.water_year.values == wy].asfreq("D") ax.plot(s.index, s.clip(lower=1e-5).values, lw=0.9, color="#2D5A7B")for name, lo in LV7: ax.axhline(lo, ls="--", lw=0.6, color="#A65A3C"if lo == LV7[0][1] else"#888") bb = [b for b in sc7.breaches if FN.water_year[b] == wy] ax.scatter(bb, [max(ps[b], 1e-5) for b in bb], s=36, facecolors="none", edgecolors="#C0392B", lw=1.4, zorder=3) ax.set_yscale("log"); ax.set_ylim(1e-5, 1.5) ax.set_xlim(pd.Timestamp(f"{wy -1}-10-01"), pd.Timestamp(f"{wy}-09-30")) ax.set_title(f"WY{wy}", fontsize=9, loc="left") ax.xaxis.set_major_formatter(plt.matplotlib.dates.DateFormatter("%b")) ax.tick_params(labelsize=7)for ax inlist(axs.flat)[len(wys7):]: ax.axis("off")fig.tight_layout();
Figure 13: Held-out daily score, by water year. Each point is a measured-level closed-mouth day scored by the model fit without that season (log scale). Red circles mark breach days; dashed lines mark the alert levels. Gaps are open-mouth days and days without a measured level.
Each breach
The held-out peak is the highest held-out score on the breach day or the day before.
Code
pd.DataFrame({"Breach": [str(b.date()) for b in peaks7.index], "Held-out peak": peaks7.values.round(4),"Rain (in)": FN.rain_in[peaks7.index].values, "Season rain (in)": FN.rain_wy[peaks7.index].values.round(2),"Days closed": FN.dc[peaks7.index].values.astype(int), "Highest held (ft)": FN.maxlvl[peaks7.index].values.round(2),"Level (ft)": FN.stage[peaks7.index].values.round(2)}).set_index("Breach")
Held-out peak
Rain (in)
Season rain (in)
Days closed
Highest held (ft)
Level (ft)
Breach
2019-01-07
0.6215
1.61
7.04
99
8.02
9.00
2020-03-16
0.7103
1.16
9.71
362
9.73
9.81
2020-04-06
0.1322
0.13
12.12
12
5.76
6.40
2021-01-28
0.8456
2.81
6.89
291
7.04
9.12
2021-12-23
0.9899
3.38
7.26
312
7.30
8.91
2022-12-31
0.7065
1.96
7.50
350
8.47
9.22
2023-02-25
0.9279
0.44
18.31
22
7.18
8.15
2023-03-10
0.9846
2.06
21.50
7
5.77
6.64
2023-03-14
0.9993
2.54
24.09
3
5.37
6.39
2024-01-22
0.8668
1.62
8.29
313
9.67
10.30
2024-02-01
0.9755
2.52
11.36
4
4.81
6.08
2024-03-06
0.2009
0.75
22.26
16
6.96
5.28
2024-03-31
0.1386
0.71
25.17
17
6.32
7.05
2025-02-13
0.9378
1.75
7.60
315
9.45
10.25
2025-03-13
0.1359
0.20
9.33
11
6.03
6.58
2025-11-15
0.2647
1.74
5.78
245
7.44
9.35
2026-02-19
0.8088
1.00
21.00
14
7.56
7.93
Levels
The lowest level starts at the lowest held-out breach peak, rounded down to the nearest 0.5%; the others sit at 0.2 and 0.5. False-alarm days are days at or above the level more than two days from any breach; days separated by gaps of three days or less form one episode.
Code
rows = []for name, lo in LV7: fa = np.where((p7 >= lo) &~sc7.near)[0] rows.append({"Level": name, "Score at least": lo, "Breaches reaching it": f"{int((peaks7 >= lo).sum())}/{nb7}","False-alarm days": len(fa), "Episodes": len(sc7.episodes(fa))})ep7 = sc7.episodes(np.where((p7 >= LV7[0][1]) &~sc7.near)[0])md(f"**False-alarm episodes at {LV7[0][0]}:** "+ ("; ".join(f"{a.date()} to {b.date()} ({n} days)"for a, b, n in ep7) or"none"))pd.DataFrame(rows).set_index("Level")
False-alarm episodes at Watch: 2019-12-22 to 2019-12-22 (1 days); 2020-01-16 to 2020-01-16 (1 days); 2020-03-10 to 2020-03-10 (1 days); 2023-12-21 to 2023-12-21 (1 days); 2025-01-26 to 2025-01-26 (1 days); 2026-02-16 to 2026-02-16 (1 days)
Score at least
Breaches reaching it
False-alarm days
Episodes
Level
Watch
0.13
17/17
6
6
Warning
0.20
14/17
3
3
Likely
0.50
12/17
1
1
Against the other models
The search covered 6,886 models (every logistic model of one to five of the 16 candidate inputs, plus the crest-curve model, refit). The best ten, then two models for comparison: rain_in + rain_prev + margin + margin_max and the crest-curve model:
The lowest level was set from all eight seasons’ breaches, so it has seen the season it scores. The season-held-out (cross-validated) check sets each season’s line only from the other seasons’ breaches.
Code
r7 = sc7.score(p7, detail=True)hrows = []for wy in sc7.seasons: c = r7["h_cut"][int(wy)] ks = np.where(sc7.bwy == wy)[0] fa = np.where((sc7.wys == wy) & (p7 >= c) &~sc7.near)[0] hrows.append({"Season": f"WY{wy}", "Line (from other seasons)": round(c, 4), "Breaches": len(ks),"Caught": int((peaks7.values[ks] >= c).sum()), "False-alarm days": len(fa),"Episodes": len(sc7.episodes(fa))})HT = pd.DataFrame(hrows).set_index("Season")HT.loc["All"] = ["", HT.Breaches.sum(), HT.Caught.sum(), HT["False-alarm days"].sum(), HT.Episodes.sum()]md(f"**Honest check: {r7['h_caught']} of {nb7} caught, {r7['h_fa_days']} false-alarm days in {r7['h_fa_ep']} episodes**, "f"missing {r7['h_missed']}.")HT
Season-held-out check: 16 of 17 caught, 4 false-alarm days in 4 episodes, missing 2020-04-06.
Line (from other seasons)
Breaches
Caught
False-alarm days
Episodes
Season
WY2019
0.1322
1
1
0
0
WY2020
0.1359
2
1
2
2
WY2021
0.1322
1
1
0
0
WY2022
0.1322
1
1
0
0
WY2023
0.1322
4
4
0
0
WY2024
0.1322
4
4
1
1
WY2025
0.1322
2
2
1
1
WY2026
0.1322
2
2
0
0
All
17
16
4
4
Inputs known in advance
Only the day’s rain is taken from the future (a perfect forecast). Level, berm age, highest level held, the already-breached flag and every simulated level come from data through the day (or hour) before. leakage_test.py rebuilds every input with all breach labels moved a day earlier and later: no input on the breach days changes. The audit, input by input, is in analysis/berm/leakage_audit.md.
Summary
Code
su = score(pu)md(f"""**Model in use (Part 7).** The website runs **rain_in + rain_wy + log_dc + maxlvl**(`models/best_daily.json`), fit on {n7:,} measured-level closed-mouth days with {nb7} breaches. With eachseason's line set only from the other seasons it catches {r7['h_caught']} of {nb7} ({r7['h_missed']} missed) with{r7['h_fa_days']} false-alarm days in {r7['h_fa_ep']} episodes. It replaces rain_in + rain_prev + margin +margin_max, whose margin_max leaked the breach label (fixed; Part 7).The crest-curve model below is the earlier model.**Breach inputs (Part 1).** Among {len(res)} logistic models with up to four inputs, the best held-out performance camefrom **{top.inputs}**: held-out log loss {top.cv_logloss:.4f} versus {clim['cv_logloss']:.4f} for a constant rate,with {int(top.caught_p50)} of {n_ev} breaches caught at 50% and {int(top.false_alarm_days_p50)} false-alarm days.The crest-curve model (nonlinear crest curve plus rain) scores {su['cv_logloss']:.4f} and catches {su['caught_p50']}of {n_ev} with {su['false_alarm_days_p50']} false-alarm days. Tide height, surge and waves{'appear'ifany(c in top.inputs for c in ['tide_max', 'surge', 'hs_max', 'wave30']) else'do not appear'} in the best model.With {n_ev} events, differences of a few thousandths in log loss are within noise; the events-per-variable limitand the bootstrap sign stability are the guardrails.**Slough rise.** Across {len(E)} storms the best leave-one-storm-out model was **rise ~ {RSEL}**(error {RS.LOO_RMSE_ft.iloc[0]:.2f} ft, versus {RM.set_index('model').loc['R0 intercept only', 'LOO_RMSE_ft']:.2f} ftfor the mean alone). The daily distributed-lag model, using {int(mb.nobs):,} closed-mouth days, puts the response at{float(np.asarray(wald.effect)[0]):.2f} ft per inch of rain spread over {best_nl} day(s), plus {mb.params['hinge']:.2f} ft perinch of a single day's rain above 0.5 in.""")
Model in use (Part 7). The website runs rain_in + rain_wy + log_dc + maxlvl (models/best_daily.json), fit on 2,404 measured-level closed-mouth days with 17 breaches. With each season’s line set only from the other seasons it catches 16 of 17 (2020-04-06 missed) with 4 false-alarm days in 4 episodes.
Breach inputs (Part 1). Among 385 logistic models with up to four inputs, the best held-out performance came from rain_in + rain_3d + log_dc + stage: held-out log loss 0.0141 versus 0.0400 for a constant rate, with 10 of 17 breaches caught at 50% and 4 false-alarm days. The crest-curve model (nonlinear crest curve plus rain) scores 0.0233 and catches 12 of 17 with 8 false-alarm days. Tide height, surge and waves do not appear in the best model. With 17 events, differences of a few thousandths in log loss are within noise; the events-per-variable limit and the bootstrap sign stability are the guardrails.
Slough rise. Across 34 storms the best leave-one-storm-out model was rise ~ total + stage0 (error 0.33 ft, versus 0.90 ft for the mean alone). The daily distributed-lag model, using 2,365 closed-mouth days, puts the response at 0.66 ft per inch of rain spread over 2 day(s), plus 0.36 ft per inch of a single day’s rain above 0.5 in.