Model Validation — Out-of-Sample Backtest
Hummingbird physics model output vs confirmed final settled loss, scope-matched. Final benchmarks are FHCF-certified FL residential wind or LA DOI certified residential wind — identical scope to the model. All dollar figures in model-run-year dollars (2020 SI).
Scope rule: No all-perils estimates, no preliminary D+7 figures, no mismatched geographies. Benchmark must be a confirmed regulatory settlement for the same peril, geography, and building class as the model output. Physics model only — no behavioral loading applied to the primary result.
🌀 Hurricane — Wind Residential · Physics Only
Final benchmark is FHCF-certified FL residential wind or LA DOI certified residential wind — identical scope to the model. No behavioral loading applied.
🌨 Hail — Residential + Auto · Ground-Up vs NCEI
Model output is ground-up loss covering residential roof damage (Ortega-corrected MRMS MESH + lognormal fragility), ancillary losses (gutters, HVAC condensers), and auto.
Benchmark is the NCEI Billion-Dollar total-economic estimate, which includes commercial property and agricultural losses the model does not cover.
Expected model coverage: 70–85% of NCEI total given residential+auto scope.
Methodology
Hurricane — Physics modelHolland B parametric wind field + HAZUS fragility curves. No behavioral loading in primary result (hb_pbl = 1.00). Surge, BI, auto, and demand surge are outside scope. Model runs against 2020 structural inventory at 2020 replacement costs.
Hurricane — Benchmark sourceFHCF (FL Hurricane Cat Fund) certifies residential wind losses for the FL reinsurance treaty — the same residential wind scope the model outputs. LA DOI residential wind reporting is the LA equivalent. Both are post-settlement regulatory figures, not industry estimates.
Hail — Physics modelMRMS MESH_Max_1440min (NOAA S3) + Ortega 0.75× bias correction + lognormal fragility (θ=46mm unrated shingle). WBZ melt correction (Brimelow 2002) applied per event. Ancillary: gutters (lognormal θ=25.4mm) + HVAC condensers (FM Global DS 1-34). Auto (θ=22mm). Residential scope only — commercial flat/membrane roofs not modeled.
Hail — Benchmark sourceNCEI Billion-Dollar Disasters total-economic estimate. Broader scope than model: includes commercial property, crops/agriculture, and infrastructure. No per-peril residential-only regulatory equivalent exists for hail. Expected model coverage: 70–85% of NCEI total (residual = commercial + ag). Fort Worth Apr 2021 excluded from this table — ICT insured benchmark only (different measurement basis).