Progress at a glance
Status as of 2026-07-24 (same day, in order: Grid Dashboard shipped,
then the landing page hub — see below). Ottawa Case Study = heat-demand
Phases 0–3 of 6 done (stock, load, electrified load; next: feeder aggregation → map → case-study page) —
no movement since the 2026-07-17 session, still the next thing to build. Retrofit Insights
(national big-picture page) shipped 2026-07-19; New Homes Explorer shipped 2026-07-15 and had its
pipeline substantially extended 2026-07-22/23 (per-fuel energy, per-component heat loss, full
heat-pump detail — see its section below). Retrofit Explorer gained a theme system, a reworked
census card, and a combo-chart fix 2026-07-19–21; its AHRI badge lookup now refreshes itself
weekly instead of needing a manual rerun (shipped 2026-07-22). Construction Tracker's first
scheduled monthly refresh ran clean 2026-07-20. ROADMAP.md remains the day-to-day
tracker; this page is the deeper "how and why" reference, refreshed alongside it.
How to read the tags on each assumption
judgment call a modelling choice Claude made — these now list the alternatives that existed and a hands-on way to verify.
source limit a limitation baked into the source data — nobody chose it, but you should know it.
calibrated a number that was cross-checked against an independent published figure.
standard factor a published constant (unit conversion, emission factor) taken from literature.
Verification effort is marked ⏱️ 2 min (click something / open a page), ⏱️ 15 min (compare against a published source), or ⏱️ deeper (re-run a script with a changed setting).
Build timeline (from git history — dates are approximate for uncommitted work)
Everything was built between June 23 and July 10, 2026 — about 2½ weeks.
Green dot = the day it went live on GitHub Pages. Not pictured: the suite kept shipping after
this chart was drawn — Retrofit Insights (2026-07-19), New Homes Explorer (2026-07-15) and the
Grid Dashboard page (2026-07-24) all went live, and the Ottawa Case Study, Retrofit Explorer polish
and grid/AHRI automation continued through
2026-07-24. See the timestamps in each section above, or git log, for the current
picture — redrawing this SVG accurately for a 5-week-and-growing history is due for a rebuild rather
than a patch.
In one sentence: pick your province or postal area and see what energy retrofits real Canadian homes actually did, and how much energy their audits say they saved.
Where the data comes from
How it was built
Full methodology, in plain language
Step 1 — turning raw audits into before/after pairs
Every audit record carries an evaluation type: D = the initial "before" audit, E = the follow-up "after" audit. The pipeline keeps homes with at least one D and at least one E, reducing a re-audited home to its earliest D and latest E (changed 2026-07-24 — it previously required exactly one of each, which discarded 149,145 re-audited homes; see the open questions on retrofits.html). It requires the E to be dated later, then applies "same home?" sanity checks: floor area changed ≤ 10%, and house type / storeys / number of units identical. Anything failing is dropped — better to lose a real pair than to compare two different houses. About 45 source columns are then mapped to friendly names and all fuels converted to kWh so they share one chart:
| Fuel | Source unit | → kWh factor |
|---|---|---|
| Natural gas | m³ | × 10.361 |
| Oil | L | × 10.2 |
| Propane | L | × 7.092 |
| Wood | tonne | × 4,166.7 |
| Total/heating energy | MJ | × 0.27778 |
Step 1 also computes the "what did they do?" flags
Insulation upgrade = post R-value > 1.10 × pre. Air-tightness upgrade = post leakage < 0.90 × pre. Deep retrofit = ≥ 50% energy saved; medium 10–50%; shallow 0–10%. Heat pump added = none before, one after. Fuel switch = primary heating fuel differs. Saving % = (before − after) ÷ before, so positive always means energy saved.
Steps 2–3 — why the data is pre-chewed
The province view never touches raw rows: a Python script pre-computes every median, count
and histogram and ships them as one summary file per province. The postal-area (FSA) view
does get raw rows (one compact file per area — arrays instead of objects, ~77% smaller)
so you can browse individual homes. This creates the suite's one sharp edge: the bin-width
contract. Python bins the province histograms; JavaScript bins the FSA ones live. Both must
use identical bucket widths (10-yr year bins, 50 m² area bins, 20 kWh/m² EUI bins, …) or the same
data looks different in the two views. The widths live in one BINS object in the page
and matching step= arguments in the Python — change one, change both.
Front end
One self-contained file. A fetch-token guard stops a slow older request from overwriting a
newer selection; payloads are cached so revisiting is instant. Postal-code quick-find, shareable
URLs (?prov=ON&fsa=K0A), colour-blind-safe pre/post palette, keyboard-accessible
Sankey. A July 2026 accuracy pass fixed a mislabelled unit (design heat loss is kW, peak
demand — not GJ/yr) and a bin mismatch, and added the selection-bias disclosure.
⚠️ Assumptions & judgment calls — check these
- source limitSavings are computer-modelled, not measured from utility bills.
Every energy number comes from NRCan's HOT2000 model of the house, not real meter readings. Real usage varies with weather and behaviour — studies of modelled-vs-metered typically find models overpredict savings for deep retrofits.
✅ Verify it yourself ⏱️ 15 min- Open the tool, enter your own postal code, and find a home similar to yours in the table.
- Compare its pre-retrofit total energy (kWh/yr) to your own annual gas + hydro bills converted to kWh (gas m³ × 10.36). Same ballpark = the modelling is sane; a big gap tells you how much to trust absolute numbers vs comparisons.
docs/RETROFITS.md §"Data notes & caveats"; also stated on the page itself. - source limitThe sample is self-selected — savings likely look better than average.
Only homes that finished their retrofit AND booked a follow-up audit are counted — mostly rebate-program participants (e.g. Greener Homes required the follow-up). A random renovation would probably save less.
✅ Verify it yourself ⏱️ 15 min- Compare the tool's median saving (~13% ON) against a published program evaluation — NRCan's Greener Homes reporting or Efficiency Canada's program reviews quote similar per-participant figures, which confirms the data is program participants, not typical renovators.
docs/RETROFITS.md §"Data notes & caveats" → "The sample is self-selected". - judgment callWhat counts as "the same home": area change ≤ 10%, same type/storeys/units.
Pairs are dropped unless the before and after audits agree on these.
Other options: stricter (≤ 5% area — fewer homes, cleaner pairs), looser (≤ 20% — more homes, more risk an addition or a different building sneaks in), or no area check at all. There's no "right" number; 10% balances sample size against contamination.
✅ Verify it yourself- ⏱️ 2 min: in any FSA view, expand a few homes and eyeball that pre/post floor area, type and storeys genuinely match.
- ⏱️ deeper: edit the 10% constant in
Python/ers_web_pipeline.py, re-run one province, and compare the median saving. If it barely moves (±1–2 points), the choice doesn't matter — that's the robustness test.
docs/RETROFITS.md §"The data pipeline" step 1, rules 2–3. - judgment callWhat counts as an "upgrade": >10% more insulation, >10% tighter air, ≥50% saved = "deep retrofit".
These cutoffs decide the headline counts ("X% added insulation"). A 9% insulation increase counts as nothing.
Other options: absolute thresholds (e.g. +0.5 RSI regardless of starting point), any-positive-change (noisy — audit measurement wobble would count), or the industry convention for "deep retrofit" (some use ≥40%, some ≥60%; 50% is the common midpoint and matches NRCan's usage).
✅ Verify it yourself- ⏱️ 2 min: expand a home flagged "roof insulation upgraded" in the FSA table and check its pre/post RSI values — is the flag intuitive?
- ⏱️ 15 min: compare the "deep retrofit" definition against NRCan / Pembina publications ("deep retrofit" ≈ 50%+ energy reduction is the standard usage).
docs/RETROFITS.md §"How each measure is flagged" (full table of rules). - standard factorAll fuels converted to kWh with published factors (gas 10.36 kWh/m³, oil 10.2 kWh/L, wood 4,167 kWh/t…).
Standard higher-heating-value energy contents.
✅ Verify it yourself ⏱️ 15 min- Cross-check against the published conversion-factor tables (natural gas ≈ 37.30 MJ/m³ ÷ 3.6 = 10.36 kWh/m³ ✓; light fuel oil ≈ 38.80 MJ/L ÷ 3.6 ≈ 10.78 ✓ — StatCan RESD 57-003-X, the same value ECCC's National Inventory Report uses). Any energy-content reference will do.
docs/RETROFITS.md §"Unit conversions" table. - source limit"Solar added" may slightly overcount.
Before-audits rarely record existing panels, so panels present at the after-audit all look "added".
✅ Verify it yourself ⏱️ 2 min- In an FSA view, look at homes with solar: the pre-audit PV column is almost always 0 even in areas where some homes certainly had panels before — that's the tell.
docs/RETROFITS.md §"Data notes & caveats" → "Solar pre vs post". - judgment callExtreme outliers are clipped from charts (energy intensity > 500 kWh/m², emissions > 30 t/yr, heat loss > 150 kW).
Purely for chart readability — the homes still exist in the data and in the medians.
Other options: log-scale axes (keeps everything visible but harder to read), percentile-based clipping (99th), or an explicit "+N above range" bar.
✅ Verify it yourself ⏱️ 2 min- Check that a clipped chart's total count still matches the header's "matched homes" count — clipping only hides bars, it doesn't drop homes from the stats.
docs/RETROFITS.md §"Data notes & caveats" → "Outliers clipped". - judgment callEmissions default to a calculated basis, not the audit's own reported figure.
The audit's
ERSGHGfield is only filled for 50.5% of matched homes (Quebec ~78%, Saskatchewan ~9%), so building every emissions chart on it would have quietly halved the sample. The page now offers 4 bases and defaults to "corrected, at year of audit" — each home's emissions recomputed from its own recorded fuel use, at factors calibrated against the ERS data for that audit year.Other options: show only the reported figure (honest but half the data, and biased by which provinces report it); use ECCC's official current factors flat across all years (comparable across eras, but overstates Ontario's historic gas and understates Alberta/Newfoundland electricity by 18–49%); or scale the reported subset up to the full population (assumes the reporting homes are representative, which is exactly what isn't known).
✅ Verify it yourself ⏱️ 3 min- Switch the "GHG basis" dropdown to "as reported" and watch the home count under the chart drop — that gap is the coverage problem this exists to solve.
- The calibrated basis was validated against the reported one where both exist: national aggregate within −0.66%, every large-sample province within ~±2% except Quebec (−5.3% on a small base).
docs/RETROFITS.md §"GHG scenarios"; retrofits.html Methodology J; open question in docs/ENERGUIDE_QUESTIONS.md §5.4.
✓ How it was checked
- A July 2026 accuracy pass audited every label against the source columns — found and fixed a mislabelled unit (design heat loss is kW, not GJ/yr) and a chart-binning mismatch between the two views.
- The province view and postal-area view are computed two different ways (Python vs JavaScript) and must agree — a documented "bin-width contract" keeps them in sync.
✅ Done since launch
- 2026-07-19/20: reworked the census card around a coverage gauge, added a Sources section, added a light/dark/colour-blind theme system, reworded the hero and trimmed the heat-pump aside.
- 2026-07-21: fixed the pre/post combo-chart's bar/area draw order and switched pre/post colours to red/green for clarity.
- 2026-07-22: the AHRI heat-pump badge lookup (
lookup/ahri_numbers.json) is now refreshed automatically every Monday viaahri-refresh.yml, re-checking existing certificates on a rolling basis (catches AHRI amending a certificate after the fact) instead of needing a manual rerun. - 2026-07-31: the Retrofit Costs proof of concept went live inside the page — an estimated retrofit cost and payback per home, priced from outside the ERS data. See its own section below.
- 2026-08-02: emissions rebuilt on 4 selectable bases instead of the audit's own half-populated figure (assumption card above), and a survivorship-bias bug in the factor derivation fixed — it had been overstating the national total by 12.8%.
- 2026-08-03: a hoverable pipeline-overview diagram added at the top of the advanced methodology, so the CSV → parquet → JSON → page path (and its emission-factor and cost branches) can be read at a glance rather than reconstructed from prose.
In one sentence: what a home's recorded upgrades would have cost, and how long they take to pay back — the one question the audit data itself cannot answer, answered from an outside source and labelled as such everywhere it appears.
Where the data comes from
How it was built
⚠️ What had to be assumed
This is the least settled thing on the site. Every item below is a reason the numbers are directional, not quotable.
- source limitThe cost data is American, unconverted.
REMDB is a US database in 2023 USD. No currency conversion and no Canadian-labour adjustment has been applied anywhere — the figures are used as-is. Canadian installed costs differ by trade, region and season in ways a flat multiplier would not capture honestly, so none was invented.
Other options: apply a headline FX rate (cheap, but implies a precision the underlying regression doesn't have); commission or source Canadian per-measure costs (the right answer, no public dataset found); or show ranges only, no point estimates.
✅ Verify it yourself ⏱️ 5 min- Compare a mid-band heat-pump figure on the page against any Canadian contractor quote you have. Expect the page to read low — that gap is roughly the size of what's missing.
docs/RETROFIT_COSTS.md; retrofits.html Methodology H. - judgment callHouse shape is assumed, because ERS records no envelope areas.
Insulation is priced per square metre, but the public ERS extract carries no wall or roof area anywhere. Areas are derived from the recorded footprint under an assumed 2:3 rectangle — a stated judgement call, not a sourced distribution of Canadian house shapes.
Other options: a square footprint (simpler, slightly lower wall area); source a real footprint-aspect distribution from assessment or building-permit data (the intended fix, on the roadmap); or decline to price envelope measures at all.
✅ Verify it yourself ⏱️ 10 min- Measure your own house's footprint and compare its true wall area against a 2:3 rectangle of the same floor area. The error is usually small; on a long narrow or L-shaped house it is not.
docs/RETROFIT_COSTS.md §"Area proxies". - source limitCoverage is partial, in three separate ways.
Eight measures are priced (roof/wall/foundation insulation, air sealing, windows, heat pump, solar PV, HRV/ERV). Furnaces, boilers, floor insulation and doors are not priced at all. Apartments, duplexes and triplexes are excluded entirely — 31,389 of the 1,451,433 matched pairs — because REMDB's regressions assume single-dwelling geometry. Of the 1,420,044 that remain, 13% end up with no priced measure. Yukon has no ERS records at all.
✅ Verify it yourself ⏱️ 2 min- In a postal-area view, compare the "homes with a cost estimate" count against the matched-home count. The difference is this gap, not a rendering bug.
docs/RETROFIT_COSTS.md §"Measures explicitly NOT priced"; retrofits.html Methodology H. - source limitThe payback denominator rests on utility rates we don't fully trust.
Payback = cost ÷ annual dollars saved, so it inherits every weakness of the rate table. The electricity source was replaced in July 2026 after the previous one was caught quoting a Saskatchewan rate ~67% below SaskPower's own schedule while flagged "high confidence"; the replacement has been cross-checked only for Saskatchewan. Natural gas rates are frozen at a 2024-10-01 effective date across every province, ~21 months stale, with no verified replacement found. Oil, propane and wood are single national screening constants. Every affected home carries a machine-readable flag.
✅ Verify it yourself ⏱️ 10 min- Look up your own utility's current residential rate and compare it to what the page uses for your province. Where they diverge, the payback figure moves proportionally.
docs/RETROFIT_COSTS.md §"Utility rates"; retrofits.html Methodology F and H. - judgment callThe HRV/ERV price is a class average, not a per-home fit.
REMDB fits ventilation cost on recovery efficiency and airflow. Neither field survives the main pipeline, so every HRV is priced at fixed midpoint metrics (0.70 SRE, 150 CFM) and tagged
flat_placeholder_metricsin the data rather than presented as a per-home number.retrofits.html Methodology H.
✓ How it was checked
- Each priced measure's REMDB area basis was audited against the raw REMDB data sheets rather than assumed — air sealing bills on house floor area, windows on the window's own area, wall and roof on surface area, all three confirmed.
- The heat pump is netted against the home's own pre-audit heating system where ERS records one, not a blanket gas-furnace assumption; two REMDB rows the database never fitted (oil furnace, electric boiler) were derived from its raw line-item workbook instead of guessed.
- Checked directly against the NRCan data dictionary that ERS carries no wall-construction-type field, so Wood/Steel Stud + Batt stays an explicitly stated assumption rather than an oversight.
🔨 What's next
- Band-specific payback (currently the mid band only).
- A better utility-rate source — electricity verified beyond Saskatchewan, and something current for natural gas.
- A real footprint aspect-ratio dataset to retire the 2:3 assumption.
- Verify the province-mode UI against deployed data.
In one sentence: the national, cross-sectional sibling of the Retrofit Explorer — the same 1,451,433 matched retrofit pairs, but zoomed out to "what actually works, and where" instead of one postal area at a time. Shipped 2026-07-19.
Where the data comes from
How it was built
Full methodology, in plain language
What's actually new vs. the Retrofit Explorer
retrofits.html is deep on one area at a time; this page is the cross-sectional view — leaderboards, a Canada-wide FSA choropleth with a metric switcher, a "what makes a retrofit successful" breakdown, climate and income-equity lenses, heat-pump geography, cold-climate equipment (an AHRI COP-vs-capacity-maintenance scatter plus a screen against the US DOE Cold Climate Heat Pump Challenge specifications — added 2026-07-30), a missed-opportunity ranking, and a program-era timeline (EnerGuide for Houses → ecoENERGY Retrofit–Homes → Canada Greener Homes Grant). Scope was deliberately kept descriptive + stratified — no fitted regression models in v1.
The single most robust finding: savings scale with # of measures
Zero-measure pairs (118,282, 8.6% of all pairs) are the audit-noise population flagged in docs/RETROFITS.md — a re-audit with no tracked upgrade, median "saving" 3.3%. Every "what worked" statistic on the page (bundles, top-decile, climate/equity medians) excludes them and instead quotes the 1,251,023 nonzero-measure pairs; the zero-measure count is reported separately, never silently dropped. Above that floor, savings rise cleanly with measure count — 1 measure ≈16% median saving, 4 measures ≈34%, 7–8 measures ≈62–69% — and heat pump alone (141,677 pairs, 25.5% median saving) clearly outperforms windows alone (66,744 pairs, 6.1%).
The missed-opportunity composite ranking
A per-FSA score combining worst stock (high pre-retrofit EUI, high GHG, older vintage) and lowest program uptake so far — "where should a program look next?" All factors are percentile-ranked 0–1 first so different units combine fairly. Reworked 2026-07-20 from continuous weights (0.30/0.25/0.20/0.25) to the same ordinal scale (Off/Low/Medium/High → 0/1/2/3, normalised to sum to 1) the page's own reader-facing priority controls use, so the shipped default ranking is something a reader can actually reproduce from the UI rather than a number they can't reach. The rework barely moved the ranking (18 of the top 20 FSAs unchanged, same #1, median rank shift 22 places out of 1,509) — evidence the reweighting was cosmetic, not a substantive change to "who's flagged".
⚠️ Assumptions & judgment calls — check these
- judgment callZero-measure pairs are excluded from every "what worked" statistic, reported only as a count.
118,282 pairs (8.6%) logged no tracked upgrade between audits — re-audits, not retrofits. Including them would dilute every measure-effectiveness number with noise.
Other options: include them as a "0 measures" bucket in the bundle table (the previous approach in retrofits.html's own stats — muddies the "what worked" question this page asks); or try to reclassify some as real (untracked) retrofits (no reliable signal to do so).
✅ Verify it yourself ⏱️ 2 min- On the page, the "what worked" section states its denominator (1,326,144) explicitly — check it's smaller than the 1,451,433 hero number, and the gap (125,289) matches the zero-measure count called out in the methodology accordion.
Python/insights_notes.md §1; docs/archive/ROADMAP_COMPLETED.md item 13 "Honesty rails". - judgment callThe missed-opportunity ranking is a reader-adjustable weighted composite — the reader can re-weight all 5 factors live and the table re-sorts client-side from stored per-FSA percentiles.
Default weights: pre-EUI High, pre-GHG Medium, pre-1980 share Low, low participation Medium, low income Off (income is emitted but not weighted by default — see next assumption).
Other options: a fixed, non-adjustable formula (simpler, but hides the judgment call instead of exposing it); a single "priority score" with no factor breakdown.
✅ Verify it yourself ⏱️ 2 min- Open the missed-opportunity section and change a factor's priority slider — the ranking should visibly re-sort without a page reload (nothing is recomputed server-side, it's arithmetic over the stored percentiles).
Python/build_insights.py "opportunity.json" section (OPP_WEIGHTS + comment); the page's priority controls. - source limitIncome correlations are ecological (FSA-level), not household-level.
"This FSA's median income is $X" describes the postal area, not the specific households that got audited. "Rich FSA" ≠ "rich participant" — a common mixing-of-levels trap the page states on-page.
✅ Verify it yourself ⏱️ 15 min- Cross-check the national dwelling-weighted median income the page implies (~$82,000) against StatCan's 2020 published Canada median (~$84,000) — close but not identical, since this is a dwelling-weighted FSA aggregate, not a direct household survey figure.
Python/insights_notes.md intro cross-checks; equity section methodology note on-page. - source limitParticipation rate mixes ~20 years of cumulative audits over a single 2021 census dwelling snapshot — it can exceed 100%.
A FSA where more homes have EVER been audited (2004–2026) than existed as dwellings in the 2021 census will show >100% participation. Not a bug; stated on-page wherever the metric appears.
✅ Verify it yourself ⏱️ 2 min- Sort the participation leaderboard and check whether any FSA sits above 100% — if so, confirm the page's own caveat text next to that metric explains why.
docs/archive/ROADMAP_COMPLETED.md item 13 "Honesty rails". - judgment call"Screened against the DOE Challenge specifications" — never "meets" or "certified for" the Challenge.
The screen checks published AHRI/ENERGY STAR certificate ratings against Table II-3 of the US DOE Cold Climate Heat Pump Challenge specifications. The Challenge itself is a verification programme with its own H11/H1N lab test protocol we do not hold, so a unit clearing every checkable criterion has rating-consistent performance and nothing more — the output column is named
screen_pass, deliberately not "meets" or "certified". Only 4 of 8 Table II-3 criteria are checkable from the ratings we hold; the other 4 (turndown ratio, compressor cut-out/cut-in, electric-heat staging, ENERGY STAR CACHP sections) are shown as explicitly not checkable rather than silently dropped. Of 15,148 models / 439,975 ERS appearances, 4 certificate rows (3 distinct models, 8 appearances, 0.00%) clear every checkable criterion — three of the four sit exactly on the COP threshold, which is expected since Challenge-qualifying products only reached market in 2024–25 and this is a historical audit record.Other options: say a passing unit "qualifies for" or "meets" the Challenge (overclaims a lab verification test we never ran); silently drop the 4 uncheckable criteria instead of showing them (hides how much of the spec we can't actually check).
✅ Verify it yourself ⏱️ 3 min- Search the "Cold-climate equipment" section and its methodology text for the word "meets" or "certified" next to "Challenge" — it should not appear; the wording is always "screened against the specifications".
- Check the qualifying-models table: it should show 4 rows but call out 3 distinct models, since GREE's GUD60W2/NHE-D(U) is AHRI-certified under two reference numbers.
HeatPump/pipeline/screen_cchp.py docstring; HeatPump/METHODOLOGY.md "US DOE Cold Climate Heat Pump Challenge screen"; docs/archive/ROADMAP_COMPLETED.md item 14.
✓ How it was checked
- National matched total in insights_json (1,451,433) cross-checked against the sum of the retrofit-explorer province files — exact match.
- Spot-checked FSAs (L3R, K2P, M5V) reproduce the same median saving % as retrofits.html's own
fsa_jsonfor those areas. - National dwelling-weighted median household income (~$82,000) lands close to StatCan's 2020 published Canada figure (~$84,000) — same-ballpark sanity check, not an exact match (different weighting methods).
In one sentence: how efficient Canada's brand-new homes actually are — as-designed (Plan) vs. as-built (blower-door-tested) EnerGuide evaluations, 223,052 matched rows. Shipped 2026-07-15; pipeline substantially extended 2026-07-22/23.
Where the data comes from
How it was built
Full methodology, in plain language
P vs. N — two very different snapshots of the same house
P ("Plan") is evaluated from architectural drawings before a shovel hits the ground — no blower-door test, purely modelled. N ("As-built") is evaluated after construction with a real blower-door air-leakage test. One row per finished, tested new home, with its matching plan record left-joined so as-designed can be compared to as-built. Homes with a plan but no matching as-built record are excluded — they're either still under construction or the paperwork never got filed, and neither case has a real "as-built" number to show.
Small enough to pair in memory
New construction is a small slice of ERS activity (~30k homes/yr, 223,052 total P/N rows nationally vs. 1.45M retrofit pairs) — the whole pairing job runs in memory, no chunking needed.
2026-07-22/23 rework: parity with the Retrofit Explorer's per-fuel detail
The Retrofit Explorer's fsa_json already carries per-fuel heating energy, per-building-component
heat loss and ventilation type (shipped 2026-07-17). This session pulled ~35 more source columns
into newhomes_pipeline.py to give New Homes the same depth: whole-house and space-heating energy
split per fuel (elec/gas/oil/propane/wood, with wood using the same
EGHFCONWOODGJ-first-then-tonnage-fallback derivation as the retrofit pipeline), annual heat loss by
component (air, ceiling, walls, foundation, exposed floor, windows/doors), insulation R-values,
window code, ventilation system type, heating-equipment efficiency (AFUE/seasonal), and full
heat-pump detail (type, COP, HSPF/SEER, capacity, AHRI certificate number). All of it now
flows into the "most efficient new homes" table's expandable per-home detail
(top20_lowest_eui in newhomes_json), which previously only showed year, floor area,
fuel and a plain heat-pump flag. A byproduct data-quality audit
(Python/ers_pn_column_fill.csv) confirms population rates per source column so gaps
are visible rather than silently null.
Year-coverage caveat baked into the page
Tier, ACH (air-changes-per-hour) and ERS-rating fields weren't collected in every audit year — the page states which years each metric actually covers rather than showing a flat "N/A" that looks like a data error.
⚠️ Assumptions & data quirks — check these
- source limitPlan-only homes (no matching as-built record) are excluded entirely.
Homes still under construction, or where the as-built evaluation was never filed, don't appear anywhere in the tool — there's no "as-built" number to show them with.
✅ Verify it yourself ⏱️ 2 min- Compare the total P (plan) record count in a raw ERS CSV for a recent year against newhomes_json's row count for that year — plan-only homes explain the gap.
docs/NEWHOMES.md "One row per finished, tested new home". - source limitTier / ACH / ERS-rating availability varies by audit year.
These fields weren't consistently collected across the full history; the page states year coverage per metric rather than implying full history everywhere.
✅ Verify it yourself ⏱️ 2 min- Filter the page to an early audit year and check whether Tier/ACH show real values or an explicit "not collected this year" state.
docs/NEWHOMES.md "Notes". - source limitShares every caveat the Retrofit Explorer has about modelled (not metered) energy and self-selection.
Same HOT2000-modelled energy numbers, same "only audited homes appear" sampling logic — see the Retrofit Explorer section above for the full detail rather than repeating it here.
docs/RETROFITS.md §"Data notes & caveats" (shared conventions); docs/NEWHOMES.md.
✓ How it was checked
- 223,052 matched P/N rows nationally (newhomes_json/CA.json total_rows) — cross-referenced against the raw ERS new-construction row counts per province during the pipeline build.
- Refreshes on the same ERS CSV drops as the Retrofit Explorer — no separate download; rerun both pipeline scripts after ers_web_pipeline.py's source files update.
In one sentence: Canada's official "where does our energy go" database (~400 government web tables) turned into one interactive page — all 5 sectors, all provinces, 2000–2023.
Where the data comes from
How it was built
Full methodology, in plain language
The design insight
NRCan publishes CEUD as ~50 static tables per sector × 5 sectors × 8 regions — roughly 400 pages. But every table is just a different 2-D slice of one underlying cube: year × energy source × end-use × building type, plus explanatory variables (households, floor space, equipment stock, heating degree-days). So the tool stores the cube once and lets you pick the slice — one interface replaces the tables. Energy intensity per household (which the raw tables make you compute yourself) is a first-class metric.
Finding a machine-readable source (Phase 0 detective work)
The CEUD landing page offers no bulk download. The copy on open.canada.ca turned out to be a
stale 2015-vintage "handbook", not the live database. The usable source: each table page has a
Download button pointing at a real .xls file containing all years at once —
one file per table per region, with a predictable URL pattern. The ETL fetches those.
The ETL's shape and safety rails
Everything is parsed into tidy long-format records ({year, energy source, end-use, value…}) and emitted as one compact JSON per sector × region (target < 200 KB, values rounded to 3 significant digits). Suppressed/confidential cells ("X") become null, never 0. Each run prints a validation report: provinces must sum to the Canada total within ~2%, end-uses must sum to the sector total, no negative values.
⚠️ Assumptions & judgment calls — check these
- judgment callSuppressed/confidential cells become gaps, never zero.
Government tables hide some provincial cells ("X"). Treating them as 0 would understate totals; they're shown as missing instead.
Other options: impute them (Canada total minus the visible provinces) — more complete but invents numbers; or drop the whole series. Null-with-a-gap is the honest middle.
✅ Verify it yourself ⏱️ 15 min- Open a provincial table on the OEE site (oee.nrcan.gc.ca → Comprehensive Energy Use Database → any provincial sector table) and find a suppressed cell.
- Check the same slice in ceud.html shows a gap, not a zero bar.
Python/ceud_source_notes.md; the ETL's suppressed-cell handling. - calibratedProvinces are checked to sum to the Canada total (within ~2%).
A built-in sanity check on every ETL run.
✅ Verify it yourself- ⏱️ 2 min: pick one number on ceud.html (say, Ontario residential energy in 2023) and compare it to the same cell in NRCan's own Table 1 web page — they should match exactly.
- ⏱️ deeper: run
python Python/ceud_etl.pyand read its printed validation report.
Python/ceud_etl.py validation printout. - source limitNo dollar figures anywhere — CEUD tracks energy and emissions, not cost.
Cost context is a planned separate project (Energy Prices layer, below).
docs/archive/CEUD_PLAN.md; the source database itself.
In one sentence: building permits → housing starts → completions for Canada, every province and 8 metro areas — plus what the building costs, who is available to build it, and how the finished homes actually perform — refreshed automatically on the 20th of each month.
Where the data comes from
How it was built
Full methodology, in plain language
Wrangling StatCan's delivery quirks
StatCan's bulk downloads are awkward at this scale — the permits table alone unzips to a
~9 GB CSV, so the ETL streams it rather than loading it in memory. Seasonally-adjusted variants
only exist for aggregate series; where none exists, the unadjusted series ships and the page says
so. The seasonally-adjusted-at-annual-rates (SAAR) starts figure lives in its own separate table
(34-10-0158). Downloads are cached; --refresh forces a re-fetch.
Storage conventions (so nothing is ambiguous)
Dollar series stored in millions; unit counts as integers; every series' units recorded
in construction_json/meta.json. Output is one compact JSON per geography (21 files,
64–95 KB each) — the page never queries StatCan live.
The provincial pipeline-gap investigation
Provincial "under construction" and "completions" appeared to stop at Dec 2022. Before accepting that, four candidate tables were queried live (34-10-0151, 0136, 0139, annual 0126) — all null after 2022. Conclusion: CMHC genuinely discontinued provincial-level reporting of those stages; only metro areas continue. The dashboard annotates the gap rather than hiding the series.
Re-verified 2026-08-27 against 34-10-0136 and 34-10-0143 through 2026-07 — still null for every province and for Canada, including the Toronto and Montréal members inside 0143. The caveat stands.
Coverage rules that shape what each card can show
Two of the newer series are published unevenly, and the page is built around that rather than papering over it. Renovation prices by project (18-10-0286) publish the 45 project types for census metropolitan areas only — provinces and the national composite carry the all-projects composite alone, and Prince Edward Island has no member at all — so that card carries its own metro picker instead of following the page geography into an empty cell. New-home energy ratings come from EnerGuide, which covers only the evaluated share of new construction, so years with fewer than 30 evaluations are suppressed rather than plotted, and the card states plainly that the two panels are not a like-for-like ratio.
Two more from the Tier-2 pass. The building-cost divisions all share one reference period, 2023 = 100, so the index is plotted as published; an earlier draft re-based every line to 2017 = 100, which was removed because it amplified whichever trade started lowest (wood, off a lumber-cycle trough) and read as a claim about the trades rather than their starting points. And the absorptions and rental-vacancy tables have no provincial members at all — they are census-metropolitan-area tables — so that card shows metros directly, shows the all-CMA aggregate on a Canada view with the label saying so, and hides itself on a provincial view.
A "not built" call, reversed after actually reading the terms
Tier 3 declined to build a CaGBC count, on the assumption that their project database needed a sign-in and carried a redistribution restriction. Both assumptions were wrong. The LEED project-search tool needs no sign-in and exports its full national list on request; and reading CaGBC's actual Terms and Conditions page plus their Member Terms & Conditions PDF in full — not just searching for the word "data" — showed the PDF is entirely about members misusing the certification mark, and the site-wide "All rights reserved" footer is the same boilerplate every page carries, not a term attached to the export function. The lesson generalizes: "I didn't find a restriction" and "I read the actual document and it doesn't address this" are different claims, and only the second should close a licence question.
Even so, the result published here is more conservative than the licence question alone required — aggregate province/program/year counts, never a project name or address — because a compiled list of names and addresses is closer to what CaGBC actually built than any single fact inside it is.
Settling the provincial pipeline gap for good
Tier 1 raised the possibility that CMHC still had the provincial under-construction series even though StatCan had stopped publishing it. Tested directly against CMHC's own portal export endpoint: Ontario under-construction for July 2026 comes back "This data series is now archived.", while the same request for December 2022 returns real data (162,813 units) and Ontario starts for July 2026 returns real data. The endpoint works; the series is gone. It is a genuine CMHC discontinuation, and the page's caveat stands with two independent confirmations behind it.
Two cities, and what their data will and won't support
City permit desks are the only address-level data on the page, and a city is not its metro
area — so the card states the ratio rather than implying the lines should meet. Toronto's cost
column turns out to hold the literal text DO NOT UPDATE OR DELETE THIS INFO FIELD
on about 45% of rows, so its dollar totals are an undercount; its permit counts and dwelling-unit
counts are sound. The cross-check therefore compares Toronto on dwelling units created and
Vancouver on permit value. Toronto also needs both its permit datasets: "cleared" means closed,
so on its own the recent months collapse to a fraction of the true count.
The one hand-typed number set on the page
NRCan's Greener Homes progress page has no data file, and each province's name sits after its figures in the markup — so a positional parser pairs them off by one and mis-assigns every province while looking perfectly plausible. The figures are therefore transcribed by hand into the repo, and a checker re-confirms each one still appears verbatim on the live page before publishing, failing loudly when NRCan posts an update. The card labels itself as transcribed and carries its as-of date.
The status doc says: do not go looking for a replacement table — it was a saga.Chart design
Where two measures share a story (e.g. starts vs interest rates), the page uses a chart pair — two stacked panels with a shared x-axis and linked hover — never a dual-axis chart. The data palette was validated for colour-blind separation; the brand navy failed as a data colour and was swapped for a lighter navy (#245A96) in charts only.
⚠️ Assumptions & data quirks — check these
- source limitProvincial "under construction" and "completions" STOP at December 2022.
CMHC discontinued those provincial series entirely — verified against four different StatCan tables before concluding it's the source, not a bug. Metro areas still have current data. The page annotates the gap.
✅ Verify it yourself ⏱️ 15 min- Open StatCan table 34-10-0151 (statcan.gc.ca → search the table number), pick any province, "Under construction".
- Confirm the values end at 2022-12 while "Starts" continues — exactly what the dashboard annotates.
docs/archive/CONSTRUCTION_STATUS.md §"The provincial pipeline-gap saga"; meta.json'sprovincial_pipeline_gap. - source limitPermit and investment history only starts 2017–2018.
StatCan redesigned those programs; older data isn't comparable. Housing starts go back to 1990. Charts mix the two histories.
✅ Verify it yourself ⏱️ 2 min- On the dashboard, note where the permits line begins vs the starts line — then confirm on StatCan table 34-10-0292 that its reference period genuinely begins in 2017.
docs/archive/CONSTRUCTION_STATUS.md milestones; docs/CONSTRUCTION.md. - judgment callStorage conventions: dollars in millions, unit counts as integers, seasonally-adjusted series preferred.
Chosen to keep files small and charts smooth.
Other options: raw unadjusted series everywhere (shows real seasonality but every chart becomes a sawtooth), or shipping both (double the file size). SA-preferred with unadjusted only as fallback is the standard dashboard convention.
✅ Verify it yourself ⏱️ 2 min- Open
construction_json/meta.json— every series' units and SA status are declared there. Spot-check one against the page.
docs/archive/CONSTRUCTION_STATUS.md "Decisions log"; construction_json/meta.json. - Open
- source limitThe Bank of Canada overnight-rate series only reaches back to 2009.
Accepted as-is for the rate-cycle chart.
✅ Verify it yourself ⏱️ 2 min- Visit the BoC Valet API page for series V39079 — its earliest observation is 2009.
docs/archive/CONSTRUCTION_STATUS.md Phase 4a note.
✓ How it was checked
- Canada and Ottawa views were compared against published StatCan figures (SAAR 261,377; permits $12.5B; under-construction 347,543 at its discontinuation) — matched.
- Zero console errors in browser verification; every toggle and view exercised.
- Ongoing check you can do monthly: after StatCan's monthly permits release (The Daily), the dashboard's headline should match The Daily's number within rounding once the 20th-of-month refresh runs.
In one sentence: a screening map of where ground-source heat pumps look promising in Ottawa — built from 55,903 water-well drilling records, the City's own data, and Hydro Ottawa's grid capacity. Screening, not engineering: a real project still needs an on-site test.
Where the data comes from
How it was built
Full methodology, in plain language
Step 1 — from 1980s drillers' paperwork to a clean database
Ontario's well archive stores depths in a mix of feet/metres/centimetres/inches with a
per-row unit column (missing unit = feet, the WWIS default); everything is normalized to metres
and L/min. Categorical fields are numeric codes — seven lookup tables were exported from the
Access database (the formation-material table alone has 82 codes; codes 60–92 are texture
modifiers like "coarse" and are deliberately not treated as rock types). Unresolvable codes pass
through visibly as code:NN rather than being silently mislabelled. The result is a
GeoPackage with 5 linked layers: wells (55,903), formations (144,719 intervals), water strikes
(62,324), pump tests (44,026), construction details (125,495).
How a well gets a conductivity estimate
Each well's "primary lithology" = its thickest bedrock layer if it has one, else its thickest layer with a usable material. That material maps to a bucket, and each bucket carries a mid-range thermal conductivity from GSHP design literature (see the assumption below for the full table). Exporting the real code tables + smarter bucketing recovered ~4,800 wells; 9,180 remain "unknown", of which 7,872 have no formation record at all — irreducible.
The open-loop screen
viable = has a static water level AND pump-test yield ≥ 15 L/min ·
possible = has a level, yield unknown · unlikely = otherwise
(dominated by missing data, not proven unsuitability).
Result: 31,775 viable / 3,602 possible / 15,431 unlikely.
Step 3 — the smooth surface
Inverse-distance weighting (12 nearest wells, distance², i.e. closer wells dominate) on a 500 m grid in UTM zone 18N. Wells are first clipped to the Ottawa bounding box — 17 mis-located records were scattering cells across Ontario. A cell is only drawn if its nearest well is ≤ 2 km away; it's flagged high confidence only when ≥ 5 wells sit within 1.5 km. Output: 13,383 cells, 1.40–3.20 W/m·K, 89% high-confidence.
Steps 4–5 — one file, and an honest check
The map embeds the grid as compact number-tuples and wells as flat arrays (that's how 132k features fit in a 6.8 MB self-contained file); only trunk sewers are embedded, and everything renders on canvas so 51k points stay responsive. Then the validation script joins every well to the City of Ottawa's own potential rating: measured yields order the City's classes correctly (High 76 → Average 37 → Low/None 27 L/min), and where the two disagree it's mostly our missing-data wells inside the City's good polygons — reassuring in both directions.
Server quirks worth remembering
The City ArcGIS server rejects GeoJSON output for sewer layers (fetched as esriJSON and converted); the zoning layer mislabels General Industrial as "Transportation Zones" (fixed from the actual by-law); everything is paged 1,000 records at a time.
⚠️ Assumptions & judgment calls — check these
- judgment callGround conductivity is GUESSED from the words in drillers' logs.
A driller wrote "limestone" in 1985 → the map assigns 2.8 W/m·K. Full table: limestone 2.8 · dolostone 3.0 · sandstone 2.3 · shale 1.9 · granite 3.2 · gneiss 3.0 · clay 1.4 · silt 1.5 · sand 2.4 · gravel 2.0 · till 1.8 · fill 1.5 · basalt 2.0 · generic "rock" 2.5. Literature mid-range values, not measurements. Good enough to compare neighbourhoods; NOT good enough to size a borehole field.
Other options: use the GSC's mapped bedrock geology instead of per-well logs (smoother, ignores real local variation — it's actually the planned fallback for well-sparse areas); carry min–max ranges per rock instead of one value (more honest, harder to map); or use only wells with actual thermal tests (there are essentially none public).
✅ Verify it yourself- ⏱️ 15 min: compare the bucket values against a published GSHP design table — the standard references are ASHRAE Applications ch. 35, IGSHPA design guides, or the widely-cited VDI 4640 ranges (e.g. limestone 2.5–4.0, granite 2.1–4.1, clay 0.9–2.3 W/m·K). Every bucket value should sit inside its published range.
- ⏱️ 2 min: on the live map, click wells and check the lithology shown in the popup against the conductivity class it got — do a granite well and a clay well disagree the right way?
Geothermal/README.md §3.1 "Lithology buckets"; §5 caveat 1. - judgment callRock-name groupings were decided by Claude (marble counts as limestone, schist as gneiss, "quicksand" as sand, generic "ROCK" gets 2.5 W/m·K…).
These mappings recovered ~4,800 wells that would otherwise read "unknown". Each is defensible geology (marble IS metamorphosed limestone), but they're calls someone should sanity-check.
Other options: leave unmapped names as "unknown" (loses 4,800 wells but assumes nothing); or have a geologist review the mapping table (the strongest option, and cheap — it's ~15 lines).
✅ Verify it yourself- ⏱️ 15 min: open the bucket dictionary in
Geothermal/scripts/combine_wells.pyand check each folded rock against any geology reference: marble→limestone ✓ metamorphic equivalent; schist→gneiss ✓ both foliated metamorphics ~3 W/m·K; conglomerate→sandstone ✓ both clastic. The one to scrutinize: generic "ROCK" = 2.5 (a guess that it's average bedrock). - ⏱️ deeper: re-run the chain with "ROCK" excluded instead, and see if the map's class counts move materially.
Geothermal/README.md §3.1; the bucket dict in combine_wells.py. - ⏱️ 15 min: open the bucket dictionary in
- judgment callInterpolation settings: 500 m grid, 12 nearest wells, distance-squared weighting, no cell further than 2 km from a well.
Tighter settings = patchier map; looser = smoother but more made-up.
Other options: kriging (statistically fancier, needs variogram fitting, harder to explain), natural-neighbour interpolation, a finer 250 m grid (4× the cells for little gain at this data density), or no interpolation at all (wells only — honest but unreadable at city scale).
✅ Verify it yourself- ⏱️ 2 min: on the map, turn on BOTH the grid and the wells layer and zoom somewhere you know — the grid colour should visibly track the nearby wells' colours.
- ⏱️ deeper: in
interpolate_conductivity.pychange k from 12 to 6 (or power 2 → 1), re-run, and compare the printed class counts (medium/high/low cell totals). Small movement = robust choice.
Geothermal/README.md §3.3; interpolate_conductivity.py. - source limit"Unlikely" for open-loop mostly means "no data", not "proven bad".
A well screens "viable" only if it has both a water level and a decent pump-test yield (≥ 15 L/min). Most "unlikely" wells are simply missing one of those records.
✅ Verify it yourself ⏱️ 2 min- Click a handful of red "unlikely" wells on the map — the popups will mostly show blank yield or blank static level rather than bad values.
Geothermal/README.md §3.1 "Open-loop screen" + §5 caveat 2. - source limitWells are rural-biased and Ontario-only.
Downtown has municipal water (few wells), so urban estimates lean on sparse, older records — watch the confidence flag. The Gatineau side is blank because the well database is Ontario's.
✅ Verify it yourself ⏱️ 2 min- Toggle the wells layer and look at Centretown vs rural Osgoode — the density difference is obvious, and downtown grid cells should show the low-confidence flag in their popups.
Geothermal/README.md §5 caveats 3–4. - judgment callThe colour classes: low < 2.0 ≤ medium ≤ 2.8 < high (W/m·K).
These cutoffs decide which cells look green vs red on the map.
Other options: a continuous colour ramp (no arbitrary cutoffs, but harder to read "is my area good?"), or data-driven terciles (would move with the data rather than meaning something physical). 2.0 ≈ "below this, boreholes get long"; 2.8 ≈ "solid rock territory" — physically meaningful lines.
✅ Verify it yourself ⏱️ 15 min- Check the cutoffs against the same GSHP design references as above: soils/clays cluster below 2.0, competent limestone/granite above 2.8 — the classes should separate "overburden" from "good bedrock".
Geothermal/README.md §3.3.
✓ How it was checked
- Every well was compared against the City of Ottawa's own geothermal potential rating: where they overlap, measured well yields order the City's classes correctly (76 L/min in "High" → 37 "Average" → 27 "Low/None"). Disagreements are mostly our missing-data cases, not contradictions. Full crosstab:
Geothermal/Data/processed/city_validation.csv. - Verified live in a browser: all 6 layers toggle, popups show real data, no errors.
- A check you can do: pick a well ID from a popup and look it up in Ontario's public Well Records search (ontario.ca → "well records") — the depth/yield/lithology should match the map's popup.
More depth: layer list & numbers
| Layer | Features |
|---|---|
| Conductivity grid (500 m cells) | 13,383 |
| Wells (coloured by open-loop status) | 50,808 |
| Hydro Ottawa feeder capacity | 3,884 |
| Industrial/employment zoning | 618 |
| Sewer lines (trunk embedded) | 63,300 |
| City open-loop potential rating | 158 |
Build log with every session: GEOTHERMAL_STATUS.md. Full method: Geothermal/README.md.
In one sentence: pick your city, house type and heat pump — the page simulates all 8,760 hours of a typical year in your browser and shows whether switching cuts emissions, and by how much. The most methodology-heavy project in the suite (7 build phases).
Where the data comes from
How it was built (7 phases)
Full methodology, in plain language (all 7 phases)
Phase 1 — how dirty is a kWh, hour by hour?
Hourly generation by fuel was fetched for Ontario (IESO), Alberta (AESO) and Quebec (HQ). Nuclear, hydro, wind, solar and biofuel count as zero combustion emissions; all direct emissions are attributed to the fossil fleet. For Ontario:
average EF(hour) = gas share of generation × 500 g CO₂e/kWh marginal EF(hour) = 500 whenever any gas is running (≈ every hour)The 500 wasn't plucked from air: The Atmospheric Fund publishes official Ontario factors, and backing the implied gas intensity out of four separate years gives 526 / 496 / 474 / 502 — a tight cluster around 500. Alberta gets coal 1050 / gas 540 the same way; Quebec's thermal remainder gets 700 (uncalibrated, but it's 0.5% of a ~99.9% clean grid — immaterial). The computed annual averages reproduce TAF's published values within 5.6% for all four overlapping years.
Phase 2 — weather, and connecting it to the grid
Two temperature datasets per city: real hourly history (ECCC, 2019–2026) and a "typical meteorological year" (CWEC2020) that the simulation runs on. The grid factors are then binned into a lookup surface by temperature (2 °C bins) × hour of day × season — so a −20 °C 7 a.m. January hour gets the emissions intensity such hours historically had. Bins with under 20 hours of history fall back to coarser bins. Check: reconstructing each year's annual average from the surface lands within ±10% of the directly-computed value.
Phase 3a — sorting 11,357 heat pumps into tiers
The NEEP cold-climate list has 180,279 rows, but that's every indoor/outdoor pairing — Carrier alone is ~110k. Collapsing to unique outdoor units (median across pairings) gives 11,357. Each gets two scores: efficiency at −15 °C (COP@5°F) and capacity retention (heat output at −15 °C ÷ rated output at +8 °C). Units are ranked on the equal-weight blend of the two percentiles and cut into equal thirds: Tier 1 "cold-climate premium" (COP ~2.07, retention ~93%), Tier 2 "mid-market" (~1.97 / 80%), Tier 3 "baseline" (~1.84 / 70%). K-means clustering was tried first and rejected — the COP scale is compressed (NEEP's own 1.75 floor) so it produced incoherent groups.
Phase 3b — from 4 test points to a full curve
Each representative model's certified points (capacity + COP at 8.3 / −8.3 / −15 °C and its coldest catalogued temperature, at max compressor speed) are connected piecewise-linearly on a −30…+15 °C grid. Below the coldest point: capacity extrapolates on the last slope, COP floors at (coldest COP − 0.3), and output is zero below the model's minimum operating temperature. A 7% defrost penalty applies on −7…+4 °C with smooth 1 °C ramps at the edges. Real quirk kept: some units' certified max capacity genuinely dips at −8 °C (a rating artefact); per-model curves stay faithful, but tier aggregates are forced monotone so the UI's "capacity rises with temperature" logic holds. Ground-source units get COP-vs-water-temperature curves digitized from two WaterFurnace catalogs, with a 9% antifreeze penalty.
Phase 4 — what is "a pre-1980 Ottawa detached home"?
From ~100k+ real audit records, filtered to each city by postal prefix, split into 4 archetypes (pre-1980 / 1980–2005 / post-2005 detached + townhouse). Each archetype's physics comes from two audited medians:
UA (heat-loss rate, W/K) = design heat loss ÷ (21 °C − design temp) hourly load = UA × max(0, Tbalance − outdoor temp) Tbalance = solved so the simulated year reproduces the audited annual energyThe solved balance points came out 8–12 °C — lower than the textbook 15–16 °C — which is flagged openly as a finding (thermostat setbacks and conservative audit heat-loss values are the likely causes). Audited "heating energy" is fuel input, so it's converted to delivered heat via each home's recorded furnace efficiency before calibration — otherwise gas homes would look artificially hungry vs electric ones.
Phase 5 — the hourly engine
For each of 8,760 hours: compute the load; the heat pump (scaled to the chosen size) delivers up to its capacity at that temperature; backup covers any gap; below the unit's lockout temperature backup takes everything. Electricity × that hour's grid factor (average or marginal, your toggle) + the base case's combustion + refrigerant leakage + upstream methane = the comparison. The engine exists twice — JavaScript (in the page) and an independent Python twin — and both must produce identical results to 4 decimal places on 5 hand-computed scenarios. The live page re-runs all 15 test vectors every time it loads.
Phase 6 — the page
One self-contained file inlining the engine; data fetched from the repo's JSON. Inputs auto-size the unit to the archetype's design load; Advanced mode exposes the average/marginal toggle, refrigerant choice and lifecycle sliders. The centrepiece chart shows heating load vs heat-pump capacity across temperature, with the balance point, cutoff and backup region shaded.
Phase 7 — does it agree with the literature?
Seasonal COP 2.32–2.54 vs 2.41 field-measured (Efficiency Canada) — direct hit. Backup share <0.5% vs "0.3% of hours" published. Ontario gas→HP on the average grid −68…−74% vs NRCan's "significant reduction". Alberta ~break-even, matching NRCan's direction. Every gap has a named cause: the marginal-vs-average choice, the 2025 grid being ~3× dirtier than the 2020 grid older studies used, and the archetype being the population median (~200 m²) vs NRCan's single smaller reference house. One real bug was found by this exercise and fixed (Quebec's near-zero baseline produced a −107,887% headline; now guarded with plain language).
⚠️ The 5 assumptions that matter most
- judgment callTHE BIG ONE: by default, heat-pump electricity is priced at the "marginal" grid rate (~500 g CO₂e/kWh in Ontario), not the much cleaner average (~97 g).
The reasoning: plugging in a new load makes the gas plants run more — the nuclear/hydro was already spoken for. This single toggle flips the Ontario gas-vs-heat-pump answer from "−68% emissions" to "+11%".
Other options: average EF (what most published studies use — flattering to heat pumps); a time-varying hourly marginal (more precise, but Ontario's margin is gas nearly every hour anyway, so it converges to ~flat 500); or a long-run marginal that credits future clean builds (speculative). The chosen flat-500-when-gas-runs is the deliberately pessimistic upper bound — honest for "what does MY new load cause today", harsh vs every headline you've read.
✅ Verify it yourself- ⏱️ 2 min: flip the Average/Marginal toggle in the page's Advanced mode and watch the verdict flip — then decide which question you're asking.
- ⏱️ 15 min: open the saved TAF guideline (
HeatPump/data/raw/TAF-Ontario-Emissions-Factors-2024.pdf) — it states marginal factors are the correct basis for fuel-switching decisions, average for inventories. The tool's default follows that guidance. - ⏱️ 15 min: sanity-check that gas really is Ontario's marginal fuel: IESO's public generator output pages show gas plants ramping with demand while nuclear/hydro run flat.
HeatPump/METHODOLOGY.md §Phase 1 "Emission factor model"; PLAN.md methodology decision #1; the in-page explainer. - calibratedOntario gas plants = a flat 500 g CO₂e/kWh, every hour, every year.
Backed out from The Atmospheric Fund's published factors for 4 separate years (came out 474–526 each time). A constant was chosen over a per-year value for simplicity. Alberta uses coal 1050 / gas 540; Quebec's tiny thermal fleet 700 (NOT independently calibrated — it's 0.5% of QC generation, immaterial).
Other options: a year-varying factor from the National Inventory Report directly (the documented TODO — the NIR PDF wasn't machine-readable this pass), or per-plant factors (overkill for a screening tool).
✅ Verify it yourself ⏱️ 15 min- Open the TAF PDF's factor table (p.11): 2023 average = 67 g/kWh.
- Divide by the 2023 gas share of Ontario generation (13.3%, in METHODOLOGY.md's table): 67 ÷ 0.133 ≈ 502. That's the whole calibration — one division you can redo for any year.
HeatPump/METHODOLOGY.md §"Gas emission factor: 500 g — calibration"; the TAF PDF is saved in the repo. - judgment callHeat-pump performance curves rest on 4 certified test points per model, connected with straight lines.
The plan was to digitize manufacturers' detailed engineering tables, but most are locked behind contractor logins (Lennox and Gree documents were downloaded and checked — no temperature tables). Fallback: the 4 AHRI-certified points every unit publishes. Real certified data, but the straight line across the 16 °C gap between −8 °C and +8 °C is an approximation.
Other options: get contractor-portal access and digitize the full tables (the upgrade path — the pipeline is built to accept more points); fit a physical compressor model (more assumptions, not fewer); or use NREL/EnergyPlus generic curves (not model-specific).
✅ Verify it yourself- ⏱️ 2 min: look at the sanity plots in
HeatPump/data/interim/hp_curve_tier1.png(etc.) — certified points overlaid on each curve. - ⏱️ 15 min: open the one public datasheet (
HeatPump/data/raw/spec_sheets/mitsubishi/…202401.pdf, Performance table p.2) and compare its numbers to the same model's curve — they agree within ~3% except one 12% COP gap explained by a different indoor unit pairing. - ⏱️ deeper: run
python HeatPump/pipeline/test_hp_curves.py— 7 automated checks including "curve passes through certified points within 3%".
HeatPump/METHODOLOGY.md §Phase 3b "Data acquisition — what is public and what is not". - ⏱️ 2 min: look at the sanity plots in
- calibratedHouses stop needing heat above ~8–12 °C outdoors — noticeably lower than the textbook 15–16 °C.
This "balance point" wasn't assumed — it was solved so each archetype's simulated year reproduces the real audited median energy use. The textbook value would predict ~1.8× too much heating (Ottawa pre-1980: ~41,000 kWh vs the observed 22,990). Likely causes: thermostat setbacks and conservative audit heat-loss figures. Flagged openly as a finding.
Other options: force 15–16 °C and scale down UA instead (same annual energy, different cold-weather shape — would change the backup-sizing story); or model setbacks explicitly (needs occupancy assumptions nobody has).
✅ Verify it yourself- ⏱️ 15 min: check against your own home — your gas bills likely drop to ~pilot-light levels by the time daily means hit 10–12 °C (May/June), not 15–16 °C. That's the same effect.
- ⏱️ deeper: open
HeatPump/data/interim/archetype_validation.csv— all 20 archetypes reproduce their audited energy, and the solved Tbalance values are listed per archetype.
HeatPump/METHODOLOGY.md §Phase 4 "Finding: calibrated Tbalance (8–12 °C)…". - judgment callA 7% efficiency penalty is applied between −7 °C and +4 °C for defrost cycles.
Frost forms on outdoor coils in that band and the unit must periodically melt it. 7% is a literature default, not a per-model measurement.
Other options: 0% (assume the certified ratings already include defrost — they don't for the max-capacity points used here, which was checked); or a per-model measured penalty (data doesn't exist publicly). Field studies report anywhere from ~3% to ~15%, so 7% is mid-range.
✅ Verify it yourself- ⏱️ 2 min: on the tier sanity plots (data/interim/), see the visible COP dip across the −7…+4 °C band.
- ⏱️ 15 min: search "defrost penalty cold climate heat pump field study" — NRCan/CanmetENERGY field monitoring reports land in the 3–15% range.
HeatPump/METHODOLOGY.md §Phase 3b "Curve construction" item 3.
All remaining assumptions (12 more, with verification steps)
- judgment callThe 3 performance tiers are equal thirds of a composite score (cold-weather efficiency + capacity retention, equally weighted).
Other options: k-means clustering (tried, rejected — 55% agreement, incoherent groups because NEEP's 1.75 COP floor compresses the scale); fixed physical thresholds (e.g. ENERGY STAR cold-climate criteria); or unequal weights favouring retention.
✅ Verify it yourself ⏱️ 15 min- Open
HeatPump/data/interim/neep_tier_report.md— tier stats and candidate models are tabulated. - Smell test: known premium units (Mitsubishi hyper-heat, −30 °C rated) should sit in Tier 1; builder-grade minimum units in Tier 3. They do.
METHODOLOGY §Phase 3a "Tiering method"; neep_tier_report.md. - Open
- calibratedThe "average installed" option = the 10 most-installed units in real Canadian retrofit audits, blended by popularity.
All 10 matched to certified ratings (100% match rate). What Canadians actually buy leans bottom-tier (weighted COP@−15 °C ≈ 1.87).
✅ Verify it yourself ⏱️ 15 min- Open
Python/ahri_numbers_seen.csv— the top AHRI number appears 5,465 times. - Search that number (206249117) on ahridirectory.org — it's the GREE GUD36, matching the report's table.
neep_tier_report.md §"AHRI popularity cross-match". - Open
- judgment call"Ottawa" = postal codes K1, K2, K4 (rural K0/K6/K7… excluded); Toronto = M; Montreal = H (incl. Laval); Calgary = T2/T3; Edmonton = T5/T6.
Other options: official census-metro boundaries (would pull in Gatineau for Ottawa — wrong province, wrong grid); include rural K0 (different housing stock would skew archetypes rural).
✅ Verify it yourself ⏱️ 2 min- Check any FSA map (Wikipedia "List of K postal codes") — K1/K2/K4 cover urban Ottawa; K0 is the rural valley. Judge whether that's the "Ottawa" you'd want.
METHODOLOGY §Phase 4 "City selection: FSA prefix" (has the reasoning per city). - judgment callHouse archetypes use PRE-retrofit audit values — the tool models replacing heating in an as-found home, not an upgraded one.
Other options: post-retrofit values (models a heat pump AFTER envelope work — a different, also-valid tool; loads would be ~15% lower), or offering both as a toggle (a possible future feature).
✅ Verify it yourself ⏱️ 2 min- In
HeatPump/pipeline/build_archetypes.py, confirm onlyPre_*columns are read.
METHODOLOGY §Phase 4 "PRE-retrofit values, not POST". - In
- judgment callDesign temperatures computed from 8 years of weather data (2.5th-percentile January hour) instead of the building code's published 30-year values.
Other options: hard-code the NBC 2020 Appendix C values (authoritative but breaks the "everything reproducible from fetched data" property). The computed values (Ottawa −22.8, Toronto −17.1, Calgary −28.6…) land within 2–4 °C of published — and note 8 recent warm years give slightly milder design temps than the 30-year normals.
✅ Verify it yourself ⏱️ 15 min- Look up NBC 2020 Appendix C (or any HVAC design-temperature table) for Ottawa: −25 °C at the 2.5% January level, vs the computed −22.8. Decide if that 2 °C matters for a screening tool (it shifts UA ~5%).
METHODOLOGY §Phase 4 "UA from design heat loss" (has the full table + reasoning). - judgment callAudited "heating energy" is fuel INPUT; it's converted to heat DELIVERED using each home's recorded furnace efficiency.
Without this, gas homes (input = delivered ÷ 0.92) would look hungrier than electric ones and the calibration would skew.
✅ Verify it yourself ⏱️ 2 min- Arithmetic check on the validation smoke run: Ottawa base gas fuel 24,985 kWh × 0.92 = 22,986 kWh delivered — exactly the archetype's annual load. The conversion is that one multiplication.
METHODOLOGY §Phase 4 "Fuel-input vs. delivered heat". - judgment callJunk-data filters on the audits: floor area 40–1,000 m², built 1900–2026, heat loss 1–50 kW. Townhouses aren't split by age (too few per cell); semi-detached homes are excluded entirely (~4–7% of rows).
Other options: percentile-based trimming (1st–99th), or folding semis into "detached" (would slightly lower the detached medians).
✅ Verify it yourself ⏱️ deeper- In
build_archetypes.py, widen a bound (e.g. floor area to 20–2,000) and re-run — medians are robust to filter choices when the filters only catch junk. The smallest archetype cell is n=143, so no cell is fragile.
METHODOLOGY §Phase 4 "Archetype definitions". - In
- standard factorRefrigerant leakage: 5%/year + 80% lost at end of a 15-year life, at each refrigerant's global-warming potential (R-410A ≈ 2,256).✅ Verify it yourself ⏱️ 15 min
- GWP: cross-check R-410A ≈ 2,256 against IPCC AR6 values (it's the 50/50 R-32/R-125 blend). Leak rates: compare against ECCC's halocarbon regulations assumptions or CARB's residential leak studies (2–10%/yr range; 5% is mid). It's a slider — try 2% and 10% and see how little the verdict moves for gas-heated homes.
METHODOLOGY §Phase 5 "Emission categories"; adjustable in the page's Advanced mode. - standard factorUpstream methane: 1–2% of gas leaks before it reaches the furnace (default 1%, slider) — this penalizes the gas baseline, the mirror image of the marginal-grid choice penalizing electricity.✅ Verify it yourself ⏱️ 15 min
- Compare against ECCC's National Inventory (~1% Canadian supply-chain estimate) and satellite-based studies (often 1.5–2.5%). The slider covers the range — check how much the gas-vs-HP verdict moves between 0% and 2%.
METHODOLOGY §Phase 5; the in-page Advanced sliders. - standard factor5% of electricity is lost in transmission lines before reaching the home.✅ Verify it yourself ⏱️ 15 min
- Hydro One / IESO publish Ontario transmission + distribution losses around 4–7%; StatCan's electricity supply-disposition table lets you compute it (losses ÷ generation).
METHODOLOGY §Phase 5 constants. - judgment callCurves use maximum compressor speed (the relevant point when a cold house calls for full heat); part-load cycling behaviour is simplified.
Other options: full variable-speed modelling with min/rated/max curves (NEEP has the points; roughly triples the curve work; mostly affects mild weather where loads and emissions are small anyway). The simplification is conservative — real part-load COPs are usually better.
✅ Verify it yourself ⏱️ 15 min- On ashp.neep.org, open any unit's detail page: it lists min/rated/max points. Compare min-speed vs max-speed COP at 8 °C — min is typically higher, confirming the simplification undersells mild-weather efficiency (i.e. errs against the heat pump).
METHODOLOGY §Phase 3b + §Phase 5 limitations. - judgment callGround-source: Ottawa loop assumed to bottom out near 0 °C in winter (undisturbed ground 8.5 °C), with a 9% antifreeze penalty — from WaterFurnace's own tables, 2 representative units.
Other options: full borefield simulation (a different project); other manufacturers' units (WaterFurnace chosen because its full performance catalogs are public).
✅ Verify it yourself ⏱️ 15 min- The source PDFs are saved in
HeatPump/data/raw/spec_sheets/gshp/— check the digitized COP-vs-water-temperature points against the catalog pages cited in METHODOLOGY (7 Series p.34, 5 Series p.62). - The 0 °C design-minimum matches CSA C448 / IGSHPA guidance for properly-sized Ottawa vertical loops — any local GSHP designer would confirm.
METHODOLOGY §Phase 3b "GSHP curves". - The source PDFs are saved in
- judgment callOne provincial grid factor applies to every city in that province (Ottawa and Toronto share Ontario's), and "the current grid" = 2025 — whose intensity is ~3× the 2020 grid older studies used.
Other options: nodal/zonal factors (data exists for ON but adds little for a screening tool); or letting the user pick the grid year (the historical levels are already stored in
ef_surface_*.json— a cheap future feature).✅ Verify it yourself ⏱️ 2 min- Electricity is provincially pooled — Ottawa and Toronto genuinely share one grid mix, so this is mostly definitional. The part worth checking: TAF's table shows the average factor tripling 2017→2023 (18 → 67 g/kWh), which is why this tool reads harsher than 2020-era studies.
METHODOLOGY §Phase 2 "Province vs. city temperature" + §Phase 7.
✓ How it was checked (unusually thoroughly)
- Grid factors: reproduce The Atmospheric Fund's published Ontario values within 5.6% for all 4 overlapping years (tolerance was 15%).
- Engine: the JavaScript engine and an independent Python twin run 5 hand-computed scenarios and must agree to 4 decimal places; the live page re-runs all 15 test vectors on every load — open the browser console (F12) on heatpump.html and look for the green "all 15 Phase-5 vectors reproduced ✓".
- Curves: 7 automated tests — run
python HeatPump/pipeline/test_hp_curves.pyyourself, takes seconds. - Whole tool vs literature (Phase 7): seasonal efficiency 2.3–2.5 matches field studies (2.41); every gap vs NRCan / Climate Institute / Efficiency Canada is explained by a named cause. One real bug was found this way and fixed (an absurd percentage on Quebec's near-zero baseline).
- Re-verified independently on 2026-07-10 in a live browser session: zero console errors, outputs match the validation tables exactly.
✅ Done since (2026-07-12)
- Real operating costs are wired in: TOU priced exactly from the engine's new month×hour electricity matrix, tiered plans priced marginally over a documented baseline, AB flagged as a screening estimate. See METHODOLOGY.md §Operating cost.
- Tool linked in README.md's tool list.
In one sentence: "what's powering the grid right now" for Ontario and Alberta, plus the average-vs-marginal emissions explainer the Heat Pump Explorer's default assumption rests on. ETL shipped 2026-07-12; the page shipped 2026-07-24.
Status
Full methodology, in plain language
Reuse, not rebuild
The heat pump project already knew how to fetch and emissions-rate IESO/AESO generation.
That logic was extracted into a shared module (HeatPump/pipeline/grid_common.py)
used by both pipelines — and the refactor was proven safe by re-running the heat pump's builds
and confirming byte-identical outputs.
What the output holds
grid_json/ carries, per province: hourly generation-by-fuel and intensity
for the last 14 days, and daily min/mean/max back one year (that compression keeps each
file ~170 KB). Ontario refreshes itself every Monday via GitHub Actions; a stale Alberta never
blocks the Ontario commit. Validation cross-checks overlapping months against the already-
validated heat-pump grid files, plus any full year against published annual references.
The page: construction.html's chart engine, not Chart.js
Page chrome (header, hero, Simple/Advanced + light/dark/colour-blind theme) reuses
assets/site-theme.css, same as retrofits.html/newhomes.html/retrofit-insights.html.
The charts themselves are hand-rolled SVG ported from construction.html's chart-pair engine
(svgEl/niceTicks/linked-hover tooltips) rather than retrofits.html's
Chart.js — the "stacked generation mix above, intensity line below, same x-axis, linked
crosshair" pattern the build brief asked for is exactly what that engine already does for
permits-vs-starts, so it was ported rather than rebuilt on a different library.
The "typical day by season" panel needed a new precompute step
grid_json/ only carries a rolling 14-day hourly window plus ~12 months of daily
min/mean/max — no way to derive a real seasonal hourly profile from that alone. The multi-year
hourly history required for it already existed, just not in a browser-shippable form: the Heat
Pump tool's own Phase-1 output (HeatPump/data/processed/grid_ef_{on,ab}.json,
2020–2026 ON / 2015–2026 AB) at 6–9 MB each. New script
Python/build_grid_seasonal.py reuses those files (no re-fetch, no re-derivation of
the EF model) and aggregates to grid_json/typical_day_{on,ab}.json — mean
AvgEF/MarginalEF/fossil-share per (season, hour-of-day), ~8 KB each. It's a manual rerun, not on
the weekly schedule.
⚠️ Known limitations — check these
- source limitAlberta is NOT live.
There is no scriptable public source for recent Alberta data — the direct-CSV lead 404'd and AESO's API needs a registered key. Alberta updates only when the monthly files are manually dropped in (current through June 2026). Ontario IS fully live.
✅ Verify it yourself ⏱️ 15 min- Try it: AESO's site offers the CSD downloads only interactively, and developer-apim.aeso.ca requires registration. If you want live AB, registering a free API key is the unblock — then the ETL can be extended.
docs/archive/ROADMAP_COMPLETED.md item 6; Python/grid_etl.py docstring; the page's own AB banner. - judgment callQuebec is a static context card, not a live series.
Other options: plot HQ's manually-supplied export file as a live-looking series (misleading — it isn't refreshed), or drop QC entirely (loses useful context for the heat pump story).
✅ Verify it yourself ⏱️ 15 min- Hydro-Québec's annual report: ~99.9% of generation is hydro/wind. A flat ~0 g/kWh card loses essentially nothing.
docs/archive/ROADMAP_COMPLETED.md item 6; grid_json/grid_qc.json; the page's Quebec context card. - calibratedThe refactor that shared code with the heat-pump pipeline was verified byte-identical.✅ Verify it yourself ⏱️ deeper
- Re-run
python HeatPump/pipeline/build_grid_ef.pyand diffdata/processed/grid_ef_on.jsonagainst git — no changes expected.
docs/archive/ROADMAP_COMPLETED.md item 6 "ETL" section. - Re-run
🗺️ What's left, in the recommended order
From ROADMAP.md (each item there has a ready-to-paste build prompt). Nothing
is queued behind this — once it ships, the next move is picking from ROADMAP.md's
"Project ideas" list.
| Order | What | Why / notes | Size |
|---|---|---|---|
| 1 | Ottawa Case Study Phases 4–6 — the flagship | Phases 0–3 done (stock, load, electrified load). Next: feeder aggregation + coincidence factor → map layers → the narrative case-study page. Prompts in HEATDEMAND_PLAN.md §5 | 3 sessions |
About this page
Generated by Claude on 2026-07-10 (expanded same day with full methodology
sections and per-assumption verification guides) by reading every project document in the repo
(ROADMAP.md, HeatPump/PLAN.md, README.md + docs/, HeatPump/METHODOLOGY.md,
Geothermal/README.md, GEOTHERMAL_STATUS.md, docs/archive/CONSTRUCTION_PLAN/STATUS.md,
docs/archive/CEUD_PLAN.md, source-note files, and the git history). Updated 2026-08-03
(two passes): first added the missing Retrofit Insights and New Homes Explorer sections (both had
shipped but were only reflected in the overview tiles, not full write-ups), refreshed every status
line against the current ROADMAP.md and commit history, and added the
ahri-refresh.yml automation layer nowhere else in the atlas mentioned it (sources:
docs/NEWHOMES.md, docs/archive/ROADMAP_COMPLETED.md item 13,
Python/insights_notes.md, Python/build_insights.py). Second pass, same day:
flipped the Grid Dashboard section to shipped once grid.html went live. Third pass, same
day: flipped the landing-page tile/todo row to shipped once index.html went live —
it has no dedicated section here (no data pipeline or assumptions to audit, just a card grid).
Regenerate it by asking
Claude to re-read those documents after major changes. Your ✓ checkmarks are stored in this browser
only (localStorage).