Weather fetch — NASA POWER + NOAA/ISD (camber.weather_source)
CAMBER's weather-dependent analytics — M&V weather normalization and OAT-sensor validation — need an
external temperature series. Until now you brought your own (a local EPW/TMY file via
mandv.weather.load_epw, or any series you already had). camber.weather_source fetches one from
either of two free, keyless providers — NASA POWER (global reanalysis; the default below) and
NOAA/ISD (real weather stations; see the NOAA/ISD section)
— in the exact °F Series shape (name="oat_f") those consumers already accept.
# no API key, global coverage; returns a °F pandas Series
python - <<'PY'
from camber.weather_source import oat_reference
ref = oat_reference(34.05, -118.24, "20240101", "20240107", tz="America/Los_Angeles")
print(ref.head())
PY
API
| Function | Returns | Use |
|---|---|---|
nasa_power_url(lat, lon, start, end, *, parameters, community) |
str |
the query URL (pure, no I/O) |
nasa_power_transport(*, timeout) |
callable(url) -> dict |
the default stdlib-urllib transport |
fetch_nasa_power(lat, lon, start, end, *, parameters, transport, tz, timeout) |
DataFrame |
oat_f (°F) + rh_pct when requested |
oat_reference(lat, lon, start, end, *, transport, tz, timeout) |
Series |
just the °F OAT reference (NaNs dropped) |
cached_transport(inner, cache_dir, *, ttl, clock) |
callable(url) -> dict |
wrap any transport with an on-disk cache |
geocode(address, *, transport, limit, user_agent, timeout) |
GeoResult |
address → top match (latitude, longitude, display_name) |
oat_reference_for(address, start, end, *, tz, geocode_transport, transport, ...) |
Series |
geocode an address, then fetch its °F OAT |
nominatim_url(address, *, limit) · nominatim_transport(*, user_agent, timeout) |
str · callable(url) -> dict |
the geocoder's URL builder + default transport |
oat_reference_isd(lat, lon, start, end, *, transport, catalog_transport, tz, ...) |
Series |
NOAA/ISD station-precise °F OAT (nearest covering station) |
isd_nearest_station(lat, lon, start, end, *, transport, stations, ...) |
IsdStation |
nearest ISD station covering the window |
isd_stations(*, transport, timeout) · fetch_isd(usaf, wban, start, end, *, ...) |
list[IsdStation] · DataFrame |
the station catalog · one station's hourly °F |
isd_transport(*, timeout) · cached_bytes_transport(inner, cache_dir, *, ttl, clock) |
callable(url) -> bytes |
the ISD default transport + its on-disk cache |
start/end accept YYYYMMDD / YYYY-MM-DD strings or date/datetime objects. parameters are NASA
POWER codes (T2M = 2 m air temperature, RH2M = 2 m relative humidity). GeoResult and IsdStation
are frozen values with .as_dict().
Geocoding — fetch by address, not just coordinates
NASA POWER is a lat/lon point query, so to fetch weather "for an address" you geocode it first, via OpenStreetMap Nominatim — also free and keyless:
from camber.weather_source import geocode, oat_reference_for
g = geocode("Chicago, IL")
print(g.display_name) # "Chicago, Cook County, Illinois, United States" — confirm the match
print(g.latitude, g.longitude)
# or, one call: geocode the address then fetch its OAT reference
ref = oat_reference_for("Chicago, IL", "2024-01-01", "2024-12-31", tz="America/Chicago")
For an uncertain address, geocode first and check .display_name before fetching. The resolved
place is also attached to ref.attrs["geocode"] (best-effort metadata).
- Precision is a non-issue. NASA POWER is a ~0.5° (~50 km) reanalysis grid, so city/ZIP-level geocoding is plenty — this is a convenience, not an address-precision claim.
- Usage policy. Nominatim requests send a descriptive
User-Agent(built in) and ask for ≤ ~1 request/second with caching — so cache your lookups:cached_transport(nominatim_transport(), cache_dir)composes withgeocodeexactly like it does with the NASA transport. - Timezone still isn't derived from the address (no dependency-light lat/lon→zone) — pass the site
IANA
tztooat_reference_for, the same load-bearing switch asoat_reference(below).
Timezone — read this before you join it to a sensor
NASA POWER hourly timestamps are UTC. BAS trend exports are naive local clock time (see
TIME-HANDLING.md), and sensordrift.compare_to_reference aligns the two by an
inner join on shared timestamps — which pandas refuses across a tz-aware/naive mismatch. So:
tz="UTC"(default) — the returned index is tz-aware UTC, with no hidden shift. Join it to a UTC sensor series.tz="<IANA zone>"(e.g."America/Los_Angeles") — the index is DST-correctly converted, then the tz is dropped, giving naive local civil time that inner-joins directly to a BAS trend index.
NASA's LST option is solar time, not clock time, so it would not line up with a DST-observing BAS export; this adapter deliberately does not use it. Getting this wrong is the one way to silently corrupt a drift bias or a normalization, so it is an explicit knob, tested for the exact hour mapping.
Two things it drops into
- Sensor validation — feed it as the reference to
sensordrift.compare_to_referenceto check the site OAT sensor against what the weather actually did (bias / drift-per-month / tracking correlation) — otherwise impossible from the BAS alone. - M&V — a fetched series feeds
mandv.weather.monthly_normals/normalized_annual_from_monthlyexactly like an EPW series (same dtype, name, and index kind). NASA POWER hourly is actual reanalysis, ideal for the reporting-period actual weather; for a typical-year (TMY) normalization baseline, keep usingmandv.weather.load_epw.
Requesting RH2M too lets you derive wet-bulb with coolingtower.stull_wetbulb_f(oat_f, rh_pct).
Dependency-light + offline-testable
The network call goes through an injectable transport (callable(url) -> parsed-JSON dict, default
stdlib urllib — the same seam as ingest.haystack.http_json_transport). Inject your own callable to
add a cache, or a canned one in tests, so every parse / unit / timezone / fill path runs with no
network. No third-party dependency (stdlib urllib / json + pandas). Missing hours (NASA's -999
sentinel) become NaN rather than a bogus -999 °C; a response with no data for a parameter raises a
clear ValueError.
Multi-year requests
NASA POWER caps a single hourly request at ~1 year, but fetch_nasa_power handles that
transparently: it splits [start, end] into consecutive calendar-year chunks (one call per
year), then concatenates them into a single, unique, sorted hourly index. A three-year request "just
works" — no extra argument. Calendar-year seams share no day (one chunk ends Dec-31, the next starts
Jan-01), so no hour is duplicated or dropped.
On-disk cache
cached_transport(inner, cache_dir) wraps any transport with a dependency-light on-disk cache, so
repeated fetches don't re-hit the API — and, combined with year-chunking, a re-run only downloads the
years missing from disk:
from camber.weather_source import fetch_nasa_power, nasa_power_transport, cached_transport
transport = cached_transport(nasa_power_transport(), "/var/cache/camber-weather")
df = fetch_nasa_power(34.05, -118.24, "20200101", "20231231", transport=transport)
Each URL's parsed JSON is memoized to <cache_dir>/<sha256(url)>.json with an atomic write; a corrupt
file self-heals (treated as a miss). NASA POWER historical reanalysis is stable, so the default is
cache-forever; the most recent ~months can be revised, so pass a ttl (a datetime.timedelta) for
windows that touch recent data. clock is injectable for deterministic TTL tests.
NOAA/ISD station data (a second, station-precise provider)
NASA POWER is a global reanalysis on a ~0.5° (~50 km) grid. When you want a real weather station
near the site, camber.weather_source also fetches NOAA's Integrated Surface Database (ISD-Lite) —
also keyless. oat_reference_isd finds the nearest station covering your window and returns the same
°F oat_f Series:
from camber.weather_source import oat_reference_isd, isd_nearest_station
st = isd_nearest_station(41.88, -87.63, "2023-01-01", "2023-12-31")
print(st.name, st.usaf, st.wban) # confirm the station it picked
ref = oat_reference_isd(41.88, -87.63, "2023-01-01", "2023-12-31", tz="America/Chicago")
# ref -> °F Series (name "oat_f"); the resolved station is on ref.attrs["isd_station"]
Station-precise but gappy — the honest trade-off. ISD is a point measurement at a real station,
higher spatial fidelity than NASA's grid when a station is nearby — but it is gappy (stations
go offline; missing hours are common → dropped to NaN) and sparse (no station near remote sites;
coverage windows vary — isd_nearest_station filters to stations whose record spans your window and
raises if none does). So: ISD when a nearby station covers the window and you want station fidelity;
NASA POWER for global coverage, a gap-free series, or anywhere without a station. They complement.
- Endpoints (keyless): the station catalog
https://www.ncei.noaa.gov/pub/data/noaa/isd-history.csv (~5 MB) and per-station-per-year gzipped
hourly files under
.../isd-lite/. Air temp is tenths of °C; the-9999missing sentinel → NaN. - Timezone is the same load-bearing switch as the NASA path (pass the site IANA
tz; see above). - Caching. ISD uses a bytes transport (gzipped/CSV, not JSON), so it has its own cache
decorator —
cached_bytes_transport(isd_transport(), cache_dir). Cache the 5 MB catalog, or resolve the station list once withisd_stations()and pass it toisd_nearest_station(..., stations=…).