What this is
Clean Tape: A morning markets dashboard and end-of-day report for US rates, credit, equities, FX and commodities. Every number passes a 7-check data-quality engine before anyone sees it.
"The tape" is trader slang for the market data feed — the stream of prices everyone reads. Clean Tape checks that feed before it reaches the screen.
It mirrors four internal tools an investment-technology team builds: a morning dashboard, a pipeline that turns vendor file drops into a clean queryable dataset, an automated end-of-day report that replaces a manual Excel workflow, and an engine that flags when a feed is late, incomplete or doesn't add up.
What's on the tape 33 series · 4 sources
| Asset class | Series on the dashboard | Source | Independent cross-check |
|---|---|---|---|
| US rates | UST 3M · UST 2Y · UST 5Y · UST 10Y · UST 30Y · 10Y breakeven · 2s10s spread · 5s30s spread · 3m10y spread | FRED | each spread = long leg − short leg |
| Credit | US HY OAS | FRED | — |
| Volatility | VIX | Yahoo Finance | — |
| Equities | S&P 500 (SPY) · Nasdaq 100 (QQQ) · Russell 2000 (IWM) · EAFE (EFA) · EM equities (EEM) | Nasdaq, Yahoo Finance | — |
| FX | EUR/USD · USD/JPY · GBP/USD · AUD/USD · USD/CAD · USD/CHF · EUR/JPY · USD broad (Fed) | FRED, Yahoo Finance | EUR/USD vs EUR/USD (ECB fix) · USD/JPY vs USD/JPY (ECB fix) · GBP/USD vs GBP/USD (ECB fix) · AUD/USD vs AUD/USD (ECB fix) · USD/CAD vs USD/CAD (ECB fix) · USD/CHF vs USD/CHF (ECB fix) · EUR/JPY vs EUR/USD × USD/JPY |
| Commodities | WTI crude · Gold · Copper · Nat gas · WTI spot (EIA via FRED) · Oil ETF (USO) · Gold ETF (GLD) · Copper ETF (CPER) · Nat gas ETF (UNG) | FRED, Nasdaq, Yahoo Finance | WTI crude vs WTI spot (EIA via FRED) |
Plus reference data the engine needs: NYSE, US bond-market (SIFMA-style), ECB TARGET and FX trading calendars, per-source publication lags, and validity bounds per instrument — all in config/. Portfolio positions are MOCK.
The check engine 7 checks · runs on every pull, before anything is shown
| Check | Question it answers | Rule (live config) |
|---|---|---|
| Freshness | Is the latest value as recent as the market calendar and the source's publishing delay say it should be? | 1 business day behind = WARN, ≥2 = FAIL |
| Completeness | Are trading days missing, did a download fail, did a file arrive with far fewer rows than usual? | gaps in last 10 sessions (≥3 = FAIL); rows ±50% vs median = WARN |
| Schema | Does each raw vendor file still have the columns and types the parser expects? | any missing column / non-numeric value = FAIL |
| Validity | Is the number physically plausible, with no nulls on trading days and no duplicate records? | outside per-instrument bounds or duplicate = FAIL; null = WARN |
| Staleness | Has a value stopped changing (a dead feed)? | same value ≥4 sessions = WARN |
| Outliers | Is today's move abnormal for this series, measured against its own last year of daily moves? | |z| > 4 = WARN, > 6 = FAIL (futures capped at WARN with a roll hint) |
| Consistency | Do independent sources agree, and do derived numbers add up? | Yahoo vs ECB FX: WARN > 1.1%–2.3%; EUR/JPY triangle: WARN > 0.10% |
Each check rates every series OK, WARN or FAIL; the run takes the worst. A FAIL strikes the number through on this site, removes it from the P&L and the top movers, puts a red box at the top of the end-of-day report, and turns the GitHub Actions run red. An empty input or a crashed check is itself a FAIL — the engine never passes silently. Cross-source tolerances are calibrated from 2021→today history (p99 / p99.9 × 1.5). Live results: Health.
How a run works
- Pull — twice each weekday (morning and evening New York time) GitHub Actions fetches FRED, Yahoo Finance (Nasdaq as fallback) and ECB data and saves every response untouched with a manifest entry.
- Clean — responses are normalised into one table of observations (date, instrument, source, value). Missing values stay missing; unfinished intraday bars are never stored.
- Check — the 7-check engine above runs.
- Publish — spreads, changes, z-scores and the mock P&L are computed, then this site and the end-of-day report (HTML + Excel) are generated with every check result attached.
Limitations
- End-of-day free data only; Yahoo is unofficial and can break or block cloud IPs.
- The bond-market calendar approximates SIFMA (federal holidays + Good Friday); early closes are not modelled.
- Futures are continuous front-month series, not roll-adjusted — display only.
- P&L and shocks are linear and simplified; the portfolio is MOCK.
- GitHub's scheduler is not punctual; runs may land late in their window or be skipped.
Instrument universe
| ID | Name | Source | Symbol | Calendar | Lag (bd) |
|---|---|---|---|---|---|
| DGS3MO | UST 3M | fred | DGS3MO | us_bond | 1 |
| DGS2 | UST 2Y | fred | DGS2 | us_bond | 1 |
| DGS5 | UST 5Y | fred | DGS5 | us_bond | 1 |
| DGS10 | UST 10Y | fred | DGS10 | us_bond | 1 |
| DGS30 | UST 30Y | fred | DGS30 | us_bond | 1 |
| T10YIE | 10Y breakeven | fred | T10YIE | us_bond | 1 |
| HY_OAS | US HY OAS | fred | BAMLH0A0HYM2 | us_bond | 1 |
| VIX | VIX | yahoo | ^VIX | nyse | 0 |
| SPY | S&P 500 (SPY) | yahoo | SPY | nyse | 0 |
| QQQ | Nasdaq 100 (QQQ) | yahoo | QQQ | nyse | 0 |
| IWM | Russell 2000 (IWM) | yahoo | IWM | nyse | 0 |
| EFA | EAFE (EFA) | yahoo | EFA | nyse | 0 |
| EEM | EM equities (EEM) | yahoo | EEM | nyse | 0 |
| EURUSD | EUR/USD | yahoo | EURUSD=X | fx_weekday | 0 |
| USDJPY | USD/JPY | yahoo | USDJPY=X | fx_weekday | 0 |
| GBPUSD | GBP/USD | yahoo | GBPUSD=X | fx_weekday | 0 |
| AUDUSD | AUD/USD | yahoo | AUDUSD=X | fx_weekday | 0 |
| USDCAD | USD/CAD | yahoo | USDCAD=X | fx_weekday | 0 |
| USDCHF | USD/CHF | yahoo | USDCHF=X | fx_weekday | 0 |
| EURJPY | EUR/JPY | yahoo | EURJPY=X | fx_weekday | 0 |
| DXY_BROAD | USD broad (Fed) | fred | DTWEXBGS | us_bond | 5 |
| EURUSD_ECB | EUR/USD (ECB fix) | ecb | EURUSD | ecb_target | 0 |
| USDJPY_ECB | USD/JPY (ECB fix) | ecb | USDJPY | ecb_target | 0 |
| GBPUSD_ECB | GBP/USD (ECB fix) | ecb | GBPUSD | ecb_target | 0 |
| AUDUSD_ECB | AUD/USD (ECB fix) | ecb | AUDUSD | ecb_target | 0 |
| USDCAD_ECB | USD/CAD (ECB fix) | ecb | USDCAD | ecb_target | 0 |
| USDCHF_ECB | USD/CHF (ECB fix) | ecb | USDCHF | ecb_target | 0 |
| EURJPY_ECB | EUR/JPY (ECB fix) | ecb | EURJPY | ecb_target | 0 |
| CL_F | WTI crude | yahoo | CL=F | nyse | 0 |
| GC_F | Gold | yahoo | GC=F | nyse | 0 |
| HG_F | Copper | yahoo | HG=F | nyse | 0 |
| NG_F | Nat gas | yahoo | NG=F | nyse | 0 |
| WTI_SPOT | WTI spot (EIA via FRED) | fred | DCOILWTICO | us_bond | 5 |
| USO | Oil ETF (USO) | yahoo | USO | nyse | 0 |
| GLD | Gold ETF (GLD) | yahoo | GLD | nyse | 0 |
| CPER | Copper ETF (CPER) | yahoo | CPER | nyse | 0 |
| UNG | Nat gas ETF (UNG) | yahoo | UNG | nyse | 0 |