Audit page

Automated research audit

Audit purpose

This automated audit scans the site and datasets for common quality issues so readers and contributors can trust the research. It runs weekly and whenever the repository is updated. You can also run the audit live in your browser using the button below.

How the audit works

The audit performs a set of reproducible checks against the site and the data files referenced in the repo. It inspects page links, asset availability, dataset schemas, and common data quality signals (missing values, duplicates, outliers). The live runner reads page DOM and fetches referenced files locally in your browser; it does not transmit private data to external services.

Checks performed

This table defines what each check evaluates. The current results and last-run timestamp are loaded from the committed audit report in Live results.

Check What it tests Implementation detail
Broken internal links Pages and assets return 200 Checks internal page and asset targets referenced by the dashboard.
Dataset schema validation Required columns present and types Validates key columns in data/processed/nhl_signed_free_agents_clean.csv and verifies expected offseason year coverage.
Missing contract values Percent of signings with null contract value Flags missing values in key contract fields so incomplete rows are visible before modeling.
Duplicate transactions Exact duplicate rows in signings table Scans expected CSVs for duplicates that can inflate counts or spending totals.
MIS calculation check MIS formula applied and totals match Checks derived movement metrics used by downstream analyses for structural consistency.
Unrendered placeholders Loading spinners or JS errors Detects unresolved loading placeholders in page HTML.
External link health Spotrac, NHL API endpoints reachable Performs best-effort reachability checks for external dependencies.
Accessibility quick check Images have alt text Flags images with missing or empty alt text for accessibility review.

Interpreting results

  • Critical: Blocks site functionality or reproducibility. Fix immediately.
  • High: Likely to affect analysis or data integrity. Prioritize.
  • Medium/Low: Usability or polish issues.

Remediation and workflow

  • For each failing check, click the remediation link to open a pre-filled GitHub issue with the failing check, timestamp, and suggested fix.
  • If the audit flags a missing data file, check the data/ folder in the repo and re-run the data extraction script in scripts/ (see GitHub).
  • For contract value gaps, run the contract_value_impute.ipynb notebook in the repo to inspect and impute missing values.

Run the audit live

Click Run audit to execute the checks in your browser. The runner will display a pass/fail table and a downloadable JSON of results.

Live audit has not been run in this browser session.

One-line bookmarklet to run locally:

javascript:(function(){fetch('https://rmallorybpc.github.io/nhl-free-agency-research/audit-runner.js').then(r=>r.text()).then(eval)})();

Live results

Loading latest committed audit report...

Category Status Checks run Issues found Details
Waiting for automated report...

CI integration

Automated Site Audit workflow status

  • The audit is already automated in GitHub Actions and runs on every push to main, on a weekly schedule, and on manual dispatch.
  • The workflow runs npm run audit and commits updated output/tables/audit_report.json directly when the report changes.
  • A strict quality-gate job runs on scheduled runs (and optional manual strict runs) and fails the workflow when audit failures are present.
  • Current failure behavior: the report records failing checks, and strict-gate executions can fail the workflow run. The workflow does not open PRs automatically.
  • Workflow file: .github/workflows/audit.yml.
  • Future option (not implemented): create an issue or PR automatically on critical failures.

Developer notes

  • Audit output format (JSON): { "timestamp": "...", "checks": [{"id":"broken_links","status":"fail","details":"..."}] }
  • Add schema files under schema/ (CSV column names and types) to make validation deterministic.
  • Add a methods.html#data anchor linking to the exact data extraction scripts and raw CSVs.

Contact

If you see an unexpected failure, open an issue: https://github.com/rmallorybpc/nhl-free-agency-research/issues/new. Include the audit timestamp and failing check name.

Live runner privacy

The live audit runs entirely in your browser and only fetches public files referenced by the site. It does not send your browsing history or local files to external servers.

Appendix

  • Example JSON output (downloadable): audit-results/latest.json
  • CI status badge links to the live audit workflow run history.