feat: typed library API over FRED with corpus-verified models#24
Merged
Conversation
Entity classes (Series/Category/Release/Source), catalog module functions, raw() escape hatch, and configure() - all 31 mapped commands returning parsed dicts (typed models land in Part 3), observations returning the polars-backed Observations frame. GeoFRED deliberately excluded; totality over the 35 CommandSpecs is test-pinned.
… extra PEP 562 lazy __init__ keeps polars off CLI startup (subprocess-pinned); py.typed ships PEP 561 typing; fredq[pandas] enables to_pandas()/to_arrow().
…ints pydantic lands as the only new runtime dependency. The gate harness (zero-nested-extras + required-set==corpus-universal + alphabetical field order + registration completeness) enforces the model law now codified in AGENTS.md. 15 models cover series/observations-meta/categories/releases/ release-dates/sources/tags plus their envelopes; 26 of 31 mapped callables now return typed models (vintage dates, updates, search-tags, and release tables follow in 3c). Evidence rulings: notes/link optional per corpus measurement; filter echo fields optional on SeriesListResult; distinct unpaginated ReleaseSourcesResult (reuse check failed as designed).
…s, release tables The typed surface is now total (test-pinned): every mapped callable returns a corpus-gated model or the Observations frame; raw() stays dict by design. Evidence rulings: series-updates reuses SeriesListResult (its envelope always echoes filter fields); release-tables is a keyed recursive Element tree whose top-level release_id arrives as a string (wire-faithful); Element's line/parent_id/series_id are required-but-nullable.
fredq is now a typed library as well as a raw-JSON CLI, so the README leads with library usage: install, a verified quickstart (observations as a polars frame, typed Series.info(), search_series, releases, error handling via FredApiError, configure(), and the raw() escape hatch), a surface table of entity classes and module functions derived from api.py, and a one-liner on the corpus-gated typed models and py.typed. The existing CLI documentation is preserved under its own section (trimmed of duplication with the new library section); GeoFRED CLI docs stay as-is since their removal is scheduled for a later part. Auth is now documented once, shared by both layers. Content is current-state- only — no migration or history narrative, which belongs in CHANGELOG.
The GeoFRED (Maps) site was sunset in 2022; the underlying API is deprecated and no longer actively maintained by FRED. Remove the `geofred` CLI command group entirely rather than keep dead surface alive: - Delete the four commands (`series-group`, `series-data`, `regional-data`, `shapes`), their CommandSpecs, and every geofred-only param constant in commands.py. - Delete the `output_to_file`/`--out` body-to-file dispatch machinery in cli.py (`_add_body_to_file_out_option`, `_write_body_to_file`, `_WriteBodyError`, the `CommandSpec.output_to_file` field, and the dispatch branch): `shapes` was its only real consumer. - Delete `fredq.api.GEOFRED_EXCLUDED`; the library's totality pin simplifies to `mapped == set(COMMANDS_BY_NAME)` now that every command is either mapped or gone. `raw()` still reaches every remaining command by name. - Delete the geofred probe cases and their 2 error captures from tools/probe.py (matrix 104 -> 94 cases; corpus prune is a separate commit). - Delete tests/test_cli_geofred.py; update test_api.py, test_core.py, test_cli_architecture.py, and test_package_data.py to drop geofred fixtures/assertions and use a mapped command's corpus capture where a stand-in FRED-error-shaped body was needed. This is a breaking CLI change (see CHANGELOG Removed). COMMANDS_BY_NAME now has 31 entries; all commands are grouped under series/category/ release/source/tag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Remove the four geofred corpus capture directories (series-group/, series-data/, regional-data/, shapes/) and their 10 manifest entries, now that tools/probe.py no longer generates them (previous commit). manifest.json rewritten byte-compatibly with _write_manifest's format (json.dumps indent=2, sort_keys=True, LF, trailing newline); _meta.case_count updated 104 -> 94. The corpus gate tests (manifest<->files equality, coverage over all 31 remaining commands, error evidence) pass unmodified against the pruned corpus. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AGENTS.md and README.md no longer mention the geofred CLI group (removed in commits 702dcac/14e2397); the corpus README annotates the removed geofred-specific curation ruling in place and corrects the case-count line to the post-prune 94 cases (85 ok / 9 http_error). cspell.json drops eight words with zero remaining references (WIPCPI, censusdivision, censusregion, necta, topojson, Highcharts, basemaps, reproject); geofred and geot stay (CHANGELOG.md/corpus README history and a live commands.py tag-group value, respectively).
An interactive dogfooding artifact built entirely on the library: a series explorer (units transforms, frequency aggregation, FredApiError surfaced in the UI), multi-series comparison with correlations, catalog search, ALFRED vintage-revision analysis with revised/new/unchanged row classification, and a mortgage-vs-Treasury spread dashboard with an unemployment overlay. Runs without adding dependencies via uv run --with marimo --with altair. examples/ is exempt from black/ruff (marimo owns the cell layout); marimo session caches are gitignored and cspell-ignored.
Owner
Author
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fc7fbdff23
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+258
to
+262
| if isinstance(value, bool): | ||
| # FRED requires lowercase 'true'/'false', not Python's | ||
| # 'True'/'False' — mirrors the CLI's own bool handling | ||
| # (cli.py:_collect_params), which bypasses coerce_param. | ||
| params[name] = "true" if value else "false" |
There was a problem hiding this comment.
When a library caller passes a bool to a non-boolean parameter, this branch bypasses coerce_param() and sends "true"/"false" directly. For example, fredq.releases(limit=True) emits limit=true instead of raising FredClientUsageError for an invalid integer, so the new typed/raw API violates its pre-request validation contract and turns local usage errors into FRED rejections. Only special-case booleans when spec.kind is ParamKind.BOOLEAN; otherwise let the normal coercion path reject the value.
Useful? React with 👍 / 👎.
Owner
Author
There was a problem hiding this comment.
Fixed in 443d483. The bypass was copied from the CLI's _collect_params, where it is safe because argparse only ever yields a bool for a boolean flag (only BOOLEAN-kind params). A library caller can pass limit=True directly, so the copy needed the guard argparse gave the CLI for free. It is now gated on spec.kind is ParamKind.BOOLEAN; a bool on any other param falls through to coerce_param and raises FredClientUsageError ("--limit expects an integer") before any request. Added a regression test asserting the client is never reached. Good catch, thanks.
A library caller can pass a bool to any parameter, unlike the CLI where argparse only yields a bool for a boolean flag. The unguarded bypass sent e.g. limit=True to FRED as limit=true instead of raising FredClientUsageError locally, violating the pre-request validation contract. Gate the bypass on spec.kind is ParamKind.BOOLEAN so non-boolean params reach coercion and reject the value. Reported by codex review on #24.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fredq becomes a fully-typed Python library on top of its existing raw-JSON CLI: import
fredqand get pydantic models, typed errors, and a polars-backed observations frame; runfredqfrom the shell and get byte-identical raw FRED JSON, unchanged except for the breaking removal described below.The library surface
fredq.Series,fredq.Category,fredq.Release,fredq.Source— plus 10 catalog module functions (search_series,search_series_tags,search_series_related_tags,series_updates,releases,release_calendar,sources,tags,tag_series,related_tags).fredq.raw(command, **params): an escape hatch that reaches every command the CLI knows, validated the same way as every typed call.fredq.configure(*, api_key=None, timeout=None)to set the shared library client's options before the first call.fredq.models(SeriesInfo,ReleaseInfo,SourceInfo,TagsResult,Element, and others) — frozen,extra="allow"so drift lands onmodel_extrainstead of raising, required/optional derived from real FRED captures rather than docs.fredq.Series.observations()returns a polars-backedObservationsframe with.to_polars(),.to_pandas(),.to_arrow(),.to_dicts(),.save_parquet(), and a typed.metaenvelope; FRED's"."sentinel is parsed to null.fredq.FredApiError/fredq.FredClientUsageError, mapped by HTTP status + response body shape, never by message wording.py.typed(PEP 561) so type checkers see real return types with no stub package.pip install "fredq[pandas]"(pandas + pyarrow) forObservations.to_pandas()/.to_arrow().import fredqstays lazy — the CLI entry point never importsapi.py,frames.py, ormodels/, sofredq --helpand every CLI command never pay the polars import cost.Evidence discipline
tests/fixtures/corpus/) drives every model: originally 104 cases, pruned to 94 with the GeoFRED removal below. Every model is registered intests/test_models_gates.pyin the commit that creates it, gated on:test_every_model_is_gated— every public model infredq.models.__all__has a gate entry).Breaking change
The
geofredCLI command group (series-group,series-data,regional-data,shapes) is removed. The GeoFRED site was sunset in 2022 and the underlying API is deprecated; it was never part of the typed library surface (raw()was its only access point, and that access point is gone too). Everything else is CLI-identical, byte-for-byte, pinned by tests. See the CHANGELOG'sRemovedentry for the full note and a pointer to FRED's own Maps API docs for anyone who still needs that data directly.Quality
test_api,test_bridge,test_core,test_corpus,test_frames,test_models_gates,test_no_skips_tool,test_packaging,test_probe,test_public_surface), plus substantial extensions totest_cli_architecture.py,test_client.py, andtest_package_data.py.tools/check_no_skips.py) wired into CI fails the build if any test is unexpectedly skipped, plus a dedicatedpandastox/CI leg that guarantees theto_pandas/to_arrowcode paths actually execute rather than silently skipping when the extra is absent.tests/test_packaging.py) — this caught a real bug where a worktree-built sdist leaked 204node_modules/files because hatchling's VCS-ignore-based exclusion silently no-ops when the build root is an absolute worktree path (.gitis a file there, not a directory). Fixed with explicit[tool.hatch.build.targets.sdist] excludeentries.uv run toxgreen across py3.10–py3.14 (black, ruff format/check, pyright, full test suite) throughout the branch; CI green on every pushed commit.Process note
Five parts, each independently gated (full tox + live re-validation before moving on): probe + corpus capture → CLI/library plumbing → response models → release prep → GeoFRED removal. 30 commits total on this branch, every one carrying a
fix:/change:/internal:/feat:prefix.🤖 Generated with Claude Code