diff --git a/README.md b/README.md index 4613b94..6a84e09 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ share no code and no process; they share a directory of files, and the format of those files is the whole of the interface between them. ```text -J-Quants API (Free plan, delayed) +J-Quants API (Free plan) | v finance (cron, 18:10 on weekdays) diff --git a/doc/BASIC_DESIGN.md b/doc/BASIC_DESIGN.md index 224e0d8..47b04c4 100644 --- a/doc/BASIC_DESIGN.md +++ b/doc/BASIC_DESIGN.md @@ -121,8 +121,9 @@ historical ratio formula. It is handed frames; it reads nothing. ### 4.9 `finance/charts.py` -`ChartRenderer` draws the two panels and writes the PNG, and `chart_prefix()` -maps a window length to a file name prefix. +`ChartRenderer` draws the price panel and, when `axis=2`, adds the oscillator +panel, then writes the PNG. `chart_prefix()` maps a window length to a file name +prefix. This is the one module that could not be ported by changing API calls. The previous implementation registered a subclass of a private pandas plotting class @@ -198,10 +199,10 @@ continue, and returning both the results and the failures. The summary pipeline: read the stock list, load the stored indicator frames, aggregate, write, and optionally keep a dated copy. -It is where the plan's delay enters the summaries. Staleness is measured against -the newest date the plan publishes, not against today; compared with today, a -source publishing weeks in arrears would drop every stock as stale and write an -empty table every evening. +It is where the configured publication window enters the summaries. Staleness is +measured against the newest date permitted by that configured window, not +against today. Otherwise a configured delay could make data that is current for +the configured subscription look stale. ### 4.14 `finance/notification.py` diff --git a/doc/DATA_CONTRACT.md b/doc/DATA_CONTRACT.md index 0dfaa58..6ebc4bf 100644 --- a/doc/DATA_CONTRACT.md +++ b/doc/DATA_CONTRACT.md @@ -17,7 +17,7 @@ is a change to a published interface and needs the consumer considered first. ## 1. The boundary ```text -J-Quants API (Free plan, delayed) +J-Quants API (Free plan) | v finance (batch, cron, 18:10 on weekdays) @@ -301,10 +301,10 @@ drawn and the caption renders as boxes. ## 9. `data_source.txt` -Where the generated data came from and how old it is. It exists because the -source publishes in arrears: the newest figure in this directory is weeks -behind the run that produced it, and a dashboard showing it without saying so -would read as live market information. +Where the generated data came from and how old it is. `generated` records when +the job ran; `last_trading_day` records the newest trading day the run actually +analysed. Keeping both lets the dashboard show data age without encoding a +subscription's current publication delay in the contract. - Tab separated, one key and value per line, no header, UTF-8. - Three keys, in this order: `source`, `generated`, `last_trading_day`. @@ -319,7 +319,7 @@ would read as live market information. | `last_trading_day` | The newest trading day the run analysed, `YYYY-MM-DD` | ```text -source J-Quants API (Free plan, delayed) +source J-Quants API (Free plan) generated 2026-07-21 last_trading_day 2026-04-24 ``` @@ -345,9 +345,9 @@ that no longer exists. Nothing replaces it. Restoring it would mean choosing a new source for fundamentals; no free source licensed for this use and offered for machine access has been adopted. Scraping one is not an option here. The link has been -removed from the dashboard so that nothing dangles, and neither repository -refers to the -file. +removed from the dashboard so that nothing dangles. No runtime code in either +repository consumes `ref_index.csv`, and the dashboard no longer links it from +the user interface. The four market indices the pipeline used to chart are withdrawn on the same grounds and by the same reasoning. See @@ -383,9 +383,9 @@ splits, and **not** for dividends. the old inconsistency deliberately, and there is no reading of the data under which that is the more correct answer. -No formula changed. `test/test_contract.py` regenerates all 42 computed columns -of the committed fixture and compares them column by column, which is the -evidence that the calculations are the same ones. +No formula changed. `test/test_contract.py` regenerates the computed indicator +columns of the committed fixture and compares them column by column, which is +the evidence that the calculations are the same ones. **Other differences of meaning, and how each is handled:** @@ -400,7 +400,7 @@ evidence that the calculations are the same ones. | Delisting | The provider stops returning rows; the summary drops the stock as stale | | Stock code | Four characters in the stock lists and in every generated file name; five, with a trailing zero, only inside the adapter | | Dates | `YYYY-MM-DD`, parsed to a tz-naive midnight index. No timezone is involved: these are trading dates, not timestamps | -| Delay | The plan publishes in arrears. Recorded in `data_source.txt`, never hidden | +| Delay | The configured publication window limits the newest date a fetch may request; `data_source.txt` records the actual `last_trading_day` rather than a copied provider delay. | **Legacy rows.** Data stored under the previous provider is not merged with this. Where the meaning of a series cannot be shown to be the same, mixing it is diff --git a/doc/VERSIONS b/doc/VERSIONS index 3832f35..819d21c 100644 --- a/doc/VERSIONS +++ b/doc/VERSIONS @@ -40,6 +40,8 @@ v1.0.1 (Release Date: TBD) - Source the API key from a root-only environment file created by deploy.sh and checked by run.sh before the job starts, and leave it empty in CI. - Preserve the last valid summary and fail when no stock can be aggregated. +- Keep data_source.txt provenance to the provider and plan, leaving data age + to last_trading_day instead of a mutable delay label. v1.0 (2026-08-14) ----------------- diff --git a/finance/analysis.py b/finance/analysis.py index 190d607..ec115d3 100644 --- a/finance/analysis.py +++ b/finance/analysis.py @@ -112,8 +112,8 @@ def last_trading_day(self) -> date | None: Return the date of the newest row this run analysed. It is the date the dashboard shows as the age of the data. The - plan publishes with a delay, so it is not today and must not be - presented as though it were. + configured publication window may place that date before today, + so it must not be filled in with today. """ if self.indicator_frame.empty: return None @@ -317,10 +317,10 @@ def run_many( """ Run several stocks, continuing past one that fails. - A nightly job covering thirty stocks must not lose twenty-nine of - them because one delisted code cannot be fetched. Each failure is - logged with its code and returned, so the caller can report a - partial run through its exit status rather than exiting zero. + A nightly list run must not lose every other stock because one code + cannot be fetched. Each failure is logged with its code and + returned, so the caller can report a partial run through its exit + status rather than exiting zero. Returns: The results that succeeded and the (code, error) pairs that did diff --git a/finance/charts.py b/finance/charts.py index d78c529..0849450 100644 --- a/finance/charts.py +++ b/finance/charts.py @@ -7,9 +7,9 @@ # Description: # Draw the PNG charts finance-dashboard displays: a candlestick price # panel with moving averages, Bollinger bands and the parabolic SAR, -# and below it an oscillator panel carrying RSI, MFI, stochastics and -# the rest, captioned with the closing figures and the two model -# outputs. +# captioned with the closing figures and the two model outputs. When +# axis=2 an oscillator panel carrying RSI, MFI, stochastics and the +# rest is added below it; axis=1 draws the price panel alone. # # This is the one module that could not be ported by changing API # calls. The old implementation registered a subclass of a private diff --git a/finance/cli/charts.py b/finance/cli/charts.py index 95c340d..3e6a958 100644 --- a/finance/cli/charts.py +++ b/finance/cli/charts.py @@ -68,6 +68,9 @@ # Build the J-Quants source from the settings and record where the # data came from. # Replace optparse, honour -u, and call the shared pipeline. +# v1.1 2026-08-24 +# Record provider and plan provenance without embedding a mutable +# publication-delay claim. # ######################################################################## @@ -105,7 +108,7 @@ # that a reader knows the figures are not live. The plan's delay is not # repeated here: it is a published property that can change, and the # last trading day beside this line is the fact that matters. -SOURCE_DESCRIPTION = "J-Quants API (Free plan, delayed)" +SOURCE_DESCRIPTION = "J-Quants API (Free plan)" logger = logging.getLogger(__name__) diff --git a/finance/stocklist.py b/finance/stocklist.py index c80cec1..eada3e5 100644 --- a/finance/stocklist.py +++ b/finance/stocklist.py @@ -5,10 +5,10 @@ # finance/stocklist.py: Stock list parsing # # Description: -# Read the comma separated listings that decide which stocks a run +# Read the comma separated stock lists that decide which stocks a run # covers: stocks.txt, topix_core30.txt and the operator's private -# holdings file. All three share one format and differ only in how many -# columns they carry. +# holdings file. These stock lists share one format and differ only in +# how many columns they carry. # # The first two columns, the code and the short name, are the ones # every file has and the only two finance-dashboard reads out of @@ -17,11 +17,10 @@ # # Every entry names a listing on the Tokyo exchange. Market indices # used to be listed here too and were fetched from a provider this -# pipeline no longer uses. The J-Quants -# Free plan does not carry index values, so there is nothing to fetch -# them from and no special case left to make for them: a code in a -# stock list is a listing, and the data source refuses anything that -# cannot be one. +# pipeline no longer uses. No index dataset has been adopted for the +# current pipeline, so there is no special case left to make for market +# indices: a code in a stock list is a listing, and the data source +# refuses anything that cannot be one. # # Author: id774 (More info: http://id774.net) # Source Code: https://github.com/id774/finance diff --git a/run.sh b/run.sh index 752e686..ecdc0c2 100755 --- a/run.sh +++ b/run.sh @@ -91,9 +91,9 @@ CORE30_LIST=${CORE30_LIST:-topix_core30.txt} # a run wants unless the operator is deliberately shortening it. START_DATE=${START_DATE:-} -# The chart windows. LONG_DAYS is inside the two years the Free plan -# keeps, and above the 300 rows that select the long_ file name; a -# larger value would draw the same chart under the same name. +# The chart windows. LONG_DAYS is above the 300 rows that select the +# long_ file name; a larger value would draw the same chart under the +# same name. DAYS=${DAYS:-240} LONG_DAYS=${LONG_DAYS:-480} SHORT_DAYS=${SHORT_DAYS:-60} diff --git a/test/test_contract.py b/test/test_contract.py index e84cd67..ed5c582 100644 --- a/test/test_contract.py +++ b/test/test_contract.py @@ -23,8 +23,8 @@ # underscores. Renaming a column breaks it; reordering does not. # # The indicator file is checked against test/ti_N225.csv, which was -# generated by the implementation this one replaces. Regenerating all -# 42 of its computed columns and finding them equal is what shows the +# generated by the implementation this one replaces. Regenerating its +# computed indicator columns and finding them equal is what shows the # contract survived the rewrite. # # Test Cases: @@ -56,6 +56,9 @@ # Pin data_source.txt, which tells the dashboard how old the data # it is showing is. # Initial release. +# v1.1 2026-08-24 +# Keep provenance text independent from data age and verify the +# generation and trading dates as separate facts. # ######################################################################## @@ -465,22 +468,21 @@ def read_as_the_dashboard_does(path) -> dict: def test_data_source_states_the_source_and_the_last_trading_day(settings, today): path = settings.data_file(DASHBOARD_DATA_SOURCE_FILE) - storage.write_data_source(path, "J-Quants API (Free plan, delayed)", today, date(2015, 3, 20)) + storage.write_data_source(path, "J-Quants API (Free plan)", today, date(2015, 3, 20)) values = read_as_the_dashboard_does(path) assert list(values) == list(DASHBOARD_DATA_SOURCE_KEYS) - assert values["source"] == "J-Quants API (Free plan, delayed)" + assert values["source"] == "J-Quants API (Free plan)" assert values["generated"] == "2015-03-23" assert values["last_trading_day"] == "2015-03-20" -def test_the_last_trading_day_is_older_than_the_day_it_was_generated(settings, today): +def test_generation_and_trading_dates_are_kept_separate(settings, today): """ - The two dates differ, and the file is what lets a reader see it. + Keep the generation date and the last trading day as separate facts. - A plan that publishes in arrears means the newest row is weeks - behind the run. Recording only one date would leave the dashboard - presenting delayed figures as though they were today's. + The two values may differ, and recording both lets the dashboard show + the age of the data without inferring it from the day the job ran. """ path = settings.data_file(DASHBOARD_DATA_SOURCE_FILE) storage.write_data_source(path, "source", today, date(2015, 3, 20))