Skip to content

[FIX]: Preserve reports with duplicate timestamps - #179

Open
Rio Yu (rioyu123) wants to merge 3 commits into
microsoft:mainfrom
rioyu123:codex/fix-report-filename-collisions
Open

[FIX]: Preserve reports with duplicate timestamps#179
Rio Yu (rioyu123) wants to merge 3 commits into
microsoft:mainfrom
rioyu123:codex/fix-report-filename-collisions

Conversation

@rioyu123

@rioyu123 Rio Yu (rioyu123) commented Aug 27, 2026

Copy link
Copy Markdown

Description

JsonFileReportSink used a seconds-precision timestamp with write_text(). Two reports emitted within the same second therefore resolved to the same path, allowing the later report to silently replace the first.

This change preserves every report while keeping the common filename concise:

  • Report filenames now use a UTC timestamp with millisecond precision, for example run_report_2026-04-25T14-30-00-123.json.
  • The timestamp path is created exclusively with mode x, so an existing report is never overwritten.
  • A UUID suffix is added only when the timestamp path already exists; the fallback path is also created exclusively.
  • Reports continue to be fully serialized before any output file is opened, so serialization failures do not leave empty report files.
  • Report content is written with explicit UTF-8 encoding instead of the platform default.

The usage guide and API documentation now describe the concise common filename, collision fallback, and remaining error cases. Tests cover repeated timestamps, existing-file preservation, a forced fallback collision, and serialization failure.

Validation:

  • uv run pre-commit run --all-files — passed
  • uv run pytest tests/unit/reporting -q — 46 passed
  • uv run pytest tests/unit -q — 1035 passed on Linux; 1033 passed and 2 skipped on Windows
  • uv run --group docs mkdocs build --strict — passed on Linux

Breaking changes

There are no public API or report-schema changes. Generated filenames now include milliseconds and may include a UUID suffix on collision. Consumers using run_report_*.json are unaffected; consumers parsing the exact timestamp format should account for the additional millisecond segment.

Checklist

  • pre-commit run --all-files passes
  • Tests added or updated for changes
  • Documentation updated

@rioyu123
Rio Yu (rioyu123) requested a review from a team August 27, 2026 09:15
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@rioyu123

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

@rioyu123
Rio Yu (rioyu123) force-pushed the codex/fix-report-filename-collisions branch from 7a7f44e to 489675a Compare August 27, 2026 12:59

@nina-msft Nina Chikanov (nina-msft) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for flagging this Rio Yu (@rioyu123)! Let me know what you think of the following comments :-)

Comment thread rampart/reporting/json_file.py Outdated
Comment thread rampart/reporting/json_file.py Outdated
Comment thread rampart/reporting/json_file.py Outdated
@nina-msft
Nina Chikanov (nina-msft) dismissed their stale review August 28, 2026 00:49

Whoops - meant to comment only instead of approval :-)

Signed-off-by: Rio Yu <52408936+rioyu123@users.noreply.github.com>
Comment thread rampart/reporting/json_file.py Outdated
filepath = self._output_dir / f"run_report_{timestamp}.json"

timestamp = datetime.now(UTC).strftime("%Y-%m-%dT%H-%M-%S-%f")[:-3]
filepath = self._output_dir / f"run_report_{timestamp}_{uuid4().hex}.json"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, could we add the UUID only when the millisecond filename collides? For example:

timestamp = datetime.now(UTC).strftime("%Y-%m-%dT%H-%M-%S-%f")[:-3]
content = json.dumps(data, indent=2, default=str)
filepath = self._output_dir / f"run_report_{timestamp}.json"

try:
    report_file = filepath.open("x", encoding="utf-8")
except FileExistsError:
    filepath = self._output_dir / f"run_report_{timestamp}_{uuid4().hex}.json"
    report_file = filepath.open("x", encoding="utf-8")

with report_file:
    report_file.write(content)

This keeps the common filename concise while preserving atomic no-overwrite behavior.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. The sink now opens run_report_<millisecond timestamp>.json exclusively first and appends a UUID only if that path already exists. The fallback is also opened exclusively, so the common filename stays concise and same-millisecond reports cannot overwrite each other. Tests cover both paths.

Comment thread docs/usage/results-and-reporting.md Outdated
Output: `.report/run_report_2026-04-25T14-30-00.json`
Output: `.report/run_report_2026-04-25T14-30-00-123_a3f18c92654d4b75ad15687d383d951b.json`

The filename contains a UTC timestamp (millisecond precision) and a random UUID. Reports created in the same millisecond receive different filenames. An exact filename collision raises `FileExistsError` instead of overwriting an existing report. Reports written within the same millisecond have no defined filename order relative to each other.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we implement behavior that I commented on below, the exact filename collision will just be avoided by adding the uuid at the end and we shouldn't raise FileExistsError :-)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. The guide now shows the concise millisecond filename and explains that a same-timestamp collision is handled by appending a UUID. It no longer says that an ordinary timestamp collision raises FileExistsError.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants