Add sglang sglang launch log summary - #10
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a new Python script, tools/maca_launch_log_summary.py, which parses and summarizes service, benchmark, or distributed runtime logs into JSON format by extracting metrics and errors. Feedback suggests reading the log files line-by-line rather than loading entire files into memory to avoid potential Out-Of-Memory (OOM) errors, and wrapping file operations in a try-except block to gracefully handle I/O errors.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| def parse(path: Path) -> dict[str, object]: | ||
| errors: list[dict[str, object]] = [] | ||
| values: list[dict[str, object]] = [] | ||
| for lineno, line in enumerate(path.read_text(encoding="utf-8", errors="replace").splitlines(), 1): | ||
| if ERROR_RE.search(line): | ||
| errors.append({"line": lineno, "text": line.strip()}) | ||
| for match in NUMBER_RE.finditer(line): | ||
| key = match.group("key").lower() | ||
| if not METRICS or any(token in key for token in METRICS): | ||
| values.append({"line": lineno, "metric": key, "value": float(match.group("value"))}) | ||
| return {"path": str(path), "metric_count": len(values), "error_count": len(errors), "metrics": values, "errors": errors} |
There was a problem hiding this comment.
Reading the entire log file into memory using path.read_text().splitlines() can lead to high memory consumption or Out-Of-Memory (OOM) errors if the log files are extremely large. Additionally, if any of the specified log files do not exist or cannot be read, the script will crash with an unhandled exception.
It is highly recommended to open the file and iterate over it line-by-line. Wrapping the file operations in a try-except block to catch OSError will also make the parser much more robust by reporting the error in the output JSON instead of crashing.
| def parse(path: Path) -> dict[str, object]: | |
| errors: list[dict[str, object]] = [] | |
| values: list[dict[str, object]] = [] | |
| for lineno, line in enumerate(path.read_text(encoding="utf-8", errors="replace").splitlines(), 1): | |
| if ERROR_RE.search(line): | |
| errors.append({"line": lineno, "text": line.strip()}) | |
| for match in NUMBER_RE.finditer(line): | |
| key = match.group("key").lower() | |
| if not METRICS or any(token in key for token in METRICS): | |
| values.append({"line": lineno, "metric": key, "value": float(match.group("value"))}) | |
| return {"path": str(path), "metric_count": len(values), "error_count": len(errors), "metrics": values, "errors": errors} | |
| def parse(path: Path) -> dict[str, object]: | |
| errors: list[dict[str, object]] = [] | |
| values: list[dict[str, object]] = [] | |
| try: | |
| with path.open(encoding="utf-8", errors="replace") as f: | |
| for lineno, line in enumerate(f, 1): | |
| if ERROR_RE.search(line): | |
| errors.append({"line": lineno, "text": line.strip()}) | |
| for match in NUMBER_RE.finditer(line): | |
| key = match.group("key").lower() | |
| if not METRICS or any(token in key for token in METRICS): | |
| values.append({"line": lineno, "metric": key, "value": float(match.group("value"))}) | |
| except OSError as e: | |
| errors.append({"line": 0, "text": f"Failed to read file: {e}"}) | |
| return {"path": str(path), "metric_count": len(values), "error_count": len(errors), "metrics": values, "errors": errors} |
References
- Avoid loading entire files into memory at once when processing line-by-line to prevent high memory usage and potential OOM errors. Always handle potential I/O errors (like FileNotFoundError or PermissionError) gracefully.
- Add SGLang MACA launch log summary - Stream MACA launch log summary input
Summary
Validation
Review notes