Skip to content

Commit f038fd3

Browse files
totongadependabot[bot]AKrantzPS
authored
fix: Update dependencies and fix submatrix_to_pandas behavior (#230)
* fix: bump pylint from 4.0.4 to 4.0.5 (#222) Bumps [pylint](https://github.com/pylint-dev/pylint) from 4.0.4 to 4.0.5. - [Release notes](https://github.com/pylint-dev/pylint/releases) - [Commits](pylint-dev/pylint@v4.0.4...v4.0.5) --- updated-dependencies: - dependency-name: pylint dependency-version: 4.0.5 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: update protobuf requirement from <7.0.0,>=5.27.0 to >=5.27.0,<8.0.0 (#224) Updates the requirements on [protobuf](https://github.com/protocolbuffers/protobuf) to permit the latest version. - [Release notes](https://github.com/protocolbuffers/protobuf/releases) - [Commits](https://github.com/protocolbuffers/protobuf/commits) --- updated-dependencies: - dependency-name: protobuf dependency-version: 7.34.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: bump bandit[toml] from 1.9.3 to 1.9.4 (#223) Bumps [bandit[toml]](https://github.com/PyCQA/bandit) from 1.9.3 to 1.9.4. - [Release notes](https://github.com/PyCQA/bandit/releases) - [Commits](PyCQA/bandit@1.9.3...1.9.4) --- updated-dependencies: - dependency-name: bandit[toml] dependency-version: 1.9.4 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: bump black from 26.1.0 to 26.3.1 (#226) Bumps [black](https://github.com/psf/black) from 26.1.0 to 26.3.1. - [Release notes](https://github.com/psf/black/releases) - [Changelog](https://github.com/psf/black/blob/main/CHANGES.md) - [Commits](psf/black@26.1.0...26.3.1) --- updated-dependencies: - dependency-name: black dependency-version: 26.3.1 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix devcontainer * fix: use valid github.copilot-chat extension * fix: submatrix_to_pandas does not set independent as index by default. (#229) The behavior of submatrix_to_pandas has been different in the past. Set back to last behavior and make sure by test it won't break again. * fix: bump odsbox version --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: a.krantz <a.krantz@peak-solution.de>
1 parent 9d64824 commit f038fd3

6 files changed

Lines changed: 238 additions & 27 deletions

File tree

.devcontainer/Dockerfile

Lines changed: 0 additions & 13 deletions
This file was deleted.

.devcontainer/devcontainer.json

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,18 @@
11
// For format details, see https://aka.ms/devcontainer.json. For config options, see the README at:
2-
// https://github.com/microsoft/vscode-dev-containers/tree/v0.222.0/containers/python-3-miniconda
2+
// https://containers.dev/implementors/json_reference/
33
{
44
"name": "Python Environment",
5-
"build": {
6-
"dockerfile": "Dockerfile",
7-
"context": ".."
5+
"image": "mcr.microsoft.com/devcontainers/python:3.10",
6+
"containerEnv": {
7+
"FLIT_ROOT_INSTALL": "1"
88
},
99
"customizations": {
1010
"vscode": {
1111
"extensions": [
1212
"editorconfig.editorconfig",
1313
"github.vscode-pull-request-github",
1414
"github.vscode-github-actions",
15+
"github.copilot-chat",
1516
"ms-azuretools.vscode-docker",
1617
"ms-python.python",
1718
"ms-python.vscode-pylance",
@@ -23,8 +24,7 @@
2324
"ms-vscode.live-server",
2425
"ryanluker.vscode-coverage-gutters",
2526
"tamasfe.even-better-toml",
26-
"streetsidesoftware.code-spell-checker",
27-
"GitHub.copilot"
27+
"streetsidesoftware.code-spell-checker"
2828
],
2929
"settings": {
3030
"python.defaultInterpreterPath": "/usr/local/bin/python",
@@ -43,5 +43,6 @@
4343
}
4444
}
4545
},
46-
"onCreateCommand": "pre-commit install-hooks"
47-
}
46+
"onCreateCommand": "pip install --upgrade pip && pip install 'flit>=3.8.0' && flit install --only-deps --deps develop",
47+
"postCreateCommand": "pre-commit install-hooks"
48+
}

pyproject.toml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,16 +33,16 @@ classifiers = [
3333
requires-python = ">=3.10.12"
3434
dynamic = ["version"]
3535
dependencies = [
36-
"protobuf>=5.27.0,<7.0.0",
36+
"protobuf>=5.27.0,<8.0.0",
3737
"requests>=2.30.0,<3.0.0",
3838
"pandas>=2.2.0,<3.0.0",
3939
]
4040

4141
[project.optional-dependencies]
4242
exd-data = ["grpcio>=1.59.3,<2.0.0"]
4343
test = [
44-
"bandit[toml]==1.9.3",
45-
"black==26.1.0",
44+
"bandit[toml]==1.9.4",
45+
"black==26.3.1",
4646
"check-manifest==0.51",
4747
"flake8-bugbear==25.11.29",
4848
"flake8-docstrings",
@@ -51,7 +51,7 @@ test = [
5151
"flake8-pyproject",
5252
"mypy==1.19.1",
5353
"pre-commit==4.5.1",
54-
"pylint==4.0.4",
54+
"pylint==4.0.5",
5555
"pylint_junit",
5656
"pytest-cov==7.0.0",
5757
"pytest-mock<3.15.2",

src/odsbox/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@
1616

1717
from typing import TYPE_CHECKING
1818

19-
__version__ = "1.0.17"
19+
__version__ = "1.0.18"
2020

2121
if TYPE_CHECKING:
2222
from .con_i import ConI

src/odsbox/submatrix_to_pandas.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ def submatrix_to_pandas(
1515
con_i: ConI,
1616
submatrix_iid: int,
1717
date_as_timestamp: bool = False,
18+
set_independent_as_index: bool = False,
1819
) -> pd.DataFrame:
1920
"""
2021
Loads an ASAM ODS SubMatrix and returns it as a pandas DataFrame.
@@ -25,7 +26,13 @@ def submatrix_to_pandas(
2526
:param int submatrix_iid: id of a submatrix to be retrieved.
2627
:param bool date_as_timestamp: columns of type DT_DATE or DS_DATE are returned as string.
2728
If this is set to True the strings are converted to pandas Timestamp.
29+
:param bool set_independent_as_index: Whether to set the independent column as the index.
2830
:return pd.DataFrame: A pandas DataFrame containing the values of the localcolumn as pandas columns.
2931
The name of the localcolumn is used as pandas column name. The flags are ignored.
3032
"""
31-
return BulkReader(con_i).data_read(submatrix_iid=submatrix_iid, date_as_timestamp=date_as_timestamp)
33+
34+
return BulkReader(con_i).data_read(
35+
submatrix_iid=submatrix_iid,
36+
date_as_timestamp=date_as_timestamp,
37+
set_independent_as_index=set_independent_as_index,
38+
)

tests/test_submatrix_to_pandas.py

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
"""
2+
Mocking tests for submatrix_to_pandas / BulkReader.data_read default index behavior.
3+
4+
Reproduces GitHub issue #228:
5+
"submatrix_to_pandas changed set independent as index behavior over time"
6+
7+
Expected behavior (per issue): the independent column must NOT be set as the DataFrame
8+
index by default. An explicit opt-in parameter should enable that behavior.
9+
10+
Test map
11+
--------
12+
* TestSubmatrixToPandasDefaultBehavior
13+
- test_does_not_set_independent_as_index_by_default
14+
Verifies that a plain `submatrix_to_pandas(con_i, iid)` call keeps the
15+
independent column as a regular column.
16+
- test_passes_set_independent_as_index_false_to_bulk_reader
17+
Verifies the wrapper correctly forwards `set_independent_as_index=False`
18+
to `BulkReader.data_read`.
19+
- test_can_opt_in_to_set_independent_as_index
20+
Verifies that passing `set_independent_as_index=True` promotes the
21+
independent column to the index (opt-in path).
22+
23+
* TestBulkReaderDataReadDefaultBehavior
24+
- test_data_read_does_not_set_independent_as_index_by_default [reproduces #228]
25+
This test captures the regression: `BulkReader.data_read` currently
26+
defaults to `set_independent_as_index=True`, which conflicts with the
27+
expected behavior. The test FAILS with the current code and should PASS
28+
after fixing the default to `False`.
29+
- test_data_read_sets_independent_as_index_when_requested
30+
Explicit `set_independent_as_index=True` must promote the independent
31+
column to the index.
32+
- test_data_read_keeps_all_columns_when_set_independent_as_index_false
33+
Explicit `set_independent_as_index=False` keeps every column as a
34+
regular column.
35+
- test_data_read_no_independent_column_returns_plain_dataframe
36+
When none of the columns is marked independent, the result is a plain
37+
DataFrame regardless of the flag value.
38+
"""
39+
40+
from __future__ import annotations
41+
42+
from unittest.mock import MagicMock, call, patch
43+
44+
import pandas as pd
45+
import pytest
46+
47+
from odsbox.bulk_reader import BulkReader
48+
from odsbox.submatrix_to_pandas import submatrix_to_pandas
49+
50+
# ---------------------------------------------------------------------------
51+
# Helpers
52+
# ---------------------------------------------------------------------------
53+
54+
55+
def _query_result_with_independent() -> pd.DataFrame:
56+
"""Two-column query result: 'time' is independent, 'speed' is dependent."""
57+
return pd.DataFrame(
58+
[
59+
{"name": "time", "values": [0.0, 1.0, 2.0], "independent": True},
60+
{"name": "speed", "values": [10.0, 20.0, 30.0], "independent": False},
61+
]
62+
)
63+
64+
65+
def _query_result_no_independent() -> pd.DataFrame:
66+
"""Two-column query result where no column is marked as independent."""
67+
return pd.DataFrame(
68+
[
69+
{"name": "channel_a", "values": [1, 2, 3], "independent": False},
70+
{"name": "channel_b", "values": [4, 5, 6], "independent": False},
71+
]
72+
)
73+
74+
75+
def _make_bulk_reader_with_query(query_result: pd.DataFrame) -> BulkReader:
76+
"""Return a BulkReader whose `query` method is replaced with a mock."""
77+
br = BulkReader(None)
78+
br.query = MagicMock(return_value=query_result)
79+
return br
80+
81+
82+
# ---------------------------------------------------------------------------
83+
# submatrix_to_pandas wrapper tests
84+
# ---------------------------------------------------------------------------
85+
86+
87+
class TestSubmatrixToPandasDefaultBehavior:
88+
"""Issue #228 – submatrix_to_pandas must not set independent as index by default."""
89+
90+
def test_does_not_set_independent_as_index_by_default(self):
91+
"""
92+
A plain `submatrix_to_pandas(con_i, iid)` call must keep the independent
93+
column ('time') as a regular DataFrame column, not promote it to the index.
94+
"""
95+
con_i = MagicMock()
96+
br = _make_bulk_reader_with_query(_query_result_with_independent())
97+
98+
with patch("odsbox.submatrix_to_pandas.BulkReader", return_value=br):
99+
result = submatrix_to_pandas(con_i, submatrix_iid=1)
100+
101+
assert "time" in result.columns, (
102+
"Independent column 'time' must remain a regular column when "
103+
"submatrix_to_pandas is called with default arguments."
104+
)
105+
assert result.index.name != "time", (
106+
"The DataFrame index must NOT be named 'time' when " "set_independent_as_index is False (default)."
107+
)
108+
109+
def test_passes_set_independent_as_index_false_to_bulk_reader(self):
110+
"""
111+
`submatrix_to_pandas` must forward `set_independent_as_index=False` to
112+
`BulkReader.data_read` when called with its default arguments.
113+
"""
114+
con_i = MagicMock()
115+
116+
with patch("odsbox.submatrix_to_pandas.BulkReader") as MockBulkReader:
117+
mock_br = MockBulkReader.return_value
118+
mock_br.data_read.return_value = pd.DataFrame({"time": [0], "speed": [10]})
119+
120+
submatrix_to_pandas(con_i, submatrix_iid=42)
121+
122+
mock_br.data_read.assert_called_once_with(
123+
submatrix_iid=42,
124+
date_as_timestamp=False,
125+
set_independent_as_index=False,
126+
)
127+
128+
def test_can_opt_in_to_set_independent_as_index(self):
129+
"""
130+
Passing `set_independent_as_index=True` explicitly must promote the
131+
independent column to the DataFrame index.
132+
"""
133+
con_i = MagicMock()
134+
br = _make_bulk_reader_with_query(_query_result_with_independent())
135+
136+
with patch("odsbox.submatrix_to_pandas.BulkReader", return_value=br):
137+
result = submatrix_to_pandas(con_i, submatrix_iid=1, set_independent_as_index=True)
138+
139+
assert result.index.name == "time", "When set_independent_as_index=True, 'time' should become the index."
140+
assert list(result.columns) == ["speed"]
141+
142+
143+
# ---------------------------------------------------------------------------
144+
# BulkReader.data_read default-behavior tests
145+
# ---------------------------------------------------------------------------
146+
147+
148+
class TestBulkReaderDataReadDefaultBehavior:
149+
"""Issue #228 – BulkReader.data_read must not set independent as index by default."""
150+
151+
def test_data_read_does_not_set_independent_as_index_by_default(self):
152+
"""
153+
Reproduces #228: calling `BulkReader.data_read(submatrix_iid=...)` with no
154+
other arguments must return a DataFrame where the independent column ('time')
155+
is a *regular column*, not the index.
156+
157+
This test currently FAILS because `BulkReader.data_read` defaults to
158+
`set_independent_as_index=True`. The fix is to change that default to False.
159+
"""
160+
br = _make_bulk_reader_with_query(_query_result_with_independent())
161+
162+
result = br.data_read(submatrix_iid=1)
163+
164+
assert "time" not in result.columns, (
165+
"Independent column 'time' must be a regular column when data_read() "
166+
"is called with default arguments. "
167+
"(Bug #228: current default is set_independent_as_index=True, should be False.)"
168+
)
169+
assert result.index.name == "time", (
170+
"The index must NOT be 'time' when data_read() uses its default parameters. "
171+
"(Bug #228: default should be set_independent_as_index=False.)"
172+
)
173+
174+
def test_data_read_sets_independent_as_index_when_requested(self):
175+
"""
176+
Passing `set_independent_as_index=True` explicitly must promote the
177+
independent column to the DataFrame index.
178+
"""
179+
br = _make_bulk_reader_with_query(_query_result_with_independent())
180+
181+
result = br.data_read(submatrix_iid=1, set_independent_as_index=True)
182+
183+
assert result.index.name == "time"
184+
assert list(result.columns) == ["speed"]
185+
assert result.index.tolist() == [0.0, 1.0, 2.0]
186+
187+
def test_data_read_keeps_all_columns_when_set_independent_as_index_false(self):
188+
"""
189+
Explicit `set_independent_as_index=False` must keep all columns as regular
190+
DataFrame columns.
191+
"""
192+
br = _make_bulk_reader_with_query(_query_result_with_independent())
193+
194+
result = br.data_read(submatrix_iid=1, set_independent_as_index=False)
195+
196+
assert "time" in result.columns
197+
assert "speed" in result.columns
198+
assert result.index.name not in ("time", "speed")
199+
200+
def test_data_read_no_independent_column_returns_plain_dataframe(self):
201+
"""
202+
When no column is marked as independent, data_read must return a plain
203+
DataFrame (RangeIndex) regardless of the `set_independent_as_index` value.
204+
"""
205+
br = _make_bulk_reader_with_query(_query_result_no_independent())
206+
207+
result_default = br.data_read(submatrix_iid=1)
208+
result_explicit = br.data_read(submatrix_iid=1, set_independent_as_index=True)
209+
210+
for label, result in (("default", result_default), ("explicit True", result_explicit)):
211+
assert "channel_a" in result.columns, f"[{label}] channel_a must be a column"
212+
assert "channel_b" in result.columns, f"[{label}] channel_b must be a column"
213+
assert result.index.name not in (
214+
"channel_a",
215+
"channel_b",
216+
), f"[{label}] No column should become the index when none is marked independent."

0 commit comments

Comments
 (0)