Add sequence parameter

[fabianliebig]

This PR adds the SequenceParameter class for modeling variable-length (including infinite-length) sequences.

[Copilot]

Pull request overview

Adds a new discrete SequenceParameter (potentially infinite) plus an encoder protocol to dynamically construct computational representations for provided sequence values, and refactors discrete-parameter encoding to a narwhals-based DiscreteParameter.transform() API (with comp_df deprecated).

Changes:

• Introduce SequenceParameter + SequenceEncoderProtocol and add targeted tests for infinite/finite behavior and encoder contract validation.
• Narwhalify discrete parameter transform() by moving encoding logic into per-parameter _encoding_table() implementations (categorical/numerical/custom/substance) and updating searchspace utilities accordingly.
• Deprecate legacy surfaces (DiscreteParameter.comp_df, CustomEncoding) and update tests/Changelog to reflect the new encoding/transform model.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
tests/validation/test_parameter_validation.py	Adds validation tests for invalid SequenceParameter constructor arguments.
tests/test_substance_parameter.py	Updates degenerate-representation test to use transform() instead of comp_df.
tests/test_sequence_parameter.py	New unit tests covering SequenceParameter finiteness, enumeration, range checks, encoder contract, and summary behavior.
tests/test_parameters.py	Adds cross-backend transform() contract tests and includes SequenceParameter in the shared suite.
tests/test_deprecations.py	Adds deprecation coverage for CustomEncoding and DiscreteParameter.comp_df.
tests/test_candidates.py	Adjusts TableCandidates tests for the updated input validation expectations.
tests/hypothesis_strategies/alternative_creation/test_searchspace.py	Updates hypothesis-based searchspace tests to the get_candidates() API.
tests/conftest.py	Fixes a tuple literal for active_values.
CHANGELOG.md	Documents API changes: narwhalified transform, comp_df deprecation, encoding changes, and kw-only field shifts.
baybe/utils/validation.py	Optimizes validate_parameter_input to avoid iterrows and align with new discrete-parameter model.
baybe/searchspace/validation.py	Updates active-values validation to use _EncodedDiscreteParameter.
baybe/searchspace/discrete.py	Migrates comp-rep aggregation and transforms to use transform() + narwhals-to-pandas conversion.
baybe/searchspace/candidates.py	Updates a TODO label related to narwhals work.
baybe/parameters/substance.py	Ports SubstanceParameter to _EncodedDiscreteParameter with _encoding_table() and comp_rep_columns.
baybe/parameters/sequence.py	New SequenceParameter implementation and SequenceEncoderProtocol.
baybe/parameters/numerical.py	Narwhalifies numerical discrete encoding via _encoding_table() and adds stronger validation for numeric fields.
baybe/parameters/enum.py	Removes generic encoding base, introduces deprecated CustomEncoding with warning-on-access behavior.
baybe/parameters/custom.py	Ports CustomDiscreteParameter to _EncodedDiscreteParameter and adds a shared helper for pandas-backed encoding tables.
baybe/parameters/categorical.py	Ports CategoricalParameter to _EncodedDiscreteParameter and implements narwhals-based encoding tables.
baybe/parameters/base.py	Introduces _EncodedDiscreteParameter, adds narwhals-based DiscreteParameter.transform(), and deprecates comp_df.
baybe/parameters/init.py	Re-exports SequenceParameter and SequenceEncoderProtocol.
baybe/exceptions.py	Adds InfiniteParameterError for non-enumerable discrete parameters.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[AdrianSosic]

Not at all finished with the review but need to jump to a meeting; hence, giving you already these few things to fix 🙃

[AdrianSosic]

Next batch of comments

[Scienfitz]

fyi I will not review this and instead review the final merge of dev/candidates in detail

[AVHopp]

Some general comments, I do not see anything major blocking although I find some design choices weird, but I guess that they are necessary

[AVHopp]

I am confused by this. According to the typing of the values property, this is intended to return a tuple with the values the parameter can take. As far as I know, tuples are always finite, so in what case is this being triggered resp. how does such a case align with values being a tuple?

[AdrianSosic]

Yes, exactly, values is a tuple, but the return types only ever describe the non-error path. In the infinite case, values will throw an InfiniteSpaceError which we catch and then conclude that the space is infinite.

Raising an error in a property is certainly not 100% nice and we may want to adjust this in the future. However, without a refactoring, this is what we can do, so I simply comply with the existing protocols

[AVHopp]

Makes sense

[fabianliebig]

I thought this was meant as a functionality test to avoid making mistakes during development. Since the sequence parameter overrides and uses is_finite to determine if the InfiniteSpaceError exception should be raised, it's now kind of an Ouroboros scenario, isn't it?

[AdrianSosic]

@fabianliebig: yeah, you are right, I did not properly think this through. So we have three options:

• Keep the current code in the base class and delete the override
• Keep the override and replace the code in base with simply return True
• Keep both --> Ouroboros

What do you think?

[fabianliebig]

I think I would prefer having an explicit assert, returning true and keeping the override. By that we avoid three things:

• No exception as flow control. Maybe that is personal preference, but I think that can mask errors and is not unavoidable in this case.
• Too many redundant implementations (only the sequence parameter needs to determine if it's finite, so a abstractmethod would be an overkill).
• We will still be noticed if something is not properly implemented in the future.

What do you think? Does that make sense to you?

[AdrianSosic]

So basically the version of base that we had before, right?

[fabianliebig]

Probably, would most appreciate a real assert actually but as we discuss that previously, the base version is also fine for me 🙃

[AdrianSosic]

Can you maybe just suggest (using github) the version you'd ideally like to have? 🙃

[fabianliebig]

Sorry, I did not phrase that really well. Ideally I would like to have an assert like assert isinstance(self.values, Sized) but it doesn't really matter because in the end, it still only produces a runtime error. So I probably should just have answered, 'Yeah, let's use the old version.' 🙈

[AVHopp]

Can you elaborate on this? The user will need to wirte an Encoder with a call that uses some backend, can't we simply infer which the user wants to use from that? Just trying it out according to an arbitrary ordering feels weird to me.

[AdrianSosic]

If you have a better solution, let me know. The problem is that the user can write their callable using arbitrary logic and the hole point is that we don't want to impose any backend on them. So without "looking into" the blackbox, I see no way how we could possibly know what they used.

The situation is essentially the following:

def my_encoder(dataframe):  # no annotations
    # code hidden

The above is what we get. So with which backend would you now call it?

[AVHopp]

Well, we could also have proper typing of the encoder as a requirement. Like "We promise that we can handle all backends, but please tell us which you use", I think this is not too hard of a requirement

[fabianliebig]

I think I would also prefer it this way. Automatic backend inference could lead to unforeseen issues, so I would rather make the backend explicit and ensure that users are aware of their responsibility.

I could only come up with a toy example, but in the following code the loop detects Polars, since both Polars and pandas share the interface used in the first encoder call. However, it later crashes because Polars Series objects do not expose values as a property:

class StatefulEncoder:
    def __init__(self) -> None:
        """Initialize the call counter."""
        self._n_calls = 0

    def __call__(self, series):
        self._n_calls += 1
        if self._n_calls == 1:
            return pd.DataFrame({"encoded": list(series)})
        return pd.DataFrame({"encoded_upper": series.values.tolist()})

def main() -> None:
    s = nw.from_native(pd.Series(["a", "b", "c"], name="x"), series_only=True)

    enc = _Encoder(encoder=StatefulEncoder())

    result1 = enc(s)
    print(f"Detected backend : {enc._implementation}")
    print(f"Result 1 columns : {result1.columns}")
    print()

    result2 = enc(s). #<-- Crash

[AdrianSosic]

That's a valid point! I think the decision we have to make is here is the balance between precision and convenience:

• Your example is perfectly valid in the sense that the current implementation has no way to fix it.
• On the other hand, the stateful case is the only one where I can see problems like this (since if a stateless one passes once, it'll also pass the subsequent calls), stateful encoders are also more on the exotic end and rather irrelevant for 99% of the users.

So one potential middle ground I could see is to simply make both the _Encoder class and its _implementation attribute public, and only set None as the default for the latter. The implications would be:

• The average user could still just pass an unlabeled callable and it'll be auto-wrapped into an Encoder with auto-inferred backend.
• A user who wants to explicitly specify the backend would manually wrap their callable into an Encoder and manually specify the backend. That step is unavoidable because we need some place to store the backend specification next to the callable itself, and the Encoder is exactly the object responsible for that.

What do you think? Implementation-wise, the only difference to the case where we drop the auto-inference completely is having the None default. In the enforced explicit case, that default value would simply be dropped.

[fabianliebig]

Yeah, that sounds like it also shapes awareness. Seems sufficient for me. I actually don't think that it will be that uncommon for something analogous to my example to appear; the most straightforward counterexample might be stateful, which is why I provided it, but I could also imagine some more complex logic that involves different sources like file systems or databases that are used in combination, just because it offers that freedom. :D

[AVHopp]

One more thing

[AVHopp]

LGTM, everything that is still open is caught in comments

[myrazma]

Looks good to me once the open discussions are solved!

[myrazma]

Statement is unclear, sounds like it raises all exceptions every time, but it is just in the case that no suitable backend is found.
If no suitable backend is found, raises exception for every tried backend.

[myrazma]

Should we add here: linear & grammar free?

[myrazma]

""".. for linear, grammar free sequence parameters."""
Should we make this more clear here or do we not want to narrow this down now?

[Scienfitz]

Release blocker