Skip to content

Print the repr these feature examples actually produce - #8564

Open
vineethsaivs wants to merge 2 commits into
huggingface:mainfrom
vineethsaivs:fix/feature-docstring-reprs
Open

Print the repr these feature examples actually produce#8564
vineethsaivs wants to merge 2 commits into
huggingface:mainfrom
vineethsaivs:fix/feature-docstring-reprs

Conversation

@vineethsaivs

Copy link
Copy Markdown
Contributor

What breaks

Four feature classes show a repr in their Examples block that they cannot print.

>>> ds.features["image"]
Image(decode=True, id=None)

Image, Pdf, Nifti and Video all declare id: Optional[str] = field(default=None, repr=False), so id=None never appears. On top of that Image has a mode field the example omits, and Video has five more fields that do print.

What they actually print:

documented actual
Image(decode=True, id=None) Image(mode=None, decode=True)
Pdf(decode=True, id=None) Pdf(decode=True)
Nifti(decode=True, id=None) Nifti(decode=True)
Video(decode=True, id=None) Video(decode=True, stream_index=None, dimension_order='NCHW', num_ffmpeg_threads=1, device='cpu', seek_mode='exact')

Two more in the same blocks:

  • video.py ends with ...[0]["video] (unterminated string), so that line does not parse.
  • image.py's last block prints a dict directly after ds = ds.cast_column(...), which is an assignment and outputs nothing. pdf.py and nifti.py have the >>> ds[0]["image"] line it is missing.

What changed

Docs only, no code. Each shown repr replaced with what the class prints, the string literal closed, and the missing line added.

Verification

By execution, not by eye: the four classes were built from this branch's source and their repr() compared against the docstring text, and every >>> line in the four blocks was compile()d.

4 wrong reprs and 1 line that does not compile on main, 0 and 0 after.

ruff format --check and ruff check clean on all four files.

Image, Pdf, Nifti and Video all declare `id` with repr=False, so none of them can
print the `X(decode=True, id=None)` their examples show. Image also has a `mode`
field the example omits, and Video has five more fields that do print.

Also close the string literal in Video's last example, which made the line
unparseable, and add the `>>> ds[0]["image"]` line Image's last block needs to
produce the dict under it, the way pdf.py and nifti.py already do.
Same drift as the four classes already in this PR, in two files it did not reach.

iterable_dataset.py prints Audio with mono=True, id=None. Audio has no mono
field at all, so the line is not just stale, it cannot be constructed, and id
carries repr=False. It also prints Image(mode=None, decode=False, id=None).

features.py Features.flatten prints List(Value('int32'), id=None) twice, and
List keeps id out of its repr as well.

Features.reorder_fields_as is a different miss. Its example builds f1 and f2
from plain dicts, but its own comment says "here List is defined at the root
level", it imports List without using it, and the documented output is
{'root': List({'b': ..., 'a': ...})}. The List wrapper was dropped from the two
constructor lines at some point and the output was never updated. Putting it
back makes the example agree with its comment, its import and its output.

Checked against main:

    Audio(sampling_rate=8000)       -> Audio(sampling_rate=8000, decode=True, num_channels=None, stream_index=None)
    Image(mode=None, decode=False)  -> Image(mode=None, decode=False)
    List(Value('int32'))            -> List(Value('int32'))
    f1.reorder_fields_as(f2)        -> {'root': List({'b': Value('string'), 'a': Value('string')})}

and f1.reorder_fields_as(f2).type == f2.type still holds, so the closing assert
in that example is unchanged.
@vineethsaivs

Copy link
Copy Markdown
Contributor Author

Pushed 101bb1a: the same drift in two files this PR had not reached, plus one example that lost its point.

iterable_dataset.py documents Audio(sampling_rate=8000, mono=True, decode=True, id=None). Audio has no mono field at all now, so that line cannot even be constructed, and id carries repr=False. Also Image(mode=None, decode=False, id=None).

features.py Features.flatten prints List(Value(int32), id=None) twice, and List keeps id out of its repr too.

Features.reorder_fields_as is a different miss. Its example builds f1 and f2 from plain dicts, yet its own comment says "here List is defined at the root level", it imports List without using it, and the documented output is {'root': List({...})}. The List wrapper was dropped from the two constructor lines and the output was never updated. Putting it back makes the example agree with its comment, its import and its output. f1.reorder_fields_as(f2).type == f2.type still holds, so the closing assert is unchanged.

Every line here was run against main, not reasoned about.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant