Skip to content

Add AMBAR music dataset to Lenskit - #1187

Open
sushobhan2024 wants to merge 12 commits into
lenskit:mainfrom
sushobhan2024:feature/ambar
Open

Add AMBAR music dataset to Lenskit#1187
sushobhan2024 wants to merge 12 commits into
lenskit:mainfrom
sushobhan2024:feature/ambar

Conversation

@sushobhan2024

@sushobhan2024 sushobhan2024 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

issue #1142

  • Added amber.py to load data from its path
  • Updated the describe.py so imported dataset can be described with the existing describe command

@sushobhan2024 sushobhan2024 self-assigned this Aug 18, 2026

@mdekstrand mdekstrand left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few things this needs off the bat:

  • tests (tagged with @mark.realdata, see the Amazon and Steam load tests for examples). The tests should probably look at it in data/ambar.
  • adding to data/convert.py.
  • adding to the data conversion CLI test script.

@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 2.43902% with 40 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.17%. Comparing base (928f7eb) to head (d9cc10c).

Files with missing lines Patch % Lines
src/lenskit/data/sources/ambar.py 0.00% 36 Missing ⚠️
src/lenskit/cli/data/describe.py 20.00% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1187      +/-   ##
==========================================
- Coverage   90.37%   90.17%   -0.21%     
==========================================
  Files         261      262       +1     
  Lines       17376    17417      +41     
==========================================
+ Hits        15704    15705       +1     
- Misses       1672     1712      +40     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mdekstrand

Copy link
Copy Markdown
Member

Now that the tests are there, they are failing: https://jynx.mde.one/repos/1/pipeline/54/3

The test environment has a copy of the AMBAR folder from the data set repo at data/ambar.

@sushobhan2024

sushobhan2024 commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

The error is due to duplicate user_ids. There are 8 duplicate user_ids.
5 have conflicts (contradictory country/continent, one even contradictory gender). The other 3 are repeated records.
I am considering removing those 5 records, and keeping first of the 3 repeated records.

Also, there are 8 different user_ids in ratings_info.csv that have no row at all in users_info.csv. Removing these as well, and passing missing="error" instead of missing="insert" in add_interactions().

conflicting user_ids (from users_info duplicates): 5
{506586384, 39168978, 128783640, 390604169, 38606714}

truly missing user_ids (in ratings, absent from users_info): 8
{271316318, 306287460, 163187558, 839347986, 871284436, 955505332, 183305018, 387251006}

overlap between the two groups: 0

ratings from conflicting users: 1018
ratings from truly missing users: 747
total distinct user_ids being excluded: 13

1,765 of 3,311,462 total ratings (0.053%) from these 13 users are now excluded.

Let me know if excluding is the right approach here.

@sushobhan2024
sushobhan2024 requested a review from aisa6148 August 20, 2026 22:29
@mdekstrand

Copy link
Copy Markdown
Member

Probably better to keep the ratings with missing users, those users just won't have attributes. On the conflicts, dropping is probably fine, we could also keep them and just pick the first or last record (if the records have timestamps, we should pick the last one by timestamp).

@mdekstrand

mdekstrand commented Aug 21, 2026

Copy link
Copy Markdown
Member

@mdekstrand just FYI, for security reasons I have to manually approve the woodpecker tests every time they run. They can't be re-triggered from within a push.

@sushobhan2024

Copy link
Copy Markdown
Contributor Author

This dataset does not have timestamps.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants