Skip to content

Improve performance of openeing files. - #6

Open
dncnmcdougall wants to merge 11 commits into
gtluu:mainfrom
rosalindfranklininstitute:main
Open

Improve performance of openeing files.#6
dncnmcdougall wants to merge 11 commits into
gtluu:mainfrom
rosalindfranklininstitute:main

Conversation

@dncnmcdougall

Copy link
Copy Markdown

This update works to improve the reading performance of large files.
See #5.
It does this simply by reading the sqlite databases in a lazy manner.
This also provdes some utilities, like join_frame and range on the analyse object that can be used for more complex queries.
In particular join_frame can be used to join another table to the Frames table along the Frame Id.
For example when reading MALDI data using frame_info = tof_data.analysis.join_frame("MaldiFrameInfo") gives the frame infor and the x and y all at the same time, reducing iteration complexity.

Secondly this adds the tims_read_scans_v2_as_array method which returns the results as a single numpy array. This is slightly faster than tims_read_scans_v2 and yields the data in a manner that is potentially faster for processing.

Unfortunately this PR also has a lot of collateral changes:

  1. The code was reformatted with Ruff
  2. A uv pyproject.toml was added
  3. The licensing was moved over to REUSE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant