Skip to content

Processing automation #23

Description

@raphidoc

Problem
The Algae-WISE data consists of 59 266 seconds (~16.48 hours) of acquisition (including transit). Considering the acquisition rate of the instruments, it takes a maximum of five seconds to create a discrete data point. So we have a potential database of 59 266 / 5 = 11 853 observations. It takes a minimum of 5 minutes to process a single discrete observation. So it would take (5x60)x11 853 = 3 555 900 seconds (~ 987 hours) to process it all (overestimate including transit data).

Solution
You can automate the process and leave the qualitative eye of the operator to deal with problematic observations only.

To make efficient automation, we first have to clean the data for transit recording:

  1. Speed_N <= 10
  2. Lu < threshold

Then we need to create the time interval that will be used to discretize the data:

  1. Start from a DateTime (x) take the next DateTime (y) with a minimum of three seconds difference and try to process.
  2. If there is not enough data (HOCR) take the next DateTime (y) (normally one second further) and try to process. Repeat
  3. If the next DateTime (y) has more than 10 seconds difference, start again with the next DateTime (y)

Describe alternatives
No alternative considered as of now

Additional context
Add any other context or screenshots about the feature request here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions