Merge branch 'master' into copilot/implement-gpu-support

Author: alexarje

docs/musicalgestures/_flow.md

@@ -75,15 +75,15 @@ Renders a dense optical flow video of the input video file using `cv2.calcOptica
 
 ### Flow().get_acceleration
 
-[[find in source code]](https://github.com/fourMs/MGT-python/blob/master/musicalgestures/_flow.py#L249)
+[[find in source code]](https://github.com/fourMs/MGT-python/blob/master/musicalgestures/_flow.py#L252)
 
 ```python
 def get_acceleration(velocity, fps):
 ```
 
 ### Flow().get_velocity
 
-[[find in source code]](https://github.com/fourMs/MGT-python/blob/master/musicalgestures/_flow.py#L259)
+[[find in source code]](https://github.com/fourMs/MGT-python/blob/master/musicalgestures/_flow.py#L262)
 
 ```python
 def get_velocity(
@@ -99,7 +99,7 @@ def get_velocity(
 
 ### Flow().sparse
 
-[[find in source code]](https://github.com/fourMs/MGT-python/blob/master/musicalgestures/_flow.py#L274)
+[[find in source code]](https://github.com/fourMs/MGT-python/blob/master/musicalgestures/_flow.py#L277)
 
 ```python
 def sparse(
@@ -137,7 +137,7 @@ Renders a sparse optical flow video of the input video file using `cv2.calcOptic
 
 ### Flow().velocity_meters_per_second
 
-[[find in source code]](https://github.com/fourMs/MGT-python/blob/master/musicalgestures/_flow.py#L267)
+[[find in source code]](https://github.com/fourMs/MGT-python/blob/master/musicalgestures/_flow.py#L270)
 
 ```python
 def velocity_meters_per_second(

docs/musicalgestures/_pose.md

@@ -18,7 +18,7 @@ Helper function to automatically download model (.caffemodel) files.
 
 ## pose
 
-[[find in source code]](https://github.com/fourMs/MGT-python/blob/master/musicalgestures/_pose.py#L14)
+[[find in source code]](https://github.com/fourMs/MGT-python/blob/master/musicalgestures/_pose.py#L30)
 
 ```python
 def pose(
@@ -37,15 +37,20 @@ def pose(
 ```
 
 Renders a video with the pose estimation (aka. "keypoint detection" or "skeleton tracking") overlaid on it.
-Outputs the predictions in a text file containing the normalized x and y coordinates of each keypoints
-(default format is csv). Uses models from the [openpose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) project.
+Outputs the predictions in a text file containing the normalized x and y coordinates of each keypoint
+(default format is csv).
+
+Supports two backends:
+
+- **MediaPipe** (`model='mediapipe'`): Uses Google's MediaPipe Pose which detects 33 landmarks entirely on CPU. Requires the optional `mediapipe` package (`pip install musicalgestures[pose]`). The model file (~8–28 MB) is auto-downloaded on first use and cached in `musicalgestures/models/`.
+- **OpenPose** (`model='body_25'`, `'coco'`, or `'mpi'`): Uses Caffe-based OpenPose models. Model weights (~200 MB) are downloaded on first use.
 
 #### Arguments
 
-- `model` *str, optional* - 'body_25' loads the model trained on the BODY_25 dataset, 'mpi' loads the model trained on the Multi-Person Dataset (MPII), 'coco' loads one trained on the COCO dataset. The BODY_25 model outputs 25 points, the MPII model outputs 15 points, while the COCO model produces 18 points. Defaults to 'body_25'.
-- `device` *str, optional* - Sets the backend to use for the neural network ('cpu' or 'gpu'). Defaults to 'gpu'.
+- `model` *str, optional* - Pose model to use. `'mediapipe'` uses MediaPipe Pose (33 landmarks, model auto-downloaded on first use). `'body_25'` loads the OpenPose BODY_25 model (25 keypoints), `'mpi'` loads the MPII model (15 keypoints), `'coco'` loads the COCO model (18 keypoints). Defaults to 'body_25'.
+- `device` *str, optional* - Sets the backend to use for the neural network ('cpu' or 'gpu'). Ignored when `model='mediapipe'` (MediaPipe always runs on CPU). Defaults to 'gpu'.
 - `threshold` *float, optional* - The normalized confidence threshold that decides whether we keep or discard a predicted point. Discarded points get substituted with (0, 0) in the output data. Defaults to 0.1.
-- `downsampling_factor` *int, optional* - Decides how much we downsample the video before we pass it to the neural network. For example `downsampling_factor=4` means that the input to the network is one-fourth the resolution of the source video. Heaviver downsampling reduces rendering time but produces lower quality pose estimation. Defaults to 2.
+- `downsampling_factor` *int, optional* - Decides how much we downsample the video before we pass it to the neural network. Ignored when `model='mediapipe'`. Defaults to 2.
 - `save_data` *bool, optional* - Whether we save the predicted pose data to a file. Defaults to True.
 - `data_format` *str, optional* - Specifies format of pose-data. Accepted values are 'csv', 'tsv' and 'txt'. For multiple output formats, use list, eg. ['csv', 'txt']. Defaults to 'csv'.
 - `save_video` *bool, optional* - Whether we save the video with the estimated pose overlaid on it. Defaults to True.

docs/musicalgestures/_pose_estimator.md

@@ -26,7 +26,7 @@ This module provides:
 * class `PoseEstimator` – an abstract base class (ABC) defining the common
   interface that all pose backends must implement.
 * class `MediaPipePoseEstimator` – a concrete backend powered by Google
-  MediaPipe Pose (33 landmarks, CPU-friendly, zero model download).
+  MediaPipe Pose (33 landmarks, CPU-friendly, auto-downloads model on first use).
 * class `OpenPosePoseEstimator` – a thin wrapper around the legacy OpenPose /
   Caffe-model implementation already present in :mod:[Pose](_pose.md#pose).
 
@@ -56,30 +56,30 @@ class MediaPipePoseEstimator(PoseEstimator):
         model_complexity: int = 1,
         min_detection_confidence: float = 0.5,
         min_tracking_confidence: float = 0.5,
-        static_image_mode: bool = False,
     ) -> None:
 ```
 
-Pose estimator backed by Google MediaPipe Pose.
+Pose estimator backed by Google MediaPipe Pose (Tasks API).
 
-Requires the optional ``mediapipe`` package
+Requires the optional ``mediapipe>=0.10`` package
 
 ```python
 pip install musicalgestures[pose]
 ```
 
+The first time you use a given complexity level the corresponding
+`.task` model file (~8–28 MB) is downloaded from Google's model
+storage and cached in `musicalgestures/models/`.
+
 Parameters
 ----------
 model_complexity:
-    MediaPipe model complexity (0, 1, or 2).  Higher = more accurate
-    but slower.  Default: 1.
+    MediaPipe model complexity (0 = lite, 1 = full, 2 = heavy).
+    Higher values are more accurate but slower.  Default: 1.
 min_detection_confidence:
     Minimum confidence for initial body detection. Default: 0.5.
 min_tracking_confidence:
     Minimum confidence for landmark tracking. Default: 0.5.
-static_image_mode:
-    If *True*, treat every frame as a static image (no tracking).
-    Default: False.
 
 Examples
 --------

musicalgestures/__init__.py

@@ -15,6 +15,7 @@
     get_length,
     generate_outfilename,
     get_cuda_device_count,
+    show_progress,
 )
 from musicalgestures._mglist import MgList
 

musicalgestures/_cropvideo.py

@@ -197,28 +197,37 @@ def mg_cropvideo_ffmpeg(
 
     if crop_movement.lower() == 'manual':
         if not in_colab():
-
-            # scale_ratio = get_box_video_ratio(filename)
-            # width, height = get_widthheight(filename)
-            # scaled_width, scaled_height = [int(elem * scale_ratio) for elem in [width, height]]
-            # first_frame_as_image = get_first_frame_as_image(filename, pict_format='.jpg')
-
-            # Cropping UI moved to another subprocess to avoid cv2.waitKey crashing Python with segmentation fault on Linux in Terminal
-            import threading
-            import queue
-
-            que = queue.Queue()
-            t = threading.Thread(target=lambda q, arg1:q.put(cropping_window(arg1)), args=(que, filename))
-
-            t.start()
-            t.join()
-
-            w, h, x, y = que.get()          
-
-            # x = threading.Thread(target=run_cropping_window, args=(first_frame_as_image, scale_ratio, scaled_width, scaled_height))
-            # run_cropping_window(first_frame_as_image, scale_ratio, scaled_width, scaled_height)
-            # x.start()
-            # x.join()
+            import sys
+            import subprocess
+            import musicalgestures
+
+            scale_ratio = get_box_video_ratio(filename)
+            width, height = get_widthheight(filename)
+            scaled_width, scaled_height = [int(elem * scale_ratio) for elem in [width, height]]
+            first_frame_as_image = get_first_frame_as_image(filename, pict_format='.jpg')
+
+            module_path = os.path.abspath(os.path.dirname(musicalgestures.__file__))
+            pyfile = os.path.join(module_path, '_cropping_window.py')
+
+            result = subprocess.run(
+                [sys.executable, pyfile, first_frame_as_image, str(scale_ratio), str(scaled_width), str(scaled_height)],
+                stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True
+            )
+
+            os.remove(first_frame_as_image)
+
+            if result.returncode != 0:
+                raise RuntimeError(
+                    f"Cropping window subprocess failed (exit code {result.returncode}):\n{result.stderr}"
+                )
+
+            res = result.stdout.strip()
+            res_array = res.split(' ')
+            if len(res_array) != 4:
+                raise RuntimeError(
+                    f"Unexpected output from cropping window: '{res}'"
+                )
+            w, h, x, y = [int(elem) for elem in res_array]
 
         else:
             x, y, w, h = manual_text_input()
@@ -228,11 +237,6 @@ def mg_cropvideo_ffmpeg(
 
     cropped_video = crop_ffmpeg(filename, w, h, x, y, target_name=target_name, overwrite=overwrite)
 
-    # if crop_movement.lower() == 'manual':
-    #     cv2.destroyAllWindows()
-    #     if not in_colab():
-    #         os.remove(first_frame_as_image)
-
     return cropped_video