Relook at the deploy-pre-release in develop volttron-core

[craigpnnl]

GitHub Actions Workflow: Deploy Pre-Release Artifacts

This document outlines the GitHub Actions workflow designed to deploy pre-release artifacts for a project. The workflow is triggered by pushes to the develop branch and includes steps for setting up the environment, managing dependencies, version bumping, and publishing artifacts.

https://github.com/eclipse-volttron/volttron-core/blob/main/.github/workflows/deploy-pre-release.yml

hash 9d97589cad285adb8bca5c3593b8eabe9d04ddd2

Sequence Diagram

The sequence diagram below provides a visual representation of the workflow process using Mermaid.

sequenceDiagram
    actor GitHub
    participant Runner as GitHub Runner

    GitHub ->> Runner: Trigger on push to develop branch
    Runner ->> Runner: Echo event details
    note right of Runner: Set up environment
    Runner ->> Runner: Checkout code
    Runner ->> Runner: Setup Python
    Runner ->> Runner: Install Poetry
    note right of Runner: Dependency Management
    Runner ->> Runner: Load cached virtualenv
    alt Cache not hit
        Runner ->> Runner: Install dependencies
    end
    Runner ->> Runner: Install root library
    note right of Runner: Version Management
    Runner ->> Runner: Bump version to prerelease
    note right of Runner: Project Build
    Runner ->> Runner: Create build artifacts
    Runner ->> Runner: Release artifacts
    note right of Runner: Publish
    alt Repository owner is eclipse-volttron
        Runner ->> PyPI: Publish pre-release
    end

Loading

Workflow Overview

Workflow Name

• Name: Deploy Pre-Release Artifacts
• Trigger: Push events on the develop branch.

Environment Setup

• Shell: Bash
• Python Version: 3.10
• Environment Variables:
  • LANG: en_US.utf-8
  • LC_ALL: en_US.utf-8

Workflow Steps and Description

1. Trigger

• The workflow is triggered by a push event on the develop branch.

2. Setup Environment

• Checkout Code:
  • Uses actions/checkout@v3.5.2 to pull code from the repository with complete history (fetch-depth: 0).
• Setup Python:
  • Configures the desired Python version (3.10) using actions/setup-python@v2.
• Install Poetry:
  • Installs Poetry with specific settings using snok/install-poetry@v1.3.3 to manage dependencies in a Python virtual environment.

3. Dependency Management

• Load Cached Virtual Environment:
  • Uses actions/cache@v2.1.7 to load cached dependencies if available, caching based on Python version and Poetry lock file.
• Install Dependencies:
  • If the cache is not hit, Poetry installs project dependencies using poetry install --no-interaction --no-root.

4. Version Management

• Bump Version:
  • Checks the latest tag, bumps the version using prerelease, or initializes it for the very first release as v0.0.1rc0.

5. Project Build

• Create Build Artifacts:
  • Uses Poetry to build the project and generate distribution packages with poetry build -vvv.

6. Release and Publish

• Release Artifacts:
  • Uses ncipollo/release-action@v1.12.0 to release the build artifacts, generate release notes, and create a pre-release tag.
• Publish to PyPI:
  • Publishes the pre-release artifacts to PyPI if the repository owner is eclipse-volttron.

Environment Variables and Tokens

• NEW_TAG: The new version tag generated during the bump version step.
• GITHUB_TOKEN: GitHub token for authenticating API requests, used in the release action.
• PYPI_TOKEN: Authentication token for publishing to PyPI, stored securely in secrets.

This workflow supports the CI/CD process by managing pre-release versions and automating the deployment, ensuring quality and consistency in the release cycle.

[schandrika]

I propose we change the "bump version to prerelease" step of the above sequence diagram to use the version number in pyproject.toml as base, run "poetry version prerelease --no-interaction" to get the next version, auto-commit the modified pyproject.toml to develop branch and the release to pypi using the newly bumped version.
We can remove the old 10.0x tags in github

[schandrika]

this is an example implementation of the proposed solution This needs to be updated to check if the source of the push is not a release branch or main.

[schandrika]

Also do not run pre-release workflow if PR is marked as "documentation"

[schandrika]

The pre-release workflow does not work with push is from a fork. It only works with branches see sub issue

Need a different approach to when the pre-release workflow is run or how it is run. We could create a new branch and create a new PR for just doing release but this would be additional PR merge.
Alternatively, we could do a periodic action that checks for changes from last release and do a pre-release push to PYPI only if there are changes other than documentation