Rust Workflow Engine

A multi-threaded, asynchronous engine for defining, scheduling, and executing complex workflows. Built with Rust and powered by Tokio, it allows you to create dynamic, dependency-driven workflows with built-in support for retries, and persistence.

Table of Contents

• Features
• Getting Started
• Usage
• Examples
• Project Structure
• Extending the Engine

Features

• Dynamic Workflow Definition: Define workflows and tasks at runtime.
• Dependency Management: Configure tasks with dependencies for sequential or parallel execution.
• Asynchronous Execution: Built with Tokio for high-performance, concurrent task execution.
• Retry Policies: Automatic task retries on failure with customizable delay.
• State Persistence: Track task and workflow status with a pluggable storage backend.

Getting Started

Prerequisites

• Rust (1.56+ recommended)
• SQLite (If using the SQLite storage backend)

Installation

As a Dependency:

[dependencies]
workflow-engine = { git = "https://github.com/romanguy13/workflow-engine" }

From Source:

git clone https://github.com/romanguy13/workflow-engine.git
cd rust-workflow-engine
cargo build

Usage

Setup storage

The engine requires a storage backend to persist workflow and task state. If you are using the SQLite storage backend, initialize a *.db file:

touch workflow.db

Defining a Task

Define a task by implementing the Task trait:

use async_trait::async_trait;
use workflow_engine::task::Task;

struct MyCustomTask {
    name: String,
}

#[async_trait]
impl Task for MyCustomTask {
    async fn execute(&self) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
        println!("Executing task: {}", self.name);
        // Do some work here
        Ok(())
    }
}

Defining a Workflow

Below is a simple example of how to define and run a workflow:

use std::sync::Arc;
use workflow_engine::{RetryPolicy, Task, Workflow};
use workflow_engine::storage::implementations::SqliteStorage;

// Define a custom task
struct SimpleTask {
    name: String,
}

#[async_trait::async_trait]
impl Task for SimpleTask {
    async fn execute(&self) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
        println!("Executing task: {}", self.name);
        // Do some work here
        Ok(())
    }
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    // Initialize storage
    let storage = Arc::new(SqliteStorage::new("workflow.db").await?);
    storage.init().await?;

    // Build workflow
    let workflow = Workflow::new("simple_workflow", storage)
        .add_task("task1", SimpleTask { name: "First Task".into() }, vec![], None)
        .add_task(
            "task2",
            SimpleTask { name: "Second Task".into() },
            vec!["task1".into()],
            Some(RetryPolicy::default().with_max_retries(2))
        );

    // Execute workflow
    workflow.execute().await?;

    Ok(())
}

Examples

Running Examples

Several examples are provided in the /examples directory. To run an example:

cargo run --example simple_example

Extending the Engine

Custom Storage

Implement the WorkflowStorage trait to create a custom storage backend. An SQLite storage implementation is provided in the storage module for reference.

#[async_trait::async_trait]
impl WorkflowStorage for MyCustomStorage {
    async fn init(&self) -> Result<(), Box<dyn Error + Send + Sync>> {
        // Initialize storage
    }

    async fn create_task_record(&self, workflow_id: &str, task_id: &str)
        -> Result<(), Box<dyn Error + Send + Sync>> {
        // Create task record
    }

    // Additional required methods...
}

Development

Code Style

• Use rustfmt to format your code:

  cargo fmt

• Follow Rust's idiomatic patterns and naming conventions.

Testing

• Run tests with:

  cargo test

License

This project is licensed under the MIT License. See the LICENSE file for details.

Contributing

All contributions are welcome! Feel free to open an issue or submit a pull request.

Happy coding! 🚀