pipeline-eds is a Python project designed to simplify API access to Emerson Enterprise Data Server (EDS) machines. It facilitates seamless data exchange between Emerson's Ovation EDS system and various external parties, including third-party contractors and internal employees. The project is distributed on PyPI under the package name pipeline-eds.
For detailed installation and usage instructions, see QUICKSTART.md.
This guide includes step-by-step commands for Windows, Linux, Termux, and developer setups.
This detailed guide provides instructions for installing and running pipeline-eds. To ensure you follow the right path, first choose the method that best fits your needs.
- For the absolute simplest setup (No Python required): If you want to run the tool without installing Python or managing dependencies, use a pre-built binary. Follow Method 1: Using Pre-Built Binaries.
- For an easy CLI setup with simple updates: If you are comfortable with the command line and want to easily keep the tool updated,
pipxis the recommended approach. Follow Method 2: CLI Installation withpipx. - For developing and contributing to the project: If you plan to modify the source code, you need a full development environment. Follow Method 3: Developer Setup with UV.
- For installing from a Git clone without UV (Advanced): If you have cloned the repository and need to install dependencies manually with
pip, follow Method 4: Installing from a Git Clone withpip.
Method 1: Using Pre-Built Binaries (`.exe`, `.elf`, `.pyz`)
This is the easiest way to get started, especially on systems where you don't have Python installed. These are standalone packages that you can download and run directly.
-
Download the appropriate binary for your system from the project's GitHub Releases page.
pipeline-eds*.exe: For Windows.pipeline-eds* (ELF has no extension): For Linux and Termux on Android.pipeline-eds*.pyz: A zipapp for any system that has Python installed.
-
Place the file in a convenient location.
# On Termux (Android) termux-setup-storage cp storage/downloads/ . to copy the file from your Android downloads folder to your $HOME folder
On iSH, you launch by default in the
rootdirectory, and the executable can be copied here (if you would like) manually using the file browser. -
Run the command from your terminal. You may need to make the
.elffile executable first (chmod +x pipeline-eds-*).# On Windows .\pipeline-eds-*.exe config # On Linux or Termux ./pipeline-eds-* config
For more details on the pros and cons of each binary type, see the Distribution and Packaging section below.
Method 2: CLI Installation with `pipx` (Recommended for CLI Users)
pipx installs and runs Python applications in isolated environments. This is the best way to get easy updates and avoid conflicts with other Python packages.
1. Install Python and pip
If you don't have them, install them using your system's package manager or an official installer.
Windows Note: If installing from the
.exeinstaller from python.org, be sure to check the box for "Add Python to PATH" during setup.
# On Windows (using a package manager in PowerShell)
winget install Python.Python.3.11
# Or with Chocolatey:
# choco install python
# On Ubuntu/Debian
sudo apt update && sudo apt install python3 python3-pip python-is-python3
# On Termux (Android)
pkg update && pkg install python
pkg install python python-cryptography # python-numpy # numpy dependency removed for 0.3.50
pip install --upgrade setuptools wheel # Just to be sure.
# Some of these are likely overkill given prepackaged cryptography, but I want you to succeed, and I will continue testing.
pkg install clang make libffi openssl-dev libffi-dev
# pkg install rust # probably not necessary, with `python-cryptography` installed
# Alterative to python-cryptography, you could `pip install cryptography` after `pip install setuptools`
# On Alpine (iSH on iOS)
apk update && apk add python3 py3-pip
apk add py3-cryptography py3-numpy py3-virtualenv
apk add openssl-dev libffi-dev2. Install pipx
Use pip to install pipx and add its scripts to your system's PATH.
python3 -m pip install --user pipx
python3 -m pipx ensurepath(You may need to restart your terminal for the PATH change to take effect.)
3. Install pipeline-eds
Install the package from PyPI using pipx.
# For all systems (Linux, macOS, Windows)
pipx install pipeline-eds
# For Termux and iSH, which require dedicated system site packages like py3-cryptography, as seen above
pipx install --system-site-packages pipeline-eds
# For Windows users who want database features from the pyodbc library (the usefulness of this has not yet been developed).
pipx install "pipeline-eds[windows]"
# If you want non-browser plotting with Matplotlib (Linux, macOS, Windows)
pipx install "pipeline-eds[mpl]"
# With the `trend` command, use the `--webplot` flag to direct the plot to plotly HTML anyways and circumvent Matplotlib.4. Run Commands
You can now use the eds alias directly in your terminal. There is also the pipeline alias and the pipeline-eds alias.
eds config
eds trend M100FI --start June3 --end June17Method 3: Developer Setup with UV
This method is for contributors who need a full development environment to modify the source code. Learn more about git. See git guide.
** Clone the Repository**
git clone https://github.com/City-of-Memphis-Wastewater/pipeline_eds.git
cd pipeline_eds** Install uv**
This project uses uv for dependency management.
** Run Commands**
uv run eds config
uv run eds pingMethod 4: Installing from a Git Clone with `pip` (Advanced)
This method is for users who have cloned the repository but prefer to manage the environment with pip and venv. This is often necessary on platforms like Termux or iSH (Alpine).
A use-case for this is for generating binaries on a system such that it is compatible with that system (Note that iSH emulates x86_64).
** Clone the Repository**
git clone https://github.com/City-of-Memphis-Wastewater/pipeline_eds.git
cd pipeline_eds- Install System Build Dependencies:
pkg update && pkg upgrade -y pkg install python python-cryptography # python-numpy # numpy is no longer needed as of 0.3.50 # The build tools (rust, clang, etc.) are generally not needed IF the # Termux-installed packages satisfy the requirements. # The below line can likely be removed if --system-site-packages is used, # but we will leave it for max compatibility. pkg install rust clang make openssl-dev libffi-dev pip install --no-cache-dir cryptography
- Create and Activate a Virtual Environment:
# CRITICAL: Use the --system-site-packages flag to access Termux's pre-compiled packages. python -m venv --system-site-packages .venv source .venv/bin/activate
- Install Python Dependencies:
pip install -r requirements.txt
- Run Commands:
python -m pipeline_eds.cli config
- Install System Build Dependencies:
apk update # Install the core Python environment tools and essential pre-compiled Python libraries # Installing 'py3-cryptography' and 'py3-numpy' via apk avoids difficult, lengthy compilation from source later. apk add python3 py3-pip py3-cryptography py3-numpy py3-virtualenv apk add gcc musl-dev build-base openssl-dev libffi-dev
- Create and Activate a Virtual Environment:
# The '--system-site-packages' flag is crucial: it allows this venv to access the # pre-compiled Python packages (like py3-cryptography) installed in the previous step # by the system package manager (apk). This satisfies their requirements without re-installing. python3 -m venv --system-site-packages .venv # Activate the virtual environment source .venv/bin/activate
- Install Python Dependencies:
# Install all project-specific dependencies defined in the requirements file. # 'pip' will install these packages into the isolated '.venv', while still # using the system packages (if needed) due to the venv's configuration. pip install -r requirements.txt
- Run Commands:
# Execute the main application command using the Python interpreter from the activated venv. python3 -m pipeline_eds.cli trend M100FI # Deactivate the virtual environment. This resets the shell's PATH to the system's # default Python environment. deactivate # NOTE: If you run the software after deactivating, the 'python3' command will only see # system-installed libraries, not the packages installed specifically for this project. # You can reactivate the environment anytime using 'source .venv/bin/activate' if you are # in the project's directory.
📦 Distribution and Packaging
This project supports multiple packaging formats to make installation flexible across platforms.
While some formats allow installation on systems without internet access, note that the application itself requires internet connectivity to call its API.
- Generated by:
build_executable.py - Variants:
- Windows
.exe: Tested on Windows 11. Runs standalone without requiring Python. - Linux
.elf:- Built on WSL2 Ubuntu for general Linux systems.
- Built on Termux for Android devices.
- Smoothest rollout on Termux: no need to install Python separately.
- Avoids the
.shortcutswidget permission error seen with.pyz.
- Windows
- Internet required for install: ❌ (binaries can be copied directly)
- Internet required for use: ✅ (API calls)
- Generated by:
build_shiv.shon WSL2 Ubuntu - Best for: Systems that already have Python installed.
- Windows support: A
.batlauncher is provided for smoother execution. - Termux notes:
- Works, but calling
.pyzfrom the Termux.shortcutswidget can trigger a permission error. - Requires Python to be installed on Termux.
- Works, but calling
- Maintainability: Easier to update regularly compared to static binaries.
- Internet required for install: ❌ (once
.pyzis copied) - Internet required for use: ✅
- Best for: Staying current with rolling changes.
- Update shortcut: On Termux, an update shortcut is available directly from the home screen widget.
- Requirements: Python and
pipxinstalled. - Internet required for install/update: ✅
- Internet required for use: ✅
- Generated by: uv
- Best for: Developers or environments where building from source is preferred.
- Internet required for install: ✅ (to fetch dependencies)
- Internet required for use: ✅
- These packages simplify installation on disconnected systems, but the application itself requires internet access to function (API calls).
To ensure a smooth and efficient experience with pipeline-eds, consider the following best practices:
Regularly upgrade your pipeline-eds installation to benefit from the latest features, bug fixes, and performance improvements.
pipx upgrade pipeline-edsThe first time you execute a command requiring access to your EDS API (e.g., eds trend), pipeline-eds will guide you through a one-time configuration process. Your sensitive API credentials (URL, username, password) are securely stored using your operating system's native keyring service. This avoids storing plaintext passwords. If your credentials change, you can re-run eds config at any time to update them.
If your EDS server is located on a private network (e.g., within your organization's internal network), you must be connected to the appropriate Virtual Private Network (VPN). Failure to do so will result in connection errors when pipeline-eds attempts to fetch data.
The eds trend command offers highly flexible date and time parsing for its --start and --end options, thanks to the pendulum package. You can use a wide variety of natural language inputs, such as:
--start "2023-09-18"--start "Sept 18"--end "now"
Experiment with different formats to suit your query needs. Remember to use quotes around values if they contain spaces.
Aside from being a CLI tool for visualing sensor data at will, pipeline is designed to be deployed as a scheduled task on a Windows server, for hourly data transmission to a third party.
- The project is executed by Windows Task Scheduler, which calls a PowerShell script (
main_eds_to_rjn_quiet.ps1) as the entry point. - The iterative timing (e.g., hourly execution) is handled by the
Task Scheduler, not by Python. - For these automated tasks, a standard
venvis used, asTask Schedulercan run under different user accounts.
The pipeline project can be installed and run on Android devices using the Termux terminal emulator.
For most users, CLI installation via pipx is the recommended method, as development is not expected in this environment and pipx provides the smoothest way to stay up to date.
There are several ways to run pipeline inside Termux, depending on your needs:
-
pipx(Recommended)- Provides rolling updates and integrates with the Termux home‑screen widget.
- As of v0.3.8, you
eds setupwill generate a shortcut button in the Termux widget to updatepipeline-edswithout needing to open the terminal. - Requires Python and
pipxto be installed. - Best choice if you want to keep up with frequent changes.
-
Native ELF Binary
- A prebuilt
.elfbinary is available specifically for Termux. - Smoothest rollout for users who just want the CLI tool without expecting updates.
- A prebuilt
-
.pyzZipapp⚠️ Known limitation: launching.pyzdirectly from the Termux.shortcutswidget may trigger a permission error.
When using pipeline-eds in Termux to generate plots (e.g., with eds trend), the visuals are displayed as web-based HTML pages using libraries like Plotly. Instead of directly opening a graphical window (which is not typically supported by Termux's command-line environment), pipeline-eds serves these HTML files via a local web server (often on localhost).
🐍 Python Version Compatibility
The pipeline-eds project is designed to support a broad range of modern Python versions, from Python 3.8 up to the latest stable releases, ensuring accessibility across various operating environments (desktop, server, and mobile environments like Termux).
The project officially supports the following CPython versions:
| Python Version | Status | Key Dependency Notes |
|---|---|---|
| 3.11 / 3.12 / 3.13 / 3.14 | ✅ Fully Supported | Runs on the latest major versions of all dependencies (e.g., keyring v25+, pendulum v3+, urllib3 v2+). |
| 3.10 | ✅ Fully Supported | Stable; uses latest dependencies, with specific numpy and matplotlib pins. |
| 3.9 | ✅ Fully Supported | Stable; this version marks the transition to modern dependency major versions. |
To maintain compatibility across this range, we pin several major dependencies based on your Python version, ensuring maximum stability.
-
Python 3.9+ (Recommended): Installations on Python 3.9 and newer automatically receive the latest, feature-rich major versions of core libraries such as
keyring,pendulum,urllib3, and the development toolspytestandpytest-cov. -
Python 3.8 (Legacy): This project no longer support Python 3.8
Python 3.11 or newer is highly recommended for the best performance, security, and access to the latest features from all third-party libraries. If you are using the Developer Setup, please target Python 3.11.9 as specified in the getting started guide.