Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 113 additions & 31 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ jobs:
outputs:
wheel-distribution: ${{ steps.wheel-distribution.outputs.path }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
with: {fetch-depth: 0} # deep clone for setuptools-scm
- uses: actions/setup-python@v5
- uses: actions/setup-python@v7
id: setup-python
with: {python-version: "3.11"}
- name: Run static analysis and format checkers
Expand All @@ -48,7 +48,7 @@ jobs:
- name: Store the distribution files for use in other stages
# `tests` and `publish` will use the same pre-built distributions,
# so we make sure to release the exact same package that was tested
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: python-distribution-files
path: dist/
Expand All @@ -66,13 +66,13 @@ jobs:
- windows-latest
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
id: setup-python
with:
python-version: ${{ matrix.python }}
- name: Retrieve pre-built distribution files
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with: {name: python-distribution-files, path: dist/}
- name: Run tests
run: >-
Expand All @@ -91,7 +91,7 @@ jobs:
parallel: true
- name: Save coverage report
if: ${{ matrix.platform == 'ubuntu-latest' && matrix.python == '3.11' }}
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: coverage
path: htmlcov
Expand All @@ -110,7 +110,7 @@ jobs:
needs: finalize
runs-on: ubuntu-latest
steps:
- uses: actions/create-github-app-token@v1
- uses: actions/create-github-app-token@v3
id: github-app-checkout
with:
app-id: ${{ secrets.BOT_GITHUB_APP_ID }}
Expand All @@ -119,19 +119,19 @@ jobs:
repositories: |
foapy-asv-results

- uses: actions/checkout@v4
- uses: actions/checkout@v7
with:
fetch-depth: 0

- uses: actions/checkout@v4
- uses: actions/checkout@v7
with:
repository: intervals-mining-lab/foapy-asv-results
path: benchmarks/results
ref: main
token: ${{ steps.github-app-checkout.outputs.token }}
persist-credentials: false

- uses: actions/setup-python@v5
- uses: actions/setup-python@v7
id: setup-python
with: {python-version: "3.11"}

Expand Down Expand Up @@ -169,15 +169,73 @@ jobs:
cd ./benchmarks
asv machine --machine gh-runner --yes || true
asv setup -v || true
asv run ALL --skip-existing --append-samples --machine gh-runner || true
# Measure both revisions on this runner so the comparison does not
# mix results from different ephemeral GitHub-hosted machines.
printf '%s\n' \
'${{ steps.context.outputs.base }}' \
'${{ steps.context.outputs.head }}' \
> ./comparison_commits.txt
asv run \
HASHFILE:comparison_commits.txt \
--interleave-rounds \
--machine gh-runner || true
asv publish

asv compare --machine gh-runner ${{ steps.context.outputs.base }} HEAD --split --only-changed > ./benchmark_report.md || true
cat ./benchmark_report.md >> $GITHUB_STEP_SUMMARY
asv compare \
--machine gh-runner \
--split \
--only-changed \
'${{ steps.context.outputs.base }}' \
'${{ steps.context.outputs.head }}' \
> ./benchmark_report.md || true

if [ -s ./benchmark_report.md ]; then
echo "has_report=true" >> $GITHUB_OUTPUT
python - <<'PY'
import os
from pathlib import Path

report = Path("benchmark_report.md").read_text(encoding="utf-8")
run_url = (
f"{os.environ['GITHUB_SERVER_URL']}/"
f"{os.environ['GITHUB_REPOSITORY']}/actions/runs/"
f"{os.environ['GITHUB_RUN_ID']}"
)
suffix = (
"\n\n_Report truncated. "
f"[Open the workflow run]({run_url}) or download the "
"`benchmark-report` artifact for the complete comparison._\n"
)


def excerpt(prefix, byte_limit):
complete = prefix + report
if len(complete.encode("utf-8")) <= byte_limit:
return complete

budget = byte_limit - len((prefix + suffix).encode("utf-8"))
lines = []
size = 0
for line in report.splitlines(keepends=True):
line_size = len(line.encode("utf-8"))
if size + line_size > budget:
break
lines.append(line)
size += line_size
return prefix + "".join(lines) + suffix


Path("benchmark_summary.md").write_text(
excerpt("", 900_000), encoding="utf-8"
)
Path("benchmark_comment.md").write_text(
excerpt("<!-- add-pr-comment:add-pr-comment -->\n\n", 60_000),
encoding="utf-8",
)
PY
cat ./benchmark_summary.md >> "$GITHUB_STEP_SUMMARY"
echo "has_report=true" >> "$GITHUB_OUTPUT"
else
echo "has_report=false" >> $GITHUB_OUTPUT
echo "has_report=false" >> "$GITHUB_OUTPUT"
fi

cd ./results
Expand All @@ -187,7 +245,7 @@ jobs:
git add .
git commit -m "Update benchmark results" || true

- uses: actions/create-github-app-token@v1
- uses: actions/create-github-app-token@v3
id: github-app-push
with:
app-id: ${{ secrets.BOT_GITHUB_APP_ID }}
Expand All @@ -204,17 +262,41 @@ jobs:
branch: ${{ steps.context.outputs.branch }}
directory: benchmarks/results

- uses: mshick/add-pr-comment@v2
if: ${{ steps.benchmark.outputs.has_report == 'true' }}
with:
message-path: benchmarks/benchmark_report.md
- name: Update benchmark PR comment
if: ${{ github.event_name == 'pull_request' && steps.benchmark.outputs.has_report == 'true' }}
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |-
comment_id="$(
gh api \
"repos/$GITHUB_REPOSITORY/issues/$PR_NUMBER/comments?per_page=100" \
--jq '[.[] | select(.body | startswith("<!-- add-pr-comment:add-pr-comment -->"))][-1].id // empty'
)"
if [ -n "$comment_id" ]; then
gh api \
--method PATCH \
"repos/$GITHUB_REPOSITORY/issues/comments/$comment_id" \
-F body=@benchmarks/benchmark_comment.md
else
gh api \
--method POST \
"repos/$GITHUB_REPOSITORY/issues/$PR_NUMBER/comments" \
-F body=@benchmarks/benchmark_comment.md
fi

- name: Save benchmark report
uses: actions/upload-artifact@v4
- name: Save benchmark website
uses: actions/upload-artifact@v7
with:
name: benchmark
path: benchmarks/html

- name: Save benchmark comparison
uses: actions/upload-artifact@v7
with:
name: benchmark-report
path: benchmarks/benchmark_report.md

docs:
needs: benchmark
runs-on: ubuntu-latest
Expand All @@ -240,27 +322,27 @@ jobs:
# (e.g. "goruha/sync-openspec-specs"), so slugify it here.
echo "docs-version=$(echo '${{ steps.context.outputs.branch }}' | tr '/' '-')" >> "$GITHUB_OUTPUT"

- uses: actions/checkout@v4
- uses: actions/checkout@v7
with: {fetch-depth: 0} # deep clone for setuptools-scm

- uses: actions/setup-python@v5
- uses: actions/setup-python@v7
id: setup-python
with: {python-version: "3.11"}

- name: Retrieve coverage report
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: coverage
path: htmlcov

- name: Retrieve benchmark report
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with:
name: benchmark
path: benchmarks/html

- name: Setup Pages
uses: actions/configure-pages@v5
uses: actions/configure-pages@v6

- name: Build package distribution files
run: |
Expand All @@ -282,11 +364,11 @@ jobs:
id-token: write
contents: write
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with: {python-version: "3.11"}
- name: Retrieve pre-built distribution files
uses: actions/download-artifact@v4
uses: actions/download-artifact@v8
with: {name: python-distribution-files, path: dist/}
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
41 changes: 21 additions & 20 deletions .specify/memory/constitution.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,18 @@
<!--
SYNC IMPACT REPORT
==================
Version change: [unversioned template] → 1.0.0
Modified principles: N/A (initial ratification)
Version change: 1.0.0 → 1.1.0
Modified principles:
- IV. Performance Requirements — prohibit every Python iteration construct in production code
Added sections:
- Core Principles (I–V)
- Development Workflow
- Quality Gates
- Governance
Removed sections: N/A
- Root AGENTS.md enforcement context
Removed sections:
- Documented production-loop exception
Templates requiring updates:
✅ .specify/templates/plan-template.md — Constitution Check section already present; gates now concrete
✅ .specify/templates/spec-template.md — Success Criteria / Performance Goals align with Principles IV and V
✅ .specify/templates/tasks-template.md — Polish phase tasks align with Principles I–IV
✅ .specify/templates/constitution-template.md — source template; no changes needed
✅ .specify/templates/plan-template.md — vectorization gate made explicit
✅ .specify/templates/tasks-template.md — production-loop audit added to final tasks
✅ .specify/templates/spec-template.md — no change required
✅ .specify/templates/constitution-template.md — no change required
Deferred TODOs: none
-->

Expand Down Expand Up @@ -72,18 +71,20 @@ signatures force defensive branching in caller code and break the substitution p

### IV. Performance Requirements

All computations on sequences MUST use vectorized numpy operations; Python-level loops over array
elements are prohibited.
All production computations MUST use C-backed vectorized NumPy operations. Python iteration
constructs (`for`, `while`, comprehensions, and generator expressions) are prohibited throughout
`src/foapy/`; loops are permitted only in tests and benchmark setup code.

- Operations on sequences up to length 10 000 MUST complete in < 100 ms on a single CPU core (no GPU
assumption).
- Memory allocation MUST be O(n) or better in sequence length; hidden quadratic allocations MUST be
eliminated before merge.
- Performance-sensitive paths (interval extraction, characteristic computation) MUST avoid
`numpy.vectorize` (which is a disguised Python loop) and MUST prefer `numpy.where`, boolean indexing,
`numpy.diff`, `numpy.unique`, or equivalent C-backed ufuncs.
- If a vectorized solution genuinely cannot express a required algorithm, a fallback loop MUST be
documented with a complexity note and flagged in the Complexity Tracking table.
`numpy.vectorize`, `numpy.apply_along_axis`, and similar disguised Python loops, and MUST prefer
`numpy.where`, boolean indexing, `numpy.diff`, `numpy.unique`, indexed ufunc updates, or equivalent
C-backed operations over complete arrays or batches.
- A feature that cannot yet be expressed without production Python iteration MUST remain unimplemented
until a vectorized design is available; a complexity note does not waive this rule.

**Rationale**: FoaPy targets research workflows where sequences can be large and many characteristics
are computed in a batch. Python loops at the inner level produce unacceptable runtimes.
Expand Down Expand Up @@ -129,8 +130,8 @@ The following gates MUST pass before any feature branch is merged to `main`:
violation is documented with a justification in the Complexity Tracking table.
4. **API consistency check**: Any new public function mirrors the signature contract defined in
Principle III; `foapy.ma` parity is maintained.
5. **Performance check**: Any new sequence-processing path uses vectorized numpy; no Python loops over
array elements without a documented justification.
5. **Performance check**: Production code contains no Python iteration constructs or disguised loop
wrappers; sequence processing uses C-backed vectorized NumPy operations over complete arrays or batches.

## Governance

Expand All @@ -155,4 +156,4 @@ In conflicts between this document and any other guidance, the constitution prev
expected to call out constitution violations explicitly; authors are expected to resolve them before
merge, not after.

**Version**: 1.0.0 | **Ratified**: 2026-03-28 | **Last Amended**: 2026-03-28
**Version**: 1.1.0 | **Ratified**: 2026-03-28 | **Last Amended**: 2026-09-10
4 changes: 3 additions & 1 deletion .specify/templates/plan-template.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,9 @@

*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*

[Gates determined based on constitution file]
- [ ] Every production array computation is designed as C-backed NumPy batch operations.
- [ ] No production `for`, `while`, comprehension, generator expression, `numpy.vectorize`, or `numpy.apply_along_axis` is planned.
- [ ] Tests and benchmarks cover correctness and performance of the vectorized design.

## Project Structure

Expand Down
1 change: 1 addition & 0 deletions .specify/templates/tasks-template.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,7 @@ Examples of foundational tasks (adjust based on your project):
- [ ] TXXX [P] Documentation updates in docs/
- [ ] TXXX Code cleanup and refactoring
- [ ] TXXX Performance optimization across all stories
- [ ] TXXX Audit production code for Python iteration constructs and disguised loop wrappers
- [ ] TXXX [P] Additional unit tests (if requested) in tests/unit/
- [ ] TXXX Security hardening
- [ ] TXXX Run quickstart.md validation
Expand Down
6 changes: 6 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# FoaPy agent instructions

- Production code under `src/foapy/` MUST NOT contain Python iteration constructs: `for`, `while`, comprehensions, or generator expressions. Use C-backed NumPy operations that process complete arrays or batches instead.
- Do not use `numpy.vectorize`, `numpy.apply_along_axis`, or similar wrappers that merely hide Python iteration.
- Python loops are permitted only in tests and benchmark setup code.
- The project constitution in `.specify/memory/constitution.md` is authoritative and must be followed for every change.
18 changes: 17 additions & 1 deletion benchmarks/benchmarks/bench_alphabet.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

from foapy import alphabet

from .cases import best_case, dna_case, normal_case, worst_case
from .cases import best_case, dna_case, normal_case, records_case, worst_case

length = [5, 50, 500, 5000, 50000, 500000, 5000000, 50000000]
skip = [
Expand Down Expand Up @@ -42,3 +42,19 @@ def time_alphabet(self, length, case):
@skip_params_if(skip, os.getenv("QUICK_BENCHMARK") == "true")
def peakmem_alphabet(self, length, case):
return alphabet(self.data)


class AxisAlphabetSuite:
params = ([5, 50, 500, 5000, 50000], [2, 8], [0, 1])
param_names = ["length", "record_width", "axis"]

data = None

def setup(self, length, record_width, axis):
self.data = records_case(length, record_width, axis)

def time_alphabet(self, length, record_width, axis):
alphabet(self.data, axis=axis)

def peakmem_alphabet(self, length, record_width, axis):
return alphabet(self.data, axis=axis)
Loading
Loading