Skip to content

Commit a18b594

Browse files
NucsNucs
committed
test: remove [OpenBugs] from 13 passing tests
Remove [OpenBugs] attribute from tests that now pass: - Add_0DScalars (BinaryOpTests.cs) - Bug32_Choice_NDArray_ReplaceFalse (NumpyAlignmentBugTests.cs) - Broadcast_SliceWithLargeIndices_Limited (LongIndexingBroadcastTest.cs) - Broadcast_Add_AllocatesFullOutput (LongIndexingBroadcastTest.cs) - Broadcast_Square_AllocatesFullOutput (LongIndexingBroadcastTest.cs) - BoolTwo2D_NDArrayOR (NDArray.OR.Test.cs) - Case1, Case2 (np.moveaxis.Test.cs) - NonUniformSample (np.random.choice.Test.cs) - UniformOneSample, NonUniformSample (np.random.seed.Test.cs) OpenBugs.Random.cs: Convert class-level [OpenBugs] to method-level for only the 3 still-failing tests: - Rand_Seed42_ShouldMatchNumPy - Randn_Seed42_ShouldMatchNumPy - Permutation_Seed42_ShouldMatchNumPy This allows 10+ previously-excluded tests to run in CI.
1 parent 3cb8eca commit a18b594

12 files changed

Lines changed: 3494 additions & 13 deletions

File tree

‎docs/TUNIT_FILTER_GUIDE.md‎

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
# TUnit `--treenode-filter` Guide
2+
3+
## Filter Path Structure
4+
5+
```
6+
/<Assembly>/<Namespace>/<ClassName>/<TestMethodName>[Property]
7+
```
8+
9+
Use `*` as wildcard. Properties filter by `[Category(...)]` attributes.
10+
11+
## 5 Concrete Examples
12+
13+
### 1. Exclude OpenBugs (CI-style run)
14+
15+
```bash
16+
dotnet test --no-build -- --treenode-filter "/*/*/*/*[Category!=OpenBugs]"
17+
```
18+
19+
Runs all tests EXCEPT those marked `[OpenBugs]`. This is what CI uses.
20+
21+
### 2. Run ONLY OpenBugs (verify bug fixes)
22+
23+
```bash
24+
dotnet test --no-build -- --treenode-filter "/*/*/*/*[Category=OpenBugs]"
25+
```
26+
27+
Runs only failing bug reproductions to check if your fix works.
28+
29+
### 3. Run single test class
30+
31+
```bash
32+
dotnet test --no-build -- --treenode-filter "/*/*/CountNonzeroTests/*"
33+
```
34+
35+
Runs all tests in `CountNonzeroTests` class regardless of namespace.
36+
37+
### 4. Run single test method
38+
39+
```bash
40+
dotnet test --no-build -- --treenode-filter "/*/*/*/Add_TwoNumbers_ReturnsSum"
41+
```
42+
43+
Runs only the test named `Add_TwoNumbers_ReturnsSum`.
44+
45+
### 5. Run tests by namespace pattern
46+
47+
```bash
48+
dotnet test --no-build -- --treenode-filter "/*/NumSharp.UnitTest.Backends.Kernels/*/*"
49+
```
50+
51+
Runs all tests in the `NumSharp.UnitTest.Backends.Kernels` namespace.
52+
53+
## Quick Reference
54+
55+
| Goal | Filter |
56+
|------|--------|
57+
| Exclude category | `/*/*/*/*[Category!=OpenBugs]` |
58+
| Include category only | `/*/*/*/*[Category=OpenBugs]` |
59+
| Single class | `/*/*/ClassName/*` |
60+
| Single method | `/*/*/*/MethodName` |
61+
| Namespace | `/*/Full.Namespace.Path/*/*` |
62+
| Multiple categories (AND) | `/*/*/*/*[(Category!=OpenBugs)&(Category!=WindowsOnly)]` |
63+
| Multiple categories (OR) | `/*/*/*/*[(Category=OpenBugs)\|(Category=Misaligned)]` |
64+
| Multiple classes | Use wildcards: `/*/*/*Comprehensive*/*` |
65+
66+
## Operators
67+
68+
| Op | Meaning | Where | Example |
69+
|----|---------|-------|---------|
70+
| `*` | Wildcard | Path & Properties | `*Tests*`, `[Category=Open*]` |
71+
| `=` | Equals | Properties | `[Category=Unit]` |
72+
| `!=` | Not equals | Properties | `[Category!=Slow]` |
73+
| `&` | AND | Properties | `[(Category=A)&(Priority=High)]` |
74+
| `\|` | OR | **Properties ONLY** | `[(Category=A)\|(Category=B)]` |
75+
76+
**Important:**
77+
- OR (`|`) only works for property filters, NOT for path segments
78+
- AND (`&`) requires outer brackets: `[(A)&(B)]` not `[A]&[B]`
79+
80+
## NumSharp Categories
81+
82+
| Category | Purpose | CI Behavior |
83+
|----------|---------|-------------|
84+
| `OpenBugs` | Known-failing bug reproductions | **Excluded** |
85+
| `Misaligned` | NumSharp vs NumPy differences (tests pass) | Runs |
86+
| `WindowsOnly` | Requires GDI+/System.Drawing | Excluded on Linux/macOS |
87+
88+
## Useful Flags
89+
90+
```bash
91+
# Stop on first failure
92+
dotnet test --no-build -- --fail-fast --treenode-filter "..."
93+
94+
# Detailed output (see passed tests too)
95+
dotnet test --no-build -- --output Detailed --treenode-filter "..."
96+
97+
# List tests without running
98+
dotnet test --no-build -- --list-tests
99+
100+
# Combine: detailed + fail-fast + filter (exclude OpenBugs and WindowsOnly)
101+
dotnet test --no-build -- --output Detailed --fail-fast \
102+
--treenode-filter "/*/*/*/*[(Category!=OpenBugs)&(Category!=WindowsOnly)]"
103+
```
104+
105+
## Advanced Filter Examples (Tested & Working)
106+
107+
### Example A: All Path Parameters Specified
108+
109+
```bash
110+
dotnet test --no-build -- --treenode-filter "/*/NumSharp.UnitTest.Backends.Kernels/VarStdComprehensiveTests/Var_2D_Axis0"
111+
```
112+
113+
**Result:** 1 test ✅
114+
115+
| Part | Value |
116+
|------|-------|
117+
| Assembly | `*` (wildcard - assembly name varies) |
118+
| Namespace | `NumSharp.UnitTest.Backends.Kernels` |
119+
| Class | `VarStdComprehensiveTests` |
120+
| Method | `Var_2D_Axis0` |
121+
122+
### Example B: All Path Parameters with Wildcards
123+
124+
```bash
125+
dotnet test --no-build -- --treenode-filter "/*/*.Backends.*/*Comprehensive*/*_2D_*"
126+
```
127+
128+
**Result:** 29 tests ✅
129+
130+
| Part | Pattern | Matches |
131+
|------|---------|---------|
132+
| Assembly | `*` | Any |
133+
| Namespace | `*.Backends.*` | Contains `.Backends.` |
134+
| Class | `*Comprehensive*` | Contains `Comprehensive` |
135+
| Method | `*_2D_*` | Contains `_2D_` |
136+
137+
### Example C: Multiple Classes via Wildcard Pattern
138+
139+
```bash
140+
dotnet test --no-build -- --treenode-filter "/*/*/VarStd*/*"
141+
```
142+
143+
**Result:** 77 tests ✅
144+
145+
Matches `VarStdComprehensiveTests` and any other class starting with `VarStd`.
146+
147+
**Note:** OR (`|`) does NOT work for path segments. Use wildcards instead.
148+
149+
### Example D: OR for Multiple Categories (Properties)
150+
151+
```bash
152+
dotnet test --no-build -- --treenode-filter "/*/*/*/*[(Category=OpenBugs)|(Category=Misaligned)]"
153+
```
154+
155+
**Result:** 277 tests ✅
156+
157+
Runs tests that have EITHER `[OpenBugs]` OR `[Misaligned]` attribute.
158+
159+
### Example E: Combined - Wildcards + Property Filters (AND)
160+
161+
```bash
162+
dotnet test --no-build -- --treenode-filter "/*/*.Backends*/*/*[(Category!=OpenBugs)&(Category!=WindowsOnly)]"
163+
```
164+
165+
**Result:** 1877 tests ✅
166+
167+
| Part | Pattern | Effect |
168+
|------|---------|--------|
169+
| Assembly | `*` | Any |
170+
| Namespace | `*.Backends*` | Backend tests only |
171+
| Class | `*` | All classes |
172+
| Method | `*` | All methods |
173+
| Properties | `[(Category!=OpenBugs)&(Category!=WindowsOnly)]` | Exclude both categories |
174+
175+
**Note:** Wrap AND conditions in outer brackets: `[(A)&(B)]`
176+
177+
## What Works vs What Doesn't
178+
179+
| Pattern Type | Works? | Example |
180+
|--------------|--------|---------|
181+
| Path wildcards | ✅ | `/*/*/Var*/*` |
182+
| Property equals | ✅ | `[Category=OpenBugs]` |
183+
| Property not equals | ✅ | `[Category!=OpenBugs]` |
184+
| Property AND (with brackets) | ✅ | `[(A=1)&(B=2)]` |
185+
| Property OR | ✅ | `[(A=1)\|(B=2)]` |
186+
| Property AND without brackets | ❌ | `[A=1]&[B=2]` → 0 matches |
187+
| Path OR `(A)\|(B)` | ❌ | Does not match |
188+
| `~=` contains | ❌ | Not supported |

0 commit comments

Comments
 (0)