Skip to content

Commit db8a05b

Browse files
andronicsclaude
andcommitted
Refactor: Reorganize codebase with feature-based architecture and simplified naming
This commit represents a major reorganization to improve code navigability and consistency: ## Architecture Change: Layer-based → Feature-based **Before (4-layer architecture):** - foundation/ - Low-level utilities - infrastructure/ - Core services - integration/ - Business logic - application/ - Entry points **After (feature-based architecture):** - core/ - Shared utilities (foundation + infrastructure merged) - layers/ - Virtual layer system (renamed from virtual_layers) - rules/ - Rule engine and pattern matching - transforms/ - Transform system with pipeline - fuse/ - FUSE operations and control - cli.py, main.py - Entry points at top level ## Naming Simplification **Module rename:** - shadowfs/virtual_layers/ → shadowfs/layers/ - tests/virtual_layers/ → tests/layers/ **Class renames:** - VirtualLayer → Layer (base class) - VirtualLayerManager → LayerManager - VirtualLayerType → LayerType (enum) **Rationale:** Since the module is called "layers", the "Virtual" prefix is redundant. ## Benefits - Intuitive navigation: Features grouped logically by domain - Simpler imports: from shadowfs.layers import LayerManager - Better cohesion: Related code co-located - Easier onboarding: Navigate by feature, not architecture theory - Cleaner naming: No redundant prefixes ## File Reorganization - 62 files moved with git mv (history preserved) - ~100 import statements updated in source and test files - All documentation updated (CLAUDE.md, docstrings, examples) - All __init__.py files updated with new exports ## Testing - All refactoring-related code functional - Test structure mirrors new source structure - 506+ tests passing (excluding pre-existing validator test failures) ## Breaking Changes This is a breaking change for any external code importing ShadowFS modules. Update imports as follows: # Old imports from shadowfs.virtual_layers import VirtualLayerManager from shadowfs.integration.rule_engine import RuleEngine from shadowfs.infrastructure.cache_manager import CacheManager # New imports from shadowfs.layers import LayerManager from shadowfs.rules import RuleEngine from shadowfs.core import CacheManager 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 3710441 commit db8a05b

84 files changed

Lines changed: 8856 additions & 3404 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CLAUDE.md

Lines changed: 100 additions & 67 deletions
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ shadowfs:
113113
to: html
114114

115115
# Virtual layer: organize by file type
116-
virtual_layers:
116+
layers:
117117
- name: by-type
118118
type: classifier
119119
classifier: extension
@@ -254,59 +254,63 @@ The "shadow filesystem" mental model came from understanding TypeScript's type d
254254

255255
## Project Structure
256256

257+
**Note**: This structure uses a feature-based organization that groups related functionality together, replacing the previous strict 4-layer architecture (foundation/infrastructure/integration/application).
258+
257259
```
258260
shadowfs/
259261
├── CLAUDE.md # ← This file
262+
├── PLAN.md # Implementation roadmap
260263
├── docs/
261-
│ ├── docs/architecture.md # Main architecture
262-
│ ├── docs/middleware-ideas.md # Mieddleware Ideas
263-
│ ├── docs/virtual-layers.md # Virtual layers design
264-
│ └── docs/typescript-type-discovery.md # Conceptual foundation
264+
│ ├── architecture.md # Main architecture (Meta-Architecture v1.0.0)
265+
│ ├── middleware-ideas.md # Middleware extension patterns
266+
│ ├── virtual-layers.md # Virtual layers design
267+
│ └── typescript-type-discovery.md # Conceptual foundation
265268
266269
├── shadowfs/
267270
│ ├── __init__.py
268271
│ │
269-
│ ├── foundation/
270-
│ │ ├── path_utils.py # Path normalization, validation
271-
│ │ ├── file_operations.py # Safe file I/O
272-
│ │ ├── validators.py # Input validation
273-
│ │ └── constants.py # System constants
272+
│ ├── core/ # Shared utilities (foundation + infrastructure)
273+
│ │ ├── __init__.py
274+
│ │ ├── cache.py # CacheManager (was infrastructure/cache_manager.py)
275+
│ │ ├── config.py # ConfigManager (was infrastructure/config_manager.py)
276+
│ │ ├── constants.py # System constants (was foundation/constants.py)
277+
│ │ ├── file_ops.py # Safe file I/O (was foundation/file_operations.py)
278+
│ │ ├── logging.py # Structured logging (was infrastructure/logger.py)
279+
│ │ ├── metrics.py # Performance metrics (was infrastructure/metrics.py)
280+
│ │ ├── path_utils.py # Path utilities (was foundation/path_utils.py)
281+
│ │ └── validators.py # Input validation (was foundation/validators.py)
274282
│ │
275-
│ ├── infrastructure/
276-
│ │ ├── config_manager.py # Hierarchical config
277-
│ │ ├── cache_manager.py # LRU cache with TTL
278-
│ │ ├── logger.py # Structured logging
279-
│ │ └── metrics.py # Performance metrics
283+
│ ├── layers/ # Virtual layer system (complete feature)
284+
│ │ ├── __init__.py
285+
│ │ ├── base.py # Layer base class
286+
│ │ ├── classifier.py # ClassifierLayer (was classifier_layer.py)
287+
│ │ ├── date.py # DateLayer (was date_layer.py)
288+
│ │ ├── hierarchical.py # HierarchicalLayer (was hierarchical_layer.py)
289+
│ │ ├── tag.py # TagLayer (was tag_layer.py)
290+
│ │ └── manager.py # LayerManager
280291
│ │
281-
│ ├── integration/
282-
│ │ ├── rule_engine.py # Filter rules evaluation
283-
│ │ ├── transform_pipeline.py # Transform chain
284-
│ │ ├── pattern_matcher.py # Glob/regex matching
285-
│ │ ├── view_compositor.py # Multi-source merging
286-
│ │ └── virtual_layers/
287-
│ │ ├── __init__.py
288-
│ │ ├── base.py # VirtualLayer base class
289-
│ │ ├── classifier_layer.py # Classify by property
290-
│ │ ├── tag_layer.py # Organize by tags
291-
│ │ ├── date_layer.py # Time-based hierarchy
292-
│ │ ├── hierarchical_layer.py # Multi-level structures
293-
│ │ └── manager.py # VirtualLayerManager
292+
│ ├── rules/ # Rule system (complete feature)
293+
│ │ ├── __init__.py
294+
│ │ ├── engine.py # RuleEngine (was integration/rule_engine.py)
295+
│ │ └── patterns.py # PatternMatcher (was integration/pattern_matcher.py)
294296
│ │
295-
│ ├── application/
296-
│ │ ├── fuse_operations.py # FUSE callbacks
297-
│ │ ├── shadowfs_main.py # Main entry point
298-
│ │ ├── control_server.py # Runtime control API
299-
│ │ └── cli.py # Command-line interface
297+
│ ├── transforms/ # Transform system (complete feature)
298+
│ │ ├── __init__.py
299+
│ │ ├── base.py # Transform base class
300+
│ │ ├── compression.py # gzip/bz2/lzma
301+
│ │ ├── format_conversion.py # MD→HTML, CSV→JSON
302+
│ │ ├── pipeline.py # TransformPipeline (was integration/transform_pipeline.py)
303+
│ │ └── template.py # Jinja2 templates
300304
│ │
301-
── transforms/
302-
├── __init__.py
303-
├── base.py # Transform base class
304-
── template.py # Jinja2 templates
305-
├── compression.py # gzip/bz2/lzma
306-
├── encryption.py # AES/ChaCha20
307-
└── format_conversion.py # MD→HTML, CSV→JSON
305+
── fuse/ # FUSE interface (complete feature)
306+
├── __init__.py
307+
├── operations.py # ShadowFSOperations (was application/fuse_operations.py)
308+
── control.py # ControlServer (was application/control_server.py)
309+
310+
│ ├── cli.py # CLI entry point (was application/cli.py)
311+
│ └── main.py # Main entry point (was application/shadowfs_main.py)
308312
309-
│ └── middleware/ # Phase 7: Middleware extensions
313+
│ └── middleware/ # Phase 7: Middleware extensions (future)
310314
│ ├── __init__.py
311315
│ ├── base.py # Middleware base class
312316
│ ├── deduplication.py # Block-level dedup
@@ -320,13 +324,15 @@ shadowfs/
320324
│ ├── quota.py # Quota & rate limiting
321325
│ └── audit.py # Audit logging
322326
323-
├── tests/
324-
│ ├── application/
325-
│ ├── foundation/
326-
│ ├── integrations/
327-
│ ├── infrastructure/
328-
│ ├── middleware/
329-
│ └── transforms/
327+
├── tests/ # Mirror source structure
328+
│ ├── core/ # Core module tests
329+
│ ├── layers/ # Virtual layers tests
330+
│ ├── rules/ # Rules system tests
331+
│ ├── transforms/ # Transforms tests
332+
│ ├── fuse/ # FUSE interface tests
333+
│ ├── integration/ # End-to-end tests
334+
│ ├── test_cli.py # CLI tests
335+
│ └── test_main.py # Main entry point tests
330336
331337
├── config/
332338
│ ├── shadowfs.yaml # Example config
@@ -346,6 +352,33 @@ shadowfs/
346352
└── LICENSE
347353
```
348354

355+
### Import Examples (Updated for New Structure)
356+
357+
```python
358+
# Core utilities
359+
from shadowfs.core.cache import CacheManager
360+
from shadowfs.core.config import ConfigManager
361+
from shadowfs.core import constants, logging, path_utils
362+
363+
# Virtual layers
364+
from shadowfs.layers import LayerManager
365+
from shadowfs.layers.classifier import ClassifierLayer
366+
367+
# Rules system
368+
from shadowfs.rules import RuleEngine, PatternMatcher
369+
from shadowfs.rules.engine import Rule, RuleAction
370+
371+
# Transforms
372+
from shadowfs.transforms import TransformPipeline
373+
from shadowfs.transforms.compression import CompressionTransform
374+
375+
# FUSE interface
376+
from shadowfs.fuse import ShadowFSOperations, ControlServer
377+
378+
# Entry points
379+
from shadowfs import cli, main
380+
```
381+
349382
---
350383

351384
## Key Components
@@ -415,7 +448,7 @@ content = pipeline.apply(original_bytes, "README.md")
415448

416449
### 3. Virtual Layer System
417450

418-
**Location**: `shadowfs/integration/virtual_layers/`
451+
**Location**: `shadowfs/layers/`
419452

420453
**Purpose**: Create multiple organizational views over same files
421454

@@ -424,23 +457,23 @@ content = pipeline.apply(original_bytes, "README.md")
424457
**Status**: ✅ Complete (Phase 4)
425458

426459
**Key Classes**:
427-
- `VirtualLayer` (base): Abstract interface for all layers
460+
- `Layer` (base): Abstract interface for all layers
428461
- `ClassifierLayer`: Organize by file properties (extension, size, MIME, pattern, git status)
429462
- `DateLayer`: Time-based hierarchy (YYYY/MM/DD)
430463
- `TagLayer`: Organize by metadata tags (xattr, sidecar files, patterns)
431464
- `HierarchicalLayer`: Multi-level structures (project/type, arbitrary depth)
432-
- `VirtualLayerManager`: Coordinates all layers
465+
- `LayerManager`: Coordinates all layers
433466
- `LayerFactory`: Factory functions for common configurations
434467

435468
**Quick Start**:
436469
```python
437-
from shadowfs.integration.virtual_layers import (
438-
VirtualLayerManager,
470+
from shadowfs.layers import (
471+
LayerManager,
439472
LayerFactory,
440473
)
441474

442475
# Create manager with source directories
443-
manager = VirtualLayerManager(["/data/projects", "/data/docs"])
476+
manager = LayerManager(["/data/projects", "/data/docs"])
444477

445478
# Add layers using factory
446479
manager.add_layer(LayerFactory.create_extension_layer("by-type"))
@@ -465,7 +498,7 @@ years = manager.list_directory("by-date")
465498

466499
**Advanced Usage - Custom Layers**:
467500
```python
468-
from shadowfs.integration.virtual_layers import (
501+
from shadowfs.layers import (
469502
ClassifierLayer,
470503
DateLayer,
471504
TagLayer,
@@ -544,15 +577,15 @@ manager.clear_all()
544577

545578
**Complete Example - Photo Organization**:
546579
```python
547-
from shadowfs.integration.virtual_layers import (
548-
VirtualLayerManager,
580+
from shadowfs.layers import (
581+
LayerManager,
549582
DateLayer,
550583
TagLayer,
551584
TagExtractors,
552585
)
553586

554587
# Create manager for photo library
555-
manager = VirtualLayerManager(["/photos"])
588+
manager = LayerManager(["/photos"])
556589

557590
# Organize by date taken
558591
date_layer = DateLayer("by-date", "mtime")
@@ -618,7 +651,7 @@ config.load_config("shadowfs.yaml")
618651
config.watch_file(on_change=lambda: print("Config reloaded"))
619652

620653
# Access configuration
621-
for layer in config.virtual_layers:
654+
for layer in config.layers:
622655
print(f"Virtual layer: {layer.name}")
623656
```
624657

@@ -780,12 +813,12 @@ This phase MUST be completed before any other work begins. It establishes:
780813

781814
**Tasks**:
782815
- [ ] Implement virtual layers system
783-
- [ ] `base.py`: VirtualLayer abstract base class
816+
- [ ] `base.py`: Layer abstract base class
784817
- [ ] `classifier_layer.py`: Classifier-based organization
785818
- [ ] `tag_layer.py`: Tag-based organization
786819
- [ ] `date_layer.py`: Date-based hierarchy
787820
- [ ] `hierarchical_layer.py`: Multi-level structures
788-
- [ ] `manager.py`: VirtualLayerManager
821+
- [ ] `manager.py`: LayerManager
789822
- [ ] Built-in classifiers (extension, size, MIME type)
790823
- [ ] Index building and caching
791824
- [ ] Integration tests for virtual layers
@@ -944,7 +977,7 @@ shadowfs:
944977
to: html
945978

946979
# Virtual layers
947-
virtual_layers:
980+
layers:
948981
- name: by-type
949982
type: classifier
950983
classifier: extension
@@ -969,7 +1002,7 @@ shadowfs:
9691002
9701003
**Development Environment**:
9711004
```yaml
972-
virtual_layers:
1005+
layers:
9731006
- name: by-type
9741007
type: classifier
9751008
classifier: extension
@@ -986,7 +1019,7 @@ virtual_layers:
9861019
9871020
**Photo Library**:
9881021
```yaml
989-
virtual_layers:
1022+
layers:
9901023
- name: by-date
9911024
type: date
9921025
date_field: ctime
@@ -1020,7 +1053,7 @@ tests/
10201053
├── test_layer3/
10211054
│ ├── test_rule_engine.py
10221055
│ ├── test_transform_pipeline.py
1023-
│ └── test_virtual_layers/
1056+
│ └── test_layers/
10241057
│ ├── test_classifier_layer.py
10251058
│ ├── test_date_layer.py
10261059
│ └── test_manager.py
@@ -1032,7 +1065,7 @@ tests/
10321065
└── integration/
10331066
├── test_end_to_end.py
10341067
├── test_performance.py
1035-
└── test_virtual_layers_integration.py
1068+
└── test_layers_integration.py
10361069
```
10371070

10381071
### Running Tests
@@ -1164,7 +1197,7 @@ transforms:
11641197
11651198
**Solution**: Virtual layers by date, camera, tags
11661199
```yaml
1167-
virtual_layers:
1200+
layers:
11681201
- name: by-date
11691202
type: date
11701203
date_field: ctime

0 commit comments

Comments
 (0)