Skip to content

Commit b48e874

Browse files
committed
v1: fixes. for tests
1 parent 18ee2d6 commit b48e874

78 files changed

Lines changed: 5300 additions & 889 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

MCP-TEST-CLIENT-SUMMARY.md

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
# MCP Test Client - Complete Implementation
2+
3+
## Overview
4+
5+
I've created a comprehensive CLI-based MCP client to sanity test all the tools provided by your MCP GitHub Project Manager server. This test client provides both automated and interactive testing capabilities.
6+
7+
## What Was Built
8+
9+
### 1. Core Test Client (`scripts/mcp-test-client.js`)
10+
- **Full MCP Protocol Implementation**: Proper JSON-RPC communication with the MCP server
11+
- **Comprehensive Tool Testing**: Tests all 30+ tools available in your server
12+
- **Sample Data Library**: Predefined test data for most tools
13+
- **Interactive Mode**: Manual testing with custom arguments
14+
- **Detailed Reporting**: Success/failure reporting with error details
15+
- **Color-coded Output**: Easy-to-read console output with colors
16+
17+
### 2. Shell Wrapper (`scripts/test-mcp.sh`)
18+
- **Environment Validation**: Checks for required environment variables
19+
- **Build Integration**: Can build the project before testing
20+
- **Auto .env Loading**: Automatically loads environment variables from .env file
21+
- **User-friendly Interface**: Simplified command-line interface
22+
23+
### 3. Demo Script (`scripts/demo-test-client.sh`)
24+
- **Interactive Tutorial**: Step-by-step demonstration of all features
25+
- **Educational Content**: Explains what each command does
26+
- **Safe Testing**: Asks before running potentially expensive operations
27+
28+
### 4. Documentation (`scripts/README-test-client.md`)
29+
- **Complete Usage Guide**: Detailed instructions for all features
30+
- **Troubleshooting Section**: Common issues and solutions
31+
- **Extension Guide**: How to add new tools and sample data
32+
33+
## Available Commands
34+
35+
### Basic Commands
36+
```bash
37+
# Show help
38+
./scripts/test-mcp.sh help
39+
40+
# List all available tools (30+ tools)
41+
./scripts/test-mcp.sh list-tools
42+
43+
# Test a specific tool
44+
./scripts/test-mcp.sh test-tool create_project
45+
46+
# Test all tools with sample data
47+
./scripts/test-mcp.sh test-all
48+
49+
# Interactive mode
50+
./scripts/test-mcp.sh interactive
51+
```
52+
53+
### Advanced Usage
54+
```bash
55+
# Build project first
56+
./scripts/test-mcp.sh build
57+
58+
# Run demo tutorial
59+
./scripts/demo-test-client.sh
60+
61+
# Direct Node.js usage
62+
node scripts/mcp-test-client.js list-tools
63+
```
64+
65+
## Tool Categories Tested
66+
67+
### 1. Project Management Tools (15+ tools)
68+
- **Projects**: create_project, list_projects, get_project, update_project, delete_project
69+
- **Milestones**: create_milestone, list_milestones, update_milestone, delete_milestone
70+
- **Issues**: create_issue, list_issues, get_issue, update_issue
71+
- **Sprints**: create_sprint, list_sprints, get_current_sprint, update_sprint
72+
- **Labels**: create_label, list_labels
73+
74+
### 2. Project Structure Tools (8+ tools)
75+
- **Fields**: create_project_field, list_project_fields, update_project_field
76+
- **Views**: create_project_view, list_project_views, update_project_view
77+
- **Items**: add_project_item, remove_project_item, list_project_items
78+
- **Values**: set_field_value, get_field_value
79+
80+
### 3. Planning & Analytics Tools (6+ tools)
81+
- **Planning**: create_roadmap, plan_sprint
82+
- **Metrics**: get_milestone_metrics, get_sprint_metrics
83+
- **Tracking**: get_overdue_milestones, get_upcoming_milestones
84+
85+
### 4. AI Task Management Tools (8+ tools)
86+
- **PRD Management**: generate_prd, parse_prd, enhance_prd
87+
- **Task Intelligence**: get_next_task, analyze_task_complexity, expand_task
88+
- **Feature Management**: add_feature, create_traceability_matrix
89+
90+
## Sample Data Included
91+
92+
The test client includes realistic sample data for:
93+
94+
### Project Tools
95+
- Creates test projects with proper visibility settings
96+
- Sample milestones with due dates
97+
- Test issues with labels and descriptions
98+
- Sprint planning with date ranges
99+
100+
### AI Tools
101+
- Complete PRD documents for task management apps
102+
- Complex task descriptions for analysis
103+
- Feature specifications with acceptance criteria
104+
- Traceability matrices with business requirements
105+
106+
### Analytics Tools
107+
- Proper query parameters for listing and filtering
108+
- Date ranges for upcoming/overdue milestone queries
109+
- Capacity and skill-based task recommendations
110+
111+
## Key Features
112+
113+
### 1. Robust Error Handling
114+
- Validates environment variables before starting
115+
- Provides clear error messages for common issues
116+
- Graceful handling of API failures and timeouts
117+
118+
### 2. Interactive Testing
119+
- Real-time tool exploration
120+
- Custom argument input with JSON validation
121+
- Schema inspection for understanding tool requirements
122+
123+
### 3. Comprehensive Reporting
124+
- Categorized tool listings
125+
- Pass/fail statistics
126+
- Detailed error reporting for failed tests
127+
- Skipped test tracking for tools without sample data
128+
129+
### 4. Developer-Friendly
130+
- Color-coded output for easy reading
131+
- Detailed help messages and examples
132+
- Extensible architecture for adding new tools
133+
134+
## Usage Examples
135+
136+
### Quick Start
137+
```bash
138+
# 1. Set environment variables
139+
export GITHUB_TOKEN=ghp_xxxxxxxxxxxx
140+
export GITHUB_OWNER=your-username
141+
export GITHUB_REPO=your-repo
142+
143+
# 2. List all tools
144+
./scripts/test-mcp.sh list-tools
145+
146+
# 3. Test a specific tool
147+
./scripts/test-mcp.sh test-tool create_project
148+
```
149+
150+
### Interactive Testing
151+
```bash
152+
./scripts/test-mcp.sh interactive
153+
> help
154+
> list
155+
> test create_project
156+
> call list_projects
157+
> schema create_milestone
158+
> exit
159+
```
160+
161+
### Comprehensive Testing
162+
```bash
163+
# Test all tools (may take several minutes)
164+
./scripts/test-mcp.sh test-all
165+
```
166+
167+
## Files Created
168+
169+
1. **`scripts/mcp-test-client.js`** - Main Node.js test client (630+ lines)
170+
2. **`scripts/test-mcp.sh`** - Shell wrapper script (120+ lines)
171+
3. **`scripts/demo-test-client.sh`** - Interactive demo script (200+ lines)
172+
4. **`scripts/README-test-client.md`** - Complete documentation (250+ lines)
173+
5. **`MCP-TEST-CLIENT-SUMMARY.md`** - This summary document
174+
175+
## Benefits
176+
177+
### For Development
178+
- **Rapid Testing**: Quickly verify all tools work correctly
179+
- **Regression Testing**: Ensure changes don't break existing functionality
180+
- **API Exploration**: Understand tool capabilities and requirements
181+
182+
### For CI/CD
183+
- **Automated Validation**: Can be integrated into build pipelines
184+
- **Quality Assurance**: Catch issues before deployment
185+
- **Documentation**: Living examples of how to use each tool
186+
187+
### For Users
188+
- **Learning Tool**: Understand available functionality
189+
- **Integration Examples**: See how to call tools programmatically
190+
- **Troubleshooting**: Verify setup and configuration
191+
192+
## Next Steps
193+
194+
1. **Run the demo**: `./scripts/demo-test-client.sh`
195+
2. **Test your specific use cases**: Use interactive mode to test relevant tools
196+
3. **Integrate with CI/CD**: Add `test-all` to your build pipeline
197+
4. **Extend sample data**: Add test cases for tools you use frequently
198+
5. **Customize for your needs**: Modify the client for specific testing scenarios
199+
200+
This test client provides a solid foundation for validating your MCP server implementation and can serve as both a development tool and a reference for users integrating with your server.

README.md

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -504,6 +504,85 @@ npm test -- --testPathPattern="enhanced"
504504
npm test
505505
```
506506

507+
## 🧪 Comprehensive E2E Testing Suite
508+
509+
The MCP GitHub Project Manager includes a comprehensive end-to-end testing suite that tests all MCP tools through the actual MCP interface with both mocked and real API calls.
510+
511+
### **Test Coverage:**
512+
-**40+ GitHub Project Management Tools** - Complete CRUD operations for projects, milestones, issues, sprints, labels, and more
513+
-**8 AI Task Management Tools** - PRD generation, task parsing, complexity analysis, feature management, and traceability
514+
-**Complex Workflow Integration** - Multi-tool workflows and real-world project management scenarios
515+
-**Real API Testing** - Optional testing with actual GitHub and AI APIs
516+
-**Schema Validation** - Comprehensive argument validation for all tools
517+
-**Error Handling** - Graceful error handling and recovery testing
518+
519+
### **Quick Start:**
520+
```bash
521+
# Run comprehensive E2E tests (mocked APIs)
522+
npm run test:e2e:tools
523+
524+
# Run with real APIs (requires credentials)
525+
npm run test:e2e:tools:real
526+
527+
# Use the interactive test runner
528+
npm run test:e2e:runner
529+
530+
# Run specific test categories
531+
npm run test:e2e:tools:github # GitHub tools only
532+
npm run test:e2e:tools:ai # AI tools only
533+
npm run test:e2e:tools:workflows # Integration workflows
534+
```
535+
536+
### **Test Runner Options:**
537+
```bash
538+
# Interactive test runner with options
539+
node scripts/run-e2e-tests.js --help
540+
541+
# Examples:
542+
node scripts/run-e2e-tests.js --real-api --github-only
543+
node scripts/run-e2e-tests.js --build --verbose --timeout 120
544+
node scripts/run-e2e-tests.js --ai-only --real-api
545+
```
546+
547+
### **Environment Setup for Real API Testing:**
548+
549+
**GitHub API (Required for GitHub tools):**
550+
```bash
551+
GITHUB_TOKEN=ghp_your_github_token
552+
GITHUB_OWNER=your-github-username
553+
GITHUB_REPO=your-test-repository
554+
```
555+
556+
**AI APIs (Required for AI tools):**
557+
```bash
558+
# At least one AI API key required
559+
ANTHROPIC_API_KEY=sk-ant-your-anthropic-key
560+
OPENAI_API_KEY=sk-your-openai-key
561+
GOOGLE_API_KEY=your-google-ai-key
562+
PERPLEXITY_API_KEY=pplx-your-perplexity-key
563+
```
564+
565+
**Enable Real API Testing:**
566+
```bash
567+
E2E_REAL_API=true npm run test:e2e:tools:real
568+
```
569+
570+
### **Test Features:**
571+
- **Tool Registration Validation** - Verify all tools are properly registered with correct schemas
572+
- **MCP Protocol Compliance** - Ensure all tools follow MCP specification
573+
- **Response Format Validation** - Validate tool responses match expected formats
574+
- **Workflow Integration Testing** - Test complex multi-tool workflows
575+
- **Credential Management** - Graceful handling of missing credentials
576+
- **Performance Monitoring** - Track tool execution performance
577+
- **Comprehensive Error Testing** - Validate error handling and recovery
578+
579+
### **Documentation:**
580+
- 📖 [Comprehensive E2E Testing Guide](docs/e2e-testing-guide.md) - Detailed testing documentation
581+
- 🔧 [Test Configuration](jest.e2e.tools.config.js) - Jest configuration for E2E tests
582+
- 🛠️ [Test Utilities](src/__tests__/e2e/utils/MCPToolTestUtils.ts) - Reusable test utilities
583+
584+
The E2E test suite ensures that all MCP tools work correctly both individually and in complex workflows, providing confidence in the reliability and integration of the entire system.
585+
507586
#### **Test Scenarios Covered:**
508587
- ✅ Default traceability-based context (no AI required)
509588
- ✅ AI-enhanced business context generation

debug-mcp.js

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ async function testMCPCommunication() {
4848
});
4949

5050
serverProcess.on('error', (error) => {
51-
console.error('Process error:', error);
51+
process.stderr.write('Process error:', error);
5252
});
5353

5454
serverProcess.on('exit', (code, signal) => {
@@ -90,4 +90,4 @@ async function testMCPCommunication() {
9090
serverProcess.kill('SIGTERM');
9191
}
9292

93-
testMCPCommunication().catch(console.error);
93+
testMCPCommunication().catch(process.stderr.write);

debug-simple.js

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ async function testServerBasic() {
3232
});
3333

3434
serverProcess.on('error', (error) => {
35-
console.error('Process error:', error);
35+
process.stderr.write('Process error:', error);
3636
});
3737

3838
serverProcess.on('exit', (code, signal) => {
@@ -85,6 +85,6 @@ testServerBasic()
8585
process.exit(result ? 0 : 1);
8686
})
8787
.catch(error => {
88-
console.error('Test error:', error);
88+
process.stderr.write('Test error:', error);
8989
process.exit(1);
9090
});

docs/api-reference/plan-sprint.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ try {
7676
if (issues.length !== issueIds.length) {
7777
const foundIds = issues.map(issue => issue.id);
7878
const missingIds = issueIds.filter(id => !foundIds.includes(id));
79-
console.error(`Issues not found: ${missingIds.join(', ')}`);
79+
process.stderr.write(`Issues not found: ${missingIds.join(', ')}`);
8080
return;
8181
}
8282

@@ -109,7 +109,7 @@ try {
109109

110110
return sprint;
111111
} catch (error) {
112-
console.error("Failed to plan sprint:", error);
112+
process.stderr.write("Failed to plan sprint:", error);
113113
throw error;
114114
}
115115
```

0 commit comments

Comments
 (0)