[Feature Request]: Implementation Plan for Root Directory

[shoummu1]

This issue outlines a detailed plan to implement the missing features and fix the broken UI for the Root Directory management in the MCP Gateway admin panel. The goal is to enable full CRUD operations, file browsing, exporting configurations, and integrating file-based resources and tools.

Implementation Tasks

The implementation is broken down into the following tasks:

Task 1: Database Schema & Persistence

Problem: Roots are only stored in memory and lost on restart.

Solution: Create database schema in mcpgateway/db.py:

class Root(Base):
    __tablename__ = "roots"
    
    id = Column(Integer, primary_key=True, index=True)
    uri = Column(String, unique=True, nullable=False, index=True)
    name = Column(String, nullable=True)
    is_active = Column(Boolean, default=True, nullable=False)
    created_at = Column(DateTime, default=datetime.utcnow)
    updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
    last_accessed = Column(DateTime, nullable=True)
    access_count = Column(Integer, default=0)
    
    # Security and validation
    max_depth = Column(Integer, default=None)  # Limit directory traversal
    allowed_extensions = Column(Text, nullable=True)  # JSON list of allowed file types
    read_only = Column(Boolean, default=True)  # Whether writes are allowed
    
    # Metadata
    description = Column(Text, nullable=True)
    tags = Column(Text, nullable=True)  # JSON list of tags

Files to modify:

• mcpgateway/db.py - Add Root model
• mcpgateway/alembic/versions/ - Create migration
• mcpgateway/services/root_service.py - Add database integration

Task 2: Enhanced RootService

Extend RootService with missing methods:

class RootService:
    # Database operations
    async def get_root(self, uri: str) -> Optional[Root]
    async def update_root(self, uri: str, data: RootUpdate) -> Root
    async def toggle_root_status(self, uri: str) -> Root
    async def get_root_stats(self, uri: str) -> RootStats
    
    # File system operations
    async def validate_root_access(self, uri: str) -> bool
    async def get_root_contents(self, uri: str, path: str = "") -> List[FileInfo]
    async def resolve_file_resource(self, uri: str) -> Optional[ResourceContent]
    
    # Security
    async def check_file_permissions(self, root_uri: str, file_path: str) -> bool
    async def sanitize_path(self, root_uri: str, requested_path: str) -> str
    
    # Tool and capability discovery
    async def discover_root_capabilities(self, root: Root) -> RootCapabilities
    async def register_file_tools(self, root: Root, tools: List[FileToolInfo])

Files to modify:

• mcpgateway/services/root_service.py - Add new methods
• mcpgateway/schemas.py - Add RootUpdate, RootStats schemas

Task 3: Resource Integration

Purpose: Files from roots become available as MCP resources.

Integration Points:

# In ResourceService.read_resource()
async def read_resource(self, db: Session, uri: str, **kwargs) -> ResourceContent:
    # Existing database resource logic...
    
    # NEW: Try file system via roots for file:// URIs
    if not resource and uri.startswith('file://'):
        from mcpgateway.main import root_service
        content = await root_service.resolve_file_resource(uri)
        if content is None:
            raise ResourceNotFoundError(f"Resource not found: {uri}")
    
    return content

# In ResourceService.list_resources()
async def list_resources(self, db: Session) -> List[ResourceRead]:
    resources = []
    
    # 1. Database resources (existing)
    db_resources = db.execute(select(DbResource)).scalars().all()
    resources.extend([self._convert_to_read(r) for r in db_resources])
    
    # 2. File system resources from roots (NEW)
    from mcpgateway.main import root_service
    for root in await root_service.list_roots():
        if root.is_active:
            file_resources = await self._discover_file_resources(root)
            resources.extend(file_resources)
    
    return resources

Files to modify:

• mcpgateway/services/resource_service.py - Add file resolution
• mcpgateway/models.py - Add file-related content types

Task 4: Tool Discovery & Integration

Purpose: Executable files in roots become MCP tools.

Tool Discovery Flow:

# When root is added with executable files
async def add_root(self, uri: str, name: str) -> Root:
    # 1. Standard root registration
    root = await self._store_root(uri, name)
    
    # 2. Discover different file types
    capabilities = await self._discover_root_capabilities(root)
    
    # 3. Register capabilities with appropriate services
    if capabilities.tools:
        await self._register_file_tools(root, capabilities.tools)
    if capabilities.resources:
        await self._register_file_resources(root, capabilities.resources)  
    if capabilities.prompts:
        await self._register_file_prompts(root, capabilities.prompts)
    
    return root

class FileToolInfo:
    name: str
    file_path: str
    execution_method: str  # "mcp_native", "script_wrapper", "binary"
    interpreter: Optional[str]  # "python", "node", "bash", etc.
    schema: Dict[str, Any]
    description: str

Files to modify:

• mcpgateway/services/root_service.py - Add tool discovery
• mcpgateway/services/tool_service.py - Add file tool execution
• mcpgateway/db.py - Extend Tool model for FILE integration_type

Task 5: Security & Validation Framework

Critical Security Requirements:

class RootSecurityValidator:
    def validate_path_safety(self, root_uri: str, requested_path: str) -> bool:
        """Prevent directory traversal attacks."""
        
    def check_file_extension(self, file_path: str, allowed_extensions: List[str]) -> bool:
        """Block dangerous file types."""
        
    def validate_file_size(self, file_path: str, max_size_mb: int = 100) -> bool:
        """Prevent serving huge files."""
        
    def check_os_permissions(self, file_path: str, operation: str) -> bool:
        """Check OS-level access rights."""
        
    async def execute_tool_safely(self, file_path: str, args: Dict[str, Any]) -> ToolResult:
        """Execute file tools in sandboxed environment."""

Files to modify:

• mcpgateway/services/root_service.py - Add security validation
• mcpgateway/utils/security.py - Create security utilities

Task 6: Admin UI Backend Endpoints

Add missing REST endpoints in admin.py:

@admin_router.get("/roots/{uri:path}/view")
async def admin_view_root(uri: str, user=Depends(get_current_user_with_permissions)):
    """Get detailed root information including file stats and capabilities."""
    root = await root_service.get_root(uri)
    if not root:
        raise HTTPException(404, "Root not found")
    
    stats = await root_service.get_root_stats(uri)
    capabilities = await root_service.get_root_capabilities(uri)
    
    return {
        "root": root,
        "stats": stats,
        "capabilities": capabilities,
        "file_count": stats.file_count,
        "directory_count": stats.directory_count,
        "total_size": stats.total_size,
        "tools_discovered": len(capabilities.tools),
        "resources_discovered": len(capabilities.resources)
    }

@admin_router.get("/roots/{uri:path}/edit") 
async def admin_get_root_for_edit(uri: str, user=Depends(get_current_user_with_permissions)):
    """Get root data for editing form."""
    
@admin_router.post("/roots/{uri:path}/edit")
async def admin_update_root(uri: str, request: Request, user=Depends(get_current_user_with_permissions)):
    """Update root properties like name, description, security settings."""
    
@admin_router.get("/roots/{uri:path}/export")
async def admin_export_root(uri: str, user=Depends(get_current_user_with_permissions)):
    """Export root configuration as JSON including all metadata and discovered capabilities."""
    
@admin_router.post("/roots/{uri:path}/deactivate")
async def admin_toggle_root_status(uri: str, user=Depends(get_current_user_with_permissions)):
    """Toggle root active/inactive status."""

@admin_router.get("/roots/{uri:path}/browse")
async def admin_browse_root_contents(uri: str, path: str = "", user=Depends(get_current_user_with_permissions)):
    """Browse directory contents within a root."""

Files to modify:

• mcpgateway/admin.py - Add missing endpoints

Task 7: Admin UI JavaScript Functions

Add missing JavaScript functions to admin.html or admin.js:

// View root details in modal
async function viewRoot(uri) {
    try {
        const response = await fetch(`${ROOT_PATH}/admin/roots/${encodeURIComponent(uri)}/view`);
        const rootData = await response.json();
        showRootDetailsModal(rootData);
    } catch (error) {
        showErrorMessage('Failed to load root details: ' + error.message);
    }
}

// Edit root properties
async function editRoot(uri) {
    try {
        const response = await fetch(`${ROOT_PATH}/admin/roots/${encodeURIComponent(uri)}/edit`);
        const rootData = await response.json();
        showEditRootModal(rootData);
    } catch (error) {
        showErrorMessage('Failed to load root for editing: ' + error.message);
    }
}

// Export root configuration
async function exportRoot(uri) {
    try {
        const response = await fetch(`${ROOT_PATH}/admin/roots/${encodeURIComponent(uri)}/export`);
        const blob = await response.blob();
        const filename = `root-${uri.replace(/[^a-zA-Z0-9]/g, '_')}.json`;
        downloadFile(blob, filename);
    } catch (error) {
        showErrorMessage('Failed to export root: ' + error.message);
    }
}

// Toggle active/inactive status
async function deactivateRoot(uri) {
    const action = confirm('Are you sure you want to deactivate this root? This will disable all file access and tools from this root.');
    if (action) {
        try {
            await fetch(`${ROOT_PATH}/admin/roots/${encodeURIComponent(uri)}/deactivate`, {
                method: 'POST'
            });
            location.reload();
        } catch (error) {
            showErrorMessage('Failed to deactivate root: ' + error.message);
        }
    }
}

// Browse root contents
async function browseRoot(uri) {
    try {
        const response = await fetch(`${ROOT_PATH}/admin/roots/${encodeURIComponent(uri)}/browse`);
        const contents = await response.json();
        showRootBrowserModal(uri, contents);
    } catch (error) {
        showErrorMessage('Failed to browse root contents: ' + error.message);
    }
}

Files to modify:

• mcpgateway/templates/admin.html - Add JavaScript functions
• mcpgateway/static/admin.js

Task 8: Enhanced UI Features

Advanced UI Components:

1. Root Details Modal:

   • File system statistics
   • Discovered tools and resources
   • Security settings
   • Access logs

2. Root Editor Modal:

   • Edit name and description
   • Configure security settings (read-only, allowed extensions)
   • Set resource limits

3. File Browser Modal:

   • Navigate directory structure
   • Preview file contents
   • View file metadata
   • Direct resource links

4. Export/Import:

   • JSON configuration export
   • Bulk root import/export
   • Migration between environments

Files to modify:

• mcpgateway/templates/admin.html - Add modal templates
• mcpgateway/static/admin.css - Add styling

Implementation Priority

Possible priorities for phased implementation:

1. 🚨 High Priority (Fix Broken UI)

   • Task 7: JavaScript functions (immediate user experience fix)
   • Task 6: Basic backend endpoints

2. 📊 Medium Priority (Core Functionality)

   • Task 1: Database persistence
   • Task 2: Enhanced RootService
   • Task 3: Resource integration

3. 🔒 Security Priority

   • Task 5: Security framework (critical for production)

4. ⚡ Enhancement Priority

   • Task 4: Tool discovery
   • Task 8: Advanced UI features

What Happens When Root is Added

Complete Flow:

User adds: file:///home/user/workspace
    ↓
1. Validate path exists and accessible
    ↓
2. Store in database with metadata
    ↓
3. Scan directory structure for capabilities:
   ├── tools/data_processor.py → Auto-register as MCP tool
   ├── docs/README.md → Available as resource
   ├── prompts/code_review.txt → Auto-register as prompt
   └── config.json → Available as resource
    ↓
4. Generate resource templates:
   - file:///home/user/workspace/{file_path}
   - docs/{document}.md
    ↓
5. Notify MCP clients of new capabilities
    ↓
6. Resources become available via resources/list
7. Tools become available via tools/list
8. Files accessible via resources/read

Testing Strategy

1. Unit Tests:

   • Root service methods
   • Security validation
   • File resolution

2. Integration Tests:

   • Database persistence
   • Resource service integration
   • Tool discovery and execution

3. E2E Tests:

   • Admin UI workflows
   • MCP client interactions
   • File serving scenarios

4. Security Tests:

   • Directory traversal attempts
   • Permission validation
   • Malicious file handling

---

[shoummu1]

@crivetimihai Could you please review the proposed solution for the root directory and share your feedback?

[shoummu1]

@madhav165 @kevalmahajan Could you please review the proposed solution for the root directory and share your feedback?