Skip to content

Commit 5c69d6e

Browse files
committed
feat: add ww export subcommand for IPFS file/directory publishing
- Implement export command equivalent to 'ipfs add -r <path>' - Add recursive directory and file export to IPFS - Print IPFS path to stdout followed by newline on success - Support configurable IPFS endpoint via --ipfs flag or WW_IPFS env var - Add comprehensive unit tests with proper temp directory handling - Include detailed documentation and usage examples - Integrate with existing ww command infrastructure The export command enables users to publish local content to IPFS for use with other Wetware commands like 'ww run'.
1 parent f839a98 commit 5c69d6e

File tree

4 files changed

+688
-0
lines changed

4 files changed

+688
-0
lines changed

cmd/ww/export/README.md

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
# Wetware Export Command
2+
3+
The `ww export` command adds files and directories to IPFS recursively, equivalent to `ipfs add -r <path>`.
4+
5+
## Overview
6+
7+
The export command provides a convenient way to publish local files and directories to IPFS, making them available via IPFS paths that can be shared, referenced, or used by other Wetware commands.
8+
9+
## Usage
10+
11+
```bash
12+
ww export <path>
13+
```
14+
15+
## Arguments
16+
17+
- **`<path>`**: The local filesystem path to export. Can be:
18+
- A file path (e.g., `./document.txt`)
19+
- A directory path (e.g., `./my-project/`)
20+
- Current directory (`.`)
21+
- Absolute paths (e.g., `/home/user/documents`)
22+
23+
## Options
24+
25+
- **`--ipfs`**: IPFS API endpoint (default: `/dns4/localhost/tcp/5001/http`)
26+
- Can also be set via `WW_IPFS` environment variable
27+
- Supports multiaddr, URL, or local path formats
28+
29+
## Examples
30+
31+
### Export a single file
32+
```bash
33+
# Export a text file
34+
ww export document.txt
35+
# Output: /ipfs/QmHash...
36+
37+
# Export with custom IPFS endpoint
38+
ww export --ipfs=/ip4/127.0.0.1/tcp/5001/http document.txt
39+
```
40+
41+
### Export a directory
42+
```bash
43+
# Export current directory
44+
ww export .
45+
46+
# Export a specific directory
47+
ww export /path/to/my-project
48+
49+
# Export with environment variable
50+
WW_IPFS=/dns4/localhost/tcp/5001/http ww export ./data/
51+
```
52+
53+
### Export for sharing
54+
```bash
55+
# Export a project directory
56+
ww export ./my-app/
57+
58+
# The resulting IPFS path can be shared with others
59+
# They can then use: ww run /ipfs/QmHash.../my-app
60+
```
61+
62+
## How It Works
63+
64+
1. **Path Resolution**: The command resolves relative paths to absolute paths
65+
2. **File Type Detection**: Determines if the path is a file or directory
66+
3. **Recursive Processing**: For directories, recursively processes all subdirectories and files
67+
4. **IPFS Addition**: Uses the IPFS Unixfs API to add content to the network
68+
5. **Output**: Prints the resulting IPFS path to stdout followed by a newline
69+
70+
## Integration with Other Commands
71+
72+
The exported IPFS paths can be used with other Wetware commands:
73+
74+
```bash
75+
# Export a binary
76+
ww export ./my-program
77+
78+
# Run the exported binary
79+
ww run /ipfs/QmHash.../my-program --help
80+
81+
# Export a project directory
82+
ww export ./my-project/
83+
84+
# Run from the exported directory
85+
ww run /ipfs/QmHash.../my-project/
86+
```
87+
88+
## Error Handling
89+
90+
The command provides clear error messages for common issues:
91+
92+
- **Missing argument**: "export requires exactly one argument: <path>"
93+
- **Path not found**: "path does not exist: /path/to/file"
94+
- **IPFS connection issues**: "failed to add to IPFS: [specific error]"
95+
96+
## Requirements
97+
98+
- IPFS daemon running and accessible
99+
- Proper IPFS API endpoint configuration
100+
- Read access to the specified path
101+
102+
## Environment Variables
103+
104+
- **`WW_IPFS`**: IPFS API endpoint (overrides `--ipfs` flag)
105+
106+
## Security Considerations
107+
108+
- The command only reads files, never modifies them
109+
- Exported content is publicly available on IPFS (unless using private networks)
110+
- Consider what content you're making publicly accessible
111+
- Use appropriate IPFS network configuration for sensitive content
112+
113+
## Troubleshooting
114+
115+
### Connection Issues
116+
```bash
117+
# Check if IPFS daemon is running
118+
ipfs id
119+
120+
# Test IPFS API endpoint
121+
curl http://localhost:5001/api/v0/version
122+
```
123+
124+
### Permission Issues
125+
```bash
126+
# Ensure read access to the path
127+
ls -la /path/to/export
128+
129+
# Check IPFS daemon permissions
130+
ipfs config show | grep "API"
131+
```
132+
133+
### Large File Handling
134+
- Large files are automatically chunked by IPFS
135+
- Progress is not displayed (use `ipfs add` directly for progress)
136+
- Memory usage scales with file size

cmd/ww/export/export.go

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
package export
2+
3+
import (
4+
"context"
5+
"fmt"
6+
"os"
7+
"path/filepath"
8+
9+
"github.com/ipfs/boxo/files"
10+
iface "github.com/ipfs/kubo/core/coreiface"
11+
"github.com/urfave/cli/v2"
12+
"github.com/wetware/go/cmd/internal/flags"
13+
"github.com/wetware/go/util"
14+
)
15+
16+
var env Env
17+
18+
func Command() *cli.Command {
19+
return &cli.Command{
20+
Name: "export",
21+
ArgsUsage: "<path>",
22+
Usage: "Add a file or directory to IPFS recursively",
23+
Description: `Add a file or directory to IPFS recursively, equivalent to 'ipfs add -r <path>'.
24+
25+
The command will:
26+
1. Read the specified path from the local filesystem
27+
2. Add it to IPFS recursively (including all subdirectories and files)
28+
3. Print the resulting IPFS path to stdout followed by a newline
29+
30+
Examples:
31+
ww export /path/to/file.txt
32+
ww export /path/to/directory
33+
ww export . # Export current directory`,
34+
Flags: append([]cli.Flag{
35+
&cli.StringFlag{
36+
Name: "ipfs",
37+
EnvVars: []string{"WW_IPFS"},
38+
Value: "/dns4/localhost/tcp/5001/http",
39+
Usage: "IPFS API endpoint",
40+
},
41+
}, flags.CapabilityFlags()...),
42+
43+
Before: func(c *cli.Context) error {
44+
return env.Boot(c.String("ipfs"))
45+
},
46+
After: func(c *cli.Context) error {
47+
return env.Close()
48+
},
49+
50+
Action: Main,
51+
}
52+
}
53+
54+
func Main(c *cli.Context) error {
55+
ctx := c.Context
56+
57+
if c.NArg() != 1 {
58+
return cli.Exit("export requires exactly one argument: <path>", 1)
59+
}
60+
61+
argPath := c.Args().First()
62+
if argPath == "" {
63+
return cli.Exit("path cannot be empty", 1)
64+
}
65+
66+
// Resolve the path to absolute
67+
absPath, err := filepath.Abs(argPath)
68+
if err != nil {
69+
return fmt.Errorf("failed to resolve path %s: %w", argPath, err)
70+
}
71+
72+
// Check if path exists
73+
if _, err := os.Stat(absPath); os.IsNotExist(err) {
74+
return cli.Exit(fmt.Sprintf("path does not exist: %s", absPath), 1)
75+
}
76+
77+
// Add the file/directory to IPFS
78+
ipfsPath, err := env.AddToIPFS(ctx, absPath)
79+
if err != nil {
80+
return fmt.Errorf("failed to add to IPFS: %w", err)
81+
}
82+
83+
// Print the IPFS path to stdout followed by a newline
84+
fmt.Println(ipfsPath)
85+
return nil
86+
}
87+
88+
type Env struct {
89+
IPFS iface.CoreAPI
90+
}
91+
92+
func (env *Env) Boot(addr string) error {
93+
var err error
94+
env.IPFS, err = util.LoadIPFSFromName(addr)
95+
return err
96+
}
97+
98+
func (env *Env) Close() error {
99+
// No cleanup needed for IPFS client
100+
return nil
101+
}
102+
103+
// AddToIPFS adds a file or directory to IPFS recursively
104+
func (env *Env) AddToIPFS(ctx context.Context, localPath string) (string, error) {
105+
// Get file info to determine if it's a directory
106+
fileInfo, err := os.Stat(localPath)
107+
if err != nil {
108+
return "", fmt.Errorf("failed to stat %s: %w", localPath, err)
109+
}
110+
111+
var node files.Node
112+
if fileInfo.IsDir() {
113+
// Handle directory
114+
node, err = env.CreateDirectoryNode(ctx, localPath)
115+
if err != nil {
116+
return "", fmt.Errorf("failed to create directory node: %w", err)
117+
}
118+
} else {
119+
// Handle single file
120+
node, err = env.CreateFileNode(ctx, localPath)
121+
if err != nil {
122+
return "", fmt.Errorf("failed to create file node: %w", err)
123+
}
124+
}
125+
126+
// Check if IPFS client is available
127+
if env.IPFS == nil {
128+
return "", fmt.Errorf("IPFS client not initialized")
129+
}
130+
131+
// Add the node to IPFS using Unixfs API
132+
path, err := env.IPFS.Unixfs().Add(ctx, node)
133+
if err != nil {
134+
return "", fmt.Errorf("failed to add to IPFS: %w", err)
135+
}
136+
137+
return path.String(), nil
138+
}
139+
140+
// CreateFileNode creates a files.Node for a single file
141+
func (env *Env) CreateFileNode(ctx context.Context, filePath string) (files.Node, error) {
142+
// Read the file content into memory
143+
content, err := os.ReadFile(filePath)
144+
if err != nil {
145+
return nil, err
146+
}
147+
148+
// Create a file node from the content
149+
return files.NewBytesFile(content), nil
150+
}
151+
152+
// CreateDirectoryNode creates a files.Node for a directory recursively
153+
func (env *Env) CreateDirectoryNode(ctx context.Context, dirPath string) (files.Node, error) {
154+
entries, err := os.ReadDir(dirPath)
155+
if err != nil {
156+
return nil, err
157+
}
158+
159+
// Create a map to hold directory contents
160+
dirMap := make(map[string]files.Node)
161+
162+
for _, entry := range entries {
163+
entryPath := filepath.Join(dirPath, entry.Name())
164+
165+
if entry.IsDir() {
166+
// Recursively handle subdirectories
167+
childNode, err := env.CreateDirectoryNode(ctx, entryPath)
168+
if err != nil {
169+
return nil, err
170+
}
171+
172+
// Add subdirectory to the map
173+
dirMap[entry.Name()] = childNode
174+
} else {
175+
// Handle files
176+
childNode, err := env.CreateFileNode(ctx, entryPath)
177+
if err != nil {
178+
return nil, err
179+
}
180+
181+
// Add file to the map
182+
dirMap[entry.Name()] = childNode
183+
}
184+
}
185+
186+
// Create directory from the map
187+
return files.NewMapDirectory(dirMap), nil
188+
}

0 commit comments

Comments
 (0)