CLI Documentation

Installation

Prerequisites

Install uv first: brew install uv

Install the Diffuse CLI in two simple steps:

Step 1: Clone the repository

git clone https://github.com/diff-use/webapp.git
cd webapp/cli

Step 2: Install with uv

uv tool install .

Verify installation:

diffuse --help

Authentication

Authenticate with GitHub to access private experiments and perform write operations:

Command	Description
`diffuse auth login`	Login via GitHub device flow
`diffuse auth status`	Check authentication and API health
`diffuse auth logout`	Clear stored credentials

Example: Login with GitHub

$ diffuse auth login
============================================================
Visit: https://github.com/login/device
Enter code: ABCD-1234
============================================================
Waiting for authorization (expires in 900s)...
✓ Authentication successful!

# Get your token from ~/.diffuse/config.json
TOKEN=$(cat ~/.diffuse/config.json | jq -r '.github_token')

# Check authentication status
curl -H "Authorization: Bearer $TOKEN" \
  https://dev.diffuse.science/api/whoami

Experiments

Create, view, edit, and manage experiments from the command line:

Command	Description
`diffuse list`	List all experiments
`diffuse view <id>`	View experiment details
`diffuse create`	Create a new experiment
`diffuse edit <id>`	Edit an experiment
`diffuse delete <id>`	Delete an experiment
`diffuse publish <id>`	Publish to public catalog
`diffuse unpublish <id>`	Unpublish from catalog

CLI:

# List all experiments
diffuse list

# Filter by public/private
diffuse list --public
diffuse list --private

# Sort experiments
diffuse list --sort recent
diffuse list --sort title

# Filter by metadata
diffuse list --filter dataset:mnist --filter epoch:100

# Output as JSON
diffuse list --format json

curl https://dev.diffuse.science/api/experiments

CLI:

# Interactive prompt
diffuse create

# With flags
diffuse create \
  --title "ResNet Training Run" \
  --summary "Training ResNet-50 on ImageNet" \
  --tags "vision,resnet,imagenet"

# With markdown file
diffuse create \
  --title "Experiment Log" \
  --markdown ./experiment.md

curl -X POST https://dev.diffuse.science/api/experiments \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "ResNet Training Run",
    "summary": "Training ResNet-50 on ImageNet",
    "tags": ["vision", "resnet", "imagenet"]
  }'

CLI:

# View experiment details
diffuse view EXP-123

# Output as JSON
diffuse view EXP-123 --format json

curl https://dev.diffuse.science/api/experiments/EXP-123

CLI:

# Update title or summary
diffuse edit EXP-123 --title "Updated Title"
diffuse edit EXP-123 --summary "New summary"

# Update tags
diffuse edit EXP-123 --tags "new,tags,here"

# Open markdown in $EDITOR
diffuse edit EXP-123 --editor

# Update from markdown file
diffuse edit EXP-123 --markdown ./updated.md

curl -X PATCH https://dev.diffuse.science/api/experiments/EXP-123 \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Updated Title",
    "summary": "New summary"
  }'

CLI:

# Publish experiment
diffuse publish EXP-123

# Unpublish experiment
diffuse unpublish EXP-123

# Publish
curl -X POST https://dev.diffuse.science/api/experiments/EXP-123/publish \
  -H "Authorization: Bearer $TOKEN"

# Unpublish
curl -X POST https://dev.diffuse.science/api/experiments/EXP-123/unpublish \
  -H "Authorization: Bearer $TOKEN"

File Uploads & Downloads

Upload artifacts and download experiment data:

Command	Description
`diffuse upload <id> <file>`	Upload a file to experiment
`diffuse download <id> <name>`	Download an artifact

Upload Example:

# Upload a file
diffuse upload EXP-123 ./model.pth

# Upload with metadata
diffuse upload EXP-123 ./model.pth \
  --metadata ./metadata.json \
  --content-type "application/octet-stream"

# 1. Create upload session
curl -X POST https://dev.diffuse.science/api/uploads/session \
  -H "Content-Type: application/json" \
  -d '{
    "experiment_id": "EXP-123",
    "filename": "model.pth",
    "content_type": "application/octet-stream",
    "size_bytes": 102400,
    "metadata_document": {}
  }'

# 2. Upload chunks to presigned URLs (from response above)
# Upload each part to its presigned URL
curl -X PUT "$PRESIGNED_URL_PART_1" \
  --data-binary @model.pth.part1 \
  -H "Content-Type: application/octet-stream"

# Capture ETag from response headers for finalization
# ETag: "abc123..."

# 3. Finalize upload
curl -X POST https://dev.diffuse.science/api/uploads/sess_xyz/finalize \
  -H "Content-Type: application/json" \
  -d '{
    "session_id": "sess_xyz",
    "parts": [
      {"part_number": 1, "etag": "abc123"},
      {"part_number": 2, "etag": "def456"}
    ],
    "metadata_document": {}
  }'

Download Example:

# Download artifact
diffuse download EXP-123 model.pth

# Save to specific location
diffuse download EXP-123 model.pth --output ./downloads/model.pth

# Get presigned download URL
curl https://dev.diffuse.science/api/experiments/EXP-123/artifacts/model.pth/download \
  -H "Authorization: Bearer $TOKEN"

# Download file using the presigned URL from response
curl -o model.pth "$PRESIGNED_DOWNLOAD_URL"

Metadata Management

View and manage experiment metadata fields:

Command	Description
`diffuse metadata get <id>`	Get experiment metadata
`diffuse metadata set <id> <key> <value>`	Set a metadata field
`diffuse metadata apply <id> -f <file>`	Apply metadata from file
`diffuse metadata fields`	List available field definitions

Example: Set metadata field

# Set individual field
diffuse metadata set EXP-123 learning_rate 0.001

# Get all metadata
diffuse metadata get EXP-123

# Apply from JSON file
echo '{"learning_rate": 0.001, "batch_size": 32}' > metadata.json
diffuse metadata apply EXP-123 -f metadata.json

# Set metadata
curl -X PATCH https://dev.diffuse.science/api/experiments/EXP-123/metadata \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "metadata": {
      "learning_rate": 0.001,
      "batch_size": 32
    }
  }'

# Get metadata
curl https://dev.diffuse.science/api/experiments/EXP-123/metadata

Activity & Audit Log

View recent activity:

# View last 20 activities
diffuse activity

# Limit results
diffuse activity --limit 50

# Output as JSON
diffuse activity --format json

curl -H "Authorization: Bearer $TOKEN" \
  https://dev.diffuse.science/api/activity

Configuration

Manage CLI configuration stored in ~/.diffuse/config.json:

Command	Description
`diffuse config view`	View current configuration
`diffuse config set <key> <value>`	Set configuration value
`diffuse config reset`	Reset configuration

Example:

# View config
diffuse config view

# Set custom API server
diffuse config set api_url https://diffuse.mycompany.com

# Use custom server with --server flag
diffuse list --server https://diffuse.mycompany.com

# Or set DIFFUSE_API_URL environment variable
export DIFFUSE_API_URL=https://diffuse.mycompany.com
diffuse list

Security Note

Your GitHub token is stored in ~/.diffuse/config.json with restricted permissions (0600). Never commit this file to version control.

Advanced Usage

Scripting & Automation

The CLI is designed for scripting and CI/CD integration:

#!/bin/bash
# Create experiment and upload results

EXP_ID=$(diffuse create \
  --title "Automated Run $(date +%Y%m%d)" \
  --tags "automation,ci" \
  --format json | jq -r '.id')

echo "Created experiment: $EXP_ID"

# Upload training artifacts
diffuse upload $EXP_ID ./model.pth
diffuse upload $EXP_ID ./metrics.json

# Set metadata
diffuse metadata set $EXP_ID commit_sha $GITHUB_SHA
diffuse metadata set $EXP_ID build_number $BUILD_NUMBER

# Publish if successful
if [ $? -eq 0 ]; then
  diffuse publish $EXP_ID
fi

GitHub Actions Integration

name: Train and Track
on: [push]
jobs:
  train:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install uv
        run: curl -LsSf https://astral.sh/uv/install.sh | sh
      - name: Install Diffuse CLI
        run: |
          git clone https://github.com/diff-use/webapp.git /tmp/diffuse
          cd /tmp/diffuse/cli
          uv tool install .
      - name: Authenticate
        run: |
          mkdir -p ~/.diffuse
          echo '{"github_token": "${{ secrets.DIFFUSE_TOKEN }}"}' > ~/.diffuse/config.json
      - name: Create experiment
        run: |
          EXP_ID=$(diffuse create --title "Run ${{ github.run_number }}" --format json | jq -r '.id')
          echo "EXP_ID=$EXP_ID" >> $GITHUB_ENV
      - name: Train model
        run: python train.py
      - name: Upload results
        run: diffuse upload ${{ env.EXP_ID }} ./model.pth