kqjy/MyFSIO
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 33 Wiki Activity
Files
9ec5797919c1928da1edc3bf18865f7cef7219e5
MyFSIO/docs.md

11 KiB
Raw Blame History

MyFSIO Rust Operations Guide

This document describes the current Rust server in rust/myfsio-engine. It replaces the older Python-oriented runbook.

The python/ implementation is deprecated as of 2026-04-21. It is retained for migration reference and legacy validation only; production usage and new development should use the Rust server.

1. What Changed

The active runtime is now Rust:

  • One process serves both the S3 API and the web UI.
  • The server entrypoint is myfsio-server.
  • The main development workflow is cargo run -p myfsio-server --.
  • API-only mode is controlled with UI_ENABLED=false.

The deprecated python/ directory may still contain older implementation code, templates, and tests, but it is not required to run the current server.

2. Quick Start

From the repository root:

cd rust/myfsio-engine
cargo run -p myfsio-server --

Default endpoints:

  • UI: http://127.0.0.1:5100/ui
  • API: http://127.0.0.1:5000/
  • Health: http://127.0.0.1:5000/myfsio/health

On first startup, MyFSIO bootstraps an admin user in data/.myfsio.sys/config/iam.json and prints the generated access key and secret key to stdout.

Windows

From PowerShell:

cd rust\myfsio-engine
cargo run -p myfsio-server --

API-only mode

UI_ENABLED=false cargo run -p myfsio-server --

There is no separate UI-only mode in the Rust server.

3. Build and Run a Binary

cd rust/myfsio-engine
cargo build --release -p myfsio-server

Run it directly:

./target/release/myfsio-server

On Windows:

.\target\release\myfsio-server.exe

4. CLI Commands

The Rust CLI supports these operational commands:

# Start serving (default command)
cargo run -p myfsio-server --

# Print version
cargo run -p myfsio-server -- version

# Show resolved configuration
cargo run -p myfsio-server -- --show-config

# Validate configuration and exit with code 1 on critical issues
cargo run -p myfsio-server -- --check-config

# Back up the current IAM file and generate fresh admin credentials
cargo run -p myfsio-server -- --reset-cred

If you are running a release build instead of cargo run, replace the cargo run ... -- prefix with the binary path.

5. Environment Files

At startup, the server tries to load environment files from these locations when they exist:

  1. /opt/myfsio/myfsio.env
  2. .env in the current directory
  3. myfsio.env in the current directory
  4. .env and myfsio.env in a few parent directories

That makes local development and systemd installs behave consistently.

6. Verified Configuration Reference

These values are taken from rust/myfsio-engine/crates/myfsio-server/src/config.rs.

Network and runtime

Variable Default Description
HOST 127.0.0.1 Bind address for both listeners
PORT 5000 S3 API port
UI_PORT 5100 Web UI port
UI_ENABLED true Disable to run API-only
API_BASE_URL unset Public-facing API base used by the UI and presigned URL generation
TEMPLATES_DIR built-in templates dir Optional override for UI templates
STATIC_DIR built-in static dir Optional override for static assets

Storage and auth

Variable Default Description
STORAGE_ROOT ./data Root for buckets and internal state
IAM_CONFIG <STORAGE_ROOT>/.myfsio.sys/config/iam.json IAM config path
AWS_REGION us-east-1 SigV4 region
SIGV4_TIMESTAMP_TOLERANCE_SECONDS 900 Allowed request time skew
PRESIGNED_URL_MIN_EXPIRY_SECONDS 1 Minimum presigned URL lifetime
PRESIGNED_URL_MAX_EXPIRY_SECONDS 604800 Maximum presigned URL lifetime
SECRET_KEY unset, then fallback to .myfsio.sys/config/.secret if present Session signing and IAM config encryption key
ADMIN_ACCESS_KEY unset Optional deterministic first-run/reset access key
ADMIN_SECRET_KEY unset Optional deterministic first-run/reset secret key

Feature toggles

Variable Default Description
ENCRYPTION_ENABLED false Enable object encryption support
KMS_ENABLED false Enable built-in KMS support
GC_ENABLED false Start the garbage collector worker
INTEGRITY_ENABLED false Start the integrity worker
LIFECYCLE_ENABLED false Start the lifecycle worker
METRICS_HISTORY_ENABLED false Persist system metrics snapshots
OPERATION_METRICS_ENABLED false Persist API operation metrics
WEBSITE_HOSTING_ENABLED false Enable website domain and hosting features
SITE_SYNC_ENABLED false Start the site sync worker

Metrics tuning

Variable Default Description
OPERATION_METRICS_INTERVAL_MINUTES 5 Snapshot interval for operation metrics
OPERATION_METRICS_RETENTION_HOURS 24 Retention window for operation metrics
METRICS_HISTORY_INTERVAL_MINUTES 5 Snapshot interval for system metrics
METRICS_HISTORY_RETENTION_HOURS 24 Retention window for system metrics

Replication and site sync

Variable Default Description
REPLICATION_CONNECT_TIMEOUT_SECONDS 5 Replication connect timeout
REPLICATION_READ_TIMEOUT_SECONDS 30 Replication read timeout
REPLICATION_MAX_RETRIES 2 Replication retry count
REPLICATION_STREAMING_THRESHOLD_BYTES 10485760 Switch to streaming for large copies
REPLICATION_MAX_FAILURES_PER_BUCKET 50 Failure budget before a bucket is skipped
SITE_SYNC_INTERVAL_SECONDS 60 Poll interval for the site sync worker
SITE_SYNC_BATCH_SIZE 100 Max objects processed per site sync batch
SITE_SYNC_CONNECT_TIMEOUT_SECONDS 10 Site sync connect timeout
SITE_SYNC_READ_TIMEOUT_SECONDS 120 Site sync read timeout
SITE_SYNC_MAX_RETRIES 2 Site sync retry count
SITE_SYNC_CLOCK_SKEW_TOLERANCE_SECONDS 1.0 Allowed skew between peers

Site identity values used by the UI

These are read directly by UI pages:

Variable Description
SITE_ID Local site identifier shown in the UI
SITE_ENDPOINT Public endpoint for this site
SITE_REGION Display region for the local site

7. Data Layout

With the default STORAGE_ROOT=./data, the Rust server writes:

data/
  <bucket>/
  .myfsio.sys/
    config/
      iam.json
      bucket_policies.json
      connections.json
      gc_history.json
      integrity_history.json
      metrics_history.json
      operation_metrics.json
    buckets/<bucket>/
      meta/
      versions/
    multipart/
    keys/

Important files:

  • data/.myfsio.sys/config/iam.json: IAM users, access keys, and inline policies
  • data/.myfsio.sys/config/bucket_policies.json: bucket policies
  • data/.myfsio.sys/config/connections.json: replication connection settings
  • data/.myfsio.sys/config/.secret: persisted secret key when one has been generated for the install

8. Background Services

The Rust server can start several workers from the same process.

Lifecycle

Enable with:

LIFECYCLE_ENABLED=true cargo run -p myfsio-server --

Current Rust behavior:

  • Runs as a Tokio background task, not a cron job
  • Default interval is 3600 seconds
  • Evaluates bucket lifecycle configuration and applies expiration and multipart abort rules

At the moment, the interval is hardcoded through LifecycleConfig::default() rather than exposed as an environment variable.

Garbage collection

Enable with:

GC_ENABLED=true cargo run -p myfsio-server --

Current Rust defaults from GcConfig::default():

  • Run every 6 hours
  • Temp files older than 24 hours are eligible for cleanup
  • Multipart uploads older than 7 days are eligible for cleanup
  • Lock files older than 1 hour are eligible for cleanup

Those GC timings are currently hardcoded defaults, not environment-driven configuration.

Integrity scanning

Enable with:

INTEGRITY_ENABLED=true cargo run -p myfsio-server --

Current Rust defaults from IntegrityConfig::default():

  • Run every 24 hours
  • Batch size 1000
  • Auto-heal disabled

Metrics history

Enable with:

METRICS_HISTORY_ENABLED=true cargo run -p myfsio-server --

Tune it with:

METRICS_HISTORY_INTERVAL_MINUTES=10
METRICS_HISTORY_RETENTION_HOURS=72

Snapshots are stored in data/.myfsio.sys/config/metrics_history.json.

Operation metrics

Enable with:

OPERATION_METRICS_ENABLED=true cargo run -p myfsio-server --

Tune it with:

OPERATION_METRICS_INTERVAL_MINUTES=5
OPERATION_METRICS_RETENTION_HOURS=24

Snapshots are stored in data/.myfsio.sys/config/operation_metrics.json.

9. Encryption and KMS

Object encryption and built-in KMS are both optional.

ENCRYPTION_ENABLED=true KMS_ENABLED=true cargo run -p myfsio-server --

Notes:

  • If ENCRYPTION_ENABLED=true and SECRET_KEY is not configured, the server still starts, but --check-config warns that secure-at-rest config encryption is unavailable.
  • KMS and the object encryption master key live under data/.myfsio.sys/keys/.

10. Docker

Build the Rust image from the rust/ directory:

docker build -t myfsio ./rust
docker run --rm \
  -p 5000:5000 \
  -p 5100:5100 \
  -v "$PWD/data:/app/data" \
  myfsio

The container entrypoint runs /usr/local/bin/myfsio-server.

Inside the image:

  • HOST=0.0.0.0
  • PORT=5000
  • STORAGE_ROOT=/app/data

If you want generated links and presigned URLs to use an external hostname, set API_BASE_URL.

11. Linux Installer

The repository includes scripts/install.sh. For the Rust server, build the binary first and pass the path explicitly:

cd rust/myfsio-engine
cargo build --release -p myfsio-server

cd ../..
sudo ./scripts/install.sh --binary ./rust/myfsio-engine/target/release/myfsio-server

The installer copies that binary to /opt/myfsio/myfsio, creates /opt/myfsio/myfsio.env, and can register a myfsio.service systemd unit.

12. Updating and Rollback

Recommended update flow:

  1. Stop the running service.
  2. Back up data/.myfsio.sys/config/.
  3. Build or download the new Rust binary.
  4. Run myfsio-server --check-config against the target environment.
  5. Start the service and verify /myfsio/health.

Example backup:

cp -r data/.myfsio.sys/config config-backup

Health check:

curl http://127.0.0.1:5000/myfsio/health

The response includes the active Rust crate version:

{
  "status": "ok",
  "version": "0.5.0"
}

13. Credential Reset

To rotate the bootstrap admin credentials:

cargo run -p myfsio-server -- --reset-cred

The command:

  • backs up the existing IAM file with a timestamped .bak-... suffix
  • writes a fresh admin config
  • respects ADMIN_ACCESS_KEY and ADMIN_SECRET_KEY if you set them

14. Testing

Run the Rust test suite:

cd rust/myfsio-engine
cargo test

If you are validating documentation changes for the UI, the most relevant coverage lives under:

  • rust/myfsio-engine/crates/myfsio-server/tests
  • rust/myfsio-engine/crates/myfsio-storage/src

15. API Notes

The Rust server exposes:

  • GET /myfsio/health
  • S3 bucket and object operations on /<bucket> and /<bucket>/<key>
  • UI routes under /ui/...
  • admin routes under /admin/...
  • KMS routes under /kms/...

For a route-level view, inspect:

  • rust/myfsio-engine/crates/myfsio-server/src/lib.rs
  • rust/myfsio-engine/crates/myfsio-server/src/handlers/
Reference in New Issue View Git Blame Copy Permalink