{% extends "base.html" %} {% block content %}

Documentation

Your guide to MyFSIO

Follow these steps to install, authenticate, master the console, and automate everything through the API.

API base URL
{{ api_base }}
01

Set up & run locally

Prepare a virtual environment, install dependencies, and launch both servers for a complete console + API experience.

  1. Install Python 3.11+ plus system build tools.
  2. Create a virtual environment and install requirements.txt.
  3. Start the services with python run.py.
python -m venv .venv
. .venv/Scripts/activate      # PowerShell: .\\.venv\\Scripts\\Activate.ps1
pip install -r requirements.txt

# Run both API and UI (Development)
python run.py

# Run in Production (Waitress server)
python run.py --prod

# Or run individually
python run.py --mode api
python run.py --mode ui

Configuration

Configuration defaults live in app/config.py. You can override them using environment variables. This is critical for production deployments behind proxies.

Variable Default Description
API_BASE_URL http://127.0.0.1:5000 The public URL of the API. Required if running behind a proxy or if the UI and API are on different domains. Ensures presigned URLs are generated correctly.
STORAGE_ROOT ./data Directory for buckets and objects.
MAX_UPLOAD_SIZE 5 GB Max request body size.
SECRET_KEY (Random) Flask session key. Set this in production.
APP_HOST 0.0.0.0 Bind interface.
APP_PORT 5000 Listen port.
ENCRYPTION_ENABLED false Enable server-side encryption support.
KMS_ENABLED false Enable KMS key management for encryption.
02

Running in background

For production or server deployments, run MyFSIO as a background service so it persists after you close the terminal.

Quick Start (nohup)

Simplest way to run in background—survives terminal close:

# Using Python
nohup python run.py --prod > /dev/null 2>&1 &

# Using compiled binary
nohup ./myfsio > /dev/null 2>&1 &

# Check if running
ps aux | grep myfsio

Screen / Tmux

Attach/detach from a persistent session:

# Start in a detached screen session
screen -dmS myfsio ./myfsio

# Attach to view logs
screen -r myfsio

# Detach: press Ctrl+A, then D

Systemd (Recommended for Production)

Create /etc/systemd/system/myfsio.service:

[Unit]
Description=MyFSIO S3-Compatible Storage
After=network.target

[Service]
Type=simple
User=myfsio
WorkingDirectory=/opt/myfsio
ExecStart=/opt/myfsio/myfsio
Restart=on-failure
RestartSec=5
Environment=MYFSIO_DATA_DIR=/var/lib/myfsio
Environment=API_BASE_URL=https://s3.example.com

[Install]
WantedBy=multi-user.target

Then enable and start:

sudo systemctl daemon-reload
sudo systemctl enable myfsio
sudo systemctl start myfsio

# Check status
sudo systemctl status myfsio
sudo journalctl -u myfsio -f   # View logs
03

Authenticate & manage IAM

MyFSIO seeds data/.myfsio.sys/config/iam.json with localadmin/localadmin. Sign in once, rotate it, then grant least-privilege access to teammates and tools.

  1. Visit /ui/login, enter the bootstrap credentials, and rotate them immediately from the IAM page.
  2. Create additional users with descriptive display names and AWS-style inline policies (for example {"bucket": "*", "actions": ["list", "read"]}).
  3. Rotate secrets when sharing with CI jobs—new secrets display once and persist to data/.myfsio.sys/config/iam.json.
  4. Bucket policies layer on top of IAM. Apply Private/Public presets or paste custom JSON; changes reload instantly.

All API calls require X-Access-Key and X-Secret-Key headers. The UI stores them in the Flask session after you log in.

04

Use the console effectively

Each workspace models an S3 workflow so you can administer buckets end-to-end.

Buckets

  • Create/delete buckets from the overview. Badges reveal IAM-only, public-read, or custom-policy states.
  • Summary stats show live object counts and total capacity; click through for inventories.

Uploads

  • Drag and drop folders or files into the upload modal. Objects above 16 MB switch to multipart automatically.
  • Progress rows highlight retries, throughput, and completion even if you close the modal.

Object details

  • Selecting an object opens the preview card with metadata, inline viewers, presign generator, and version history.
  • Trigger downloads, deletes, restores, or metadata refreshes without leaving the panel.

Policies & versioning

  • Toggle versioning (requires write access). Archived-only keys are flagged so you can restore them quickly.
  • The policy editor saves drafts, ships with presets, and hot-reloads data/.myfsio.sys/config/bucket_policies.json.
05

Automate with CLI & tools

Point standard S3 clients at {{ api_base }} and reuse the same IAM credentials.

AWS CLI

aws configure set aws_access_key_id <access_key>
aws configure set aws_secret_access_key <secret_key>
aws configure set default.region us-east-1

aws --endpoint-url {{ api_base }} s3 ls
aws --endpoint-url {{ api_base }} s3api create-bucket --bucket demo
aws --endpoint-url {{ api_base }} s3 cp ./sample.txt s3://demo/sample.txt

s3cmd

cat > ~/.s3cfg-myfsio <<'EOF'
host_base = {{ api_host }}
host_bucket = %(bucket)s.{{ api_host }}
access_key = <access_key>
secret_key = <secret_key>
use_https = False
signature_v2 = False
EOF

s3cmd --config ~/.s3cfg-myfsio ls
s3cmd --config ~/.s3cfg-myfsio put notes.txt s3://demo/notes.txt

curl / HTTPie

curl {{ api_base }}/ \
  -H "X-Access-Key: <access_key>" \
  -H "X-Secret-Key: <secret_key>"

curl -X PUT {{ api_base }}/demo/notes.txt \
  -H "X-Access-Key: <access_key>" \
  -H "X-Secret-Key: <secret_key>" \
  --data-binary @notes.txt

curl -X POST {{ api_base }}/presign/demo/notes.txt \
  -H "Content-Type: application/json" \
  -H "X-Access-Key: <access_key>" \
  -H "X-Secret-Key: <secret_key>" \
  -d '{"method":"GET", "expires_in": 900}'
06

Key REST endpoints

Method Path Purpose
GET / List buckets accessible to the caller.
PUT /<bucket> Create a bucket.
DELETE /<bucket> Delete a bucket (must be empty).
GET /<bucket> List objects (supports prefix / max-keys queries).
PUT /<bucket>/<key> Upload or overwrite an object; UI helper handles multipart flows.
GET /<bucket>/<key> Download an object (UI adds ?download=1 to force attachment).
DELETE /<bucket>/<key> Delete an object.
GET/PUT/DELETE /bucket-policy/<bucket> Fetch, upsert, or remove a bucket policy.
POST /presign/<bucket>/<key> Generate SigV4 URLs for GET/PUT/DELETE with custom expiry.

All responses include X-Request-Id for tracing. Logs land in logs/api.log and logs/ui.log.

07

API Examples

Common operations using boto3.

Multipart Upload
import boto3

s3 = boto3.client('s3', endpoint_url='{{ api_base }}')

# Initiate
response = s3.create_multipart_upload(Bucket='mybucket', Key='large.bin')
upload_id = response['UploadId']

# Upload parts
parts = []
chunks = [b'chunk1', b'chunk2'] # Example data chunks
for part_number, chunk in enumerate(chunks, start=1):
    response = s3.upload_part(
        Bucket='mybucket',
        Key='large.bin',
        PartNumber=part_number,
        UploadId=upload_id,
        Body=chunk
    )
    parts.append({'PartNumber': part_number, 'ETag': response['ETag']})

# Complete
s3.complete_multipart_upload(
    Bucket='mybucket',
    Key='large.bin',
    UploadId=upload_id,
    MultipartUpload={'Parts': parts}
)
08

Site Replication

Automatically copy new objects to another MyFSIO instance or S3-compatible service for backup or disaster recovery.

Setup Guide

  1. Prepare Target: On the destination server, create a bucket (e.g., backup-bucket) and an IAM user with write permissions.
  2. Connect Source: On this server, go to Connections and add the target's API URL and credentials.
  3. Enable Rule: Go to the source bucket's Replication tab, select the connection, and enter the target bucket name.
Headless Target Setup?

If your target server has no UI, use the Python API directly to bootstrap credentials. See docs.md in the project root for the setup_target.py script.

Bidirectional Replication (Active-Active)

To set up two-way replication (Server A ↔ Server B):

  1. Follow the steps above to replicate A → B.
  2. Repeat the process on Server B to replicate B → A (create a connection to A, enable rule).

Loop Prevention: The system automatically detects replication traffic using a custom User-Agent (S3ReplicationAgent). This prevents infinite loops where an object replicated from A to B is immediately replicated back to A.
Deletes: Deleting an object on one server will propagate the deletion to the other server.

09

Encryption

Protect data at rest with server-side encryption using AES-256-GCM. Objects are encrypted before being written to disk and decrypted transparently on read.

Encryption Types

Type Description
AES-256 (SSE-S3) Server-managed encryption using a local master key
KMS (SSE-KMS) Encryption using customer-managed keys via the built-in KMS

Enabling Encryption

  1. Set environment variables: 
    # PowerShell
$env:ENCRYPTION_ENABLED = "true"
$env:KMS_ENABLED = "true"  # Optional
python run.py

# Bash
export ENCRYPTION_ENABLED=true
export KMS_ENABLED=true
python run.py
  2. Configure bucket encryption: Navigate to your bucket → Properties tab → Default Encryption card → Click Enable Encryption.
  3. Choose algorithm: Select AES-256 for server-managed keys or aws:kms to use a KMS-managed key.
Important: Only new uploads after enabling encryption will be encrypted. Existing objects remain unencrypted.

KMS Key Management

When KMS_ENABLED=true, manage encryption keys via the API:

# Create a new KMS key
curl -X POST {{ api_base }}/kms/keys \
  -H "Content-Type: application/json" \
  -H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>" \
  -d '{"alias": "my-key", "description": "Production key"}'

# List all keys
curl {{ api_base }}/kms/keys \
  -H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>"

# Rotate a key (creates new key material)
curl -X POST {{ api_base }}/kms/keys/{key-id}/rotate \
  -H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>"

# Disable/Enable a key
curl -X POST {{ api_base }}/kms/keys/{key-id}/disable \
  -H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>"

# Schedule key deletion (30-day waiting period)
curl -X DELETE "{{ api_base }}/kms/keys/{key-id}?waiting_period_days=30" \
  -H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>"

How It Works

Envelope Encryption: Each object is encrypted with a unique Data Encryption Key (DEK). The DEK is then encrypted (wrapped) by the master key or KMS key and stored alongside the ciphertext. On read, the DEK is unwrapped and used to decrypt the object transparently.

10

Troubleshooting & tips

Symptom Likely cause Fix
403 from API despite Public preset Policy not saved or ARN mismatch Reapply the preset and confirm arn:aws:s3:::bucket/* matches the bucket name.
UI shows stale policy/object data Browser cached prior state Refresh; the server hot-reloads data/.myfsio.sys/config/bucket_policies.json and storage metadata.
Presign dialog returns 403 User lacks required read/write/delete action or bucket policy denies Update IAM inline policies or remove conflicting deny statements.
Large uploads fail instantly MAX_UPLOAD_SIZE exceeded Raise the env var or split the object.
Requests hit the wrong host Proxy headers missing or API_BASE_URL incorrect Ensure your proxy sends X-Forwarded-Host/Proto headers, or explicitly set API_BASE_URL to your public domain.
{% endblock %}