Update home
89
home.md
89
home.md
@@ -145,13 +145,15 @@ All configuration is done via environment variables. The table below lists every
|
|||||||
|
|
||||||
| Variable | Default | Notes |
|
| Variable | Default | Notes |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| `IAM_CONFIG` | `data/.myfsio.sys/config/iam.json` | Stores users, secrets, and inline policies. |
|
| `IAM_CONFIG` | `data/.myfsio.sys/config/iam.json` | Stores users, secrets, and inline policies. Encrypted at rest when `SECRET_KEY` is set. |
|
||||||
| `BUCKET_POLICY_PATH` | `data/.myfsio.sys/config/bucket_policies.json` | Bucket policy store (auto hot-reload). |
|
| `BUCKET_POLICY_PATH` | `data/.myfsio.sys/config/bucket_policies.json` | Bucket policy store (auto hot-reload). |
|
||||||
| `AUTH_MAX_ATTEMPTS` | `5` | Failed login attempts before lockout. |
|
| `AUTH_MAX_ATTEMPTS` | `5` | Failed login attempts before lockout. |
|
||||||
| `AUTH_LOCKOUT_MINUTES` | `15` | Lockout duration after max failed attempts. |
|
| `AUTH_LOCKOUT_MINUTES` | `15` | Lockout duration after max failed attempts. |
|
||||||
| `SESSION_LIFETIME_DAYS` | `30` | How long UI sessions remain valid. |
|
| `SESSION_LIFETIME_DAYS` | `30` | How long UI sessions remain valid. |
|
||||||
| `SECRET_TTL_SECONDS` | `300` | TTL for ephemeral secrets (presigned URLs). |
|
| `SECRET_TTL_SECONDS` | `300` | TTL for ephemeral secrets (presigned URLs). |
|
||||||
| `UI_ENFORCE_BUCKET_POLICIES` | `false` | Whether the UI should enforce bucket policies. |
|
| `UI_ENFORCE_BUCKET_POLICIES` | `false` | Whether the UI should enforce bucket policies. |
|
||||||
|
| `ADMIN_ACCESS_KEY` | (none) | Custom access key for the admin user on first run or credential reset. If unset, a random key is generated. |
|
||||||
|
| `ADMIN_SECRET_KEY` | (none) | Custom secret key for the admin user on first run or credential reset. If unset, a random key is generated. |
|
||||||
|
|
||||||
### CORS (Cross-Origin Resource Sharing)
|
### CORS (Cross-Origin Resource Sharing)
|
||||||
|
|
||||||
@@ -250,6 +252,60 @@ Once enabled, configure lifecycle rules via:
|
|||||||
</LifecycleConfiguration>
|
</LifecycleConfiguration>
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Garbage Collection
|
||||||
|
|
||||||
|
The garbage collector (GC) automatically cleans up orphaned data that accumulates over time: stale temporary files from failed uploads, abandoned multipart uploads, stale lock files, orphaned metadata entries, orphaned version files, and empty directories.
|
||||||
|
|
||||||
|
### Enabling GC
|
||||||
|
|
||||||
|
By default, GC is disabled. Enable it by setting:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
GC_ENABLED=true python run.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Or in your `myfsio.env` file:
|
||||||
|
```
|
||||||
|
GC_ENABLED=true
|
||||||
|
GC_INTERVAL_HOURS=6 # Run every 6 hours (default)
|
||||||
|
GC_TEMP_FILE_MAX_AGE_HOURS=24 # Delete temp files older than 24h
|
||||||
|
GC_MULTIPART_MAX_AGE_DAYS=7 # Delete orphaned multipart uploads older than 7 days
|
||||||
|
GC_LOCK_FILE_MAX_AGE_HOURS=1 # Delete stale lock files older than 1h
|
||||||
|
GC_DRY_RUN=false # Set to true to log without deleting
|
||||||
|
```
|
||||||
|
|
||||||
|
### What Gets Cleaned
|
||||||
|
|
||||||
|
| Type | Location | Condition |
|
||||||
|
|------|----------|-----------|
|
||||||
|
| **Temp files** | `.myfsio.sys/tmp/` | Older than `GC_TEMP_FILE_MAX_AGE_HOURS` |
|
||||||
|
| **Orphaned multipart uploads** | `.myfsio.sys/multipart/` and `<bucket>/.multipart/` | Older than `GC_MULTIPART_MAX_AGE_DAYS` |
|
||||||
|
| **Stale lock files** | `.myfsio.sys/buckets/<bucket>/locks/` | Older than `GC_LOCK_FILE_MAX_AGE_HOURS` |
|
||||||
|
| **Orphaned metadata** | `.myfsio.sys/buckets/<bucket>/meta/` and `<bucket>/.meta/` | Object file no longer exists |
|
||||||
|
| **Orphaned versions** | `.myfsio.sys/buckets/<bucket>/versions/` and `<bucket>/.versions/` | Main object no longer exists |
|
||||||
|
| **Empty directories** | Various internal directories | Directory is empty after cleanup |
|
||||||
|
|
||||||
|
### Admin API
|
||||||
|
|
||||||
|
All GC endpoints require admin (`iam:*`) permissions.
|
||||||
|
|
||||||
|
| Method | Route | Description |
|
||||||
|
|--------|-------|-------------|
|
||||||
|
| `GET` | `/admin/gc/status` | Get GC status and configuration |
|
||||||
|
| `POST` | `/admin/gc/run` | Trigger a manual GC run (body: `{"dry_run": true}` for preview) |
|
||||||
|
| `GET` | `/admin/gc/history` | Get GC execution history (query: `?limit=50&offset=0`) |
|
||||||
|
|
||||||
|
### Dry Run Mode
|
||||||
|
|
||||||
|
Set `GC_DRY_RUN=true` to log what would be deleted without actually removing anything. You can also trigger a one-time dry run via the admin API:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST "http://localhost:5000/admin/gc/run" \
|
||||||
|
-H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"dry_run": true}'
|
||||||
|
```
|
||||||
|
|
||||||
### Performance Tuning
|
### Performance Tuning
|
||||||
|
|
||||||
| Variable | Default | Notes |
|
| Variable | Default | Notes |
|
||||||
@@ -277,13 +333,14 @@ API responses for JSON, XML, HTML, CSS, and JavaScript are automatically gzip-co
|
|||||||
|
|
||||||
Before deploying to production, ensure you:
|
Before deploying to production, ensure you:
|
||||||
|
|
||||||
1. **Set `SECRET_KEY`** - Use a strong, unique value (e.g., `openssl rand -base64 32`)
|
1. **Set `SECRET_KEY`** - Use a strong, unique value (e.g., `openssl rand -base64 32`). This also enables IAM config encryption at rest.
|
||||||
2. **Restrict CORS** - Set `CORS_ORIGINS` to your specific domains instead of `*`
|
2. **Restrict CORS** - Set `CORS_ORIGINS` to your specific domains instead of `*`
|
||||||
3. **Configure `API_BASE_URL`** - Required for correct presigned URLs behind proxies
|
3. **Configure `API_BASE_URL`** - Required for correct presigned URLs behind proxies
|
||||||
4. **Enable HTTPS** - Use a reverse proxy (nginx, Cloudflare) with TLS termination
|
4. **Enable HTTPS** - Use a reverse proxy (nginx, Cloudflare) with TLS termination
|
||||||
5. **Review rate limits** - Adjust `RATE_LIMIT_DEFAULT` based on your needs
|
5. **Review rate limits** - Adjust `RATE_LIMIT_DEFAULT` based on your needs
|
||||||
6. **Secure master keys** - Back up `ENCRYPTION_MASTER_KEY_PATH` if using encryption
|
6. **Secure master keys** - Back up `ENCRYPTION_MASTER_KEY_PATH` if using encryption
|
||||||
7. **Use `--prod` flag** - Runs with Waitress instead of Flask dev server
|
7. **Use `--prod` flag** - Runs with Waitress instead of Flask dev server
|
||||||
|
8. **Set credential expiry** - Assign `expires_at` to non-admin users for time-limited access
|
||||||
|
|
||||||
### Proxy Configuration
|
### Proxy Configuration
|
||||||
|
|
||||||
@@ -633,9 +690,10 @@ MyFSIO implements a comprehensive Identity and Access Management (IAM) system th
|
|||||||
|
|
||||||
### Getting Started
|
### Getting Started
|
||||||
|
|
||||||
1. On first boot, `data/.myfsio.sys/config/iam.json` is created with a randomly generated admin user. The access key and secret key are printed to the console during first startup. If you miss it, check the `iam.json` file directly—credentials are stored in plaintext.
|
1. On first boot, `data/.myfsio.sys/config/iam.json` is created with a randomly generated admin user. The access key and secret key are printed to the console during first startup. You can set `ADMIN_ACCESS_KEY` and `ADMIN_SECRET_KEY` environment variables to use custom credentials instead of random ones. If `SECRET_KEY` is configured, the IAM config file is encrypted at rest using AES (Fernet). To reset admin credentials later, run `python run.py --reset-cred`.
|
||||||
2. Sign into the UI using the generated credentials, then open **IAM**:
|
2. Sign into the UI using the generated credentials, then open **IAM**:
|
||||||
- **Create user**: supply a display name and optional JSON inline policy array.
|
- **Create user**: supply a display name, optional JSON inline policy array, and optional credential expiry date.
|
||||||
|
- **Set expiry**: assign an expiration date to any user's credentials. Expired credentials are rejected at authentication time. The UI shows expiry badges and preset durations (1h, 24h, 7d, 30d, 90d).
|
||||||
- **Rotate secret**: generates a new secret key; the UI surfaces it once.
|
- **Rotate secret**: generates a new secret key; the UI surfaces it once.
|
||||||
- **Policy editor**: select a user, paste an array of objects (`{"bucket": "*", "actions": ["list", "read"]}`), and submit. Alias support includes AWS-style verbs (e.g., `s3:GetObject`).
|
- **Policy editor**: select a user, paste an array of objects (`{"bucket": "*", "actions": ["list", "read"]}`), and submit. Alias support includes AWS-style verbs (e.g., `s3:GetObject`).
|
||||||
3. Wildcard action `iam:*` is supported for admin user definitions.
|
3. Wildcard action `iam:*` is supported for admin user definitions.
|
||||||
@@ -653,8 +711,11 @@ The API expects every request to include authentication headers. The UI persists
|
|||||||
|
|
||||||
**Security Features:**
|
**Security Features:**
|
||||||
- **Lockout Protection**: After `AUTH_MAX_ATTEMPTS` (default: 5) failed login attempts, the account is locked for `AUTH_LOCKOUT_MINUTES` (default: 15 minutes).
|
- **Lockout Protection**: After `AUTH_MAX_ATTEMPTS` (default: 5) failed login attempts, the account is locked for `AUTH_LOCKOUT_MINUTES` (default: 15 minutes).
|
||||||
|
- **Credential Expiry**: Each user can have an optional `expires_at` timestamp (ISO 8601). Once expired, all API requests using those credentials are rejected. Set or clear expiry via the UI or API.
|
||||||
|
- **IAM Config Encryption**: When `SECRET_KEY` is set, the IAM config file (`iam.json`) is encrypted at rest using Fernet (AES-256-CBC with HMAC). Existing plaintext configs are automatically encrypted on next load.
|
||||||
- **Session Management**: UI sessions remain valid for `SESSION_LIFETIME_DAYS` (default: 30 days).
|
- **Session Management**: UI sessions remain valid for `SESSION_LIFETIME_DAYS` (default: 30 days).
|
||||||
- **Hot Reload**: IAM configuration changes take effect immediately without restart.
|
- **Hot Reload**: IAM configuration changes take effect immediately without restart.
|
||||||
|
- **Credential Reset**: Run `python run.py --reset-cred` to reset admin credentials. Supports `ADMIN_ACCESS_KEY` and `ADMIN_SECRET_KEY` env vars for deterministic keys.
|
||||||
|
|
||||||
### Permission Model
|
### Permission Model
|
||||||
|
|
||||||
@@ -814,7 +875,8 @@ curl -X POST http://localhost:5000/iam/users \
|
|||||||
-H "X-Access-Key: ..." -H "X-Secret-Key: ..." \
|
-H "X-Access-Key: ..." -H "X-Secret-Key: ..." \
|
||||||
-d '{
|
-d '{
|
||||||
"display_name": "New User",
|
"display_name": "New User",
|
||||||
"policies": [{"bucket": "*", "actions": ["list", "read"]}]
|
"policies": [{"bucket": "*", "actions": ["list", "read"]}],
|
||||||
|
"expires_at": "2026-12-31T23:59:59Z"
|
||||||
}'
|
}'
|
||||||
|
|
||||||
# Rotate user secret (requires iam:rotate_key)
|
# Rotate user secret (requires iam:rotate_key)
|
||||||
@@ -827,6 +889,18 @@ curl -X PUT http://localhost:5000/iam/users/<access-key>/policies \
|
|||||||
-H "X-Access-Key: ..." -H "X-Secret-Key: ..." \
|
-H "X-Access-Key: ..." -H "X-Secret-Key: ..." \
|
||||||
-d '[{"bucket": "*", "actions": ["list", "read", "write"]}]'
|
-d '[{"bucket": "*", "actions": ["list", "read", "write"]}]'
|
||||||
|
|
||||||
|
# Update credential expiry (requires iam:update_policy)
|
||||||
|
curl -X POST http://localhost:5000/iam/users/<access-key>/expiry \
|
||||||
|
-H "Content-Type: application/x-www-form-urlencoded" \
|
||||||
|
-H "X-Access-Key: ..." -H "X-Secret-Key: ..." \
|
||||||
|
-d 'expires_at=2026-12-31T23:59:59Z'
|
||||||
|
|
||||||
|
# Remove credential expiry (never expires)
|
||||||
|
curl -X POST http://localhost:5000/iam/users/<access-key>/expiry \
|
||||||
|
-H "Content-Type: application/x-www-form-urlencoded" \
|
||||||
|
-H "X-Access-Key: ..." -H "X-Secret-Key: ..." \
|
||||||
|
-d 'expires_at='
|
||||||
|
|
||||||
# Delete a user (requires iam:delete_user)
|
# Delete a user (requires iam:delete_user)
|
||||||
curl -X DELETE http://localhost:5000/iam/users/<access-key> \
|
curl -X DELETE http://localhost:5000/iam/users/<access-key> \
|
||||||
-H "X-Access-Key: ..." -H "X-Secret-Key: ..."
|
-H "X-Access-Key: ..." -H "X-Secret-Key: ..."
|
||||||
@@ -838,8 +912,9 @@ When a request is made, permissions are evaluated in this order:
|
|||||||
|
|
||||||
1. **Authentication** – Verify the access key and secret key are valid
|
1. **Authentication** – Verify the access key and secret key are valid
|
||||||
2. **Lockout Check** – Ensure the account is not locked due to failed attempts
|
2. **Lockout Check** – Ensure the account is not locked due to failed attempts
|
||||||
3. **IAM Policy Check** – Verify the user has the required action for the target bucket
|
3. **Expiry Check** – Reject requests if the user's credentials have expired (`expires_at`)
|
||||||
4. **Bucket Policy Check** – If a bucket policy exists, verify it allows the action
|
4. **IAM Policy Check** – Verify the user has the required action for the target bucket
|
||||||
|
5. **Bucket Policy Check** – If a bucket policy exists, verify it allows the action
|
||||||
|
|
||||||
A request is allowed only if:
|
A request is allowed only if:
|
||||||
- The IAM policy grants the action, AND
|
- The IAM policy grants the action, AND
|
||||||
|
|||||||
Reference in New Issue
Block a user