Update docs
This commit is contained in:
69
docs.md
69
docs.md
@@ -139,6 +139,7 @@ All configuration is done via environment variables. The table below lists every
|
||||
| `API_BASE_URL` | `http://127.0.0.1:5000` | Internal S3 API URL used by the web UI proxy. Also used for presigned URL generation. Set to your public URL if running behind a reverse proxy. |
|
||||
| `AWS_REGION` | `us-east-1` | Region embedded in SigV4 credential scope. |
|
||||
| `AWS_SERVICE` | `s3` | Service string for SigV4. |
|
||||
| `DISPLAY_TIMEZONE` | `UTC` | Timezone for timestamps in the web UI (e.g., `US/Eastern`, `Asia/Tokyo`). |
|
||||
|
||||
### IAM & Security
|
||||
|
||||
@@ -170,6 +171,7 @@ All configuration is done via environment variables. The table below lists every
|
||||
| `RATE_LIMIT_BUCKET_OPS` | `120 per minute` | Rate limit for bucket operations (PUT/DELETE/GET/POST on `/<bucket>`). |
|
||||
| `RATE_LIMIT_OBJECT_OPS` | `240 per minute` | Rate limit for object operations (PUT/GET/DELETE/POST on `/<bucket>/<key>`). |
|
||||
| `RATE_LIMIT_HEAD_OPS` | `100 per minute` | Rate limit for HEAD requests (bucket and object). |
|
||||
| `RATE_LIMIT_ADMIN` | `60 per minute` | Rate limit for admin API endpoints (`/admin/*`). |
|
||||
| `RATE_LIMIT_STORAGE_URI` | `memory://` | Storage backend for rate limits. Use `redis://host:port` for distributed setups. |
|
||||
|
||||
### Server Configuration
|
||||
@@ -256,6 +258,12 @@ Once enabled, configure lifecycle rules via:
|
||||
| `MULTIPART_MIN_PART_SIZE` | `5242880` (5 MB) | Minimum part size for multipart uploads. |
|
||||
| `BUCKET_STATS_CACHE_TTL` | `60` | Seconds to cache bucket statistics. |
|
||||
| `BULK_DELETE_MAX_KEYS` | `500` | Maximum keys per bulk delete request. |
|
||||
| `BULK_DOWNLOAD_MAX_BYTES` | `1073741824` (1 GiB) | Maximum total size for bulk ZIP downloads. |
|
||||
| `OBJECT_CACHE_TTL` | `60` | Seconds to cache object metadata. |
|
||||
|
||||
#### Gzip Compression
|
||||
|
||||
API responses for JSON, XML, HTML, CSS, and JavaScript are automatically gzip-compressed when the client sends `Accept-Encoding: gzip`. Compression activates for responses larger than 500 bytes and is handled by a WSGI middleware (`app/compression.py`). Binary object downloads and streaming responses are never compressed. No configuration is needed.
|
||||
|
||||
### Server Settings
|
||||
|
||||
@@ -285,6 +293,12 @@ If running behind a reverse proxy (e.g., Nginx, Cloudflare, or a tunnel), ensure
|
||||
|
||||
The application automatically trusts these headers to generate correct presigned URLs (e.g., `https://s3.example.com/...` instead of `http://127.0.0.1:5000/...`). Alternatively, you can explicitly set `API_BASE_URL` to your public endpoint.
|
||||
|
||||
| Variable | Default | Notes |
|
||||
| --- | --- | --- |
|
||||
| `NUM_TRUSTED_PROXIES` | `1` | Number of trusted reverse proxies for `X-Forwarded-*` header processing. |
|
||||
| `ALLOWED_REDIRECT_HOSTS` | `""` | Comma-separated whitelist of safe redirect targets. Empty allows only same-host redirects. |
|
||||
| `ALLOW_INTERNAL_ENDPOINTS` | `false` | Allow connections to internal/private IPs for webhooks and replication targets. **Keep disabled in production unless needed.** |
|
||||
|
||||
## 4. Upgrading and Updates
|
||||
|
||||
### Version Checking
|
||||
@@ -912,7 +926,7 @@ Objects with forward slashes (`/`) in their keys are displayed as a folder hiera
|
||||
|
||||
- Select multiple objects using checkboxes
|
||||
- **Bulk Delete**: Delete multiple objects at once
|
||||
- **Bulk Download**: Download selected objects as individual files
|
||||
- **Bulk Download**: Download selected objects as a single ZIP archive (up to `BULK_DOWNLOAD_MAX_BYTES`, default 1 GiB)
|
||||
|
||||
#### Search & Filter
|
||||
|
||||
@@ -985,6 +999,7 @@ MyFSIO supports **server-side encryption at rest** to protect your data. When en
|
||||
|------|-------------|
|
||||
| **AES-256 (SSE-S3)** | Server-managed encryption using a local master key |
|
||||
| **KMS (SSE-KMS)** | Encryption using customer-managed keys via the built-in KMS |
|
||||
| **SSE-C** | Server-side encryption with customer-provided keys (per-request) |
|
||||
|
||||
### Enabling Encryption
|
||||
|
||||
@@ -1083,6 +1098,44 @@ encrypted, metadata = ClientEncryptionHelper.encrypt_for_upload(plaintext, key)
|
||||
decrypted = ClientEncryptionHelper.decrypt_from_download(encrypted, metadata, key)
|
||||
```
|
||||
|
||||
### SSE-C (Customer-Provided Keys)
|
||||
|
||||
With SSE-C, you provide your own 256-bit AES encryption key with each request. The server encrypts/decrypts using your key but never stores it. You must supply the same key for both upload and download.
|
||||
|
||||
**Required headers:**
|
||||
|
||||
| Header | Value |
|
||||
|--------|-------|
|
||||
| `x-amz-server-side-encryption-customer-algorithm` | `AES256` |
|
||||
| `x-amz-server-side-encryption-customer-key` | Base64-encoded 256-bit key |
|
||||
| `x-amz-server-side-encryption-customer-key-MD5` | Base64-encoded MD5 of the key |
|
||||
|
||||
```bash
|
||||
# Generate a 256-bit key
|
||||
KEY=$(openssl rand -base64 32)
|
||||
KEY_MD5=$(echo -n "$KEY" | base64 -d | openssl dgst -md5 -binary | base64)
|
||||
|
||||
# Upload with SSE-C
|
||||
curl -X PUT "http://localhost:5000/my-bucket/secret.txt" \
|
||||
-H "X-Access-Key: ..." -H "X-Secret-Key: ..." \
|
||||
-H "x-amz-server-side-encryption-customer-algorithm: AES256" \
|
||||
-H "x-amz-server-side-encryption-customer-key: $KEY" \
|
||||
-H "x-amz-server-side-encryption-customer-key-MD5: $KEY_MD5" \
|
||||
--data-binary @secret.txt
|
||||
|
||||
# Download with SSE-C (same key required)
|
||||
curl "http://localhost:5000/my-bucket/secret.txt" \
|
||||
-H "X-Access-Key: ..." -H "X-Secret-Key: ..." \
|
||||
-H "x-amz-server-side-encryption-customer-algorithm: AES256" \
|
||||
-H "x-amz-server-side-encryption-customer-key: $KEY" \
|
||||
-H "x-amz-server-side-encryption-customer-key-MD5: $KEY_MD5"
|
||||
```
|
||||
|
||||
**Key points:**
|
||||
- SSE-C does not require `ENCRYPTION_ENABLED` or `KMS_ENABLED` — the key is provided per-request
|
||||
- If you lose your key, the data is irrecoverable
|
||||
- The MD5 header is optional but recommended for integrity verification
|
||||
|
||||
### Important Notes
|
||||
|
||||
- **Existing objects are NOT encrypted** - Only new uploads after enabling encryption are encrypted
|
||||
@@ -1959,6 +2012,20 @@ curl -X PUT "http://localhost:5000/my-bucket/file.txt" \
|
||||
-H "x-amz-meta-newkey: newvalue"
|
||||
```
|
||||
|
||||
### MoveObject (UI)
|
||||
|
||||
Move an object to a different key or bucket. This is a UI-only convenience operation that performs a copy followed by a delete of the source. Requires `read` and `delete` on the source, and `write` on the destination.
|
||||
|
||||
```bash
|
||||
# Move via UI API
|
||||
curl -X POST "http://localhost:5100/ui/buckets/my-bucket/objects/old-path/file.txt/move" \
|
||||
-H "Content-Type: application/json" \
|
||||
--cookie "session=..." \
|
||||
-d '{"dest_bucket": "other-bucket", "dest_key": "new-path/file.txt"}'
|
||||
```
|
||||
|
||||
The move is atomic from the caller's perspective: if the copy succeeds but the delete fails, the object exists in both locations (no data loss).
|
||||
|
||||
### UploadPartCopy
|
||||
|
||||
Copy data from an existing object into a multipart upload part:
|
||||
|
||||
@@ -52,6 +52,11 @@
|
||||
<li><a href="#acls">Access Control Lists</a></li>
|
||||
<li><a href="#tagging">Object & Bucket Tagging</a></li>
|
||||
<li><a href="#website-hosting">Static Website Hosting</a></li>
|
||||
<li><a href="#cors-config">CORS Configuration</a></li>
|
||||
<li><a href="#post-object">PostObject (Form Upload)</a></li>
|
||||
<li><a href="#list-objects-v2">List Objects API v2</a></li>
|
||||
<li><a href="#upgrading">Upgrading & Updates</a></li>
|
||||
<li><a href="#api-matrix">Full API Reference</a></li>
|
||||
</ul>
|
||||
</div>
|
||||
</div>
|
||||
@@ -126,6 +131,11 @@ python run.py --mode ui
|
||||
<td><code>5000</code></td>
|
||||
<td>Listen port (UI uses 5100).</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>DISPLAY_TIMEZONE</code></td>
|
||||
<td><code>UTC</code></td>
|
||||
<td>Timezone for UI timestamps (e.g., <code>US/Eastern</code>, <code>Asia/Tokyo</code>).</td>
|
||||
</tr>
|
||||
<tr class="table-secondary">
|
||||
<td colspan="3" class="fw-semibold">CORS Settings</td>
|
||||
</tr>
|
||||
@@ -187,6 +197,11 @@ python run.py --mode ui
|
||||
<td><code>100 per minute</code></td>
|
||||
<td>Rate limit for HEAD requests.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>RATE_LIMIT_ADMIN</code></td>
|
||||
<td><code>60 per minute</code></td>
|
||||
<td>Rate limit for admin API endpoints (<code>/admin/*</code>).</td>
|
||||
</tr>
|
||||
<tr class="table-secondary">
|
||||
<td colspan="3" class="fw-semibold">Server Settings</td>
|
||||
</tr>
|
||||
@@ -338,6 +353,24 @@ python run.py --mode ui
|
||||
<td><code>604800</code></td>
|
||||
<td>Maximum presigned URL expiry time (7 days).</td>
|
||||
</tr>
|
||||
<tr class="table-secondary">
|
||||
<td colspan="3" class="fw-semibold">Proxy & Network Settings</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>NUM_TRUSTED_PROXIES</code></td>
|
||||
<td><code>1</code></td>
|
||||
<td>Number of trusted reverse proxies for <code>X-Forwarded-*</code> headers.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ALLOWED_REDIRECT_HOSTS</code></td>
|
||||
<td>(empty)</td>
|
||||
<td>Comma-separated whitelist of safe redirect targets.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ALLOW_INTERNAL_ENDPOINTS</code></td>
|
||||
<td><code>false</code></td>
|
||||
<td>Allow connections to internal/private IPs (webhooks, replication).</td>
|
||||
</tr>
|
||||
<tr class="table-secondary">
|
||||
<td colspan="3" class="fw-semibold">Storage Limits</td>
|
||||
</tr>
|
||||
@@ -366,6 +399,16 @@ python run.py --mode ui
|
||||
<td><code>50</code></td>
|
||||
<td>Max lifecycle history records per bucket.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>OBJECT_CACHE_TTL</code></td>
|
||||
<td><code>60</code></td>
|
||||
<td>Seconds to cache object metadata.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>BULK_DOWNLOAD_MAX_BYTES</code></td>
|
||||
<td><code>1 GB</code></td>
|
||||
<td>Max total size for bulk ZIP downloads.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ENCRYPTION_CHUNK_SIZE_BYTES</code></td>
|
||||
<td><code>65536</code></td>
|
||||
@@ -491,7 +534,7 @@ sudo journalctl -u myfsio -f # View logs</code></pre>
|
||||
<ul>
|
||||
<li>Navigate folder hierarchies using breadcrumbs. Objects with <code>/</code> in keys display as folders.</li>
|
||||
<li>Infinite scroll loads more objects automatically. Choose batch size (50–250) from the footer dropdown.</li>
|
||||
<li>Bulk select objects for multi-delete or multi-download. Filter by name using the search box.</li>
|
||||
<li>Bulk select objects for multi-delete or multi-download (ZIP archive, up to 1 GiB). Filter by name using the search box.</li>
|
||||
<li>If loading fails, click <strong>Retry</strong> to attempt again—no page refresh needed.</li>
|
||||
</ul>
|
||||
</div>
|
||||
@@ -613,15 +656,75 @@ curl -X PUT {{ api_base }}/demo/notes.txt \
|
||||
<td><code>/<bucket>/<key></code></td>
|
||||
<td>Delete an object.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>HEAD</td>
|
||||
<td><code>/<bucket></code></td>
|
||||
<td>Check if a bucket exists.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>HEAD</td>
|
||||
<td><code>/<bucket>/<key></code></td>
|
||||
<td>Get object metadata without downloading.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>POST</td>
|
||||
<td><code>/<bucket>?delete</code></td>
|
||||
<td>Bulk delete objects (XML body).</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GET/PUT/DELETE</td>
|
||||
<td><code>/<bucket>?policy</code></td>
|
||||
<td>Fetch, upsert, or remove a bucket policy (S3-compatible).</td>
|
||||
<td>Bucket policy management.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GET/PUT</td>
|
||||
<td><code>/<bucket>?versioning</code></td>
|
||||
<td>Versioning status.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GET/PUT/DELETE</td>
|
||||
<td><code>/<bucket>?lifecycle</code></td>
|
||||
<td>Lifecycle rules.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GET/PUT/DELETE</td>
|
||||
<td><code>/<bucket>?cors</code></td>
|
||||
<td>CORS configuration.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GET/PUT/DELETE</td>
|
||||
<td><code>/<bucket>?encryption</code></td>
|
||||
<td>Default encryption.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GET/PUT</td>
|
||||
<td><code>/<bucket>?acl</code></td>
|
||||
<td>Bucket ACL.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GET/PUT/DELETE</td>
|
||||
<td><code>/<bucket>?tagging</code></td>
|
||||
<td>Bucket tags.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GET/PUT/DELETE</td>
|
||||
<td><code>/<bucket>/<key>?tagging</code></td>
|
||||
<td>Object tags.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>POST</td>
|
||||
<td><code>/<bucket>/<key>?uploads</code></td>
|
||||
<td>Initiate multipart upload.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>POST</td>
|
||||
<td><code>/<bucket>/<key>?select</code></td>
|
||||
<td>SQL query (SelectObjectContent).</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<p class="small text-muted mt-3 mb-0">All responses include <code>X-Request-Id</code> for tracing. Logs land in <code>logs/api.log</code> and <code>logs/ui.log</code>.</p>
|
||||
<p class="small text-muted mt-3 mb-0">All responses include <code>X-Request-Id</code> for tracing. See the <a href="#api-matrix">Full API Reference</a> for the complete endpoint list. Logs land in <code>logs/api.log</code> and <code>logs/ui.log</code>.</p>
|
||||
</div>
|
||||
</article>
|
||||
<article id="examples" class="card shadow-sm docs-section">
|
||||
@@ -1311,6 +1414,10 @@ curl -X PUT "{{ api_base }}/bucket/<bucket>?quota" \
|
||||
<td><strong>KMS (SSE-KMS)</strong></td>
|
||||
<td>Encryption using customer-managed keys via the built-in KMS</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>SSE-C</strong></td>
|
||||
<td>Server-side encryption with customer-provided keys (per-request)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
@@ -1377,6 +1484,54 @@ curl -X DELETE "{{ api_base }}/kms/keys/{key-id}?waiting_period_days=30" \
|
||||
<p class="small text-muted mb-0">
|
||||
<strong>Envelope Encryption:</strong> Each object is encrypted with a unique Data Encryption Key (DEK). The DEK is then encrypted (wrapped) by the master key or KMS key and stored alongside the ciphertext. On read, the DEK is unwrapped and used to decrypt the object transparently.
|
||||
</p>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">SSE-C (Customer-Provided Keys)</h3>
|
||||
<p class="small text-muted">With SSE-C, you supply your own 256-bit AES key with each request. The server encrypts/decrypts using your key but never stores it. You must provide the same key for both upload and download.</p>
|
||||
<div class="table-responsive mb-3">
|
||||
<table class="table table-sm table-bordered small">
|
||||
<thead class="table-light">
|
||||
<tr>
|
||||
<th>Header</th>
|
||||
<th>Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>x-amz-server-side-encryption-customer-algorithm</code></td>
|
||||
<td><code>AES256</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>x-amz-server-side-encryption-customer-key</code></td>
|
||||
<td>Base64-encoded 256-bit key</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>x-amz-server-side-encryption-customer-key-MD5</code></td>
|
||||
<td>Base64-encoded MD5 of the key</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
<pre class="mb-3"><code class="language-bash"># Generate a 256-bit key
|
||||
KEY=$(openssl rand -base64 32)
|
||||
KEY_MD5=$(echo -n "$KEY" | base64 -d | openssl dgst -md5 -binary | base64)
|
||||
|
||||
# Upload with SSE-C
|
||||
curl -X PUT "{{ api_base }}/my-bucket/secret.txt" \
|
||||
-H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>" \
|
||||
-H "x-amz-server-side-encryption-customer-algorithm: AES256" \
|
||||
-H "x-amz-server-side-encryption-customer-key: $KEY" \
|
||||
-H "x-amz-server-side-encryption-customer-key-MD5: $KEY_MD5" \
|
||||
--data-binary @secret.txt
|
||||
|
||||
# Download with SSE-C (same key required)
|
||||
curl "{{ api_base }}/my-bucket/secret.txt" \
|
||||
-H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>" \
|
||||
-H "x-amz-server-side-encryption-customer-algorithm: AES256" \
|
||||
-H "x-amz-server-side-encryption-customer-key: $KEY" \
|
||||
-H "x-amz-server-side-encryption-customer-key-MD5: $KEY_MD5"</code></pre>
|
||||
<div class="alert alert-light border mb-0 small">
|
||||
<strong>Note:</strong> SSE-C does not require <code>ENCRYPTION_ENABLED</code> or <code>KMS_ENABLED</code>. If you lose your key, the data is irrecoverable.
|
||||
</div>
|
||||
</div>
|
||||
</article>
|
||||
<article id="lifecycle" class="card shadow-sm docs-section">
|
||||
@@ -1926,7 +2081,7 @@ curl -X POST "{{ api_base }}/<bucket>/data.csv?select" \
|
||||
<span class="docs-section-kicker">22</span>
|
||||
<h2 class="h4 mb-0">Advanced S3 Operations</h2>
|
||||
</div>
|
||||
<p class="text-muted">Copy objects, upload part copies, and use range requests for partial downloads.</p>
|
||||
<p class="text-muted">Copy, move, and partially download objects using advanced S3 operations.</p>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">CopyObject</h3>
|
||||
<pre class="mb-3"><code class="language-bash"># Copy within same bucket
|
||||
@@ -1941,6 +2096,13 @@ curl -X PUT "{{ api_base }}/<bucket>/file.txt" \
|
||||
-H "x-amz-metadata-directive: REPLACE" \
|
||||
-H "x-amz-meta-newkey: newvalue"</code></pre>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">MoveObject (UI)</h3>
|
||||
<p class="small text-muted">Move an object to a different key or bucket via the UI. Performs a copy then deletes the source. Requires <code>read</code>+<code>delete</code> on source and <code>write</code> on destination.</p>
|
||||
<pre class="mb-3"><code class="language-bash"># Move via UI API (session-authenticated)
|
||||
curl -X POST "http://localhost:5100/ui/buckets/<bucket>/objects/<key>/move" \
|
||||
-H "Content-Type: application/json" --cookie "session=..." \
|
||||
-d '{"dest_bucket": "other-bucket", "dest_key": "new-path/file.txt"}'</code></pre>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">UploadPartCopy</h3>
|
||||
<p class="small text-muted">Copy data from an existing object into a multipart upload part:</p>
|
||||
<pre class="mb-3"><code class="language-bash"># Copy bytes 0-10485759 from source as part 1
|
||||
@@ -2193,6 +2355,279 @@ server {
|
||||
</div>
|
||||
</div>
|
||||
</article>
|
||||
<article id="cors-config" class="card shadow-sm docs-section">
|
||||
<div class="card-body">
|
||||
<div class="d-flex align-items-center gap-2 mb-3">
|
||||
<span class="docs-section-kicker">26</span>
|
||||
<h2 class="h4 mb-0">CORS Configuration</h2>
|
||||
</div>
|
||||
<p class="text-muted">Configure per-bucket Cross-Origin Resource Sharing rules to control which origins can access your bucket from a browser.</p>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">Setting CORS Rules</h3>
|
||||
<pre class="mb-3"><code class="language-bash"># Set CORS configuration
|
||||
curl -X PUT "{{ api_base }}/<bucket>?cors" \
|
||||
-H "Content-Type: application/xml" \
|
||||
-H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>" \
|
||||
-d '<CORSConfiguration>
|
||||
<CORSRule>
|
||||
<AllowedOrigin>https://example.com</AllowedOrigin>
|
||||
<AllowedMethod>GET</AllowedMethod>
|
||||
<AllowedMethod>PUT</AllowedMethod>
|
||||
<AllowedHeader>*</AllowedHeader>
|
||||
<ExposeHeader>ETag</ExposeHeader>
|
||||
<MaxAgeSeconds>3600</MaxAgeSeconds>
|
||||
</CORSRule>
|
||||
</CORSConfiguration>'
|
||||
|
||||
# Get CORS configuration
|
||||
curl "{{ api_base }}/<bucket>?cors" \
|
||||
-H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>"
|
||||
|
||||
# Delete CORS configuration
|
||||
curl -X DELETE "{{ api_base }}/<bucket>?cors" \
|
||||
-H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>"</code></pre>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">Rule Fields</h3>
|
||||
<div class="table-responsive mb-0">
|
||||
<table class="table table-sm table-bordered small mb-0">
|
||||
<thead class="table-light">
|
||||
<tr>
|
||||
<th>Field</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>AllowedOrigin</code></td>
|
||||
<td>Origins allowed to make requests (supports <code>*</code> wildcard)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>AllowedMethod</code></td>
|
||||
<td>HTTP methods: <code>GET</code>, <code>PUT</code>, <code>POST</code>, <code>DELETE</code>, <code>HEAD</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>AllowedHeader</code></td>
|
||||
<td>Request headers allowed in preflight (supports <code>*</code>)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ExposeHeader</code></td>
|
||||
<td>Response headers visible to the browser (e.g., <code>ETag</code>, <code>x-amz-request-id</code>)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>MaxAgeSeconds</code></td>
|
||||
<td>How long the browser caches preflight results</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</article>
|
||||
<article id="post-object" class="card shadow-sm docs-section">
|
||||
<div class="card-body">
|
||||
<div class="d-flex align-items-center gap-2 mb-3">
|
||||
<span class="docs-section-kicker">27</span>
|
||||
<h2 class="h4 mb-0">PostObject (HTML Form Upload)</h2>
|
||||
</div>
|
||||
<p class="text-muted">Upload objects directly from an HTML form using browser-based POST uploads with policy-based authorization.</p>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">Form Fields</h3>
|
||||
<div class="table-responsive mb-3">
|
||||
<table class="table table-sm table-bordered small">
|
||||
<thead class="table-light">
|
||||
<tr>
|
||||
<th>Field</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr><td><code>key</code></td><td>Object key (supports <code>${filename}</code> variable)</td></tr>
|
||||
<tr><td><code>file</code></td><td>The file to upload</td></tr>
|
||||
<tr><td><code>policy</code></td><td>Base64-encoded policy document (JSON)</td></tr>
|
||||
<tr><td><code>x-amz-signature</code></td><td>HMAC-SHA256 signature of the policy</td></tr>
|
||||
<tr><td><code>x-amz-credential</code></td><td>Access key / date / region / s3 / aws4_request</td></tr>
|
||||
<tr><td><code>x-amz-algorithm</code></td><td><code>AWS4-HMAC-SHA256</code></td></tr>
|
||||
<tr><td><code>x-amz-date</code></td><td>ISO 8601 date (e.g., <code>20250101T000000Z</code>)</td></tr>
|
||||
<tr><td><code>Content-Type</code></td><td>MIME type of the uploaded file</td></tr>
|
||||
<tr><td><code>x-amz-meta-*</code></td><td>Custom metadata headers</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">Simple Upload (No Signing)</h3>
|
||||
<pre class="mb-3"><code class="language-html"><form action="{{ api_base }}/my-bucket" method="POST" enctype="multipart/form-data">
|
||||
<input type="hidden" name="key" value="uploads/${filename}">
|
||||
<input type="file" name="file">
|
||||
<button type="submit">Upload</button>
|
||||
</form></code></pre>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">Signed Upload (With Policy)</h3>
|
||||
<p class="small text-muted mb-0">For authenticated uploads, include a base64-encoded policy and SigV4 signature fields. The policy constrains allowed keys, content types, and size limits. See docs.md Section 20 for full signing examples.</p>
|
||||
</div>
|
||||
</article>
|
||||
<article id="list-objects-v2" class="card shadow-sm docs-section">
|
||||
<div class="card-body">
|
||||
<div class="d-flex align-items-center gap-2 mb-3">
|
||||
<span class="docs-section-kicker">28</span>
|
||||
<h2 class="h4 mb-0">List Objects API v2</h2>
|
||||
</div>
|
||||
<p class="text-muted">Use the v2 list API for improved pagination with continuation tokens instead of markers.</p>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">Usage</h3>
|
||||
<pre class="mb-3"><code class="language-bash"># List with v2 API
|
||||
curl "{{ api_base }}/<bucket>?list-type=2&prefix=logs/&delimiter=/&max-keys=100" \
|
||||
-H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>"
|
||||
|
||||
# Paginate with continuation token
|
||||
curl "{{ api_base }}/<bucket>?list-type=2&continuation-token=<token>" \
|
||||
-H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>"
|
||||
|
||||
# Start listing after a specific key
|
||||
curl "{{ api_base }}/<bucket>?list-type=2&start-after=photos/2025/" \
|
||||
-H "X-Access-Key: <key>" -H "X-Secret-Key: <secret>"</code></pre>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">Query Parameters</h3>
|
||||
<div class="table-responsive mb-0">
|
||||
<table class="table table-sm table-bordered small mb-0">
|
||||
<thead class="table-light">
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr><td><code>list-type=2</code></td><td>Enables v2 API (required)</td></tr>
|
||||
<tr><td><code>prefix</code></td><td>Filter to keys starting with this prefix</td></tr>
|
||||
<tr><td><code>delimiter</code></td><td>Group keys by delimiter (typically <code>/</code> for folders)</td></tr>
|
||||
<tr><td><code>max-keys</code></td><td>Maximum objects to return (default 1000)</td></tr>
|
||||
<tr><td><code>continuation-token</code></td><td>Token from previous response for pagination</td></tr>
|
||||
<tr><td><code>start-after</code></td><td>Start listing after this key (first page only)</td></tr>
|
||||
<tr><td><code>fetch-owner</code></td><td>Include owner info in response</td></tr>
|
||||
<tr><td><code>encoding-type</code></td><td>Set to <code>url</code> to URL-encode keys in response</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
</div>
|
||||
</article>
|
||||
<article id="upgrading" class="card shadow-sm docs-section">
|
||||
<div class="card-body">
|
||||
<div class="d-flex align-items-center gap-2 mb-3">
|
||||
<span class="docs-section-kicker">29</span>
|
||||
<h2 class="h4 mb-0">Upgrading & Updates</h2>
|
||||
</div>
|
||||
<p class="text-muted">How to safely update MyFSIO to a new version.</p>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">Pre-Update Backup</h3>
|
||||
<p class="small text-muted">Always back up before updating:</p>
|
||||
<pre class="mb-3"><code class="language-bash"># Back up configuration
|
||||
cp -r data/.myfsio.sys/config/ config-backup/
|
||||
|
||||
# Back up data (optional, for critical deployments)
|
||||
tar czf myfsio-backup-$(date +%Y%m%d).tar.gz data/
|
||||
|
||||
# Back up logs
|
||||
cp -r logs/ logs-backup/</code></pre>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">Update Procedure</h3>
|
||||
<ol class="docs-steps mb-3">
|
||||
<li><strong>Stop the service:</strong> <code>sudo systemctl stop myfsio</code> (or kill the process)</li>
|
||||
<li><strong>Pull new version:</strong> <code>git pull origin main</code> or download the new binary</li>
|
||||
<li><strong>Install dependencies:</strong> <code>pip install -r requirements.txt</code></li>
|
||||
<li><strong>Validate config:</strong> <code>python run.py --check-config</code></li>
|
||||
<li><strong>Start the service:</strong> <code>sudo systemctl start myfsio</code></li>
|
||||
<li><strong>Verify:</strong> <code>curl http://localhost:5000/myfsio/health</code></li>
|
||||
</ol>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">Docker Update</h3>
|
||||
<pre class="mb-3"><code class="language-bash">docker pull myfsio:latest
|
||||
docker stop myfsio && docker rm myfsio
|
||||
docker run -d --name myfsio -v myfsio-data:/app/data -p 5000:5000 -p 5100:5100 myfsio:latest</code></pre>
|
||||
|
||||
<h3 class="h6 text-uppercase text-muted mt-4">Rollback</h3>
|
||||
<p class="small text-muted mb-0">If something goes wrong, stop the service, restore the backed-up config and data directories, then restart with the previous binary or code version. See <code>docs.md</code> Section 4 for detailed rollback procedures including blue-green deployment strategies.</p>
|
||||
</div>
|
||||
</article>
|
||||
<article id="api-matrix" class="card shadow-sm docs-section">
|
||||
<div class="card-body">
|
||||
<div class="d-flex align-items-center gap-2 mb-3">
|
||||
<span class="docs-section-kicker">30</span>
|
||||
<h2 class="h4 mb-0">Full API Reference</h2>
|
||||
</div>
|
||||
<p class="text-muted">Complete list of all S3-compatible, admin, and KMS endpoints.</p>
|
||||
<pre class="mb-0"><code class="language-text"># Service
|
||||
GET /myfsio/health # Health check
|
||||
|
||||
# Bucket Operations
|
||||
GET / # List buckets
|
||||
PUT /<bucket> # Create bucket
|
||||
DELETE /<bucket> # Delete bucket
|
||||
GET /<bucket> # List objects (?list-type=2)
|
||||
HEAD /<bucket> # Check bucket exists
|
||||
POST /<bucket> # POST object / form upload
|
||||
POST /<bucket>?delete # Bulk delete
|
||||
|
||||
# Bucket Configuration
|
||||
GET|PUT|DELETE /<bucket>?policy # Bucket policy
|
||||
GET|PUT /<bucket>?quota # Bucket quota
|
||||
GET|PUT /<bucket>?versioning # Versioning
|
||||
GET|PUT|DELETE /<bucket>?lifecycle # Lifecycle rules
|
||||
GET|PUT|DELETE /<bucket>?cors # CORS config
|
||||
GET|PUT|DELETE /<bucket>?encryption # Default encryption
|
||||
GET|PUT /<bucket>?acl # Bucket ACL
|
||||
GET|PUT|DELETE /<bucket>?tagging # Bucket tags
|
||||
GET|PUT|DELETE /<bucket>?replication # Replication rules
|
||||
GET|PUT /<bucket>?logging # Access logging
|
||||
GET|PUT /<bucket>?notification # Event notifications
|
||||
GET|PUT /<bucket>?object-lock # Object lock config
|
||||
GET|PUT|DELETE /<bucket>?website # Static website
|
||||
GET /<bucket>?uploads # List multipart uploads
|
||||
GET /<bucket>?versions # List object versions
|
||||
GET /<bucket>?location # Bucket region
|
||||
|
||||
# Object Operations
|
||||
PUT /<bucket>/<key> # Upload object
|
||||
GET /<bucket>/<key> # Download (Range supported)
|
||||
DELETE /<bucket>/<key> # Delete object
|
||||
HEAD /<bucket>/<key> # Object metadata
|
||||
POST /<bucket>/<key>?select # SQL query (SelectObjectContent)
|
||||
|
||||
# Object Configuration
|
||||
GET|PUT|DELETE /<bucket>/<key>?tagging # Object tags
|
||||
GET|PUT /<bucket>/<key>?acl # Object ACL
|
||||
GET|PUT /<bucket>/<key>?retention # Object retention
|
||||
GET|PUT /<bucket>/<key>?legal-hold # Legal hold
|
||||
|
||||
# Multipart Upload
|
||||
POST /<bucket>/<key>?uploads # Initiate
|
||||
PUT /<bucket>/<key>?uploadId=X&partNumber=N # Upload part
|
||||
POST /<bucket>/<key>?uploadId=X # Complete
|
||||
DELETE /<bucket>/<key>?uploadId=X # Abort
|
||||
GET /<bucket>/<key>?uploadId=X # List parts
|
||||
|
||||
# Copy (via x-amz-copy-source header)
|
||||
PUT /<bucket>/<key> # CopyObject
|
||||
PUT /<bucket>/<key>?uploadId&partNumber # UploadPartCopy
|
||||
|
||||
# Admin API
|
||||
GET|PUT /admin/site # Local site config
|
||||
GET /admin/sites # List peers
|
||||
POST /admin/sites # Register peer
|
||||
GET|PUT|DELETE /admin/sites/<id> # Manage peer
|
||||
GET /admin/sites/<id>/health # Peer health
|
||||
GET /admin/topology # Cluster topology
|
||||
GET|POST|PUT|DELETE /admin/website-domains # Domain mappings
|
||||
|
||||
# KMS API
|
||||
GET|POST /kms/keys # List / Create keys
|
||||
GET|DELETE /kms/keys/<id> # Get / Delete key
|
||||
POST /kms/keys/<id>/enable # Enable key
|
||||
POST /kms/keys/<id>/disable # Disable key
|
||||
POST /kms/keys/<id>/rotate # Rotate key
|
||||
POST /kms/encrypt # Encrypt data
|
||||
POST /kms/decrypt # Decrypt data
|
||||
POST /kms/generate-data-key # Generate data key
|
||||
POST /kms/generate-random # Generate random bytes</code></pre>
|
||||
</div>
|
||||
</article>
|
||||
</div>
|
||||
<div class="col-xl-4 docs-sidebar-col">
|
||||
<aside class="card shadow-sm docs-sidebar">
|
||||
@@ -2224,6 +2659,11 @@ server {
|
||||
<li><a href="#acls">Access Control Lists</a></li>
|
||||
<li><a href="#tagging">Object & Bucket Tagging</a></li>
|
||||
<li><a href="#website-hosting">Static Website Hosting</a></li>
|
||||
<li><a href="#cors-config">CORS Configuration</a></li>
|
||||
<li><a href="#post-object">PostObject (Form Upload)</a></li>
|
||||
<li><a href="#list-objects-v2">List Objects API v2</a></li>
|
||||
<li><a href="#upgrading">Upgrading & Updates</a></li>
|
||||
<li><a href="#api-matrix">Full API Reference</a></li>
|
||||
</ul>
|
||||
<div class="docs-sidebar-callouts">
|
||||
<div>
|
||||
|
||||
Reference in New Issue
Block a user