Merge pull request 'MyFSIO v0.2.2 Release' (#14) from next into main

Reviewed-on: #14

.dockerignore

@@ -11,7 +11,3 @@ htmlcov
 logs
 data
 tmp
-tests
-myfsio_core/target
-Dockerfile
-.dockerignore

.gitignore

@@ -26,13 +26,6 @@ dist/
 *.egg-info/
 .eggs/
 
-# Rust / maturin build artifacts
-python/myfsio_core/target/
-python/myfsio_core/Cargo.lock
-
-# Rust engine build artifacts
-rust/myfsio-engine/target/
-
 # Local runtime artifacts
 logs/
 *.log

Dockerfile

@@ -0,0 +1,37 @@
+# syntax=docker/dockerfile:1.7
+FROM python:3.12.12-slim
+
+ENV PYTHONDONTWRITEBYTECODE=1 \
+    PYTHONUNBUFFERED=1
+
+WORKDIR /app
+
+# Install build deps for any wheels that need compilation, then clean up
+RUN apt-get update \ 
+    && apt-get install -y --no-install-recommends build-essential \ 
+    && rm -rf /var/lib/apt/lists/*
+
+COPY requirements.txt ./
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY . .
+
+# Make entrypoint executable
+RUN chmod +x docker-entrypoint.sh
+
+# Create data directory and set permissions
+RUN mkdir -p /app/data \
+    && useradd -m -u 1000 myfsio \ 
+    && chown -R myfsio:myfsio /app
+
+USER myfsio
+
+EXPOSE 5000 5100
+ENV APP_HOST=0.0.0.0 \
+    FLASK_ENV=production \
+    FLASK_DEBUG=0
+
+HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
+    CMD python -c "import requests; requests.get('http://localhost:5000/myfsio/health', timeout=2)"
+
+CMD ["./docker-entrypoint.sh"]

README.md

@@ -1,212 +1,245 @@
 # MyFSIO
 
-MyFSIO is an S3-compatible object storage server with a Rust runtime and a filesystem-backed storage engine. The active server lives under `rust/myfsio-engine` and serves both the S3 API and the built-in web UI from a single process.
+A lightweight, S3-compatible object storage system built with Flask. MyFSIO implements core AWS S3 REST API operations with filesystem-backed storage, making it ideal for local development, testing, and self-hosted storage scenarios.
-
-The `python/` implementation is deprecated as of 2026-04-21. It remains in the repository for migration reference and legacy tests, but new development and supported runtime usage should target the Rust server.
 
 ## Features
 
-- S3-compatible REST API with Signature Version 4 authentication
+**Core Storage**
-- Browser UI for buckets, objects, IAM users, policies, replication, metrics, and site administration
+- S3-compatible REST API with AWS Signature Version 4 authentication
-- Filesystem-backed storage rooted at `data/`
+- Bucket and object CRUD operations
-- Bucket versioning, multipart uploads, presigned URLs, CORS, object and bucket tagging
+- Object versioning with version history
-- Server-side encryption and built-in KMS support
+- Multipart uploads for large files
-- Optional background services for lifecycle, garbage collection, integrity scanning, operation metrics, and system metrics history
+- Presigned URLs (1 second to 7 days validity)
-- Replication, site sync, and static website hosting support
 
-## Runtime Model
+**Security & Access Control**
+- IAM users with access key management and rotation
+- Bucket policies (AWS Policy Version 2012-10-17)
+- Server-side encryption (SSE-S3 and SSE-KMS)
+- Built-in Key Management Service (KMS)
+- Rate limiting per endpoint
 
-MyFSIO now runs as one Rust process:
+**Advanced Features**
+- Cross-bucket replication to remote S3-compatible endpoints
+- Hot-reload for bucket policies (no restart required)
+- CORS configuration per bucket
 
-- API listener on `HOST` + `PORT` (default `127.0.0.1:5000`)
+**Management UI**
-- UI listener on `HOST` + `UI_PORT` (default `127.0.0.1:5100`)
+- Web console for bucket and object management
-- Shared state for storage, IAM, policies, sessions, metrics, and background workers
+- IAM dashboard for user administration
+- Inline JSON policy editor with presets
+- Object browser with folder navigation and bulk operations
+- Dark mode support
 
-If you want API-only mode, set `UI_ENABLED=false`. There is no separate "UI-only" runtime anymore.
+## Architecture
+
+```
++------------------+         +------------------+
+|   API Server     |         |   UI Server      |
+|   (port 5000)    |         |   (port 5100)    |
+|                  |         |                  |
+|  - S3 REST API   |<------->|  - Web Console   |
+|  - SigV4 Auth    |         |  - IAM Dashboard |
+|  - Presign URLs  |         |  - Bucket Editor |
++--------+---------+         +------------------+
+         |
+         v
++------------------+         +------------------+
+| Object Storage   |         | System Metadata  |
+| (filesystem)     |         | (.myfsio.sys/)   |
+|                  |         |                  |
+| data/<bucket>/   |         | - IAM config     |
+|   <objects>      |         | - Bucket policies|
+|                  |         | - Encryption keys|
++------------------+         +------------------+
+```
 
 ## Quick Start
 
-From the repository root:
-
 ```bash
-cd rust/myfsio-engine
+# Clone and setup
-cargo run -p myfsio-server --
+git clone https://gitea.jzwsite.com/kqjy/MyFSIO
+cd s3
+python -m venv .venv
+
+# Activate virtual environment
+# Windows PowerShell:
+.\.venv\Scripts\Activate.ps1
+# Windows CMD:
+.venv\Scripts\activate.bat
+# Linux/macOS:
+source .venv/bin/activate
+
+# Install dependencies
+pip install -r requirements.txt
+
+# Start both servers
+python run.py
+
+# Or start individually
+python run.py --mode api   # API only (port 5000)
+python run.py --mode ui    # UI only (port 5100)
 ```
 
-Useful URLs:
+**Default Credentials:** `localadmin` / `localadmin`
 
-- UI: `http://127.0.0.1:5100/ui`
+- **Web Console:** http://127.0.0.1:5100/ui
-- API: `http://127.0.0.1:5000/`
+- **API Endpoint:** http://127.0.0.1:5000
-- Health: `http://127.0.0.1:5000/myfsio/health`
-
-On first boot, MyFSIO creates `data/.myfsio.sys/config/iam.json` and prints the generated admin access key and secret key to the console.
-
-### Common CLI commands
-
-```bash
-# Show resolved configuration
-cargo run -p myfsio-server -- --show-config
-
-# Validate configuration and exit non-zero on critical issues
-cargo run -p myfsio-server -- --check-config
-
-# Reset admin credentials
-cargo run -p myfsio-server -- --reset-cred
-
-# API only
-UI_ENABLED=false cargo run -p myfsio-server --
-```
-
-## Building a Binary
-
-```bash
-cd rust/myfsio-engine
-cargo build --release -p myfsio-server
-```
-
-Binary locations:
-
-- Linux/macOS: `rust/myfsio-engine/target/release/myfsio-server`
-- Windows: `rust/myfsio-engine/target/release/myfsio-server.exe`
-
-Run the built binary directly:
-
-```bash
-./target/release/myfsio-server
-```
 
 ## Configuration
 
-The server reads environment variables from the process environment and also loads, when present:
-
-- `/opt/myfsio/myfsio.env`
-- `.env`
-- `myfsio.env`
-
-Core settings:
-
 | Variable | Default | Description |
-| --- | --- | --- |
+|----------|---------|-------------|
-| `HOST` | `127.0.0.1` | Bind address for API and UI listeners |
+| `STORAGE_ROOT` | `./data` | Filesystem root for bucket storage |
-| `PORT` | `5000` | API port |
+| `IAM_CONFIG` | `.myfsio.sys/config/iam.json` | IAM user and policy store |
-| `UI_PORT` | `5100` | UI port |
+| `BUCKET_POLICY_PATH` | `.myfsio.sys/config/bucket_policies.json` | Bucket policy store |
-| `UI_ENABLED` | `true` | Disable to run API-only |
+| `API_BASE_URL` | `http://127.0.0.1:5000` | API endpoint for UI calls |
-| `STORAGE_ROOT` | `./data` | Root directory for buckets and system metadata |
+| `MAX_UPLOAD_SIZE` | `1073741824` | Maximum upload size in bytes (1 GB) |
-| `IAM_CONFIG` | `<STORAGE_ROOT>/.myfsio.sys/config/iam.json` | IAM config path |
+| `MULTIPART_MIN_PART_SIZE` | `5242880` | Minimum multipart part size (5 MB) |
-| `API_BASE_URL` | unset | Public API base used by the UI and presigned URL generation |
+| `UI_PAGE_SIZE` | `100` | Default page size for listings |
-| `AWS_REGION` | `us-east-1` | Region used in SigV4 scope |
+| `SECRET_KEY` | `dev-secret-key` | Flask session secret |
-| `SIGV4_TIMESTAMP_TOLERANCE_SECONDS` | `900` | Allowed request time skew |
+| `AWS_REGION` | `us-east-1` | Region for SigV4 signing |
-| `PRESIGNED_URL_MIN_EXPIRY_SECONDS` | `1` | Minimum presigned URL expiry |
+| `AWS_SERVICE` | `s3` | Service name for SigV4 signing |
-| `PRESIGNED_URL_MAX_EXPIRY_SECONDS` | `604800` | Maximum presigned URL expiry |
+| `ENCRYPTION_ENABLED` | `false` | Enable server-side encryption |
-| `SECRET_KEY` | loaded from `.myfsio.sys/config/.secret` if present | Session signing key and IAM-at-rest encryption key |
+| `KMS_ENABLED` | `false` | Enable Key Management Service |
-| `ADMIN_ACCESS_KEY` | unset | Optional first-run or reset access key |
+| `LOG_LEVEL` | `INFO` | Logging verbosity |
-| `ADMIN_SECRET_KEY` | unset | Optional first-run or reset secret key |
-
-Feature toggles:
-
-| Variable | Default |
-| --- | --- |
-| `ENCRYPTION_ENABLED` | `false` |
-| `KMS_ENABLED` | `false` |
-| `GC_ENABLED` | `false` |
-| `INTEGRITY_ENABLED` | `false` |
-| `LIFECYCLE_ENABLED` | `false` |
-| `METRICS_HISTORY_ENABLED` | `false` |
-| `OPERATION_METRICS_ENABLED` | `false` |
-| `WEBSITE_HOSTING_ENABLED` | `false` |
-| `SITE_SYNC_ENABLED` | `false` |
-
-Metrics and replication tuning:
-
-| Variable | Default |
-| --- | --- |
-| `OPERATION_METRICS_INTERVAL_MINUTES` | `5` |
-| `OPERATION_METRICS_RETENTION_HOURS` | `24` |
-| `METRICS_HISTORY_INTERVAL_MINUTES` | `5` |
-| `METRICS_HISTORY_RETENTION_HOURS` | `24` |
-| `REPLICATION_CONNECT_TIMEOUT_SECONDS` | `5` |
-| `REPLICATION_READ_TIMEOUT_SECONDS` | `30` |
-| `REPLICATION_MAX_RETRIES` | `2` |
-| `REPLICATION_STREAMING_THRESHOLD_BYTES` | `10485760` |
-| `REPLICATION_MAX_FAILURES_PER_BUCKET` | `50` |
-| `SITE_SYNC_INTERVAL_SECONDS` | `60` |
-| `SITE_SYNC_BATCH_SIZE` | `100` |
-| `SITE_SYNC_CONNECT_TIMEOUT_SECONDS` | `10` |
-| `SITE_SYNC_READ_TIMEOUT_SECONDS` | `120` |
-| `SITE_SYNC_MAX_RETRIES` | `2` |
-| `SITE_SYNC_CLOCK_SKEW_TOLERANCE_SECONDS` | `1.0` |
-
-UI asset overrides:
-
-| Variable | Default |
-| --- | --- |
-| `TEMPLATES_DIR` | built-in crate templates directory |
-| `STATIC_DIR` | built-in crate static directory |
-
-See [docs.md](./docs.md) for the full Rust-side operations guide.
 
 ## Data Layout
 
-```text
-data/
-  <bucket>/
-  .myfsio.sys/
-    config/
-      iam.json
-      bucket_policies.json
-      connections.json
-      operation_metrics.json
-      metrics_history.json
-    buckets/<bucket>/
-      meta/
-      versions/
-    multipart/
-    keys/
 ```
+data/
+├── <bucket>/                    # User buckets with objects
+└── .myfsio.sys/                 # System metadata
+    ├── config/
+    │   ├── iam.json             # IAM users and policies
+    │   ├── bucket_policies.json # Bucket policies
+    │   ├── replication_rules.json
+    │   └── connections.json     # Remote S3 connections
+    ├── buckets/<bucket>/
+    │   ├── meta/                # Object metadata (.meta.json)
+    │   ├── versions/            # Archived object versions
+    │   └── .bucket.json         # Bucket config (versioning, CORS)
+    ├── multipart/               # Active multipart uploads
+    └── keys/                    # Encryption keys (SSE-S3/KMS)
+```
+
+## API Reference
+
+All endpoints require AWS Signature Version 4 authentication unless using presigned URLs or public bucket policies.
+
+### Bucket Operations
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| `GET` | `/` | List all buckets |
+| `PUT` | `/<bucket>` | Create bucket |
+| `DELETE` | `/<bucket>` | Delete bucket (must be empty) |
+| `HEAD` | `/<bucket>` | Check bucket exists |
+
+### Object Operations
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| `GET` | `/<bucket>` | List objects (supports `list-type=2`) |
+| `PUT` | `/<bucket>/<key>` | Upload object |
+| `GET` | `/<bucket>/<key>` | Download object |
+| `DELETE` | `/<bucket>/<key>` | Delete object |
+| `HEAD` | `/<bucket>/<key>` | Get object metadata |
+| `POST` | `/<bucket>/<key>?uploads` | Initiate multipart upload |
+| `PUT` | `/<bucket>/<key>?partNumber=N&uploadId=X` | Upload part |
+| `POST` | `/<bucket>/<key>?uploadId=X` | Complete multipart upload |
+| `DELETE` | `/<bucket>/<key>?uploadId=X` | Abort multipart upload |
+
+### Bucket Policies (S3-compatible)
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| `GET` | `/<bucket>?policy` | Get bucket policy |
+| `PUT` | `/<bucket>?policy` | Set bucket policy |
+| `DELETE` | `/<bucket>?policy` | Delete bucket policy |
+
+### Versioning
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| `GET` | `/<bucket>/<key>?versionId=X` | Get specific version |
+| `DELETE` | `/<bucket>/<key>?versionId=X` | Delete specific version |
+| `GET` | `/<bucket>?versions` | List object versions |
+
+### Health Check
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| `GET` | `/myfsio/health` | Health check endpoint |
+
+## IAM & Access Control
+
+### Users and Access Keys
+
+On first run, MyFSIO creates a default admin user (`localadmin`/`localadmin`). Use the IAM dashboard to:
+
+- Create and delete users
+- Generate and rotate access keys
+- Attach inline policies to users
+- Control IAM management permissions
+
+### Bucket Policies
+
+Bucket policies follow AWS policy grammar (Version `2012-10-17`) with support for:
+
+- Principal-based access (`*` for anonymous, specific users)
+- Action-based permissions (`s3:GetObject`, `s3:PutObject`, etc.)
+- Resource patterns (`arn:aws:s3:::bucket/*`)
+- Condition keys
+
+**Policy Presets:**
+- **Public:** Grants anonymous read access (`s3:GetObject`, `s3:ListBucket`)
+- **Private:** Removes bucket policy (IAM-only access)
+- **Custom:** Manual policy editing with draft preservation
+
+Policies hot-reload when the JSON file changes.
+
+## Server-Side Encryption
+
+MyFSIO supports two encryption modes:
+
+- **SSE-S3:** Server-managed keys with automatic key rotation
+- **SSE-KMS:** Customer-managed keys via built-in KMS
+
+Enable encryption with:
+```bash
+ENCRYPTION_ENABLED=true python run.py
+```
+
+## Cross-Bucket Replication
+
+Replicate objects to remote S3-compatible endpoints:
+
+1. Configure remote connections in the UI
+2. Create replication rules specifying source/destination
+3. Objects are automatically replicated on upload
 
 ## Docker
 
-Build the Rust image from the `rust/` directory:
-
 ```bash
-docker build -t myfsio ./rust
+docker build -t myfsio .
-docker run --rm -p 5000:5000 -p 5100:5100 -v "${PWD}/data:/app/data" myfsio
+docker run -p 5000:5000 -p 5100:5100 -v ./data:/app/data myfsio
 ```
 
-If the instance sits behind a reverse proxy, set `API_BASE_URL` to the public S3 endpoint.
-
-## Linux Installation
-
-The repository includes `scripts/install.sh` for systemd-style Linux installs. Build the Rust binary first, then pass it to the installer:
-
-```bash
-cd rust/myfsio-engine
-cargo build --release -p myfsio-server
-
-cd ../..
-sudo ./scripts/install.sh --binary ./rust/myfsio-engine/target/release/myfsio-server
-```
-
-The installer copies the binary into `/opt/myfsio/myfsio`, writes `/opt/myfsio/myfsio.env`, and can register a `myfsio.service` unit.
-
 ## Testing
 
-Run the Rust test suite from the workspace:
-
 ```bash
-cd rust/myfsio-engine
+# Run all tests
-cargo test
+pytest tests/ -v
+
+# Run specific test file
+pytest tests/test_api.py -v
+
+# Run with coverage
+pytest tests/ --cov=app --cov-report=html
 ```
 
-## Health Check
+## References
 
-`GET /myfsio/health` returns:
+- [Amazon S3 Documentation](https://docs.aws.amazon.com/s3/)
-
+- [AWS Signature Version 4](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html)
-```json
+- [S3 Bucket Policy Examples](https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html)
-{
-  "status": "ok",
-  "version": "0.5.0"
-}
-```
-
-The `version` field comes from the Rust crate version in `rust/myfsio-engine/crates/myfsio-server/Cargo.toml`.

app/__init__.py

@@ -0,0 +1,408 @@
+from __future__ import annotations
+
+import logging
+import shutil
+import sys
+import time
+import uuid
+from logging.handlers import RotatingFileHandler
+from pathlib import Path
+from datetime import timedelta
+from typing import Any, Dict, List, Optional
+
+from flask import Flask, g, has_request_context, redirect, render_template, request, url_for
+from flask_cors import CORS
+from flask_wtf.csrf import CSRFError
+from werkzeug.middleware.proxy_fix import ProxyFix
+
+from .access_logging import AccessLoggingService
+from .operation_metrics import OperationMetricsCollector, classify_endpoint
+from .compression import GzipMiddleware
+from .acl import AclService
+from .bucket_policies import BucketPolicyStore
+from .config import AppConfig
+from .connections import ConnectionStore
+from .encryption import EncryptionManager
+from .extensions import limiter, csrf
+from .iam import IamService
+from .kms import KMSManager
+from .lifecycle import LifecycleManager
+from .notifications import NotificationService
+from .object_lock import ObjectLockService
+from .replication import ReplicationManager
+from .secret_store import EphemeralSecretStore
+from .storage import ObjectStorage
+from .version import get_version
+
+
+def _migrate_config_file(active_path: Path, legacy_paths: List[Path]) -> Path:
+    """Migrate config file from legacy locations to the active path.
+    
+    Checks each legacy path in order and moves the first one found to the active path.
+    This ensures backward compatibility for users upgrading from older versions.
+    """
+    active_path.parent.mkdir(parents=True, exist_ok=True)
+    
+    if active_path.exists():
+        return active_path
+    
+    for legacy_path in legacy_paths:
+        if legacy_path.exists():
+            try:
+                shutil.move(str(legacy_path), str(active_path))
+            except OSError:
+                shutil.copy2(legacy_path, active_path)
+                try:
+                    legacy_path.unlink(missing_ok=True)
+                except OSError:
+                    pass
+            break
+    
+    return active_path
+
+
+def create_app(
+    test_config: Optional[Dict[str, Any]] = None,
+    *,
+    include_api: bool = True,
+    include_ui: bool = True,
+) -> Flask:
+    """Create and configure the Flask application."""
+    config = AppConfig.from_env(test_config)
+
+    if getattr(sys, "frozen", False):
+        project_root = Path(sys._MEIPASS)
+    else:
+        project_root = Path(__file__).resolve().parent.parent
+
+    app = Flask(
+        __name__,
+        static_folder=str(project_root / "static"),
+        template_folder=str(project_root / "templates"),
+    )
+    app.config.update(config.to_flask_config())
+    if test_config:
+        app.config.update(test_config)
+    app.config.setdefault("APP_VERSION", get_version())
+    app.permanent_session_lifetime = timedelta(days=int(app.config.get("SESSION_LIFETIME_DAYS", 30)))
+    if app.config.get("TESTING"):
+        app.config.setdefault("WTF_CSRF_ENABLED", False)
+
+    # Trust X-Forwarded-* headers from proxies
+    app.wsgi_app = ProxyFix(app.wsgi_app, x_for=1, x_proto=1, x_host=1, x_prefix=1)
+
+    # Enable gzip compression for responses (10-20x smaller JSON payloads)
+    if app.config.get("ENABLE_GZIP", True):
+        app.wsgi_app = GzipMiddleware(app.wsgi_app, compression_level=6)
+
+    _configure_cors(app)
+    _configure_logging(app)
+
+    limiter.init_app(app)
+    csrf.init_app(app)
+
+    storage = ObjectStorage(
+        Path(app.config["STORAGE_ROOT"]),
+        cache_ttl=app.config.get("OBJECT_CACHE_TTL", 5),
+    )
+
+    if app.config.get("WARM_CACHE_ON_STARTUP", True) and not app.config.get("TESTING"):
+        storage.warm_cache_async()
+
+    iam = IamService(
+        Path(app.config["IAM_CONFIG"]),
+        auth_max_attempts=app.config.get("AUTH_MAX_ATTEMPTS", 5),
+        auth_lockout_minutes=app.config.get("AUTH_LOCKOUT_MINUTES", 15),
+    )
+    bucket_policies = BucketPolicyStore(Path(app.config["BUCKET_POLICY_PATH"]))
+    secret_store = EphemeralSecretStore(default_ttl=app.config.get("SECRET_TTL_SECONDS", 300))
+    
+    storage_root = Path(app.config["STORAGE_ROOT"])
+    config_dir = storage_root / ".myfsio.sys" / "config"
+    config_dir.mkdir(parents=True, exist_ok=True)
+    
+    connections_path = _migrate_config_file(
+        active_path=config_dir / "connections.json",
+        legacy_paths=[
+            storage_root / ".myfsio.sys" / "connections.json",
+            storage_root / ".connections.json",
+        ],
+    )
+    replication_rules_path = _migrate_config_file(
+        active_path=config_dir / "replication_rules.json",
+        legacy_paths=[
+            storage_root / ".myfsio.sys" / "replication_rules.json",
+            storage_root / ".replication_rules.json",
+        ],
+    )
+    
+    connections = ConnectionStore(connections_path)
+    replication = ReplicationManager(storage, connections, replication_rules_path, storage_root)
+    
+    encryption_config = {
+        "encryption_enabled": app.config.get("ENCRYPTION_ENABLED", False),
+        "encryption_master_key_path": app.config.get("ENCRYPTION_MASTER_KEY_PATH"),
+        "default_encryption_algorithm": app.config.get("DEFAULT_ENCRYPTION_ALGORITHM", "AES256"),
+    }
+    encryption_manager = EncryptionManager(encryption_config)
+    
+    kms_manager = None
+    if app.config.get("KMS_ENABLED", False):
+        kms_keys_path = Path(app.config.get("KMS_KEYS_PATH", ""))
+        kms_master_key_path = Path(app.config.get("ENCRYPTION_MASTER_KEY_PATH", ""))
+        kms_manager = KMSManager(kms_keys_path, kms_master_key_path)
+        encryption_manager.set_kms_provider(kms_manager)
+
+    if app.config.get("ENCRYPTION_ENABLED", False):
+        from .encrypted_storage import EncryptedObjectStorage
+        storage = EncryptedObjectStorage(storage, encryption_manager)
+
+    acl_service = AclService(storage_root)
+    object_lock_service = ObjectLockService(storage_root)
+    notification_service = NotificationService(storage_root)
+    access_logging_service = AccessLoggingService(storage_root)
+    access_logging_service.set_storage(storage)
+
+    lifecycle_manager = None
+    if app.config.get("LIFECYCLE_ENABLED", False):
+        base_storage = storage.storage if hasattr(storage, 'storage') else storage
+        lifecycle_manager = LifecycleManager(
+            base_storage,
+            interval_seconds=app.config.get("LIFECYCLE_INTERVAL_SECONDS", 3600),
+            storage_root=storage_root,
+        )
+        lifecycle_manager.start()
+
+    app.extensions["object_storage"] = storage
+    app.extensions["iam"] = iam
+    app.extensions["bucket_policies"] = bucket_policies
+    app.extensions["secret_store"] = secret_store
+    app.extensions["limiter"] = limiter
+    app.extensions["connections"] = connections
+    app.extensions["replication"] = replication
+    app.extensions["encryption"] = encryption_manager
+    app.extensions["kms"] = kms_manager
+    app.extensions["acl"] = acl_service
+    app.extensions["lifecycle"] = lifecycle_manager
+    app.extensions["object_lock"] = object_lock_service
+    app.extensions["notifications"] = notification_service
+    app.extensions["access_logging"] = access_logging_service
+
+    operation_metrics_collector = None
+    if app.config.get("OPERATION_METRICS_ENABLED", False):
+        operation_metrics_collector = OperationMetricsCollector(
+            storage_root,
+            interval_minutes=app.config.get("OPERATION_METRICS_INTERVAL_MINUTES", 5),
+            retention_hours=app.config.get("OPERATION_METRICS_RETENTION_HOURS", 24),
+        )
+    app.extensions["operation_metrics"] = operation_metrics_collector
+
+    @app.errorhandler(500)
+    def internal_error(error):
+        return render_template('500.html'), 500
+
+    @app.errorhandler(CSRFError)
+    def handle_csrf_error(e):
+        return render_template('csrf_error.html', reason=e.description), 400
+
+    @app.template_filter("filesizeformat")
+    def filesizeformat(value: int) -> str:
+        """Format bytes as human-readable file size."""
+        for unit in ["B", "KB", "MB", "GB", "TB", "PB"]:
+            if abs(value) < 1024.0 or unit == "PB":
+                if unit == "B":
+                    return f"{int(value)} {unit}"
+                return f"{value:.1f} {unit}"
+            value /= 1024.0
+        return f"{value:.1f} PB"
+
+    @app.template_filter("timestamp_to_datetime")
+    def timestamp_to_datetime(value: float) -> str:
+        """Format Unix timestamp as human-readable datetime in configured timezone."""
+        from datetime import datetime, timezone as dt_timezone
+        from zoneinfo import ZoneInfo
+        if not value:
+            return "Never"
+        try:
+            dt_utc = datetime.fromtimestamp(value, dt_timezone.utc)
+            display_tz = app.config.get("DISPLAY_TIMEZONE", "UTC")
+            if display_tz and display_tz != "UTC":
+                try:
+                    tz = ZoneInfo(display_tz)
+                    dt_local = dt_utc.astimezone(tz)
+                    return dt_local.strftime("%Y-%m-%d %H:%M:%S")
+                except (KeyError, ValueError):
+                    pass 
+            return dt_utc.strftime("%Y-%m-%d %H:%M:%S UTC")
+        except (ValueError, OSError):
+            return "Unknown"
+
+    @app.template_filter("format_datetime")
+    def format_datetime_filter(dt, include_tz: bool = True) -> str:
+        """Format datetime object as human-readable string in configured timezone."""
+        from datetime import datetime, timezone as dt_timezone
+        from zoneinfo import ZoneInfo
+        if not dt:
+            return ""
+        try:
+            display_tz = app.config.get("DISPLAY_TIMEZONE", "UTC")
+            if display_tz and display_tz != "UTC":
+                try:
+                    tz = ZoneInfo(display_tz)
+                    if dt.tzinfo is None:
+                        dt = dt.replace(tzinfo=dt_timezone.utc)
+                    dt = dt.astimezone(tz)
+                except (KeyError, ValueError):
+                    pass
+            tz_abbr = dt.strftime("%Z") or "UTC"
+            if include_tz:
+                return f"{dt.strftime('%b %d, %Y %H:%M')} ({tz_abbr})"
+            return dt.strftime("%b %d, %Y %H:%M")
+        except (ValueError, AttributeError):
+            return str(dt)
+
+    if include_api:
+        from .s3_api import s3_api_bp
+        from .kms_api import kms_api_bp
+
+        app.register_blueprint(s3_api_bp)
+        app.register_blueprint(kms_api_bp)
+        csrf.exempt(s3_api_bp)
+        csrf.exempt(kms_api_bp)
+
+    if include_ui:
+        from .ui import ui_bp
+
+        app.register_blueprint(ui_bp)
+        if not include_api:
+            @app.get("/")
+            def ui_root_redirect():
+                return redirect(url_for("ui.buckets_overview"))
+
+    @app.errorhandler(404)
+    def handle_not_found(error):
+        wants_html = request.accept_mimetypes.accept_html
+        path = request.path or ""
+        if include_ui and wants_html:
+            if not include_api or path.startswith("/ui") or path == "/":
+                return render_template("404.html"), 404
+        return error
+
+    @app.get("/myfsio/health")
+    def healthcheck() -> Dict[str, str]:
+        return {"status": "ok"}
+
+    return app
+
+
+def create_api_app(test_config: Optional[Dict[str, Any]] = None) -> Flask:
+    return create_app(test_config, include_api=True, include_ui=False)
+
+
+def create_ui_app(test_config: Optional[Dict[str, Any]] = None) -> Flask:
+    return create_app(test_config, include_api=False, include_ui=True)
+
+
+def _configure_cors(app: Flask) -> None:
+    origins = app.config.get("CORS_ORIGINS", ["*"])
+    methods = app.config.get("CORS_METHODS", ["GET", "PUT", "POST", "DELETE", "OPTIONS", "HEAD"])
+    allow_headers = app.config.get("CORS_ALLOW_HEADERS", ["*"])
+    expose_headers = app.config.get("CORS_EXPOSE_HEADERS", ["*"])
+    CORS(
+        app,
+        resources={r"/*": {"origins": origins, "methods": methods, "allow_headers": allow_headers, "expose_headers": expose_headers}},
+        supports_credentials=True,
+    )
+
+
+class _RequestContextFilter(logging.Filter):
+    """Inject request-specific attributes into log records."""
+
+    def filter(self, record: logging.LogRecord) -> bool:  
+        if has_request_context():
+            record.request_id = getattr(g, "request_id", "-")
+            record.path = request.path
+            record.method = request.method
+            record.remote_addr = request.remote_addr or "-"
+        else:
+            record.request_id = getattr(record, "request_id", "-")
+            record.path = getattr(record, "path", "-")
+            record.method = getattr(record, "method", "-")
+            record.remote_addr = getattr(record, "remote_addr", "-")
+        return True
+
+
+def _configure_logging(app: Flask) -> None:
+    formatter = logging.Formatter(
+        "%(asctime)s | %(levelname)s | %(request_id)s | %(method)s %(path)s | %(message)s"
+    )
+
+    stream_handler = logging.StreamHandler(sys.stdout)
+    stream_handler.setFormatter(formatter)
+    stream_handler.addFilter(_RequestContextFilter())
+
+    logger = app.logger
+    for handler in logger.handlers[:]:
+        handler.close()
+    logger.handlers.clear()
+    logger.addHandler(stream_handler)
+
+    if app.config.get("LOG_TO_FILE"):
+        log_file = Path(app.config["LOG_FILE"])
+        log_file.parent.mkdir(parents=True, exist_ok=True)
+        file_handler = RotatingFileHandler(
+            log_file,
+            maxBytes=int(app.config.get("LOG_MAX_BYTES", 5 * 1024 * 1024)),
+            backupCount=int(app.config.get("LOG_BACKUP_COUNT", 3)),
+            encoding="utf-8",
+        )
+        file_handler.setFormatter(formatter)
+        file_handler.addFilter(_RequestContextFilter())
+        logger.addHandler(file_handler)
+
+    logger.setLevel(getattr(logging, app.config.get("LOG_LEVEL", "INFO"), logging.INFO))
+
+    @app.before_request
+    def _log_request_start() -> None:
+        g.request_id = uuid.uuid4().hex
+        g.request_started_at = time.perf_counter()
+        g.request_bytes_in = request.content_length or 0
+        app.logger.info(
+            "Request started",
+            extra={"path": request.path, "method": request.method, "remote_addr": request.remote_addr},
+        )
+
+    @app.after_request
+    def _log_request_end(response):
+        duration_ms = 0.0
+        if hasattr(g, "request_started_at"):
+            duration_ms = (time.perf_counter() - g.request_started_at) * 1000
+        request_id = getattr(g, "request_id", uuid.uuid4().hex)
+        response.headers.setdefault("X-Request-ID", request_id)
+        app.logger.info(
+            "Request completed",
+            extra={
+                "path": request.path,
+                "method": request.method,
+                "remote_addr": request.remote_addr,
+            },
+        )
+        response.headers["X-Request-Duration-ms"] = f"{duration_ms:.2f}"
+
+        operation_metrics = app.extensions.get("operation_metrics")
+        if operation_metrics:
+            bytes_in = getattr(g, "request_bytes_in", 0)
+            bytes_out = response.content_length or 0
+            error_code = getattr(g, "s3_error_code", None)
+            endpoint_type = classify_endpoint(request.path)
+            operation_metrics.record_request(
+                method=request.method,
+                endpoint_type=endpoint_type,
+                status_code=response.status_code,
+                latency_ms=duration_ms,
+                bytes_in=bytes_in,
+                bytes_out=bytes_out,
+                error_code=error_code,
+            )
+
+        return response

app/bucket_policies.py

@@ -2,12 +2,10 @@ from __future__ import annotations
 
 import ipaddress
 import json
-import os
 import re
 import time
 from dataclasses import dataclass, field
 from fnmatch import fnmatch, translate
-from functools import lru_cache
 from pathlib import Path
 from typing import Any, Dict, Iterable, List, Optional, Pattern, Sequence, Tuple
 
@@ -15,14 +13,9 @@ from typing import Any, Dict, Iterable, List, Optional, Pattern, Sequence, Tuple
 RESOURCE_PREFIX = "arn:aws:s3:::"
 
 
-@lru_cache(maxsize=256)
-def _compile_pattern(pattern: str) -> Pattern[str]:
-    return re.compile(translate(pattern), re.IGNORECASE)
-
-
 def _match_string_like(value: str, pattern: str) -> bool:
-    compiled = _compile_pattern(pattern)
+    regex = translate(pattern)
-    return bool(compiled.match(value))
+    return bool(re.match(regex, value, re.IGNORECASE))
 
 
 def _ip_in_cidr(ip_str: str, cidr: str) -> bool:
@@ -76,7 +69,7 @@ def _evaluate_condition_operator(
         expected_null = condition_values[0].lower() in ("true", "1", "yes") if condition_values else True
         return is_null == expected_null
 
-    return False
+    return True
 
 ACTION_ALIASES = {
     "s3:listbucket": "list",
@@ -269,7 +262,7 @@ class BucketPolicyStore:
         self._last_mtime = self._current_mtime()
         # Performance: Avoid stat() on every request
         self._last_stat_check = 0.0
-        self._stat_check_interval = float(os.environ.get("BUCKET_POLICY_STAT_CHECK_INTERVAL_SECONDS", "2.0"))
+        self._stat_check_interval = 1.0  # Only check mtime every 1 second
 
     def maybe_reload(self) -> None:
         # Performance: Skip stat check if we checked recently

app/compression.py

@@ -36,11 +36,10 @@ class GzipMiddleware:
         content_type = None
         content_length = None
         should_compress = False
-        passthrough = False
         exc_info_holder = [None]
 
         def custom_start_response(status: str, headers: List[Tuple[str, str]], exc_info=None):
-            nonlocal response_started, status_code, response_headers, content_type, content_length, should_compress, passthrough
+            nonlocal response_started, status_code, response_headers, content_type, content_length, should_compress
             response_started = True
             status_code = int(status.split(' ', 1)[0])
             response_headers = list(headers)
@@ -51,32 +50,18 @@ class GzipMiddleware:
                 if name_lower == 'content-type':
                     content_type = value.split(';')[0].strip().lower()
                 elif name_lower == 'content-length':
-                    try:
                     content_length = int(value)
-                    except (ValueError, TypeError):
-                        pass
                 elif name_lower == 'content-encoding':
-                    passthrough = True
+                    should_compress = False
-                    return start_response(status, headers, exc_info)
-                elif name_lower == 'x-stream-response':
-                    passthrough = True
                     return start_response(status, headers, exc_info)
 
             if content_type and content_type in COMPRESSIBLE_MIMES:
                 if content_length is None or content_length >= self.min_size:
                     should_compress = True
-            else:
-                passthrough = True
-                return start_response(status, headers, exc_info)
 
             return None
 
-        app_iter = self.app(environ, custom_start_response)
+        response_body = b''.join(self.app(environ, custom_start_response))
-
-        if passthrough:
-            return app_iter
-
-        response_body = b''.join(app_iter)
 
         if not response_started:
             return [response_body]

app/config.py

@@ -0,0 +1,374 @@
+from __future__ import annotations
+
+import os
+import re
+import secrets
+import shutil
+import sys
+import warnings
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any, Dict, Optional
+
+
+def _validate_rate_limit(value: str) -> str:
+    pattern = r"^\d+\s+per\s+(second|minute|hour|day)$"
+    if not re.match(pattern, value):
+        raise ValueError(f"Invalid rate limit format: {value}. Expected format: '200 per minute'")
+    return value
+
+if getattr(sys, "frozen", False):
+    # Running in a PyInstaller bundle
+    PROJECT_ROOT = Path(sys._MEIPASS)
+else:
+    # Running in a normal Python environment
+    PROJECT_ROOT = Path(__file__).resolve().parent.parent
+
+
+def _prepare_config_file(active_path: Path, legacy_path: Optional[Path] = None) -> Path:
+    """Ensure config directories exist and migrate legacy files when possible."""
+    active_path = Path(active_path)
+    active_path.parent.mkdir(parents=True, exist_ok=True)
+    if legacy_path:
+        legacy_path = Path(legacy_path)
+        if not active_path.exists() and legacy_path.exists():
+            legacy_path.parent.mkdir(parents=True, exist_ok=True)
+            try:
+                shutil.move(str(legacy_path), str(active_path))
+            except OSError:
+                shutil.copy2(legacy_path, active_path)
+                try:
+                    legacy_path.unlink(missing_ok=True)
+                except OSError:
+                    pass
+    return active_path
+
+
+@dataclass
+class AppConfig:
+    storage_root: Path
+    max_upload_size: int
+    ui_page_size: int
+    secret_key: str
+    iam_config_path: Path
+    bucket_policy_path: Path
+    api_base_url: Optional[str]
+    aws_region: str
+    aws_service: str
+    ui_enforce_bucket_policies: bool
+    log_level: str
+    log_to_file: bool
+    log_path: Path
+    log_max_bytes: int
+    log_backup_count: int
+    ratelimit_default: str
+    ratelimit_storage_uri: str
+    cors_origins: list[str]
+    cors_methods: list[str]
+    cors_allow_headers: list[str]
+    cors_expose_headers: list[str]
+    session_lifetime_days: int
+    auth_max_attempts: int
+    auth_lockout_minutes: int
+    bulk_delete_max_keys: int
+    secret_ttl_seconds: int
+    stream_chunk_size: int
+    multipart_min_part_size: int
+    bucket_stats_cache_ttl: int
+    object_cache_ttl: int
+    encryption_enabled: bool
+    encryption_master_key_path: Path
+    kms_enabled: bool
+    kms_keys_path: Path
+    default_encryption_algorithm: str
+    display_timezone: str
+    lifecycle_enabled: bool
+    lifecycle_interval_seconds: int
+    metrics_history_enabled: bool
+    metrics_history_retention_hours: int
+    metrics_history_interval_minutes: int
+    operation_metrics_enabled: bool
+    operation_metrics_interval_minutes: int
+    operation_metrics_retention_hours: int
+
+    @classmethod
+    def from_env(cls, overrides: Optional[Dict[str, Any]] = None) -> "AppConfig":
+        overrides = overrides or {}
+
+        def _get(name: str, default: Any) -> Any:
+            return overrides.get(name, os.getenv(name, default))
+
+        storage_root = Path(_get("STORAGE_ROOT", PROJECT_ROOT / "data")).resolve()
+        max_upload_size = int(_get("MAX_UPLOAD_SIZE", 1024 * 1024 * 1024)) 
+        ui_page_size = int(_get("UI_PAGE_SIZE", 100))
+        auth_max_attempts = int(_get("AUTH_MAX_ATTEMPTS", 5))
+        auth_lockout_minutes = int(_get("AUTH_LOCKOUT_MINUTES", 15))
+        bulk_delete_max_keys = int(_get("BULK_DELETE_MAX_KEYS", 500))
+        secret_ttl_seconds = int(_get("SECRET_TTL_SECONDS", 300))
+        stream_chunk_size = int(_get("STREAM_CHUNK_SIZE", 64 * 1024))
+        multipart_min_part_size = int(_get("MULTIPART_MIN_PART_SIZE", 5 * 1024 * 1024))
+        lifecycle_enabled = _get("LIFECYCLE_ENABLED", "false").lower() in ("true", "1", "yes")
+        lifecycle_interval_seconds = int(_get("LIFECYCLE_INTERVAL_SECONDS", 3600))
+        default_secret = "dev-secret-key"
+        secret_key = str(_get("SECRET_KEY", default_secret))
+        
+        if not secret_key or secret_key == default_secret:
+            secret_file = storage_root / ".myfsio.sys" / "config" / ".secret"
+            if secret_file.exists():
+                secret_key = secret_file.read_text().strip()
+            else:
+                generated = secrets.token_urlsafe(32)
+                if secret_key == default_secret:
+                    warnings.warn("Using insecure default SECRET_KEY. A random value has been generated and persisted; set SECRET_KEY for production", RuntimeWarning)
+                try:
+                    secret_file.parent.mkdir(parents=True, exist_ok=True)
+                    secret_file.write_text(generated)
+                    try:
+                        os.chmod(secret_file, 0o600)
+                    except OSError:
+                        pass
+                    secret_key = generated
+                except OSError:
+                    secret_key = generated
+
+        iam_env_override = "IAM_CONFIG" in overrides or "IAM_CONFIG" in os.environ
+        bucket_policy_override = "BUCKET_POLICY_PATH" in overrides or "BUCKET_POLICY_PATH" in os.environ
+
+        default_iam_path = storage_root / ".myfsio.sys" / "config" / "iam.json"
+        default_bucket_policy_path = storage_root / ".myfsio.sys" / "config" / "bucket_policies.json"
+
+        iam_config_path = Path(_get("IAM_CONFIG", default_iam_path)).resolve()
+        bucket_policy_path = Path(_get("BUCKET_POLICY_PATH", default_bucket_policy_path)).resolve()
+
+        iam_config_path = _prepare_config_file(
+            iam_config_path,
+            legacy_path=None if iam_env_override else storage_root / "iam.json",
+        )
+        bucket_policy_path = _prepare_config_file(
+            bucket_policy_path,
+            legacy_path=None if bucket_policy_override else storage_root / "bucket_policies.json",
+        )
+        api_base_url = _get("API_BASE_URL", None)
+        if api_base_url:
+            api_base_url = str(api_base_url)
+        
+        aws_region = str(_get("AWS_REGION", "us-east-1"))
+        aws_service = str(_get("AWS_SERVICE", "s3"))
+        enforce_ui_policies = str(_get("UI_ENFORCE_BUCKET_POLICIES", "0")).lower() in {"1", "true", "yes", "on"}
+        log_level = str(_get("LOG_LEVEL", "INFO")).upper()
+        log_to_file = str(_get("LOG_TO_FILE", "1")).lower() in {"1", "true", "yes", "on"}
+        log_dir = Path(_get("LOG_DIR", storage_root.parent / "logs")).resolve()
+        log_dir.mkdir(parents=True, exist_ok=True)
+        log_path = log_dir / str(_get("LOG_FILE", "app.log"))
+        log_max_bytes = int(_get("LOG_MAX_BYTES", 5 * 1024 * 1024))
+        log_backup_count = int(_get("LOG_BACKUP_COUNT", 3))
+        ratelimit_default = _validate_rate_limit(str(_get("RATE_LIMIT_DEFAULT", "200 per minute")))
+        ratelimit_storage_uri = str(_get("RATE_LIMIT_STORAGE_URI", "memory://"))
+
+        def _csv(value: str, default: list[str]) -> list[str]:
+            if not value:
+                return default
+            parts = [segment.strip() for segment in value.split(",") if segment.strip()]
+            return parts or default
+
+        cors_origins = _csv(str(_get("CORS_ORIGINS", "*")), ["*"])
+        cors_methods = _csv(str(_get("CORS_METHODS", "GET,PUT,POST,DELETE,OPTIONS,HEAD")), ["GET", "PUT", "POST", "DELETE", "OPTIONS", "HEAD"])
+        cors_allow_headers = _csv(str(_get("CORS_ALLOW_HEADERS", "*")), ["*"])
+        cors_expose_headers = _csv(str(_get("CORS_EXPOSE_HEADERS", "*")), ["*"])
+        session_lifetime_days = int(_get("SESSION_LIFETIME_DAYS", 30))
+        bucket_stats_cache_ttl = int(_get("BUCKET_STATS_CACHE_TTL", 60))
+        object_cache_ttl = int(_get("OBJECT_CACHE_TTL", 5))
+
+        encryption_enabled = str(_get("ENCRYPTION_ENABLED", "0")).lower() in {"1", "true", "yes", "on"}
+        encryption_keys_dir = storage_root / ".myfsio.sys" / "keys"
+        encryption_master_key_path = Path(_get("ENCRYPTION_MASTER_KEY_PATH", encryption_keys_dir / "master.key")).resolve()
+        kms_enabled = str(_get("KMS_ENABLED", "0")).lower() in {"1", "true", "yes", "on"}
+        kms_keys_path = Path(_get("KMS_KEYS_PATH", encryption_keys_dir / "kms_keys.json")).resolve()
+        default_encryption_algorithm = str(_get("DEFAULT_ENCRYPTION_ALGORITHM", "AES256"))
+        display_timezone = str(_get("DISPLAY_TIMEZONE", "UTC"))
+        metrics_history_enabled = str(_get("METRICS_HISTORY_ENABLED", "0")).lower() in {"1", "true", "yes", "on"}
+        metrics_history_retention_hours = int(_get("METRICS_HISTORY_RETENTION_HOURS", 24))
+        metrics_history_interval_minutes = int(_get("METRICS_HISTORY_INTERVAL_MINUTES", 5))
+        operation_metrics_enabled = str(_get("OPERATION_METRICS_ENABLED", "0")).lower() in {"1", "true", "yes", "on"}
+        operation_metrics_interval_minutes = int(_get("OPERATION_METRICS_INTERVAL_MINUTES", 5))
+        operation_metrics_retention_hours = int(_get("OPERATION_METRICS_RETENTION_HOURS", 24))
+
+        return cls(storage_root=storage_root,
+                   max_upload_size=max_upload_size,
+                   ui_page_size=ui_page_size,
+                   secret_key=secret_key,
+                   iam_config_path=iam_config_path,
+                   bucket_policy_path=bucket_policy_path,
+                   api_base_url=api_base_url,
+                   aws_region=aws_region,
+                   aws_service=aws_service,
+                   ui_enforce_bucket_policies=enforce_ui_policies,
+                   log_level=log_level,
+                   log_to_file=log_to_file,
+                   log_path=log_path,
+                   log_max_bytes=log_max_bytes,
+                   log_backup_count=log_backup_count,
+                   ratelimit_default=ratelimit_default,
+                   ratelimit_storage_uri=ratelimit_storage_uri,
+                   cors_origins=cors_origins,
+                   cors_methods=cors_methods,
+                   cors_allow_headers=cors_allow_headers,
+                   cors_expose_headers=cors_expose_headers,
+                   session_lifetime_days=session_lifetime_days,
+                   auth_max_attempts=auth_max_attempts,
+                   auth_lockout_minutes=auth_lockout_minutes,
+                   bulk_delete_max_keys=bulk_delete_max_keys,
+                   secret_ttl_seconds=secret_ttl_seconds,
+                   stream_chunk_size=stream_chunk_size,
+                   multipart_min_part_size=multipart_min_part_size,
+                   bucket_stats_cache_ttl=bucket_stats_cache_ttl,
+                   object_cache_ttl=object_cache_ttl,
+                   encryption_enabled=encryption_enabled,
+                   encryption_master_key_path=encryption_master_key_path,
+                   kms_enabled=kms_enabled,
+                   kms_keys_path=kms_keys_path,
+                   default_encryption_algorithm=default_encryption_algorithm,
+                   display_timezone=display_timezone,
+                   lifecycle_enabled=lifecycle_enabled,
+                   lifecycle_interval_seconds=lifecycle_interval_seconds,
+                   metrics_history_enabled=metrics_history_enabled,
+                   metrics_history_retention_hours=metrics_history_retention_hours,
+                   metrics_history_interval_minutes=metrics_history_interval_minutes,
+                   operation_metrics_enabled=operation_metrics_enabled,
+                   operation_metrics_interval_minutes=operation_metrics_interval_minutes,
+                   operation_metrics_retention_hours=operation_metrics_retention_hours)
+
+    def validate_and_report(self) -> list[str]:
+        """Validate configuration and return a list of warnings/issues.
+        
+        Call this at startup to detect potential misconfigurations before
+        the application fully commits to running.
+        """
+        issues = []
+        
+        try:
+            test_file = self.storage_root / ".write_test"
+            test_file.touch()
+            test_file.unlink()
+        except (OSError, PermissionError) as e:
+            issues.append(f"CRITICAL: STORAGE_ROOT '{self.storage_root}' is not writable: {e}")
+        
+        storage_str = str(self.storage_root).lower()
+        if "/tmp" in storage_str or "\\temp" in storage_str or "appdata\\local\\temp" in storage_str:
+            issues.append(f"WARNING: STORAGE_ROOT '{self.storage_root}' appears to be a temporary directory. Data may be lost on reboot!")
+        
+        try:
+            self.iam_config_path.relative_to(self.storage_root)
+        except ValueError:
+            issues.append(f"WARNING: IAM_CONFIG '{self.iam_config_path}' is outside STORAGE_ROOT '{self.storage_root}'. Consider setting IAM_CONFIG explicitly or ensuring paths are aligned.")
+        
+        try:
+            self.bucket_policy_path.relative_to(self.storage_root)
+        except ValueError:
+            issues.append(f"WARNING: BUCKET_POLICY_PATH '{self.bucket_policy_path}' is outside STORAGE_ROOT '{self.storage_root}'. Consider setting BUCKET_POLICY_PATH explicitly.")
+        
+        try:
+            self.log_path.parent.mkdir(parents=True, exist_ok=True)
+            test_log = self.log_path.parent / ".write_test"
+            test_log.touch()
+            test_log.unlink()
+        except (OSError, PermissionError) as e:
+            issues.append(f"WARNING: Log directory '{self.log_path.parent}' is not writable: {e}")
+        
+        log_str = str(self.log_path).lower()
+        if "/tmp" in log_str or "\\temp" in log_str or "appdata\\local\\temp" in log_str:
+            issues.append(f"WARNING: LOG_DIR '{self.log_path.parent}' appears to be a temporary directory. Logs may be lost on reboot!")
+        
+        if self.encryption_enabled:
+            try:
+                self.encryption_master_key_path.relative_to(self.storage_root)
+            except ValueError:
+                issues.append(f"WARNING: ENCRYPTION_MASTER_KEY_PATH '{self.encryption_master_key_path}' is outside STORAGE_ROOT. Ensure proper backup procedures.")
+        
+        if self.kms_enabled:
+            try:
+                self.kms_keys_path.relative_to(self.storage_root)
+            except ValueError:
+                issues.append(f"WARNING: KMS_KEYS_PATH '{self.kms_keys_path}' is outside STORAGE_ROOT. Ensure proper backup procedures.")
+        
+        if self.secret_key == "dev-secret-key":
+            issues.append("WARNING: Using default SECRET_KEY. Set SECRET_KEY environment variable for production.")
+        
+        if "*" in self.cors_origins:
+            issues.append("INFO: CORS_ORIGINS is set to '*'. Consider restricting to specific domains in production.")
+        
+        return issues
+
+    def print_startup_summary(self) -> None:
+        """Print a summary of the configuration at startup."""
+        print("\n" + "=" * 60)
+        print("MyFSIO Configuration Summary")
+        print("=" * 60)
+        print(f"  STORAGE_ROOT:     {self.storage_root}")
+        print(f"  IAM_CONFIG:       {self.iam_config_path}")
+        print(f"  BUCKET_POLICY:    {self.bucket_policy_path}")
+        print(f"  LOG_PATH:         {self.log_path}")
+        if self.api_base_url:
+            print(f"  API_BASE_URL:     {self.api_base_url}")
+        if self.encryption_enabled:
+            print(f"  ENCRYPTION:       Enabled (Master key: {self.encryption_master_key_path})")
+        if self.kms_enabled:
+            print(f"  KMS:              Enabled (Keys: {self.kms_keys_path})")
+        print("=" * 60)
+        
+        issues = self.validate_and_report()
+        if issues:
+            print("\nConfiguration Issues Detected:")
+            for issue in issues:
+                print(f"  • {issue}")
+            print()
+        else:
+            print("  ✓ Configuration validated successfully\n")
+
+    def to_flask_config(self) -> Dict[str, Any]:
+        return {
+            "STORAGE_ROOT": str(self.storage_root),
+            "MAX_CONTENT_LENGTH": self.max_upload_size,
+            "UI_PAGE_SIZE": self.ui_page_size,
+            "SECRET_KEY": self.secret_key,
+            "IAM_CONFIG": str(self.iam_config_path),
+            "BUCKET_POLICY_PATH": str(self.bucket_policy_path),
+            "API_BASE_URL": self.api_base_url,
+            "AWS_REGION": self.aws_region,
+            "AWS_SERVICE": self.aws_service,
+            "UI_ENFORCE_BUCKET_POLICIES": self.ui_enforce_bucket_policies,
+            "AUTH_MAX_ATTEMPTS": self.auth_max_attempts,
+            "AUTH_LOCKOUT_MINUTES": self.auth_lockout_minutes,
+            "BULK_DELETE_MAX_KEYS": self.bulk_delete_max_keys,
+            "SECRET_TTL_SECONDS": self.secret_ttl_seconds,
+            "STREAM_CHUNK_SIZE": self.stream_chunk_size,
+            "MULTIPART_MIN_PART_SIZE": self.multipart_min_part_size,
+            "BUCKET_STATS_CACHE_TTL": self.bucket_stats_cache_ttl,
+            "OBJECT_CACHE_TTL": self.object_cache_ttl,
+            "LOG_LEVEL": self.log_level,
+            "LOG_TO_FILE": self.log_to_file,
+            "LOG_FILE": str(self.log_path),
+            "LOG_MAX_BYTES": self.log_max_bytes,
+            "LOG_BACKUP_COUNT": self.log_backup_count,
+            "RATELIMIT_DEFAULT": self.ratelimit_default,
+            "RATELIMIT_STORAGE_URI": self.ratelimit_storage_uri,
+            "CORS_ORIGINS": self.cors_origins,
+            "CORS_METHODS": self.cors_methods,
+            "CORS_ALLOW_HEADERS": self.cors_allow_headers,
+            "CORS_EXPOSE_HEADERS": self.cors_expose_headers,
+            "SESSION_LIFETIME_DAYS": self.session_lifetime_days,
+            "ENCRYPTION_ENABLED": self.encryption_enabled,
+            "ENCRYPTION_MASTER_KEY_PATH": str(self.encryption_master_key_path),
+            "KMS_ENABLED": self.kms_enabled,
+            "KMS_KEYS_PATH": str(self.kms_keys_path),
+            "DEFAULT_ENCRYPTION_ALGORITHM": self.default_encryption_algorithm,
+            "DISPLAY_TIMEZONE": self.display_timezone,
+            "LIFECYCLE_ENABLED": self.lifecycle_enabled,
+            "LIFECYCLE_INTERVAL_SECONDS": self.lifecycle_interval_seconds,
+            "METRICS_HISTORY_ENABLED": self.metrics_history_enabled,
+            "METRICS_HISTORY_RETENTION_HOURS": self.metrics_history_retention_hours,
+            "METRICS_HISTORY_INTERVAL_MINUTES": self.metrics_history_interval_minutes,
+            "OPERATION_METRICS_ENABLED": self.operation_metrics_enabled,
+            "OPERATION_METRICS_INTERVAL_MINUTES": self.operation_metrics_interval_minutes,
+            "OPERATION_METRICS_RETENTION_HOURS": self.operation_metrics_retention_hours,
+        }

app/encrypted_storage.py

@@ -190,15 +190,6 @@ class EncryptedObjectStorage:
     def list_objects(self, bucket_name: str, **kwargs):
         return self.storage.list_objects(bucket_name, **kwargs)
     
-    def list_objects_shallow(self, bucket_name: str, **kwargs):
-        return self.storage.list_objects_shallow(bucket_name, **kwargs)
-
-    def iter_objects_shallow(self, bucket_name: str, **kwargs):
-        return self.storage.iter_objects_shallow(bucket_name, **kwargs)
-
-    def search_objects(self, bucket_name: str, query: str, **kwargs):
-        return self.storage.search_objects(bucket_name, query, **kwargs)
-
     def list_objects_all(self, bucket_name: str):
         return self.storage.list_objects_all(bucket_name)
     
@@ -283,11 +274,5 @@ class EncryptedObjectStorage:
     def set_bucket_quota(self, bucket_name: str, *, max_bytes=None, max_objects=None):
         return self.storage.set_bucket_quota(bucket_name, max_bytes=max_bytes, max_objects=max_objects)
     
-    def get_bucket_website(self, bucket_name: str):
-        return self.storage.get_bucket_website(bucket_name)
-
-    def set_bucket_website(self, bucket_name: str, website_config):
-        return self.storage.set_bucket_website(bucket_name, website_config)
-    
     def _compute_etag(self, path: Path) -> str:
         return self.storage._compute_etag(path)

app/encryption.py

@@ -1,55 +1,15 @@
+"""Encryption providers for server-side and client-side encryption."""
 from __future__ import annotations
 
 import base64
 import io
 import json
-import logging
-import os
 import secrets
-import subprocess
-import sys
 from dataclasses import dataclass
 from pathlib import Path
 from typing import Any, BinaryIO, Dict, Generator, Optional
 
 from cryptography.hazmat.primitives.ciphers.aead import AESGCM
-from cryptography.hazmat.primitives.kdf.hkdf import HKDF
-from cryptography.hazmat.primitives import hashes
-
-if sys.platform != "win32":
-    import fcntl
-
-try:
-    import myfsio_core as _rc
-    if not all(hasattr(_rc, f) for f in (
-        "encrypt_stream_chunked", "decrypt_stream_chunked",
-    )):
-        raise ImportError("myfsio_core is outdated, rebuild with: cd myfsio_core && maturin develop --release")
-    _HAS_RUST = True
-except ImportError:
-    _rc = None
-    _HAS_RUST = False
-
-logger = logging.getLogger(__name__)
-
-
-def _set_secure_file_permissions(file_path: Path) -> None:
-    """Set restrictive file permissions (owner read/write only)."""
-    if sys.platform == "win32":
-        try:
-            username = os.environ.get("USERNAME", "")
-            if username:
-                subprocess.run(
-                    ["icacls", str(file_path), "/inheritance:r",
-                     "/grant:r", f"{username}:F"],
-                    check=True, capture_output=True
-                )
-            else:
-                logger.warning("Could not set secure permissions on %s: USERNAME not set", file_path)
-        except (subprocess.SubprocessError, OSError) as exc:
-            logger.warning("Failed to set secure permissions on %s: %s", file_path, exc)
-    else:
-        os.chmod(file_path, 0o600)
 
 
 class EncryptionError(Exception):
@@ -115,18 +75,6 @@ class EncryptionProvider:
         """
         raise NotImplementedError
 
-    def decrypt_data_key(self, encrypted_data_key: bytes, key_id: str | None = None) -> bytes:
-        """Decrypt an encrypted data key.
-
-        Args:
-            encrypted_data_key: The encrypted data key bytes
-            key_id: Optional key identifier (used by KMS providers)
-
-        Returns:
-            The decrypted data key
-        """
-        raise NotImplementedError
-
 
 class LocalKeyEncryption(EncryptionProvider):
     """SSE-S3 style encryption using a local master key.
@@ -151,46 +99,26 @@ class LocalKeyEncryption(EncryptionProvider):
         return self._master_key
     
     def _load_or_create_master_key(self) -> bytes:
-        """Load master key from file or generate a new one (with file locking)."""
+        """Load master key from file or generate a new one."""
-        lock_path = self.master_key_path.with_suffix(".lock")
-        lock_path.parent.mkdir(parents=True, exist_ok=True)
-
-        try:
-            with open(lock_path, "w") as lock_file:
-                if sys.platform == "win32":
-                    import msvcrt
-                    msvcrt.locking(lock_file.fileno(), msvcrt.LK_LOCK, 1)
-                else:
-                    fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX)
-                try:
         if self.master_key_path.exists():
             try:
                 return base64.b64decode(self.master_key_path.read_text().strip())
             except Exception as exc:
                 raise EncryptionError(f"Failed to load master key: {exc}") from exc
+        
         key = secrets.token_bytes(32)
         try:
+            self.master_key_path.parent.mkdir(parents=True, exist_ok=True)
             self.master_key_path.write_text(base64.b64encode(key).decode())
-                        _set_secure_file_permissions(self.master_key_path)
         except OSError as exc:
             raise EncryptionError(f"Failed to save master key: {exc}") from exc
         return key
-                finally:
-                    if sys.platform == "win32":
-                        import msvcrt
-                        msvcrt.locking(lock_file.fileno(), msvcrt.LK_UNLCK, 1)
-                    else:
-                        fcntl.flock(lock_file.fileno(), fcntl.LOCK_UN)
-        except OSError as exc:
-            raise EncryptionError(f"Failed to acquire lock for master key: {exc}") from exc
-    
-    DATA_KEY_AAD = b'{"purpose":"data_key","version":1}'
     
     def _encrypt_data_key(self, data_key: bytes) -> bytes:
         """Encrypt the data key with the master key."""
         aesgcm = AESGCM(self.master_key)
         nonce = secrets.token_bytes(12)
-        encrypted = aesgcm.encrypt(nonce, data_key, self.DATA_KEY_AAD)
+        encrypted = aesgcm.encrypt(nonce, data_key, None)
         return nonce + encrypted
     
     def _decrypt_data_key(self, encrypted_data_key: bytes) -> bytes:
@@ -200,18 +128,11 @@ class LocalKeyEncryption(EncryptionProvider):
         aesgcm = AESGCM(self.master_key)
         nonce = encrypted_data_key[:12]
         ciphertext = encrypted_data_key[12:]
-        try:
-            return aesgcm.decrypt(nonce, ciphertext, self.DATA_KEY_AAD)
-        except Exception:
         try:
             return aesgcm.decrypt(nonce, ciphertext, None)
         except Exception as exc:
             raise EncryptionError(f"Failed to decrypt data key: {exc}") from exc
     
-    def decrypt_data_key(self, encrypted_data_key: bytes, key_id: str | None = None) -> bytes:
-        """Decrypt an encrypted data key (key_id ignored for local encryption)."""
-        return self._decrypt_data_key(encrypted_data_key)
-
     def generate_data_key(self) -> tuple[bytes, bytes]:
         """Generate a data key and its encrypted form."""
         plaintext_key = secrets.token_bytes(32)
@@ -224,8 +145,7 @@ class LocalKeyEncryption(EncryptionProvider):
         
         aesgcm = AESGCM(data_key)
         nonce = secrets.token_bytes(12)
-        aad = json.dumps(context, sort_keys=True).encode() if context else None
+        ciphertext = aesgcm.encrypt(nonce, plaintext, None)
-        ciphertext = aesgcm.encrypt(nonce, plaintext, aad)
         
         return EncryptionResult(
             ciphertext=ciphertext,
@@ -239,11 +159,10 @@ class LocalKeyEncryption(EncryptionProvider):
         """Decrypt data using envelope encryption."""
         data_key = self._decrypt_data_key(encrypted_data_key)
         aesgcm = AESGCM(data_key)
-        aad = json.dumps(context, sort_keys=True).encode() if context else None
         try:
-            return aesgcm.decrypt(nonce, ciphertext, aad)
+            return aesgcm.decrypt(nonce, ciphertext, None)
         except Exception as exc:
-            raise EncryptionError("Failed to decrypt data") from exc
+            raise EncryptionError(f"Failed to decrypt data: {exc}") from exc
 
 
 class StreamingEncryptor:
@@ -261,14 +180,12 @@ class StreamingEncryptor:
         self.chunk_size = chunk_size
     
     def _derive_chunk_nonce(self, base_nonce: bytes, chunk_index: int) -> bytes:
-        """Derive a unique nonce for each chunk using HKDF."""
+        """Derive a unique nonce for each chunk.
-        hkdf = HKDF(
+
-            algorithm=hashes.SHA256(),
+        Performance: Use direct byte manipulation instead of full int conversion.
-            length=12,
+        """
-            salt=base_nonce,
+        # Performance: Only modify last 4 bytes instead of full 12-byte conversion
-            info=chunk_index.to_bytes(4, "big"),
+        return base_nonce[:8] + (chunk_index ^ int.from_bytes(base_nonce[8:], "big")).to_bytes(4, "big")
-        )
-        return hkdf.derive(b"chunk_nonce")
 
     def encrypt_stream(self, stream: BinaryIO,
                        context: Dict[str, str] | None = None) -> tuple[BinaryIO, EncryptionMetadata]:
@@ -317,7 +234,10 @@ class StreamingEncryptor:
 
         Performance: Writes chunks directly to output buffer instead of accumulating in list.
         """
-        data_key = self.provider.decrypt_data_key(metadata.encrypted_data_key, metadata.key_id)
+        if isinstance(self.provider, LocalKeyEncryption):
+            data_key = self.provider._decrypt_data_key(metadata.encrypted_data_key)
+        else:
+            raise EncryptionError("Unsupported provider for streaming decryption")
 
         aesgcm = AESGCM(data_key)
         base_nonce = metadata.nonce
@@ -349,69 +269,6 @@ class StreamingEncryptor:
         output.seek(0)
         return output
 
-    def encrypt_file(self, input_path: str, output_path: str) -> EncryptionMetadata:
-        data_key, encrypted_data_key = self.provider.generate_data_key()
-        base_nonce = secrets.token_bytes(12)
-
-        if _HAS_RUST:
-            _rc.encrypt_stream_chunked(
-                input_path, output_path, data_key, base_nonce, self.chunk_size
-            )
-        else:
-            with open(input_path, "rb") as stream:
-                aesgcm = AESGCM(data_key)
-                with open(output_path, "wb") as out:
-                    out.write(b"\x00\x00\x00\x00")
-                    chunk_index = 0
-                    while True:
-                        chunk = stream.read(self.chunk_size)
-                        if not chunk:
-                            break
-                        chunk_nonce = self._derive_chunk_nonce(base_nonce, chunk_index)
-                        encrypted_chunk = aesgcm.encrypt(chunk_nonce, chunk, None)
-                        out.write(len(encrypted_chunk).to_bytes(self.HEADER_SIZE, "big"))
-                        out.write(encrypted_chunk)
-                        chunk_index += 1
-                    out.seek(0)
-                    out.write(chunk_index.to_bytes(4, "big"))
-
-        return EncryptionMetadata(
-            algorithm="AES256",
-            key_id=self.provider.KEY_ID if hasattr(self.provider, "KEY_ID") else "local",
-            nonce=base_nonce,
-            encrypted_data_key=encrypted_data_key,
-        )
-
-    def decrypt_file(self, input_path: str, output_path: str,
-                     metadata: EncryptionMetadata) -> None:
-        data_key = self.provider.decrypt_data_key(metadata.encrypted_data_key, metadata.key_id)
-        base_nonce = metadata.nonce
-
-        if _HAS_RUST:
-            _rc.decrypt_stream_chunked(input_path, output_path, data_key, base_nonce)
-        else:
-            with open(input_path, "rb") as stream:
-                chunk_count_bytes = stream.read(4)
-                if len(chunk_count_bytes) < 4:
-                    raise EncryptionError("Invalid encrypted stream: missing header")
-                chunk_count = int.from_bytes(chunk_count_bytes, "big")
-                aesgcm = AESGCM(data_key)
-                with open(output_path, "wb") as out:
-                    for chunk_index in range(chunk_count):
-                        size_bytes = stream.read(self.HEADER_SIZE)
-                        if len(size_bytes) < self.HEADER_SIZE:
-                            raise EncryptionError(f"Invalid encrypted stream: truncated at chunk {chunk_index}")
-                        chunk_size = int.from_bytes(size_bytes, "big")
-                        encrypted_chunk = stream.read(chunk_size)
-                        if len(encrypted_chunk) < chunk_size:
-                            raise EncryptionError(f"Invalid encrypted stream: incomplete chunk {chunk_index}")
-                        chunk_nonce = self._derive_chunk_nonce(base_nonce, chunk_index)
-                        try:
-                            decrypted_chunk = aesgcm.decrypt(chunk_nonce, encrypted_chunk, None)
-                            out.write(decrypted_chunk)
-                        except Exception as exc:
-                            raise EncryptionError(f"Failed to decrypt chunk {chunk_index}: {exc}") from exc
-
 
 class EncryptionManager:
     """Manages encryption providers and operations."""
@@ -453,8 +310,7 @@ class EncryptionManager:
     
     def get_streaming_encryptor(self) -> StreamingEncryptor:
         if self._streaming_encryptor is None:
-            chunk_size = self.config.get("encryption_chunk_size_bytes", 64 * 1024)
+            self._streaming_encryptor = StreamingEncryptor(self.get_local_provider())
-            self._streaming_encryptor = StreamingEncryptor(self.get_local_provider(), chunk_size=chunk_size)
         return self._streaming_encryptor
     
     def encrypt_object(self, data: bytes, algorithm: str = "AES256",
@@ -547,8 +403,7 @@ class SSECEncryption(EncryptionProvider):
     def encrypt(self, plaintext: bytes, context: Dict[str, str] | None = None) -> EncryptionResult:
         aesgcm = AESGCM(self.customer_key)
         nonce = secrets.token_bytes(12)
-        aad = json.dumps(context, sort_keys=True).encode() if context else None
+        ciphertext = aesgcm.encrypt(nonce, plaintext, None)
-        ciphertext = aesgcm.encrypt(nonce, plaintext, aad)
 
         return EncryptionResult(
             ciphertext=ciphertext,
@@ -560,11 +415,10 @@ class SSECEncryption(EncryptionProvider):
     def decrypt(self, ciphertext: bytes, nonce: bytes, encrypted_data_key: bytes,
                 key_id: str, context: Dict[str, str] | None = None) -> bytes:
         aesgcm = AESGCM(self.customer_key)
-        aad = json.dumps(context, sort_keys=True).encode() if context else None
         try:
-            return aesgcm.decrypt(nonce, ciphertext, aad)
+            return aesgcm.decrypt(nonce, ciphertext, None)
         except Exception as exc:
-            raise EncryptionError("SSE-C decryption failed") from exc
+            raise EncryptionError(f"SSE-C decryption failed: {exc}") from exc
 
     def generate_data_key(self) -> tuple[bytes, bytes]:
         return self.customer_key, b""
@@ -618,7 +472,7 @@ class ClientEncryptionHelper:
         }
     
     @staticmethod
-    def encrypt_with_key(plaintext: bytes, key_b64: str, context: Dict[str, str] | None = None) -> Dict[str, str]:
+    def encrypt_with_key(plaintext: bytes, key_b64: str) -> Dict[str, str]:
         """Encrypt data with a client-provided key."""
         key = base64.b64decode(key_b64)
         if len(key) != 32:
@@ -626,8 +480,7 @@ class ClientEncryptionHelper:
         
         aesgcm = AESGCM(key)
         nonce = secrets.token_bytes(12)
-        aad = json.dumps(context, sort_keys=True).encode() if context else None
+        ciphertext = aesgcm.encrypt(nonce, plaintext, None)
-        ciphertext = aesgcm.encrypt(nonce, plaintext, aad)
         
         return {
             "ciphertext": base64.b64encode(ciphertext).decode(),
@@ -636,7 +489,7 @@ class ClientEncryptionHelper:
         }
     
     @staticmethod
-    def decrypt_with_key(ciphertext_b64: str, nonce_b64: str, key_b64: str, context: Dict[str, str] | None = None) -> bytes:
+    def decrypt_with_key(ciphertext_b64: str, nonce_b64: str, key_b64: str) -> bytes:
         """Decrypt data with a client-provided key."""
         key = base64.b64decode(key_b64)
         nonce = base64.b64decode(nonce_b64)
@@ -646,8 +499,7 @@ class ClientEncryptionHelper:
             raise EncryptionError("Key must be 256 bits (32 bytes)")
         
         aesgcm = AESGCM(key)
-        aad = json.dumps(context, sort_keys=True).encode() if context else None
         try:
-            return aesgcm.decrypt(nonce, ciphertext, aad)
+            return aesgcm.decrypt(nonce, ciphertext, None)
         except Exception as exc:
-            raise EncryptionError("Decryption failed") from exc
+            raise EncryptionError(f"Decryption failed: {exc}") from exc

app/errors.py

@@ -6,7 +6,6 @@ from typing import Optional, Dict, Any
 from xml.etree.ElementTree import Element, SubElement, tostring
 
 from flask import Response, jsonify, request, flash, redirect, url_for, g
-from flask_limiter import RateLimitExceeded
 
 logger = logging.getLogger(__name__)
 
@@ -173,29 +172,9 @@ def handle_app_error(error: AppError) -> Response:
         return error.to_xml_response()
 
 
-def handle_rate_limit_exceeded(e: RateLimitExceeded) -> Response:
-    g.s3_error_code = "SlowDown"
-    if request.path.startswith("/ui") or request.path.startswith("/buckets"):
-        wants_json = (
-            request.is_json or
-            request.headers.get("X-Requested-With") == "XMLHttpRequest" or
-            "application/json" in request.accept_mimetypes.values()
-        )
-        if wants_json:
-            return jsonify({"success": False, "error": {"code": "SlowDown", "message": "Please reduce your request rate."}}), 429
-    error = Element("Error")
-    SubElement(error, "Code").text = "SlowDown"
-    SubElement(error, "Message").text = "Please reduce your request rate."
-    SubElement(error, "Resource").text = request.path
-    SubElement(error, "RequestId").text = getattr(g, "request_id", "")
-    xml_bytes = tostring(error, encoding="utf-8")
-    return Response(xml_bytes, status="429 Too Many Requests", mimetype="application/xml")
-
-
 def register_error_handlers(app):
     """Register error handlers with a Flask app."""
     app.register_error_handler(AppError, handle_app_error)
-    app.register_error_handler(RateLimitExceeded, handle_rate_limit_exceeded)
     
     for error_class in [
         BucketNotFoundError, BucketAlreadyExistsError, BucketNotEmptyError,

app/iam.py

@@ -0,0 +1,539 @@
+from __future__ import annotations
+
+import hmac
+import json
+import math
+import secrets
+import time
+from collections import deque
+from dataclasses import dataclass
+from datetime import datetime, timedelta, timezone
+from pathlib import Path
+from typing import Any, Deque, Dict, Iterable, List, Optional, Sequence, Set, Tuple
+
+
+class IamError(RuntimeError):
+    """Raised when authentication or authorization fails."""
+
+
+S3_ACTIONS = {"list", "read", "write", "delete", "share", "policy", "replication", "lifecycle", "cors"}
+IAM_ACTIONS = {
+    "iam:list_users",
+    "iam:create_user",
+    "iam:delete_user",
+    "iam:rotate_key",
+    "iam:update_policy",
+}
+ALLOWED_ACTIONS = (S3_ACTIONS | IAM_ACTIONS) | {"iam:*"}
+
+ACTION_ALIASES = {
+    "list": "list",
+    "s3:listbucket": "list",
+    "s3:listallmybuckets": "list",
+    "s3:listbucketversions": "list",
+    "s3:listmultipartuploads": "list",
+    "s3:listparts": "list",
+    "read": "read",
+    "s3:getobject": "read",
+    "s3:getobjectversion": "read",
+    "s3:getobjecttagging": "read",
+    "s3:getobjectversiontagging": "read",
+    "s3:getobjectacl": "read",
+    "s3:getbucketversioning": "read",
+    "s3:headobject": "read",
+    "s3:headbucket": "read",
+    "write": "write",
+    "s3:putobject": "write",
+    "s3:createbucket": "write",
+    "s3:putobjecttagging": "write",
+    "s3:putbucketversioning": "write",
+    "s3:createmultipartupload": "write",
+    "s3:uploadpart": "write",
+    "s3:completemultipartupload": "write",
+    "s3:abortmultipartupload": "write",
+    "s3:copyobject": "write",
+    "delete": "delete",
+    "s3:deleteobject": "delete",
+    "s3:deleteobjectversion": "delete",
+    "s3:deletebucket": "delete",
+    "s3:deleteobjecttagging": "delete",
+    "share": "share",
+    "s3:putobjectacl": "share",
+    "s3:putbucketacl": "share",
+    "s3:getbucketacl": "share",
+    "policy": "policy",
+    "s3:putbucketpolicy": "policy",
+    "s3:getbucketpolicy": "policy",
+    "s3:deletebucketpolicy": "policy",
+    "replication": "replication",
+    "s3:getreplicationconfiguration": "replication",
+    "s3:putreplicationconfiguration": "replication",
+    "s3:deletereplicationconfiguration": "replication",
+    "s3:replicateobject": "replication",
+    "s3:replicatetags": "replication",
+    "s3:replicatedelete": "replication",
+    "lifecycle": "lifecycle",
+    "s3:getlifecycleconfiguration": "lifecycle",
+    "s3:putlifecycleconfiguration": "lifecycle",
+    "s3:deletelifecycleconfiguration": "lifecycle",
+    "s3:getbucketlifecycle": "lifecycle",
+    "s3:putbucketlifecycle": "lifecycle",
+    "cors": "cors",
+    "s3:getbucketcors": "cors",
+    "s3:putbucketcors": "cors",
+    "s3:deletebucketcors": "cors",
+    "iam:listusers": "iam:list_users",
+    "iam:createuser": "iam:create_user",
+    "iam:deleteuser": "iam:delete_user",
+    "iam:rotateaccesskey": "iam:rotate_key",
+    "iam:putuserpolicy": "iam:update_policy",
+    "iam:*": "iam:*",
+}
+
+
+@dataclass
+class Policy:
+    bucket: str
+    actions: Set[str]
+
+
+@dataclass
+class Principal:
+    access_key: str
+    display_name: str
+    policies: List[Policy]
+
+
+class IamService:
+    """Loads IAM configuration, manages users, and evaluates policies."""
+
+    def __init__(self, config_path: Path, auth_max_attempts: int = 5, auth_lockout_minutes: int = 15) -> None:
+        self.config_path = Path(config_path)
+        self.auth_max_attempts = auth_max_attempts
+        self.auth_lockout_window = timedelta(minutes=auth_lockout_minutes)
+        self.config_path.parent.mkdir(parents=True, exist_ok=True)
+        if not self.config_path.exists():
+            self._write_default()
+        self._users: Dict[str, Dict[str, Any]] = {}
+        self._raw_config: Dict[str, Any] = {}
+        self._failed_attempts: Dict[str, Deque[datetime]] = {}
+        self._last_load_time = 0.0
+        self._credential_cache: Dict[str, Tuple[str, Principal, float]] = {}
+        self._cache_ttl = 60.0 
+        self._last_stat_check = 0.0
+        self._stat_check_interval = 1.0
+        self._sessions: Dict[str, Dict[str, Any]] = {}
+        self._load()
+
+    def _maybe_reload(self) -> None:
+        """Reload configuration if the file has changed on disk."""
+        now = time.time()
+        if now - self._last_stat_check < self._stat_check_interval:
+            return
+        self._last_stat_check = now
+        try:
+            if self.config_path.stat().st_mtime > self._last_load_time:
+                self._load()
+                self._credential_cache.clear() 
+        except OSError:
+            pass
+
+    def authenticate(self, access_key: str, secret_key: str) -> Principal:
+        self._maybe_reload()
+        access_key = (access_key or "").strip()
+        secret_key = (secret_key or "").strip()
+        if not access_key or not secret_key:
+            raise IamError("Missing access credentials")
+        if self._is_locked_out(access_key):
+            seconds = self._seconds_until_unlock(access_key)
+            raise IamError(
+                f"Access temporarily locked. Try again in {seconds} seconds."
+            )
+        record = self._users.get(access_key)
+        if not record or not hmac.compare_digest(record["secret_key"], secret_key):
+            self._record_failed_attempt(access_key)
+            raise IamError("Invalid credentials")
+        self._clear_failed_attempts(access_key)
+        return self._build_principal(access_key, record)
+
+    def _record_failed_attempt(self, access_key: str) -> None:
+        if not access_key:
+            return
+        attempts = self._failed_attempts.setdefault(access_key, deque())
+        self._prune_attempts(attempts)
+        attempts.append(datetime.now(timezone.utc))
+
+    def _clear_failed_attempts(self, access_key: str) -> None:
+        if not access_key:
+            return
+        self._failed_attempts.pop(access_key, None)
+
+    def _prune_attempts(self, attempts: Deque[datetime]) -> None:
+        cutoff = datetime.now(timezone.utc) - self.auth_lockout_window
+        while attempts and attempts[0] < cutoff:
+            attempts.popleft()
+
+    def _is_locked_out(self, access_key: str) -> bool:
+        if not access_key:
+            return False
+        attempts = self._failed_attempts.get(access_key)
+        if not attempts:
+            return False
+        self._prune_attempts(attempts)
+        return len(attempts) >= self.auth_max_attempts
+
+    def _seconds_until_unlock(self, access_key: str) -> int:
+        attempts = self._failed_attempts.get(access_key)
+        if not attempts:
+            return 0
+        self._prune_attempts(attempts)
+        if len(attempts) < self.auth_max_attempts:
+            return 0
+        oldest = attempts[0]
+        elapsed = (datetime.now(timezone.utc) - oldest).total_seconds()
+        return int(max(0, self.auth_lockout_window.total_seconds() - elapsed))
+
+    def create_session_token(self, access_key: str, duration_seconds: int = 3600) -> str:
+        """Create a temporary session token for an access key."""
+        self._maybe_reload()
+        record = self._users.get(access_key)
+        if not record:
+            raise IamError("Unknown access key")
+        self._cleanup_expired_sessions()
+        token = secrets.token_urlsafe(32)
+        expires_at = time.time() + duration_seconds
+        self._sessions[token] = {
+            "access_key": access_key,
+            "expires_at": expires_at,
+        }
+        return token
+
+    def validate_session_token(self, access_key: str, session_token: str) -> bool:
+        """Validate a session token for an access key."""
+        session = self._sessions.get(session_token)
+        if not session:
+            return False
+        if session["access_key"] != access_key:
+            return False
+        if time.time() > session["expires_at"]:
+            del self._sessions[session_token]
+            return False
+        return True
+
+    def _cleanup_expired_sessions(self) -> None:
+        """Remove expired session tokens."""
+        now = time.time()
+        expired = [token for token, data in self._sessions.items() if now > data["expires_at"]]
+        for token in expired:
+            del self._sessions[token]
+
+    def principal_for_key(self, access_key: str) -> Principal:
+        now = time.time()
+        cached = self._credential_cache.get(access_key)
+        if cached:
+            secret, principal, cached_time = cached
+            if now - cached_time < self._cache_ttl:
+                return principal
+
+        self._maybe_reload()
+        record = self._users.get(access_key)
+        if not record:
+            raise IamError("Unknown access key")
+        principal = self._build_principal(access_key, record)
+        self._credential_cache[access_key] = (record["secret_key"], principal, now)
+        return principal
+
+    def secret_for_key(self, access_key: str) -> str:
+        now = time.time()
+        cached = self._credential_cache.get(access_key)
+        if cached:
+            secret, principal, cached_time = cached
+            if now - cached_time < self._cache_ttl:
+                return secret
+
+        self._maybe_reload()
+        record = self._users.get(access_key)
+        if not record:
+            raise IamError("Unknown access key")
+        principal = self._build_principal(access_key, record)
+        self._credential_cache[access_key] = (record["secret_key"], principal, now)
+        return record["secret_key"]
+
+    def authorize(self, principal: Principal, bucket_name: str | None, action: str) -> None:
+        action = self._normalize_action(action)
+        if action not in ALLOWED_ACTIONS:
+            raise IamError(f"Unknown action '{action}'")
+        bucket_name = bucket_name or "*"
+        normalized = bucket_name.lower() if bucket_name != "*" else bucket_name
+        if not self._is_allowed(principal, normalized, action):
+            raise IamError(f"Access denied for action '{action}' on bucket '{bucket_name}'")
+
+    def buckets_for_principal(self, principal: Principal, buckets: Iterable[str]) -> List[str]:
+        return [bucket for bucket in buckets if self._is_allowed(principal, bucket, "list")]
+
+    def _is_allowed(self, principal: Principal, bucket_name: str, action: str) -> bool:
+        bucket_name = bucket_name.lower()
+        for policy in principal.policies:
+            if policy.bucket not in {"*", bucket_name}:
+                continue
+            if "*" in policy.actions or action in policy.actions:
+                return True
+            if "iam:*" in policy.actions and action.startswith("iam:"):
+                return True
+        return False
+
+    def list_users(self) -> List[Dict[str, Any]]:
+        listing: List[Dict[str, Any]] = []
+        for access_key, record in self._users.items():
+            listing.append(
+                {
+                    "access_key": access_key,
+                    "display_name": record["display_name"],
+                    "policies": [
+                        {"bucket": policy.bucket, "actions": sorted(policy.actions)}
+                        for policy in record["policies"]
+                    ],
+                }
+            )
+        return listing
+
+    def create_user(
+        self,
+        *,
+        display_name: str,
+        policies: Optional[Sequence[Dict[str, Any]]] = None,
+        access_key: str | None = None,
+        secret_key: str | None = None,
+    ) -> Dict[str, str]:
+        access_key = (access_key or self._generate_access_key()).strip()
+        if not access_key:
+            raise IamError("Access key cannot be empty")
+        if access_key in self._users:
+            raise IamError("Access key already exists")
+        secret_key = secret_key or self._generate_secret_key()
+        sanitized_policies = self._prepare_policy_payload(policies)
+        record = {
+            "access_key": access_key,
+            "secret_key": secret_key,
+            "display_name": display_name or access_key,
+            "policies": sanitized_policies,
+        }
+        self._raw_config.setdefault("users", []).append(record)
+        self._save()
+        self._load()
+        return {"access_key": access_key, "secret_key": secret_key}
+
+    def rotate_secret(self, access_key: str) -> str:
+        user = self._get_raw_user(access_key)
+        new_secret = self._generate_secret_key()
+        user["secret_key"] = new_secret
+        self._save()
+        self._load()
+        return new_secret
+
+    def update_user(self, access_key: str, display_name: str) -> None:
+        user = self._get_raw_user(access_key)
+        user["display_name"] = display_name
+        self._save()
+        self._load()
+
+    def delete_user(self, access_key: str) -> None:
+        users = self._raw_config.get("users", [])
+        if len(users) <= 1:
+            raise IamError("Cannot delete the only user")
+        remaining = [user for user in users if user["access_key"] != access_key]
+        if len(remaining) == len(users):
+            raise IamError("User not found")
+        self._raw_config["users"] = remaining
+        self._save()
+        self._load()
+
+    def update_user_policies(self, access_key: str, policies: Sequence[Dict[str, Any]]) -> None:
+        user = self._get_raw_user(access_key)
+        user["policies"] = self._prepare_policy_payload(policies)
+        self._save()
+        self._load()
+
+    def _load(self) -> None:
+        try:
+            self._last_load_time = self.config_path.stat().st_mtime
+            content = self.config_path.read_text(encoding='utf-8')
+            raw = json.loads(content)
+        except FileNotFoundError:
+            raise IamError(f"IAM config not found: {self.config_path}")
+        except json.JSONDecodeError as e:
+            raise IamError(f"Corrupted IAM config (invalid JSON): {e}")
+        except PermissionError as e:
+            raise IamError(f"Cannot read IAM config (permission denied): {e}")
+        except (OSError, ValueError) as e:
+            raise IamError(f"Failed to load IAM config: {e}")
+        
+        users: Dict[str, Dict[str, Any]] = {}
+        for user in raw.get("users", []):
+            policies = self._build_policy_objects(user.get("policies", []))
+            users[user["access_key"]] = {
+                "secret_key": user["secret_key"],
+                "display_name": user.get("display_name", user["access_key"]),
+                "policies": policies,
+            }
+        if not users:
+            raise IamError("IAM configuration contains no users")
+        self._users = users
+        self._raw_config = {
+            "users": [
+                {
+                    "access_key": entry["access_key"],
+                    "secret_key": entry["secret_key"],
+                    "display_name": entry.get("display_name", entry["access_key"]),
+                    "policies": entry.get("policies", []),
+                }
+                for entry in raw.get("users", [])
+            ]
+        }
+
+    def _save(self) -> None:
+        try:
+            temp_path = self.config_path.with_suffix('.json.tmp')
+            temp_path.write_text(json.dumps(self._raw_config, indent=2), encoding='utf-8')
+            temp_path.replace(self.config_path)
+        except (OSError, PermissionError) as e:
+            raise IamError(f"Cannot save IAM config: {e}")
+
+    def config_summary(self) -> Dict[str, Any]:
+        return {
+            "path": str(self.config_path),
+            "user_count": len(self._users),
+            "allowed_actions": sorted(ALLOWED_ACTIONS),
+        }
+
+    def export_config(self, mask_secrets: bool = True) -> Dict[str, Any]:
+        payload: Dict[str, Any] = {"users": []}
+        for user in self._raw_config.get("users", []):
+            record = dict(user)
+            if mask_secrets and "secret_key" in record:
+                record["secret_key"] = "••••••••••"
+            payload["users"].append(record)
+        return payload
+
+    def _build_policy_objects(self, policies: Sequence[Dict[str, Any]]) -> List[Policy]:
+        entries: List[Policy] = []
+        for policy in policies:
+            bucket = str(policy.get("bucket", "*")).lower()
+            raw_actions = policy.get("actions", [])
+            if isinstance(raw_actions, str):
+                raw_actions = [raw_actions]
+            action_set: Set[str] = set()
+            for action in raw_actions:
+                canonical = self._normalize_action(action)
+                if canonical == "*":
+                    action_set = set(ALLOWED_ACTIONS)
+                    break
+                if canonical:
+                    action_set.add(canonical)
+            if action_set:
+                entries.append(Policy(bucket=bucket, actions=action_set))
+        return entries
+
+    def _prepare_policy_payload(self, policies: Optional[Sequence[Dict[str, Any]]]) -> List[Dict[str, Any]]:
+        if not policies:
+            policies = (
+                {
+                    "bucket": "*",
+                    "actions": ["list", "read", "write", "delete", "share", "policy"],
+                },
+            )
+        sanitized: List[Dict[str, Any]] = []
+        for policy in policies:
+            bucket = str(policy.get("bucket", "*")).lower()
+            raw_actions = policy.get("actions", [])
+            if isinstance(raw_actions, str):
+                raw_actions = [raw_actions]
+            action_set: Set[str] = set()
+            for action in raw_actions:
+                canonical = self._normalize_action(action)
+                if canonical == "*":
+                    action_set = set(ALLOWED_ACTIONS)
+                    break
+                if canonical:
+                    action_set.add(canonical)
+            if not action_set:
+                continue
+            sanitized.append({"bucket": bucket, "actions": sorted(action_set)})
+        if not sanitized:
+            raise IamError("At least one policy with valid actions is required")
+        return sanitized
+
+    def _build_principal(self, access_key: str, record: Dict[str, Any]) -> Principal:
+        return Principal(
+            access_key=access_key,
+            display_name=record["display_name"],
+            policies=record["policies"],
+        )
+
+    def _normalize_action(self, action: str) -> str:
+        if not action:
+            return ""
+        lowered = action.strip().lower()
+        if lowered == "*":
+            return "*"
+        candidate = ACTION_ALIASES.get(lowered, lowered)
+        return candidate if candidate in ALLOWED_ACTIONS else ""
+
+    def _write_default(self) -> None:
+        default = {
+            "users": [
+                {
+                    "access_key": "localadmin",
+                    "secret_key": "localadmin",
+                    "display_name": "Local Admin",
+                    "policies": [
+                        {"bucket": "*", "actions": list(ALLOWED_ACTIONS)}
+                    ],
+                }
+            ]
+        }
+        self.config_path.write_text(json.dumps(default, indent=2))
+
+    def _generate_access_key(self) -> str:
+        return secrets.token_hex(8)
+
+    def _generate_secret_key(self) -> str:
+        return secrets.token_urlsafe(24)
+
+    def _get_raw_user(self, access_key: str) -> Dict[str, Any]:
+        for user in self._raw_config.get("users", []):
+            if user["access_key"] == access_key:
+                return user
+        raise IamError("User not found")
+
+    def get_secret_key(self, access_key: str) -> str | None:
+        now = time.time()
+        cached = self._credential_cache.get(access_key)
+        if cached:
+            secret, principal, cached_time = cached
+            if now - cached_time < self._cache_ttl:
+                return secret
+
+        self._maybe_reload()
+        record = self._users.get(access_key)
+        if record:
+            principal = self._build_principal(access_key, record)
+            self._credential_cache[access_key] = (record["secret_key"], principal, now)
+            return record["secret_key"]
+        return None
+
+    def get_principal(self, access_key: str) -> Principal | None:
+        now = time.time()
+        cached = self._credential_cache.get(access_key)
+        if cached:
+            secret, principal, cached_time = cached
+            if now - cached_time < self._cache_ttl:
+                return principal
+
+        self._maybe_reload()
+        record = self._users.get(access_key)
+        if record:
+            principal = self._build_principal(access_key, record)
+            self._credential_cache[access_key] = (record["secret_key"], principal, now)
+            return principal
+        return None

app/kms.py

@@ -2,11 +2,7 @@ from __future__ import annotations
 
 import base64
 import json
-import logging
-import os
 import secrets
-import subprocess
-import sys
 import uuid
 from dataclasses import dataclass, field
 from datetime import datetime, timezone
@@ -17,30 +13,6 @@ from cryptography.hazmat.primitives.ciphers.aead import AESGCM
 
 from .encryption import EncryptionError, EncryptionProvider, EncryptionResult
 
-if sys.platform != "win32":
-    import fcntl
-
-logger = logging.getLogger(__name__)
-
-
-def _set_secure_file_permissions(file_path: Path) -> None:
-    """Set restrictive file permissions (owner read/write only)."""
-    if sys.platform == "win32":
-        try:
-            username = os.environ.get("USERNAME", "")
-            if username:
-                subprocess.run(
-                    ["icacls", str(file_path), "/inheritance:r",
-                     "/grant:r", f"{username}:F"],
-                    check=True, capture_output=True
-                )
-            else:
-                logger.warning("Could not set secure permissions on %s: USERNAME not set", file_path)
-        except (subprocess.SubprocessError, OSError) as exc:
-            logger.warning("Failed to set secure permissions on %s: %s", file_path, exc)
-    else:
-        os.chmod(file_path, 0o600)
-
 
 @dataclass
 class KMSKey:
@@ -106,7 +78,7 @@ class KMSEncryptionProvider(EncryptionProvider):
         aesgcm = AESGCM(data_key)
         nonce = secrets.token_bytes(12)
         ciphertext = aesgcm.encrypt(nonce, plaintext, 
-                                     json.dumps(context, sort_keys=True).encode() if context else None)
+                                     json.dumps(context).encode() if context else None)
         
         return EncryptionResult(
             ciphertext=ciphertext,
@@ -118,26 +90,15 @@ class KMSEncryptionProvider(EncryptionProvider):
     def decrypt(self, ciphertext: bytes, nonce: bytes, encrypted_data_key: bytes,
                 key_id: str, context: Dict[str, str] | None = None) -> bytes:
         """Decrypt data using envelope encryption with KMS."""
+        # Note: Data key is encrypted without context (AAD), so we decrypt without context
         data_key = self.kms.decrypt_data_key(key_id, encrypted_data_key, context=None)
-        if len(data_key) != 32:
-            raise EncryptionError("Invalid data key size")
         
         aesgcm = AESGCM(data_key)
         try:
             return aesgcm.decrypt(nonce, ciphertext,
-                                  json.dumps(context, sort_keys=True).encode() if context else None)
+                                  json.dumps(context).encode() if context else None)
         except Exception as exc:
-            logger.debug("KMS decryption failed: %s", exc)
+            raise EncryptionError(f"Failed to decrypt data: {exc}") from exc
-            raise EncryptionError("Failed to decrypt data") from exc
-
-    def decrypt_data_key(self, encrypted_data_key: bytes, key_id: str | None = None) -> bytes:
-        """Decrypt an encrypted data key using KMS."""
-        if key_id is None:
-            key_id = self.key_id
-        data_key = self.kms.decrypt_data_key(key_id, encrypted_data_key, context=None)
-        if len(data_key) != 32:
-            raise EncryptionError("Invalid data key size")
-        return data_key
 
 
 class KMSManager:
@@ -147,52 +108,27 @@ class KMSManager:
     Keys are stored encrypted on disk.
     """
     
-    def __init__(
+    def __init__(self, keys_path: Path, master_key_path: Path):
-        self,
-        keys_path: Path,
-        master_key_path: Path,
-        generate_data_key_min_bytes: int = 1,
-        generate_data_key_max_bytes: int = 1024,
-    ):
         self.keys_path = keys_path
         self.master_key_path = master_key_path
-        self.generate_data_key_min_bytes = generate_data_key_min_bytes
-        self.generate_data_key_max_bytes = generate_data_key_max_bytes
         self._keys: Dict[str, KMSKey] = {}
         self._master_key: bytes | None = None
-        self._master_aesgcm: AESGCM | None = None
         self._loaded = False
     
     @property
     def master_key(self) -> bytes:
-        """Load or create the master key for encrypting KMS keys (with file locking)."""
+        """Load or create the master key for encrypting KMS keys."""
         if self._master_key is None:
-            lock_path = self.master_key_path.with_suffix(".lock")
-            lock_path.parent.mkdir(parents=True, exist_ok=True)
-            with open(lock_path, "w") as lock_file:
-                if sys.platform == "win32":
-                    import msvcrt
-                    msvcrt.locking(lock_file.fileno(), msvcrt.LK_LOCK, 1)
-                else:
-                    fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX)
-                try:
             if self.master_key_path.exists():
                 self._master_key = base64.b64decode(
                     self.master_key_path.read_text().strip()
                 )
             else:
                 self._master_key = secrets.token_bytes(32)
+                self.master_key_path.parent.mkdir(parents=True, exist_ok=True)
                 self.master_key_path.write_text(
                     base64.b64encode(self._master_key).decode()
                 )
-                        _set_secure_file_permissions(self.master_key_path)
-                finally:
-                    if sys.platform == "win32":
-                        import msvcrt
-                        msvcrt.locking(lock_file.fileno(), msvcrt.LK_UNLCK, 1)
-                    else:
-                        fcntl.flock(lock_file.fileno(), fcntl.LOCK_UN)
-            self._master_aesgcm = AESGCM(self._master_key)
         return self._master_key
     
     def _load_keys(self) -> None:
@@ -209,10 +145,8 @@ class KMSManager:
                         encrypted = base64.b64decode(key_data["EncryptedKeyMaterial"])
                         key.key_material = self._decrypt_key_material(encrypted)
                     self._keys[key.key_id] = key
-            except json.JSONDecodeError as exc:
+            except Exception:
-                logger.error("Failed to parse KMS keys file: %s", exc)
+                pass
-            except (ValueError, KeyError) as exc:
-                logger.error("Invalid KMS key data: %s", exc)
         
         self._loaded = True
     
@@ -230,19 +164,20 @@ class KMSManager:
             json.dumps({"keys": keys_data}, indent=2),
             encoding="utf-8"
         )
-        _set_secure_file_permissions(self.keys_path)
     
     def _encrypt_key_material(self, key_material: bytes) -> bytes:
-        _ = self.master_key
+        """Encrypt key material with the master key."""
+        aesgcm = AESGCM(self.master_key)
         nonce = secrets.token_bytes(12)
-        ciphertext = self._master_aesgcm.encrypt(nonce, key_material, None)
+        ciphertext = aesgcm.encrypt(nonce, key_material, None)
         return nonce + ciphertext
     
     def _decrypt_key_material(self, encrypted: bytes) -> bytes:
-        _ = self.master_key
+        """Decrypt key material with the master key."""
+        aesgcm = AESGCM(self.master_key)
         nonce = encrypted[:12]
         ciphertext = encrypted[12:]
-        return self._master_aesgcm.decrypt(nonce, ciphertext, None)
+        return aesgcm.decrypt(nonce, ciphertext, None)
     
     def create_key(self, description: str = "", key_id: str | None = None) -> KMSKey:
         """Create a new KMS key."""
@@ -334,7 +269,7 @@ class KMSManager:
         
         aesgcm = AESGCM(key.key_material)
         nonce = secrets.token_bytes(12)
-        aad = json.dumps(context, sort_keys=True).encode() if context else None
+        aad = json.dumps(context).encode() if context else None
         ciphertext = aesgcm.encrypt(nonce, plaintext, aad)
         
         key_id_bytes = key_id.encode("utf-8")
@@ -363,24 +298,17 @@ class KMSManager:
         encrypted = rest[12:]
         
         aesgcm = AESGCM(key.key_material)
-        aad = json.dumps(context, sort_keys=True).encode() if context else None
+        aad = json.dumps(context).encode() if context else None
         try:
             plaintext = aesgcm.decrypt(nonce, encrypted, aad)
             return plaintext, key_id
         except Exception as exc:
-            logger.debug("KMS decrypt operation failed: %s", exc)
+            raise EncryptionError(f"Decryption failed: {exc}") from exc
-            raise EncryptionError("Decryption failed") from exc
     
     def generate_data_key(self, key_id: str,
-                          context: Dict[str, str] | None = None,
+                          context: Dict[str, str] | None = None) -> tuple[bytes, bytes]:
-                          key_spec: str = "AES_256") -> tuple[bytes, bytes]:
         """Generate a data key and return both plaintext and encrypted versions.
         
-        Args:
-            key_id: The KMS key ID to use for encryption
-            context: Optional encryption context
-            key_spec: Key specification - AES_128 or AES_256 (default)
-
         Returns:
             Tuple of (plaintext_key, encrypted_key)
         """
@@ -391,8 +319,7 @@ class KMSManager:
         if not key.enabled:
             raise EncryptionError(f"Key is disabled: {key_id}")
         
-        key_bytes = 32 if key_spec == "AES_256" else 16
+        plaintext_key = secrets.token_bytes(32)
-        plaintext_key = secrets.token_bytes(key_bytes)
 
         encrypted_key = self.encrypt(key_id, plaintext_key, context)
         
@@ -404,6 +331,22 @@ class KMSManager:
         plaintext, _ = self.decrypt(encrypted_key, context)
         return plaintext
     
+    def get_provider(self, key_id: str | None = None) -> KMSEncryptionProvider:
+        """Get an encryption provider for a specific key."""
+        self._load_keys()
+        
+        if key_id is None:
+            if not self._keys:
+                key = self.create_key("Default KMS Key")
+                key_id = key.key_id
+            else:
+                key_id = next(iter(self._keys.keys()))
+        
+        if key_id not in self._keys:
+            raise EncryptionError(f"Key not found: {key_id}")
+        
+        return KMSEncryptionProvider(self, key_id)
+    
     def re_encrypt(self, ciphertext: bytes, destination_key_id: str,
                    source_context: Dict[str, str] | None = None,
                    destination_context: Dict[str, str] | None = None) -> bytes:
@@ -415,8 +358,6 @@ class KMSManager:
     
     def generate_random(self, num_bytes: int = 32) -> bytes:
         """Generate cryptographically secure random bytes."""
-        if num_bytes < self.generate_data_key_min_bytes or num_bytes > self.generate_data_key_max_bytes:
+        if num_bytes < 1 or num_bytes > 1024:
-            raise EncryptionError(
+            raise EncryptionError("Number of bytes must be between 1 and 1024")
-                f"Number of bytes must be between {self.generate_data_key_min_bytes} and {self.generate_data_key_max_bytes}"
-            )
         return secrets.token_bytes(num_bytes)

app/lifecycle.py

@@ -71,9 +71,10 @@ class LifecycleExecutionRecord:
 
 
 class LifecycleHistoryStore:
-    def __init__(self, storage_root: Path, max_history_per_bucket: int = 50) -> None:
+    MAX_HISTORY_PER_BUCKET = 50
+
+    def __init__(self, storage_root: Path) -> None:
         self.storage_root = storage_root
-        self.max_history_per_bucket = max_history_per_bucket
         self._lock = threading.Lock()
 
     def _get_history_path(self, bucket_name: str) -> Path:
@@ -94,7 +95,7 @@ class LifecycleHistoryStore:
     def save_history(self, bucket_name: str, records: List[LifecycleExecutionRecord]) -> None:
         path = self._get_history_path(bucket_name)
         path.parent.mkdir(parents=True, exist_ok=True)
-        data = {"executions": [r.to_dict() for r in records[:self.max_history_per_bucket]]}
+        data = {"executions": [r.to_dict() for r in records[:self.MAX_HISTORY_PER_BUCKET]]}
         try:
             with open(path, "w") as f:
                 json.dump(data, f, indent=2)
@@ -113,20 +114,14 @@ class LifecycleHistoryStore:
 
 
 class LifecycleManager:
-    def __init__(
+    def __init__(self, storage: ObjectStorage, interval_seconds: int = 3600, storage_root: Optional[Path] = None):
-        self,
-        storage: ObjectStorage,
-        interval_seconds: int = 3600,
-        storage_root: Optional[Path] = None,
-        max_history_per_bucket: int = 50,
-    ):
         self.storage = storage
         self.interval_seconds = interval_seconds
         self.storage_root = storage_root
         self._timer: Optional[threading.Timer] = None
         self._shutdown = False
         self._lock = threading.Lock()
-        self.history_store = LifecycleHistoryStore(storage_root, max_history_per_bucket) if storage_root else None
+        self.history_store = LifecycleHistoryStore(storage_root) if storage_root else None
 
     def start(self) -> None:
         if self._timer is not None:

app/notifications.py

@@ -1,10 +1,8 @@
 from __future__ import annotations
 
-import ipaddress
 import json
 import logging
 import queue
-import socket
 import threading
 import time
 import uuid
@@ -15,71 +13,6 @@ from typing import Any, Dict, List, Optional
 from urllib.parse import urlparse
 
 import requests
-from urllib3.util.connection import create_connection as _urllib3_create_connection
-
-
-def _resolve_and_check_url(url: str, allow_internal: bool = False) -> Optional[str]:
-    try:
-        parsed = urlparse(url)
-        hostname = parsed.hostname
-        if not hostname:
-            return None
-        cloud_metadata_hosts = {
-            "metadata.google.internal",
-            "169.254.169.254",
-        }
-        if hostname.lower() in cloud_metadata_hosts:
-            return None
-        if allow_internal:
-            return hostname
-        blocked_hosts = {
-            "localhost",
-            "127.0.0.1",
-            "0.0.0.0",
-            "::1",
-            "[::1]",
-        }
-        if hostname.lower() in blocked_hosts:
-            return None
-        try:
-            resolved_ip = socket.gethostbyname(hostname)
-            ip = ipaddress.ip_address(resolved_ip)
-            if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
-                return None
-            return resolved_ip
-        except (socket.gaierror, ValueError):
-            return None
-    except Exception:
-        return None
-
-
-def _is_safe_url(url: str, allow_internal: bool = False) -> bool:
-    return _resolve_and_check_url(url, allow_internal) is not None
-
-
-_dns_pin_lock = threading.Lock()
-
-
-def _pinned_post(url: str, pinned_ip: str, **kwargs: Any) -> requests.Response:
-    parsed = urlparse(url)
-    hostname = parsed.hostname or ""
-    session = requests.Session()
-    original_create = _urllib3_create_connection
-
-    def _create_pinned(address: Any, *args: Any, **kw: Any) -> Any:
-        host, req_port = address
-        if host == hostname:
-            return original_create((pinned_ip, req_port), *args, **kw)
-        return original_create(address, *args, **kw)
-
-    import urllib3.util.connection as _conn_mod
-    with _dns_pin_lock:
-        _conn_mod.create_connection = _create_pinned
-        try:
-            return session.post(url, **kwargs)
-        finally:
-            _conn_mod.create_connection = original_create
-
 
 logger = logging.getLogger(__name__)
 
@@ -232,9 +165,8 @@ class NotificationConfiguration:
 
 
 class NotificationService:
-    def __init__(self, storage_root: Path, worker_count: int = 2, allow_internal_endpoints: bool = False):
+    def __init__(self, storage_root: Path, worker_count: int = 2):
         self.storage_root = storage_root
-        self._allow_internal_endpoints = allow_internal_endpoints
         self._configs: Dict[str, List[NotificationConfiguration]] = {}
         self._queue: queue.Queue[tuple[NotificationEvent, WebhookDestination]] = queue.Queue()
         self._workers: List[threading.Thread] = []
@@ -367,18 +299,14 @@ class NotificationService:
                 self._queue.task_done()
 
     def _send_notification(self, event: NotificationEvent, destination: WebhookDestination) -> None:
-        resolved_ip = _resolve_and_check_url(destination.url, allow_internal=self._allow_internal_endpoints)
-        if not resolved_ip:
-            raise RuntimeError(f"Blocked request (SSRF protection): {destination.url}")
         payload = event.to_s3_event()
         headers = {"Content-Type": "application/json", **destination.headers}
 
         last_error = None
         for attempt in range(destination.retry_count):
             try:
-                response = _pinned_post(
+                response = requests.post(
                     destination.url,
-                    resolved_ip,
                     json=payload,
                     headers=headers,
                     timeout=destination.timeout_seconds,

app/operation_metrics.py

@@ -2,17 +2,13 @@ from __future__ import annotations
 
 import json
 import logging
-import random
 import threading
 import time
-from collections import defaultdict
 from dataclasses import dataclass, field
 from datetime import datetime, timezone
 from pathlib import Path
 from typing import Any, Dict, List, Optional
 
-MAX_LATENCY_SAMPLES = 5000
-
 logger = logging.getLogger(__name__)
 
 
@@ -26,17 +22,6 @@ class OperationStats:
     latency_max_ms: float = 0.0
     bytes_in: int = 0
     bytes_out: int = 0
-    latency_samples: List[float] = field(default_factory=list)
-
-    @staticmethod
-    def _compute_percentile(sorted_data: List[float], p: float) -> float:
-        if not sorted_data:
-            return 0.0
-        k = (len(sorted_data) - 1) * (p / 100.0)
-        f = int(k)
-        c = min(f + 1, len(sorted_data) - 1)
-        d = k - f
-        return sorted_data[f] + d * (sorted_data[c] - sorted_data[f])
 
     def record(self, latency_ms: float, success: bool, bytes_in: int = 0, bytes_out: int = 0) -> None:
         self.count += 1
@@ -51,17 +36,10 @@ class OperationStats:
             self.latency_max_ms = latency_ms
         self.bytes_in += bytes_in
         self.bytes_out += bytes_out
-        if len(self.latency_samples) < MAX_LATENCY_SAMPLES:
-            self.latency_samples.append(latency_ms)
-        else:
-            j = random.randint(0, self.count - 1)
-            if j < MAX_LATENCY_SAMPLES:
-                self.latency_samples[j] = latency_ms
 
     def to_dict(self) -> Dict[str, Any]:
         avg_latency = self.latency_sum_ms / self.count if self.count > 0 else 0.0
         min_latency = self.latency_min_ms if self.latency_min_ms != float("inf") else 0.0
-        sorted_latencies = sorted(self.latency_samples)
         return {
             "count": self.count,
             "success_count": self.success_count,
@@ -69,9 +47,6 @@ class OperationStats:
             "latency_avg_ms": round(avg_latency, 2),
             "latency_min_ms": round(min_latency, 2),
             "latency_max_ms": round(self.latency_max_ms, 2),
-            "latency_p50_ms": round(self._compute_percentile(sorted_latencies, 50), 2),
-            "latency_p95_ms": round(self._compute_percentile(sorted_latencies, 95), 2),
-            "latency_p99_ms": round(self._compute_percentile(sorted_latencies, 99), 2),
             "bytes_in": self.bytes_in,
             "bytes_out": self.bytes_out,
         }
@@ -87,11 +62,6 @@ class OperationStats:
             self.latency_max_ms = other.latency_max_ms
         self.bytes_in += other.bytes_in
         self.bytes_out += other.bytes_out
-        combined = self.latency_samples + other.latency_samples
-        if len(combined) > MAX_LATENCY_SAMPLES:
-            random.shuffle(combined)
-            combined = combined[:MAX_LATENCY_SAMPLES]
-        self.latency_samples = combined
 
 
 @dataclass
@@ -139,8 +109,8 @@ class OperationMetricsCollector:
         self.interval_seconds = interval_minutes * 60
         self.retention_hours = retention_hours
         self._lock = threading.Lock()
-        self._by_method: Dict[str, OperationStats] = defaultdict(OperationStats)
+        self._by_method: Dict[str, OperationStats] = {}
-        self._by_endpoint: Dict[str, OperationStats] = defaultdict(OperationStats)
+        self._by_endpoint: Dict[str, OperationStats] = {}
         self._by_status_class: Dict[str, int] = {}
         self._error_codes: Dict[str, int] = {}
         self._totals = OperationStats()
@@ -212,8 +182,8 @@ class OperationMetricsCollector:
             self._prune_old_snapshots()
             self._save_history()
 
-            self._by_method = defaultdict(OperationStats)
+            self._by_method.clear()
-            self._by_endpoint = defaultdict(OperationStats)
+            self._by_endpoint.clear()
             self._by_status_class.clear()
             self._error_codes.clear()
             self._totals = OperationStats()
@@ -233,7 +203,12 @@ class OperationMetricsCollector:
         status_class = f"{status_code // 100}xx"
 
         with self._lock:
+            if method not in self._by_method:
+                self._by_method[method] = OperationStats()
             self._by_method[method].record(latency_ms, success, bytes_in, bytes_out)
+
+            if endpoint_type not in self._by_endpoint:
+                self._by_endpoint[endpoint_type] = OperationStats()
             self._by_endpoint[endpoint_type].record(latency_ms, success, bytes_in, bytes_out)
 
             self._by_status_class[status_class] = self._by_status_class.get(status_class, 0) + 1

app/replication.py

@@ -21,20 +21,15 @@ from .storage import ObjectStorage, StorageError
 logger = logging.getLogger(__name__)
 
 REPLICATION_USER_AGENT = "S3ReplicationAgent/1.0"
+REPLICATION_CONNECT_TIMEOUT = 5
+REPLICATION_READ_TIMEOUT = 30
+STREAMING_THRESHOLD_BYTES = 10 * 1024 * 1024 
 
 REPLICATION_MODE_NEW_ONLY = "new_only"
 REPLICATION_MODE_ALL = "all"
-REPLICATION_MODE_BIDIRECTIONAL = "bidirectional"
 
 
-def _create_s3_client(
+def _create_s3_client(connection: RemoteConnection, *, health_check: bool = False) -> Any:
-    connection: RemoteConnection,
-    *,
-    health_check: bool = False,
-    connect_timeout: int = 5,
-    read_timeout: int = 30,
-    max_retries: int = 2,
-) -> Any:
     """Create a boto3 S3 client for the given connection.
     Args:
         connection: Remote S3 connection configuration
@@ -42,9 +37,9 @@ def _create_s3_client(
     """
     config = Config(
         user_agent_extra=REPLICATION_USER_AGENT,
-        connect_timeout=connect_timeout,
+        connect_timeout=REPLICATION_CONNECT_TIMEOUT,
-        read_timeout=read_timeout,
+        read_timeout=REPLICATION_READ_TIMEOUT,
-        retries={'max_attempts': 1 if health_check else max_retries},
+        retries={'max_attempts': 1 if health_check else 2},
         signature_version='s3v4',
         s3={'addressing_style': 'path'},
         request_checksum_calculation='when_required',
@@ -135,9 +130,6 @@ class ReplicationRule:
     mode: str = REPLICATION_MODE_NEW_ONLY 
     created_at: Optional[float] = None
     stats: ReplicationStats = field(default_factory=ReplicationStats)
-    sync_deletions: bool = True
-    last_pull_at: Optional[float] = None
-    filter_prefix: Optional[str] = None
     
     def to_dict(self) -> dict:
         return {
@@ -148,9 +140,6 @@ class ReplicationRule:
             "mode": self.mode,
             "created_at": self.created_at,
             "stats": self.stats.to_dict(),
-            "sync_deletions": self.sync_deletions,
-            "last_pull_at": self.last_pull_at,
-            "filter_prefix": self.filter_prefix,
         }
     
     @classmethod
@@ -160,28 +149,22 @@ class ReplicationRule:
             data["mode"] = REPLICATION_MODE_NEW_ONLY
         if "created_at" not in data:
             data["created_at"] = None
-        if "sync_deletions" not in data:
-            data["sync_deletions"] = True
-        if "last_pull_at" not in data:
-            data["last_pull_at"] = None
-        if "filter_prefix" not in data:
-            data["filter_prefix"] = None
         rule = cls(**data)
         rule.stats = ReplicationStats.from_dict(stats_data) if stats_data else ReplicationStats()
         return rule
 
 
 class ReplicationFailureStore:
-    def __init__(self, storage_root: Path, max_failures_per_bucket: int = 50) -> None:
+    MAX_FAILURES_PER_BUCKET = 50
+
+    def __init__(self, storage_root: Path) -> None:
         self.storage_root = storage_root
-        self.max_failures_per_bucket = max_failures_per_bucket
         self._lock = threading.Lock()
-        self._cache: Dict[str, List[ReplicationFailure]] = {}
 
     def _get_failures_path(self, bucket_name: str) -> Path:
         return self.storage_root / ".myfsio.sys" / "buckets" / bucket_name / "replication_failures.json"
 
-    def _load_from_disk(self, bucket_name: str) -> List[ReplicationFailure]:
+    def load_failures(self, bucket_name: str) -> List[ReplicationFailure]:
         path = self._get_failures_path(bucket_name)
         if not path.exists():
             return []
@@ -193,28 +176,16 @@ class ReplicationFailureStore:
             logger.error(f"Failed to load replication failures for {bucket_name}: {e}")
             return []
 
-    def _save_to_disk(self, bucket_name: str, failures: List[ReplicationFailure]) -> None:
+    def save_failures(self, bucket_name: str, failures: List[ReplicationFailure]) -> None:
         path = self._get_failures_path(bucket_name)
         path.parent.mkdir(parents=True, exist_ok=True)
-        data = {"failures": [f.to_dict() for f in failures[:self.max_failures_per_bucket]]}
+        data = {"failures": [f.to_dict() for f in failures[:self.MAX_FAILURES_PER_BUCKET]]}
         try:
             with open(path, "w") as f:
                 json.dump(data, f, indent=2)
         except OSError as e:
             logger.error(f"Failed to save replication failures for {bucket_name}: {e}")
 
-    def load_failures(self, bucket_name: str) -> List[ReplicationFailure]:
-        if bucket_name in self._cache:
-            return list(self._cache[bucket_name])
-        failures = self._load_from_disk(bucket_name)
-        self._cache[bucket_name] = failures
-        return list(failures)
-
-    def save_failures(self, bucket_name: str, failures: List[ReplicationFailure]) -> None:
-        trimmed = failures[:self.max_failures_per_bucket]
-        self._cache[bucket_name] = trimmed
-        self._save_to_disk(bucket_name, trimmed)
-
     def add_failure(self, bucket_name: str, failure: ReplicationFailure) -> None:
         with self._lock:
             failures = self.load_failures(bucket_name)
@@ -240,7 +211,6 @@ class ReplicationFailureStore:
 
     def clear_failures(self, bucket_name: str) -> None:
         with self._lock:
-            self._cache.pop(bucket_name, None)
             path = self._get_failures_path(bucket_name)
             if path.exists():
                 path.unlink()
@@ -254,43 +224,18 @@ class ReplicationFailureStore:
 
 
 class ReplicationManager:
-    def __init__(
+    def __init__(self, storage: ObjectStorage, connections: ConnectionStore, rules_path: Path, storage_root: Path) -> None:
-        self,
-        storage: ObjectStorage,
-        connections: ConnectionStore,
-        rules_path: Path,
-        storage_root: Path,
-        connect_timeout: int = 5,
-        read_timeout: int = 30,
-        max_retries: int = 2,
-        streaming_threshold_bytes: int = 10 * 1024 * 1024,
-        max_failures_per_bucket: int = 50,
-    ) -> None:
         self.storage = storage
         self.connections = connections
         self.rules_path = rules_path
         self.storage_root = storage_root
-        self.connect_timeout = connect_timeout
-        self.read_timeout = read_timeout
-        self.max_retries = max_retries
-        self.streaming_threshold_bytes = streaming_threshold_bytes
         self._rules: Dict[str, ReplicationRule] = {}
         self._stats_lock = threading.Lock()
         self._executor = ThreadPoolExecutor(max_workers=4, thread_name_prefix="ReplicationWorker")
         self._shutdown = False
-        self.failure_store = ReplicationFailureStore(storage_root, max_failures_per_bucket)
+        self.failure_store = ReplicationFailureStore(storage_root)
         self.reload_rules()
 
-    def _create_client(self, connection: RemoteConnection, *, health_check: bool = False) -> Any:
-        """Create an S3 client with the manager's configured timeouts."""
-        return _create_s3_client(
-            connection,
-            health_check=health_check,
-            connect_timeout=self.connect_timeout,
-            read_timeout=self.read_timeout,
-            max_retries=self.max_retries,
-        )
-
     def shutdown(self, wait: bool = True) -> None:
         """Shutdown the replication executor gracefully.
 
@@ -326,7 +271,7 @@ class ReplicationManager:
         Uses short timeouts to prevent blocking.
         """
         try:
-            s3 = self._create_client(connection, health_check=True)
+            s3 = _create_s3_client(connection, health_check=True)
             s3.list_buckets()
             return True
         except Exception as e:
@@ -336,9 +281,6 @@ class ReplicationManager:
     def get_rule(self, bucket_name: str) -> Optional[ReplicationRule]:
         return self._rules.get(bucket_name)
 
-    def list_rules(self) -> List[ReplicationRule]:
-        return list(self._rules.values())
-
     def set_rule(self, rule: ReplicationRule) -> None:
         old_rule = self._rules.get(rule.bucket_name)
         was_all_mode = old_rule and old_rule.mode == REPLICATION_MODE_ALL if old_rule else False
@@ -378,7 +320,7 @@ class ReplicationManager:
             source_objects = self.storage.list_objects_all(bucket_name)
             source_keys = {obj.key: obj.size for obj in source_objects}
 
-            s3 = self._create_client(connection)
+            s3 = _create_s3_client(connection)
 
             dest_keys = set()
             bytes_synced = 0
@@ -444,7 +386,7 @@ class ReplicationManager:
             raise ValueError(f"Connection {connection_id} not found")
 
         try:
-            s3 = self._create_client(connection)
+            s3 = _create_s3_client(connection)
             s3.create_bucket(Bucket=bucket_name)
         except ClientError as e:
             logger.error(f"Failed to create remote bucket {bucket_name}: {e}")
@@ -487,7 +429,7 @@ class ReplicationManager:
             return
 
         try:
-            s3 = self._create_client(conn)
+            s3 = _create_s3_client(conn)
 
             if action == "delete":
                 try:
@@ -530,7 +472,7 @@ class ReplicationManager:
                 if content_type:
                     extra_args["ContentType"] = content_type
 
-                if file_size >= self.streaming_threshold_bytes:
+                if file_size >= STREAMING_THRESHOLD_BYTES:
                     s3.upload_file(
                         str(path),
                         rule.target_bucket,

app/secret_store.py

@@ -18,18 +18,6 @@ class EphemeralSecretStore:
         self._store[token] = (payload, expires_at)
         return token
 
-    def peek(self, token: str | None) -> Any | None:
-        if not token:
-            return None
-        entry = self._store.get(token)
-        if not entry:
-            return None
-        payload, expires_at = entry
-        if expires_at < time.time():
-            self._store.pop(token, None)
-            return None
-        return payload
-
     def pop(self, token: str | None) -> Any | None:
         if not token:
             return None

app/version.py

@@ -1,6 +1,6 @@
 from __future__ import annotations
 
-APP_VERSION = "0.4.3"
+APP_VERSION = "0.2.2"
 
 
 def get_version() -> str:

docker-entrypoint.sh

@@ -0,0 +1,5 @@
+#!/bin/sh
+set -e
+
+# Run both services using the python runner in production mode
+exec python run.py --prod

python/Dockerfile

@@ -1,56 +0,0 @@
-FROM python:3.14.3-slim AS builder
-
-ENV PYTHONDONTWRITEBYTECODE=1 \
-    PYTHONUNBUFFERED=1
-
-WORKDIR /build
-
-RUN apt-get update \
-    && apt-get install -y --no-install-recommends build-essential curl \
-    && curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal \
-    && rm -rf /var/lib/apt/lists/*
-
-ENV PATH="/root/.cargo/bin:${PATH}"
-
-RUN pip install --no-cache-dir maturin
-
-COPY myfsio_core ./myfsio_core
-RUN cd myfsio_core \
-    && maturin build --release --out /wheels
-
-
-FROM python:3.14.3-slim
-
-ENV PYTHONDONTWRITEBYTECODE=1 \
-    PYTHONUNBUFFERED=1
-
-WORKDIR /app
-
-COPY requirements.txt ./
-RUN pip install --no-cache-dir -r requirements.txt
-
-COPY --from=builder /wheels/*.whl /tmp/
-RUN pip install --no-cache-dir /tmp/*.whl && rm /tmp/*.whl
-
-COPY app ./app
-COPY templates ./templates
-COPY static ./static
-COPY run.py ./
-COPY docker-entrypoint.sh ./
-
-RUN chmod +x docker-entrypoint.sh \
-    && mkdir -p /app/data \
-    && useradd -m -u 1000 myfsio \
-    && chown -R myfsio:myfsio /app
-
-USER myfsio
-
-EXPOSE 5000 5100
-ENV APP_HOST=0.0.0.0 \
-    FLASK_ENV=production \
-    FLASK_DEBUG=0
-
-HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
-    CMD python -c "import requests; requests.get('http://localhost:5000/myfsio/health', timeout=2)"
-
-CMD ["./docker-entrypoint.sh"]

python/README.md

@@ -1,14 +0,0 @@
-# Deprecated Python Implementation
-
-The Python implementation of MyFSIO is deprecated as of 2026-04-21.
-
-The supported server runtime now lives in `../rust/myfsio-engine` and serves the S3 API and web UI from the Rust `myfsio-server` binary. Keep this tree for migration reference, compatibility checks, and legacy tests only.
-
-For normal development and operations, run:
-
-```bash
-cd ../rust/myfsio-engine
-cargo run -p myfsio-server --
-```
-
-Do not add new product features to the Python implementation unless they are needed to unblock a migration or compare behavior with the Rust server.

python/app/__init__.py

@@ -1,763 +0,0 @@
-from __future__ import annotations
-
-import itertools
-import logging
-import mimetypes
-import os
-import shutil
-import sys
-import time
-from logging.handlers import RotatingFileHandler
-from pathlib import Path
-from datetime import timedelta
-from typing import Any, Dict, List, Optional
-
-from flask import Flask, Response, g, has_request_context, redirect, render_template, request, url_for
-from flask_cors import CORS
-from flask_wtf.csrf import CSRFError
-from werkzeug.middleware.proxy_fix import ProxyFix
-
-import io
-
-from .access_logging import AccessLoggingService
-from .operation_metrics import OperationMetricsCollector, classify_endpoint
-from .compression import GzipMiddleware
-from .acl import AclService
-from .bucket_policies import BucketPolicyStore
-from .config import AppConfig
-from .connections import ConnectionStore
-from .encryption import EncryptionManager
-from .extensions import limiter, csrf
-from .iam import IamService
-from .kms import KMSManager
-from .gc import GarbageCollector
-from .integrity import IntegrityChecker
-from .lifecycle import LifecycleManager
-from .notifications import NotificationService
-from .object_lock import ObjectLockService
-from .replication import ReplicationManager
-from .secret_store import EphemeralSecretStore
-from .site_registry import SiteRegistry, SiteInfo
-from .storage import ObjectStorage, StorageError
-from .version import get_version
-from .website_domains import WebsiteDomainStore
-
-_request_counter = itertools.count(1)
-
-
-class _ChunkedTransferMiddleware:
-
-    def __init__(self, app):
-        self.app = app
-
-    def __call__(self, environ, start_response):
-        if environ.get("REQUEST_METHOD") not in ("PUT", "POST"):
-            return self.app(environ, start_response)
-
-        transfer_encoding = environ.get("HTTP_TRANSFER_ENCODING", "")
-        content_length = environ.get("CONTENT_LENGTH")
-
-        if "chunked" in transfer_encoding.lower():
-            if content_length:
-                del environ["HTTP_TRANSFER_ENCODING"]
-            else:
-                raw = environ.get("wsgi.input")
-                if raw:
-                    try:
-                        if hasattr(raw, "seek"):
-                            raw.seek(0)
-                        body = raw.read()
-                    except Exception:
-                        body = b""
-                    if body:
-                        environ["wsgi.input"] = io.BytesIO(body)
-                        environ["CONTENT_LENGTH"] = str(len(body))
-                del environ["HTTP_TRANSFER_ENCODING"]
-
-        content_length = environ.get("CONTENT_LENGTH")
-        if not content_length or content_length == "0":
-            sha256 = environ.get("HTTP_X_AMZ_CONTENT_SHA256", "")
-            decoded_len = environ.get("HTTP_X_AMZ_DECODED_CONTENT_LENGTH", "")
-            content_encoding = environ.get("HTTP_CONTENT_ENCODING", "")
-            if ("STREAMING" in sha256.upper() or decoded_len
-                    or "aws-chunked" in content_encoding.lower()):
-                raw = environ.get("wsgi.input")
-                if raw:
-                    try:
-                        if hasattr(raw, "seek"):
-                            raw.seek(0)
-                        body = raw.read()
-                    except Exception:
-                        body = b""
-                    if body:
-                        environ["wsgi.input"] = io.BytesIO(body)
-                        environ["CONTENT_LENGTH"] = str(len(body))
-
-        raw = environ.get("wsgi.input")
-        if raw and hasattr(raw, "seek"):
-            try:
-                raw.seek(0)
-            except Exception:
-                pass
-
-        return self.app(environ, start_response)
-
-
-def _migrate_config_file(active_path: Path, legacy_paths: List[Path]) -> Path:
-    """Migrate config file from legacy locations to the active path.
-    
-    Checks each legacy path in order and moves the first one found to the active path.
-    This ensures backward compatibility for users upgrading from older versions.
-    """
-    active_path.parent.mkdir(parents=True, exist_ok=True)
-    
-    if active_path.exists():
-        return active_path
-    
-    for legacy_path in legacy_paths:
-        if legacy_path.exists():
-            try:
-                shutil.move(str(legacy_path), str(active_path))
-            except OSError:
-                shutil.copy2(legacy_path, active_path)
-                try:
-                    legacy_path.unlink(missing_ok=True)
-                except OSError:
-                    pass
-            break
-    
-    return active_path
-
-
-def create_app(
-    test_config: Optional[Dict[str, Any]] = None,
-    *,
-    include_api: bool = True,
-    include_ui: bool = True,
-) -> Flask:
-    """Create and configure the Flask application."""
-    config = AppConfig.from_env(test_config)
-
-    if getattr(sys, "frozen", False):
-        project_root = Path(sys._MEIPASS)
-    else:
-        project_root = Path(__file__).resolve().parent.parent
-
-    app = Flask(
-        __name__,
-        static_folder=str(project_root / "static"),
-        template_folder=str(project_root / "templates"),
-    )
-    app.config.update(config.to_flask_config())
-    if test_config:
-        app.config.update(test_config)
-    app.config.setdefault("APP_VERSION", get_version())
-    app.permanent_session_lifetime = timedelta(days=int(app.config.get("SESSION_LIFETIME_DAYS", 30)))
-    if app.config.get("TESTING"):
-        app.config.setdefault("WTF_CSRF_ENABLED", False)
-
-    # Trust X-Forwarded-* headers from proxies
-    num_proxies = app.config.get("NUM_TRUSTED_PROXIES", 1)
-    if num_proxies:
-        if "NUM_TRUSTED_PROXIES" not in os.environ:
-            logging.getLogger(__name__).warning(
-                "NUM_TRUSTED_PROXIES not set, defaulting to 1. "
-                "Set NUM_TRUSTED_PROXIES=0 if not behind a reverse proxy."
-            )
-        app.wsgi_app = ProxyFix(app.wsgi_app, x_for=num_proxies, x_proto=num_proxies, x_host=num_proxies, x_prefix=num_proxies)
-
-    if app.config.get("ENABLE_GZIP", True):
-        app.wsgi_app = GzipMiddleware(app.wsgi_app, compression_level=6)
-
-    app.wsgi_app = _ChunkedTransferMiddleware(app.wsgi_app)
-
-    _configure_cors(app)
-    _configure_logging(app)
-
-    limiter.init_app(app)
-    csrf.init_app(app)
-
-    storage = ObjectStorage(
-        Path(app.config["STORAGE_ROOT"]),
-        cache_ttl=app.config.get("OBJECT_CACHE_TTL", 60),
-        object_cache_max_size=app.config.get("OBJECT_CACHE_MAX_SIZE", 100),
-        bucket_config_cache_ttl=app.config.get("BUCKET_CONFIG_CACHE_TTL_SECONDS", 30.0),
-        object_key_max_length_bytes=app.config.get("OBJECT_KEY_MAX_LENGTH_BYTES", 1024),
-        meta_read_cache_max=app.config.get("META_READ_CACHE_MAX", 2048),
-    )
-
-    if app.config.get("WARM_CACHE_ON_STARTUP", True) and not app.config.get("TESTING"):
-        storage.warm_cache_async()
-
-    iam = IamService(
-        Path(app.config["IAM_CONFIG"]),
-        auth_max_attempts=app.config.get("AUTH_MAX_ATTEMPTS", 5),
-        auth_lockout_minutes=app.config.get("AUTH_LOCKOUT_MINUTES", 15),
-        encryption_key=app.config.get("SECRET_KEY"),
-    )
-    bucket_policies = BucketPolicyStore(Path(app.config["BUCKET_POLICY_PATH"]))
-    secret_store = EphemeralSecretStore(default_ttl=app.config.get("SECRET_TTL_SECONDS", 300))
-    
-    storage_root = Path(app.config["STORAGE_ROOT"])
-    config_dir = storage_root / ".myfsio.sys" / "config"
-    config_dir.mkdir(parents=True, exist_ok=True)
-    
-    connections_path = _migrate_config_file(
-        active_path=config_dir / "connections.json",
-        legacy_paths=[
-            storage_root / ".myfsio.sys" / "connections.json",
-            storage_root / ".connections.json",
-        ],
-    )
-    replication_rules_path = _migrate_config_file(
-        active_path=config_dir / "replication_rules.json",
-        legacy_paths=[
-            storage_root / ".myfsio.sys" / "replication_rules.json",
-            storage_root / ".replication_rules.json",
-        ],
-    )
-    
-    connections = ConnectionStore(connections_path)
-    replication = ReplicationManager(
-        storage,
-        connections,
-        replication_rules_path,
-        storage_root,
-        connect_timeout=app.config.get("REPLICATION_CONNECT_TIMEOUT_SECONDS", 5),
-        read_timeout=app.config.get("REPLICATION_READ_TIMEOUT_SECONDS", 30),
-        max_retries=app.config.get("REPLICATION_MAX_RETRIES", 2),
-        streaming_threshold_bytes=app.config.get("REPLICATION_STREAMING_THRESHOLD_BYTES", 10 * 1024 * 1024),
-        max_failures_per_bucket=app.config.get("REPLICATION_MAX_FAILURES_PER_BUCKET", 50),
-    )
-
-    site_registry_path = config_dir / "site_registry.json"
-    site_registry = SiteRegistry(site_registry_path)
-    if app.config.get("SITE_ID") and not site_registry.get_local_site():
-        site_registry.set_local_site(SiteInfo(
-            site_id=app.config["SITE_ID"],
-            endpoint=app.config.get("SITE_ENDPOINT") or "",
-            region=app.config.get("SITE_REGION", "us-east-1"),
-            priority=app.config.get("SITE_PRIORITY", 100),
-        ))
-
-    encryption_config = {
-        "encryption_enabled": app.config.get("ENCRYPTION_ENABLED", False),
-        "encryption_master_key_path": app.config.get("ENCRYPTION_MASTER_KEY_PATH"),
-        "default_encryption_algorithm": app.config.get("DEFAULT_ENCRYPTION_ALGORITHM", "AES256"),
-        "encryption_chunk_size_bytes": app.config.get("ENCRYPTION_CHUNK_SIZE_BYTES", 64 * 1024),
-    }
-    encryption_manager = EncryptionManager(encryption_config)
-    
-    kms_manager = None
-    if app.config.get("KMS_ENABLED", False):
-        kms_keys_path = Path(app.config.get("KMS_KEYS_PATH", ""))
-        kms_master_key_path = Path(app.config.get("ENCRYPTION_MASTER_KEY_PATH", ""))
-        kms_manager = KMSManager(
-            kms_keys_path,
-            kms_master_key_path,
-            generate_data_key_min_bytes=app.config.get("KMS_GENERATE_DATA_KEY_MIN_BYTES", 1),
-            generate_data_key_max_bytes=app.config.get("KMS_GENERATE_DATA_KEY_MAX_BYTES", 1024),
-        )
-        encryption_manager.set_kms_provider(kms_manager)
-
-    if app.config.get("ENCRYPTION_ENABLED", False):
-        from .encrypted_storage import EncryptedObjectStorage
-        storage = EncryptedObjectStorage(storage, encryption_manager)
-
-    acl_service = AclService(storage_root)
-    object_lock_service = ObjectLockService(storage_root)
-    notification_service = NotificationService(
-        storage_root,
-        allow_internal_endpoints=app.config.get("ALLOW_INTERNAL_ENDPOINTS", False),
-    )
-    access_logging_service = AccessLoggingService(storage_root)
-    access_logging_service.set_storage(storage)
-
-    lifecycle_manager = None
-    if app.config.get("LIFECYCLE_ENABLED", False):
-        base_storage = storage.storage if hasattr(storage, 'storage') else storage
-        lifecycle_manager = LifecycleManager(
-            base_storage,
-            interval_seconds=app.config.get("LIFECYCLE_INTERVAL_SECONDS", 3600),
-            storage_root=storage_root,
-            max_history_per_bucket=app.config.get("LIFECYCLE_MAX_HISTORY_PER_BUCKET", 50),
-        )
-        lifecycle_manager.start()
-
-    gc_collector = None
-    if app.config.get("GC_ENABLED", False):
-        gc_collector = GarbageCollector(
-            storage_root=storage_root,
-            interval_hours=app.config.get("GC_INTERVAL_HOURS", 6.0),
-            temp_file_max_age_hours=app.config.get("GC_TEMP_FILE_MAX_AGE_HOURS", 24.0),
-            multipart_max_age_days=app.config.get("GC_MULTIPART_MAX_AGE_DAYS", 7),
-            lock_file_max_age_hours=app.config.get("GC_LOCK_FILE_MAX_AGE_HOURS", 1.0),
-            dry_run=app.config.get("GC_DRY_RUN", False),
-            io_throttle_ms=app.config.get("GC_IO_THROTTLE_MS", 10),
-        )
-        gc_collector.start()
-
-    integrity_checker = None
-    if app.config.get("INTEGRITY_ENABLED", False):
-        integrity_checker = IntegrityChecker(
-            storage_root=storage_root,
-            interval_hours=app.config.get("INTEGRITY_INTERVAL_HOURS", 24.0),
-            batch_size=app.config.get("INTEGRITY_BATCH_SIZE", 1000),
-            auto_heal=app.config.get("INTEGRITY_AUTO_HEAL", False),
-            dry_run=app.config.get("INTEGRITY_DRY_RUN", False),
-            io_throttle_ms=app.config.get("INTEGRITY_IO_THROTTLE_MS", 10),
-        )
-        integrity_checker.start()
-
-    app.extensions["object_storage"] = storage
-    app.extensions["iam"] = iam
-    app.extensions["bucket_policies"] = bucket_policies
-    app.extensions["secret_store"] = secret_store
-    app.extensions["limiter"] = limiter
-    app.extensions["connections"] = connections
-    app.extensions["replication"] = replication
-    app.extensions["encryption"] = encryption_manager
-    app.extensions["kms"] = kms_manager
-    app.extensions["acl"] = acl_service
-    app.extensions["lifecycle"] = lifecycle_manager
-    app.extensions["gc"] = gc_collector
-    app.extensions["integrity"] = integrity_checker
-    app.extensions["object_lock"] = object_lock_service
-    app.extensions["notifications"] = notification_service
-    app.extensions["access_logging"] = access_logging_service
-    app.extensions["site_registry"] = site_registry
-
-    website_domains_store = None
-    if app.config.get("WEBSITE_HOSTING_ENABLED", False):
-        website_domains_path = config_dir / "website_domains.json"
-        website_domains_store = WebsiteDomainStore(website_domains_path)
-    app.extensions["website_domains"] = website_domains_store
-
-    from .s3_client import S3ProxyClient
-    api_base = app.config.get("API_BASE_URL") or "http://127.0.0.1:5000"
-    app.extensions["s3_proxy"] = S3ProxyClient(
-        api_base_url=api_base,
-        region=app.config.get("AWS_REGION", "us-east-1"),
-    )
-
-    operation_metrics_collector = None
-    if app.config.get("OPERATION_METRICS_ENABLED", False):
-        operation_metrics_collector = OperationMetricsCollector(
-            storage_root,
-            interval_minutes=app.config.get("OPERATION_METRICS_INTERVAL_MINUTES", 5),
-            retention_hours=app.config.get("OPERATION_METRICS_RETENTION_HOURS", 24),
-        )
-    app.extensions["operation_metrics"] = operation_metrics_collector
-
-    system_metrics_collector = None
-    if app.config.get("METRICS_HISTORY_ENABLED", False):
-        from .system_metrics import SystemMetricsCollector
-        system_metrics_collector = SystemMetricsCollector(
-            storage_root,
-            interval_minutes=app.config.get("METRICS_HISTORY_INTERVAL_MINUTES", 5),
-            retention_hours=app.config.get("METRICS_HISTORY_RETENTION_HOURS", 24),
-        )
-        system_metrics_collector.set_storage(storage)
-    app.extensions["system_metrics"] = system_metrics_collector
-
-    site_sync_worker = None
-    if app.config.get("SITE_SYNC_ENABLED", False):
-        from .site_sync import SiteSyncWorker
-        site_sync_worker = SiteSyncWorker(
-            storage=storage,
-            connections=connections,
-            replication_manager=replication,
-            storage_root=storage_root,
-            interval_seconds=app.config.get("SITE_SYNC_INTERVAL_SECONDS", 60),
-            batch_size=app.config.get("SITE_SYNC_BATCH_SIZE", 100),
-            connect_timeout=app.config.get("SITE_SYNC_CONNECT_TIMEOUT_SECONDS", 10),
-            read_timeout=app.config.get("SITE_SYNC_READ_TIMEOUT_SECONDS", 120),
-            max_retries=app.config.get("SITE_SYNC_MAX_RETRIES", 2),
-            clock_skew_tolerance_seconds=app.config.get("SITE_SYNC_CLOCK_SKEW_TOLERANCE_SECONDS", 1.0),
-        )
-        site_sync_worker.start()
-    app.extensions["site_sync"] = site_sync_worker
-
-    @app.errorhandler(500)
-    def internal_error(error):
-        wants_html = request.accept_mimetypes.accept_html
-        path = request.path or ""
-        if include_ui and wants_html and (path.startswith("/ui") or path == "/"):
-            return render_template('500.html'), 500
-        error_xml = (
-            '<?xml version="1.0" encoding="UTF-8"?>'
-            '<Error>'
-            '<Code>InternalError</Code>'
-            '<Message>An internal server error occurred</Message>'
-            f'<Resource>{path}</Resource>'
-            f'<RequestId>{getattr(g, "request_id", "-")}</RequestId>'
-            '</Error>'
-        )
-        return error_xml, 500, {'Content-Type': 'application/xml'}
-
-    @app.errorhandler(CSRFError)
-    def handle_csrf_error(e):
-        wants_html = request.accept_mimetypes.accept_html
-        path = request.path or ""
-        if include_ui and wants_html and (path.startswith("/ui") or path == "/"):
-            return render_template('csrf_error.html', reason=e.description), 400
-        error_xml = (
-            '<?xml version="1.0" encoding="UTF-8"?>'
-            '<Error>'
-            '<Code>CSRFError</Code>'
-            f'<Message>{e.description}</Message>'
-            f'<Resource>{path}</Resource>'
-            f'<RequestId>{getattr(g, "request_id", "-")}</RequestId>'
-            '</Error>'
-        )
-        return error_xml, 400, {'Content-Type': 'application/xml'}
-
-    @app.template_filter("filesizeformat")
-    def filesizeformat(value: int) -> str:
-        """Format bytes as human-readable file size."""
-        for unit in ["B", "KB", "MB", "GB", "TB", "PB"]:
-            if abs(value) < 1024.0 or unit == "PB":
-                if unit == "B":
-                    return f"{int(value)} {unit}"
-                return f"{value:.1f} {unit}"
-            value /= 1024.0
-        return f"{value:.1f} PB"
-
-    @app.template_filter("timestamp_to_datetime")
-    def timestamp_to_datetime(value: float) -> str:
-        """Format Unix timestamp as human-readable datetime in configured timezone."""
-        from datetime import datetime, timezone as dt_timezone
-        from zoneinfo import ZoneInfo
-        if not value:
-            return "Never"
-        try:
-            dt_utc = datetime.fromtimestamp(value, dt_timezone.utc)
-            display_tz = app.config.get("DISPLAY_TIMEZONE", "UTC")
-            if display_tz and display_tz != "UTC":
-                try:
-                    tz = ZoneInfo(display_tz)
-                    dt_local = dt_utc.astimezone(tz)
-                    return dt_local.strftime("%Y-%m-%d %H:%M:%S")
-                except (KeyError, ValueError):
-                    pass 
-            return dt_utc.strftime("%Y-%m-%d %H:%M:%S UTC")
-        except (ValueError, OSError):
-            return "Unknown"
-
-    @app.template_filter("format_datetime")
-    def format_datetime_filter(dt, include_tz: bool = True) -> str:
-        """Format datetime object as human-readable string in configured timezone."""
-        from datetime import datetime, timezone as dt_timezone
-        from zoneinfo import ZoneInfo
-        if not dt:
-            return ""
-        try:
-            display_tz = app.config.get("DISPLAY_TIMEZONE", "UTC")
-            if display_tz and display_tz != "UTC":
-                try:
-                    tz = ZoneInfo(display_tz)
-                    if dt.tzinfo is None:
-                        dt = dt.replace(tzinfo=dt_timezone.utc)
-                    dt = dt.astimezone(tz)
-                except (KeyError, ValueError):
-                    pass
-            tz_abbr = dt.strftime("%Z") or "UTC"
-            if include_tz:
-                return f"{dt.strftime('%b %d, %Y %H:%M')} ({tz_abbr})"
-            return dt.strftime("%b %d, %Y %H:%M")
-        except (ValueError, AttributeError):
-            return str(dt)
-
-    if include_api:
-        from .s3_api import s3_api_bp
-        from .kms_api import kms_api_bp
-        from .admin_api import admin_api_bp
-
-        app.register_blueprint(s3_api_bp)
-        app.register_blueprint(kms_api_bp)
-        app.register_blueprint(admin_api_bp)
-        csrf.exempt(s3_api_bp)
-        csrf.exempt(kms_api_bp)
-        csrf.exempt(admin_api_bp)
-
-    if include_ui:
-        from .ui import ui_bp
-
-        app.register_blueprint(ui_bp)
-        if not include_api:
-            @app.get("/")
-            def ui_root_redirect():
-                return redirect(url_for("ui.buckets_overview"))
-
-    @app.errorhandler(404)
-    def handle_not_found(error):
-        wants_html = request.accept_mimetypes.accept_html
-        path = request.path or ""
-        if include_ui and wants_html:
-            if not include_api or path.startswith("/ui") or path == "/":
-                return render_template("404.html"), 404
-        return error
-
-    @app.get("/myfsio/health")
-    def healthcheck() -> Dict[str, str]:
-        return {"status": "ok"}
-
-    return app
-
-
-def create_api_app(test_config: Optional[Dict[str, Any]] = None) -> Flask:
-    return create_app(test_config, include_api=True, include_ui=False)
-
-
-def create_ui_app(test_config: Optional[Dict[str, Any]] = None) -> Flask:
-    return create_app(test_config, include_api=False, include_ui=True)
-
-
-def _configure_cors(app: Flask) -> None:
-    origins = app.config.get("CORS_ORIGINS", ["*"])
-    methods = app.config.get("CORS_METHODS", ["GET", "PUT", "POST", "DELETE", "OPTIONS", "HEAD"])
-    allow_headers = app.config.get("CORS_ALLOW_HEADERS", ["*"])
-    expose_headers = app.config.get("CORS_EXPOSE_HEADERS", ["*"])
-    CORS(
-        app,
-        resources={r"/*": {"origins": origins, "methods": methods, "allow_headers": allow_headers, "expose_headers": expose_headers}},
-        supports_credentials=True,
-    )
-
-
-class _RequestContextFilter(logging.Filter):
-    """Inject request-specific attributes into log records."""
-
-    def filter(self, record: logging.LogRecord) -> bool:  
-        if has_request_context():
-            record.request_id = getattr(g, "request_id", "-")
-            record.path = request.path
-            record.method = request.method
-            record.remote_addr = request.remote_addr or "-"
-        else:
-            record.request_id = getattr(record, "request_id", "-")
-            record.path = getattr(record, "path", "-")
-            record.method = getattr(record, "method", "-")
-            record.remote_addr = getattr(record, "remote_addr", "-")
-        return True
-
-
-def _configure_logging(app: Flask) -> None:
-    formatter = logging.Formatter(
-        "%(asctime)s | %(levelname)s | %(request_id)s | %(method)s %(path)s | %(message)s"
-    )
-
-    stream_handler = logging.StreamHandler(sys.stdout)
-    stream_handler.setFormatter(formatter)
-    stream_handler.addFilter(_RequestContextFilter())
-
-    logger = app.logger
-    for handler in logger.handlers[:]:
-        handler.close()
-    logger.handlers.clear()
-    logger.addHandler(stream_handler)
-
-    if app.config.get("LOG_TO_FILE"):
-        log_file = Path(app.config["LOG_FILE"])
-        log_file.parent.mkdir(parents=True, exist_ok=True)
-        file_handler = RotatingFileHandler(
-            log_file,
-            maxBytes=int(app.config.get("LOG_MAX_BYTES", 5 * 1024 * 1024)),
-            backupCount=int(app.config.get("LOG_BACKUP_COUNT", 3)),
-            encoding="utf-8",
-        )
-        file_handler.setFormatter(formatter)
-        file_handler.addFilter(_RequestContextFilter())
-        logger.addHandler(file_handler)
-
-    logger.setLevel(getattr(logging, app.config.get("LOG_LEVEL", "INFO"), logging.INFO))
-
-    @app.before_request
-    def _log_request_start() -> None:
-        g.request_id = f"{os.getpid():x}{next(_request_counter):012x}"
-        g.request_started_at = time.perf_counter()
-        g.request_bytes_in = request.content_length or 0
-
-    @app.before_request
-    def _maybe_serve_website():
-        if not app.config.get("WEBSITE_HOSTING_ENABLED"):
-            return None
-        if request.method not in {"GET", "HEAD"}:
-            return None
-        host = request.host
-        if ":" in host:
-            host = host.rsplit(":", 1)[0]
-        host = host.lower()
-        store = app.extensions.get("website_domains")
-        if not store:
-            return None
-        bucket = store.get_bucket(host)
-        if not bucket:
-            return None
-        storage = app.extensions["object_storage"]
-        if not storage.bucket_exists(bucket):
-            return _website_error_response(404, "Not Found")
-        website_config = storage.get_bucket_website(bucket)
-        if not website_config:
-            return _website_error_response(404, "Not Found")
-        index_doc = website_config.get("index_document", "index.html")
-        error_doc = website_config.get("error_document")
-        req_path = request.path.lstrip("/")
-        if not req_path or req_path.endswith("/"):
-            object_key = req_path + index_doc
-        else:
-            object_key = req_path
-        try:
-            obj_path = storage.get_object_path(bucket, object_key)
-        except (StorageError, OSError):
-            if object_key == req_path:
-                try:
-                    obj_path = storage.get_object_path(bucket, req_path + "/" + index_doc)
-                    object_key = req_path + "/" + index_doc
-                except (StorageError, OSError):
-                    return _serve_website_error(storage, bucket, error_doc, 404)
-            else:
-                return _serve_website_error(storage, bucket, error_doc, 404)
-        content_type = mimetypes.guess_type(object_key)[0] or "application/octet-stream"
-        is_encrypted = False
-        try:
-            metadata = storage.get_object_metadata(bucket, object_key)
-            is_encrypted = "x-amz-server-side-encryption" in metadata
-        except (StorageError, OSError):
-            pass
-        if is_encrypted and hasattr(storage, "get_object_data"):
-            try:
-                data, _ = storage.get_object_data(bucket, object_key)
-                file_size = len(data)
-            except (StorageError, OSError):
-                return _website_error_response(500, "Internal Server Error")
-        else:
-            data = None
-            try:
-                stat = obj_path.stat()
-                file_size = stat.st_size
-            except OSError:
-                return _website_error_response(500, "Internal Server Error")
-        if request.method == "HEAD":
-            response = Response(status=200)
-            response.headers["Content-Length"] = file_size
-            response.headers["Content-Type"] = content_type
-            response.headers["Accept-Ranges"] = "bytes"
-            return response
-        from .s3_api import _parse_range_header
-        range_header = request.headers.get("Range")
-        if range_header:
-            ranges = _parse_range_header(range_header, file_size)
-            if ranges is None:
-                return Response(status=416, headers={"Content-Range": f"bytes */{file_size}"})
-            start, end = ranges[0]
-            length = end - start + 1
-            if data is not None:
-                partial_data = data[start:end + 1]
-                response = Response(partial_data, status=206, mimetype=content_type)
-            else:
-                def _stream_range(file_path, start_pos, length_to_read):
-                    with file_path.open("rb") as f:
-                        f.seek(start_pos)
-                        remaining = length_to_read
-                        while remaining > 0:
-                            chunk = f.read(min(262144, remaining))
-                            if not chunk:
-                                break
-                            remaining -= len(chunk)
-                            yield chunk
-                response = Response(_stream_range(obj_path, start, length), status=206, mimetype=content_type, direct_passthrough=True)
-            response.headers["Content-Range"] = f"bytes {start}-{end}/{file_size}"
-            response.headers["Content-Length"] = length
-            response.headers["Accept-Ranges"] = "bytes"
-            return response
-        if data is not None:
-            response = Response(data, mimetype=content_type)
-            response.headers["Content-Length"] = file_size
-            response.headers["Accept-Ranges"] = "bytes"
-            return response
-        def _stream(file_path):
-            with file_path.open("rb") as f:
-                while True:
-                    chunk = f.read(65536)
-                    if not chunk:
-                        break
-                    yield chunk
-        response = Response(_stream(obj_path), mimetype=content_type, direct_passthrough=True)
-        response.headers["Content-Length"] = file_size
-        response.headers["Accept-Ranges"] = "bytes"
-        return response
-
-    def _serve_website_error(storage, bucket, error_doc_key, status_code):
-        if not error_doc_key:
-            return _website_error_response(status_code, "Not Found" if status_code == 404 else "Error")
-        try:
-            obj_path = storage.get_object_path(bucket, error_doc_key)
-        except (StorageError, OSError):
-            return _website_error_response(status_code, "Not Found")
-        content_type = mimetypes.guess_type(error_doc_key)[0] or "text/html"
-        is_encrypted = False
-        try:
-            metadata = storage.get_object_metadata(bucket, error_doc_key)
-            is_encrypted = "x-amz-server-side-encryption" in metadata
-        except (StorageError, OSError):
-            pass
-        if is_encrypted and hasattr(storage, "get_object_data"):
-            try:
-                data, _ = storage.get_object_data(bucket, error_doc_key)
-                response = Response(data, status=status_code, mimetype=content_type)
-                response.headers["Content-Length"] = len(data)
-                return response
-            except (StorageError, OSError):
-                return _website_error_response(status_code, "Not Found")
-        try:
-            data = obj_path.read_bytes()
-            response = Response(data, status=status_code, mimetype=content_type)
-            response.headers["Content-Length"] = len(data)
-            return response
-        except OSError:
-            return _website_error_response(status_code, "Not Found")
-
-    def _website_error_response(status_code, message):
-        if status_code == 404:
-            body = "<h1>404 page not found</h1>"
-        else:
-            body = f"{status_code} {message}"
-        return Response(body, status=status_code, mimetype="text/html")
-
-    @app.after_request
-    def _log_request_end(response):
-        duration_ms = 0.0
-        if hasattr(g, "request_started_at"):
-            duration_ms = (time.perf_counter() - g.request_started_at) * 1000
-        request_id = getattr(g, "request_id", f"{os.getpid():x}{next(_request_counter):012x}")
-        response.headers.setdefault("X-Request-ID", request_id)
-        if app.logger.isEnabledFor(logging.INFO):
-            app.logger.info(
-                "Request completed",
-                extra={
-                    "path": request.path,
-                    "method": request.method,
-                    "remote_addr": request.remote_addr,
-                },
-            )
-        response.headers["X-Request-Duration-ms"] = f"{duration_ms:.2f}"
-        response.headers["Server"] = "MyFSIO"
-
-        operation_metrics = app.extensions.get("operation_metrics")
-        if operation_metrics:
-            bytes_in = getattr(g, "request_bytes_in", 0)
-            bytes_out = response.content_length or 0
-            error_code = getattr(g, "s3_error_code", None)
-            endpoint_type = classify_endpoint(request.path)
-            operation_metrics.record_request(
-                method=request.method,
-                endpoint_type=endpoint_type,
-                status_code=response.status_code,
-                latency_ms=duration_ms,
-                bytes_in=bytes_in,
-                bytes_out=bytes_out,
-                error_code=error_code,
-            )
-
-        return response

python/app/admin_api.py

@@ -1,984 +0,0 @@
-from __future__ import annotations
-
-import ipaddress
-import json
-import logging
-import re
-import socket
-import time
-from typing import Any, Dict, Optional, Tuple
-from urllib.parse import urlparse
-
-import requests
-from flask import Blueprint, Response, current_app, jsonify, request
-
-from .connections import ConnectionStore
-from .extensions import limiter
-from .gc import GarbageCollector
-from .integrity import IntegrityChecker
-from .iam import IamError, Principal
-from .replication import ReplicationManager
-from .site_registry import PeerSite, SiteInfo, SiteRegistry
-from .website_domains import WebsiteDomainStore, normalize_domain, is_valid_domain
-
-
-def _is_safe_url(url: str, allow_internal: bool = False) -> bool:
-    """Check if a URL is safe to make requests to (not internal/private).
-
-    Args:
-        url: The URL to check.
-        allow_internal: If True, allows internal/private IP addresses.
-                       Use for self-hosted deployments on internal networks.
-    """
-    try:
-        parsed = urlparse(url)
-        hostname = parsed.hostname
-        if not hostname:
-            return False
-        cloud_metadata_hosts = {
-            "metadata.google.internal",
-            "169.254.169.254",
-        }
-        if hostname.lower() in cloud_metadata_hosts:
-            return False
-        if allow_internal:
-            return True
-        blocked_hosts = {
-            "localhost",
-            "127.0.0.1",
-            "0.0.0.0",
-            "::1",
-            "[::1]",
-        }
-        if hostname.lower() in blocked_hosts:
-            return False
-        try:
-            resolved_ip = socket.gethostbyname(hostname)
-            ip = ipaddress.ip_address(resolved_ip)
-            if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
-                return False
-        except (socket.gaierror, ValueError):
-            return False
-        return True
-    except Exception:
-        return False
-
-
-def _validate_endpoint(endpoint: str) -> Optional[str]:
-    """Validate endpoint URL format. Returns error message or None."""
-    try:
-        parsed = urlparse(endpoint)
-        if not parsed.scheme or parsed.scheme not in ("http", "https"):
-            return "Endpoint must be http or https URL"
-        if not parsed.netloc:
-            return "Endpoint must have a host"
-        return None
-    except Exception:
-        return "Invalid endpoint URL"
-
-
-def _validate_priority(priority: Any) -> Optional[str]:
-    """Validate priority value. Returns error message or None."""
-    try:
-        p = int(priority)
-        if p < 0 or p > 1000:
-            return "Priority must be between 0 and 1000"
-        return None
-    except (TypeError, ValueError):
-        return "Priority must be an integer"
-
-
-def _validate_region(region: str) -> Optional[str]:
-    """Validate region format. Returns error message or None."""
-    if not re.match(r"^[a-z]{2,}-[a-z]+-\d+$", region):
-        return "Region must match format like us-east-1"
-    return None
-
-
-def _validate_site_id(site_id: str) -> Optional[str]:
-    """Validate site_id format. Returns error message or None."""
-    if not site_id or len(site_id) > 63:
-        return "site_id must be 1-63 characters"
-    if not re.match(r'^[a-zA-Z0-9][a-zA-Z0-9_-]*$', site_id):
-        return "site_id must start with alphanumeric and contain only alphanumeric, hyphens, underscores"
-    return None
-
-
-logger = logging.getLogger(__name__)
-
-admin_api_bp = Blueprint("admin_api", __name__, url_prefix="/admin")
-
-
-def _require_principal() -> Tuple[Optional[Principal], Optional[Tuple[Dict[str, Any], int]]]:
-    from .s3_api import _require_principal as s3_require_principal
-    return s3_require_principal()
-
-
-def _require_admin() -> Tuple[Optional[Principal], Optional[Tuple[Dict[str, Any], int]]]:
-    principal, error = _require_principal()
-    if error:
-        return None, error
-
-    try:
-        _iam().authorize(principal, None, "iam:*")
-        return principal, None
-    except IamError:
-        return None, _json_error("AccessDenied", "Admin access required", 403)
-
-
-def _site_registry() -> SiteRegistry:
-    return current_app.extensions["site_registry"]
-
-
-def _connections() -> ConnectionStore:
-    return current_app.extensions["connections"]
-
-
-def _replication() -> ReplicationManager:
-    return current_app.extensions["replication"]
-
-
-def _iam():
-    return current_app.extensions["iam"]
-
-
-def _json_error(code: str, message: str, status: int) -> Tuple[Dict[str, Any], int]:
-    return {"error": {"code": code, "message": message}}, status
-
-
-def _get_admin_rate_limit() -> str:
-    return current_app.config.get("RATE_LIMIT_ADMIN", "60 per minute")
-
-
-@admin_api_bp.route("/site", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def get_local_site():
-    principal, error = _require_admin()
-    if error:
-        return error
-
-    registry = _site_registry()
-    local_site = registry.get_local_site()
-
-    if local_site:
-        return jsonify(local_site.to_dict())
-
-    config_site_id = current_app.config.get("SITE_ID")
-    config_endpoint = current_app.config.get("SITE_ENDPOINT")
-
-    if config_site_id:
-        return jsonify({
-            "site_id": config_site_id,
-            "endpoint": config_endpoint or "",
-            "region": current_app.config.get("SITE_REGION", "us-east-1"),
-            "priority": current_app.config.get("SITE_PRIORITY", 100),
-            "display_name": config_site_id,
-            "source": "environment",
-        })
-
-    return _json_error("NotFound", "Local site not configured", 404)
-
-
-@admin_api_bp.route("/site", methods=["PUT"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def update_local_site():
-    principal, error = _require_admin()
-    if error:
-        return error
-
-    payload = request.get_json(silent=True) or {}
-
-    site_id = payload.get("site_id")
-    endpoint = payload.get("endpoint")
-
-    if not site_id:
-        return _json_error("ValidationError", "site_id is required", 400)
-
-    site_id_error = _validate_site_id(site_id)
-    if site_id_error:
-        return _json_error("ValidationError", site_id_error, 400)
-
-    if endpoint:
-        endpoint_error = _validate_endpoint(endpoint)
-        if endpoint_error:
-            return _json_error("ValidationError", endpoint_error, 400)
-
-    if "priority" in payload:
-        priority_error = _validate_priority(payload["priority"])
-        if priority_error:
-            return _json_error("ValidationError", priority_error, 400)
-
-    if "region" in payload:
-        region_error = _validate_region(payload["region"])
-        if region_error:
-            return _json_error("ValidationError", region_error, 400)
-
-    registry = _site_registry()
-    existing = registry.get_local_site()
-
-    site = SiteInfo(
-        site_id=site_id,
-        endpoint=endpoint or "",
-        region=payload.get("region", "us-east-1"),
-        priority=payload.get("priority", 100),
-        display_name=payload.get("display_name", site_id),
-        created_at=existing.created_at if existing else None,
-    )
-
-    registry.set_local_site(site)
-
-    logger.info("Local site updated", extra={"site_id": site_id, "principal": principal.access_key})
-    return jsonify(site.to_dict())
-
-
-@admin_api_bp.route("/sites", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def list_all_sites():
-    principal, error = _require_admin()
-    if error:
-        return error
-
-    registry = _site_registry()
-    local = registry.get_local_site()
-    peers = registry.list_peers()
-
-    result = {
-        "local": local.to_dict() if local else None,
-        "peers": [peer.to_dict() for peer in peers],
-        "total_peers": len(peers),
-    }
-
-    return jsonify(result)
-
-
-@admin_api_bp.route("/sites", methods=["POST"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def register_peer_site():
-    principal, error = _require_admin()
-    if error:
-        return error
-
-    payload = request.get_json(silent=True) or {}
-
-    site_id = payload.get("site_id")
-    endpoint = payload.get("endpoint")
-
-    if not site_id:
-        return _json_error("ValidationError", "site_id is required", 400)
-
-    site_id_error = _validate_site_id(site_id)
-    if site_id_error:
-        return _json_error("ValidationError", site_id_error, 400)
-
-    if not endpoint:
-        return _json_error("ValidationError", "endpoint is required", 400)
-
-    endpoint_error = _validate_endpoint(endpoint)
-    if endpoint_error:
-        return _json_error("ValidationError", endpoint_error, 400)
-
-    region = payload.get("region", "us-east-1")
-    region_error = _validate_region(region)
-    if region_error:
-        return _json_error("ValidationError", region_error, 400)
-
-    priority = payload.get("priority", 100)
-    priority_error = _validate_priority(priority)
-    if priority_error:
-        return _json_error("ValidationError", priority_error, 400)
-
-    registry = _site_registry()
-
-    if registry.get_peer(site_id):
-        return _json_error("AlreadyExists", f"Peer site '{site_id}' already exists", 409)
-
-    connection_id = payload.get("connection_id")
-    if connection_id:
-        if not _connections().get(connection_id):
-            return _json_error("ValidationError", f"Connection '{connection_id}' not found", 400)
-
-    peer = PeerSite(
-        site_id=site_id,
-        endpoint=endpoint,
-        region=region,
-        priority=int(priority),
-        display_name=payload.get("display_name", site_id),
-        connection_id=connection_id,
-    )
-
-    registry.add_peer(peer)
-
-    logger.info("Peer site registered", extra={"site_id": site_id, "principal": principal.access_key})
-    return jsonify(peer.to_dict()), 201
-
-
-@admin_api_bp.route("/sites/<site_id>", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def get_peer_site(site_id: str):
-    principal, error = _require_admin()
-    if error:
-        return error
-
-    registry = _site_registry()
-    peer = registry.get_peer(site_id)
-
-    if not peer:
-        return _json_error("NotFound", f"Peer site '{site_id}' not found", 404)
-
-    return jsonify(peer.to_dict())
-
-
-@admin_api_bp.route("/sites/<site_id>", methods=["PUT"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def update_peer_site(site_id: str):
-    principal, error = _require_admin()
-    if error:
-        return error
-
-    registry = _site_registry()
-    existing = registry.get_peer(site_id)
-
-    if not existing:
-        return _json_error("NotFound", f"Peer site '{site_id}' not found", 404)
-
-    payload = request.get_json(silent=True) or {}
-
-    if "endpoint" in payload:
-        endpoint_error = _validate_endpoint(payload["endpoint"])
-        if endpoint_error:
-            return _json_error("ValidationError", endpoint_error, 400)
-
-    if "priority" in payload:
-        priority_error = _validate_priority(payload["priority"])
-        if priority_error:
-            return _json_error("ValidationError", priority_error, 400)
-
-    if "region" in payload:
-        region_error = _validate_region(payload["region"])
-        if region_error:
-            return _json_error("ValidationError", region_error, 400)
-
-    if "connection_id" in payload:
-        if payload["connection_id"] and not _connections().get(payload["connection_id"]):
-            return _json_error("ValidationError", f"Connection '{payload['connection_id']}' not found", 400)
-
-    peer = PeerSite(
-        site_id=site_id,
-        endpoint=payload.get("endpoint", existing.endpoint),
-        region=payload.get("region", existing.region),
-        priority=payload.get("priority", existing.priority),
-        display_name=payload.get("display_name", existing.display_name),
-        connection_id=payload.get("connection_id", existing.connection_id),
-        created_at=existing.created_at,
-        is_healthy=existing.is_healthy,
-        last_health_check=existing.last_health_check,
-    )
-
-    registry.update_peer(peer)
-
-    logger.info("Peer site updated", extra={"site_id": site_id, "principal": principal.access_key})
-    return jsonify(peer.to_dict())
-
-
-@admin_api_bp.route("/sites/<site_id>", methods=["DELETE"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def delete_peer_site(site_id: str):
-    principal, error = _require_admin()
-    if error:
-        return error
-
-    registry = _site_registry()
-
-    if not registry.delete_peer(site_id):
-        return _json_error("NotFound", f"Peer site '{site_id}' not found", 404)
-
-    logger.info("Peer site deleted", extra={"site_id": site_id, "principal": principal.access_key})
-    return Response(status=204)
-
-
-@admin_api_bp.route("/sites/<site_id>/health", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def check_peer_health(site_id: str):
-    principal, error = _require_admin()
-    if error:
-        return error
-
-    registry = _site_registry()
-    peer = registry.get_peer(site_id)
-
-    if not peer:
-        return _json_error("NotFound", f"Peer site '{site_id}' not found", 404)
-
-    is_healthy = False
-    error_message = None
-
-    if peer.connection_id:
-        connection = _connections().get(peer.connection_id)
-        if connection:
-            is_healthy = _replication().check_endpoint_health(connection)
-        else:
-            error_message = f"Connection '{peer.connection_id}' not found"
-    else:
-        error_message = "No connection configured for this peer"
-
-    registry.update_health(site_id, is_healthy)
-
-    result = {
-        "site_id": site_id,
-        "is_healthy": is_healthy,
-        "checked_at": time.time(),
-    }
-    if error_message:
-        result["error"] = error_message
-
-    return jsonify(result)
-
-
-@admin_api_bp.route("/topology", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def get_topology():
-    principal, error = _require_admin()
-    if error:
-        return error
-
-    registry = _site_registry()
-    local = registry.get_local_site()
-    peers = registry.list_peers()
-
-    sites = []
-
-    if local:
-        sites.append({
-            **local.to_dict(),
-            "is_local": True,
-            "is_healthy": True,
-        })
-
-    for peer in peers:
-        sites.append({
-            **peer.to_dict(),
-            "is_local": False,
-        })
-
-    sites.sort(key=lambda s: s.get("priority", 100))
-
-    return jsonify({
-        "sites": sites,
-        "total": len(sites),
-        "healthy_count": sum(1 for s in sites if s.get("is_healthy")),
-    })
-
-
-@admin_api_bp.route("/sites/<site_id>/bidirectional-status", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def check_bidirectional_status(site_id: str):
-    principal, error = _require_admin()
-    if error:
-        return error
-
-    registry = _site_registry()
-    peer = registry.get_peer(site_id)
-
-    if not peer:
-        return _json_error("NotFound", f"Peer site '{site_id}' not found", 404)
-
-    local_site = registry.get_local_site()
-    replication = _replication()
-    local_rules = replication.list_rules()
-
-    local_bidir_rules = []
-    for rule in local_rules:
-        if rule.target_connection_id == peer.connection_id and rule.mode == "bidirectional":
-            local_bidir_rules.append({
-                "bucket_name": rule.bucket_name,
-                "target_bucket": rule.target_bucket,
-                "enabled": rule.enabled,
-            })
-
-    result = {
-        "site_id": site_id,
-        "local_site_id": local_site.site_id if local_site else None,
-        "local_endpoint": local_site.endpoint if local_site else None,
-        "local_bidirectional_rules": local_bidir_rules,
-        "local_site_sync_enabled": current_app.config.get("SITE_SYNC_ENABLED", False),
-        "remote_status": None,
-        "issues": [],
-        "is_fully_configured": False,
-    }
-
-    if not local_site or not local_site.site_id:
-        result["issues"].append({
-            "code": "NO_LOCAL_SITE_ID",
-            "message": "Local site identity not configured",
-            "severity": "error",
-        })
-
-    if not local_site or not local_site.endpoint:
-        result["issues"].append({
-            "code": "NO_LOCAL_ENDPOINT",
-            "message": "Local site endpoint not configured (remote site cannot reach back)",
-            "severity": "error",
-        })
-
-    if not peer.connection_id:
-        result["issues"].append({
-            "code": "NO_CONNECTION",
-            "message": "No connection configured for this peer",
-            "severity": "error",
-        })
-        return jsonify(result)
-
-    connection = _connections().get(peer.connection_id)
-    if not connection:
-        result["issues"].append({
-            "code": "CONNECTION_NOT_FOUND",
-            "message": f"Connection '{peer.connection_id}' not found",
-            "severity": "error",
-        })
-        return jsonify(result)
-
-    if not local_bidir_rules:
-        result["issues"].append({
-            "code": "NO_LOCAL_BIDIRECTIONAL_RULES",
-            "message": "No bidirectional replication rules configured on this site",
-            "severity": "warning",
-        })
-
-    if not result["local_site_sync_enabled"]:
-        result["issues"].append({
-            "code": "SITE_SYNC_DISABLED",
-            "message": "Site sync worker is disabled (SITE_SYNC_ENABLED=false). Pull operations will not work.",
-            "severity": "warning",
-        })
-
-    if not replication.check_endpoint_health(connection):
-        result["issues"].append({
-            "code": "REMOTE_UNREACHABLE",
-            "message": "Remote endpoint is not reachable",
-            "severity": "error",
-        })
-        return jsonify(result)
-
-    allow_internal = current_app.config.get("ALLOW_INTERNAL_ENDPOINTS", False)
-    if not _is_safe_url(peer.endpoint, allow_internal=allow_internal):
-        result["issues"].append({
-            "code": "ENDPOINT_NOT_ALLOWED",
-            "message": "Peer endpoint points to cloud metadata service (SSRF protection)",
-            "severity": "error",
-        })
-        return jsonify(result)
-
-    try:
-        admin_url = peer.endpoint.rstrip("/") + "/admin/sites"
-        resp = requests.get(
-            admin_url,
-            timeout=10,
-            headers={
-                "Accept": "application/json",
-                "X-Access-Key": connection.access_key,
-                "X-Secret-Key": connection.secret_key,
-            },
-        )
-
-        if resp.status_code == 200:
-            try:
-                remote_data = resp.json()
-                if not isinstance(remote_data, dict):
-                    raise ValueError("Expected JSON object")
-                remote_local = remote_data.get("local")
-                if remote_local is not None and not isinstance(remote_local, dict):
-                    raise ValueError("Expected 'local' to be an object")
-                remote_peers = remote_data.get("peers", [])
-                if not isinstance(remote_peers, list):
-                    raise ValueError("Expected 'peers' to be a list")
-            except (ValueError, json.JSONDecodeError) as e:
-                logger.warning("Invalid JSON from remote admin API: %s", e)
-                result["remote_status"] = {"reachable": True, "invalid_response": True}
-                result["issues"].append({
-                    "code": "REMOTE_INVALID_RESPONSE",
-                    "message": "Remote admin API returned invalid JSON",
-                    "severity": "warning",
-                })
-                return jsonify(result)
-
-            result["remote_status"] = {
-                "reachable": True,
-                "local_site": remote_local,
-                "site_sync_enabled": None,
-                "has_peer_for_us": False,
-                "peer_connection_configured": False,
-                "has_bidirectional_rules_for_us": False,
-            }
-
-            for rp in remote_peers:
-                if not isinstance(rp, dict):
-                    continue
-                if local_site and (
-                    rp.get("site_id") == local_site.site_id or
-                    rp.get("endpoint") == local_site.endpoint
-                ):
-                    result["remote_status"]["has_peer_for_us"] = True
-                    result["remote_status"]["peer_connection_configured"] = bool(rp.get("connection_id"))
-                    break
-
-            if not result["remote_status"]["has_peer_for_us"]:
-                result["issues"].append({
-                    "code": "REMOTE_NO_PEER_FOR_US",
-                    "message": "Remote site does not have this site registered as a peer",
-                    "severity": "error",
-                })
-            elif not result["remote_status"]["peer_connection_configured"]:
-                result["issues"].append({
-                    "code": "REMOTE_NO_CONNECTION_FOR_US",
-                    "message": "Remote site has us as peer but no connection configured (cannot push back)",
-                    "severity": "error",
-                })
-        elif resp.status_code == 401 or resp.status_code == 403:
-            result["remote_status"] = {
-                "reachable": True,
-                "admin_access_denied": True,
-            }
-            result["issues"].append({
-                "code": "REMOTE_ADMIN_ACCESS_DENIED",
-                "message": "Cannot verify remote configuration (admin access denied)",
-                "severity": "warning",
-            })
-        else:
-            result["remote_status"] = {
-                "reachable": True,
-                "admin_api_error": resp.status_code,
-            }
-            result["issues"].append({
-                "code": "REMOTE_ADMIN_API_ERROR",
-                "message": f"Remote admin API returned status {resp.status_code}",
-                "severity": "warning",
-            })
-    except requests.RequestException as e:
-        logger.warning("Remote admin API unreachable: %s", e)
-        result["remote_status"] = {
-            "reachable": False,
-            "error": "Connection failed",
-        }
-        result["issues"].append({
-            "code": "REMOTE_ADMIN_UNREACHABLE",
-            "message": "Could not reach remote admin API",
-            "severity": "warning",
-        })
-    except Exception as e:
-        logger.warning("Error checking remote bidirectional status: %s", e, exc_info=True)
-        result["issues"].append({
-            "code": "VERIFICATION_ERROR",
-            "message": "Internal error during verification",
-            "severity": "warning",
-        })
-
-    error_issues = [i for i in result["issues"] if i["severity"] == "error"]
-    result["is_fully_configured"] = len(error_issues) == 0 and len(local_bidir_rules) > 0
-
-    return jsonify(result)
-
-
-def _website_domains() -> WebsiteDomainStore:
-    return current_app.extensions["website_domains"]
-
-
-def _storage():
-    return current_app.extensions["object_storage"]
-
-
-def _require_iam_action(action: str):
-    principal, error = _require_principal()
-    if error:
-        return None, error
-    try:
-        _iam().authorize(principal, None, action)
-        return principal, None
-    except IamError:
-        return None, _json_error("AccessDenied", f"Requires {action} permission", 403)
-
-
-@admin_api_bp.route("/iam/users", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def iam_list_users():
-    principal, error = _require_iam_action("iam:list_users")
-    if error:
-        return error
-    return jsonify({"users": _iam().list_users()})
-
-
-@admin_api_bp.route("/iam/users/<identifier>", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def iam_get_user(identifier):
-    principal, error = _require_iam_action("iam:get_user")
-    if error:
-        return error
-    try:
-        user_id = _iam().resolve_user_id(identifier)
-        return jsonify(_iam().get_user_by_id(user_id))
-    except IamError as exc:
-        return _json_error("NotFound", str(exc), 404)
-
-
-@admin_api_bp.route("/iam/users/<identifier>/policies", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def iam_get_user_policies(identifier):
-    principal, error = _require_iam_action("iam:get_policy")
-    if error:
-        return error
-    try:
-        return jsonify({"policies": _iam().get_user_policies(identifier)})
-    except IamError as exc:
-        return _json_error("NotFound", str(exc), 404)
-
-
-@admin_api_bp.route("/iam/users/<identifier>/keys", methods=["POST"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def iam_create_access_key(identifier):
-    principal, error = _require_iam_action("iam:create_key")
-    if error:
-        return error
-    try:
-        result = _iam().create_access_key(identifier)
-        logger.info("Access key created for %s by %s", identifier, principal.access_key)
-        return jsonify(result), 201
-    except IamError as exc:
-        return _json_error("InvalidRequest", str(exc), 400)
-
-
-@admin_api_bp.route("/iam/users/<identifier>/keys/<access_key>", methods=["DELETE"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def iam_delete_access_key(identifier, access_key):
-    principal, error = _require_iam_action("iam:delete_key")
-    if error:
-        return error
-    try:
-        _iam().delete_access_key(access_key)
-        logger.info("Access key %s deleted by %s", access_key, principal.access_key)
-        return "", 204
-    except IamError as exc:
-        return _json_error("InvalidRequest", str(exc), 400)
-
-
-@admin_api_bp.route("/iam/users/<identifier>/disable", methods=["POST"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def iam_disable_user(identifier):
-    principal, error = _require_iam_action("iam:disable_user")
-    if error:
-        return error
-    try:
-        _iam().disable_user(identifier)
-        logger.info("User %s disabled by %s", identifier, principal.access_key)
-        return jsonify({"status": "disabled"})
-    except IamError as exc:
-        return _json_error("InvalidRequest", str(exc), 400)
-
-
-@admin_api_bp.route("/iam/users/<identifier>/enable", methods=["POST"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def iam_enable_user(identifier):
-    principal, error = _require_iam_action("iam:disable_user")
-    if error:
-        return error
-    try:
-        _iam().enable_user(identifier)
-        logger.info("User %s enabled by %s", identifier, principal.access_key)
-        return jsonify({"status": "enabled"})
-    except IamError as exc:
-        return _json_error("InvalidRequest", str(exc), 400)
-
-
-@admin_api_bp.route("/website-domains", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def list_website_domains():
-    principal, error = _require_admin()
-    if error:
-        return error
-    if not current_app.config.get("WEBSITE_HOSTING_ENABLED", False):
-        return _json_error("InvalidRequest", "Website hosting is not enabled", 400)
-    return jsonify(_website_domains().list_all())
-
-
-@admin_api_bp.route("/website-domains", methods=["POST"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def create_website_domain():
-    principal, error = _require_admin()
-    if error:
-        return error
-    if not current_app.config.get("WEBSITE_HOSTING_ENABLED", False):
-        return _json_error("InvalidRequest", "Website hosting is not enabled", 400)
-    payload = request.get_json(silent=True) or {}
-    domain = normalize_domain(payload.get("domain") or "")
-    bucket = (payload.get("bucket") or "").strip()
-    if not domain:
-        return _json_error("ValidationError", "domain is required", 400)
-    if not is_valid_domain(domain):
-        return _json_error("ValidationError", f"Invalid domain: '{domain}'", 400)
-    if not bucket:
-        return _json_error("ValidationError", "bucket is required", 400)
-    storage = _storage()
-    if not storage.bucket_exists(bucket):
-        return _json_error("NoSuchBucket", f"Bucket '{bucket}' does not exist", 404)
-    store = _website_domains()
-    existing = store.get_bucket(domain)
-    if existing:
-        return _json_error("Conflict", f"Domain '{domain}' is already mapped to bucket '{existing}'", 409)
-    store.set_mapping(domain, bucket)
-    logger.info("Website domain mapping created: %s -> %s", domain, bucket)
-    return jsonify({"domain": domain, "bucket": bucket}), 201
-
-
-@admin_api_bp.route("/website-domains/<domain>", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def get_website_domain(domain: str):
-    principal, error = _require_admin()
-    if error:
-        return error
-    if not current_app.config.get("WEBSITE_HOSTING_ENABLED", False):
-        return _json_error("InvalidRequest", "Website hosting is not enabled", 400)
-    domain = normalize_domain(domain)
-    bucket = _website_domains().get_bucket(domain)
-    if not bucket:
-        return _json_error("NotFound", f"No mapping found for domain '{domain}'", 404)
-    return jsonify({"domain": domain, "bucket": bucket})
-
-
-@admin_api_bp.route("/website-domains/<domain>", methods=["PUT"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def update_website_domain(domain: str):
-    principal, error = _require_admin()
-    if error:
-        return error
-    if not current_app.config.get("WEBSITE_HOSTING_ENABLED", False):
-        return _json_error("InvalidRequest", "Website hosting is not enabled", 400)
-    domain = normalize_domain(domain)
-    payload = request.get_json(silent=True) or {}
-    bucket = (payload.get("bucket") or "").strip()
-    if not bucket:
-        return _json_error("ValidationError", "bucket is required", 400)
-    storage = _storage()
-    if not storage.bucket_exists(bucket):
-        return _json_error("NoSuchBucket", f"Bucket '{bucket}' does not exist", 404)
-    store = _website_domains()
-    if not store.get_bucket(domain):
-        return _json_error("NotFound", f"No mapping found for domain '{domain}'", 404)
-    store.set_mapping(domain, bucket)
-    logger.info("Website domain mapping updated: %s -> %s", domain, bucket)
-    return jsonify({"domain": domain, "bucket": bucket})
-
-
-@admin_api_bp.route("/website-domains/<domain>", methods=["DELETE"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def delete_website_domain(domain: str):
-    principal, error = _require_admin()
-    if error:
-        return error
-    if not current_app.config.get("WEBSITE_HOSTING_ENABLED", False):
-        return _json_error("InvalidRequest", "Website hosting is not enabled", 400)
-    domain = normalize_domain(domain)
-    if not _website_domains().delete_mapping(domain):
-        return _json_error("NotFound", f"No mapping found for domain '{domain}'", 404)
-    logger.info("Website domain mapping deleted: %s", domain)
-    return Response(status=204)
-
-
-def _gc() -> Optional[GarbageCollector]:
-    return current_app.extensions.get("gc")
-
-
-@admin_api_bp.route("/gc/status", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def gc_status():
-    principal, error = _require_admin()
-    if error:
-        return error
-    gc = _gc()
-    if not gc:
-        return jsonify({"enabled": False, "message": "GC is not enabled. Set GC_ENABLED=true to enable."})
-    return jsonify(gc.get_status())
-
-
-@admin_api_bp.route("/gc/run", methods=["POST"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def gc_run_now():
-    principal, error = _require_admin()
-    if error:
-        return error
-    gc = _gc()
-    if not gc:
-        return _json_error("InvalidRequest", "GC is not enabled", 400)
-    payload = request.get_json(silent=True) or {}
-    started = gc.run_async(dry_run=payload.get("dry_run"))
-    logger.info("GC manual run by %s", principal.access_key)
-    if not started:
-        return _json_error("Conflict", "GC is already in progress", 409)
-    return jsonify({"status": "started"})
-
-
-@admin_api_bp.route("/gc/history", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def gc_history():
-    principal, error = _require_admin()
-    if error:
-        return error
-    gc = _gc()
-    if not gc:
-        return jsonify({"executions": []})
-    limit = min(int(request.args.get("limit", 50)), 200)
-    offset = int(request.args.get("offset", 0))
-    records = gc.get_history(limit=limit, offset=offset)
-    return jsonify({"executions": records})
-
-
-def _integrity() -> Optional[IntegrityChecker]:
-    return current_app.extensions.get("integrity")
-
-
-@admin_api_bp.route("/integrity/status", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def integrity_status():
-    principal, error = _require_admin()
-    if error:
-        return error
-    checker = _integrity()
-    if not checker:
-        return jsonify({"enabled": False, "message": "Integrity checker is not enabled. Set INTEGRITY_ENABLED=true to enable."})
-    return jsonify(checker.get_status())
-
-
-@admin_api_bp.route("/integrity/run", methods=["POST"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def integrity_run_now():
-    principal, error = _require_admin()
-    if error:
-        return error
-    checker = _integrity()
-    if not checker:
-        return _json_error("InvalidRequest", "Integrity checker is not enabled", 400)
-    payload = request.get_json(silent=True) or {}
-    override_dry_run = payload.get("dry_run")
-    override_auto_heal = payload.get("auto_heal")
-    started = checker.run_async(
-        auto_heal=override_auto_heal if override_auto_heal is not None else None,
-        dry_run=override_dry_run if override_dry_run is not None else None,
-    )
-    logger.info("Integrity manual run by %s", principal.access_key)
-    if not started:
-        return _json_error("Conflict", "A scan is already in progress", 409)
-    return jsonify({"status": "started"})
-
-
-@admin_api_bp.route("/integrity/history", methods=["GET"])
-@limiter.limit(lambda: _get_admin_rate_limit())
-def integrity_history():
-    principal, error = _require_admin()
-    if error:
-        return error
-    checker = _integrity()
-    if not checker:
-        return jsonify({"executions": []})
-    limit = min(int(request.args.get("limit", 50)), 200)
-    offset = int(request.args.get("offset", 0))
-    records = checker.get_history(limit=limit, offset=offset)
-    return jsonify({"executions": records})
-
-

python/app/config.py

@@ -1,683 +0,0 @@
-from __future__ import annotations
-
-import os
-import re
-import secrets
-import shutil
-import sys
-import warnings
-from dataclasses import dataclass
-from pathlib import Path
-from typing import Any, Dict, Optional
-
-import psutil
-
-
-def _calculate_auto_threads() -> int:
-    cpu_count = psutil.cpu_count(logical=True) or 4
-    return max(1, min(cpu_count * 2, 64))
-
-
-def _calculate_auto_connection_limit() -> int:
-    available_mb = psutil.virtual_memory().available / (1024 * 1024)
-    calculated = int(available_mb / 5)
-    return max(20, min(calculated, 1000))
-
-
-def _calculate_auto_backlog(connection_limit: int) -> int:
-    return max(128, min(connection_limit * 2, 4096))
-
-
-def _validate_rate_limit(value: str) -> str:
-    pattern = r"^\d+\s+per\s+(second|minute|hour|day)$"
-    if not re.match(pattern, value):
-        raise ValueError(f"Invalid rate limit format: {value}. Expected format: '200 per minute'")
-    return value
-
-if getattr(sys, "frozen", False):
-    # Running in a PyInstaller bundle
-    PROJECT_ROOT = Path(sys._MEIPASS)
-else:
-    # Running in a normal Python environment
-    PROJECT_ROOT = Path(__file__).resolve().parent.parent
-
-
-def _prepare_config_file(active_path: Path, legacy_path: Optional[Path] = None) -> Path:
-    """Ensure config directories exist and migrate legacy files when possible."""
-    active_path = Path(active_path)
-    active_path.parent.mkdir(parents=True, exist_ok=True)
-    if legacy_path:
-        legacy_path = Path(legacy_path)
-        if not active_path.exists() and legacy_path.exists():
-            legacy_path.parent.mkdir(parents=True, exist_ok=True)
-            try:
-                shutil.move(str(legacy_path), str(active_path))
-            except OSError:
-                shutil.copy2(legacy_path, active_path)
-                try:
-                    legacy_path.unlink(missing_ok=True)
-                except OSError:
-                    pass
-    return active_path
-
-
-@dataclass
-class AppConfig:
-    storage_root: Path
-    max_upload_size: int
-    ui_page_size: int
-    secret_key: str
-    iam_config_path: Path
-    bucket_policy_path: Path
-    api_base_url: Optional[str]
-    aws_region: str
-    aws_service: str
-    ui_enforce_bucket_policies: bool
-    log_level: str
-    log_to_file: bool
-    log_path: Path
-    log_max_bytes: int
-    log_backup_count: int
-    ratelimit_default: str
-    ratelimit_storage_uri: str
-    ratelimit_list_buckets: str
-    ratelimit_bucket_ops: str
-    ratelimit_object_ops: str
-    ratelimit_head_ops: str
-    cors_origins: list[str]
-    cors_methods: list[str]
-    cors_allow_headers: list[str]
-    cors_expose_headers: list[str]
-    session_lifetime_days: int
-    auth_max_attempts: int
-    auth_lockout_minutes: int
-    bulk_delete_max_keys: int
-    secret_ttl_seconds: int
-    stream_chunk_size: int
-    multipart_min_part_size: int
-    bucket_stats_cache_ttl: int
-    object_cache_ttl: int
-    encryption_enabled: bool
-    encryption_master_key_path: Path
-    kms_enabled: bool
-    kms_keys_path: Path
-    default_encryption_algorithm: str
-    display_timezone: str
-    lifecycle_enabled: bool
-    lifecycle_interval_seconds: int
-    metrics_history_enabled: bool
-    metrics_history_retention_hours: int
-    metrics_history_interval_minutes: int
-    operation_metrics_enabled: bool
-    operation_metrics_interval_minutes: int
-    operation_metrics_retention_hours: int
-    server_threads: int
-    server_connection_limit: int
-    server_backlog: int
-    server_channel_timeout: int
-    server_max_buffer_size: int
-    server_threads_auto: bool
-    server_connection_limit_auto: bool
-    server_backlog_auto: bool
-    site_sync_enabled: bool
-    site_sync_interval_seconds: int
-    site_sync_batch_size: int
-    sigv4_timestamp_tolerance_seconds: int
-    presigned_url_min_expiry_seconds: int
-    presigned_url_max_expiry_seconds: int
-    replication_connect_timeout_seconds: int
-    replication_read_timeout_seconds: int
-    replication_max_retries: int
-    replication_streaming_threshold_bytes: int
-    replication_max_failures_per_bucket: int
-    site_sync_connect_timeout_seconds: int
-    site_sync_read_timeout_seconds: int
-    site_sync_max_retries: int
-    site_sync_clock_skew_tolerance_seconds: float
-    object_key_max_length_bytes: int
-    object_cache_max_size: int
-    meta_read_cache_max: int
-    bucket_config_cache_ttl_seconds: float
-    object_tag_limit: int
-    encryption_chunk_size_bytes: int
-    kms_generate_data_key_min_bytes: int
-    kms_generate_data_key_max_bytes: int
-    lifecycle_max_history_per_bucket: int
-    site_id: Optional[str]
-    site_endpoint: Optional[str]
-    site_region: str
-    site_priority: int
-    ratelimit_admin: str
-    num_trusted_proxies: int
-    allowed_redirect_hosts: list[str]
-    allow_internal_endpoints: bool
-    website_hosting_enabled: bool
-    gc_enabled: bool
-    gc_interval_hours: float
-    gc_temp_file_max_age_hours: float
-    gc_multipart_max_age_days: int
-    gc_lock_file_max_age_hours: float
-    gc_dry_run: bool
-    gc_io_throttle_ms: int
-    integrity_enabled: bool
-    integrity_interval_hours: float
-    integrity_batch_size: int
-    integrity_auto_heal: bool
-    integrity_dry_run: bool
-    integrity_io_throttle_ms: int
-
-    @classmethod
-    def from_env(cls, overrides: Optional[Dict[str, Any]] = None) -> "AppConfig":
-        overrides = overrides or {}
-
-        def _get(name: str, default: Any) -> Any:
-            return overrides.get(name, os.getenv(name, default))
-
-        storage_root = Path(_get("STORAGE_ROOT", PROJECT_ROOT / "data")).resolve()
-        max_upload_size = int(_get("MAX_UPLOAD_SIZE", 1024 * 1024 * 1024)) 
-        ui_page_size = int(_get("UI_PAGE_SIZE", 100))
-        auth_max_attempts = int(_get("AUTH_MAX_ATTEMPTS", 5))
-        auth_lockout_minutes = int(_get("AUTH_LOCKOUT_MINUTES", 15))
-        bulk_delete_max_keys = int(_get("BULK_DELETE_MAX_KEYS", 500))
-        secret_ttl_seconds = int(_get("SECRET_TTL_SECONDS", 300))
-        stream_chunk_size = int(_get("STREAM_CHUNK_SIZE", 64 * 1024))
-        multipart_min_part_size = int(_get("MULTIPART_MIN_PART_SIZE", 5 * 1024 * 1024))
-        lifecycle_enabled = _get("LIFECYCLE_ENABLED", "false").lower() in ("true", "1", "yes")
-        lifecycle_interval_seconds = int(_get("LIFECYCLE_INTERVAL_SECONDS", 3600))
-        default_secret = "dev-secret-key"
-        secret_key = str(_get("SECRET_KEY", default_secret))
-        
-        if not secret_key or secret_key == default_secret:
-            secret_file = storage_root / ".myfsio.sys" / "config" / ".secret"
-            if secret_file.exists():
-                secret_key = secret_file.read_text().strip()
-            else:
-                generated = secrets.token_urlsafe(32)
-                if secret_key == default_secret:
-                    warnings.warn("Using insecure default SECRET_KEY. A random value has been generated and persisted; set SECRET_KEY for production", RuntimeWarning)
-                try:
-                    secret_file.parent.mkdir(parents=True, exist_ok=True)
-                    secret_file.write_text(generated)
-                    try:
-                        os.chmod(secret_file, 0o600)
-                    except OSError:
-                        pass
-                    secret_key = generated
-                except OSError:
-                    secret_key = generated
-
-        iam_env_override = "IAM_CONFIG" in overrides or "IAM_CONFIG" in os.environ
-        bucket_policy_override = "BUCKET_POLICY_PATH" in overrides or "BUCKET_POLICY_PATH" in os.environ
-
-        default_iam_path = storage_root / ".myfsio.sys" / "config" / "iam.json"
-        default_bucket_policy_path = storage_root / ".myfsio.sys" / "config" / "bucket_policies.json"
-
-        iam_config_path = Path(_get("IAM_CONFIG", default_iam_path)).resolve()
-        bucket_policy_path = Path(_get("BUCKET_POLICY_PATH", default_bucket_policy_path)).resolve()
-
-        iam_config_path = _prepare_config_file(
-            iam_config_path,
-            legacy_path=None if iam_env_override else storage_root / "iam.json",
-        )
-        bucket_policy_path = _prepare_config_file(
-            bucket_policy_path,
-            legacy_path=None if bucket_policy_override else storage_root / "bucket_policies.json",
-        )
-        api_base_url = _get("API_BASE_URL", None)
-        if api_base_url:
-            api_base_url = str(api_base_url)
-        
-        aws_region = str(_get("AWS_REGION", "us-east-1"))
-        aws_service = str(_get("AWS_SERVICE", "s3"))
-        enforce_ui_policies = str(_get("UI_ENFORCE_BUCKET_POLICIES", "0")).lower() in {"1", "true", "yes", "on"}
-        log_level = str(_get("LOG_LEVEL", "INFO")).upper()
-        log_to_file = str(_get("LOG_TO_FILE", "1")).lower() in {"1", "true", "yes", "on"}
-        log_dir = Path(_get("LOG_DIR", storage_root.parent / "logs")).resolve()
-        log_dir.mkdir(parents=True, exist_ok=True)
-        log_path = log_dir / str(_get("LOG_FILE", "app.log"))
-        log_max_bytes = int(_get("LOG_MAX_BYTES", 5 * 1024 * 1024))
-        log_backup_count = int(_get("LOG_BACKUP_COUNT", 3))
-        ratelimit_default = _validate_rate_limit(str(_get("RATE_LIMIT_DEFAULT", "200 per minute")))
-        ratelimit_storage_uri = str(_get("RATE_LIMIT_STORAGE_URI", "memory://"))
-        ratelimit_list_buckets = _validate_rate_limit(str(_get("RATE_LIMIT_LIST_BUCKETS", "60 per minute")))
-        ratelimit_bucket_ops = _validate_rate_limit(str(_get("RATE_LIMIT_BUCKET_OPS", "120 per minute")))
-        ratelimit_object_ops = _validate_rate_limit(str(_get("RATE_LIMIT_OBJECT_OPS", "240 per minute")))
-        ratelimit_head_ops = _validate_rate_limit(str(_get("RATE_LIMIT_HEAD_OPS", "100 per minute")))
-
-        def _csv(value: str, default: list[str]) -> list[str]:
-            if not value:
-                return default
-            parts = [segment.strip() for segment in value.split(",") if segment.strip()]
-            return parts or default
-
-        cors_origins = _csv(str(_get("CORS_ORIGINS", "*")), ["*"])
-        cors_methods = _csv(str(_get("CORS_METHODS", "GET,PUT,POST,DELETE,OPTIONS,HEAD")), ["GET", "PUT", "POST", "DELETE", "OPTIONS", "HEAD"])
-        cors_allow_headers = _csv(str(_get("CORS_ALLOW_HEADERS", "*")), ["*"])
-        cors_expose_headers = _csv(str(_get("CORS_EXPOSE_HEADERS", "*")), ["*"])
-        session_lifetime_days = int(_get("SESSION_LIFETIME_DAYS", 30))
-        bucket_stats_cache_ttl = int(_get("BUCKET_STATS_CACHE_TTL", 60))
-        object_cache_ttl = int(_get("OBJECT_CACHE_TTL", 60))
-
-        encryption_enabled = str(_get("ENCRYPTION_ENABLED", "0")).lower() in {"1", "true", "yes", "on"}
-        encryption_keys_dir = storage_root / ".myfsio.sys" / "keys"
-        encryption_master_key_path = Path(_get("ENCRYPTION_MASTER_KEY_PATH", encryption_keys_dir / "master.key")).resolve()
-        kms_enabled = str(_get("KMS_ENABLED", "0")).lower() in {"1", "true", "yes", "on"}
-        kms_keys_path = Path(_get("KMS_KEYS_PATH", encryption_keys_dir / "kms_keys.json")).resolve()
-        default_encryption_algorithm = str(_get("DEFAULT_ENCRYPTION_ALGORITHM", "AES256"))
-        display_timezone = str(_get("DISPLAY_TIMEZONE", "UTC"))
-        metrics_history_enabled = str(_get("METRICS_HISTORY_ENABLED", "0")).lower() in {"1", "true", "yes", "on"}
-        metrics_history_retention_hours = int(_get("METRICS_HISTORY_RETENTION_HOURS", 24))
-        metrics_history_interval_minutes = int(_get("METRICS_HISTORY_INTERVAL_MINUTES", 5))
-        operation_metrics_enabled = str(_get("OPERATION_METRICS_ENABLED", "0")).lower() in {"1", "true", "yes", "on"}
-        operation_metrics_interval_minutes = int(_get("OPERATION_METRICS_INTERVAL_MINUTES", 5))
-        operation_metrics_retention_hours = int(_get("OPERATION_METRICS_RETENTION_HOURS", 24))
-
-        _raw_threads = int(_get("SERVER_THREADS", 0))
-        if _raw_threads == 0:
-            server_threads = _calculate_auto_threads()
-            server_threads_auto = True
-        else:
-            server_threads = _raw_threads
-            server_threads_auto = False
-
-        _raw_conn_limit = int(_get("SERVER_CONNECTION_LIMIT", 0))
-        if _raw_conn_limit == 0:
-            server_connection_limit = _calculate_auto_connection_limit()
-            server_connection_limit_auto = True
-        else:
-            server_connection_limit = _raw_conn_limit
-            server_connection_limit_auto = False
-
-        _raw_backlog = int(_get("SERVER_BACKLOG", 0))
-        if _raw_backlog == 0:
-            server_backlog = _calculate_auto_backlog(server_connection_limit)
-            server_backlog_auto = True
-        else:
-            server_backlog = _raw_backlog
-            server_backlog_auto = False
-
-        server_channel_timeout = int(_get("SERVER_CHANNEL_TIMEOUT", 120))
-        server_max_buffer_size = int(_get("SERVER_MAX_BUFFER_SIZE", 1024 * 1024 * 128))
-        site_sync_enabled = str(_get("SITE_SYNC_ENABLED", "0")).lower() in {"1", "true", "yes", "on"}
-        site_sync_interval_seconds = int(_get("SITE_SYNC_INTERVAL_SECONDS", 60))
-        site_sync_batch_size = int(_get("SITE_SYNC_BATCH_SIZE", 100))
-
-        sigv4_timestamp_tolerance_seconds = int(_get("SIGV4_TIMESTAMP_TOLERANCE_SECONDS", 900))
-        presigned_url_min_expiry_seconds = int(_get("PRESIGNED_URL_MIN_EXPIRY_SECONDS", 1))
-        presigned_url_max_expiry_seconds = int(_get("PRESIGNED_URL_MAX_EXPIRY_SECONDS", 604800))
-        replication_connect_timeout_seconds = int(_get("REPLICATION_CONNECT_TIMEOUT_SECONDS", 5))
-        replication_read_timeout_seconds = int(_get("REPLICATION_READ_TIMEOUT_SECONDS", 30))
-        replication_max_retries = int(_get("REPLICATION_MAX_RETRIES", 2))
-        replication_streaming_threshold_bytes = int(_get("REPLICATION_STREAMING_THRESHOLD_BYTES", 10 * 1024 * 1024))
-        replication_max_failures_per_bucket = int(_get("REPLICATION_MAX_FAILURES_PER_BUCKET", 50))
-        site_sync_connect_timeout_seconds = int(_get("SITE_SYNC_CONNECT_TIMEOUT_SECONDS", 10))
-        site_sync_read_timeout_seconds = int(_get("SITE_SYNC_READ_TIMEOUT_SECONDS", 120))
-        site_sync_max_retries = int(_get("SITE_SYNC_MAX_RETRIES", 2))
-        site_sync_clock_skew_tolerance_seconds = float(_get("SITE_SYNC_CLOCK_SKEW_TOLERANCE_SECONDS", 1.0))
-        object_key_max_length_bytes = int(_get("OBJECT_KEY_MAX_LENGTH_BYTES", 1024))
-        object_cache_max_size = int(_get("OBJECT_CACHE_MAX_SIZE", 100))
-        meta_read_cache_max = int(_get("META_READ_CACHE_MAX", 2048))
-        bucket_config_cache_ttl_seconds = float(_get("BUCKET_CONFIG_CACHE_TTL_SECONDS", 30.0))
-        object_tag_limit = int(_get("OBJECT_TAG_LIMIT", 50))
-        encryption_chunk_size_bytes = int(_get("ENCRYPTION_CHUNK_SIZE_BYTES", 64 * 1024))
-        kms_generate_data_key_min_bytes = int(_get("KMS_GENERATE_DATA_KEY_MIN_BYTES", 1))
-        kms_generate_data_key_max_bytes = int(_get("KMS_GENERATE_DATA_KEY_MAX_BYTES", 1024))
-        lifecycle_max_history_per_bucket = int(_get("LIFECYCLE_MAX_HISTORY_PER_BUCKET", 50))
-
-        site_id_raw = _get("SITE_ID", None)
-        site_id = str(site_id_raw).strip() if site_id_raw else None
-        site_endpoint_raw = _get("SITE_ENDPOINT", None)
-        site_endpoint = str(site_endpoint_raw).strip() if site_endpoint_raw else None
-        site_region = str(_get("SITE_REGION", "us-east-1"))
-        site_priority = int(_get("SITE_PRIORITY", 100))
-        ratelimit_admin = _validate_rate_limit(str(_get("RATE_LIMIT_ADMIN", "60 per minute")))
-        num_trusted_proxies = int(_get("NUM_TRUSTED_PROXIES", 1))
-        allowed_redirect_hosts_raw = _get("ALLOWED_REDIRECT_HOSTS", "")
-        allowed_redirect_hosts = [h.strip() for h in str(allowed_redirect_hosts_raw).split(",") if h.strip()]
-        allow_internal_endpoints = str(_get("ALLOW_INTERNAL_ENDPOINTS", "0")).lower() in {"1", "true", "yes", "on"}
-        website_hosting_enabled = str(_get("WEBSITE_HOSTING_ENABLED", "0")).lower() in {"1", "true", "yes", "on"}
-        gc_enabled = str(_get("GC_ENABLED", "0")).lower() in {"1", "true", "yes", "on"}
-        gc_interval_hours = float(_get("GC_INTERVAL_HOURS", 6.0))
-        gc_temp_file_max_age_hours = float(_get("GC_TEMP_FILE_MAX_AGE_HOURS", 24.0))
-        gc_multipart_max_age_days = int(_get("GC_MULTIPART_MAX_AGE_DAYS", 7))
-        gc_lock_file_max_age_hours = float(_get("GC_LOCK_FILE_MAX_AGE_HOURS", 1.0))
-        gc_dry_run = str(_get("GC_DRY_RUN", "0")).lower() in {"1", "true", "yes", "on"}
-        gc_io_throttle_ms = int(_get("GC_IO_THROTTLE_MS", 10))
-        integrity_enabled = str(_get("INTEGRITY_ENABLED", "0")).lower() in {"1", "true", "yes", "on"}
-        integrity_interval_hours = float(_get("INTEGRITY_INTERVAL_HOURS", 24.0))
-        integrity_batch_size = int(_get("INTEGRITY_BATCH_SIZE", 1000))
-        integrity_auto_heal = str(_get("INTEGRITY_AUTO_HEAL", "0")).lower() in {"1", "true", "yes", "on"}
-        integrity_dry_run = str(_get("INTEGRITY_DRY_RUN", "0")).lower() in {"1", "true", "yes", "on"}
-        integrity_io_throttle_ms = int(_get("INTEGRITY_IO_THROTTLE_MS", 10))
-
-        return cls(storage_root=storage_root,
-                   max_upload_size=max_upload_size,
-                   ui_page_size=ui_page_size,
-                   secret_key=secret_key,
-                   iam_config_path=iam_config_path,
-                   bucket_policy_path=bucket_policy_path,
-                   api_base_url=api_base_url,
-                   aws_region=aws_region,
-                   aws_service=aws_service,
-                   ui_enforce_bucket_policies=enforce_ui_policies,
-                   log_level=log_level,
-                   log_to_file=log_to_file,
-                   log_path=log_path,
-                   log_max_bytes=log_max_bytes,
-                   log_backup_count=log_backup_count,
-                   ratelimit_default=ratelimit_default,
-                   ratelimit_storage_uri=ratelimit_storage_uri,
-                   ratelimit_list_buckets=ratelimit_list_buckets,
-                   ratelimit_bucket_ops=ratelimit_bucket_ops,
-                   ratelimit_object_ops=ratelimit_object_ops,
-                   ratelimit_head_ops=ratelimit_head_ops,
-                   cors_origins=cors_origins,
-                   cors_methods=cors_methods,
-                   cors_allow_headers=cors_allow_headers,
-                   cors_expose_headers=cors_expose_headers,
-                   session_lifetime_days=session_lifetime_days,
-                   auth_max_attempts=auth_max_attempts,
-                   auth_lockout_minutes=auth_lockout_minutes,
-                   bulk_delete_max_keys=bulk_delete_max_keys,
-                   secret_ttl_seconds=secret_ttl_seconds,
-                   stream_chunk_size=stream_chunk_size,
-                   multipart_min_part_size=multipart_min_part_size,
-                   bucket_stats_cache_ttl=bucket_stats_cache_ttl,
-                   object_cache_ttl=object_cache_ttl,
-                   encryption_enabled=encryption_enabled,
-                   encryption_master_key_path=encryption_master_key_path,
-                   kms_enabled=kms_enabled,
-                   kms_keys_path=kms_keys_path,
-                   default_encryption_algorithm=default_encryption_algorithm,
-                   display_timezone=display_timezone,
-                   lifecycle_enabled=lifecycle_enabled,
-                   lifecycle_interval_seconds=lifecycle_interval_seconds,
-                   metrics_history_enabled=metrics_history_enabled,
-                   metrics_history_retention_hours=metrics_history_retention_hours,
-                   metrics_history_interval_minutes=metrics_history_interval_minutes,
-                   operation_metrics_enabled=operation_metrics_enabled,
-                   operation_metrics_interval_minutes=operation_metrics_interval_minutes,
-                   operation_metrics_retention_hours=operation_metrics_retention_hours,
-                   server_threads=server_threads,
-                   server_connection_limit=server_connection_limit,
-                   server_backlog=server_backlog,
-                   server_channel_timeout=server_channel_timeout,
-                   server_max_buffer_size=server_max_buffer_size,
-                   server_threads_auto=server_threads_auto,
-                   server_connection_limit_auto=server_connection_limit_auto,
-                   server_backlog_auto=server_backlog_auto,
-                   site_sync_enabled=site_sync_enabled,
-                   site_sync_interval_seconds=site_sync_interval_seconds,
-                   site_sync_batch_size=site_sync_batch_size,
-                   sigv4_timestamp_tolerance_seconds=sigv4_timestamp_tolerance_seconds,
-                   presigned_url_min_expiry_seconds=presigned_url_min_expiry_seconds,
-                   presigned_url_max_expiry_seconds=presigned_url_max_expiry_seconds,
-                   replication_connect_timeout_seconds=replication_connect_timeout_seconds,
-                   replication_read_timeout_seconds=replication_read_timeout_seconds,
-                   replication_max_retries=replication_max_retries,
-                   replication_streaming_threshold_bytes=replication_streaming_threshold_bytes,
-                   replication_max_failures_per_bucket=replication_max_failures_per_bucket,
-                   site_sync_connect_timeout_seconds=site_sync_connect_timeout_seconds,
-                   site_sync_read_timeout_seconds=site_sync_read_timeout_seconds,
-                   site_sync_max_retries=site_sync_max_retries,
-                   site_sync_clock_skew_tolerance_seconds=site_sync_clock_skew_tolerance_seconds,
-                   object_key_max_length_bytes=object_key_max_length_bytes,
-                   object_cache_max_size=object_cache_max_size,
-                   meta_read_cache_max=meta_read_cache_max,
-                   bucket_config_cache_ttl_seconds=bucket_config_cache_ttl_seconds,
-                   object_tag_limit=object_tag_limit,
-                   encryption_chunk_size_bytes=encryption_chunk_size_bytes,
-                   kms_generate_data_key_min_bytes=kms_generate_data_key_min_bytes,
-                   kms_generate_data_key_max_bytes=kms_generate_data_key_max_bytes,
-                   lifecycle_max_history_per_bucket=lifecycle_max_history_per_bucket,
-                   site_id=site_id,
-                   site_endpoint=site_endpoint,
-                   site_region=site_region,
-                   site_priority=site_priority,
-                   ratelimit_admin=ratelimit_admin,
-                   num_trusted_proxies=num_trusted_proxies,
-                   allowed_redirect_hosts=allowed_redirect_hosts,
-                   allow_internal_endpoints=allow_internal_endpoints,
-                   website_hosting_enabled=website_hosting_enabled,
-                   gc_enabled=gc_enabled,
-                   gc_interval_hours=gc_interval_hours,
-                   gc_temp_file_max_age_hours=gc_temp_file_max_age_hours,
-                   gc_multipart_max_age_days=gc_multipart_max_age_days,
-                   gc_lock_file_max_age_hours=gc_lock_file_max_age_hours,
-                   gc_dry_run=gc_dry_run,
-                   gc_io_throttle_ms=gc_io_throttle_ms,
-                   integrity_enabled=integrity_enabled,
-                   integrity_interval_hours=integrity_interval_hours,
-                   integrity_batch_size=integrity_batch_size,
-                   integrity_auto_heal=integrity_auto_heal,
-                   integrity_dry_run=integrity_dry_run,
-                   integrity_io_throttle_ms=integrity_io_throttle_ms)
-
-    def validate_and_report(self) -> list[str]:
-        """Validate configuration and return a list of warnings/issues.
-        
-        Call this at startup to detect potential misconfigurations before
-        the application fully commits to running.
-        """
-        issues = []
-        
-        try:
-            test_file = self.storage_root / ".write_test"
-            test_file.touch()
-            test_file.unlink()
-        except (OSError, PermissionError) as e:
-            issues.append(f"CRITICAL: STORAGE_ROOT '{self.storage_root}' is not writable: {e}")
-        
-        storage_str = str(self.storage_root).lower()
-        if "/tmp" in storage_str or "\\temp" in storage_str or "appdata\\local\\temp" in storage_str:
-            issues.append(f"WARNING: STORAGE_ROOT '{self.storage_root}' appears to be a temporary directory. Data may be lost on reboot!")
-        
-        try:
-            self.iam_config_path.relative_to(self.storage_root)
-        except ValueError:
-            issues.append(f"WARNING: IAM_CONFIG '{self.iam_config_path}' is outside STORAGE_ROOT '{self.storage_root}'. Consider setting IAM_CONFIG explicitly or ensuring paths are aligned.")
-        
-        try:
-            self.bucket_policy_path.relative_to(self.storage_root)
-        except ValueError:
-            issues.append(f"WARNING: BUCKET_POLICY_PATH '{self.bucket_policy_path}' is outside STORAGE_ROOT '{self.storage_root}'. Consider setting BUCKET_POLICY_PATH explicitly.")
-        
-        try:
-            self.log_path.parent.mkdir(parents=True, exist_ok=True)
-            test_log = self.log_path.parent / ".write_test"
-            test_log.touch()
-            test_log.unlink()
-        except (OSError, PermissionError) as e:
-            issues.append(f"WARNING: Log directory '{self.log_path.parent}' is not writable: {e}")
-        
-        log_str = str(self.log_path).lower()
-        if "/tmp" in log_str or "\\temp" in log_str or "appdata\\local\\temp" in log_str:
-            issues.append(f"WARNING: LOG_DIR '{self.log_path.parent}' appears to be a temporary directory. Logs may be lost on reboot!")
-        
-        if self.encryption_enabled:
-            try:
-                self.encryption_master_key_path.relative_to(self.storage_root)
-            except ValueError:
-                issues.append(f"WARNING: ENCRYPTION_MASTER_KEY_PATH '{self.encryption_master_key_path}' is outside STORAGE_ROOT. Ensure proper backup procedures.")
-        
-        if self.kms_enabled:
-            try:
-                self.kms_keys_path.relative_to(self.storage_root)
-            except ValueError:
-                issues.append(f"WARNING: KMS_KEYS_PATH '{self.kms_keys_path}' is outside STORAGE_ROOT. Ensure proper backup procedures.")
-        
-        if self.secret_key == "dev-secret-key":
-            issues.append("WARNING: Using default SECRET_KEY. Set SECRET_KEY environment variable for production.")
-        
-        if "*" in self.cors_origins:
-            issues.append("INFO: CORS_ORIGINS is set to '*'. Consider restricting to specific domains in production.")
-
-        if not (1 <= self.server_threads <= 64):
-            issues.append(f"CRITICAL: SERVER_THREADS={self.server_threads} is outside valid range (1-64). Server cannot start.")
-        if not (10 <= self.server_connection_limit <= 1000):
-            issues.append(f"CRITICAL: SERVER_CONNECTION_LIMIT={self.server_connection_limit} is outside valid range (10-1000). Server cannot start.")
-        if not (128 <= self.server_backlog <= 4096):
-            issues.append(f"CRITICAL: SERVER_BACKLOG={self.server_backlog} is outside valid range (128-4096). Server cannot start.")
-        if not (10 <= self.server_channel_timeout <= 300):
-            issues.append(f"CRITICAL: SERVER_CHANNEL_TIMEOUT={self.server_channel_timeout} is outside valid range (10-300). Server cannot start.")
-        if self.server_max_buffer_size < 1024 * 1024:
-            issues.append(f"WARNING: SERVER_MAX_BUFFER_SIZE={self.server_max_buffer_size} is less than 1MB. Large uploads will fail.")
-
-        if sys.platform != "win32":
-            try:
-                import resource
-                soft_limit, _ = resource.getrlimit(resource.RLIMIT_NOFILE)
-                threshold = int(soft_limit * 0.8)
-                if self.server_connection_limit > threshold:
-                    issues.append(f"WARNING: SERVER_CONNECTION_LIMIT={self.server_connection_limit} exceeds 80% of system file descriptor limit (soft={soft_limit}). Consider running 'ulimit -n {self.server_connection_limit + 100}'.")
-            except (ImportError, OSError):
-                pass
-
-        try:
-            import psutil
-            available_mb = psutil.virtual_memory().available / (1024 * 1024)
-            estimated_mb = self.server_threads * 50
-            if estimated_mb > available_mb * 0.5:
-                issues.append(f"WARNING: SERVER_THREADS={self.server_threads} may require ~{estimated_mb}MB memory, exceeding 50% of available RAM ({int(available_mb)}MB).")
-        except ImportError:
-            pass
-
-        return issues
-
-    def print_startup_summary(self) -> None:
-        """Print a summary of the configuration at startup."""
-        print("\n" + "=" * 60)
-        print("MyFSIO Configuration Summary")
-        print("=" * 60)
-        print(f"  STORAGE_ROOT:     {self.storage_root}")
-        print(f"  IAM_CONFIG:       {self.iam_config_path}")
-        print(f"  BUCKET_POLICY:    {self.bucket_policy_path}")
-        print(f"  LOG_PATH:         {self.log_path}")
-        if self.api_base_url:
-            print(f"  API_BASE_URL:     {self.api_base_url}")
-        if self.encryption_enabled:
-            print(f"  ENCRYPTION:       Enabled (Master key: {self.encryption_master_key_path})")
-        if self.kms_enabled:
-            print(f"  KMS:              Enabled (Keys: {self.kms_keys_path})")
-        if self.website_hosting_enabled:
-            print(f"  WEBSITE_HOSTING:  Enabled")
-        def _auto(flag: bool) -> str:
-            return " (auto)" if flag else ""
-        print(f"  SERVER_THREADS:   {self.server_threads}{_auto(self.server_threads_auto)}")
-        print(f"  CONNECTION_LIMIT: {self.server_connection_limit}{_auto(self.server_connection_limit_auto)}")
-        print(f"  BACKLOG:          {self.server_backlog}{_auto(self.server_backlog_auto)}")
-        print(f"  CHANNEL_TIMEOUT:  {self.server_channel_timeout}s")
-        print(f"  MAX_BUFFER_SIZE:  {self.server_max_buffer_size // (1024 * 1024)}MB")
-        print("=" * 60)
-        
-        issues = self.validate_and_report()
-        if issues:
-            print("\nConfiguration Issues Detected:")
-            for issue in issues:
-                print(f"  • {issue}")
-            print()
-        else:
-            print("  ✓ Configuration validated successfully\n")
-
-    def to_flask_config(self) -> Dict[str, Any]:
-        return {
-            "STORAGE_ROOT": str(self.storage_root),
-            "MAX_CONTENT_LENGTH": self.max_upload_size,
-            "UI_PAGE_SIZE": self.ui_page_size,
-            "SECRET_KEY": self.secret_key,
-            "IAM_CONFIG": str(self.iam_config_path),
-            "BUCKET_POLICY_PATH": str(self.bucket_policy_path),
-            "API_BASE_URL": self.api_base_url,
-            "AWS_REGION": self.aws_region,
-            "AWS_SERVICE": self.aws_service,
-            "UI_ENFORCE_BUCKET_POLICIES": self.ui_enforce_bucket_policies,
-            "AUTH_MAX_ATTEMPTS": self.auth_max_attempts,
-            "AUTH_LOCKOUT_MINUTES": self.auth_lockout_minutes,
-            "BULK_DELETE_MAX_KEYS": self.bulk_delete_max_keys,
-            "SECRET_TTL_SECONDS": self.secret_ttl_seconds,
-            "STREAM_CHUNK_SIZE": self.stream_chunk_size,
-            "MULTIPART_MIN_PART_SIZE": self.multipart_min_part_size,
-            "BUCKET_STATS_CACHE_TTL": self.bucket_stats_cache_ttl,
-            "OBJECT_CACHE_TTL": self.object_cache_ttl,
-            "LOG_LEVEL": self.log_level,
-            "LOG_TO_FILE": self.log_to_file,
-            "LOG_FILE": str(self.log_path),
-            "LOG_MAX_BYTES": self.log_max_bytes,
-            "LOG_BACKUP_COUNT": self.log_backup_count,
-            "RATELIMIT_DEFAULT": self.ratelimit_default,
-            "RATELIMIT_STORAGE_URI": self.ratelimit_storage_uri,
-            "RATELIMIT_LIST_BUCKETS": self.ratelimit_list_buckets,
-            "RATELIMIT_BUCKET_OPS": self.ratelimit_bucket_ops,
-            "RATELIMIT_OBJECT_OPS": self.ratelimit_object_ops,
-            "RATELIMIT_HEAD_OPS": self.ratelimit_head_ops,
-            "CORS_ORIGINS": self.cors_origins,
-            "CORS_METHODS": self.cors_methods,
-            "CORS_ALLOW_HEADERS": self.cors_allow_headers,
-            "CORS_EXPOSE_HEADERS": self.cors_expose_headers,
-            "SESSION_LIFETIME_DAYS": self.session_lifetime_days,
-            "ENCRYPTION_ENABLED": self.encryption_enabled,
-            "ENCRYPTION_MASTER_KEY_PATH": str(self.encryption_master_key_path),
-            "KMS_ENABLED": self.kms_enabled,
-            "KMS_KEYS_PATH": str(self.kms_keys_path),
-            "DEFAULT_ENCRYPTION_ALGORITHM": self.default_encryption_algorithm,
-            "DISPLAY_TIMEZONE": self.display_timezone,
-            "LIFECYCLE_ENABLED": self.lifecycle_enabled,
-            "LIFECYCLE_INTERVAL_SECONDS": self.lifecycle_interval_seconds,
-            "METRICS_HISTORY_ENABLED": self.metrics_history_enabled,
-            "METRICS_HISTORY_RETENTION_HOURS": self.metrics_history_retention_hours,
-            "METRICS_HISTORY_INTERVAL_MINUTES": self.metrics_history_interval_minutes,
-            "OPERATION_METRICS_ENABLED": self.operation_metrics_enabled,
-            "OPERATION_METRICS_INTERVAL_MINUTES": self.operation_metrics_interval_minutes,
-            "OPERATION_METRICS_RETENTION_HOURS": self.operation_metrics_retention_hours,
-            "SERVER_THREADS": self.server_threads,
-            "SERVER_CONNECTION_LIMIT": self.server_connection_limit,
-            "SERVER_BACKLOG": self.server_backlog,
-            "SERVER_CHANNEL_TIMEOUT": self.server_channel_timeout,
-            "SERVER_MAX_BUFFER_SIZE": self.server_max_buffer_size,
-            "SITE_SYNC_ENABLED": self.site_sync_enabled,
-            "SITE_SYNC_INTERVAL_SECONDS": self.site_sync_interval_seconds,
-            "SITE_SYNC_BATCH_SIZE": self.site_sync_batch_size,
-            "SIGV4_TIMESTAMP_TOLERANCE_SECONDS": self.sigv4_timestamp_tolerance_seconds,
-            "PRESIGNED_URL_MIN_EXPIRY_SECONDS": self.presigned_url_min_expiry_seconds,
-            "PRESIGNED_URL_MAX_EXPIRY_SECONDS": self.presigned_url_max_expiry_seconds,
-            "REPLICATION_CONNECT_TIMEOUT_SECONDS": self.replication_connect_timeout_seconds,
-            "REPLICATION_READ_TIMEOUT_SECONDS": self.replication_read_timeout_seconds,
-            "REPLICATION_MAX_RETRIES": self.replication_max_retries,
-            "REPLICATION_STREAMING_THRESHOLD_BYTES": self.replication_streaming_threshold_bytes,
-            "REPLICATION_MAX_FAILURES_PER_BUCKET": self.replication_max_failures_per_bucket,
-            "SITE_SYNC_CONNECT_TIMEOUT_SECONDS": self.site_sync_connect_timeout_seconds,
-            "SITE_SYNC_READ_TIMEOUT_SECONDS": self.site_sync_read_timeout_seconds,
-            "SITE_SYNC_MAX_RETRIES": self.site_sync_max_retries,
-            "SITE_SYNC_CLOCK_SKEW_TOLERANCE_SECONDS": self.site_sync_clock_skew_tolerance_seconds,
-            "OBJECT_KEY_MAX_LENGTH_BYTES": self.object_key_max_length_bytes,
-            "OBJECT_CACHE_MAX_SIZE": self.object_cache_max_size,
-            "META_READ_CACHE_MAX": self.meta_read_cache_max,
-            "BUCKET_CONFIG_CACHE_TTL_SECONDS": self.bucket_config_cache_ttl_seconds,
-            "OBJECT_TAG_LIMIT": self.object_tag_limit,
-            "ENCRYPTION_CHUNK_SIZE_BYTES": self.encryption_chunk_size_bytes,
-            "KMS_GENERATE_DATA_KEY_MIN_BYTES": self.kms_generate_data_key_min_bytes,
-            "KMS_GENERATE_DATA_KEY_MAX_BYTES": self.kms_generate_data_key_max_bytes,
-            "LIFECYCLE_MAX_HISTORY_PER_BUCKET": self.lifecycle_max_history_per_bucket,
-            "SITE_ID": self.site_id,
-            "SITE_ENDPOINT": self.site_endpoint,
-            "SITE_REGION": self.site_region,
-            "SITE_PRIORITY": self.site_priority,
-            "RATE_LIMIT_ADMIN": self.ratelimit_admin,
-            "NUM_TRUSTED_PROXIES": self.num_trusted_proxies,
-            "ALLOWED_REDIRECT_HOSTS": self.allowed_redirect_hosts,
-            "ALLOW_INTERNAL_ENDPOINTS": self.allow_internal_endpoints,
-            "WEBSITE_HOSTING_ENABLED": self.website_hosting_enabled,
-            "GC_ENABLED": self.gc_enabled,
-            "GC_INTERVAL_HOURS": self.gc_interval_hours,
-            "GC_TEMP_FILE_MAX_AGE_HOURS": self.gc_temp_file_max_age_hours,
-            "GC_MULTIPART_MAX_AGE_DAYS": self.gc_multipart_max_age_days,
-            "GC_LOCK_FILE_MAX_AGE_HOURS": self.gc_lock_file_max_age_hours,
-            "GC_DRY_RUN": self.gc_dry_run,
-            "GC_IO_THROTTLE_MS": self.gc_io_throttle_ms,
-            "INTEGRITY_ENABLED": self.integrity_enabled,
-            "INTEGRITY_INTERVAL_HOURS": self.integrity_interval_hours,
-            "INTEGRITY_BATCH_SIZE": self.integrity_batch_size,
-            "INTEGRITY_AUTO_HEAL": self.integrity_auto_heal,
-            "INTEGRITY_DRY_RUN": self.integrity_dry_run,
-            "INTEGRITY_IO_THROTTLE_MS": self.integrity_io_throttle_ms,
-        }

python/app/gc.py

@@ -1,596 +0,0 @@
-from __future__ import annotations
-
-import json
-import logging
-import os
-import shutil
-import threading
-import time
-from dataclasses import dataclass, field
-from datetime import datetime, timezone
-from pathlib import Path
-from typing import Any, Dict, List, Optional
-
-logger = logging.getLogger(__name__)
-
-
-@dataclass
-class GCResult:
-    temp_files_deleted: int = 0
-    temp_bytes_freed: int = 0
-    multipart_uploads_deleted: int = 0
-    multipart_bytes_freed: int = 0
-    lock_files_deleted: int = 0
-    orphaned_metadata_deleted: int = 0
-    orphaned_versions_deleted: int = 0
-    orphaned_version_bytes_freed: int = 0
-    empty_dirs_removed: int = 0
-    errors: List[str] = field(default_factory=list)
-    execution_time_seconds: float = 0.0
-
-    def to_dict(self) -> dict:
-        return {
-            "temp_files_deleted": self.temp_files_deleted,
-            "temp_bytes_freed": self.temp_bytes_freed,
-            "multipart_uploads_deleted": self.multipart_uploads_deleted,
-            "multipart_bytes_freed": self.multipart_bytes_freed,
-            "lock_files_deleted": self.lock_files_deleted,
-            "orphaned_metadata_deleted": self.orphaned_metadata_deleted,
-            "orphaned_versions_deleted": self.orphaned_versions_deleted,
-            "orphaned_version_bytes_freed": self.orphaned_version_bytes_freed,
-            "empty_dirs_removed": self.empty_dirs_removed,
-            "errors": self.errors,
-            "execution_time_seconds": self.execution_time_seconds,
-        }
-
-    @property
-    def total_bytes_freed(self) -> int:
-        return self.temp_bytes_freed + self.multipart_bytes_freed + self.orphaned_version_bytes_freed
-
-    @property
-    def has_work(self) -> bool:
-        return (
-            self.temp_files_deleted > 0
-            or self.multipart_uploads_deleted > 0
-            or self.lock_files_deleted > 0
-            or self.orphaned_metadata_deleted > 0
-            or self.orphaned_versions_deleted > 0
-            or self.empty_dirs_removed > 0
-        )
-
-
-@dataclass
-class GCExecutionRecord:
-    timestamp: float
-    result: dict
-    dry_run: bool
-
-    def to_dict(self) -> dict:
-        return {
-            "timestamp": self.timestamp,
-            "result": self.result,
-            "dry_run": self.dry_run,
-        }
-
-    @classmethod
-    def from_dict(cls, data: dict) -> GCExecutionRecord:
-        return cls(
-            timestamp=data["timestamp"],
-            result=data["result"],
-            dry_run=data.get("dry_run", False),
-        )
-
-
-class GCHistoryStore:
-    def __init__(self, storage_root: Path, max_records: int = 50) -> None:
-        self.storage_root = storage_root
-        self.max_records = max_records
-        self._lock = threading.Lock()
-
-    def _get_path(self) -> Path:
-        return self.storage_root / ".myfsio.sys" / "config" / "gc_history.json"
-
-    def load(self) -> List[GCExecutionRecord]:
-        path = self._get_path()
-        if not path.exists():
-            return []
-        try:
-            with open(path, "r", encoding="utf-8") as f:
-                data = json.load(f)
-                return [GCExecutionRecord.from_dict(d) for d in data.get("executions", [])]
-        except (OSError, ValueError, KeyError) as e:
-            logger.error("Failed to load GC history: %s", e)
-            return []
-
-    def save(self, records: List[GCExecutionRecord]) -> None:
-        path = self._get_path()
-        path.parent.mkdir(parents=True, exist_ok=True)
-        data = {"executions": [r.to_dict() for r in records[: self.max_records]]}
-        try:
-            with open(path, "w", encoding="utf-8") as f:
-                json.dump(data, f, indent=2)
-        except OSError as e:
-            logger.error("Failed to save GC history: %s", e)
-
-    def add(self, record: GCExecutionRecord) -> None:
-        with self._lock:
-            records = self.load()
-            records.insert(0, record)
-            self.save(records)
-
-    def get_history(self, limit: int = 50, offset: int = 0) -> List[GCExecutionRecord]:
-        return self.load()[offset : offset + limit]
-
-
-def _dir_size(path: Path) -> int:
-    total = 0
-    try:
-        for f in path.rglob("*"):
-            if f.is_file():
-                try:
-                    total += f.stat().st_size
-                except OSError:
-                    pass
-    except OSError:
-        pass
-    return total
-
-
-def _file_age_hours(path: Path) -> float:
-    try:
-        mtime = path.stat().st_mtime
-        return (time.time() - mtime) / 3600.0
-    except OSError:
-        return 0.0
-
-
-class GarbageCollector:
-    SYSTEM_ROOT = ".myfsio.sys"
-    SYSTEM_TMP_DIR = "tmp"
-    SYSTEM_MULTIPART_DIR = "multipart"
-    SYSTEM_BUCKETS_DIR = "buckets"
-    BUCKET_META_DIR = "meta"
-    BUCKET_VERSIONS_DIR = "versions"
-    INTERNAL_FOLDERS = {".meta", ".versions", ".multipart"}
-
-    def __init__(
-        self,
-        storage_root: Path,
-        interval_hours: float = 6.0,
-        temp_file_max_age_hours: float = 24.0,
-        multipart_max_age_days: int = 7,
-        lock_file_max_age_hours: float = 1.0,
-        dry_run: bool = False,
-        max_history: int = 50,
-        io_throttle_ms: int = 10,
-    ) -> None:
-        self.storage_root = Path(storage_root)
-        self.interval_seconds = interval_hours * 3600.0
-        self.temp_file_max_age_hours = temp_file_max_age_hours
-        self.multipart_max_age_days = multipart_max_age_days
-        self.lock_file_max_age_hours = lock_file_max_age_hours
-        self.dry_run = dry_run
-        self._timer: Optional[threading.Timer] = None
-        self._shutdown = False
-        self._lock = threading.Lock()
-        self._scanning = False
-        self._scan_start_time: Optional[float] = None
-        self._io_throttle = max(0, io_throttle_ms) / 1000.0
-        self.history_store = GCHistoryStore(storage_root, max_records=max_history)
-
-    def start(self) -> None:
-        if self._timer is not None:
-            return
-        self._shutdown = False
-        self._schedule_next()
-        logger.info(
-            "GC started: interval=%.1fh, temp_max_age=%.1fh, multipart_max_age=%dd, lock_max_age=%.1fh, dry_run=%s",
-            self.interval_seconds / 3600.0,
-            self.temp_file_max_age_hours,
-            self.multipart_max_age_days,
-            self.lock_file_max_age_hours,
-            self.dry_run,
-        )
-
-    def stop(self) -> None:
-        self._shutdown = True
-        if self._timer:
-            self._timer.cancel()
-            self._timer = None
-        logger.info("GC stopped")
-
-    def _schedule_next(self) -> None:
-        if self._shutdown:
-            return
-        self._timer = threading.Timer(self.interval_seconds, self._run_cycle)
-        self._timer.daemon = True
-        self._timer.start()
-
-    def _run_cycle(self) -> None:
-        if self._shutdown:
-            return
-        try:
-            self.run_now()
-        except Exception as e:
-            logger.error("GC cycle failed: %s", e)
-        finally:
-            self._schedule_next()
-
-    def run_now(self, dry_run: Optional[bool] = None) -> GCResult:
-        if not self._lock.acquire(blocking=False):
-            raise RuntimeError("GC is already in progress")
-
-        effective_dry_run = dry_run if dry_run is not None else self.dry_run
-
-        try:
-            self._scanning = True
-            self._scan_start_time = time.time()
-
-            start = self._scan_start_time
-            result = GCResult()
-
-            original_dry_run = self.dry_run
-            self.dry_run = effective_dry_run
-            try:
-                self._clean_temp_files(result)
-                self._clean_orphaned_multipart(result)
-                self._clean_stale_locks(result)
-                self._clean_orphaned_metadata(result)
-                self._clean_orphaned_versions(result)
-                self._clean_empty_dirs(result)
-            finally:
-                self.dry_run = original_dry_run
-
-            result.execution_time_seconds = time.time() - start
-
-            if result.has_work or result.errors:
-                logger.info(
-                    "GC completed in %.2fs: temp=%d (%.1f MB), multipart=%d (%.1f MB), "
-                    "locks=%d, meta=%d, versions=%d (%.1f MB), dirs=%d, errors=%d%s",
-                    result.execution_time_seconds,
-                    result.temp_files_deleted,
-                    result.temp_bytes_freed / (1024 * 1024),
-                    result.multipart_uploads_deleted,
-                    result.multipart_bytes_freed / (1024 * 1024),
-                    result.lock_files_deleted,
-                    result.orphaned_metadata_deleted,
-                    result.orphaned_versions_deleted,
-                    result.orphaned_version_bytes_freed / (1024 * 1024),
-                    result.empty_dirs_removed,
-                    len(result.errors),
-                    " (dry run)" if effective_dry_run else "",
-                )
-
-            record = GCExecutionRecord(
-                timestamp=time.time(),
-                result=result.to_dict(),
-                dry_run=effective_dry_run,
-            )
-            self.history_store.add(record)
-
-            return result
-        finally:
-            self._scanning = False
-            self._scan_start_time = None
-            self._lock.release()
-
-    def run_async(self, dry_run: Optional[bool] = None) -> bool:
-        if self._scanning:
-            return False
-        t = threading.Thread(target=self.run_now, args=(dry_run,), daemon=True)
-        t.start()
-        return True
-
-    def _system_path(self) -> Path:
-        return self.storage_root / self.SYSTEM_ROOT
-
-    def _throttle(self) -> bool:
-        if self._shutdown:
-            return True
-        if self._io_throttle > 0:
-            time.sleep(self._io_throttle)
-        return self._shutdown
-
-    def _list_bucket_names(self) -> List[str]:
-        names = []
-        try:
-            for entry in self.storage_root.iterdir():
-                if entry.is_dir() and entry.name != self.SYSTEM_ROOT:
-                    names.append(entry.name)
-        except OSError:
-            pass
-        return names
-
-    def _clean_temp_files(self, result: GCResult) -> None:
-        tmp_dir = self._system_path() / self.SYSTEM_TMP_DIR
-        if not tmp_dir.exists():
-            return
-        try:
-            for entry in tmp_dir.iterdir():
-                if self._throttle():
-                    return
-                if not entry.is_file():
-                    continue
-                age = _file_age_hours(entry)
-                if age < self.temp_file_max_age_hours:
-                    continue
-                try:
-                    size = entry.stat().st_size
-                    if not self.dry_run:
-                        entry.unlink()
-                    result.temp_files_deleted += 1
-                    result.temp_bytes_freed += size
-                except OSError as e:
-                    result.errors.append(f"temp file {entry.name}: {e}")
-        except OSError as e:
-            result.errors.append(f"scan tmp dir: {e}")
-
-    def _clean_orphaned_multipart(self, result: GCResult) -> None:
-        cutoff_hours = self.multipart_max_age_days * 24.0
-        bucket_names = self._list_bucket_names()
-
-        for bucket_name in bucket_names:
-            if self._shutdown:
-                return
-            for multipart_root in (
-                self._system_path() / self.SYSTEM_MULTIPART_DIR / bucket_name,
-                self.storage_root / bucket_name / ".multipart",
-            ):
-                if not multipart_root.exists():
-                    continue
-                try:
-                    for upload_dir in multipart_root.iterdir():
-                        if self._throttle():
-                            return
-                        if not upload_dir.is_dir():
-                            continue
-                        self._maybe_clean_upload(upload_dir, cutoff_hours, result)
-                except OSError as e:
-                    result.errors.append(f"scan multipart {bucket_name}: {e}")
-
-    def _maybe_clean_upload(self, upload_dir: Path, cutoff_hours: float, result: GCResult) -> None:
-        manifest_path = upload_dir / "manifest.json"
-        age = _file_age_hours(manifest_path) if manifest_path.exists() else _file_age_hours(upload_dir)
-
-        if age < cutoff_hours:
-            return
-
-        dir_bytes = _dir_size(upload_dir)
-        try:
-            if not self.dry_run:
-                shutil.rmtree(upload_dir, ignore_errors=True)
-            result.multipart_uploads_deleted += 1
-            result.multipart_bytes_freed += dir_bytes
-        except OSError as e:
-            result.errors.append(f"multipart {upload_dir.name}: {e}")
-
-    def _clean_stale_locks(self, result: GCResult) -> None:
-        buckets_root = self._system_path() / self.SYSTEM_BUCKETS_DIR
-        if not buckets_root.exists():
-            return
-
-        try:
-            for bucket_dir in buckets_root.iterdir():
-                if self._shutdown:
-                    return
-                if not bucket_dir.is_dir():
-                    continue
-                locks_dir = bucket_dir / "locks"
-                if not locks_dir.exists():
-                    continue
-                try:
-                    for lock_file in locks_dir.iterdir():
-                        if self._throttle():
-                            return
-                        if not lock_file.is_file() or not lock_file.name.endswith(".lock"):
-                            continue
-                        age = _file_age_hours(lock_file)
-                        if age < self.lock_file_max_age_hours:
-                            continue
-                        try:
-                            if not self.dry_run:
-                                lock_file.unlink(missing_ok=True)
-                            result.lock_files_deleted += 1
-                        except OSError as e:
-                            result.errors.append(f"lock {lock_file.name}: {e}")
-                except OSError as e:
-                    result.errors.append(f"scan locks {bucket_dir.name}: {e}")
-        except OSError as e:
-            result.errors.append(f"scan buckets for locks: {e}")
-
-    def _clean_orphaned_metadata(self, result: GCResult) -> None:
-        bucket_names = self._list_bucket_names()
-
-        for bucket_name in bucket_names:
-            if self._shutdown:
-                return
-            legacy_meta = self.storage_root / bucket_name / ".meta"
-            if legacy_meta.exists():
-                self._clean_legacy_metadata(bucket_name, legacy_meta, result)
-
-            new_meta = self._system_path() / self.SYSTEM_BUCKETS_DIR / bucket_name / self.BUCKET_META_DIR
-            if new_meta.exists():
-                self._clean_index_metadata(bucket_name, new_meta, result)
-
-    def _clean_legacy_metadata(self, bucket_name: str, meta_root: Path, result: GCResult) -> None:
-        bucket_path = self.storage_root / bucket_name
-        try:
-            for meta_file in meta_root.rglob("*.meta.json"):
-                if self._throttle():
-                    return
-                if not meta_file.is_file():
-                    continue
-                try:
-                    rel = meta_file.relative_to(meta_root)
-                    object_key = rel.as_posix().removesuffix(".meta.json")
-                    object_path = bucket_path / object_key
-                    if not object_path.exists():
-                        if not self.dry_run:
-                            meta_file.unlink(missing_ok=True)
-                        result.orphaned_metadata_deleted += 1
-                except (OSError, ValueError) as e:
-                    result.errors.append(f"legacy meta {bucket_name}/{meta_file.name}: {e}")
-        except OSError as e:
-            result.errors.append(f"scan legacy meta {bucket_name}: {e}")
-
-    def _clean_index_metadata(self, bucket_name: str, meta_root: Path, result: GCResult) -> None:
-        bucket_path = self.storage_root / bucket_name
-        try:
-            for index_file in meta_root.rglob("_index.json"):
-                if self._throttle():
-                    return
-                if not index_file.is_file():
-                    continue
-                try:
-                    with open(index_file, "r", encoding="utf-8") as f:
-                        index_data = json.load(f)
-                except (OSError, json.JSONDecodeError):
-                    continue
-
-                keys_to_remove = []
-                for key in index_data:
-                    rel_dir = index_file.parent.relative_to(meta_root)
-                    if rel_dir == Path("."):
-                        full_key = key
-                    else:
-                        full_key = rel_dir.as_posix() + "/" + key
-                    object_path = bucket_path / full_key
-                    if not object_path.exists():
-                        keys_to_remove.append(key)
-
-                if keys_to_remove:
-                    if not self.dry_run:
-                        for k in keys_to_remove:
-                            index_data.pop(k, None)
-                        if index_data:
-                            try:
-                                with open(index_file, "w", encoding="utf-8") as f:
-                                    json.dump(index_data, f)
-                            except OSError as e:
-                                result.errors.append(f"write index {bucket_name}: {e}")
-                                continue
-                        else:
-                            try:
-                                index_file.unlink(missing_ok=True)
-                            except OSError:
-                                pass
-                    result.orphaned_metadata_deleted += len(keys_to_remove)
-        except OSError as e:
-            result.errors.append(f"scan index meta {bucket_name}: {e}")
-
-    def _clean_orphaned_versions(self, result: GCResult) -> None:
-        bucket_names = self._list_bucket_names()
-
-        for bucket_name in bucket_names:
-            if self._shutdown:
-                return
-            bucket_path = self.storage_root / bucket_name
-            for versions_root in (
-                self._system_path() / self.SYSTEM_BUCKETS_DIR / bucket_name / self.BUCKET_VERSIONS_DIR,
-                self.storage_root / bucket_name / ".versions",
-            ):
-                if not versions_root.exists():
-                    continue
-                try:
-                    for key_dir in versions_root.iterdir():
-                        if self._throttle():
-                            return
-                        if not key_dir.is_dir():
-                            continue
-                        self._clean_versions_for_key(bucket_path, versions_root, key_dir, result)
-                except OSError as e:
-                    result.errors.append(f"scan versions {bucket_name}: {e}")
-
-    def _clean_versions_for_key(
-        self, bucket_path: Path, versions_root: Path, key_dir: Path, result: GCResult
-    ) -> None:
-        try:
-            rel = key_dir.relative_to(versions_root)
-        except ValueError:
-            return
-
-        object_path = bucket_path / rel
-        if object_path.exists():
-            return
-
-        version_files = list(key_dir.glob("*.bin")) + list(key_dir.glob("*.json"))
-        if not version_files:
-            return
-
-        for vf in version_files:
-            try:
-                size = vf.stat().st_size if vf.suffix == ".bin" else 0
-                if not self.dry_run:
-                    vf.unlink(missing_ok=True)
-                if vf.suffix == ".bin":
-                    result.orphaned_version_bytes_freed += size
-                result.orphaned_versions_deleted += 1
-            except OSError as e:
-                result.errors.append(f"version file {vf.name}: {e}")
-
-    def _clean_empty_dirs(self, result: GCResult) -> None:
-        targets = [
-            self._system_path() / self.SYSTEM_TMP_DIR,
-            self._system_path() / self.SYSTEM_MULTIPART_DIR,
-            self._system_path() / self.SYSTEM_BUCKETS_DIR,
-        ]
-        for bucket_name in self._list_bucket_names():
-            targets.append(self.storage_root / bucket_name / ".meta")
-            targets.append(self.storage_root / bucket_name / ".versions")
-            targets.append(self.storage_root / bucket_name / ".multipart")
-
-        for root in targets:
-            if not root.exists():
-                continue
-            self._remove_empty_dirs_recursive(root, root, result)
-
-    def _remove_empty_dirs_recursive(self, path: Path, stop_at: Path, result: GCResult) -> bool:
-        if self._shutdown:
-            return False
-        if not path.is_dir():
-            return False
-
-        try:
-            children = list(path.iterdir())
-        except OSError:
-            return False
-
-        all_empty = True
-        for child in children:
-            if self._throttle():
-                return False
-            if child.is_dir():
-                if not self._remove_empty_dirs_recursive(child, stop_at, result):
-                    all_empty = False
-            else:
-                all_empty = False
-
-        if all_empty and path != stop_at:
-            try:
-                if not self.dry_run:
-                    path.rmdir()
-                result.empty_dirs_removed += 1
-                return True
-            except OSError:
-                return False
-        return all_empty
-
-    def get_history(self, limit: int = 50, offset: int = 0) -> List[dict]:
-        records = self.history_store.get_history(limit, offset)
-        return [r.to_dict() for r in records]
-
-    def get_status(self) -> dict:
-        status: Dict[str, Any] = {
-            "enabled": not self._shutdown or self._timer is not None,
-            "running": self._timer is not None and not self._shutdown,
-            "scanning": self._scanning,
-            "interval_hours": self.interval_seconds / 3600.0,
-            "temp_file_max_age_hours": self.temp_file_max_age_hours,
-            "multipart_max_age_days": self.multipart_max_age_days,
-            "lock_file_max_age_hours": self.lock_file_max_age_hours,
-            "dry_run": self.dry_run,
-            "io_throttle_ms": round(self._io_throttle * 1000),
-        }
-        if self._scanning and self._scan_start_time:
-            status["scan_elapsed_seconds"] = time.time() - self._scan_start_time
-        return status

python/app/integrity.py

@@ -1,995 +0,0 @@
-from __future__ import annotations
-
-import hashlib
-import json
-import logging
-import os
-import threading
-import time
-from dataclasses import dataclass, field
-from pathlib import Path
-from typing import Any, Dict, List, Optional
-
-try:
-    import myfsio_core as _rc
-    if not hasattr(_rc, "md5_file"):
-        raise ImportError("myfsio_core is outdated, rebuild with: cd myfsio_core && maturin develop --release")
-    _HAS_RUST = True
-except ImportError:
-    _HAS_RUST = False
-
-logger = logging.getLogger(__name__)
-
-
-def _compute_etag(path: Path) -> str:
-    if _HAS_RUST:
-        return _rc.md5_file(str(path))
-    checksum = hashlib.md5()
-    with path.open("rb") as handle:
-        for chunk in iter(lambda: handle.read(8192), b""):
-            checksum.update(chunk)
-    return checksum.hexdigest()
-
-
-@dataclass
-class IntegrityIssue:
-    issue_type: str
-    bucket: str
-    key: str
-    detail: str
-    healed: bool = False
-    heal_action: str = ""
-
-    def to_dict(self) -> dict:
-        return {
-            "issue_type": self.issue_type,
-            "bucket": self.bucket,
-            "key": self.key,
-            "detail": self.detail,
-            "healed": self.healed,
-            "heal_action": self.heal_action,
-        }
-
-
-@dataclass
-class IntegrityResult:
-    corrupted_objects: int = 0
-    orphaned_objects: int = 0
-    phantom_metadata: int = 0
-    stale_versions: int = 0
-    etag_cache_inconsistencies: int = 0
-    legacy_metadata_drifts: int = 0
-    issues_healed: int = 0
-    issues: List[IntegrityIssue] = field(default_factory=list)
-    errors: List[str] = field(default_factory=list)
-    objects_scanned: int = 0
-    buckets_scanned: int = 0
-    execution_time_seconds: float = 0.0
-
-    def to_dict(self) -> dict:
-        return {
-            "corrupted_objects": self.corrupted_objects,
-            "orphaned_objects": self.orphaned_objects,
-            "phantom_metadata": self.phantom_metadata,
-            "stale_versions": self.stale_versions,
-            "etag_cache_inconsistencies": self.etag_cache_inconsistencies,
-            "legacy_metadata_drifts": self.legacy_metadata_drifts,
-            "issues_healed": self.issues_healed,
-            "issues": [i.to_dict() for i in self.issues],
-            "errors": self.errors,
-            "objects_scanned": self.objects_scanned,
-            "buckets_scanned": self.buckets_scanned,
-            "execution_time_seconds": self.execution_time_seconds,
-        }
-
-    @property
-    def total_issues(self) -> int:
-        return (
-            self.corrupted_objects
-            + self.orphaned_objects
-            + self.phantom_metadata
-            + self.stale_versions
-            + self.etag_cache_inconsistencies
-            + self.legacy_metadata_drifts
-        )
-
-    @property
-    def has_issues(self) -> bool:
-        return self.total_issues > 0
-
-
-@dataclass
-class IntegrityExecutionRecord:
-    timestamp: float
-    result: dict
-    dry_run: bool
-    auto_heal: bool
-
-    def to_dict(self) -> dict:
-        return {
-            "timestamp": self.timestamp,
-            "result": self.result,
-            "dry_run": self.dry_run,
-            "auto_heal": self.auto_heal,
-        }
-
-    @classmethod
-    def from_dict(cls, data: dict) -> IntegrityExecutionRecord:
-        return cls(
-            timestamp=data["timestamp"],
-            result=data["result"],
-            dry_run=data.get("dry_run", False),
-            auto_heal=data.get("auto_heal", False),
-        )
-
-
-class IntegrityHistoryStore:
-    def __init__(self, storage_root: Path, max_records: int = 50) -> None:
-        self.storage_root = storage_root
-        self.max_records = max_records
-        self._lock = threading.Lock()
-
-    def _get_path(self) -> Path:
-        return self.storage_root / ".myfsio.sys" / "config" / "integrity_history.json"
-
-    def load(self) -> List[IntegrityExecutionRecord]:
-        path = self._get_path()
-        if not path.exists():
-            return []
-        try:
-            with open(path, "r", encoding="utf-8") as f:
-                data = json.load(f)
-                return [IntegrityExecutionRecord.from_dict(d) for d in data.get("executions", [])]
-        except (OSError, ValueError, KeyError) as e:
-            logger.error("Failed to load integrity history: %s", e)
-            return []
-
-    def save(self, records: List[IntegrityExecutionRecord]) -> None:
-        path = self._get_path()
-        path.parent.mkdir(parents=True, exist_ok=True)
-        data = {"executions": [r.to_dict() for r in records[: self.max_records]]}
-        try:
-            with open(path, "w", encoding="utf-8") as f:
-                json.dump(data, f, indent=2)
-        except OSError as e:
-            logger.error("Failed to save integrity history: %s", e)
-
-    def add(self, record: IntegrityExecutionRecord) -> None:
-        with self._lock:
-            records = self.load()
-            records.insert(0, record)
-            self.save(records)
-
-    def get_history(self, limit: int = 50, offset: int = 0) -> List[IntegrityExecutionRecord]:
-        return self.load()[offset : offset + limit]
-
-
-class IntegrityCursorStore:
-    def __init__(self, storage_root: Path) -> None:
-        self.storage_root = storage_root
-        self._lock = threading.Lock()
-
-    def _get_path(self) -> Path:
-        return self.storage_root / ".myfsio.sys" / "config" / "integrity_cursor.json"
-
-    def load(self) -> Dict[str, Any]:
-        path = self._get_path()
-        if not path.exists():
-            return {"buckets": {}}
-        try:
-            with open(path, "r", encoding="utf-8") as f:
-                data = json.load(f)
-                if not isinstance(data.get("buckets"), dict):
-                    return {"buckets": {}}
-                return data
-        except (OSError, ValueError, KeyError):
-            return {"buckets": {}}
-
-    def save(self, data: Dict[str, Any]) -> None:
-        path = self._get_path()
-        path.parent.mkdir(parents=True, exist_ok=True)
-        try:
-            with open(path, "w", encoding="utf-8") as f:
-                json.dump(data, f, indent=2)
-        except OSError as e:
-            logger.error("Failed to save integrity cursor: %s", e)
-
-    def update_bucket(
-        self,
-        bucket_name: str,
-        timestamp: float,
-        last_key: Optional[str] = None,
-        completed: bool = False,
-    ) -> None:
-        with self._lock:
-            data = self.load()
-            entry = data["buckets"].get(bucket_name, {})
-            if completed:
-                entry["last_scanned"] = timestamp
-                entry.pop("last_key", None)
-                entry["completed"] = True
-            else:
-                entry["last_scanned"] = timestamp
-                if last_key is not None:
-                    entry["last_key"] = last_key
-                entry["completed"] = False
-            data["buckets"][bucket_name] = entry
-            self.save(data)
-
-    def clean_stale(self, existing_buckets: List[str]) -> None:
-        with self._lock:
-            data = self.load()
-            existing_set = set(existing_buckets)
-            stale_keys = [k for k in data["buckets"] if k not in existing_set]
-            if stale_keys:
-                for k in stale_keys:
-                    del data["buckets"][k]
-                self.save(data)
-
-    def get_last_key(self, bucket_name: str) -> Optional[str]:
-        data = self.load()
-        entry = data.get("buckets", {}).get(bucket_name)
-        if entry is None:
-            return None
-        return entry.get("last_key")
-
-    def get_bucket_order(self, bucket_names: List[str]) -> List[str]:
-        data = self.load()
-        buckets_info = data.get("buckets", {})
-
-        incomplete = []
-        complete = []
-        for name in bucket_names:
-            entry = buckets_info.get(name)
-            if entry is None:
-                incomplete.append((name, 0.0))
-            elif entry.get("last_key") is not None:
-                incomplete.append((name, entry.get("last_scanned", 0.0)))
-            else:
-                complete.append((name, entry.get("last_scanned", 0.0)))
-
-        incomplete.sort(key=lambda x: x[1])
-        complete.sort(key=lambda x: x[1])
-
-        return [n for n, _ in incomplete] + [n for n, _ in complete]
-
-    def get_info(self) -> Dict[str, Any]:
-        data = self.load()
-        buckets = data.get("buckets", {})
-        return {
-            "tracked_buckets": len(buckets),
-            "buckets": {
-                name: {
-                    "last_scanned": info.get("last_scanned"),
-                    "last_key": info.get("last_key"),
-                    "completed": info.get("completed", False),
-                }
-                for name, info in buckets.items()
-            },
-        }
-
-
-MAX_ISSUES = 500
-
-
-class IntegrityChecker:
-    SYSTEM_ROOT = ".myfsio.sys"
-    SYSTEM_BUCKETS_DIR = "buckets"
-    BUCKET_META_DIR = "meta"
-    BUCKET_VERSIONS_DIR = "versions"
-    INTERNAL_FOLDERS = {".meta", ".versions", ".multipart"}
-
-    def __init__(
-        self,
-        storage_root: Path,
-        interval_hours: float = 24.0,
-        batch_size: int = 1000,
-        auto_heal: bool = False,
-        dry_run: bool = False,
-        max_history: int = 50,
-        io_throttle_ms: int = 10,
-    ) -> None:
-        self.storage_root = Path(storage_root)
-        self.interval_seconds = interval_hours * 3600.0
-        self.batch_size = batch_size
-        self.auto_heal = auto_heal
-        self.dry_run = dry_run
-        self._timer: Optional[threading.Timer] = None
-        self._shutdown = False
-        self._lock = threading.Lock()
-        self._scanning = False
-        self._scan_start_time: Optional[float] = None
-        self._io_throttle = max(0, io_throttle_ms) / 1000.0
-        self.history_store = IntegrityHistoryStore(storage_root, max_records=max_history)
-        self.cursor_store = IntegrityCursorStore(self.storage_root)
-
-    def start(self) -> None:
-        if self._timer is not None:
-            return
-        self._shutdown = False
-        self._schedule_next()
-        logger.info(
-            "Integrity checker started: interval=%.1fh, batch_size=%d, auto_heal=%s, dry_run=%s",
-            self.interval_seconds / 3600.0,
-            self.batch_size,
-            self.auto_heal,
-            self.dry_run,
-        )
-
-    def stop(self) -> None:
-        self._shutdown = True
-        if self._timer:
-            self._timer.cancel()
-            self._timer = None
-        logger.info("Integrity checker stopped")
-
-    def _schedule_next(self) -> None:
-        if self._shutdown:
-            return
-        self._timer = threading.Timer(self.interval_seconds, self._run_cycle)
-        self._timer.daemon = True
-        self._timer.start()
-
-    def _run_cycle(self) -> None:
-        if self._shutdown:
-            return
-        try:
-            self.run_now()
-        except Exception as e:
-            logger.error("Integrity check cycle failed: %s", e)
-        finally:
-            self._schedule_next()
-
-    def run_now(self, auto_heal: Optional[bool] = None, dry_run: Optional[bool] = None) -> IntegrityResult:
-        if not self._lock.acquire(blocking=False):
-            raise RuntimeError("Integrity scan is already in progress")
-
-        try:
-            self._scanning = True
-            self._scan_start_time = time.time()
-
-            effective_auto_heal = auto_heal if auto_heal is not None else self.auto_heal
-            effective_dry_run = dry_run if dry_run is not None else self.dry_run
-
-            start = self._scan_start_time
-            result = IntegrityResult()
-
-            bucket_names = self._list_bucket_names()
-            self.cursor_store.clean_stale(bucket_names)
-            ordered_buckets = self.cursor_store.get_bucket_order(bucket_names)
-
-            for bucket_name in ordered_buckets:
-                if self._batch_exhausted(result):
-                    break
-                result.buckets_scanned += 1
-                cursor_key = self.cursor_store.get_last_key(bucket_name)
-                key_corrupted = self._check_corrupted_objects(bucket_name, result, effective_auto_heal, effective_dry_run, cursor_key)
-                key_orphaned = self._check_orphaned_objects(bucket_name, result, effective_auto_heal, effective_dry_run, cursor_key)
-                key_phantom = self._check_phantom_metadata(bucket_name, result, effective_auto_heal, effective_dry_run, cursor_key)
-                self._check_stale_versions(bucket_name, result, effective_auto_heal, effective_dry_run)
-                self._check_etag_cache(bucket_name, result, effective_auto_heal, effective_dry_run)
-                self._check_legacy_metadata(bucket_name, result, effective_auto_heal, effective_dry_run)
-                returned_keys = [k for k in (key_corrupted, key_orphaned, key_phantom) if k is not None]
-                bucket_exhausted = self._batch_exhausted(result)
-                if bucket_exhausted and returned_keys:
-                    self.cursor_store.update_bucket(bucket_name, time.time(), last_key=min(returned_keys))
-                else:
-                    self.cursor_store.update_bucket(bucket_name, time.time(), completed=True)
-
-            result.execution_time_seconds = time.time() - start
-
-            if result.has_issues or result.errors:
-                logger.info(
-                    "Integrity check completed in %.2fs: corrupted=%d, orphaned=%d, phantom=%d, "
-                    "stale_versions=%d, etag_cache=%d, legacy_drift=%d, healed=%d, errors=%d%s",
-                    result.execution_time_seconds,
-                    result.corrupted_objects,
-                    result.orphaned_objects,
-                    result.phantom_metadata,
-                    result.stale_versions,
-                    result.etag_cache_inconsistencies,
-                    result.legacy_metadata_drifts,
-                    result.issues_healed,
-                    len(result.errors),
-                    " (dry run)" if effective_dry_run else "",
-                )
-
-            record = IntegrityExecutionRecord(
-                timestamp=time.time(),
-                result=result.to_dict(),
-                dry_run=effective_dry_run,
-                auto_heal=effective_auto_heal,
-            )
-            self.history_store.add(record)
-
-            return result
-        finally:
-            self._scanning = False
-            self._scan_start_time = None
-            self._lock.release()
-
-    def run_async(self, auto_heal: Optional[bool] = None, dry_run: Optional[bool] = None) -> bool:
-        if self._scanning:
-            return False
-        t = threading.Thread(target=self.run_now, args=(auto_heal, dry_run), daemon=True)
-        t.start()
-        return True
-
-    def _system_path(self) -> Path:
-        return self.storage_root / self.SYSTEM_ROOT
-
-    def _list_bucket_names(self) -> List[str]:
-        names = []
-        try:
-            for entry in self.storage_root.iterdir():
-                if entry.is_dir() and entry.name != self.SYSTEM_ROOT:
-                    names.append(entry.name)
-        except OSError:
-            pass
-        return names
-
-    def _throttle(self) -> bool:
-        if self._shutdown:
-            return True
-        if self._io_throttle > 0:
-            time.sleep(self._io_throttle)
-        return self._shutdown
-
-    def _batch_exhausted(self, result: IntegrityResult) -> bool:
-        return self._shutdown or result.objects_scanned >= self.batch_size
-
-    def _add_issue(self, result: IntegrityResult, issue: IntegrityIssue) -> None:
-        if len(result.issues) < MAX_ISSUES:
-            result.issues.append(issue)
-
-    def _collect_index_keys(
-        self, meta_root: Path, cursor_key: Optional[str] = None,
-    ) -> Dict[str, Dict[str, Any]]:
-        all_keys: Dict[str, Dict[str, Any]] = {}
-        if not meta_root.exists():
-            return all_keys
-        try:
-            for index_file in meta_root.rglob("_index.json"):
-                if not index_file.is_file():
-                    continue
-                rel_dir = index_file.parent.relative_to(meta_root)
-                dir_prefix = "" if rel_dir == Path(".") else rel_dir.as_posix()
-                if cursor_key is not None and dir_prefix:
-                    full_prefix = dir_prefix + "/"
-                    if not cursor_key.startswith(full_prefix) and cursor_key > full_prefix:
-                        continue
-                try:
-                    index_data = json.loads(index_file.read_text(encoding="utf-8"))
-                except (OSError, json.JSONDecodeError):
-                    continue
-                for key_name, entry in index_data.items():
-                    full_key = (dir_prefix + "/" + key_name) if dir_prefix else key_name
-                    if cursor_key is not None and full_key <= cursor_key:
-                        continue
-                    all_keys[full_key] = {
-                        "entry": entry,
-                        "index_file": index_file,
-                        "key_name": key_name,
-                    }
-        except OSError:
-            pass
-        return all_keys
-
-    def _walk_bucket_files_sorted(
-        self, bucket_path: Path, cursor_key: Optional[str] = None,
-    ):
-        def _walk(dir_path: Path, prefix: str):
-            try:
-                entries = list(os.scandir(dir_path))
-            except OSError:
-                return
-
-            def _sort_key(e):
-                if e.is_dir(follow_symlinks=False):
-                    return e.name + "/"
-                return e.name
-
-            entries.sort(key=_sort_key)
-
-            for entry in entries:
-                if entry.is_dir(follow_symlinks=False):
-                    if not prefix and entry.name in self.INTERNAL_FOLDERS:
-                        continue
-                    new_prefix = (prefix + "/" + entry.name) if prefix else entry.name
-                    if cursor_key is not None:
-                        full_prefix = new_prefix + "/"
-                        if not cursor_key.startswith(full_prefix) and cursor_key > full_prefix:
-                            continue
-                    yield from _walk(Path(entry.path), new_prefix)
-                elif entry.is_file(follow_symlinks=False):
-                    full_key = (prefix + "/" + entry.name) if prefix else entry.name
-                    if cursor_key is not None and full_key <= cursor_key:
-                        continue
-                    yield full_key
-
-        yield from _walk(bucket_path, "")
-
-    def _check_corrupted_objects(
-        self, bucket_name: str, result: IntegrityResult, auto_heal: bool, dry_run: bool,
-        cursor_key: Optional[str] = None,
-    ) -> Optional[str]:
-        if self._batch_exhausted(result):
-            return None
-        bucket_path = self.storage_root / bucket_name
-        meta_root = self._system_path() / self.SYSTEM_BUCKETS_DIR / bucket_name / self.BUCKET_META_DIR
-
-        if not meta_root.exists():
-            return None
-
-        last_key = None
-        try:
-            all_keys = self._collect_index_keys(meta_root, cursor_key)
-            sorted_keys = sorted(all_keys.keys())
-
-            for full_key in sorted_keys:
-                if self._throttle():
-                    return last_key
-                if self._batch_exhausted(result):
-                    return last_key
-
-                info = all_keys[full_key]
-                entry = info["entry"]
-                index_file = info["index_file"]
-                key_name = info["key_name"]
-
-                object_path = bucket_path / full_key
-                if not object_path.exists():
-                    continue
-
-                result.objects_scanned += 1
-                last_key = full_key
-
-                meta = entry.get("metadata", {}) if isinstance(entry, dict) else {}
-                stored_etag = meta.get("__etag__")
-                if not stored_etag:
-                    continue
-
-                try:
-                    actual_etag = _compute_etag(object_path)
-                except OSError:
-                    continue
-
-                if actual_etag != stored_etag:
-                    result.corrupted_objects += 1
-                    issue = IntegrityIssue(
-                        issue_type="corrupted_object",
-                        bucket=bucket_name,
-                        key=full_key,
-                        detail=f"stored_etag={stored_etag} actual_etag={actual_etag}",
-                    )
-
-                    if auto_heal and not dry_run:
-                        try:
-                            stat = object_path.stat()
-                            meta["__etag__"] = actual_etag
-                            meta["__size__"] = str(stat.st_size)
-                            meta["__last_modified__"] = str(stat.st_mtime)
-                            try:
-                                index_data = json.loads(index_file.read_text(encoding="utf-8"))
-                            except (OSError, json.JSONDecodeError):
-                                index_data = {}
-                            index_data[key_name] = {"metadata": meta}
-                            self._atomic_write_index(index_file, index_data)
-                            issue.healed = True
-                            issue.heal_action = "updated etag in index"
-                            result.issues_healed += 1
-                        except OSError as e:
-                            result.errors.append(f"heal corrupted {bucket_name}/{full_key}: {e}")
-
-                    self._add_issue(result, issue)
-        except OSError as e:
-            result.errors.append(f"check corrupted {bucket_name}: {e}")
-        return last_key
-
-    def _check_orphaned_objects(
-        self, bucket_name: str, result: IntegrityResult, auto_heal: bool, dry_run: bool,
-        cursor_key: Optional[str] = None,
-    ) -> Optional[str]:
-        if self._batch_exhausted(result):
-            return None
-        bucket_path = self.storage_root / bucket_name
-        meta_root = self._system_path() / self.SYSTEM_BUCKETS_DIR / bucket_name / self.BUCKET_META_DIR
-
-        last_key = None
-        try:
-            for full_key in self._walk_bucket_files_sorted(bucket_path, cursor_key):
-                if self._throttle():
-                    return last_key
-                if self._batch_exhausted(result):
-                    return last_key
-
-                result.objects_scanned += 1
-                last_key = full_key
-                key_path = Path(full_key)
-                key_name = key_path.name
-                parent = key_path.parent
-
-                if parent == Path("."):
-                    index_path = meta_root / "_index.json"
-                else:
-                    index_path = meta_root / parent / "_index.json"
-
-                has_entry = False
-                if index_path.exists():
-                    try:
-                        index_data = json.loads(index_path.read_text(encoding="utf-8"))
-                        has_entry = key_name in index_data
-                    except (OSError, json.JSONDecodeError):
-                        pass
-
-                if not has_entry:
-                    result.orphaned_objects += 1
-                    issue = IntegrityIssue(
-                        issue_type="orphaned_object",
-                        bucket=bucket_name,
-                        key=full_key,
-                        detail="file exists without metadata entry",
-                    )
-
-                    if auto_heal and not dry_run:
-                        try:
-                            object_path = bucket_path / full_key
-                            etag = _compute_etag(object_path)
-                            stat = object_path.stat()
-                            meta = {
-                                "__etag__": etag,
-                                "__size__": str(stat.st_size),
-                                "__last_modified__": str(stat.st_mtime),
-                            }
-                            index_data = {}
-                            if index_path.exists():
-                                try:
-                                    index_data = json.loads(index_path.read_text(encoding="utf-8"))
-                                except (OSError, json.JSONDecodeError):
-                                    pass
-                            index_data[key_name] = {"metadata": meta}
-                            self._atomic_write_index(index_path, index_data)
-                            issue.healed = True
-                            issue.heal_action = "created metadata entry"
-                            result.issues_healed += 1
-                        except OSError as e:
-                            result.errors.append(f"heal orphaned {bucket_name}/{full_key}: {e}")
-
-                    self._add_issue(result, issue)
-        except OSError as e:
-            result.errors.append(f"check orphaned {bucket_name}: {e}")
-        return last_key
-
-    def _check_phantom_metadata(
-        self, bucket_name: str, result: IntegrityResult, auto_heal: bool, dry_run: bool,
-        cursor_key: Optional[str] = None,
-    ) -> Optional[str]:
-        if self._batch_exhausted(result):
-            return None
-        bucket_path = self.storage_root / bucket_name
-        meta_root = self._system_path() / self.SYSTEM_BUCKETS_DIR / bucket_name / self.BUCKET_META_DIR
-
-        if not meta_root.exists():
-            return None
-
-        last_key = None
-        try:
-            all_keys = self._collect_index_keys(meta_root, cursor_key)
-            sorted_keys = sorted(all_keys.keys())
-
-            heal_by_index: Dict[Path, List[str]] = {}
-
-            for full_key in sorted_keys:
-                if self._batch_exhausted(result):
-                    break
-
-                result.objects_scanned += 1
-                last_key = full_key
-
-                object_path = bucket_path / full_key
-                if not object_path.exists():
-                    result.phantom_metadata += 1
-                    info = all_keys[full_key]
-                    issue = IntegrityIssue(
-                        issue_type="phantom_metadata",
-                        bucket=bucket_name,
-                        key=full_key,
-                        detail="metadata entry without file on disk",
-                    )
-                    if auto_heal and not dry_run:
-                        index_file = info["index_file"]
-                        heal_by_index.setdefault(index_file, []).append(info["key_name"])
-                        issue.healed = True
-                        issue.heal_action = "removed stale index entry"
-                        result.issues_healed += 1
-                    self._add_issue(result, issue)
-
-            if heal_by_index and auto_heal and not dry_run:
-                for index_file, keys_to_remove in heal_by_index.items():
-                    try:
-                        index_data = json.loads(index_file.read_text(encoding="utf-8"))
-                        for k in keys_to_remove:
-                            index_data.pop(k, None)
-                        if index_data:
-                            self._atomic_write_index(index_file, index_data)
-                        else:
-                            index_file.unlink(missing_ok=True)
-                    except OSError as e:
-                        result.errors.append(f"heal phantom {bucket_name}: {e}")
-        except OSError as e:
-            result.errors.append(f"check phantom {bucket_name}: {e}")
-        return last_key
-
-    def _check_stale_versions(
-        self, bucket_name: str, result: IntegrityResult, auto_heal: bool, dry_run: bool
-    ) -> None:
-        if self._batch_exhausted(result):
-            return
-        versions_root = self._system_path() / self.SYSTEM_BUCKETS_DIR / bucket_name / self.BUCKET_VERSIONS_DIR
-
-        if not versions_root.exists():
-            return
-
-        try:
-            for key_dir in versions_root.rglob("*"):
-                if self._throttle():
-                    return
-                if self._batch_exhausted(result):
-                    return
-                if not key_dir.is_dir():
-                    continue
-
-                bin_files = {f.stem: f for f in key_dir.glob("*.bin")}
-                json_files = {f.stem: f for f in key_dir.glob("*.json")}
-
-                for stem, bin_file in bin_files.items():
-                    if self._batch_exhausted(result):
-                        return
-                    result.objects_scanned += 1
-                    if stem not in json_files:
-                        result.stale_versions += 1
-                        issue = IntegrityIssue(
-                            issue_type="stale_version",
-                            bucket=bucket_name,
-                            key=f"{key_dir.relative_to(versions_root).as_posix()}/{bin_file.name}",
-                            detail="version data without manifest",
-                        )
-                        if auto_heal and not dry_run:
-                            try:
-                                bin_file.unlink(missing_ok=True)
-                                issue.healed = True
-                                issue.heal_action = "removed orphaned version data"
-                                result.issues_healed += 1
-                            except OSError as e:
-                                result.errors.append(f"heal stale version {bin_file}: {e}")
-                        self._add_issue(result, issue)
-
-                for stem, json_file in json_files.items():
-                    if self._batch_exhausted(result):
-                        return
-                    result.objects_scanned += 1
-                    if stem not in bin_files:
-                        result.stale_versions += 1
-                        issue = IntegrityIssue(
-                            issue_type="stale_version",
-                            bucket=bucket_name,
-                            key=f"{key_dir.relative_to(versions_root).as_posix()}/{json_file.name}",
-                            detail="version manifest without data",
-                        )
-                        if auto_heal and not dry_run:
-                            try:
-                                json_file.unlink(missing_ok=True)
-                                issue.healed = True
-                                issue.heal_action = "removed orphaned version manifest"
-                                result.issues_healed += 1
-                            except OSError as e:
-                                result.errors.append(f"heal stale version {json_file}: {e}")
-                        self._add_issue(result, issue)
-        except OSError as e:
-            result.errors.append(f"check stale versions {bucket_name}: {e}")
-
-    def _check_etag_cache(
-        self, bucket_name: str, result: IntegrityResult, auto_heal: bool, dry_run: bool
-    ) -> None:
-        if self._batch_exhausted(result):
-            return
-        etag_index_path = self._system_path() / self.SYSTEM_BUCKETS_DIR / bucket_name / "etag_index.json"
-
-        if not etag_index_path.exists():
-            return
-
-        meta_root = self._system_path() / self.SYSTEM_BUCKETS_DIR / bucket_name / self.BUCKET_META_DIR
-        if not meta_root.exists():
-            return
-
-        try:
-            etag_cache = json.loads(etag_index_path.read_text(encoding="utf-8"))
-        except (OSError, json.JSONDecodeError):
-            return
-
-        found_mismatch = False
-
-        for full_key, cached_etag in etag_cache.items():
-            if self._batch_exhausted(result):
-                break
-            result.objects_scanned += 1
-            key_path = Path(full_key)
-            key_name = key_path.name
-            parent = key_path.parent
-
-            if parent == Path("."):
-                index_path = meta_root / "_index.json"
-            else:
-                index_path = meta_root / parent / "_index.json"
-
-            if not index_path.exists():
-                continue
-
-            try:
-                index_data = json.loads(index_path.read_text(encoding="utf-8"))
-            except (OSError, json.JSONDecodeError):
-                continue
-
-            entry = index_data.get(key_name)
-            if not entry:
-                continue
-
-            meta = entry.get("metadata", {}) if isinstance(entry, dict) else {}
-            stored_etag = meta.get("__etag__")
-
-            if stored_etag and cached_etag != stored_etag:
-                result.etag_cache_inconsistencies += 1
-                found_mismatch = True
-                issue = IntegrityIssue(
-                    issue_type="etag_cache_inconsistency",
-                    bucket=bucket_name,
-                    key=full_key,
-                    detail=f"cached_etag={cached_etag} index_etag={stored_etag}",
-                )
-                self._add_issue(result, issue)
-
-        if found_mismatch and auto_heal and not dry_run:
-            try:
-                etag_index_path.unlink(missing_ok=True)
-                for issue in result.issues:
-                    if issue.issue_type == "etag_cache_inconsistency" and issue.bucket == bucket_name and not issue.healed:
-                        issue.healed = True
-                        issue.heal_action = "deleted etag_index.json"
-                        result.issues_healed += 1
-            except OSError as e:
-                result.errors.append(f"heal etag cache {bucket_name}: {e}")
-
-    def _check_legacy_metadata(
-        self, bucket_name: str, result: IntegrityResult, auto_heal: bool, dry_run: bool
-    ) -> None:
-        if self._batch_exhausted(result):
-            return
-        legacy_meta_root = self.storage_root / bucket_name / ".meta"
-        if not legacy_meta_root.exists():
-            return
-
-        meta_root = self._system_path() / self.SYSTEM_BUCKETS_DIR / bucket_name / self.BUCKET_META_DIR
-
-        try:
-            for meta_file in legacy_meta_root.rglob("*.meta.json"):
-                if self._throttle():
-                    return
-                if self._batch_exhausted(result):
-                    return
-                if not meta_file.is_file():
-                    continue
-
-                result.objects_scanned += 1
-                try:
-                    rel = meta_file.relative_to(legacy_meta_root)
-                except ValueError:
-                    continue
-
-                full_key = rel.as_posix().removesuffix(".meta.json")
-                key_path = Path(full_key)
-                key_name = key_path.name
-                parent = key_path.parent
-
-                if parent == Path("."):
-                    index_path = meta_root / "_index.json"
-                else:
-                    index_path = meta_root / parent / "_index.json"
-
-                try:
-                    legacy_data = json.loads(meta_file.read_text(encoding="utf-8"))
-                except (OSError, json.JSONDecodeError):
-                    continue
-
-                index_entry = None
-                if index_path.exists():
-                    try:
-                        index_data = json.loads(index_path.read_text(encoding="utf-8"))
-                        index_entry = index_data.get(key_name)
-                    except (OSError, json.JSONDecodeError):
-                        pass
-
-                if index_entry is None:
-                    result.legacy_metadata_drifts += 1
-                    issue = IntegrityIssue(
-                        issue_type="legacy_metadata_drift",
-                        bucket=bucket_name,
-                        key=full_key,
-                        detail="unmigrated legacy .meta.json",
-                    )
-
-                    if auto_heal and not dry_run:
-                        try:
-                            index_data = {}
-                            if index_path.exists():
-                                try:
-                                    index_data = json.loads(index_path.read_text(encoding="utf-8"))
-                                except (OSError, json.JSONDecodeError):
-                                    pass
-                            index_data[key_name] = {"metadata": legacy_data}
-                            self._atomic_write_index(index_path, index_data)
-                            meta_file.unlink(missing_ok=True)
-                            issue.healed = True
-                            issue.heal_action = "migrated to index and deleted legacy file"
-                            result.issues_healed += 1
-                        except OSError as e:
-                            result.errors.append(f"heal legacy {bucket_name}/{full_key}: {e}")
-
-                    self._add_issue(result, issue)
-                else:
-                    index_meta = index_entry.get("metadata", {}) if isinstance(index_entry, dict) else {}
-                    if legacy_data != index_meta:
-                        result.legacy_metadata_drifts += 1
-                        issue = IntegrityIssue(
-                            issue_type="legacy_metadata_drift",
-                            bucket=bucket_name,
-                            key=full_key,
-                            detail="legacy .meta.json differs from index entry",
-                        )
-
-                        if auto_heal and not dry_run:
-                            try:
-                                meta_file.unlink(missing_ok=True)
-                                issue.healed = True
-                                issue.heal_action = "deleted legacy file (index is authoritative)"
-                                result.issues_healed += 1
-                            except OSError as e:
-                                result.errors.append(f"heal legacy drift {bucket_name}/{full_key}: {e}")
-
-                        self._add_issue(result, issue)
-        except OSError as e:
-            result.errors.append(f"check legacy meta {bucket_name}: {e}")
-
-    @staticmethod
-    def _atomic_write_index(index_path: Path, data: Dict[str, Any]) -> None:
-        index_path.parent.mkdir(parents=True, exist_ok=True)
-        tmp_path = index_path.with_suffix(".tmp")
-        try:
-            with open(tmp_path, "w", encoding="utf-8") as f:
-                json.dump(data, f)
-            os.replace(str(tmp_path), str(index_path))
-        except BaseException:
-            try:
-                tmp_path.unlink(missing_ok=True)
-            except OSError:
-                pass
-            raise
-
-    def get_history(self, limit: int = 50, offset: int = 0) -> List[dict]:
-        records = self.history_store.get_history(limit, offset)
-        return [r.to_dict() for r in records]
-
-    def get_status(self) -> dict:
-        status: Dict[str, Any] = {
-            "enabled": not self._shutdown or self._timer is not None,
-            "running": self._timer is not None and not self._shutdown,
-            "scanning": self._scanning,
-            "interval_hours": self.interval_seconds / 3600.0,
-            "batch_size": self.batch_size,
-            "auto_heal": self.auto_heal,
-            "dry_run": self.dry_run,
-            "io_throttle_ms": round(self._io_throttle * 1000),
-        }
-        if self._scanning and self._scan_start_time is not None:
-            status["scan_elapsed_seconds"] = round(time.time() - self._scan_start_time, 1)
-        status["cursor"] = self.cursor_store.get_info()
-        return status

python/app/s3_client.py

@@ -1,296 +0,0 @@
-from __future__ import annotations
-
-import json
-import logging
-import threading
-import time
-from typing import Any, Generator, Optional
-
-import boto3
-from botocore.config import Config
-from botocore.exceptions import ClientError, EndpointConnectionError, ConnectionClosedError
-from flask import current_app, session
-
-logger = logging.getLogger(__name__)
-
-UI_PROXY_USER_AGENT = "MyFSIO-UIProxy/1.0"
-
-_BOTO_ERROR_MAP = {
-    "NoSuchBucket": 404,
-    "NoSuchKey": 404,
-    "NoSuchUpload": 404,
-    "BucketAlreadyExists": 409,
-    "BucketAlreadyOwnedByYou": 409,
-    "BucketNotEmpty": 409,
-    "AccessDenied": 403,
-    "InvalidAccessKeyId": 403,
-    "SignatureDoesNotMatch": 403,
-    "InvalidBucketName": 400,
-    "InvalidArgument": 400,
-    "MalformedXML": 400,
-    "EntityTooLarge": 400,
-    "QuotaExceeded": 403,
-}
-
-_UPLOAD_REGISTRY_MAX_AGE = 86400
-_UPLOAD_REGISTRY_CLEANUP_INTERVAL = 3600
-
-
-class UploadRegistry:
-    def __init__(self) -> None:
-        self._entries: dict[str, tuple[str, str, float]] = {}
-        self._lock = threading.Lock()
-        self._last_cleanup = time.monotonic()
-
-    def register(self, upload_id: str, bucket_name: str, object_key: str) -> None:
-        with self._lock:
-            self._entries[upload_id] = (bucket_name, object_key, time.monotonic())
-            self._maybe_cleanup()
-
-    def get_key(self, upload_id: str, bucket_name: str) -> Optional[str]:
-        with self._lock:
-            entry = self._entries.get(upload_id)
-            if entry is None:
-                return None
-            stored_bucket, key, created_at = entry
-            if stored_bucket != bucket_name:
-                return None
-            if time.monotonic() - created_at > _UPLOAD_REGISTRY_MAX_AGE:
-                del self._entries[upload_id]
-                return None
-            return key
-
-    def remove(self, upload_id: str) -> None:
-        with self._lock:
-            self._entries.pop(upload_id, None)
-
-    def _maybe_cleanup(self) -> None:
-        now = time.monotonic()
-        if now - self._last_cleanup < _UPLOAD_REGISTRY_CLEANUP_INTERVAL:
-            return
-        self._last_cleanup = now
-        cutoff = now - _UPLOAD_REGISTRY_MAX_AGE
-        stale = [uid for uid, (_, _, ts) in self._entries.items() if ts < cutoff]
-        for uid in stale:
-            del self._entries[uid]
-
-
-class S3ProxyClient:
-    def __init__(self, api_base_url: str, region: str = "us-east-1") -> None:
-        if not api_base_url:
-            raise ValueError("api_base_url is required for S3ProxyClient")
-        self._api_base_url = api_base_url.rstrip("/")
-        self._region = region
-        self.upload_registry = UploadRegistry()
-
-    @property
-    def api_base_url(self) -> str:
-        return self._api_base_url
-
-    def get_client(self, access_key: str, secret_key: str) -> Any:
-        if not access_key or not secret_key:
-            raise ValueError("Both access_key and secret_key are required")
-        config = Config(
-            user_agent_extra=UI_PROXY_USER_AGENT,
-            connect_timeout=5,
-            read_timeout=30,
-            retries={"max_attempts": 0},
-            signature_version="s3v4",
-            s3={"addressing_style": "path"},
-            request_checksum_calculation="when_required",
-            response_checksum_validation="when_required",
-        )
-        return boto3.client(
-            "s3",
-            endpoint_url=self._api_base_url,
-            aws_access_key_id=access_key,
-            aws_secret_access_key=secret_key,
-            region_name=self._region,
-            config=config,
-        )
-
-
-def _get_proxy() -> S3ProxyClient:
-    proxy = current_app.extensions.get("s3_proxy")
-    if proxy is None:
-        raise RuntimeError(
-            "S3 proxy not configured. Set API_BASE_URL or run both API and UI servers."
-        )
-    return proxy
-
-
-def _get_session_creds() -> tuple[str, str]:
-    secret_store = current_app.extensions["secret_store"]
-    secret_store.purge_expired()
-    token = session.get("cred_token")
-    if not token:
-        raise PermissionError("Not authenticated")
-    creds = secret_store.peek(token)
-    if not creds:
-        raise PermissionError("Session expired")
-    access_key = creds.get("access_key", "")
-    secret_key = creds.get("secret_key", "")
-    if not access_key or not secret_key:
-        raise PermissionError("Invalid session credentials")
-    return access_key, secret_key
-
-
-def get_session_s3_client() -> Any:
-    proxy = _get_proxy()
-    access_key, secret_key = _get_session_creds()
-    return proxy.get_client(access_key, secret_key)
-
-
-def get_upload_registry() -> UploadRegistry:
-    return _get_proxy().upload_registry
-
-
-def handle_client_error(exc: ClientError) -> tuple[dict[str, str], int]:
-    error_info = exc.response.get("Error", {})
-    code = error_info.get("Code", "InternalError")
-    message = error_info.get("Message") or "S3 operation failed"
-    http_status = _BOTO_ERROR_MAP.get(code)
-    if http_status is None:
-        http_status = exc.response.get("ResponseMetadata", {}).get("HTTPStatusCode", 500)
-    return {"error": message}, http_status
-
-
-def handle_connection_error(exc: Exception) -> tuple[dict[str, str], int]:
-    logger.error("S3 API connection failed: %s", exc)
-    return {"error": "S3 API server is unreachable. Ensure the API server is running."}, 502
-
-
-def format_datetime_display(dt: Any, display_tz: str = "UTC") -> str:
-    from .ui import _format_datetime_display
-    return _format_datetime_display(dt, display_tz)
-
-
-def format_datetime_iso(dt: Any, display_tz: str = "UTC") -> str:
-    from .ui import _format_datetime_iso
-    return _format_datetime_iso(dt, display_tz)
-
-
-def build_url_templates(bucket_name: str) -> dict[str, str]:
-    from flask import url_for
-    preview_t = url_for("ui.object_preview", bucket_name=bucket_name, object_key="KEY_PLACEHOLDER")
-    delete_t = url_for("ui.delete_object", bucket_name=bucket_name, object_key="KEY_PLACEHOLDER")
-    presign_t = url_for("ui.object_presign", bucket_name=bucket_name, object_key="KEY_PLACEHOLDER")
-    versions_t = url_for("ui.object_versions", bucket_name=bucket_name, object_key="KEY_PLACEHOLDER")
-    restore_t = url_for(
-        "ui.restore_object_version",
-        bucket_name=bucket_name,
-        object_key="KEY_PLACEHOLDER",
-        version_id="VERSION_ID_PLACEHOLDER",
-    )
-    tags_t = url_for("ui.object_tags", bucket_name=bucket_name, object_key="KEY_PLACEHOLDER")
-    copy_t = url_for("ui.copy_object", bucket_name=bucket_name, object_key="KEY_PLACEHOLDER")
-    move_t = url_for("ui.move_object", bucket_name=bucket_name, object_key="KEY_PLACEHOLDER")
-    metadata_t = url_for("ui.object_metadata", bucket_name=bucket_name, object_key="KEY_PLACEHOLDER")
-    return {
-        "preview": preview_t,
-        "download": preview_t + "?download=1",
-        "presign": presign_t,
-        "delete": delete_t,
-        "versions": versions_t,
-        "restore": restore_t,
-        "tags": tags_t,
-        "copy": copy_t,
-        "move": move_t,
-        "metadata": metadata_t,
-    }
-
-
-def translate_list_objects(
-    boto3_response: dict[str, Any],
-    url_templates: dict[str, str],
-    display_tz: str = "UTC",
-    versioning_enabled: bool = False,
-) -> dict[str, Any]:
-    objects_data = []
-    for obj in boto3_response.get("Contents", []):
-        last_mod = obj["LastModified"]
-        objects_data.append({
-            "key": obj["Key"],
-            "size": obj["Size"],
-            "last_modified": last_mod.isoformat(),
-            "last_modified_display": format_datetime_display(last_mod, display_tz),
-            "last_modified_iso": format_datetime_iso(last_mod, display_tz),
-            "etag": obj.get("ETag", "").strip('"'),
-        })
-    return {
-        "objects": objects_data,
-        "is_truncated": boto3_response.get("IsTruncated", False),
-        "next_continuation_token": boto3_response.get("NextContinuationToken"),
-        "total_count": boto3_response.get("KeyCount", len(objects_data)),
-        "versioning_enabled": versioning_enabled,
-        "url_templates": url_templates,
-    }
-
-
-def get_versioning_via_s3(client: Any, bucket_name: str) -> bool:
-    try:
-        resp = client.get_bucket_versioning(Bucket=bucket_name)
-        return resp.get("Status") == "Enabled"
-    except ClientError as exc:
-        code = exc.response.get("Error", {}).get("Code", "")
-        if code != "NoSuchBucket":
-            logger.warning("Failed to check versioning for %s: %s", bucket_name, code)
-        return False
-
-
-def stream_objects_ndjson(
-    client: Any,
-    bucket_name: str,
-    prefix: Optional[str],
-    url_templates: dict[str, str],
-    display_tz: str = "UTC",
-    versioning_enabled: bool = False,
-    delimiter: Optional[str] = None,
-) -> Generator[str, None, None]:
-    meta_line = json.dumps({
-        "type": "meta",
-        "versioning_enabled": versioning_enabled,
-        "url_templates": url_templates,
-    }) + "\n"
-    yield meta_line
-
-    yield json.dumps({"type": "count", "total_count": 0}) + "\n"
-
-    kwargs: dict[str, Any] = {"Bucket": bucket_name, "MaxKeys": 1000}
-    if prefix:
-        kwargs["Prefix"] = prefix
-    if delimiter:
-        kwargs["Delimiter"] = delimiter
-
-    running_count = 0
-    try:
-        paginator = client.get_paginator("list_objects_v2")
-        for page in paginator.paginate(**kwargs):
-            for cp in page.get("CommonPrefixes", []):
-                yield json.dumps({
-                    "type": "folder",
-                    "prefix": cp["Prefix"],
-                }) + "\n"
-            page_contents = page.get("Contents", [])
-            for obj in page_contents:
-                last_mod = obj["LastModified"]
-                yield json.dumps({
-                    "type": "object",
-                    "key": obj["Key"],
-                    "size": obj["Size"],
-                    "last_modified": last_mod.isoformat(),
-                    "last_modified_display": format_datetime_display(last_mod, display_tz),
-                    "last_modified_iso": format_datetime_iso(last_mod, display_tz),
-                    "etag": obj.get("ETag", "").strip('"'),
-                }) + "\n"
-            running_count += len(page_contents)
-            yield json.dumps({"type": "count", "total_count": running_count}) + "\n"
-    except ClientError as exc:
-        error_msg = exc.response.get("Error", {}).get("Message", "S3 operation failed")
-        yield json.dumps({"type": "error", "error": error_msg}) + "\n"
-        return
-    except (EndpointConnectionError, ConnectionClosedError):
-        yield json.dumps({"type": "error", "error": "S3 API server is unreachable"}) + "\n"
-        return
-
-    yield json.dumps({"type": "done"}) + "\n"

python/app/select_content.py

@@ -1,171 +0,0 @@
-"""S3 SelectObjectContent SQL query execution using DuckDB."""
-from __future__ import annotations
-
-import json
-from pathlib import Path
-from typing import Any, Dict, Generator, Optional
-
-try:
-    import duckdb
-    DUCKDB_AVAILABLE = True
-except ImportError:
-    DUCKDB_AVAILABLE = False
-
-
-class SelectError(Exception):
-    """Error during SELECT query execution."""
-    pass
-
-
-def execute_select_query(
-    file_path: Path,
-    expression: str,
-    input_format: str,
-    input_config: Dict[str, Any],
-    output_format: str,
-    output_config: Dict[str, Any],
-    chunk_size: int = 65536,
-) -> Generator[bytes, None, None]:
-    """Execute SQL query on object content."""
-    if not DUCKDB_AVAILABLE:
-        raise SelectError("DuckDB is not installed. Install with: pip install duckdb")
-
-    conn = duckdb.connect(":memory:")
-
-    try:
-        if input_format == "CSV":
-            _load_csv(conn, file_path, input_config)
-        elif input_format == "JSON":
-            _load_json(conn, file_path, input_config)
-        elif input_format == "Parquet":
-            _load_parquet(conn, file_path)
-        else:
-            raise SelectError(f"Unsupported input format: {input_format}")
-
-        normalized_expression = expression.replace("s3object", "data").replace("S3Object", "data")
-
-        try:
-            result = conn.execute(normalized_expression)
-        except duckdb.Error as exc:
-            raise SelectError(f"SQL execution error: {exc}")
-
-        if output_format == "CSV":
-            yield from _output_csv(result, output_config, chunk_size)
-        elif output_format == "JSON":
-            yield from _output_json(result, output_config, chunk_size)
-        else:
-            raise SelectError(f"Unsupported output format: {output_format}")
-
-    finally:
-        conn.close()
-
-
-def _load_csv(conn, file_path: Path, config: Dict[str, Any]) -> None:
-    """Load CSV file into DuckDB."""
-    file_header_info = config.get("file_header_info", "NONE")
-    delimiter = config.get("field_delimiter", ",")
-    quote = config.get("quote_character", '"')
-
-    header = file_header_info in ("USE", "IGNORE")
-    path_str = str(file_path).replace("\\", "/")
-
-    conn.execute(f"""
-        CREATE TABLE data AS
-        SELECT * FROM read_csv('{path_str}',
-            header={header},
-            delim='{delimiter}',
-            quote='{quote}'
-        )
-    """)
-
-
-def _load_json(conn, file_path: Path, config: Dict[str, Any]) -> None:
-    """Load JSON file into DuckDB."""
-    json_type = config.get("type", "DOCUMENT")
-    path_str = str(file_path).replace("\\", "/")
-
-    if json_type == "LINES":
-        conn.execute(f"""
-            CREATE TABLE data AS
-            SELECT * FROM read_json_auto('{path_str}', format='newline_delimited')
-        """)
-    else:
-        conn.execute(f"""
-            CREATE TABLE data AS
-            SELECT * FROM read_json_auto('{path_str}', format='array')
-        """)
-
-
-def _load_parquet(conn, file_path: Path) -> None:
-    """Load Parquet file into DuckDB."""
-    path_str = str(file_path).replace("\\", "/")
-    conn.execute(f"CREATE TABLE data AS SELECT * FROM read_parquet('{path_str}')")
-
-
-def _output_csv(
-    result,
-    config: Dict[str, Any],
-    chunk_size: int,
-) -> Generator[bytes, None, None]:
-    """Output query results as CSV."""
-    delimiter = config.get("field_delimiter", ",")
-    record_delimiter = config.get("record_delimiter", "\n")
-    quote = config.get("quote_character", '"')
-
-    buffer = ""
-
-    while True:
-        rows = result.fetchmany(1000)
-        if not rows:
-            break
-
-        for row in rows:
-            fields = []
-            for value in row:
-                if value is None:
-                    fields.append("")
-                elif isinstance(value, str):
-                    if delimiter in value or quote in value or record_delimiter in value:
-                        escaped = value.replace(quote, quote + quote)
-                        fields.append(f'{quote}{escaped}{quote}')
-                    else:
-                        fields.append(value)
-                else:
-                    fields.append(str(value))
-
-            buffer += delimiter.join(fields) + record_delimiter
-
-            while len(buffer) >= chunk_size:
-                yield buffer[:chunk_size].encode("utf-8")
-                buffer = buffer[chunk_size:]
-
-    if buffer:
-        yield buffer.encode("utf-8")
-
-
-def _output_json(
-    result,
-    config: Dict[str, Any],
-    chunk_size: int,
-) -> Generator[bytes, None, None]:
-    """Output query results as JSON Lines."""
-    record_delimiter = config.get("record_delimiter", "\n")
-    columns = [desc[0] for desc in result.description]
-
-    buffer = ""
-
-    while True:
-        rows = result.fetchmany(1000)
-        if not rows:
-            break
-
-        for row in rows:
-            record = dict(zip(columns, row))
-            buffer += json.dumps(record, default=str) + record_delimiter
-
-            while len(buffer) >= chunk_size:
-                yield buffer[:chunk_size].encode("utf-8")
-                buffer = buffer[chunk_size:]
-
-    if buffer:
-        yield buffer.encode("utf-8")

python/app/site_registry.py

@@ -1,177 +0,0 @@
-from __future__ import annotations
-
-import json
-import time
-from dataclasses import dataclass, field
-from pathlib import Path
-from typing import Any, Dict, List, Optional
-
-
-@dataclass
-class SiteInfo:
-    site_id: str
-    endpoint: str
-    region: str = "us-east-1"
-    priority: int = 100
-    display_name: str = ""
-    created_at: Optional[float] = None
-    updated_at: Optional[float] = None
-
-    def __post_init__(self) -> None:
-        if not self.display_name:
-            self.display_name = self.site_id
-        if self.created_at is None:
-            self.created_at = time.time()
-
-    def to_dict(self) -> Dict[str, Any]:
-        return {
-            "site_id": self.site_id,
-            "endpoint": self.endpoint,
-            "region": self.region,
-            "priority": self.priority,
-            "display_name": self.display_name,
-            "created_at": self.created_at,
-            "updated_at": self.updated_at,
-        }
-
-    @classmethod
-    def from_dict(cls, data: Dict[str, Any]) -> SiteInfo:
-        return cls(
-            site_id=data["site_id"],
-            endpoint=data.get("endpoint", ""),
-            region=data.get("region", "us-east-1"),
-            priority=data.get("priority", 100),
-            display_name=data.get("display_name", ""),
-            created_at=data.get("created_at"),
-            updated_at=data.get("updated_at"),
-        )
-
-
-@dataclass
-class PeerSite:
-    site_id: str
-    endpoint: str
-    region: str = "us-east-1"
-    priority: int = 100
-    display_name: str = ""
-    created_at: Optional[float] = None
-    updated_at: Optional[float] = None
-    connection_id: Optional[str] = None
-    is_healthy: Optional[bool] = None
-    last_health_check: Optional[float] = None
-
-    def __post_init__(self) -> None:
-        if not self.display_name:
-            self.display_name = self.site_id
-        if self.created_at is None:
-            self.created_at = time.time()
-
-    def to_dict(self) -> Dict[str, Any]:
-        return {
-            "site_id": self.site_id,
-            "endpoint": self.endpoint,
-            "region": self.region,
-            "priority": self.priority,
-            "display_name": self.display_name,
-            "created_at": self.created_at,
-            "updated_at": self.updated_at,
-            "connection_id": self.connection_id,
-            "is_healthy": self.is_healthy,
-            "last_health_check": self.last_health_check,
-        }
-
-    @classmethod
-    def from_dict(cls, data: Dict[str, Any]) -> PeerSite:
-        return cls(
-            site_id=data["site_id"],
-            endpoint=data.get("endpoint", ""),
-            region=data.get("region", "us-east-1"),
-            priority=data.get("priority", 100),
-            display_name=data.get("display_name", ""),
-            created_at=data.get("created_at"),
-            updated_at=data.get("updated_at"),
-            connection_id=data.get("connection_id"),
-            is_healthy=data.get("is_healthy"),
-            last_health_check=data.get("last_health_check"),
-        )
-
-
-class SiteRegistry:
-    def __init__(self, config_path: Path) -> None:
-        self.config_path = config_path
-        self._local_site: Optional[SiteInfo] = None
-        self._peers: Dict[str, PeerSite] = {}
-        self.reload()
-
-    def reload(self) -> None:
-        if not self.config_path.exists():
-            self._local_site = None
-            self._peers = {}
-            return
-
-        try:
-            with open(self.config_path, "r", encoding="utf-8") as f:
-                data = json.load(f)
-
-            if data.get("local"):
-                self._local_site = SiteInfo.from_dict(data["local"])
-            else:
-                self._local_site = None
-
-            self._peers = {}
-            for peer_data in data.get("peers", []):
-                peer = PeerSite.from_dict(peer_data)
-                self._peers[peer.site_id] = peer
-
-        except (OSError, json.JSONDecodeError, KeyError):
-            self._local_site = None
-            self._peers = {}
-
-    def save(self) -> None:
-        self.config_path.parent.mkdir(parents=True, exist_ok=True)
-        data = {
-            "local": self._local_site.to_dict() if self._local_site else None,
-            "peers": [peer.to_dict() for peer in self._peers.values()],
-        }
-        with open(self.config_path, "w", encoding="utf-8") as f:
-            json.dump(data, f, indent=2)
-
-    def get_local_site(self) -> Optional[SiteInfo]:
-        return self._local_site
-
-    def set_local_site(self, site: SiteInfo) -> None:
-        site.updated_at = time.time()
-        self._local_site = site
-        self.save()
-
-    def list_peers(self) -> List[PeerSite]:
-        return list(self._peers.values())
-
-    def get_peer(self, site_id: str) -> Optional[PeerSite]:
-        return self._peers.get(site_id)
-
-    def add_peer(self, peer: PeerSite) -> None:
-        peer.created_at = peer.created_at or time.time()
-        self._peers[peer.site_id] = peer
-        self.save()
-
-    def update_peer(self, peer: PeerSite) -> None:
-        if peer.site_id not in self._peers:
-            raise ValueError(f"Peer {peer.site_id} not found")
-        peer.updated_at = time.time()
-        self._peers[peer.site_id] = peer
-        self.save()
-
-    def delete_peer(self, site_id: str) -> bool:
-        if site_id in self._peers:
-            del self._peers[site_id]
-            self.save()
-            return True
-        return False
-
-    def update_health(self, site_id: str, is_healthy: bool) -> None:
-        peer = self._peers.get(site_id)
-        if peer:
-            peer.is_healthy = is_healthy
-            peer.last_health_check = time.time()
-            self.save()

python/app/site_sync.py

@@ -1,416 +0,0 @@
-from __future__ import annotations
-
-import json
-import logging
-import tempfile
-import threading
-import time
-from dataclasses import dataclass, field
-from datetime import datetime, timezone
-from pathlib import Path
-from typing import Any, Dict, List, Optional, TYPE_CHECKING
-
-import boto3
-from botocore.config import Config
-from botocore.exceptions import ClientError
-
-if TYPE_CHECKING:
-    from .connections import ConnectionStore, RemoteConnection
-    from .replication import ReplicationManager, ReplicationRule
-    from .storage import ObjectStorage
-
-logger = logging.getLogger(__name__)
-
-SITE_SYNC_USER_AGENT = "SiteSyncAgent/1.0"
-
-
-@dataclass
-class SyncedObjectInfo:
-    last_synced_at: float
-    remote_etag: str
-    source: str
-
-    def to_dict(self) -> Dict[str, Any]:
-        return {
-            "last_synced_at": self.last_synced_at,
-            "remote_etag": self.remote_etag,
-            "source": self.source,
-        }
-
-    @classmethod
-    def from_dict(cls, data: Dict[str, Any]) -> "SyncedObjectInfo":
-        return cls(
-            last_synced_at=data["last_synced_at"],
-            remote_etag=data["remote_etag"],
-            source=data["source"],
-        )
-
-
-@dataclass
-class SyncState:
-    synced_objects: Dict[str, SyncedObjectInfo] = field(default_factory=dict)
-    last_full_sync: Optional[float] = None
-
-    def to_dict(self) -> Dict[str, Any]:
-        return {
-            "synced_objects": {k: v.to_dict() for k, v in self.synced_objects.items()},
-            "last_full_sync": self.last_full_sync,
-        }
-
-    @classmethod
-    def from_dict(cls, data: Dict[str, Any]) -> "SyncState":
-        synced_objects = {}
-        for k, v in data.get("synced_objects", {}).items():
-            synced_objects[k] = SyncedObjectInfo.from_dict(v)
-        return cls(
-            synced_objects=synced_objects,
-            last_full_sync=data.get("last_full_sync"),
-        )
-
-
-@dataclass
-class SiteSyncStats:
-    last_sync_at: Optional[float] = None
-    objects_pulled: int = 0
-    objects_skipped: int = 0
-    conflicts_resolved: int = 0
-    deletions_applied: int = 0
-    errors: int = 0
-
-    def to_dict(self) -> Dict[str, Any]:
-        return {
-            "last_sync_at": self.last_sync_at,
-            "objects_pulled": self.objects_pulled,
-            "objects_skipped": self.objects_skipped,
-            "conflicts_resolved": self.conflicts_resolved,
-            "deletions_applied": self.deletions_applied,
-            "errors": self.errors,
-        }
-
-
-@dataclass
-class RemoteObjectMeta:
-    key: str
-    size: int
-    last_modified: datetime
-    etag: str
-
-    @classmethod
-    def from_s3_object(cls, obj: Dict[str, Any]) -> "RemoteObjectMeta":
-        return cls(
-            key=obj["Key"],
-            size=obj.get("Size", 0),
-            last_modified=obj["LastModified"],
-            etag=obj.get("ETag", "").strip('"'),
-        )
-
-
-def _create_sync_client(
-    connection: "RemoteConnection",
-    *,
-    connect_timeout: int = 10,
-    read_timeout: int = 120,
-    max_retries: int = 2,
-) -> Any:
-    config = Config(
-        user_agent_extra=SITE_SYNC_USER_AGENT,
-        connect_timeout=connect_timeout,
-        read_timeout=read_timeout,
-        retries={"max_attempts": max_retries},
-        signature_version="s3v4",
-        s3={"addressing_style": "path"},
-        request_checksum_calculation="when_required",
-        response_checksum_validation="when_required",
-    )
-    return boto3.client(
-        "s3",
-        endpoint_url=connection.endpoint_url,
-        aws_access_key_id=connection.access_key,
-        aws_secret_access_key=connection.secret_key,
-        region_name=connection.region or "us-east-1",
-        config=config,
-    )
-
-
-class SiteSyncWorker:
-    def __init__(
-        self,
-        storage: "ObjectStorage",
-        connections: "ConnectionStore",
-        replication_manager: "ReplicationManager",
-        storage_root: Path,
-        interval_seconds: int = 60,
-        batch_size: int = 100,
-        connect_timeout: int = 10,
-        read_timeout: int = 120,
-        max_retries: int = 2,
-        clock_skew_tolerance_seconds: float = 1.0,
-    ):
-        self.storage = storage
-        self.connections = connections
-        self.replication_manager = replication_manager
-        self.storage_root = storage_root
-        self.interval_seconds = interval_seconds
-        self.batch_size = batch_size
-        self.connect_timeout = connect_timeout
-        self.read_timeout = read_timeout
-        self.max_retries = max_retries
-        self.clock_skew_tolerance_seconds = clock_skew_tolerance_seconds
-        self._lock = threading.Lock()
-        self._shutdown = threading.Event()
-        self._sync_thread: Optional[threading.Thread] = None
-        self._bucket_stats: Dict[str, SiteSyncStats] = {}
-
-    def _create_client(self, connection: "RemoteConnection") -> Any:
-        """Create an S3 client with the worker's configured timeouts."""
-        return _create_sync_client(
-            connection,
-            connect_timeout=self.connect_timeout,
-            read_timeout=self.read_timeout,
-            max_retries=self.max_retries,
-        )
-
-    def start(self) -> None:
-        if self._sync_thread is not None and self._sync_thread.is_alive():
-            return
-        self._shutdown.clear()
-        self._sync_thread = threading.Thread(
-            target=self._sync_loop, name="site-sync-worker", daemon=True
-        )
-        self._sync_thread.start()
-        logger.info("Site sync worker started (interval=%ds)", self.interval_seconds)
-
-    def shutdown(self) -> None:
-        self._shutdown.set()
-        if self._sync_thread is not None:
-            self._sync_thread.join(timeout=10.0)
-        logger.info("Site sync worker shut down")
-
-    def trigger_sync(self, bucket_name: str) -> Optional[SiteSyncStats]:
-        from .replication import REPLICATION_MODE_BIDIRECTIONAL
-        rule = self.replication_manager.get_rule(bucket_name)
-        if not rule or rule.mode != REPLICATION_MODE_BIDIRECTIONAL or not rule.enabled:
-            return None
-        return self._sync_bucket(rule)
-
-    def get_stats(self, bucket_name: str) -> Optional[SiteSyncStats]:
-        with self._lock:
-            return self._bucket_stats.get(bucket_name)
-
-    def _sync_loop(self) -> None:
-        while not self._shutdown.is_set():
-            self._shutdown.wait(timeout=self.interval_seconds)
-            if self._shutdown.is_set():
-                break
-            self._run_sync_cycle()
-
-    def _run_sync_cycle(self) -> None:
-        from .replication import REPLICATION_MODE_BIDIRECTIONAL
-        for bucket_name, rule in list(self.replication_manager._rules.items()):
-            if self._shutdown.is_set():
-                break
-            if rule.mode != REPLICATION_MODE_BIDIRECTIONAL or not rule.enabled:
-                continue
-            try:
-                stats = self._sync_bucket(rule)
-                with self._lock:
-                    self._bucket_stats[bucket_name] = stats
-            except Exception as e:
-                logger.exception("Site sync failed for bucket %s: %s", bucket_name, e)
-
-    def _sync_bucket(self, rule: "ReplicationRule") -> SiteSyncStats:
-        stats = SiteSyncStats()
-        connection = self.connections.get(rule.target_connection_id)
-        if not connection:
-            logger.warning("Connection %s not found for bucket %s", rule.target_connection_id, rule.bucket_name)
-            stats.errors += 1
-            return stats
-
-        try:
-            local_objects = self._list_local_objects(rule.bucket_name)
-        except Exception as e:
-            logger.error("Failed to list local objects for %s: %s", rule.bucket_name, e)
-            stats.errors += 1
-            return stats
-
-        try:
-            remote_objects = self._list_remote_objects(rule, connection)
-        except Exception as e:
-            logger.error("Failed to list remote objects for %s: %s", rule.bucket_name, e)
-            stats.errors += 1
-            return stats
-
-        sync_state = self._load_sync_state(rule.bucket_name)
-        local_keys = set(local_objects.keys())
-        remote_keys = set(remote_objects.keys())
-
-        to_pull = []
-        for key in remote_keys:
-            remote_meta = remote_objects[key]
-            local_meta = local_objects.get(key)
-            if local_meta is None:
-                to_pull.append(key)
-            else:
-                resolution = self._resolve_conflict(local_meta, remote_meta)
-                if resolution == "pull":
-                    to_pull.append(key)
-                    stats.conflicts_resolved += 1
-                else:
-                    stats.objects_skipped += 1
-
-        pulled_count = 0
-        for key in to_pull:
-            if self._shutdown.is_set():
-                break
-            if pulled_count >= self.batch_size:
-                break
-            remote_meta = remote_objects[key]
-            success = self._pull_object(rule, key, connection, remote_meta)
-            if success:
-                stats.objects_pulled += 1
-                pulled_count += 1
-                sync_state.synced_objects[key] = SyncedObjectInfo(
-                    last_synced_at=time.time(),
-                    remote_etag=remote_meta.etag,
-                    source="remote",
-                )
-            else:
-                stats.errors += 1
-
-        if rule.sync_deletions:
-            for key in list(sync_state.synced_objects.keys()):
-                if key not in remote_keys and key in local_keys:
-                    tracked = sync_state.synced_objects[key]
-                    if tracked.source == "remote":
-                        local_meta = local_objects.get(key)
-                        if local_meta and local_meta.last_modified.timestamp() <= tracked.last_synced_at:
-                            success = self._apply_remote_deletion(rule.bucket_name, key)
-                            if success:
-                                stats.deletions_applied += 1
-                                del sync_state.synced_objects[key]
-
-        sync_state.last_full_sync = time.time()
-        self._save_sync_state(rule.bucket_name, sync_state)
-
-        with self.replication_manager._stats_lock:
-            rule.last_pull_at = time.time()
-            self.replication_manager.save_rules()
-
-        stats.last_sync_at = time.time()
-        logger.info(
-            "Site sync completed for %s: pulled=%d, skipped=%d, conflicts=%d, deletions=%d, errors=%d",
-            rule.bucket_name,
-            stats.objects_pulled,
-            stats.objects_skipped,
-            stats.conflicts_resolved,
-            stats.deletions_applied,
-            stats.errors,
-        )
-        return stats
-
-    def _list_local_objects(self, bucket_name: str) -> Dict[str, Any]:
-        from .storage import ObjectMeta
-        objects = self.storage.list_objects_all(bucket_name)
-        return {obj.key: obj for obj in objects}
-
-    def _list_remote_objects(self, rule: "ReplicationRule", connection: "RemoteConnection") -> Dict[str, RemoteObjectMeta]:
-        s3 = self._create_client(connection)
-        result: Dict[str, RemoteObjectMeta] = {}
-        paginator = s3.get_paginator("list_objects_v2")
-        try:
-            for page in paginator.paginate(Bucket=rule.target_bucket):
-                for obj in page.get("Contents", []):
-                    meta = RemoteObjectMeta.from_s3_object(obj)
-                    result[meta.key] = meta
-        except ClientError as e:
-            if e.response["Error"]["Code"] == "NoSuchBucket":
-                return {}
-            raise
-        return result
-
-    def _resolve_conflict(self, local_meta: Any, remote_meta: RemoteObjectMeta) -> str:
-        local_ts = local_meta.last_modified.timestamp()
-        remote_ts = remote_meta.last_modified.timestamp()
-
-        if abs(remote_ts - local_ts) < self.clock_skew_tolerance_seconds:
-            local_etag = local_meta.etag or ""
-            if remote_meta.etag == local_etag:
-                return "skip"
-            return "pull" if remote_meta.etag > local_etag else "keep"
-
-        return "pull" if remote_ts > local_ts else "keep"
-
-    def _pull_object(
-        self,
-        rule: "ReplicationRule",
-        object_key: str,
-        connection: "RemoteConnection",
-        remote_meta: RemoteObjectMeta,
-    ) -> bool:
-        s3 = self._create_client(connection)
-        tmp_path = None
-        try:
-            tmp_dir = self.storage_root / ".myfsio.sys" / "tmp"
-            tmp_dir.mkdir(parents=True, exist_ok=True)
-            with tempfile.NamedTemporaryFile(dir=tmp_dir, delete=False) as tmp_file:
-                tmp_path = Path(tmp_file.name)
-
-            s3.download_file(rule.target_bucket, object_key, str(tmp_path))
-
-            head_response = s3.head_object(Bucket=rule.target_bucket, Key=object_key)
-            user_metadata = head_response.get("Metadata", {})
-
-            with open(tmp_path, "rb") as f:
-                self.storage.put_object(
-                    rule.bucket_name,
-                    object_key,
-                    f,
-                    metadata=user_metadata if user_metadata else None,
-                )
-
-            logger.debug("Pulled object %s/%s from remote", rule.bucket_name, object_key)
-            return True
-
-        except ClientError as e:
-            logger.error("Failed to pull %s/%s: %s", rule.bucket_name, object_key, e)
-            return False
-        except Exception as e:
-            logger.error("Failed to store pulled object %s/%s: %s", rule.bucket_name, object_key, e)
-            return False
-        finally:
-            if tmp_path and tmp_path.exists():
-                try:
-                    tmp_path.unlink()
-                except OSError:
-                    pass
-
-    def _apply_remote_deletion(self, bucket_name: str, object_key: str) -> bool:
-        try:
-            self.storage.delete_object(bucket_name, object_key)
-            logger.debug("Applied remote deletion for %s/%s", bucket_name, object_key)
-            return True
-        except Exception as e:
-            logger.error("Failed to apply remote deletion for %s/%s: %s", bucket_name, object_key, e)
-            return False
-
-    def _sync_state_path(self, bucket_name: str) -> Path:
-        return self.storage_root / ".myfsio.sys" / "buckets" / bucket_name / "site_sync_state.json"
-
-    def _load_sync_state(self, bucket_name: str) -> SyncState:
-        path = self._sync_state_path(bucket_name)
-        if not path.exists():
-            return SyncState()
-        try:
-            data = json.loads(path.read_text(encoding="utf-8"))
-            return SyncState.from_dict(data)
-        except (json.JSONDecodeError, OSError, KeyError) as e:
-            logger.warning("Failed to load sync state for %s: %s", bucket_name, e)
-            return SyncState()
-
-    def _save_sync_state(self, bucket_name: str, state: SyncState) -> None:
-        path = self._sync_state_path(bucket_name)
-        path.parent.mkdir(parents=True, exist_ok=True)
-        try:
-            path.write_text(json.dumps(state.to_dict(), indent=2), encoding="utf-8")
-        except OSError as e:
-            logger.warning("Failed to save sync state for %s: %s", bucket_name, e)

python/app/system_metrics.py

@@ -1,215 +0,0 @@
-from __future__ import annotations
-
-import json
-import logging
-import threading
-import time
-from dataclasses import dataclass
-from datetime import datetime, timezone
-from pathlib import Path
-from typing import Any, Dict, List, Optional, TYPE_CHECKING
-
-import psutil
-
-if TYPE_CHECKING:
-    from .storage import ObjectStorage
-
-logger = logging.getLogger(__name__)
-
-
-@dataclass
-class SystemMetricsSnapshot:
-    timestamp: datetime
-    cpu_percent: float
-    memory_percent: float
-    disk_percent: float
-    storage_bytes: int
-
-    def to_dict(self) -> Dict[str, Any]:
-        return {
-            "timestamp": self.timestamp.strftime("%Y-%m-%dT%H:%M:%SZ"),
-            "cpu_percent": round(self.cpu_percent, 2),
-            "memory_percent": round(self.memory_percent, 2),
-            "disk_percent": round(self.disk_percent, 2),
-            "storage_bytes": self.storage_bytes,
-        }
-
-    @classmethod
-    def from_dict(cls, data: Dict[str, Any]) -> "SystemMetricsSnapshot":
-        timestamp_str = data["timestamp"]
-        if timestamp_str.endswith("Z"):
-            timestamp_str = timestamp_str[:-1] + "+00:00"
-        return cls(
-            timestamp=datetime.fromisoformat(timestamp_str),
-            cpu_percent=data.get("cpu_percent", 0.0),
-            memory_percent=data.get("memory_percent", 0.0),
-            disk_percent=data.get("disk_percent", 0.0),
-            storage_bytes=data.get("storage_bytes", 0),
-        )
-
-
-class SystemMetricsCollector:
-    def __init__(
-        self,
-        storage_root: Path,
-        interval_minutes: int = 5,
-        retention_hours: int = 24,
-    ):
-        self.storage_root = storage_root
-        self.interval_seconds = interval_minutes * 60
-        self.retention_hours = retention_hours
-        self._lock = threading.Lock()
-        self._shutdown = threading.Event()
-        self._snapshots: List[SystemMetricsSnapshot] = []
-        self._storage_ref: Optional["ObjectStorage"] = None
-
-        self._load_history()
-
-        self._snapshot_thread = threading.Thread(
-            target=self._snapshot_loop,
-            name="system-metrics-snapshot",
-            daemon=True,
-        )
-        self._snapshot_thread.start()
-
-    def set_storage(self, storage: "ObjectStorage") -> None:
-        with self._lock:
-            self._storage_ref = storage
-
-    def _config_path(self) -> Path:
-        return self.storage_root / ".myfsio.sys" / "config" / "metrics_history.json"
-
-    def _load_history(self) -> None:
-        config_path = self._config_path()
-        if not config_path.exists():
-            return
-        try:
-            data = json.loads(config_path.read_text(encoding="utf-8"))
-            history_data = data.get("history", [])
-            self._snapshots = [SystemMetricsSnapshot.from_dict(s) for s in history_data]
-            self._prune_old_snapshots()
-        except (json.JSONDecodeError, OSError, KeyError) as e:
-            logger.warning(f"Failed to load system metrics history: {e}")
-
-    def _save_history(self) -> None:
-        config_path = self._config_path()
-        config_path.parent.mkdir(parents=True, exist_ok=True)
-        try:
-            data = {"history": [s.to_dict() for s in self._snapshots]}
-            config_path.write_text(json.dumps(data, indent=2), encoding="utf-8")
-        except OSError as e:
-            logger.warning(f"Failed to save system metrics history: {e}")
-
-    def _prune_old_snapshots(self) -> None:
-        if not self._snapshots:
-            return
-        cutoff = datetime.now(timezone.utc).timestamp() - (self.retention_hours * 3600)
-        self._snapshots = [
-            s for s in self._snapshots if s.timestamp.timestamp() > cutoff
-        ]
-
-    def _snapshot_loop(self) -> None:
-        while not self._shutdown.is_set():
-            self._shutdown.wait(timeout=self.interval_seconds)
-            if not self._shutdown.is_set():
-                self._take_snapshot()
-
-    def _take_snapshot(self) -> None:
-        try:
-            cpu_percent = psutil.cpu_percent(interval=0.1)
-            memory = psutil.virtual_memory()
-            disk = psutil.disk_usage(str(self.storage_root))
-
-            storage_bytes = 0
-            with self._lock:
-                storage = self._storage_ref
-            if storage:
-                try:
-                    buckets = storage.list_buckets()
-                    for bucket in buckets:
-                        stats = storage.bucket_stats(bucket.name, cache_ttl=60)
-                        storage_bytes += stats.get("total_bytes", stats.get("bytes", 0))
-                except Exception as e:
-                    logger.warning(f"Failed to collect bucket stats: {e}")
-
-            snapshot = SystemMetricsSnapshot(
-                timestamp=datetime.now(timezone.utc),
-                cpu_percent=cpu_percent,
-                memory_percent=memory.percent,
-                disk_percent=disk.percent,
-                storage_bytes=storage_bytes,
-            )
-
-            with self._lock:
-                self._snapshots.append(snapshot)
-                self._prune_old_snapshots()
-                self._save_history()
-
-            logger.debug(f"System metrics snapshot taken: CPU={cpu_percent:.1f}%, Memory={memory.percent:.1f}%")
-        except Exception as e:
-            logger.warning(f"Failed to take system metrics snapshot: {e}")
-
-    def get_current(self) -> Dict[str, Any]:
-        cpu_percent = psutil.cpu_percent(interval=0.1)
-        memory = psutil.virtual_memory()
-        disk = psutil.disk_usage(str(self.storage_root))
-        boot_time = psutil.boot_time()
-        uptime_seconds = time.time() - boot_time
-        uptime_days = int(uptime_seconds / 86400)
-
-        total_buckets = 0
-        total_objects = 0
-        total_bytes_used = 0
-        total_versions = 0
-
-        with self._lock:
-            storage = self._storage_ref
-        if storage:
-            try:
-                buckets = storage.list_buckets()
-                total_buckets = len(buckets)
-                for bucket in buckets:
-                    stats = storage.bucket_stats(bucket.name, cache_ttl=60)
-                    total_objects += stats.get("total_objects", stats.get("objects", 0))
-                    total_bytes_used += stats.get("total_bytes", stats.get("bytes", 0))
-                    total_versions += stats.get("version_count", 0)
-            except Exception as e:
-                logger.warning(f"Failed to collect current bucket stats: {e}")
-
-        return {
-            "cpu_percent": round(cpu_percent, 2),
-            "memory": {
-                "total": memory.total,
-                "available": memory.available,
-                "used": memory.used,
-                "percent": round(memory.percent, 2),
-            },
-            "disk": {
-                "total": disk.total,
-                "free": disk.free,
-                "used": disk.used,
-                "percent": round(disk.percent, 2),
-            },
-            "app": {
-                "buckets": total_buckets,
-                "objects": total_objects,
-                "versions": total_versions,
-                "storage_bytes": total_bytes_used,
-                "uptime_days": uptime_days,
-            },
-        }
-
-    def get_history(self, hours: Optional[int] = None) -> List[Dict[str, Any]]:
-        with self._lock:
-            snapshots = list(self._snapshots)
-
-        if hours:
-            cutoff = datetime.now(timezone.utc).timestamp() - (hours * 3600)
-            snapshots = [s for s in snapshots if s.timestamp.timestamp() > cutoff]
-
-        return [s.to_dict() for s in snapshots]
-
-    def shutdown(self) -> None:
-        self._shutdown.set()
-        self._take_snapshot()
-        self._snapshot_thread.join(timeout=5.0)

python/app/website_domains.py

@@ -1,108 +0,0 @@
-from __future__ import annotations
-
-import json
-import re
-import threading
-from pathlib import Path
-from typing import Dict, List, Optional
-
-_DOMAIN_RE = re.compile(
-    r"^(?!-)[a-z0-9]([a-z0-9-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9-]*[a-z0-9])?)*$"
-)
-
-
-def normalize_domain(raw: str) -> str:
-    raw = raw.strip().lower()
-    for prefix in ("https://", "http://"):
-        if raw.startswith(prefix):
-            raw = raw[len(prefix):]
-    raw = raw.split("/", 1)[0]
-    raw = raw.split("?", 1)[0]
-    raw = raw.split("#", 1)[0]
-    if ":" in raw:
-        raw = raw.rsplit(":", 1)[0]
-    return raw
-
-
-def is_valid_domain(domain: str) -> bool:
-    if not domain or len(domain) > 253:
-        return False
-    return bool(_DOMAIN_RE.match(domain))
-
-
-class WebsiteDomainStore:
-    def __init__(self, config_path: Path) -> None:
-        self.config_path = config_path
-        self._lock = threading.Lock()
-        self._domains: Dict[str, str] = {}
-        self._last_mtime: float = 0.0
-        self.reload()
-
-    def reload(self) -> None:
-        if not self.config_path.exists():
-            self._domains = {}
-            self._last_mtime = 0.0
-            return
-        try:
-            self._last_mtime = self.config_path.stat().st_mtime
-            with open(self.config_path, "r", encoding="utf-8") as f:
-                data = json.load(f)
-                if isinstance(data, dict):
-                    self._domains = {k.lower(): v for k, v in data.items()}
-                else:
-                    self._domains = {}
-        except (OSError, json.JSONDecodeError):
-            self._domains = {}
-
-    def _maybe_reload(self) -> None:
-        try:
-            if self.config_path.exists():
-                mtime = self.config_path.stat().st_mtime
-                if mtime != self._last_mtime:
-                    self._last_mtime = mtime
-                    with open(self.config_path, "r", encoding="utf-8") as f:
-                        data = json.load(f)
-                    if isinstance(data, dict):
-                        self._domains = {k.lower(): v for k, v in data.items()}
-                    else:
-                        self._domains = {}
-            elif self._domains:
-                self._domains = {}
-                self._last_mtime = 0.0
-        except (OSError, json.JSONDecodeError):
-            pass
-
-    def _save(self) -> None:
-        self.config_path.parent.mkdir(parents=True, exist_ok=True)
-        with open(self.config_path, "w", encoding="utf-8") as f:
-            json.dump(self._domains, f, indent=2)
-        self._last_mtime = self.config_path.stat().st_mtime
-
-    def list_all(self) -> List[Dict[str, str]]:
-        with self._lock:
-            self._maybe_reload()
-            return [{"domain": d, "bucket": b} for d, b in self._domains.items()]
-
-    def get_bucket(self, domain: str) -> Optional[str]:
-        with self._lock:
-            self._maybe_reload()
-            return self._domains.get(domain.lower())
-
-    def get_domains_for_bucket(self, bucket: str) -> List[str]:
-        with self._lock:
-            self._maybe_reload()
-            return [d for d, b in self._domains.items() if b == bucket]
-
-    def set_mapping(self, domain: str, bucket: str) -> None:
-        with self._lock:
-            self._domains[domain.lower()] = bucket
-            self._save()
-
-    def delete_mapping(self, domain: str) -> bool:
-        with self._lock:
-            key = domain.lower()
-            if key not in self._domains:
-                return False
-            del self._domains[key]
-            self._save()
-            return True

python/docker-entrypoint.sh

@@ -1,4 +0,0 @@
-#!/bin/sh
-set -e
-
-exec python run.py --prod

python/myfsio_core/Cargo.toml

@@ -1,24 +0,0 @@
-[package]
-name = "myfsio_core"
-version = "0.1.0"
-edition = "2021"
-
-[lib]
-name = "myfsio_core"
-crate-type = ["cdylib"]
-
-[dependencies]
-pyo3 = { version = "0.28", features = ["extension-module"] }
-hmac = "0.12"
-sha2 = "0.10"
-md-5 = "0.10"
-hex = "0.4"
-unicode-normalization = "0.1"
-serde_json = "1"
-regex = "1"
-lru = "0.14"
-parking_lot = "0.12"
-percent-encoding = "2"
-aes-gcm = "0.10"
-hkdf = "0.12"
-uuid = { version = "1", features = ["v4"] }

python/myfsio_core/pyproject.toml

@@ -1,11 +0,0 @@
-[build-system]
-requires = ["maturin>=1.0,<2.0"]
-build-backend = "maturin"
-
-[project]
-name = "myfsio_core"
-version = "0.1.0"
-requires-python = ">=3.10"
-
-[tool.maturin]
-features = ["pyo3/extension-module"]

python/myfsio_core/src/crypto.rs

@@ -1,192 +0,0 @@
-use aes_gcm::aead::Aead;
-use aes_gcm::{Aes256Gcm, KeyInit, Nonce};
-use hkdf::Hkdf;
-use pyo3::exceptions::{PyIOError, PyValueError};
-use pyo3::prelude::*;
-use sha2::Sha256;
-use std::fs::File;
-use std::io::{Read, Seek, SeekFrom, Write};
-
-const DEFAULT_CHUNK_SIZE: usize = 65536;
-const HEADER_SIZE: usize = 4;
-
-fn read_exact_chunk(reader: &mut impl Read, buf: &mut [u8]) -> std::io::Result<usize> {
-    let mut filled = 0;
-    while filled < buf.len() {
-        match reader.read(&mut buf[filled..]) {
-            Ok(0) => break,
-            Ok(n) => filled += n,
-            Err(ref e) if e.kind() == std::io::ErrorKind::Interrupted => continue,
-            Err(e) => return Err(e),
-        }
-    }
-    Ok(filled)
-}
-
-fn derive_chunk_nonce(base_nonce: &[u8], chunk_index: u32) -> Result<[u8; 12], String> {
-    let hkdf = Hkdf::<Sha256>::new(Some(base_nonce), b"chunk_nonce");
-    let mut okm = [0u8; 12];
-    hkdf.expand(&chunk_index.to_be_bytes(), &mut okm)
-        .map_err(|e| format!("HKDF expand failed: {}", e))?;
-    Ok(okm)
-}
-
-#[pyfunction]
-#[pyo3(signature = (input_path, output_path, key, base_nonce, chunk_size=DEFAULT_CHUNK_SIZE))]
-pub fn encrypt_stream_chunked(
-    py: Python<'_>,
-    input_path: &str,
-    output_path: &str,
-    key: &[u8],
-    base_nonce: &[u8],
-    chunk_size: usize,
-) -> PyResult<u32> {
-    if key.len() != 32 {
-        return Err(PyValueError::new_err(format!(
-            "Key must be 32 bytes, got {}",
-            key.len()
-        )));
-    }
-    if base_nonce.len() != 12 {
-        return Err(PyValueError::new_err(format!(
-            "Base nonce must be 12 bytes, got {}",
-            base_nonce.len()
-        )));
-    }
-
-    let chunk_size = if chunk_size == 0 {
-        DEFAULT_CHUNK_SIZE
-    } else {
-        chunk_size
-    };
-
-    let inp = input_path.to_owned();
-    let out = output_path.to_owned();
-    let key_arr: [u8; 32] = key.try_into().unwrap();
-    let nonce_arr: [u8; 12] = base_nonce.try_into().unwrap();
-
-    py.detach(move || {
-        let cipher = Aes256Gcm::new(&key_arr.into());
-
-        let mut infile = File::open(&inp)
-            .map_err(|e| PyIOError::new_err(format!("Failed to open input: {}", e)))?;
-        let mut outfile = File::create(&out)
-            .map_err(|e| PyIOError::new_err(format!("Failed to create output: {}", e)))?;
-
-        outfile
-            .write_all(&[0u8; 4])
-            .map_err(|e| PyIOError::new_err(format!("Failed to write header: {}", e)))?;
-
-        let mut buf = vec![0u8; chunk_size];
-        let mut chunk_index: u32 = 0;
-
-        loop {
-            let n = read_exact_chunk(&mut infile, &mut buf)
-                .map_err(|e| PyIOError::new_err(format!("Failed to read: {}", e)))?;
-            if n == 0 {
-                break;
-            }
-
-            let nonce_bytes = derive_chunk_nonce(&nonce_arr, chunk_index)
-                .map_err(|e| PyValueError::new_err(e))?;
-            let nonce = Nonce::from_slice(&nonce_bytes);
-
-            let encrypted = cipher
-                .encrypt(nonce, &buf[..n])
-                .map_err(|e| PyValueError::new_err(format!("Encrypt failed: {}", e)))?;
-
-            let size = encrypted.len() as u32;
-            outfile
-                .write_all(&size.to_be_bytes())
-                .map_err(|e| PyIOError::new_err(format!("Failed to write chunk size: {}", e)))?;
-            outfile
-                .write_all(&encrypted)
-                .map_err(|e| PyIOError::new_err(format!("Failed to write chunk: {}", e)))?;
-
-            chunk_index += 1;
-        }
-
-        outfile
-            .seek(SeekFrom::Start(0))
-            .map_err(|e| PyIOError::new_err(format!("Failed to seek: {}", e)))?;
-        outfile
-            .write_all(&chunk_index.to_be_bytes())
-            .map_err(|e| PyIOError::new_err(format!("Failed to write chunk count: {}", e)))?;
-
-        Ok(chunk_index)
-    })
-}
-
-#[pyfunction]
-pub fn decrypt_stream_chunked(
-    py: Python<'_>,
-    input_path: &str,
-    output_path: &str,
-    key: &[u8],
-    base_nonce: &[u8],
-) -> PyResult<u32> {
-    if key.len() != 32 {
-        return Err(PyValueError::new_err(format!(
-            "Key must be 32 bytes, got {}",
-            key.len()
-        )));
-    }
-    if base_nonce.len() != 12 {
-        return Err(PyValueError::new_err(format!(
-            "Base nonce must be 12 bytes, got {}",
-            base_nonce.len()
-        )));
-    }
-
-    let inp = input_path.to_owned();
-    let out = output_path.to_owned();
-    let key_arr: [u8; 32] = key.try_into().unwrap();
-    let nonce_arr: [u8; 12] = base_nonce.try_into().unwrap();
-
-    py.detach(move || {
-        let cipher = Aes256Gcm::new(&key_arr.into());
-
-        let mut infile = File::open(&inp)
-            .map_err(|e| PyIOError::new_err(format!("Failed to open input: {}", e)))?;
-        let mut outfile = File::create(&out)
-            .map_err(|e| PyIOError::new_err(format!("Failed to create output: {}", e)))?;
-
-        let mut header = [0u8; HEADER_SIZE];
-        infile
-            .read_exact(&mut header)
-            .map_err(|e| PyIOError::new_err(format!("Failed to read header: {}", e)))?;
-        let chunk_count = u32::from_be_bytes(header);
-
-        let mut size_buf = [0u8; HEADER_SIZE];
-        for chunk_index in 0..chunk_count {
-            infile
-                .read_exact(&mut size_buf)
-                .map_err(|e| {
-                    PyIOError::new_err(format!(
-                        "Failed to read chunk {} size: {}",
-                        chunk_index, e
-                    ))
-                })?;
-            let chunk_size = u32::from_be_bytes(size_buf) as usize;
-
-            let mut encrypted = vec![0u8; chunk_size];
-            infile.read_exact(&mut encrypted).map_err(|e| {
-                PyIOError::new_err(format!("Failed to read chunk {}: {}", chunk_index, e))
-            })?;
-
-            let nonce_bytes = derive_chunk_nonce(&nonce_arr, chunk_index)
-                .map_err(|e| PyValueError::new_err(e))?;
-            let nonce = Nonce::from_slice(&nonce_bytes);
-
-            let decrypted = cipher.decrypt(nonce, encrypted.as_ref()).map_err(|e| {
-                PyValueError::new_err(format!("Decrypt chunk {} failed: {}", chunk_index, e))
-            })?;
-
-            outfile.write_all(&decrypted).map_err(|e| {
-                PyIOError::new_err(format!("Failed to write chunk {}: {}", chunk_index, e))
-            })?;
-        }
-
-        Ok(chunk_count)
-    })
-}

python/myfsio_core/src/hashing.rs

@@ -1,90 +0,0 @@
-use md5::{Digest, Md5};
-use pyo3::exceptions::PyIOError;
-use pyo3::prelude::*;
-use sha2::Sha256;
-use std::fs::File;
-use std::io::Read;
-
-const CHUNK_SIZE: usize = 65536;
-
-#[pyfunction]
-pub fn md5_file(py: Python<'_>, path: &str) -> PyResult<String> {
-    let path = path.to_owned();
-    py.detach(move || {
-        let mut file = File::open(&path)
-            .map_err(|e| PyIOError::new_err(format!("Failed to open file: {}", e)))?;
-        let mut hasher = Md5::new();
-        let mut buf = vec![0u8; CHUNK_SIZE];
-        loop {
-            let n = file
-                .read(&mut buf)
-                .map_err(|e| PyIOError::new_err(format!("Failed to read file: {}", e)))?;
-            if n == 0 {
-                break;
-            }
-            hasher.update(&buf[..n]);
-        }
-        Ok(format!("{:x}", hasher.finalize()))
-    })
-}
-
-#[pyfunction]
-pub fn md5_bytes(data: &[u8]) -> String {
-    let mut hasher = Md5::new();
-    hasher.update(data);
-    format!("{:x}", hasher.finalize())
-}
-
-#[pyfunction]
-pub fn sha256_file(py: Python<'_>, path: &str) -> PyResult<String> {
-    let path = path.to_owned();
-    py.detach(move || {
-        let mut file = File::open(&path)
-            .map_err(|e| PyIOError::new_err(format!("Failed to open file: {}", e)))?;
-        let mut hasher = Sha256::new();
-        let mut buf = vec![0u8; CHUNK_SIZE];
-        loop {
-            let n = file
-                .read(&mut buf)
-                .map_err(|e| PyIOError::new_err(format!("Failed to read file: {}", e)))?;
-            if n == 0 {
-                break;
-            }
-            hasher.update(&buf[..n]);
-        }
-        Ok(format!("{:x}", hasher.finalize()))
-    })
-}
-
-#[pyfunction]
-pub fn sha256_bytes(data: &[u8]) -> String {
-    let mut hasher = Sha256::new();
-    hasher.update(data);
-    format!("{:x}", hasher.finalize())
-}
-
-#[pyfunction]
-pub fn md5_sha256_file(py: Python<'_>, path: &str) -> PyResult<(String, String)> {
-    let path = path.to_owned();
-    py.detach(move || {
-        let mut file = File::open(&path)
-            .map_err(|e| PyIOError::new_err(format!("Failed to open file: {}", e)))?;
-        let mut md5_hasher = Md5::new();
-        let mut sha_hasher = Sha256::new();
-        let mut buf = vec![0u8; CHUNK_SIZE];
-        loop {
-            let n = file
-                .read(&mut buf)
-                .map_err(|e| PyIOError::new_err(format!("Failed to read file: {}", e)))?;
-            if n == 0 {
-                break;
-            }
-            md5_hasher.update(&buf[..n]);
-            sha_hasher.update(&buf[..n]);
-        }
-        Ok((
-            format!("{:x}", md5_hasher.finalize()),
-            format!("{:x}", sha_hasher.finalize()),
-        ))
-    })
-}

python/myfsio_core/src/lib.rs

@@ -1,51 +0,0 @@
-mod crypto;
-mod hashing;
-mod metadata;
-mod sigv4;
-mod storage;
-mod streaming;
-mod validation;
-
-use pyo3::prelude::*;
-
-#[pymodule]
-mod myfsio_core {
-    use super::*;
-
-    #[pymodule_init]
-    fn init(m: &Bound<'_, PyModule>) -> PyResult<()> {
-        m.add_function(wrap_pyfunction!(sigv4::verify_sigv4_signature, m)?)?;
-        m.add_function(wrap_pyfunction!(sigv4::derive_signing_key, m)?)?;
-        m.add_function(wrap_pyfunction!(sigv4::compute_signature, m)?)?;
-        m.add_function(wrap_pyfunction!(sigv4::build_string_to_sign, m)?)?;
-        m.add_function(wrap_pyfunction!(sigv4::constant_time_compare, m)?)?;
-        m.add_function(wrap_pyfunction!(sigv4::clear_signing_key_cache, m)?)?;
-
-        m.add_function(wrap_pyfunction!(hashing::md5_file, m)?)?;
-        m.add_function(wrap_pyfunction!(hashing::md5_bytes, m)?)?;
-        m.add_function(wrap_pyfunction!(hashing::sha256_file, m)?)?;
-        m.add_function(wrap_pyfunction!(hashing::sha256_bytes, m)?)?;
-        m.add_function(wrap_pyfunction!(hashing::md5_sha256_file, m)?)?;
-
-        m.add_function(wrap_pyfunction!(validation::validate_object_key, m)?)?;
-        m.add_function(wrap_pyfunction!(validation::validate_bucket_name, m)?)?;
-
-        m.add_function(wrap_pyfunction!(metadata::read_index_entry, m)?)?;
-
-        m.add_function(wrap_pyfunction!(storage::write_index_entry, m)?)?;
-        m.add_function(wrap_pyfunction!(storage::delete_index_entry, m)?)?;
-        m.add_function(wrap_pyfunction!(storage::check_bucket_contents, m)?)?;
-        m.add_function(wrap_pyfunction!(storage::shallow_scan, m)?)?;
-        m.add_function(wrap_pyfunction!(storage::bucket_stats_scan, m)?)?;
-        m.add_function(wrap_pyfunction!(storage::search_objects_scan, m)?)?;
-        m.add_function(wrap_pyfunction!(storage::build_object_cache, m)?)?;
-
-        m.add_function(wrap_pyfunction!(streaming::stream_to_file_with_md5, m)?)?;
-        m.add_function(wrap_pyfunction!(streaming::assemble_parts_with_md5, m)?)?;
-
-        m.add_function(wrap_pyfunction!(crypto::encrypt_stream_chunked, m)?)?;
-        m.add_function(wrap_pyfunction!(crypto::decrypt_stream_chunked, m)?)?;
-
-        Ok(())
-    }
-}

python/myfsio_core/src/metadata.rs

@@ -1,71 +0,0 @@
-use pyo3::exceptions::PyValueError;
-use pyo3::prelude::*;
-use pyo3::types::{PyDict, PyList, PyString};
-use serde_json::Value;
-use std::fs;
-
-const MAX_DEPTH: u32 = 64;
-
-fn value_to_py(py: Python<'_>, v: &Value, depth: u32) -> PyResult<Py<PyAny>> {
-    if depth > MAX_DEPTH {
-        return Err(PyValueError::new_err("JSON nesting too deep"));
-    }
-    match v {
-        Value::Null => Ok(py.None()),
-        Value::Bool(b) => Ok((*b).into_pyobject(py)?.to_owned().into_any().unbind()),
-        Value::Number(n) => {
-            if let Some(i) = n.as_i64() {
-                Ok(i.into_pyobject(py)?.into_any().unbind())
-            } else if let Some(f) = n.as_f64() {
-                Ok(f.into_pyobject(py)?.into_any().unbind())
-            } else {
-                Ok(py.None())
-            }
-        }
-        Value::String(s) => Ok(PyString::new(py, s).into_any().unbind()),
-        Value::Array(arr) => {
-            let list = PyList::empty(py);
-            for item in arr {
-                list.append(value_to_py(py, item, depth + 1)?)?;
-            }
-            Ok(list.into_any().unbind())
-        }
-        Value::Object(map) => {
-            let dict = PyDict::new(py);
-            for (k, val) in map {
-                dict.set_item(k, value_to_py(py, val, depth + 1)?)?;
-            }
-            Ok(dict.into_any().unbind())
-        }
-    }
-}
-
-#[pyfunction]
-pub fn read_index_entry(
-    py: Python<'_>,
-    path: &str,
-    entry_name: &str,
-) -> PyResult<Option<Py<PyAny>>> {
-    let path_owned = path.to_owned();
-    let entry_owned = entry_name.to_owned();
-
-    let entry: Option<Value> = py.detach(move || -> PyResult<Option<Value>> {
-        let content = match fs::read_to_string(&path_owned) {
-            Ok(c) => c,
-            Err(_) => return Ok(None),
-        };
-        let parsed: Value = match serde_json::from_str(&content) {
-            Ok(v) => v,
-            Err(_) => return Ok(None),
-        };
-        match parsed {
-            Value::Object(mut map) => Ok(map.remove(&entry_owned)),
-            _ => Ok(None),
-        }
-    })?;
-
-    match entry {
-        Some(val) => Ok(Some(value_to_py(py, &val, 0)?)),
-        None => Ok(None),
-    }
-}

python/myfsio_core/src/sigv4.rs

@@ -1,193 +0,0 @@
-use hmac::{Hmac, Mac};
-use lru::LruCache;
-use parking_lot::Mutex;
-use percent_encoding::{percent_encode, AsciiSet, NON_ALPHANUMERIC};
-use pyo3::prelude::*;
-use sha2::{Digest, Sha256};
-use std::num::NonZeroUsize;
-use std::sync::LazyLock;
-use std::time::Instant;
-
-type HmacSha256 = Hmac<Sha256>;
-
-struct CacheEntry {
-    key: Vec<u8>,
-    created: Instant,
-}
-
-static SIGNING_KEY_CACHE: LazyLock<Mutex<LruCache<(String, String, String, String), CacheEntry>>> =
-    LazyLock::new(|| Mutex::new(LruCache::new(NonZeroUsize::new(256).unwrap())));
-
-const CACHE_TTL_SECS: u64 = 60;
-
-const AWS_ENCODE_SET: &AsciiSet = &NON_ALPHANUMERIC
-    .remove(b'-')
-    .remove(b'_')
-    .remove(b'.')
-    .remove(b'~');
-
-fn hmac_sha256(key: &[u8], msg: &[u8]) -> Vec<u8> {
-    let mut mac = HmacSha256::new_from_slice(key).expect("HMAC key length is always valid");
-    mac.update(msg);
-    mac.finalize().into_bytes().to_vec()
-}
-
-fn sha256_hex(data: &[u8]) -> String {
-    let mut hasher = Sha256::new();
-    hasher.update(data);
-    hex::encode(hasher.finalize())
-}
-
-fn aws_uri_encode(input: &str) -> String {
-    percent_encode(input.as_bytes(), AWS_ENCODE_SET).to_string()
-}
-
-fn derive_signing_key_cached(
-    secret_key: &str,
-    date_stamp: &str,
-    region: &str,
-    service: &str,
-) -> Vec<u8> {
-    let cache_key = (
-        secret_key.to_owned(),
-        date_stamp.to_owned(),
-        region.to_owned(),
-        service.to_owned(),
-    );
-
-    {
-        let mut cache = SIGNING_KEY_CACHE.lock();
-        if let Some(entry) = cache.get(&cache_key) {
-            if entry.created.elapsed().as_secs() < CACHE_TTL_SECS {
-                return entry.key.clone();
-            }
-            cache.pop(&cache_key);
-        }
-    }
-
-    let k_date = hmac_sha256(format!("AWS4{}", secret_key).as_bytes(), date_stamp.as_bytes());
-    let k_region = hmac_sha256(&k_date, region.as_bytes());
-    let k_service = hmac_sha256(&k_region, service.as_bytes());
-    let k_signing = hmac_sha256(&k_service, b"aws4_request");
-
-    {
-        let mut cache = SIGNING_KEY_CACHE.lock();
-        cache.put(
-            cache_key,
-            CacheEntry {
-                key: k_signing.clone(),
-                created: Instant::now(),
-            },
-        );
-    }
-
-    k_signing
-}
-
-fn constant_time_compare_inner(a: &[u8], b: &[u8]) -> bool {
-    if a.len() != b.len() {
-        return false;
-    }
-    let mut result: u8 = 0;
-    for (x, y) in a.iter().zip(b.iter()) {
-        result |= x ^ y;
-    }
-    result == 0
-}
-
-#[pyfunction]
-pub fn verify_sigv4_signature(
-    method: &str,
-    canonical_uri: &str,
-    query_params: Vec<(String, String)>,
-    signed_headers_str: &str,
-    header_values: Vec<(String, String)>,
-    payload_hash: &str,
-    amz_date: &str,
-    date_stamp: &str,
-    region: &str,
-    service: &str,
-    secret_key: &str,
-    provided_signature: &str,
-) -> bool {
-    let mut sorted_params = query_params;
-    sorted_params.sort_by(|a, b| a.0.cmp(&b.0).then_with(|| a.1.cmp(&b.1)));
-
-    let canonical_query_string = sorted_params
-        .iter()
-        .map(|(k, v)| format!("{}={}", aws_uri_encode(k), aws_uri_encode(v)))
-        .collect::<Vec<_>>()
-        .join("&");
-
-    let mut canonical_headers = String::new();
-    for (name, value) in &header_values {
-        let lower_name = name.to_lowercase();
-        let normalized = value.split_whitespace().collect::<Vec<_>>().join(" ");
-        let final_value = if lower_name == "expect" && normalized.is_empty() {
-            "100-continue"
-        } else {
-            &normalized
-        };
-        canonical_headers.push_str(&lower_name);
-        canonical_headers.push(':');
-        canonical_headers.push_str(final_value);
-        canonical_headers.push('\n');
-    }
-
-    let canonical_request = format!(
-        "{}\n{}\n{}\n{}\n{}\n{}",
-        method, canonical_uri, canonical_query_string, canonical_headers, signed_headers_str, payload_hash
-    );
-
-    let credential_scope = format!("{}/{}/{}/aws4_request", date_stamp, region, service);
-    let cr_hash = sha256_hex(canonical_request.as_bytes());
-    let string_to_sign = format!(
-        "AWS4-HMAC-SHA256\n{}\n{}\n{}",
-        amz_date, credential_scope, cr_hash
-    );
-
-    let signing_key = derive_signing_key_cached(secret_key, date_stamp, region, service);
-    let calculated = hmac_sha256(&signing_key, string_to_sign.as_bytes());
-    let calculated_hex = hex::encode(&calculated);
-
-    constant_time_compare_inner(calculated_hex.as_bytes(), provided_signature.as_bytes())
-}
-
-#[pyfunction]
-pub fn derive_signing_key(
-    secret_key: &str,
-    date_stamp: &str,
-    region: &str,
-    service: &str,
-) -> Vec<u8> {
-    derive_signing_key_cached(secret_key, date_stamp, region, service)
-}
-
-#[pyfunction]
-pub fn compute_signature(signing_key: &[u8], string_to_sign: &str) -> String {
-    let sig = hmac_sha256(signing_key, string_to_sign.as_bytes());
-    hex::encode(sig)
-}
-
-#[pyfunction]
-pub fn build_string_to_sign(
-    amz_date: &str,
-    credential_scope: &str,
-    canonical_request: &str,
-) -> String {
-    let cr_hash = sha256_hex(canonical_request.as_bytes());
-    format!(
-        "AWS4-HMAC-SHA256\n{}\n{}\n{}",
-        amz_date, credential_scope, cr_hash
-    )
-}
-
-#[pyfunction]
-pub fn constant_time_compare(a: &str, b: &str) -> bool {
-    constant_time_compare_inner(a.as_bytes(), b.as_bytes())
-}
-
-#[pyfunction]
-pub fn clear_signing_key_cache() {
-    SIGNING_KEY_CACHE.lock().clear();
-}

python/myfsio_core/src/storage.rs

@@ -1,817 +0,0 @@
-use pyo3::exceptions::PyIOError;
-use pyo3::prelude::*;
-use pyo3::types::{PyDict, PyList, PyString, PyTuple};
-use serde_json::Value;
-use std::collections::HashMap;
-use std::fs;
-use std::path::Path;
-use std::time::SystemTime;
-
-const INTERNAL_FOLDERS: &[&str] = &[".meta", ".versions", ".multipart"];
-
-fn system_time_to_epoch(t: SystemTime) -> f64 {
-    t.duration_since(std::time::UNIX_EPOCH)
-        .map(|d| d.as_secs_f64())
-        .unwrap_or(0.0)
-}
-
-fn extract_etag_from_meta_bytes(content: &[u8]) -> Option<String> {
-    let marker = b"\"__etag__\"";
-    let idx = content.windows(marker.len()).position(|w| w == marker)?;
-    let after = &content[idx + marker.len()..];
-    let start = after.iter().position(|&b| b == b'"')? + 1;
-    let rest = &after[start..];
-    let end = rest.iter().position(|&b| b == b'"')?;
-    std::str::from_utf8(&rest[..end]).ok().map(|s| s.to_owned())
-}
-
-fn has_any_file(root: &str) -> bool {
-    let root_path = Path::new(root);
-    if !root_path.is_dir() {
-        return false;
-    }
-    let mut stack = vec![root_path.to_path_buf()];
-    while let Some(current) = stack.pop() {
-        let entries = match fs::read_dir(&current) {
-            Ok(e) => e,
-            Err(_) => continue,
-        };
-        for entry_result in entries {
-            let entry = match entry_result {
-                Ok(e) => e,
-                Err(_) => continue,
-            };
-            let ft = match entry.file_type() {
-                Ok(ft) => ft,
-                Err(_) => continue,
-            };
-            if ft.is_file() {
-                return true;
-            }
-            if ft.is_dir() && !ft.is_symlink() {
-                stack.push(entry.path());
-            }
-        }
-    }
-    false
-}
-
-#[pyfunction]
-pub fn write_index_entry(
-    py: Python<'_>,
-    path: &str,
-    entry_name: &str,
-    entry_data_json: &str,
-) -> PyResult<()> {
-    let path_owned = path.to_owned();
-    let entry_owned = entry_name.to_owned();
-    let data_owned = entry_data_json.to_owned();
-
-    py.detach(move || -> PyResult<()> {
-        let entry_value: Value = serde_json::from_str(&data_owned)
-            .map_err(|e| PyIOError::new_err(format!("Failed to parse entry data: {}", e)))?;
-
-        if let Some(parent) = Path::new(&path_owned).parent() {
-            let _ = fs::create_dir_all(parent);
-        }
-
-        let mut index_data: serde_json::Map<String, Value> = match fs::read_to_string(&path_owned)
-        {
-            Ok(content) => serde_json::from_str(&content).unwrap_or_default(),
-            Err(_) => serde_json::Map::new(),
-        };
-
-        index_data.insert(entry_owned, entry_value);
-
-        let serialized = serde_json::to_string(&Value::Object(index_data))
-            .map_err(|e| PyIOError::new_err(format!("Failed to serialize index: {}", e)))?;
-
-        fs::write(&path_owned, serialized)
-            .map_err(|e| PyIOError::new_err(format!("Failed to write index: {}", e)))?;
-
-        Ok(())
-    })
-}
-
-#[pyfunction]
-pub fn delete_index_entry(py: Python<'_>, path: &str, entry_name: &str) -> PyResult<bool> {
-    let path_owned = path.to_owned();
-    let entry_owned = entry_name.to_owned();
-
-    py.detach(move || -> PyResult<bool> {
-        let content = match fs::read_to_string(&path_owned) {
-            Ok(c) => c,
-            Err(_) => return Ok(false),
-        };
-
-        let mut index_data: serde_json::Map<String, Value> =
-            match serde_json::from_str(&content) {
-                Ok(v) => v,
-                Err(_) => return Ok(false),
-            };
-
-        if index_data.remove(&entry_owned).is_none() {
-            return Ok(false);
-        }
-
-        if index_data.is_empty() {
-            let _ = fs::remove_file(&path_owned);
-            return Ok(true);
-        }
-
-        let serialized = serde_json::to_string(&Value::Object(index_data))
-            .map_err(|e| PyIOError::new_err(format!("Failed to serialize index: {}", e)))?;
-
-        fs::write(&path_owned, serialized)
-            .map_err(|e| PyIOError::new_err(format!("Failed to write index: {}", e)))?;
-
-        Ok(true)
-    })
-}
-
-#[pyfunction]
-pub fn check_bucket_contents(
-    py: Python<'_>,
-    bucket_path: &str,
-    version_roots: Vec<String>,
-    multipart_roots: Vec<String>,
-) -> PyResult<(bool, bool, bool)> {
-    let bucket_owned = bucket_path.to_owned();
-
-    py.detach(move || -> PyResult<(bool, bool, bool)> {
-        let mut has_objects = false;
-        let bucket_p = Path::new(&bucket_owned);
-        if bucket_p.is_dir() {
-            let mut stack = vec![bucket_p.to_path_buf()];
-            'obj_scan: while let Some(current) = stack.pop() {
-                let is_root = current == bucket_p;
-                let entries = match fs::read_dir(&current) {
-                    Ok(e) => e,
-                    Err(_) => continue,
-                };
-                for entry_result in entries {
-                    let entry = match entry_result {
-                        Ok(e) => e,
-                        Err(_) => continue,
-                    };
-                    let ft = match entry.file_type() {
-                        Ok(ft) => ft,
-                        Err(_) => continue,
-                    };
-                    if is_root {
-                        if let Some(name) = entry.file_name().to_str() {
-                            if INTERNAL_FOLDERS.contains(&name) {
-                                continue;
-                            }
-                        }
-                    }
-                    if ft.is_file() && !ft.is_symlink() {
-                        has_objects = true;
-                        break 'obj_scan;
-                    }
-                    if ft.is_dir() && !ft.is_symlink() {
-                        stack.push(entry.path());
-                    }
-                }
-            }
-        }
-
-        let mut has_versions = false;
-        for root in &version_roots {
-            if has_versions {
-                break;
-            }
-            has_versions = has_any_file(root);
-        }
-
-        let mut has_multipart = false;
-        for root in &multipart_roots {
-            if has_multipart {
-                break;
-            }
-            has_multipart = has_any_file(root);
-        }
-
-        Ok((has_objects, has_versions, has_multipart))
-    })
-}
-
-#[pyfunction]
-pub fn shallow_scan(
-    py: Python<'_>,
-    target_dir: &str,
-    prefix: &str,
-    meta_cache_json: &str,
-) -> PyResult<Py<PyAny>> {
-    let target_owned = target_dir.to_owned();
-    let prefix_owned = prefix.to_owned();
-    let cache_owned = meta_cache_json.to_owned();
-
-    let result: (
-        Vec<(String, u64, f64, Option<String>)>,
-        Vec<String>,
-        Vec<(String, bool)>,
-    ) = py.detach(move || -> PyResult<(
-        Vec<(String, u64, f64, Option<String>)>,
-        Vec<String>,
-        Vec<(String, bool)>,
-    )> {
-        let meta_cache: HashMap<String, String> =
-            serde_json::from_str(&cache_owned).unwrap_or_default();
-
-        let mut files: Vec<(String, u64, f64, Option<String>)> = Vec::new();
-        let mut dirs: Vec<String> = Vec::new();
-
-        let entries = match fs::read_dir(&target_owned) {
-            Ok(e) => e,
-            Err(_) => return Ok((files, dirs, Vec::new())),
-        };
-
-        for entry_result in entries {
-            let entry = match entry_result {
-                Ok(e) => e,
-                Err(_) => continue,
-            };
-            let name = match entry.file_name().into_string() {
-                Ok(n) => n,
-                Err(_) => continue,
-            };
-            if INTERNAL_FOLDERS.contains(&name.as_str()) {
-                continue;
-            }
-            let ft = match entry.file_type() {
-                Ok(ft) => ft,
-                Err(_) => continue,
-            };
-            if ft.is_dir() && !ft.is_symlink() {
-                let cp = format!("{}{}/", prefix_owned, name);
-                dirs.push(cp);
-            } else if ft.is_file() && !ft.is_symlink() {
-                let key = format!("{}{}", prefix_owned, name);
-                let md = match entry.metadata() {
-                    Ok(m) => m,
-                    Err(_) => continue,
-                };
-                let size = md.len();
-                let mtime = md
-                    .modified()
-                    .map(system_time_to_epoch)
-                    .unwrap_or(0.0);
-                let etag = meta_cache.get(&key).cloned();
-                files.push((key, size, mtime, etag));
-            }
-        }
-
-        files.sort_by(|a, b| a.0.cmp(&b.0));
-        dirs.sort();
-
-        let mut merged: Vec<(String, bool)> = Vec::with_capacity(files.len() + dirs.len());
-        let mut fi = 0;
-        let mut di = 0;
-        while fi < files.len() && di < dirs.len() {
-            if files[fi].0 <= dirs[di] {
-                merged.push((files[fi].0.clone(), false));
-                fi += 1;
-            } else {
-                merged.push((dirs[di].clone(), true));
-                di += 1;
-            }
-        }
-        while fi < files.len() {
-            merged.push((files[fi].0.clone(), false));
-            fi += 1;
-        }
-        while di < dirs.len() {
-            merged.push((dirs[di].clone(), true));
-            di += 1;
-        }
-
-        Ok((files, dirs, merged))
-    })?;
-
-    let (files, dirs, merged) = result;
-
-    let dict = PyDict::new(py);
-
-    let files_list = PyList::empty(py);
-    for (key, size, mtime, etag) in &files {
-        let etag_py: Py<PyAny> = match etag {
-            Some(e) => PyString::new(py, e).into_any().unbind(),
-            None => py.None(),
-        };
-        let tuple = PyTuple::new(py, &[
-            PyString::new(py, key).into_any().unbind(),
-            size.into_pyobject(py)?.into_any().unbind(),
-            mtime.into_pyobject(py)?.into_any().unbind(),
-            etag_py,
-        ])?;
-        files_list.append(tuple)?;
-    }
-    dict.set_item("files", files_list)?;
-
-    let dirs_list = PyList::empty(py);
-    for d in &dirs {
-        dirs_list.append(PyString::new(py, d))?;
-    }
-    dict.set_item("dirs", dirs_list)?;
-
-    let merged_list = PyList::empty(py);
-    for (key, is_dir) in &merged {
-        let bool_obj: Py<PyAny> = if *is_dir {
-            true.into_pyobject(py)?.to_owned().into_any().unbind()
-        } else {
-            false.into_pyobject(py)?.to_owned().into_any().unbind()
-        };
-        let tuple = PyTuple::new(py, &[
-            PyString::new(py, key).into_any().unbind(),
-            bool_obj,
-        ])?;
-        merged_list.append(tuple)?;
-    }
-    dict.set_item("merged_keys", merged_list)?;
-
-    Ok(dict.into_any().unbind())
-}
-
-#[pyfunction]
-pub fn bucket_stats_scan(
-    py: Python<'_>,
-    bucket_path: &str,
-    versions_root: &str,
-) -> PyResult<(u64, u64, u64, u64)> {
-    let bucket_owned = bucket_path.to_owned();
-    let versions_owned = versions_root.to_owned();
-
-    py.detach(move || -> PyResult<(u64, u64, u64, u64)> {
-        let mut object_count: u64 = 0;
-        let mut total_bytes: u64 = 0;
-
-        let bucket_p = Path::new(&bucket_owned);
-        if bucket_p.is_dir() {
-            let mut stack = vec![bucket_p.to_path_buf()];
-            while let Some(current) = stack.pop() {
-                let is_root = current == bucket_p;
-                let entries = match fs::read_dir(&current) {
-                    Ok(e) => e,
-                    Err(_) => continue,
-                };
-                for entry_result in entries {
-                    let entry = match entry_result {
-                        Ok(e) => e,
-                        Err(_) => continue,
-                    };
-                    if is_root {
-                        if let Some(name) = entry.file_name().to_str() {
-                            if INTERNAL_FOLDERS.contains(&name) {
-                                continue;
-                            }
-                        }
-                    }
-                    let ft = match entry.file_type() {
-                        Ok(ft) => ft,
-                        Err(_) => continue,
-                    };
-                    if ft.is_dir() && !ft.is_symlink() {
-                        stack.push(entry.path());
-                    } else if ft.is_file() && !ft.is_symlink() {
-                        object_count += 1;
-                        if let Ok(md) = entry.metadata() {
-                            total_bytes += md.len();
-                        }
-                    }
-                }
-            }
-        }
-
-        let mut version_count: u64 = 0;
-        let mut version_bytes: u64 = 0;
-
-        let versions_p = Path::new(&versions_owned);
-        if versions_p.is_dir() {
-            let mut stack = vec![versions_p.to_path_buf()];
-            while let Some(current) = stack.pop() {
-                let entries = match fs::read_dir(&current) {
-                    Ok(e) => e,
-                    Err(_) => continue,
-                };
-                for entry_result in entries {
-                    let entry = match entry_result {
-                        Ok(e) => e,
-                        Err(_) => continue,
-                    };
-                    let ft = match entry.file_type() {
-                        Ok(ft) => ft,
-                        Err(_) => continue,
-                    };
-                    if ft.is_dir() && !ft.is_symlink() {
-                        stack.push(entry.path());
-                    } else if ft.is_file() && !ft.is_symlink() {
-                        if let Some(name) = entry.file_name().to_str() {
-                            if name.ends_with(".bin") {
-                                version_count += 1;
-                                if let Ok(md) = entry.metadata() {
-                                    version_bytes += md.len();
-                                }
-                            }
-                        }
-                    }
-                }
-            }
-        }
-
-        Ok((object_count, total_bytes, version_count, version_bytes))
-    })
-}
-
-#[pyfunction]
-#[pyo3(signature = (bucket_path, search_root, query, limit))]
-pub fn search_objects_scan(
-    py: Python<'_>,
-    bucket_path: &str,
-    search_root: &str,
-    query: &str,
-    limit: usize,
-) -> PyResult<Py<PyAny>> {
-    let bucket_owned = bucket_path.to_owned();
-    let search_owned = search_root.to_owned();
-    let query_owned = query.to_owned();
-
-    let result: (Vec<(String, u64, f64)>, bool) = py.detach(
-        move || -> PyResult<(Vec<(String, u64, f64)>, bool)> {
-            let query_lower = query_owned.to_lowercase();
-            let bucket_len = bucket_owned.len() + 1;
-            let scan_limit = limit * 4;
-            let mut matched: usize = 0;
-            let mut results: Vec<(String, u64, f64)> = Vec::new();
-
-            let search_p = Path::new(&search_owned);
-            if !search_p.is_dir() {
-                return Ok((results, false));
-            }
-
-            let bucket_p = Path::new(&bucket_owned);
-            let mut stack = vec![search_p.to_path_buf()];
-
-            'scan: while let Some(current) = stack.pop() {
-                let is_bucket_root = current == bucket_p;
-                let entries = match fs::read_dir(&current) {
-                    Ok(e) => e,
-                    Err(_) => continue,
-                };
-                for entry_result in entries {
-                    let entry = match entry_result {
-                        Ok(e) => e,
-                        Err(_) => continue,
-                    };
-                    if is_bucket_root {
-                        if let Some(name) = entry.file_name().to_str() {
-                            if INTERNAL_FOLDERS.contains(&name) {
-                                continue;
-                            }
-                        }
-                    }
-                    let ft = match entry.file_type() {
-                        Ok(ft) => ft,
-                        Err(_) => continue,
-                    };
-                    if ft.is_dir() && !ft.is_symlink() {
-                        stack.push(entry.path());
-                    } else if ft.is_file() && !ft.is_symlink() {
-                        let full_path = entry.path();
-                        let full_str = full_path.to_string_lossy();
-                        if full_str.len() <= bucket_len {
-                            continue;
-                        }
-                        let key = full_str[bucket_len..].replace('\\', "/");
-                        if key.to_lowercase().contains(&query_lower) {
-                            if let Ok(md) = entry.metadata() {
-                                let size = md.len();
-                                let mtime = md
-                                    .modified()
-                                    .map(system_time_to_epoch)
-                                    .unwrap_or(0.0);
-                                results.push((key, size, mtime));
-                                matched += 1;
-                            }
-                        }
-                        if matched >= scan_limit {
-                            break 'scan;
-                        }
-                    }
-                }
-            }
-
-            results.sort_by(|a, b| a.0.cmp(&b.0));
-            let truncated = results.len() > limit;
-            results.truncate(limit);
-
-            Ok((results, truncated))
-        },
-    )?;
-
-    let (results, truncated) = result;
-
-    let dict = PyDict::new(py);
-
-    let results_list = PyList::empty(py);
-    for (key, size, mtime) in &results {
-        let tuple = PyTuple::new(py, &[
-            PyString::new(py, key).into_any().unbind(),
-            size.into_pyobject(py)?.into_any().unbind(),
-            mtime.into_pyobject(py)?.into_any().unbind(),
-        ])?;
-        results_list.append(tuple)?;
-    }
-    dict.set_item("results", results_list)?;
-    dict.set_item("truncated", truncated)?;
-
-    Ok(dict.into_any().unbind())
-}
-
-#[pyfunction]
-pub fn build_object_cache(
-    py: Python<'_>,
-    bucket_path: &str,
-    meta_root: &str,
-    etag_index_path: &str,
-) -> PyResult<Py<PyAny>> {
-    let bucket_owned = bucket_path.to_owned();
-    let meta_owned = meta_root.to_owned();
-    let index_path_owned = etag_index_path.to_owned();
-
-    let result: (HashMap<String, String>, Vec<(String, u64, f64, Option<String>)>, bool) =
-        py.detach(move || -> PyResult<(
-            HashMap<String, String>,
-            Vec<(String, u64, f64, Option<String>)>,
-            bool,
-        )> {
-            let mut meta_cache: HashMap<String, String> = HashMap::new();
-            let mut index_mtime: f64 = 0.0;
-            let mut etag_cache_changed = false;
-
-            let index_p = Path::new(&index_path_owned);
-            if index_p.is_file() {
-                if let Ok(md) = fs::metadata(&index_path_owned) {
-                    index_mtime = md
-                        .modified()
-                        .map(system_time_to_epoch)
-                        .unwrap_or(0.0);
-                }
-                if let Ok(content) = fs::read_to_string(&index_path_owned) {
-                    if let Ok(parsed) = serde_json::from_str::<HashMap<String, String>>(&content) {
-                        meta_cache = parsed;
-                    }
-                }
-            }
-
-            let meta_p = Path::new(&meta_owned);
-            let mut needs_rebuild = false;
-
-            if meta_p.is_dir() && index_mtime > 0.0 {
-                fn check_newer(dir: &Path, index_mtime: f64) -> bool {
-                    let entries = match fs::read_dir(dir) {
-                        Ok(e) => e,
-                        Err(_) => return false,
-                    };
-                    for entry_result in entries {
-                        let entry = match entry_result {
-                            Ok(e) => e,
-                            Err(_) => continue,
-                        };
-                        let ft = match entry.file_type() {
-                            Ok(ft) => ft,
-                            Err(_) => continue,
-                        };
-                        if ft.is_dir() && !ft.is_symlink() {
-                            if check_newer(&entry.path(), index_mtime) {
-                                return true;
-                            }
-                        } else if ft.is_file() {
-                            if let Some(name) = entry.file_name().to_str() {
-                                if name.ends_with(".meta.json") || name == "_index.json" {
-                                    if let Ok(md) = entry.metadata() {
-                                        let mt = md
-                                            .modified()
-                                            .map(system_time_to_epoch)
-                                            .unwrap_or(0.0);
-                                        if mt > index_mtime {
-                                            return true;
-                                        }
-                                    }
-                                }
-                            }
-                        }
-                    }
-                    false
-                }
-                needs_rebuild = check_newer(meta_p, index_mtime);
-            } else if meta_cache.is_empty() {
-                needs_rebuild = true;
-            }
-
-            if needs_rebuild && meta_p.is_dir() {
-                let meta_str = meta_owned.clone();
-                let meta_len = meta_str.len() + 1;
-                let mut index_files: Vec<String> = Vec::new();
-                let mut legacy_meta_files: Vec<(String, String)> = Vec::new();
-
-                fn collect_meta(
-                    dir: &Path,
-                    meta_len: usize,
-                    index_files: &mut Vec<String>,
-                    legacy_meta_files: &mut Vec<(String, String)>,
-                ) {
-                    let entries = match fs::read_dir(dir) {
-                        Ok(e) => e,
-                        Err(_) => return,
-                    };
-                    for entry_result in entries {
-                        let entry = match entry_result {
-                            Ok(e) => e,
-                            Err(_) => continue,
-                        };
-                        let ft = match entry.file_type() {
-                            Ok(ft) => ft,
-                            Err(_) => continue,
-                        };
-                        if ft.is_dir() && !ft.is_symlink() {
-                            collect_meta(&entry.path(), meta_len, index_files, legacy_meta_files);
-                        } else if ft.is_file() {
-                            if let Some(name) = entry.file_name().to_str() {
-                                let full = entry.path().to_string_lossy().to_string();
-                                if name == "_index.json" {
-                                    index_files.push(full);
-                                } else if name.ends_with(".meta.json") {
-                                    if full.len() > meta_len {
-                                        let rel = &full[meta_len..];
-                                        let key = if rel.len() > 10 {
-                                            rel[..rel.len() - 10].replace('\\', "/")
-                                        } else {
-                                            continue;
-                                        };
-                                        legacy_meta_files.push((key, full));
-                                    }
-                                }
-                            }
-                        }
-                    }
-                }
-
-                collect_meta(
-                    meta_p,
-                    meta_len,
-                    &mut index_files,
-                    &mut legacy_meta_files,
-                );
-
-                meta_cache.clear();
-
-                for idx_path in &index_files {
-                    if let Ok(content) = fs::read_to_string(idx_path) {
-                        if let Ok(idx_data) = serde_json::from_str::<HashMap<String, Value>>(&content) {
-                            let rel_dir = if idx_path.len() > meta_len {
-                                let r = &idx_path[meta_len..];
-                                r.replace('\\', "/")
-                            } else {
-                                String::new()
-                            };
-                            let dir_prefix = if rel_dir.ends_with("/_index.json") {
-                                &rel_dir[..rel_dir.len() - "/_index.json".len()]
-                            } else {
-                                ""
-                            };
-                            for (entry_name, entry_data) in &idx_data {
-                                let key = if dir_prefix.is_empty() {
-                                    entry_name.clone()
-                                } else {
-                                    format!("{}/{}", dir_prefix, entry_name)
-                                };
-                                if let Some(meta_obj) = entry_data.get("metadata") {
-                                    if let Some(etag) = meta_obj.get("__etag__") {
-                                        if let Some(etag_str) = etag.as_str() {
-                                            meta_cache.insert(key, etag_str.to_owned());
-                                        }
-                                    }
-                                }
-                            }
-                        }
-                    }
-                }
-
-                for (key, path) in &legacy_meta_files {
-                    if meta_cache.contains_key(key) {
-                        continue;
-                    }
-                    if let Ok(content) = fs::read(path) {
-                        if let Some(etag) = extract_etag_from_meta_bytes(&content) {
-                            meta_cache.insert(key.clone(), etag);
-                        }
-                    }
-                }
-
-                etag_cache_changed = true;
-            }
-
-            let bucket_p = Path::new(&bucket_owned);
-            let bucket_len = bucket_owned.len() + 1;
-            let mut objects: Vec<(String, u64, f64, Option<String>)> = Vec::new();
-
-            if bucket_p.is_dir() {
-                let mut stack = vec![bucket_p.to_path_buf()];
-                while let Some(current) = stack.pop() {
-                    let entries = match fs::read_dir(&current) {
-                        Ok(e) => e,
-                        Err(_) => continue,
-                    };
-                    for entry_result in entries {
-                        let entry = match entry_result {
-                            Ok(e) => e,
-                            Err(_) => continue,
-                        };
-                        let ft = match entry.file_type() {
-                            Ok(ft) => ft,
-                            Err(_) => continue,
-                        };
-                        if ft.is_dir() && !ft.is_symlink() {
-                            let full = entry.path();
-                            let full_str = full.to_string_lossy();
-                            if full_str.len() > bucket_len {
-                                let first_part: &str = if let Some(sep_pos) =
-                                    full_str[bucket_len..].find(|c: char| c == '\\' || c == '/')
-                                {
-                                    &full_str[bucket_len..bucket_len + sep_pos]
-                                } else {
-                                    &full_str[bucket_len..]
-                                };
-                                if INTERNAL_FOLDERS.contains(&first_part) {
-                                    continue;
-                                }
-                            } else if let Some(name) = entry.file_name().to_str() {
-                                if INTERNAL_FOLDERS.contains(&name) {
-                                    continue;
-                                }
-                            }
-                            stack.push(full);
-                        } else if ft.is_file() && !ft.is_symlink() {
-                            let full = entry.path();
-                            let full_str = full.to_string_lossy();
-                            if full_str.len() <= bucket_len {
-                                continue;
-                            }
-                            let rel = &full_str[bucket_len..];
-                            let first_part: &str =
-                                if let Some(sep_pos) = rel.find(|c: char| c == '\\' || c == '/') {
-                                    &rel[..sep_pos]
-                                } else {
-                                    rel
-                                };
-                            if INTERNAL_FOLDERS.contains(&first_part) {
-                                continue;
-                            }
-                            let key = rel.replace('\\', "/");
-                            if let Ok(md) = entry.metadata() {
-                                let size = md.len();
-                                let mtime = md
-                                    .modified()
-                                    .map(system_time_to_epoch)
-                                    .unwrap_or(0.0);
-                                let etag = meta_cache.get(&key).cloned();
-                                objects.push((key, size, mtime, etag));
-                            }
-                        }
-                    }
-                }
-            }
-
-            Ok((meta_cache, objects, etag_cache_changed))
-        })?;
-
-    let (meta_cache, objects, etag_cache_changed) = result;
-
-    let dict = PyDict::new(py);
-
-    let cache_dict = PyDict::new(py);
-    for (k, v) in &meta_cache {
-        cache_dict.set_item(k, v)?;
-    }
-    dict.set_item("etag_cache", cache_dict)?;
-
-    let objects_list = PyList::empty(py);
-    for (key, size, mtime, etag) in &objects {
-        let etag_py: Py<PyAny> = match etag {
-            Some(e) => PyString::new(py, e).into_any().unbind(),
-            None => py.None(),
-        };
-        let tuple = PyTuple::new(py, &[
-            PyString::new(py, key).into_any().unbind(),
-            size.into_pyobject(py)?.into_any().unbind(),
-            mtime.into_pyobject(py)?.into_any().unbind(),
-            etag_py,
-        ])?;
-        objects_list.append(tuple)?;
-    }
-    dict.set_item("objects", objects_list)?;
-    dict.set_item("etag_cache_changed", etag_cache_changed)?;
-
-    Ok(dict.into_any().unbind())
-}

python/myfsio_core/src/streaming.rs

@@ -1,112 +0,0 @@
-use md5::{Digest, Md5};
-use pyo3::exceptions::{PyIOError, PyValueError};
-use pyo3::prelude::*;
-use std::fs::{self, File};
-use std::io::{Read, Write};
-use uuid::Uuid;
-
-const DEFAULT_CHUNK_SIZE: usize = 262144;
-
-#[pyfunction]
-#[pyo3(signature = (stream, tmp_dir, chunk_size=DEFAULT_CHUNK_SIZE))]
-pub fn stream_to_file_with_md5(
-    py: Python<'_>,
-    stream: &Bound<'_, PyAny>,
-    tmp_dir: &str,
-    chunk_size: usize,
-) -> PyResult<(String, String, u64)> {
-    let chunk_size = if chunk_size == 0 {
-        DEFAULT_CHUNK_SIZE
-    } else {
-        chunk_size
-    };
-
-    fs::create_dir_all(tmp_dir)
-        .map_err(|e| PyIOError::new_err(format!("Failed to create tmp dir: {}", e)))?;
-
-    let tmp_name = format!("{}.tmp", Uuid::new_v4().as_hyphenated());
-    let tmp_path_buf = std::path::PathBuf::from(tmp_dir).join(&tmp_name);
-    let tmp_path = tmp_path_buf.to_string_lossy().into_owned();
-
-    let mut file = File::create(&tmp_path)
-        .map_err(|e| PyIOError::new_err(format!("Failed to create temp file: {}", e)))?;
-    let mut hasher = Md5::new();
-    let mut total_bytes: u64 = 0;
-
-    let result: PyResult<()> = (|| {
-        loop {
-            let chunk: Vec<u8> = stream.call_method1("read", (chunk_size,))?.extract()?;
-            if chunk.is_empty() {
-                break;
-            }
-            hasher.update(&chunk);
-            file.write_all(&chunk)
-                .map_err(|e| PyIOError::new_err(format!("Failed to write: {}", e)))?;
-            total_bytes += chunk.len() as u64;
-
-            py.check_signals()?;
-        }
-        file.sync_all()
-            .map_err(|e| PyIOError::new_err(format!("Failed to fsync: {}", e)))?;
-        Ok(())
-    })();
-
-    if let Err(e) = result {
-        drop(file);
-        let _ = fs::remove_file(&tmp_path);
-        return Err(e);
-    }
-
-    drop(file);
-
-    let md5_hex = format!("{:x}", hasher.finalize());
-    Ok((tmp_path, md5_hex, total_bytes))
-}
-
-#[pyfunction]
-pub fn assemble_parts_with_md5(
-    py: Python<'_>,
-    part_paths: Vec<String>,
-    dest_path: &str,
-) -> PyResult<String> {
-    if part_paths.is_empty() {
-        return Err(PyValueError::new_err("No parts to assemble"));
-    }
-
-    let dest = dest_path.to_owned();
-    let parts = part_paths;
-
-    py.detach(move || {
-        if let Some(parent) = std::path::Path::new(&dest).parent() {
-            fs::create_dir_all(parent)
-                .map_err(|e| PyIOError::new_err(format!("Failed to create dest dir: {}", e)))?;
-        }
-
-        let mut target = File::create(&dest)
-            .map_err(|e| PyIOError::new_err(format!("Failed to create dest file: {}", e)))?;
-        let mut hasher = Md5::new();
-        let mut buf = vec![0u8; 1024 * 1024];
-
-        for part_path in &parts {
-            let mut part = File::open(part_path)
-                .map_err(|e| PyIOError::new_err(format!("Failed to open part {}: {}", part_path, e)))?;
-            loop {
-                let n = part
-                    .read(&mut buf)
-                    .map_err(|e| PyIOError::new_err(format!("Failed to read part: {}", e)))?;
-                if n == 0 {
-                    break;
-                }
-                hasher.update(&buf[..n]);
-                target
-                    .write_all(&buf[..n])
-                    .map_err(|e| PyIOError::new_err(format!("Failed to write: {}", e)))?;
-            }
-        }
-
-        target.sync_all()
-            .map_err(|e| PyIOError::new_err(format!("Failed to fsync: {}", e)))?;
-
-        Ok(format!("{:x}", hasher.finalize()))
-    })
-}

python/myfsio_core/src/validation.rs

@@ -1,149 +0,0 @@
-use pyo3::prelude::*;
-use std::sync::LazyLock;
-use unicode_normalization::UnicodeNormalization;
-
-const WINDOWS_RESERVED: &[&str] = &[
-    "CON", "PRN", "AUX", "NUL", "COM0", "COM1", "COM2", "COM3", "COM4", "COM5", "COM6", "COM7",
-    "COM8", "COM9", "LPT0", "LPT1", "LPT2", "LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8",
-    "LPT9",
-];
-
-const WINDOWS_ILLEGAL_CHARS: &[char] = &['<', '>', ':', '"', '/', '\\', '|', '?', '*'];
-
-const INTERNAL_FOLDERS: &[&str] = &[".meta", ".versions", ".multipart"];
-const SYSTEM_ROOT: &str = ".myfsio.sys";
-
-static IP_REGEX: LazyLock<regex::Regex> =
-    LazyLock::new(|| regex::Regex::new(r"^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$").unwrap());
-
-#[pyfunction]
-#[pyo3(signature = (object_key, max_length_bytes=1024, is_windows=false, reserved_prefixes=None))]
-pub fn validate_object_key(
-    object_key: &str,
-    max_length_bytes: usize,
-    is_windows: bool,
-    reserved_prefixes: Option<Vec<String>>,
-) -> PyResult<Option<String>> {
-    if object_key.is_empty() {
-        return Ok(Some("Object key required".to_string()));
-    }
-
-    if object_key.contains('\0') {
-        return Ok(Some("Object key contains null bytes".to_string()));
-    }
-
-    let normalized: String = object_key.nfc().collect();
-
-    if normalized.len() > max_length_bytes {
-        return Ok(Some(format!(
-            "Object key exceeds maximum length of {} bytes",
-            max_length_bytes
-        )));
-    }
-
-    if normalized.starts_with('/') || normalized.starts_with('\\') {
-        return Ok(Some("Object key cannot start with a slash".to_string()));
-    }
-
-    let parts: Vec<&str> = if cfg!(windows) || is_windows {
-        normalized.split(['/', '\\']).collect()
-    } else {
-        normalized.split('/').collect()
-    };
-
-    for part in &parts {
-        if part.is_empty() {
-            continue;
-        }
-
-        if *part == ".." {
-            return Ok(Some(
-                "Object key contains parent directory references".to_string(),
-            ));
-        }
-
-        if *part == "." {
-            return Ok(Some("Object key contains invalid segments".to_string()));
-        }
-
-        if part.chars().any(|c| (c as u32) < 32) {
-            return Ok(Some(
-                "Object key contains control characters".to_string(),
-            ));
-        }
-
-        if is_windows {
-            if part.chars().any(|c| WINDOWS_ILLEGAL_CHARS.contains(&c)) {
-                return Ok(Some(
-                    "Object key contains characters not supported on Windows filesystems"
-                        .to_string(),
-                ));
-            }
-            if part.ends_with(' ') || part.ends_with('.') {
-                return Ok(Some(
-                    "Object key segments cannot end with spaces or periods on Windows".to_string(),
-                ));
-            }
-            let trimmed = part.trim_end_matches(['.', ' ']).to_uppercase();
-            if WINDOWS_RESERVED.contains(&trimmed.as_str()) {
-                return Ok(Some(format!("Invalid filename segment: {}", part)));
-            }
-        }
-    }
-
-    let non_empty_parts: Vec<&str> = parts.iter().filter(|p| !p.is_empty()).copied().collect();
-    if let Some(top) = non_empty_parts.first() {
-        if INTERNAL_FOLDERS.contains(top) || *top == SYSTEM_ROOT {
-            return Ok(Some("Object key uses a reserved prefix".to_string()));
-        }
-
-        if let Some(ref prefixes) = reserved_prefixes {
-            for prefix in prefixes {
-                if *top == prefix.as_str() {
-                    return Ok(Some("Object key uses a reserved prefix".to_string()));
-                }
-            }
-        }
-    }
-
-    Ok(None)
-}
-
-#[pyfunction]
-pub fn validate_bucket_name(bucket_name: &str) -> Option<String> {
-    let len = bucket_name.len();
-    if len < 3 || len > 63 {
-        return Some("Bucket name must be between 3 and 63 characters".to_string());
-    }
-
-    let bytes = bucket_name.as_bytes();
-    if !bytes[0].is_ascii_lowercase() && !bytes[0].is_ascii_digit() {
-        return Some(
-            "Bucket name must start and end with a lowercase letter or digit".to_string(),
-        );
-    }
-    if !bytes[len - 1].is_ascii_lowercase() && !bytes[len - 1].is_ascii_digit() {
-        return Some(
-            "Bucket name must start and end with a lowercase letter or digit".to_string(),
-        );
-    }
-
-    for &b in bytes {
-        if !b.is_ascii_lowercase() && !b.is_ascii_digit() && b != b'.' && b != b'-' {
-            return Some(
-                "Bucket name can only contain lowercase letters, digits, dots, and hyphens"
-                    .to_string(),
-            );
-        }
-    }
-
-    if bucket_name.contains("..") {
-        return Some("Bucket name must not contain consecutive periods".to_string());
-    }
-
-    if IP_REGEX.is_match(bucket_name) {
-        return Some("Bucket name must not be formatted as an IP address".to_string());
-    }
-
-    None
-}

python/run.py

@@ -1,315 +0,0 @@
-"""Helper script to run the API server, UI server, or both."""
-from __future__ import annotations
-
-import argparse
-import atexit
-import os
-import signal
-import sys
-import warnings
-import multiprocessing
-from multiprocessing import Process
-from pathlib import Path
-
-from dotenv import load_dotenv
-
-for _env_file in [
-    Path("/opt/myfsio/myfsio.env"),
-    Path.cwd() / ".env",
-    Path.cwd() / "myfsio.env",
-]:
-    if _env_file.exists():
-        load_dotenv(_env_file, override=True)
-
-from typing import Optional
-
-from app import create_api_app, create_ui_app
-from app.config import AppConfig
-from app.iam import IamService, IamError, ALLOWED_ACTIONS, _derive_fernet_key
-from app.version import get_version
-
-PYTHON_DEPRECATION_MESSAGE = (
-    "The Python MyFSIO runtime is deprecated as of 2026-04-21. "
-    "Use the Rust server in rust/myfsio-engine for supported development and production usage."
-)
-
-
-def _server_host() -> str:
-    """Return the bind host for API and UI servers."""
-    return os.getenv("APP_HOST", "0.0.0.0")
-
-
-def _is_debug_enabled() -> bool:
-    return os.getenv("FLASK_DEBUG", "0").lower() in ("1", "true", "yes")
-
-
-def _is_frozen() -> bool:
-    """Check if running as a compiled binary (PyInstaller/Nuitka)."""
-    return getattr(sys, 'frozen', False) or '__compiled__' in globals()
-
-
-def _serve_granian(target: str, port: int, config: Optional[AppConfig] = None) -> None:
-    from granian import Granian
-    from granian.constants import Interfaces
-    from granian.http import HTTP1Settings
-
-    kwargs: dict = {
-        "target": target,
-        "address": _server_host(),
-        "port": port,
-        "interface": Interfaces.WSGI,
-        "factory": True,
-        "workers": 1,
-    }
-
-    if config:
-        kwargs["blocking_threads"] = config.server_threads
-        kwargs["backlog"] = config.server_backlog
-        kwargs["backpressure"] = config.server_connection_limit
-        kwargs["http1_settings"] = HTTP1Settings(
-            header_read_timeout=config.server_channel_timeout * 1000,
-            max_buffer_size=config.server_max_buffer_size,
-        )
-    else:
-        kwargs["http1_settings"] = HTTP1Settings(
-            max_buffer_size=1024 * 1024 * 128,
-        )
-
-    server = Granian(**kwargs)
-    server.serve()
-
-
-def serve_api(port: int, prod: bool = False, config: Optional[AppConfig] = None) -> None:
-    if prod:
-        _serve_granian("app:create_api_app", port, config)
-    else:
-        app = create_api_app()
-        debug = _is_debug_enabled()
-        if debug:
-            warnings.warn("DEBUG MODE ENABLED - DO NOT USE IN PRODUCTION", RuntimeWarning)
-        app.run(host=_server_host(), port=port, debug=debug)
-
-
-def serve_ui(port: int, prod: bool = False, config: Optional[AppConfig] = None) -> None:
-    if prod:
-        _serve_granian("app:create_ui_app", port, config)
-    else:
-        app = create_ui_app()
-        debug = _is_debug_enabled()
-        if debug:
-            warnings.warn("DEBUG MODE ENABLED - DO NOT USE IN PRODUCTION", RuntimeWarning)
-        app.run(host=_server_host(), port=port, debug=debug)
-
-
-def reset_credentials() -> None:
-    import json
-    import secrets
-    from cryptography.fernet import Fernet
-
-    config = AppConfig.from_env()
-    iam_path = config.iam_config_path
-    encryption_key = config.secret_key
-
-    access_key = os.environ.get("ADMIN_ACCESS_KEY", "").strip() or secrets.token_hex(12)
-    secret_key = os.environ.get("ADMIN_SECRET_KEY", "").strip() or secrets.token_urlsafe(32)
-    custom_keys = bool(os.environ.get("ADMIN_ACCESS_KEY", "").strip())
-
-    fernet = Fernet(_derive_fernet_key(encryption_key)) if encryption_key else None
-
-    raw_config = None
-    if iam_path.exists():
-        try:
-            raw_bytes = iam_path.read_bytes()
-            from app.iam import _IAM_ENCRYPTED_PREFIX
-            if raw_bytes.startswith(_IAM_ENCRYPTED_PREFIX):
-                if fernet:
-                    try:
-                        content = fernet.decrypt(raw_bytes[len(_IAM_ENCRYPTED_PREFIX):]).decode("utf-8")
-                        raw_config = json.loads(content)
-                    except Exception:
-                        print("WARNING: Could not decrypt existing IAM config. Creating fresh config.")
-                else:
-                    print("WARNING: IAM config is encrypted but no SECRET_KEY available. Creating fresh config.")
-            else:
-                try:
-                    raw_config = json.loads(raw_bytes.decode("utf-8"))
-                except json.JSONDecodeError:
-                    print("WARNING: Existing IAM config is corrupted. Creating fresh config.")
-        except OSError:
-            pass
-
-    if raw_config and raw_config.get("users"):
-        is_v2 = raw_config.get("version", 1) >= 2
-        admin_user = None
-        for user in raw_config["users"]:
-            policies = user.get("policies", [])
-            for p in policies:
-                actions = p.get("actions", [])
-                if "iam:*" in actions or "*" in actions:
-                    admin_user = user
-                    break
-            if admin_user:
-                break
-        if not admin_user:
-            admin_user = raw_config["users"][0]
-
-        if is_v2:
-            admin_keys = admin_user.get("access_keys", [])
-            if admin_keys:
-                admin_keys[0]["access_key"] = access_key
-                admin_keys[0]["secret_key"] = secret_key
-            else:
-                from datetime import datetime as _dt, timezone as _tz
-                admin_user["access_keys"] = [{
-                    "access_key": access_key,
-                    "secret_key": secret_key,
-                    "status": "active",
-                    "created_at": _dt.now(_tz.utc).isoformat(),
-                }]
-        else:
-            admin_user["access_key"] = access_key
-            admin_user["secret_key"] = secret_key
-    else:
-        from datetime import datetime as _dt, timezone as _tz
-        raw_config = {
-            "version": 2,
-            "users": [
-                {
-                    "user_id": f"u-{secrets.token_hex(8)}",
-                    "display_name": "Local Admin",
-                    "enabled": True,
-                    "access_keys": [
-                        {
-                            "access_key": access_key,
-                            "secret_key": secret_key,
-                            "status": "active",
-                            "created_at": _dt.now(_tz.utc).isoformat(),
-                        }
-                    ],
-                    "policies": [
-                        {"bucket": "*", "actions": list(ALLOWED_ACTIONS)}
-                    ],
-                }
-            ]
-        }
-
-    json_text = json.dumps(raw_config, indent=2)
-    iam_path.parent.mkdir(parents=True, exist_ok=True)
-    temp_path = iam_path.with_suffix(".json.tmp")
-    if fernet:
-        from app.iam import _IAM_ENCRYPTED_PREFIX
-        encrypted = fernet.encrypt(json_text.encode("utf-8"))
-        temp_path.write_bytes(_IAM_ENCRYPTED_PREFIX + encrypted)
-    else:
-        temp_path.write_text(json_text, encoding="utf-8")
-    temp_path.replace(iam_path)
-
-    print(f"\n{'='*60}")
-    print("MYFSIO - ADMIN CREDENTIALS RESET")
-    print(f"{'='*60}")
-    if custom_keys:
-        print(f"Access Key: {access_key} (from ADMIN_ACCESS_KEY)")
-        print(f"Secret Key: {'(from ADMIN_SECRET_KEY)' if os.environ.get('ADMIN_SECRET_KEY', '').strip() else secret_key}")
-    else:
-        print(f"Access Key: {access_key}")
-        print(f"Secret Key: {secret_key}")
-    print(f"{'='*60}")
-    if fernet:
-        print("IAM config saved (encrypted).")
-    else:
-        print(f"IAM config saved to: {iam_path}")
-    print(f"{'='*60}\n")
-
-
-if __name__ == "__main__":
-    multiprocessing.freeze_support()
-    if _is_frozen():
-        multiprocessing.set_start_method("spawn", force=True)
-
-    parser = argparse.ArgumentParser(description="Run the S3 clone services.")
-    parser.add_argument("--mode", choices=["api", "ui", "both", "reset-cred"], default="both")
-    parser.add_argument("--api-port", type=int, default=5000)
-    parser.add_argument("--ui-port", type=int, default=5100)
-    parser.add_argument("--prod", action="store_true", help="Run in production mode using Granian")
-    parser.add_argument("--dev", action="store_true", help="Force development mode (Flask dev server)")
-    parser.add_argument("--check-config", action="store_true", help="Validate configuration and exit")
-    parser.add_argument("--show-config", action="store_true", help="Show configuration summary and exit")
-    parser.add_argument("--reset-cred", action="store_true", help="Reset admin credentials and exit")
-    parser.add_argument("--version", action="version", version=f"MyFSIO {get_version()}")
-    args = parser.parse_args()
-
-    warnings.warn(PYTHON_DEPRECATION_MESSAGE, DeprecationWarning, stacklevel=1)
-
-    if args.reset_cred or args.mode == "reset-cred":
-        reset_credentials()
-        sys.exit(0)
-
-    if args.check_config or args.show_config:
-        config = AppConfig.from_env()
-        config.print_startup_summary()
-        if args.check_config:
-            issues = config.validate_and_report()
-            critical = [i for i in issues if i.startswith("CRITICAL:")]
-            sys.exit(1 if critical else 0)
-        sys.exit(0)
-
-    prod_mode = args.prod or (_is_frozen() and not args.dev)
-    
-    config = AppConfig.from_env()
-    
-    first_run_marker = config.storage_root / ".myfsio.sys" / ".initialized"
-    is_first_run = not first_run_marker.exists()
-    
-    if is_first_run:
-        config.print_startup_summary()
-        
-        issues = config.validate_and_report()
-        critical_issues = [i for i in issues if i.startswith("CRITICAL:")]
-        if critical_issues:
-            print("ABORTING: Critical configuration issues detected. Please fix them before starting.")
-            sys.exit(1)
-        
-        try:
-            first_run_marker.parent.mkdir(parents=True, exist_ok=True)
-            first_run_marker.write_text(f"Initialized on {__import__('datetime').datetime.now().isoformat()}\n")
-        except OSError:
-            pass
-    
-    if prod_mode:
-        print("Running in production mode (Granian)")
-        issues = config.validate_and_report()
-        critical_issues = [i for i in issues if i.startswith("CRITICAL:")]
-        if critical_issues:
-            for issue in critical_issues:
-                print(f"  {issue}")
-            print("ABORTING: Critical configuration issues detected. Please fix them before starting.")
-            sys.exit(1)
-    else:
-        print("Running in development mode (Flask dev server)")
-
-    if args.mode in {"api", "both"}:
-        print(f"Starting API server on port {args.api_port}...")
-        api_proc = Process(target=serve_api, args=(args.api_port, prod_mode, config))
-        api_proc.start()
-    else:
-        api_proc = None
-
-    def _cleanup_api():
-        if api_proc and api_proc.is_alive():
-            api_proc.terminate()
-            api_proc.join(timeout=5)
-            if api_proc.is_alive():
-                api_proc.kill()
-
-    if api_proc:
-        atexit.register(_cleanup_api)
-        signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
-
-    if args.mode in {"ui", "both"}:
-        print(f"Starting UI server on port {args.ui_port}...")
-        serve_ui(args.ui_port, prod_mode, config)
-    elif api_proc:
-        try:
-            api_proc.join()
-        except KeyboardInterrupt:
-            pass

python/static/js/connections-management.js

@@ -1,343 +0,0 @@
-window.ConnectionsManagement = (function() {
-  'use strict';
-
-  var endpoints = {};
-  var csrfToken = '';
-
-  function init(config) {
-    endpoints = config.endpoints || {};
-    csrfToken = config.csrfToken || '';
-
-    setupEventListeners();
-    checkAllConnectionHealth();
-  }
-
-  function togglePassword(id) {
-    var input = document.getElementById(id);
-    if (input) {
-      input.type = input.type === 'password' ? 'text' : 'password';
-    }
-  }
-
-  async function testConnection(formId, resultId) {
-    var form = document.getElementById(formId);
-    var resultDiv = document.getElementById(resultId);
-    if (!form || !resultDiv) return;
-
-    var formData = new FormData(form);
-    var data = {};
-    formData.forEach(function(value, key) {
-      if (key !== 'csrf_token') {
-        data[key] = value;
-      }
-    });
-
-    resultDiv.innerHTML = '<div class="text-info"><span class="spinner-border spinner-border-sm" role="status" aria-hidden="true"></span> Testing connection...</div>';
-
-    var controller = new AbortController();
-    var timeoutId = setTimeout(function() { controller.abort(); }, 20000);
-
-    try {
-      var response = await fetch(endpoints.test, {
-        method: 'POST',
-        headers: {
-          'Content-Type': 'application/json',
-          'X-CSRFToken': csrfToken
-        },
-        body: JSON.stringify(data),
-        signal: controller.signal
-      });
-      clearTimeout(timeoutId);
-
-      var result = await response.json();
-      if (response.ok) {
-        resultDiv.innerHTML = '<div class="text-success">' +
-          '<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="me-1" viewBox="0 0 16 16">' +
-          '<path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zm-3.97-3.03a.75.75 0 0 0-1.08.022L7.477 9.417 5.384 7.323a.75.75 0 0 0-1.06 1.06L6.97 11.03a.75.75 0 0 0 1.079-.02l3.992-4.99a.75.75 0 0 0-.01-1.05z"/>' +
-          '</svg>' + window.UICore.escapeHtml(result.message) + '</div>';
-      } else {
-        resultDiv.innerHTML = '<div class="text-danger">' +
-          '<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="me-1" viewBox="0 0 16 16">' +
-          '<path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zM5.354 4.646a.5.5 0 1 0-.708.708L7.293 8l-2.647 2.646a.5.5 0 0 0 .708.708L8 8.707l2.646 2.647a.5.5 0 0 0 .708-.708L8.707 8l2.647-2.646a.5.5 0 0 0-.708-.708L8 7.293 5.354 4.646z"/>' +
-          '</svg>' + window.UICore.escapeHtml(result.message) + '</div>';
-      }
-    } catch (error) {
-      clearTimeout(timeoutId);
-      var message = error.name === 'AbortError'
-        ? 'Connection test timed out - endpoint may be unreachable'
-        : 'Connection failed: Network error';
-      resultDiv.innerHTML = '<div class="text-danger">' +
-        '<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="me-1" viewBox="0 0 16 16">' +
-        '<path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zM5.354 4.646a.5.5 0 1 0-.708.708L7.293 8l-2.647 2.646a.5.5 0 0 0 .708.708L8 8.707l2.646 2.647a.5.5 0 0 0 .708-.708L8.707 8l2.647-2.646a.5.5 0 0 0-.708-.708L8 7.293 5.354 4.646z"/>' +
-        '</svg>' + message + '</div>';
-    }
-  }
-
-  async function checkConnectionHealth(connectionId, statusEl) {
-    if (!statusEl) return;
-
-    try {
-      var controller = new AbortController();
-      var timeoutId = setTimeout(function() { controller.abort(); }, 10000);
-
-      var response = await fetch(endpoints.healthTemplate.replace('CONNECTION_ID', connectionId), {
-        signal: controller.signal
-      });
-      clearTimeout(timeoutId);
-
-      var data = await response.json();
-      if (data.healthy) {
-        statusEl.innerHTML = '<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-success" viewBox="0 0 16 16">' +
-          '<path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zm-3.97-3.03a.75.75 0 0 0-1.08.022L7.477 9.417 5.384 7.323a.75.75 0 0 0-1.06 1.06L6.97 11.03a.75.75 0 0 0 1.079-.02l3.992-4.99a.75.75 0 0 0-.01-1.05z"/></svg>';
-        statusEl.setAttribute('data-status', 'healthy');
-        statusEl.setAttribute('title', 'Connected');
-      } else {
-        statusEl.innerHTML = '<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-danger" viewBox="0 0 16 16">' +
-          '<path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zM5.354 4.646a.5.5 0 1 0-.708.708L7.293 8l-2.647 2.646a.5.5 0 0 0 .708.708L8 8.707l2.646 2.647a.5.5 0 0 0 .708-.708L8.707 8l2.647-2.646a.5.5 0 0 0-.708-.708L8 7.293 5.354 4.646z"/></svg>';
-        statusEl.setAttribute('data-status', 'unhealthy');
-        statusEl.setAttribute('title', data.error || 'Unreachable');
-      }
-    } catch (error) {
-      statusEl.innerHTML = '<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-warning" viewBox="0 0 16 16">' +
-        '<path d="M8.982 1.566a1.13 1.13 0 0 0-1.96 0L.165 13.233c-.457.778.091 1.767.98 1.767h13.713c.889 0 1.438-.99.98-1.767L8.982 1.566zM8 5c.535 0 .954.462.9.995l-.35 3.507a.552.552 0 0 1-1.1 0L7.1 5.995A.905.905 0 0 1 8 5zm.002 6a1 1 0 1 1 0 2 1 1 0 0 1 0-2z"/></svg>';
-      statusEl.setAttribute('data-status', 'unknown');
-      statusEl.setAttribute('title', 'Could not check status');
-    }
-  }
-
-  function checkAllConnectionHealth() {
-    var rows = document.querySelectorAll('tr[data-connection-id]');
-    rows.forEach(function(row, index) {
-      var connectionId = row.getAttribute('data-connection-id');
-      var statusEl = row.querySelector('.connection-status');
-      if (statusEl) {
-        setTimeout(function() {
-          checkConnectionHealth(connectionId, statusEl);
-        }, index * 200);
-      }
-    });
-  }
-
-  function updateConnectionCount() {
-    var countBadge = document.querySelector('.badge.bg-primary.bg-opacity-10.text-primary.fs-6');
-    if (countBadge) {
-      var remaining = document.querySelectorAll('tr[data-connection-id]').length;
-      countBadge.textContent = remaining + ' connection' + (remaining !== 1 ? 's' : '');
-    }
-  }
-
-  function createConnectionRowHtml(conn) {
-    var ak = conn.access_key || '';
-    var maskedKey = ak.length > 12 ? ak.slice(0, 8) + '...' + ak.slice(-4) : ak;
-
-    return '<tr data-connection-id="' + window.UICore.escapeHtml(conn.id) + '">' +
-      '<td class="text-center">' +
-      '<span class="connection-status" data-status="checking" title="Checking...">' +
-      '<span class="spinner-border spinner-border-sm text-muted" role="status" style="width: 12px; height: 12px;"></span>' +
-      '</span></td>' +
-      '<td><div class="d-flex align-items-center gap-2">' +
-      '<div class="connection-icon"><svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" viewBox="0 0 16 16">' +
-      '<path d="M4.406 3.342A5.53 5.53 0 0 1 8 2c2.69 0 4.923 2 5.166 4.579C14.758 6.804 16 8.137 16 9.773 16 11.569 14.502 13 12.687 13H3.781C1.708 13 0 11.366 0 9.318c0-1.763 1.266-3.223 2.942-3.593.143-.863.698-1.723 1.464-2.383z"/></svg></div>' +
-      '<span class="fw-medium">' + window.UICore.escapeHtml(conn.name) + '</span>' +
-      '</div></td>' +
-      '<td><span class="text-muted small text-truncate d-inline-block" style="max-width: 200px;" title="' + window.UICore.escapeHtml(conn.endpoint_url) + '">' + window.UICore.escapeHtml(conn.endpoint_url) + '</span></td>' +
-      '<td><span class="badge bg-primary bg-opacity-10 text-primary">' + window.UICore.escapeHtml(conn.region) + '</span></td>' +
-      '<td><code class="small">' + window.UICore.escapeHtml(maskedKey) + '</code></td>' +
-      '<td class="text-end"><div class="btn-group btn-group-sm" role="group">' +
-      '<button type="button" class="btn btn-outline-secondary" data-bs-toggle="modal" data-bs-target="#editConnectionModal" ' +
-      'data-id="' + window.UICore.escapeHtml(conn.id) + '" data-name="' + window.UICore.escapeHtml(conn.name) + '" ' +
-      'data-endpoint="' + window.UICore.escapeHtml(conn.endpoint_url) + '" data-region="' + window.UICore.escapeHtml(conn.region) + '" ' +
-      'data-access="' + window.UICore.escapeHtml(conn.access_key) + '" title="Edit connection">' +
-      '<svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" viewBox="0 0 16 16">' +
-      '<path d="M12.146.146a.5.5 0 0 1 .708 0l3 3a.5.5 0 0 1 0 .708l-10 10a.5.5 0 0 1-.168.11l-5 2a.5.5 0 0 1-.65-.65l2-5a.5.5 0 0 1 .11-.168l10-10zM11.207 2.5 13.5 4.793 14.793 3.5 12.5 1.207 11.207 2.5zm1.586 3L10.5 3.207 4 9.707V10h.5a.5.5 0 0 1 .5.5v.5h.5a.5.5 0 0 1 .5.5v.5h.293l6.5-6.5z"/></svg></button>' +
-      '<button type="button" class="btn btn-outline-danger" data-bs-toggle="modal" data-bs-target="#deleteConnectionModal" ' +
-      'data-id="' + window.UICore.escapeHtml(conn.id) + '" data-name="' + window.UICore.escapeHtml(conn.name) + '" title="Delete connection">' +
-      '<svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" viewBox="0 0 16 16">' +
-      '<path d="M5.5 5.5A.5.5 0 0 1 6 6v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm2.5 0a.5.5 0 0 1 .5.5v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm3 .5a.5.5 0 0 0-1 0v6a.5.5 0 0 0 1 0V6z"/>' +
-      '<path fill-rule="evenodd" d="M14.5 3a1 1 0 0 1-1 1H13v9a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V4h-.5a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1H6a1 1 0 0 1 1-1h2a1 1 0 0 1 1 1h3.5a1 1 0 0 1 1 1v1zM4.118 4 4 4.059V13a1 1 0 0 0 1 1h6a1 1 0 0 0 1-1V4.059L11.882 4H4.118zM2.5 3V2h11v1h-11z"/></svg></button>' +
-      '</div></td></tr>';
-  }
-
-  function setupEventListeners() {
-    var testBtn = document.getElementById('testConnectionBtn');
-    if (testBtn) {
-      testBtn.addEventListener('click', function() {
-        testConnection('createConnectionForm', 'testResult');
-      });
-    }
-
-    var editTestBtn = document.getElementById('editTestConnectionBtn');
-    if (editTestBtn) {
-      editTestBtn.addEventListener('click', function() {
-        testConnection('editConnectionForm', 'editTestResult');
-      });
-    }
-
-    var editModal = document.getElementById('editConnectionModal');
-    if (editModal) {
-      editModal.addEventListener('show.bs.modal', function(event) {
-        var button = event.relatedTarget;
-        if (!button) return;
-
-        var id = button.getAttribute('data-id');
-
-        document.getElementById('edit_name').value = button.getAttribute('data-name') || '';
-        document.getElementById('edit_endpoint_url').value = button.getAttribute('data-endpoint') || '';
-        document.getElementById('edit_region').value = button.getAttribute('data-region') || '';
-        document.getElementById('edit_access_key').value = button.getAttribute('data-access') || '';
-        document.getElementById('edit_secret_key').value = '';
-        document.getElementById('edit_secret_key').placeholder = '(unchanged — leave blank to keep current)';
-        document.getElementById('edit_secret_key').required = false;
-        document.getElementById('editTestResult').innerHTML = '';
-
-        var form = document.getElementById('editConnectionForm');
-        form.action = endpoints.updateTemplate.replace('CONNECTION_ID', id);
-      });
-    }
-
-    var deleteModal = document.getElementById('deleteConnectionModal');
-    if (deleteModal) {
-      deleteModal.addEventListener('show.bs.modal', function(event) {
-        var button = event.relatedTarget;
-        if (!button) return;
-
-        var id = button.getAttribute('data-id');
-        var name = button.getAttribute('data-name');
-
-        document.getElementById('deleteConnectionName').textContent = name;
-        var form = document.getElementById('deleteConnectionForm');
-        form.action = endpoints.deleteTemplate.replace('CONNECTION_ID', id);
-      });
-    }
-
-    var createForm = document.getElementById('createConnectionForm');
-    if (createForm) {
-      createForm.addEventListener('submit', function(e) {
-        e.preventDefault();
-        window.UICore.submitFormAjax(createForm, {
-          successMessage: 'Connection created',
-          onSuccess: function(data) {
-            createForm.reset();
-            document.getElementById('testResult').innerHTML = '';
-
-            if (data.connection) {
-              var emptyState = document.querySelector('.empty-state');
-              if (emptyState) {
-                var cardBody = emptyState.closest('.card-body');
-                if (cardBody) {
-                  cardBody.innerHTML = '<div class="table-responsive"><table class="table table-hover align-middle mb-0">' +
-                    '<thead class="table-light"><tr>' +
-                    '<th scope="col" style="width: 50px;">Status</th>' +
-                    '<th scope="col">Name</th><th scope="col">Endpoint</th>' +
-                    '<th scope="col">Region</th><th scope="col">Access Key</th>' +
-                    '<th scope="col" class="text-end">Actions</th></tr></thead>' +
-                    '<tbody></tbody></table></div>';
-                }
-              }
-
-              var tbody = document.querySelector('table tbody');
-              if (tbody) {
-                tbody.insertAdjacentHTML('beforeend', createConnectionRowHtml(data.connection));
-                var newRow = tbody.lastElementChild;
-                var statusEl = newRow.querySelector('.connection-status');
-                if (statusEl) {
-                  checkConnectionHealth(data.connection.id, statusEl);
-                }
-              }
-              updateConnectionCount();
-            } else {
-              location.reload();
-            }
-          }
-        });
-      });
-    }
-
-    var editForm = document.getElementById('editConnectionForm');
-    if (editForm) {
-      editForm.addEventListener('submit', function(e) {
-        e.preventDefault();
-        window.UICore.submitFormAjax(editForm, {
-          successMessage: 'Connection updated',
-          onSuccess: function(data) {
-            var modal = bootstrap.Modal.getInstance(document.getElementById('editConnectionModal'));
-            if (modal) modal.hide();
-
-            var connId = editForm.action.split('/').slice(-2)[0];
-            var row = document.querySelector('tr[data-connection-id="' + connId + '"]');
-            if (row && data.connection) {
-              var nameCell = row.querySelector('.fw-medium');
-              if (nameCell) nameCell.textContent = data.connection.name;
-
-              var endpointCell = row.querySelector('.text-truncate');
-              if (endpointCell) {
-                endpointCell.textContent = data.connection.endpoint_url;
-                endpointCell.title = data.connection.endpoint_url;
-              }
-
-              var regionBadge = row.querySelector('.badge.bg-primary');
-              if (regionBadge) regionBadge.textContent = data.connection.region;
-
-              var accessCode = row.querySelector('code.small');
-              if (accessCode && data.connection.access_key) {
-                var ak = data.connection.access_key;
-                accessCode.textContent = ak.slice(0, 8) + '...' + ak.slice(-4);
-              }
-
-              var editBtn = row.querySelector('[data-bs-target="#editConnectionModal"]');
-              if (editBtn) {
-                editBtn.setAttribute('data-name', data.connection.name);
-                editBtn.setAttribute('data-endpoint', data.connection.endpoint_url);
-                editBtn.setAttribute('data-region', data.connection.region);
-                editBtn.setAttribute('data-access', data.connection.access_key);
-              }
-
-              var deleteBtn = row.querySelector('[data-bs-target="#deleteConnectionModal"]');
-              if (deleteBtn) {
-                deleteBtn.setAttribute('data-name', data.connection.name);
-              }
-
-              var statusEl = row.querySelector('.connection-status');
-              if (statusEl) {
-                checkConnectionHealth(connId, statusEl);
-              }
-            }
-          }
-        });
-      });
-    }
-
-    var deleteForm = document.getElementById('deleteConnectionForm');
-    if (deleteForm) {
-      deleteForm.addEventListener('submit', function(e) {
-        e.preventDefault();
-        window.UICore.submitFormAjax(deleteForm, {
-          successMessage: 'Connection deleted',
-          onSuccess: function(data) {
-            var modal = bootstrap.Modal.getInstance(document.getElementById('deleteConnectionModal'));
-            if (modal) modal.hide();
-
-            var connId = deleteForm.action.split('/').slice(-2)[0];
-            var row = document.querySelector('tr[data-connection-id="' + connId + '"]');
-            if (row) {
-              row.remove();
-            }
-
-            updateConnectionCount();
-
-            if (document.querySelectorAll('tr[data-connection-id]').length === 0) {
-              location.reload();
-            }
-          }
-        });
-      });
-    }
-  }
-
-  return {
-    init: init,
-    togglePassword: togglePassword,
-    testConnection: testConnection,
-    checkConnectionHealth: checkConnectionHealth
-  };
-})();

python/templates/replication_wizard.html

@@ -1,270 +0,0 @@
-{% extends "base.html" %}
-
-{% block title %}Set Up Replication - S3 Compatible Storage{% endblock %}
-
-{% block content %}
-<div class="page-header d-flex justify-content-between align-items-center mb-4">
-    <div>
-        <nav aria-label="breadcrumb">
-            <ol class="breadcrumb mb-1">
-                <li class="breadcrumb-item"><a href="{{ url_for('ui.sites_dashboard') }}">Sites</a></li>
-                <li class="breadcrumb-item active" aria-current="page">Replication Wizard</li>
-            </ol>
-        </nav>
-        <h1 class="h3 mb-1 d-flex align-items-center gap-2">
-            <svg xmlns="http://www.w3.org/2000/svg" width="28" height="28" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-                <path d="M8 4a.5.5 0 0 1 .5.5V6a.5.5 0 0 1-1 0V4.5A.5.5 0 0 1 8 4zM3.732 5.732a.5.5 0 0 1 .707 0l.915.914a.5.5 0 1 1-.708.708l-.914-.915a.5.5 0 0 1 0-.707zM2 10a.5.5 0 0 1 .5-.5h1.586a.5.5 0 0 1 0 1H2.5A.5.5 0 0 1 2 10zm9.5 0a.5.5 0 0 1 .5-.5h1.5a.5.5 0 0 1 0 1H12a.5.5 0 0 1-.5-.5zm.754-4.246a.389.389 0 0 0-.527-.02L7.547 9.31a.91.91 0 1 0 1.302 1.258l3.434-4.297a.389.389 0 0 0-.029-.518z"/>
-                <path fill-rule="evenodd" d="M0 10a8 8 0 1 1 15.547 2.661c-.442 1.253-1.845 1.602-2.932 1.25C11.309 13.488 9.475 13 8 13c-1.474 0-3.31.488-4.615.911-1.087.352-2.49.003-2.932-1.25A7.988 7.988 0 0 1 0 10zm8-7a7 7 0 0 0-6.603 9.329c.203.575.923.876 1.68.63C4.397 12.533 6.358 12 8 12s3.604.532 4.923.96c.757.245 1.477-.056 1.68-.631A7 7 0 0 0 8 3z"/>
-            </svg>
-            Set Up Replication
-        </h1>
-        <p class="text-muted mb-0 mt-1">Configure bucket replication to <strong>{{ peer.display_name or peer.site_id }}</strong></p>
-    </div>
-</div>
-
-<div class="row g-4">
-    <div class="col-lg-4 col-md-5">
-        <div class="card shadow-sm border-0 mb-4" style="border-radius: 1rem;">
-            <div class="card-header bg-transparent border-0 pt-4 pb-0 px-4">
-                <h5 class="fw-semibold d-flex align-items-center gap-2 mb-1">
-                    <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-                        <path d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8m7.5-6.923c-.67.204-1.335.82-1.887 1.855A8 8 0 0 0 5.145 4H7.5zM4.09 4a9.3 9.3 0 0 1 .64-1.539 7 7 0 0 1 .597-.933A7.03 7.03 0 0 0 2.255 4zm-.582 3.5c.03-.877.138-1.718.312-2.5H1.674a7 7 0 0 0-.656 2.5zM4.847 5a12.5 12.5 0 0 0-.338 2.5H7.5V5zM8.5 5v2.5h2.99a12.5 12.5 0 0 0-.337-2.5zM4.51 8.5a12.5 12.5 0 0 0 .337 2.5H7.5V8.5zm3.99 0V11h2.653c.187-.765.306-1.608.338-2.5zM5.145 12q.208.58.468 1.068c.552 1.035 1.218 1.65 1.887 1.855V12zm.182 2.472a7 7 0 0 1-.597-.933A9.3 9.3 0 0 1 4.09 12H2.255a7 7 0 0 0 3.072 2.472M3.82 11a13.7 13.7 0 0 1-.312-2.5h-2.49a7 7 0 0 0 .656 2.5zM8.5 12v2.923c.67-.204 1.335-.82 1.887-1.855q.26-.487.468-1.068zm3.68-1h2.146c.365-.767.594-1.61.656-2.5h-2.49a13.7 13.7 0 0 1-.312 2.5m2.802-3.5a7 7 0 0 0-.656-2.5H12.18c.174.782.282 1.623.312 2.5zM11.27 2.461c.247.464.462.98.64 1.539h1.835a7 7 0 0 0-3.072-2.472c.218.284.418.598.597.933M10.855 4a8 8 0 0 0-.468-1.068C9.835 1.897 9.17 1.282 8.5 1.077V4z"/>
-                    </svg>
-                    Peer Site
-                </h5>
-            </div>
-            <div class="card-body px-4 pb-4">
-                <dl class="mb-0">
-                    <dt class="text-muted small">Site ID</dt>
-                    <dd class="mb-2">{{ peer.site_id }}</dd>
-                    <dt class="text-muted small">Endpoint</dt>
-                    <dd class="mb-2 text-truncate" title="{{ peer.endpoint }}">{{ peer.endpoint }}</dd>
-                    <dt class="text-muted small">Region</dt>
-                    <dd class="mb-2"><span class="badge bg-primary bg-opacity-10 text-primary">{{ peer.region }}</span></dd>
-                    <dt class="text-muted small">Connection</dt>
-                    <dd class="mb-0"><span class="badge bg-secondary bg-opacity-10 text-secondary">{{ connection.name }}</span></dd>
-                </dl>
-            </div>
-        </div>
-
-        <div class="card shadow-sm border-0" style="border-radius: 1rem;">
-            <div class="card-header bg-transparent border-0 pt-4 pb-0 px-4">
-                <h5 class="fw-semibold d-flex align-items-center gap-2 mb-1">
-                    <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-muted" viewBox="0 0 16 16">
-                        <path d="M8 16A8 8 0 1 0 8 0a8 8 0 0 0 0 16zm.93-9.412-1 4.705c-.07.34.029.533.304.533.194 0 .487-.07.686-.246l-.088.416c-.287.346-.92.598-1.465.598-.703 0-1.002-.422-.808-1.319l.738-3.468c.064-.293.006-.399-.287-.47l-.451-.081.082-.381 2.29-.287zM8 5.5a1 1 0 1 1 0-2 1 1 0 0 1 0 2z"/>
-                    </svg>
-                    Replication Modes
-                </h5>
-            </div>
-            <div class="card-body px-4 pb-4 small">
-                <p class="mb-2"><strong>New Only:</strong> Only replicate new objects uploaded after the rule is created.</p>
-                <p class="mb-2"><strong>All Objects:</strong> Replicate all existing objects plus new uploads.</p>
-                <p class="mb-0"><strong>Bidirectional:</strong> Two-way sync between sites. Changes on either side are synchronized.</p>
-            </div>
-        </div>
-    </div>
-
-    <div class="col-lg-8 col-md-7">
-        <div class="card shadow-sm border-0" style="border-radius: 1rem;">
-            <div class="card-header bg-transparent border-0 pt-4 pb-0 px-4">
-                <h5 class="fw-semibold d-flex align-items-center gap-2 mb-1">
-                    <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-muted" viewBox="0 0 16 16">
-                        <path d="M2.522 5H2a.5.5 0 0 0-.494.574l1.372 9.149A1.5 1.5 0 0 0 4.36 16h7.278a1.5 1.5 0 0 0 1.483-1.277l1.373-9.149A.5.5 0 0 0 14 5h-.522A5.5 5.5 0 0 0 2.522 5zm1.005 0a4.5 4.5 0 0 1 8.945 0H3.527z"/>
-                    </svg>
-                    Select Buckets to Replicate
-                </h5>
-                <p class="text-muted small mb-0">Choose which buckets should be replicated to this peer site</p>
-            </div>
-            <div class="card-body px-4 pb-4">
-                {% if buckets %}
-                <form method="POST" action="{{ url_for('ui.create_peer_replication_rules', site_id=peer.site_id) }}">
-                    <input type="hidden" name="csrf_token" value="{{ csrf_token() }}"/>
-
-                    <div class="mb-4">
-                        <label for="mode" class="form-label fw-medium">Replication Mode</label>
-                        <select class="form-select" id="mode" name="mode">
-                            <option value="new_only">New Objects Only</option>
-                            <option value="all">All Objects (includes existing)</option>
-                            <option value="bidirectional">Bidirectional Sync</option>
-                        </select>
-                    </div>
-
-                    <div id="bidirWarning" class="alert alert-warning d-none mb-4" role="alert">
-                        <h6 class="alert-heading fw-bold d-flex align-items-center gap-2">
-                            <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" viewBox="0 0 16 16">
-                                <path fill-rule="evenodd" d="M1 11.5a.5.5 0 0 0 .5.5h11.793l-3.147 3.146a.5.5 0 0 0 .708.708l4-4a.5.5 0 0 0 0-.708l-4-4a.5.5 0 0 0-.708.708L13.293 11H1.5a.5.5 0 0 0-.5.5zm14-7a.5.5 0 0 1-.5.5H2.707l3.147 3.146a.5.5 0 1 1-.708.708l-4-4a.5.5 0 0 1 0-.708l4-4a.5.5 0 1 1 .708.708L2.707 4H14.5a.5.5 0 0 1 .5.5z"/>
-                            </svg>
-                            Bidirectional Sync Requires Configuration on Both Sites
-                        </h6>
-                        <p class="mb-2">For bidirectional sync to work properly, you must configure <strong>both</strong> sites. This wizard only configures one direction.</p>
-                        <hr class="my-2">
-                        <p class="mb-2 fw-semibold">After completing this wizard, you must also:</p>
-                        <ol class="mb-2 ps-3">
-                            <li>Go to <strong>{{ peer.display_name or peer.site_id }}</strong>'s admin UI</li>
-                            <li>Register <strong>this site</strong> as a peer (with a connection)</li>
-                            <li>Create matching bidirectional replication rules pointing back to this site</li>
-                            <li>Ensure <code>SITE_SYNC_ENABLED=true</code> is set on both sites</li>
-                        </ol>
-                        <div class="d-flex align-items-center gap-2 mt-3">
-                            <span class="badge bg-light text-dark border">Local Site ID: <strong>{{ local_site.site_id if local_site else 'Not configured' }}</strong></span>
-                            <span class="badge bg-light text-dark border">Local Endpoint: <strong>{{ local_site.endpoint if local_site and local_site.endpoint else 'Not configured' }}</strong></span>
-                        </div>
-                        {% if not local_site or not local_site.site_id or not local_site.endpoint %}
-                        <div class="alert alert-danger mt-3 mb-0 py-2">
-                            <small><strong>Warning:</strong> Your local site identity is not fully configured. The remote site won't be able to connect back. <a href="{{ url_for('ui.sites_dashboard') }}">Configure it now</a>.</small>
-                        </div>
-                        {% endif %}
-                    </div>
-
-                    <div class="table-responsive">
-                        <table class="table table-hover align-middle mb-0">
-                            <thead class="table-light">
-                                <tr>
-                                    <th scope="col" style="width: 40px;">
-                                        <input type="checkbox" class="form-check-input" id="selectAll">
-                                    </th>
-                                    <th scope="col">Local Bucket</th>
-                                    <th scope="col">Target Bucket Name</th>
-                                    <th scope="col">Status</th>
-                                </tr>
-                            </thead>
-                            <tbody>
-                                {% for bucket in buckets %}
-                                <tr>
-                                    <td>
-                                        <input type="checkbox" class="form-check-input bucket-checkbox"
-                                            name="buckets" value="{{ bucket.name }}"
-                                            {% if bucket.has_rule %}disabled{% endif %}>
-                                    </td>
-                                    <td>
-                                        <div class="d-flex align-items-center gap-2">
-                                            <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-muted" viewBox="0 0 16 16">
-                                                <path d="M2.522 5H2a.5.5 0 0 0-.494.574l1.372 9.149A1.5 1.5 0 0 0 4.36 16h7.278a1.5 1.5 0 0 0 1.483-1.277l1.373-9.149A.5.5 0 0 0 14 5h-.522A5.5 5.5 0 0 0 2.522 5zm1.005 0a4.5 4.5 0 0 1 8.945 0H3.527z"/>
-                                            </svg>
-                                            <span class="fw-medium">{{ bucket.name }}</span>
-                                        </div>
-                                    </td>
-                                    <td>
-                                        <input type="text" class="form-control form-control-sm"
-                                            name="target_{{ bucket.name }}"
-                                            value="{{ bucket.existing_target or bucket.name }}"
-                                            placeholder="{{ bucket.name }}"
-                                            {% if bucket.has_rule %}disabled{% endif %}>
-                                    </td>
-                                    <td>
-                                        {% if bucket.has_rule %}
-                                        <span class="badge bg-info bg-opacity-10 text-info">
-                                            Already configured ({{ bucket.existing_mode }})
-                                        </span>
-                                        {% else %}
-                                        <span class="badge bg-secondary bg-opacity-10 text-secondary">
-                                            Not configured
-                                        </span>
-                                        {% endif %}
-                                    </td>
-                                </tr>
-                                {% endfor %}
-                            </tbody>
-                        </table>
-                    </div>
-
-                    <div class="d-flex gap-2 mt-4 pt-3 border-top">
-                        <button type="submit" class="btn btn-primary" id="submitBtn" disabled>
-                            <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1" viewBox="0 0 16 16">
-                                <path d="M10.97 4.97a.75.75 0 0 1 1.07 1.05l-3.99 4.99a.75.75 0 0 1-1.08.02L4.324 8.384a.75.75 0 1 1 1.06-1.06l2.094 2.093 3.473-4.425a.267.267 0 0 1 .02-.022z"/>
-                            </svg>
-                            Create Replication Rules
-                        </button>
-                        <a href="{{ url_for('ui.sites_dashboard') }}" class="btn btn-outline-secondary">
-                            Skip for Now
-                        </a>
-                    </div>
-                </form>
-                {% else %}
-                <div class="empty-state text-center py-5">
-                    <div class="empty-state-icon mx-auto mb-3">
-                        <svg xmlns="http://www.w3.org/2000/svg" width="48" height="48" fill="currentColor" viewBox="0 0 16 16">
-                            <path d="M2.522 5H2a.5.5 0 0 0-.494.574l1.372 9.149A1.5 1.5 0 0 0 4.36 16h7.278a1.5 1.5 0 0 0 1.483-1.277l1.373-9.149A.5.5 0 0 0 14 5h-.522A5.5 5.5 0 0 0 2.522 5zm1.005 0a4.5 4.5 0 0 1 8.945 0H3.527z"/>
-                        </svg>
-                    </div>
-                    <h5 class="fw-semibold mb-2">No buckets yet</h5>
-                    <p class="text-muted mb-3">Create some buckets first, then come back to set up replication.</p>
-                    <a href="{{ url_for('ui.buckets_overview') }}" class="btn btn-primary">
-                        Go to Buckets
-                    </a>
-                </div>
-                {% endif %}
-            </div>
-        </div>
-    </div>
-</div>
-
-<script>
-(function() {
-    const selectAllCheckbox = document.getElementById('selectAll');
-    const bucketCheckboxes = document.querySelectorAll('.bucket-checkbox:not(:disabled)');
-    const submitBtn = document.getElementById('submitBtn');
-    const modeSelect = document.getElementById('mode');
-    const bidirWarning = document.getElementById('bidirWarning');
-
-    function updateBidirWarning() {
-        if (modeSelect && bidirWarning) {
-            if (modeSelect.value === 'bidirectional') {
-                bidirWarning.classList.remove('d-none');
-            } else {
-                bidirWarning.classList.add('d-none');
-            }
-        }
-    }
-
-    if (modeSelect) {
-        modeSelect.addEventListener('change', updateBidirWarning);
-        updateBidirWarning();
-    }
-
-    function updateSubmitButton() {
-        const checkedCount = document.querySelectorAll('.bucket-checkbox:checked').length;
-        if (submitBtn) {
-            submitBtn.disabled = checkedCount === 0;
-            const text = checkedCount > 0
-                ? `Create ${checkedCount} Replication Rule${checkedCount > 1 ? 's' : ''}`
-                : 'Create Replication Rules';
-            submitBtn.innerHTML = `
-                <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1" viewBox="0 0 16 16">
-                    <path d="M10.97 4.97a.75.75 0 0 1 1.07 1.05l-3.99 4.99a.75.75 0 0 1-1.08.02L4.324 8.384a.75.75 0 1 1 1.06-1.06l2.094 2.093 3.473-4.425a.267.267 0 0 1 .02-.022z"/>
-                </svg>
-                ${text}
-            `;
-        }
-    }
-
-    function updateSelectAll() {
-        if (selectAllCheckbox && bucketCheckboxes.length > 0) {
-            const allChecked = Array.from(bucketCheckboxes).every(cb => cb.checked);
-            const someChecked = Array.from(bucketCheckboxes).some(cb => cb.checked);
-            selectAllCheckbox.checked = allChecked;
-            selectAllCheckbox.indeterminate = someChecked && !allChecked;
-        }
-    }
-
-    if (selectAllCheckbox) {
-        selectAllCheckbox.addEventListener('change', function() {
-            bucketCheckboxes.forEach(cb => {
-                cb.checked = this.checked;
-            });
-            updateSubmitButton();
-        });
-    }
-
-    bucketCheckboxes.forEach(cb => {
-        cb.addEventListener('change', function() {
-            updateSelectAll();
-            updateSubmitButton();
-        });
-    });
-
-    updateSelectAll();
-    updateSubmitButton();
-})();
-</script>
-{% endblock %}

python/templates/sites.html

@@ -1,891 +0,0 @@
-{% extends "base.html" %}
-
-{% block title %}Sites - S3 Compatible Storage{% endblock %}
-
-{% block content %}
-<div class="page-header d-flex justify-content-between align-items-center mb-4">
-  <div>
-    <p class="text-uppercase text-muted small mb-1">Geo-Distribution</p>
-    <h1 class="h3 mb-1 d-flex align-items-center gap-2">
-      <svg xmlns="http://www.w3.org/2000/svg" width="28" height="28" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-        <path d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8zm7.5-6.923c-.67.204-1.335.82-1.887 1.855A7.97 7.97 0 0 0 5.145 4H7.5V1.077zM4.09 4a9.267 9.267 0 0 1 .64-1.539 6.7 6.7 0 0 1 .597-.933A7.025 7.025 0 0 0 2.255 4H4.09zm-.582 3.5c.03-.877.138-1.718.312-2.5H1.674a6.958 6.958 0 0 0-.656 2.5h2.49zM4.847 5a12.5 12.5 0 0 0-.338 2.5H7.5V5H4.847zM8.5 5v2.5h2.99a12.495 12.495 0 0 0-.337-2.5H8.5zM4.51 8.5a12.5 12.5 0 0 0 .337 2.5H7.5V8.5H4.51zm3.99 0V11h2.653c.187-.765.306-1.608.338-2.5H8.5zM5.145 12c.138.386.295.744.468 1.068.552 1.035 1.218 1.65 1.887 1.855V12H5.145zm.182 2.472a6.696 6.696 0 0 1-.597-.933A9.268 9.268 0 0 1 4.09 12H2.255a7.024 7.024 0 0 0 3.072 2.472zM3.82 11a13.652 13.652 0 0 1-.312-2.5h-2.49c.062.89.291 1.733.656 2.5H3.82zm6.853 3.472A7.024 7.024 0 0 0 13.745 12H11.91a9.27 9.27 0 0 1-.64 1.539 6.688 6.688 0 0 1-.597.933zM8.5 12v2.923c.67-.204 1.335-.82 1.887-1.855.173-.324.33-.682.468-1.068H8.5zm3.68-1h2.146c.365-.767.594-1.61.656-2.5h-2.49a13.65 13.65 0 0 1-.312 2.5zm2.802-3.5a6.959 6.959 0 0 0-.656-2.5H12.18c.174.782.282 1.623.312 2.5h2.49zM11.27 2.461c.247.464.462.98.64 1.539h1.835a7.024 7.024 0 0 0-3.072-2.472c.218.284.418.598.597.933zM10.855 4a7.966 7.966 0 0 0-.468-1.068C9.835 1.897 9.17 1.282 8.5 1.077V4h2.355z"/>
-      </svg>
-      Site Registry
-    </h1>
-    <p class="text-muted mb-0 mt-1">Configure this site's identity and manage peer sites for geo-distribution.</p>
-  </div>
-  <div class="d-none d-md-flex align-items-center gap-2">
-    {% if local_site and local_site.site_id %}
-    <span class="badge bg-secondary bg-opacity-10 text-secondary fs-6 px-3 py-2">
-      {{ local_site.site_id }}
-    </span>
-    <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-muted" viewBox="0 0 16 16">
-      <path fill-rule="evenodd" d="M1 8a.5.5 0 0 1 .5-.5h11.793l-3.147-3.146a.5.5 0 0 1 .708-.708l4 4a.5.5 0 0 1 0 .708l-4 4a.5.5 0 0 1-.708-.708L13.293 8.5H1.5A.5.5 0 0 1 1 8z"/>
-    </svg>
-    {% endif %}
-    <span class="badge bg-primary bg-opacity-10 text-primary fs-6 px-3 py-2">
-      {{ peers|length }} peer{{ 's' if peers|length != 1 else '' }}
-    </span>
-  </div>
-</div>
-
-<div class="row g-4">
-    <div class="col-lg-4 col-md-5">
-        <div class="card shadow-sm border-0 mb-4" style="border-radius: 1rem;">
-            <div class="card-header bg-transparent border-0 pt-4 pb-0 px-4">
-                <h5 class="fw-semibold d-flex align-items-center gap-2 mb-1">
-                    <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-                        <path d="M8 16s6-5.686 6-10A6 6 0 0 0 2 6c0 4.314 6 10 6 10zm0-7a3 3 0 1 1 0-6 3 3 0 0 1 0 6z"/>
-                    </svg>
-                    Local Site Identity
-                </h5>
-                <p class="text-muted small mb-0">This site's configuration</p>
-            </div>
-            <div class="card-body px-4 pb-4">
-                <form method="POST" action="{{ url_for('ui.update_local_site') }}" id="localSiteForm">
-                    <input type="hidden" name="csrf_token" value="{{ csrf_token() }}"/>
-                    <div class="mb-3">
-                        <label for="site_id" class="form-label fw-medium">Site ID</label>
-                        <input type="text" class="form-control" id="site_id" name="site_id" required
-                            value="{{ local_site.site_id if local_site else config_site_id or '' }}"
-                            placeholder="us-west-1">
-                        <div class="form-text">Unique identifier for this site</div>
-                    </div>
-                    <div class="mb-3">
-                        <label for="endpoint" class="form-label fw-medium">Endpoint URL</label>
-                        <input type="url" class="form-control" id="endpoint" name="endpoint"
-                            value="{{ local_site.endpoint if local_site else config_site_endpoint or '' }}"
-                            placeholder="https://s3.us-west-1.example.com">
-                        <div class="form-text">Public URL for this site</div>
-                    </div>
-                    <div class="mb-3">
-                        <label for="region" class="form-label fw-medium">Region</label>
-                        <input type="text" class="form-control" id="region" name="region"
-                            value="{{ local_site.region if local_site else config_site_region }}">
-                    </div>
-                    <div class="row mb-3">
-                        <div class="col-6">
-                            <label for="priority" class="form-label fw-medium">Priority</label>
-                            <input type="number" class="form-control" id="priority" name="priority"
-                                value="{{ local_site.priority if local_site else 100 }}" min="0">
-                            <div class="form-text">Lower = preferred</div>
-                        </div>
-                        <div class="col-6">
-                            <label for="display_name" class="form-label fw-medium">Display Name</label>
-                            <input type="text" class="form-control" id="display_name" name="display_name"
-                                value="{{ local_site.display_name if local_site else '' }}"
-                                placeholder="US West Primary">
-                        </div>
-                    </div>
-                    <div class="d-grid">
-                        <button type="submit" class="btn btn-primary">
-                            <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1" viewBox="0 0 16 16">
-                                <path d="M10.97 4.97a.75.75 0 0 1 1.07 1.05l-3.99 4.99a.75.75 0 0 1-1.08.02L4.324 8.384a.75.75 0 1 1 1.06-1.06l2.094 2.093 3.473-4.425a.267.267 0 0 1 .02-.022z"/>
-                            </svg>
-                            Save Local Site
-                        </button>
-                    </div>
-                </form>
-            </div>
-        </div>
-
-        <div class="card shadow-sm border-0" style="border-radius: 1rem;">
-            <div class="card-header bg-transparent border-0 pt-3 pb-0 px-4">
-                <button class="btn btn-link text-decoration-none p-0 w-100 d-flex align-items-center justify-content-between"
-                    type="button" data-bs-toggle="collapse" data-bs-target="#addPeerCollapse"
-                    aria-expanded="false" aria-controls="addPeerCollapse">
-                    <span class="d-flex align-items-center gap-2">
-                        <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-                            <path fill-rule="evenodd" d="M8 2a.5.5 0 0 1 .5.5v5h5a.5.5 0 0 1 0 1h-5v5a.5.5 0 0 1-1 0v-5h-5a.5.5 0 0 1 0-1h5v-5A.5.5 0 0 1 8 2Z"/>
-                        </svg>
-                        <span class="fw-semibold h5 mb-0">Add Peer Site</span>
-                    </span>
-                    <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-muted add-peer-chevron" viewBox="0 0 16 16">
-                        <path fill-rule="evenodd" d="M1.646 4.646a.5.5 0 0 1 .708 0L8 10.293l5.646-5.647a.5.5 0 0 1 .708.708l-6 6a.5.5 0 0 1-.708 0l-6-6a.5.5 0 0 1 0-.708z"/>
-                    </svg>
-                </button>
-                <p class="text-muted small mb-0 mt-1">Register a remote site</p>
-            </div>
-            <div class="collapse" id="addPeerCollapse">
-                <div class="card-body px-4 pb-4">
-                    <form method="POST" action="{{ url_for('ui.add_peer_site') }}" id="addPeerForm">
-                        <input type="hidden" name="csrf_token" value="{{ csrf_token() }}"/>
-                        <div class="mb-3">
-                            <label for="peer_site_id" class="form-label fw-medium">Site ID</label>
-                            <input type="text" class="form-control" id="peer_site_id" name="site_id" required placeholder="us-east-1">
-                        </div>
-                        <div class="mb-3">
-                            <label for="peer_endpoint" class="form-label fw-medium">Endpoint URL</label>
-                            <input type="url" class="form-control" id="peer_endpoint" name="endpoint" required placeholder="https://s3.us-east-1.example.com">
-                        </div>
-                        <div class="mb-3">
-                            <label for="peer_region" class="form-label fw-medium">Region</label>
-                            <input type="text" class="form-control" id="peer_region" name="region" value="us-east-1">
-                        </div>
-                        <div class="row mb-3">
-                            <div class="col-6">
-                                <label for="peer_priority" class="form-label fw-medium">Priority</label>
-                                <input type="number" class="form-control" id="peer_priority" name="priority" value="100" min="0">
-                            </div>
-                            <div class="col-6">
-                                <label for="peer_display_name" class="form-label fw-medium">Display Name</label>
-                                <input type="text" class="form-control" id="peer_display_name" name="display_name" placeholder="US East DR">
-                            </div>
-                        </div>
-                        <div class="mb-3">
-                            <label for="peer_connection_id" class="form-label fw-medium">Connection</label>
-                            <select class="form-select" id="peer_connection_id" name="connection_id">
-                                <option value="">No connection</option>
-                                {% for conn in connections %}
-                                <option value="{{ conn.id }}">{{ conn.name }} ({{ conn.endpoint_url }})</option>
-                                {% endfor %}
-                            </select>
-                            <div class="form-text">Link to a remote connection for health checks</div>
-                        </div>
-                        <div class="d-grid">
-                            <button type="submit" class="btn btn-primary">
-                                <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1" viewBox="0 0 16 16">
-                                    <path fill-rule="evenodd" d="M8 2a.5.5 0 0 1 .5.5v5h5a.5.5 0 0 1 0 1h-5v5a.5.5 0 0 1-1 0v-5h-5a.5.5 0 0 1 0-1h5v-5A.5.5 0 0 1 8 2Z"/>
-                                </svg>
-                                Add Peer Site
-                            </button>
-                        </div>
-                    </form>
-                </div>
-            </div>
-        </div>
-    </div>
-
-    <div class="col-lg-8 col-md-7">
-        <div class="card shadow-sm border-0" style="border-radius: 1rem;">
-            <div class="card-header bg-transparent border-0 pt-4 pb-0 px-4 d-flex justify-content-between align-items-start">
-                <div>
-                    <h5 class="fw-semibold d-flex align-items-center gap-2 mb-1">
-                        <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-muted" viewBox="0 0 16 16">
-                            <path fill-rule="evenodd" d="M6 3.5A1.5 1.5 0 0 1 7.5 2h1A1.5 1.5 0 0 1 10 3.5v1A1.5 1.5 0 0 1 8.5 6v1H14a.5.5 0 0 1 .5.5v1a.5.5 0 0 1-1 0V8h-5v.5a.5.5 0 0 1-1 0V8h-5v.5a.5.5 0 0 1-1 0v-1A.5.5 0 0 1 2 7h5.5V6A1.5 1.5 0 0 1 6 4.5v-1zM8.5 5a.5.5 0 0 0 .5-.5v-1a.5.5 0 0 0-.5-.5h-1a.5.5 0 0 0-.5.5v1a.5.5 0 0 0 .5.5h1zM0 11.5A1.5 1.5 0 0 1 1.5 10h1A1.5 1.5 0 0 1 4 11.5v1A1.5 1.5 0 0 1 2.5 14h-1A1.5 1.5 0 0 1 0 12.5v-1zm1.5-.5a.5.5 0 0 0-.5.5v1a.5.5 0 0 0 .5.5h1a.5.5 0 0 0 .5-.5v-1a.5.5 0 0 0-.5-.5h-1zm4.5.5A1.5 1.5 0 0 1 7.5 10h1a1.5 1.5 0 0 1 1.5 1.5v1A1.5 1.5 0 0 1 8.5 14h-1A1.5 1.5 0 0 1 6 12.5v-1zm1.5-.5a.5.5 0 0 0-.5.5v1a.5.5 0 0 0 .5.5h1a.5.5 0 0 0 .5-.5v-1a.5.5 0 0 0-.5-.5h-1zm4.5.5a1.5 1.5 0 0 1 1.5-1.5h1a1.5 1.5 0 0 1 1.5 1.5v1a1.5 1.5 0 0 1-1.5 1.5h-1a1.5 1.5 0 0 1-1.5-1.5v-1zm1.5-.5a.5.5 0 0 0-.5.5v1a.5.5 0 0 0 .5.5h1a.5.5 0 0 0 .5-.5v-1a.5.5 0 0 0-.5-.5h-1z"/>
-                        </svg>
-                        Peer Sites
-                    </h5>
-                    <p class="text-muted small mb-0">Known remote sites in the cluster</p>
-                </div>
-                {% if peers %}
-                <button type="button" class="btn btn-outline-secondary btn-sm" id="btnCheckAllHealth" title="Check health of all peers">
-                    <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1" viewBox="0 0 16 16">
-                        <path d="M11.251.068a.5.5 0 0 1 .227.58L9.677 6.5H13a.5.5 0 0 1 .364.843l-8 8.5a.5.5 0 0 1-.842-.49L6.323 9.5H3a.5.5 0 0 1-.364-.843l8-8.5a.5.5 0 0 1 .615-.09z"/>
-                    </svg>
-                    Check All
-                </button>
-                {% endif %}
-            </div>
-            <div class="card-body px-4 pb-4">
-                {% if peers %}
-                <div class="table-responsive">
-                    <table class="table table-hover align-middle mb-0">
-                        <thead class="table-light">
-                            <tr>
-                                <th scope="col" style="width: 50px;">Health</th>
-                                <th scope="col">Site ID</th>
-                                <th scope="col">Endpoint</th>
-                                <th scope="col">Region</th>
-                                <th scope="col">Priority</th>
-                                <th scope="col">Sync Status</th>
-                                <th scope="col" class="text-end">Actions</th>
-                            </tr>
-                        </thead>
-                        <tbody>
-                            {% for item in peers_with_stats %}
-                            {% set peer = item.peer %}
-                            <tr data-site-id="{{ peer.site_id }}">
-                                <td class="text-center">
-                                    <span class="peer-health-status" data-site-id="{{ peer.site_id }}"
-                                        data-last-checked="{{ peer.last_health_check or '' }}"
-                                        title="{% if peer.is_healthy == true %}Healthy{% elif peer.is_healthy == false %}Unhealthy{% else %}Not checked{% endif %}{% if peer.last_health_check %} (checked {{ peer.last_health_check }}){% endif %}"
-                                        style="cursor: help;">
-                                        {% if peer.is_healthy == true %}
-                                        <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-success" viewBox="0 0 16 16">
-                                            <path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zm-3.97-3.03a.75.75 0 0 0-1.08.022L7.477 9.417 5.384 7.323a.75.75 0 0 0-1.06 1.06L6.97 11.03a.75.75 0 0 0 1.079-.02l3.992-4.99a.75.75 0 0 0-.01-1.05z"/>
-                                        </svg>
-                                        {% elif peer.is_healthy == false %}
-                                        <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-danger" viewBox="0 0 16 16">
-                                            <path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zM5.354 4.646a.5.5 0 1 0-.708.708L7.293 8l-2.647 2.646a.5.5 0 0 0 .708.708L8 8.707l2.646 2.647a.5.5 0 0 0 .708-.708L8.707 8l2.647-2.646a.5.5 0 0 0-.708-.708L8 7.293 5.354 4.646z"/>
-                                        </svg>
-                                        {% else %}
-                                        <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-muted" viewBox="0 0 16 16">
-                                            <path d="M8 15A7 7 0 1 1 8 1a7 7 0 0 1 0 14zm0 1A8 8 0 1 0 8 0a8 8 0 0 0 0 16z"/>
-                                            <path d="M5.255 5.786a.237.237 0 0 0 .241.247h.825c.138 0 .248-.113.266-.25.09-.656.54-1.134 1.342-1.134.686 0 1.314.343 1.314 1.168 0 .635-.374.927-.965 1.371-.673.489-1.206 1.06-1.168 1.987l.003.217a.25.25 0 0 0 .25.246h.811a.25.25 0 0 0 .25-.25v-.105c0-.718.273-.927 1.01-1.486.609-.463 1.244-.977 1.244-2.056 0-1.511-1.276-2.241-2.673-2.241-1.267 0-2.655.59-2.75 2.286zm1.557 5.763c0 .533.425.927 1.01.927.609 0 1.028-.394 1.028-.927 0-.552-.42-.94-1.029-.94-.584 0-1.009.388-1.009.94z"/>
-                                        </svg>
-                                        {% endif %}
-                                    </span>
-                                </td>
-                                <td>
-                                    <div class="d-flex align-items-center gap-2">
-                                        <div class="peer-icon">
-                                            <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" viewBox="0 0 16 16">
-                                                <path d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8zm7.5-6.923c-.67.204-1.335.82-1.887 1.855A7.97 7.97 0 0 0 5.145 4H7.5V1.077zM4.09 4a9.267 9.267 0 0 1 .64-1.539 6.7 6.7 0 0 1 .597-.933A7.025 7.025 0 0 0 2.255 4H4.09zm-.582 3.5c.03-.877.138-1.718.312-2.5H1.674a6.958 6.958 0 0 0-.656 2.5h2.49zM4.847 5a12.5 12.5 0 0 0-.338 2.5H7.5V5H4.847zM8.5 5v2.5h2.99a12.495 12.495 0 0 0-.337-2.5H8.5zM4.51 8.5a12.5 12.5 0 0 0 .337 2.5H7.5V8.5H4.51zm3.99 0V11h2.653c.187-.765.306-1.608.338-2.5H8.5z"/>
-                                            </svg>
-                                        </div>
-                                        <div>
-                                            <span class="fw-medium">{{ peer.display_name or peer.site_id }}</span>
-                                            {% if peer.display_name and peer.display_name != peer.site_id %}
-                                            <br><small class="text-muted">{{ peer.site_id }}</small>
-                                            {% endif %}
-                                        </div>
-                                    </div>
-                                </td>
-                                <td>
-                                    <span class="endpoint-display text-muted small" data-full-url="{{ peer.endpoint }}" title="{{ peer.endpoint }}" style="cursor: pointer;">
-                                        {% set parsed = peer.endpoint.split('//') %}
-                                        {% if parsed|length > 1 %}{{ parsed[1].split('/')[0] }}{% else %}{{ peer.endpoint }}{% endif %}
-                                    </span>
-                                    <button type="button" class="btn btn-link btn-sm p-0 ms-1 btn-copy-endpoint" data-url="{{ peer.endpoint }}" title="Copy full URL">
-                                        <svg xmlns="http://www.w3.org/2000/svg" width="12" height="12" fill="currentColor" class="text-muted" viewBox="0 0 16 16">
-                                            <path d="M4 1.5H3a2 2 0 0 0-2 2V14a2 2 0 0 0 2 2h10a2 2 0 0 0 2-2V3.5a2 2 0 0 0-2-2h-1v1h1a1 1 0 0 1 1 1V14a1 1 0 0 1-1 1H3a1 1 0 0 1-1-1V3.5a1 1 0 0 1 1-1h1v-1z"/>
-                                            <path d="M9.5 1a.5.5 0 0 1 .5.5v1a.5.5 0 0 1-.5.5h-3a.5.5 0 0 1-.5-.5v-1a.5.5 0 0 1 .5-.5h3zm-3-1A1.5 1.5 0 0 0 5 1.5v1A1.5 1.5 0 0 0 6.5 4h3A1.5 1.5 0 0 0 11 2.5v-1A1.5 1.5 0 0 0 9.5 0h-3z"/>
-                                        </svg>
-                                    </button>
-                                </td>
-                                <td><span class="text-muted small">{{ peer.region }}</span></td>
-                                <td><span class="text-muted small">{{ peer.priority }}</span></td>
-                                <td class="sync-stats-cell" data-site-id="{{ peer.site_id }}">
-                                    {% if item.has_connection %}
-                                    <div class="d-flex align-items-center gap-2">
-                                        <span class="badge bg-primary bg-opacity-10 text-primary">{{ item.buckets_syncing }} bucket{{ 's' if item.buckets_syncing != 1 else '' }}</span>
-                                        {% if item.has_bidirectional %}
-                                        <span class="bidir-status-icon" data-site-id="{{ peer.site_id }}" title="Bidirectional sync - click to verify">
-                                            <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="text-info" viewBox="0 0 16 16" style="cursor: pointer;">
-                                                <path fill-rule="evenodd" d="M1 11.5a.5.5 0 0 0 .5.5h11.793l-3.147 3.146a.5.5 0 0 0 .708.708l4-4a.5.5 0 0 0 0-.708l-4-4a.5.5 0 0 0-.708.708L13.293 11H1.5a.5.5 0 0 0-.5.5zm14-7a.5.5 0 0 1-.5.5H2.707l3.147 3.146a.5.5 0 1 1-.708.708l-4-4a.5.5 0 0 1 0-.708l4-4a.5.5 0 1 1 .708.708L2.707 4H14.5a.5.5 0 0 1 .5.5z"/>
-                                            </svg>
-                                        </span>
-                                        {% endif %}
-                                    </div>
-                                    <div class="sync-stats-detail d-none mt-2 small" id="stats-{{ peer.site_id }}">
-                                        <span class="spinner-border spinner-border-sm text-muted" style="width: 12px; height: 12px;"></span>
-                                    </div>
-                                    {% else %}
-                                    <a href="#" class="text-muted small link-no-connection"
-                                        data-site-id="{{ peer.site_id }}"
-                                        title="Click to link a connection">Link a connection</a>
-                                    {% endif %}
-                                </td>
-                                <td class="text-end">
-                                    <div class="d-flex align-items-center justify-content-end gap-1">
-                                        <button type="button" class="btn btn-outline-secondary btn-sm"
-                                            data-bs-toggle="modal"
-                                            data-bs-target="#editPeerModal"
-                                            data-site-id="{{ peer.site_id }}"
-                                            data-endpoint="{{ peer.endpoint }}"
-                                            data-region="{{ peer.region }}"
-                                            data-priority="{{ peer.priority }}"
-                                            data-display-name="{{ peer.display_name }}"
-                                            data-connection-id="{{ peer.connection_id or '' }}"
-                                            title="Edit peer">
-                                            <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" viewBox="0 0 16 16">
-                                                <path d="M12.146.146a.5.5 0 0 1 .708 0l3 3a.5.5 0 0 1 0 .708l-10 10a.5.5 0 0 1-.168.11l-5 2a.5.5 0 0 1-.65-.65l2-5a.5.5 0 0 1 .11-.168l10-10zM11.207 2.5 13.5 4.793 14.793 3.5 12.5 1.207 11.207 2.5zm1.586 3L10.5 3.207 4 9.707V10h.5a.5.5 0 0 1 .5.5v.5h.5a.5.5 0 0 1 .5.5v.5h.293l6.5-6.5z"/>
-                                            </svg>
-                                        </button>
-                                        <div class="dropdown peer-actions-dropdown">
-                                            <button class="btn btn-outline-secondary btn-sm" type="button" data-bs-toggle="dropdown" aria-expanded="false" title="More actions">
-                                                <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" viewBox="0 0 16 16">
-                                                    <path d="M3 9.5a1.5 1.5 0 1 1 0-3 1.5 1.5 0 0 1 0 3zm5 0a1.5 1.5 0 1 1 0-3 1.5 1.5 0 0 1 0 3zm5 0a1.5 1.5 0 1 1 0-3 1.5 1.5 0 0 1 0 3z"/>
-                                                </svg>
-                                            </button>
-                                            <ul class="dropdown-menu dropdown-menu-end">
-                                                <li>
-                                                    <button type="button" class="dropdown-item btn-check-health" data-site-id="{{ peer.site_id }}">
-                                                        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-2 text-warning" viewBox="0 0 16 16">
-                                                            <path d="M11.251.068a.5.5 0 0 1 .227.58L9.677 6.5H13a.5.5 0 0 1 .364.843l-8 8.5a.5.5 0 0 1-.842-.49L6.323 9.5H3a.5.5 0 0 1-.364-.843l8-8.5a.5.5 0 0 1 .615-.09z"/>
-                                                        </svg>
-                                                        Check Health
-                                                    </button>
-                                                </li>
-                                                <li>
-                                                    <button type="button" class="dropdown-item btn-check-bidir {% if not item.has_connection %}disabled{% endif %}"
-                                                        data-site-id="{{ peer.site_id }}"
-                                                        data-display-name="{{ peer.display_name or peer.site_id }}">
-                                                        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-2 text-info" viewBox="0 0 16 16">
-                                                            <path fill-rule="evenodd" d="M1 11.5a.5.5 0 0 0 .5.5h11.793l-3.147 3.146a.5.5 0 0 0 .708.708l4-4a.5.5 0 0 0 0-.708l-4-4a.5.5 0 0 0-.708.708L13.293 11H1.5a.5.5 0 0 0-.5.5zm14-7a.5.5 0 0 1-.5.5H2.707l3.147 3.146a.5.5 0 1 1-.708.708l-4-4a.5.5 0 0 1 0-.708l4-4a.5.5 0 1 1 .708.708L2.707 4H14.5a.5.5 0 0 1 .5.5z"/>
-                                                        </svg>
-                                                        Bidirectional Status
-                                                    </button>
-                                                </li>
-                                                {% if item.has_connection and item.buckets_syncing > 0 %}
-                                                <li>
-                                                    <button type="button" class="dropdown-item btn-load-stats" data-site-id="{{ peer.site_id }}">
-                                                        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-2 text-primary" viewBox="0 0 16 16">
-                                                            <path fill-rule="evenodd" d="M8 3a5 5 0 1 0 4.546 2.914.5.5 0 0 1 .908-.417A6 6 0 1 1 8 2v1z"/>
-                                                            <path d="M8 4.466V.534a.25.25 0 0 1 .41-.192l2.36 1.966c.12.1.12.284 0 .384L8.41 4.658A.25.25 0 0 1 8 4.466z"/>
-                                                        </svg>
-                                                        Load Sync Stats
-                                                    </button>
-                                                </li>
-                                                {% endif %}
-                                                <li>
-                                                    <a href="{{ url_for('ui.replication_wizard', site_id=peer.site_id) }}"
-                                                        class="dropdown-item {% if not item.has_connection %}disabled{% endif %}">
-                                                        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-2 text-primary" viewBox="0 0 16 16">
-                                                            <path d="M11.534 7h3.932a.25.25 0 0 1 .192.41l-1.966 2.36a.25.25 0 0 1-.384 0l-1.966-2.36a.25.25 0 0 1 .192-.41zm-11 2h3.932a.25.25 0 0 0 .192-.41L2.692 6.23a.25.25 0 0 0-.384 0L.342 8.59A.25.25 0 0 0 .534 9z"/>
-                                                            <path fill-rule="evenodd" d="M8 3c-1.552 0-2.94.707-3.857 1.818a.5.5 0 1 1-.771-.636A6.002 6.002 0 0 1 13.917 7H12.9A5.002 5.002 0 0 0 8 3zM3.1 9a5.002 5.002 0 0 0 8.757 2.182.5.5 0 1 1 .771.636A6.002 6.002 0 0 1 2.083 9H3.1z"/>
-                                                        </svg>
-                                                        Replication Wizard
-                                                    </a>
-                                                </li>
-                                                <li><hr class="dropdown-divider"></li>
-                                                <li>
-                                                    <button type="button" class="dropdown-item text-danger"
-                                                        data-bs-toggle="modal"
-                                                        data-bs-target="#deletePeerModal"
-                                                        data-site-id="{{ peer.site_id }}"
-                                                        data-display-name="{{ peer.display_name or peer.site_id }}">
-                                                        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-2" viewBox="0 0 16 16">
-                                                            <path d="M5.5 5.5A.5.5 0 0 1 6 6v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm2.5 0a.5.5 0 0 1 .5.5v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm3 .5a.5.5 0 0 0-1 0v6a.5.5 0 0 0 1 0V6z"/>
-                                                            <path fill-rule="evenodd" d="M14.5 3a1 1 0 0 1-1 1H13v9a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V4h-.5a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1H6a1 1 0 0 1 1-1h2a1 1 0 0 1 1 1h3.5a1 1 0 0 1 1 1v1zM4.118 4 4 4.059V13a1 1 0 0 0 1 1h6a1 1 0 0 0 1-1V4.059L11.882 4H4.118zM2.5 3V2h11v1h-11z"/>
-                                                        </svg>
-                                                        Delete Peer
-                                                    </button>
-                                                </li>
-                                            </ul>
-                                        </div>
-                                    </div>
-                                </td>
-                            </tr>
-                            {% endfor %}
-                        </tbody>
-                    </table>
-                </div>
-                {% else %}
-                <div class="empty-state text-center py-5">
-                    <div class="empty-state-icon mx-auto mb-3">
-                        <svg xmlns="http://www.w3.org/2000/svg" width="48" height="48" fill="currentColor" viewBox="0 0 16 16">
-                            <path d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8zm7.5-6.923c-.67.204-1.335.82-1.887 1.855A7.97 7.97 0 0 0 5.145 4H7.5V1.077zM4.09 4a9.267 9.267 0 0 1 .64-1.539 6.7 6.7 0 0 1 .597-.933A7.025 7.025 0 0 0 2.255 4H4.09zm-.582 3.5c.03-.877.138-1.718.312-2.5H1.674a6.958 6.958 0 0 0-.656 2.5h2.49zM4.847 5a12.5 12.5 0 0 0-.338 2.5H7.5V5H4.847zM8.5 5v2.5h2.99a12.495 12.495 0 0 0-.337-2.5H8.5zM4.51 8.5a12.5 12.5 0 0 0 .337 2.5H7.5V8.5H4.51zm3.99 0V11h2.653c.187-.765.306-1.608.338-2.5H8.5zM5.145 12c.138.386.295.744.468 1.068.552 1.035 1.218 1.65 1.887 1.855V12H5.145zm.182 2.472a6.696 6.696 0 0 1-.597-.933A9.268 9.268 0 0 1 4.09 12H2.255a7.024 7.024 0 0 0 3.072 2.472z"/>
-                        </svg>
-                    </div>
-                    <h5 class="fw-semibold mb-2">No peer sites yet</h5>
-                    <p class="text-muted mb-0">Add peer sites to enable geo-distribution and site-to-site replication.</p>
-                </div>
-                {% endif %}
-            </div>
-        </div>
-    </div>
-</div>
-
-<div class="modal fade" id="editPeerModal" tabindex="-1" aria-hidden="true">
-    <div class="modal-dialog modal-dialog-centered">
-        <div class="modal-content">
-            <div class="modal-header border-0 pb-0">
-                <h5 class="modal-title fw-semibold">
-                    <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-                        <path d="M12.146.146a.5.5 0 0 1 .708 0l3 3a.5.5 0 0 1 0 .708l-10 10a.5.5 0 0 1-.168.11l-5 2a.5.5 0 0 1-.65-.65l2-5a.5.5 0 0 1 .11-.168l10-10zM11.207 2.5 13.5 4.793 14.793 3.5 12.5 1.207 11.207 2.5zm1.586 3L10.5 3.207 4 9.707V10h.5a.5.5 0 0 1 .5.5v.5h.5a.5.5 0 0 1 .5.5v.5h.293l6.5-6.5zm-9.761 5.175-.106.106-1.528 3.821 3.821-1.528.106-.106A.5.5 0 0 1 5 12.5V12h-.5a.5.5 0 0 1-.5-.5V11h-.5a.5.5 0 0 1-.468-.325z"/>
-                    </svg>
-                    Edit Peer Site
-                </h5>
-                <button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="Close"></button>
-            </div>
-            <form method="POST" id="editPeerForm">
-                <input type="hidden" name="csrf_token" value="{{ csrf_token() }}"/>
-                <div class="modal-body">
-                    <div class="mb-3">
-                        <label class="form-label fw-medium">Site ID</label>
-                        <input type="text" class="form-control" id="edit_site_id" readonly>
-                    </div>
-                    <div class="mb-3">
-                        <label for="edit_endpoint" class="form-label fw-medium">Endpoint URL</label>
-                        <input type="url" class="form-control" id="edit_endpoint" name="endpoint" required>
-                    </div>
-                    <div class="mb-3">
-                        <label for="edit_region" class="form-label fw-medium">Region</label>
-                        <input type="text" class="form-control" id="edit_region" name="region" required>
-                    </div>
-                    <div class="row mb-3">
-                        <div class="col-6">
-                            <label for="edit_priority" class="form-label fw-medium">Priority</label>
-                            <input type="number" class="form-control" id="edit_priority" name="priority" min="0">
-                        </div>
-                        <div class="col-6">
-                            <label for="edit_display_name" class="form-label fw-medium">Display Name</label>
-                            <input type="text" class="form-control" id="edit_display_name" name="display_name">
-                        </div>
-                    </div>
-                    <div class="mb-3">
-                        <label for="edit_connection_id" class="form-label fw-medium">Connection</label>
-                        <select class="form-select" id="edit_connection_id" name="connection_id">
-                            <option value="">No connection</option>
-                            {% for conn in connections %}
-                            <option value="{{ conn.id }}">{{ conn.name }} ({{ conn.endpoint_url }})</option>
-                            {% endfor %}
-                        </select>
-                    </div>
-                </div>
-                <div class="modal-footer">
-                    <button type="button" class="btn btn-outline-secondary" data-bs-dismiss="modal">Cancel</button>
-                    <button type="submit" class="btn btn-primary">
-                        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1" viewBox="0 0 16 16">
-                            <path d="M10.97 4.97a.75.75 0 0 1 1.07 1.05l-3.99 4.99a.75.75 0 0 1-1.08.02L4.324 8.384a.75.75 0 1 1 1.06-1.06l2.094 2.093 3.473-4.425a.267.267 0 0 1 .02-.022z"/>
-                        </svg>
-                        Save
-                    </button>
-                </div>
-            </form>
-        </div>
-    </div>
-</div>
-
-<div class="modal fade" id="deletePeerModal" tabindex="-1" aria-hidden="true">
-    <div class="modal-dialog modal-dialog-centered">
-        <div class="modal-content">
-            <div class="modal-header border-0 pb-0">
-                <h5 class="modal-title fw-semibold">
-                    <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-danger" viewBox="0 0 16 16">
-                        <path d="M5.5 5.5A.5.5 0 0 1 6 6v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm2.5 0a.5.5 0 0 1 .5.5v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm3 .5a.5.5 0 0 0-1 0v6a.5.5 0 0 0 1 0V6z"/>
-                        <path fill-rule="evenodd" d="M14.5 3a1 1 0 0 1-1 1H13v9a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V4h-.5a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1H6a1 1 0 0 1 1-1h2a1 1 0 0 1 1 1h3.5a1 1 0 0 1 1 1v1zM4.118 4 4 4.059V13a1 1 0 0 0 1 1h6a1 1 0 0 0 1-1V4.059L11.882 4H4.118zM2.5 3V2h11v1h-11z"/>
-                    </svg>
-                    Delete Peer Site
-                </h5>
-                <button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="Close"></button>
-            </div>
-            <div class="modal-body">
-                <p>Are you sure you want to delete <strong id="deletePeerName"></strong>?</p>
-                <div class="alert alert-warning d-flex align-items-start small" role="alert">
-                    <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="flex-shrink-0 me-2 mt-0" viewBox="0 0 16 16">
-                        <path d="M8 16A8 8 0 1 0 8 0a8 8 0 0 0 0 16zm.93-9.412-1 4.705c-.07.34.029.533.304.533.194 0 .487-.07.686-.246l-.088.416c-.287.346-.92.598-1.465.598-.703 0-1.002-.422-.808-1.319l.738-3.468c.064-.293.006-.399-.287-.47l-.451-.081.082-.381 2.29-.287zM8 5.5a1 1 0 1 1 0-2 1 1 0 0 1 0 2z"/>
-                    </svg>
-                    <div>This will remove the peer from the site registry. Any site sync configurations may be affected.</div>
-                </div>
-            </div>
-            <div class="modal-footer">
-                <button type="button" class="btn btn-outline-secondary" data-bs-dismiss="modal">Cancel</button>
-                <form method="POST" id="deletePeerForm">
-                    <input type="hidden" name="csrf_token" value="{{ csrf_token() }}"/>
-                    <button type="submit" class="btn btn-danger">
-                        <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1" viewBox="0 0 16 16">
-                            <path d="M5.5 5.5A.5.5 0 0 1 6 6v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm2.5 0a.5.5 0 0 1 .5.5v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm3 .5a.5.5 0 0 0-1 0v6a.5.5 0 0 0 1 0V6z"/>
-                            <path fill-rule="evenodd" d="M14.5 3a1 1 0 0 1-1 1H13v9a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V4h-.5a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1H6a1 1 0 0 1 1-1h2a1 1 0 0 1 1 1h3.5a1 1 0 0 1 1 1v1zM4.118 4 4 4.059V13a1 1 0 0 0 1 1h6a1 1 0 0 0 1-1V4.059L11.882 4H4.118zM2.5 3V2h11v1h-11z"/>
-                        </svg>
-                        Delete
-                    </button>
-                </form>
-            </div>
-        </div>
-    </div>
-</div>
-
-<div class="modal fade" id="bidirStatusModal" tabindex="-1" aria-hidden="true">
-    <div class="modal-dialog modal-dialog-centered modal-lg">
-        <div class="modal-content">
-            <div class="modal-header border-0 pb-0">
-                <h5 class="modal-title fw-semibold">
-                    <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-info me-2" viewBox="0 0 16 16">
-                        <path fill-rule="evenodd" d="M1 11.5a.5.5 0 0 0 .5.5h11.793l-3.147 3.146a.5.5 0 0 0 .708.708l4-4a.5.5 0 0 0 0-.708l-4-4a.5.5 0 0 0-.708.708L13.293 11H1.5a.5.5 0 0 0-.5.5zm14-7a.5.5 0 0 1-.5.5H2.707l3.147 3.146a.5.5 0 1 1-.708.708l-4-4a.5.5 0 0 1 0-.708l4-4a.5.5 0 1 1 .708.708L2.707 4H14.5a.5.5 0 0 1 .5.5z"/>
-                    </svg>
-                    Bidirectional Sync Status
-                </h5>
-                <button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="Close"></button>
-            </div>
-            <div class="modal-body">
-                <div id="bidirStatusContent">
-                    <div class="text-center py-4">
-                        <span class="spinner-border text-primary" role="status"></span>
-                        <p class="text-muted mt-2 mb-0">Checking configuration...</p>
-                    </div>
-                </div>
-            </div>
-            <div class="modal-footer">
-                <button type="button" class="btn btn-outline-secondary" data-bs-dismiss="modal">Close</button>
-                <a href="#" id="bidirWizardLink" class="btn btn-primary d-none">
-                    <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1" viewBox="0 0 16 16">
-                        <path d="M9.828.722a.5.5 0 0 1 .354.146l4.95 4.95a.5.5 0 0 1 0 .707c-.48.48-1.072.588-1.503.588-.177 0-.335-.018-.46-.039l-3.134 3.134a5.927 5.927 0 0 1 .16 1.013c.046.702-.032 1.687-.72 2.375a.5.5 0 0 1-.707 0l-2.829-2.828-3.182 3.182c-.195.195-1.219.902-1.414.707-.195-.195.512-1.22.707-1.414l3.182-3.182-2.828-2.829a.5.5 0 0 1 0-.707c.688-.688 1.673-.767 2.375-.72a5.922 5.922 0 0 1 1.013.16l3.134-3.133a2.772 2.772 0 0 1-.04-.461c0-.43.108-1.022.589-1.503a.5.5 0 0 1 .353-.146z"/>
-                    </svg>
-                    Run Setup Wizard
-                </a>
-            </div>
-        </div>
-    </div>
-</div>
-
-<script>
-(function() {
-    var escapeHtml = window.UICore ? window.UICore.escapeHtml : function(s) { return s; };
-
-    var editPeerModal = document.getElementById('editPeerModal');
-    if (editPeerModal) {
-        editPeerModal.addEventListener('show.bs.modal', function (event) {
-            var button = event.relatedTarget;
-            var siteId = button.getAttribute('data-site-id');
-            document.getElementById('edit_site_id').value = siteId;
-            document.getElementById('edit_endpoint').value = button.getAttribute('data-endpoint');
-            document.getElementById('edit_region').value = button.getAttribute('data-region');
-            document.getElementById('edit_priority').value = button.getAttribute('data-priority');
-            document.getElementById('edit_display_name').value = button.getAttribute('data-display-name');
-            document.getElementById('edit_connection_id').value = button.getAttribute('data-connection-id');
-            document.getElementById('editPeerForm').action = '/ui/sites/peers/' + encodeURIComponent(siteId) + '/update';
-        });
-    }
-
-    document.querySelectorAll('.link-no-connection').forEach(function(link) {
-        link.addEventListener('click', function(e) {
-            e.preventDefault();
-            var siteId = this.getAttribute('data-site-id');
-            var row = this.closest('tr[data-site-id]');
-            if (row) {
-                var btn = row.querySelector('.btn[data-bs-target="#editPeerModal"]');
-                if (btn) btn.click();
-            }
-        });
-    });
-
-    var deletePeerModal = document.getElementById('deletePeerModal');
-    if (deletePeerModal) {
-        deletePeerModal.addEventListener('show.bs.modal', function (event) {
-            var button = event.relatedTarget;
-            var siteId = button.getAttribute('data-site-id');
-            var displayName = button.getAttribute('data-display-name');
-            document.getElementById('deletePeerName').textContent = displayName;
-            document.getElementById('deletePeerForm').action = '/ui/sites/peers/' + encodeURIComponent(siteId) + '/delete';
-        });
-    }
-
-    function formatTimeAgo(date) {
-        var seconds = Math.floor((new Date() - date) / 1000);
-        if (seconds < 60) return 'just now';
-        var minutes = Math.floor(seconds / 60);
-        if (minutes < 60) return minutes + 'm ago';
-        var hours = Math.floor(minutes / 60);
-        if (hours < 24) return hours + 'h ago';
-        return Math.floor(hours / 24) + 'd ago';
-    }
-
-    function doHealthCheck(siteId) {
-        var row = document.querySelector('tr[data-site-id="' + CSS.escape(siteId) + '"]');
-        var statusSpan = row ? row.querySelector('.peer-health-status') : null;
-        if (!statusSpan) return Promise.resolve();
-
-        statusSpan.innerHTML = '<span class="spinner-border spinner-border-sm text-muted" role="status" style="width: 14px; height: 14px;"></span>';
-
-        return fetch('/ui/sites/peers/' + encodeURIComponent(siteId) + '/health')
-            .then(function(response) { return response.json(); })
-            .then(function(data) {
-                var now = new Date();
-                statusSpan.setAttribute('data-last-checked', now.toISOString());
-                if (data.is_healthy) {
-                    statusSpan.innerHTML = '<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-success" viewBox="0 0 16 16"><path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zm-3.97-3.03a.75.75 0 0 0-1.08.022L7.477 9.417 5.384 7.323a.75.75 0 0 0-1.06 1.06L6.97 11.03a.75.75 0 0 0 1.079-.02l3.992-4.99a.75.75 0 0 0-.01-1.05z"/></svg>';
-                    statusSpan.title = 'Healthy (checked ' + formatTimeAgo(now) + ')';
-                    return { siteId: siteId, healthy: true };
-                } else {
-                    statusSpan.innerHTML = '<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-danger" viewBox="0 0 16 16"><path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zM5.354 4.646a.5.5 0 1 0-.708.708L7.293 8l-2.647 2.646a.5.5 0 0 0 .708.708L8 8.707l2.646 2.647a.5.5 0 0 0 .708-.708L8.707 8l2.647-2.646a.5.5 0 0 0-.708-.708L8 7.293 5.354 4.646z"/></svg>';
-                    statusSpan.title = 'Unhealthy' + (data.error ? ': ' + data.error : '') + ' (checked ' + formatTimeAgo(now) + ')';
-                    return { siteId: siteId, healthy: false, error: data.error };
-                }
-            })
-            .catch(function(err) {
-                statusSpan.innerHTML = '<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-muted" viewBox="0 0 16 16"><path d="M8 15A7 7 0 1 1 8 1a7 7 0 0 1 0 14zm0 1A8 8 0 1 0 8 0a8 8 0 0 0 0 16z"/><path d="M5.255 5.786a.237.237 0 0 0 .241.247h.825c.138 0 .248-.113.266-.25.09-.656.54-1.134 1.342-1.134.686 0 1.314.343 1.314 1.168 0 .635-.374.927-.965 1.371-.673.489-1.206 1.06-1.168 1.987l.003.217a.25.25 0 0 0 .25.246h.811a.25.25 0 0 0 .25-.25v-.105c0-.718.273-.927 1.01-1.486.609-.463 1.244-.977 1.244-2.056 0-1.511-1.276-2.241-2.673-2.241-1.267 0-2.655.59-2.75 2.286zm1.557 5.763c0 .533.425.927 1.01.927.609 0 1.028-.394 1.028-.927 0-.552-.42-.94-1.029-.94-.584 0-1.009.388-1.009.94z"/></svg>';
-                statusSpan.title = 'Check failed';
-                return { siteId: siteId, healthy: null };
-            });
-    }
-
-    document.querySelectorAll('.btn-check-health').forEach(function(btn) {
-        btn.addEventListener('click', function() {
-            var siteId = this.getAttribute('data-site-id');
-            doHealthCheck(siteId).then(function(result) {
-                if (!result) return;
-                if (result.healthy === true) {
-                    if (window.showToast) window.showToast('Peer site is healthy', 'Health Check', 'success');
-                } else if (result.healthy === false) {
-                    if (window.showToast) window.showToast(result.error || 'Peer site is unhealthy', 'Health Check', 'error');
-                } else {
-                    if (window.showToast) window.showToast('Failed to check health', 'Health Check', 'error');
-                }
-            });
-        });
-    });
-
-    var checkAllBtn = document.getElementById('btnCheckAllHealth');
-    if (checkAllBtn) {
-        checkAllBtn.addEventListener('click', function() {
-            var btn = this;
-            var originalHtml = btn.innerHTML;
-            btn.disabled = true;
-            btn.innerHTML = '<span class="spinner-border spinner-border-sm me-1"></span>Checking...';
-
-            var siteIds = [];
-            document.querySelectorAll('.peer-health-status').forEach(function(el) {
-                siteIds.push(el.getAttribute('data-site-id'));
-            });
-
-            var promises = siteIds.map(function(id) { return doHealthCheck(id); });
-            Promise.all(promises).then(function(results) {
-                var healthy = results.filter(function(r) { return r && r.healthy === true; }).length;
-                var unhealthy = results.filter(function(r) { return r && r.healthy === false; }).length;
-                var failed = results.filter(function(r) { return r && r.healthy === null; }).length;
-
-                var msg = healthy + ' healthy';
-                if (unhealthy > 0) msg += ', ' + unhealthy + ' unhealthy';
-                if (failed > 0) msg += ', ' + failed + ' failed';
-                if (window.showToast) window.showToast(msg, 'Health Check', unhealthy > 0 ? 'warning' : 'success');
-
-                btn.disabled = false;
-                btn.innerHTML = originalHtml;
-            });
-        });
-    }
-
-    document.querySelectorAll('.btn-load-stats').forEach(function(btn) {
-        btn.addEventListener('click', function() {
-            var siteId = this.getAttribute('data-site-id');
-            var detailDiv = document.getElementById('stats-' + siteId);
-            if (!detailDiv) return;
-
-            detailDiv.classList.remove('d-none');
-            detailDiv.innerHTML = '<span class="spinner-border spinner-border-sm text-muted" style="width: 12px; height: 12px;"></span> Loading...';
-
-            fetch('/ui/sites/peers/' + encodeURIComponent(siteId) + '/sync-stats')
-                .then(function(response) { return response.json(); })
-                .then(function(data) {
-                    if (data.error) {
-                        detailDiv.innerHTML = '<span class="text-danger">' + escapeHtml(data.error) + '</span>';
-                    } else {
-                        var lastSync = data.last_sync_at
-                            ? new Date(data.last_sync_at * 1000).toLocaleString()
-                            : 'Never';
-                        detailDiv.innerHTML =
-                            '<div class="d-flex flex-wrap gap-2 mb-1">' +
-                                '<span class="text-success"><strong>' + escapeHtml(String(data.objects_synced)) + '</strong> synced</span>' +
-                                '<span class="text-warning"><strong>' + escapeHtml(String(data.objects_pending)) + '</strong> pending</span>' +
-                                '<span class="text-danger"><strong>' + escapeHtml(String(data.objects_failed)) + '</strong> failed</span>' +
-                            '</div>' +
-                            '<div class="text-muted" style="font-size: 0.75rem;">Last sync: ' + escapeHtml(lastSync) + '</div>';
-                    }
-                })
-                .catch(function() {
-                    detailDiv.innerHTML = '<span class="text-danger">Failed to load stats</span>';
-                });
-        });
-    });
-
-    document.querySelectorAll('.bidir-status-icon').forEach(function(icon) {
-        icon.addEventListener('click', function() {
-            var siteId = this.getAttribute('data-site-id');
-            var row = this.closest('tr[data-site-id]');
-            var btn = row ? row.querySelector('.btn-check-bidir') : null;
-            if (btn) btn.click();
-        });
-    });
-
-    document.querySelectorAll('.btn-check-bidir').forEach(function(btn) {
-        btn.addEventListener('click', function() {
-            var siteId = this.getAttribute('data-site-id');
-            var displayName = this.getAttribute('data-display-name');
-            var modal = new bootstrap.Modal(document.getElementById('bidirStatusModal'));
-            var contentDiv = document.getElementById('bidirStatusContent');
-            var wizardLink = document.getElementById('bidirWizardLink');
-
-            contentDiv.innerHTML =
-                '<div class="text-center py-4">' +
-                    '<span class="spinner-border text-primary" role="status"></span>' +
-                    '<p class="text-muted mt-2 mb-0">Checking bidirectional configuration with ' + escapeHtml(displayName) + '...</p>' +
-                '</div>';
-            wizardLink.classList.add('d-none');
-            modal.show();
-
-            fetch('/ui/sites/peers/' + encodeURIComponent(siteId) + '/bidirectional-status')
-                .then(function(response) { return response.json(); })
-                .then(function(data) {
-                    var html = '';
-
-                    if (data.is_fully_configured) {
-                        html += '<div class="alert alert-success d-flex align-items-center mb-4" role="alert">' +
-                            '<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" fill="currentColor" class="flex-shrink-0 me-2" viewBox="0 0 16 16">' +
-                                '<path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zm-3.97-3.03a.75.75 0 0 0-1.08.022L7.477 9.417 5.384 7.323a.75.75 0 0 0-1.06 1.06L6.97 11.03a.75.75 0 0 0 1.079-.02l3.992-4.99a.75.75 0 0 0-.01-1.05z"/>' +
-                            '</svg>' +
-                            '<div><strong>Bidirectional sync is fully configured!</strong><br><small>Both sites are set up to sync data in both directions.</small></div>' +
-                        '</div>';
-                    } else if (data.issues && data.issues.length > 0) {
-                        var errors = data.issues.filter(function(i) { return i.severity === 'error'; });
-                        var warnings = data.issues.filter(function(i) { return i.severity === 'warning'; });
-
-                        if (errors.length > 0) {
-                            html += '<div class="alert alert-danger mb-3" role="alert">' +
-                                '<h6 class="alert-heading fw-bold mb-2">' +
-                                    '<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="me-1" viewBox="0 0 16 16"><path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zM5.354 4.646a.5.5 0 1 0-.708.708L7.293 8l-2.647 2.646a.5.5 0 0 0 .708.708L8 8.707l2.646 2.647a.5.5 0 0 0 .708-.708L8.707 8l2.647-2.646a.5.5 0 0 0-.708-.708L8 7.293 5.354 4.646z"/></svg>' +
-                                    ' Configuration Errors</h6><ul class="mb-0 ps-3">';
-                            errors.forEach(function(issue) {
-                                html += '<li><strong>' + escapeHtml(issue.code) + ':</strong> ' + escapeHtml(issue.message) + '</li>';
-                            });
-                            html += '</ul></div>';
-                        }
-
-                        if (warnings.length > 0) {
-                            html += '<div class="alert alert-warning mb-3" role="alert">' +
-                                '<h6 class="alert-heading fw-bold mb-2">' +
-                                    '<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="me-1" viewBox="0 0 16 16"><path d="M8.982 1.566a1.13 1.13 0 0 0-1.96 0L.165 13.233c-.457.778.091 1.767.98 1.767h13.713c.889 0 1.438-.99.98-1.767L8.982 1.566zM8 5c.535 0 .954.462.9.995l-.35 3.507a.552.552 0 0 1-1.1 0L7.1 5.995A.905.905 0 0 1 8 5zm.002 6a1 1 0 1 1 0 2 1 1 0 0 1 0-2z"/></svg>' +
-                                    ' Warnings</h6><ul class="mb-0 ps-3">';
-                            warnings.forEach(function(issue) {
-                                html += '<li><strong>' + escapeHtml(issue.code) + ':</strong> ' + escapeHtml(issue.message) + '</li>';
-                            });
-                            html += '</ul></div>';
-                        }
-                    }
-
-                    html += '<div class="row g-3">';
-                    html += '<div class="col-md-6"><div class="card h-100"><div class="card-header bg-light py-2"><strong>This Site (Local)</strong></div>' +
-                        '<div class="card-body small">' +
-                            '<p class="mb-1"><strong>Site ID:</strong> ' + (data.local_site_id ? escapeHtml(data.local_site_id) : '<span class="text-danger">Not configured</span>') + '</p>' +
-                            '<p class="mb-1"><strong>Endpoint:</strong> ' + (data.local_endpoint ? escapeHtml(data.local_endpoint) : '<span class="text-danger">Not configured</span>') + '</p>' +
-                            '<p class="mb-1"><strong>Site Sync Worker:</strong> ' + (data.local_site_sync_enabled ? '<span class="text-success">Enabled</span>' : '<span class="text-warning">Disabled</span>') + '</p>' +
-                            '<p class="mb-0"><strong>Bidirectional Rules:</strong> ' + (data.local_bidirectional_rules ? data.local_bidirectional_rules.length : 0) + '</p>' +
-                        '</div></div></div>';
-
-                    if (data.remote_status) {
-                        var rs = data.remote_status;
-                        html += '<div class="col-md-6"><div class="card h-100"><div class="card-header bg-light py-2"><strong>Remote Site (' + escapeHtml(displayName) + ')</strong></div>' +
-                            '<div class="card-body small">';
-                        if (rs.admin_access_denied) {
-                            html += '<p class="text-warning mb-0">Admin access denied - cannot verify remote configuration</p>';
-                        } else if (rs.reachable === false) {
-                            html += '<p class="text-danger mb-0">Could not reach remote admin API</p>';
-                        } else {
-                            html += '<p class="mb-1"><strong>Has Peer Entry for Us:</strong> ' + (rs.has_peer_for_us ? '<span class="text-success">Yes</span>' : '<span class="text-danger">No</span>') + '</p>' +
-                                '<p class="mb-1"><strong>Connection Configured:</strong> ' + (rs.peer_connection_configured ? '<span class="text-success">Yes</span>' : '<span class="text-danger">No</span>') + '</p>';
-                        }
-                        html += '</div></div></div>';
-                    } else {
-                        html += '<div class="col-md-6"><div class="card h-100"><div class="card-header bg-light py-2"><strong>Remote Site (' + escapeHtml(displayName) + ')</strong></div>' +
-                            '<div class="card-body small"><p class="text-muted mb-0">Could not check remote status</p></div></div></div>';
-                    }
-                    html += '</div>';
-
-                    if (data.local_bidirectional_rules && data.local_bidirectional_rules.length > 0) {
-                        html += '<div class="mt-3"><h6 class="fw-semibold">Local Bidirectional Rules</h6>' +
-                            '<table class="table table-sm table-bordered mb-0"><thead class="table-light"><tr><th>Source Bucket</th><th>Target Bucket</th><th>Status</th></tr></thead><tbody>';
-                        data.local_bidirectional_rules.forEach(function(rule) {
-                            html += '<tr><td>' + escapeHtml(rule.bucket_name) + '</td><td>' + escapeHtml(rule.target_bucket) + '</td>' +
-                                '<td>' + (rule.enabled ? '<span class="badge bg-success">Enabled</span>' : '<span class="badge bg-secondary">Disabled</span>') + '</td></tr>';
-                        });
-                        html += '</tbody></table></div>';
-                    }
-
-                    if (!data.is_fully_configured) {
-                        html += '<div class="alert alert-info mt-3 mb-0" role="alert">' +
-                            '<h6 class="alert-heading fw-bold">How to Fix</h6>' +
-                            '<ol class="mb-0 ps-3">' +
-                                '<li>Ensure this site has a Site ID and Endpoint URL configured</li>' +
-                                '<li>On the remote site, register this site as a peer with a connection</li>' +
-                                '<li>Create bidirectional replication rules on both sites</li>' +
-                                '<li>Enable SITE_SYNC_ENABLED=true on both sites</li>' +
-                            '</ol></div>';
-                        var blockingErrors = ['NO_CONNECTION', 'CONNECTION_NOT_FOUND', 'REMOTE_UNREACHABLE', 'ENDPOINT_NOT_ALLOWED'];
-                        var hasBlockingError = data.issues && data.issues.some(function(i) { return blockingErrors.indexOf(i.code) !== -1; });
-                        if (!hasBlockingError) {
-                            wizardLink.href = '/ui/sites/peers/' + encodeURIComponent(siteId) + '/replication-wizard';
-                            wizardLink.classList.remove('d-none');
-                        }
-                    }
-
-                    contentDiv.innerHTML = html;
-                })
-                .catch(function(err) {
-                    contentDiv.innerHTML = '<div class="alert alert-danger" role="alert"><strong>Error:</strong> Failed to check bidirectional status. ' + escapeHtml(err.message || '') + '</div>';
-                });
-        });
-    });
-
-    document.querySelectorAll('.btn-copy-endpoint').forEach(function(btn) {
-        btn.addEventListener('click', function(e) {
-            e.stopPropagation();
-            var url = this.getAttribute('data-url');
-            if (window.UICore && window.UICore.copyToClipboard) {
-                window.UICore.copyToClipboard(url).then(function(ok) {
-                    if (ok && window.showToast) window.showToast('Endpoint URL copied', 'Copied', 'success');
-                });
-            }
-        });
-    });
-
-    var localSiteForm = document.getElementById('localSiteForm');
-    if (localSiteForm) {
-        localSiteForm.addEventListener('submit', function(e) {
-            e.preventDefault();
-            window.UICore.submitFormAjax(this, {
-                successMessage: 'Local site configuration updated',
-                onSuccess: function() {
-                    setTimeout(function() { location.reload(); }, 800);
-                }
-            });
-        });
-    }
-
-    var addPeerForm = document.getElementById('addPeerForm');
-    if (addPeerForm) {
-        addPeerForm.addEventListener('submit', function(e) {
-            e.preventDefault();
-            window.UICore.submitFormAjax(this, {
-                successMessage: 'Peer site added',
-                onSuccess: function(data) {
-                    if (data.redirect) {
-                        setTimeout(function() { window.location.href = data.redirect; }, 800);
-                    } else {
-                        setTimeout(function() { location.reload(); }, 800);
-                    }
-                }
-            });
-        });
-    }
-
-    var editPeerForm = document.getElementById('editPeerForm');
-    if (editPeerForm) {
-        editPeerForm.addEventListener('submit', function(e) {
-            e.preventDefault();
-            var modal = bootstrap.Modal.getInstance(document.getElementById('editPeerModal'));
-            window.UICore.submitFormAjax(this, {
-                successMessage: 'Peer site updated',
-                onSuccess: function() {
-                    if (modal) modal.hide();
-                    setTimeout(function() { location.reload(); }, 800);
-                }
-            });
-        });
-    }
-
-    var deletePeerForm = document.getElementById('deletePeerForm');
-    if (deletePeerForm) {
-        deletePeerForm.addEventListener('submit', function(e) {
-            e.preventDefault();
-            var modal = bootstrap.Modal.getInstance(document.getElementById('deletePeerModal'));
-            window.UICore.submitFormAjax(this, {
-                successMessage: 'Peer site deleted',
-                onSuccess: function() {
-                    if (modal) modal.hide();
-                    setTimeout(function() { location.reload(); }, 800);
-                }
-            });
-        });
-    }
-    document.querySelectorAll('.peer-actions-dropdown').forEach(function(dd) {
-        dd.addEventListener('shown.bs.dropdown', function() {
-            var toggle = dd.querySelector('[data-bs-toggle="dropdown"]');
-            var menu = dd.querySelector('.dropdown-menu');
-            if (!toggle || !menu) return;
-            var rect = toggle.getBoundingClientRect();
-            menu.style.top = rect.bottom + 'px';
-            menu.style.left = (rect.right - menu.offsetWidth) + 'px';
-        });
-    });
-})();
-</script>
-
-<style>
-.add-peer-chevron {
-    transition: transform 0.2s ease;
-}
-[aria-expanded="true"] .add-peer-chevron {
-    transform: rotate(180deg);
-}
-.endpoint-display:hover {
-    text-decoration: underline;
-}
-.peer-actions-dropdown .dropdown-menu {
-    position: fixed !important;
-    inset: auto !important;
-    transform: none !important;
-}
-</style>
-{% endblock %}

python/templates/system.html

@@ -1,750 +0,0 @@
-{% extends "base.html" %}
-
-{% block title %}System - MyFSIO Console{% endblock %}
-
-{% block content %}
-<div class="page-header d-flex justify-content-between align-items-center mb-4">
-  <div>
-    <p class="text-uppercase text-muted small mb-1">Administration</p>
-    <h1 class="h3 mb-1 d-flex align-items-center gap-2">
-      <svg xmlns="http://www.w3.org/2000/svg" width="28" height="28" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-        <path d="M9.405 1.05c-.413-1.4-2.397-1.4-2.81 0l-.1.34a1.464 1.464 0 0 1-2.105.872l-.31-.17c-1.283-.698-2.686.705-1.987 1.987l.169.311c.446.82.023 1.841-.872 2.105l-.34.1c-1.4.413-1.4 2.397 0 2.81l.34.1a1.464 1.464 0 0 1 .872 2.105l-.17.31c-.698 1.283.705 2.686 1.987 1.987l.311-.169a1.464 1.464 0 0 1 2.105.872l.1.34c.413 1.4 2.397 1.4 2.81 0l.1-.34a1.464 1.464 0 0 1 2.105-.872l.31.17c1.283.698 2.686-.705 1.987-1.987l-.169-.311a1.464 1.464 0 0 1 .872-2.105l.34-.1c1.4-.413 1.4-2.397 0-2.81l-.34-.1a1.464 1.464 0 0 1-.872-2.105l.17-.31c.698-1.283-.705-2.686-1.987-1.987l-.311.169a1.464 1.464 0 0 1-2.105-.872l-.1-.34zM8 10.93a2.929 2.929 0 1 1 0-5.86 2.929 2.929 0 0 1 0 5.858z"/>
-      </svg>
-      System
-    </h1>
-    <p class="text-muted mb-0 mt-1">Server information, feature flags, and maintenance tools.</p>
-  </div>
-  <div class="d-none d-md-block">
-    <span class="badge bg-primary bg-opacity-10 text-primary fs-6 px-3 py-2">v{{ app_version }}</span>
-  </div>
-</div>
-
-<div class="row g-4 mb-4">
-  <div class="col-lg-6">
-    <div class="card shadow-sm border-0" style="border-radius: 1rem;">
-      <div class="card-header bg-transparent border-0 pt-4 pb-0 px-4">
-        <h5 class="fw-semibold d-flex align-items-center gap-2 mb-1">
-          <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-            <path d="M5 0a.5.5 0 0 1 .5.5V2h1V.5a.5.5 0 0 1 1 0V2h1V.5a.5.5 0 0 1 1 0V2h1V.5a.5.5 0 0 1 1 0V2A2.5 2.5 0 0 1 14 4.5h1.5a.5.5 0 0 1 0 1H14v1h1.5a.5.5 0 0 1 0 1H14v1h1.5a.5.5 0 0 1 0 1H14v1h1.5a.5.5 0 0 1 0 1H14a2.5 2.5 0 0 1-2.5 2.5v1.5a.5.5 0 0 1-1 0V14h-1v1.5a.5.5 0 0 1-1 0V14h-1v1.5a.5.5 0 0 1-1 0V14h-1v1.5a.5.5 0 0 1-1 0V14A2.5 2.5 0 0 1 2 11.5H.5a.5.5 0 0 1 0-1H2v-1H.5a.5.5 0 0 1 0-1H2v-1H.5a.5.5 0 0 1 0-1H2v-1H.5a.5.5 0 0 1 0-1H2A2.5 2.5 0 0 1 4.5 2V.5A.5.5 0 0 1 5 0zm-.5 3A1.5 1.5 0 0 0 3 4.5v7A1.5 1.5 0 0 0 4.5 13h7a1.5 1.5 0 0 0 1.5-1.5v-7A1.5 1.5 0 0 0 11.5 3h-7zM5 6.5A1.5 1.5 0 0 1 6.5 5h3A1.5 1.5 0 0 1 11 6.5v3A1.5 1.5 0 0 1 9.5 11h-3A1.5 1.5 0 0 1 5 9.5v-3zM6.5 6a.5.5 0 0 0-.5.5v3a.5.5 0 0 0 .5.5h3a.5.5 0 0 0 .5-.5v-3a.5.5 0 0 0-.5-.5h-3z"/>
-          </svg>
-          Server Information
-        </h5>
-        <p class="text-muted small mb-0">Runtime environment and configuration</p>
-      </div>
-      <div class="card-body px-4 pb-4">
-        <table class="table table-sm mb-0">
-          <tbody>
-            <tr><td class="text-muted" style="width:40%">Version</td><td class="fw-medium">{{ app_version }}</td></tr>
-            <tr><td class="text-muted">Storage Root</td><td><code>{{ storage_root }}</code></td></tr>
-            <tr><td class="text-muted">Platform</td><td>{{ platform }}</td></tr>
-            <tr><td class="text-muted">Python</td><td>{{ python_version }}</td></tr>
-            <tr><td class="text-muted">Rust Extension</td><td>
-              {% if has_rust %}
-              <span class="badge bg-success bg-opacity-10 text-success">Loaded</span>
-              {% else %}
-              <span class="badge bg-secondary bg-opacity-10 text-secondary">Not loaded</span>
-              {% endif %}
-            </td></tr>
-          </tbody>
-        </table>
-      </div>
-    </div>
-  </div>
-
-  <div class="col-lg-6">
-    <div class="card shadow-sm border-0" style="border-radius: 1rem;">
-      <div class="card-header bg-transparent border-0 pt-4 pb-0 px-4">
-        <h5 class="fw-semibold d-flex align-items-center gap-2 mb-1">
-          <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-            <path fill-rule="evenodd" d="M11.5 2a1.5 1.5 0 1 0 0 3 1.5 1.5 0 0 0 0-3zM9.05 3a2.5 2.5 0 0 1 4.9 0H16v1h-2.05a2.5 2.5 0 0 1-4.9 0H0V3h9.05zM4.5 7a1.5 1.5 0 1 0 0 3 1.5 1.5 0 0 0 0-3zM2.05 8a2.5 2.5 0 0 1 4.9 0H16v1H6.95a2.5 2.5 0 0 1-4.9 0H0V8h2.05zm9.45 4a1.5 1.5 0 1 0 0 3 1.5 1.5 0 0 0 0-3zm-2.45 1a2.5 2.5 0 0 1 4.9 0H16v1h-2.05a2.5 2.5 0 0 1-4.9 0H0v-1h9.05z"/>
-          </svg>
-          Feature Flags
-        </h5>
-        <p class="text-muted small mb-0">Features configured via environment variables</p>
-      </div>
-      <div class="card-body px-4 pb-4">
-        <table class="table table-sm mb-0">
-          <tbody>
-            {% for feat in features %}
-            <tr>
-              <td class="text-muted" style="width:55%">{{ feat.label }}</td>
-              <td class="text-end">
-                {% if feat.enabled %}
-                <span class="badge bg-success bg-opacity-10 text-success">Enabled</span>
-                {% else %}
-                <span class="badge bg-secondary bg-opacity-10 text-secondary">Disabled</span>
-                {% endif %}
-              </td>
-            </tr>
-            {% endfor %}
-          </tbody>
-        </table>
-      </div>
-    </div>
-  </div>
-</div>
-
-<div class="row g-4 mb-4">
-  <div class="col-lg-6">
-    <div class="card shadow-sm border-0" style="border-radius: 1rem;">
-      <div class="card-header bg-transparent border-0 pt-4 pb-0 px-4">
-        <div class="d-flex justify-content-between align-items-start">
-          <div>
-            <h5 class="fw-semibold d-flex align-items-center gap-2 mb-1">
-              <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-                <path d="M2.5 1a1 1 0 0 0-1 1v1a1 1 0 0 0 1 1H3v9a2 2 0 0 0 2 2h6a2 2 0 0 0 2-2V4h.5a1 1 0 0 0 1-1V2a1 1 0 0 0-1-1H10a1 1 0 0 0-1-1H7a1 1 0 0 0-1 1H2.5zm3 4a.5.5 0 0 1 .5.5v7a.5.5 0 0 1-1 0v-7a.5.5 0 0 1 .5-.5zM8 5a.5.5 0 0 1 .5.5v7a.5.5 0 0 1-1 0v-7A.5.5 0 0 1 8 5zm3 .5v7a.5.5 0 0 1-1 0v-7a.5.5 0 0 1 1 0z"/>
-              </svg>
-              Garbage Collection
-            </h5>
-            <p class="text-muted small mb-0">Clean up temporary files, orphaned uploads, and stale locks</p>
-          </div>
-          <div>
-            {% if gc_status.enabled %}
-            <span class="badge bg-success bg-opacity-10 text-success">Active</span>
-            {% else %}
-            <span class="badge bg-secondary bg-opacity-10 text-secondary">Disabled</span>
-            {% endif %}
-          </div>
-        </div>
-      </div>
-      <div class="card-body px-4 pb-4">
-        {% if gc_status.enabled %}
-        <div class="d-flex gap-2 mb-3">
-          <button class="btn btn-primary btn-sm d-inline-flex align-items-center" id="gcRunBtn" onclick="runGC(false)">
-            <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1 flex-shrink-0" viewBox="0 0 16 16">
-              <path fill-rule="evenodd" d="M8 3a5 5 0 1 0 4.546 2.914.5.5 0 0 1 .908-.417A6 6 0 1 1 8 2v1z"/>
-              <path d="M8 4.466V.534a.25.25 0 0 1 .41-.192l2.36 1.966c.12.1.12.284 0 .384L8.41 4.658A.25.25 0 0 1 8 4.466z"/>
-            </svg>
-            Run Now
-          </button>
-          <button class="btn btn-outline-secondary btn-sm" id="gcDryRunBtn" onclick="runGC(true)">
-            Dry Run
-          </button>
-        </div>
-
-        <div id="gcScanningBanner" class="mb-3 {% if not gc_status.scanning %}d-none{% endif %}">
-          <div class="alert alert-info mb-0 small d-flex align-items-center gap-2">
-            <div class="spinner-border spinner-border-sm text-info" role="status"></div>
-            <span>GC in progress<span id="gcScanElapsed"></span></span>
-          </div>
-        </div>
-
-        <div id="gcResult" class="mb-3 d-none">
-          <div class="alert mb-0 small" id="gcResultAlert">
-            <div class="d-flex justify-content-between align-items-start">
-              <div class="fw-semibold mb-1" id="gcResultTitle"></div>
-              <button type="button" class="btn-close btn-close-sm" style="font-size:0.65rem" onclick="document.getElementById('gcResult').classList.add('d-none')"></button>
-            </div>
-            <div id="gcResultBody"></div>
-          </div>
-        </div>
-
-        <div class="border rounded p-3 mb-3" style="background: var(--bs-tertiary-bg, #f8f9fa);">
-          <div class="d-flex align-items-center gap-2 mb-2">
-            <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="text-muted" viewBox="0 0 16 16">
-              <path d="M9.405 1.05c-.413-1.4-2.397-1.4-2.81 0l-.1.34a1.464 1.464 0 0 1-2.105.872l-.31-.17c-1.283-.698-2.686.705-1.987 1.987l.169.311c.446.82.023 1.841-.872 2.105l-.34.1c-1.4.413-1.4 2.397 0 2.81l.34.1a1.464 1.464 0 0 1 .872 2.105l-.17.31c-.698 1.283.705 2.686 1.987 1.987l.311-.169a1.464 1.464 0 0 1 2.105.872l.1.34c.413 1.4 2.397 1.4 2.81 0l.1-.34a1.464 1.464 0 0 1 2.105-.872l.31.17c1.283.698 2.686-.705 1.987-1.987l-.169-.311a1.464 1.464 0 0 1 .872-2.105l.34-.1c1.4-.413 1.4-2.397 0-2.81l-.34-.1a1.464 1.464 0 0 1-.872-2.105l.17-.31c.698-1.283-.705-2.686-1.987-1.987l-.311.169a1.464 1.464 0 0 1-2.105-.872l-.1-.34zM8 10.93a2.929 2.929 0 1 1 0-5.86 2.929 2.929 0 0 1 0 5.858z"/>
-            </svg>
-            <span class="small fw-semibold text-muted">Configuration</span>
-          </div>
-          <div class="row small">
-            <div class="col-6 mb-1"><span class="text-muted">Interval:</span> {{ gc_status.interval_hours }}h</div>
-            <div class="col-6 mb-1"><span class="text-muted">Dry run:</span> {{ "Yes" if gc_status.dry_run else "No" }}</div>
-            <div class="col-6 mb-1"><span class="text-muted">Temp max age:</span> {{ gc_status.temp_file_max_age_hours }}h</div>
-            <div class="col-6 mb-1"><span class="text-muted">Lock max age:</span> {{ gc_status.lock_file_max_age_hours }}h</div>
-            <div class="col-6"><span class="text-muted">Multipart max age:</span> {{ gc_status.multipart_max_age_days }}d</div>
-          </div>
-        </div>
-
-        <div id="gcHistoryContainer">
-        {% if gc_history %}
-        <h6 class="fw-semibold small text-muted mb-2 d-flex align-items-center gap-2">
-          <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" viewBox="0 0 16 16">
-            <path d="M8.515 1.019A7 7 0 0 0 8 1V0a8 8 0 0 1 .589.022l-.074.997zm2.004.45a7.003 7.003 0 0 0-.985-.299l.219-.976c.383.086.76.2 1.126.342l-.36.933zm1.37.71a7.01 7.01 0 0 0-.439-.27l.493-.87a8.025 8.025 0 0 1 .979.654l-.615.789a6.996 6.996 0 0 0-.418-.302zm1.834 1.79a6.99 6.99 0 0 0-.653-.796l.724-.69c.27.285.52.59.747.91l-.818.576zm.744 1.352a7.08 7.08 0 0 0-.214-.468l.893-.45a7.976 7.976 0 0 1 .45 1.088l-.95.313a7.023 7.023 0 0 0-.179-.483zm.53 2.507a6.991 6.991 0 0 0-.1-1.025l.985-.17c.067.386.106.778.116 1.17l-1 .025zm-.131 1.538c.033-.17.06-.339.081-.51l.993.123a7.957 7.957 0 0 1-.23 1.155l-.964-.267c.046-.165.086-.332.12-.501zm-.952 2.379c.184-.29.346-.594.486-.908l.914.405c-.16.36-.345.706-.555 1.038l-.845-.535zm-.964 1.205c.122-.122.239-.248.35-.378l.758.653a8.073 8.073 0 0 1-.401.432l-.707-.707z"/>
-            <path d="M8 1a7 7 0 1 0 4.95 11.95l.707.707A8.001 8.001 0 1 1 8 0v1z"/>
-            <path d="M7.5 3a.5.5 0 0 1 .5.5v5.21l3.248 1.856a.5.5 0 0 1-.496.868l-3.5-2A.5.5 0 0 1 7 8V3.5a.5.5 0 0 1 .5-.5z"/>
-          </svg>
-          Recent Executions
-        </h6>
-        <div class="table-responsive">
-          <table class="table table-sm small mb-0">
-            <thead class="table-light">
-              <tr>
-                <th>Time</th>
-                <th class="text-center">Cleaned</th>
-                <th class="text-center">Freed</th>
-                <th class="text-center">Mode</th>
-              </tr>
-            </thead>
-            <tbody>
-              {% for exec in gc_history %}
-              <tr>
-                <td class="text-nowrap">{{ exec.timestamp_display }}</td>
-                <td class="text-center">
-                  {% set r = exec.result %}
-                  {{ (r.temp_files_deleted|d(0)) + (r.multipart_uploads_deleted|d(0)) + (r.lock_files_deleted|d(0)) + (r.orphaned_metadata_deleted|d(0)) + (r.orphaned_versions_deleted|d(0)) + (r.empty_dirs_removed|d(0)) }}
-                </td>
-                <td class="text-center">{{ exec.bytes_freed_display }}</td>
-                <td class="text-center">
-                  {% if exec.dry_run %}
-                  <span class="badge bg-warning bg-opacity-10 text-warning">Dry run</span>
-                  {% else %}
-                  <span class="badge bg-primary bg-opacity-10 text-primary">Live</span>
-                  {% endif %}
-                </td>
-              </tr>
-              {% endfor %}
-            </tbody>
-          </table>
-        </div>
-        {% else %}
-        <div class="text-center py-2">
-          <p class="text-muted small mb-0">No executions recorded yet.</p>
-        </div>
-        {% endif %}
-        </div>
-
-        {% else %}
-        <div class="text-center py-4">
-          <svg xmlns="http://www.w3.org/2000/svg" width="40" height="40" fill="currentColor" class="text-muted mb-2 opacity-50" viewBox="0 0 16 16">
-            <path d="M2.5 1a1 1 0 0 0-1 1v1a1 1 0 0 0 1 1H3v9a2 2 0 0 0 2 2h6a2 2 0 0 0 2-2V4h.5a1 1 0 0 0 1-1V2a1 1 0 0 0-1-1H10a1 1 0 0 0-1-1H7a1 1 0 0 0-1 1H2.5zm3 4a.5.5 0 0 1 .5.5v7a.5.5 0 0 1-1 0v-7a.5.5 0 0 1 .5-.5zM8 5a.5.5 0 0 1 .5.5v7a.5.5 0 0 1-1 0v-7A.5.5 0 0 1 8 5zm3 .5v7a.5.5 0 0 1-1 0v-7a.5.5 0 0 1 1 0z"/>
-          </svg>
-          <p class="text-muted mb-1">Garbage collection is not enabled.</p>
-          <p class="text-muted small mb-0">Set <code>GC_ENABLED=true</code> to enable automatic cleanup.</p>
-        </div>
-        {% endif %}
-      </div>
-    </div>
-  </div>
-
-  <div class="col-lg-6">
-    <div class="card shadow-sm border-0" style="border-radius: 1rem;">
-      <div class="card-header bg-transparent border-0 pt-4 pb-0 px-4">
-        <div class="d-flex justify-content-between align-items-start">
-          <div>
-            <h5 class="fw-semibold d-flex align-items-center gap-2 mb-1">
-              <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-                <path d="M5.338 1.59a61.44 61.44 0 0 0-2.837.856.481.481 0 0 0-.328.39c-.554 4.157.726 7.19 2.253 9.188a10.725 10.725 0 0 0 2.287 2.233c.346.244.652.42.893.533.12.057.218.095.293.118a.55.55 0 0 0 .101.025.615.615 0 0 0 .1-.025c.076-.023.174-.061.294-.118.24-.113.547-.29.893-.533a10.726 10.726 0 0 0 2.287-2.233c1.527-1.997 2.807-5.031 2.253-9.188a.48.48 0 0 0-.328-.39c-.651-.213-1.75-.56-2.837-.855C9.552 1.29 8.531 1.067 8 1.067c-.53 0-1.552.223-2.662.524zM5.072.56C6.157.265 7.31 0 8 0s1.843.265 2.928.56c1.11.3 2.229.655 2.887.87a1.54 1.54 0 0 1 1.044 1.262c.596 4.477-.787 7.795-2.465 9.99a11.775 11.775 0 0 1-2.517 2.453 7.159 7.159 0 0 1-1.048.625c-.28.132-.581.24-.829.24s-.548-.108-.829-.24a7.158 7.158 0 0 1-1.048-.625 11.777 11.777 0 0 1-2.517-2.453C1.928 10.487.545 7.169 1.141 2.692A1.54 1.54 0 0 1 2.185 1.43 62.456 62.456 0 0 1 5.072.56z"/>
-                <path d="M10.854 5.146a.5.5 0 0 1 0 .708l-3 3a.5.5 0 0 1-.708 0l-1.5-1.5a.5.5 0 1 1 .708-.708L7.5 7.793l2.646-2.647a.5.5 0 0 1 .708 0z"/>
-              </svg>
-              Integrity Scanner
-            </h5>
-            <p class="text-muted small mb-0">Detect and heal corrupted objects, orphaned files, and metadata drift</p>
-          </div>
-          <div>
-            {% if integrity_status.enabled %}
-            <span class="badge bg-success bg-opacity-10 text-success">Active</span>
-            {% else %}
-            <span class="badge bg-secondary bg-opacity-10 text-secondary">Disabled</span>
-            {% endif %}
-          </div>
-        </div>
-      </div>
-      <div class="card-body px-4 pb-4">
-        {% if integrity_status.enabled %}
-        <div class="d-flex gap-2 flex-wrap mb-3">
-          <button class="btn btn-primary btn-sm d-inline-flex align-items-center" id="integrityRunBtn" onclick="runIntegrity(false, false)" {% if integrity_status.scanning %}disabled{% endif %}>
-            <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1 flex-shrink-0" viewBox="0 0 16 16">
-              <path fill-rule="evenodd" d="M8 3a5 5 0 1 0 4.546 2.914.5.5 0 0 1 .908-.417A6 6 0 1 1 8 2v1z"/>
-              <path d="M8 4.466V.534a.25.25 0 0 1 .41-.192l2.36 1.966c.12.1.12.284 0 .384L8.41 4.658A.25.25 0 0 1 8 4.466z"/>
-            </svg>
-            Scan Now
-          </button>
-          <button class="btn btn-outline-warning btn-sm" id="integrityHealBtn" onclick="runIntegrity(false, true)" {% if integrity_status.scanning %}disabled{% endif %}>
-            Scan &amp; Heal
-          </button>
-          <button class="btn btn-outline-secondary btn-sm" id="integrityDryRunBtn" onclick="runIntegrity(true, false)" {% if integrity_status.scanning %}disabled{% endif %}>
-            Dry Run
-          </button>
-        </div>
-
-        <div id="integrityScanningBanner" class="mb-3 {% if not integrity_status.scanning %}d-none{% endif %}">
-          <div class="alert alert-info mb-0 small d-flex align-items-center gap-2">
-            <div class="spinner-border spinner-border-sm text-info" role="status"></div>
-            <span>Scan in progress<span id="integrityScanElapsed"></span></span>
-          </div>
-        </div>
-
-        <div id="integrityResult" class="mb-3 d-none">
-          <div class="alert mb-0 small" id="integrityResultAlert">
-            <div class="d-flex justify-content-between align-items-start">
-              <div class="fw-semibold mb-1" id="integrityResultTitle"></div>
-              <button type="button" class="btn-close btn-close-sm" style="font-size:0.65rem" onclick="document.getElementById('integrityResult').classList.add('d-none')"></button>
-            </div>
-            <div id="integrityResultBody"></div>
-          </div>
-        </div>
-
-        <div class="border rounded p-3 mb-3" style="background: var(--bs-tertiary-bg, #f8f9fa);">
-          <div class="d-flex align-items-center gap-2 mb-2">
-            <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="text-muted" viewBox="0 0 16 16">
-              <path d="M9.405 1.05c-.413-1.4-2.397-1.4-2.81 0l-.1.34a1.464 1.464 0 0 1-2.105.872l-.31-.17c-1.283-.698-2.686.705-1.987 1.987l.169.311c.446.82.023 1.841-.872 2.105l-.34.1c-1.4.413-1.4 2.397 0 2.81l.34.1a1.464 1.464 0 0 1 .872 2.105l-.17.31c-.698 1.283.705 2.686 1.987 1.987l.311-.169a1.464 1.464 0 0 1 2.105.872l.1.34c.413 1.4 2.397 1.4 2.81 0l.1-.34a1.464 1.464 0 0 1 2.105-.872l.31.17c1.283.698 2.686-.705 1.987-1.987l-.169-.311a1.464 1.464 0 0 1 .872-2.105l.34-.1c1.4-.413 1.4-2.397 0-2.81l-.34-.1a1.464 1.464 0 0 1-.872-2.105l.17-.31c.698-1.283-.705-2.686-1.987-1.987l-.311.169a1.464 1.464 0 0 1-2.105-.872l-.1-.34zM8 10.93a2.929 2.929 0 1 1 0-5.86 2.929 2.929 0 0 1 0 5.858z"/>
-            </svg>
-            <span class="small fw-semibold text-muted">Configuration</span>
-          </div>
-          <div class="row small">
-            <div class="col-6 mb-1"><span class="text-muted">Interval:</span> {{ integrity_status.interval_hours }}h</div>
-            <div class="col-6 mb-1"><span class="text-muted">Dry run:</span> {{ "Yes" if integrity_status.dry_run else "No" }}</div>
-            <div class="col-6"><span class="text-muted">Batch size:</span> {{ integrity_status.batch_size }}</div>
-            <div class="col-6"><span class="text-muted">Auto-heal:</span> {{ "Yes" if integrity_status.auto_heal else "No" }}</div>
-          </div>
-        </div>
-
-        <div id="integrityHistoryContainer">
-        {% if integrity_history %}
-        <h6 class="fw-semibold small text-muted mb-2 d-flex align-items-center gap-2">
-          <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" viewBox="0 0 16 16">
-            <path d="M8.515 1.019A7 7 0 0 0 8 1V0a8 8 0 0 1 .589.022l-.074.997zm2.004.45a7.003 7.003 0 0 0-.985-.299l.219-.976c.383.086.76.2 1.126.342l-.36.933zm1.37.71a7.01 7.01 0 0 0-.439-.27l.493-.87a8.025 8.025 0 0 1 .979.654l-.615.789a6.996 6.996 0 0 0-.418-.302zm1.834 1.79a6.99 6.99 0 0 0-.653-.796l.724-.69c.27.285.52.59.747.91l-.818.576zm.744 1.352a7.08 7.08 0 0 0-.214-.468l.893-.45a7.976 7.976 0 0 1 .45 1.088l-.95.313a7.023 7.023 0 0 0-.179-.483zm.53 2.507a6.991 6.991 0 0 0-.1-1.025l.985-.17c.067.386.106.778.116 1.17l-1 .025zm-.131 1.538c.033-.17.06-.339.081-.51l.993.123a7.957 7.957 0 0 1-.23 1.155l-.964-.267c.046-.165.086-.332.12-.501zm-.952 2.379c.184-.29.346-.594.486-.908l.914.405c-.16.36-.345.706-.555 1.038l-.845-.535zm-.964 1.205c.122-.122.239-.248.35-.378l.758.653a8.073 8.073 0 0 1-.401.432l-.707-.707z"/>
-            <path d="M8 1a7 7 0 1 0 4.95 11.95l.707.707A8.001 8.001 0 1 1 8 0v1z"/>
-            <path d="M7.5 3a.5.5 0 0 1 .5.5v5.21l3.248 1.856a.5.5 0 0 1-.496.868l-3.5-2A.5.5 0 0 1 7 8V3.5a.5.5 0 0 1 .5-.5z"/>
-          </svg>
-          Recent Scans
-        </h6>
-        <div class="table-responsive">
-          <table class="table table-sm small mb-0">
-            <thead class="table-light">
-              <tr>
-                <th>Time</th>
-                <th class="text-center">Scanned</th>
-                <th class="text-center">Issues</th>
-                <th class="text-center">Healed</th>
-                <th class="text-center">Mode</th>
-              </tr>
-            </thead>
-            <tbody>
-              {% for exec in integrity_history %}
-              <tr>
-                <td class="text-nowrap">{{ exec.timestamp_display }}</td>
-                <td class="text-center">{{ exec.result.objects_scanned|d(0) }}</td>
-                <td class="text-center">
-                  {% set total_issues = (exec.result.corrupted_objects|d(0)) + (exec.result.orphaned_objects|d(0)) + (exec.result.phantom_metadata|d(0)) + (exec.result.stale_versions|d(0)) + (exec.result.etag_cache_inconsistencies|d(0)) + (exec.result.legacy_metadata_drifts|d(0)) %}
-                  {% if total_issues > 0 %}
-                  <span class="text-danger fw-medium">{{ total_issues }}</span>
-                  {% else %}
-                  <span class="text-success">0</span>
-                  {% endif %}
-                </td>
-                <td class="text-center">{{ exec.result.issues_healed|d(0) }}</td>
-                <td class="text-center">
-                  {% if exec.dry_run %}
-                  <span class="badge bg-warning bg-opacity-10 text-warning">Dry</span>
-                  {% elif exec.auto_heal %}
-                  <span class="badge bg-success bg-opacity-10 text-success">Heal</span>
-                  {% else %}
-                  <span class="badge bg-primary bg-opacity-10 text-primary">Scan</span>
-                  {% endif %}
-                </td>
-              </tr>
-              {% endfor %}
-            </tbody>
-          </table>
-        </div>
-        {% else %}
-        <div class="text-center py-2">
-          <p class="text-muted small mb-0">No scans recorded yet.</p>
-        </div>
-        {% endif %}
-        </div>
-
-        {% else %}
-        <div class="text-center py-4">
-          <svg xmlns="http://www.w3.org/2000/svg" width="40" height="40" fill="currentColor" class="text-muted mb-2 opacity-50" viewBox="0 0 16 16">
-            <path d="M5.338 1.59a61.44 61.44 0 0 0-2.837.856.481.481 0 0 0-.328.39c-.554 4.157.726 7.19 2.253 9.188a10.725 10.725 0 0 0 2.287 2.233c.346.244.652.42.893.533.12.057.218.095.293.118a.55.55 0 0 0 .101.025.615.615 0 0 0 .1-.025c.076-.023.174-.061.294-.118.24-.113.547-.29.893-.533a10.726 10.726 0 0 0 2.287-2.233c1.527-1.997 2.807-5.031 2.253-9.188a.48.48 0 0 0-.328-.39c-.651-.213-1.75-.56-2.837-.855C9.552 1.29 8.531 1.067 8 1.067c-.53 0-1.552.223-2.662.524zM5.072.56C6.157.265 7.31 0 8 0s1.843.265 2.928.56c1.11.3 2.229.655 2.887.87a1.54 1.54 0 0 1 1.044 1.262c.596 4.477-.787 7.795-2.465 9.99a11.775 11.775 0 0 1-2.517 2.453 7.159 7.159 0 0 1-1.048.625c-.28.132-.581.24-.829.24s-.548-.108-.829-.24a7.158 7.158 0 0 1-1.048-.625 11.777 11.777 0 0 1-2.517-2.453C1.928 10.487.545 7.169 1.141 2.692A1.54 1.54 0 0 1 2.185 1.43 62.456 62.456 0 0 1 5.072.56z"/>
-            <path d="M10.854 5.146a.5.5 0 0 1 0 .708l-3 3a.5.5 0 0 1-.708 0l-1.5-1.5a.5.5 0 1 1 .708-.708L7.5 7.793l2.646-2.647a.5.5 0 0 1 .708 0z"/>
-          </svg>
-          <p class="text-muted mb-1">Integrity scanner is not enabled.</p>
-          <p class="text-muted small mb-0">Set <code>INTEGRITY_ENABLED=true</code> to enable automatic scanning.</p>
-        </div>
-        {% endif %}
-      </div>
-    </div>
-  </div>
-</div>
-{% endblock %}
-
-{% block extra_scripts %}
-<script>
-(function () {
-  var csrfToken = document.querySelector('meta[name="csrf-token"]')?.getAttribute('content') || '';
-
-  function setLoading(btnId, loading, spinnerOnly) {
-    var btn = document.getElementById(btnId);
-    if (!btn) return;
-    btn.disabled = loading;
-    if (loading && !spinnerOnly) {
-      btn.dataset.originalHtml = btn.innerHTML;
-      btn.innerHTML = '<span class="spinner-border spinner-border-sm me-1" role="status"></span>Running...';
-    } else if (!loading && btn.dataset.originalHtml) {
-      btn.innerHTML = btn.dataset.originalHtml;
-    }
-  }
-
-  function formatBytes(bytes) {
-    if (!bytes || bytes === 0) return '0 B';
-    var units = ['B', 'KB', 'MB', 'GB'];
-    var i = 0;
-    var b = bytes;
-    while (b >= 1024 && i < units.length - 1) { b /= 1024; i++; }
-    return (i === 0 ? b : b.toFixed(1)) + ' ' + units[i];
-  }
-
-  var _displayTimezone = {{ display_timezone|tojson }};
-
-  function formatTimestamp(ts) {
-    var d = new Date(ts * 1000);
-    try {
-      var opts = {year: 'numeric', month: 'short', day: '2-digit', hour: '2-digit', minute: '2-digit', hour12: false, timeZone: _displayTimezone, timeZoneName: 'short'};
-      return d.toLocaleString('en-US', opts);
-    } catch (e) {
-      var pad = function (n) { return n < 10 ? '0' + n : '' + n; };
-      return d.getUTCFullYear() + '-' + pad(d.getUTCMonth() + 1) + '-' + pad(d.getUTCDate()) +
-             ' ' + pad(d.getUTCHours()) + ':' + pad(d.getUTCMinutes()) + ' UTC';
-    }
-  }
-
-  var _gcHistoryIcon = '<svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" viewBox="0 0 16 16">' +
-    '<path d="M8.515 1.019A7 7 0 0 0 8 1V0a8 8 0 0 1 .589.022l-.074.997zm2.004.45a7.003 7.003 0 0 0-.985-.299l.219-.976c.383.086.76.2 1.126.342l-.36.933zm1.37.71a7.01 7.01 0 0 0-.439-.27l.493-.87a8.025 8.025 0 0 1 .979.654l-.615.789a6.996 6.996 0 0 0-.418-.302zm1.834 1.79a6.99 6.99 0 0 0-.653-.796l.724-.69c.27.285.52.59.747.91l-.818.576zm.744 1.352a7.08 7.08 0 0 0-.214-.468l.893-.45a7.976 7.976 0 0 1 .45 1.088l-.95.313a7.023 7.023 0 0 0-.179-.483zm.53 2.507a6.991 6.991 0 0 0-.1-1.025l.985-.17c.067.386.106.778.116 1.17l-1 .025zm-.131 1.538c.033-.17.06-.339.081-.51l.993.123a7.957 7.957 0 0 1-.23 1.155l-.964-.267c.046-.165.086-.332.12-.501zm-.952 2.379c.184-.29.346-.594.486-.908l.914.405c-.16.36-.345.706-.555 1.038l-.845-.535zm-.964 1.205c.122-.122.239-.248.35-.378l.758.653a8.073 8.073 0 0 1-.401.432l-.707-.707z"/>' +
-    '<path d="M8 1a7 7 0 1 0 4.95 11.95l.707.707A8.001 8.001 0 1 1 8 0v1z"/>' +
-    '<path d="M7.5 3a.5.5 0 0 1 .5.5v5.21l3.248 1.856a.5.5 0 0 1-.496.868l-3.5-2A.5.5 0 0 1 7 8V3.5a.5.5 0 0 1 .5-.5z"/></svg>';
-
-  function _gcRefreshHistory() {
-    fetch('{{ url_for("ui.system_gc_history") }}?limit=10', {
-      headers: {'X-CSRFToken': csrfToken}
-    })
-    .then(function (r) { return r.json(); })
-    .then(function (hist) {
-      var container = document.getElementById('gcHistoryContainer');
-      if (!container) return;
-      var execs = hist.executions || [];
-      if (execs.length === 0) {
-        container.innerHTML = '<div class="text-center py-2"><p class="text-muted small mb-0">No executions recorded yet.</p></div>';
-        return;
-      }
-      var html = '<h6 class="fw-semibold small text-muted mb-2 d-flex align-items-center gap-2">' +
-        _gcHistoryIcon + ' Recent Executions</h6>' +
-        '<div class="table-responsive"><table class="table table-sm small mb-0">' +
-        '<thead class="table-light"><tr><th>Time</th><th class="text-center">Cleaned</th>' +
-        '<th class="text-center">Freed</th><th class="text-center">Mode</th></tr></thead><tbody>';
-      execs.forEach(function (exec) {
-        var r = exec.result || {};
-        var cleaned = (r.temp_files_deleted || 0) + (r.multipart_uploads_deleted || 0) +
-          (r.lock_files_deleted || 0) + (r.orphaned_metadata_deleted || 0) +
-          (r.orphaned_versions_deleted || 0) + (r.empty_dirs_removed || 0);
-        var freed = (r.temp_bytes_freed || 0) + (r.multipart_bytes_freed || 0) +
-          (r.orphaned_version_bytes_freed || 0);
-        var mode = exec.dry_run
-          ? '<span class="badge bg-warning bg-opacity-10 text-warning">Dry run</span>'
-          : '<span class="badge bg-primary bg-opacity-10 text-primary">Live</span>';
-        html += '<tr><td class="text-nowrap">' + formatTimestamp(exec.timestamp) + '</td>' +
-          '<td class="text-center">' + cleaned + '</td>' +
-          '<td class="text-center">' + formatBytes(freed) + '</td>' +
-          '<td class="text-center">' + mode + '</td></tr>';
-      });
-      html += '</tbody></table></div>';
-      container.innerHTML = html;
-    })
-    .catch(function () {});
-  }
-
-  function _integrityRefreshHistory() {
-    fetch('{{ url_for("ui.system_integrity_history") }}?limit=10', {
-      headers: {'X-CSRFToken': csrfToken}
-    })
-    .then(function (r) { return r.json(); })
-    .then(function (hist) {
-      var container = document.getElementById('integrityHistoryContainer');
-      if (!container) return;
-      var execs = hist.executions || [];
-      if (execs.length === 0) {
-        container.innerHTML = '<div class="text-center py-2"><p class="text-muted small mb-0">No scans recorded yet.</p></div>';
-        return;
-      }
-      var html = '<h6 class="fw-semibold small text-muted mb-2 d-flex align-items-center gap-2">' +
-        _gcHistoryIcon + ' Recent Scans</h6>' +
-        '<div class="table-responsive"><table class="table table-sm small mb-0">' +
-        '<thead class="table-light"><tr><th>Time</th><th class="text-center">Scanned</th>' +
-        '<th class="text-center">Issues</th><th class="text-center">Healed</th>' +
-        '<th class="text-center">Mode</th></tr></thead><tbody>';
-      execs.forEach(function (exec) {
-        var r = exec.result || {};
-        var issues = (r.corrupted_objects || 0) + (r.orphaned_objects || 0) +
-          (r.phantom_metadata || 0) + (r.stale_versions || 0) +
-          (r.etag_cache_inconsistencies || 0) + (r.legacy_metadata_drifts || 0);
-        var issueHtml = issues > 0
-          ? '<span class="text-danger fw-medium">' + issues + '</span>'
-          : '<span class="text-success">0</span>';
-        var mode = exec.dry_run
-          ? '<span class="badge bg-warning bg-opacity-10 text-warning">Dry</span>'
-          : (exec.auto_heal
-            ? '<span class="badge bg-success bg-opacity-10 text-success">Heal</span>'
-            : '<span class="badge bg-primary bg-opacity-10 text-primary">Scan</span>');
-        html += '<tr><td class="text-nowrap">' + formatTimestamp(exec.timestamp) + '</td>' +
-          '<td class="text-center">' + (r.objects_scanned || 0) + '</td>' +
-          '<td class="text-center">' + issueHtml + '</td>' +
-          '<td class="text-center">' + (r.issues_healed || 0) + '</td>' +
-          '<td class="text-center">' + mode + '</td></tr>';
-      });
-      html += '</tbody></table></div>';
-      container.innerHTML = html;
-    })
-    .catch(function () {});
-  }
-
-  var _gcPollTimer = null;
-  var _gcLastDryRun = false;
-
-  function _gcSetScanning(scanning) {
-    var banner = document.getElementById('gcScanningBanner');
-    var btns = ['gcRunBtn', 'gcDryRunBtn'];
-    if (scanning) {
-      banner.classList.remove('d-none');
-      btns.forEach(function (id) {
-        var el = document.getElementById(id);
-        if (el) el.disabled = true;
-      });
-    } else {
-      banner.classList.add('d-none');
-      document.getElementById('gcScanElapsed').textContent = '';
-      btns.forEach(function (id) {
-        var el = document.getElementById(id);
-        if (el) el.disabled = false;
-      });
-    }
-  }
-
-  function _gcShowResult(data, dryRun) {
-    var container = document.getElementById('gcResult');
-    var alert = document.getElementById('gcResultAlert');
-    var title = document.getElementById('gcResultTitle');
-    var body = document.getElementById('gcResultBody');
-    container.classList.remove('d-none');
-
-    var totalItems = (data.temp_files_deleted || 0) + (data.multipart_uploads_deleted || 0) +
-                      (data.lock_files_deleted || 0) + (data.orphaned_metadata_deleted || 0) +
-                      (data.orphaned_versions_deleted || 0) + (data.empty_dirs_removed || 0);
-    var totalFreed = (data.temp_bytes_freed || 0) + (data.multipart_bytes_freed || 0) +
-                     (data.orphaned_version_bytes_freed || 0);
-
-    alert.className = totalItems > 0 ? 'alert alert-success mb-0 small' : 'alert alert-info mb-0 small';
-    title.textContent = (dryRun ? '[Dry Run] ' : '') + 'Completed in ' + (data.execution_time_seconds || 0).toFixed(2) + 's';
-
-    var lines = [];
-    if (data.temp_files_deleted) lines.push('Temp files: ' + data.temp_files_deleted + ' (' + formatBytes(data.temp_bytes_freed) + ')');
-    if (data.multipart_uploads_deleted) lines.push('Multipart uploads: ' + data.multipart_uploads_deleted + ' (' + formatBytes(data.multipart_bytes_freed) + ')');
-    if (data.lock_files_deleted) lines.push('Lock files: ' + data.lock_files_deleted);
-    if (data.orphaned_metadata_deleted) lines.push('Orphaned metadata: ' + data.orphaned_metadata_deleted);
-    if (data.orphaned_versions_deleted) lines.push('Orphaned versions: ' + data.orphaned_versions_deleted + ' (' + formatBytes(data.orphaned_version_bytes_freed) + ')');
-    if (data.empty_dirs_removed) lines.push('Empty directories: ' + data.empty_dirs_removed);
-    if (totalItems === 0) lines.push('Nothing to clean up.');
-    if (totalFreed > 0) lines.push('Total freed: ' + formatBytes(totalFreed));
-    if (data.errors && data.errors.length > 0) lines.push('Errors: ' + data.errors.join(', '));
-
-    body.innerHTML = lines.join('<br>');
-  }
-
-  function _gcPoll() {
-    fetch('{{ url_for("ui.system_gc_status") }}', {
-      headers: {'X-CSRFToken': csrfToken}
-    })
-    .then(function (r) { return r.json(); })
-    .then(function (status) {
-      if (status.scanning) {
-        var elapsed = status.scan_elapsed_seconds || 0;
-        document.getElementById('gcScanElapsed').textContent = ' (' + elapsed.toFixed(0) + 's)';
-        _gcPollTimer = setTimeout(_gcPoll, 2000);
-      } else {
-        _gcSetScanning(false);
-        _gcRefreshHistory();
-        fetch('{{ url_for("ui.system_gc_history") }}?limit=1', {
-          headers: {'X-CSRFToken': csrfToken}
-        })
-        .then(function (r) { return r.json(); })
-        .then(function (hist) {
-          if (hist.executions && hist.executions.length > 0) {
-            var latest = hist.executions[0];
-            _gcShowResult(latest.result, latest.dry_run);
-          }
-        })
-        .catch(function () {});
-      }
-    })
-    .catch(function () {
-      _gcPollTimer = setTimeout(_gcPoll, 3000);
-    });
-  }
-
-  window.runGC = function (dryRun) {
-    _gcLastDryRun = dryRun;
-    document.getElementById('gcResult').classList.add('d-none');
-    _gcSetScanning(true);
-
-    fetch('{{ url_for("ui.system_gc_run") }}', {
-      method: 'POST',
-      headers: {'Content-Type': 'application/json', 'X-CSRFToken': csrfToken},
-      body: JSON.stringify({dry_run: dryRun})
-    })
-    .then(function (r) { return r.json(); })
-    .then(function (data) {
-      if (data.error) {
-        _gcSetScanning(false);
-        var container = document.getElementById('gcResult');
-        var alert = document.getElementById('gcResultAlert');
-        var title = document.getElementById('gcResultTitle');
-        var body = document.getElementById('gcResultBody');
-        container.classList.remove('d-none');
-        alert.className = 'alert alert-danger mb-0 small';
-        title.textContent = 'Error';
-        body.textContent = data.error;
-        return;
-      }
-      _gcPollTimer = setTimeout(_gcPoll, 2000);
-    })
-    .catch(function (err) {
-      _gcSetScanning(false);
-      var container = document.getElementById('gcResult');
-      var alert = document.getElementById('gcResultAlert');
-      var title = document.getElementById('gcResultTitle');
-      var body = document.getElementById('gcResultBody');
-      container.classList.remove('d-none');
-      alert.className = 'alert alert-danger mb-0 small';
-      title.textContent = 'Error';
-      body.textContent = err.message;
-    });
-  };
-
-  {% if gc_status.scanning %}
-  _gcSetScanning(true);
-  _gcPollTimer = setTimeout(_gcPoll, 2000);
-  {% endif %}
-
-  var _integrityPollTimer = null;
-  var _integrityLastMode = {dryRun: false, autoHeal: false};
-
-  function _integritySetScanning(scanning) {
-    var banner = document.getElementById('integrityScanningBanner');
-    var btns = ['integrityRunBtn', 'integrityHealBtn', 'integrityDryRunBtn'];
-    if (scanning) {
-      banner.classList.remove('d-none');
-      btns.forEach(function (id) {
-        var el = document.getElementById(id);
-        if (el) el.disabled = true;
-      });
-    } else {
-      banner.classList.add('d-none');
-      document.getElementById('integrityScanElapsed').textContent = '';
-      btns.forEach(function (id) {
-        var el = document.getElementById(id);
-        if (el) el.disabled = false;
-      });
-    }
-  }
-
-  function _integrityShowResult(data, dryRun, autoHeal) {
-    var container = document.getElementById('integrityResult');
-    var alert = document.getElementById('integrityResultAlert');
-    var title = document.getElementById('integrityResultTitle');
-    var body = document.getElementById('integrityResultBody');
-    container.classList.remove('d-none');
-
-    var totalIssues = (data.corrupted_objects || 0) + (data.orphaned_objects || 0) +
-                       (data.phantom_metadata || 0) + (data.stale_versions || 0) +
-                       (data.etag_cache_inconsistencies || 0) + (data.legacy_metadata_drifts || 0);
-
-    var prefix = dryRun ? '[Dry Run] ' : (autoHeal ? '[Heal] ' : '');
-    alert.className = totalIssues > 0 ? 'alert alert-warning mb-0 small' : 'alert alert-success mb-0 small';
-    title.textContent = prefix + 'Completed in ' + (data.execution_time_seconds || 0).toFixed(2) + 's';
-
-    var lines = [];
-    lines.push('Scanned: ' + (data.objects_scanned || 0) + ' objects in ' + (data.buckets_scanned || 0) + ' buckets');
-    if (totalIssues === 0) {
-      lines.push('No issues found.');
-    } else {
-      if (data.corrupted_objects) lines.push('Corrupted objects: ' + data.corrupted_objects);
-      if (data.orphaned_objects) lines.push('Orphaned objects: ' + data.orphaned_objects);
-      if (data.phantom_metadata) lines.push('Phantom metadata: ' + data.phantom_metadata);
-      if (data.stale_versions) lines.push('Stale versions: ' + data.stale_versions);
-      if (data.etag_cache_inconsistencies) lines.push('ETag inconsistencies: ' + data.etag_cache_inconsistencies);
-      if (data.legacy_metadata_drifts) lines.push('Legacy metadata drifts: ' + data.legacy_metadata_drifts);
-      if (data.issues_healed) lines.push('Issues healed: ' + data.issues_healed);
-    }
-    if (data.errors && data.errors.length > 0) lines.push('Errors: ' + data.errors.join(', '));
-
-    body.innerHTML = lines.join('<br>');
-  }
-
-  function _integrityPoll() {
-    fetch('{{ url_for("ui.system_integrity_status") }}', {
-      headers: {'X-CSRFToken': csrfToken}
-    })
-    .then(function (r) { return r.json(); })
-    .then(function (status) {
-      if (status.scanning) {
-        var elapsed = status.scan_elapsed_seconds || 0;
-        document.getElementById('integrityScanElapsed').textContent = ' (' + elapsed.toFixed(0) + 's)';
-        _integrityPollTimer = setTimeout(_integrityPoll, 2000);
-      } else {
-        _integritySetScanning(false);
-        _integrityRefreshHistory();
-        fetch('{{ url_for("ui.system_integrity_history") }}?limit=1', {
-          headers: {'X-CSRFToken': csrfToken}
-        })
-        .then(function (r) { return r.json(); })
-        .then(function (hist) {
-          if (hist.executions && hist.executions.length > 0) {
-            var latest = hist.executions[0];
-            _integrityShowResult(latest.result, latest.dry_run, latest.auto_heal);
-          }
-        })
-        .catch(function () {});
-      }
-    })
-    .catch(function () {
-      _integrityPollTimer = setTimeout(_integrityPoll, 3000);
-    });
-  }
-
-  window.runIntegrity = function (dryRun, autoHeal) {
-    _integrityLastMode = {dryRun: dryRun, autoHeal: autoHeal};
-    document.getElementById('integrityResult').classList.add('d-none');
-    _integritySetScanning(true);
-
-    fetch('{{ url_for("ui.system_integrity_run") }}', {
-      method: 'POST',
-      headers: {'Content-Type': 'application/json', 'X-CSRFToken': csrfToken},
-      body: JSON.stringify({dry_run: dryRun, auto_heal: autoHeal})
-    })
-    .then(function (r) { return r.json(); })
-    .then(function (data) {
-      if (data.error) {
-        _integritySetScanning(false);
-        var container = document.getElementById('integrityResult');
-        var alert = document.getElementById('integrityResultAlert');
-        var title = document.getElementById('integrityResultTitle');
-        var body = document.getElementById('integrityResultBody');
-        container.classList.remove('d-none');
-        alert.className = 'alert alert-danger mb-0 small';
-        title.textContent = 'Error';
-        body.textContent = data.error;
-        return;
-      }
-      _integrityPollTimer = setTimeout(_integrityPoll, 2000);
-    })
-    .catch(function (err) {
-      _integritySetScanning(false);
-      var container = document.getElementById('integrityResult');
-      var alert = document.getElementById('integrityResultAlert');
-      var title = document.getElementById('integrityResultTitle');
-      var body = document.getElementById('integrityResultBody');
-      container.classList.remove('d-none');
-      alert.className = 'alert alert-danger mb-0 small';
-      title.textContent = 'Error';
-      body.textContent = err.message;
-    });
-  };
-
-  {% if integrity_status.scanning %}
-  _integritySetScanning(true);
-  _integrityPollTimer = setTimeout(_integrityPoll, 2000);
-  {% endif %}
-})();
-</script>
-{% endblock %}

python/templates/website_domains.html

@@ -1,367 +0,0 @@
-{% extends "base.html" %}
-
-{% block title %}Website Domains - MyFSIO Console{% endblock %}
-
-{% block content %}
-<div class="page-header d-flex justify-content-between align-items-center mb-4">
-  <div>
-    <p class="text-uppercase text-muted small mb-1">Website Hosting</p>
-    <h1 class="h3 mb-1 d-flex align-items-center gap-2">
-      <svg xmlns="http://www.w3.org/2000/svg" width="28" height="28" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-        <path d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8zm7.5-6.923c-.67.204-1.335.82-1.887 1.855A7.97 7.97 0 0 0 5.145 4H7.5V1.077zM4.09 4a9.267 9.267 0 0 1 .64-1.539 6.7 6.7 0 0 1 .597-.933A7.025 7.025 0 0 0 2.255 4H4.09zm-.582 3.5c.03-.877.138-1.718.312-2.5H1.674a6.958 6.958 0 0 0-.656 2.5h2.49zM4.847 5a12.5 12.5 0 0 0-.338 2.5H7.5V5H4.847zM8.5 5v2.5h2.99a12.495 12.495 0 0 0-.337-2.5H8.5zM4.51 8.5a12.5 12.5 0 0 0 .337 2.5H7.5V8.5H4.51zm3.99 0V11h2.653c.187-.765.306-1.608.338-2.5H8.5zM5.145 12c.138.386.295.744.468 1.068.552 1.035 1.218 1.65 1.887 1.855V12H5.145zm.182 2.472a6.696 6.696 0 0 1-.597-.933A9.268 9.268 0 0 1 4.09 12H2.255a7.024 7.024 0 0 0 3.072 2.472zM3.82 11a13.652 13.652 0 0 1-.312-2.5h-2.49c.062.89.291 1.733.656 2.5H3.82zm6.853 3.472A7.024 7.024 0 0 0 13.745 12H11.91a9.27 9.27 0 0 1-.64 1.539 6.688 6.688 0 0 1-.597.933zM8.5 12v2.923c.67-.204 1.335-.82 1.887-1.855.173-.324.33-.682.468-1.068H8.5zm3.68-1h2.146c.365-.767.594-1.61.656-2.5h-2.49a13.65 13.65 0 0 1-.312 2.5zm2.802-3.5a6.959 6.959 0 0 0-.656-2.5H12.18c.174.782.282 1.623.312 2.5h2.49zM11.27 2.461c.247.464.462.98.64 1.539h1.835a7.024 7.024 0 0 0-3.072-2.472c.218.284.418.598.597.933zM10.855 4a7.966 7.966 0 0 0-.468-1.068C9.835 1.897 9.17 1.282 8.5 1.077V4h2.355z"/>
-      </svg>
-      Domain Mappings
-    </h1>
-    <p class="text-muted mb-0 mt-1">Map custom domains to buckets for static website hosting.</p>
-  </div>
-  <div class="d-none d-md-block">
-    <span class="badge bg-primary bg-opacity-10 text-primary fs-6 px-3 py-2">
-      {{ mappings|length }} mapping{{ 's' if mappings|length != 1 else '' }}
-    </span>
-  </div>
-</div>
-
-<div class="row g-4">
-  <div class="col-lg-4 col-md-5">
-    <div class="card shadow-sm border-0" style="border-radius: 1rem;">
-      <div class="card-header bg-transparent border-0 pt-4 pb-0 px-4">
-        <h5 class="fw-semibold d-flex align-items-center gap-2 mb-1">
-          <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-            <path fill-rule="evenodd" d="M8 2a.5.5 0 0 1 .5.5v5h5a.5.5 0 0 1 0 1h-5v5a.5.5 0 0 1-1 0v-5h-5a.5.5 0 0 1 0-1h5v-5A.5.5 0 0 1 8 2Z"/>
-          </svg>
-          Add Domain Mapping
-        </h5>
-        <p class="text-muted small mb-0">Point a custom domain to a bucket</p>
-      </div>
-      <div class="card-body px-4 pb-4">
-        <form method="POST" action="{{ url_for('ui.create_website_domain') }}" id="createDomainForm">
-          <input type="hidden" name="csrf_token" value="{{ csrf_token() }}"/>
-          <div class="mb-3">
-            <label for="domain" class="form-label fw-medium">Domain</label>
-            <input type="text" class="form-control" id="domain" name="domain" required
-                   placeholder="www.example.com"
-                   pattern="^[a-zA-Z0-9]([a-zA-Z0-9\-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9\-]*[a-zA-Z0-9])?)*$"
-                   title="Enter a valid hostname (e.g. www.example.com). Do not include http:// or trailing slashes.">
-            <div class="form-text">Hostname only &mdash; no <code>http://</code> prefix or trailing slash.</div>
-            <div class="invalid-feedback">Enter a valid hostname like www.example.com</div>
-          </div>
-          <div id="domainPreview" class="alert alert-light border small py-2 px-3 mb-3 d-none">
-            <span class="text-muted">Will be accessible at:</span>
-            <code id="domainPreviewUrl" class="ms-1"></code>
-          </div>
-          <div class="mb-3">
-            <label for="bucket" class="form-label fw-medium">Bucket</label>
-            {% if buckets %}
-            <select class="form-select" id="bucket" name="bucket" required>
-              <option value="" selected disabled>Select a bucket</option>
-              {% for b in buckets %}
-              <option value="{{ b }}">{{ b }}</option>
-              {% endfor %}
-            </select>
-            {% else %}
-            <input type="text" class="form-control" id="bucket" name="bucket" required placeholder="my-site-bucket">
-            {% endif %}
-            <div class="form-text">The bucket must have website hosting enabled.</div>
-          </div>
-          <div class="d-grid">
-            <button type="submit" class="btn btn-primary" id="addMappingBtn">
-              <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1" viewBox="0 0 16 16">
-                <path fill-rule="evenodd" d="M8 2a.5.5 0 0 1 .5.5v5h5a.5.5 0 0 1 0 1h-5v5a.5.5 0 0 1-1 0v-5h-5a.5.5 0 0 1 0-1h5v-5A.5.5 0 0 1 8 2Z"/>
-              </svg>
-              Add Mapping
-            </button>
-          </div>
-        </form>
-      </div>
-    </div>
-
-    <div class="card shadow-sm border-0 mt-4" style="border-radius: 1rem;">
-      <div class="card-body px-4 py-3">
-        <h6 class="fw-semibold mb-2 d-flex align-items-center gap-2">
-          <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-muted" viewBox="0 0 16 16">
-            <path d="M8 16A8 8 0 1 0 8 0a8 8 0 0 0 0 16zm.93-9.412-1 4.705c-.07.34.029.533.304.533.194 0 .487-.07.686-.246l-.088.416c-.287.346-.92.598-1.465.598-.703 0-1.002-.422-.808-1.319l.738-3.468c.064-.293.006-.399-.287-.47l-.451-.081.082-.381 2.29-.287zM8 5.5a1 1 0 1 1 0-2 1 1 0 0 1 0 2z"/>
-          </svg>
-          How it works
-        </h6>
-        <ol class="small text-muted mb-0 ps-3">
-          <li class="mb-1">Enable website hosting on a bucket (Properties tab)</li>
-          <li class="mb-1">Create a domain mapping here</li>
-          <li>Point your DNS (A/CNAME) to this server</li>
-        </ol>
-      </div>
-    </div>
-  </div>
-
-  <div class="col-lg-8 col-md-7">
-    <div class="card shadow-sm border-0" style="border-radius: 1rem;">
-      <div class="card-header bg-transparent border-0 pt-4 pb-0 px-4">
-        <div class="d-flex justify-content-between align-items-center mb-1">
-          <h5 class="fw-semibold d-flex align-items-center gap-2 mb-0">
-            <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-muted" viewBox="0 0 16 16">
-              <path d="M4.715 6.542 3.343 7.914a3 3 0 1 0 4.243 4.243l1.828-1.829A3 3 0 0 0 8.586 5.5L8 6.086a1.002 1.002 0 0 0-.154.199 2 2 0 0 1 .861 3.337L6.88 11.45a2 2 0 1 1-2.83-2.83l.793-.792a4.018 4.018 0 0 1-.128-1.287z"/>
-              <path d="M6.586 4.672A3 3 0 0 0 7.414 9.5l.775-.776a2 2 0 0 1-.896-3.346L9.12 3.55a2 2 0 1 1 2.83 2.83l-.793.792c.112.42.155.855.128 1.287l1.372-1.372a3 3 0 1 0-4.243-4.243L6.586 4.672z"/>
-            </svg>
-            Active Mappings
-          </h5>
-        </div>
-        <p class="text-muted small mb-0">Domains currently serving website content</p>
-        {% if mappings|length > 3 %}
-        <div class="mt-3">
-          <div class="search-input-wrapper">
-            <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="search-icon" viewBox="0 0 16 16">
-              <path d="M11.742 10.344a6.5 6.5 0 1 0-1.397 1.398h-.001c.03.04.062.078.098.115l3.85 3.85a1 1 0 0 0 1.415-1.414l-3.85-3.85a1.007 1.007 0 0 0-.115-.1zM12 6.5a5.5 5.5 0 1 1-11 0 5.5 5.5 0 0 1 11 0z"/>
-            </svg>
-            <input type="text" class="form-control" id="domainSearch" placeholder="Filter by domain or bucket..." autocomplete="off" />
-          </div>
-        </div>
-        {% endif %}
-      </div>
-      <div class="card-body px-4 pb-4">
-        {% if mappings %}
-        <div class="table-responsive">
-          <table class="table table-hover align-middle mb-0" id="domainTable">
-            <thead class="table-light">
-              <tr>
-                <th scope="col">Domain</th>
-                <th scope="col">Bucket</th>
-                <th scope="col" class="text-end">Actions</th>
-              </tr>
-            </thead>
-            <tbody>
-              {% for m in mappings %}
-              <tr data-domain="{{ m.domain }}" data-bucket="{{ m.bucket }}">
-                <td>
-                  <div class="d-flex align-items-center gap-2">
-                    <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="text-success flex-shrink-0" viewBox="0 0 16 16">
-                      <path d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8zm7.5-6.923c-.67.204-1.335.82-1.887 1.855A7.97 7.97 0 0 0 5.145 4H7.5V1.077zM4.09 4a9.267 9.267 0 0 1 .64-1.539 6.7 6.7 0 0 1 .597-.933A7.025 7.025 0 0 0 2.255 4H4.09zm-.582 3.5c.03-.877.138-1.718.312-2.5H1.674a6.958 6.958 0 0 0-.656 2.5h2.49zM4.847 5a12.5 12.5 0 0 0-.338 2.5H7.5V5H4.847zM8.5 5v2.5h2.99a12.495 12.495 0 0 0-.337-2.5H8.5zM4.51 8.5a12.5 12.5 0 0 0 .337 2.5H7.5V8.5H4.51zm3.99 0V11h2.653c.187-.765.306-1.608.338-2.5H8.5zM5.145 12c.138.386.295.744.468 1.068.552 1.035 1.218 1.65 1.887 1.855V12H5.145zm.182 2.472a6.696 6.696 0 0 1-.597-.933A9.268 9.268 0 0 1 4.09 12H2.255a7.024 7.024 0 0 0 3.072 2.472zM3.82 11a13.652 13.652 0 0 1-.312-2.5h-2.49c.062.89.291 1.733.656 2.5H3.82zm6.853 3.472A7.024 7.024 0 0 0 13.745 12H11.91a9.27 9.27 0 0 1-.64 1.539 6.688 6.688 0 0 1-.597.933zM8.5 12v2.923c.67-.204 1.335-.82 1.887-1.855.173-.324.33-.682.468-1.068H8.5zm3.68-1h2.146c.365-.767.594-1.61.656-2.5h-2.49a13.65 13.65 0 0 1-.312 2.5zm2.802-3.5a6.959 6.959 0 0 0-.656-2.5H12.18c.174.782.282 1.623.312 2.5h2.49zM11.27 2.461c.247.464.462.98.64 1.539h1.835a7.024 7.024 0 0 0-3.072-2.472c.218.284.418.598.597.933zM10.855 4a7.966 7.966 0 0 0-.468-1.068C9.835 1.897 9.17 1.282 8.5 1.077V4h2.355z"/>
-                    </svg>
-                    <div>
-                      <code class="fw-medium">{{ m.domain }}</code>
-                      <div class="text-muted small">http://{{ m.domain }}</div>
-                    </div>
-                  </div>
-                </td>
-                <td><span class="badge bg-primary bg-opacity-10 text-primary">{{ m.bucket }}</span></td>
-                <td class="text-end">
-                  <div class="btn-group btn-group-sm" role="group">
-                    <button type="button" class="btn btn-outline-secondary"
-                            data-bs-toggle="modal"
-                            data-bs-target="#editDomainModal"
-                            data-domain="{{ m.domain }}"
-                            data-bucket="{{ m.bucket }}"
-                            title="Edit mapping">
-                      <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" viewBox="0 0 16 16">
-                        <path d="M12.146.146a.5.5 0 0 1 .708 0l3 3a.5.5 0 0 1 0 .708l-10 10a.5.5 0 0 1-.168.11l-5 2a.5.5 0 0 1-.65-.65l2-5a.5.5 0 0 1 .11-.168l10-10zM11.207 2.5 13.5 4.793 14.793 3.5 12.5 1.207 11.207 2.5zm1.586 3L10.5 3.207 4 9.707V10h.5a.5.5 0 0 1 .5.5v.5h.5a.5.5 0 0 1 .5.5v.5h.293l6.5-6.5z"/>
-                      </svg>
-                    </button>
-                    <button type="button" class="btn btn-outline-danger"
-                            data-bs-toggle="modal"
-                            data-bs-target="#deleteDomainModal"
-                            data-domain="{{ m.domain }}"
-                            data-bucket="{{ m.bucket }}"
-                            title="Delete mapping">
-                      <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" viewBox="0 0 16 16">
-                        <path d="M5.5 5.5A.5.5 0 0 1 6 6v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm2.5 0a.5.5 0 0 1 .5.5v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm3 .5a.5.5 0 0 0-1 0v6a.5.5 0 0 0 1 0V6z"/>
-                        <path fill-rule="evenodd" d="M14.5 3a1 1 0 0 1-1 1H13v9a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V4h-.5a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1H6a1 1 0 0 1 1-1h2a1 1 0 0 1 1 1h3.5a1 1 0 0 1 1 1v1zM4.118 4 4 4.059V13a1 1 0 0 0 1 1h6a1 1 0 0 0 1-1V4.059L11.882 4H4.118zM2.5 3V2h11v1h-11z"/>
-                      </svg>
-                    </button>
-                  </div>
-                </td>
-              </tr>
-              {% endfor %}
-            </tbody>
-          </table>
-        </div>
-        <div id="noSearchResults" class="text-center py-4 d-none">
-          <p class="text-muted mb-0">No mappings match your search.</p>
-        </div>
-        {% else %}
-        <div class="empty-state text-center py-5">
-          <div class="empty-state-icon mx-auto mb-3">
-            <svg xmlns="http://www.w3.org/2000/svg" width="48" height="48" fill="currentColor" class="text-muted" viewBox="0 0 16 16">
-              <path d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8zm7.5-6.923c-.67.204-1.335.82-1.887 1.855A7.97 7.97 0 0 0 5.145 4H7.5V1.077zM4.09 4a9.267 9.267 0 0 1 .64-1.539 6.7 6.7 0 0 1 .597-.933A7.025 7.025 0 0 0 2.255 4H4.09zm-.582 3.5c.03-.877.138-1.718.312-2.5H1.674a6.958 6.958 0 0 0-.656 2.5h2.49zM4.847 5a12.5 12.5 0 0 0-.338 2.5H7.5V5H4.847zM8.5 5v2.5h2.99a12.495 12.495 0 0 0-.337-2.5H8.5zM4.51 8.5a12.5 12.5 0 0 0 .337 2.5H7.5V8.5H4.51zm3.99 0V11h2.653c.187-.765.306-1.608.338-2.5H8.5zM5.145 12c.138.386.295.744.468 1.068.552 1.035 1.218 1.65 1.887 1.855V12H5.145zm.182 2.472a6.696 6.696 0 0 1-.597-.933A9.268 9.268 0 0 1 4.09 12H2.255a7.024 7.024 0 0 0 3.072 2.472zM3.82 11a13.652 13.652 0 0 1-.312-2.5h-2.49c.062.89.291 1.733.656 2.5H3.82zm6.853 3.472A7.024 7.024 0 0 0 13.745 12H11.91a9.27 9.27 0 0 1-.64 1.539 6.688 6.688 0 0 1-.597.933zM8.5 12v2.923c.67-.204 1.335-.82 1.887-1.855.173-.324.33-.682.468-1.068H8.5zm3.68-1h2.146c.365-.767.594-1.61.656-2.5h-2.49a13.65 13.65 0 0 1-.312 2.5zm2.802-3.5a6.959 6.959 0 0 0-.656-2.5H12.18c.174.782.282 1.623.312 2.5h2.49zM11.27 2.461c.247.464.462.98.64 1.539h1.835a7.024 7.024 0 0 0-3.072-2.472c.218.284.418.598.597.933zM10.855 4a7.966 7.966 0 0 0-.468-1.068C9.835 1.897 9.17 1.282 8.5 1.077V4h2.355z"/>
-            </svg>
-          </div>
-          <h5 class="fw-semibold mb-2">No domain mappings yet</h5>
-          <p class="text-muted mb-0">Add your first domain mapping to serve a bucket as a static website.</p>
-        </div>
-        {% endif %}
-      </div>
-    </div>
-  </div>
-</div>
-
-<div class="modal fade" id="editDomainModal" tabindex="-1" aria-hidden="true">
-  <div class="modal-dialog modal-dialog-centered">
-    <div class="modal-content">
-      <div class="modal-header border-0 pb-0">
-        <h5 class="modal-title fw-semibold">
-          <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-primary" viewBox="0 0 16 16">
-            <path d="M12.146.146a.5.5 0 0 1 .708 0l3 3a.5.5 0 0 1 0 .708l-10 10a.5.5 0 0 1-.168.11l-5 2a.5.5 0 0 1-.65-.65l2-5a.5.5 0 0 1 .11-.168l10-10zM11.207 2.5 13.5 4.793 14.793 3.5 12.5 1.207 11.207 2.5zm1.586 3L10.5 3.207 4 9.707V10h.5a.5.5 0 0 1 .5.5v.5h.5a.5.5 0 0 1 .5.5v.5h.293l6.5-6.5zm-9.761 5.175-.106.106-1.528 3.821 3.821-1.528.106-.106A.5.5 0 0 1 5 12.5V12h-.5a.5.5 0 0 1-.5-.5V11h-.5a.5.5 0 0 1-.468-.325z"/>
-          </svg>
-          Edit Domain Mapping
-        </h5>
-        <button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="Close"></button>
-      </div>
-      <form method="POST" id="editDomainForm">
-        <input type="hidden" name="csrf_token" value="{{ csrf_token() }}"/>
-        <div class="modal-body">
-          <div class="mb-3">
-            <label class="form-label fw-medium">Domain</label>
-            <input type="text" class="form-control bg-light" id="editDomainName" disabled>
-          </div>
-          <div class="mb-3">
-            <label for="editBucket" class="form-label fw-medium">Bucket</label>
-            {% if buckets %}
-            <select class="form-select" id="editBucket" name="bucket" required>
-              {% for b in buckets %}
-              <option value="{{ b }}">{{ b }}</option>
-              {% endfor %}
-            </select>
-            {% else %}
-            <input type="text" class="form-control" id="editBucket" name="bucket" required>
-            {% endif %}
-          </div>
-        </div>
-        <div class="modal-footer">
-          <button type="button" class="btn btn-outline-secondary" data-bs-dismiss="modal">Cancel</button>
-          <button type="submit" class="btn btn-primary">
-            <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1" viewBox="0 0 16 16">
-              <path d="M10.97 4.97a.75.75 0 0 1 1.07 1.05l-3.99 4.99a.75.75 0 0 1-1.08.02L4.324 8.384a.75.75 0 1 1 1.06-1.06l2.094 2.093 3.473-4.425a.267.267 0 0 1 .02-.022z"/>
-            </svg>
-            Save
-          </button>
-        </div>
-      </form>
-    </div>
-  </div>
-</div>
-
-<div class="modal fade" id="deleteDomainModal" tabindex="-1" aria-hidden="true">
-  <div class="modal-dialog modal-dialog-centered">
-    <div class="modal-content">
-      <form method="POST" id="deleteDomainForm">
-        <input type="hidden" name="csrf_token" value="{{ csrf_token() }}"/>
-        <div class="modal-header border-0 pb-0">
-          <h5 class="modal-title fw-semibold">
-            <svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" class="text-danger" viewBox="0 0 16 16">
-              <path d="M5.5 5.5A.5.5 0 0 1 6 6v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm2.5 0a.5.5 0 0 1 .5.5v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm3 .5a.5.5 0 0 0-1 0v6a.5.5 0 0 0 1 0V6z"/>
-              <path fill-rule="evenodd" d="M14.5 3a1 1 0 0 1-1 1H13v9a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V4h-.5a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1H6a1 1 0 0 1 1-1h2a1 1 0 0 1 1 1h3.5a1 1 0 0 1 1 1v1zM4.118 4 4 4.059V13a1 1 0 0 0 1 1h6a1 1 0 0 0 1-1V4.059L11.882 4H4.118zM2.5 3V2h11v1h-11z"/>
-            </svg>
-            Delete Domain Mapping
-          </h5>
-          <button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="Close"></button>
-        </div>
-        <div class="modal-body">
-          <p>Are you sure you want to delete the mapping for <strong><code id="deleteDomainName"></code></strong>?</p>
-          <p class="text-muted small mb-0">Mapped to bucket: <code id="deleteBucketName"></code></p>
-          <div class="alert alert-warning d-flex align-items-start small mt-3 mb-0" role="alert">
-            <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" class="flex-shrink-0 me-2 mt-0" viewBox="0 0 16 16">
-              <path d="M8.982 1.566a1.13 1.13 0 0 0-1.96 0L.165 13.233c-.457.778.091 1.767.98 1.767h13.713c.889 0 1.438-.99.98-1.767L8.982 1.566zM8 5c.535 0 .954.462.9.995l-.35 3.507a.552.552 0 0 1-1.1 0L7.1 5.995A.905.905 0 0 1 8 5zm.002 6a1 1 0 1 1 0 2 1 1 0 0 1 0-2z"/>
-            </svg>
-            <div>This domain will stop serving website content immediately.</div>
-          </div>
-        </div>
-        <div class="modal-footer">
-          <button type="button" class="btn btn-outline-secondary" data-bs-dismiss="modal">Cancel</button>
-          <button type="submit" class="btn btn-danger">
-            <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" fill="currentColor" class="me-1" viewBox="0 0 16 16">
-              <path d="M5.5 5.5A.5.5 0 0 1 6 6v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm2.5 0a.5.5 0 0 1 .5.5v6a.5.5 0 0 1-1 0V6a.5.5 0 0 1 .5-.5zm3 .5a.5.5 0 0 0-1 0v6a.5.5 0 0 0 1 0V6z"/>
-              <path fill-rule="evenodd" d="M14.5 3a1 1 0 0 1-1 1H13v9a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V4h-.5a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1H6a1 1 0 0 1 1-1h2a1 1 0 0 1 1 1h3.5a1 1 0 0 1 1 1v1zM4.118 4 4 4.059V13a1 1 0 0 0 1 1h6a1 1 0 0 0 1-1V4.059L11.882 4H4.118zM2.5 3V2h11v1h-11z"/>
-            </svg>
-            Delete
-          </button>
-        </div>
-      </form>
-    </div>
-  </div>
-</div>
-{% endblock %}
-
-{% block extra_scripts %}
-<script>
-(function () {
-  function normalizeDomain(val) {
-    val = val.trim().toLowerCase();
-    if (val.indexOf('https://') === 0) val = val.substring(8);
-    else if (val.indexOf('http://') === 0) val = val.substring(7);
-    var slashIdx = val.indexOf('/');
-    if (slashIdx !== -1) val = val.substring(0, slashIdx);
-    var qIdx = val.indexOf('?');
-    if (qIdx !== -1) val = val.substring(0, qIdx);
-    var hIdx = val.indexOf('#');
-    if (hIdx !== -1) val = val.substring(0, hIdx);
-    var colonIdx = val.indexOf(':');
-    if (colonIdx !== -1) val = val.substring(0, colonIdx);
-    return val;
-  }
-
-  var domainInput = document.getElementById('domain');
-  var preview = document.getElementById('domainPreview');
-  var previewUrl = document.getElementById('domainPreviewUrl');
-  if (domainInput && preview) {
-    domainInput.addEventListener('input', function () {
-      var clean = normalizeDomain(this.value);
-      if (clean && /^[a-z0-9]([a-z0-9\-]*[a-z0-9])?(\.[a-z0-9]([a-z0-9\-]*[a-z0-9])?)*$/.test(clean)) {
-        previewUrl.textContent = 'http://' + clean;
-        preview.classList.remove('d-none');
-      } else {
-        preview.classList.add('d-none');
-      }
-    });
-
-    var createForm = document.getElementById('createDomainForm');
-    if (createForm) {
-      createForm.addEventListener('submit', function () {
-        domainInput.value = normalizeDomain(domainInput.value);
-      });
-    }
-  }
-
-  var editModal = document.getElementById('editDomainModal');
-  if (editModal) {
-    editModal.addEventListener('show.bs.modal', function (event) {
-      var btn = event.relatedTarget;
-      var domain = btn.getAttribute('data-domain');
-      var bucket = btn.getAttribute('data-bucket');
-      document.getElementById('editDomainName').value = domain;
-      var editBucket = document.getElementById('editBucket');
-      editBucket.value = bucket;
-      document.getElementById('editDomainForm').action = '{{ url_for("ui.update_website_domain", domain="__DOMAIN__") }}'.replace('__DOMAIN__', encodeURIComponent(domain));
-    });
-  }
-
-  var deleteModal = document.getElementById('deleteDomainModal');
-  if (deleteModal) {
-    deleteModal.addEventListener('show.bs.modal', function (event) {
-      var btn = event.relatedTarget;
-      var domain = btn.getAttribute('data-domain');
-      var bucket = btn.getAttribute('data-bucket') || '';
-      document.getElementById('deleteDomainName').textContent = domain;
-      document.getElementById('deleteBucketName').textContent = bucket;
-      document.getElementById('deleteDomainForm').action = '{{ url_for("ui.delete_website_domain", domain="__DOMAIN__") }}'.replace('__DOMAIN__', encodeURIComponent(domain));
-    });
-  }
-
-  var searchInput = document.getElementById('domainSearch');
-  if (searchInput) {
-    searchInput.addEventListener('input', function () {
-      var q = this.value.toLowerCase();
-      var rows = document.querySelectorAll('#domainTable tbody tr');
-      var visible = 0;
-      rows.forEach(function (row) {
-        var domain = (row.getAttribute('data-domain') || '').toLowerCase();
-        var bucket = (row.getAttribute('data-bucket') || '').toLowerCase();
-        var match = !q || domain.indexOf(q) !== -1 || bucket.indexOf(q) !== -1;
-        row.style.display = match ? '' : 'none';
-        if (match) visible++;
-      });
-      var noResults = document.getElementById('noSearchResults');
-      if (noResults) {
-        noResults.classList.toggle('d-none', visible > 0);
-      }
-    });
-  }
-})();
-</script>
-{% endblock %}

python/tests/test_conditional_headers.py

@@ -1,156 +0,0 @@
-import hashlib
-import time
-
-import pytest
-
-
-@pytest.fixture()
-def bucket(client, signer):
-    headers = signer("PUT", "/cond-test")
-    client.put("/cond-test", headers=headers)
-    return "cond-test"
-
-
-@pytest.fixture()
-def uploaded(client, signer, bucket):
-    body = b"hello conditional"
-    etag = hashlib.md5(body).hexdigest()
-    headers = signer("PUT", f"/{bucket}/obj.txt", body=body)
-    resp = client.put(f"/{bucket}/obj.txt", headers=headers, data=body)
-    last_modified = resp.headers.get("Last-Modified")
-    return {"etag": etag, "last_modified": last_modified}
-
-
-class TestIfMatch:
-    def test_get_matching_etag(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={"If-Match": f'"{uploaded["etag"]}"'})
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 200
-
-    def test_get_non_matching_etag(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={"If-Match": '"wrongetag"'})
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 412
-
-    def test_head_matching_etag(self, client, signer, bucket, uploaded):
-        headers = signer("HEAD", f"/{bucket}/obj.txt", headers={"If-Match": f'"{uploaded["etag"]}"'})
-        resp = client.head(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 200
-
-    def test_head_non_matching_etag(self, client, signer, bucket, uploaded):
-        headers = signer("HEAD", f"/{bucket}/obj.txt", headers={"If-Match": '"wrongetag"'})
-        resp = client.head(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 412
-
-    def test_wildcard_match(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={"If-Match": "*"})
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 200
-
-    def test_multiple_etags_one_matches(self, client, signer, bucket, uploaded):
-        etag_list = f'"bad1", "{uploaded["etag"]}", "bad2"'
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={"If-Match": etag_list})
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 200
-
-    def test_multiple_etags_none_match(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={"If-Match": '"bad1", "bad2"'})
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 412
-
-
-class TestIfNoneMatch:
-    def test_get_matching_etag_returns_304(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={"If-None-Match": f'"{uploaded["etag"]}"'})
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 304
-        assert uploaded["etag"] in resp.headers.get("ETag", "")
-
-    def test_get_non_matching_etag_returns_200(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={"If-None-Match": '"wrongetag"'})
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 200
-
-    def test_head_matching_etag_returns_304(self, client, signer, bucket, uploaded):
-        headers = signer("HEAD", f"/{bucket}/obj.txt", headers={"If-None-Match": f'"{uploaded["etag"]}"'})
-        resp = client.head(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 304
-
-    def test_head_non_matching_etag_returns_200(self, client, signer, bucket, uploaded):
-        headers = signer("HEAD", f"/{bucket}/obj.txt", headers={"If-None-Match": '"wrongetag"'})
-        resp = client.head(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 200
-
-    def test_wildcard_returns_304(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={"If-None-Match": "*"})
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 304
-
-
-class TestIfModifiedSince:
-    def test_not_modified_returns_304(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={"If-Modified-Since": "Sun, 01 Jan 2034 00:00:00 GMT"})
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 304
-        assert "ETag" in resp.headers
-
-    def test_modified_returns_200(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={"If-Modified-Since": "Sun, 01 Jan 2000 00:00:00 GMT"})
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 200
-
-    def test_head_not_modified(self, client, signer, bucket, uploaded):
-        headers = signer("HEAD", f"/{bucket}/obj.txt", headers={"If-Modified-Since": "Sun, 01 Jan 2034 00:00:00 GMT"})
-        resp = client.head(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 304
-
-    def test_if_none_match_takes_precedence(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={
-            "If-None-Match": '"wrongetag"',
-            "If-Modified-Since": "Sun, 01 Jan 2034 00:00:00 GMT",
-        })
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 200
-
-
-class TestIfUnmodifiedSince:
-    def test_unmodified_returns_200(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={"If-Unmodified-Since": "Sun, 01 Jan 2034 00:00:00 GMT"})
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 200
-
-    def test_modified_returns_412(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={"If-Unmodified-Since": "Sun, 01 Jan 2000 00:00:00 GMT"})
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 412
-
-    def test_head_modified_returns_412(self, client, signer, bucket, uploaded):
-        headers = signer("HEAD", f"/{bucket}/obj.txt", headers={"If-Unmodified-Since": "Sun, 01 Jan 2000 00:00:00 GMT"})
-        resp = client.head(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 412
-
-    def test_if_match_takes_precedence(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={
-            "If-Match": f'"{uploaded["etag"]}"',
-            "If-Unmodified-Since": "Sun, 01 Jan 2000 00:00:00 GMT",
-        })
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 200
-
-
-class TestConditionalWithRange:
-    def test_if_match_with_range(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={
-            "If-Match": f'"{uploaded["etag"]}"',
-            "Range": "bytes=0-4",
-        })
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 206
-
-    def test_if_match_fails_with_range(self, client, signer, bucket, uploaded):
-        headers = signer("GET", f"/{bucket}/obj.txt", headers={
-            "If-Match": '"wrongetag"',
-            "Range": "bytes=0-4",
-        })
-        resp = client.get(f"/{bucket}/obj.txt", headers=headers)
-        assert resp.status_code == 412

python/tests/test_gc.py

@@ -1,356 +0,0 @@
-import json
-import os
-import time
-from pathlib import Path
-
-import pytest
-
-from app.gc import GarbageCollector, GCResult
-
-
-@pytest.fixture
-def storage_root(tmp_path):
-    root = tmp_path / "data"
-    root.mkdir()
-    sys_root = root / ".myfsio.sys"
-    sys_root.mkdir()
-    (sys_root / "config").mkdir(parents=True)
-    (sys_root / "tmp").mkdir()
-    (sys_root / "multipart").mkdir()
-    (sys_root / "buckets").mkdir()
-    return root
-
-
-@pytest.fixture
-def gc(storage_root):
-    return GarbageCollector(
-        storage_root=storage_root,
-        interval_hours=1.0,
-        temp_file_max_age_hours=1.0,
-        multipart_max_age_days=1,
-        lock_file_max_age_hours=0.5,
-        dry_run=False,
-    )
-
-
-def _make_old(path, hours=48):
-    old_time = time.time() - hours * 3600
-    os.utime(path, (old_time, old_time))
-
-
-class TestTempFileCleanup:
-    def test_old_temp_files_deleted(self, storage_root, gc):
-        tmp_dir = storage_root / ".myfsio.sys" / "tmp"
-        old_file = tmp_dir / "abc123.tmp"
-        old_file.write_bytes(b"x" * 1000)
-        _make_old(old_file, hours=48)
-
-        result = gc.run_now()
-        assert result.temp_files_deleted == 1
-        assert result.temp_bytes_freed == 1000
-        assert not old_file.exists()
-
-    def test_recent_temp_files_kept(self, storage_root, gc):
-        tmp_dir = storage_root / ".myfsio.sys" / "tmp"
-        new_file = tmp_dir / "recent.tmp"
-        new_file.write_bytes(b"data")
-
-        result = gc.run_now()
-        assert result.temp_files_deleted == 0
-        assert new_file.exists()
-
-    def test_dry_run_keeps_files(self, storage_root, gc):
-        gc.dry_run = True
-        tmp_dir = storage_root / ".myfsio.sys" / "tmp"
-        old_file = tmp_dir / "stale.tmp"
-        old_file.write_bytes(b"x" * 500)
-        _make_old(old_file, hours=48)
-
-        result = gc.run_now()
-        assert result.temp_files_deleted == 1
-        assert result.temp_bytes_freed == 500
-        assert old_file.exists()
-
-
-class TestMultipartCleanup:
-    def test_old_orphaned_multipart_deleted(self, storage_root, gc):
-        bucket = storage_root / "test-bucket"
-        bucket.mkdir()
-        mp_root = storage_root / ".myfsio.sys" / "multipart" / "test-bucket"
-        mp_root.mkdir(parents=True)
-        upload_dir = mp_root / "upload-123"
-        upload_dir.mkdir()
-        manifest = upload_dir / "manifest.json"
-        manifest.write_text(json.dumps({"upload_id": "upload-123", "object_key": "foo.txt"}))
-        part = upload_dir / "part-00001.part"
-        part.write_bytes(b"x" * 2000)
-        _make_old(manifest, hours=200)
-        _make_old(part, hours=200)
-        _make_old(upload_dir, hours=200)
-
-        result = gc.run_now()
-        assert result.multipart_uploads_deleted == 1
-        assert result.multipart_bytes_freed > 0
-        assert not upload_dir.exists()
-
-    def test_recent_multipart_kept(self, storage_root, gc):
-        bucket = storage_root / "test-bucket"
-        bucket.mkdir()
-        mp_root = storage_root / ".myfsio.sys" / "multipart" / "test-bucket"
-        mp_root.mkdir(parents=True)
-        upload_dir = mp_root / "upload-new"
-        upload_dir.mkdir()
-        manifest = upload_dir / "manifest.json"
-        manifest.write_text(json.dumps({"upload_id": "upload-new", "object_key": "bar.txt"}))
-
-        result = gc.run_now()
-        assert result.multipart_uploads_deleted == 0
-        assert upload_dir.exists()
-
-    def test_legacy_multipart_cleaned(self, storage_root, gc):
-        bucket = storage_root / "test-bucket"
-        bucket.mkdir()
-        legacy_mp = bucket / ".multipart" / "upload-old"
-        legacy_mp.mkdir(parents=True)
-        part = legacy_mp / "part-00001.part"
-        part.write_bytes(b"y" * 500)
-        _make_old(part, hours=200)
-        _make_old(legacy_mp, hours=200)
-
-        result = gc.run_now()
-        assert result.multipart_uploads_deleted == 1
-
-
-class TestLockFileCleanup:
-    def test_stale_lock_files_deleted(self, storage_root, gc):
-        locks_dir = storage_root / ".myfsio.sys" / "buckets" / "test-bucket" / "locks"
-        locks_dir.mkdir(parents=True)
-        lock = locks_dir / "some_key.lock"
-        lock.write_text("")
-        _make_old(lock, hours=2)
-
-        result = gc.run_now()
-        assert result.lock_files_deleted == 1
-        assert not lock.exists()
-
-    def test_recent_lock_kept(self, storage_root, gc):
-        locks_dir = storage_root / ".myfsio.sys" / "buckets" / "test-bucket" / "locks"
-        locks_dir.mkdir(parents=True)
-        lock = locks_dir / "active.lock"
-        lock.write_text("")
-
-        result = gc.run_now()
-        assert result.lock_files_deleted == 0
-        assert lock.exists()
-
-
-class TestOrphanedMetadataCleanup:
-    def test_legacy_orphaned_metadata_deleted(self, storage_root, gc):
-        bucket = storage_root / "test-bucket"
-        bucket.mkdir()
-        meta_dir = bucket / ".meta"
-        meta_dir.mkdir()
-        orphan = meta_dir / "deleted_file.txt.meta.json"
-        orphan.write_text(json.dumps({"etag": "abc"}))
-
-        result = gc.run_now()
-        assert result.orphaned_metadata_deleted == 1
-        assert not orphan.exists()
-
-    def test_valid_metadata_kept(self, storage_root, gc):
-        bucket = storage_root / "test-bucket"
-        bucket.mkdir()
-        obj = bucket / "exists.txt"
-        obj.write_text("hello")
-        meta_dir = bucket / ".meta"
-        meta_dir.mkdir()
-        meta = meta_dir / "exists.txt.meta.json"
-        meta.write_text(json.dumps({"etag": "abc"}))
-
-        result = gc.run_now()
-        assert result.orphaned_metadata_deleted == 0
-        assert meta.exists()
-
-    def test_index_orphaned_entries_cleaned(self, storage_root, gc):
-        bucket = storage_root / "test-bucket"
-        bucket.mkdir()
-        obj = bucket / "keep.txt"
-        obj.write_text("hello")
-
-        meta_dir = storage_root / ".myfsio.sys" / "buckets" / "test-bucket" / "meta"
-        meta_dir.mkdir(parents=True)
-        index = meta_dir / "_index.json"
-        index.write_text(json.dumps({"keep.txt": {"etag": "a"}, "gone.txt": {"etag": "b"}}))
-
-        result = gc.run_now()
-        assert result.orphaned_metadata_deleted == 1
-
-        updated = json.loads(index.read_text())
-        assert "keep.txt" in updated
-        assert "gone.txt" not in updated
-
-
-class TestOrphanedVersionsCleanup:
-    def test_orphaned_versions_deleted(self, storage_root, gc):
-        bucket = storage_root / "test-bucket"
-        bucket.mkdir()
-        versions_dir = storage_root / ".myfsio.sys" / "buckets" / "test-bucket" / "versions" / "deleted_obj.txt"
-        versions_dir.mkdir(parents=True)
-        v_bin = versions_dir / "v1.bin"
-        v_json = versions_dir / "v1.json"
-        v_bin.write_bytes(b"old data" * 100)
-        v_json.write_text(json.dumps({"version_id": "v1", "size": 800}))
-
-        result = gc.run_now()
-        assert result.orphaned_versions_deleted == 2
-        assert result.orphaned_version_bytes_freed == 800
-
-    def test_active_versions_kept(self, storage_root, gc):
-        bucket = storage_root / "test-bucket"
-        bucket.mkdir()
-        obj = bucket / "active.txt"
-        obj.write_text("current")
-        versions_dir = storage_root / ".myfsio.sys" / "buckets" / "test-bucket" / "versions" / "active.txt"
-        versions_dir.mkdir(parents=True)
-        v_bin = versions_dir / "v1.bin"
-        v_bin.write_bytes(b"old version")
-
-        result = gc.run_now()
-        assert result.orphaned_versions_deleted == 0
-        assert v_bin.exists()
-
-
-class TestEmptyDirCleanup:
-    def test_empty_dirs_removed(self, storage_root, gc):
-        empty = storage_root / ".myfsio.sys" / "buckets" / "test-bucket" / "locks" / "sub"
-        empty.mkdir(parents=True)
-
-        result = gc.run_now()
-        assert result.empty_dirs_removed > 0
-        assert not empty.exists()
-
-
-class TestHistory:
-    def test_history_recorded(self, storage_root, gc):
-        gc.run_now()
-        history = gc.get_history()
-        assert len(history) == 1
-        assert "result" in history[0]
-        assert "timestamp" in history[0]
-
-    def test_multiple_runs(self, storage_root, gc):
-        gc.run_now()
-        gc.run_now()
-        gc.run_now()
-        history = gc.get_history()
-        assert len(history) == 3
-        assert history[0]["timestamp"] >= history[1]["timestamp"]
-
-
-class TestStatus:
-    def test_get_status(self, storage_root, gc):
-        status = gc.get_status()
-        assert status["interval_hours"] == 1.0
-        assert status["dry_run"] is False
-        assert status["temp_file_max_age_hours"] == 1.0
-        assert status["multipart_max_age_days"] == 1
-        assert status["lock_file_max_age_hours"] == 0.5
-
-
-class TestGCResult:
-    def test_total_bytes_freed(self):
-        r = GCResult(temp_bytes_freed=100, multipart_bytes_freed=200, orphaned_version_bytes_freed=300)
-        assert r.total_bytes_freed == 600
-
-    def test_has_work(self):
-        assert not GCResult().has_work
-        assert GCResult(temp_files_deleted=1).has_work
-        assert GCResult(lock_files_deleted=1).has_work
-        assert GCResult(empty_dirs_removed=1).has_work
-
-
-class TestAdminAPI:
-    @pytest.fixture
-    def gc_app(self, tmp_path):
-        from app import create_api_app
-        storage_root = tmp_path / "data"
-        iam_config = tmp_path / "iam.json"
-        bucket_policies = tmp_path / "bucket_policies.json"
-        iam_payload = {
-            "users": [
-                {
-                    "access_key": "admin",
-                    "secret_key": "adminsecret",
-                    "display_name": "Admin",
-                    "policies": [{"bucket": "*", "actions": ["list", "read", "write", "delete", "policy", "iam:*"]}],
-                }
-            ]
-        }
-        iam_config.write_text(json.dumps(iam_payload))
-        flask_app = create_api_app({
-            "TESTING": True,
-            "SECRET_KEY": "testing",
-            "STORAGE_ROOT": storage_root,
-            "IAM_CONFIG": iam_config,
-            "BUCKET_POLICY_PATH": bucket_policies,
-            "GC_ENABLED": True,
-            "GC_INTERVAL_HOURS": 1.0,
-        })
-        yield flask_app
-        gc = flask_app.extensions.get("gc")
-        if gc:
-            gc.stop()
-
-    def test_gc_status(self, gc_app):
-        client = gc_app.test_client()
-        resp = client.get("/admin/gc/status", headers={"X-Access-Key": "admin", "X-Secret-Key": "adminsecret"})
-        assert resp.status_code == 200
-        data = resp.get_json()
-        assert data["enabled"] is True
-
-    def test_gc_run(self, gc_app):
-        client = gc_app.test_client()
-        resp = client.post(
-            "/admin/gc/run",
-            headers={"X-Access-Key": "admin", "X-Secret-Key": "adminsecret"},
-            content_type="application/json",
-        )
-        assert resp.status_code == 200
-        data = resp.get_json()
-        assert data["status"] == "started"
-
-    def test_gc_dry_run(self, gc_app):
-        client = gc_app.test_client()
-        resp = client.post(
-            "/admin/gc/run",
-            headers={"X-Access-Key": "admin", "X-Secret-Key": "adminsecret"},
-            data=json.dumps({"dry_run": True}),
-            content_type="application/json",
-        )
-        assert resp.status_code == 200
-        data = resp.get_json()
-        assert data["status"] == "started"
-
-    def test_gc_history(self, gc_app):
-        import time
-        client = gc_app.test_client()
-        client.post("/admin/gc/run", headers={"X-Access-Key": "admin", "X-Secret-Key": "adminsecret"})
-        for _ in range(50):
-            time.sleep(0.1)
-            status = client.get("/admin/gc/status", headers={"X-Access-Key": "admin", "X-Secret-Key": "adminsecret"}).get_json()
-            if not status.get("scanning"):
-                break
-        resp = client.get("/admin/gc/history", headers={"X-Access-Key": "admin", "X-Secret-Key": "adminsecret"})
-        assert resp.status_code == 200
-        data = resp.get_json()
-        assert len(data["executions"]) >= 1
-
-    def test_gc_requires_admin(self, gc_app):
-        iam = gc_app.extensions["iam"]
-        user = iam.create_user(display_name="Regular")
-        client = gc_app.test_client()
-        resp = client.get(
-            "/admin/gc/status",
-            headers={"X-Access-Key": user["access_key"], "X-Secret-Key": user["secret_key"]},
-        )
-        assert resp.status_code == 403

python/tests/test_integrity.py

@@ -1,788 +0,0 @@
-import hashlib
-import json
-import os
-import sys
-import time
-from pathlib import Path
-
-import pytest
-
-sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
-
-from app.integrity import IntegrityChecker, IntegrityCursorStore, IntegrityResult
-
-
-def _wait_scan_done(client, headers, timeout=10):
-    deadline = time.time() + timeout
-    while time.time() < deadline:
-        resp = client.get("/admin/integrity/status", headers=headers)
-        data = resp.get_json()
-        if not data.get("scanning"):
-            return
-        time.sleep(0.1)
-    raise TimeoutError("scan did not complete")
-
-
-def _md5(data: bytes) -> str:
-    return hashlib.md5(data).hexdigest()
-
-
-def _setup_bucket(storage_root: Path, bucket_name: str, objects: dict[str, bytes]) -> None:
-    bucket_path = storage_root / bucket_name
-    bucket_path.mkdir(parents=True, exist_ok=True)
-    meta_root = storage_root / ".myfsio.sys" / "buckets" / bucket_name / "meta"
-    meta_root.mkdir(parents=True, exist_ok=True)
-    bucket_json = storage_root / ".myfsio.sys" / "buckets" / bucket_name / ".bucket.json"
-    bucket_json.write_text(json.dumps({"created": "2025-01-01"}))
-
-    for key, data in objects.items():
-        obj_path = bucket_path / key
-        obj_path.parent.mkdir(parents=True, exist_ok=True)
-        obj_path.write_bytes(data)
-
-        etag = _md5(data)
-        stat = obj_path.stat()
-        meta = {
-            "__etag__": etag,
-            "__size__": str(stat.st_size),
-            "__last_modified__": str(stat.st_mtime),
-        }
-
-        key_path = Path(key)
-        parent = key_path.parent
-        key_name = key_path.name
-        if parent == Path("."):
-            index_path = meta_root / "_index.json"
-        else:
-            index_path = meta_root / parent / "_index.json"
-        index_path.parent.mkdir(parents=True, exist_ok=True)
-
-        index_data = {}
-        if index_path.exists():
-            index_data = json.loads(index_path.read_text())
-        index_data[key_name] = {"metadata": meta}
-        index_path.write_text(json.dumps(index_data))
-
-
-def _issues_of_type(result, issue_type):
-    return [i for i in result.issues if i.issue_type == issue_type]
-
-
-@pytest.fixture
-def storage_root(tmp_path):
-    root = tmp_path / "data"
-    root.mkdir()
-    (root / ".myfsio.sys" / "config").mkdir(parents=True, exist_ok=True)
-    return root
-
-
-@pytest.fixture
-def checker(storage_root):
-    return IntegrityChecker(
-        storage_root=storage_root,
-        interval_hours=24.0,
-        batch_size=1000,
-        auto_heal=False,
-        dry_run=False,
-    )
-
-
-class TestCorruptedObjects:
-    def test_detect_corrupted(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello world"})
-        (storage_root / "mybucket" / "file.txt").write_bytes(b"corrupted data")
-
-        result = checker.run_now()
-        assert result.corrupted_objects == 1
-        issues = _issues_of_type(result, "corrupted_object")
-        assert len(issues) == 1
-        assert issues[0].bucket == "mybucket"
-        assert issues[0].key == "file.txt"
-        assert not issues[0].healed
-
-    def test_heal_corrupted(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello world"})
-        (storage_root / "mybucket" / "file.txt").write_bytes(b"corrupted data")
-
-        result = checker.run_now(auto_heal=True)
-        assert result.corrupted_objects == 1
-        assert result.issues_healed == 1
-        issues = _issues_of_type(result, "corrupted_object")
-        assert issues[0].healed
-
-        result2 = checker.run_now()
-        assert result2.corrupted_objects == 0
-
-    def test_valid_objects_pass(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello world"})
-
-        result = checker.run_now()
-        assert result.corrupted_objects == 0
-        assert result.objects_scanned >= 1
-
-    def test_corrupted_nested_key(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"sub/dir/file.txt": b"nested content"})
-        (storage_root / "mybucket" / "sub" / "dir" / "file.txt").write_bytes(b"bad")
-
-        result = checker.run_now()
-        assert result.corrupted_objects == 1
-        issues = _issues_of_type(result, "corrupted_object")
-        assert issues[0].key == "sub/dir/file.txt"
-
-
-class TestOrphanedObjects:
-    def test_detect_orphaned(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {})
-        (storage_root / "mybucket" / "orphan.txt").write_bytes(b"orphan data")
-
-        result = checker.run_now()
-        assert result.orphaned_objects == 1
-        issues = _issues_of_type(result, "orphaned_object")
-        assert len(issues) == 1
-
-    def test_heal_orphaned(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {})
-        (storage_root / "mybucket" / "orphan.txt").write_bytes(b"orphan data")
-
-        result = checker.run_now(auto_heal=True)
-        assert result.orphaned_objects == 1
-        assert result.issues_healed == 1
-        issues = _issues_of_type(result, "orphaned_object")
-        assert issues[0].healed
-
-        result2 = checker.run_now()
-        assert result2.orphaned_objects == 0
-        assert result2.objects_scanned >= 1
-
-
-class TestPhantomMetadata:
-    def test_detect_phantom(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-        (storage_root / "mybucket" / "file.txt").unlink()
-
-        result = checker.run_now()
-        assert result.phantom_metadata == 1
-        issues = _issues_of_type(result, "phantom_metadata")
-        assert len(issues) == 1
-
-    def test_heal_phantom(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-        (storage_root / "mybucket" / "file.txt").unlink()
-
-        result = checker.run_now(auto_heal=True)
-        assert result.phantom_metadata == 1
-        assert result.issues_healed == 1
-
-        result2 = checker.run_now()
-        assert result2.phantom_metadata == 0
-
-
-class TestStaleVersions:
-    def test_manifest_without_data(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-        versions_root = storage_root / ".myfsio.sys" / "buckets" / "mybucket" / "versions" / "file.txt"
-        versions_root.mkdir(parents=True)
-        (versions_root / "v1.json").write_text(json.dumps({"etag": "abc"}))
-
-        result = checker.run_now()
-        assert result.stale_versions == 1
-        issues = _issues_of_type(result, "stale_version")
-        assert "manifest without data" in issues[0].detail
-
-    def test_data_without_manifest(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-        versions_root = storage_root / ".myfsio.sys" / "buckets" / "mybucket" / "versions" / "file.txt"
-        versions_root.mkdir(parents=True)
-        (versions_root / "v1.bin").write_bytes(b"old data")
-
-        result = checker.run_now()
-        assert result.stale_versions == 1
-        issues = _issues_of_type(result, "stale_version")
-        assert "data without manifest" in issues[0].detail
-
-    def test_heal_stale_versions(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-        versions_root = storage_root / ".myfsio.sys" / "buckets" / "mybucket" / "versions" / "file.txt"
-        versions_root.mkdir(parents=True)
-        (versions_root / "v1.json").write_text(json.dumps({"etag": "abc"}))
-        (versions_root / "v2.bin").write_bytes(b"old data")
-
-        result = checker.run_now(auto_heal=True)
-        assert result.stale_versions == 2
-        assert result.issues_healed == 2
-        assert not (versions_root / "v1.json").exists()
-        assert not (versions_root / "v2.bin").exists()
-
-    def test_valid_versions_pass(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-        versions_root = storage_root / ".myfsio.sys" / "buckets" / "mybucket" / "versions" / "file.txt"
-        versions_root.mkdir(parents=True)
-        (versions_root / "v1.json").write_text(json.dumps({"etag": "abc"}))
-        (versions_root / "v1.bin").write_bytes(b"old data")
-
-        result = checker.run_now()
-        assert result.stale_versions == 0
-
-
-class TestEtagCache:
-    def test_detect_mismatch(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-        etag_path = storage_root / ".myfsio.sys" / "buckets" / "mybucket" / "etag_index.json"
-        etag_path.write_text(json.dumps({"file.txt": "wrong_etag"}))
-
-        result = checker.run_now()
-        assert result.etag_cache_inconsistencies == 1
-        issues = _issues_of_type(result, "etag_cache_inconsistency")
-        assert len(issues) == 1
-
-    def test_heal_mismatch(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-        etag_path = storage_root / ".myfsio.sys" / "buckets" / "mybucket" / "etag_index.json"
-        etag_path.write_text(json.dumps({"file.txt": "wrong_etag"}))
-
-        result = checker.run_now(auto_heal=True)
-        assert result.etag_cache_inconsistencies == 1
-        assert result.issues_healed == 1
-        assert not etag_path.exists()
-
-
-class TestLegacyMetadata:
-    def test_detect_unmigrated(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-
-        legacy_meta = storage_root / "mybucket" / ".meta" / "file.txt.meta.json"
-        legacy_meta.parent.mkdir(parents=True)
-        legacy_meta.write_text(json.dumps({"__etag__": "different_value"}))
-
-        meta_root = storage_root / ".myfsio.sys" / "buckets" / "mybucket" / "meta"
-        index_path = meta_root / "_index.json"
-        index_path.unlink()
-
-        result = checker.run_now()
-        assert result.legacy_metadata_drifts == 1
-        issues = _issues_of_type(result, "legacy_metadata_drift")
-        assert len(issues) == 1
-        assert issues[0].detail == "unmigrated legacy .meta.json"
-
-    def test_detect_drift(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-
-        legacy_meta = storage_root / "mybucket" / ".meta" / "file.txt.meta.json"
-        legacy_meta.parent.mkdir(parents=True)
-        legacy_meta.write_text(json.dumps({"__etag__": "different_value"}))
-
-        result = checker.run_now()
-        assert result.legacy_metadata_drifts == 1
-        issues = _issues_of_type(result, "legacy_metadata_drift")
-        assert "differs from index" in issues[0].detail
-
-    def test_heal_unmigrated(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-
-        legacy_meta = storage_root / "mybucket" / ".meta" / "file.txt.meta.json"
-        legacy_meta.parent.mkdir(parents=True)
-        legacy_data = {"__etag__": _md5(b"hello"), "__size__": "5"}
-        legacy_meta.write_text(json.dumps(legacy_data))
-
-        meta_root = storage_root / ".myfsio.sys" / "buckets" / "mybucket" / "meta"
-        index_path = meta_root / "_index.json"
-        index_path.unlink()
-
-        result = checker.run_now(auto_heal=True)
-        assert result.legacy_metadata_drifts == 1
-        legacy_issues = _issues_of_type(result, "legacy_metadata_drift")
-        assert len(legacy_issues) == 1
-        assert legacy_issues[0].healed
-        assert not legacy_meta.exists()
-
-        index_data = json.loads(index_path.read_text())
-        assert "file.txt" in index_data
-        assert index_data["file.txt"]["metadata"]["__etag__"] == _md5(b"hello")
-
-    def test_heal_drift(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-
-        legacy_meta = storage_root / "mybucket" / ".meta" / "file.txt.meta.json"
-        legacy_meta.parent.mkdir(parents=True)
-        legacy_meta.write_text(json.dumps({"__etag__": "different_value"}))
-
-        result = checker.run_now(auto_heal=True)
-        assert result.legacy_metadata_drifts == 1
-        legacy_issues = _issues_of_type(result, "legacy_metadata_drift")
-        assert legacy_issues[0].healed
-        assert not legacy_meta.exists()
-
-
-class TestDryRun:
-    def test_dry_run_no_changes(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-        (storage_root / "mybucket" / "file.txt").write_bytes(b"corrupted")
-        (storage_root / "mybucket" / "orphan.txt").write_bytes(b"orphan")
-
-        result = checker.run_now(auto_heal=True, dry_run=True)
-        assert result.corrupted_objects == 1
-        assert result.orphaned_objects == 1
-        assert result.issues_healed == 0
-
-        meta_root = storage_root / ".myfsio.sys" / "buckets" / "mybucket" / "meta"
-        index_data = json.loads((meta_root / "_index.json").read_text())
-        assert "orphan.txt" not in index_data
-
-
-class TestBatchSize:
-    def test_batch_limits_scan(self, storage_root):
-        objects = {f"file{i}.txt": f"data{i}".encode() for i in range(10)}
-        _setup_bucket(storage_root, "mybucket", objects)
-
-        checker = IntegrityChecker(
-            storage_root=storage_root,
-            batch_size=3,
-        )
-        result = checker.run_now()
-        assert result.objects_scanned <= 3
-
-
-class TestHistory:
-    def test_history_recorded(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-
-        checker.run_now()
-        history = checker.get_history()
-        assert len(history) == 1
-        assert "corrupted_objects" in history[0]["result"]
-
-    def test_history_multiple(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-
-        checker.run_now()
-        checker.run_now()
-        checker.run_now()
-        history = checker.get_history()
-        assert len(history) == 3
-
-    def test_history_pagination(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-
-        for _ in range(5):
-            checker.run_now()
-
-        history = checker.get_history(limit=2, offset=1)
-        assert len(history) == 2
-
-
-AUTH_HEADERS = {"X-Access-Key": "admin", "X-Secret-Key": "adminsecret"}
-
-
-class TestAdminAPI:
-    @pytest.fixture
-    def integrity_app(self, tmp_path):
-        from app import create_api_app
-        storage_root = tmp_path / "data"
-        iam_config = tmp_path / "iam.json"
-        bucket_policies = tmp_path / "bucket_policies.json"
-        iam_payload = {
-            "users": [
-                {
-                    "access_key": "admin",
-                    "secret_key": "adminsecret",
-                    "display_name": "Admin",
-                    "policies": [{"bucket": "*", "actions": ["list", "read", "write", "delete", "policy", "iam:*"]}],
-                }
-            ]
-        }
-        iam_config.write_text(json.dumps(iam_payload))
-        flask_app = create_api_app({
-            "TESTING": True,
-            "SECRET_KEY": "testing",
-            "STORAGE_ROOT": storage_root,
-            "IAM_CONFIG": iam_config,
-            "BUCKET_POLICY_PATH": bucket_policies,
-            "API_BASE_URL": "http://testserver",
-            "INTEGRITY_ENABLED": True,
-            "INTEGRITY_AUTO_HEAL": False,
-            "INTEGRITY_DRY_RUN": False,
-        })
-        yield flask_app
-        storage = flask_app.extensions.get("object_storage")
-        if storage:
-            base = getattr(storage, "storage", storage)
-            if hasattr(base, "shutdown_stats"):
-                base.shutdown_stats()
-        ic = flask_app.extensions.get("integrity")
-        if ic:
-            ic.stop()
-
-    def test_status_endpoint(self, integrity_app):
-        client = integrity_app.test_client()
-        resp = client.get("/admin/integrity/status", headers=AUTH_HEADERS)
-        assert resp.status_code == 200
-        data = resp.get_json()
-        assert data["enabled"] is True
-        assert "interval_hours" in data
-
-    def test_run_endpoint(self, integrity_app):
-        client = integrity_app.test_client()
-        resp = client.post("/admin/integrity/run", headers=AUTH_HEADERS, json={})
-        assert resp.status_code == 200
-        data = resp.get_json()
-        assert data["status"] == "started"
-        _wait_scan_done(client, AUTH_HEADERS)
-        resp = client.get("/admin/integrity/history?limit=1", headers=AUTH_HEADERS)
-        hist = resp.get_json()
-        assert len(hist["executions"]) >= 1
-        assert "corrupted_objects" in hist["executions"][0]["result"]
-        assert "objects_scanned" in hist["executions"][0]["result"]
-
-    def test_run_with_overrides(self, integrity_app):
-        client = integrity_app.test_client()
-        resp = client.post(
-            "/admin/integrity/run",
-            headers=AUTH_HEADERS,
-            json={"dry_run": True, "auto_heal": True},
-        )
-        assert resp.status_code == 200
-        _wait_scan_done(client, AUTH_HEADERS)
-
-    def test_history_endpoint(self, integrity_app):
-        client = integrity_app.test_client()
-        client.post("/admin/integrity/run", headers=AUTH_HEADERS, json={})
-        _wait_scan_done(client, AUTH_HEADERS)
-        resp = client.get("/admin/integrity/history", headers=AUTH_HEADERS)
-        assert resp.status_code == 200
-        data = resp.get_json()
-        assert "executions" in data
-        assert len(data["executions"]) >= 1
-
-    def test_auth_required(self, integrity_app):
-        client = integrity_app.test_client()
-        resp = client.get("/admin/integrity/status")
-        assert resp.status_code in (401, 403)
-
-    def test_disabled_status(self, tmp_path):
-        from app import create_api_app
-        storage_root = tmp_path / "data2"
-        iam_config = tmp_path / "iam2.json"
-        bucket_policies = tmp_path / "bp2.json"
-        iam_payload = {
-            "users": [
-                {
-                    "access_key": "admin",
-                    "secret_key": "adminsecret",
-                    "display_name": "Admin",
-                    "policies": [{"bucket": "*", "actions": ["list", "read", "write", "delete", "policy", "iam:*"]}],
-                }
-            ]
-        }
-        iam_config.write_text(json.dumps(iam_payload))
-        flask_app = create_api_app({
-            "TESTING": True,
-            "SECRET_KEY": "testing",
-            "STORAGE_ROOT": storage_root,
-            "IAM_CONFIG": iam_config,
-            "BUCKET_POLICY_PATH": bucket_policies,
-            "API_BASE_URL": "http://testserver",
-            "INTEGRITY_ENABLED": False,
-        })
-        c = flask_app.test_client()
-        resp = c.get("/admin/integrity/status", headers=AUTH_HEADERS)
-        assert resp.status_code == 200
-        data = resp.get_json()
-        assert data["enabled"] is False
-
-        storage = flask_app.extensions.get("object_storage")
-        if storage:
-            base = getattr(storage, "storage", storage)
-            if hasattr(base, "shutdown_stats"):
-                base.shutdown_stats()
-
-
-class TestMultipleBuckets:
-    def test_scans_multiple_buckets(self, storage_root, checker):
-        _setup_bucket(storage_root, "bucket1", {"a.txt": b"aaa"})
-        _setup_bucket(storage_root, "bucket2", {"b.txt": b"bbb"})
-
-        result = checker.run_now()
-        assert result.buckets_scanned == 2
-        assert result.objects_scanned >= 2
-        assert result.corrupted_objects == 0
-
-
-class TestGetStatus:
-    def test_status_fields(self, checker):
-        status = checker.get_status()
-        assert "enabled" in status
-        assert "running" in status
-        assert "interval_hours" in status
-        assert "batch_size" in status
-        assert "auto_heal" in status
-        assert "dry_run" in status
-
-    def test_status_includes_cursor(self, storage_root, checker):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-        checker.run_now()
-        status = checker.get_status()
-        assert "cursor" in status
-        assert status["cursor"]["tracked_buckets"] == 1
-        assert "mybucket" in status["cursor"]["buckets"]
-
-
-class TestUnifiedBatchCounter:
-    def test_orphaned_objects_count_toward_batch(self, storage_root):
-        _setup_bucket(storage_root, "mybucket", {})
-        for i in range(10):
-            (storage_root / "mybucket" / f"orphan{i}.txt").write_bytes(f"data{i}".encode())
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=3)
-        result = checker.run_now()
-        assert result.objects_scanned <= 3
-
-    def test_phantom_metadata_counts_toward_batch(self, storage_root):
-        objects = {f"file{i}.txt": f"data{i}".encode() for i in range(10)}
-        _setup_bucket(storage_root, "mybucket", objects)
-        for i in range(10):
-            (storage_root / "mybucket" / f"file{i}.txt").unlink()
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=5)
-        result = checker.run_now()
-        assert result.objects_scanned <= 5
-
-    def test_all_check_types_contribute(self, storage_root):
-        _setup_bucket(storage_root, "mybucket", {"valid.txt": b"hello"})
-        (storage_root / "mybucket" / "orphan.txt").write_bytes(b"orphan")
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=1000)
-        result = checker.run_now()
-        assert result.objects_scanned > 2
-
-
-class TestCursorRotation:
-    def test_oldest_bucket_scanned_first(self, storage_root):
-        _setup_bucket(storage_root, "bucket-a", {"a.txt": b"aaa"})
-        _setup_bucket(storage_root, "bucket-b", {"b.txt": b"bbb"})
-        _setup_bucket(storage_root, "bucket-c", {"c.txt": b"ccc"})
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=5)
-
-        checker.cursor_store.update_bucket("bucket-a", 1000.0)
-        checker.cursor_store.update_bucket("bucket-b", 3000.0)
-        checker.cursor_store.update_bucket("bucket-c", 2000.0)
-
-        ordered = checker.cursor_store.get_bucket_order(["bucket-a", "bucket-b", "bucket-c"])
-        assert ordered[0] == "bucket-a"
-        assert ordered[1] == "bucket-c"
-        assert ordered[2] == "bucket-b"
-
-    def test_never_scanned_buckets_first(self, storage_root):
-        _setup_bucket(storage_root, "bucket-old", {"a.txt": b"aaa"})
-        _setup_bucket(storage_root, "bucket-new", {"b.txt": b"bbb"})
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=1000)
-
-        checker.cursor_store.update_bucket("bucket-old", time.time())
-
-        ordered = checker.cursor_store.get_bucket_order(["bucket-old", "bucket-new"])
-        assert ordered[0] == "bucket-new"
-
-    def test_rotation_covers_all_buckets(self, storage_root):
-        for name in ["bucket-a", "bucket-b", "bucket-c"]:
-            _setup_bucket(storage_root, name, {f"{name}.txt": name.encode()})
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=4)
-
-        result1 = checker.run_now()
-        scanned_buckets_1 = set()
-        for issue_bucket in [storage_root]:
-            pass
-        assert result1.buckets_scanned >= 1
-
-        result2 = checker.run_now()
-        result3 = checker.run_now()
-
-        cursor_info = checker.cursor_store.get_info()
-        assert cursor_info["tracked_buckets"] == 3
-
-    def test_cursor_persistence(self, storage_root):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-
-        checker1 = IntegrityChecker(storage_root=storage_root, batch_size=1000)
-        checker1.run_now()
-
-        cursor1 = checker1.cursor_store.get_info()
-        assert cursor1["tracked_buckets"] == 1
-        assert "mybucket" in cursor1["buckets"]
-
-        checker2 = IntegrityChecker(storage_root=storage_root, batch_size=1000)
-        cursor2 = checker2.cursor_store.get_info()
-        assert cursor2["tracked_buckets"] == 1
-        assert "mybucket" in cursor2["buckets"]
-
-    def test_stale_cursor_cleanup(self, storage_root):
-        _setup_bucket(storage_root, "bucket-a", {"a.txt": b"aaa"})
-        _setup_bucket(storage_root, "bucket-b", {"b.txt": b"bbb"})
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=1000)
-        checker.run_now()
-
-        import shutil
-        shutil.rmtree(storage_root / "bucket-b")
-        meta_b = storage_root / ".myfsio.sys" / "buckets" / "bucket-b"
-        if meta_b.exists():
-            shutil.rmtree(meta_b)
-
-        checker.run_now()
-
-        cursor_info = checker.cursor_store.get_info()
-        assert "bucket-b" not in cursor_info["buckets"]
-        assert "bucket-a" in cursor_info["buckets"]
-
-    def test_cursor_updates_after_scan(self, storage_root):
-        _setup_bucket(storage_root, "mybucket", {"file.txt": b"hello"})
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=1000)
-        before = time.time()
-        checker.run_now()
-        after = time.time()
-
-        cursor_info = checker.cursor_store.get_info()
-        entry = cursor_info["buckets"]["mybucket"]
-        assert before <= entry["last_scanned"] <= after
-        assert entry["completed"] is True
-
-
-class TestIntraBucketCursor:
-    def test_resumes_from_cursor_key(self, storage_root):
-        objects = {f"file_{chr(ord('a') + i)}.txt": f"data{i}".encode() for i in range(10)}
-        _setup_bucket(storage_root, "mybucket", objects)
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=3)
-        result1 = checker.run_now()
-        assert result1.objects_scanned == 3
-
-        cursor_info = checker.cursor_store.get_info()
-        entry = cursor_info["buckets"]["mybucket"]
-        assert entry["last_key"] is not None
-        assert entry["completed"] is False
-
-        result2 = checker.run_now()
-        assert result2.objects_scanned == 3
-
-        cursor_after = checker.cursor_store.get_info()["buckets"]["mybucket"]
-        assert cursor_after["last_key"] > entry["last_key"]
-
-    def test_cursor_resets_after_full_pass(self, storage_root):
-        objects = {f"file_{i}.txt": f"data{i}".encode() for i in range(3)}
-        _setup_bucket(storage_root, "mybucket", objects)
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=100)
-        checker.run_now()
-
-        cursor_info = checker.cursor_store.get_info()
-        entry = cursor_info["buckets"]["mybucket"]
-        assert entry["last_key"] is None
-        assert entry["completed"] is True
-
-    def test_full_coverage_across_cycles(self, storage_root):
-        objects = {f"obj_{chr(ord('a') + i)}.txt": f"data{i}".encode() for i in range(10)}
-        _setup_bucket(storage_root, "mybucket", objects)
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=4)
-        all_scanned = 0
-        for _ in range(10):
-            result = checker.run_now()
-            all_scanned += result.objects_scanned
-            if checker.cursor_store.get_info()["buckets"]["mybucket"]["completed"]:
-                break
-
-        assert all_scanned >= 10
-
-    def test_deleted_cursor_key_skips_gracefully(self, storage_root):
-        objects = {f"file_{chr(ord('a') + i)}.txt": f"data{i}".encode() for i in range(6)}
-        _setup_bucket(storage_root, "mybucket", objects)
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=3)
-        checker.run_now()
-
-        cursor_info = checker.cursor_store.get_info()
-        cursor_key = cursor_info["buckets"]["mybucket"]["last_key"]
-        assert cursor_key is not None
-
-        obj_path = storage_root / "mybucket" / cursor_key
-        meta_root = storage_root / ".myfsio.sys" / "buckets" / "mybucket" / "meta"
-        key_path = Path(cursor_key)
-        index_path = meta_root / key_path.parent / "_index.json" if key_path.parent != Path(".") else meta_root / "_index.json"
-        if key_path.parent == Path("."):
-            index_path = meta_root / "_index.json"
-        else:
-            index_path = meta_root / key_path.parent / "_index.json"
-        if obj_path.exists():
-            obj_path.unlink()
-        if index_path.exists():
-            index_data = json.loads(index_path.read_text())
-            index_data.pop(key_path.name, None)
-            index_path.write_text(json.dumps(index_data))
-
-        result2 = checker.run_now()
-        assert result2.objects_scanned > 0
-
-    def test_incomplete_buckets_prioritized(self, storage_root):
-        _setup_bucket(storage_root, "bucket-a", {f"a{i}.txt": b"a" for i in range(5)})
-        _setup_bucket(storage_root, "bucket-b", {f"b{i}.txt": b"b" for i in range(5)})
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=3)
-        checker.run_now()
-
-        cursor_info = checker.cursor_store.get_info()
-        incomplete = [
-            name for name, info in cursor_info["buckets"].items()
-            if info.get("last_key") is not None
-        ]
-        assert len(incomplete) >= 1
-
-        result2 = checker.run_now()
-        assert result2.objects_scanned > 0
-
-    def test_cursor_skips_nested_directories(self, storage_root):
-        objects = {
-            "aaa/file1.txt": b"a1",
-            "aaa/file2.txt": b"a2",
-            "bbb/file1.txt": b"b1",
-            "bbb/file2.txt": b"b2",
-            "ccc/file1.txt": b"c1",
-            "ccc/file2.txt": b"c2",
-        }
-        _setup_bucket(storage_root, "mybucket", objects)
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=4)
-        result1 = checker.run_now()
-        assert result1.objects_scanned == 4
-
-        cursor_info = checker.cursor_store.get_info()
-        cursor_key = cursor_info["buckets"]["mybucket"]["last_key"]
-        assert cursor_key is not None
-        assert cursor_key.startswith("aaa/") or cursor_key.startswith("bbb/")
-
-        result2 = checker.run_now()
-        assert result2.objects_scanned >= 2
-
-        all_scanned = result1.objects_scanned + result2.objects_scanned
-        for _ in range(10):
-            if checker.cursor_store.get_info()["buckets"]["mybucket"]["completed"]:
-                break
-            r = checker.run_now()
-            all_scanned += r.objects_scanned
-
-        assert all_scanned >= 6
-
-    def test_sorted_walk_order(self, storage_root):
-        objects = {
-            "bar.txt": b"bar",
-            "bar/inner.txt": b"inner",
-            "abc.txt": b"abc",
-            "zzz/deep.txt": b"deep",
-        }
-        _setup_bucket(storage_root, "mybucket", objects)
-
-        checker = IntegrityChecker(storage_root=storage_root, batch_size=100)
-        result = checker.run_now()
-        assert result.objects_scanned >= 4
-        assert result.total_issues == 0

python/tests/test_rust_extensions.py

@@ -1,350 +0,0 @@
-import hashlib
-import io
-import os
-import secrets
-import sys
-from pathlib import Path
-
-import pytest
-
-sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
-
-try:
-    import myfsio_core as _rc
-    HAS_RUST = True
-except ImportError:
-    _rc = None
-    HAS_RUST = False
-
-pytestmark = pytest.mark.skipif(not HAS_RUST, reason="myfsio_core not available")
-
-
-class TestStreamToFileWithMd5:
-    def test_basic_write(self, tmp_path):
-        data = b"hello world" * 1000
-        stream = io.BytesIO(data)
-        tmp_dir = str(tmp_path / "tmp")
-
-        tmp_path_str, md5_hex, size = _rc.stream_to_file_with_md5(stream, tmp_dir)
-
-        assert size == len(data)
-        assert md5_hex == hashlib.md5(data).hexdigest()
-        assert Path(tmp_path_str).exists()
-        assert Path(tmp_path_str).read_bytes() == data
-
-    def test_empty_stream(self, tmp_path):
-        stream = io.BytesIO(b"")
-        tmp_dir = str(tmp_path / "tmp")
-
-        tmp_path_str, md5_hex, size = _rc.stream_to_file_with_md5(stream, tmp_dir)
-
-        assert size == 0
-        assert md5_hex == hashlib.md5(b"").hexdigest()
-        assert Path(tmp_path_str).read_bytes() == b""
-
-    def test_large_data(self, tmp_path):
-        data = os.urandom(1024 * 1024 * 2)
-        stream = io.BytesIO(data)
-        tmp_dir = str(tmp_path / "tmp")
-
-        tmp_path_str, md5_hex, size = _rc.stream_to_file_with_md5(stream, tmp_dir)
-
-        assert size == len(data)
-        assert md5_hex == hashlib.md5(data).hexdigest()
-
-    def test_custom_chunk_size(self, tmp_path):
-        data = b"x" * 10000
-        stream = io.BytesIO(data)
-        tmp_dir = str(tmp_path / "tmp")
-
-        tmp_path_str, md5_hex, size = _rc.stream_to_file_with_md5(
-            stream, tmp_dir, chunk_size=128
-        )
-
-        assert size == len(data)
-        assert md5_hex == hashlib.md5(data).hexdigest()
-
-
-class TestAssemblePartsWithMd5:
-    def test_basic_assembly(self, tmp_path):
-        parts = []
-        combined = b""
-        for i in range(3):
-            data = f"part{i}data".encode() * 100
-            combined += data
-            p = tmp_path / f"part{i}"
-            p.write_bytes(data)
-            parts.append(str(p))
-
-        dest = str(tmp_path / "output")
-        md5_hex = _rc.assemble_parts_with_md5(parts, dest)
-
-        assert md5_hex == hashlib.md5(combined).hexdigest()
-        assert Path(dest).read_bytes() == combined
-
-    def test_single_part(self, tmp_path):
-        data = b"single part data"
-        p = tmp_path / "part0"
-        p.write_bytes(data)
-
-        dest = str(tmp_path / "output")
-        md5_hex = _rc.assemble_parts_with_md5([str(p)], dest)
-
-        assert md5_hex == hashlib.md5(data).hexdigest()
-        assert Path(dest).read_bytes() == data
-
-    def test_empty_parts_list(self):
-        with pytest.raises(ValueError, match="No parts"):
-            _rc.assemble_parts_with_md5([], "dummy")
-
-    def test_missing_part_file(self, tmp_path):
-        with pytest.raises(OSError):
-            _rc.assemble_parts_with_md5(
-                [str(tmp_path / "nonexistent")], str(tmp_path / "out")
-            )
-
-    def test_large_parts(self, tmp_path):
-        parts = []
-        combined = b""
-        for i in range(5):
-            data = os.urandom(512 * 1024)
-            combined += data
-            p = tmp_path / f"part{i}"
-            p.write_bytes(data)
-            parts.append(str(p))
-
-        dest = str(tmp_path / "output")
-        md5_hex = _rc.assemble_parts_with_md5(parts, dest)
-
-        assert md5_hex == hashlib.md5(combined).hexdigest()
-        assert Path(dest).read_bytes() == combined
-
-
-class TestEncryptDecryptStreamChunked:
-    def _python_derive_chunk_nonce(self, base_nonce, chunk_index):
-        from cryptography.hazmat.primitives.kdf.hkdf import HKDF
-        from cryptography.hazmat.primitives import hashes
-        hkdf = HKDF(
-            algorithm=hashes.SHA256(),
-            length=12,
-            salt=base_nonce,
-            info=chunk_index.to_bytes(4, "big"),
-        )
-        return hkdf.derive(b"chunk_nonce")
-
-    def test_encrypt_decrypt_roundtrip(self, tmp_path):
-        data = b"Hello, encryption!" * 500
-        key = secrets.token_bytes(32)
-        base_nonce = secrets.token_bytes(12)
-
-        input_path = str(tmp_path / "plaintext")
-        encrypted_path = str(tmp_path / "encrypted")
-        decrypted_path = str(tmp_path / "decrypted")
-
-        Path(input_path).write_bytes(data)
-
-        chunk_count = _rc.encrypt_stream_chunked(
-            input_path, encrypted_path, key, base_nonce
-        )
-        assert chunk_count > 0
-
-        chunk_count_dec = _rc.decrypt_stream_chunked(
-            encrypted_path, decrypted_path, key, base_nonce
-        )
-        assert chunk_count_dec == chunk_count
-        assert Path(decrypted_path).read_bytes() == data
-
-    def test_empty_file(self, tmp_path):
-        key = secrets.token_bytes(32)
-        base_nonce = secrets.token_bytes(12)
-
-        input_path = str(tmp_path / "empty")
-        encrypted_path = str(tmp_path / "encrypted")
-        decrypted_path = str(tmp_path / "decrypted")
-
-        Path(input_path).write_bytes(b"")
-
-        chunk_count = _rc.encrypt_stream_chunked(
-            input_path, encrypted_path, key, base_nonce
-        )
-        assert chunk_count == 0
-
-        chunk_count_dec = _rc.decrypt_stream_chunked(
-            encrypted_path, decrypted_path, key, base_nonce
-        )
-        assert chunk_count_dec == 0
-        assert Path(decrypted_path).read_bytes() == b""
-
-    def test_custom_chunk_size(self, tmp_path):
-        data = os.urandom(10000)
-        key = secrets.token_bytes(32)
-        base_nonce = secrets.token_bytes(12)
-
-        input_path = str(tmp_path / "plaintext")
-        encrypted_path = str(tmp_path / "encrypted")
-        decrypted_path = str(tmp_path / "decrypted")
-
-        Path(input_path).write_bytes(data)
-
-        chunk_count = _rc.encrypt_stream_chunked(
-            input_path, encrypted_path, key, base_nonce, chunk_size=1024
-        )
-        assert chunk_count == 10
-
-        _rc.decrypt_stream_chunked(encrypted_path, decrypted_path, key, base_nonce)
-        assert Path(decrypted_path).read_bytes() == data
-
-    def test_invalid_key_length(self, tmp_path):
-        input_path = str(tmp_path / "in")
-        Path(input_path).write_bytes(b"data")
-
-        with pytest.raises(ValueError, match="32 bytes"):
-            _rc.encrypt_stream_chunked(
-                input_path, str(tmp_path / "out"), b"short", secrets.token_bytes(12)
-            )
-
-    def test_invalid_nonce_length(self, tmp_path):
-        input_path = str(tmp_path / "in")
-        Path(input_path).write_bytes(b"data")
-
-        with pytest.raises(ValueError, match="12 bytes"):
-            _rc.encrypt_stream_chunked(
-                input_path, str(tmp_path / "out"), secrets.token_bytes(32), b"short"
-            )
-
-    def test_wrong_key_fails_decrypt(self, tmp_path):
-        data = b"sensitive data"
-        key = secrets.token_bytes(32)
-        wrong_key = secrets.token_bytes(32)
-        base_nonce = secrets.token_bytes(12)
-
-        input_path = str(tmp_path / "plaintext")
-        encrypted_path = str(tmp_path / "encrypted")
-        decrypted_path = str(tmp_path / "decrypted")
-
-        Path(input_path).write_bytes(data)
-        _rc.encrypt_stream_chunked(input_path, encrypted_path, key, base_nonce)
-
-        with pytest.raises((ValueError, OSError)):
-            _rc.decrypt_stream_chunked(
-                encrypted_path, decrypted_path, wrong_key, base_nonce
-            )
-
-    def test_cross_compat_python_encrypt_rust_decrypt(self, tmp_path):
-        from cryptography.hazmat.primitives.ciphers.aead import AESGCM
-
-        data = b"cross compat test data" * 100
-        key = secrets.token_bytes(32)
-        base_nonce = secrets.token_bytes(12)
-        chunk_size = 1024
-
-        encrypted_path = str(tmp_path / "py_encrypted")
-        with open(encrypted_path, "wb") as f:
-            f.write(b"\x00\x00\x00\x00")
-            aesgcm = AESGCM(key)
-            chunk_index = 0
-            offset = 0
-            while offset < len(data):
-                chunk = data[offset:offset + chunk_size]
-                nonce = self._python_derive_chunk_nonce(base_nonce, chunk_index)
-                enc = aesgcm.encrypt(nonce, chunk, None)
-                f.write(len(enc).to_bytes(4, "big"))
-                f.write(enc)
-                chunk_index += 1
-                offset += chunk_size
-            f.seek(0)
-            f.write(chunk_index.to_bytes(4, "big"))
-
-        decrypted_path = str(tmp_path / "rust_decrypted")
-        _rc.decrypt_stream_chunked(encrypted_path, decrypted_path, key, base_nonce)
-        assert Path(decrypted_path).read_bytes() == data
-
-    def test_cross_compat_rust_encrypt_python_decrypt(self, tmp_path):
-        from cryptography.hazmat.primitives.ciphers.aead import AESGCM
-
-        data = b"cross compat reverse test" * 100
-        key = secrets.token_bytes(32)
-        base_nonce = secrets.token_bytes(12)
-        chunk_size = 1024
-
-        input_path = str(tmp_path / "plaintext")
-        encrypted_path = str(tmp_path / "rust_encrypted")
-        Path(input_path).write_bytes(data)
-
-        chunk_count = _rc.encrypt_stream_chunked(
-            input_path, encrypted_path, key, base_nonce, chunk_size=chunk_size
-        )
-
-        aesgcm = AESGCM(key)
-        with open(encrypted_path, "rb") as f:
-            count_bytes = f.read(4)
-            assert int.from_bytes(count_bytes, "big") == chunk_count
-
-            decrypted = b""
-            for i in range(chunk_count):
-                size = int.from_bytes(f.read(4), "big")
-                enc_chunk = f.read(size)
-                nonce = self._python_derive_chunk_nonce(base_nonce, i)
-                decrypted += aesgcm.decrypt(nonce, enc_chunk, None)
-
-        assert decrypted == data
-
-    def test_large_file_roundtrip(self, tmp_path):
-        data = os.urandom(1024 * 1024)
-        key = secrets.token_bytes(32)
-        base_nonce = secrets.token_bytes(12)
-
-        input_path = str(tmp_path / "large")
-        encrypted_path = str(tmp_path / "encrypted")
-        decrypted_path = str(tmp_path / "decrypted")
-
-        Path(input_path).write_bytes(data)
-
-        _rc.encrypt_stream_chunked(input_path, encrypted_path, key, base_nonce)
-        _rc.decrypt_stream_chunked(encrypted_path, decrypted_path, key, base_nonce)
-
-        assert Path(decrypted_path).read_bytes() == data
-
-
-class TestStreamingEncryptorFileMethods:
-    def test_encrypt_file_decrypt_file_roundtrip(self, tmp_path):
-        from app.encryption import LocalKeyEncryption, StreamingEncryptor
-
-        master_key_path = tmp_path / "master.key"
-        provider = LocalKeyEncryption(master_key_path)
-        encryptor = StreamingEncryptor(provider, chunk_size=512)
-
-        data = b"file method test data" * 200
-        input_path = str(tmp_path / "input")
-        encrypted_path = str(tmp_path / "encrypted")
-        decrypted_path = str(tmp_path / "decrypted")
-
-        Path(input_path).write_bytes(data)
-
-        metadata = encryptor.encrypt_file(input_path, encrypted_path)
-        assert metadata.algorithm == "AES256"
-
-        encryptor.decrypt_file(encrypted_path, decrypted_path, metadata)
-        assert Path(decrypted_path).read_bytes() == data
-
-    def test_encrypt_file_matches_encrypt_stream(self, tmp_path):
-        from app.encryption import LocalKeyEncryption, StreamingEncryptor
-
-        master_key_path = tmp_path / "master.key"
-        provider = LocalKeyEncryption(master_key_path)
-        encryptor = StreamingEncryptor(provider, chunk_size=512)
-
-        data = b"stream vs file comparison" * 100
-        input_path = str(tmp_path / "input")
-        Path(input_path).write_bytes(data)
-
-        file_encrypted_path = str(tmp_path / "file_enc")
-        metadata_file = encryptor.encrypt_file(input_path, file_encrypted_path)
-
-        file_decrypted_path = str(tmp_path / "file_dec")
-        encryptor.decrypt_file(file_encrypted_path, file_decrypted_path, metadata_file)
-        assert Path(file_decrypted_path).read_bytes() == data
-
-        stream_enc, metadata_stream = encryptor.encrypt_stream(io.BytesIO(data))
-        stream_dec = encryptor.decrypt_stream(stream_enc, metadata_stream)
-        assert stream_dec.read() == data

python/tests/test_site_sync.py

@@ -1,460 +0,0 @@
-import io
-import json
-import time
-from datetime import datetime, timezone
-from pathlib import Path
-from unittest.mock import MagicMock, patch
-
-import pytest
-
-from app.connections import ConnectionStore, RemoteConnection
-from app.replication import (
-    ReplicationManager,
-    ReplicationRule,
-    REPLICATION_MODE_BIDIRECTIONAL,
-    REPLICATION_MODE_NEW_ONLY,
-)
-from app.site_sync import (
-    SiteSyncWorker,
-    SyncState,
-    SyncedObjectInfo,
-    SiteSyncStats,
-    RemoteObjectMeta,
-)
-from app.storage import ObjectStorage
-
-
-@pytest.fixture
-def storage(tmp_path: Path):
-    storage_root = tmp_path / "data"
-    storage_root.mkdir(parents=True)
-    return ObjectStorage(storage_root)
-
-
-@pytest.fixture
-def connections(tmp_path: Path):
-    connections_path = tmp_path / "connections.json"
-    store = ConnectionStore(connections_path)
-    conn = RemoteConnection(
-        id="test-conn",
-        name="Test Remote",
-        endpoint_url="http://localhost:9000",
-        access_key="remote-access",
-        secret_key="remote-secret",
-        region="us-east-1",
-    )
-    store.add(conn)
-    return store
-
-
-@pytest.fixture
-def replication_manager(storage, connections, tmp_path):
-    rules_path = tmp_path / "replication_rules.json"
-    storage_root = tmp_path / "data"
-    storage_root.mkdir(exist_ok=True)
-    manager = ReplicationManager(storage, connections, rules_path, storage_root)
-    yield manager
-    manager.shutdown(wait=False)
-
-
-@pytest.fixture
-def site_sync_worker(storage, connections, replication_manager, tmp_path):
-    storage_root = tmp_path / "data"
-    worker = SiteSyncWorker(
-        storage=storage,
-        connections=connections,
-        replication_manager=replication_manager,
-        storage_root=storage_root,
-        interval_seconds=60,
-        batch_size=100,
-    )
-    yield worker
-    worker.shutdown()
-
-
-class TestSyncedObjectInfo:
-    def test_to_dict(self):
-        info = SyncedObjectInfo(
-            last_synced_at=1234567890.0,
-            remote_etag="abc123",
-            source="remote",
-        )
-        result = info.to_dict()
-        assert result["last_synced_at"] == 1234567890.0
-        assert result["remote_etag"] == "abc123"
-        assert result["source"] == "remote"
-
-    def test_from_dict(self):
-        data = {
-            "last_synced_at": 9876543210.0,
-            "remote_etag": "def456",
-            "source": "local",
-        }
-        info = SyncedObjectInfo.from_dict(data)
-        assert info.last_synced_at == 9876543210.0
-        assert info.remote_etag == "def456"
-        assert info.source == "local"
-
-
-class TestSyncState:
-    def test_to_dict(self):
-        state = SyncState(
-            synced_objects={
-                "test.txt": SyncedObjectInfo(
-                    last_synced_at=1000.0,
-                    remote_etag="etag1",
-                    source="remote",
-                )
-            },
-            last_full_sync=2000.0,
-        )
-        result = state.to_dict()
-        assert "test.txt" in result["synced_objects"]
-        assert result["synced_objects"]["test.txt"]["remote_etag"] == "etag1"
-        assert result["last_full_sync"] == 2000.0
-
-    def test_from_dict(self):
-        data = {
-            "synced_objects": {
-                "file.txt": {
-                    "last_synced_at": 3000.0,
-                    "remote_etag": "etag2",
-                    "source": "remote",
-                }
-            },
-            "last_full_sync": 4000.0,
-        }
-        state = SyncState.from_dict(data)
-        assert "file.txt" in state.synced_objects
-        assert state.synced_objects["file.txt"].remote_etag == "etag2"
-        assert state.last_full_sync == 4000.0
-
-    def test_from_dict_empty(self):
-        state = SyncState.from_dict({})
-        assert state.synced_objects == {}
-        assert state.last_full_sync is None
-
-
-class TestSiteSyncStats:
-    def test_to_dict(self):
-        stats = SiteSyncStats(
-            last_sync_at=1234567890.0,
-            objects_pulled=10,
-            objects_skipped=5,
-            conflicts_resolved=2,
-            deletions_applied=1,
-            errors=0,
-        )
-        result = stats.to_dict()
-        assert result["objects_pulled"] == 10
-        assert result["objects_skipped"] == 5
-        assert result["conflicts_resolved"] == 2
-        assert result["deletions_applied"] == 1
-        assert result["errors"] == 0
-
-
-class TestRemoteObjectMeta:
-    def test_from_s3_object(self):
-        obj = {
-            "Key": "test/file.txt",
-            "Size": 1024,
-            "LastModified": datetime(2025, 1, 1, 12, 0, 0, tzinfo=timezone.utc),
-            "ETag": '"abc123def456"',
-        }
-        meta = RemoteObjectMeta.from_s3_object(obj)
-        assert meta.key == "test/file.txt"
-        assert meta.size == 1024
-        assert meta.last_modified == datetime(2025, 1, 1, 12, 0, 0, tzinfo=timezone.utc)
-        assert meta.etag == "abc123def456"
-
-
-class TestReplicationRuleBidirectional:
-    def test_rule_with_bidirectional_mode(self):
-        rule = ReplicationRule(
-            bucket_name="sync-bucket",
-            target_connection_id="test-conn",
-            target_bucket="remote-bucket",
-            enabled=True,
-            mode=REPLICATION_MODE_BIDIRECTIONAL,
-            sync_deletions=True,
-        )
-        assert rule.mode == REPLICATION_MODE_BIDIRECTIONAL
-        assert rule.sync_deletions is True
-        assert rule.last_pull_at is None
-
-    def test_rule_to_dict_includes_new_fields(self):
-        rule = ReplicationRule(
-            bucket_name="sync-bucket",
-            target_connection_id="test-conn",
-            target_bucket="remote-bucket",
-            mode=REPLICATION_MODE_BIDIRECTIONAL,
-            sync_deletions=False,
-            last_pull_at=1234567890.0,
-        )
-        result = rule.to_dict()
-        assert result["mode"] == REPLICATION_MODE_BIDIRECTIONAL
-        assert result["sync_deletions"] is False
-        assert result["last_pull_at"] == 1234567890.0
-
-    def test_rule_from_dict_with_new_fields(self):
-        data = {
-            "bucket_name": "sync-bucket",
-            "target_connection_id": "test-conn",
-            "target_bucket": "remote-bucket",
-            "mode": REPLICATION_MODE_BIDIRECTIONAL,
-            "sync_deletions": False,
-            "last_pull_at": 1234567890.0,
-        }
-        rule = ReplicationRule.from_dict(data)
-        assert rule.mode == REPLICATION_MODE_BIDIRECTIONAL
-        assert rule.sync_deletions is False
-        assert rule.last_pull_at == 1234567890.0
-
-    def test_rule_from_dict_defaults_new_fields(self):
-        data = {
-            "bucket_name": "sync-bucket",
-            "target_connection_id": "test-conn",
-            "target_bucket": "remote-bucket",
-        }
-        rule = ReplicationRule.from_dict(data)
-        assert rule.sync_deletions is True
-        assert rule.last_pull_at is None
-
-
-class TestSiteSyncWorker:
-    def test_start_and_shutdown(self, site_sync_worker):
-        site_sync_worker.start()
-        assert site_sync_worker._sync_thread is not None
-        assert site_sync_worker._sync_thread.is_alive()
-        site_sync_worker.shutdown()
-        assert not site_sync_worker._sync_thread.is_alive()
-
-    def test_trigger_sync_no_rule(self, site_sync_worker):
-        result = site_sync_worker.trigger_sync("nonexistent-bucket")
-        assert result is None
-
-    def test_trigger_sync_wrong_mode(self, site_sync_worker, replication_manager):
-        rule = ReplicationRule(
-            bucket_name="new-only-bucket",
-            target_connection_id="test-conn",
-            target_bucket="remote-bucket",
-            mode=REPLICATION_MODE_NEW_ONLY,
-            enabled=True,
-        )
-        replication_manager.set_rule(rule)
-        result = site_sync_worker.trigger_sync("new-only-bucket")
-        assert result is None
-
-    def test_trigger_sync_disabled_rule(self, site_sync_worker, replication_manager):
-        rule = ReplicationRule(
-            bucket_name="disabled-bucket",
-            target_connection_id="test-conn",
-            target_bucket="remote-bucket",
-            mode=REPLICATION_MODE_BIDIRECTIONAL,
-            enabled=False,
-        )
-        replication_manager.set_rule(rule)
-        result = site_sync_worker.trigger_sync("disabled-bucket")
-        assert result is None
-
-    def test_get_stats_no_sync(self, site_sync_worker):
-        stats = site_sync_worker.get_stats("nonexistent")
-        assert stats is None
-
-    def test_resolve_conflict_remote_newer(self, site_sync_worker):
-        local_meta = MagicMock()
-        local_meta.last_modified = datetime(2025, 1, 1, 12, 0, 0, tzinfo=timezone.utc)
-        local_meta.etag = "local123"
-
-        remote_meta = RemoteObjectMeta(
-            key="test.txt",
-            size=100,
-            last_modified=datetime(2025, 1, 2, 12, 0, 0, tzinfo=timezone.utc),
-            etag="remote456",
-        )
-
-        result = site_sync_worker._resolve_conflict(local_meta, remote_meta)
-        assert result == "pull"
-
-    def test_resolve_conflict_local_newer(self, site_sync_worker):
-        local_meta = MagicMock()
-        local_meta.last_modified = datetime(2025, 1, 2, 12, 0, 0, tzinfo=timezone.utc)
-        local_meta.etag = "local123"
-
-        remote_meta = RemoteObjectMeta(
-            key="test.txt",
-            size=100,
-            last_modified=datetime(2025, 1, 1, 12, 0, 0, tzinfo=timezone.utc),
-            etag="remote456",
-        )
-
-        result = site_sync_worker._resolve_conflict(local_meta, remote_meta)
-        assert result == "keep"
-
-    def test_resolve_conflict_same_time_same_etag(self, site_sync_worker):
-        ts = datetime(2025, 1, 1, 12, 0, 0, tzinfo=timezone.utc)
-        local_meta = MagicMock()
-        local_meta.last_modified = ts
-        local_meta.etag = "same123"
-
-        remote_meta = RemoteObjectMeta(
-            key="test.txt",
-            size=100,
-            last_modified=ts,
-            etag="same123",
-        )
-
-        result = site_sync_worker._resolve_conflict(local_meta, remote_meta)
-        assert result == "skip"
-
-    def test_resolve_conflict_same_time_different_etag(self, site_sync_worker):
-        ts = datetime(2025, 1, 1, 12, 0, 0, tzinfo=timezone.utc)
-        local_meta = MagicMock()
-        local_meta.last_modified = ts
-        local_meta.etag = "aaa"
-
-        remote_meta = RemoteObjectMeta(
-            key="test.txt",
-            size=100,
-            last_modified=ts,
-            etag="zzz",
-        )
-
-        result = site_sync_worker._resolve_conflict(local_meta, remote_meta)
-        assert result == "pull"
-
-    def test_sync_state_persistence(self, site_sync_worker, tmp_path):
-        bucket_name = "test-bucket"
-        state = SyncState(
-            synced_objects={
-                "file1.txt": SyncedObjectInfo(
-                    last_synced_at=time.time(),
-                    remote_etag="etag1",
-                    source="remote",
-                )
-            },
-            last_full_sync=time.time(),
-        )
-
-        site_sync_worker._save_sync_state(bucket_name, state)
-
-        loaded = site_sync_worker._load_sync_state(bucket_name)
-        assert "file1.txt" in loaded.synced_objects
-        assert loaded.synced_objects["file1.txt"].remote_etag == "etag1"
-
-    def test_load_sync_state_nonexistent(self, site_sync_worker):
-        state = site_sync_worker._load_sync_state("nonexistent-bucket")
-        assert state.synced_objects == {}
-        assert state.last_full_sync is None
-
-    @patch("app.site_sync._create_sync_client")
-    def test_list_remote_objects(self, mock_create_client, site_sync_worker, connections, replication_manager):
-        mock_client = MagicMock()
-        mock_paginator = MagicMock()
-        mock_paginator.paginate.return_value = [
-            {
-                "Contents": [
-                    {
-                        "Key": "file1.txt",
-                        "Size": 100,
-                        "LastModified": datetime(2025, 1, 1, tzinfo=timezone.utc),
-                        "ETag": '"etag1"',
-                    },
-                    {
-                        "Key": "file2.txt",
-                        "Size": 200,
-                        "LastModified": datetime(2025, 1, 2, tzinfo=timezone.utc),
-                        "ETag": '"etag2"',
-                    },
-                ]
-            }
-        ]
-        mock_client.get_paginator.return_value = mock_paginator
-        mock_create_client.return_value = mock_client
-
-        rule = ReplicationRule(
-            bucket_name="local-bucket",
-            target_connection_id="test-conn",
-            target_bucket="remote-bucket",
-            mode=REPLICATION_MODE_BIDIRECTIONAL,
-        )
-        conn = connections.get("test-conn")
-
-        result = site_sync_worker._list_remote_objects(rule, conn)
-
-        assert "file1.txt" in result
-        assert "file2.txt" in result
-        assert result["file1.txt"].size == 100
-        assert result["file2.txt"].size == 200
-
-    def test_list_local_objects(self, site_sync_worker, storage):
-        storage.create_bucket("test-bucket")
-        storage.put_object("test-bucket", "file1.txt", io.BytesIO(b"content1"))
-        storage.put_object("test-bucket", "file2.txt", io.BytesIO(b"content2"))
-
-        result = site_sync_worker._list_local_objects("test-bucket")
-
-        assert "file1.txt" in result
-        assert "file2.txt" in result
-
-    @patch("app.site_sync._create_sync_client")
-    def test_sync_bucket_connection_not_found(self, mock_create_client, site_sync_worker, replication_manager):
-        rule = ReplicationRule(
-            bucket_name="test-bucket",
-            target_connection_id="missing-conn",
-            target_bucket="remote-bucket",
-            mode=REPLICATION_MODE_BIDIRECTIONAL,
-            enabled=True,
-        )
-        replication_manager.set_rule(rule)
-
-        stats = site_sync_worker._sync_bucket(rule)
-        assert stats.errors == 1
-
-
-class TestSiteSyncIntegration:
-    @patch("app.site_sync._create_sync_client")
-    def test_full_sync_cycle(self, mock_create_client, site_sync_worker, storage, connections, replication_manager):
-        storage.create_bucket("sync-bucket")
-        storage.put_object("sync-bucket", "local-only.txt", io.BytesIO(b"local content"))
-
-        mock_client = MagicMock()
-        mock_paginator = MagicMock()
-        mock_paginator.paginate.return_value = [
-            {
-                "Contents": [
-                    {
-                        "Key": "remote-only.txt",
-                        "Size": 100,
-                        "LastModified": datetime(2025, 1, 15, tzinfo=timezone.utc),
-                        "ETag": '"remoteetag"',
-                    },
-                ]
-            }
-        ]
-        mock_client.get_paginator.return_value = mock_paginator
-        mock_client.head_object.return_value = {"Metadata": {}}
-
-        def mock_download(bucket, key, path):
-            Path(path).write_bytes(b"remote content")
-
-        mock_client.download_file.side_effect = mock_download
-        mock_create_client.return_value = mock_client
-
-        rule = ReplicationRule(
-            bucket_name="sync-bucket",
-            target_connection_id="test-conn",
-            target_bucket="remote-bucket",
-            mode=REPLICATION_MODE_BIDIRECTIONAL,
-            enabled=True,
-        )
-        replication_manager.set_rule(rule)
-
-        stats = site_sync_worker._sync_bucket(rule)
-
-        assert stats.objects_pulled == 1
-        assert stats.errors == 0
-
-        objects = site_sync_worker._list_local_objects("sync-bucket")
-        assert "local-only.txt" in objects
-        assert "remote-only.txt" in objects

python/tests/test_ui_bulk_delete.py

@@ -1,125 +0,0 @@
-import io
-import json
-import threading
-from pathlib import Path
-
-from werkzeug.serving import make_server
-
-from app import create_app
-from app.s3_client import S3ProxyClient
-
-
-def _build_app(tmp_path: Path):
-    storage_root = tmp_path / "data"
-    iam_config = tmp_path / "iam.json"
-    bucket_policies = tmp_path / "bucket_policies.json"
-    iam_payload = {
-        "users": [
-            {
-                "access_key": "test",
-                "secret_key": "secret",
-                "display_name": "Bulk Tester",
-                "policies": [{"bucket": "*", "actions": ["list", "read", "write", "delete", "policy"]}],
-            }
-        ]
-    }
-    iam_config.write_text(json.dumps(iam_payload))
-    app = create_app(
-        {
-            "TESTING": True,
-            "STORAGE_ROOT": storage_root,
-            "IAM_CONFIG": iam_config,
-            "BUCKET_POLICY_PATH": bucket_policies,
-            "API_BASE_URL": "http://127.0.0.1:0",
-            "SECRET_KEY": "testing",
-            "WTF_CSRF_ENABLED": False,
-        }
-    )
-
-    server = make_server("127.0.0.1", 0, app)
-    host, port = server.server_address
-    api_url = f"http://{host}:{port}"
-    app.config["API_BASE_URL"] = api_url
-    app.extensions["s3_proxy"] = S3ProxyClient(api_base_url=api_url)
-
-    thread = threading.Thread(target=server.serve_forever, daemon=True)
-    thread.start()
-
-    app._test_server = server
-    app._test_thread = thread
-    return app
-
-
-def _shutdown_app(app):
-    if hasattr(app, "_test_server"):
-        app._test_server.shutdown()
-        app._test_thread.join(timeout=2)
-
-
-def _login(client):
-    return client.post(
-        "/ui/login",
-        data={"access_key": "test", "secret_key": "secret"},
-        follow_redirects=True,
-    )
-
-
-def test_bulk_delete_json_route(tmp_path: Path):
-    app = _build_app(tmp_path)
-    try:
-        storage = app.extensions["object_storage"]
-        storage.create_bucket("demo")
-        storage.put_object("demo", "first.txt", io.BytesIO(b"first"))
-        storage.put_object("demo", "second.txt", io.BytesIO(b"second"))
-
-        client = app.test_client()
-        assert _login(client).status_code == 200
-
-        response = client.post(
-            "/ui/buckets/demo/objects/bulk-delete",
-            json={"keys": ["first.txt", "missing.txt"]},
-            headers={"X-Requested-With": "XMLHttpRequest"},
-        )
-        assert response.status_code == 200
-        payload = response.get_json()
-        assert payload["status"] == "ok"
-        assert set(payload["deleted"]) == {"first.txt", "missing.txt"}
-        assert payload["errors"] == []
-
-        listing = storage.list_objects_all("demo")
-        assert {meta.key for meta in listing} == {"second.txt"}
-    finally:
-        _shutdown_app(app)
-
-
-def test_bulk_delete_validation(tmp_path: Path):
-    app = _build_app(tmp_path)
-    try:
-        storage = app.extensions["object_storage"]
-        storage.create_bucket("demo")
-        storage.put_object("demo", "keep.txt", io.BytesIO(b"keep"))
-
-        client = app.test_client()
-        assert _login(client).status_code == 200
-
-        bad_response = client.post(
-            "/ui/buckets/demo/objects/bulk-delete",
-            json={"keys": []},
-            headers={"X-Requested-With": "XMLHttpRequest"},
-        )
-        assert bad_response.status_code == 400
-        assert bad_response.get_json()["status"] == "error"
-
-        too_many = [f"obj-{index}.txt" for index in range(501)]
-        limit_response = client.post(
-            "/ui/buckets/demo/objects/bulk-delete",
-            json={"keys": too_many},
-            headers={"X-Requested-With": "XMLHttpRequest"},
-        )
-        assert limit_response.status_code == 400
-        assert limit_response.get_json()["status"] == "error"
-
-        still_there = storage.list_objects_all("demo")
-        assert {meta.key for meta in still_there} == {"keep.txt"}
-    finally:
-        _shutdown_app(app)

python/tests/test_ui_encryption.py

@@ -1,263 +0,0 @@
-"""Tests for UI-based encryption configuration."""
-import json
-import threading
-from pathlib import Path
-
-import pytest
-from werkzeug.serving import make_server
-
-from app import create_app
-from app.s3_client import S3ProxyClient
-
-
-def get_csrf_token(response):
-    """Extract CSRF token from response HTML."""
-    html = response.data.decode("utf-8")
-    import re
-    match = re.search(r'name="csrf_token"\s+value="([^"]+)"', html)
-    return match.group(1) if match else None
-
-
-def _make_encryption_app(tmp_path: Path, *, kms_enabled: bool = True):
-    """Create an app with encryption enabled."""
-    storage_root = tmp_path / "data"
-    iam_config = tmp_path / "iam.json"
-    bucket_policies = tmp_path / "bucket_policies.json"
-    iam_payload = {
-        "users": [
-            {
-                "access_key": "test",
-                "secret_key": "secret",
-                "display_name": "Test User",
-                "policies": [{"bucket": "*", "actions": ["list", "read", "write", "delete", "policy"]}],
-            },
-            {
-                "access_key": "readonly",
-                "secret_key": "secret",
-                "display_name": "Read Only User",
-                "policies": [{"bucket": "*", "actions": ["list", "read"]}],
-            },
-        ]
-    }
-    iam_config.write_text(json.dumps(iam_payload))
-
-    config = {
-        "TESTING": True,
-        "STORAGE_ROOT": storage_root,
-        "IAM_CONFIG": iam_config,
-        "BUCKET_POLICY_PATH": bucket_policies,
-        "API_BASE_URL": "http://127.0.0.1:0",
-        "SECRET_KEY": "testing",
-        "ENCRYPTION_ENABLED": True,
-        "WTF_CSRF_ENABLED": False,
-    }
-
-    if kms_enabled:
-        config["KMS_ENABLED"] = True
-        config["KMS_KEYS_PATH"] = str(tmp_path / "kms_keys.json")
-        config["ENCRYPTION_MASTER_KEY_PATH"] = str(tmp_path / "master.key")
-
-    app = create_app(config)
-
-    server = make_server("127.0.0.1", 0, app)
-    host, port = server.server_address
-    api_url = f"http://{host}:{port}"
-    app.config["API_BASE_URL"] = api_url
-    app.extensions["s3_proxy"] = S3ProxyClient(api_base_url=api_url)
-
-    thread = threading.Thread(target=server.serve_forever, daemon=True)
-    thread.start()
-
-    app._test_server = server
-    app._test_thread = thread
-
-    storage = app.extensions["object_storage"]
-    storage.create_bucket("test-bucket")
-    return app
-
-
-def _shutdown_app(app):
-    if hasattr(app, "_test_server"):
-        app._test_server.shutdown()
-        app._test_thread.join(timeout=2)
-
-
-class TestUIBucketEncryption:
-    """Test bucket encryption configuration via UI."""
-
-    def test_bucket_detail_shows_encryption_card(self, tmp_path):
-        """Encryption card should be visible on bucket detail page."""
-        app = _make_encryption_app(tmp_path)
-        try:
-            client = app.test_client()
-
-            client.post("/ui/login", data={"access_key": "test", "secret_key": "secret"}, follow_redirects=True)
-
-            response = client.get("/ui/buckets/test-bucket?tab=properties")
-            assert response.status_code == 200
-
-            html = response.data.decode("utf-8")
-            assert "Default Encryption" in html
-            assert "Encryption Algorithm" in html or "Default encryption disabled" in html
-        finally:
-            _shutdown_app(app)
-
-    def test_enable_aes256_encryption(self, tmp_path):
-        """Should be able to enable AES-256 encryption."""
-        app = _make_encryption_app(tmp_path)
-        try:
-            client = app.test_client()
-
-            client.post("/ui/login", data={"access_key": "test", "secret_key": "secret"}, follow_redirects=True)
-
-            response = client.post(
-                "/ui/buckets/test-bucket/encryption",
-                data={
-                    "action": "enable",
-                    "algorithm": "AES256",
-                },
-                follow_redirects=True,
-            )
-
-            assert response.status_code == 200
-            html = response.data.decode("utf-8")
-            assert "AES-256" in html or "encryption enabled" in html.lower()
-        finally:
-            _shutdown_app(app)
-
-    def test_enable_kms_encryption(self, tmp_path):
-        """Should be able to enable KMS encryption."""
-        app = _make_encryption_app(tmp_path, kms_enabled=True)
-        try:
-            with app.app_context():
-                kms = app.extensions.get("kms")
-                if kms:
-                    key = kms.create_key("test-key")
-                    key_id = key.key_id
-                else:
-                    pytest.skip("KMS not available")
-
-            client = app.test_client()
-            client.post("/ui/login", data={"access_key": "test", "secret_key": "secret"}, follow_redirects=True)
-
-            response = client.post(
-                "/ui/buckets/test-bucket/encryption",
-                data={
-                    "action": "enable",
-                    "algorithm": "aws:kms",
-                    "kms_key_id": key_id,
-                },
-                follow_redirects=True,
-            )
-
-            assert response.status_code == 200
-            html = response.data.decode("utf-8")
-            assert "KMS" in html or "encryption enabled" in html.lower()
-        finally:
-            _shutdown_app(app)
-
-    def test_disable_encryption(self, tmp_path):
-        """Should be able to disable encryption."""
-        app = _make_encryption_app(tmp_path)
-        try:
-            client = app.test_client()
-
-            client.post("/ui/login", data={"access_key": "test", "secret_key": "secret"}, follow_redirects=True)
-
-            client.post(
-                "/ui/buckets/test-bucket/encryption",
-                data={
-                    "action": "enable",
-                    "algorithm": "AES256",
-                },
-            )
-
-            response = client.post(
-                "/ui/buckets/test-bucket/encryption",
-                data={
-                    "action": "disable",
-                },
-                follow_redirects=True,
-            )
-
-            assert response.status_code == 200
-            html = response.data.decode("utf-8")
-            assert "disabled" in html.lower() or "Default encryption disabled" in html
-        finally:
-            _shutdown_app(app)
-
-    def test_invalid_algorithm_rejected(self, tmp_path):
-        """Invalid encryption algorithm should be rejected."""
-        app = _make_encryption_app(tmp_path)
-        try:
-            client = app.test_client()
-
-            client.post("/ui/login", data={"access_key": "test", "secret_key": "secret"}, follow_redirects=True)
-
-            response = client.post(
-                "/ui/buckets/test-bucket/encryption",
-                data={
-                    "action": "enable",
-                    "algorithm": "INVALID",
-                },
-                follow_redirects=True,
-            )
-
-            assert response.status_code == 200
-            html = response.data.decode("utf-8")
-            assert "Invalid" in html or "danger" in html
-        finally:
-            _shutdown_app(app)
-
-    def test_encryption_persists_in_config(self, tmp_path):
-        """Encryption config should persist in bucket config."""
-        app = _make_encryption_app(tmp_path)
-        try:
-            client = app.test_client()
-
-            client.post("/ui/login", data={"access_key": "test", "secret_key": "secret"}, follow_redirects=True)
-
-            client.post(
-                "/ui/buckets/test-bucket/encryption",
-                data={
-                    "action": "enable",
-                    "algorithm": "AES256",
-                },
-            )
-
-            with app.app_context():
-                storage = app.extensions["object_storage"]
-                config = storage.get_bucket_encryption("test-bucket")
-
-                assert "Rules" in config
-                assert len(config["Rules"]) == 1
-                assert config["Rules"][0]["SSEAlgorithm"] == "AES256"
-        finally:
-            _shutdown_app(app)
-
-
-class TestUIEncryptionWithoutPermission:
-    """Test encryption UI when user lacks permissions."""
-
-    def test_readonly_user_cannot_change_encryption(self, tmp_path):
-        """Read-only user should not be able to change encryption settings."""
-        app = _make_encryption_app(tmp_path)
-        try:
-            client = app.test_client()
-
-            client.post("/ui/login", data={"access_key": "readonly", "secret_key": "secret"}, follow_redirects=True)
-
-            response = client.post(
-                "/ui/buckets/test-bucket/encryption",
-                data={
-                    "action": "enable",
-                    "algorithm": "AES256",
-                },
-                follow_redirects=True,
-            )
-
-            assert response.status_code == 200
-            html = response.data.decode("utf-8")
-            assert "Access denied" in html or "permission" in html.lower() or "not authorized" in html.lower()
-        finally:
-            _shutdown_app(app)

python/tests/test_ui_pagination.py

@@ -1,212 +0,0 @@
-"""Tests for UI pagination of bucket objects."""
-import json
-import threading
-from io import BytesIO
-from pathlib import Path
-
-import pytest
-from werkzeug.serving import make_server
-
-from app import create_app
-from app.s3_client import S3ProxyClient
-
-
-def _make_app(tmp_path: Path):
-    """Create an app for testing with a live API server."""
-    storage_root = tmp_path / "data"
-    iam_config = tmp_path / "iam.json"
-    bucket_policies = tmp_path / "bucket_policies.json"
-    iam_payload = {
-        "users": [
-            {
-                "access_key": "test",
-                "secret_key": "secret",
-                "display_name": "Test User",
-                "policies": [{"bucket": "*", "actions": ["list", "read", "write", "delete", "policy"]}],
-            },
-        ]
-    }
-    iam_config.write_text(json.dumps(iam_payload))
-
-    flask_app = create_app(
-        {
-            "TESTING": True,
-            "SECRET_KEY": "testing",
-            "WTF_CSRF_ENABLED": False,
-            "STORAGE_ROOT": storage_root,
-            "IAM_CONFIG": iam_config,
-            "BUCKET_POLICY_PATH": bucket_policies,
-            "API_BASE_URL": "http://127.0.0.1:0",
-        }
-    )
-
-    server = make_server("127.0.0.1", 0, flask_app)
-    host, port = server.server_address
-    api_url = f"http://{host}:{port}"
-    flask_app.config["API_BASE_URL"] = api_url
-    flask_app.extensions["s3_proxy"] = S3ProxyClient(api_base_url=api_url)
-
-    thread = threading.Thread(target=server.serve_forever, daemon=True)
-    thread.start()
-
-    flask_app._test_server = server
-    flask_app._test_thread = thread
-    return flask_app
-
-
-def _shutdown_app(app):
-    if hasattr(app, "_test_server"):
-        app._test_server.shutdown()
-        app._test_thread.join(timeout=2)
-
-
-class TestPaginatedObjectListing:
-    """Test paginated object listing API."""
-
-    def test_objects_api_returns_paginated_results(self, tmp_path):
-        """Objects API should return paginated results."""
-        app = _make_app(tmp_path)
-        try:
-            storage = app.extensions["object_storage"]
-            storage.create_bucket("test-bucket")
-
-            for i in range(10):
-                storage.put_object("test-bucket", f"file{i:02d}.txt", BytesIO(b"content"))
-
-            with app.test_client() as client:
-                client.post("/ui/login", data={"access_key": "test", "secret_key": "secret"}, follow_redirects=True)
-
-                resp = client.get("/ui/buckets/test-bucket/objects?max_keys=3")
-                assert resp.status_code == 200
-
-                data = resp.get_json()
-                assert len(data["objects"]) == 3
-                assert data["is_truncated"] is True
-                assert data["next_continuation_token"] is not None
-        finally:
-            _shutdown_app(app)
-
-    def test_objects_api_pagination_continuation(self, tmp_path):
-        """Objects API should support continuation tokens."""
-        app = _make_app(tmp_path)
-        try:
-            storage = app.extensions["object_storage"]
-            storage.create_bucket("test-bucket")
-
-            for i in range(5):
-                storage.put_object("test-bucket", f"file{i:02d}.txt", BytesIO(b"content"))
-
-            with app.test_client() as client:
-                client.post("/ui/login", data={"access_key": "test", "secret_key": "secret"}, follow_redirects=True)
-
-                resp = client.get("/ui/buckets/test-bucket/objects?max_keys=2")
-                assert resp.status_code == 200
-                data = resp.get_json()
-
-                first_page_keys = [obj["key"] for obj in data["objects"]]
-                assert len(first_page_keys) == 2
-                assert data["is_truncated"] is True
-
-                token = data["next_continuation_token"]
-                resp = client.get(f"/ui/buckets/test-bucket/objects?max_keys=2&continuation_token={token}")
-                assert resp.status_code == 200
-                data = resp.get_json()
-
-                second_page_keys = [obj["key"] for obj in data["objects"]]
-                assert len(second_page_keys) == 2
-
-                assert set(first_page_keys).isdisjoint(set(second_page_keys))
-        finally:
-            _shutdown_app(app)
-
-    def test_objects_api_prefix_filter(self, tmp_path):
-        """Objects API should support prefix filtering."""
-        app = _make_app(tmp_path)
-        try:
-            storage = app.extensions["object_storage"]
-            storage.create_bucket("test-bucket")
-
-            storage.put_object("test-bucket", "logs/access.log", BytesIO(b"log"))
-            storage.put_object("test-bucket", "logs/error.log", BytesIO(b"log"))
-            storage.put_object("test-bucket", "data/file.txt", BytesIO(b"data"))
-
-            with app.test_client() as client:
-                client.post("/ui/login", data={"access_key": "test", "secret_key": "secret"}, follow_redirects=True)
-
-                resp = client.get("/ui/buckets/test-bucket/objects?prefix=logs/")
-                assert resp.status_code == 200
-                data = resp.get_json()
-
-                keys = [obj["key"] for obj in data["objects"]]
-                assert all(k.startswith("logs/") for k in keys)
-                assert len(keys) == 2
-        finally:
-            _shutdown_app(app)
-
-    def test_objects_api_requires_authentication(self, tmp_path):
-        """Objects API should require login."""
-        app = _make_app(tmp_path)
-        try:
-            storage = app.extensions["object_storage"]
-            storage.create_bucket("test-bucket")
-
-            with app.test_client() as client:
-                resp = client.get("/ui/buckets/test-bucket/objects")
-                assert resp.status_code == 302
-                assert "/ui/login" in resp.headers.get("Location", "")
-        finally:
-            _shutdown_app(app)
-
-    def test_objects_api_returns_object_metadata(self, tmp_path):
-        """Objects API should return complete object metadata."""
-        app = _make_app(tmp_path)
-        try:
-            storage = app.extensions["object_storage"]
-            storage.create_bucket("test-bucket")
-            storage.put_object("test-bucket", "test.txt", BytesIO(b"test content"))
-
-            with app.test_client() as client:
-                client.post("/ui/login", data={"access_key": "test", "secret_key": "secret"}, follow_redirects=True)
-
-                resp = client.get("/ui/buckets/test-bucket/objects")
-                assert resp.status_code == 200
-                data = resp.get_json()
-
-                assert len(data["objects"]) == 1
-                obj = data["objects"][0]
-
-                assert obj["key"] == "test.txt"
-                assert obj["size"] == 12
-                assert "last_modified" in obj
-                assert "last_modified_display" in obj
-                assert "etag" in obj
-
-                assert "url_templates" in data
-                templates = data["url_templates"]
-                assert "preview" in templates
-                assert "download" in templates
-                assert "delete" in templates
-                assert "KEY_PLACEHOLDER" in templates["preview"]
-        finally:
-            _shutdown_app(app)
-
-    def test_bucket_detail_page_loads_without_objects(self, tmp_path):
-        """Bucket detail page should load even with many objects."""
-        app = _make_app(tmp_path)
-        try:
-            storage = app.extensions["object_storage"]
-            storage.create_bucket("test-bucket")
-
-            for i in range(100):
-                storage.put_object("test-bucket", f"file{i:03d}.txt", BytesIO(b"x"))
-
-            with app.test_client() as client:
-                client.post("/ui/login", data={"access_key": "test", "secret_key": "secret"}, follow_redirects=True)
-
-                resp = client.get("/ui/buckets/test-bucket")
-                assert resp.status_code == 200
-
-                html = resp.data.decode("utf-8")
-                assert "bucket-detail-main.js" in html
-        finally:
-            _shutdown_app(app)

python/tests/test_website_hosting.py

@@ -1,442 +0,0 @@
-import io
-import json
-from pathlib import Path
-from xml.etree.ElementTree import fromstring
-
-import pytest
-
-from app import create_api_app
-from app.website_domains import WebsiteDomainStore
-
-
-def _stream(data: bytes):
-    return io.BytesIO(data)
-
-
-@pytest.fixture()
-def website_app(tmp_path: Path):
-    storage_root = tmp_path / "data"
-    iam_config = tmp_path / "iam.json"
-    bucket_policies = tmp_path / "bucket_policies.json"
-    iam_payload = {
-        "users": [
-            {
-                "access_key": "test",
-                "secret_key": "secret",
-                "display_name": "Test User",
-                "policies": [{"bucket": "*", "actions": ["list", "read", "write", "delete", "policy", "iam:*"]}],
-            }
-        ]
-    }
-    iam_config.write_text(json.dumps(iam_payload))
-    flask_app = create_api_app(
-        {
-            "TESTING": True,
-            "SECRET_KEY": "testing",
-            "STORAGE_ROOT": storage_root,
-            "IAM_CONFIG": iam_config,
-            "BUCKET_POLICY_PATH": bucket_policies,
-            "API_BASE_URL": "http://testserver",
-            "WEBSITE_HOSTING_ENABLED": True,
-        }
-    )
-    yield flask_app
-
-
-@pytest.fixture()
-def website_client(website_app):
-    return website_app.test_client()
-
-
-@pytest.fixture()
-def storage(website_app):
-    return website_app.extensions["object_storage"]
-
-
-class TestWebsiteDomainStore:
-    def test_empty_store(self, tmp_path):
-        store = WebsiteDomainStore(tmp_path / "domains.json")
-        assert store.list_all() == []
-        assert store.get_bucket("example.com") is None
-
-    def test_set_and_get_mapping(self, tmp_path):
-        store = WebsiteDomainStore(tmp_path / "domains.json")
-        store.set_mapping("example.com", "my-site")
-        assert store.get_bucket("example.com") == "my-site"
-
-    def test_case_insensitive(self, tmp_path):
-        store = WebsiteDomainStore(tmp_path / "domains.json")
-        store.set_mapping("Example.COM", "my-site")
-        assert store.get_bucket("example.com") == "my-site"
-        assert store.get_bucket("EXAMPLE.COM") == "my-site"
-
-    def test_list_all(self, tmp_path):
-        store = WebsiteDomainStore(tmp_path / "domains.json")
-        store.set_mapping("a.com", "bucket-a")
-        store.set_mapping("b.com", "bucket-b")
-        result = store.list_all()
-        domains = {item["domain"] for item in result}
-        assert domains == {"a.com", "b.com"}
-
-    def test_delete_mapping(self, tmp_path):
-        store = WebsiteDomainStore(tmp_path / "domains.json")
-        store.set_mapping("example.com", "my-site")
-        assert store.delete_mapping("example.com") is True
-        assert store.get_bucket("example.com") is None
-
-    def test_delete_nonexistent(self, tmp_path):
-        store = WebsiteDomainStore(tmp_path / "domains.json")
-        assert store.delete_mapping("nope.com") is False
-
-    def test_overwrite_mapping(self, tmp_path):
-        store = WebsiteDomainStore(tmp_path / "domains.json")
-        store.set_mapping("example.com", "old-bucket")
-        store.set_mapping("example.com", "new-bucket")
-        assert store.get_bucket("example.com") == "new-bucket"
-
-    def test_persistence(self, tmp_path):
-        path = tmp_path / "domains.json"
-        store1 = WebsiteDomainStore(path)
-        store1.set_mapping("example.com", "my-site")
-        store2 = WebsiteDomainStore(path)
-        assert store2.get_bucket("example.com") == "my-site"
-
-    def test_corrupt_file(self, tmp_path):
-        path = tmp_path / "domains.json"
-        path.write_text("not json")
-        store = WebsiteDomainStore(path)
-        assert store.list_all() == []
-
-    def test_non_dict_file(self, tmp_path):
-        path = tmp_path / "domains.json"
-        path.write_text('["not", "a", "dict"]')
-        store = WebsiteDomainStore(path)
-        assert store.list_all() == []
-
-
-class TestStorageWebsiteConfig:
-    def test_get_website_no_config(self, storage):
-        storage.create_bucket("test-bucket")
-        assert storage.get_bucket_website("test-bucket") is None
-
-    def test_set_and_get_website(self, storage):
-        storage.create_bucket("test-bucket")
-        config = {"index_document": "index.html", "error_document": "error.html"}
-        storage.set_bucket_website("test-bucket", config)
-        result = storage.get_bucket_website("test-bucket")
-        assert result["index_document"] == "index.html"
-        assert result["error_document"] == "error.html"
-
-    def test_delete_website_config(self, storage):
-        storage.create_bucket("test-bucket")
-        storage.set_bucket_website("test-bucket", {"index_document": "index.html"})
-        storage.set_bucket_website("test-bucket", None)
-        assert storage.get_bucket_website("test-bucket") is None
-
-    def test_nonexistent_bucket(self, storage):
-        with pytest.raises(Exception):
-            storage.get_bucket_website("no-such-bucket")
-
-
-class TestS3WebsiteAPI:
-    def test_put_website_config(self, website_client, signer):
-        headers = signer("PUT", "/site-bucket")
-        assert website_client.put("/site-bucket", headers=headers).status_code == 200
-
-        xml_body = b"""<WebsiteConfiguration>
-            <IndexDocument><Suffix>index.html</Suffix></IndexDocument>
-            <ErrorDocument><Key>404.html</Key></ErrorDocument>
-        </WebsiteConfiguration>"""
-        headers = signer("PUT", "/site-bucket?website",
-                         headers={"Content-Type": "application/xml"}, body=xml_body)
-        resp = website_client.put("/site-bucket", query_string={"website": ""},
-                                  headers=headers, data=xml_body, content_type="application/xml")
-        assert resp.status_code == 200
-
-    def test_get_website_config(self, website_client, signer, storage):
-        storage.create_bucket("site-bucket")
-        storage.set_bucket_website("site-bucket", {
-            "index_document": "index.html",
-            "error_document": "error.html",
-        })
-
-        headers = signer("GET", "/site-bucket?website")
-        resp = website_client.get("/site-bucket", query_string={"website": ""}, headers=headers)
-        assert resp.status_code == 200
-
-        root = fromstring(resp.data)
-        suffix = root.find(".//{http://s3.amazonaws.com/doc/2006-03-01/}Suffix")
-        if suffix is None:
-            suffix = root.find(".//Suffix")
-        assert suffix is not None
-        assert suffix.text == "index.html"
-
-    def test_get_website_config_not_set(self, website_client, signer, storage):
-        storage.create_bucket("no-website")
-        headers = signer("GET", "/no-website?website")
-        resp = website_client.get("/no-website", query_string={"website": ""}, headers=headers)
-        assert resp.status_code == 404
-
-    def test_delete_website_config(self, website_client, signer, storage):
-        storage.create_bucket("site-bucket")
-        storage.set_bucket_website("site-bucket", {"index_document": "index.html"})
-
-        headers = signer("DELETE", "/site-bucket?website")
-        resp = website_client.delete("/site-bucket", query_string={"website": ""}, headers=headers)
-        assert resp.status_code == 204
-        assert storage.get_bucket_website("site-bucket") is None
-
-    def test_put_website_missing_index(self, website_client, signer, storage):
-        storage.create_bucket("site-bucket")
-        xml_body = b"""<WebsiteConfiguration>
-            <ErrorDocument><Key>error.html</Key></ErrorDocument>
-        </WebsiteConfiguration>"""
-        headers = signer("PUT", "/site-bucket?website",
-                         headers={"Content-Type": "application/xml"}, body=xml_body)
-        resp = website_client.put("/site-bucket", query_string={"website": ""},
-                                  headers=headers, data=xml_body, content_type="application/xml")
-        assert resp.status_code == 400
-
-    def test_put_website_slash_in_suffix(self, website_client, signer, storage):
-        storage.create_bucket("site-bucket")
-        xml_body = b"""<WebsiteConfiguration>
-            <IndexDocument><Suffix>path/index.html</Suffix></IndexDocument>
-        </WebsiteConfiguration>"""
-        headers = signer("PUT", "/site-bucket?website",
-                         headers={"Content-Type": "application/xml"}, body=xml_body)
-        resp = website_client.put("/site-bucket", query_string={"website": ""},
-                                  headers=headers, data=xml_body, content_type="application/xml")
-        assert resp.status_code == 400
-
-    def test_put_website_malformed_xml(self, website_client, signer, storage):
-        storage.create_bucket("site-bucket")
-        xml_body = b"not xml at all"
-        headers = signer("PUT", "/site-bucket?website",
-                         headers={"Content-Type": "application/xml"}, body=xml_body)
-        resp = website_client.put("/site-bucket", query_string={"website": ""},
-                                  headers=headers, data=xml_body, content_type="application/xml")
-        assert resp.status_code == 400
-
-    def test_website_disabled(self, client, signer):
-        headers = signer("PUT", "/test-bucket")
-        assert client.put("/test-bucket", headers=headers).status_code == 200
-        headers = signer("GET", "/test-bucket?website")
-        resp = client.get("/test-bucket", query_string={"website": ""}, headers=headers)
-        assert resp.status_code == 400
-        assert b"not enabled" in resp.data
-
-
-class TestAdminWebsiteDomains:
-    def _admin_headers(self, signer):
-        return signer("GET", "/admin/website-domains")
-
-    def test_list_empty(self, website_client, signer):
-        headers = self._admin_headers(signer)
-        resp = website_client.get("/admin/website-domains", headers=headers)
-        assert resp.status_code == 200
-        assert resp.get_json() == []
-
-    def test_create_mapping(self, website_client, signer, storage):
-        storage.create_bucket("my-site")
-        headers = signer("POST", "/admin/website-domains",
-                         headers={"Content-Type": "application/json"},
-                         body=json.dumps({"domain": "example.com", "bucket": "my-site"}).encode())
-        resp = website_client.post("/admin/website-domains",
-                                   headers=headers,
-                                   json={"domain": "example.com", "bucket": "my-site"})
-        assert resp.status_code == 201
-        data = resp.get_json()
-        assert data["domain"] == "example.com"
-        assert data["bucket"] == "my-site"
-
-    def test_create_duplicate(self, website_client, signer, storage):
-        storage.create_bucket("my-site")
-        body = json.dumps({"domain": "dup.com", "bucket": "my-site"}).encode()
-        headers = signer("POST", "/admin/website-domains",
-                         headers={"Content-Type": "application/json"}, body=body)
-        website_client.post("/admin/website-domains", headers=headers,
-                            json={"domain": "dup.com", "bucket": "my-site"})
-        headers = signer("POST", "/admin/website-domains",
-                         headers={"Content-Type": "application/json"}, body=body)
-        resp = website_client.post("/admin/website-domains", headers=headers,
-                                   json={"domain": "dup.com", "bucket": "my-site"})
-        assert resp.status_code == 409
-
-    def test_create_missing_domain(self, website_client, signer, storage):
-        storage.create_bucket("my-site")
-        body = json.dumps({"bucket": "my-site"}).encode()
-        headers = signer("POST", "/admin/website-domains",
-                         headers={"Content-Type": "application/json"}, body=body)
-        resp = website_client.post("/admin/website-domains", headers=headers,
-                                   json={"bucket": "my-site"})
-        assert resp.status_code == 400
-
-    def test_create_nonexistent_bucket(self, website_client, signer):
-        body = json.dumps({"domain": "x.com", "bucket": "no-such"}).encode()
-        headers = signer("POST", "/admin/website-domains",
-                         headers={"Content-Type": "application/json"}, body=body)
-        resp = website_client.post("/admin/website-domains", headers=headers,
-                                   json={"domain": "x.com", "bucket": "no-such"})
-        assert resp.status_code == 404
-
-    def test_get_mapping(self, website_client, signer, storage):
-        storage.create_bucket("my-site")
-        body = json.dumps({"domain": "get.com", "bucket": "my-site"}).encode()
-        headers = signer("POST", "/admin/website-domains",
-                         headers={"Content-Type": "application/json"}, body=body)
-        website_client.post("/admin/website-domains", headers=headers,
-                            json={"domain": "get.com", "bucket": "my-site"})
-
-        headers = signer("GET", "/admin/website-domains/get.com")
-        resp = website_client.get("/admin/website-domains/get.com", headers=headers)
-        assert resp.status_code == 200
-        assert resp.get_json()["bucket"] == "my-site"
-
-    def test_get_nonexistent(self, website_client, signer):
-        headers = signer("GET", "/admin/website-domains/nope.com")
-        resp = website_client.get("/admin/website-domains/nope.com", headers=headers)
-        assert resp.status_code == 404
-
-    def test_update_mapping(self, website_client, signer, storage):
-        storage.create_bucket("old-bucket")
-        storage.create_bucket("new-bucket")
-        body = json.dumps({"domain": "upd.com", "bucket": "old-bucket"}).encode()
-        headers = signer("POST", "/admin/website-domains",
-                         headers={"Content-Type": "application/json"}, body=body)
-        website_client.post("/admin/website-domains", headers=headers,
-                            json={"domain": "upd.com", "bucket": "old-bucket"})
-
-        body = json.dumps({"bucket": "new-bucket"}).encode()
-        headers = signer("PUT", "/admin/website-domains/upd.com",
-                         headers={"Content-Type": "application/json"}, body=body)
-        resp = website_client.put("/admin/website-domains/upd.com", headers=headers,
-                                  json={"bucket": "new-bucket"})
-        assert resp.status_code == 200
-        assert resp.get_json()["bucket"] == "new-bucket"
-
-    def test_delete_mapping(self, website_client, signer, storage):
-        storage.create_bucket("del-bucket")
-        body = json.dumps({"domain": "del.com", "bucket": "del-bucket"}).encode()
-        headers = signer("POST", "/admin/website-domains",
-                         headers={"Content-Type": "application/json"}, body=body)
-        website_client.post("/admin/website-domains", headers=headers,
-                            json={"domain": "del.com", "bucket": "del-bucket"})
-
-        headers = signer("DELETE", "/admin/website-domains/del.com")
-        resp = website_client.delete("/admin/website-domains/del.com", headers=headers)
-        assert resp.status_code == 204
-
-    def test_delete_nonexistent(self, website_client, signer):
-        headers = signer("DELETE", "/admin/website-domains/nope.com")
-        resp = website_client.delete("/admin/website-domains/nope.com", headers=headers)
-        assert resp.status_code == 404
-
-    def test_disabled(self, website_client, signer):
-        with website_client.application.test_request_context():
-            website_client.application.config["WEBSITE_HOSTING_ENABLED"] = False
-        headers = signer("GET", "/admin/website-domains")
-        resp = website_client.get("/admin/website-domains", headers=headers)
-        assert resp.status_code == 400
-        website_client.application.config["WEBSITE_HOSTING_ENABLED"] = True
-
-
-class TestWebsiteServing:
-    def _setup_website(self, storage, website_app):
-        storage.create_bucket("my-site")
-        storage.put_object("my-site", "index.html", _stream(b"<h1>Home</h1>"))
-        storage.put_object("my-site", "about.html", _stream(b"<h1>About</h1>"))
-        storage.put_object("my-site", "assets/style.css", _stream(b"body { color: red; }"))
-        storage.put_object("my-site", "sub/index.html", _stream(b"<h1>Sub</h1>"))
-        storage.put_object("my-site", "404.html", _stream(b"<h1>Not Found</h1>"))
-        storage.set_bucket_website("my-site", {
-            "index_document": "index.html",
-            "error_document": "404.html",
-        })
-        store = website_app.extensions["website_domains"]
-        store.set_mapping("mysite.example.com", "my-site")
-
-    def test_serve_index(self, website_client, storage, website_app):
-        self._setup_website(storage, website_app)
-        resp = website_client.get("/", headers={"Host": "mysite.example.com"})
-        assert resp.status_code == 200
-        assert b"<h1>Home</h1>" in resp.data
-        assert "text/html" in resp.content_type
-
-    def test_serve_specific_file(self, website_client, storage, website_app):
-        self._setup_website(storage, website_app)
-        resp = website_client.get("/about.html", headers={"Host": "mysite.example.com"})
-        assert resp.status_code == 200
-        assert b"<h1>About</h1>" in resp.data
-
-    def test_serve_css(self, website_client, storage, website_app):
-        self._setup_website(storage, website_app)
-        resp = website_client.get("/assets/style.css", headers={"Host": "mysite.example.com"})
-        assert resp.status_code == 200
-        assert b"body { color: red; }" in resp.data
-        assert "text/css" in resp.content_type
-
-    def test_serve_subdirectory_index(self, website_client, storage, website_app):
-        self._setup_website(storage, website_app)
-        resp = website_client.get("/sub/", headers={"Host": "mysite.example.com"})
-        assert resp.status_code == 200
-        assert b"<h1>Sub</h1>" in resp.data
-
-    def test_serve_subdirectory_no_trailing_slash(self, website_client, storage, website_app):
-        self._setup_website(storage, website_app)
-        resp = website_client.get("/sub", headers={"Host": "mysite.example.com"})
-        assert resp.status_code == 200
-        assert b"<h1>Sub</h1>" in resp.data
-
-    def test_serve_error_document(self, website_client, storage, website_app):
-        self._setup_website(storage, website_app)
-        resp = website_client.get("/nonexistent.html", headers={"Host": "mysite.example.com"})
-        assert resp.status_code == 404
-        assert b"<h1>Not Found</h1>" in resp.data
-
-    def test_unmapped_host_passes_through(self, website_client, storage, website_app):
-        self._setup_website(storage, website_app)
-        resp = website_client.get("/", headers={"Host": "unknown.example.com"})
-        assert resp.status_code != 200 or b"<h1>Home</h1>" not in resp.data
-
-    def test_head_request(self, website_client, storage, website_app):
-        self._setup_website(storage, website_app)
-        resp = website_client.head("/index.html", headers={"Host": "mysite.example.com"})
-        assert resp.status_code == 200
-        assert "Content-Length" in resp.headers
-        assert resp.data == b""
-
-    def test_post_not_intercepted(self, website_client, storage, website_app):
-        self._setup_website(storage, website_app)
-        resp = website_client.post("/index.html", headers={"Host": "mysite.example.com"})
-        assert resp.status_code != 200 or b"<h1>Home</h1>" not in resp.data
-
-    def test_bucket_deleted(self, website_client, storage, website_app):
-        self._setup_website(storage, website_app)
-        for obj in storage.list_objects_all("my-site"):
-            storage.delete_object("my-site", obj.key)
-        storage.delete_bucket("my-site")
-        resp = website_client.get("/", headers={"Host": "mysite.example.com"})
-        assert resp.status_code == 404
-
-    def test_no_website_config(self, website_client, storage, website_app):
-        storage.create_bucket("bare-bucket")
-        store = website_app.extensions["website_domains"]
-        store.set_mapping("bare.example.com", "bare-bucket")
-        resp = website_client.get("/", headers={"Host": "bare.example.com"})
-        assert resp.status_code == 404
-
-    def test_host_with_port(self, website_client, storage, website_app):
-        self._setup_website(storage, website_app)
-        resp = website_client.get("/", headers={"Host": "mysite.example.com:5000"})
-        assert resp.status_code == 200
-        assert b"<h1>Home</h1>" in resp.data
-
-    def test_no_error_document(self, website_client, storage, website_app):
-        storage.create_bucket("no-err")
-        storage.put_object("no-err", "index.html", _stream(b"<h1>Home</h1>"))
-        storage.set_bucket_website("no-err", {"index_document": "index.html"})
-        store = website_app.extensions["website_domains"]
-        store.set_mapping("noerr.example.com", "no-err")
-        resp = website_client.get("/missing.html", headers={"Host": "noerr.example.com"})
-        assert resp.status_code == 404
-        assert resp.data == b"<h1>404 page not found</h1>"

requirements.txt

@@ -1,4 +1,4 @@
-Flask>=3.1.3
+Flask>=3.1.2
 Flask-Limiter>=4.1.1
 Flask-Cors>=6.0.2
 Flask-WTF>=1.2.2
@@ -6,8 +6,7 @@ python-dotenv>=1.2.1
 pytest>=9.0.2
 requests>=2.32.5
 boto3>=1.42.14
-granian>=2.7.2
+waitress>=3.0.2
-psutil>=7.2.2
+psutil>=7.1.3
-cryptography>=46.0.5
+cryptography>=46.0.3
 defusedxml>=0.7.1
-duckdb>=1.5.1

run.py

@@ -0,0 +1,131 @@
+"""Helper script to run the API server, UI server, or both."""
+from __future__ import annotations
+
+import argparse
+import os
+import sys
+import warnings
+from multiprocessing import Process
+from pathlib import Path
+
+from dotenv import load_dotenv
+
+for _env_file in [
+    Path("/opt/myfsio/myfsio.env"),
+    Path.cwd() / ".env",
+    Path.cwd() / "myfsio.env",
+]:
+    if _env_file.exists():
+        load_dotenv(_env_file, override=True)
+
+from app import create_api_app, create_ui_app
+from app.config import AppConfig
+
+
+def _server_host() -> str:
+    """Return the bind host for API and UI servers."""
+    return os.getenv("APP_HOST", "0.0.0.0")
+
+
+def _is_debug_enabled() -> bool:
+    return os.getenv("FLASK_DEBUG", "0").lower() in ("1", "true", "yes")
+
+
+def _is_frozen() -> bool:
+    """Check if running as a compiled binary (PyInstaller/Nuitka)."""
+    return getattr(sys, 'frozen', False) or '__compiled__' in globals()
+
+
+def serve_api(port: int, prod: bool = False) -> None:
+    app = create_api_app()
+    if prod:
+        from waitress import serve
+        serve(app, host=_server_host(), port=port, ident="MyFSIO")
+    else:
+        debug = _is_debug_enabled()
+        if debug:
+            warnings.warn("DEBUG MODE ENABLED - DO NOT USE IN PRODUCTION", RuntimeWarning)
+        app.run(host=_server_host(), port=port, debug=debug)
+
+
+def serve_ui(port: int, prod: bool = False) -> None:
+    app = create_ui_app()
+    if prod:
+        from waitress import serve
+        serve(app, host=_server_host(), port=port, ident="MyFSIO")
+    else:
+        debug = _is_debug_enabled()
+        if debug:
+            warnings.warn("DEBUG MODE ENABLED - DO NOT USE IN PRODUCTION", RuntimeWarning)
+        app.run(host=_server_host(), port=port, debug=debug)
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(description="Run the S3 clone services.")
+    parser.add_argument("--mode", choices=["api", "ui", "both"], default="both")
+    parser.add_argument("--api-port", type=int, default=5000)
+    parser.add_argument("--ui-port", type=int, default=5100)
+    parser.add_argument("--prod", action="store_true", help="Run in production mode using Waitress")
+    parser.add_argument("--dev", action="store_true", help="Force development mode (Flask dev server)")
+    parser.add_argument("--check-config", action="store_true", help="Validate configuration and exit")
+    parser.add_argument("--show-config", action="store_true", help="Show configuration summary and exit")
+    args = parser.parse_args()
+
+    # Handle config check/show modes
+    if args.check_config or args.show_config:
+        config = AppConfig.from_env()
+        config.print_startup_summary()
+        if args.check_config:
+            issues = config.validate_and_report()
+            critical = [i for i in issues if i.startswith("CRITICAL:")]
+            sys.exit(1 if critical else 0)
+        sys.exit(0)
+
+    # Default to production mode when running as compiled binary
+    # unless --dev is explicitly passed
+    prod_mode = args.prod or (_is_frozen() and not args.dev)
+    
+    # Validate configuration before starting
+    config = AppConfig.from_env()
+    
+    # Show startup summary only on first run (when marker file doesn't exist)
+    first_run_marker = config.storage_root / ".myfsio.sys" / ".initialized"
+    is_first_run = not first_run_marker.exists()
+    
+    if is_first_run:
+        config.print_startup_summary()
+        
+        # Check for critical issues that should prevent startup
+        issues = config.validate_and_report()
+        critical_issues = [i for i in issues if i.startswith("CRITICAL:")]
+        if critical_issues:
+            print("ABORTING: Critical configuration issues detected. Fix them before starting.")
+            sys.exit(1)
+        
+        # Create the marker file to indicate successful first run
+        try:
+            first_run_marker.parent.mkdir(parents=True, exist_ok=True)
+            first_run_marker.write_text(f"Initialized on {__import__('datetime').datetime.now().isoformat()}\n")
+        except OSError:
+            pass  # Non-critical, just skip marker creation
+    
+    if prod_mode:
+        print("Running in production mode (Waitress)")
+    else:
+        print("Running in development mode (Flask dev server)")
+
+    if args.mode in {"api", "both"}:
+        print(f"Starting API server on port {args.api_port}...")
+        api_proc = Process(target=serve_api, args=(args.api_port, prod_mode), daemon=True)
+        api_proc.start()
+    else:
+        api_proc = None
+
+    if args.mode in {"ui", "both"}:
+        print(f"Starting UI server on port {args.ui_port}...")
+        serve_ui(args.ui_port, prod_mode)
+    elif api_proc:
+        try:
+            api_proc.join()
+        except KeyboardInterrupt:
+            pass

rust/.dockerignore

@@ -1,9 +0,0 @@
-.git
-.gitignore
-logs
-data
-tmp
-myfsio-engine/target
-myfsio-engine/tests
-Dockerfile
-.dockerignore

rust/Dockerfile

@@ -1,50 +0,0 @@
-FROM rust:1-slim-bookworm AS builder
-
-WORKDIR /build
-
-RUN apt-get update \
-    && apt-get install -y --no-install-recommends build-essential pkg-config libssl-dev \
-    && rm -rf /var/lib/apt/lists/*
-
-COPY myfsio-engine ./myfsio-engine
-
-RUN cd myfsio-engine \
-    && cargo build --release --bin myfsio-server \
-    && strip target/release/myfsio-server
-
-
-FROM debian:bookworm-slim
-
-WORKDIR /app
-
-RUN apt-get update \
-    && apt-get install -y --no-install-recommends ca-certificates curl \
-    && rm -rf /var/lib/apt/lists/* \
-    && mkdir -p /app/data \
-    && useradd -m -u 1000 myfsio \
-    && chown -R myfsio:myfsio /app
-
-COPY --from=builder /build/myfsio-engine/target/release/myfsio-server /usr/local/bin/myfsio-server
-COPY --from=builder /build/myfsio-engine/crates/myfsio-server/templates /app/templates
-COPY --from=builder /build/myfsio-engine/crates/myfsio-server/static /app/static
-COPY docker-entrypoint.sh /app/docker-entrypoint.sh
-
-RUN chmod +x /app/docker-entrypoint.sh \
-    && chown -R myfsio:myfsio /app
-
-USER myfsio
-
-EXPOSE 5000
-EXPOSE 5100
-ENV HOST=0.0.0.0 \
-    PORT=5000 \
-    UI_PORT=5100 \
-    STORAGE_ROOT=/app/data \
-    TEMPLATES_DIR=/app/templates \
-    STATIC_DIR=/app/static \
-    RUST_LOG=info
-
-HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
-    CMD curl -fsS "http://localhost:${PORT}/myfsio/health" || exit 1
-
-CMD ["/app/docker-entrypoint.sh"]

rust/docker-entrypoint.sh

@@ -1,4 +0,0 @@
-#!/bin/sh
-set -e
-
-exec /usr/local/bin/myfsio-server

rust/myfsio-engine/Cargo.toml

@@ -1,61 +0,0 @@
-[workspace]
-resolver = "2"
-members = [
-    "crates/myfsio-common",
-    "crates/myfsio-auth",
-    "crates/myfsio-crypto",
-    "crates/myfsio-storage",
-    "crates/myfsio-xml",
-    "crates/myfsio-server",
-]
-
-[workspace.package]
-version = "0.4.3"
-edition = "2021"
-
-[workspace.dependencies]
-tokio = { version = "1", features = ["full"] }
-axum = { version = "0.8" }
-tower = { version = "0.5" }
-tower-http = { version = "0.6", features = ["cors", "trace", "fs", "compression-gzip"] }
-hyper = { version = "1" }
-bytes = "1"
-serde = { version = "1", features = ["derive"] }
-serde_json = "1"
-quick-xml = { version = "0.37", features = ["serialize"] }
-hmac = "0.12"
-sha2 = "0.10"
-md-5 = "0.10"
-hex = "0.4"
-aes = "0.8"
-aes-gcm = "0.10"
-cbc = { version = "0.1", features = ["alloc"] }
-hkdf = "0.12"
-uuid = { version = "1", features = ["v4"] }
-parking_lot = "0.12"
-lru = "0.14"
-percent-encoding = "2"
-regex = "1"
-unicode-normalization = "0.1"
-tracing = "0.1"
-tracing-subscriber = "0.3"
-thiserror = "2"
-chrono = { version = "0.4", features = ["serde"] }
-base64 = "0.22"
-tokio-util = { version = "0.7", features = ["io"] }
-futures = "0.3"
-dashmap = "6"
-crc32fast = "1"
-duckdb = { version = "1", features = ["bundled"] }
-reqwest = { version = "0.12", default-features = false, features = ["stream", "rustls-tls", "json"] }
-aws-sdk-s3 = { version = "1", features = ["behavior-version-latest", "rt-tokio"] }
-aws-config = { version = "1", features = ["behavior-version-latest"] }
-aws-credential-types = "1"
-aws-smithy-runtime-api = "1"
-aws-smithy-types = "1"
-async-trait = "0.1"
-tera = "1"
-cookie = "0.18"
-subtle = "2"
-clap = { version = "4", features = ["derive"] }
-dotenvy = "0.15"

rust/myfsio-engine/crates/myfsio-auth/Cargo.toml

@@ -1,27 +0,0 @@
-[package]
-name = "myfsio-auth"
-version.workspace = true
-edition.workspace = true
-
-[dependencies]
-myfsio-common = { path = "../myfsio-common" }
-hmac = { workspace = true }
-sha2 = { workspace = true }
-hex = { workspace = true }
-aes = { workspace = true }
-cbc = { workspace = true }
-base64 = { workspace = true }
-pbkdf2 = "0.12"
-rand = "0.8"
-lru = { workspace = true }
-parking_lot = { workspace = true }
-percent-encoding = { workspace = true }
-serde = { workspace = true }
-serde_json = { workspace = true }
-thiserror = { workspace = true }
-chrono = { workspace = true }
-tracing = { workspace = true }
-uuid = { workspace = true }
-
-[dev-dependencies]
-tempfile = "3"

rust/myfsio-engine/crates/myfsio-auth/src/fernet.rs

@@ -1,118 +0,0 @@
-use aes::cipher::{block_padding::Pkcs7, BlockDecryptMut, BlockEncryptMut, KeyIvInit};
-use base64::{engine::general_purpose::URL_SAFE, Engine};
-use hmac::{Hmac, Mac};
-use rand::RngCore;
-use sha2::Sha256;
-
-type Aes128CbcDec = cbc::Decryptor<aes::Aes128>;
-type Aes128CbcEnc = cbc::Encryptor<aes::Aes128>;
-type HmacSha256 = Hmac<Sha256>;
-
-pub fn derive_fernet_key(secret: &str) -> String {
-    let mut derived = [0u8; 32];
-    pbkdf2::pbkdf2_hmac::<Sha256>(
-        secret.as_bytes(),
-        b"myfsio-iam-encryption",
-        100_000,
-        &mut derived,
-    );
-    URL_SAFE.encode(derived)
-}
-
-pub fn decrypt(key_b64: &str, token: &str) -> Result<Vec<u8>, &'static str> {
-    let key_bytes = URL_SAFE
-        .decode(key_b64)
-        .map_err(|_| "invalid fernet key base64")?;
-    if key_bytes.len() != 32 {
-        return Err("fernet key must be 32 bytes");
-    }
-
-    let signing_key = &key_bytes[..16];
-    let encryption_key = &key_bytes[16..];
-
-    let token_bytes = URL_SAFE
-        .decode(token)
-        .map_err(|_| "invalid fernet token base64")?;
-
-    if token_bytes.len() < 57 {
-        return Err("fernet token too short");
-    }
-
-    if token_bytes[0] != 0x80 {
-        return Err("invalid fernet version");
-    }
-
-    let hmac_offset = token_bytes.len() - 32;
-    let payload = &token_bytes[..hmac_offset];
-    let expected_hmac = &token_bytes[hmac_offset..];
-
-    let mut mac = HmacSha256::new_from_slice(signing_key).map_err(|_| "hmac key error")?;
-    mac.update(payload);
-    mac.verify_slice(expected_hmac)
-        .map_err(|_| "HMAC verification failed")?;
-
-    let iv = &token_bytes[9..25];
-    let ciphertext = &token_bytes[25..hmac_offset];
-
-    let plaintext = Aes128CbcDec::new(encryption_key.into(), iv.into())
-        .decrypt_padded_vec_mut::<Pkcs7>(ciphertext)
-        .map_err(|_| "AES-CBC decryption failed")?;
-
-    Ok(plaintext)
-}
-
-pub fn encrypt(key_b64: &str, plaintext: &[u8]) -> Result<String, &'static str> {
-    let key_bytes = URL_SAFE
-        .decode(key_b64)
-        .map_err(|_| "invalid fernet key base64")?;
-    if key_bytes.len() != 32 {
-        return Err("fernet key must be 32 bytes");
-    }
-
-    let signing_key = &key_bytes[..16];
-    let encryption_key = &key_bytes[16..];
-
-    let mut iv = [0u8; 16];
-    rand::thread_rng().fill_bytes(&mut iv);
-
-    let timestamp = std::time::SystemTime::now()
-        .duration_since(std::time::UNIX_EPOCH)
-        .map_err(|_| "system time error")?
-        .as_secs();
-
-    let ciphertext = Aes128CbcEnc::new(encryption_key.into(), (&iv).into())
-        .encrypt_padded_vec_mut::<Pkcs7>(plaintext);
-
-    let mut payload = Vec::with_capacity(1 + 8 + 16 + ciphertext.len());
-    payload.push(0x80);
-    payload.extend_from_slice(&timestamp.to_be_bytes());
-    payload.extend_from_slice(&iv);
-    payload.extend_from_slice(&ciphertext);
-
-    let mut mac = HmacSha256::new_from_slice(signing_key).map_err(|_| "hmac key error")?;
-    mac.update(&payload);
-    let tag = mac.finalize().into_bytes();
-
-    let mut token_bytes = payload;
-    token_bytes.extend_from_slice(&tag);
-    Ok(URL_SAFE.encode(&token_bytes))
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-
-    #[test]
-    fn test_derive_fernet_key_format() {
-        let key = derive_fernet_key("test-secret");
-        let decoded = URL_SAFE.decode(&key).unwrap();
-        assert_eq!(decoded.len(), 32);
-    }
-
-    #[test]
-    fn test_roundtrip_with_python_compat() {
-        let key = derive_fernet_key("dev-secret-key");
-        let decoded = URL_SAFE.decode(&key).unwrap();
-        assert_eq!(decoded.len(), 32);
-    }
-}

rust/myfsio-engine/crates/myfsio-auth/src/lib.rs

@@ -1,4 +0,0 @@
-mod fernet;
-pub mod iam;
-pub mod principal;
-pub mod sigv4;

rust/myfsio-engine/crates/myfsio-auth/src/principal.rs

@@ -1 +0,0 @@
-pub use myfsio_common::types::Principal;

rust/myfsio-engine/crates/myfsio-auth/src/sigv4.rs

@@ -1,287 +0,0 @@
-use hmac::{Hmac, Mac};
-use lru::LruCache;
-use parking_lot::Mutex;
-use percent_encoding::{percent_encode, AsciiSet, NON_ALPHANUMERIC};
-use sha2::{Digest, Sha256};
-use std::num::NonZeroUsize;
-use std::sync::LazyLock;
-use std::time::Instant;
-
-type HmacSha256 = Hmac<Sha256>;
-
-struct CacheEntry {
-    key: Vec<u8>,
-    created: Instant,
-}
-
-static SIGNING_KEY_CACHE: LazyLock<Mutex<LruCache<(String, String, String, String), CacheEntry>>> =
-    LazyLock::new(|| Mutex::new(LruCache::new(NonZeroUsize::new(256).unwrap())));
-
-const CACHE_TTL_SECS: u64 = 60;
-
-const AWS_ENCODE_SET: &AsciiSet = &NON_ALPHANUMERIC
-    .remove(b'-')
-    .remove(b'_')
-    .remove(b'.')
-    .remove(b'~');
-
-fn hmac_sha256(key: &[u8], msg: &[u8]) -> Vec<u8> {
-    let mut mac = HmacSha256::new_from_slice(key).expect("HMAC key length is always valid");
-    mac.update(msg);
-    mac.finalize().into_bytes().to_vec()
-}
-
-fn sha256_hex(data: &[u8]) -> String {
-    let mut hasher = Sha256::new();
-    hasher.update(data);
-    hex::encode(hasher.finalize())
-}
-
-fn aws_uri_encode(input: &str) -> String {
-    percent_encode(input.as_bytes(), AWS_ENCODE_SET).to_string()
-}
-
-pub fn derive_signing_key_cached(
-    secret_key: &str,
-    date_stamp: &str,
-    region: &str,
-    service: &str,
-) -> Vec<u8> {
-    let cache_key = (
-        secret_key.to_owned(),
-        date_stamp.to_owned(),
-        region.to_owned(),
-        service.to_owned(),
-    );
-
-    {
-        let mut cache = SIGNING_KEY_CACHE.lock();
-        if let Some(entry) = cache.get(&cache_key) {
-            if entry.created.elapsed().as_secs() < CACHE_TTL_SECS {
-                return entry.key.clone();
-            }
-            cache.pop(&cache_key);
-        }
-    }
-
-    let k_date = hmac_sha256(
-        format!("AWS4{}", secret_key).as_bytes(),
-        date_stamp.as_bytes(),
-    );
-    let k_region = hmac_sha256(&k_date, region.as_bytes());
-    let k_service = hmac_sha256(&k_region, service.as_bytes());
-    let k_signing = hmac_sha256(&k_service, b"aws4_request");
-
-    {
-        let mut cache = SIGNING_KEY_CACHE.lock();
-        cache.put(
-            cache_key,
-            CacheEntry {
-                key: k_signing.clone(),
-                created: Instant::now(),
-            },
-        );
-    }
-
-    k_signing
-}
-
-fn constant_time_compare_inner(a: &[u8], b: &[u8]) -> bool {
-    if a.len() != b.len() {
-        return false;
-    }
-    let mut result: u8 = 0;
-    for (x, y) in a.iter().zip(b.iter()) {
-        result |= x ^ y;
-    }
-    result == 0
-}
-
-pub fn verify_sigv4_signature(
-    method: &str,
-    canonical_uri: &str,
-    query_params: &[(String, String)],
-    signed_headers_str: &str,
-    header_values: &[(String, String)],
-    payload_hash: &str,
-    amz_date: &str,
-    date_stamp: &str,
-    region: &str,
-    service: &str,
-    secret_key: &str,
-    provided_signature: &str,
-) -> bool {
-    let mut sorted_params = query_params.to_vec();
-    sorted_params.sort_by(|a, b| a.0.cmp(&b.0).then_with(|| a.1.cmp(&b.1)));
-
-    let canonical_query_string = sorted_params
-        .iter()
-        .map(|(k, v)| format!("{}={}", aws_uri_encode(k), aws_uri_encode(v)))
-        .collect::<Vec<_>>()
-        .join("&");
-
-    let mut canonical_headers = String::new();
-    for (name, value) in header_values {
-        let lower_name = name.to_lowercase();
-        let normalized = value.split_whitespace().collect::<Vec<_>>().join(" ");
-        let final_value = if lower_name == "expect" && normalized.is_empty() {
-            "100-continue"
-        } else {
-            &normalized
-        };
-        canonical_headers.push_str(&lower_name);
-        canonical_headers.push(':');
-        canonical_headers.push_str(final_value);
-        canonical_headers.push('\n');
-    }
-
-    let canonical_request = format!(
-        "{}\n{}\n{}\n{}\n{}\n{}",
-        method,
-        canonical_uri,
-        canonical_query_string,
-        canonical_headers,
-        signed_headers_str,
-        payload_hash
-    );
-
-    let credential_scope = format!("{}/{}/{}/aws4_request", date_stamp, region, service);
-    let cr_hash = sha256_hex(canonical_request.as_bytes());
-    let string_to_sign = format!(
-        "AWS4-HMAC-SHA256\n{}\n{}\n{}",
-        amz_date, credential_scope, cr_hash
-    );
-
-    let signing_key = derive_signing_key_cached(secret_key, date_stamp, region, service);
-    let calculated = hmac_sha256(&signing_key, string_to_sign.as_bytes());
-    let calculated_hex = hex::encode(&calculated);
-
-    constant_time_compare_inner(calculated_hex.as_bytes(), provided_signature.as_bytes())
-}
-
-pub fn derive_signing_key(
-    secret_key: &str,
-    date_stamp: &str,
-    region: &str,
-    service: &str,
-) -> Vec<u8> {
-    derive_signing_key_cached(secret_key, date_stamp, region, service)
-}
-
-pub fn compute_signature(signing_key: &[u8], string_to_sign: &str) -> String {
-    let sig = hmac_sha256(signing_key, string_to_sign.as_bytes());
-    hex::encode(sig)
-}
-
-pub fn compute_post_policy_signature(signing_key: &[u8], policy_b64: &str) -> String {
-    let sig = hmac_sha256(signing_key, policy_b64.as_bytes());
-    hex::encode(sig)
-}
-
-pub fn build_string_to_sign(
-    amz_date: &str,
-    credential_scope: &str,
-    canonical_request: &str,
-) -> String {
-    let cr_hash = sha256_hex(canonical_request.as_bytes());
-    format!(
-        "AWS4-HMAC-SHA256\n{}\n{}\n{}",
-        amz_date, credential_scope, cr_hash
-    )
-}
-
-pub fn constant_time_compare(a: &str, b: &str) -> bool {
-    constant_time_compare_inner(a.as_bytes(), b.as_bytes())
-}
-
-pub fn clear_signing_key_cache() {
-    SIGNING_KEY_CACHE.lock().clear();
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-
-    #[test]
-    fn test_derive_signing_key() {
-        let key = derive_signing_key(
-            "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
-            "20130524",
-            "us-east-1",
-            "s3",
-        );
-        assert_eq!(key.len(), 32);
-    }
-
-    #[test]
-    fn test_derive_signing_key_cached() {
-        let key1 = derive_signing_key("secret", "20240101", "us-east-1", "s3");
-        let key2 = derive_signing_key("secret", "20240101", "us-east-1", "s3");
-        assert_eq!(key1, key2);
-    }
-
-    #[test]
-    fn test_constant_time_compare() {
-        assert!(constant_time_compare("abc", "abc"));
-        assert!(!constant_time_compare("abc", "abd"));
-        assert!(!constant_time_compare("abc", "abcd"));
-    }
-
-    #[test]
-    fn test_build_string_to_sign() {
-        let result = build_string_to_sign(
-            "20130524T000000Z",
-            "20130524/us-east-1/s3/aws4_request",
-            "GET\n/\n\nhost:example.com\n\nhost\nUNSIGNED-PAYLOAD",
-        );
-        assert!(result.starts_with("AWS4-HMAC-SHA256\n"));
-        assert!(result.contains("20130524T000000Z"));
-    }
-
-    #[test]
-    fn test_aws_uri_encode() {
-        assert_eq!(aws_uri_encode("hello world"), "hello%20world");
-        assert_eq!(aws_uri_encode("test-file_name.txt"), "test-file_name.txt");
-        assert_eq!(aws_uri_encode("a/b"), "a%2Fb");
-    }
-
-    #[test]
-    fn test_verify_sigv4_roundtrip() {
-        let secret = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY";
-        let date_stamp = "20130524";
-        let region = "us-east-1";
-        let service = "s3";
-        let amz_date = "20130524T000000Z";
-
-        let signing_key = derive_signing_key(secret, date_stamp, region, service);
-
-        let canonical_request =
-            "GET\n/\n\nhost:examplebucket.s3.amazonaws.com\n\nhost\nUNSIGNED-PAYLOAD";
-        let string_to_sign = build_string_to_sign(
-            amz_date,
-            &format!("{}/{}/{}/aws4_request", date_stamp, region, service),
-            canonical_request,
-        );
-
-        let signature = compute_signature(&signing_key, &string_to_sign);
-
-        let result = verify_sigv4_signature(
-            "GET",
-            "/",
-            &[],
-            "host",
-            &[(
-                "host".to_string(),
-                "examplebucket.s3.amazonaws.com".to_string(),
-            )],
-            "UNSIGNED-PAYLOAD",
-            amz_date,
-            date_stamp,
-            region,
-            service,
-            secret,
-            &signature,
-        );
-        assert!(result);
-    }
-}

rust/myfsio-engine/crates/myfsio-common/Cargo.toml

@@ -1,11 +0,0 @@
-[package]
-name = "myfsio-common"
-version.workspace = true
-edition.workspace = true
-
-[dependencies]
-thiserror = { workspace = true }
-serde = { workspace = true }
-serde_json = { workspace = true }
-chrono = { workspace = true }
-uuid = { workspace = true }

rust/myfsio-engine/crates/myfsio-common/src/constants.rs

@@ -1,20 +0,0 @@
-pub const SYSTEM_ROOT: &str = ".myfsio.sys";
-pub const SYSTEM_BUCKETS_DIR: &str = "buckets";
-pub const SYSTEM_MULTIPART_DIR: &str = "multipart";
-pub const BUCKET_META_DIR: &str = "meta";
-pub const BUCKET_VERSIONS_DIR: &str = "versions";
-pub const BUCKET_CONFIG_FILE: &str = ".bucket.json";
-pub const STATS_FILE: &str = "stats.json";
-pub const ETAG_INDEX_FILE: &str = "etag_index.json";
-pub const INDEX_FILE: &str = "_index.json";
-pub const MANIFEST_FILE: &str = "manifest.json";
-
-pub const INTERNAL_FOLDERS: &[&str] = &[".meta", ".versions", ".multipart"];
-
-pub const DEFAULT_REGION: &str = "us-east-1";
-pub const AWS_SERVICE: &str = "s3";
-
-pub const DEFAULT_MAX_KEYS: usize = 1000;
-pub const DEFAULT_OBJECT_KEY_MAX_BYTES: usize = 1024;
-pub const DEFAULT_CHUNK_SIZE: usize = 65536;
-pub const STREAM_CHUNK_SIZE: usize = 1_048_576;

rust/myfsio-engine/crates/myfsio-common/src/error.rs

@@ -1,233 +0,0 @@
-use std::fmt;
-
-#[derive(Debug, Clone, Copy, PartialEq, Eq)]
-pub enum S3ErrorCode {
-    AccessDenied,
-    BadDigest,
-    BucketAlreadyExists,
-    BucketNotEmpty,
-    EntityTooLarge,
-    InternalError,
-    InvalidAccessKeyId,
-    InvalidArgument,
-    InvalidBucketName,
-    InvalidKey,
-    InvalidPolicyDocument,
-    InvalidRange,
-    InvalidRequest,
-    InvalidTag,
-    MalformedXML,
-    MethodNotAllowed,
-    NoSuchBucket,
-    NoSuchKey,
-    NoSuchUpload,
-    NoSuchVersion,
-    NoSuchTagSet,
-    PreconditionFailed,
-    NotModified,
-    QuotaExceeded,
-    SignatureDoesNotMatch,
-    SlowDown,
-}
-
-impl S3ErrorCode {
-    pub fn http_status(&self) -> u16 {
-        match self {
-            Self::AccessDenied => 403,
-            Self::BadDigest => 400,
-            Self::BucketAlreadyExists => 409,
-            Self::BucketNotEmpty => 409,
-            Self::EntityTooLarge => 413,
-            Self::InternalError => 500,
-            Self::InvalidAccessKeyId => 403,
-            Self::InvalidArgument => 400,
-            Self::InvalidBucketName => 400,
-            Self::InvalidKey => 400,
-            Self::InvalidPolicyDocument => 400,
-            Self::InvalidRange => 416,
-            Self::InvalidRequest => 400,
-            Self::InvalidTag => 400,
-            Self::MalformedXML => 400,
-            Self::MethodNotAllowed => 405,
-            Self::NoSuchBucket => 404,
-            Self::NoSuchKey => 404,
-            Self::NoSuchUpload => 404,
-            Self::NoSuchVersion => 404,
-            Self::NoSuchTagSet => 404,
-            Self::PreconditionFailed => 412,
-            Self::NotModified => 304,
-            Self::QuotaExceeded => 403,
-            Self::SignatureDoesNotMatch => 403,
-            Self::SlowDown => 429,
-        }
-    }
-
-    pub fn as_str(&self) -> &'static str {
-        match self {
-            Self::AccessDenied => "AccessDenied",
-            Self::BadDigest => "BadDigest",
-            Self::BucketAlreadyExists => "BucketAlreadyExists",
-            Self::BucketNotEmpty => "BucketNotEmpty",
-            Self::EntityTooLarge => "EntityTooLarge",
-            Self::InternalError => "InternalError",
-            Self::InvalidAccessKeyId => "InvalidAccessKeyId",
-            Self::InvalidArgument => "InvalidArgument",
-            Self::InvalidBucketName => "InvalidBucketName",
-            Self::InvalidKey => "InvalidKey",
-            Self::InvalidPolicyDocument => "InvalidPolicyDocument",
-            Self::InvalidRange => "InvalidRange",
-            Self::InvalidRequest => "InvalidRequest",
-            Self::InvalidTag => "InvalidTag",
-            Self::MalformedXML => "MalformedXML",
-            Self::MethodNotAllowed => "MethodNotAllowed",
-            Self::NoSuchBucket => "NoSuchBucket",
-            Self::NoSuchKey => "NoSuchKey",
-            Self::NoSuchUpload => "NoSuchUpload",
-            Self::NoSuchVersion => "NoSuchVersion",
-            Self::NoSuchTagSet => "NoSuchTagSet",
-            Self::PreconditionFailed => "PreconditionFailed",
-            Self::NotModified => "NotModified",
-            Self::QuotaExceeded => "QuotaExceeded",
-            Self::SignatureDoesNotMatch => "SignatureDoesNotMatch",
-            Self::SlowDown => "SlowDown",
-        }
-    }
-
-    pub fn default_message(&self) -> &'static str {
-        match self {
-            Self::AccessDenied => "Access Denied",
-            Self::BadDigest => "The Content-MD5 or checksum value you specified did not match what we received",
-            Self::BucketAlreadyExists => "The requested bucket name is not available",
-            Self::BucketNotEmpty => "The bucket you tried to delete is not empty",
-            Self::EntityTooLarge => "Your proposed upload exceeds the maximum allowed size",
-            Self::InternalError => "We encountered an internal error. Please try again.",
-            Self::InvalidAccessKeyId => "The access key ID you provided does not exist",
-            Self::InvalidArgument => "Invalid argument",
-            Self::InvalidBucketName => "The specified bucket is not valid",
-            Self::InvalidKey => "The specified key is not valid",
-            Self::InvalidPolicyDocument => "The content of the form does not meet the conditions specified in the policy document",
-            Self::InvalidRange => "The requested range is not satisfiable",
-            Self::InvalidRequest => "Invalid request",
-            Self::InvalidTag => "The Tagging header is invalid",
-            Self::MalformedXML => "The XML you provided was not well-formed",
-            Self::MethodNotAllowed => "The specified method is not allowed against this resource",
-            Self::NoSuchBucket => "The specified bucket does not exist",
-            Self::NoSuchKey => "The specified key does not exist",
-            Self::NoSuchUpload => "The specified multipart upload does not exist",
-            Self::NoSuchVersion => "The specified version does not exist",
-            Self::NoSuchTagSet => "The TagSet does not exist",
-            Self::PreconditionFailed => "At least one of the preconditions you specified did not hold",
-            Self::NotModified => "Not Modified",
-            Self::QuotaExceeded => "The bucket quota has been exceeded",
-            Self::SignatureDoesNotMatch => "The request signature we calculated does not match the signature you provided",
-            Self::SlowDown => "Please reduce your request rate",
-        }
-    }
-}
-
-impl fmt::Display for S3ErrorCode {
-    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-        f.write_str(self.as_str())
-    }
-}
-
-#[derive(Debug, Clone)]
-pub struct S3Error {
-    pub code: S3ErrorCode,
-    pub message: String,
-    pub resource: String,
-    pub request_id: String,
-}
-
-impl S3Error {
-    pub fn new(code: S3ErrorCode, message: impl Into<String>) -> Self {
-        Self {
-            code,
-            message: message.into(),
-            resource: String::new(),
-            request_id: String::new(),
-        }
-    }
-
-    pub fn from_code(code: S3ErrorCode) -> Self {
-        Self::new(code, code.default_message())
-    }
-
-    pub fn with_resource(mut self, resource: impl Into<String>) -> Self {
-        self.resource = resource.into();
-        self
-    }
-
-    pub fn with_request_id(mut self, request_id: impl Into<String>) -> Self {
-        self.request_id = request_id.into();
-        self
-    }
-
-    pub fn http_status(&self) -> u16 {
-        self.code.http_status()
-    }
-
-    pub fn to_xml(&self) -> String {
-        format!(
-            "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\
-            <Error>\
-            <Code>{}</Code>\
-            <Message>{}</Message>\
-            <Resource>{}</Resource>\
-            <RequestId>{}</RequestId>\
-            </Error>",
-            self.code.as_str(),
-            xml_escape(&self.message),
-            xml_escape(&self.resource),
-            xml_escape(&self.request_id),
-        )
-    }
-}
-
-impl fmt::Display for S3Error {
-    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-        write!(f, "{}: {}", self.code, self.message)
-    }
-}
-
-impl std::error::Error for S3Error {}
-
-fn xml_escape(s: &str) -> String {
-    s.replace('&', "&amp;")
-        .replace('<', "&lt;")
-        .replace('>', "&gt;")
-        .replace('"', "&quot;")
-        .replace('\'', "&apos;")
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-
-    #[test]
-    fn test_error_codes() {
-        assert_eq!(S3ErrorCode::NoSuchKey.http_status(), 404);
-        assert_eq!(S3ErrorCode::AccessDenied.http_status(), 403);
-        assert_eq!(S3ErrorCode::NoSuchBucket.as_str(), "NoSuchBucket");
-    }
-
-    #[test]
-    fn test_error_to_xml() {
-        let err = S3Error::from_code(S3ErrorCode::NoSuchKey)
-            .with_resource("/test-bucket/test-key")
-            .with_request_id("abc123");
-        let xml = err.to_xml();
-        assert!(xml.contains("<Code>NoSuchKey</Code>"));
-        assert!(xml.contains("<Resource>/test-bucket/test-key</Resource>"));
-        assert!(xml.contains("<RequestId>abc123</RequestId>"));
-    }
-
-    #[test]
-    fn test_xml_escape() {
-        let err = S3Error::new(S3ErrorCode::InvalidArgument, "key <test> & \"value\"")
-            .with_resource("/bucket/key&amp");
-        let xml = err.to_xml();
-        assert!(xml.contains("&lt;test&gt;"));
-        assert!(xml.contains("&amp;"));
-    }
-}

rust/myfsio-engine/crates/myfsio-common/src/lib.rs

@@ -1,3 +0,0 @@
-pub mod constants;
-pub mod error;
-pub mod types;

rust/myfsio-engine/crates/myfsio-common/src/types.rs

@@ -1,178 +0,0 @@
-use std::collections::HashMap;
-
-use chrono::{DateTime, Utc};
-use serde::{Deserialize, Serialize};
-
-#[derive(Debug, Clone, Serialize, Deserialize)]
-pub struct ObjectMeta {
-    pub key: String,
-    pub size: u64,
-    pub last_modified: DateTime<Utc>,
-    pub etag: Option<String>,
-    pub content_type: Option<String>,
-    pub storage_class: Option<String>,
-    pub metadata: HashMap<String, String>,
-}
-
-impl ObjectMeta {
-    pub fn new(key: String, size: u64, last_modified: DateTime<Utc>) -> Self {
-        Self {
-            key,
-            size,
-            last_modified,
-            etag: None,
-            content_type: None,
-            storage_class: Some("STANDARD".to_string()),
-            metadata: HashMap::new(),
-        }
-    }
-}
-
-#[derive(Debug, Clone, Serialize, Deserialize)]
-pub struct BucketMeta {
-    pub name: String,
-    pub creation_date: DateTime<Utc>,
-}
-
-#[derive(Debug, Clone, Default)]
-pub struct BucketStats {
-    pub objects: u64,
-    pub bytes: u64,
-    pub version_count: u64,
-    pub version_bytes: u64,
-}
-
-impl BucketStats {
-    pub fn total_objects(&self) -> u64 {
-        self.objects + self.version_count
-    }
-
-    pub fn total_bytes(&self) -> u64 {
-        self.bytes + self.version_bytes
-    }
-}
-
-#[derive(Debug, Clone)]
-pub struct ListObjectsResult {
-    pub objects: Vec<ObjectMeta>,
-    pub is_truncated: bool,
-    pub next_continuation_token: Option<String>,
-}
-
-#[derive(Debug, Clone)]
-pub struct ShallowListResult {
-    pub objects: Vec<ObjectMeta>,
-    pub common_prefixes: Vec<String>,
-    pub is_truncated: bool,
-    pub next_continuation_token: Option<String>,
-}
-
-#[derive(Debug, Clone, Default)]
-pub struct ListParams {
-    pub max_keys: usize,
-    pub continuation_token: Option<String>,
-    pub prefix: Option<String>,
-    pub start_after: Option<String>,
-}
-
-#[derive(Debug, Clone, Default)]
-pub struct ShallowListParams {
-    pub prefix: String,
-    pub delimiter: String,
-    pub max_keys: usize,
-    pub continuation_token: Option<String>,
-}
-
-#[derive(Debug, Clone, Serialize, Deserialize)]
-pub struct PartMeta {
-    pub part_number: u32,
-    pub etag: String,
-    pub size: u64,
-    pub last_modified: Option<DateTime<Utc>>,
-}
-
-#[derive(Debug, Clone)]
-pub struct PartInfo {
-    pub part_number: u32,
-    pub etag: String,
-}
-
-#[derive(Debug, Clone, Serialize, Deserialize)]
-pub struct MultipartUploadInfo {
-    pub upload_id: String,
-    pub key: String,
-    pub initiated: DateTime<Utc>,
-}
-
-#[derive(Debug, Clone, Serialize, Deserialize)]
-pub struct VersionInfo {
-    pub version_id: String,
-    pub key: String,
-    pub size: u64,
-    pub last_modified: DateTime<Utc>,
-    pub etag: Option<String>,
-    pub is_latest: bool,
-    #[serde(default)]
-    pub is_delete_marker: bool,
-}
-
-#[derive(Debug, Clone, Serialize, Deserialize)]
-pub struct Tag {
-    pub key: String,
-    pub value: String,
-}
-
-#[derive(Debug, Clone, Default, Serialize, Deserialize)]
-pub struct BucketConfig {
-    #[serde(default)]
-    pub versioning_enabled: bool,
-    #[serde(default)]
-    pub tags: Vec<Tag>,
-    #[serde(default)]
-    pub cors: Option<serde_json::Value>,
-    #[serde(default)]
-    pub encryption: Option<serde_json::Value>,
-    #[serde(default)]
-    pub lifecycle: Option<serde_json::Value>,
-    #[serde(default)]
-    pub website: Option<serde_json::Value>,
-    #[serde(default)]
-    pub quota: Option<QuotaConfig>,
-    #[serde(default)]
-    pub acl: Option<serde_json::Value>,
-    #[serde(default)]
-    pub notification: Option<serde_json::Value>,
-    #[serde(default)]
-    pub logging: Option<serde_json::Value>,
-    #[serde(default)]
-    pub object_lock: Option<serde_json::Value>,
-    #[serde(default)]
-    pub policy: Option<serde_json::Value>,
-    #[serde(default)]
-    pub replication: Option<serde_json::Value>,
-}
-
-#[derive(Debug, Clone, Serialize, Deserialize)]
-pub struct QuotaConfig {
-    pub max_bytes: Option<u64>,
-    pub max_objects: Option<u64>,
-}
-
-#[derive(Debug, Clone)]
-pub struct Principal {
-    pub access_key: String,
-    pub user_id: String,
-    pub display_name: String,
-    pub is_admin: bool,
-}
-
-impl Principal {
-    pub fn new(access_key: String, user_id: String, display_name: String, is_admin: bool) -> Self {
-        Self {
-            access_key,
-            user_id,
-            display_name,
-            is_admin,
-        }
-    }
-}

rust/myfsio-engine/crates/myfsio-crypto/Cargo.toml

@@ -1,24 +0,0 @@
-[package]
-name = "myfsio-crypto"
-version.workspace = true
-edition.workspace = true
-
-[dependencies]
-myfsio-common = { path = "../myfsio-common" }
-md-5 = { workspace = true }
-sha2 = { workspace = true }
-hex = { workspace = true }
-aes-gcm = { workspace = true }
-hkdf = { workspace = true }
-thiserror = { workspace = true }
-tokio = { workspace = true }
-serde = { workspace = true }
-serde_json = { workspace = true }
-uuid = { workspace = true }
-chrono = { workspace = true }
-base64 = { workspace = true }
-rand = "0.8"
-
-[dev-dependencies]
-tokio = { workspace = true, features = ["macros", "rt-multi-thread"] }
-tempfile = "3"

rust/myfsio-engine/crates/myfsio-crypto/src/aes_gcm.rs

@@ -1,253 +0,0 @@
-use aes_gcm::aead::Aead;
-use aes_gcm::{Aes256Gcm, KeyInit, Nonce};
-use hkdf::Hkdf;
-use sha2::Sha256;
-use std::fs::File;
-use std::io::{Read, Seek, SeekFrom, Write};
-use std::path::Path;
-use thiserror::Error;
-
-const DEFAULT_CHUNK_SIZE: usize = 65536;
-const HEADER_SIZE: usize = 4;
-
-#[derive(Debug, Error)]
-pub enum CryptoError {
-    #[error("IO error: {0}")]
-    Io(#[from] std::io::Error),
-    #[error("Invalid key size: expected 32 bytes, got {0}")]
-    InvalidKeySize(usize),
-    #[error("Invalid nonce size: expected 12 bytes, got {0}")]
-    InvalidNonceSize(usize),
-    #[error("Encryption failed: {0}")]
-    EncryptionFailed(String),
-    #[error("Decryption failed at chunk {0}")]
-    DecryptionFailed(u32),
-    #[error("HKDF expand failed: {0}")]
-    HkdfFailed(String),
-}
-
-fn read_exact_chunk(reader: &mut impl Read, buf: &mut [u8]) -> std::io::Result<usize> {
-    let mut filled = 0;
-    while filled < buf.len() {
-        match reader.read(&mut buf[filled..]) {
-            Ok(0) => break,
-            Ok(n) => filled += n,
-            Err(ref e) if e.kind() == std::io::ErrorKind::Interrupted => continue,
-            Err(e) => return Err(e),
-        }
-    }
-    Ok(filled)
-}
-
-fn derive_chunk_nonce(base_nonce: &[u8], chunk_index: u32) -> Result<[u8; 12], CryptoError> {
-    let hkdf = Hkdf::<Sha256>::new(Some(base_nonce), b"chunk_nonce");
-    let mut okm = [0u8; 12];
-    hkdf.expand(&chunk_index.to_be_bytes(), &mut okm)
-        .map_err(|e| CryptoError::HkdfFailed(e.to_string()))?;
-    Ok(okm)
-}
-
-pub fn encrypt_stream_chunked(
-    input_path: &Path,
-    output_path: &Path,
-    key: &[u8],
-    base_nonce: &[u8],
-    chunk_size: Option<usize>,
-) -> Result<u32, CryptoError> {
-    if key.len() != 32 {
-        return Err(CryptoError::InvalidKeySize(key.len()));
-    }
-    if base_nonce.len() != 12 {
-        return Err(CryptoError::InvalidNonceSize(base_nonce.len()));
-    }
-
-    let chunk_size = chunk_size.unwrap_or(DEFAULT_CHUNK_SIZE);
-    let key_arr: [u8; 32] = key.try_into().unwrap();
-    let nonce_arr: [u8; 12] = base_nonce.try_into().unwrap();
-    let cipher = Aes256Gcm::new(&key_arr.into());
-
-    let mut infile = File::open(input_path)?;
-    let mut outfile = File::create(output_path)?;
-
-    outfile.write_all(&[0u8; 4])?;
-
-    let mut buf = vec![0u8; chunk_size];
-    let mut chunk_index: u32 = 0;
-
-    loop {
-        let n = read_exact_chunk(&mut infile, &mut buf)?;
-        if n == 0 {
-            break;
-        }
-
-        let nonce_bytes = derive_chunk_nonce(&nonce_arr, chunk_index)?;
-        let nonce = Nonce::from_slice(&nonce_bytes);
-
-        let encrypted = cipher
-            .encrypt(nonce, &buf[..n])
-            .map_err(|e| CryptoError::EncryptionFailed(e.to_string()))?;
-
-        let size = encrypted.len() as u32;
-        outfile.write_all(&size.to_be_bytes())?;
-        outfile.write_all(&encrypted)?;
-
-        chunk_index += 1;
-    }
-
-    outfile.seek(SeekFrom::Start(0))?;
-    outfile.write_all(&chunk_index.to_be_bytes())?;
-
-    Ok(chunk_index)
-}
-
-pub fn decrypt_stream_chunked(
-    input_path: &Path,
-    output_path: &Path,
-    key: &[u8],
-    base_nonce: &[u8],
-) -> Result<u32, CryptoError> {
-    if key.len() != 32 {
-        return Err(CryptoError::InvalidKeySize(key.len()));
-    }
-    if base_nonce.len() != 12 {
-        return Err(CryptoError::InvalidNonceSize(base_nonce.len()));
-    }
-
-    let key_arr: [u8; 32] = key.try_into().unwrap();
-    let nonce_arr: [u8; 12] = base_nonce.try_into().unwrap();
-    let cipher = Aes256Gcm::new(&key_arr.into());
-
-    let mut infile = File::open(input_path)?;
-    let mut outfile = File::create(output_path)?;
-
-    let mut header = [0u8; HEADER_SIZE];
-    infile.read_exact(&mut header)?;
-    let chunk_count = u32::from_be_bytes(header);
-
-    let mut size_buf = [0u8; HEADER_SIZE];
-    for chunk_index in 0..chunk_count {
-        infile.read_exact(&mut size_buf)?;
-        let chunk_size = u32::from_be_bytes(size_buf) as usize;
-
-        let mut encrypted = vec![0u8; chunk_size];
-        infile.read_exact(&mut encrypted)?;
-
-        let nonce_bytes = derive_chunk_nonce(&nonce_arr, chunk_index)?;
-        let nonce = Nonce::from_slice(&nonce_bytes);
-
-        let decrypted = cipher
-            .decrypt(nonce, encrypted.as_ref())
-            .map_err(|_| CryptoError::DecryptionFailed(chunk_index))?;
-
-        outfile.write_all(&decrypted)?;
-    }
-
-    Ok(chunk_count)
-}
-
-pub async fn encrypt_stream_chunked_async(
-    input_path: &Path,
-    output_path: &Path,
-    key: &[u8],
-    base_nonce: &[u8],
-    chunk_size: Option<usize>,
-) -> Result<u32, CryptoError> {
-    let input_path = input_path.to_owned();
-    let output_path = output_path.to_owned();
-    let key = key.to_vec();
-    let base_nonce = base_nonce.to_vec();
-    tokio::task::spawn_blocking(move || {
-        encrypt_stream_chunked(&input_path, &output_path, &key, &base_nonce, chunk_size)
-    })
-    .await
-    .map_err(|e| CryptoError::Io(std::io::Error::new(std::io::ErrorKind::Other, e)))?
-}
-
-pub async fn decrypt_stream_chunked_async(
-    input_path: &Path,
-    output_path: &Path,
-    key: &[u8],
-    base_nonce: &[u8],
-) -> Result<u32, CryptoError> {
-    let input_path = input_path.to_owned();
-    let output_path = output_path.to_owned();
-    let key = key.to_vec();
-    let base_nonce = base_nonce.to_vec();
-    tokio::task::spawn_blocking(move || {
-        decrypt_stream_chunked(&input_path, &output_path, &key, &base_nonce)
-    })
-    .await
-    .map_err(|e| CryptoError::Io(std::io::Error::new(std::io::ErrorKind::Other, e)))?
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-    use std::io::Write as IoWrite;
-
-    #[test]
-    fn test_encrypt_decrypt_roundtrip() {
-        let dir = tempfile::tempdir().unwrap();
-        let input = dir.path().join("input.bin");
-        let encrypted = dir.path().join("encrypted.bin");
-        let decrypted = dir.path().join("decrypted.bin");
-
-        let data = b"Hello, this is a test of AES-256-GCM chunked encryption!";
-        std::fs::File::create(&input)
-            .unwrap()
-            .write_all(data)
-            .unwrap();
-
-        let key = [0x42u8; 32];
-        let nonce = [0x01u8; 12];
-
-        let chunks = encrypt_stream_chunked(&input, &encrypted, &key, &nonce, Some(16)).unwrap();
-        assert!(chunks > 0);
-
-        let chunks2 = decrypt_stream_chunked(&encrypted, &decrypted, &key, &nonce).unwrap();
-        assert_eq!(chunks, chunks2);
-
-        let result = std::fs::read(&decrypted).unwrap();
-        assert_eq!(result, data);
-    }
-
-    #[test]
-    fn test_invalid_key_size() {
-        let dir = tempfile::tempdir().unwrap();
-        let input = dir.path().join("input.bin");
-        std::fs::File::create(&input)
-            .unwrap()
-            .write_all(b"test")
-            .unwrap();
-
-        let result = encrypt_stream_chunked(
-            &input,
-            &dir.path().join("out"),
-            &[0u8; 16],
-            &[0u8; 12],
-            None,
-        );
-        assert!(matches!(result, Err(CryptoError::InvalidKeySize(16))));
-    }
-
-    #[test]
-    fn test_wrong_key_fails_decrypt() {
-        let dir = tempfile::tempdir().unwrap();
-        let input = dir.path().join("input.bin");
-        let encrypted = dir.path().join("encrypted.bin");
-        let decrypted = dir.path().join("decrypted.bin");
-
-        std::fs::File::create(&input)
-            .unwrap()
-            .write_all(b"secret data")
-            .unwrap();
-
-        let key = [0x42u8; 32];
-        let nonce = [0x01u8; 12];
-        encrypt_stream_chunked(&input, &encrypted, &key, &nonce, None).unwrap();
-
-        let wrong_key = [0x43u8; 32];
-        let result = decrypt_stream_chunked(&encrypted, &decrypted, &wrong_key, &nonce);
-        assert!(matches!(result, Err(CryptoError::DecryptionFailed(_))));
-    }
-}

rust/myfsio-engine/crates/myfsio-crypto/src/encryption.rs

@@ -1,377 +0,0 @@
-use base64::engine::general_purpose::STANDARD as B64;
-use base64::Engine;
-use rand::RngCore;
-use std::collections::HashMap;
-use std::path::Path;
-
-use crate::aes_gcm::{decrypt_stream_chunked, encrypt_stream_chunked, CryptoError};
-use crate::kms::KmsService;
-
-#[derive(Debug, Clone, PartialEq)]
-pub enum SseAlgorithm {
-    Aes256,
-    AwsKms,
-    CustomerProvided,
-}
-
-impl SseAlgorithm {
-    pub fn as_str(&self) -> &'static str {
-        match self {
-            SseAlgorithm::Aes256 => "AES256",
-            SseAlgorithm::AwsKms => "aws:kms",
-            SseAlgorithm::CustomerProvided => "AES256",
-        }
-    }
-}
-
-#[derive(Debug, Clone)]
-pub struct EncryptionContext {
-    pub algorithm: SseAlgorithm,
-    pub kms_key_id: Option<String>,
-    pub customer_key: Option<Vec<u8>>,
-}
-
-#[derive(Debug, Clone)]
-pub struct EncryptionMetadata {
-    pub algorithm: String,
-    pub nonce: String,
-    pub encrypted_data_key: Option<String>,
-    pub kms_key_id: Option<String>,
-}
-
-impl EncryptionMetadata {
-    pub fn to_metadata_map(&self) -> HashMap<String, String> {
-        let mut map = HashMap::new();
-        map.insert(
-            "x-amz-server-side-encryption".to_string(),
-            self.algorithm.clone(),
-        );
-        map.insert("x-amz-encryption-nonce".to_string(), self.nonce.clone());
-        if let Some(ref dk) = self.encrypted_data_key {
-            map.insert("x-amz-encrypted-data-key".to_string(), dk.clone());
-        }
-        if let Some(ref kid) = self.kms_key_id {
-            map.insert("x-amz-encryption-key-id".to_string(), kid.clone());
-        }
-        map
-    }
-
-    pub fn from_metadata(meta: &HashMap<String, String>) -> Option<Self> {
-        let algorithm = meta.get("x-amz-server-side-encryption")?;
-        let nonce = meta.get("x-amz-encryption-nonce")?;
-        Some(Self {
-            algorithm: algorithm.clone(),
-            nonce: nonce.clone(),
-            encrypted_data_key: meta.get("x-amz-encrypted-data-key").cloned(),
-            kms_key_id: meta.get("x-amz-encryption-key-id").cloned(),
-        })
-    }
-
-    pub fn is_encrypted(meta: &HashMap<String, String>) -> bool {
-        meta.contains_key("x-amz-server-side-encryption")
-    }
-
-    pub fn clean_metadata(meta: &mut HashMap<String, String>) {
-        meta.remove("x-amz-server-side-encryption");
-        meta.remove("x-amz-encryption-nonce");
-        meta.remove("x-amz-encrypted-data-key");
-        meta.remove("x-amz-encryption-key-id");
-    }
-}
-
-pub struct EncryptionService {
-    master_key: [u8; 32],
-    kms: Option<std::sync::Arc<KmsService>>,
-}
-
-impl EncryptionService {
-    pub fn new(master_key: [u8; 32], kms: Option<std::sync::Arc<KmsService>>) -> Self {
-        Self { master_key, kms }
-    }
-
-    pub fn generate_data_key(&self) -> ([u8; 32], [u8; 12]) {
-        let mut data_key = [0u8; 32];
-        let mut nonce = [0u8; 12];
-        rand::thread_rng().fill_bytes(&mut data_key);
-        rand::thread_rng().fill_bytes(&mut nonce);
-        (data_key, nonce)
-    }
-
-    pub fn wrap_data_key(&self, data_key: &[u8; 32]) -> Result<String, CryptoError> {
-        use aes_gcm::aead::Aead;
-        use aes_gcm::{Aes256Gcm, KeyInit, Nonce};
-
-        let cipher = Aes256Gcm::new((&self.master_key).into());
-        let mut nonce_bytes = [0u8; 12];
-        rand::thread_rng().fill_bytes(&mut nonce_bytes);
-        let nonce = Nonce::from_slice(&nonce_bytes);
-
-        let encrypted = cipher
-            .encrypt(nonce, data_key.as_slice())
-            .map_err(|e| CryptoError::EncryptionFailed(e.to_string()))?;
-
-        let mut combined = Vec::with_capacity(12 + encrypted.len());
-        combined.extend_from_slice(&nonce_bytes);
-        combined.extend_from_slice(&encrypted);
-        Ok(B64.encode(&combined))
-    }
-
-    pub fn unwrap_data_key(&self, wrapped_b64: &str) -> Result<[u8; 32], CryptoError> {
-        use aes_gcm::aead::Aead;
-        use aes_gcm::{Aes256Gcm, KeyInit, Nonce};
-
-        let combined = B64.decode(wrapped_b64).map_err(|e| {
-            CryptoError::EncryptionFailed(format!("Bad wrapped key encoding: {}", e))
-        })?;
-        if combined.len() < 12 {
-            return Err(CryptoError::EncryptionFailed(
-                "Wrapped key too short".to_string(),
-            ));
-        }
-
-        let (nonce_bytes, ciphertext) = combined.split_at(12);
-        let cipher = Aes256Gcm::new((&self.master_key).into());
-        let nonce = Nonce::from_slice(nonce_bytes);
-
-        let plaintext = cipher
-            .decrypt(nonce, ciphertext)
-            .map_err(|_| CryptoError::DecryptionFailed(0))?;
-
-        if plaintext.len() != 32 {
-            return Err(CryptoError::InvalidKeySize(plaintext.len()));
-        }
-        let mut key = [0u8; 32];
-        key.copy_from_slice(&plaintext);
-        Ok(key)
-    }
-
-    pub async fn encrypt_object(
-        &self,
-        input_path: &Path,
-        output_path: &Path,
-        ctx: &EncryptionContext,
-    ) -> Result<EncryptionMetadata, CryptoError> {
-        let (data_key, nonce) = self.generate_data_key();
-
-        let (encrypted_data_key, kms_key_id) = match ctx.algorithm {
-            SseAlgorithm::Aes256 => {
-                let wrapped = self.wrap_data_key(&data_key)?;
-                (Some(wrapped), None)
-            }
-            SseAlgorithm::AwsKms => {
-                let kms = self
-                    .kms
-                    .as_ref()
-                    .ok_or_else(|| CryptoError::EncryptionFailed("KMS not available".into()))?;
-                let kid = ctx
-                    .kms_key_id
-                    .as_ref()
-                    .ok_or_else(|| CryptoError::EncryptionFailed("No KMS key ID".into()))?;
-                let ciphertext = kms.encrypt_data(kid, &data_key).await?;
-                (Some(B64.encode(&ciphertext)), Some(kid.clone()))
-            }
-            SseAlgorithm::CustomerProvided => (None, None),
-        };
-
-        let actual_key = if ctx.algorithm == SseAlgorithm::CustomerProvided {
-            let ck = ctx
-                .customer_key
-                .as_ref()
-                .ok_or_else(|| CryptoError::EncryptionFailed("No customer key provided".into()))?;
-            if ck.len() != 32 {
-                return Err(CryptoError::InvalidKeySize(ck.len()));
-            }
-            let mut k = [0u8; 32];
-            k.copy_from_slice(ck);
-            k
-        } else {
-            data_key
-        };
-
-        let ip = input_path.to_owned();
-        let op = output_path.to_owned();
-        let ak = actual_key;
-        let n = nonce;
-        tokio::task::spawn_blocking(move || encrypt_stream_chunked(&ip, &op, &ak, &n, None))
-            .await
-            .map_err(|e| CryptoError::Io(std::io::Error::new(std::io::ErrorKind::Other, e)))??;
-
-        Ok(EncryptionMetadata {
-            algorithm: ctx.algorithm.as_str().to_string(),
-            nonce: B64.encode(nonce),
-            encrypted_data_key,
-            kms_key_id,
-        })
-    }
-
-    pub async fn decrypt_object(
-        &self,
-        input_path: &Path,
-        output_path: &Path,
-        enc_meta: &EncryptionMetadata,
-        customer_key: Option<&[u8]>,
-    ) -> Result<(), CryptoError> {
-        let nonce_bytes = B64
-            .decode(&enc_meta.nonce)
-            .map_err(|e| CryptoError::EncryptionFailed(format!("Bad nonce encoding: {}", e)))?;
-        if nonce_bytes.len() != 12 {
-            return Err(CryptoError::InvalidNonceSize(nonce_bytes.len()));
-        }
-
-        let data_key: [u8; 32] = if let Some(ck) = customer_key {
-            if ck.len() != 32 {
-                return Err(CryptoError::InvalidKeySize(ck.len()));
-            }
-            let mut k = [0u8; 32];
-            k.copy_from_slice(ck);
-            k
-        } else if enc_meta.algorithm == "aws:kms" {
-            let kms = self
-                .kms
-                .as_ref()
-                .ok_or_else(|| CryptoError::EncryptionFailed("KMS not available".into()))?;
-            let kid = enc_meta
-                .kms_key_id
-                .as_ref()
-                .ok_or_else(|| CryptoError::EncryptionFailed("No KMS key ID in metadata".into()))?;
-            let encrypted_dk = enc_meta.encrypted_data_key.as_ref().ok_or_else(|| {
-                CryptoError::EncryptionFailed("No encrypted data key in metadata".into())
-            })?;
-            let ct = B64.decode(encrypted_dk).map_err(|e| {
-                CryptoError::EncryptionFailed(format!("Bad data key encoding: {}", e))
-            })?;
-            let dk = kms.decrypt_data(kid, &ct).await?;
-            if dk.len() != 32 {
-                return Err(CryptoError::InvalidKeySize(dk.len()));
-            }
-            let mut k = [0u8; 32];
-            k.copy_from_slice(&dk);
-            k
-        } else {
-            let wrapped = enc_meta.encrypted_data_key.as_ref().ok_or_else(|| {
-                CryptoError::EncryptionFailed("No encrypted data key in metadata".into())
-            })?;
-            self.unwrap_data_key(wrapped)?
-        };
-
-        let ip = input_path.to_owned();
-        let op = output_path.to_owned();
-        let nb: [u8; 12] = nonce_bytes.try_into().unwrap();
-        tokio::task::spawn_blocking(move || decrypt_stream_chunked(&ip, &op, &data_key, &nb))
-            .await
-            .map_err(|e| CryptoError::Io(std::io::Error::new(std::io::ErrorKind::Other, e)))??;
-
-        Ok(())
-    }
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-    use std::io::Write;
-
-    fn test_master_key() -> [u8; 32] {
-        [0x42u8; 32]
-    }
-
-    #[test]
-    fn test_wrap_unwrap_data_key() {
-        let svc = EncryptionService::new(test_master_key(), None);
-        let dk = [0xAAu8; 32];
-        let wrapped = svc.wrap_data_key(&dk).unwrap();
-        let unwrapped = svc.unwrap_data_key(&wrapped).unwrap();
-        assert_eq!(dk, unwrapped);
-    }
-
-    #[tokio::test]
-    async fn test_encrypt_decrypt_object_sse_s3() {
-        let dir = tempfile::tempdir().unwrap();
-        let input = dir.path().join("plain.bin");
-        let encrypted = dir.path().join("enc.bin");
-        let decrypted = dir.path().join("dec.bin");
-
-        let data = b"SSE-S3 encrypted content for testing!";
-        std::fs::File::create(&input)
-            .unwrap()
-            .write_all(data)
-            .unwrap();
-
-        let svc = EncryptionService::new(test_master_key(), None);
-
-        let ctx = EncryptionContext {
-            algorithm: SseAlgorithm::Aes256,
-            kms_key_id: None,
-            customer_key: None,
-        };
-
-        let meta = svc.encrypt_object(&input, &encrypted, &ctx).await.unwrap();
-        assert_eq!(meta.algorithm, "AES256");
-        assert!(meta.encrypted_data_key.is_some());
-
-        svc.decrypt_object(&encrypted, &decrypted, &meta, None)
-            .await
-            .unwrap();
-
-        let result = std::fs::read(&decrypted).unwrap();
-        assert_eq!(result, data);
-    }
-
-    #[tokio::test]
-    async fn test_encrypt_decrypt_object_sse_c() {
-        let dir = tempfile::tempdir().unwrap();
-        let input = dir.path().join("plain.bin");
-        let encrypted = dir.path().join("enc.bin");
-        let decrypted = dir.path().join("dec.bin");
-
-        let data = b"SSE-C encrypted content!";
-        std::fs::File::create(&input)
-            .unwrap()
-            .write_all(data)
-            .unwrap();
-
-        let customer_key = [0xBBu8; 32];
-        let svc = EncryptionService::new(test_master_key(), None);
-
-        let ctx = EncryptionContext {
-            algorithm: SseAlgorithm::CustomerProvided,
-            kms_key_id: None,
-            customer_key: Some(customer_key.to_vec()),
-        };
-
-        let meta = svc.encrypt_object(&input, &encrypted, &ctx).await.unwrap();
-        assert!(meta.encrypted_data_key.is_none());
-
-        svc.decrypt_object(&encrypted, &decrypted, &meta, Some(&customer_key))
-            .await
-            .unwrap();
-
-        let result = std::fs::read(&decrypted).unwrap();
-        assert_eq!(result, data);
-    }
-
-    #[test]
-    fn test_encryption_metadata_roundtrip() {
-        let meta = EncryptionMetadata {
-            algorithm: "AES256".to_string(),
-            nonce: "dGVzdG5vbmNlMTI=".to_string(),
-            encrypted_data_key: Some("c29tZWtleQ==".to_string()),
-            kms_key_id: None,
-        };
-        let map = meta.to_metadata_map();
-        let restored = EncryptionMetadata::from_metadata(&map).unwrap();
-        assert_eq!(restored.algorithm, "AES256");
-        assert_eq!(restored.nonce, meta.nonce);
-        assert_eq!(restored.encrypted_data_key, meta.encrypted_data_key);
-    }
-
-    #[test]
-    fn test_is_encrypted() {
-        let mut meta = HashMap::new();
-        assert!(!EncryptionMetadata::is_encrypted(&meta));
-        meta.insert(
-            "x-amz-server-side-encryption".to_string(),
-            "AES256".to_string(),
-        );
-        assert!(EncryptionMetadata::is_encrypted(&meta));
-    }
-}

rust/myfsio-engine/crates/myfsio-crypto/src/hashing.rs

@@ -1,138 +0,0 @@
-use md5::{Digest, Md5};
-use sha2::Sha256;
-use std::io::Read;
-use std::path::Path;
-
-const CHUNK_SIZE: usize = 65536;
-
-pub fn md5_file(path: &Path) -> std::io::Result<String> {
-    let mut file = std::fs::File::open(path)?;
-    let mut hasher = Md5::new();
-    let mut buf = vec![0u8; CHUNK_SIZE];
-    loop {
-        let n = file.read(&mut buf)?;
-        if n == 0 {
-            break;
-        }
-        hasher.update(&buf[..n]);
-    }
-    Ok(format!("{:x}", hasher.finalize()))
-}
-
-pub fn md5_bytes(data: &[u8]) -> String {
-    let mut hasher = Md5::new();
-    hasher.update(data);
-    format!("{:x}", hasher.finalize())
-}
-
-pub fn sha256_file(path: &Path) -> std::io::Result<String> {
-    let mut file = std::fs::File::open(path)?;
-    let mut hasher = Sha256::new();
-    let mut buf = vec![0u8; CHUNK_SIZE];
-    loop {
-        let n = file.read(&mut buf)?;
-        if n == 0 {
-            break;
-        }
-        hasher.update(&buf[..n]);
-    }
-    Ok(format!("{:x}", hasher.finalize()))
-}
-
-pub fn sha256_bytes(data: &[u8]) -> String {
-    let mut hasher = Sha256::new();
-    hasher.update(data);
-    format!("{:x}", hasher.finalize())
-}
-
-pub fn md5_sha256_file(path: &Path) -> std::io::Result<(String, String)> {
-    let mut file = std::fs::File::open(path)?;
-    let mut md5_hasher = Md5::new();
-    let mut sha_hasher = Sha256::new();
-    let mut buf = vec![0u8; CHUNK_SIZE];
-    loop {
-        let n = file.read(&mut buf)?;
-        if n == 0 {
-            break;
-        }
-        md5_hasher.update(&buf[..n]);
-        sha_hasher.update(&buf[..n]);
-    }
-    Ok((
-        format!("{:x}", md5_hasher.finalize()),
-        format!("{:x}", sha_hasher.finalize()),
-    ))
-}
-
-pub async fn md5_file_async(path: &Path) -> std::io::Result<String> {
-    let path = path.to_owned();
-    tokio::task::spawn_blocking(move || md5_file(&path))
-        .await
-        .map_err(|e| std::io::Error::new(std::io::ErrorKind::Other, e))?
-}
-
-pub async fn sha256_file_async(path: &Path) -> std::io::Result<String> {
-    let path = path.to_owned();
-    tokio::task::spawn_blocking(move || sha256_file(&path))
-        .await
-        .map_err(|e| std::io::Error::new(std::io::ErrorKind::Other, e))?
-}
-
-pub async fn md5_sha256_file_async(path: &Path) -> std::io::Result<(String, String)> {
-    let path = path.to_owned();
-    tokio::task::spawn_blocking(move || md5_sha256_file(&path))
-        .await
-        .map_err(|e| std::io::Error::new(std::io::ErrorKind::Other, e))?
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-    use std::io::Write;
-
-    #[test]
-    fn test_md5_bytes() {
-        assert_eq!(md5_bytes(b""), "d41d8cd98f00b204e9800998ecf8427e");
-        assert_eq!(md5_bytes(b"hello"), "5d41402abc4b2a76b9719d911017c592");
-    }
-
-    #[test]
-    fn test_sha256_bytes() {
-        let hash = sha256_bytes(b"hello");
-        assert_eq!(
-            hash,
-            "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"
-        );
-    }
-
-    #[test]
-    fn test_md5_file() {
-        let mut tmp = tempfile::NamedTempFile::new().unwrap();
-        tmp.write_all(b"hello").unwrap();
-        tmp.flush().unwrap();
-        let hash = md5_file(tmp.path()).unwrap();
-        assert_eq!(hash, "5d41402abc4b2a76b9719d911017c592");
-    }
-
-    #[test]
-    fn test_md5_sha256_file() {
-        let mut tmp = tempfile::NamedTempFile::new().unwrap();
-        tmp.write_all(b"hello").unwrap();
-        tmp.flush().unwrap();
-        let (md5, sha) = md5_sha256_file(tmp.path()).unwrap();
-        assert_eq!(md5, "5d41402abc4b2a76b9719d911017c592");
-        assert_eq!(
-            sha,
-            "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"
-        );
-    }
-
-    #[tokio::test]
-    async fn test_md5_file_async() {
-        let mut tmp = tempfile::NamedTempFile::new().unwrap();
-        tmp.write_all(b"hello").unwrap();
-        tmp.flush().unwrap();
-        let hash = md5_file_async(tmp.path()).await.unwrap();
-        assert_eq!(hash, "5d41402abc4b2a76b9719d911017c592");
-    }
-}

rust/myfsio-engine/crates/myfsio-crypto/src/kms.rs

@@ -1,451 +0,0 @@
-use aes_gcm::aead::Aead;
-use aes_gcm::{Aes256Gcm, KeyInit, Nonce};
-use base64::engine::general_purpose::STANDARD as B64;
-use base64::Engine;
-use chrono::{DateTime, Utc};
-use rand::RngCore;
-use serde::{Deserialize, Serialize};
-use std::path::{Path, PathBuf};
-use std::sync::Arc;
-use tokio::sync::RwLock;
-
-use crate::aes_gcm::CryptoError;
-
-#[derive(Debug, Clone, Serialize, Deserialize)]
-pub struct KmsKey {
-    #[serde(rename = "KeyId")]
-    pub key_id: String,
-    #[serde(rename = "Arn")]
-    pub arn: String,
-    #[serde(rename = "Description")]
-    pub description: String,
-    #[serde(rename = "CreationDate")]
-    pub creation_date: DateTime<Utc>,
-    #[serde(rename = "Enabled")]
-    pub enabled: bool,
-    #[serde(rename = "KeyState")]
-    pub key_state: String,
-    #[serde(rename = "KeyUsage")]
-    pub key_usage: String,
-    #[serde(rename = "KeySpec")]
-    pub key_spec: String,
-    #[serde(rename = "EncryptedKeyMaterial")]
-    pub encrypted_key_material: String,
-}
-
-#[derive(Debug, Clone, Serialize, Deserialize)]
-struct KmsStore {
-    keys: Vec<KmsKey>,
-}
-
-pub struct KmsService {
-    keys_path: PathBuf,
-    master_key: Arc<RwLock<[u8; 32]>>,
-    keys: Arc<RwLock<Vec<KmsKey>>>,
-}
-
-impl KmsService {
-    pub async fn new(keys_dir: &Path) -> Result<Self, CryptoError> {
-        std::fs::create_dir_all(keys_dir).map_err(CryptoError::Io)?;
-
-        let keys_path = keys_dir.join("kms_keys.json");
-
-        let master_key = Self::load_or_create_master_key(&keys_dir.join("kms_master.key"))?;
-
-        let keys = if keys_path.exists() {
-            let data = std::fs::read_to_string(&keys_path).map_err(CryptoError::Io)?;
-            let store: KmsStore = serde_json::from_str(&data)
-                .map_err(|e| CryptoError::EncryptionFailed(format!("Bad KMS store: {}", e)))?;
-            store.keys
-        } else {
-            Vec::new()
-        };
-
-        Ok(Self {
-            keys_path,
-            master_key: Arc::new(RwLock::new(master_key)),
-            keys: Arc::new(RwLock::new(keys)),
-        })
-    }
-
-    fn load_or_create_master_key(path: &Path) -> Result<[u8; 32], CryptoError> {
-        if path.exists() {
-            let encoded = std::fs::read_to_string(path).map_err(CryptoError::Io)?;
-            let decoded = B64.decode(encoded.trim()).map_err(|e| {
-                CryptoError::EncryptionFailed(format!("Bad master key encoding: {}", e))
-            })?;
-            if decoded.len() != 32 {
-                return Err(CryptoError::InvalidKeySize(decoded.len()));
-            }
-            let mut key = [0u8; 32];
-            key.copy_from_slice(&decoded);
-            Ok(key)
-        } else {
-            let mut key = [0u8; 32];
-            rand::thread_rng().fill_bytes(&mut key);
-            let encoded = B64.encode(key);
-            std::fs::write(path, &encoded).map_err(CryptoError::Io)?;
-            Ok(key)
-        }
-    }
-
-    fn encrypt_key_material(
-        master_key: &[u8; 32],
-        plaintext_key: &[u8],
-    ) -> Result<String, CryptoError> {
-        let cipher = Aes256Gcm::new(master_key.into());
-        let mut nonce_bytes = [0u8; 12];
-        rand::thread_rng().fill_bytes(&mut nonce_bytes);
-        let nonce = Nonce::from_slice(&nonce_bytes);
-
-        let ciphertext = cipher
-            .encrypt(nonce, plaintext_key)
-            .map_err(|e| CryptoError::EncryptionFailed(e.to_string()))?;
-
-        let mut combined = Vec::with_capacity(12 + ciphertext.len());
-        combined.extend_from_slice(&nonce_bytes);
-        combined.extend_from_slice(&ciphertext);
-        Ok(B64.encode(&combined))
-    }
-
-    fn decrypt_key_material(
-        master_key: &[u8; 32],
-        encrypted_b64: &str,
-    ) -> Result<Vec<u8>, CryptoError> {
-        let combined = B64.decode(encrypted_b64).map_err(|e| {
-            CryptoError::EncryptionFailed(format!("Bad key material encoding: {}", e))
-        })?;
-        if combined.len() < 12 {
-            return Err(CryptoError::EncryptionFailed(
-                "Encrypted key material too short".to_string(),
-            ));
-        }
-
-        let (nonce_bytes, ciphertext) = combined.split_at(12);
-        let cipher = Aes256Gcm::new(master_key.into());
-        let nonce = Nonce::from_slice(nonce_bytes);
-
-        cipher
-            .decrypt(nonce, ciphertext)
-            .map_err(|_| CryptoError::DecryptionFailed(0))
-    }
-
-    async fn save(&self) -> Result<(), CryptoError> {
-        let keys = self.keys.read().await;
-        let store = KmsStore { keys: keys.clone() };
-        let json = serde_json::to_string_pretty(&store)
-            .map_err(|e| CryptoError::EncryptionFailed(e.to_string()))?;
-        std::fs::write(&self.keys_path, json).map_err(CryptoError::Io)?;
-        Ok(())
-    }
-
-    pub async fn create_key(&self, description: &str) -> Result<KmsKey, CryptoError> {
-        let key_id = uuid::Uuid::new_v4().to_string();
-        let arn = format!("arn:aws:kms:local:000000000000:key/{}", key_id);
-
-        let mut plaintext_key = [0u8; 32];
-        rand::thread_rng().fill_bytes(&mut plaintext_key);
-
-        let master = self.master_key.read().await;
-        let encrypted = Self::encrypt_key_material(&master, &plaintext_key)?;
-
-        let kms_key = KmsKey {
-            key_id: key_id.clone(),
-            arn,
-            description: description.to_string(),
-            creation_date: Utc::now(),
-            enabled: true,
-            key_state: "Enabled".to_string(),
-            key_usage: "ENCRYPT_DECRYPT".to_string(),
-            key_spec: "SYMMETRIC_DEFAULT".to_string(),
-            encrypted_key_material: encrypted,
-        };
-
-        self.keys.write().await.push(kms_key.clone());
-        self.save().await?;
-        Ok(kms_key)
-    }
-
-    pub async fn list_keys(&self) -> Vec<KmsKey> {
-        self.keys.read().await.clone()
-    }
-
-    pub async fn get_key(&self, key_id: &str) -> Option<KmsKey> {
-        let keys = self.keys.read().await;
-        keys.iter()
-            .find(|k| k.key_id == key_id || k.arn == key_id)
-            .cloned()
-    }
-
-    pub async fn delete_key(&self, key_id: &str) -> Result<bool, CryptoError> {
-        let mut keys = self.keys.write().await;
-        let len_before = keys.len();
-        keys.retain(|k| k.key_id != key_id && k.arn != key_id);
-        let removed = keys.len() < len_before;
-        drop(keys);
-        if removed {
-            self.save().await?;
-        }
-        Ok(removed)
-    }
-
-    pub async fn enable_key(&self, key_id: &str) -> Result<bool, CryptoError> {
-        let mut keys = self.keys.write().await;
-        if let Some(key) = keys.iter_mut().find(|k| k.key_id == key_id) {
-            key.enabled = true;
-            key.key_state = "Enabled".to_string();
-            drop(keys);
-            self.save().await?;
-            Ok(true)
-        } else {
-            Ok(false)
-        }
-    }
-
-    pub async fn disable_key(&self, key_id: &str) -> Result<bool, CryptoError> {
-        let mut keys = self.keys.write().await;
-        if let Some(key) = keys.iter_mut().find(|k| k.key_id == key_id) {
-            key.enabled = false;
-            key.key_state = "Disabled".to_string();
-            drop(keys);
-            self.save().await?;
-            Ok(true)
-        } else {
-            Ok(false)
-        }
-    }
-
-    pub async fn decrypt_data_key(&self, key_id: &str) -> Result<Vec<u8>, CryptoError> {
-        let keys = self.keys.read().await;
-        let key = keys
-            .iter()
-            .find(|k| k.key_id == key_id || k.arn == key_id)
-            .ok_or_else(|| CryptoError::EncryptionFailed("KMS key not found".to_string()))?;
-
-        if !key.enabled {
-            return Err(CryptoError::EncryptionFailed(
-                "KMS key is disabled".to_string(),
-            ));
-        }
-
-        let master = self.master_key.read().await;
-        Self::decrypt_key_material(&master, &key.encrypted_key_material)
-    }
-
-    pub async fn encrypt_data(
-        &self,
-        key_id: &str,
-        plaintext: &[u8],
-    ) -> Result<Vec<u8>, CryptoError> {
-        let data_key = self.decrypt_data_key(key_id).await?;
-        if data_key.len() != 32 {
-            return Err(CryptoError::InvalidKeySize(data_key.len()));
-        }
-
-        let key_arr: [u8; 32] = data_key.try_into().unwrap();
-        let cipher = Aes256Gcm::new(&key_arr.into());
-        let mut nonce_bytes = [0u8; 12];
-        rand::thread_rng().fill_bytes(&mut nonce_bytes);
-        let nonce = Nonce::from_slice(&nonce_bytes);
-
-        let ciphertext = cipher
-            .encrypt(nonce, plaintext)
-            .map_err(|e| CryptoError::EncryptionFailed(e.to_string()))?;
-
-        let mut result = Vec::with_capacity(12 + ciphertext.len());
-        result.extend_from_slice(&nonce_bytes);
-        result.extend_from_slice(&ciphertext);
-        Ok(result)
-    }
-
-    pub async fn decrypt_data(
-        &self,
-        key_id: &str,
-        ciphertext: &[u8],
-    ) -> Result<Vec<u8>, CryptoError> {
-        if ciphertext.len() < 12 {
-            return Err(CryptoError::EncryptionFailed(
-                "Ciphertext too short".to_string(),
-            ));
-        }
-
-        let data_key = self.decrypt_data_key(key_id).await?;
-        if data_key.len() != 32 {
-            return Err(CryptoError::InvalidKeySize(data_key.len()));
-        }
-
-        let key_arr: [u8; 32] = data_key.try_into().unwrap();
-        let (nonce_bytes, ct) = ciphertext.split_at(12);
-        let cipher = Aes256Gcm::new(&key_arr.into());
-        let nonce = Nonce::from_slice(nonce_bytes);
-
-        cipher
-            .decrypt(nonce, ct)
-            .map_err(|_| CryptoError::DecryptionFailed(0))
-    }
-
-    pub async fn generate_data_key(
-        &self,
-        key_id: &str,
-        num_bytes: usize,
-    ) -> Result<(Vec<u8>, Vec<u8>), CryptoError> {
-        let kms_key = self.decrypt_data_key(key_id).await?;
-        if kms_key.len() != 32 {
-            return Err(CryptoError::InvalidKeySize(kms_key.len()));
-        }
-
-        let mut plaintext_key = vec![0u8; num_bytes];
-        rand::thread_rng().fill_bytes(&mut plaintext_key);
-
-        let key_arr: [u8; 32] = kms_key.try_into().unwrap();
-        let cipher = Aes256Gcm::new(&key_arr.into());
-        let mut nonce_bytes = [0u8; 12];
-        rand::thread_rng().fill_bytes(&mut nonce_bytes);
-        let nonce = Nonce::from_slice(&nonce_bytes);
-
-        let encrypted = cipher
-            .encrypt(nonce, plaintext_key.as_slice())
-            .map_err(|e| CryptoError::EncryptionFailed(e.to_string()))?;
-
-        let mut wrapped = Vec::with_capacity(12 + encrypted.len());
-        wrapped.extend_from_slice(&nonce_bytes);
-        wrapped.extend_from_slice(&encrypted);
-
-        Ok((plaintext_key, wrapped))
-    }
-}
-
-pub async fn load_or_create_master_key(keys_dir: &Path) -> Result<[u8; 32], CryptoError> {
-    std::fs::create_dir_all(keys_dir).map_err(CryptoError::Io)?;
-    let path = keys_dir.join("master.key");
-
-    if path.exists() {
-        let encoded = std::fs::read_to_string(&path).map_err(CryptoError::Io)?;
-        let decoded = B64.decode(encoded.trim()).map_err(|e| {
-            CryptoError::EncryptionFailed(format!("Bad master key encoding: {}", e))
-        })?;
-        if decoded.len() != 32 {
-            return Err(CryptoError::InvalidKeySize(decoded.len()));
-        }
-        let mut key = [0u8; 32];
-        key.copy_from_slice(&decoded);
-        Ok(key)
-    } else {
-        let mut key = [0u8; 32];
-        rand::thread_rng().fill_bytes(&mut key);
-        let encoded = B64.encode(key);
-        std::fs::write(&path, &encoded).map_err(CryptoError::Io)?;
-        Ok(key)
-    }
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-
-    #[tokio::test]
-    async fn test_create_and_list_keys() {
-        let dir = tempfile::tempdir().unwrap();
-        let kms = KmsService::new(dir.path()).await.unwrap();
-
-        let key = kms.create_key("test key").await.unwrap();
-        assert!(key.enabled);
-        assert_eq!(key.description, "test key");
-        assert!(key.key_id.len() > 0);
-
-        let keys = kms.list_keys().await;
-        assert_eq!(keys.len(), 1);
-        assert_eq!(keys[0].key_id, key.key_id);
-    }
-
-    #[tokio::test]
-    async fn test_enable_disable_key() {
-        let dir = tempfile::tempdir().unwrap();
-        let kms = KmsService::new(dir.path()).await.unwrap();
-
-        let key = kms.create_key("toggle").await.unwrap();
-        assert!(key.enabled);
-
-        kms.disable_key(&key.key_id).await.unwrap();
-        let k = kms.get_key(&key.key_id).await.unwrap();
-        assert!(!k.enabled);
-
-        kms.enable_key(&key.key_id).await.unwrap();
-        let k = kms.get_key(&key.key_id).await.unwrap();
-        assert!(k.enabled);
-    }
-
-    #[tokio::test]
-    async fn test_delete_key() {
-        let dir = tempfile::tempdir().unwrap();
-        let kms = KmsService::new(dir.path()).await.unwrap();
-
-        let key = kms.create_key("doomed").await.unwrap();
-        assert!(kms.delete_key(&key.key_id).await.unwrap());
-        assert!(kms.get_key(&key.key_id).await.is_none());
-        assert_eq!(kms.list_keys().await.len(), 0);
-    }
-
-    #[tokio::test]
-    async fn test_encrypt_decrypt_data() {
-        let dir = tempfile::tempdir().unwrap();
-        let kms = KmsService::new(dir.path()).await.unwrap();
-
-        let key = kms.create_key("enc-key").await.unwrap();
-        let plaintext = b"Hello, KMS!";
-
-        let ciphertext = kms.encrypt_data(&key.key_id, plaintext).await.unwrap();
-        assert_ne!(&ciphertext, plaintext);
-
-        let decrypted = kms.decrypt_data(&key.key_id, &ciphertext).await.unwrap();
-        assert_eq!(decrypted, plaintext);
-    }
-
-    #[tokio::test]
-    async fn test_generate_data_key() {
-        let dir = tempfile::tempdir().unwrap();
-        let kms = KmsService::new(dir.path()).await.unwrap();
-
-        let key = kms.create_key("data-key-gen").await.unwrap();
-        let (plaintext, wrapped) = kms.generate_data_key(&key.key_id, 32).await.unwrap();
-
-        assert_eq!(plaintext.len(), 32);
-        assert!(wrapped.len() > 32);
-    }
-
-    #[tokio::test]
-    async fn test_disabled_key_cannot_encrypt() {
-        let dir = tempfile::tempdir().unwrap();
-        let kms = KmsService::new(dir.path()).await.unwrap();
-
-        let key = kms.create_key("disabled").await.unwrap();
-        kms.disable_key(&key.key_id).await.unwrap();
-
-        let result = kms.encrypt_data(&key.key_id, b"test").await;
-        assert!(result.is_err());
-    }
-
-    #[tokio::test]
-    async fn test_persistence_across_reload() {
-        let dir = tempfile::tempdir().unwrap();
-
-        let key_id = {
-            let kms = KmsService::new(dir.path()).await.unwrap();
-            let key = kms.create_key("persistent").await.unwrap();
-            key.key_id
-        };
-
-        let kms2 = KmsService::new(dir.path()).await.unwrap();
-        let key = kms2.get_key(&key_id).await;
-        assert!(key.is_some());
-        assert_eq!(key.unwrap().description, "persistent");
-    }
-
-    #[tokio::test]
-    async fn test_master_key_roundtrip() {
-        let dir = tempfile::tempdir().unwrap();
-        let key1 = load_or_create_master_key(dir.path()).await.unwrap();
-        let key2 = load_or_create_master_key(dir.path()).await.unwrap();
-        assert_eq!(key1, key2);
-    }
-}

rust/myfsio-engine/crates/myfsio-crypto/src/lib.rs

@@ -1,4 +0,0 @@
-pub mod aes_gcm;
-pub mod encryption;
-pub mod hashing;
-pub mod kms;

rust/myfsio-engine/crates/myfsio-server/Cargo.toml

@@ -1,56 +0,0 @@
-[package]
-name = "myfsio-server"
-version.workspace = true
-edition.workspace = true
-
-[dependencies]
-myfsio-common = { path = "../myfsio-common" }
-myfsio-auth = { path = "../myfsio-auth" }
-myfsio-crypto = { path = "../myfsio-crypto" }
-myfsio-storage = { path = "../myfsio-storage" }
-myfsio-xml = { path = "../myfsio-xml" }
-base64 = { workspace = true }
-md-5 = { workspace = true }
-axum = { workspace = true }
-tokio = { workspace = true }
-tower = { workspace = true }
-tower-http = { workspace = true }
-hyper = { workspace = true }
-bytes = { workspace = true }
-serde = { workspace = true }
-serde_json = { workspace = true }
-tracing = { workspace = true }
-tracing-subscriber = { workspace = true }
-tokio-util = { workspace = true }
-chrono = { workspace = true }
-uuid = { workspace = true }
-futures = { workspace = true }
-http-body-util = "0.1"
-percent-encoding = { workspace = true }
-quick-xml = { workspace = true }
-mime_guess = "2"
-crc32fast = { workspace = true }
-sha2 = { workspace = true }
-duckdb = { workspace = true }
-roxmltree = "0.20"
-parking_lot = { workspace = true }
-regex = "1"
-multer = "3"
-reqwest = { workspace = true }
-aws-sdk-s3 = { workspace = true }
-aws-config = { workspace = true }
-aws-credential-types = { workspace = true }
-aws-smithy-types = { workspace = true }
-async-trait = { workspace = true }
-rand = "0.8"
-tera = { workspace = true }
-cookie = { workspace = true }
-subtle = { workspace = true }
-clap = { workspace = true }
-dotenvy = { workspace = true }
-sysinfo = "0.32"
-aes-gcm = { workspace = true }
-
-[dev-dependencies]
-tempfile = "3"
-tower = { workspace = true, features = ["util"] }

rust/myfsio-engine/crates/myfsio-server/src/config.rs

@@ -1,227 +0,0 @@
-use std::net::SocketAddr;
-use std::path::PathBuf;
-
-#[derive(Debug, Clone)]
-pub struct ServerConfig {
-    pub bind_addr: SocketAddr,
-    pub ui_bind_addr: SocketAddr,
-    pub storage_root: PathBuf,
-    pub region: String,
-    pub iam_config_path: PathBuf,
-    pub sigv4_timestamp_tolerance_secs: u64,
-    pub presigned_url_min_expiry: u64,
-    pub presigned_url_max_expiry: u64,
-    pub secret_key: Option<String>,
-    pub encryption_enabled: bool,
-    pub kms_enabled: bool,
-    pub gc_enabled: bool,
-    pub integrity_enabled: bool,
-    pub metrics_enabled: bool,
-    pub metrics_history_enabled: bool,
-    pub metrics_interval_minutes: u64,
-    pub metrics_retention_hours: u64,
-    pub metrics_history_interval_minutes: u64,
-    pub metrics_history_retention_hours: u64,
-    pub lifecycle_enabled: bool,
-    pub website_hosting_enabled: bool,
-    pub replication_connect_timeout_secs: u64,
-    pub replication_read_timeout_secs: u64,
-    pub replication_max_retries: u32,
-    pub replication_streaming_threshold_bytes: u64,
-    pub replication_max_failures_per_bucket: usize,
-    pub site_sync_enabled: bool,
-    pub site_sync_interval_secs: u64,
-    pub site_sync_batch_size: usize,
-    pub site_sync_connect_timeout_secs: u64,
-    pub site_sync_read_timeout_secs: u64,
-    pub site_sync_max_retries: u32,
-    pub site_sync_clock_skew_tolerance: f64,
-    pub ui_enabled: bool,
-    pub templates_dir: PathBuf,
-    pub static_dir: PathBuf,
-}
-
-impl ServerConfig {
-    pub fn from_env() -> Self {
-        let host = std::env::var("HOST").unwrap_or_else(|_| "127.0.0.1".to_string());
-        let port: u16 = std::env::var("PORT")
-            .unwrap_or_else(|_| "5000".to_string())
-            .parse()
-            .unwrap_or(5000);
-        let ui_port: u16 = std::env::var("UI_PORT")
-            .unwrap_or_else(|_| "5100".to_string())
-            .parse()
-            .unwrap_or(5100);
-        let storage_root = std::env::var("STORAGE_ROOT").unwrap_or_else(|_| "./data".to_string());
-        let region = std::env::var("AWS_REGION").unwrap_or_else(|_| "us-east-1".to_string());
-
-        let storage_path = PathBuf::from(&storage_root);
-        let iam_config_path = std::env::var("IAM_CONFIG")
-            .map(PathBuf::from)
-            .unwrap_or_else(|_| {
-                storage_path
-                    .join(".myfsio.sys")
-                    .join("config")
-                    .join("iam.json")
-            });
-
-        let sigv4_timestamp_tolerance_secs: u64 =
-            std::env::var("SIGV4_TIMESTAMP_TOLERANCE_SECONDS")
-                .unwrap_or_else(|_| "900".to_string())
-                .parse()
-                .unwrap_or(900);
-
-        let presigned_url_min_expiry: u64 = std::env::var("PRESIGNED_URL_MIN_EXPIRY_SECONDS")
-            .unwrap_or_else(|_| "1".to_string())
-            .parse()
-            .unwrap_or(1);
-
-        let presigned_url_max_expiry: u64 = std::env::var("PRESIGNED_URL_MAX_EXPIRY_SECONDS")
-            .unwrap_or_else(|_| "604800".to_string())
-            .parse()
-            .unwrap_or(604800);
-
-        let secret_key = {
-            let env_key = std::env::var("SECRET_KEY").ok();
-            match env_key {
-                Some(k) if !k.is_empty() && k != "dev-secret-key" => Some(k),
-                _ => {
-                    let secret_file = storage_path
-                        .join(".myfsio.sys")
-                        .join("config")
-                        .join(".secret");
-                    std::fs::read_to_string(&secret_file)
-                        .ok()
-                        .map(|s| s.trim().to_string())
-                }
-            }
-        };
-
-        let encryption_enabled = parse_bool_env("ENCRYPTION_ENABLED", false);
-
-        let kms_enabled = parse_bool_env("KMS_ENABLED", false);
-
-        let gc_enabled = parse_bool_env("GC_ENABLED", false);
-
-        let integrity_enabled = parse_bool_env("INTEGRITY_ENABLED", false);
-
-        let metrics_enabled = parse_bool_env("OPERATION_METRICS_ENABLED", false);
-
-        let metrics_history_enabled = parse_bool_env("METRICS_HISTORY_ENABLED", false);
-
-        let metrics_interval_minutes = parse_u64_env("OPERATION_METRICS_INTERVAL_MINUTES", 5);
-        let metrics_retention_hours = parse_u64_env("OPERATION_METRICS_RETENTION_HOURS", 24);
-        let metrics_history_interval_minutes = parse_u64_env("METRICS_HISTORY_INTERVAL_MINUTES", 5);
-        let metrics_history_retention_hours = parse_u64_env("METRICS_HISTORY_RETENTION_HOURS", 24);
-
-        let lifecycle_enabled = parse_bool_env("LIFECYCLE_ENABLED", false);
-
-        let website_hosting_enabled = parse_bool_env("WEBSITE_HOSTING_ENABLED", false);
-
-        let replication_connect_timeout_secs =
-            parse_u64_env("REPLICATION_CONNECT_TIMEOUT_SECONDS", 5);
-        let replication_read_timeout_secs = parse_u64_env("REPLICATION_READ_TIMEOUT_SECONDS", 30);
-        let replication_max_retries = parse_u64_env("REPLICATION_MAX_RETRIES", 2) as u32;
-        let replication_streaming_threshold_bytes =
-            parse_u64_env("REPLICATION_STREAMING_THRESHOLD_BYTES", 10_485_760);
-        let replication_max_failures_per_bucket =
-            parse_u64_env("REPLICATION_MAX_FAILURES_PER_BUCKET", 50) as usize;
-
-        let site_sync_enabled = parse_bool_env("SITE_SYNC_ENABLED", false);
-        let site_sync_interval_secs = parse_u64_env("SITE_SYNC_INTERVAL_SECONDS", 60);
-        let site_sync_batch_size = parse_u64_env("SITE_SYNC_BATCH_SIZE", 100) as usize;
-        let site_sync_connect_timeout_secs = parse_u64_env("SITE_SYNC_CONNECT_TIMEOUT_SECONDS", 10);
-        let site_sync_read_timeout_secs = parse_u64_env("SITE_SYNC_READ_TIMEOUT_SECONDS", 120);
-        let site_sync_max_retries = parse_u64_env("SITE_SYNC_MAX_RETRIES", 2) as u32;
-        let site_sync_clock_skew_tolerance: f64 =
-            std::env::var("SITE_SYNC_CLOCK_SKEW_TOLERANCE_SECONDS")
-                .ok()
-                .and_then(|s| s.parse().ok())
-                .unwrap_or(1.0);
-
-        let ui_enabled = parse_bool_env("UI_ENABLED", true);
-        let templates_dir = std::env::var("TEMPLATES_DIR")
-            .map(PathBuf::from)
-            .unwrap_or_else(|_| default_templates_dir());
-        let static_dir = std::env::var("STATIC_DIR")
-            .map(PathBuf::from)
-            .unwrap_or_else(|_| default_static_dir());
-
-        let host_ip: std::net::IpAddr = host.parse().unwrap();
-        Self {
-            bind_addr: SocketAddr::new(host_ip, port),
-            ui_bind_addr: SocketAddr::new(host_ip, ui_port),
-            storage_root: storage_path,
-            region,
-            iam_config_path,
-            sigv4_timestamp_tolerance_secs,
-            presigned_url_min_expiry,
-            presigned_url_max_expiry,
-            secret_key,
-            encryption_enabled,
-            kms_enabled,
-            gc_enabled,
-            integrity_enabled,
-            metrics_enabled,
-            metrics_history_enabled,
-            metrics_interval_minutes,
-            metrics_retention_hours,
-            metrics_history_interval_minutes,
-            metrics_history_retention_hours,
-            lifecycle_enabled,
-            website_hosting_enabled,
-            replication_connect_timeout_secs,
-            replication_read_timeout_secs,
-            replication_max_retries,
-            replication_streaming_threshold_bytes,
-            replication_max_failures_per_bucket,
-            site_sync_enabled,
-            site_sync_interval_secs,
-            site_sync_batch_size,
-            site_sync_connect_timeout_secs,
-            site_sync_read_timeout_secs,
-            site_sync_max_retries,
-            site_sync_clock_skew_tolerance,
-            ui_enabled,
-            templates_dir,
-            static_dir,
-        }
-    }
-}
-
-fn default_templates_dir() -> PathBuf {
-    let manifest_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
-    manifest_dir.join("templates")
-}
-
-fn default_static_dir() -> PathBuf {
-    let manifest_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
-    for candidate in [
-        manifest_dir.join("static"),
-        manifest_dir.join("..").join("..").join("..").join("static"),
-    ] {
-        if candidate.exists() {
-            return candidate;
-        }
-    }
-    manifest_dir.join("static")
-}
-
-fn parse_u64_env(key: &str, default: u64) -> u64 {
-    std::env::var(key)
-        .ok()
-        .and_then(|s| s.parse().ok())
-        .unwrap_or(default)
-}
-
-fn parse_bool_env(key: &str, default: bool) -> bool {
-    std::env::var(key)
-        .ok()
-        .map(|value| {
-            matches!(
-                value.trim().to_ascii_lowercase().as_str(),
-                "1" | "true" | "yes" | "on"
-            )
-        })
-        .unwrap_or(default)
-}