Skip to main content

SSH Key Sync Security Boundary

Related Issues: #2182, #2328

Security Guarantee​

Open ACE never propagates root private keys to workspace users.

When secure SSH sync is unavailable or fails, the system:

  1. Skips all file sync (it never falls back to the legacy sync logic)
  2. Writes a structured security log to /var/log/openace/ssh-sync-failure.json
  3. Creates a deployment alert file /var/log/openace/ssh-sync-failure.warning
  4. Returns a non-zero exit status to signal failure

Under no circumstances does the system fall back to an implementation that copies root private keys. This is the core of the fail-closed security principle.

Overview​

Open ACE implements a secure SSH key synchronization mechanism that prevents private keys of the platform control plane from being propagated to workspace users.

Security Boundary​

┌─────────────────────────────────────────────────────────────┐
│  Platform Layer (root)                                      │
│  /root/.ssh                                                 │
│  ├── id_rsa (sync denied)                                   │
│  ├── id_ed25519 (sync denied)                               │
│  ├── *.pem, *.key (sync denied)                             │
│  ├── known_hosts (sync allowed)                             │
└── allowed_keys/*.pub (allowed only when explicitly enabled) │
├─────────────────────────────────────────────────────────────┤
│  User/Tenant Layer (per-user)                               │
│  /home/<user>/.ssh                                          │
│  ├── known_hosts (synced from platform)                     │
│  ├── id_rsa (user-configured, not synced from root)         │
  └── deploy_key_<project> (per-project or explicitly mounted)│
└─────────────────────────────────────────────────────────────┘

Core Principles​

1. Default Deny​

  • By default, no files are synced to a user's ~/.ssh
  • Only files explicitly listed in the allowlist may be synced
  • Files in the denylist can never be synced, even if configured to be

2. Allowlist Mechanism​

Safe files allowed to sync by default:

  • known_hosts - known hosts list
  • known_hosts.old - known_hosts backup
  • known_hosts.old.* - historical known_hosts backups

Extended allowlist (requires security review):

  • Company public key files (*.pub)
  • SSH config snippets that have passed a security review

3. Denylist​

Files that are absolutely forbidden from syncing:

Private key files:

  • id_rsa, id_dsa, id_ecdsa, id_ed25519
  • id_* (all private key patterns)
  • *_rsa, *_dsa, *_ecdsa, *_ed25519

Certificate/key files:

  • *.pem, *.key, *.p12, *.pfx

Socket files:

  • *.socket, agent.*, control_*

Token files:

  • token_*, *.token

SSH config files:

  • config, config_*

File types:

  • Symbolic links (symlink)
  • Hard links
  • Unix sockets
  • Device files
  • Named pipes (FIFO)

Security Mechanisms​

1. TOCTOU Protection​

File descriptor operations are used to prevent race conditions:

  • Use os.open() with the O_NOFOLLOW flag
  • Use fstat() instead of stat()
  • Read data from the file descriptor

Detect st_nlink > 1 to guard against hard-link attacks.

3. Content Detection​

Check whether file content contains private key markers:

  • RSA private key format (markers starting with -----BEGIN RSA...)
  • OpenSSH private key format (markers starting with -----BEGIN OPENSSH...)
  • Other private key formats

Note: Ellipses are used above to avoid complete markers, so that this document itself does not trigger security detection.

4. Path Validation​

Validate the canonical path:

  • The source path must be under /root/.ssh
  • The destination path must be under /home/<user>/.ssh
  • Prevents path escape attacks

5. Owner Validation​

Ensure synced files have the correct owner:

  • Verify the user exists
  • Verify the UID/GID are valid
  • Set the correct owner/group

Upgrade Handling​

Legacy Private Key Detection​

When upgrading from an older version, the system detects:

  1. Private keys with file permission 600
  2. File names matching the id_* pattern
  3. Content identical to the file with the same name in /root/.ssh (content fingerprint)

Handling Policies​

  • warn: warn only, do not delete
  • backup: back up to ~/.ssh/legacy_backup_YYYYMMDD_HHMMSS/, then delete
  • delete: delete directly (only files whose content fingerprint matches)

Secure Alternatives​

Generate a dedicated deploy key for each user:

ssh-keygen -t ed25519 -f /home/<user>/.ssh/id_ed25519_<project>

Add the public key to the deploy keys of the Git repository.

Option B: Explicit Mount for a Specific User​

Configure in Docker Compose:

volumes:
  - ./secrets/user1/.ssh:/home/user1/.ssh:ro

Option C: SSH Certificate Broker (enterprise grade)​

Deploy a standalone SSH broker service that issues short-lived certificates.

Audit Log​

All sync operations are recorded in: /var/log/openace/ssh-sync.log

Log format:

[TIMESTAMP] [LEVEL] [USER] MESSAGE

Example:

[2026-01-15T10:30:00] [INFO] [alice] SYNC_SUCCESS file=known_hosts
[2026-01-15T10:30:00] [WARN] [alice] SYNC_DENIED file=id_rsa reason=denylist
[2026-01-15T10:30:00] [ERROR] [alice] LEGACY_KEY_DETECTED file=/home/alice/.ssh/id_rsa

Environment Variables​

VariableDescriptionDefault
OPENACE_SSH_UPGRADE_ACTIONUpgrade handling policybackup
OPENACE_SSH_UPGRADE_REQUIRE_CONFIRMWhether manual confirmation is requiredfalse
OPENACE_SSH_DETECT_ENCODED_KEYSWhether to detect encoded private keysfalse

Best Practices​

  1. Do not put private keys into /root/.ssh

    • Use per-user deploy keys
    • Or use explicit mounts

  2. Review audit logs regularly

    • Check SYNC_DENIED records
    • Check LEGACY_KEY_DETECTED alerts

  3. Back up data before upgrading

    • Back up user ~/.ssh directories
    • Check upgrade alerts

  4. Use secure alternatives

    • Avoid depending on root private keys
    • Configure dedicated credentials for each user/project

Changelog​

  • 2026-01-15: Initial version (Issue #2182)
    • Implemented the secure SSH key sync mechanism
    • Blocked private key sync by default
    • Provided the per-user deploy key solution