Skip to content

Commit 1c939a2

Browse files
drewdrew
committed
feat(sandbox): enable port forwarding and setup openclaw (!33)
1 parent f143972 commit 1c939a2

29 files changed

Lines changed: 1313 additions & 226 deletions

File tree

‎CONTRIBUTING.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,16 @@ To connect to a running sandbox with SSH, use:
5757
navigator sandbox connect <sandbox-id>
5858
```
5959

60+
To forward a local port into a sandbox (e.g., port 18789):
61+
62+
```bash
63+
navigator sandbox forward start 18789 <sandbox-name>
64+
```
65+
66+
This opens a local SSH tunnel so connections to `127.0.0.1:18789` on the host
67+
are forwarded to `127.0.0.1:18789` inside the sandbox. The command stays
68+
attached until interrupted (Ctrl+C). Add `-d` to run in the background.
69+
6070
Relevant environment variables:
6171

6272
- `NAVIGATOR_SSH_GATEWAY_HOST`, `NAVIGATOR_SSH_GATEWAY_PORT`, `NAVIGATOR_SSH_CONNECT_PATH`

‎Cargo.lock‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎architecture/sandbox-connect.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,62 @@ sequenceDiagram
9696
- HMAC is SHA-256 over `token|timestamp|nonce` with the shared secret.
9797
- Sandbox verifies timestamp is within `ssh_handshake_skew_secs` and HMAC matches.
9898

99+
## Port Forwarding (`sandbox forward start`)
100+
101+
`nav sandbox forward start <port> <name>` opens a local SSH tunnel so connections to `127.0.0.1:<port>`
102+
on the host are forwarded to `127.0.0.1:<port>` inside the sandbox.
103+
104+
### CLI
105+
106+
- Reuses the same `ProxyCommand` path as `sandbox connect`.
107+
- Invokes OpenSSH with `-N -L <port>:127.0.0.1:<port> sandbox`.
108+
- By default stays attached in foreground until interrupted (Ctrl+C).
109+
- With `-d`/`--background`, SSH forks after auth and the CLI exits. The PID is
110+
tracked in `~/.config/navigator/forwards/<name>-<port>.pid` along with sandbox id metadata.
111+
- `nav sandbox forward stop <port> <name>` validates PID ownership and then kills a background forward.
112+
- `nav sandbox forward list` shows all tracked forwards.
113+
- `nav sandbox forward stop` and `nav sandbox forward list` are local operations and do not require
114+
resolving an active cluster.
115+
- `nav sandbox create --forward <port>` starts a background forward before connect/exec, including
116+
when no trailing command is provided.
117+
- `nav sandbox delete` auto-stops any active forwards for the deleted sandbox.
118+
119+
### Supervisor `direct-tcpip` handling
120+
121+
The sandbox SSH server (`crates/navigator-sandbox/src/ssh.rs`) implements
122+
`channel_open_direct_tcpip` from the russh `Handler` trait.
123+
124+
- **Loopback-only**: only `127.0.0.1`, `localhost`, and `::1` destinations are accepted.
125+
Non-loopback destinations are rejected (`Ok(false)`) to prevent the sandbox from being
126+
used as a generic proxy.
127+
- **Bridge**: accepted channels spawn a tokio task that connects a `TcpStream` to the
128+
target address and uses `copy_bidirectional` between the SSH channel stream and the
129+
TCP stream.
130+
- No additional state is stored on `SshHandler` — the `Channel<Msg>` object from russh is
131+
self-contained, so forwarding channels are fully independent of session channels.
132+
133+
### Flow
134+
135+
```mermaid
136+
sequenceDiagram
137+
participant App as Local Application
138+
participant SSH as OpenSSH Client
139+
participant GW as Gateway (CONNECT)
140+
participant SSHD as Sandbox SSH
141+
participant SVC as Service in Sandbox
142+
143+
SSH->>GW: CONNECT /connect/ssh
144+
GW->>SSHD: TCP + Preface handshake
145+
SSH->>SSHD: direct-tcpip channel (127.0.0.1:port)
146+
SSHD->>SVC: TcpStream::connect(127.0.0.1:port)
147+
App->>SSH: connect to 127.0.0.1:port (local)
148+
SSH->>SSHD: channel data
149+
SSHD->>SVC: TCP data
150+
SVC-->>SSHD: TCP response
151+
SSHD-->>SSH: channel data
152+
SSH-->>App: response
153+
```
154+
99155
## Authentication Model
100156

101157
- The SSH server accepts any SSH key or none; the gateway handles authorization.

‎architecture/sandbox.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -138,6 +138,23 @@ The sandbox supervisor can run as a more privileged user while the child process
138138
privileged account before `exec`. Configure this via `process.run_as_user` and
139139
`process.run_as_group` in the policy. If unset, the child inherits the supervisor's user/group.
140140

141+
## Zombie Reaping (PID 1 Init Duties)
142+
143+
`navigator-sandbox` runs as PID 1 inside the container. In Linux, when a process exits, its
144+
parent must call `waitpid()` to collect the exit status; otherwise the process remains as a zombie.
145+
Orphaned processes (whose parent exits first) are reparented to PID 1, which becomes responsible
146+
for reaping them.
147+
148+
Coding agents running inside the sandbox (OpenClaw, Claude, Codex) frequently spawn background
149+
daemons and child processes. When these grandchildren are orphaned, they become PID 1's
150+
responsibility. Without reaping, they accumulate as zombies for the lifetime of the container.
151+
152+
The sandbox supervisor registers a `SIGCHLD` handler at startup and runs a background reaper task.
153+
On each signal, it first inspects exited children with `waitid(..., WNOWAIT)` and checks whether
154+
the PID belongs to a managed child with an explicit waiter (entrypoint or SSH session child). If
155+
the PID is managed, it leaves the status for that waiter. Otherwise, it reaps the orphaned child.
156+
This avoids `ECHILD` races with explicit `child.wait()` calls while still collecting orphan zombies.
157+
141158
## Platform Extensibility
142159

143160
Platform-specific implementations are wired through `crates/navigator-sandbox/src/sandbox/mod.rs`.

‎build/scripts/cluster-deploy-fast.sh‎

Lines changed: 33 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ elif [[ "${explicit_target}" == "0" ]]; then
102102
crates/navigator-server/*|deploy/docker/Dockerfile.server)
103103
build_server=1
104104
;;
105-
crates/navigator-sandbox/*|deploy/docker/Dockerfile.sandbox|python/*|pyproject.toml|uv.lock|dev-sandbox-policy.rego)
105+
crates/navigator-sandbox/*|deploy/docker/Dockerfile.sandbox|deploy/docker/openclaw-start.sh|python/*|pyproject.toml|uv.lock|dev-sandbox-policy.rego)
106106
build_sandbox=1
107107
;;
108108
deploy/docker/Dockerfile.pki-job)
@@ -131,11 +131,14 @@ fi
131131

132132
build_start=$(date +%s)
133133

134-
# Capture the sandbox image ID before rebuild so we can detect if it changed.
135-
sandbox_image_id_before=""
136-
if [[ "${build_sandbox}" == "1" ]]; then
137-
sandbox_image_id_before=$(docker images -q "navigator-sandbox:${IMAGE_TAG}" 2>/dev/null || true)
138-
fi
134+
# Capture image IDs before rebuild so we can detect what changed.
135+
declare -A image_id_before=()
136+
for component in server sandbox pki-job; do
137+
var="build_${component//-/_}"
138+
if [[ "${!var}" == "1" ]]; then
139+
image_id_before[${component}]=$(docker images -q "navigator-${component}:${IMAGE_TAG}" 2>/dev/null || true)
140+
fi
141+
done
139142

140143
server_pid=""
141144
sandbox_pid=""
@@ -174,36 +177,22 @@ build_end=$(date +%s)
174177
log_duration "Image builds" "${build_start}" "${build_end}"
175178

176179
declare -a pushed_images=()
177-
178-
# Detect whether the sandbox image actually changed by comparing the Docker
179-
# image ID before and after the build. This is a content-addressable hash so
180-
# identical builds produce the same ID regardless of registry digest quirks.
181-
sandbox_image_changed=0
182-
if [[ "${build_sandbox}" == "1" ]]; then
183-
sandbox_image_id_after=$(docker images -q "navigator-sandbox:${IMAGE_TAG}" 2>/dev/null || true)
184-
if [[ -n "${sandbox_image_id_before}" && -n "${sandbox_image_id_after}" \
185-
&& "${sandbox_image_id_before}" != "${sandbox_image_id_after}" ]]; then
186-
sandbox_image_changed=1
187-
elif [[ -z "${sandbox_image_id_before}" && -n "${sandbox_image_id_after}" ]]; then
188-
# First build — treat as changed
189-
sandbox_image_changed=1
180+
declare -a changed_images=()
181+
182+
for component in server sandbox pki-job; do
183+
var="build_${component//-/_}"
184+
if [[ "${!var}" == "1" ]]; then
185+
docker tag "navigator-${component}:${IMAGE_TAG}" "${IMAGE_REPO_BASE}/${component}:${IMAGE_TAG}"
186+
pushed_images+=("${IMAGE_REPO_BASE}/${component}:${IMAGE_TAG}")
187+
188+
# Detect whether the image actually changed by comparing Docker image IDs.
189+
id_after=$(docker images -q "navigator-${component}:${IMAGE_TAG}" 2>/dev/null || true)
190+
id_before=${image_id_before[${component}]:-}
191+
if [[ -z "${id_before}" || "${id_before}" != "${id_after}" ]]; then
192+
changed_images+=("${component}")
193+
fi
190194
fi
191-
fi
192-
193-
if [[ "${build_server}" == "1" ]]; then
194-
docker tag "navigator-server:${IMAGE_TAG}" "${IMAGE_REPO_BASE}/server:${IMAGE_TAG}"
195-
pushed_images+=("${IMAGE_REPO_BASE}/server:${IMAGE_TAG}")
196-
fi
197-
198-
if [[ "${build_sandbox}" == "1" ]]; then
199-
docker tag "navigator-sandbox:${IMAGE_TAG}" "${IMAGE_REPO_BASE}/sandbox:${IMAGE_TAG}"
200-
pushed_images+=("${IMAGE_REPO_BASE}/sandbox:${IMAGE_TAG}")
201-
fi
202-
203-
if [[ "${build_pki_job}" == "1" ]]; then
204-
docker tag "navigator-pki-job:${IMAGE_TAG}" "${IMAGE_REPO_BASE}/pki-job:${IMAGE_TAG}"
205-
pushed_images+=("${IMAGE_REPO_BASE}/pki-job:${IMAGE_TAG}")
206-
fi
195+
done
207196

208197
if [[ "${#pushed_images[@]}" -gt 0 ]]; then
209198
push_start=$(date +%s)
@@ -215,13 +204,15 @@ if [[ "${#pushed_images[@]}" -gt 0 ]]; then
215204
log_duration "Image push" "${push_start}" "${push_end}"
216205
fi
217206

218-
# If the sandbox image changed, evict the stale copy from k3s's containerd
219-
# store so new sandbox pods pull the updated image from the registry.
220-
# Without this, k3s uses its cached copy (imagePullPolicy defaults to
221-
# IfNotPresent for non-:latest tags) and sandbox pods run stale code.
222-
if [[ "${sandbox_image_changed}" == "1" ]]; then
223-
echo "Sandbox image changed (${sandbox_image_id_before:-<none>} -> ${sandbox_image_id_after}), evicting stale image from k3s..."
224-
docker exec "${CONTAINER_NAME}" crictl rmi "${IMAGE_REPO_BASE}/sandbox:${IMAGE_TAG}" >/dev/null 2>&1 || true
207+
# Evict stale images from k3s's containerd store so new pods pull the
208+
# updated image from the registry. Without this, k3s uses its cached copy
209+
# (imagePullPolicy defaults to IfNotPresent for non-:latest tags) and pods
210+
# run stale code.
211+
if [[ "${#changed_images[@]}" -gt 0 ]]; then
212+
echo "Evicting stale images from k3s: ${changed_images[*]}"
213+
for component in "${changed_images[@]}"; do
214+
docker exec "${CONTAINER_NAME}" crictl rmi "${IMAGE_REPO_BASE}/${component}:${IMAGE_TAG}" >/dev/null 2>&1 || true
215+
done
225216
fi
226217

227218
if [[ "${needs_helm_upgrade}" == "1" ]]; then

‎build/scripts/cluster-push-component.sh‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,15 @@ esac
1818

1919
IMAGE_TAG=${IMAGE_TAG:-dev}
2020
IMAGE_REPO_BASE=${IMAGE_REPO_BASE:-${NAVIGATOR_REGISTRY:-localhost:5000/navigator}}
21+
CLUSTER_NAME=${CLUSTER_NAME:-$(basename "$PWD")}
22+
CONTAINER_NAME="navigator-cluster-${CLUSTER_NAME}"
2123

2224
docker tag "navigator-${component}:${IMAGE_TAG}" "${IMAGE_REPO_BASE}/${component}:${IMAGE_TAG}"
2325
docker push "${IMAGE_REPO_BASE}/${component}:${IMAGE_TAG}"
26+
27+
# Evict the stale image from k3s's containerd cache so new pods pull the
28+
# updated image. Without this, k3s uses its cached copy (imagePullPolicy
29+
# defaults to IfNotPresent for non-:latest tags) and pods run stale code.
30+
if docker ps -q --filter "name=${CONTAINER_NAME}" | grep -q .; then
31+
docker exec "${CONTAINER_NAME}" crictl rmi "${IMAGE_REPO_BASE}/${component}:${IMAGE_TAG}" >/dev/null 2>&1 || true
32+
fi

‎build/test.toml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,8 @@ description = "Run sandbox end-to-end tests"
1919
depends = ["python:proto", "cluster:deploy"]
2020
env = { UV_NO_SYNC = "1", PYTHONPATH = "python" }
2121
run = "uv run pytest -o python_files='test_*.py' e2e/python"
22+
23+
["test:e2e:port-forward"]
24+
description = "Run port-forward integration test"
25+
depends = ["cluster:deploy"]
26+
run = "bash e2e/bash/test_port_forward.sh"

‎crates/navigator-cli/Cargo.toml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,10 @@ tracing-subscriber = { workspace = true }
6262
workspace = true
6363

6464
[dev-dependencies]
65+
futures = { workspace = true }
6566
rcgen = { version = "0.13", features = ["crypto", "pem"] }
67+
reqwest = { workspace = true }
68+
serde_json = { workspace = true }
6669
tempfile = "3"
6770
tokio-stream = { workspace = true }
71+
url = { workspace = true }

0 commit comments

Comments
 (0)