Skip to content

Commit 2700e1b

Browse files
matuscongradymatuscongrady
committed
Qualify isolated Console browser pilot
1 parent 732539d commit 2700e1b

6 files changed

Lines changed: 76 additions & 29 deletions

File tree

‎apps/console‎

‎docs/testing.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -48,10 +48,10 @@ and snapshots that merely repeat the implementation.
4848

4949
## Shared dev reservation
5050

51-
An isolated full Console browser/API harness is a [planned improvement](testing/e2e.md#isolated-console-application).
52-
Today, API/UI journeys use `pnpm dev:console`; UI-only work can use `pnpm dev:console:ui` when deployed dev supports its
53-
unchanged contract. The source CLI defaults to deployed dev even when a local API is running; select
54-
[the intended API explicitly](testing/console.md#prove-the-changed-revision).
51+
An [isolated Console browser/API pilot](testing/e2e.md#isolated-console-application) covers issue-state persistence and
52+
tenant denial against disposable PostgreSQL. Other API/UI journeys use `pnpm dev:console`; UI-only work can use
53+
`pnpm dev:console:ui` when deployed dev supports its unchanged contract. The source CLI defaults to deployed dev even
54+
when a local API is running; select [the intended API explicitly](testing/console.md#prove-the-changed-revision).
5555

5656
[Localhost login](testing/console.md#localhost-login) uses the existing dev test identities and SSM-backed Playwright
5757
helper. The current shared-dev browser tests authenticate through the real sign-in form.

‎docs/testing/console.md‎

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,12 @@
11
# Console testing
22

33
Read the [test-selection policy](../testing.md) first. Use only the sections needed for the changed behavior. Run
4-
`pnpm test:doctor -- --for=console` before a long Console lane; the default doctor checks workspace tools only. The
5-
[Console development skill](../../.agents/skills/console-development/SKILL.md) owns local startup and recovery; the
6-
private [Console E2E guide](../../apps/console/e2e/README.md) owns test identities, fixture setup and scenario commands.
7-
The [isolated application harness](e2e.md#isolated-console-application) is planned; the shared-dev commands below remain
8-
the current procedure for full browser/API journeys.
4+
`pnpm test:doctor -- --for=console` before a shared-dev Console lane; it checks the dev AWS account, login and
5+
parameters. For the isolated browser/database lane, use the default `pnpm test:doctor`, which checks workspace tools
6+
only. The [Console development skill](../../.agents/skills/console-development/SKILL.md) owns local startup and
7+
recovery; the private [Console E2E guide](../../apps/console/e2e/README.md) owns test identities, fixture setup and
8+
scenario commands. The [isolated application pilot](e2e.md#isolated-console-application) covers one browser/API/database
9+
issue journey; the shared-dev commands below remain the procedure for hosted identity and external integrations.
910

1011
## Console API and PostgreSQL
1112

@@ -30,9 +31,16 @@ pnpm --filter @stacktape/console-api-app test:db --incident-agent
3031

3132
Pass the flag directly: an extra `--` is forwarded to this script and rejected. Choose the suite whose assertions cover
3233
the change, extending it when necessary. These suites use the disposable database, not shared dev. The runner currently
33-
defaults to PostgreSQL 16 while Console config specifies RDS 15.14; matching the production major version is part of the
34-
[suite migration](strategy.md#replacing-the-existing-tests). `pnpm dev:console` instead exercises the real shared dev
35-
data plane.
34+
defaults to a pinned PostgreSQL 15.14 image matching Console's configured RDS major version. Override it only to qualify
35+
a deliberate database upgrade. `pnpm dev:console` instead exercises the real shared dev data plane.
36+
37+
The isolated issue browser pilot uses the same disposable database runner:
38+
39+
```sh
40+
pnpm --filter @stacktape/console-api-app test:db --isolated-browser
41+
```
42+
43+
It starts and stops the real local UI and API itself, so it needs neither dev credentials nor a shared-dev reservation.
3644

3745
After a schema change passes locally, apply its committed migration with `pnpm migrate:console:dev` and test the
3846
affected Console flow. Production migration remains separately authorized.

‎docs/testing/e2e.md‎

Lines changed: 31 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -60,29 +60,45 @@ separate from the scenario's allowed service traffic.
6060

6161
## Isolated Console application
6262

63-
This is the highest-value missing harness. Current full local mode uses shared dev resources; retain its reservation and
64-
setup rules until the isolated path exists.
63+
The first isolated browser journey is available through the private Console database runner. It uses a disposable
64+
PostgreSQL container, the real Console UI and Fastify router, and locally signed Cognito-shaped tokens. It does not
65+
replace shared dev for hosted sign-in, callbacks, AWS execution or deployed configuration.
6566

66-
Implement incrementally:
67+
```sh
68+
pnpm --filter @stacktape/console-api-app test:db --isolated-browser
69+
```
70+
71+
The runner owns PostgreSQL and verifies container removal. The test owns a scratch database, API server, Vite process
72+
and browser; it disposes each after success or failure. Two independently seeded tenant journeys run concurrently. Each
73+
resolves an issue through the UI, checks the database, reloads, and checks the persisted state. A second signed identity
74+
is denied through the same HTTP server both before and after its own organization membership check. The local session
75+
fixture writes Amplify's token keys before page startup, and an assertion checks that the UI sends the accepted bearer
76+
token. Only the test startup substitutes PostgreSQL's TLS connection options; Prisma, token verification, authorization
77+
and the production router remain real. The lane has no shared-dev reservation or AWS credential requirement.
78+
79+
Run the command twice to check repeatability. `STP_ISOLATED_BROWSER_FAIL_AFTER_START=1` triggers a deliberate failure
80+
after all services start, for checking teardown; that invocation must fail while still reporting removal of its owned
81+
database container. The fixture currently tests one issue flow. Extend it only for cases that need a
82+
browser/API/database boundary; keep provider and hosted identity qualifications in their existing lanes.
83+
84+
When extending this pilot:
6785

6886
1. Reuse the real migrated PostgreSQL and locally signed-token setup already demonstrated by the incident-agent tests.
6987
The normal verifier must still check signatures, token use, issuer, audience and expiry. Local issuance qualifies our
7088
verification and authorization logic, not Cognito's hosted login or federation.
7189
2. Reuse the production-router HTTP bootstrap in
72-
[incident-agent runtime tests](../../apps/console/api/scripts/incident-agent-runtime.test.ts). For reusable startup,
73-
replace the database-factory module mock with a narrow connection/configuration seam where needed. Keep the real
74-
router and token verifier. Do not add an auth bypass, test-only public endpoint, mutable global table of mocked
75-
services or a second implementation of permission checks.
90+
[incident-agent runtime tests](../../apps/console/api/scripts/incident-agent-runtime.test.ts). Keep the real router
91+
and token verifier. Do not add an auth bypass, test-only public endpoint, mutable global table of mocked services or
92+
a second implementation of permission checks.
7693
3. Start the real UI against that API. Seed an organization, project and restricted identities. The UI uses Amplify's
7794
Cognito token storage, `fetchAuthSession` for bearer headers, and its own stored email for initial UI state. First
78-
prototype a test-owned session fixture compatible with those consumers, using newly issued synthetic ID/access tokens
79-
and the configured test pool/client. Verify the actual client sends an accepted request after reload without reaching
80-
Cognito; an arbitrary JWT in local storage is insufficient. Keep storage details inside one helper and check it when
81-
upgrading Amplify. If a narrow identity adapter is needed, wire it only into the isolated app startup. Expiry/refresh
82-
and hosted sign-in/sign-out need separate scenarios; Amplify may refresh expired tokens over the network.
83-
[Amplify session behavior](https://docs.amplify.aws/javascript/frontend/auth/manage-user-sessions/).
84-
4. First prove a small flow such as changing an issue's state, rereading it over HTTP, and seeing the persisted value
85-
after browser reload. A second identity from another project/organization must be denied through the same server.
95+
session fixture uses newly issued synthetic ID/access tokens and the configured test pool/client. Verify the actual
96+
client sends an accepted request after reload without reaching Cognito. Keep storage details inside one helper and
97+
check it when upgrading Amplify. If a narrow identity adapter is needed, wire it only into the isolated app startup.
98+
Expiry/refresh and hosted sign-in/sign-out need separate scenarios; Amplify may refresh expired tokens over the
99+
network. [Amplify session behavior](https://docs.amplify.aws/javascript/frontend/auth/manage-user-sessions/).
100+
4. Keep the issue flow's state change, database read and persistence after browser reload. A second identity from
101+
another project/organization must be denied through the same server.
86102
5. Run two independent scenarios concurrently and repeat them from clean state. Verify all child processes, containers
87103
and data are disposed, including after an intentionally failed assertion. Only then expand parallel coverage.
88104

‎scripts/workspace/test-plan.test.ts‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,18 @@ test('browser scenario and configuration edits require executing the browser lan
104104
}
105105
});
106106

107+
test('isolated Console issue journeys select the disposable browser and database lane', () => {
108+
for (const path of [
109+
'apps/console/ui/e2e/isolated-console.test.ts',
110+
'apps/console/ui/src/pages/IssuesPage/IssueDetailPage.tsx',
111+
'apps/console/api/scripts/run-db-integration.ts'
112+
]) {
113+
const ids = new Set(createTestPlan([path]).map(({ id }) => id));
114+
assert.ok(ids.has('console-browser-isolated'));
115+
if (path.endsWith('isolated-console.test.ts')) assert.ok(!ids.has('console-browser-dev-api'));
116+
}
117+
});
118+
107119
test('selects semantic synthesis and live AWS evidence for CloudFormation changes', () => {
108120
const ids = new Set(createTestPlan(['packages/cloudformation/src/template.ts']).map(({ id }) => id));
109121
assert.ok(ids.has('synthesis'));

‎scripts/workspace/test-plan.ts‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -139,14 +139,25 @@ const RULES: Rule[] = [
139139
matches: (path) =>
140140
path.startsWith('apps/console/api/src/') || path.startsWith('packages/console-api/') || isConsoleStartupPath(path)
141141
},
142+
{
143+
id: 'console-browser-isolated',
144+
proves:
145+
'Issue-state persistence and tenant denial through the real UI, local HTTP router and disposable PostgreSQL.',
146+
commands: ['pnpm --filter @stacktape/console-api-app test:db --isolated-browser'],
147+
matches: (path) =>
148+
path === 'apps/console/ui/e2e/isolated-console.test.ts' ||
149+
path === 'apps/console/api/scripts/incident-agent-fixtures.ts' ||
150+
path === 'apps/console/api/scripts/run-db-integration.ts' ||
151+
path.startsWith('apps/console/ui/src/pages/IssuesPage/')
152+
},
142153
{
143154
id: 'console-browser-dev-api',
144155
proves:
145156
'Authenticated projects navigation against the dev API. Extend the browser scenario to cover the changed customer flow.',
146157
commands: ['pnpm test:console:browser:dev-api'],
147158
matches: (path) =>
148159
path.startsWith('apps/console/ui/src/') ||
149-
path.startsWith('apps/console/ui/e2e/') ||
160+
(path.startsWith('apps/console/ui/e2e/') && path !== 'apps/console/ui/e2e/isolated-console.test.ts') ||
150161
/^apps\/console\/ui\/playwright.*\.ts$/.test(path) ||
151162
path.startsWith('packages/ui-react/')
152163
},

0 commit comments

Comments
 (0)