Spin up one Ubuntu 24.04 EC2 instance (Docker pre-installed via cloud-init), a
stable Elastic IP, a Route53 A record, and an imported SSH key pair (generated
locally), then run the
on-prem Compose stack (deploy/compose) on it. This is the environment used
for the P3-10 stage-5 completion gate (“stand it up from the release bundle on a clean
host”). It is a single-VM host — not the ECS adapter (P3-7).
Only the AWS CLI is required (no Terraform). Tear everything down with one command.
Generate an SSH key locally (the private key never leaves your machine — only the
public key is imported, so no ssm:PutParameter permission is needed):
DNS gotcha. The Route53 A record only works if the domain is actually
delegated to that Route53 hosted zone (registrar NS = the zone’s awsdns-*
servers). If the domain lives at another DNS provider, either add the A record
there instead, or skip DNS entirely and use sslip.io on the Elastic IP:
set Fqdn / PUBLIC_DOMAIN to <dash-ip>.sslip.io (e.g.
203-0-113-7.sslip.io) — it resolves to the IP with zero setup and still
gets a real Let’s Encrypt cert.
Minimal IAM for the deploying user: EC2 + Route53 + CloudFormation (no SSM write,
no IAM caps needed).
Caddy obtains a real Let’s Encrypt cert for <Fqdn> (ports 80/443 are open).
Register https://<Fqdn>/oauth2/callback in the login IdP’s client (that one URI
serves every provider you enable), then open https://<Fqdn> and sign in.
AF_OAUTH_ALLOWED_DOMAINS is what gets your first administrator in; after
that, somebody invited to a tenant in the Admin panel signs in without being
listed there too. Per-tenant sign-in pages live at https://<Fqdn>/login/<slug>.
SUPER_ADMIN_EMAILS is read once at startup and is the only source of truth —
removing an address from it and restarting also revokes the role in the database.
The Elastic IP, security group, instance, Route53 record and key pair are all
deleted with the stack.
KeyPair delete gotcha. The AWS::EC2::KeyPair handler touches SSM
Parameter Store even for imported keys; if the deploying user lacks ssm:*
(e.g. only AmazonSSMReadOnlyAccess), the KeyPair fails to delete with an
opaque InternalFailure and the stack goes DELETE_FAILED. Work around it by
retaining that one resource and deleting the key pair directly:
(Or grant the deploying user ssm:PutParameter/ssm:DeleteParameter on
/ec2/keypair/* to avoid it entirely.)
Also delete the throwaway login IdP client (the Google Cloud Console OAuth
client, or the Entra/Okta/… app registration) — its secret was used during testing.
The instance requires IMDSv2 with a hop limit of 1 (MetadataOptions in cfn.yaml). The
template attaches no instance profile; the setting is there so that one attached later
(Bedrock, the single-VM engines of ADR 0099) reaches the host-network Control Plane (one hop)
and not the workspaces, which sit on docker bridges (two hops). Workspaces are also started
with AWS_EC2_METADATA_DISABLED=true, unless the Control Plane has AF_WS_WORKLOAD_AWS=1;
that opt-in does not lift the hop limit, so the instance profile stays out of reach either way.
For a stack created before this setting, apply it to the running instance rather than by
updating the stack: LatestUbuntuAmi resolves again on every update, and a newer AMI replaces
the instance.
Workspaces that were already running keep their old container and environment until they are
Stopped and Started in the Console (docker compose up -d does not recreate them); do that
after the upgrade, then check from a workspace shell (with AF_WS_WORKLOAD_AWS off, the
default) that echo ${AWS_EC2_METADATA_DISABLED:-unset}
prints true, that aws sts get-caller-identity with no profile fails with “Unable to locate
credentials”, and that the token request below fails. With the opt-in on, the variable is
unset by design, but the token request must still fail.
Check the network block from a workspace shell: curl -s -o /dev/null -m 3 -w '%{http_code}' -X PUT -H 'X-aws-ec2-metadata-token-ttl-seconds: 60' http://169.254.169.254/latest/api/token must not
print 200.
This template does not offer provided routes (ADR 0129: Bedrock the deployment pays for). The
Control Plane would need an AWS identity of its own to assume the route role, and the template
attaches no instance profile and creates no route role. A stored access key is not an option:
ADR 0129 rules it out. The ECS stack’s ProvidedRoutes=on is the supported way
(deploy/aws/ecs/cfn/PARAMETERS.md, “Provided routes”).
