Add EC2 ops workflow driving the testnet node over SSM - #157
Conversation
PaulLaux
left a comment
PaulLaux
left a comment
There was a problem hiding this comment.
Added comments.
Also,
No environment, all other three workflows use environment: dev. Do we need an env?
| --parameters "commands=[\"bash /opt/zebra/ops.sh ${{ inputs.action }} ${{ inputs.image_tag }}\"]" \ | ||
| --query Command.CommandId --output text) | ||
| aws ssm wait command-executed --command-id "$CMD" \ | ||
| --instance-id "${{ steps.target.outputs.id }}" || true |
There was a problem hiding this comment.
true will swallow a fail here, is that what we want?
There was a problem hiding this comment.
Yes for the true itself, as otherwise the step aborts before printing the output. So I added an explicit status check that still prints the result and fails the job on anything except Success
| contents: read | ||
|
|
||
| env: | ||
| AWS_REGION: eu-central-1 |
There was a problem hiding this comment.
Switched to ${{ vars.AWS_REGION }}
| ops: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: aws-actions/configure-aws-credentials@v4 |
| --instance-id "${{ steps.target.outputs.id }}" || true | ||
| aws ssm get-command-invocation --command-id "$CMD" \ | ||
| --instance-id "${{ steps.target.outputs.id }}" \ | ||
| --query '[Status,StandardOutputContent,StandardErrorContent]' --output text \ |
There was a problem hiding this comment.
StandardOutputContent is capped at 24,000 chars. Is it enough?
There was a problem hiding this comment.
Yes, even the logs which have an internal cap that's smaller than that
| --document-name AWS-RunShellScript \ | ||
| --parameters "commands=[\"bash /opt/zebra/ops.sh ${{ inputs.action }} ${{ inputs.image_tag }}\"]" \ | ||
| --query Command.CommandId --output text) | ||
| aws ssm wait command-executed --command-id "$CMD" \ |
There was a problem hiding this comment.
aws ssm wait command-executed gives up after 100 seconds. is it enough?
There was a problem hiding this comment.
Yes except for deploy, for that changed to 15min
| run: | | ||
| CMD=$(aws ssm send-command --instance-ids "${{ steps.target.outputs.id }}" \ | ||
| --document-name AWS-RunShellScript \ | ||
| --parameters "commands=[\"bash /opt/zebra/ops.sh ${{ inputs.action }} ${{ inputs.image_tag }}\"]" \ |
There was a problem hiding this comment.
image_tag is a potential shell injection into the testnet node - nothing is checked and the user can input any value.
Constrain the image_tag using printf '%s' "$IMAGE_TAG" | grep -Eq '^[A-Za-z0-9._-]{1,128}$' || exit 1 or similar - no free text and escape chars.
There was a problem hiding this comment.
Good point, and instance_id and confirm had a similar problem.
All inputs now go via env, so they arrive as variables, not spliced text, and added your regex on image_tag
…iter && Pin configure-aws-credentials to a commit SHA
Fixed all comments, it's not mandatory, but added env to align. |
2 participants
Add EC2 ops workflow
One manual dispatch (Zebra EC2 Ops) to operate the EC2 zebra-testnet node,
replacing the ECS
deploy-ecs/stop-ecsworkflows.The box has no inbound SSH and no key: commands run over SSM Run Command, which
invokes
/opt/zebra/ops.shon the instance. Auth is OIDC - no AWS access keys.Inputs:
action,image_tag(deploy only),instance_name_tag,confirm.Actions
deploy.envat the given ECR tag, pulls it, recreates the containerrestartstopstartrecreatelogsstatusdocker compose psplus the running image IDwipe-stateconfirm=wipe-statereboot-instanceOnly
reboot-instancetouches the machine; the rest act on the container.Notes
env, never interpolated intorunbodies(template-injection sink).
deployprepends a freshdocker login: the box's boot-time ECR tokenexpires after 12h.