add S3 preview setup and teardown actions #8
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| name: 'Setup S3 Preview' | ||
| description: 'Creates an S3 preview prefix by copying from the main prefix' | ||
| inputs: | ||
| namespace: | ||
| description: 'Kubernetes namespace' | ||
| required: true | ||
| preview-number: | ||
| description: 'Preview number (PR number)' | ||
| required: true | ||
| configmap-name: | ||
| description: 'Name of the ConfigMap' | ||
| required: false | ||
| default: 'app-spring-deployment-environments' | ||
| secret-name: | ||
| description: 'Name of the secret containing credentials' | ||
| required: false | ||
| default: 'app-secrets' | ||
| s3-bucket-key: | ||
| description: 'Key for S3_BUCKET in ConfigMap' | ||
| required: false | ||
| default: 'S3_BUCKET' | ||
| s3-endpoint-key: | ||
| description: 'Key for S3_ENDPOINT in ConfigMap' | ||
| required: false | ||
| default: 'S3_ENDPOINT' | ||
| s3-region-key: | ||
| description: 'Key for S3_REGION in ConfigMap' | ||
| required: false | ||
| default: 'S3_REGION' | ||
| s3-root-folder-key: | ||
| description: 'Key for S3_ROOT_FOLDER in ConfigMap' | ||
| required: false | ||
| default: 'S3_ROOT_FOLDER' | ||
|
Comment on lines
+18
to
+33
Member
There was a problem hiding this comment. I would place these next to the config map reference, since they are related. |
||
| s3-force-path-style-access-key: | ||
| description: 'Key for S3_FORCE_PATH_STYLE_ACCESS in ConfigMap' | ||
| required: false | ||
| default: 'S3_FORCE_PATH_STYLE_ACCESS' | ||
| s3-access-key-key: | ||
| description: 'Key for S3_ACCESS_KEY in Secret' | ||
| required: false | ||
| default: 'S3_ACCESS_KEY' | ||
| s3-secret-key-key: | ||
| description: 'Key for S3_SECRET_KEY in Secret' | ||
| required: false | ||
| default: 'S3_SECRET_KEY' | ||
|
|
||
| runs: | ||
| using: "composite" | ||
| steps: | ||
| - name: Prepare setup-s3 job | ||
| env: | ||
| JOB_NAME: "prepare-s3-preview-${{ inputs.preview-number }}" | ||
| CONFIGMAP_NAME: "${{ inputs.configmap-name }}" | ||
| SECRET_NAME: "${{ inputs.secret-name }}" | ||
| S3_BUCKET_KEY: "${{ inputs.s3-bucket-key }}" | ||
| S3_ENDPOINT_KEY: "${{ inputs.s3-endpoint-key }}" | ||
| S3_REGION_KEY: "${{ inputs.s3-region-key }}" | ||
| S3_ROOT_FOLDER_KEY: "${{ inputs.s3-root-folder-key }}" | ||
| S3_FORCE_PATH_STYLE_ACCESS_KEY: "${{ inputs.s3-force-path-style-access-key }}" | ||
| S3_ACCESS_KEY_KEY: "${{ inputs.s3-access-key-key }}" | ||
| S3_SECRET_KEY_KEY: "${{ inputs.s3-secret-key-key }}" | ||
| PREVIEW_NUMBER: "${{ inputs.preview-number }}" | ||
| run: | | ||
| envsubst ' | ||
| $JOB_NAME | ||
| $CONFIGMAP_NAME | ||
| $SECRET_NAME | ||
| $S3_BUCKET_KEY | ||
| $S3_ENDPOINT_KEY | ||
| $S3_REGION_KEY | ||
| $S3_ROOT_FOLDER_KEY | ||
| $S3_FORCE_PATH_STYLE_ACCESS_KEY | ||
| $S3_ACCESS_KEY_KEY | ||
| $S3_SECRET_KEY_KEY | ||
| $PREVIEW_NUMBER | ||
| ' < ${{ github.action_path }}/job-template.yml > job-setup-s3.yml | ||
|
|
||
| echo "Prepared job manifest: job-setup-s3.yml" | ||
| shell: bash | ||
|
|
||
| - name: Create setup-s3 job | ||
| run: | | ||
| kubectl apply --namespace ${{ inputs.namespace }} -f job-setup-s3.yml | ||
| shell: bash | ||
|
|
||
| - name: Wait for setup-s3 job to complete | ||
| env: | ||
| JOB_NAME: "prepare-s3-preview-${{ inputs.preview-number }}" | ||
| NAMESPACE: "${{ inputs.namespace }}" | ||
| run: | | ||
| echo "Waiting for job $JOB_NAME in namespace $NAMESPACE..." | ||
|
|
||
| if kubectl wait --namespace $NAMESPACE --for=condition=complete --timeout=10m job/$JOB_NAME; then | ||
| echo "Job finished with status: Complete" | ||
| kubectl logs job/$JOB_NAME --namespace $NAMESPACE | ||
| elif kubectl wait --namespace $NAMESPACE --for=condition=failed --timeout=1s job/$JOB_NAME; then | ||
| echo "Job finished with status: Failed" | ||
| kubectl logs job/$JOB_NAME --namespace $NAMESPACE | ||
| exit 1 | ||
| else | ||
| echo "Timeout waiting for job to complete." | ||
| kubectl logs job/$JOB_NAME --namespace $NAMESPACE || true | ||
| exit 1 | ||
| fi | ||
| shell: bash | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| apiVersion: batch/v1 | ||
| kind: Job | ||
| metadata: | ||
| name: ${JOB_NAME} | ||
| labels: | ||
| app.kubernetes.io/managed-by: github-actions | ||
| app.kubernetes.io/component: s3-preview-setup | ||
| spec: | ||
| ttlSecondsAfterFinished: 60 | ||
| template: | ||
| spec: | ||
| restartPolicy: Never | ||
| containers: | ||
| - name: prepare-s3-preview | ||
| image: amazon/aws-cli:2.34.31 | ||
| command: [ "bash", "-c" ] | ||
| args: | ||
| - | | ||
| set -e | ||
|
|
||
| echo "### START $(date --iso-8601=seconds) ###" | ||
|
|
||
| if [ "$S3_FORCE_PATH_STYLE_ACCESS" = "true" ]; then | ||
| aws configure set default.s3.addressing_style path | ||
| fi | ||
|
|
||
| S3_ARGS="" | ||
| if [ -n "$S3_ENDPOINT" ]; then | ||
| S3_ARGS="--endpoint-url $S3_ENDPOINT" | ||
| fi | ||
|
|
||
| S3_SOURCE_PREFIX="" | ||
| if [ -n "$S3_ROOT_FOLDER" ]; then | ||
| S3_SOURCE_PREFIX="s3://$S3_BUCKET/${S3_ROOT_FOLDER%/}/" | ||
| else | ||
| S3_SOURCE_PREFIX="s3://$S3_BUCKET/" | ||
| fi | ||
|
|
||
| S3_PREVIEW_PREFIX="s3://$S3_BUCKET/preview-${PREVIEW_NUMBER}/" | ||
|
|
||
| echo "Checking if $S3_PREVIEW_PREFIX already exists..." | ||
| if [ "$(aws s3 $S3_ARGS ls "$S3_PREVIEW_PREFIX" | wc -l)" -gt 0 ]; then | ||
| echo "$S3_PREVIEW_PREFIX already exists. Skipping setup." | ||
| echo "Script finished successfully!" | ||
| echo "### END $(date --iso-8601=seconds) ###" | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "Syncing from $S3_SOURCE_PREFIX to $S3_PREVIEW_PREFIX ..." | ||
| aws s3 $S3_ARGS sync "$S3_SOURCE_PREFIX" "$S3_PREVIEW_PREFIX" | ||
|
|
||
| echo "Script finished successfully!" | ||
| echo "### END $(date --iso-8601=seconds) ###" | ||
| env: | ||
| - name: S3_BUCKET | ||
| valueFrom: | ||
| configMapKeyRef: | ||
| name: ${CONFIGMAP_NAME} | ||
| key: ${S3_BUCKET_KEY} | ||
| - name: S3_ENDPOINT | ||
| valueFrom: | ||
| configMapKeyRef: | ||
| name: ${CONFIGMAP_NAME} | ||
| key: ${S3_ENDPOINT_KEY} | ||
| optional: true | ||
| - name: AWS_DEFAULT_REGION | ||
| valueFrom: | ||
| configMapKeyRef: | ||
| name: ${CONFIGMAP_NAME} | ||
| key: ${S3_REGION_KEY} | ||
| - name: S3_ROOT_FOLDER | ||
| valueFrom: | ||
| configMapKeyRef: | ||
| name: ${CONFIGMAP_NAME} | ||
| key: ${S3_ROOT_FOLDER_KEY} | ||
| optional: true | ||
| - name: S3_FORCE_PATH_STYLE_ACCESS | ||
| valueFrom: | ||
| configMapKeyRef: | ||
| name: ${CONFIGMAP_NAME} | ||
| key: ${S3_FORCE_PATH_STYLE_ACCESS_KEY} | ||
| optional: true | ||
| - name: AWS_ACCESS_KEY_ID | ||
| valueFrom: | ||
| secretKeyRef: | ||
| name: ${SECRET_NAME} | ||
| key: ${S3_ACCESS_KEY_KEY} | ||
| - name: AWS_SECRET_ACCESS_KEY | ||
| valueFrom: | ||
| secretKeyRef: | ||
| name: ${SECRET_NAME} | ||
| key: ${S3_SECRET_KEY_KEY} | ||
| - name: PREVIEW_NUMBER | ||
| value: "${PREVIEW_NUMBER}" |
|
Member
There was a problem hiding this comment. Same here with the inputs order. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| name: 'Teardown S3 Preview' | ||
| description: 'Deletes the S3 preview prefix' | ||
| inputs: | ||
| namespace: | ||
| description: 'Kubernetes namespace' | ||
| required: true | ||
| preview-number: | ||
| description: 'Preview number (PR number)' | ||
| required: true | ||
| configmap-name: | ||
| description: 'Name of the ConfigMap' | ||
| required: false | ||
| default: 'app-spring-deployment-environments' | ||
| secret-name: | ||
| description: 'Name of the secret containing credentials' | ||
| required: false | ||
| default: 'app-secrets' | ||
| s3-bucket-key: | ||
| description: 'Key for S3_BUCKET in ConfigMap' | ||
| required: false | ||
| default: 'S3_BUCKET' | ||
| s3-endpoint-key: | ||
| description: 'Key for S3_ENDPOINT in ConfigMap' | ||
| required: false | ||
| default: 'S3_ENDPOINT' | ||
| s3-region-key: | ||
| description: 'Key for S3_REGION in ConfigMap' | ||
| required: false | ||
| default: 'S3_REGION' | ||
| s3-force-path-style-access-key: | ||
| description: 'Key for S3_FORCE_PATH_STYLE_ACCESS in ConfigMap' | ||
| required: false | ||
| default: 'S3_FORCE_PATH_STYLE_ACCESS' | ||
| s3-access-key-key: | ||
| description: 'Key for S3_ACCESS_KEY in Secret' | ||
| required: false | ||
| default: 'S3_ACCESS_KEY' | ||
| s3-secret-key-key: | ||
| description: 'Key for S3_SECRET_KEY in Secret' | ||
| required: false | ||
| default: 'S3_SECRET_KEY' | ||
|
|
||
| runs: | ||
| using: "composite" | ||
| steps: | ||
| - name: Prepare teardown-s3 job | ||
| env: | ||
| JOB_NAME: "teardown-s3-preview-${{ inputs.preview-number }}" | ||
| CONFIGMAP_NAME: "${{ inputs.configmap-name }}" | ||
| SECRET_NAME: "${{ inputs.secret-name }}" | ||
| S3_BUCKET_KEY: "${{ inputs.s3-bucket-key }}" | ||
| S3_ENDPOINT_KEY: "${{ inputs.s3-endpoint-key }}" | ||
| S3_REGION_KEY: "${{ inputs.s3-region-key }}" | ||
| S3_FORCE_PATH_STYLE_ACCESS_KEY: "${{ inputs.s3-force-path-style-access-key }}" | ||
| S3_ACCESS_KEY_KEY: "${{ inputs.s3-access-key-key }}" | ||
| S3_SECRET_KEY_KEY: "${{ inputs.s3-secret-key-key }}" | ||
| PREVIEW_NUMBER: "${{ inputs.preview-number }}" | ||
| run: | | ||
| envsubst ' | ||
| $JOB_NAME | ||
| $CONFIGMAP_NAME | ||
| $SECRET_NAME | ||
| $S3_BUCKET_KEY | ||
| $S3_ENDPOINT_KEY | ||
| $S3_REGION_KEY | ||
| $S3_FORCE_PATH_STYLE_ACCESS_KEY | ||
| $S3_ACCESS_KEY_KEY | ||
| $S3_SECRET_KEY_KEY | ||
| $PREVIEW_NUMBER | ||
| ' < ${{ github.action_path }}/job-template.yml > job-teardown-s3.yml | ||
|
|
||
| echo "Prepared job manifest: job-teardown-s3.yml" | ||
| shell: bash | ||
|
|
||
| - name: Create teardown-s3 job | ||
| run: | | ||
| kubectl apply --namespace ${{ inputs.namespace }} -f job-teardown-s3.yml | ||
| shell: bash | ||
|
|
||
| - name: Wait for teardown-s3 job to complete | ||
| env: | ||
| JOB_NAME: "teardown-s3-preview-${{ inputs.preview-number }}" | ||
| NAMESPACE: "${{ inputs.namespace }}" | ||
| run: | | ||
| echo "Waiting for job $JOB_NAME in namespace $NAMESPACE..." | ||
|
|
||
| if kubectl wait --namespace $NAMESPACE --for=condition=complete --timeout=5m job/$JOB_NAME; then | ||
| echo "Job finished with status: Complete" | ||
| kubectl logs job/$JOB_NAME --namespace $NAMESPACE | ||
| elif kubectl wait --namespace $NAMESPACE --for=condition=failed --timeout=1s job/$JOB_NAME; then | ||
| echo "Job finished with status: Failed" | ||
| kubectl logs job/$JOB_NAME --namespace $NAMESPACE | ||
| exit 1 | ||
| else | ||
| echo "Timeout waiting for job to complete." | ||
| kubectl logs job/$JOB_NAME --namespace $NAMESPACE || true | ||
| exit 1 | ||
| fi | ||
| shell: bash |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would place the namespace always at top level.