No valid credential source for S3 backend found with GitHub OIDC - amazon-web-services

Question

I am working with Github OIDC to login to AWS and Deploy our terraform code, I am stuck on terraform init, most of the solutions on the internet point towards deleting the credentials file or providing the credentials explicitly, I can't do any of those since the credentials file does not exist with OIDC and I don't want to explicitly provide the Access_key and Secret_ID explicitly in the backend moduel either since that could lead to a security risk, Here's my GitHub Deployment file:
name: AWS Terraform Plan & Deploy
on:
push:
paths:
- "infrastructure/**"
# branches-ignore:
# - '**'
pull_request:
env:
tf_actions_working_dir: infrastructure/env/dev-slb-alpha/dev
tf_actions_working_dir_prod: infrastructure/env/prod-slb-prod/prod
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TF_WORKSPACE: "default"
TF_ACTION_COMMENT: 1
plan: "plan.tfplan"
BUCKET_NAME : "slb-dev-terraform-state"
AWS_REGION : "us-east-1"
jobs:
build:
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
steps:
- run: sleep 5 # there's still a race condition for now
- name: Clone Repository (Latest)
uses: actions/checkout#v2
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials#v1
with:
aws-region: us-east-1
role-to-assume: arn:aws:iam::262267462662:role/slb-dev-github-actions-role
role-session-name: GithubActionsSession
# - name: Configure AWS
# run: |
# export AWS_ROLE_ARN=arn:aws:iam::262267462662:role/slb-dev-github-actions-role
# # export AWS_WEB_IDENTITY_TOKEN_FILE=/tmp/awscreds
# export AWS_DEFAULT_REGION=us-east-1
# # echo AWS_WEB_IDENTITY_TOKEN_FILE=$AWS_WEB_IDENTITY_TOKEN_FILE >> $GITHUB_ENV
# echo AWS_ROLE_ARN=$AWS_ROLE_ARN >> $GITHUB_ENV
# echo AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION >> $GITHUB_ENV
- run: aws sts get-caller-identity
setup:
runs-on: ubuntu-latest
environment:
name: Dev
url: https://dev.test.com
name: checkov-action-dev
steps:
- name: Checkout repo
uses: actions/checkout#master
with:
submodules: 'true'
# - name: Add Space to Dev
# run: |
# sysconfig -r proc exec_disable_arg_limit=1
# shell: bash
- name: Run Checkov action
run: |
pip3 install checkov
checkov --directory /infrastructure
id: checkov
# uses: bridgecrewio/checkov-action#master
# with:
# directory: infrastructure/
#skip_check: CKV_AWS_1
# quiet: true
# soft_fail: true
#framework: terraform
tfsec:
name: tfsec
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
# - name: Terraform security scan
# uses: aquasecurity/tfsec-pr-commenter-action#v0.1.10
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: tfsec
uses: tfsec/tfsec-sarif-action#master
with:
# sarif_file: tfsec.sarif
github_token: ${{ secrets.INPUT_GITHUB_TOKEN }}
# - name: Upload SARIF file
# uses: github/codeql-action/upload-sarif#v1
# with:
# sarif_file: tfsec.sarif
superlinter:
name: superlinter
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Scan Code Base
# uses: github/super-linter#v4
# env:
# VALIDATE_ALL_CODEBASE: false
# # DEFAULT_BRANCH: master
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# VALIDATE_TERRAFORM_TERRASCAN: false
uses: terraform-linters/setup-tflint#v1
with:
tflint_version: v0.29.0
terrascan:
name: terrascan
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Run Terrascan
id: terrascan
uses: accurics/terrascan-action#v1
with:
iac_type: "terraform"
iac_version: "v15"
policy_type: "aws"
only_warn: true
#iac_dir:
#policy_path:
#skip_rules:
#config_path:
terraform:
defaults:
run:
working-directory: ${{ env.tf_actions_working_dir}}
name: "Terraform"
runs-on: ubuntu-latest
needs: build
steps:
- name: Clone Repository (Latest)
uses: actions/checkout#v2
if: github.event.inputs.git-ref == ''
- name: Clone Repository (Custom Ref)
uses: actions/checkout#v2
if: github.event.inputs.git-ref != ''
with:
ref: ${{ github.event.inputs.git-ref }}
- name: Setup Terraform
uses: hashicorp/setup-terraform#v1
with:
terraform_version: 1.1.2
- name: Terraform Format
id: fmt
run: terraform fmt -check
- name: Terraform Init
id: init
run: |
# # cat ~/.aws/crendentials
# # export AWS_PROFILE=pki-aws-informatics
# aws configure list-profiles
#terraform init -backend-config="bucket=slb-dev-terraform-state"
terraform init -backend-config="access_key=${{ env.AWS_ACCESS_KEY_ID}}" -backend-config="secret_key=${{ env.AWS_SECRET_ACCESS_KEY}}"
terraform init --backend-config="access_key=${{ env.AWS_ACCESS_KEY_ID}}" --backend-config="secret_key=${{ env.AWS_SECRET_ACCESS_KEY}}"
- name: Terraform Validate
id: validate
run: terraform validate -no-color
- name: Terraform Plan
id: plan
run: terraform plan -var-file="terraform.tfvars" -out=${{ env.plan }}
- uses: actions/github-script#0.9.0
if: github.event_name == 'pull_request'
env:
PLAN: "terraform\n${{ steps.plan.outputs.stdout }}"
with:
github-token: ${{ secrets.INPUT_GITHUB_TOKEN }}
script: |
const output = `#### Terraform Format and Style 🖌\`${{ steps.fmt.outcome }}\`
#### Terraform Initialization ⚙️\`${{ steps.init.outcome }}\`
#### Terraform Validation 🤖${{ steps.validate.outputs.stdout }}
#### Terraform Plan 📖\`${{ steps.plan.outcome }}\`
<details><summary>Show Plan</summary>
\`\`\`${process.env.PLAN}\`\`\`
</details>
*Pusher: #${{ github.actor }}, Action: \`${{ github.event_name }}\`, Working Directory: \`${{ env.tf_actions_working_dir }}\`, Workflow: \`${{ github.workflow }}\`*`;
github.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: output
})
As you can see I have tried it a couple of ways and still end up with the same error, which is , I have made sure that the profile we using is correct,I also cannot proivde credentials in the init command itself,it is validating to the correct profile since it is fetching the correct arn for the profile I need it to work on, I also read somewhere that the credentials for aws profiles and S3 could be different and if that is the case how can I integrate OIDC in ythat project, not sure what or where I might be going wrong otherwise, appreciate any help or headers,

Answer 1

I can't give advice specific to Github (since I'm using Bitbucket), but if you're using OIDC for access to AWS from your SCM of choice the same principals apply. The S3 backend for Terraform itself doesn't allow specifying any of the normal configuration for OIDC, but you can set this with environment variables and have it work:
AWS_WEB_IDENTITY_TOKEN_FILE=<web-identity-token-file>
AWS_ROLE_ARN=arn:aws:iam::<account-id>:role/<role-name>
For Bitbucket Pipelines users:
Specify oidc: true in your pipelines config
Write the OIDC token file using e.g. echo $BITBUCKET_STEP_OIDC_TOKEN > $(pwd)/web-identity-token
Export the environment variables as above
I've split my S3 backend storage away from the account that has resources, so will need to look at configuring the actual AWS provider separately - it does have options for assume_role.web_identity_token_file and assume_role.role_arn