When building CI/CD Pipelines to automate the process of provisioning or updating resources in AWS, one of the challenges we are facing is how to authenticate to the AWS from the pipeline. Traditionally, the method of connecting to AWS service is by using IAM user Access Key ID and Secret Access Key, saved in your gitlab repository variables. These keys need to be rotated as best-practices after 90 days to avoid potential threat of its leakage which adds up to the maintenance work. This might be fine if you only have one repository, but in real life situations you might have thousands or hundreds of repos.
From a security standpoint, hardcoding credentials is not a well-embraced approach and it should be avoided by all means. To solve this recurring issue, AWS has support for OpenID Connect using external identity providers like Gitlab or Github and assuming IAM roles with temporary credentials.
In this blog post, we will demonstrate how to use OpenID Connect (OIDC) to allow your GitLab runners pipelines to access resources in Amazon Web Services (AWS), without needing to store the AWS credentials
what is OpenID Connect (OIDC)?
OpenID allows clients of all types, including Web-based, mobile, and JavaScript clients to verify the identity of the End-User based on the authentication performed by an Authorization Server, as well as to obtain basic profile information about the End-User in an interoperable and REST-like manner.
How to set up OIDC
Add GitLab as an OpenID Connect (OIDC) provider in AWS
1. On the IAM console, under Access management in the navigation pane, choose Identity providers.
2. Choose Create provider.
3. For the Provider type, choose OpenID Connect
4. For the Provider URL, enter https://gitlab.com
or the address of your self hosted GitLab instance
5. For the Audience enter https://gitlab.com
or the address of your self hosted GitLab instance (The address must include http://
or https://
)
6. Click on Add provider
Create IAM role and Policy
1. On the IAM console, under Access management in the navigation pane, choose Policies.
2. Choose Create policy.
3. On the JSON tab, enter the following policy code as an example we will give full access to S3 service to our pipeline:
{ "Version": "2012-10-17", "Statement": [ { "Action": [ "s3:*" ], "Effect": "Allow", "Resource": "*" } ] }
4. Choose Review policy.
5. For Name, enter Gitalb-mypoject-S3FullAccess-policy
.
6. Choose Create policy.
7. In the navigation pane, choose Roles.
8. For Select type of trusted entity, choose Custom trust policy
9. Enter the following trust policy code:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::ACCOUNTID:oidc-provider/gitlab.com" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringLike": { "gitlab.com:sub": "project_path:GITLAB-USERID/PROJECTNAME:ref_type:branch:ref:BRANCH" } } } ] }
You can limit the authorization to a specific group, project, branch, or tag. For the full list of supported filtering types, see Connect to cloud services.
10. Choose Next.
11. Search for and select the policy you created (Gitalb-mypoject-S3FullAccess-policy
).
12. Choose Next.
13. For Role name, enter Gitalb-mypoject-S3FullAccess-role
.
14. Review the attached policies and choose Create role.
Test the GitLab OpenID Connect provider from a CI CD Pipeline
To test whether the GitLab OpenID Connect provider can connect AWS and assume the role to retrieve temporary credentials, let create a simple pipeline as below
stages: - deploy .authenticate: &authenticate - echo "$CI_JOB_JWT_V2" > $AWS_WEB_IDENTITY_TOKEN_FILE deploy: stage: deploy image: name: public.ecr.aws/aws-cli/aws-cli:latest variables: AWS_ROLE_ARN: arn:aws:iam:::role/Gitalb-mypoject-S3FullAccess-role # or ${ROLE_ARN} injected by CI-CD variables AWS_WEB_IDENTITY_TOKEN_FILE: /tmp/web-identity-token AWS_DEFAULT_REGION: eu-west-1 script: - *authenticate - aws s3 ls
Summary
In this tutorial, we showed you how to use a GitLab CI/CD job with a JSON web token (JWT) to retrieve temporary credentials from AWS without needing to store secrets. We configured OpenID Connect (OIDC) for ID federation between GitLab and AWS and created an AWS IAM role and policy.