If you are still using AWS CodeBuild managed service for testing and debugging, then you have the scope to save cost and expedite the build-test-debug cycle loop using AWS CodeBuild local agent. Troubleshooting and fixing issues in BuildSpec file is so easy on the local development machine.
In this post, we'll see how to install and configure the AWS CodeBuild local agent and build a sample Java project.
Youtube Shorts:
Installation:
Clone the AWS CodeBuild Local docker image GitHub repo.
abhijit@AwsJunkie:~$ git clone https://github.com/aws/aws-codebuild-docker-images.git
Cloning into 'aws-codebuild-docker-images'...
remote: Enumerating objects: 2844, done.
remote: Counting objects: 100% (190/190), done.
remote: Compressing objects: 100% (82/82), done.
remote: Total 2844 (delta 77), reused 170 (delta 74), pack-reused 2654
Receiving objects: 100% (2844/2844), 748.04 KiB | 1.42 MiB/s, done.
Resolving deltas: 100% (1328/1328), done.
Based on the requirement of runtimes, identify the docker image provided by AWS CodeBuild. For the sample Java project I have chosen Ubuntu 22.04 (aws/codebuild/standard:7.0) which supports java corretto17.
Change directory to the location contains AWS CodeBuild Standard 7.0 Dockerfile with Ubuntu base.
abhijit@AwsJunkie:~$ cd aws-codebuild-docker-images/ubuntu/standard/7.0
abhijit@AwsJunkie:~/aws-codebuild-docker-images/ubuntu/standard/7.0$
Run docker build. It will take some time.
abhijit@AwsJunkie:~/aws-codebuild-docker-images/ubuntu/standard/7.0$ docker build -t aws/codebuild/standard:7.0 .
docker build -t aws/codebuild/standard:7.0 .
[+] Building 66.4s (5/46)
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 18.50kB 0.0s
=> [internal] load metadata for public.ecr.aws/ubuntu/ubuntu:22.04 1.8s
=> [internal] load build context 0.1s
=> => transferring context: 14.17kB 0.0s
=> [core 1/3] FROM public.ecr.aws/ubuntu/ubuntu:22.04@sha256:5f6b39af2870c0357181b0625ee9f524af2085965151b63435e 3.4s
=> => resolve public.ecr.aws/ubuntu/ubuntu:22.04@sha256:5f6b39af2870c0357181b0625ee9f524af2085965151b63435e17971 0.1s
=> => sha256:3153aa388d026c26a2235e1ed0163e350e451f41a8a313e1804d7e1afb857ab4 29.53MB / 29.53MB 2.4s
=> => sha256:5f6b39af2870c0357181b0625ee9f524af2085965151b63435e17971fcb4f136 1.19kB / 1.19kB 0.0s
=> => sha256:b060fffe8e1561c9c3e6dea6db487b900100fc26830b9ea2ec966c151ab4c020 424B / 424B 0.0s
=> => sha256:5a81c4b8502e4979e75bd8f91343b95b0d695ab67f241dbed0d1530a35bde1eb 2.30kB / 2.30kB 0.0s
=> => extracting sha256:3153aa388d026c26a2235e1ed0163e350e451f41a8a313e1804d7e1afb857ab4
We should see aws/codebuild/standard docker image in the image list.
abhijit@AwsJunkie:~/aws-codebuild-docker-images/ubuntu/standard/7.0$ docker images | grep codebuild
aws/codebuild/standard 7.0 b7ab77099289 2 days ago 5.89GB
Change directory to local_builds
abhijit@AwsJunkie:~$ cd ~/aws-codebuild-docker-images/local_builds/
abhijit@AwsJunkie:~/aws-codebuild-docker-images/local_builds$
Give execute permission to codebuild_build.sh
abhijit@AwsJunkie:~/aws-codebuild-docker-images/local_builds$ chmod +x codebuild_build.sh
Verify:
Sample SpringBoot java project contains the following buildspec.yml file.
version: 0.2
phases:
build:
commands:
- echo AWS CodeBuild started locally on `date`
- echo Strting Maven Build for a dummy Java SpringBoot project
- mvn install
artifacts:
files:
- target/aws-local-codebuild-1.0.jar
Run ./codebuild_build.sh -i <container-image-name> -a <artifact-output-directory> -s <project-code-directory> to run AWS CodeBuild local agent.
abhijit@AwsJunkie:~/aws-codebuild-docker-images/local_builds$ ls
AwsLocalCodeBuild NOTICE.txt THIRD-PARTY.txt codebuild_build.sh
LICENSE.txt README.md
abhijit@AwsJunkie:~/aws-codebuild-docker-images/local_builds$ ./codebuild_build.sh -i aws/codebuild/standard:7.0 -a artifacts -s AwsLocalCodeBuild
Build Command:
docker run -it -v /var/run/docker.sock:/var/run/docker.sock -e "IMAGE_NAME=aws/codebuild/standard:7.0" -e "ARTIFACTS=/home/abhijit/aws-codebuild-docker-images/local_builds/artifacts" -e "SOURCE=/home/abhijit/aws-codebuild-docker-images/local_builds/AwsLocalCodeBuild" -e "INITIATOR=abhijit" public.ecr.aws/codebuild/local-builds:latest
Removing agent-resources_build_1 ... done
Removing agent-resources_agent_1 ... done
Removing network agent-resources_default
Removing volume agent-resources_source_volume
Removing volume agent-resources_user_volume
Creating network "agent-resources_default" with the default driver
Creating volume "agent-resources_source_volume" with local driver
Creating volume "agent-resources_user_volume" with local driver
Creating agent-resources_agent_1 ... done
Creating agent-resources_build_1 ... done
Attaching to agent-resources_agent_1, agent-resources_build_1
agent_1 | [Container] 2023/07/18 00:50:45 Waiting for agent ping
agent_1 | [Container] 2023/07/18 00:50:45 Waiting for DOWNLOAD_SOURCE
agent_1 | [Container] 2023/07/18 00:50:46 Phase is DOWNLOAD_SOURCE
agent_1 | [Container] 2023/07/18 00:50:46 CODEBUILD_SRC_DIR=/codebuild/output/src117800177/src
agent_1 | [Container] 2023/07/18 00:50:46 yamlDoc
agent_1 | [Container] 2023/07/18 00:50:46 YAML location is /codebuild/output/srcDownload/src/buildspec.yml
agent_1 | [Container] 2023/07/18 00:50:46 Processing environment variables
agent_1 | [Container] 2023/07/18 00:50:46 Moving to directory /codebuild/output/src117800177/src
agent_1 | [Container] 2023/07/18 00:50:46 Registering with agent
agent_1 | [Container] 2023/07/18 00:50:46 Phases found in YAML: 1
agent_1 | [Container] 2023/07/18 00:50:46 BUILD: 3 commands
agent_1 | [Container] 2023/07/18 00:50:46 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED
agent_1 | [Container] 2023/07/18 00:50:46 Phase context status code: Message:
agent_1 | [Container] 2023/07/18 00:50:46 Entering phase INSTALL
agent_1 | [Container] 2023/07/18 00:50:46 Phase complete: INSTALL State: SUCCEEDED
agent_1 | [Container] 2023/07/18 00:50:46 Phase context status code: Message:
agent_1 | [Container] 2023/07/18 00:50:46 Entering phase PRE_BUILD
agent_1 | [Container] 2023/07/18 00:50:46 Phase complete: PRE_BUILD State: SUCCEEDED
agent_1 | [Container] 2023/07/18 00:50:46 Phase context status code: Message:
agent_1 | [Container] 2023/07/18 00:50:46 Entering phase BUILD
agent_1 | [Container] 2023/07/18 00:50:46 Running command echo AWS CodeBuild started locally on `date`
agent_1 | AWS CodeBuild started locally on Tue Jul 18 00:50:46 UTC 2023
agent_1 |
agent_1 | [Container] 2023/07/18 00:50:46 Running command echo Strting Maven Build for a dummy Java SpringBoot project
agent_1 | Strting Maven Build for a dummy Java SpringBoot project
agent_1 |
agent_1 | [Container] 2023/07/18 00:50:46 Running command mvn install
agent_1 | [INFO] Scanning for projects...
agent_1 | Downloading from central: https://repo.maven.apache.org/maven2/org/springframework/boot/spring-boot-starter-parent/3.1.1/spring-boot-starter-parent-3.1.1.pom
:
:
agent_1 | [INFO] Replacing main artifact /codebuild/output/srcDownload/src/target/aws-local-codebuild-1.0.jar with repackaged archive, adding nested dependencies in BOOT-INF/.
agent_1 | [INFO] The original artifact has been renamed to /codebuild/output/srcDownload/src/target/aws-local-codebuild-1.0.jar.original
agent_1 | [INFO]
agent_1 | [INFO] --- install:3.1.1:install (default-install) @ aws-local-codebuild ---
agent_1 | [INFO] Installing /codebuild/output/srcDownload/src/pom.xml to /root/.m2/repository/com/awsjunkie/aws-local-codebuild/1.0/aws-local-codebuild-1.0.pom
agent_1 | [INFO] Installing /codebuild/output/srcDownload/src/target/aws-local-codebuild-1.0.jar to /root/.m2/repository/com/awsjunkie/aws-local-codebuild/1.0/aws-local-codebuild-1.0.jar
agent_1 | [INFO] ------------------------------------------------------------------------
agent_1 | [INFO] BUILD SUCCESS
agent_1 | [INFO] ------------------------------------------------------------------------
agent_1 | [INFO] Total time: 01:02 min
agent_1 | [INFO] Finished at: 2023-07-18T00:51:51Z
agent_1 | [INFO] ------------------------------------------------------------------------
agent_1 |
agent_1 | [Container] 2023/07/18 00:51:51 Phase complete: BUILD State: SUCCEEDED
agent_1 | [Container] 2023/07/18 00:51:51 Phase context status code: Message:
agent_1 | [Container] 2023/07/18 00:51:51 Entering phase POST_BUILD
agent_1 | [Container] 2023/07/18 00:51:51 Phase complete: POST_BUILD State: SUCCEEDED
agent_1 | [Container] 2023/07/18 00:51:51 Phase context status code: Message:
agent_1 | [Container] 2023/07/18 00:51:51 Expanding base directory path: .
agent_1 | [Container] 2023/07/18 00:51:51 Assembling file list
agent_1 | [Container] 2023/07/18 00:51:51 Expanding .
agent_1 | [Container] 2023/07/18 00:51:51 Expanding file paths for base directory .
agent_1 | [Container] 2023/07/18 00:51:51 Assembling file list
agent_1 | [Container] 2023/07/18 00:51:51 Expanding target/aws-local-codebuild-1.0.jar
agent_1 | [Container] 2023/07/18 00:51:51 Found 1 file(s)
agent_1 | [Container] 2023/07/18 00:51:52 Preparing to copy secondary artifacts
agent_1 | [Container] 2023/07/18 00:51:52 No secondary artifacts defined in buildspec
agent_1 | [Container] 2023/07/18 00:51:52 Phase complete: UPLOAD_ARTIFACTS State: SUCCEEDED
agent_1 | [Container] 2023/07/18 00:51:52 Phase context status code: Message:
agent-resources_build_1 exited with code 0
Aborting on container exit...
Soon, we'll see "Phase complete: BUILD State: SUCCEEDED" and "Phase complete: UPLOAD_ARTIFACTS State: SUCCEEDED".
Let's verify the artifact output directory to confirm.
abhijit@AwsJunkie:~/aws-codebuild-docker-images/local_builds$ ls artifacts/
artifacts.zip
Command Usage:
abhijit@AwsJunkie:~/aws-codebuild-docker-images/local_builds$ ./codebuild_build.sh -h
usage: codebuild_build.sh [-i image_name] [-a artifact_output_directory] [options]
Required:
-i Used to specify the customer build container image.
-a Used to specify an artifact output directory.
Options:
-l IMAGE Used to override the default local agent image.
-r Used to specify a report output directory.
-s Used to specify source information. Defaults to the current working directory for primary source.
* First (-s) is for primary source
* Use additional (-s) in <sourceIdentifier>:<sourceLocation> format for secondary source
* For sourceIdentifier, use a value that is fewer than 128 characters and contains only alphanumeric characters and underscores
-c Use the AWS configuration and credentials from your local host. This includes ~/.aws and any AWS_* environment variables.
-p Used to specify the AWS CLI Profile.
-b FILE Used to specify a buildspec override file. Defaults to buildspec.yml in the source directory.
-m Used to mount the source directory to the customer build container directly.
-d Used to run the build container in docker privileged mode.
-e FILE Used to specify a file containing environment variables.
(-e) File format expectations:
* Each line is in VAR=VAL format
* Lines beginning with # are processed as comments and ignored
* Blank lines are ignored
* File can be of type .env or .txt
* There is no special handling of quotation marks, meaning they will be part of the VAL
![[new capability] Amazon EC2 supports disabling Amazon Machine Image (or AMI)](https://cdn.awsjunkie.com/content/2023/10/disable-ami-new-amazon-ec2-capability.jpg)
