If you are still using AWS CodeBuild managed service for testing and debugging, then you have the scope to save cost and expedite the build-test-debug cycle loop using AWS CodeBuild local agent. Troubleshooting and fixing issues in BuildSpec file is so easy on the local development machine.
In this post, we'll see how to install and configure the AWS CodeBuild local agent and build a sample Java project.
Youtube Shorts:
Installation:
Clone the AWS CodeBuild Local docker image GitHub repo.
abhijit@AwsJunkie:~$ git clone https://github.com/aws/aws-codebuild-docker-images.git
Cloning into 'aws-codebuild-docker-images'...
remote: Enumerating objects: 2844, done.
remote: Counting objects: 100% (190/190), done.
remote: Compressing objects: 100% (82/82), done.
remote: Total 2844 (delta 77), reused 170 (delta 74), pack-reused 2654
Receiving objects: 100% (2844/2844), 748.04 KiB | 1.42 MiB/s, done.
Resolving deltas: 100% (1328/1328), done.
Based on the requirement of runtimes, identify the docker image provided by AWS CodeBuild. For the sample Java project I have chosen Ubuntu 22.04 (aws/codebuild/standard:7.0) which supports java corretto17.
Change directory to the location contains AWS CodeBuild Standard 7.0 Dockerfile with Ubuntu base.
abhijit@AwsJunkie:~$ cd aws-codebuild-docker-images/ubuntu/standard/7.0
abhijit@AwsJunkie:~/aws-codebuild-docker-images/ubuntu/standard/7.0$
Run docker build. It will take some time.
abhijit@AwsJunkie:~/aws-codebuild-docker-images/ubuntu/standard/7.0$ docker build -t aws/codebuild/standard:7.0 .
docker build -t aws/codebuild/standard:7.0 .
[+] Building 66.4s (5/46)
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 18.50kB 0.0s
=> [internal] load metadata for public.ecr.aws/ubuntu/ubuntu:22.04 1.8s
=> [internal] load build context 0.1s
=> => transferring context: 14.17kB 0.0s
=> [core 1/3] FROM public.ecr.aws/ubuntu/ubuntu:22.04@sha256:5f6b39af2870c0357181b0625ee9f524af2085965151b63435e 3.4s
=> => resolve public.ecr.aws/ubuntu/ubuntu:22.04@sha256:5f6b39af2870c0357181b0625ee9f524af2085965151b63435e17971 0.1s
=> => sha256:3153aa388d026c26a2235e1ed0163e350e451f41a8a313e1804d7e1afb857ab4 29.53MB / 29.53MB 2.4s
=> => sha256:5f6b39af2870c0357181b0625ee9f524af2085965151b63435e17971fcb4f136 1.19kB / 1.19kB 0.0s
=> => sha256:b060fffe8e1561c9c3e6dea6db487b900100fc26830b9ea2ec966c151ab4c020 424B / 424B 0.0s
=> => sha256:5a81c4b8502e4979e75bd8f91343b95b0d695ab67f241dbed0d1530a35bde1eb 2.30kB / 2.30kB 0.0s
=> => extracting sha256:3153aa388d026c26a2235e1ed0163e350e451f41a8a313e1804d7e1afb857ab4
We should see aws/codebuild/standard docker image in the image list.
abhijit@AwsJunkie:~/aws-codebuild-docker-images/ubuntu/standard/7.0$ docker images | grep codebuild
aws/codebuild/standard 7.0 b7ab77099289 2 days ago 5.89GB
Change directory to local_builds
abhijit@AwsJunkie:~$ cd ~/aws-codebuild-docker-images/local_builds/
abhijit@AwsJunkie:~/aws-codebuild-docker-images/local_builds$
Give execute permission to codebuild_build.sh
abhijit@AwsJunkie:~/aws-codebuild-docker-images/local_builds$ chmod +x codebuild_build.sh
Verify:
Sample SpringBoot java project contains the following buildspec.yml file.
version: 0.2
phases: build: commands: - echo AWS CodeBuild started locally ondate- echo Strting Maven Build for a dummy Java SpringBoot project - mvn install artifacts: files: - target/aws-local-codebuild-1.0.jar
Run ./codebuild_build.sh -i <container-image-name> -a <artifact-output-directory> -s <project-code-directory> to run AWS CodeBuild local agent.
abhijit@AwsJunkie:~/aws-codebuild-docker-images/local_builds$ lsAwsLocalCodeBuild NOTICE.txt THIRD-PARTY.txt codebuild_build.sh LICENSE.txt README.md abhijit@AwsJunkie:~/aws-codebuild-docker-images/local_builds$ ./codebuild_build.sh -i aws/codebuild/standard:7.0 -a artifacts -s AwsLocalCodeBuild Build Command:
docker run -it -v /var/run/docker.sock:/var/run/docker.sock -e “IMAGE_NAME=aws/codebuild/standard:7.0” -e “ARTIFACTS=/home/abhijit/aws-codebuild-docker-images/local_builds/artifacts” -e “SOURCE=/home/abhijit/aws-codebuild-docker-images/local_builds/AwsLocalCodeBuild” -e “INITIATOR=abhijit” public.ecr.aws/codebuild/local-builds:latest
Removing agent-resources_build_1 … done Removing agent-resources_agent_1 … done Removing network agent-resources_default Removing volume agent-resources_source_volume Removing volume agent-resources_user_volume Creating network “agent-resources_default” with the default driver Creating volume “agent-resources_source_volume” with local driver Creating volume “agent-resources_user_volume” with local driver Creating agent-resources_agent_1 … done Creating agent-resources_build_1 … done Attaching to agent-resources_agent_1, agent-resources_build_1 agent_1 | [Container] 2023/07/18 00:50:45 Waiting for agent ping agent_1 | [Container] 2023/07/18 00:50:45 Waiting for DOWNLOAD_SOURCE agent_1 | [Container] 2023/07/18 00:50:46 Phase is DOWNLOAD_SOURCE agent_1 | [Container] 2023/07/18 00:50:46 CODEBUILD_SRC_DIR=/codebuild/output/src117800177/src agent_1 | [Container] 2023/07/18 00:50:46 yamlDoc agent_1 | [Container] 2023/07/18 00:50:46 YAML location is /codebuild/output/srcDownload/src/buildspec.yml agent_1 | [Container] 2023/07/18 00:50:46 Processing environment variables agent_1 | [Container] 2023/07/18 00:50:46 Moving to directory /codebuild/output/src117800177/src agent_1 | [Container] 2023/07/18 00:50:46 Registering with agent agent_1 | [Container] 2023/07/18 00:50:46 Phases found in YAML: 1 agent_1 | [Container] 2023/07/18 00:50:46 BUILD: 3 commands agent_1 | [Container] 2023/07/18 00:50:46 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED agent_1 | [Container] 2023/07/18 00:50:46 Phase context status code: Message: agent_1 | [Container] 2023/07/18 00:50:46 Entering phase INSTALL agent_1 | [Container] 2023/07/18 00:50:46 Phase complete: INSTALL State: SUCCEEDED agent_1 | [Container] 2023/07/18 00:50:46 Phase context status code: Message: agent_1 | [Container] 2023/07/18 00:50:46 Entering phase PRE_BUILD agent_1 | [Container] 2023/07/18 00:50:46 Phase complete: PRE_BUILD State: SUCCEEDED agent_1 | [Container] 2023/07/18 00:50:46 Phase context status code: Message: agent_1 | [Container] 2023/07/18 00:50:46 Entering phase BUILD agent_1 | [Container] 2023/07/18 00:50:46 Running command echo AWS CodeBuild started locally ondateagent_1 | AWS CodeBuild started locally on Tue Jul 18 00:50:46 UTC 2023 agent_1 | agent_1 | [Container] 2023/07/18 00:50:46 Running command echo Strting Maven Build for a dummy Java SpringBoot project agent_1 | Strting Maven Build for a dummy Java SpringBoot project agent_1 | agent_1 | [Container] 2023/07/18 00:50:46 Running command mvn install agent_1 | [INFO] Scanning for projects… agent_1 | Downloading from central: https://repo.maven.apache.org/maven2/org/springframework/boot/spring-boot-starter-parent/3.1.1/spring-boot-starter-parent-3.1.1.pom : : agent_1 | [INFO] Replacing main artifact /codebuild/output/srcDownload/src/target/aws-local-codebuild-1.0.jar with repackaged archive, adding nested dependencies in BOOT-INF/. agent_1 | [INFO] The original artifact has been renamed to /codebuild/output/srcDownload/src/target/aws-local-codebuild-1.0.jar.original agent_1 | [INFO] agent_1 | [INFO] --- install:3.1.1:install (default-install) @ aws-local-codebuild --- agent_1 | [INFO] Installing /codebuild/output/srcDownload/src/pom.xml to /root/.m2/repository/com/awsjunkie/aws-local-codebuild/1.0/aws-local-codebuild-1.0.pom agent_1 | [INFO] Installing /codebuild/output/srcDownload/src/target/aws-local-codebuild-1.0.jar to /root/.m2/repository/com/awsjunkie/aws-local-codebuild/1.0/aws-local-codebuild-1.0.jar agent_1 | [INFO] ------------------------------------------------------------------------ agent_1 | [INFO] BUILD SUCCESS agent_1 | [INFO] ------------------------------------------------------------------------ agent_1 | [INFO] Total time: 01:02 min agent_1 | [INFO] Finished at: 2023-07-18T00:51:51Z agent_1 | [INFO] ------------------------------------------------------------------------ agent_1 | agent_1 | [Container] 2023/07/18 00:51:51 Phase complete: BUILD State: SUCCEEDED agent_1 | [Container] 2023/07/18 00:51:51 Phase context status code: Message: agent_1 | [Container] 2023/07/18 00:51:51 Entering phase POST_BUILD agent_1 | [Container] 2023/07/18 00:51:51 Phase complete: POST_BUILD State: SUCCEEDED agent_1 | [Container] 2023/07/18 00:51:51 Phase context status code: Message: agent_1 | [Container] 2023/07/18 00:51:51 Expanding base directory path: . agent_1 | [Container] 2023/07/18 00:51:51 Assembling file list agent_1 | [Container] 2023/07/18 00:51:51 Expanding . agent_1 | [Container] 2023/07/18 00:51:51 Expanding file paths for base directory . agent_1 | [Container] 2023/07/18 00:51:51 Assembling file list agent_1 | [Container] 2023/07/18 00:51:51 Expanding target/aws-local-codebuild-1.0.jar agent_1 | [Container] 2023/07/18 00:51:51 Found 1 file(s) agent_1 | [Container] 2023/07/18 00:51:52 Preparing to copy secondary artifacts agent_1 | [Container] 2023/07/18 00:51:52 No secondary artifacts defined in buildspec agent_1 | [Container] 2023/07/18 00:51:52 Phase complete: UPLOAD_ARTIFACTS State: SUCCEEDED agent_1 | [Container] 2023/07/18 00:51:52 Phase context status code: Message: agent-resources_build_1 exited with code 0 Aborting on container exit…
Soon, we'll see "Phase complete: BUILD State: SUCCEEDED" and "Phase complete: UPLOAD_ARTIFACTS State: SUCCEEDED".
Let's verify the artifact output directory to confirm.
abhijit@AwsJunkie:~/aws-codebuild-docker-images/local_builds$ ls artifacts/
artifacts.zip
Command Usage:
abhijit@AwsJunkie:~/aws-codebuild-docker-images/local_builds$ ./codebuild_build.sh -h
usage: codebuild_build.sh [-i image_name] [-a artifact_output_directory] [options] Required: -i Used to specify the customer build container image. -a Used to specify an artifact output directory. Options: -l IMAGE Used to override the default local agent image. -r Used to specify a report output directory. -s Used to specify source information. Defaults to the current working directory for primary source. * First (-s) is for primary source * Use additional (-s) in <sourceIdentifier>:<sourceLocation> format for secondary source * For sourceIdentifier, use a value that is fewer than 128 characters and contains only alphanumeric characters and underscores -c Use the AWS configuration and credentials from your local host. This includes ~/.aws and any AWS_* environment variables. -p Used to specify the AWS CLI Profile. -b FILE Used to specify a buildspec override file. Defaults to buildspec.yml in the source directory. -m Used to mount the source directory to the customer build container directly. -d Used to run the build container in docker privileged mode. -e FILE Used to specify a file containing environment variables. (-e) File format expectations: * Each line is in VAR=VAL format * Lines beginning with # are processed as comments and ignored * Blank lines are ignored * File can be of type .env or .txt * There is no special handling of quotation marks, meaning they will be part of the VAL
![AWS CodeBuild - GitHub Actions: COMMAND_EXECUTION_ERROR Message: Error while executing command | [error]Docker pull failed with exit code 1](https://cdn.awsjunkie.com/content/2023/07/aws-codebuild-github-action-docker-daemon-error.jpg)