Building Docker Images with Kaniko Pushing to Docker Registries

We can build a Docker image with kaniko and push it to Docker Hub or any other standard Docker registry.

Running kaniko from a Docker daemon does not provide much advantage over just running a docker build, but it is useful for testing or validation. It also helps understand how kaniko works and how it supports the different registries and authentication mechanisms.

git clone https://github.com/carlossg/kaniko-demo.git
cd kaniko-demo
# if you just want to test the build, no pushing
docker run \
    -v `pwd`:/workspace gcr.io/kaniko-project/executor:v1.0.0 \
    --no-push

Building by itself is not very useful, so we want to push to a remote Docker registry.

To push to DockerHub or any other username and password Docker registries we need to mount the Docker config.json file that contains the credentials. Caching will not work for DockerHub as it does not support repositories with more than 2 path sections (acme/myimage/cache), but it will work in Artifactory and maybe other registry implementations.

DOCKER_USERNAME=[...]
DOCKER_PASSWORD=[...]
AUTH=$(echo -n "${DOCKER_USERNAME}:${DOCKER_PASSWORD}" | base64)
cat << EOF > config.json
{
    "auths": {
        "https://index.docker.io/v1/": {
            "auth": "${AUTH}"
        }
    }
}
EOF
docker run \
    -v `pwd`/config.json:/kaniko/.docker/config.json:ro \
    -v `pwd`:/workspace \
    gcr.io/kaniko-project/executor:v1.0.0 \
    --destination $DOCKER_USERNAME/kaniko-demo:kaniko-docker

In Kubernetes

In Kubernetes we can manually create a pod that will do our Docker image build. We need to provide the build context, containing the same files that we would put in the directory used when building a Docker image with a Docker daemon. It should contain the Dockerfile and any other files used to build the image, ie. referenced in COPY commands.

As build context we can use multiple sources

  • GCS Bucket (as a tar.gz file)
    • gs://kaniko-bucket/path/to/context.tar.gz
  • S3 Bucket (as a tar.gz file) `
    • s3://kaniko-bucket/path/to/context.tar.gz
  • Azure Blob Storage (as a tar.gz file)
  • Local Directory, mounted in the /workspace dir as shown above
    • dir:///workspace
  • Git Repository
    • git://github.com/acme/myproject.git#refs/heads/mybranch

Depending on where we want to push to, we will also need to create the corresponding secrets and config maps.

We are going to show examples building from a git repository as it will be the most typical use case.

Deploying to Docker Hub or a Docker registry

We will need the Docker registry credentials in a config.json file, the same way that we need them to pull images from a private registry in Kubernetes.

DOCKER_USERNAME=[...]
DOCKER_PASSWORD=[...]
DOCKER_SERVER=https://index.docker.io/v1/
kubectl create secret docker-registry regcred \
    --docker-server=${DOCKER_SERVER} \
    --docker-username=${DOCKER_USERNAME} \
    --docker-password=${DOCKER_PASSWORD}

cat << EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: kaniko-docker
spec:
  restartPolicy: Never
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:v1.0.0
    imagePullPolicy: Always
    args: ["--dockerfile=Dockerfile",
            "--context=git://github.com/carlossg/kaniko-demo.git",
            "--destination=${DOCKER_USERNAME}/kaniko-demo"]
    volumeMounts:
      - name: docker-config
        mountPath: /kaniko/.docker
    resources:
      limits:
        cpu: 1
        memory: 1Gi
  volumes:
  - name: docker-config
    projected:
      sources:
      - secret:
          name: regcred
          items:
            - key: .dockerconfigjson
              path: config.json
EOF

Building Docker Images with Kaniko

This is the first post in a series about kaniko.

kaniko is a tool to build container images from a Dockerfile, similar to docker build, but without needing a Docker daemon. kaniko builds the images inside a container, executing the Dockerfile commands in userspace, so it allows us to build the images in standard Kubernetes clusters.

This means that in a containerized environment, be it a Kubernetes cluster, a Jenkins agent running in Docker, or any other container scheduler, we no longer need to use Docker in Docker nor do the build in the host system by mounting the Docker socket, simplifying and improving the security of container image builds.

Still, kaniko does not make it safe to run untrusted container image builds, but it relies on the security features of the container runtime. If you have a minimal base image that doesn’t require permissions to unpack, and your Dockerfile doesn’t execute any commands as the root user, you can run Kaniko without root permissions.

kaniko builds the container image inside a container, so it needs a way to get the build context (the directory where the Dockerfile and any other files that we want to copy into the container are) and to push the resulting image to a registry.

The build context can be a compressed tar in a Google Cloud Storage or AWS S3 bucket, a local directory inside the kaniko container, that we need to mount ourselves, or a git repository.

kaniko can be run in Docker, Kubernetes, Google Cloud Build (sending our image build to Google Cloud), or gVisor. gVisor is an OCI sandbox runtime that provides a virtualized container environment. It provides an additional security boundary for our container image builds.

Images can be pushed to any standard Docker registry but also Google GCR and AWS ECR are directly supported.

With Docker daemon image builds (docker build) we have caching. Each layer generated by RUN commands in the Dockerfile is kept and reused if the commands don’t change. In kaniko, because the image builds happen inside a container that is gone after the build we lose anything built locally. To solve this, kaniko can push these intermediate layers resulting from RUN commands to the remote registry when using the --cache flag.

In this series I will be covering using kaniko with several container registries.