Tag Archives: docker

Building Docker Images with Kaniko Pushing to Amazon Elastic Container Registry (ECR)

Posted on by Carlos Sanchez
1

To deploy to Amazon Elastic Container Registry (ECR) we can create a secret with AWS credentials or we can run with more secure IAM node instance roles.

When running on EKS we would have an EKS worker node IAM role (NodeInstanceRole), we need to add the IAM permissions to be able to pull and push from ECR. These permissions are grouped in the arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPowerUser policy, that can be attached to the node instance role.

When using instance roles we no longer need a secret, but we still need to configure kaniko to authenticate to AWS, by using a config.json containing just { "credsStore": "ecr-login" }, mounted in /kaniko/.docker/.

We also need to create the ECR repository beforehand, and, if using caching, another one for the cache.

ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
REPOSITORY=kanikorepo
REGION=us-east-1
# create the repository to push to
aws ecr create-repository --repository-name ${REPOSITORY}/kaniko-demo --region ${REGION}
# when using cache we need another repository for it
aws ecr create-repository --repository-name ${REPOSITORY}/kaniko-demo/cache --region ${REGION}

cat << EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: kaniko-eks
spec:
  restartPolicy: Never
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:v1.0.0
    imagePullPolicy: Always
    args: ["--dockerfile=Dockerfile",
            "--context=git://github.com/carlossg/kaniko-demo.git",
            "--destination=${ACCOUNT}.dkr.ecr.${REGION}.amazonaws.com/${REPOSITORY}/kaniko-demo:latest",
            "--cache=true"]
    volumeMounts:
      - name: docker-config
        mountPath: /kaniko/.docker/
    resources:
      limits:
        cpu: 1
        memory: 1Gi
  volumes:
    - name: docker-config
      configMap:
        name: docker-config
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: docker-config
data:
  config.json: |-
    { "credsStore": "ecr-login" }
EOF

Building Docker Images with Kaniko Pushing to Azure Container Registry (ACR)

Posted on by Carlos Sanchez
Reply

To push to Azure Container Registry (ACR) we can create an admin password for the ACR registry and use the standard Docker registry method or we can use a token. We use that token to craft both the standard Docker config file at /kaniko/.docker/config.json plus the ACR specific file used by the Docker ACR credential helper in /kaniko/.docker/acr/config.json. ACR does support caching and so it will push the intermediate layers to ${REGISTRY_NAME}.azurecr.io/kaniko-demo/cache:_some_large_uuid_ to be reused in subsequent builds.

RESOURCE_GROUP=kaniko-demo
REGISTRY_NAME=kaniko-demo
LOCATION=eastus
az login
# Create the resource group
az group create --name $RESOURCE_GROUP -l $LOCATION
# Create the ACR registry
az acr create --resource-group $RESOURCE_GROUP --name $REGISTRY_NAME --sku Basic
# If we want to enable password based authentication
# az acr update -n $REGISTRY_NAME --admin-enabled true

# Get the token
token=$(az acr login --name $REGISTRY_NAME --expose-token | jq -r '.accessToken')

And to build the image with kaniko

git clone https://github.com/carlossg/kaniko-demo.git
cd kaniko-demo

cat << EOF > config.json
{
  "auths": {
		"${REGISTRY_NAME}.azurecr.io": {}
	},
	"credsStore": "acr"
}
EOF
cat << EOF > config-acr.json
{
	"auths": {
		"${REGISTRY_NAME}.azurecr.io": {
			"identitytoken": "${token}"
		}
	}
}
EOF
docker run \
    -v `pwd`/config.json:/kaniko/.docker/config.json:ro \
    -v `pwd`/config-acr.json:/kaniko/.docker/acr/config.json:ro \
    -v `pwd`:/workspace \
    gcr.io/kaniko-project/executor:v1.0.0 \
    --destination $REGISTRY_NAME.azurecr.io/kaniko-demo:kaniko-docker \
    --cache

In Kubernetes

If you want to create a new Kubernetes cluster

az aks create --resource-group $RESOURCE_GROUP \
    --name AKSKanikoCluster \
    --generate-ssh-keys \
    --node-count 2
az aks get-credentials --resource-group $RESOURCE_GROUP --name AKSKanikoCluster --admin

In Kubernetes we need to mount the docker config file and the ACR config file with the token.

token=$(az acr login --name $REGISTRY_NAME --expose-token | jq -r '.accessToken')
cat << EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: kaniko-aks
spec:
  restartPolicy: Never
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:v1.0.0
    imagePullPolicy: Always
    args: ["--dockerfile=Dockerfile",
            "--context=git://github.com/carlossg/kaniko-demo.git",
            "--destination=${REGISTRY_NAME}.azurecr.io/kaniko-demo:latest",
            "--cache=true"]
    volumeMounts:
    - name: docker-config
      mountPath: /kaniko/.docker/
    - name: docker-acr-config
      mountPath: /kaniko/.docker/acr/
    resources:
      limits:
        cpu: 1
        memory: 1Gi
  volumes:
  - name: docker-config
    configMap:
      name: docker-config
  - name: docker-acr-config
    secret:
      name: kaniko-secret
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: docker-config
data:
  config.json: |-
    {
      "auths": {
    		"${REGISTRY_NAME}.azurecr.io": {}
    	},
    	"credsStore": "acr"
    }
---
apiVersion: v1
kind: Secret
metadata:
  name: kaniko-secret
stringData:
  config.json: |-
    {
    	"auths": {
    		"${REGISTRY_NAME}.azurecr.io": {
    			"identitytoken": "${token}"
    		}
    	}
    }
EOF

Building Docker Images with Kaniko Pushing to Google Container Registry (GCR)

Posted on by Carlos Sanchez
1

To push to Google Container Registry (GCR) we need to login to Google Cloud and mount our local $HOME/.config/gcloud containing our credentials into the kaniko container so it can push to GCR. GCR does support caching and so it will push the intermediate layers to gcr.io/$PROJECT/kaniko-demo/cache:_some_large_uuid_ to be reused in subsequent builds.

git clone https://github.com/carlossg/kaniko-demo.git
cd kaniko-demo

gcloud auth application-default login # get the Google Cloud credentials
PROJECT=$(gcloud config get-value project 2> /dev/null) # Your Google Cloud project id
docker run \
    -v $HOME/.config/gcloud:/root/.config/gcloud:ro \
    -v `pwd`:/workspace \
    gcr.io/kaniko-project/executor:v1.0.0 \
    --destination gcr.io/$PROJECT/kaniko-demo:kaniko-docker \
    --cache

kaniko can cache layers created by RUN commands in a remote repository. Before executing a command, kaniko checks the cache for the layer. If it exists, kaniko will pull and extract the cached layer instead of executing the command. If not, kaniko will execute the command and then push the newly created layer to the cache.

We can see in the output how kaniko uploads the intermediate layers to the cache.

INFO[0001] Resolved base name golang to build-env
INFO[0001] Retrieving image manifest golang
INFO[0001] Retrieving image golang
INFO[0004] Retrieving image manifest golang
INFO[0004] Retrieving image golang
INFO[0006] No base image, nothing to extract
INFO[0006] Built cross stage deps: map[0:[/src/bin/kaniko-demo]]
INFO[0006] Retrieving image manifest golang
INFO[0006] Retrieving image golang
INFO[0008] Retrieving image manifest golang
INFO[0008] Retrieving image golang
INFO[0010] Executing 0 build triggers
INFO[0010] Using files from context: [/workspace]
INFO[0011] Checking for cached layer gcr.io/api-project-642841493686/kaniko-demo/cache:0ab16b2e8a90e3820282b9f1ef6faf5b9a083e1fbfe8a445c36abcca00236b4f...
INFO[0011] No cached layer found for cmd RUN cd /src && make
INFO[0011] Unpacking rootfs as cmd ADD . /src requires it.
INFO[0051] Using files from context: [/workspace]
INFO[0051] ADD . /src
INFO[0051] Taking snapshot of files...
INFO[0051] RUN cd /src && make
INFO[0051] Taking snapshot of full filesystem...
INFO[0061] cmd: /bin/sh
INFO[0061] args: [-c cd /src && make]
INFO[0061] Running: [/bin/sh -c cd /src && make]
CGO_ENABLED=0 go build -ldflags '' -o bin/kaniko-demo main.go
INFO[0065] Taking snapshot of full filesystem...
INFO[0070] Pushing layer gcr.io/api-project-642841493686/kaniko-demo/cache:0ab16b2e8a90e3820282b9f1ef6faf5b9a083e1fbfe8a445c36abcca00236b4f to cache now
INFO[0144] Saving file src/bin/kaniko-demo for later use
INFO[0144] Deleting filesystem...
INFO[0145] No base image, nothing to extract
INFO[0145] Executing 0 build triggers
INFO[0145] cmd: EXPOSE
INFO[0145] Adding exposed port: 8080/tcp
INFO[0145] Checking for cached layer gcr.io/api-project-642841493686/kaniko-demo/cache:6ec16d3475b976bd7cbd41b74000c5d2543bdc2a35a635907415a0995784676d...
INFO[0146] No cached layer found for cmd COPY --from=build-env /src/bin/kaniko-demo /
INFO[0146] Unpacking rootfs as cmd COPY --from=build-env /src/bin/kaniko-demo / requires it.
INFO[0146] EXPOSE 8080
INFO[0146] cmd: EXPOSE
INFO[0146] Adding exposed port: 8080/tcp
INFO[0146] No files changed in this command, skipping snapshotting.
INFO[0146] ENTRYPOINT ["/kaniko-demo"]
INFO[0146] No files changed in this command, skipping snapshotting.
INFO[0146] COPY --from=build-env /src/bin/kaniko-demo /
INFO[0146] Taking snapshot of files...
INFO[0146] Pushing layer gcr.io/api-project-642841493686/kaniko-demo/cache:6ec16d3475b976bd7cbd41b74000c5d2543bdc2a35a635907415a0995784676d to cache now

If we run kaniko twice we can see how the cached layers are pulled instead of rebuilt.

INFO[0001] Resolved base name golang to build-env
INFO[0001] Retrieving image manifest golang
INFO[0001] Retrieving image golang
INFO[0004] Retrieving image manifest golang
INFO[0004] Retrieving image golang
INFO[0006] No base image, nothing to extract
INFO[0006] Built cross stage deps: map[0:[/src/bin/kaniko-demo]]
INFO[0006] Retrieving image manifest golang
INFO[0006] Retrieving image golang
INFO[0008] Retrieving image manifest golang
INFO[0008] Retrieving image golang
INFO[0010] Executing 0 build triggers
INFO[0010] Using files from context: [/workspace]
INFO[0010] Checking for cached layer gcr.io/api-project-642841493686/kaniko-demo/cache:0ab16b2e8a90e3820282b9f1ef6faf5b9a083e1fbfe8a445c36abcca00236b4f...
INFO[0012] Using caching version of cmd: RUN cd /src && make
INFO[0012] Unpacking rootfs as cmd ADD . /src requires it.
INFO[0049] Using files from context: [/workspace]
INFO[0049] ADD . /src
INFO[0049] Taking snapshot of files...
INFO[0049] RUN cd /src && make
INFO[0049] Found cached layer, extracting to filesystem
INFO[0051] Saving file src/bin/kaniko-demo for later use
INFO[0051] Deleting filesystem...
INFO[0052] No base image, nothing to extract
INFO[0052] Executing 0 build triggers
INFO[0052] cmd: EXPOSE
INFO[0052] Adding exposed port: 8080/tcp
INFO[0052] Checking for cached layer gcr.io/api-project-642841493686/kaniko-demo/cache:6ec16d3475b976bd7cbd41b74000c5d2543bdc2a35a635907415a0995784676d...
INFO[0054] Using caching version of cmd: COPY --from=build-env /src/bin/kaniko-demo /
INFO[0054] Skipping unpacking as no commands require it.
INFO[0054] EXPOSE 8080
INFO[0054] cmd: EXPOSE
INFO[0054] Adding exposed port: 8080/tcp
INFO[0054] No files changed in this command, skipping snapshotting.
INFO[0054] ENTRYPOINT ["/kaniko-demo"]
INFO[0054] No files changed in this command, skipping snapshotting.
INFO[0054] COPY --from=build-env /src/bin/kaniko-demo /
INFO[0054] Found cached layer, extracting to filesystem

In Kubernetes

To deploy to GCR we can use a service account and mount it as a Kubernetes secret, but when running on Google Kubernetes Engine (GKE) it is more convenient and safe to use the node pool service account.

When creating the GKE node pool the default configuration only includes read-only access to Storage API, and we need full access in order to push to GCR. This is something that we need to change under Add a new node pool – Security – Access scopes – Set access for each API – Storage – Full. Note that the scopes cannot be changed once the node pool has been created.

If the nodes have the correct service account with full storage access scope then we do not need to do anything extra on our kaniko pod, as it will be able to push to GCR just fine.

PROJECT=$(gcloud config get-value project 2> /dev/null)

cat << EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: kaniko-gcr
spec:
  restartPolicy: Never
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:v1.0.0
    imagePullPolicy: Always
    args: ["--dockerfile=Dockerfile",
            "--context=git://github.com/carlossg/kaniko-demo.git",
            "--destination=gcr.io/${PROJECT}/kaniko-demo:latest",
            "--cache=true"]
    resources:
      limits:
        cpu: 1
        memory: 1Gi
EOF

Building Docker Images with Kaniko Pushing to Docker Registries

Posted on by Carlos Sanchez
1

We can build a Docker image with kaniko and push it to Docker Hub or any other standard Docker registry.

Running kaniko from a Docker daemon does not provide much advantage over just running a docker build, but it is useful for testing or validation. It also helps understand how kaniko works and how it supports the different registries and authentication mechanisms.

git clone https://github.com/carlossg/kaniko-demo.git
cd kaniko-demo
# if you just want to test the build, no pushing
docker run \
    -v `pwd`:/workspace gcr.io/kaniko-project/executor:v1.0.0 \
    --no-push

Building by itself is not very useful, so we want to push to a remote Docker registry.

To push to DockerHub or any other username and password Docker registries we need to mount the Docker config.json file that contains the credentials. Caching will not work for DockerHub as it does not support repositories with more than 2 path sections (acme/myimage/cache), but it will work in Artifactory and maybe other registry implementations.

DOCKER_USERNAME=[...]
DOCKER_PASSWORD=[...]
AUTH=$(echo -n "${DOCKER_USERNAME}:${DOCKER_PASSWORD}" | base64)
cat << EOF > config.json
{
    "auths": {
        "https://index.docker.io/v1/": {
            "auth": "${AUTH}"
        }
    }
}
EOF
docker run \
    -v `pwd`/config.json:/kaniko/.docker/config.json:ro \
    -v `pwd`:/workspace \
    gcr.io/kaniko-project/executor:v1.0.0 \
    --destination $DOCKER_USERNAME/kaniko-demo:kaniko-docker

In Kubernetes

In Kubernetes we can manually create a pod that will do our Docker image build. We need to provide the build context, containing the same files that we would put in the directory used when building a Docker image with a Docker daemon. It should contain the Dockerfile and any other files used to build the image, ie. referenced in COPY commands.

As build context we can use multiple sources

  • GCS Bucket (as a tar.gz file)
    • gs://kaniko-bucket/path/to/context.tar.gz
  • S3 Bucket (as a tar.gz file) `
    • s3://kaniko-bucket/path/to/context.tar.gz
  • Azure Blob Storage (as a tar.gz file)
    • https://myaccount.blob.core.windows.net/container/path/to/context.tar.gz
  • Local Directory, mounted in the /workspace dir as shown above
    • dir:///workspace
  • Git Repository
    • git://github.com/acme/myproject.git#refs/heads/mybranch

Depending on where we want to push to, we will also need to create the corresponding secrets and config maps.

We are going to show examples building from a git repository as it will be the most typical use case.

Deploying to Docker Hub or a Docker registry

We will need the Docker registry credentials in a config.json file, the same way that we need them to pull images from a private registry in Kubernetes.

DOCKER_USERNAME=[...]
DOCKER_PASSWORD=[...]
DOCKER_SERVER=https://index.docker.io/v1/
kubectl create secret docker-registry regcred \
    --docker-server=${DOCKER_SERVER} \
    --docker-username=${DOCKER_USERNAME} \
    --docker-password=${DOCKER_PASSWORD}

cat << EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: kaniko-docker
spec:
  restartPolicy: Never
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:v1.0.0
    imagePullPolicy: Always
    args: ["--dockerfile=Dockerfile",
            "--context=git://github.com/carlossg/kaniko-demo.git",
            "--destination=${DOCKER_USERNAME}/kaniko-demo"]
    volumeMounts:
      - name: docker-config
        mountPath: /kaniko/.docker
    resources:
      limits:
        cpu: 1
        memory: 1Gi
  volumes:
  - name: docker-config
    projected:
      sources:
      - secret:
          name: regcred
          items:
            - key: .dockerconfigjson
              path: config.json
EOF

Building Docker Images with Kaniko

Posted on by Carlos Sanchez
Reply

This is the first post in a series about kaniko.

kaniko is a tool to build container images from a Dockerfile, similar to docker build, but without needing a Docker daemon. kaniko builds the images inside a container, executing the Dockerfile commands in userspace, so it allows us to build the images in standard Kubernetes clusters.

This means that in a containerized environment, be it a Kubernetes cluster, a Jenkins agent running in Docker, or any other container scheduler, we no longer need to use Docker in Docker nor do the build in the host system by mounting the Docker socket, simplifying and improving the security of container image builds.

Still, kaniko does not make it safe to run untrusted container image builds, but it relies on the security features of the container runtime. If you have a minimal base image that doesn’t require permissions to unpack, and your Dockerfile doesn’t execute any commands as the root user, you can run Kaniko without root permissions.

kaniko builds the container image inside a container, so it needs a way to get the build context (the directory where the Dockerfile and any other files that we want to copy into the container are) and to push the resulting image to a registry.

The build context can be a compressed tar in a Google Cloud Storage or AWS S3 bucket, a local directory inside the kaniko container, that we need to mount ourselves, or a git repository.

kaniko can be run in Docker, Kubernetes, Google Cloud Build (sending our image build to Google Cloud), or gVisor. gVisor is an OCI sandbox runtime that provides a virtualized container environment. It provides an additional security boundary for our container image builds.

Images can be pushed to any standard Docker registry but also Google GCR and AWS ECR are directly supported.

With Docker daemon image builds (docker build) we have caching. Each layer generated by RUN commands in the Dockerfile is kept and reused if the commands don’t change. In kaniko, because the image builds happen inside a container that is gone after the build we lose anything built locally. To solve this, kaniko can push these intermediate layers resulting from RUN commands to the remote registry when using the --cache flag.

In this series I will be covering using kaniko with several container registries.

Running a JVM in a Container Without Getting Killed II

Posted on by Carlos Sanchez
4

A follow up to Running a JVM in a Container Without Getting Killed

In Java 10 there is improved container integration.
No need to add extra flags, the JVM will use 1/4 of the container memory for heap.

$ docker run -m 1GB openjdk:10 java -XshowSettings:vm \
    -version
VM settings:
    Max. Heap Size (Estimated): 247.50M
    Using VM: OpenJDK 64-Bit Server VM

openjdk version "10.0.1" 2018-04-17
OpenJDK Runtime Environment (build 10.0.1+10-Debian-4)
OpenJDK 64-Bit Server VM (build 10.0.1+10-Debian-4, mixed mode)

Java 10 obsoletes the -XX:MaxRAM parameter, as the JVM will correctly detect the value.

You can still use the -XX:MaxRAMFraction=1 option to squeeze all the memory from the container.

$ docker run -m 1GB openjdk:10 java -XshowSettings:vm \
    -XX:MaxRAMFraction=1 -version
OpenJDK 64-Bit Server VM warning: Option MaxRAMFraction was deprecated in version 10.0 and will likely be removed in a future release.
VM settings:
    Max. Heap Size (Estimated): 989.88M
    Using VM: OpenJDK 64-Bit Server VM

openjdk version "10.0.1" 2018-04-17
OpenJDK Runtime Environment (build 10.0.1+10-Debian-4)
OpenJDK 64-Bit Server VM (build 10.0.1+10-Debian-4, mixed mode)

But it can be risky if your container uses off heap memory, as almost all the container memory is allocated to heap. You would have to either set -XX:MaxRAMFraction=2 and use only 50% of the container memory for heap, or resort to Xmx.

Building Docker Images without Docker

Posted on by Carlos Sanchez
Reply

Kaniko-Logo
Kaniko is a project launched by Google that allows building Dockerfiles without Docker or the Docker daemon.

Kaniko can be used inside Kubernetes to build a Docker image and push it to a registry, supporting Docker registry, Google Container Registry and AWS ECR, as well as any other registry supported by Docker credential helpers.

This solution is still not safe, as containers run as root, but it is way better than mounting the Docker socket and launching containers in the host. For one there are no leaked resources or containers running outside the scheduler.

To launch Kaniko from Jenkins in Kubernetes just need an agent template that uses the debug Kaniko image (just to have cat and nohup) and a Kubernetes secret with the image registry credentials, as shown in this example pipeline.

UPDATED: some changes needed for the latest Kaniko

/**
 * This pipeline will build and deploy a Docker image with Kaniko
 * https://github.com/GoogleContainerTools/kaniko
 * without needing a Docker host
 *
 * You need to create a jenkins-docker-cfg secret with your docker config
 * as described in
 * https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/#create-a-secret-in-the-cluster-that-holds-your-authorization-token
 */

 def label = "kaniko-${UUID.randomUUID().toString()}"

 podTemplate(name: 'kaniko', label: label, yaml: """
kind: Pod
metadata:
  name: kaniko
spec:
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:debug
    imagePullPolicy: Always
    command:
    - /busybox/cat
    tty: true
    volumeMounts:
      - name: jenkins-docker-cfg
        mountPath: /root
  volumes:
  - name: jenkins-docker-cfg
    projected:
      sources:
      - secret:
          name: regcred
          items:
            - key: .dockerconfigjson
              path: .docker/config.json
"""
  ) {

   node(label) {
     stage('Build with Kaniko') {
       git 'https://github.com/jenkinsci/docker-jnlp-slave.git'
       container(name: 'kaniko', shell: '/busybox/sh') {
           sh '''#!/busybox/sh
           /kaniko/executor -f `pwd`/Dockerfile -c `pwd` --insecure-skip-tls-verify --destination=mydockerregistry:5000/myorg/myimage
           '''
       }
     }
   }
 }

Pros:

  • No need to mount docker socket or have docker binary
  • No stray containers running outside of the scheduler

Cons:

  • Still not secure
  • Does not support the full Dockerfile syntax yet

Skaffold also has support for Kaniko, and can be used in your Jenkins X pipelines, which use Skaffold to abstract the image building.

Speaking Trips on DevOps, Kubernetes, Jenkins

Posted on by Carlos Sanchez
Reply

This 2nd half of the year speaking season is starting and you’ll find me speaking about DevOps, Kubernetes, Jenkins,… at

If you organize a conference and would like me to give a talk in 2018 you can find me @csanchez.

Screen Shot 2017-08-24 at 17.07.45.png

Running a JVM in a Container Without Getting Killed

Posted on by Carlos Sanchez
24

No pun intended

The JDK 8u131 has backported a nice feature in JDK 9, which is the ability of the JVM to detect how much memory is available when running inside a Docker container.

I have talked multiple times about the problems running a JVM inside a container, how it will default in most cases to a max heap of 1/4 of the host memory, not the container.

For example in my machine

$ docker run -m 100MB openjdk:8u121 java -XshowSettings:vm -version
VM settings:
    Max. Heap Size (Estimated): 444.50M
    Ergonomics Machine Class: server
    Using VM: OpenJDK 64-Bit Server VM

Wait, WAT? I set a container memory of 100MB and my JVM sets a max heap of 444M ? It is very likely that it is going to cause the Kernel to kill my JVM at some point.

Let’s try the JDK 8u131 with the experimental option -XX:+UseCGroupMemoryLimitForHeap

$ docker run -m 100MB openjdk:8u131 java \
  -XX:+UnlockExperimentalVMOptions \
  -XX:+UseCGroupMemoryLimitForHeap \
  -XshowSettings:vm -version
VM settings:
    Max. Heap Size (Estimated): 44.50M
    Ergonomics Machine Class: server
    Using VM: OpenJDK 64-Bit Server VM

Ok this makes more sense, the JVM was able to detect the container has only 100MB and set the max heap to 44M.

Let’s try in a bigger container

$ docker run -m 1GB openjdk:8u131 java \
  -XX:+UnlockExperimentalVMOptions \
  -XX:+UseCGroupMemoryLimitForHeap \
  -XshowSettings:vm -version
VM settings:
    Max. Heap Size (Estimated): 228.00M
    Ergonomics Machine Class: server
    Using VM: OpenJDK 64-Bit Server VM

Mmm, now the container has 1GB but JVM is only using 228M as max heap. Can we optimize this even more, given that nothing else other than the JVM is running in the container? Yes we can!

$ docker run -m 1GB openjdk:8u131 java \
  -XX:+UnlockExperimentalVMOptions \
  -XX:+UseCGroupMemoryLimitForHeap \
  -XX:MaxRAMFraction=1 -XshowSettings:vm -version
VM settings:
    Max. Heap Size (Estimated): 910.50M
    Ergonomics Machine Class: server
    Using VM: OpenJDK 64-Bit Server VM

Using -XX:MaxRAMFraction we are telling the JVM to use available memory/MaxRAMFraction as max heap. Using -XX:MaxRAMFraction=1 we are using almost all the available memory as max heap.

UPDATE: follow up for Java 10+ at Running a JVM in a Container Without Getting Killed II

Docker Registry with Let’s Encrypt Certificate

Posted on by Carlos Sanchez
1

A one-liner to run a SSL Docker registry generating a Let’s Encrypt certificate.

This command will create a registry proxying the Docker hub, caching the images in a registry volume.

LetsEncrypt certificate will be auto generated and stored in the host dir as letsencrypt.json. You could also use a Docker volume to store it.

In order for the certificate generation to work the registry needs to be accessible from the internet in port 443. After the certificate is generated that’s no longer needed.

docker run -d -p 443:5000 --name registry \
  -v `pwd`:/etc/docker/registry/ \
  -v registry:/var/lib/registry \
  -e REGISTRY_HTTP_ADDR=0.0.0.0:5000 \
  -e REGISTRY_HTTP_HOST=https://docker.example.com \
  -e REGISTRY_HTTP_TLS_LETSENCRYPT_CACHEFILE=/etc/docker/registry/letsencrypt.json \
  -e REGISTRY_HTTP_TLS_LETSENCRYPT_EMAIL=admin@example.com \
  -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io \
  registry:2

You can also create a config.yml in this dir and run the registry using the file instead of environment variables

version: 0.1
storage:
  filesystem:
http:
  addr: 0.0.0.0:5000
  host: https://docker.example.com
  tls:
    letsencrypt:
      cachefile: /etc/docker/registry/letsencrypt.json
      email: admin@example.com
proxy:
  remoteurl: https://registry-1.docker.io

Then run

docker run -d -p 443:5000 --name registry \
  -v `pwd`:/etc/docker/registry/ \
  -v registry:/var/lib/registry \
  registry:2

If you want to use this as a remote repository and not just for proxying, remove the proxy entry in the configuration