Running a JVM in a Container Without Getting Killed

No pun intended

The JDK 8u131 has backported a nice feature in JDK 9, which is the ability of the JVM to detect how much memory is available when running inside a Docker container.

I have talked multiple times about the problems running a JVM inside a container, how it will default in most cases to a max heap of 1/4 of the host memory, not the container.

For example in my machine

$ docker run -m 100MB openjdk:8u121 java -XshowSettings:vm -version
VM settings:
    Max. Heap Size (Estimated): 444.50M
    Ergonomics Machine Class: server
    Using VM: OpenJDK 64-Bit Server VM

Wait, WAT? I set a container memory of 100MB and my JVM sets a max heap of 444M ? It is very likely that it is going to cause the Kernel to kill my JVM at some point.

Let’s try the JDK 8u131 with the experimental option -XX:+UseCGroupMemoryLimitForHeap

$ docker run -m 100MB openjdk:8u131 java \
  -XX:+UnlockExperimentalVMOptions \
  -XX:+UseCGroupMemoryLimitForHeap \
  -XshowSettings:vm -version
VM settings:
    Max. Heap Size (Estimated): 44.50M
    Ergonomics Machine Class: server
    Using VM: OpenJDK 64-Bit Server VM

Ok this makes more sense, the JVM was able to detect the container has only 100MB and set the max heap to 44M.

Let’s try in a bigger container

$ docker run -m 1GB openjdk:8u131 java \
  -XX:+UnlockExperimentalVMOptions \
  -XX:+UseCGroupMemoryLimitForHeap \
  -XshowSettings:vm -version
VM settings:
    Max. Heap Size (Estimated): 228.00M
    Ergonomics Machine Class: server
    Using VM: OpenJDK 64-Bit Server VM

Mmm, now the container has 1GB but JVM is only using 228M as max heap. Can we optimize this even more, given that nothing else other than the JVM is running in the container? Yes we can!

$ docker run -m 1GB openjdk:8u131 java \
  -XX:+UnlockExperimentalVMOptions \
  -XX:+UseCGroupMemoryLimitForHeap \
  -XX:MaxRAMFraction=1 -XshowSettings:vm -version
VM settings:
    Max. Heap Size (Estimated): 910.50M
    Ergonomics Machine Class: server
    Using VM: OpenJDK 64-Bit Server VM

Using -XX:MaxRAMFraction we are telling the JVM to use available memory/MaxRAMFraction as max heap. Using -XX:MaxRAMFraction=1 we are using almost all the available memory as max heap.

UPDATE: follow up for Java 10+ at Running a JVM in a Container Without Getting Killed II

24 thoughts on “Running a JVM in a Container Without Getting Killed”

won’t having your max heap take the total ram still usually result in your process getting killed because of other parts of the memory that get used like “Metaspace”?

Reply ↓

Carlos Sanchez on June 6, 2017 at 09:01 said:

Supposedly the MaxRAMFraction=1 will still leave some room for the other non-heap memory. I haven’t done extensive testing though

Reply ↓

The paragraph after the 2nd 100MB example says “Ok this makes more sense, the JVM was able to detect the container has only 1000MB and set the max heap to 44M.” There’s an extra zero in that memory amount.

Reply ↓

Carlos Sanchez on June 6, 2017 at 23:04 said:

fixed, thanks!

Reply ↓

Seems capped to 1GB though. Given a container memory of 3GB or larger and fraction = 1, it still sets max to 962MB. Looking for a way to run bigger containers, like 16GB, and let the JVM have most of it, without hardcoding with -Xmx.

Reply ↓

Carlos Sanchez on July 7, 2017 at 10:27 said:

You are probably not using the latest jdk

$ docker run -m 2G -ti --rm openjdk:8-jdk java -XshowSettings:vm -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -version Max. Heap Size (Estimated): 1.74G

Reply ↓
- ulsaulsa on July 7, 2017 at 11:47 said:
  
  Yes, I was, but I had capped the VirtualBox running Docker to 1GB. 🙂 Increased it and it now works fine. Thanks!

Hi, great article. But it looks like JVM continues to allocate max 1.74Gb heap memory even for 4Gb of container memory (with -XX:MaxRAMFraction=1) Any ideas?

Reply ↓

Carlos Sanchez on January 4, 2018 at 14:04 said:

you need to set both -XX:+UseCGroupMemoryLimitForHeap and XX:MaxRAMFraction to use all the memory in the container

Reply ↓
- evox on January 4, 2018 at 14:53 said:
  
  exactly, i set XX:MaxRAMFraction=1
  
  docker run -m 4GB java8:2.3.0 java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -version
  VM settings:
  Max. Heap Size (Estimated): 1.74G
  Ergonomics Machine Class: server
  Using VM: OpenJDK 64-Bit Server VM
  
  openjdk version “1.8.0_141”
  OpenJDK Runtime Environment (build 1.8.0_141-8u141-b15-1~deb9u1-b15)
  OpenJDK 64-Bit Server VM (build 25.141-b15, mixed mode)
- evox on January 4, 2018 at 14:59 said:
  
  I thought it could be my jvm version but also openjdk:8u131 can allocate only 1.74 GB memory for a container has 4GB memory.
  
  docker run -m 4GB openjdk:8u131 java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -version
  VM settings:
  Max. Heap Size (Estimated): 1.74G
  Ergonomics Machine Class: server
  Using VM: OpenJDK 64-Bit Server VM
  
  openjdk version “1.8.0_131”
  OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2-b11)
  OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)

Have you allowed more than 2GB to Docker? It’s not enough to just say `-m 4GB`, you also need to allocate at least 4GB in whatever virtual machine you’re using. I had 2GB allocated, so I got the same result as you with your command, until I increased it in my Docker (for Mac) advanced preferences.

Before:
$ docker run –rm -m 4GB openjdk:8u141 java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -version
VM settings:
Max. Heap Size (Estimated): 1.74G

After:
$ docker run –rm -m 4GB openjdk:8u141 java -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -version
VM settings:
Max. Heap Size (Estimated): 3.43G

Reply ↓

evox on January 4, 2018 at 15:22 said:

Yes, this is exactly the case. Thank you so much.

Reply ↓
Carlos Sanchez on January 4, 2018 at 15:24 said:

That is correct, thanks!

Reply ↓

Looks like the calculation doesn’t work with the G1 garbage collector. The JVM uses the entire memory for the heap and there is no memory left for off-heap. Does anyone have experience?

$ docker run -m 1GB openjdk:8u151 java -XX:+UseG1GC -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -version
VM settings:
Max. Heap Size (Estimated): 1.00G
Ergonomics Machine Class: server
Using VM: OpenJDK 64-Bit Server VM

openjdk version “1.8.0_151”
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-1~deb9u1-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

Reply ↓

i am seeing same result
docker run -m 1GB openjdk:8u151 java -XX:+UseG1GC -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -version
VM settings:
Max. Heap Size (Estimated): 1.00G
Ergonomics Machine Class: server
Using VM: OpenJDK 64-Bit Server VM

openjdk version “1.8.0_151”
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-1~deb9u1-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

docker run -m 1.5GB openjdk:8u151 java -XX:+UseG1GC -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -version
VM settings:
Max. Heap Size (Estimated): 1.50G
Ergonomics Machine Class: server
Using VM: OpenJDK 64-Bit Server VM

openjdk version “1.8.0_151”
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-1~deb9u1-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

docker run -m 2GB openjdk:8u151 java -XX:+UseG1GC -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -version
VM settings:
Max. Heap Size (Estimated): 1.95G
Ergonomics Machine Class: server
Using VM: OpenJDK 64-Bit Server VM

openjdk version “1.8.0_151”
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-1~deb9u1-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

Reply ↓

Pingback: Running a JVM in a Container Without Getting Killed II | Carlos Sanchez's Weblog

Pingback: Java标志Xms和Xmx是否覆盖标志XX:+ UseCGroupMemoryLimitForHeap？ – FIXBBS

Pingback: 如何使用Docker确定Java微服务的堆大小/非堆大小？ – FIXBBS

Pingback: Java 17 Features and Migration Considerations - techmark.pk

Pingback: [SOLVED] Do java flags Xms and Xmx overwrite flag XX:+UseCGroupMemoryLimitForHeap? – BugsFixing