Kubernetes Plugin for Jenkins 1.0

Includes support to get container logs from the pod, Kubernetes API auto configuration and lots of bug fixes

The full changelog:

  • containerLog step to get the logs of a container running in the agent pod JENKINS-46085 #195
  • Autoconfigure cloud if kubernetes url is not set #208
  • Change containerCap and instanceCap 0 to mean do not use JENKINS-45845 #199
  • Add environment variables to container from a secret JENKINS-39867 #162
  • Deprecate containerEnvVar for envVar and added secretEnvVar
  • Enable setting slaveConnectTimeout in podTemplate defined in pipeline #213
  • Read Jenkins URL from cloud configuration or KUBERNETES_JENKINS_URL env var #216
  • Make withEnv work inside a container JENKINS-46278 #204
  • Close resource leak, fix broken pipe error. Make number of concurrent requests to Kubernetes configurable JENKINS-40825 #182
  • Delete pods in the cloud namespace when pod namespace is not defined JENKINS-45910 #192
  • Use Util.replaceMacro instead of our custom replacement logic. Behavior change: when a var is not defined it is not replaced, ie. ${key1} or ${key2} or ${key3} -> value1 or value2 or ${key3} #198
  • Allow to create non-configurable instances programmatically #191
  • Do not cache kubernetes connection to reflect config changes and credential expiration JENKINS-39867 #189
  • Inherit podAnnotations when inheriting pod templates #209
  • Remove unneeded plugin dependencies, make pipeline-model-extensions optional #214

Speaking Trips on DevOps, Kubernetes, Jenkins

This 2nd half of the year speaking season is starting and you’ll find me speaking about DevOps, Kubernetes, Jenkins,… at

If you organize a conference and would like me to give a talk in 2018 you can find me @csanchez.

Screen Shot 2017-08-24 at 17.07.45.png

Kubernetes Plugin for Jenkins 0.12

Includes declarative pipeline support (note that you need Jenkins 2.66+ for it to work) and lots of bug fixes

The full changelog:

  • Add an experimental Declarative Agent extension for Kubernetes JENKINS-41758 #127
  • Implement Port mapping #165
  • Support idleMinutes field in pipeline #154
  • Add command liveness probe support #158
  • Add toggle for node usage mode #158
  • Add namespace support on PodTemplate.
  • Make PodTemplate optional within pipeline JENKINS-42315
  • Make Slave Jenkins connection timeout configurable #141
  • Fix durable pipeline PID NumberFormatException JENKINS-42048 #157
  • Don’t provision nodes if there are no PodTemplates set to usage mode Normal #171
  • Refactoring add/set methods in PodTemplate #173
  • Delete the build pod after we have finished with the template block #172
  • Default to use the kubernetes.default.svc.cluster.local endpoint
  • Do not print stack trace on ConnectException
  • Upgrade kubernetes client to 2.3.1 JENKINS-44189
  • Step namespace should have priority over anything else #161
  • Wait for pod to exist up to 60 seconds before erroring #155
  • Catch IOException on ContainerExecProc#kill
  • Do not print stack trace on connection exception
  • Restore random naming for pipeline managed pod templates.
  • Dir context is not honored by shell step JENKINS-40925 #146
  • Limit pod name to 63 characters, and change the randomly generated string #143
  • Fix workingDir inheritance error #136
  • Use name instead of label for the nesting stack #137
  • Exception in configure page when ‘Kubernetes URL’ isn’t filled JENKINS-45282 #174
  • kubectl temporary config file should work where Jenkins project contains spaces #178
  • Thread/connection leak #177

Jenkins Kubernetes Plugin 0.10 Released

The 0.10 release is mostly a bugfix release

Changelog for 0.10:

  • Fixing checkbox serialization by jelly views #110
  • Do not throw exceptions in the test configuration page #107
  • Upgrade to the latest kubernetes-client version. #106
  • feat: make pipeline support instanceCap field #102
  • Instantiating Kubernetes Client with proper config in Container Steps #104
  • Fix NPE when slaves are read from disk #103
  • [JENKINS-39867] Upgrade fabric8 to 1.4.26 #101
  • The pod watcher now checks readiness of the right pod. #97
  • Fix logic for waitUntilContainerIsReady #95
  • instanceCap is not used in pipeline #92
  • Allow nesting of templates for inheritance. #94
  • Wait until all containers are in ready state before starting the slave #93
  • Adding basic retention for idle slaves, using the idleTimeout setting properly #91
  • Improve the inheritFrom functionality to better cover containers and volumes. #84
  • Fix null pointer exceptions. #89
  • fix PvcVolume jelly templates path #90
  • Added tool installations to the pod template. #85
  • fix configmap volume name #87
  • set the serviceAccount when creating new pods #86
  • Read and connect timeout are now correctly used to configure the client. #82
  • Fix nodeSelector in podTemplate #83
  • Use the client’s namespace when deleting a pod (fixes a regression preventing pods to delete). #81

Next Week: DevOps Pro, Moscow and ApacheCON Europe, Seville

Next week is a busy one, will be speaking at DevOps Pro in Moscow then ApacheCON Europe in Seville. If you are around give me a shout!

And if you are organizing an event next year and are interested in me giving a talk, let me know.

From Monolith to Docker Distributed Applications

Docker is revolutionizing the way people think about applications and deployments. It provides a simple way to run and distribute Linux containers for a variety of use cases, from lightweight virtual machines to complex distributed micro-services architectures.

Containers allow to run services in isolation with a minimum performance penalty, increased speed, easier configuration and less complexity, making it ideal for continuous integration and continuous delivery based workloads. But migrating an existing application to a distributed micro-services architecture is no easy task, requiring a shift in the software development, networking and storage to accommodate the new architecture.

We will provide insight on our experience creating a Jenkins platform based on distributed Docker containers running on Apache Mesos and comparing other solutions as Kubernetes or Docker Swarm, applicable for all types of applications, but specially Java and JVM based nones.

CI and CD at Scale: Scaling Jenkins with Docker and Apache Mesos

As Continuous Integration and Continuous Delivery usage grows, the underlying infrastructure needs to grow too, scaling horizontally across multiple hosts and dynamically provisioned as needed.

Docker is revolutionizing the way people think about applications and deployments. Apache Mesos enables fault-tolerant distributed systems and allows managing a cluster of Linux servers as a single system, running Docker containers across multiple hosts, offering elasticity and machine abstractions.

By using Apache Mesos and Marathon, an Apache Mesos framework for long-running applications, the Jenkins Continuous Integration environment can be run at scale, dynamically.

Jenkins masters can be run on Docker containers distributed on Marathon, while the Jenkins Mesos and Docker plugins allow slaves and jobs to run in containers distributed across the multiple hosts, and isolating job execution.

 

Jenkins Kubernetes Plugin 0.9 Released

New features released in 0.9 include pipeline support and multiple containers per pod.

So now it is possible to define all the containers used in a job in your Jenkinsfile, for instance building a Maven project and a golang project in the same node without having to create any specific Docker image \o/

podTemplate(label: 'mypod', containers: [
    containerTemplate(name: 'maven', image: 'maven:3.3.9-jdk-8-alpine', ttyEnabled: true, command: 'cat'),
    containerTemplate(name: 'golang', image: 'golang:1.6.3-alpine', ttyEnabled: true, command: 'cat')
  ],
  volumes: [secretVolume(secretName: 'shared-secrets', mountPath: '/etc/shared-secrets')]) {

  node ('mypod') {
    stage 'Get a Maven project'
    git 'https://github.com/jenkinsci/kubernetes-plugin.git'
    container('maven') {
      stage 'Build a Maven project'
      sh 'mvn clean install'
    }

    stage 'Get a Golang project'
    git url: 'https://github.com/hashicorp/terraform.git'
    container('golang') {
      stage 'Build a Go project'
      sh """
      mkdir -p /go/src/github.com/hashicorp
      ln -s `pwd` /go/src/github.com/hashicorp/terraform
      cd /go/src/github.com/hashicorp/terraform && make core-dev
      """
    }

  }
}

Changelog:

  • Make it possible to define more than one container inside a pod.
  • Add new pod template step which allows defining / overriding a pod template from a pipeline script.
  • Introduce pipeline step that allows choosing one of the containers of the pod and have all ‘sh’ steps executed there.
  • allow setting dynamic pod volumes in pipelines
  • Add support for persistent volume claims
  • Add support for containerEnvVar’s in pipelines
  • [JENKINS-37087] Handle multiple labels per pod correctly
  • [JENKINS-37087] Iterate over all matching templates
  • Fix slave description and labels
  • [JENKINS-38829] Add help text for Kubernetes server certificate
  • #59: Allow blank namespace and reuse whatever is discovered by the client.
  • Ensure instanceCap defaults to unlimited
  • Add Jenkins computer name to the container env vars
  • Split arguments having quotes into account
  • Allow the user to enable pseudo-TTY on container level.
  • Use provided arguments without forcing jnlpmac and name into them. Provide placeholders for jnlpmac and name for the user to use. Fallback container uses as default arguments jnlpmac and name.
  • Split volume classes into their own package (#77)

 

Scaling Jenkins with Docker and Apache Mesos @ O’Reilly Media

I will be delivering this online course for O’Reilly media, October 18, 2016 7:00pm CEST

Scaling Jenkins with Docker and Apache Mesos

Continuous integration and continuous delivery at scale

Join Carlos Sanchez for a study in running Jenkins at scale. He’ll share his experience using Docker and Apache Mesos to create one of the biggest Jenkins clusters to date. You’ll drill down into the details with Carlos to get a better understanding of how Apache Mesos works. Together you’ll explore the challenges of running containerized and distributed applications (particularly JVM ones) through a real-world use case. By the end of this course, you’ll have a solid grounding in using these popular open source technologies for continuous integration and continuous delivery at scale.

What you’ll learn—and how you can apply it

By the end of this live, online course, you’ll understand:

  • How Apache Mesos works and how Docker containers are executed in a Mesos cluster
  • How Jenkins can use a Mesos cluster as a provider to provision build agents on demand
  • How Java applications behave inside a Docker container

And you’ll be able to:

  • Create a Apache Mesos cluster for local development using Docker Compose
  • Create Jenkins jobs that are executed dynamically based on demand
  • Use Jenkins Pipelines to execute jobs in one or more Docker containers

This course is for you because…

  • You’re a build/release engineer or are interested in deploying Docker at scale
  • You work with Jenkins or other Java applications
  • You want to become a Docker expert!