Kubash

Build, provision, initialize, add common components, and tear down a cluster PDQ.

old waffle (still has quite a backlog)

Dev

Build production ready clusters using a variety of technologies along the way.

By default, this will build an image using packer, then rebase that image for your nodes. Then initialize them using kubeadm, and install charts using helm.

Oneliner installation

Install with one easy line:

curl -L git.io/kubash|bash

Get started by making the example clusters:

cd ~/.kubash
make all-examples
ls -l clusters/

or to generate just the main example:

kubash -n example yaml2cluster examples/example-cluster.yaml
ls -l clusters/example

Now build the image kubash build --target-os kubeadm

Then kubash provision -n example

And finally kubash -n example init

By default kubash is quiet unless an error is hit (though many of the programs called by kubash might not be very quiet so there is still lot’s of noise at VERBOSITY=0). If you like watching noisy output crank up the verbosity by adding a few v flags (i.e. -vvvv) or secify the verbosity --verbosity 100 or export it as a environment variable e.g.

export VERBOSITY=100

kubash output will be denoted by appending #s in front of various verbosity levels e.g.

############# Kubash, by Josh Cox

Alternative pipelines

There are also alternative methods available for the steps, for coreos there is an alternative builder that merely downloads the official images. And for initializing the default is to directly initialize with kubeadm, or can be alternatively done through ansible with the openshift or kubespray or kubeadm2ha methods.

Other provisioning beyond KVM/qemu is also being looked at, suggestions welcome in the issues. Keep in mind this started life as a ten line script of me just trying to duplicate the official instructions after both the kubespray and openshift playbooks went sideways on me and I determined I needed to refamiliarize myself with spinning up a cluster with another method. Somewhere along the way I came across this google doc

And I decided to combine all of the above into a unified forkable pipeline that automates the entire proces of building the images using a similar method to the one I used for CoreOS, using one of the initialization methods in a very repeatable way. And throw some of the additional common components I tend to add to every cluster.

Provisioners

KVM/qemu is used first and foremost because it is standard and built into Linux itself. I intend to add virtualbox, vmware, etc builders and provisioners in the future.

Builders

Right now you can build these OSs as your base image:

1. Ubuntu
2. Debian
3. Centos
4. CoreOS

Packer will build ubuntu, debian, and centos. And there is also a basic downloader for the CoreOS images.

Initializers

Although this script can just utilize ssh to run the kubeadm commands directly on the VMs, you can optionally use kubespray or openshift’s ansible playbooks instead, I have had various issues with both and that’s why i wrote this script so I can choose amongst a few different methods in my regular daily builds (I’m the sort of guy who likes to spin up clusters while sipping my morning coffee).

Configs

By default there is now a .kubash folder in your home directory. Inside this directory is a folder called clusters, make example in the .kubash directory will build the default example cluster.

Usage

This script automates the setup and maintenance of a kubernetes cluster

kubash -n clustername COMMAND

Commands:

yaml2cluster - Build a cluster directory from a yaml cluster file

json2cluster - Build a cluster directory from a json cluster file

build - Build a base image

build-all - Build all the base images in parallel

provision - Provision individual nodes

init - Initialize the cluster

reset - Reset the cluster by running kubeadm reset on all the hosts

decommission - Tear down the cluster and decommission nodes

copy - copy the built images to the provisioning hosts

ping - Perform ansible ping to all hosts

auto - Full auto will provision and initialize all hosts

hostnamer - Will rehostname all the hosts

refresh - will search for all the hosts using the appropriate method and recreate hosts.csv

Options

These options are parsed using GNU getopt

options:

 -h --help - Print usage

 -n --clustername - work with a named cluster (or by default it will use a cluster name of 'default')

 -c --csv FILE - Set the csv file to be parsed

 --parallel NUMBER_OF_THREADS - set the number of parallel jobs for tasks that support it

 -v --verbose - Increase the verbosity (can set multiple times to incrementally increase e.g. `-vvvv`

 --verbosity NUMBER - or you can set the verbosity directly

 --debug - adds the debug flag

 --oidc - enable the oidc auths

There is an example csv file in this repo which shows how to compose this file

Debugging

First try kubash COMMAND --help

See the debugging page for more

Interactive Mode

kubash – alone will invoke an interactive shell (or if you prefer kubash interactive)

see the Interactive Mode documentation

Parallel jobs

To set the number of concurrent jobs export PARALLEL_JOBS e.g.

export PARALLEL_JOBS=10

GNU Parallel

This project takes advantage of GNU Parallel gnu parallel and so should you, for more info see:

  O. Tange (2011): GNU Parallel - The Command-Line Power Tool,                                                                                                                                                     
  ;login: The USENIX Magazine, February 2011:42-47.