Creating a Vagrant, Virtualbox & Ansible environment in the Windows Linux Subsystem

I’ve just been given a new Windows corporate laptop, with a huge amount of RAM (64GB), a large number of cores, and I wanted to start using this as my main development virtualisation platform. I do a lot of stuff with Vagrant, Ansible and VirtualBox and Windows hasn’t always been a welcome home for this setup. A more welcoming experience can be received through the Windows Linux Subsystem (WLSS) and is a big improvement over Cygwin. The instructions here used Debian 9.5 but should work on many other Linux distributions with minor modifications (i.e. package manager).

First install Debian (or other distro) from the official instructions (easiest way is through the MS App Store).

Then update the OS…

sudo apt-get update && sudo apt-get upgrade;

The install Python and pip…

sudo apt install python;
sudo apt install python-pip;

Now Ansible can be installed through pip…

sudo pip install ansible

Next download Vagrant and install it…

VAGRANT="https://releases.hashicorp.com/vagrant/2.1.5/vagrant_2.1.5_x86_64.deb";
wget "$VAGRANT";
sudo apt install ./$(basename "$VAGRANT");

If you’re behind a proxy you probably need this Vagrant plugin

vagrant plugin install vagrant-proxyconf;

Next install git…

sudo apt install git;

Finally we need to install VirtualBox. Don’t rush ahead and install the Linux version. I did and this and it does not work. Grab the latest Windows version and install that on the host system in the usual way.

Next back in the WLSS Debian shell we need to make a few modification to allow the Windows Version of VirtualBox to be used. Basically we allow Vagrant to access the Windows version of VB and set the proxy. Edit or delete these as necessary…

cd
echo 'export VAGRANT_WSL_ENABLE_WINDOWS_ACCESS="1"' >> .profile
echo 'export PATH="$PATH:/mnt/c/Program Files/Oracle/VirtualBox"' >> .profile
echo 'export VAGRANT_HTTP_PROXY=${http_proxy}' >> .profile
source .profile

Next let’s clone a Vagrant / Ansible / Virtual project to test the setup out…

mkdir git && cd git;
git clone https://github.com/rhysmeister/Jenkins.git
cd Jenkins
vagrant up

This is one my own projects setting up a Jenkins instance. It’s fairly simple but it will test all components of the setup we have just installed.

Vagrant: Create a series of VMs from a hostname array

I couldn’t find any examples of creating VMs from a array of strings online so sat down to work something out myself. Here’s how you do it…

Vagrant.configure("2") do |config|

  [ "web1",
    "db1",
    "web2",
    "db2",
    "backup1",
    "backup2",
    "admin1" ].each do |host|
      config.vm.define "#{host}" do |nrpe|
        nrpe.vm.box = "bento/centos-7.5"
        nrpe.vm.provider :virtualbox do |vb|
          vb.customize [
            "modifyvm", :id,
            "--name", "#{host}",
            "--memory", "1024"
          ]
          vb.cpus = 2
        end
        config.vm.hostname = "#{host}"
        config.vm.provision :ansible do |ansible|
          ansible.playbook = "basic.yml"
        end
      end
  end
end

This Vagrantfile will create a VM for each hostname in the array as well as running the basic.yml Ansible playbook against it. You can fire it up with…

vagrant up

Once booted you can view the status of the created vms…

vagrant status
Current machine states:

web1                   running (virtualbox)
db1                    running (virtualbox)
web2                   running (virtualbox)
db2                    running (virtualbox)
backup1                running (virtualbox)
backup2                running (virtualbox)
admin1                 running (virtualbox)

Automate ssh-copy-id with numbered hosts

Here’s a script I use to automate ssh-copy-id when I need to add a series of hosts using a incremental node number. For example…

prod-db-server001
prod-db-server002
prod-db-server003

and so on. The script uses expect to perform its work. To adjust this for your own purposes you simply need to change the SSH_USER variable, the number of hosts in the for loop and of course the hostname scheme. Once you execute the script you’ll enter your password once and ssh-copy-id will be performed for all the hosts in sequence.

#!/bin/bash

set -x;

export SSH_USER="admin"
read -s PASSWORD
export PASSWORD

for node in {1..91}; do
        if (( $node <= 9 )); then
                export HOST=hostname00${node}.domain.ch
        else
                export HOST=hostname0${node}.domain.ch
        fi;

    expect -c '
    set SSH_USER $env(SSH_USER)
    set HOST $env(HOST)
    set PASSWORD $env(PASSWORD)
    spawn ssh-copy-id $SSH_USER@$HOST
    expect {
        "continue" {
            send "yes\n";
            exp_continue
        }
        "assword:" {
            send "$PASSWORD\n";
        }
    }
    expect eof'

    echo "Done $HOST"
done;

Ansible Playbook for Raspberry Pi Headphones Setup

I’ve created another Ansible Playbook for the Raspberry Pi to setup Headphones. It’s hosted over on my Github: PiHeadphones

The playbook can be execute with the following command…

ansible-playbook -i inventory headphones.yaml

The inventory file should contain the name of your Raspberry Pi and should already be setup for ssh. The playbook will clone the git repo and setup Headphones as a service. Afterwards there’s a little manual configuration to do in the web interface which is available at http://yourraspberrypi:8181/

Ansible: Find files newer than another

I needed to figure out a way of identifying files newer than another one in Ansible. Here’s an outline of the solution I came up with.

First we need to create a bunch of directories and folder, with modified mtime values, that we can work with.

mkdir dir1;
mkdir dir2;
# Set the time on this dir to 3 days ago
touch -t $(date -v-3d +'%Y%m%d%H%M') dir1;

# create a bunch of files in dir2 and modify the dates
touch -t $(date -v-7d +'%Y%m%d%H%M') dir2/file1.txt;
touch -t $(date -v-6d +'%Y%m%d%H%M') dir2/file2.txt;
touch -t $(date -v-5d +'%Y%m%d%H%M') dir2/file3.txt;
touch -t $(date -v-4d +'%Y%m%d%H%M') dir2/file4.txt;
touch -t $(date -v-3d +'%Y%m%d%H%M') dir2/file5.txt;
touch -t $(date -v-2d +'%Y%m%d%H%M') dir2/file6.txt;
touch -t $(date -v-1d +'%Y%m%d%H%M') dir2/file7.txt;
touch dir2/file8.txt;

Let’s check the mtime value on our files…

stat -x dir2/*.txt | egrep 'File:|Modify:'
  File: "dir2/file1.txt"
Modify: Mon Jun 18 13:29:00 2018
  File: "dir2/file2.txt"
Modify: Tue Jun 19 13:29:00 2018
  File: "dir2/file3.txt"
Modify: Wed Jun 20 13:29:00 2018
  File: "dir2/file4.txt"
Modify: Thu Jun 21 13:29:00 2018
  File: "dir2/file5.txt"
Modify: Fri Jun 22 13:29:00 2018
  File: "dir2/file6.txt"
Modify: Sat Jun 23 13:29:00 2018
  File: "dir2/file7.txt"
Modify: Sun Jun 24 13:29:00 2018
  File: "dir2/file8.txt"
Modify: Mon Jun 25 13:29:33 2018

If we want to return files in dir2 than are newer than dir1, which has an mtime of Fri Jun 22 13:29:00 2018, then we would expect to return file6.txt, file7.txt and file8.txt. Here’s the playbook that will do just that;

---
  - hosts: localhost
    become: no

    tasks:

    - name: Get the mtime of the snapshot for later use
      stat:
        path: dir1/
      register: snapshot_stat
      failed_when: snapshot_stat.stat.exists == False

    - set_fact: myage={{ ansible_date_time.epoch|int - snapshot_stat.stat.mtime|int}}

    - debug:
        msg: "{{ myage }}"

    - name: Find files newer than the snapshot_dir mtime
      find:
        paths: dir2/
        age: "-{{ myage }}"
        age_stamp: mtime
        file_type: file
      register: found_files

    - debug:
        var: found_files

Not the myage variable above has a minus in front of it. This is makes it return files newer than this ‘age’. The documentation page for the find module described the age parameter as follows;

“Select files whose age is equal to or greater than the specified time. Use a negative age to find files equal to or less than the specified time. You can choose seconds, minutes, hours, days, or weeks by specifying the first letter of any of those words (e.g., “1w”).”

I found this explanation to be a little confusing with the mixed up semantics between age and time. Basically specifying a value of ‘1d’ would return all files older than 1 day, while ‘-1d’ would return all files less than a day old.