Using Ansible Modules with Testinfra

I’ve been looking at improving the quality of the testing I do with Molecule and Testinfra. Simple checks like service.is_running or package.is_installed have their place but they’re pretty limited as to what assurances they provide us. Part of the issue I have is that some tests need a fair bit of setup to make them worthwhile. Attempting to tackle that with raw Python might be a little bit tricky. A better approach is to use the Ansible module available within TestInfra. We can call an ansible module with the following Python code:

ansible = host.ansible(module name,
                       module arguments,
                       check mode,
                       become)

Some of the stuff I do involves Nagios Plugins that parse log files for interesting content. I can test the the plugin is doing what it’s supposed to by setting log file content and then running the plugin and checking the output. Here’s an example of that…

def test_nagios_plugin_test(host):
    with host.sudo():
        cmd = host.run("rm -f /var/log/mylogfile.log")
        assert cmd.rc == 0

    with host.sudo("reaper"):
        ansible = host.ansible("copy",
                               "src=dummy_log_content.log dest=/var/log/mylogfile.log",
                               check=False,
                               become=True)
        assert ansible['changed'] == True

    with host.sudo("nrpe"):
        cmd = host.run("/usr/lib64/nagios/plugins/check_logfile_custom")
        assert cmd.rc == 0
        assert cmd.stdout.strip() == "OK: All looks good in /var/log/mylogfile.log"

The dummy_log_content.log is placed into the roles molecule/default/files directory. By providing a range of dummy log files I can ensure the Nagios Plugin is working as expected.

Using Bash brace expansion to generate multiple files

I needed to generate a whole bunch of files, with identical content, for a recent task. You might automatically think of using a loop for such a task but there’s a much simpler method using brace expansion in the shell.

I wanted to generate files in the following format…

rhys-tmp01.txt
rhys-tmp02.txt
rhys-tmp03.txt
...
rhys-tmp91..txt
rhys-tmp92.txt

This is achievable with a simple one-liner once we have created the source file rhys-tmp01.txt:

tee rhys-tmp{02..92}.txt < rhys-tmp01.txt

Note that this uses zero-padding and this won't work in old versions of bash (probably needs to be at least version 4).

Linux Server checks with Goss

I’ve been playing a little with goss recently. Goss is similar to TestInfra in that it allows you to write tests to validate your infrastructure. Goss uses yaml to specify the expected state rather than python code unittests like Testinfra. It also has a couple of other interesting features making it stand out from the crowd…

The first is a test auto-generation feature. Have a service you want to monitor? Simply run this…

goss autoadd nagios

Goss will autodiscover various things about that service. Here’s what it found out about the Nagios service….

package:
  nagios:
    installed: true
    versions:
    - 4.4.3
service:
  nagios:
    enabled: true
    running: true
user:
  nagios:
    exists: true
    uid: 999
    gid: 998
    groups:
    - nagios
    home: /var/spool/nagios
    shell: /sbin/nologin
group:
  nagios:
    exists: true
    gid: 998
process:
  nagios:
    running: true

The state of the nagios service can then be validated with…

goss validate
.............

Total Duration: 0.037s
Count: 13, Failed: 0, Skipped: 0

We can also setup a http health endpoint…

goss serve &
2019/08/14 15:05:42 Starting to listen on: :8080

When we hit the endpoint the tests are executed…

curl http://localhost:8080/healthz
2019/08/14 15:06:48 127.0.0.1:42792: requesting health probe
2019/08/14 15:06:48 127.0.0.1:42792: Stale cache, running tests
.............

Total Duration: 0.036s
Count: 13, Failed: 0, Skipped: 0

We can also run the tests as a Nagios check…

goss validate --format nagios
GOSS OK - Count: 13, Failed: 0, Skipped: 0, Duration: 0.032s

Note the execution speeds above. Goss outperforms python-based testing tools like TestInfra by a significant margin.

Wait for processes to end with Ansible

I’ve been doing a lot in stuff in ansible recently where I needed to fire up, kill and relaunch a bunch of processes. I wanted to find a quick and reliable way of managing this…

This is possible using a combination of the pids and wait_for modules…

First get the pids of your process…

- name: Getting pids for mongod
  pids:
      name: mongod
  register: pids_of_mongod

The pids module returns a list with which we can iterate over with with_items.Then we can use the wait_for task and the /proc filesystem to ensure all the processes have exited…

- name: Wait for all mongod processes to exit
  wait_for:
    path: "/proc/{{ item }}/status"
    state: absent
  with_items: "{{ pids_of_mongod.pids }}"

After this last task complete you can be sure that the Linux OS has cleaned up all your processes.

Broken sudo?

If you somehow add a dodgy sudo rule you might end up breaking it completely…

sudo su -

>>> /etc/sudoers.d/new_sudo_rule: syntax error near line 1 <<<

sudo: parse error in /etc/sudoers.d/new_sudo_rule near line 1

[sudo] password for rhys:

rhys is not in the sudoers file.  This incident will be reported.

You need to sudo to fix sudo? You might first think of booting into rescue mode. That would work but luckily there's an easier way...

pkexec mv /etc/sudoers.d/new_sudo_rule .

This will move the dodgy sudo rule out of harms way. See more on pkexec.