Ansible: Find files newer than another

I needed to figure out a way of identifying files newer than another one in Ansible. Here’s an outline of the solution I came up with.

First we need to create a bunch of directories and folder, with modified mtime values, that we can work with.

mkdir dir1;
mkdir dir2;
# Set the time on this dir to 3 days ago
touch -t $(date -v-3d +'%Y%m%d%H%M') dir1;
 
# create a bunch of files in dir2 and modify the dates
touch -t $(date -v-7d +'%Y%m%d%H%M') dir2/file1.txt;
touch -t $(date -v-6d +'%Y%m%d%H%M') dir2/file2.txt;
touch -t $(date -v-5d +'%Y%m%d%H%M') dir2/file3.txt;
touch -t $(date -v-4d +'%Y%m%d%H%M') dir2/file4.txt;
touch -t $(date -v-3d +'%Y%m%d%H%M') dir2/file5.txt;
touch -t $(date -v-2d +'%Y%m%d%H%M') dir2/file6.txt;
touch -t $(date -v-1d +'%Y%m%d%H%M') dir2/file7.txt;
touch dir2/file8.txt;

Let’s check the mtime value on our files…

stat -x dir2/*.txt | egrep 'File:|Modify:'
  File: "dir2/file1.txt"
Modify: Mon Jun 18 13:29:00 2018
  File: "dir2/file2.txt"
Modify: Tue Jun 19 13:29:00 2018
  File: "dir2/file3.txt"
Modify: Wed Jun 20 13:29:00 2018
  File: "dir2/file4.txt"
Modify: Thu Jun 21 13:29:00 2018
  File: "dir2/file5.txt"
Modify: Fri Jun 22 13:29:00 2018
  File: "dir2/file6.txt"
Modify: Sat Jun 23 13:29:00 2018
  File: "dir2/file7.txt"
Modify: Sun Jun 24 13:29:00 2018
  File: "dir2/file8.txt"
Modify: Mon Jun 25 13:29:33 2018

If we want to return files in dir2 than are newer than dir1, which has an mtime of Fri Jun 22 13:29:00 2018, then we would expect to return file6.txt, file7.txt and file8.txt. Here’s the playbook that will do just that;

---
  - hosts: localhost
    become: no
 
    tasks:
 
    - name: Get the mtime of the snapshot for later use
      stat:
        path: dir1/
      register: snapshot_stat
      failed_when: snapshot_stat.stat.exists == False
 
    - set_fact: myage={{ ansible_date_time.epoch|int - snapshot_stat.stat.mtime|int}}
 
    - debug:
        msg: "{{ myage }}"
 
    - name: Find files newer than the snapshot_dir mtime
      find:
        paths: dir2/
        age: "-{{ myage }}"
        age_stamp: mtime
        file_type: file
      register: found_files
 
    - debug:
        var: found_files

Not the myage variable above has a minus in front of it. This is makes it return files newer than this ‘age’. The documentation page for the find module described the age parameter as follows;

“Select files whose age is equal to or greater than the specified time. Use a negative age to find files equal to or less than the specified time. You can choose seconds, minutes, hours, days, or weeks by specifying the first letter of any of those words (e.g., “1w”).”

I found this explanation to be a little confusing with the mixed up semantics between age and time. Basically specifying a value of ‘1d’ would return all files older than 1 day, while ‘-1d’ would return all files less than a day old.

Ansible Raspberry Pi Projects

I’ve been playing around with a few RaspberryPi units and thought I’d share the Ansible projects I’ve created here. All these Ansible roles are intended for the Raspian OS.

  • RaspberryPiBase – Perform some basic setup tasks on the Pi Raspian OS. Currently changes pi user password and updates the OS.
  • PiTV – Install Sabnzbd, SickBeard and CouchPotato on the Pi. A little manual configuration to get all the apps speaking to each other is required. Also adds an external hdd to the mounts.
  • PiPlex – Install the Plex Media Server onto the Raspian OS. Also mounts storage, on another pi, via sshfs. Please setup passwordless login between the two Pi computers.

 

Setting up a Ansible Module Test Environment

I’ve begun developing some Ansible modules and have created a Vagrant environment to help with testing. You can check it over over on my github. The environment has been created to test some MongoDB modules but can easily be repurposed to another use. It’s quite simple to get started;

git clone https://github.com/rhysmeister/AnsibleTest.git
cd AnsibleTest
vagrant up

This will fire up a VM and install two versions of Python; 2.6 and 3.5. These are the two versions of python that all Ansible modules should be tested against. These version can be activated with the following bash aliases; py26 and py35. These aliases will set the version of python to the appropriate one as well as setting up the ansible environment so your custom modules can be found (those that are in the git repo cloned during setup).

The tasks/main.yml file does contain a bunch of other tasks, to deploy some bash scripts and ansible playbooks, to help with module testing. These can be removed or modified to suit your own purposes.

Getting JConsole working over ssh with Cassandra

I’ve been working with Cassandra recently and wanted to start using JConsole. JConsole exposes some pretty useful information about Cassandra and the Java environment in which it runs. It operates over JMX and by default it only accepts connections from the localhost. Since we don’t run GUIs on our servers and we didn’t want to open up the JMX service to remote connections we needed to do a little wizardry to connect. First I thought of tunneling over ssh;

ssh -N -L 7199:localhost:7199 remotehost

This is a pretty standard way to connect a service that only listens for connections from the localhost. However this would not work at all. After much troubleshooting I came across a solution.

The solution was to connect via a SOCKS proxy;

ssh -fN -D 7199 remotehost

This essentially setup up a proxy, running on port 7199 (this can be anything, it does not have to match the JMX port), to the remote node (cassandra in this case). Then we can connect with JConsole like so…

jconsole -J-DsocksProxyHost=localhost -J-DsocksProxyPort=7199 service:jmx:rmi:///jndi/rmi://localhost:7199/jmxrmi -J-DsocksNonProxyHosts=

This allows JConsole to connect to the JMX service running on the remote Cassandra node;

JConsole connectted via JMX to a remote Cassandra Node with a SOCKS proxy

Jenkins test instance with Vagrant & Ansible

Here’s yet another project using Vagrant and Ansible. This time it’s for a Jenkins instance. To get started head on over to the Jenkins test instance github page. Consult the readme for setup instructions.