Groesbeek, view of the 'National Liberation Museum 1944-1945' in Groesbeek. © Ton Kersten
Fork me on GitHub

Ansible with loops or lookup

2019-02-23 (150) by Ton Kersten, tagged as ansible, sysadm

Since Ansible version 2.5 there is a lot of discussion and confusion about the loop syntax. There is also discussion if with_...: will be replaced by loop: deprecating the with_... keywords. Even Ansibles documentation is not clear about this.

Should I use loop: or with_...:, in fact nobody really knows. What would the correct syntax be?

- name: Loops with with_ and lookup
  hosts: localhost
  connection: local
  gather_facts: no
      - john
      - paul
      - mary
      - beer
      - wine
      - whisky

    - name: with nested
        msg: "with_nested: item[0] is '{{ item[0] }}' and item[1] is '{{ item[1] }}'"
        - "{{ people }}"
        - "{{ drinks }}"

    - name: nested and loop
        msg: "nested_loop: item[0] is '{{ item[0] }}' and item[1] is '{{ item[1] }}'"
        - "{{ people }}"
        - "{{ drinks }}"

Read more »

Ansible: One Role to Rule them All

2019-02-07 (149) by Ton Kersten, tagged as ansible, sysadm

I am a long time Ansible user and contributor (since 2012) and I have been struggling with a decent setup for a multi-environment case. I have been designing and re-designing a lot, until I came up with this design. And what a coincidence, a customer wanted a setup that was exactly this. So this concept is a real world setup, working in a production environment.

Did I get your attention? Read after the break, but take your time. it is a long read.

Read more »

Running it through Tattr (part 2)

2018-08-08 (148) by Ton Kersten, tagged as ansible, sysadm

Some time ago I created a playbook to show the content of a rendered template. When you keep digging in the Ansible documentation, you suddenly stumble over the template lookup-plugin. And then it turns out that my playbook is a bit clumsy.

A nicer and shorter way to do it:

# This playbook renders a template and shows the results
# Run this playbook with:
#       ansible-playbook -e templ=<name of the template> template_test.yml
- hosts: localhost
  become: false
  connection: local

    - fail:
        msg: "Bailing out. The play requires a template name (templ=...)"
      when: templ is undefined

    - name: show templating results
        msg: "{{ lookup('template', templ) }}"

Ansible, loop in loop in loop in loop in loop

2018-06-08 (147) by Ton Kersten, tagged as ansible, loop, sysadm

A couple of days ago a client asked me if I could solve the following problem:

They have a large number of web servers, all running a plethora of PHP versions. These machines are locally managed with DirectAdmin, which manages the PHP configuration files as well. They are also running Ansible for all kind of configuration tasks. What they want is a simple playbook that ensures a certain line in all PHP ini files for all PHP versions on all webservers.

All the PHP directories match the pattern /etc/php[0-9][0-9].d.

Thinking about this, I came up with this solution (took me some time, though) smiley

- name: find all ini files in all /etc/php directories
  hosts: webservers
  user: ansible
  become: True
  become_user: root

    - name: get php directories
        file_type: directory
          - /etc
           - php[0-9][0-9].d
      register: dirs

    - name: get files in php directories
          - "{{ item.path }}"
          - "*.ini"
      loop: "{{ dirs.files }}"
      register: phpfiles

    - name: show all found files
        msg: "Files is {{ item.1.path }}"
        - "{{ phpfiles.results }}"
        - files

The part with the with_subelements did the trick. Of course this line can be written as:

loop: "{{ query('subelements', phpfiles.results, files) }}"

Ditched Disqus

2018-05-31 (146) by Ton Kersten, tagged as gdpr, privacy

As the new GDPR finds it’s way all over Europe I decided to have a closer look at my website. I was using the Disqus comment system for some time now, but hardly ever someone really takes the time to comment.

As the Disqus systems uses a lot of Javascript and cookies, I decided it was time to get rid of these tools and make my site fly, again.

At Disqus: So long and thanks for all the fish.

Did you run it through TAttr

2017-08-15 (145) by Ton Kersten, tagged as ansible, sysadm

During my last Ansible training the students needed to create some Ansible templates for them selfs. As I do not want to run a testing template against some, or all, machines under Ansible control I created a small Ansible playbook to test templates.

Read more »

Stupid Fedora

2016-05-26 (144) by Ton Kersten, tagged as sysadm

Yesterday I removed a simple package from my Fedora 23 machine and after that I got the message

error: Failed to initialize NSS library


Searching the interwebs I found out I wasn’t the first, and probably not the last, to run into this problem.

It seems that, one way or another, the DNF package doesn’t know about the dependency it has on SQLite. So, when a package removal requests to remove SQLite, DNF removes it without questions. Ans thus break itself.

But how to fix this? DNF doesn’t work, but RPM doesn’t either, so there is no way to reinstall the SQLite packages.

Tinkering and probing I found this solution:


wget ${url}/sqlite-${ver}.fc23.x86_64.rpm
wget ${url}/sqlite-libs-${ver}.fc23.x86_64.rpm
rpm2cpio sqlite-${ver}.fc23.x86_64.rpm | cpio -idmv
rpm2cpio sqlite-libs-${ver}.fc23.x86_64.rpm | cpio -idmv
cp -Rp usr /
dnf --best --allowerasing install sqlite.x86_64

This downloads the SQLite package and SQLite library packages, extracts them and copies the missing files to their /usr destination. After doing that, DNF and RPM get working again. It could be that I downloaded an older version of the SQLite stuff, so to make sure I have a current version I reinstall SQLite again.

Maybe a good idea to fix that in DNF!

Building an Ergodox

2015-03-03 (143) by Ton Kersten, tagged as news

After a lot of thought I decided it was time for a new project, one I would enjoy and a project that would be useful for a long time.

Searching the web and reading articles I found the ErgoDox.

The ErgoDox is a split-hand ergonomic keyboard with mechanical switches and open source, layer-based firmware running on a Teensy microcontroller. While other keyboards offer dip-switches or GUI config tools, the firmware and layouts can be built from source on the command line or through a layout configuration tool. Flashing a new build onto the ErgoDox is easy with the multi-platform Teensy loader.

I immediately got interested and after searching a bit more I ordered the kit at Falbatech. Unfortunately they do not supply the keycaps, so I ordered them from Signature Plastics.

Read more »

Stable Internet

2014-10-01 (142) by Ton Kersten, tagged as internet

My stable internet connection

Since a couple of years I’m running a fiber connection to the Internet, supplied by XMS-Net.

I also have an Atlas probe to do some internet measurements for RIPE.

Today I got a status email from RIPE with the connection status of last month. I guess I can say I have a stable internet connection. smiley

This is your monthly availability report for probe xxxx (TonKs Atlas).

Calculation interval    : 2014-09-01 00:00:00 - 2014-10-01 00:00:00
Total Connected Time    :  30d 00:00
Total Disconnected Time :   0d 00:00
Total Availability      :    100.00%

| Connected (UTC)     | Disconnected (UTC)  | Connected  | Disconnected |
| 2014-08-26 07:09:17 | Still up            |  30d 00:00 |     0d 00:00 |

Puppet environments

2014-05-26 (141) by Ton Kersten, tagged as puppet

For my job I do a lot of Puppet and I thought it was about time to write some tips and tricks down.

First part of this post is about my environment setup. In my test setup I use a lot of environments. They are not at all useful, but that’s not the point. It’s my lab environment so things need to break once in a while. But with multiple environments Puppetlabs says that you should switch to directory environments (PuppetDoc) but some way or another I cannot get that to work in a good way with my PE version (3.4.3 (Puppet Enterprise 3.2.3)). So I started implementing dynamic environments, which is a simple way of specifying the directories for your environments.

Part of my puppet.conf looks like

    environment = production
    manifest    = $confdir/environments/$environment/manifests/site.pp
    manifestdir = $confdir/environments/$environment/manifests
    modulepath  = $confdir/environments/$environment/modules:/usr/share/puppet/modules
    templatedir = $confdir/environments/$environment/templates

So, my default environment is production and a client can specify another environment to be in. The command

puppet agent --environment=test

would place this node in the test environment. A simple module places a new puppet.conf file on the client stating this new environment. Couldn’t be more simple.

Well, that’s what you think. But what if you need to deploy 10.000+ hosts of which there are about a third in environment test and about a 1000 in environment development? It would take a lot of time to ssh into all these servers and run Puppet with the correct environment.

There has to be a way around that. And, of course, there is. In Puppet version 3 and up Hiera is integrated into Puppet and we already use that a lot. Why not integrate the environment in Hiera? Well, our hiera.yaml is now:

    - "%{environment}/hiera/%{::fqdn}"
    - "%{environment}/hiera/%{::hostname}"
    - "%{environment}/hiera/%{::domainname}"
    - "%{environment}/hiera/%{::systemtype}"
    - "%{environment}/hiera/%{::osfamily}"
    - "%{environment}/hiera/common"

    - yaml


This challenges me with a chicken and egg problem. To get the environment I need to know the environment. But what if I make Hiera into an ENC and let this one deliver the environment? Can this be done? Yes, it can.

This is how I did it:

First create a part of the Hiera structure that’s not in the current environment, for example like this:

    - "hiera/%{::fqdn}"
    - "hiera/default"
    - "%{environment}/hiera/%{::fqdn}"
    - "%{environment}/hiera/%{::hostname}"
    - "%{environment}/hiera/%{::domainname}"
    - "%{environment}/hiera/%{::systemtype}"
    - "%{environment}/hiera/%{::osfamily}"
    - "%{environment}/hiera/common"

    - yaml


And in the directory /etc/puppetlabs/puppet/environments/hiera I place a very small file, called default.yaml, which contains:

environment: 'production'

This makes sure that any node without a specific file, will get the production environment. This is the default for Puppet as well, so nothing changes for that.

To test this, run:

hiera environment ::fqdn=$(hostname -f)

This will give you something like environment: production. For every host in another environment as the production one, create a small file named the FQDN of the host with the contents stating the wanted environment.

(Watch for the :: in front of the fqdn. This means that the fqdn variable is a top scope variable, as all facter variables are.

Now integrate this into Puppet. First create a little script that executes the command above and returns the wanted output.

My script is called getenv and placed in /etc/puppetlabs/puppet/bin


penv="$(/opt/puppet/bin/hiera                       \
            -c /etc/puppetlabs/puppet/hiera.yaml    \
            environment ::fqdn="${1}")"

echo "environment: ${penv}"

This returns a string like environment: production.

And last, but not least, place this settings in the [master] of your puppet.conf

node_terminus  = exec
external_nodes = /etc/puppetlabs/puppet/bin/getenv

It took some work to get things started, but a small shell thingy read the file with all 10.000+ hosts and required environments, that created all the Hiera files for all nodes that are not in the production environment.

Just one thing to do: When I have a lot of host-files in a single directory, this could become slow. I could place all definitions in a simple database, but things would get complicated again, and that’s not what I want. I also could split things up per letter, but I’m not sure yet if I really want that.

When I have resolved this, this entry will be continued.