SaltStack Execution Modules

SaltStack (or Salt, for short) is a Python-based open-source configuration management software and remote execution engine. It supports the “Infrastructure as Code” approach (the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools) to deployment and cloud management.

I’ll describe in this short article how you can extend the Salt functionalities through the addition of a new execution module.

Just to clarify, you don’t need to be able to write Python or other code to use Salt in the normal usage case. Adding extensions to Salt is an “advanced”, but quite interesting topic.

A bit of theory

A Salt execution module is a Python (2.6+) or Cython module, though some specificities do exist, placed in a directory called _modules at the root of the Salt file server, usually /srv/salt.

An execution module usually define a __virtual__() function, to determine whether the requirements for that module are met, and the __virtualname__ string variable, that is used by the documentation build system to know the virtual name of a module without calling the __virtual__ function.

The following example from the official documentation should clarify this point.

A huge and consistent corpus of libraries and functions are packaged in the Salt framework. You can speed up and highly simplify the development of your modules by making use of those resources.

# Some examples of import:
import salt.utils
import salt.utils.itertools
import salt.utils.url
import salt.fileserver
from salt.utils.odict import OrderedDict

See the official documentation for more information, or… wait for a future article.

As an example, we can write a simple module returning some information about the CPU architecture of a Linux host.

Step by step development

We start by importing some Python and Salt libraries and by defining the  __virtualname__ variable and a __virtual__() function.

# Import Python libs
import logging
# Import Salt libs
from salt.exceptions import CommandExecutionError

__virtualname__ = 'cpuinfo'

def __virtual__():
    '''
    Only run on Linux systems
    '''
    if __grains__['kernel'] != 'Linux':
        return (False,
            'The {0} execution module cannot be loaded: '
            'only available on Linux systems.'.format(
            __virtualname__))
    return __virtualname__

As you can see, __virtual__() returns  False  when the operating system is not Linux. This means that the module cpuinfo will be only available for Linux minions and hidden otherwise.

Salt comes with an interface to derive information about the underlying system. This is called the grains interface, because it presents salt with grains of information. Grains are collected for the operating system, domain name, IP address, kernel, OS type, memory, and many other system properties. The __grains__ dictionary contains the grains data generated by the minion that is currently being worked with.

It’s time now to implement the logic of our module.

log = logging.getLogger(__name__)

def _verify_run(out):
    '''
    Crash to the log if command execution was not
    successful.
    '''
    if out.get('retcode', 0) and out['stderr']:
        log.debug('Return code: {0}'.format(
            out.get('retcode')))
        log.debug('Error output\n{0}'.format(
            out.get('stderr', 'N/A')))
        raise CommandExecutionError(out['stderr'])

def _lscpu():
    '''
    Get available CPU information.
    '''
    try:
        out = __salt__['cmd.run_all']("lscpu")
    except:
        return None

    _verify_run(out)

    data = dict()
    for descr, value in [elm.split(":", 1) \
        for elm in out['stdout'].split(os.linesep)]:
            data[descr.strip()] = value.strip()

    cpus = data.get('CPU(s)')
    sockets = data.get('Socket(s)')
    cores = data.get('Core(s) per socket')
    return (cpus, sockets, cores)

Note that the functions _lscpu() and _verify_run() have their names starting with an underscore (Python weak “internal use” indicator) and thus, by convention, will not be exported by Salt to the public interface.

The Salt method cmd.run_all is used here to execute an external binary (lscpu) and grasp its standard output and error.

The function _verify_run() aims to catch any system error and, when necessary, abort the module execution. This code snippet shows the usage of the Python exceptions in Salt. We raise here a CommandExecutionError exception, declared in the Salt library salt.exceptions if a system error has occurred.

To end our module we implement a function which just calls _lscpu() and parses the user command line arguments (if any), or the module extra arguments, when out module is called by another script. A CommandExecutionError exception is raised for any invalid argument passed to our function.

def lscpu(*args):
    (cpus, sockets, cores) = _lscpu()
    infos = {
        'cores': cores,
        'logicals': cpus,
        'sockets': sockets
    }

    if not args:
        return infos

    try:
        ret = dict((arg, infos[arg]) for arg in args)
    except:
        raise CommandExecutionError(
            'Invalid flag passed to {0}.proc'.format(
            __virtualname__))
    return ret

This function lscpu() is public and will be available on all the Linux minions managed by Salt. Any public method that you define in a module can be invoked by prefixing its name with the corresponding virtual module (cpuinfo in our case):

salt '*' cpuinfo.lscpu

or, if you just need the number of logical CPUs:

salt '*' cpuinfo.lscpu logicals

We have extended Salt.

The final module

When we put all of the preceding code together, we end up with the following code:

'''
SaltStack module returning some information about the CPU
architecture.  This module parses the output of the command lscpu.
'''
# Import Python libs
import logging
# Import salt libs
from salt.exceptions import CommandExecutionError

__virtualname__ = 'cpuinfo'

def __virtual__():
    '''
    Only run on Linux systems
    '''
    if __grains__['kernel'] != 'Linux':
        return (False,
            'The {0} execution module cannot be loaded: '
            'only available on Linux systems.'.format(
            __virtualname__))
    return __virtualname__

log = logging.getLogger(__name__)

def _verify_run(out):
    '''
    Crash to the log if command execution was not
    successful.
    '''
    if out.get('retcode', 0) and out['stderr']:
        log.debug('Return code: {0}'.format(
            out.get('retcode')))
        log.debug('Error output\n{0}'.format(
            out.get('stderr', 'N/A')))
        raise CommandExecutionError(out['stderr'])

def _lscpu():
    '''
    Get available CPU information.
    '''
    try:
        out = __salt__['cmd.run_all']("lscpu")
    except:
        return None
    _verify_run(out)

    data = dict()
    for descr, value in [elm.split(":", 1) \
        for elm in out['stdout'].split(os.linesep)]:
        data[descr.strip()] = value.strip()

    cpus = data.get('CPU(s)')
    sockets = data.get('Socket(s)')
    cores = data.get('Core(s) per socket')

    return (cpus, sockets, cores)

def lscpu(*args):
    '''
    Return the number of core, logical, and CPU sockets,
    by parsing the lscpu command and following back to
    /proc/cpuinfo when this tool is not available.

    CLI Example:

        .. code-block:: bash

            salt '*' cpuinfo.lscpu
            salt '*' cpuinfo.lscpu logicals
    '''
    (cpus, sockets, cores) = _lscpu()
    infos = {
        'cores': cores,
        'logicals': cpus,
        'sockets': sockets
    }
    if not args:
        return infos
    try:
        ret = dict((arg, infos[arg]) for arg in args)
    except:
        raise CommandExecutionError(
            'Invalid flag passed to {0}.proc'.format(
            __virtualname__))
    return ret

You can find other examples in this GitHub page.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s