Intro to SaltStack

The New Configuration Management System

Presented by Justin Carmony / @JustinCarmony

NOTE: Clone & Setup the Vagrant Project

Slides split up into multiple sections.

Slides both vertical and horizontal.

Test Vertical Slide


We will be using Vagrant for this tutorial! If you haven't downloaded the project & started it up, please do so now!

About Presenter

  • Director of Development
    @ Deseret Digital Media
  • President of Utah PHP Usergroup
  • 9+ years of professional web development
  • Goofy dad

About This Tutorial

  • All Slides & code examples will be posted online
  • Feel free to ask questions
  • I want more of a discussion than in a typical presentation
  • Feel free to talk and/or contact me
    afterward the tutorial

Our Goal

  • Understand Some Server Challenges Facing Teams
  • Basic introduction to Salt about what it does
  • Implement some basic salt functionality for running PHP
  • Introduce you to some more advanced abilities for salt

Lets Try Breaking the Ice

Since lots of times you guys can be a really dead audience.

Who here has used a server config management tool before?

Number of Servers You Manage / Work With?

Have a Stage Environment that Mirrors Production?

What are some of the things you're hoping to get out of this tutorial?

Tutorial Setup Check

Lets Start

With a Story

You're a Happy Developer

You're Part of a
Small Team of Devs

You Have a New Website

On a small cloud server

You Start Getting Lots of Users ...

... and you need to quickly scale.

You ask your boss to hire a sys admin.

"We don't have the budget!"

So your team decides to ask:

who can run operations?

... and you start deploying more servers ...

... and some more ...

... and maybe just a few more ...

Until it is a giant mess!

Scenario: John gets Fired

You have a dozen servers

He has SSH keys on half of them.

How are you going to ensure you've

revoked his access everywhere?

The Problem:

Ad-Hoc Server Management

The Cloud

Problem: It's Easy!

  • Doesn't Require a Sys Admin
  • Simple to deploy many servers
  • But complex to solve large scaling issues
  • Long-term management requires
    fore-thought & planning

The Developer

Problem: Lack of Ops Experience

  • Can relatively easily setup a production environment
  • Typically doesn't know how to manage many servers long-term
  • Busy programming, last priority is "sysadmin stuff"

The Business

Problem: Lack of Money

  • Hard to justify a "Sys Admin"
    when devs can do an "okay" job.
  • Usually hit scaling problems
    before profitability

The Solution:

Developers need to think like Ops


We've been putting more dev into ops, but we need to put more ops into dev. Ops is a state of mind. Theo Schlossnagle

Need to Apply Best Practices

For Server Environments

  • Source Control for Configuration
  • Ensure Servers are Configured The Same
  • Automate Upgrades, Changes, & new Deployments


Configuration Management

Programmatic Approach
for Configuring Servers

What Can Configuration
Management Do?

  • Manage Packages
  • Manage Services
  • Manage Configuration
  • Manage Files
  • Manage Users

Popular Tools

  • Puppet
  • Chef
  • Ansible

So what is salt?

and how is it going

to help with my problems?

What Is Salt

SaltStack takes a new approach to infrastructure management by developing software that is easy enough to get running in minutes, scalable enough to manage tens of thousands of servers, and fast enough to communicate with them in seconds. SaltStack delivers a dynamic infrastructure communication bus used for orchestration, remote execution, configuration management and much more. About SaltStack,

A Different Approach

Salt is a simple system that focuses on two things:

  • Scalable, Fast Communication (Flow)
  • Quickly get information / data from a server (State)

Salt Basics

Master / Minion Setup

  • Minions connect to Master via ZeroMQ
  • Minions authenticate with master via AES (pub/priv) keys
  • Master can send module commands to minions to execute
  • Execution is in parallel
  • Modules are just python modules

What does this give us?

  • Extremely Scalable (~10k Minions per Server)
  • Great Flexibility (just write a python module for new functionality)
  • Simple, Avoids Complexity
  • Great Building Blocks for Automation


New since version 0.17

  • No need for a master or minion
  • Create a roster file
  • Authenticate & Communicate over SSH
  • Supports all functionality that doesn't require master/minion communication (i.e. peer-to-peer minion)

Built On Top of Salt

  • Configuration Management
  • Remote Execution
  • VM / Cloud Management

New / Upcoming Functionality

  • Monitoring
  • Web UI

100% Open Source

  • Everything is open source! Apache License
  • No "watered-down" community edition
  • 8th Most Unique Contributors on GitHub in 2012
  • Extremely Open & Friendly Community

Who Uses Salt

Installing Salt

Its so easy even I can do it

Basic Salt Setup

Setting Up The Master

  • Install salt-master via Salt Bootstrap, Pip, Apt, Yum, Source, etc
  • Edit Configuration (i.e. /etc/salt/master)
    • File Roots
          - /srv/salt
    • Pillar Roots
          - /srv/pillar
  • Restart salt-master (i.e. /etc/init.d/salt-master restart)

Setting Up The Minions

  • Install salt-minion via Salt Bootstrap, Pip, Apt, Yum, Source, etc
  • Set FQDN on the server (OR set ID Manually in Config)
  • Edit Configuration (i.e. /etc/salt/minion)
    • Set Master Server
    • Pillar Roots
          - /srv/pillar
  • Restart Minion

View Minion Keys

On the salt master, list the current keys

user@salt-master:~$ sudo salt-key -L
Accepted Keys:
Unaccepted Keys:
Rejected Keys:

Accept Minion Keys

Accept the key (view it before hand to make sure it's legit)

user@salt-master:~$ sudo salt-key -a
Accepted Keys:
Unaccepted Keys:
Rejected Keys:

That's it! We're ready to rock & roll.

Tutorial Project Review

  • Vagrant provisions 3 servers: master, minion1, minion2
  • Vagrant installs salt on the master & minions
  • Vagrant sets the configuration files for the master & minions
  • Minions connect to master, ready for their keys to be accepted

Vagrant Mounts

Vagrant will mount the project folder to the /vagrant directory on the servers.

Files we'll be modifying are under the saltstack directory:

  • etc/ - configuration files for master & minions
  • pillar/ - pillar data for minions
  • salt/ - sls state files & resource files

Tutorial Hands On

Accept the Minion Keys

Remote Execution

"Do my biding, minions!"

Targeting Minions

Use "salt" command on master to communicate to minions

Example: Test Ping to All Minions

user@salt-master:~$ sudo salt '*'

Targeting types of minions

salt 'db*'
salt -L,
salt -G 'os:Ubuntu'
salt -C 'G@os:Debian and web*'

Execute Command on Servers

Example: Restart Apache

user@master:~$ sudo salt 'web*' "/etc/init.d/apache2 restart"

Execute Modules on Servers

Example: Restart Apache

user@salt-master:~$ sudo salt 'web*' apache.signal restart

Lots of Modules

Here are just some of them:

apache, apt, cron, disk, file, mount, mysql, network, puppet, service, solr, state, test, useradd, win_disk, win_service, win_useradd

Tutorial Step

Lets try to run some commands!

salt \* "vmstat"
salt \* pkg.install "siege"
salt \* "siege -c 5 -r 5"

Configuration Management

Keeping your servers configured the same


  • Define a State Tree
  • Assign Parts of State Tree to Servers
  • Minion will download it's state definition from the tree
  • Minion will compare its current state vs state tree
  • Minion will make changes to match state tree & report back

What is State?

Example using English:

  • State Tree: Apache is installed & running
  • Server Minion: Apache is not installed nor running

Top.sls File

Defines which servers have which parts of the state tree applied to them.

Example top.sls file:

        - core.users
        - apache2
        - php5

Different Environments

        # Stuff for all servers
        # stuff for dev stage
        # stuff for stable stage
        # stuff for prod stage

Tree Maps to Files

Base is your file_roots

  • apache2 => apache2/init.sls
  • core => core/init.sls
  • => core/tools.sls

Anatomy of a State Declaration

Example of a user definition

justin:                         ## ID
    user:                       ## Type
        - present               ## Function
        - shell: /bin/bash
        - home: /home/justin
        - password: $1$MG4PbsH9$.w68M3yd/kUmESwq3cRM91

Lets Install & Manage Apache2

        - installed
        - running

        - managed
        - source: salt://files/etc/apache2/apache2.conf
        - watch_in:
            - service: apache2

Jinja Templates

    {% if grains['os'] == 'RedHat' %}
    - name: httpd
    {% elif grains['os'] == 'Ubuntu' %}
    - name: apache2
    {% endif %}

Applying the High State

You just call the state.highstate function

salt '*' state.highstate

What does High State Mean?

High state is one of the layers in the salt state system. If you really want to see under the hood you can read the docs.

Tutorial Hands-On

git checkout -b step1-states origin/step1-states

Steps in this section

  • Review salt/top.sls file
  • Review salt/common/init.sls file
  • Uncomment different includes in salt/common/init.sls
  • Execute salt \* state.highstate

More Advanced States

Lets try to manage as much as possible about your server.

Data Available in States

  • Grains
  • Pillar

Salt Grains

Static bits of information that a minion collects about the system when the minion first starts.

Random Tips for Using Salt

Common pitfalls while getting started with Salt

Debugging Salt

Run things in the foreground:

# Run master in foreground
/etc/init.d/salt-master stop
salt-master -L

# Run minion in foreground
/etc/init.d/salt-minion stop
salt-minion -L

Debugging Salt

Run things manualy from the minion

salt-call -l debug state.highstate

Other Tidbits

  • yaml chokes on tabs, make sure you use spaces
  • I use sublime text configured to always use spaces
  • Use in hosts file to set FQDN
  • Test what is the FQDN with:

python -c 'import socket;print(socket.getfqdn());'

You've been introduced to Salt

Now What?

Checkout My Salt AWS Demo

Uses Vagrant & AWS to setup a test 5 server environment.

Checkout Salt's Documentation & Tutorials

Salt has some very good docs (and getting better all the time).

Join the Super Friendly IRC Channel #salt


Blog/tweet/discuss your experience

It's Open Source, Get Involved

Very Pull Request friendly

(They even have accepted PR's from me!)