SaltStack SSH

SaltStack is an automation solution, useful for data center orchestration and configuration management. This guide teaches how to install salt-ssh, the agentless module of SaltStack.

Requirements

There is a low chance that the requirements are not already met, but I will still include this part, just in case:

- x64 system for the server machine - SSH server on the target machines - python on server and target machines - pip tool for managing python on server machine - your private ssh key on the server machine and your public one installed on the target ones

For ssh server, you can use OpenSSH if you have nothing installed:

sudo apt-get install openssh-server 

For python, we first have to install some dependencies:

sudo apt-get install build-essential checkinstall
sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev

Then, download the package and extract:

wget http://python.org/ftp/python/2.7.5/Python-2.7.5.tgz
tar -xvf Python-2.7.5.tgz
cd Python-2.7.5

Lastly, use the following commands:

./configure
make
sudo checkinstall

Now you can install pip:

sudo python get-pip.py

You have instruction about how to set up a ssh key pair here: [1]

Install and Set up Salt-SSH

The next command sets up the salt-ssh package:

pip install salt-ssh

Salt-ssh uses a roster file to connect to other machines via ssh. This file shall be located in /etc/salt/ directory. if it does not exist, create it:

cd /etc
sudo mkdir salt
cd salt
sudo nano roster

Inside this file you can set up the target systems as follows:

System_Name:
  host: ip_address
  user: User_name

Change

 System_Name 

with the name you want to identify the machine with (can be any name you want),

 ip_address 

with the target's ip address and

 User_name 

with the name of the user that has ssh access to that system.

Testing

A simple way to test the setup is running the following command:

 sudo salt-ssh -i '*' test.ping 

The outcome, if successful should be the next output:

System_Name:
      True

Useful Scripts

Roster_Loader

Provided a text file "list.txt" with an ip address on each row, here is a script that formats the roster file in etc/salt/ ,creating minions for each address, named minion1, minion2 and so on, that asks for a username for the user fields and also with the sudo flag set for true. feel free to modify it as to suit your needs:

#!/bin/bash
# Program name: roster_loader

if  [ -z "$1" ]; then echo "Need file of machines to add to roster, one address per line" && echo "roster_loader /path/to/file" && exit;
else echo "Please insert the username from the remote host"
read u
i=1
 
cat $1 |  while read output
do
    echo "minion$i:" >>/etc/salt/roster
    echo "  host: $output" >>/etc/salt/roster
    echo "  user: "$u >>/etc/salt/roster
    echo "  sudo: True" >>/etc/salt/roster
    echo"" >>/etc/salt/roster
((i++))
done
fi

Instructions for Installing Salt Minions via SSH

Master

Firstly, let's install Salt Master:

curl -L https://bootstrap.saltstack.com -o install_salt.sh

sudo sh install_salt.sh -M # the -M flag symbolises that the master will be installed as well

The minions

Now, provided the fact that you have a roster with all machines configured and responding properly, you can use the following commands to install salt minions on all the supervised nodes (may need permission hacks to allow existing non-root user key to work). The '*' means that the command will apply to all the hosts in the roster, you can use an IP address or hostname instead. cmd.run is the salt command that means "run the next thing in shell" so whatever is between will take bash effect.

salt-ssh '*' cmd.run 'curl -L https://bootstrap.saltstack.com -o install_salt.sh' # deploying the repo

salt-ssh '*' cmd.run 'sudo sh install_salt.sh' # installing the minion.

salt-ssh '*' cmd.run 'rm -rvf /etc/salt/minion.d/masterless.conf' # removes the masterless.conf. without this command the minion won't look for a master, and will "listen to itself". deleting it removes the issue

salt-ssh '*' -i cmd.run "echo 'your_master_ip salt'>> /etc/hosts " # adds the ip of your master under the name 'salt'; that's what the minion looks for. replace your_master_ip with the actual ip

salt-ssh '*' -i cmd.run "sudo systemctl restart salt-minion" 

Now the minions are sending the keys to the master for authentication. The master-minion system works in the opposite direction, the minions call the master. Execute the next command as root. Salt-Master listens only to root when accepting keys:

salt-key -A

That should be all. to test, run the next command:

sudo salt '*' test.ping

Renaming a minion

When the hostname of a minion is changed the name on salt-master will remain the same, but it can be changed manually:

salt-key -d <minion-old-name>     #on salt-master
vi /etc/salt/minion_id     #on salt-minion, change the name in that file
systemctl restart salt-minion      #on salt-minion
salt-key -A     #on salt-master

Troubleshooting

If

 sudo salt-ssh -i '*' test.ping 

fails with

 Permission denied (publickey,password). 

but connecting by ssh still works, then the home/.ssh/ may not have the right ownership. Try

 sudo chown -R username:username /home/username/.ssh 

where username is the rightful

owner of the key and directory. Also useful in the case of not being able to deploy any ssh key . Changing the settings in sudoers (by sudo visudo)

so that the user has paswordless sudo may also solve the problem.

Also, you may try to manually add the salt key if sudo salt-ssh -i '*' test.ping does not work:

ssh-copy-id -i /etc/salt/pki/master/ssh/salt-ssh.rsa.pub user@target_ip

In the end, if all else fail, and the "Permission denied (publickey,password)." message persists, try:

sudo salt-ssh -l debug '*' test.ping

This will give you an insight about what happens in the back. most of the time it's a permission issue, or the lack of some ssh capabilities.

Errors

If initial deployment stage fails with any of the following messages in the output, please follow steps described to resolve. NB unsupported OS versions i.e. end of life ones are unlikely to be supportable by Salt.

bin/bash: curl: command not found

Try:

salt-ssh -i '*' cmd.run 'sudo apt-get update'
salt-ssh -i '*' cmd.run 'sudo apt-get install curl'

Or manually installing curl on the machines that have this error logged, then try step 1 of Salt deployment again.

ImportError: No module named backports.ssl_match_hostname

On each machine affected, do

apt-get update
apt-get install python-pip
pip install backports.ssl_match_hostname

Then try step 1 of Salt deployment again.

NB python-pip is in the Universe repo so may need to be enabled on the machines with this error.

locale.Error: unsupported locale setting

If you get this error when using pip install the following command worked for me:

export LC_ALL=C