Ansible -- UDMY

 +++++++++++++++++++++++++++++++++++++++++++++++++++

 ANSIBLE :

Credit : James Spurin

This documentation is made for my reference from the video creator named : Named James Spurtin.

So every thing and anything related to Ansible is the contributor mentioned above.

I made this reference note so that I don't have to sit over video again and again and helps me to refresh my learning once again.

+++++++++++++++++++++++++++++++++++++++++++++++++++ 

Section 1 : Installation and Configuration of Ansible :

Ansible Requirements - Python , PIP , and Virtual Environments .

Ansible can be installed in a verity of different ways. for different OS and the advantages and disadvantages to these operations . We will learn PIP - Python Package Manager and how this could be used in a effective means at a managing and maintaining Ansible releases.

Finally we will learn about Virtual Environments in Python . And how we can make use of them with ansible

Ansible is Automated Framework written in Python . Of coure knowledge of python a benefit when we look into some of the advanced topics of this course . It is not a requirement for lerning and using Ansible.

Pythion 2.6 or  2.7 can be used with Ansible. Python 3 is being actively developed as a specific branch in the main Ansible github repository . At the time of creating this video it is already very stable. However this is still treated as a development branch . This course focuses on the use of Python 2 with the deruns wherever possible between Python 2 and Python 3 , where it involves development in Python . If you do wish to use python 3 with this course most I expect you will be sucessful with the use of the content . It is work in progress . Your knowledge may vary .

There are variety of ways in which Ansible can be installed.

Each with different Merits . The following slide summaries a free common approaches

Python VM:  In python a VM is an isolated area in which packages can be installed . They are created using an existing python installation. This could be version 2 or version 3 . The virtual environment will relate to the version of Python that was used to built it. You can create multiple virtual environments.

When the virtual environments are activated , when we install a package . It will install into a virtual environment for example if we use PIP . Issue the command

$ pip install ansible

This will install a cub version of Ansible available from PIPE PI pacjage Index into the virtual environment . if we want to install a specific version for example an earlier version of Ansible 2 . we can give this specific package of Ansible to PIP .  We can also do the same if we install a later version.

Virtual environment separate your working environments from the main Python Installation.  Providing a verity of advantages . It keeps your working environment clean as all changes are applied to your virtual environment and not on the system that has installed Python. There are also less dependencies using a super user. As virtual environments can be created by non root users. Finally virtual environment provide a convenient means for installing Ansible. In the corresponding videos we are going throigh a process of setting up an environment for the use of Ansible through out this course. 

ANSIBLE COURSE : ENVIRONMENT SETUP 

In this course we are going to take a look at 

1. Oracle Virtual Box

The video sessions make use of 8 hosts .

Oracle Virtual Box is a x86 Hypervisor, allowing you to run VMs on windows, MacOs , Linux and Solaris .

The videos sessions make use of 8 hosts

• Ubuntu C -  Is our Ansible Control Host. with -c being added to signify that this the control host
• CentOS 1 , 2 & 3 - Are all CentOS hosts
• Unbunti 1, 2 & 3 - Three Ubuntu hosts
• dnsmasq --  finally there is a host called dnsmasq as the name suggest is been confiured with DNS mask service.This acts as a named server for local hosts and as a forwarding DNS server. It is important to know that this machines acts as DNS server to all of the other machies. You will have to have this machine running for other virtual machines to run as expected,
  
• The virtual machines can be downloaded from the 

URL : https://goo.gl/b68auU

 Pause the video and download each machine in the background 

The 8 machines are been configured on virtual box using two Network adapters . As shown in the blue and green arrows. Both virtual machine adapter must be bridges with the same network adapter on your system . And that adapter must be providing an IP assignment with an internet connectivity through DHCP . And it is typically used in home internet routers to automatically provide IP addresses and internet connectivity . If you access the internet on your computer through your WIFI router then the adapter will be bridged in the adapter the two adapters too in your vrtual machine will be this WIFI adapter

If you connect an eithernet cable to this computer and automatically assign an IP and connectivity will be used in this adapter as a bridge to both the adapters in the virtual machines.

Adapter 1 signified by the green arrows has been configured in the system to use DHCP get an IP address , so when this is bridged . DHCP enabled adapter . it should automatically assign an IP address on adapetr1 for each of the instances

 Adapter 2  : signified by the blue arrow has been configured with static ip addresses , each of this instance has a 192.168.0.4x with dnsmasq bing 192.168.0.40

As this adapter is also bridged on your main network.

 

If you were able to set an ip address on your system or create a virtual interface on your system

For all the machine there are these users configured on them

Ansible on Linux :

Running Ansible as a control host on Linux .

Installing Ansible in a Ubuntu machine:

$ sudo apt-get update

$ sudo apt install ansible

installation location is shown by the below command.

$ which ansible

Check the version of the Ansible Installed.

Installing using APT Packages.

what are  going to do is use the APT to check for Ansible personal archives.to see this how this improves

We have restored that machines back to its original state . Now the ansible is removed and the system is without ansible. This time we are going to setup personal packages archive.

$ sudo apt-get update

$ sudo apt-get install software-properties-common

Now we add in the APT repository 

$ sudo apt-add-repository ppa:ansible/ansible

In comparison to the earlier installation this time you have more details about the installation.

He again restored it back to the earlier state :

Installing Ansible by VM and PIP

$ sudo apt-get update

The we will install the dependencies .

$ sudo apt-get install python-minimal virtualenv python-dev build-essential

we will make a directory to store 

$ mkdir ansible

$ cd ansible

we will create a virtual environment .

$ virtualenv venv27

We will not activate our virtual environment

$ source venv27/bin/activate

Check the python installion

$ which python 

$ which pip

We are not doing s pip install ansible

Now if you install anything using PIP will be installed in its virtual environment

$ pip install ansible

$ which ansible

This path is relevant to the virtual environment this is no system path

$ ansible --version

One of the advantages of have pip is that we have loads of control over pip.

I am going to uninstall ansible

$ pip uninstall ansible

Checking if git is installed

$ git 

we are going to install git

$ sudo apt install git  // installed using apt package manager

We can again install using PIP

$ pip install git+https://github.com/ansible/ansible

This can take a while for you are pulling it from Github. internet speed and stuff like that will depend on this installation.

$ ansible --version

Now that covers up our three main approaches of installing ansible on Ubuntu

Now we will see the other approaches of installing Ansible 

CentOS system

Installing Ansible by the creating your own RPM

First you are going to install your dependencies 

$ sudo yum install asciidoc python-jinja2 PyYAML git python-setuptools rpm-build python2-devel

we are just going to the /tmp directory

$ cd /tmp

 and we are going to clone the current version of Ansible.

$ git clone git://gitub.com/ansible/ansible.git

$ cd /ansible // go into the ansible directory that we cloned now.

 Now we will create the RPM, Now this will not work. But we will proceed to check how we can solve this.

$ make rpm

This has build our RPM -- however

RPM is installed in RPM

$ ls rpm_build    

Installing dependencies below

$ sudo rpm -Uvh rpm-build/ansible-2.5.0-100.git201712021324.1faa0a7.devel.e17.centos.noarch.rpm

What we see here is that this is build in with some dependencies . So for us to actually progress with this we need to install certain dependencies .

$ yum install python-paramiko python-six python2-cryptography sshpass

now again run the previous commands.

$  sudo rpm -Uvh rpm-build/ansible-2.5.0-100.git201712021324.1faa0a7.devel.e17.centos.noarch.rpm

$ which ansible

/usr/bin/ansible

$ ansible --version

 For centOS

$ sudo yum install epel-release 

$ sudo yum install ansible

$ ansible --version

To do this installation we must install , in order to do this is compiler and the corresponding compiler utilities . 

$ sudo yum install 'Development Tools'

 Install dependencies 

$ sudo yum install python-virtualenv  libffi-devel openssl-devel python-cffi PyYAML

$ mkdir ansible

$ cd ansible

 Create virtual environment

$ virtualenv venv27

$ source venv27/bin/activate

$ which python

/root/ansible/venv27/bin/python

$ which pip

Sometimes in CentOS and RHEL . we have issue installing using PIP. 

There is an easy fix for this .

$ pip install --upgrade setuptools

$ pip install ansible

7 : Validating Ansible Installation 

 We are going to look at ansible configuration shell command to check the os and then execute the script accordingly ion file and how it is going to vary depending on our installation .

 If you have installed Ansible , if you have configured your own ansible control host - by one of the methods.

• You should have an ansible folder  , 
• Within which you should have a sub directory venv27
• If so activate your virtual machine : source venv27/bin/activate

If instead you have decide to go with an alternative approach such as installing Ansible through system package . Please create an ansible directory now. You can continue with out the virtual environment now.

 We are going to use the same command to validate our system but this time we are going to work with the remote system and we are going to make use of an Ansible configuration and inventory file. Before we get to that when going for an installation process    

The above is by order of precedence

If there was an environment variable called ansible_config . it will attempt to use whatever file name this refers to. The  next priority is ./ansible.cfg . The dot represents the current directory. So it will look at the current directory for the file called ansible.cfg . The third priority is ~/.ansible.cfg . Current users home directory 

echo ~   -- will show the users path home directory 

Last is in the system location of etc : /etc/ansible/ansible.cfg  -- You will not be able to write to this location unless you are super user or executing commands in a way that provides super user access,

If we list we have all of these configuration files

There is another default file this is with relations to the hosts file . /etc/ansible/hosts . we are not going to use this approach , instead what will are going to doing is create our own inventory file and within that inventory file we are going to specify the hosts location.

He uses this method, again it is a system user path and would need super user privileges to use it.  /etc/ansible/hosts -- we are going to create our own hosts file.

The host file can be structured in a variety of different ways , the most common format is like any file in windows . It can be any YAML or JSON file. From now I am going to use the any format. 

vi  hosts

[ALL] -- Build in session called 

[all]

centos1                  -- added my host centos one

Technically here we didnt need to add the [All] GROUP , because all hosts is assigned the all group in ansible

we are going to use a file called ansible.cfg  

vim ansible.cfg

Creating a default session.

[default]

inventory = hosts   -- we are defining an inventory of hosts.

And lets try to ping the host : At this time it is going to fail but we go on by rectifying the issue as we go.

$ ansible all -m ping 

Because there is no SSH that is configure it is unable to make a connection to the host machine.

Once you run this command your ansible controller makes note of the hosts machine in the file below

$ cat /.ssh/known_hosts

Creating a SSH key 

$ ssh-keygen -H -F centos1

If we do the same again with the IP address

$ ssh-keygen -H -F 192.168.0.45

If you above and see this careful - the centos1. If we were to remove these entries . Let me just take them out of it.

$ vim ~/.ssh/known_hosts

remove all the entries from here.  And again if we re-run .

$ ansible all -m ping

 It once again prompts 

At the moment it is not ideal from an ansible perspective because every host that we connect to it is going to ask us whether or not you want to establish and continue connecting .

what we can do here is set an environment variable ANSIBLE_HOST_KEY_CHECKING=False . by setting it before the command 

$ ANSIBLE_HOST_KEY_CHEKKING=False ansible all -m ping 

This failed once again. But this time It has automatically added the host key to the node hosts 

Though this is a solution but not an ideal solution. 

what we want here is a permanent solution . I will remove it from the ssh key file again

$ ~/.ssh/known_hosts

$ vim ansible.cfg  -- lets edit our ansible configuration file.

added

host_key_checking = False

it fails again but it has automatically added that host file 

To solve this we need to configure SSH key.

Lets check out ssh connection  .For now I will feed in the password

$ ssh-keygen

At the moment the remote host doesn't even have a .ssh directory.

If we now have a look at centos1

And with in that is the authorised key fie

And remember what i was saying here about the permissions

And it is working as expected.

Now with the trust relationship in place. If we try our ping module. We can successfully ping

$ ansible all -m ping

I mentioned earlier by default all hosts belong to group all group . To prove this we are going to toggle the host file and remove the [all] from the file

[all]   -- removed this from the host file

centos1

And now we re execute the command once again

Before we go any further I want to show you that ansible is flexible in the way it can be used. For reference the equivalent can be done using the command line tool with out an inventory and with out a configuration file.

$ ansible all -i 

The -i expects an inventory file but we give a hostname provinding that we end this with a comma

$ ansible all  -i centos1, -m  ping 

 This with the exception of the host key checking is equivalent of what we have in the inventory and the ansible configuration file.

This makes ansible make it a very powerful command line tool. Depending of the scope of the task that you may need to perform. And may be viable to choose any one approach than the other .

The adhoc command line tool can be used to with any of the ansible modules 

The ansible "debug" module is used to print statements during its execution.

$ ansible all -m debug

By default it is using a message there "Hello World" , because we haven't supplied one.

Lets use the command line tool to look at the documentation for this.

$ ansible-doc debug

You can see here the command line tool accepts a variety of different options. 

Therefore ansible command line tool accepts arguments and we can pass arguments by 

-- args  / -A

$ ansible all debug --args='msg="This is a custom debug message"'

Lets use verbosity 

If we were to set the debug module parameter for verbosity to 3 , it will only run when ansible is running in a particular verbosity mode. For example; 

The output there shows it was skipped

Ansible command accepts verbosity through the increments of V with a proceeding - So when there is no V option given . The verbosity level is 0 .

1v -- v -- means verbosity level 1

2 v -- vv -- means verbosity level 2

3v --vvv -- means verbosity level 3

Lets setup other host machines for our lab -- got to hosts file

if we run the ping module against all group now . it is going to succeed for one and fail for all of the others.

We can also do this for individual groups 

$ ansible centos -m ping

Again it is going to succeed for one and fail for other two.

You can also refer to the [all] group with '*' sign

$ ansible '*' -m ping

to resolve this we need the ssh keys on all of these hosts machines. And the easiest way for us to do this is to just be with some simple bash loopin here.

Writing a loop script

We have now successfully copied the shh key to the all the host with the for loop.

And now lets check it up 

$ ansible all -m ping -o

-o flag reduces the output to a single line .

You can check the list of host that you will be executing the code via ansible command line.

$ ansible centos  --list-host

$ ansible centos  --list-host

$ ansible all --list-host

It can also be used with individual hosts

You have an option where you can use pattern match.

$ ansible ~.*3  --list-host

8 : Ansible Inventories :

Ansible Inventories :

Now that we have our environment setup . We can get in to master Ansible. 

As you can see there is a directory for each of the session in this course. 

You can navigate into the directories 

here you can specify the user specifically to indicate that the user that we will use to connect to this hosts is root

so if you ping the centos group this should fail for we haven't setup our root ssh keys

Adding the root ssh keys to the centos hosts

Enter the password for once and it will be copied to the hosts.

Now our connectivity looks like the below

For unbuntu we will be using a different approach , we will be connecting as packt user and then connecting as root.

Looks at the changes to the hosts file

Later we will be talking about Ansible Vault to secure the password in a better way.

After applying the changes our configuration now looks like the following.

The ping command that we just ran is doing a lot more in the background behind the scenes . For unbuntu hosts we are connecting as packt and then we sudo in as root. 

I recommend you to take a look at the following URL there is a variety of content here, 

There is a variety of useful content here which you should cover regardless of your setup . For example if you are running on different SSH ports , have different SSH keys that you use in for different system , something that is very common with cloud providers , or if your python installation is in a different location on the remote host . This page has a loads of different scenarios as to how to handle them .

I am going to change one of our system intentionally to show you how this could be of use. I am going to change centos1 so that it is going to run on a different ssh port . So we will use 2222 instead of 22 

Connect across to centos1

let me just do that once again . I want to connect as root

Lets edit the sshd config

vim /etc/ssh/sshd_config

to let the linux system know that we have changed the ssh key port we need to copy the above mentioned command and run the same.

If we run this command this command is going to fail , and the reason it fails is that it does not have semanage utility 

We can quickly resolve this 

$ yum  -y install policycoreutilis-python

$ semanage port -a -t ssh_port_t -p tcp 2222

we are also running on centos7 so we also need to open the firewall for these ports .

$ firewall-cmd --permanent --add-port=2222/tcp

we will not reload the firewall

$ firewall-cmd --reload

The last thing you want to do i start the ssh service. 

$ service sshd restart

easy way of checking it 

$ ssh 0 -p 2222    -- 0 is a short form for 0.0.0.0 which is technically local host.

With those changes , the ping command is going to fail for centos and succeed for all of the others.

$ ansible all -m ping

Lets check if we can connect from unbun2 to centos 2222

$ telnet centos1 2222

Now we need to accommodate that port change in our ansible configuration file .

And now if we can run the ping command it should be able to connect to the centos machine.

If you look at section 5 there is another way to specify this

After the configuration , our configuration now looks like the following.

As well as using SSH as the connection type we can where applicable use local to connect to our local system like we did when we were using the -c with the ansible tool. In our validating ansible video.

If we take a look at revision 6

$ ansible control -m ping

if you look at the host file there is , we have commonality between all of the unbuntu hosts and centos 2 & 3 

with in the host file you can simplify this with in ranges .  example below

Just to see this still works

We still however have some duplication , where we have ansible_user=root for both we can mitigate this in the centos group where where we have ansible_user defined for both . We can mitigate this by use of group vars . making every host in the centos group automatically receive the host file of ansible_user = root. We will also look how to do this for unbuntu group.

Now that group variable will be applied to every centos group.

What we have now is a mixture of our control host and what could be considered our linux host - the centos and the ubuntu . It would be good if we easily distinguish between the linux groups . Fortunately 

Ansible has a children declaration which us very useful for this.

So what we are going to do now is define a parent group of linux with both centos and ubuntu as the children .

[linux:children]

centos

ubuntu

because of that , this group will inherit all of the members of centos and ubuntu.

And if we perform a ping against that specific group 

$ ansible linux -m ping -o

You can see we have targeted the ubuntu and the centos groups.

If you recall from section 1 , we saw the all group and just a reminder technically all hosts are the part of all group. We can use this knowledge to apply variables to all hosts  by using [all:vars]

Of course 1234 is not a valid port for this case. And our connections will fail but it is a good way of demonstrating this functionality . If you look back here at centos1 we have an ansible port entry. specific to that host. And specific host variables take precedence  over all .

So if we run our ping against all our hosts

$ ansible all -m ping

You can apply group var the parents and all children will receive the vars , but as we saw all groups hosts specific hosts like the centos1 take precedence .

we should have the same outcome .

You can write a  host file into a YAML file or a JSON file. Example of a yaml file.

if you check the ansible.cfg

[defaults]

inventory = hosts.yml

host_key_checking = False

The important thing here is that you must have this extension . you need to have this yml . Ansible uses this to work out the format of the file.

And that is using the yaml format and that is working as expected. 

JSON file :

In section 14 : we have reverted our ansible file to text file

you can also specify the inventory in the command line.

$ ansible all -i hosts -m ping -o

$ ansible all -i hosts.yml -m ping -o

JSON version

$ ansible all -i hosts.json -m ping -o

Overwriting existing variables set in the inventory, we set that ansible_port = 2222

$ ansible linux -m ping -e 'ansible_port=22'

except for centos -- which is listening on 2222 , all other host will take up 22 and connect to it 

since cenos1 is explicity mentioned in the hosts file it will fail

Ansible Module :

we will be looking at the following module.

And finally we will be looking at the Ansible-doc module to quickly check the options. 

The first one that we look at is the setup module. Used for gathering facts when executing play-book.

$ ansible centos1 -m setup   --   we are giving a direct host

And that comes back with a loads of information. 

$ ansible centos1 -m setup  | more

$ ansible centos1 -m setup | more

we can see that it is got the IP information , we can see information about the devices that are available to the systems , you can see sda, sda1, sda2 and more device information and information about the Ansible distribution as well . Loads of information that gets gathered here .

Next Module : is File module

$ date 

$ ansible all -m file -a 'path=/tmp/test state=touch'

If you look at this you will see it is a zero length file. And the timestamp currently leads to the date that we saw in the previous command  

You can see the out put in colour

Ansible used the following colour schemes 

Lets write a new command for changing the file permissions

$ ansible all -m file -a  "path=/tmp/test state=file mode=600'

The change is made across all the servers.

If you change the permission and rerun the command once again there will be no change . This i called idem-potency where a change will not be made twice.

The second time when you run the same command you will have a green output.

we can run this command over and over and it will be deemed a success. As the file is in the correct state

Lets put this to test , if we change the mode of this file.

if we change the mode on the local control host alone to 644 and rerun the below command 

then the change will be applied only to the local host where you have changed the mode on. The other servers since you didn't make any changes on them then continue to remain in the same state.

$ ansible all -m file -a "path=/tmp/test state=file mode=600'

You will find 

The next module that you are looking at it is the copy module. 

We simply created a file called temp/x

$ ansible all -m copy -a 'src=/tmp/x dest=/tmp/x'

The copy modules uses checksums to validate whether the copying was needed or not.

We can also use ansible to copy file from the remote to the remote - COPYING FILES ON THE REMOTE MACHINE TO THE SAME REMOTE MACHINE.

$ ansible all -m copy -a 'remote_src=yes src=/tmp/x dest=/temp/y'

remote_src=yes  -- just to tell ansible that we are working on the remote source

One of my favourite module in Ansible is the command module. 

it allows you to use your ansible configurations to execute remote commands . 

$ ansible all -m command -a 'hostname' -o

A return code - rc of 0 states that the commands were successful. and the standout shows the hostname .

the command module will be the default module that will be used for Ansible adhoc commands.

why we say that because even if you skip the -m flag your command module will work by default

$ anisble all -a 'hostname' -o

we get the same out put 

If you recall in the previous session we originally configured our ssh keys so we can connect to all of our guests that is packt and then we updated this to connect as root . We can verify that we have root connectivity using the command module

$ ansible all -a 'id'

we do not have to specify the -m command , we just need to specify the unix module id

A feature of the command module which is particularly useful that is the creates and removes variable and can be used for idempotence 

command module

and also the remove option . 

lets take a look at this. 

$ ansible all -a 'touch /tmp/test_copy_module creates=/tmp/test_copy_module'

Now if we rerun this again

what it actually did is that it skipped the file since that skip copy module file already exists. 

The file already exists there therefore you , therefore it is termed successful. You are wondering why we haven't had those yellow colour for there is nothing changing here. In all case we have rc = 0 , then it is deemed successful. 

removing the file now

$ ansible all -a  rm '/tmp/test_copy_module removes=/tmp/est_copy_module'

If you run this once again.

if you scroll up Ansible has give out a warning , stating what would be the approach to achieve the desired outcome .

If you follow the ansible's recommandation.

$ anisble all -m file -a 'path=/tmp/test_copy_module state=absent' 

go to the ansible-website looks for fetch module. used to fetch a file from the remote machine.

we are creating a file on one of the host centos1

$ ansible centos1 -m file -a 'path=/tmp/test_module.txt state=touch mode=600' 

And now we want to pull out that file from the remote centos1 system to your syste, 

$ ansible centos1 -m fetch -a 'src=/temp/test_module.txt dest=/tmp/test_module' 

The above command will copy the file from the remote host and place it in the folder you have set .

This command has actually made a directory for us. You may assume that is has copied the file to the directory but instead it creates a directory and then places the file inside it.

ansible-doc command 

$ ansible-doc file

10 : YAML