diff --git a/Doc/Common/glossary.xml b/Doc/Common/glossary.xml index a4a27fb1bf08ad8ea0e9f8beafee455ebc7c1299..257b79f7106494bd0675d1e1468c1b55877a39d1 100644 --- a/Doc/Common/glossary.xml +++ b/Doc/Common/glossary.xml @@ -703,6 +703,16 @@ </glossdef> </glossentry> + <glossentry xml:id="glo_SNMP"> + <glossterm><acronym>SNMP</acronym></glossterm> + + <glossdef> + <para><link + xlink:href="https://en.wikipedia.org/wiki/Simple_Network_Management_Protocol">Simple + Network Management Protocol</link></para> + </glossdef> + </glossentry> + <glossentry xml:id="glo_Soft"> <glossterm>Software</glossterm> @@ -818,7 +828,7 @@ </glossentry> <glossentry xml:id="glo_ssh"> - <glossterm><acronym>Ssh</acronym></glossterm> + <glossterm><acronym>ssh</acronym></glossterm> <glossdef> <para><link diff --git a/Doc/Sdi/gettingStarted.xml b/Doc/Sdi/gettingStarted.xml index 27f5472053c04fed70ace8c0cb1872a7a2461e35..992bf8d4d23825a10919080eb0dbfc474d7db973 100644 --- a/Doc/Sdi/gettingStarted.xml +++ b/Doc/Sdi/gettingStarted.xml @@ -10,7 +10,7 @@ xmlns:db="http://docbook.org/ns/docbook"> <title>Getting started</title> - <section xml:id="sdiAccessVm"> + <section xml:id="sdi_sect_AccessVm"> <title>Accessing your virtual machine</title> <para>This lecture requires practical exercises. Each group will have @@ -97,9 +97,10 @@ Welcome to Ubuntu 14.04.2 LTS (GNU/Linux 2.6.32-19-pve x86_64) * Documentation: https://help.ubuntu.com/ Last login: Fri Mar 27 08:29:40 2015 from 192.168.1.66</screen> - <para>Since password access is generally being considered insecure (e.g. - due to insufficient length or poor choice of password) we will configure - public key authentication by a public/private key pair:</para> + <para>Since password access is generally being considered insecure + (<abbrev>e.g.</abbrev> due to insufficient length or poor choice of + password) we will configure public key authentication by a public/private + key pair:</para> <orderedlist> <listitem> @@ -177,4 +178,14 @@ Permission denied (publickey).</screen> </listitem> </orderedlist> </section> + + <section xml:id="sdi_sect_configVm"> + <title>Global configurations</title> + + <para>Locale issues as being described in <link + xlink:href="https://ubuntuforums.org/showthread.php?t=1720356">LC_ALL = + (unset). Googled solutions don't work</link> may be fixed by <link + xlink:href="https://ubuntuforums.org/showthread.php?t=1720356#post_10632770">»Answering + myself«</link> reply.</para> + </section> </chapter> diff --git a/Doc/Sdi/icinga.xml b/Doc/Sdi/icinga.xml index 3b9e89e5ea6663245a391315615a2907743f970e..4922d0f6cf1df9c0550dccde4f554d56481dc0c9 100644 --- a/Doc/Sdi/icinga.xml +++ b/Doc/Sdi/icinga.xml @@ -12,9 +12,8 @@ <title>Icinga</title> <para>With respect to more current modules we start from the more recent - icinga2 packages provided by the <link - xlink:href="https://packages.icinga.com/debian/#indexlist">Icinga Debian - package repository</link>.</para> + <link xlink:href="https://packages.icinga.com/debian/#indexlist">icinga2 + packages</link>.</para> <itemizedlist> <listitem> @@ -67,12 +66,12 @@ <orderedlist> <listitem> - <para>Base system + icinga web</para> + <para>Base system + <productname>Icinga</productname> web</para> </listitem> <listitem> - <para>Setting up <productname>Icinga 2</productname> API (prerequisite - for the <quote>director</quote> module)</para> + <para>Optional: Setting up <productname>Icinga 2</productname> API + (prerequisite for the <quote>director</quote> module)</para> </listitem> <listitem> @@ -87,7 +86,8 @@ <productname>Nginx</productname> configuration can be achieved using:</para> - <screen>icingacli setup config webserver nginx --document-root /usr/share/icingaweb2/public</screen> + <screen>icingacli setup config webserver {<emphasis role="red">apache</emphasis>|<emphasis + role="red">nginx</emphasis>} --document-root /usr/share/icingaweb2/public</screen> </listitem> <listitem> @@ -122,9 +122,277 @@ </itemizedlist> </tip> - <section xml:id="sdi_icinga2_checkBySsh"> - <title>Remote check by ssh</title> + <section xml:id="sdi_icinga_sect_functional"> + <title>Functional checks</title> + + <orderedlist> + <listitem> + <para>Install <productname>Nagios</productname> plugins by executing + <command>aptitude</command> <option>install</option> + <option>nagios-plugins</option>.</para> + </listitem> + </orderedlist> + </section> + + <section xml:id="sdi_icinga_sect_snmp"> + <title><xref linkend="glo_SNMP"/> based checks</title> <para/> </section> + + <section xml:id="sdi_icinga2_checkBySsh"> + <title><xref linkend="glo_ssh"/> based checks</title> + + <para>Our scenario involves checking <filename>/var/log</filename> not + exceeding a given file system size. A violation may indicate + <abbrev>e.g.</abbrev> <command + xlink:href="https://linux.die.net/man/8/logrotate">logrotate</command> not + being set up properly. The subsequently described steps assume the + <productname>Icinga</productname> software running on host + sdi10a.mi.hdm-stuttgart.de / 141.62.75.120 querying target host + sdi10b.mi.hdm-stuttgart.de.</para> + + <para>Target host configuration steps:</para> + + <orderedlist> + <listitem> + <para>Download the <filename>check_file_size.sh</filename> plugin to + <filename>/usr/lib/nagios/plugins</filename>.</para> + </listitem> + + <listitem> + <para>Test <filename>check_file_size.sh</filename> manually by + choosing appropriate warning and critical threshold values:</para> + + <screen>sdi10b#> /usr/lib/nagios/plugins/check_file_size.sh /var/log --maxwarn 1000 --maxcrit <emphasis + role="red">1500</emphasis> +FILE Critical: Size of <emphasis role="red">1610 > 1500</emphasis> for /var/log + +sdi10b#> /usr/lib/nagios/plugins/check_file_size.sh /var/log --maxwarn <emphasis + role="red">1000</emphasis> --maxcrit 2000 +FILE Warning: Size of <emphasis role="red">1610 > 1000</emphasis> for /var/log + +sdi10b#> /usr/lib/nagios/plugins/check_file_size.sh /var/log --maxwarn 2000 --maxcrit 4000 +FILE OK: All files (1) fall within requested parameters +</screen> + </listitem> + + <listitem xml:id="sdi_icinga_listitemRemoteExecScript"> + <para>Prepare for remote execution. Create a directory + <filename>/etc/nagiosBySsh</filename> and the following executable + bash script <filename>/etc/nagiosBySsh/nagioscheckssh</filename> + within:</para> + + <programlisting language="bash">#!/bin/bash + +if [ -z "$SSH_ORIGINAL_COMMAND" ]; then + echo "Environment variable »SSH_ORIGINAL_COMMAND« undefined" +else + exec $SSH_ORIGINAL_COMMAND +fi + +#end</programlisting> + + <para>This script will later be executed by a remote <xref + linkend="glo_ssh"/> call. The actual command to be executed will be + provided by the environment variable + <varname>SSH_ORIGINAL_COMMAND</varname>.</para> + </listitem> + + <listitem> + <para>Simulate remote execution by using <command + xlink:href="https://linux.die.net/man/1/env">env</command> for setting + the desired environment variable:</para> + + <screen>sdi10b#> env SSH_ORIGINAL_COMMAND='/usr/lib/nagios/plugins/check_file_size.sh /var/log --maxwarn 1000 --maxcrit 1500' /etc/nagiosBySsh/nagioscheckssh +</screen> + + <para>Depending on the chosen parameter values the resulting output + should be similar to:</para> + + <screen>FILE Critical: Size of 1610 > 1500 for /var/log</screen> + </listitem> + + <listitem> + <para>For the sake of security a non-privileged account will be used + for remote <xref linkend="glo_ssh"/> execution. Create a system + account <code>nagioscheck</code> for this purpose:</para> + + <screen><command>adduser</command> <option>--shell</option> <option>/bin/bash</option> <option>--system</option> <option>nagioscheck</option></screen> + </listitem> + + <listitem> + <para>As user <code>nagioscheck</code> create an <xref + linkend="glo_ssh"/> pair of keys using an empty pass phrase:</para> + + <screen>sdi10b#> su - nagioscheck +sdi10b$> ssh-keygen +Generating public/private rsa key pair. +Enter file in which to save the key (/home/nagioscheck/.ssh/id_rsa): +Created directory '/home/nagioscheck/.ssh'. +Enter passphrase (empty for no passphrase): +Enter same passphrase again: +Your identification has been saved in <emphasis role="red">/home/nagioscheck/.ssh/id_rsa</emphasis>. <co + xml:id="sdi_icinga_ncheckPrivateKey"/> +Your public key has been saved in <emphasis role="red">/home/nagioscheck/.ssh/id_rsa.pub</emphasis>. <co + xml:id="sdi_icinga_ncheckPublicKey"/> +The key fingerprint is: ...</screen> + </listitem> + + <listitem> + <para>The private <xref linkend="glo_ssh"/> key <coref + linkend="sdi_icinga_ncheckPrivateKey"/> will later be used for remote + query invocations. After moving it to the querying host it may deleted + on the target host for security reasons.</para> + + <para>Configure the corresponding public key <coref + linkend="sdi_icinga_ncheckPublicKey"/> to allow for ordinary remote + <xref linkend="glo_ssh"/> login:</para> + + <screen>sdi10b#> su - nagioscheck +sdi10b> cd ~/.ssh +sdi10b> cp id_rsa.pub authorized_keys</screen> + </listitem> + + <listitem> + <para>We now restrict remote logins to execution of + <filename>/etc/nagiosBySsh/nagioscheckssh</filename> from <xref + linkend="sdi_icinga_listitemRemoteExecScript"/>. Modify + <filename>/home/nagioscheck/.ssh/authorized_keys</filename> to + contain:</para> + + <programlisting language="bash">from="141.62.75.110",command="/etc/nagiosBySsh/nagioscheckssh" ssh-rsa AAAAB3N...LKPFSJo5 nagioscheck@sdi10b</programlisting> + + <para>Explanation: <xref linkend="glo_ssh"/> connections origination + from 141.62.75.110 will receive the execution result of + <filename>/etc/nagiosBySsh/nagioscheckssh</filename>. In particular no + login shell will be provided.</para> + + <caution> + <para>The <option>from</option> option really requires an IP rather + than a <xref linkend="glo_DNS"/> name.</para> + </caution> + </listitem> + </orderedlist> + + <para><productname>Icinga</productname> host configuration steps:</para> + + <orderedlist> + <listitem> + <para>Copy the target host's + <filename>/home/nagioscheck/.ssh/id_rsa</filename> private key to + <filename>/etc/icinga2/nagioscheck_id_rsa</filename>. Since the + <productname>Icinga</productname> process running with the nagios user + is requires read access we change the ownnership accordingly: </para> + + <screen>chown nagios.nagios nagioscheck_id_rsa</screen> + </listitem> + + <listitem> + <para>The <productname>Icinga</productname> daemon runs with the + effective user's id <code>nagios</code>.We thus have to assure remote + <xref linkend="glo_ssh"/> login as user <code>nagios</code>:</para> + + <screen>ssh -i /etc/icinga2/nagioscheck_id_rsa nagioscheck@sdi10b.mi.hdm-stuttgart.de \ + "/usr/lib/nagios/plugins/check_file_size.sh /var/log \ + --maxwarn 1000 --maxcrit 1500" +The authenticity of host 'sdi10b.mi.hdm-stuttgart.de (141.62.75.120)' can't be established. +ECDSA key fingerprint is SHA256:4L5rfTIJPu1lr1gpTyvaywDE01W55roZjNGKkni/060. +Are you sure you want to continue connecting (yes/no)? yes +Warning: Permanently added 'sdi10b.mi.hdm-stuttgart.de,141.62.75.120' (ECDSA) to the list of known hosts. +du: cannot read directory '/var/log/samba': Permission denied +du: cannot read directory '/var/log/unattended-upgrades': Permission denied +FILE Critical: Size of 1742 > 1500 for /var/log</screen> + + <para>Explanation: Remote command execution of + <filename>/usr/lib/nagios/plugins/check_file_size.sh</filename> as + user <code>nagioscheck</code> lacks privileges to completely read the + entire /var/log directory tree. This will be fixed in the next + step.</para> + </listitem> + + <listitem> + <para>Turn back to your target server and allow <command + xlink:href="https://linux.die.net/man/8/sudo">sudo</command> execution + of <filename>/usr/lib/nagios/plugins/check_file_size.sh</filename> to + user <code>nagioscheck</code> without providing a password. Start + by:</para> + + <screen>sdi10b#> apt-get install sudo</screen> + + <para>Subsequently create a file + <filename>/etc/sudoers.d/nagios</filename> to contain:</para> + + <programlisting language="unset">nagioscheck ALL=NOPASSWD : /usr/lib/nagios/plugins/check_file_size.sh</programlisting> + + <para>You should now be able to initiate <command + xlink:href="https://linux.die.net/man/8/sudo">sudo</command> + privileged execution from your <productname>Icinga</productname> + host:</para> + + <screen>ssh -i /etc/icinga2/nagioscheck_id_rsa nagioscheck@sdi10b.mi.hdm-stuttgart.de \ + "<emphasis role="red">sudo</emphasis> /usr/lib/nagios/plugins/check_file_size.sh /var/log \ + --maxwarn 1000 --maxcrit 1500" +FILE Critical: Size of 1726 > 1500 for /var/log</screen> + </listitem> + + <listitem> + <para>Configure an <productname>Icinga</productname> check command in + <filename>conf.d/commands.conf</filename>:</para> + + <programlisting language="none">object CheckCommand "<emphasis + role="red">by_ssh_file_size</emphasis>" { + import "by_ssh" + + vars.by_ssh_command = "sudo /usr/lib/nagios/plugins/check_file_size.sh --maxwarn $by_ssh_file_size_warn$ --maxcrit $by_ssh_file_size_crit$ $by_ssh_file_size_path$" + vars.by_ssh_identity = "/etc/icinga2/nagioscheck_id_rsa" + vars.by_ssh_logname = "nagioscheck" + +# Parameters by_ssh_file_size_warn, vars.by_ssh_file_size_crit and +# by_ssh_file_size_path will be defined in service or host definition +}</programlisting> + </listitem> + + <listitem> + <para>Define an <productname>Icinga</productname> template in + <filename>conf.d/services.conf</filename>:</para> + + <programlisting language="none">apply Service for (path => config in <emphasis + role="red">host.vars.paths</emphasis>) { + import "generic-service" + + check_command = "<emphasis role="red">by_ssh_file_size</emphasis>" + + vars += config +}</programlisting> + </listitem> + + <listitem> + <para>Finally add two suitable host service checks in + <filename>conf.d/hosts.conf</filename>:</para> + + <programlisting language="none">object Host "sdi10b.mi.hdm-stuttgart.de" { + import "generic-host" + + address = "sdi10b.mi.hdm-stuttgart.de" + + vars.paths["/var/log size"] = { + by_ssh_file_size_warn = "500" + by_ssh_file_size_crit = "5500" + by_ssh_file_size_path = "<emphasis role="red">/var/log</emphasis>" + } + + vars.paths["/var/lib size"] = { + by_ssh_file_size_warn = "50000" + by_ssh_file_size_crit = "900000" + by_ssh_file_size_path = "<emphasis role="red">/var/lib</emphasis>" + } + + vars.notification["mail"] = { + groups = [ "icingaadmins" ] + } +}</programlisting> + </listitem> + </orderedlist> + </section> </chapter> diff --git a/Doc/lectures.xml b/Doc/lectures.xml index b6563212ca117aab80be91a3ee5f9ac72b8deab1..05a9a5bb7fa71879c07abba30acee30013bc1fe4 100644 --- a/Doc/lectures.xml +++ b/Doc/lectures.xml @@ -196,7 +196,7 @@ <xi:include href="Sdi/docker.xml" xpointer="element(/1)"/> - <xi:include href="Sdi/surveillance.xml" xpointer="element(/1)"/> + <xi:include href="Sdi/icinga.xml" xpointer="element(/1)"/> </part> <part xml:id="persist">