<?xml version="1.0" encoding="UTF-8"?>
<chapter annotations="slide" version="5.1" xml:id="sdi_cloudProvider"
xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xila="http://www.w3.org/2001/XInclude/local-attributes"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:trans="http://docbook.org/ns/transclusion"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns:m="http://www.w3.org/1998/Math/MathML"
xmlns:html="http://www.w3.org/1999/xhtml"
xmlns:db="http://docbook.org/ns/docbook">
<title>Cloud provider</title>
<section xml:id="sdi_cloudProvider_webAdminGui">
<title><orgname xlink:href="https://hetzner.com">Hetzner</orgname> cloud
administration GUI</title>
<figure xml:id="sdi_cloudProvider_webAdminGui_createSshKeyPair">
<title>Preliminary: Create an <command>ssh</command> key pair</title>
<screen>sdiuser@martin-pc-dachboden:~$ <command
xlink:href="https://linux.die.net/man/1/ssh-keygen">ssh-keygen</command> -t ed25519 <co
linkends="sdi_cloudProvider_webAdminGui_createSshKeyPair-1"
xml:id="sdi_cloudProvider_webAdminGui_createSshKeyPair-1-co"/>
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/sdiuser/.ssh/id_ed25519):
Created directory '/home/sdiuser/.ssh'.
Enter passphrase (empty for no passphrase): <co
linkends="sdi_cloudProvider_webAdminGui_createSshKeyPair-2"
xml:id="sdi_cloudProvider_webAdminGui_createSshKeyPair-2-co"/>
Enter same passphrase again:
Your identification has been saved in /home/sdiuser/.ssh/id_ed25519 <co
linkends="sdi_cloudProvider_webAdminGui_createSshKeyPair-3"
xml:id="sdi_cloudProvider_webAdminGui_createSshKeyPair-3-co"/>
Your public key has been saved in /home/sdiuser/.ssh/id_ed25519.pub <co
linkends="sdi_cloudProvider_webAdminGui_createSshKeyPair-4"
xml:id="sdi_cloudProvider_webAdminGui_createSshKeyPair-4-co"/></screen>
<calloutlist role="slideExclude">
<callout arearefs="sdi_cloudProvider_webAdminGui_createSshKeyPair-1-co"
xml:id="sdi_cloudProvider_webAdminGui_createSshKeyPair-1">
<para>Create an elliptic rather than default <abbrev>RSA</abbrev>
type key.</para>
</callout>
<callout arearefs="sdi_cloudProvider_webAdminGui_createSshKeyPair-2-co"
xml:id="sdi_cloudProvider_webAdminGui_createSshKeyPair-2">
<para>Security aware folks will choose a decent passphrase
protecting the private key being generated.</para>
</callout>
<callout arearefs="sdi_cloudProvider_webAdminGui_createSshKeyPair-3-co"
xml:id="sdi_cloudProvider_webAdminGui_createSshKeyPair-3">
<para>The generated private key.</para>
</callout>
<callout arearefs="sdi_cloudProvider_webAdminGui_createSshKeyPair-4-co"
xml:id="sdi_cloudProvider_webAdminGui_createSshKeyPair-4">
<para>The generated public key.</para>
<note>
<para>Different implementations like e.g. <command
xlink:href="https://www.putty.org">putty</command> may use
different key storage formats.</para>
</note>
</callout>
</calloutlist>
</figure>
<figure xml:id="sdi_cloudProvider_webAdminGui_hetznerSignUp">
<title>Create a <orgname>Hetzner</orgname> account</title>
<itemizedlist>
<listitem>
<para>Sign up at <link
xlink:href="https://accounts.hetzner.com/signUp">https://accounts.hetzner.com/signUp</link>
using an account name of your choice.</para>
</listitem>
<listitem>
<para>Optionally: Activate 2-factor authentication.</para>
</listitem>
<listitem>
<para>You may validate your account by ID card or similar. No
payment required!</para>
</listitem>
<listitem>
<para>Publish your <orgname>Hetzner</orgname> account's username to
your SDI course's group at <link
xlink:href="https://learn.mi.hdm-stuttgart.de">https://learn.mi.hdm-stuttgart.de</link>.</para>
</listitem>
</itemizedlist>
</figure>
<figure xml:id="sdi_cloudProvider_webAdminGui_accessProject">
<title>Access your project space</title>
<para>Upon confirmation your <orgname>Hetzner</orgname> project space
sdi_gxy (e.g. sdi_g01 corresponding to group 1) should be
accessible.</para>
</figure>
<figure xml:id="sdi_cloudProvider_webAdminGui_">
<title>Create a server</title>
<informaltable border="0">
<tr>
<td valign="top"><orderedlist>
<listitem>
<para>Create a default firewall allowing <command
xlink:href="https://linux.die.net/man/8/ping">ping</command>
and <command
xlink:href="https://linux.die.net/man/1/ssh">ssh</command></para>
</listitem>
<listitem>
<para><productname>Ubuntu</productname> latest</para>
</listitem>
<listitem>
<para>Shared vCPU / x86 / CX11 (<link
xlink:href="https://www.hetzner.com/cloud/#pricing">the
cheapest</link>)</para>
</listitem>
<listitem>
<para>Add your personal <command>ssh</command> public key from
<xref
linkend="sdi_cloudProvider_webAdminGui_createSshKeyPair"/></para>
</listitem>
</orderedlist></td>
<td valign="top"><orderedlist continuation="continues">
<listitem>
<para>Omit volume, labels and cloud configuration</para>
</listitem>
<listitem>
<para>Note the <guimenuitem>Networking</guimenuitem> /
<guisubmenu>Public IPv4</guisubmenu> address for later
reference</para>
</listitem>
<listitem>
<para>Click »Create & Buy now«</para>
</listitem>
</orderedlist></td>
</tr>
</informaltable>
</figure>
<figure xml:id="sdi_cloudProvider_webAdminGui_accessServer">
<title>Access your server</title>
<itemizedlist>
<listitem>
<para>Ping your server:</para>
<note>
<para>The IP 91.107.232.156 serves just as a sample value
irrespective of your individual actual server IP.</para>
</note>
<screen>sdiuser:~$ ping 91.107.232.156
PING 91.107.232.156 (91.107.232.156) 56(84) bytes of data.
64 bytes from 91.107.232.156: icmp_seq=1 ttl=49 time=18.3 ms
64 bytes from 91.107.232.156 ...</screen>
</listitem>
<listitem>
<para>Login via <command>ssh</command>:</para>
<screen>ssh root@91.107.232.156</screen>
</listitem>
</itemizedlist>
</figure>
<figure xml:id="sdi_cloudProvider_webAdminGui_updateServer">
<title>Update and reboot</title>
<orderedlist>
<listitem>
<para>apt update</para>
</listitem>
<listitem>
<para>apt upgrade</para>
</listitem>
<listitem>
<para>reboot</para>
</listitem>
</orderedlist>
</figure>
<figure xml:id="sdi_cloudProvider_webAdminGui_installNginx">
<title>Install a web server</title>
<screen>root@topsy:~# apt install nginx</screen>
</figure>
<figure xml:id="sdi_cloudProvider_webAdminGui_localHttpAccess">
<title>Check local <acronym>http</acronym> web access</title>
<screen>root@topsy:~# wget -O - 91.107.232.156
--2024-04-07 18:59:13-- http://91.107.232.156/
Connecting to 91.107.232.156:80... connected.
<html>
<head>
<title>Welcome to nginx!</title> ...</screen>
</figure>
<figure xml:id="sdi_cloudProvider_cloudAdminGui_externHttp">
<title>External <acronym>http</acronym> web access</title>
<para>Point your browser to http://91.107.232.156.</para>
<screen>sdiuser:~$ telnet 91.107.232.156 80
Trying 91.107.232.156...</screen>
<para>Why is there no answer?</para>
</figure>
<figure xml:id="sdi_cloudProvider_cloudAdminGui_allowHttp">
<title>Add port 80 / <acronym>http</acronym> firewall rule</title>
<screen>sdiuser:~$ telnet 91.107.232.156 80
Trying 91.107.232.156...
Connected to 91.107.232.156.
Escape character is '^]'</screen>
<para>Congrats: External Browser access is working now!</para>
</figure>
<figure xml:id="sdi_cloudProvider_cloudAdminGui_cleanUp">
<title>Cleaning up!</title>
<caution>
<para>This is about <emphasis role="red">$$$ MONEY
$$$</emphasis></para>
</caution>
<itemizedlist>
<listitem>
<para>Delete your server including the IPv4 address.</para>
</listitem>
<listitem>
<para>You may delete your firewall</para>
</listitem>
</itemizedlist>
</figure>
</section>
<section xml:id="sdi_cloudProvider_terra">
<title>Working with <productname
xlink:href="https://www.terraform.io">Terraform</productname></title>
<figure xml:id="sdi_cloudProvider_terra_pledge">
<title>What's it all about?</title>
<para><link
xlink:href="https://developer.hashicorp.com/terraform/intro">Quote:</link></para>
<para><quote><productname>Terraform</productname> is an infrastructure
as code tool that lets you build, change, and version cloud and
<abbrev>on-prem</abbrev> resources safely and
efficiently.</quote></para>
</figure>
<figure xml:id="sdi_cloudProvider_terra_installSoftware">
<title><productname>Terraform</productname> resources</title>
<itemizedlist>
<listitem>
<para><link
xlink:href="https://developer.hashicorp.com/terraform/intro#why-terraform">Why
<productname>Terraform</productname>?</link></para>
</listitem>
<listitem>
<para><link
xlink:href="https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli">Install
<productname>Terraform</productname></link>.</para>
</listitem>
</itemizedlist>
</figure>
<figure xml:id="sdi_cloudProvider_terra_HetznerApiToken">
<title><orgname>Hetzner</orgname> API token</title>
<itemizedlist>
<listitem>
<para>Access you cloud project using the <orgname>Hetzner</orgname>
<xref linkend="glo_GUI"/> interface.</para>
</listitem>
<listitem>
<para>Go to <option>Security</option> --> <option>API
Tokens</option> --> <option>Generate API token</option></para>
</listitem>
<listitem>
<para>Provide a name and hit Generate API token.</para>
</listitem>
<listitem>
<para>Copy the generated token's value and store it in a secure
location e.g. a password manager.</para>
<caution>
<para>The <orgname>Hetzner</orgname> <xref linkend="glo_GUI"/>
blocks future access to the token.</para>
</caution>
</listitem>
</itemizedlist>
</figure>
<figure xml:id="sdi_cloudProvider_terra_minimalConfig">
<title>Minimal <productname>Terraform</productname>
configuration</title>
<programlisting language="terraform"># Define Hetzner cloud provider
terraform {
required_providers {
hcloud = {
source = "hetznercloud/hcloud"
}
}
required_version = ">= 0.13"
}
# Configure the Hetzner Cloud API token
provider "hcloud" {
token = "<emphasis role="red">your_api_token_goes_here</emphasis>"
}
# Create a server
resource "hcloud_server" "helloServer" {
name = "hello"
image = "debian-12"
server_type = "cx11"
}</programlisting>
</figure>
<figure xml:id="sdi_cloudProvider_terra_init">
<title><productname>Terraform</productname> <command
xlink:href="https://developer.hashicorp.com/terraform/cli/commands/init">init</command></title>
<programlisting language="terraform">$ terraform <command
xlink:href="https://developer.hashicorp.com/terraform/cli/commands/init">init</command>
Initializing the backend...
Initializing provider plugins...
- Finding latest version of hetznercloud/hcloud...
- Installing hetznercloud/hcloud v1.46.1...
- Installed hetznercloud/hcloud v1.46.1 (signed by a HashiCorp partner, key ID 5219EACB3A77198B)
...
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. ...</programlisting>
</figure>
<figure xml:id="sdi_cloudProvider_terra_plan">
<title><productname>Terraform</productname> <command
xlink:href="https://developer.hashicorp.com/terraform/cli/commands/plan">plan</command></title>
<programlisting language="terraform">$ terraform <command
xlink:href="https://developer.hashicorp.com/terraform/cli/commands/plan">plan</command>
Terraform used the selected providers to generate the following execution plan. Resource actions ...
+ create
Terraform will perform the following actions:
# hcloud_server.helloServer will be created
+ resource "hcloud_server" "helloServer" {
+ allow_deprecated_images = false
+ backup_window = (known after apply)
...
}
Plan: 1 to add, 0 to change, 0 to destroy.</programlisting>
</figure>
<figure xml:id="sdi_cloudProvider_terra_apply">
<title><productname>Terraform</productname> <command
xlink:href="https://developer.hashicorp.com/terraform/cli/commands/apply">apply</command></title>
<programlisting language="terraform">$ terraform <command
xlink:href="https://developer.hashicorp.com/terraform/cli/commands/apply">apply</command>
...
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: <emphasis role="red">yes</emphasis>
hcloud_server.helloServer: Creating...
hcloud_server.helloServer: Still creating... [10s elapsed]
hcloud_server.helloServer: Creation complete after 14s [id=45822789]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.</programlisting>
</figure>
<figure xml:id="sdi_cloudProvider_terra_hello_email">
<title><productname>Credentials by E-Mail</productname></title>
<screen>Your server "hello" was created!
You can access your server with the following credentials:
IPv4 <emphasis role="red">128.140.108.60</emphasis>
IPv6 2a01:4f8:1c1c:8e3a::/64
User root
Password rJ3pNvJXbqMp3XNTvFdq
You will be prompted to change your password on your first login.
To improve security, we recommend that you add an SSH key when creating a server.</screen>
</figure>
<figure xml:id="sdi_cloudProvider_terra_hello_problems">
<title><productname>Problems</productname>: 😟</title>
<itemizedlist>
<listitem>
<para>Firewall blocks <command
xlink:href="https://linux.die.net/man/1/ssh">ssh</command> server
access:</para>
<screen>$ ssh root@<emphasis role="red">128.140.108.60</emphasis>
ssh: connect to host 128.140.108.60 port 22: Connection refused</screen>
<para>Access by <xref linkend="glo_Vnc"/> <link
xlink:href="https://docs.hetzner.com/cloud/servers/getting-started/vnc-console">console
login</link> only</para>
</listitem>
<listitem>
<para>IP and (initial) credentials by email 😱</para>
</listitem>
</itemizedlist>
<para>Solution:</para>
<orderedlist>
<listitem>
<para>Add firewall inbound <command
xlink:href="https://linux.die.net/man/1/ssh">ssh</command> access
rule.</para>
</listitem>
<listitem>
<para>Configure <command
xlink:href="https://linux.die.net/man/1/ssh">ssh</command> public
key login.</para>
</listitem>
</orderedlist>
</figure>
<figure xml:id="sdi_cloudProvider_terra_hello_sshAccessFw">
<title><command
xlink:href="https://linux.die.net/man/1/ssh">ssh</command> access,
firewall</title>
<programlisting language="terraform">resource "hcloud_firewall" "<emphasis
role="red">sshFw</emphasis>" {
name = "ssh-firewall"
rule {
direction = "in"
protocol = "tcp"
port = "22"
source_ips = ["0.0.0.0/0", "::/0"]
}
}
...
resource "hcloud_server" "helloServer" {
...
firewall_ids = [hcloud_firewall.<emphasis role="red">sshFw</emphasis>.id]
}</programlisting>
</figure>
<figure xml:id="sdi_cloudProvider_terra_hello_sshAccessKey">
<title><command
xlink:href="https://linux.die.net/man/1/ssh">ssh</command> access,
public key</title>
<programlisting language="terraform">resource "hcloud_ssh_key" "<emphasis
role="red">loginUser</emphasis>" {
name = "goik@hdm-stuttgart.de"
public_key = file("~/.ssh/id_ed25519.pub")
}
...
resource "hcloud_server" "helloServer" {
...
ssh_keys = [hcloud_ssh_key.<emphasis role="red">loginUser</emphasis>.id]
}</programlisting>
</figure>
<figure xml:id="sdi_cloudProvider_terra_hello_sshApply">
<title>Apply <command
xlink:href="https://linux.die.net/man/1/ssh">ssh</command> key
access</title>
<screen>$ terraform apply
# hcloud_firewall.sshFw will be created
+ resource "hcloud_firewall" "sshFw" {
...
# hcloud_server.helloServer will be created
+ resource "hcloud_server" "helloServer" {
...
# hcloud_ssh_key.goik will be created
+ resource "hcloud_ssh_key" "loginUser" {
...
Plan: 3 to add, 0 to change, 0 to destroy.
...
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.</screen>
</figure>
<figure xml:id="sdi_cloudProvider_terra_hello_sshApplyOutputValuesDetails1">
<title>Output data details #1/2</title>
<para>See <link
xlink:href="https://developer.hashicorp.com/terraform/language/values/outputs">
terraform output documentation</link>:</para>
<informaltable border="1">
<tr>
<th>File <filename>outputs.tf</filename></th>
<th>Result</th>
</tr>
<tr>
<td rowspan="2" valign="top"><programlisting language="terraform">output "hello_ip_addr" {
value = hcloud_server.helloServer.ipv4_address
description = "The server's IPv4 address"
}
output "hello_datacenter" {
value = hcloud_server.helloServer.datacenter
description = "The server's datacenter"
}</programlisting></td>
<td valign="top"><screen language="properties">$ <command>terraform</command> output
hello_datacenter = "nbg1-dc3"
hello_ip_addr = "159.69.152.37"</screen></td>
</tr>
<tr>
<td valign="top"><screen language="properties">$ <command>terraform</command> output hello_ip_addr
"159.69.152.37"</screen></td>
</tr>
</informaltable>
</figure>
<figure xml:id="sdi_cloudProvider_terra_hello_sshApplyOutputValuesDetails">
<title>Output data details #2/2</title>
<informaltable border="1">
<tr>
<th>File <filename>outputs.tf</filename></th>
<th><command>terraform</command> output <option>-json</option></th>
</tr>
<tr>
<td valign="top"><programlisting language="terraform">output "hello_ip_addr" {
value = hcloud_server.helloServer.ipv4_address
description = "The server's IPv4 address"
}
output "hello_datacenter" {
value = hcloud_server.helloServer.datacenter
description = "The server's datacenter"
}</programlisting></td>
<td valign="top"><programlisting language="json">{
"hello_datacenter": {
"sensitive": false,
"type": "string",
"value": "nbg1-dc3"
},
"hello_ip_addr": {
"sensitive": false,
"type": "string",
"value": "159.69.152.37"
}
}</programlisting></td>
</tr>
</informaltable>
</figure>
<figure xml:id="sdi_cloudProvider_terra_hello_sshProblemApiToken">
<title><productname>Problem 2</productname>: <xref linkend="glo_VCS"/>
and visible provider API token 😱</title>
<para><emphasis role="red">Versioned</emphasis> file
<filename>main.tf</filename>:</para>
<programlisting language="terraform">...
provider "hcloud" { token = "<emphasis role="red">xdaGfz9LmwO8SWkg ... </emphasis>"}
...</programlisting>
<para>Solution:</para>
<itemizedlist>
<listitem>
<para>Declare a <link
xlink:href="https://developer.hashicorp.com/terraform/language/values/variables">variable</link>
<varname>hcloud_token</varname> in a
<filename>variables.tf</filename> file</para>
</listitem>
<listitem>
<para>Add a non-versioned file
<filename>secrets.auto.tfvars</filename>.</para>
</listitem>
<listitem>
<para>Optional: Provide a versioned
<filename>secrets.auto.tfvars.template</filename> documenting
file</para>
</listitem>
</itemizedlist>
</figure>
<figure xml:id="sdi_cloudProvider_terra_hello_sshProblemApiTokenSolve">
<title>Solution</title>
<informaltable border="0">
<tr>
<td valign="top"><para>Declaring <emphasis
role="red"><varname>hcloud_token</varname></emphasis> in
<filename>variables.tf</filename>:</para><programlisting
language="terraform">variable "<emphasis role="red">hcloud_token</emphasis>" { # See secret.auto.tfvars
nullable = false
sensitive = true
}</programlisting></td>
<td valign="top"><para>Defining <emphasis
role="red"><varname>hcloud_token</varname></emphasis>'s value in
<filename>secrets.auto.tfvars</filename>:</para><programlisting
language="terraform"><emphasis role="red">hcloud_token</emphasis>="xdaGfz9LmwO8SWkg ... "</programlisting></td>
</tr>
<tr>
<td valign="top"><para>Using <emphasis
role="red"><varname>hcloud_token</varname></emphasis> in
<filename>main.tf</filename>:</para><programlisting
language="terraform">provider "hcloud" { token = var.<emphasis
role="red">hcloud_token</emphasis> }</programlisting></td>
<td valign="top"><para>Example in
<filename>secrets.auto.tfvars.template:</filename></para><programlisting
language="terraform"><emphasis role="red">hcloud_token</emphasis>="your_api_token_goes_here"</programlisting></td>
</tr>
</informaltable>
</figure>
<qandaset defaultlabel="qanda"
xml:id="sdi_cloudProvider_terra_qandaBasicSystem">
<title>Incrementally creating a base system</title>
<qandadiv>
<qandaentry>
<question>
<para>Follow the subsequent steps creating basic server based on
<xref linkend="glo_Terraform"/>:</para>
<orderedlist>
<listitem>
<para>Start from <xref
linkend="sdi_cloudProvider_terra_minimalConfig"/> adding a
<xref linkend="glo_ssh"/> inbound firewall rule. Enter your
<orgname>Hetzner</orgname> provider token and create the
server.</para>
<para>On success you'll receive an e-mail containing your
server's IP address and the <code>root</code> user's password
for <xref linkend="glo_ssh"/> login. Why does this happen? Log
in to your server.</para>
</listitem>
<listitem>
<para>Subject your configuration to version control in a
<productname>Git</productname> project. Versioning the
previous <xref linkend="glo_Terraform"/> configuration might
expose your cloud provider's API token. Circumvent this
problem by following the steps outlined in <xref
linkend="sdi_cloudProvider_terra_hello_sshProblemApiTokenSolve"/>.</para>
</listitem>
<listitem>
<para>Ditch unsafe (and tedious) <xref linkend="glo_ssh"/>
password login in favour of public/private key access.</para>
<tip>
<para>Create a <link
xlink:href="https://registry.terraform.io/providers/hetznercloud/hcloud/latest/docs/resources/ssh_key">resource
"hcloud_ssh_key" ...</link> and read your <link
xlink:href="https://registry.terraform.io/providers/hetznercloud/hcloud/latest/docs/resources/server">hcloud_server</link>
documentation regarding <xref linkend="glo_ssh"/> public key
configuration.</para>
</tip>
<para>On success you should be able to log in using your <xref
linkend="glo_ssh"/> private key.</para>
</listitem>
<listitem>
<para>Currently when executing the <command
xlink:href="https://developer.hashicorp.com/terraform/cli/commands/apply">terraform
apply</command> command both your newly created server's IP
and <link
xlink:href="https://registry.terraform.io/providers/hetznercloud/hcloud/latest/docs/resources/server#datacenter">data
center location</link> are not being shown. Add an
<filename>outputs.tf</filename> file containing two
corresponding <code
xlink:href="https://developer.hashicorp.com/terraform/language/values/outputs">output</code>
entries.</para>
<para>On success when executing <command
xlink:href="https://developer.hashicorp.com/terraform/cli/commands/apply">terraform
apply</command> you should see something like:</para>
<screen>terraform apply
...
hcloud_server.helloServer: Still creating... [10s elapsed]
hcloud_server.helloServer: Creation complete after 14s [id=46961197]
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Outputs:
<emphasis role="red">hello_datacenter = "hel1-dc2"
hello_ip_addr = "95.217.154.104"</emphasis></screen>
</listitem>
</orderedlist>
</question>
</qandaentry>
</qandadiv>
</qandaset>
</section>
<section xml:id="sdi_cloudProvider_cloudInit">
<title><productname>Cloud-init</productname></title>
<figure xml:id="sdi_cloudProvider_cloudInit_cloudStackTalk">
<title>Introduction and reference</title>
<itemizedlist>
<listitem>
<para>Cloud Stack <link
xlink:href="https://mirror.mi.hdm-stuttgart.de/Videos/Cloud/cloud-init_CloudStackCollaborationConference2022.mp4">Conference
talk</link>.</para>
</listitem>
<listitem>
<para><link
xlink:href="https://cloudinit.readthedocs.io/en/latest">Cloud-init
documentation</link></para>
</listitem>
</itemizedlist>
</figure>
<figure xml:id="sdi_cloudProvider_cloudInit_nutshell">
<title>In a nutshell</title>
<itemizedlist>
<listitem>
<para>Distribution image containing pre-installed <productname>Cloud
Init</productname></para>
</listitem>
<listitem>
<para>Script configurable installation options</para>
</listitem>
</itemizedlist>
</figure>
<figure xml:id="sdi_cloudProvider_cloudInit_configOverview">
<title>Configuration options</title>
<informaltable border="0">
<tr>
<td valign="top"><itemizedlist>
<listitem>
<para>Individual CRUD file operations</para>
</listitem>
<listitem>
<para>Supplying <productname>ssh</productname> user and host
keys.</para>
</listitem>
<listitem>
<para>Adding users</para>
</listitem>
<listitem>
<para>...</para>
</listitem>
</itemizedlist></td>
<td valign="top"><itemizedlist>
<listitem>
<para>Installing packages</para>
</listitem>
<listitem>
<para>System Upgrade + reboot</para>
</listitem>
<listitem>
<para>Arbitrary command execution</para>
</listitem>
</itemizedlist></td>
</tr>
</informaltable>
</figure>
<figure xml:id="sdi_cloudProvider_cloudInit_terraformInterfaceCloudinit">
<title><productname>Terraform</productname> interface to
<productname>Cloud Init</productname></title>
<programlisting language="terraform">resource "hcloud_server" "web" {
name = var.server_name
...
user_data = file("userData.yml")
}</programlisting>
</figure>
<figure xml:id="sdi_cloudProvider_cloudInit_helloWorld">
<title>»hello, world ...« <filename>userData.yml</filename> file</title>
<programlisting language="yaml">#cloud-config
packages:
- nginx
runcmd:
- systemctl enable nginx
- rm /var/www/html/*
- >
echo "I'm Nginx @ $(dig -4 TXT +short o-o.myaddr.l.google.com @ns1.google.com)
created $(date -u)" >> /var/www/html/index.html</programlisting>
</figure>
<figure xml:id="sdi_cloudProvider_cloudInit_kownHostsDuplicateProblem">
<title>Duplicate known_hosts entry on re-creating server</title>
<para>Problem of repeated <command
linkend="glo_Terraform">terraform</command>
<option>apply</option>:</para>
<screen>$ ssh root@128.140.108.60
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (<emphasis role="red">man-in-the-middle attack</emphasis>)!</screen>
</figure>
<figure xml:id="sdi_cloudProvider_cloudInit_watchOutForBadGuys">
<title>Watch out for your enemies!</title>
<screen>root@hello:~# journalctl -f
May 06 04:41:20 hello cloud-init[898]: Cloud-init v. 22.4.2 finished at Mon, 06 May 2024 04:41:20 +0000. Datasource DataSourceHetzner. Up 11.78 seconds
...
May 06 04:46:16 hello sshd[927]: Invalid user abc from 43.163.218.130 port 33408
May 06 04:46:17 hello sshd[927]: Received disconnect from 43.163.218.130 port 33408:11: Bye Bye [preauth]
May 06 04:46:17 hello sshd[927]: Disconnected from invalid user abc 43.163.218.130 port 33408 [preauth]
...
May 06 04:50:54 hello sshd[930]: fatal: Timeout before authentication for 27.128.243.225 port 59866
...
May 06 04:52:45 hello sshd[933]: Invalid user cos from 43.163.218.130 port 59776
...
May 06 04:53:04 hello sshd[935]: Invalid user admin from 194.169.175.35 port 51128
May 06 04:53:49 hello sshd[937]: User root from 43.163.218.130 not allowed because not listed in AllowUsers
May 06 04:53:49 hello sshd[937]: Disconnected from invalid user root 43.163.218.130 port 50592 [preauth]</screen>
</figure>
<qandaset defaultlabel="qanda"
xml:id="sdi_cloudProvider_cloudInit_qanda_gettingStarted">
<title>Working on <productname>Cloud-init</productname></title>
<qandadiv>
<qandaentry>
<question>
<para>We continue our exercise series <xref
linkend="sdi_cloudProvider_terra_qandaBasicSystem"/> by adding a
<productname>Cloud-init</productname> configuration:</para>
<orderedlist>
<listitem>
<para>Follow <xref
linkend="sdi_cloudProvider_cloudInit_terraformInterfaceCloudinit"/>
and <xref linkend="sdi_cloudProvider_cloudInit_helloWorld"/>
to create simple web server.</para>
<tip>
<para>You will have to extend your current firewall
configuration allowing inbound traffic to port 80.</para>
</tip>
<para>On success pointing your web browser of choice to
<uri>http://<your server's IP></uri> should result in
something similar to:</para>
<screen>I'm Nginx @ "95.217.154.104" created Sun May 5 06:58:37 PM UTC 2024</screen>
</listitem>
<listitem>
<para>With respect to <xref
linkend="sdi_cloudProvider_cloudInit_watchOutForBadGuys"/>
inspect the output of <command>journalctl -f</command> on your
own server for a while. Then modify your current
<command>sshd</command> configuration:</para>
<itemizedlist>
<listitem>
<para>Read the <link
xlink:href="https://linux.die.net/man/5/sshd_config">sshd_config(5)
- Linux man page</link></para>
</listitem>
<listitem>
<para>Disallow <xref linkend="glo_ssh"/> password based
logins</para>
</listitem>
<listitem>
<para>Disallow <code>root</code> login altogether</para>
</listitem>
<listitem>
<para>Create a user <code>devops</code> eligible for both
<xref linkend="glo_ssh"/> login and
<command>sudo</command> execution.</para>
</listitem>
</itemizedlist>
<tip>
<itemizedlist>
<listitem>
<para><link
xlink:href="https://www.digitalocean.com/community/tutorials/how-to-use-cloud-config-for-your-initial-server-setup#configuring-the-ssh-daemon-through-focused-changes">Modifying
ssh configuration files</link></para>
</listitem>
<listitem>
<para><link
xlink:href="https://www.digitalocean.com/community/tutorials/how-to-use-cloud-config-for-your-initial-server-setup#set-up-sudo-access">Set
up sudo access</link></para>
</listitem>
</itemizedlist>
</tip>
<para>On success the following sequence should be
possible:</para>
<screen>$ ssh -v devops@95.217.154.104
...
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: <emphasis role="red">Authentications that can continue: publickey</emphasis> <co
linkends="sdi_cloudProvider_cloudInit_qanda_gettingStarted_ssh-1"
xml:id="sdi_cloudProvider_cloudInit_qanda_gettingStarted_ssh-1-co"/>
debug1: Next authentication method: publickey
...
Last login: Sun May 5 19:21:12 2024 from 217.245.243.187
devops@hello:~$ sudo su - <co
linkends="sdi_cloudProvider_cloudInit_qanda_gettingStarted_ssh-2"
xml:id="sdi_cloudProvider_cloudInit_qanda_gettingStarted_ssh-2-co"/>
root@hello:~# hostname
hello</screen>
<calloutlist>
<callout arearefs="sdi_cloudProvider_cloudInit_qanda_gettingStarted_ssh-1-co"
xml:id="sdi_cloudProvider_cloudInit_qanda_gettingStarted_ssh-1">
<para><code>password</code> is not among the list of
allowed authentication methods.</para>
</callout>
<callout arearefs="sdi_cloudProvider_cloudInit_qanda_gettingStarted_ssh-2-co"
xml:id="sdi_cloudProvider_cloudInit_qanda_gettingStarted_ssh-2">
<para>User <code>devops</code> may execute
<command>sudo</command> commands by virtue of his
membership in group <code>sudo</code>.</para>
</callout>
</calloutlist>
<para>On contrary <xref linkend="glo_ssh"/> root login must be
prohibited:</para>
<screen>$ ssh root@95.217.154.104
root@95.217.154.104: Permission denied (publickey).</screen>
</listitem>
<listitem>
<para>Read the <link
xlink:href="https://cloudinit.readthedocs.io/en/latest">cloud-init
documentation</link> and/or related tutorials to:</para>
<itemizedlist>
<listitem>
<para>Currently your (most likely outdated) cloud provider
supplied distribution does not get upgraded on
installation time:</para>
<screen>$ ./bin/ssh
...
devops@hello:~$ sudo su -
root@hello:~# apt update
Hit:1 http://security.debian.org/debian-security bookworm-security InRelease
Hit:2 http://deb.debian.org/debian bookworm InRelease
...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
<emphasis role="red">6 packages can be upgraded. Run 'apt list --upgradable' to see them.</emphasis>
# apt list --upgradable
Listing... Done
less/stable-security,stable-security 590-2.1~deb12u2 amd64 [upgradable from: 590-2]
libc-bin/stable-security,stable-security 2.36-9+deb12u7 amd64 [upgradable from: 2.36-9+deb12u6]
libc-l10n/stable-security,stable-security 2.36-9+deb12u7 all [upgradable from: 2.36-9+deb12u6]
libc6/stable-security,stable-security 2.36-9+deb12u7 amd64 [upgradable from: 2.36-9+deb12u6]
locales-all/stable-security,stable-security 2.36-9+deb12u7 amd64 [upgradable from: 2.36-9+deb12u6]
locales/stable-security,stable-security 2.36-9+deb12u7 all [upgradable from: 2.36-9+deb12u6]</screen>
<para>Modify your <productname>Cloud-init</productname>
configuration to upgrade your distribution at server
creation time. If so required your system should also
reboot.</para>
</listitem>
<listitem>
<para>Install and configure fail2ban limiting <xref
linkend="glo_ssh"/> failed connection attempts.</para>
<tip>
<itemizedlist>
<listitem>
<para>Develop your desired configuration manually on
an already created server. Then automate the
process.</para>
</listitem>
<listitem>
<para>Read the answer <link
xlink:href="https://superuser.com/questions/1830245/i-cant-get-fail2ban-working-on-debian-12#answer-1830273">On
Debian 12 there are a couple of things you have to
do to make it work</link>.</para>
</listitem>
</itemizedlist>
</tip>
</listitem>
<listitem>
<para>Install the <code>plocate</code> file indexer
package and initialize it.</para>
</listitem>
</itemizedlist>
<para>On success all packages should be up to date:</para>
<screen>$ ./bin/ssh
...
devops@hello:~$ sudo su -
root@hello:~# apt update
Hit:1 http://security.debian.org/debian-security bookworm-security InRelease
Hit:2 http://deb.debian.org/debian bookworm InRelease
...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
<emphasis role="red">All packages are up to date.</emphasis></screen>
<para>Failed login attempts should be banned: Keep a second
login open in advance when trying to simulate login failures!
You should then see a report similar to:</para>
<screen>root@hello:~# fail2ban-client status sshd
Status for the jail: sshd
|- Filter
| |- Currently failed: 2
| |- Total failed: 14
| `- Journal matches: _SYSTEMD_UNIT=sshd.service + _COMM=sshd
`- Actions
|- Currently banned: 2
|- Total banned: 2
`- <emphasis role="red">Banned IP list: 170.64.133.30 213.136.94.219</emphasis></screen>
<para>Searching for file name components should work like
e.g.:</para>
<screen>root@hello:~# locate <emphasis role="red">ssh_host</emphasis>
/etc/ssh/<emphasis role="red">ssh_host</emphasis>_ed25519_key
/etc/ssh/<emphasis role="red">ssh_host</emphasis>_ed25519_key.pub</screen>
</listitem>
</orderedlist>
</question>
</qandaentry>
</qandadiv>
</qandaset>
</section>
<section xml:id="sdi_cloudProvider_volume">
<title>Volumes</title>
<figure xml:id="sdi_cloudProvider_volumeTheEasyWay">
<title>A volume: The easy way</title>
<informaltable border="0">
<tr>
<td valign="top"><programlisting language="tf">resource "<emphasis
role="red">hcloud_server</emphasis>" "<emphasis role="red">helloServer</emphasis>" {
server_type = "cx11"
...
}
resource "hcloud_volume" "volume01" {
name = "volume1"
size = 10
server_id = <emphasis role="red">hcloud_server</emphasis>.<emphasis
role="red">helloServer</emphasis>.id
automount = true
format = "xfs"
}</programlisting></td>
<td valign="top"><screen>df
...
/mnt/HC_Volume_100723816</screen></td>
</tr>
</informaltable>
</figure>
<figure xml:id="sdi_cloudProvider_volumeEasyDetails">
<title>Volume details</title>
<informaltable border="0">
<tr>
<td valign="top"><programlisting language="tf">output "volume_id" {
value=hcloud_volume.volume01.id
description = "The volume's id"
}</programlisting></td>
<td valign="top"><screen># ls /dev/disk/by-id/*<emphasis role="red">100723816</emphasis>
/dev/disk/by-id/scsi-0HC_Volume_<emphasis role="red">100723816</emphasis></screen></td>
</tr>
<tr>
<td valign="top"><screen>terraform apply
...
hello_ip_addr="37.27.22.189"
volume_id="<emphasis role="red">100723816</emphasis>"</screen></td>
<td valign="top"><para>Desired
<code>/etc/fstab</code>:</para><programlisting language="none">/dev/disk/by-id/scsi-0HC_Volume_<emphasis
role="red">100723816</emphasis>
/volume01 xfs discard,nofail,defaults 0 0</programlisting></td>
</tr>
</informaltable>
</figure>
<qandaset defaultlabel="qanda"
xml:id="sdi_cloudProvider_volume_qanda_automount">
<title>Auto mounting a volume</title>
<qandadiv>
<qandaentry>
<question>
<para>Follow <xref linkend="sdi_cloudProvider_volumeTheEasyWay"/>
adding an a auto mounted volume to an existing server.</para>
<tip>
<para>Due to a <productname>Cloud-init</productname>
implementation quirk consider the <link
xlink:href="https://github.com/hetznercloud/terraform-provider-hcloud/issues/473#issuecomment-971535629-permalink">auto
mount workaround</link> hint.</para>
</tip>
</question>
</qandaentry>
</qandadiv>
</qandaset>
<figure xml:id="sdi_cloudProvider_volumeMountPointNameStrategy">
<title>Providing a mount point's name</title>
<informaltable border="0">
<tr>
<td valign="top"><programlisting language="tf">resource "hcloud_server" "helloServer" {
...
user_data = templatefile("tpl/userData.yml", {
...
volume01Id = hcloud_volume.volume01.id
})
}
resource "hcloud_volume" "volume01" {
server_id = hcloud_server.helloServer.id
...
}</programlisting></td>
<td valign="top"><para>Problem: Cyclic dependency</para><screen>helloServer <--> volume01</screen></td>
</tr>
</informaltable>
</figure>
<figure xml:id="sdi_cloudProvider_volumeServerIndependent">
<title>Solution: Independent resource creation</title>
<programlisting language="tf">resource "hcloud_volume" "<emphasis
role="red">volume01</emphasis>" {
size = 10
....
}
resource "hcloud_server" "helloServer" {
user_data = templatefile("tpl/userData.yml", {
volume01Id = hcloud_volume.<emphasis role="red">volume01</emphasis>.id # No cycle any more
...
}) ...
}
resource "hcloud_volume_attachment" "main" {
volume_id = hcloud_volume.<emphasis role="red">volume01</emphasis>.id
server_id = hcloud_server.helloServer.id
}</programlisting>
</figure>
<qandaset defaultlabel="qanda"
xml:id="sdi_cloudProvider_volume_qanda_mountPointName">
<title>Mount point's name specification</title>
<qandadiv>
<qandaentry>
<question>
<para>In <xref
linkend="sdi_cloudProvider_volume_qanda_automount"/> the mount
point's name had been auto generated by
<productname>Cloud-init</productname>. Modify your setup and
define <filename>/volume01</filename> as the new mount point.
Avoiding the dependency cycle from <xref
linkend="sdi_cloudProvider_volumeMountPointNameStrategy"/> you may
follow:</para>
<itemizedlist>
<listitem>
<para>Create volume and server independently</para>
<tip>
<para>Choose a common <code
xlink:href="https://registry.terraform.io/providers/hetznercloud/hcloud/latest/docs/resources/volume#location">location</code>
value for server and volume e.g. <code>nbg1</code>.</para>
</tip>
</listitem>
<listitem>
<para>Attach volume to server by virtue of <code
xlink:href="https://registry.terraform.io/providers/hetznercloud/hcloud/latest/docs/resources/volume_attachment">hcloud_volume_attachment</code>
setting <code>automount = false</code>.</para>
</listitem>
<listitem>
<para>Pass the volume's id to your
<productname>Cloud-init</productname> template and add a
corresponding line to <filename>/etc/fstab</filename>.</para>
<tip>
<para>You may have to execute <command>systemctl</command>
<option>daemon-reload</option> after modifying
<filename>/etc/fstab</filename></para>
</tip>
</listitem>
<listitem>
<para>Execute <command>mount</command> <option>-a</option> for
taking your changes into effect</para>
</listitem>
</itemizedlist>
</question>
</qandaentry>
</qandadiv>
</qandaset>
</section>
</chapter>