Using and configuring Chef on Ubuntu

• Chef
• Ubuntu
• Xenial
• Focal-Fossa

Chef is a configuration management tool that is written in Ruby and Erlang. It is capable of managing both your on-premise machines and your Scaleway resources with ease.

As your infrastructure requirements grow, managing each server individually becomes an increasingly difficult task. Configuration management solutions are designed to turn your infrastructure administration into a code base. Instead of performing individual tasks on several machines, these tools allow you to commit your requirements to a central location where each component can connect, pull down their configuration, and apply it.

You can easily manage up to 10,000 nodes using Chef. Chef also makes it easy to automate the replication of infrastructure components.

Before you start

To complete the actions presented below, you must have:

• A Scaleway account logged into the console
• Owner status or IAM permissions allowing you to perform actions in the intended Organization
• An SSH key
• One of the following remote machines running Ubuntu (this tutorial was validated for machines running Ubuntu Xenial or Ubuntu Focal Fossa):
  • An Instance
  • An Elastic Metal server
  • A Dedibox dedicated server
• A second machine (a local machine or a second remote machine) on which to install Chef SDK

Core concepts

• Chef Server: This central server holds all configuration data that the nodes will use for configuration.
• Workstation: This machine holds all the configuration data that can later be pushed to the central Chef server. Several Chef command line utilities are available in this system, which can be used to interact with nodes, update configurations, etc.
• Node: This is a client-server/system that will be registered to the central Chef server, from which it can pull configuration data to apply.

Configuring a Chef server

Accessing the server by hostname

1. Connect to your remote machine via SSH. Follow the instructions depending on your remote machine type: Instance, Elastic Metal, or Dedibox.

   Note

   This tutorial assumes that you connect to your machine as root. If this is not the case, you may need to add sudo to the commands shown.

   To install Chef, the hostname of the machine must be a resolvable fully qualified domain name or IP address. Check that in the following steps:

2. On the machine where you plan to install Chef, launch:

   hostname -f
   

   If the command does not return an address where the server can be reached, follow steps 3-5 to resolve this. Otherwise, proceed to the next section of this tutorial.

3. Open the etc/hosts configuration file in a text editor:

   nano /etc/hosts
   

4. Edit the configuration file to add a domain name or IP address where the server can be reached, following the example below:

   127.0.1.1 fqdn_or_IP_address host_alias
   127.0.0.1 localhost
   IP_address fqdn_or_IP_address host_alias
   

   Tip

   You can find the IP address of your Instance or Elastic Metal server in its Overview tab in the Scaleway console. For an Instance with host alias scw-flamboyant-boyd, IP address 192.158.1.38, and no fully qualified domain name, the configuration file once edited might look like this:

   127.0.1.1 192.158.1.38 scw-flamboyant-boyd
   127.0.0.1 localhost
   192.158.1.38 192.158.1.38 scw-flamboyant-boyd
   # The following lines are desirable for IPv6 capable hosts
   ::1 ip6-localhost ip6-loopback
   fe00::0 ip6-localnet
   ff00::0 ip6-mcastprefix
   ff02::1 ip6-allnodes
   ff02::2 ip6-allrouters
   ff02::3 ip6-allhosts
   127.0.0.1 scw-flamboyant-boyd
   

5. Check that the value was set correctly:

   hostname -f
   

   The command returns the value that you can use to reach your Chef server from anywhere in your infrastructure. This could be an IP address or a fully-qualified domain name.

Installing Chef server

1. Update the Ubuntu package manager:

   apt-get update
   

2. Upgrade the Ubuntu packages already installed:

   apt-get upgrade
   

3. Go to the Chef Infra Server Download Page, from your local machine’s browser, and click Download next to the version of Ubuntu you are running.

4. Complete your information in the form that displays and click Download.

5. Right-click Download Chef Infra Server directly and copy the link.

6. Launch the installation using wget + the link you copied, from the terminal of your remote machine. Make sure to use your own link and not the one shown below as an example:

   wget https://packages.chef.io/files/stable/chef-server/12.19.31/ubuntu/18.04/chef-server-core_12.19.31-1_amd64.deb
   

7. Install the package you downloaded:

   dpkg -i chef-server*
   

8. Call the reconfigure command, which configures the components that make up the server to work together in your specific environment:

   chef-server-ctl reconfigure
   

   You are given a link to the license agreement and prompted to accept its terms.

   The configuration may take some minutes to complete.

Creating an Admin User and Organization

Creating an admin user enables a specific user to make changes to the infrastructure components in the organization. An Organization is a grouping of infrastructure and configuration within Chef.

1. Launch the chef-server-ctlwith the user-create subcommand:

   chef-server-ctl user-create USERNAME FIRST_NAME LAST_NAME EMAIL PASSWORD
   

For our example, we will create a user with the following information:

• Username: testuser
• First Name: test
• Last Name: user
• Email: testuser@example.com
• Password: pwdexample
• Filename: testuser.pem

We will also add -f option to specify a filename in which to output our new user’s private RSA key. We will need this to authenticate using the knife management command later. In the end, the command translates to:

chef-server-ctl user-create testuser test user testuser@example.com pwdexample -f testuser.pem

2. Launch the chef-server-ctl with the org-create subcommand:

   chef-server-ctl org-create SHORTNAME LONGNAME --association_user USERNAME
   

   For our example, we will create an organization with the following information:

   • Short Name (refers to the organization within Chef): scaleway
   • Long Name (name of the organization): Scaleway, Inc.
   • Association User (username that has access to administer the organization): testuser
   • Filename: scaleway-file.pem

   Again, we will add the -f flag to specify the private key file location. The key that will be created is used to validate new clients as part of the organization until they can get their own unique client key. In the end, the command translates to:

   chef-server-ctl org-create scaleway "Scaleway, Inc." --association_user testuser -f scaleway-file.pem
   

The Chef server installation is now complete.

Configuring a Chef workstation

The actual infrastructure coordination and configuration do not take place on the Chef server. This work is done on a workstation which then uploads the data to the server to influence the Chef environment. This workstation can be your local machine or a remote one.

Installing Chef Workstation

The Chef Workstations is a suite of software designed for Chef workstations.

1. Go to the Chef Workstation Ubuntu Linux Download Page, and click Download next to the version of Ubuntu you are running.
2. Complete your information in the form that displays and click Download.
3. Click Download ChefDK directly and copy the link.
4. Launch the installation using wget + the link you copied on the machine you want to install ChefDK.

   wget https://packages.chef.io/files/stable/chef-workstation/0.4.2/ubuntu/18.04/chef-workstation_0.4.2-1_amd64.deb
   

5. Install the package you downloaded:

   dpkg -i chef-workstation_*
   

6. Verify that all the components are available in their expected location through the new chef command:

   chef verify
   

7. Enter yes, when you are prompted to accept the license agreement, to agree.
   Note

   If your workstation is primarily used to manage Chef for your infrastructure, you will likely want to default to the version of Ruby installed with Chef. You can do this by modifying your .bash_profile so that Chef’s Ruby takes precedence:

   echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile
   

   Afterward, you can source your .bash_profile file to set the correct environmental variables for the current session:

   source ~/.bash_profile
   

Cloning the Chef repository

The Chef configuration for your infrastructure is maintained in a hierarchical file structure known as a Chef repo. As the Chef GitHub repository is deprecated, we recommend using the chef generate repo command that comes with ChefDK.

chef generate repo chef_repo

Downloading the authentication keys to the workstation

At this point, your workstation has all the software needed to interact with a Chef server and compose infrastructure configurations. However, it is not yet configured to interact with your Chef server and your environment.

Create the hidden directory where you can store the user key and the organization validator key that we created on the Chef server.

mkdir ~/chef_repo/.chef

The method that you use to connect to the Chef server will determine how to download the keys.

Option 1: (Recommended) Downloading keys when connecting to a Chef server using SSH keys

1. Leave your workstation

   exit
   

The next steps should be carried out from the machine that holds the private SSH key needed to connect to your Chef server. 2. Add the SSH key you use to connect to the Chef server to an SSH agent:

eval $(ssh-agent)

which should start the agent and return something similar to

Agent pid 2893

3. Add your SSH key to the agent:

   ssh-add
   

   This keeps your SSH key stored in memory.

4. Forward the stored key to your workstation using the -A option with SSH. This also connects you to your workstation:

   ssh -A username@workstation_domain_or_IP
   

   Now you can connect to your Chef server from your workstation without needing a password, using the forwarded SSH credentials.

5. Copy the testuser.pem and scaleway-file.pem files from the Chef server to the folder you created in your Chef workstation with the following commands:

   scp root@chef_server_domain_or_IP:/root/testuser.pem ~/chef_repo/.chef
   scp root@chef_server_domain_or_IP:/root/scaleway-file.pem ~/chef_repo/.chef
   

   If you connect to your Chef server using a non-root user, the commands will look more like this:

   scp username@chef_server_domain_or_IP:/home/username/testuser.pem ~/chef_repo/.chef
   scp username@chef_server_domain_or_IP:/home/username/scaleway-file.pem ~/chef_repo/.chef
   

Option 2: Downloading keys when connecting to a Chef server with passwords

In this case, we will construct the scp commands directly. When executing the scp commands, you will be asked for the password to connect to your Chef server.

From your Chef workstation, construct the scp commands as follows:

1. Specify the username and domain name or IP address used to connect to the Chef server.

2. Follow this immediately with a colon (:) and the path to the file you wish to download.

3. After adding a space, indicate the directory on the local computer where you wish the downloaded files to be placed (~/chef_repo/.chef in our case).

   If you log into the Chef server using the root user account, your commands will look something like this.

   scp root@chef_server_domain_or_IP:/root/testuser.pem ~/chef_repo/.chef
   scp root@chef_server_domain_or_IP:/root/scaleway-file.pem ~/chef_repo/.chef
   

   If you connect to your Chef server using a non-root user, the commands will look more like this:

   scp username@schef_server_domain_or_IP:/home/username/testuser.pem ~/chef_repo/.chef
   scp username@chef_server_domain_or_IP:/home/username/scaleway-file.pem ~/chef_repo/
   

Configuring Knife to manage your Chef environment

knife is a command-line tool that provides an interface between a local Chef repo and the Chef server.

Now that you have your Chef credentials available on your workstation, we can configure the knife command with the information it needs to connect to and control your Chef infrastructure.

This is done through a knife.rb file that we will place in the ~/chef_repo/.chef directory along with our keys.

1. Create and open a file called knife.rb in that directory in your text editor:

   nano ~/chef_repo/.chef/knife.rb
   

2. Paste the following information replacing the information with your infrastructure information:

   current_dir = File.dirname(__FILE__)
   log_level                :info
   log_location             STDOUT
   node_name                "name_for_workstation"
   client_key               "#{current_dir}/name_of_user_key"
   validation_client_name   "organization_validator_name"
   validation_key           "#{current_dir}/organization_validator_key"
   chef_server_url          "https://server_domain_or_public_IP/organizations/organization_name"
   syntax_check_cache_path  "#{ENV['HOME']}/.chef/syntaxcache"
   cookbook_path            ["#{current_dir}/../cookbooks"]
   

   • node_name: Specifies the name that knife will use to connect to your Chef server. This should match your username.
   • client_key: Name and path to the user key that you copied over from the Chef server. We can use the #{current_dir} snippet to fill in the path if the key is in the same directory as the knife.rb file.
   • validation_client_name: Name of the validation client that knife will use to bootstrap new nodes. This will take the form of your organization short name, followed by -validator.
   • validation_key: Like the client_key, this includes the name and path to the validation key you copied from the Chef server. Again, you can use the #{current_dir}.
   • chef_server_url: This is the URL where the Chef server can be reached. It should begin with https://, followed by your Chef server’s domain name or Public IP address. Afterward, the path to your organization should be specified by appending /organizations/your_organization_name.

   For our tutorial, the file looks like this:

   current_dir = File.dirname(__FILE__)
   log_level                :info
   log_location             STDOUT
   node_name                "testuser"
   client_key               "#{current_dir}/testuser.pem"
   validation_client_name   "scaleway-validator"
   validation_key           "#{current_dir}/scaleway-file.pem"
   chef_server_url          "https://chef_server_domainname_or_IP/organizations/scaleway"
   syntax_check_cache_path  "#{ENV['HOME']}/.chef/syntaxcache"
   cookbook_path            ["#{current_dir}/../cookbooks"]
   

3. Save and close the knife.rb file.

4. Test the configuration file. We need to be in our ~/chef-repo directory for our configuration file to be read correctly:

   cd ~/chef_repo
   knife client list
   

   which returns an error similar to:

   ERROR: SSL Validation failure connecting to host: 51.15.240.56 - SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate)
   ERROR: Could not establish a secure connection to the server.
   Use `knife ssl check` to troubleshoot your SSL configuration.
   If your Chef Server uses a self-signed certificate, you can use
   `knife ssl fetch` to make knife trust the server's certificates.
   Original Exception: OpenSSL::SSL::SSLError: SSL Error connecting to https://51.15.240.56/organizations/scaleway/clients - SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate)
   

   This occurs because we do not have our Chef server’s SSL certificate on our workstation.

5. Launch the following command:

   knife ssl fetch
   

   which returns

   WARNING: Certificates from public_IP will be fetched and placed in your trusted_cert
   directory (/root/chef_repo/.chef/trusted_certs).
   Knife has no means to verify these are the correct certificates. You should
   verify the authenticity of these certificates after downloading.
   Adding certificate for public_IP in /root/chef_repo/.chef/trusted_certs/public_IP.crt
   

   After the SSL certificate has been fetched, the previous command should now work.

6. Launch the following command:

   knife client list
   

   which returns

   scaleway-validator
   

If the above command correctly returns, your workstation is now set up to control your Chef environment.