Lab 2.1: Setting up Linux server

Overview

In this lab, you will learn howto create a Linux server, access it through ssh (for linux), and copy files to/form the server. You will also compile and install an Apache web server. The goal of this lab is to give a real experience on deploying a server application such as Apache web server. Other servers (such as SMPT, LDAP..etc) follow similar set up process, so try to grasp the underlying mechanism.

In the subsequent labs, we use the term server for different things depending on the context. Some times we refer to a physical machine. We may also refer to an application running on a remote machine that listens to a port for requests. Such an application is also called service or daemon.

Setup the server

For this lab, you will be configuring you Ubuntu laptop to act as server using these steps.

  1. Install openssh-server which lets you access your computer securely over LAN or internet.
    1
    sudo apt-get install openssh-server openssh-client
  2. Edit the ssh config settings to disable root login for extra security:
    1
    sudo nano /etc/ssh/sshd_config

    This will open the configuration file. Find the following line in the file and change its value to ‘no’ as shown. Press CTRL-W and type PermitRootLogin to quickly find it:
    1
    PermitRootLogin no

    Now press CTRL-X and select Yes to save your changes.
  3. Check the ssh settings are valid by running:
    1
    sudo sshd -t

    The command will tell you if you’ve made any syntax error in the config file. If you don’t get any output, it means there are no errors in the configuration file.
  4. At this point you should be able to connect to the laptop (i.e., server) from itself in a terminal by:
    1
    2
    3
    ssh localhost
    # or equivalently
    ssh 127.0.0.1

    Type your password when asked and you will log in. Now, try it from another machine on your network (for example, from another student’s or TA’s laptop). First find your IP by running the following command in a terminal and note down the value of inet addr for the WiFi interface. WiFi interface is either named wlan0 or wlp3s0:
    1
    ifconfig

    Now you can log in from another machine using:
    1
    ssh [ip-address]
  5. It is possible to configure your server to allow your computer to access it without providing a password each time you try to connect. To do this you need to generate an SSH key. First, check whether there are already keys on your computer:
    1
    ls ~/.ssh

    If you see files named id_rsa.pub or id_dsa.pub you have keys set up already, so you can skip the generating keys step and go to the next step. To generate new SSH keys enter the following command. Replace key-name with your name or any other valid string of your choice.
    1
    ssh-keygen -t rsa -C key-name

    Upon entering this command, you’ll be asked a series of questions. Just keep hitting Enter to go with the default values. Now you should see the files id_rsa and id_rsa.pub in your .ssh directory in your home folder:
    1
    2
    3
    ls ~/.ssh
    # the out should look like this
    authorized_keys  id_rsa  id_rsa.pub  known_hosts

    The id_rsa file is your private key. Keep this on your computer. The id_rsa.pub file is your public key. This is what you put on machines you want to connect to. When the machine you try to connect to matches up your public and private key, it will allow you to connect.
  6. Append the public key to your authorized_keys file on the server, sending it over SSH:
    1
    cat ~/.ssh/id_rsa.pub | ssh localhost 'cat >> .ssh/authorized_keys'

    Note that this time you will have to authenticate with your password.
    Now you should be able to onnect without a password prompt:
    1
    ssh localhost

Copy files to/from server

  • Create and copy a file to a server
    1
    2
    3
    4
    mkdir ~/server
    touch testFile.txt
    echo "testing scp (server copy command)" | tee testFile.txt
    scp testFile.txt localhost:~/server

    Note: the tailing colon ‘:’ indicates the location relative to user’s home directory. If you add a slash ‘/’ after the colon, you specify the directory from the root.
  • If you want to copy a directory, simply add option ‘-r’ (means recursive):
    1
    2
    3
    4
    mkdir ~/recursivescp
    mv testscp.txt recursivescp/
    # scp -r local/directory/location remoteServer:/remote/directory
    scp -r ~/recursivescp localhost:~/server

    Similarly, if you want to copy from the server, all you need is to switch orders in the command.
    1
    2
    #scp -r remoteServer:/remote/directory local/directory/
    scp -r localhost:~/server/recursivescp ~/

Compiling & Installing Apache Web Server

You can always install servers using package managers such as apt-get. However, these package managers provide binaries that are compiled for generic CPU using few instruction sets which is usually a subset of your CPU instruction set. Therefore, they tend to be slower as it doesn’t fully utilize the CPU.

First, execute below command to login to your Amazon instance. Replace “[your-key].pem” below with your private key, and “[ec2-instance-address]” with the address of your EC2 server.

1
ssh localhost

Install compilers

  • First of all, we need to install some compiler to compile our open source applications. Namely, we need gcc for C applications and g++ for C++. The easiest way in Ubuntu is to type:
    1
    2
    sudo apt-get update
    sudo apt-get install gcc g++
  • Install make tool
    1
    sudo apt-get install make

Install Apache

  • Now, go to Apache website and find the latest stable httpd server. You can either download on your local machine, and then send it using scp to the server, or simply ssh to your server and type
    1
    2
    cd # change to home directory
    wget http://www.eu.apache.org/dist//httpd/httpd-2.4.25.tar.bz2
  • Its also good habit to always check the file integrity to make sure that the file is not corrupted.
    1
    sha1sum httpd-2.4.25.tar.bz2

    Now compare first few digits with the one given in apache website (note that checking first few digits is fairly enough as hash functions such as sha1 have very random behavior and unlikely to find collision between two different hashes) .
  • Extract the tar package installed
    1
    mkdir httpd2 && tar xvf httpd-2.4.25.tar.bz2 -C httpd2 --strip-components 1

    To learn more about ‘tar’ tool, type ‘man tar’ in a terminal. Basically, option ‘x’ means extract, ‘j’ is setting archive type to bzip2, and ‘v’ mean verbose (Command prints more messages).
  • Now its time to configure and then compile the server.
    1
    2
    cd ~/httpd2
    ./configure

    Usually before compiling any open source package, you need to execute ./configure script to check the availability of all dependencies and whether your server has all required libraries. This is good for code portability and its quite standard way in Linux environment.
  • Now if you get some errors, read carefully and try to search and download the missing libraries or packages. You can use your package manager to download the binaries, but if you are performance enthusiastic then you should download each missing package manually. Since we are doing server configuration, performance is very important, so will download and install missing packages manually.
  • In our case, we need APR package (Apache Run-Time)
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    cd ~/httpd2
    wget http://www.eu.apache.org/dist//apr/apr-1.5.2.tar.gz
    #extract the archive
    mkdir apr1 && tar xvf apr-1.5.2.tar.gz -C apr1 --strip-components 1
    cd apr1
    #run configure
    ./configure
    #if configure is not reporting errors. Compile by make
    make -j4
    #If compilation is successful
    sudo make install
  • Now, lets try to configure our Apache httpd server again
    1
    2
    cd ~/httpd2
    ./configure
  • This time it requires apr-util. Go to Apache site, and find the latest apr-util tar file:
    1
    2
    3
    4
    5
    6
    7
    8
    cd ~/httpd2
    wget http://www.us.apache.org/dist//apr/apr-util-1.5.4.tar.gz
    #check the hash for integrity using md5sum, sha1sum or sha2sum ..etc
    mkdir apr-util1 && tar xvf apr-util-1.5.4.tar.gz -C apr-util1 --strip-components 1
    cd apr-util1
    ./configure --with-apr=/usr/local/apr/bin/apr-1-config
    make -j4
    sudo make install
  • Try again to configure the httpd server. Again, it may complain about PCRE. If it complains, visit pcre.org and download latest package:
    1
    2
    3
    4
    5
    6
    cd ~/httpd2
    wget http://sourceforge.net/projects/pcre/files/pcre/8.40/pcre-8.40.tar.bz2
    mkdir pcre8 && tar xjvf pcre-8.40.tar.bz2 -C pcre8 --strip-components 1
    cd pcre8
    ./configure && make -j4 && sudo make install
    #in bash script, && means if the previous process is successful, then execute the next. || means the opposite
  • Configure httpd server. This time it should not complain anymore (though it may complain on different Linux distribution Check the list of requirement for apache). Now, you need to compile and install apache:
    1
    2
    3
    4
    5
    6
    #use ldconfig to autmatically detect other libraries and headers locations. If it didn't help, use --with- option in configure
    cd ~/httpd2
    sudo ldconfig
    ./configure
    make -j4 # -j4 means use 4 cores to speed up the process
    sudo make install
  • Finally, launch your apache server:
    1
    sudo /usr/local/apache2/bin/httpd
  • Launch your browser and type your server URL (i.e. http://localhost). If you see “it works!” then your installation was successful.

Configure Apache

Auto start

Usually, daemons (or services) such as Apache run automatically at your system start up, so the administrator doesn’t need to invoke manually on every reboot. Usually this done in Linux by putting a script in ‘/etc/init.d/<daemon_name>’. To do so, locate the httpd init.d script and copy it to /etc/init.d directory:

1
2
3
4
5
sudo cp /usr/local/apache2/bin/apachectl /etc/init.d/httpd
# to stop manually
sudo /etc/init.d/httpd stop
# to start manually
sudo /etc/init.d/httpd start

To enable automatic start at boot time, you need to link the copied script to default run levels for your system.

1
sudo update-rc.d httpd defaults

Note: A Linux system goes through several run levels from boot (run level 0) to halt (In ubuntu distribution is run level 6). Each run level contains a set of daemons that should be running. You can find a set of directories in “/etc” named “rc.d/”. Each of these contain symbolic links to scripts in /etc/init.d/. ‘update-rc.d’ simply creates/delete symbolic links in these directories. If you want to check your current run level, simply type ‘runlevel’ in your terminal.

To check whether your service is running or not, use netstat command with options ‘-ltcp’ as root (check ‘man netstat’ for options meanings). You will find httpd process listening to port 80. If you want to see the processes running by httpd daemon, type ‘ps aux | grep httpd’. Again, check for ‘aux’ meanings using ‘man’ command.

Finally, create a sym link for httpd.conf in /etc directory. This will make your life easier.

1
sudo ln -s /usr/local/apache2/conf/httpd.conf /etc/httpd.conf

Update Apache Configuration File (httpd.conf)

  • In your terminal create a directory “/var/www”
    1
    sudo mkdir /var/www
  • open /etc/httpd.conf as a root (use “nano” editor if you never tried any command line editor)
    1
    sudo nano /etc/httpd.conf
  • Add the following at the end of file to get rid of the warning message that you probably have noticed by now:
    1
    ServerName localhost
  • Locate “DocumentRoot” directive. In nano editor, press CTRL+W and type DocumentRoot to find it. Change the value to “/var/www”. Also, in the line below (i.e. “Directory” tag), change the string to “/var/www/”. It should look like this.
    1
    2
    DocumentRoot "/var/www"
    <Directory "/var/www">
  • Save the file (CTRL+X => ‘Y’ => ‘ENTER’) and restart httpd service
    1
    sudo /etc/init.d/httpd restart
  • Now, create your first HTML page. Create a file “index.html” in “/var/www/” diretory:
    1
    sudo touch /var/www/index.html
  • Write some html code in “index.html”.
    1
    echo "hello world, my name is [student name]" | sudo tee /var/www/index.html
  • In your browser check your first page (open your server address in browser).
  • If you are using Firefox, then you should stop Firefox from caching pages using the instructions provided here. When caching is on, Firefox might just display the cached version of ‘http://localhost’ and you might not see your changes taking effect.
  • In Linux, any process must belong to a user. Our Apache2 server is linked to a user called “daemon” which may have more permission than required. Usually we need to give least permission possible to secure our server. Imagine a hacker exploits a vulnerability in apache 2.4 and launches a shell, then he will have full permission of user “daemon”. We will create a user called “httpd” with only read/write/execute on “/var/www” directory, no more and no less.Create a new user httpd
    1
    sudo useradd httpd
  • Open httpd.conf file (sudo nano /etc/httpd.conf), and find the following lines:
    1
    2
    User daemon
    Group daemon
  • Change above two lines as below
    1
    2
    User httpd
    Group httpd
  • change /var/www/ owner
    1
    sudo chown -R httpd /var/www
  • Restart apache2, and check which user the process belongs to (sudo ps aux | grep httpd).

Backup

From time to time during this course, you may need to backup your system. In Linux you can use this command as a super user to backup the root file system.

1
sudo tar cvpzf ~/backup.tgz /var/www

Then copy the tar file to your local machine.

1
scp localhost:backup.tgz ~/server

Apache Directory structure

By default Apache is installed in ‘/usr/local/apache2/’, though you could have changed it before before compilation (using ./configure –prefix=<directory>). However, this is quite standard place for applications. Here is directory structure for apache:
—— bin
—— build
—— cgi-bin
—— conf
———— extra
———— original
—————— extra
—— error
———— include
—— fcgi-bin
—— htdocs
—— icons
———— small
—— include
—— logs
———— fcgidsock
—— man
———— man1
———— man8
—— manual
———— developer
———— faq
———— howto
———— images
———— misc
———— mod
———— platform
———— programs
———— rewrite
———— ssl
———— style
—————— css
—————— lang
—————— latex
—————— scripts
—————— xsl
———————— util
———— vhosts
———— modules

The main configuration file is in conf/httpd.conf. Open this file by your favorite command line editor (say nono or vim). Here are few important directives:

  • Listen: By default it listens to port 80
  • LoadModule: In Apache you can add modules to run php scripts, python scripts..etc. Modules are simply compiled libraries that extend Apache functionality.
  • DocumentRoot: Sets the location where web pages are served. By default, it is /usr/local/apache2/htdocs.

You can find the complete list of directives here.

Exercise 2.1

  • You are asked to finish the lab successfully and serve a simple html page (hello world, my name is <student name>).
  • Do not use “apt-get install apache”. You are asked to manually configure the server, not the other way around. We will verify your installation and will discover whether its automatic installation or manual.
  • The deadline is before the next lab.

Reference

http://httpd.apache.org/docs/2.4/