Lab 2.1: Setting up Linux server
Overview
In this lab, you will learn howto create a Linux server, access it through ssh (for linux), and copy files to/form the server. You will also compile and install an Apache web server. The goal of this lab is to give a real experience on deploying a server application such as Apache web server. Other servers (such as SMPT, LDAP..etc) follow similar set up process, so try to grasp the underlying mechanism.
In the subsequent labs, we use the term server for different things depending on the context. Some times we refer to a physical machine. We may also refer to an application running on a remote machine that listens to a port for requests. Such an application is also called service or daemon.
Setup the server
For this lab, you will be configuring you Ubuntu laptop to act as server using these steps.
- Install openssh-server which lets you access your computer securely over LAN or internet.
1sudo apt-get install openssh-server openssh-client - Edit the ssh config settings to disable root login for extra security:
1sudo nano /etc/ssh/sshd_config
This will open the configuration file. Find the following line in the file and change its value to ‘no’ as shown. Press CTRL-W and type PermitRootLogin to quickly find it:
1PermitRootLogin no
Now press CTRL-X and select Yes to save your changes. - Check the ssh settings are valid by running:
1sudo sshd -t
The command will tell you if you’ve made any syntax error in the config file. If you don’t get any output, it means there are no errors in the configuration file. - At this point you should be able to connect to the laptop (i.e., server) from itself in a terminal by:
123ssh localhost# or equivalentlyssh 127.0.0.1
Type your password when asked and you will log in. Now, try it from another machine on your network (for example, from another student’s or TA’s laptop). First find your IP by running the following command in a terminal and note down the value of inet addr for the WiFi interface. WiFi interface is either named wlan0 or wlp3s0:
1ifconfig
Now you can log in from another machine using:
1ssh [ip-address] - It is possible to configure your server to allow your computer to access it without providing a password each time you try to connect. To do this you need to generate an SSH key. First, check whether there are already keys on your computer:
1ls ~/.ssh
If you see files namedid_rsa.pub
orid_dsa.pub
you have keys set up already, so you can skip the generating keys step and go to the next step. To generate new SSH keys enter the following command. Replacekey-name
with your name or any other valid string of your choice.
1ssh-keygen -t rsa -C key-name
Upon entering this command, you’ll be asked a series of questions. Just keep hittingEnter
to go with the default values. Now you should see the filesid_rsa
andid_rsa.pub
in your.ssh
directory in your home folder:
123ls ~/.ssh# the out should look like thisauthorized_keys id_rsa id_rsa.pub known_hosts
Theid_rsa
file is your private key. Keep this on your computer. Theid_rsa.pub
file is your public key. This is what you put on machines you want to connect to. When the machine you try to connect to matches up your public and private key, it will allow you to connect. - Append the public key to your
authorized_keys
file on the server, sending it over SSH:
1cat ~/.ssh/id_rsa.pub | ssh localhost 'cat >> .ssh/authorized_keys'
Note that this time you will have to authenticate with your password.
Now you should be able to onnect without a password prompt:
1ssh localhost
Copy files to/from server
- Create and copy a file to a server
1234mkdir ~/servertouch testFile.txtecho "testing scp (server copy command)" | tee testFile.txtscp testFile.txt localhost:~/server
Note: the tailing colon ‘:’ indicates the location relative to user’s home directory. If you add a slash ‘/’ after the colon, you specify the directory from the root. - If you want to copy a directory, simply add option ‘-r’ (means recursive):
1234mkdir ~/recursivescpmv testscp.txt recursivescp/# scp -r local/directory/location remoteServer:/remote/directoryscp -r ~/recursivescp localhost:~/server
Similarly, if you want to copy from the server, all you need is to switch orders in the command.
12#scp -r remoteServer:/remote/directory local/directory/scp -r localhost:~/server/recursivescp ~/
Compiling & Installing Apache Web Server
You can always install servers using package managers such as apt-get. However, these package managers provide binaries that are compiled for generic CPU using few instruction sets which is usually a subset of your CPU instruction set. Therefore, they tend to be slower as it doesn’t fully utilize the CPU.
First, execute below command to login to your Amazon instance. Replace “[your-key].pem” below with your private key, and “[ec2-instance-address]” with the address of your EC2 server.
1 |
ssh localhost |
Install compilers
- First of all, we need to install some compiler to compile our open source applications. Namely, we need gcc for C applications and g++ for C++. The easiest way in Ubuntu is to type:
12sudo apt-get updatesudo apt-get install gcc g++ - Install make tool
1sudo apt-get install make
Install Apache
- Now, go to Apache website and find the latest stable httpd server. You can either download on your local machine, and then send it using scp to the server, or simply ssh to your server and type
12cd # change to home directorywget http://www.eu.apache.org/dist//httpd/httpd-2.4.25.tar.bz2 - Its also good habit to always check the file integrity to make sure that the file is not corrupted.
1sha1sum httpd-2.4.25.tar.bz2
Now compare first few digits with the one given in apache website (note that checking first few digits is fairly enough as hash functions such as sha1 have very random behavior and unlikely to find collision between two different hashes) . - Extract the tar package installed
1mkdir httpd2 && tar xvf httpd-2.4.25.tar.bz2 -C httpd2 --strip-components 1
To learn more about ‘tar’ tool, type ‘man tar’ in a terminal. Basically, option ‘x’ means extract, ‘j’ is setting archive type to bzip2, and ‘v’ mean verbose (Command prints more messages). - Now its time to configure and then compile the server.
12cd ~/httpd2./configure
Usually before compiling any open source package, you need to execute ./configure script to check the availability of all dependencies and whether your server has all required libraries. This is good for code portability and its quite standard way in Linux environment. - Now if you get some errors, read carefully and try to search and download the missing libraries or packages. You can use your package manager to download the binaries, but if you are performance enthusiastic then you should download each missing package manually. Since we are doing server configuration, performance is very important, so will download and install missing packages manually.
- In our case, we need APR package (Apache Run-Time)
1234567891011cd ~/httpd2wget http://www.eu.apache.org/dist//apr/apr-1.5.2.tar.gz#extract the archivemkdir apr1 && tar xvf apr-1.5.2.tar.gz -C apr1 --strip-components 1cd apr1#run configure./configure#if configure is not reporting errors. Compile by makemake -j4#If compilation is successfulsudo make install - Now, lets try to configure our Apache httpd server again
12cd ~/httpd2./configure - This time it requires apr-util. Go to Apache site, and find the latest apr-util tar file:
12345678cd ~/httpd2wget http://www.us.apache.org/dist//apr/apr-util-1.5.4.tar.gz#check the hash for integrity using md5sum, sha1sum or sha2sum ..etcmkdir apr-util1 && tar xvf apr-util-1.5.4.tar.gz -C apr-util1 --strip-components 1cd apr-util1./configure --with-apr=/usr/local/apr/bin/apr-1-configmake -j4sudo make install - Try again to configure the httpd server. Again, it may complain about PCRE. If it complains, visit pcre.org and download latest package:
123456cd ~/httpd2wget http://sourceforge.net/projects/pcre/files/pcre/8.40/pcre-8.40.tar.bz2mkdir pcre8 && tar xjvf pcre-8.40.tar.bz2 -C pcre8 --strip-components 1cd pcre8./configure && make -j4 && sudo make install#in bash script, && means if the previous process is successful, then execute the next. || means the opposite - Configure httpd server. This time it should not complain anymore (though it may complain on different Linux distribution Check the list of requirement for apache). Now, you need to compile and install apache:
123456#use ldconfig to autmatically detect other libraries and headers locations. If it didn't help, use --with- option in configurecd ~/httpd2sudo ldconfig./configuremake -j4 # -j4 means use 4 cores to speed up the processsudo make install
- Finally, launch your apache server:
1sudo /usr/local/apache2/bin/httpd - Launch your browser and type your server URL (i.e. http://localhost). If you see “it works!” then your installation was successful.
Configure Apache
Auto start
Usually, daemons (or services) such as Apache run automatically at your system start up, so the administrator doesn’t need to invoke manually on every reboot. Usually this done in Linux by putting a script in ‘/etc/init.d/<daemon_name>’. To do so, locate the httpd init.d script and copy it to /etc/init.d directory:
1 2 3 4 5 |
sudo cp /usr/local/apache2/bin/apachectl /etc/init.d/httpd # to stop manually sudo /etc/init.d/httpd stop # to start manually sudo /etc/init.d/httpd start |
To enable automatic start at boot time, you need to link the copied script to default run levels for your system.
1 |
sudo update-rc.d httpd defaults |
Note: A Linux system goes through several run levels from boot (run level 0) to halt (In ubuntu distribution is run level 6). Each run level contains a set of daemons that should be running. You can find a set of directories in “/etc” named “rc.d/”. Each of these contain symbolic links to scripts in /etc/init.d/. ‘update-rc.d’ simply creates/delete symbolic links in these directories. If you want to check your current run level, simply type ‘runlevel’ in your terminal.
To check whether your service is running or not, use netstat command with options ‘-ltcp’ as root (check ‘man netstat’ for options meanings). You will find httpd process listening to port 80. If you want to see the processes running by httpd daemon, type ‘ps aux | grep httpd’. Again, check for ‘aux’ meanings using ‘man’ command.
Finally, create a sym link for httpd.conf in /etc directory. This will make your life easier.
1 |
sudo ln -s /usr/local/apache2/conf/httpd.conf /etc/httpd.conf |
Update Apache Configuration File (httpd.conf)
- In your terminal create a directory “/var/www”
1sudo mkdir /var/www - open /etc/httpd.conf as a root (use “nano” editor if you never tried any command line editor)
1sudo nano /etc/httpd.conf - Add the following at the end of file to get rid of the warning message that you probably have noticed by now:
1ServerName localhost - Locate “DocumentRoot” directive. In nano editor, press CTRL+W and type DocumentRoot to find it. Change the value to “/var/www”. Also, in the line below (i.e. “Directory” tag), change the string to “/var/www/”. It should look like this.
12DocumentRoot "/var/www"<Directory "/var/www"> - Save the file (CTRL+X => ‘Y’ => ‘ENTER’) and restart httpd service
1sudo /etc/init.d/httpd restart - Now, create your first HTML page. Create a file “index.html” in “/var/www/” diretory:
1sudo touch /var/www/index.html - Write some html code in “index.html”.
1echo "hello world, my name is [student name]" | sudo tee /var/www/index.html - In your browser check your first page (open your server address in browser).
- If you are using Firefox, then you should stop Firefox from caching pages using the instructions provided here. When caching is on, Firefox might just display the cached version of ‘http://localhost’ and you might not see your changes taking effect.
- In Linux, any process must belong to a user. Our Apache2 server is linked to a user called “daemon” which may have more permission than required. Usually we need to give least permission possible to secure our server. Imagine a hacker exploits a vulnerability in apache 2.4 and launches a shell, then he will have full permission of user “daemon”. We will create a user called “httpd” with only read/write/execute on “/var/www” directory, no more and no less.Create a new user httpd
1sudo useradd httpd - Open httpd.conf file (sudo nano /etc/httpd.conf), and find the following lines:
12User daemonGroup daemon - Change above two lines as below
12User httpdGroup httpd - change /var/www/ owner
1sudo chown -R httpd /var/www - Restart apache2, and check which user the process belongs to (sudo ps aux | grep httpd).
Backup
From time to time during this course, you may need to backup your system. In Linux you can use this command as a super user to backup the root file system.
1 |
sudo tar cvpzf ~/backup.tgz /var/www |
Then copy the tar file to your local machine.
1 |
scp localhost:backup.tgz ~/server |
Apache Directory structure
By default Apache is installed in ‘/usr/local/apache2/’, though you could have changed it before before compilation (using ./configure –prefix=<directory>). However, this is quite standard place for applications. Here is directory structure for apache:
—— bin
—— build
—— cgi-bin
—— conf
———— extra
———— original
—————— extra
—— error
———— include
—— fcgi-bin
—— htdocs
—— icons
———— small
—— include
—— logs
———— fcgidsock
—— man
———— man1
———— man8
—— manual
———— developer
———— faq
———— howto
———— images
———— misc
———— mod
———— platform
———— programs
———— rewrite
———— ssl
———— style
—————— css
—————— lang
—————— latex
—————— scripts
—————— xsl
———————— util
———— vhosts
———— modules
The main configuration file is in conf/httpd.conf. Open this file by your favorite command line editor (say nono or vim). Here are few important directives:
- Listen: By default it listens to port 80
- LoadModule: In Apache you can add modules to run php scripts, python scripts..etc. Modules are simply compiled libraries that extend Apache functionality.
- DocumentRoot: Sets the location where web pages are served. By default, it is /usr/local/apache2/htdocs.
You can find the complete list of directives here.
Exercise 2.1
- You are asked to finish the lab successfully and serve a simple html page (hello world, my name is <student name>).
- Do not use “apt-get install apache”. You are asked to manually configure the server, not the other way around. We will verify your installation and will discover whether its automatic installation or manual.
- The deadline is before the next lab.
Reference
http://httpd.apache.org/docs/2.4/