My internship has finished. This blog will not be updated anymore. x

Week 10 at Thinline

I finished the highcharts statistics, implemented some check-in exceptions, I fiddled with the configuration of Apache and Nginx webserver on the Raspberry Pi and I learned how to deploy my webapp to a remote web server by using Capistrano. 

Exceptions

On monday I finished the highcharts configuration. I also implemented "exceptions": when an employee is sick, the admin has to make an exception for this user on a specific date. The admin can also make global exceptions, which will apply to all the users, for example a one week vacation. When calculating the score and different statistics, the functionality will now skip the calculation for that period of time for which the user or the global group has an exception. tl;dr The user won't lose points when he's not checked in (and has a valid reason, causing an exception to be set).

Running a webserver on the Raspberry Pi

I thought the webapplication now had enough functionality to do a first test run on the actual webserver of the raspberry pi. When I think of webservers, I mostly think of an Apache installation with PHP and mysql (a LAMP stack). So I began the configuration of my LAMP webserver on the raspberry pi.

Apache webserver

Installing an Apache webserver really isn't that hard. The first thing I did was get Raspbian (Debian) up to date with the latest OS and packages lists.

sudo apt-get install -y rpi-update
sudo apt-get update -y && sudo apt-get upgrade -y

Next, I installed Apache and PHP and then MySQL:

sudo apt-get install apache2 php5 libapache2-mod-php5
sudo apt-get install mysql-server mysql-client php5-mysql

And then I made my permissions:

sudo groupadd -f -g33 www-data
sudo chown pi:www-data /var/www
sudo chmod 775 /var/www

I made a quick php file with phpinfo(); and placed this file into /var/www. It worked! You'll get something like this:

Apache Phpinfo

I configured my setup by editting the default file: sudo nano  /etc/apache2/sites-available/default and don't forget to restart apache with sudo service apache2 restart.

Installing other tools

I also needed git to clone my repository, so I installed it with:

sudo apt-get install git-core

I also need a working PhpMyAdmin installation to manage my databases in a user-friendly way:

# Install phpmyadmin
sudo apt-get install phpmyadmin

# Include it into the apache config
sudo nano /etc/apache2/apache2.conf

# Add this line to the bottom
Include /etc/phpmyadmin/apache.conf

# Restart apache
sudo service apache2 restart

File sharing right in your Finder window would also be nice, so I installed the Netatalk service:

# Installation
sudo apt-get install netatalk

# Edit config
sudo /etc/init.d/netatalk stop
sudo nano /etc/netatalk/AppleVolumes.default

# Add web server path
~/ "Home Directory"
/var/www "Webserver"

# Start netatalk, it might take a few minutes to appear in the network
sudo /etc/init.d/netatalk start

I admit that sharing the webserver folder on the internal network is a large security breach, but it made things easier for me to fiddle around in the www folder right on my mac with Sublime Text 2, instead of using vi or nano with an SSH connection, just to edit php files. 

I also made the mysql accessible from my mac, instead of only allowing 127.0.0.1 connections on the RPi:

# Open the config
sudo nano /etc/mysql/my.cnf

# Comment out this line:
#bind-address = 127.0.0.1

# Restart mysql
sudo service mysql restart

You'll also need to create a user with his machine location set to "%" and with his own privileges. 

Speeding things up with Nginx & APC

I deployed my Silex webapplication with Capistrano to the raspberry pi and when I ran it on Apache, the webserver wasn't really fast. When I demonstrated the raspberry pi with the webserver and checkin app to the Thinline team, I was asked if I could speed things up a little. 

I started by doing some basic configurations, like doing the memory split configuration of the raspberry pi. On the raspberry pi the GPU (video) and system memory are shared. Since my RPi doesn’t use the GUI, I set my GPU memory to be 16MB. If you are using a GUI, you may want to go with 64, 128, or even 256MB of GPU ram. I made this configuration by opening the raspi config wizard you get on your first boot: sudo raspi-config

I was reading an interesting article about a comparison with benchmarks for different webservers on the raspberry pi. The article compares the webservers Apache, Lighttpd, Monkey and Nginx and the conclusion was that the fastest and most reliable solution is Nginx webserver. Although Monkey webserver is catching up fast with the big guys (it was actually the first time I heard about Monkey webserver), but I think it's still not fully ready for production and that's why I chose to install Nginx. 

INSTALLation of Nginx + PHP-FPM

sudo apt-get install nginx php5-fpm php5-cgi php5-cli php5-common

The configuration files are located into /etc/nginx/sites-available but to enable the config you have to make a symlink to /etc/nginx/sites-enabled. After the installation, there's a Default configuration in these folders, so I removed the symlink in the sites-enabled folder to disable this config. I made my own configuration "checkinapp" in the sites-available folder and symlinked it to enable it.

touch /etc/nginx/sites-available/checkinapp
sudo ln -s /etc/nginx/sites-available/checkinapp /etc/nginx/sites-enabled/checkinapp

Edit this file by doing sudo nano /etc/nginx/sites-available/checkinapp and make your configuration. Nginx doesn't work with .htaccess and url rewriting the way apache does. Everything is defined in the config file. Here's mine, including what I came up with after some intensive googling (config file caching, block htaccess and htpasswords, ...)

server {
		# listen for ipv4; this line is default and implied
		listen   80;

		# Working directory: web folder of Silex app
		root /var/www/current/www/web;

		# Index files
		index index.php index.html index.htm;

		# Make site accessible from http://localhost/
		server_name localhost;

		# Logging
		access_log /var/log/nginx/raspberry.access_log;
		error_log /var/log/nginx/raspberry.error_log info;

		# static file 404's aren't logged and expires header is set to maximum age
		location ~* \.(jpg|jpeg|gif|css|png|js|ico|html)$ {
				access_log off;
				expires max;
		}

		# caching of files
		location ~* \.(ico|pdf|flv)$ {
			expires 1y;
		}

		# Make sure files with the following extensions do not get loaded by nginx because nginx would display the source code, and these files can contain PASSWORDS!
		location ~* \.(engine|inc|info|install|make|module|profile|test|po|sh|.*sql|theme|tpl(\.php)?|xtmpl)$|^(\..*|Entries.*|Repository|Root|Tag|Template)$|\.php_ {
			deny all;
		}

		# gzip
		gzip on;
		gzip_disable "MSIE [1-6]\.(?!.*SV1)"; # disables gzip compression for browsers that don't support it (in this case MS Internet Explorer before version 6 SV1).
		gzip_http_version 1.1;
		gzip_vary on; # This sets the response header Vary: Accept-Encoding. Some proxies have a bug in that they serve compressed content to browsers that don't support it. By setting the Vary: Accept-Encoding header, you instruct proxies to store both a compressed and uncompressed version of the content.
		gzip_comp_level 6;
		gzip_proxied any;
		gzip_types text/plain text/html text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript text/x-js ;
		gzip_buffers 16 8k;

		# Pass PHP requests through PHP fast-cgi process manager (PHP-FPM)
		location ~ \.php$ {
			include fastcgi_params;
			fastcgi_intercept_errors on;
			fastcgi_pass 127.0.0.1:9000;
		}

		# Deny .htaccess and .htpassword files
		location ~ /\.ht {
			deny  all;
		}

		# Silex: route everything through index.php
		location / {
			autoindex on;
			if (-f $request_filename) {
					expires max;
					break;
			}
			rewrite ^(.*) /index.php last;
		}


		# PhpMyAdmin path routing
		location /phpmyadmin {
			root /usr/share/;
			index index.php index.html index.htm;
			location ~ ^/phpmyadmin/(.+\.php)$ {
				try_files $uri =404;
				root /usr/share/;
				fastcgi_pass 127.0.0.1:9000;
				include fastcgi_params;
				fastcgi_intercept_errors on;
			}
			location ~* ^/phpmyadmin/(.+\.(jpg|jpeg|gif|css|png|js|ico|html|xml|txt))$ {
				root /usr/share/;
			}
		}

}

Now, we will need to tell PHP-FPM to listen on port 9000 instead of the socket. That’s simple. We’ll have to edit just one line.

nano /etc/php5/fpm/pool.d/www.conf 

# Comment out this line and add a new line, like this:
#listen = /var/run/php5-fpm.sock
listen = 9000

I also editted my Nginx config file and have set my Nginx worker_processes to 1 instead of 4, because the raspberry pi only has a single CPU core working at 700 MHz. 

Some rules:

worker_processes [number of processor cores];
max clients = worker_processes * worker_connections

What I've done:

sudo nano /etc/nginx/nginx.conf
worker_processes  1; # 1 worker process instead of default 4
worker_connections  128;  #128 connections instead of default 768

Finally, restart both nginx and PHP-FPM. 

sudo service php5-fpm restart 
sudo service nginx restart

At this point I didn't need Apache2 to boot on startup, so I disabled this:

update-rc.d -f apache2 remove

With Nginx on the raspberry pi, static HTML pages now load superfast and php webpages load pretty fast

 

Setting up caching with PHP-APC

The usage of Alternative PHP Cache (APC), a free and open opcode cache for PHP, can significantly speed up your PHP application. I read about Wordpress installations on the raspberry pi with a 200% faster (!) response time with APC enabled. PHP-APC is a free, open and robust framework for caching and optimizing PHP code. The PHP code isn't parsed everytime it is requested, but instead it's saved in cache and only parsed the first time. 

To install APC:

sudo apt-get install php-apc

Then I editted the configuration file and added the following lines to enable APC and set the cache size:

sudo nano /etc/php5/conf.d/20-apc.ini

[apc]
extension=apc.so
apc.enabled=1
apc.shm_size=12M  ; the desired cache size
apc.ttl=7200
apc.user_ttl=7200

;Setting this to 0 will give you the best performance, as APC will
;not have to check the IO for changes. However, you must clear
;the APC cache to recompile already cached files. If you are still developing, set it to 1.
apc.stat=1

I restarted Nginx and I opened my phpinfo() page and saw that APC was now enabled. Hurray!

I already had the Symfony2 webprofiler installed in my Silex webapplication, so I used the debugger toolbar to get an idea of the execution time and memory usage. I started my Apache2 server and stopped Nginx and I loaded the page a few times. I did the same for Nginx. These are some results and while they're probably not fully accurate, I got a basic idea of the speed I gained by using Nginx with APC. 

 
Apache webserver Nginx webserver with APC
4615 ms 2485 ms
4041 ms 1922 ms
3577 ms 1753 ms
4209 ms 1773 ms
4091 ms 1764 ms

Capistrano deployment

I didn't want to copy my entire project or use ftp upload every time I wanted to update the webapp on the raspberry pi. Actually, FTP is slow, stupid, and lacks an undo feature. Capistrano can be used to do secure and efficient deployments, but also rollback in case a problem occurs. The past couple of months, I wanted to learn to work with Capistrano but I haven't found the time yet. So I installed Capistrano this week on my mac, started reading on how to make your configuration, and I made my first succesful deployment! 

While Capistrano is an automation tool originally written for the Rails community, it's perfectly suitable for other languages such as PHP. You'll need Ruby installed though, because the application & config files are written in Ruby. If you are running OS X or Linux, this is already installed. Windows users will have to install Ruby. 

You can install Capistrano with the RubyGems package manager:

gem install capistrano

Once installed, you can create the capistrano deployment files by executing the capify command in your current working directory. This will generate a Capfile and /config/deploy.rb file. Here's my configuration for deploy.rb. Note: I'm just getting started with Capistrano so I don't expect this config to be absolutely perfect, but it gets the job done right now. I also created a few tasks, like symlinking my config.yml file from the Shared folder to the current release folder, and installing the vendors. Maybe I'll let the vendors folder install into the Shared folder and symlink it to the current release, so I won't have to install my vendors every time. 

# What is the name of the local application?
set :application, "CheckinApp"

# What user is connecting to the remote server?
set :user, "pi"

# Where is the local repository?
set :local_repository, "."

# How are the project files being transferred?
set :deploy_via, :copy

# What is the production server domain?
role :web, "10.0.1.xxx"

# What remote directory hosts the production website?
set :deploy_to,   "/var/www"

# Is sudo required to manipulate files on the remote server?
set :use_sudo, false

# What version control solution does the project use?
set :scm,        :git
set :branch,     'master'
set :repository,  "[email protected]:xxx/checkinapp.git"

# Keep 3 releases
set :keep_releases, 3

# Maintain a local repository cache. Speeds up the copy process.
set :copy_cache, true

# Ignore any local files?
set :copy_exclude, %w(.git)

# run composer install
before "deploy:restart", "deploy:install"
namespace :deploy do
	desc "run composer install and ensure all dependencies are installed"
	task :install do
        run "cd #{release_path}/www && composer install"
    end
end

# Create symlink to the Silex config yaml file
after "deploy:update_code", "checkinapp:symlink"
namespace :checkinapp do
  desc "Create symlink to the Silex config yaml file"
  task :symlink do
    run "ln -s #{shared_path}/config #{release_path}/www/config"
  end
end

To initialize the deployment on the webserver (create the folder structure), I just have to run:

cap deploy:setup

And every time I've committed & pushed some code to the git repository I can now easily deploy by doing:

cap deploy

That's really all there is to deploy your webapplication! You can do A LOT with capistrano, e.g do a rollback, clean your releases, ... I plan on using this for every website I'll be making in the future. 

Next week?

I'm currently converting the badges class from my old code to the newly created BadgesController in my Silex app. I've also setup crontab to periodically run my cronjobs to calculate the userscore.

The NFC/RFID break-out board, the cobbler kit and the LCD display kit were out of stock two weeks ago, and finally got back in stock this week. I suppose they will be delivered on friday or monday, which means I can work on the hardware part next week, doing some soldering and some cursing to make the electronics work together. Looking forward to this!

Write a comment