Cloud Computing Definition

What is Cloud Computing ?

The Definition of the Cloud Computing is depending on what you serve/use of the Cloud Technology.
So If you are used storage service; then the Cloud will be a free space on the Internet that is accessible to you whenever you want.
Buy if you used MapReduce to process some data; then the definition will be A computing Power that serves you to process Data.
all of these definitions are not describing the real Cloud Computing.

The definition of Cloud Computing from My perspective is:

Computing Services On-Demand.

Yes. it is as simple as these words, and I will proof it at the end of the article.

Cloud Computing as defined by ” NIST ” -National Institute of Standards and Technology- is :

a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction.

NIST identifies five Essential Cloud Computing Characteristics:

  1. On-demand self-service
  2. Broad network access.
  3. Resource pooling.
  4. Rapid elasticity.
  5. Measured service.

And This is My Explanation for these Characteristics.

  • On-demand self-service means that Anybody can provision Any service at Any time without the interaction of Anybody.
  • Broad network Access means that Client can access his provisioned services using heterogeneous -varied- platforms.
  • Resource pooling means that the computing resources can be dynamically assigned and reassigned to the consumers according to consumers Demands.
  • Rapid elasticity means that the cloud provisions 100% of the consumers requests as soon as they Demands it.
  • Measured service means that Cloud systems automatically control and optimize “resource use” by leveraging a metering capability appropriate to the type of service -use metering capability to maximum advantage-.

The Cloud Computing providers offer their services according to several fundamental models:

  1. Software as a Service (SaaS).
  2. Platform as a Service (PaaS).
  3. Infrastructure as a Service (IaaS).
  • Software as a Service is providing the consumer with Applications that are running on a Cloud Infrastructure. (Facebook, Twitter)
  • Platform as a Service is enabling the consumer to deploy Web applications without the complexity of building and maintaining the infrastructure . (AWS Elastic Beanstalk, Google app engine)
  • Infrastructure as a Service is enabling the consumer to provision computing resources on-demand and giving him the ability to use these resources to build his own environment. (AWS EC2).


This Image describes what is Cloud Computing.

Now, we have reached the end of the article so I have to prove my perspective of cloud computing Definition.

if you read the Characteristics section carefully you will find that three of the five characteristics are depending on the consumer Demands, and the last characteristic was about Measuring the demanded services, so Cloud Computing key characteristic is to be demanded.

And if we checked how these huge computing power can be served to the consumers we will find that the infrastructure, software, and platform have to be served as services as per the service model.

so this is the reason of defining Cloud Computing as “Computing Services On-Demand

References:

  1. “The NIST Definition of Cloud Computing”
  2. http://en.wikipedia.org/wiki/Cloud_computing

Original Post By me at LinkedIn.com

Advertisements

Live Sync Deamon

LSYNCD…

What is LSYNCD ?

LSYNCD is a Service which uses Rsync to sync the changes between 2 Directories in real time .

why to use LSYNCD ?

If you need to replicate data in some location to another one in a very short time.

example of usage :

if you need to create 2 load balanced replicated servers .

Installing LSYNCD on CENTOS:

The first step is to install yum atomic repository .

so follow the steps :

# cd /etc/yum.repos.d/

# touch atomic.repo

#vim atomic.repo

now paste these Lines int atomic.repo

# Name: Atomic Rocket Turtle RPM Repository for CentOS / Red Hat Enterprise Linux 6 –
# URL: http://www.atomicrocketturtle.com/
# Note: This isn’t covered by ASL support. -Scott
[atomic]
name = CentOS / Red Hat Enterprise Linux $releasever – atomicrocketturtle.com
mirrorlist = http://www.atomicorp.com/mirrorlist/atomic/centos-6-$basearch
#mirrorlist = http://www.atomicorp.com/channels/atomic/centos/6/mirrors-atomic
enabled = 1
priority = 1
protect = 0
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY.art.txt
gpgcheck = 1

# Almost Stable, release candidates for [atomic]
# Note these are not supported by anyone. -Scott
[atomic-testing]
name = CentOS / Red Hat Enterprise Linux $releasever – atomicrocketturtle.com – (Testing)
mirrorlist = http://www.atomicorp.com/mirrorlist/atomic-testing/centos-6-$basearch
enabled = 1
priority = 1
protect = 0
gpgcheck = 0

after saving the file you have to install the LSYNCD using yum.

yum -y install LSYNCD

now you have the LSYNCD installed on your machine.

we have to write the configuration file now.

so we will create the configuration file in /etc/lsyncd

#mkdir /etc/lsyncd

#vim /etc/lsyncd/NAME.conf

now write the following:

sync{
default.rsync,
source=”/Source/Full/Path/”,
target=”/Destination/Full/path/”,
delay=0,#this is the time delay between the change and the sync trigger
rsyncOpts={“–delete”,”-az”,”-o”,”-g”},#normal Rsync Options
logfile = ‘/var/log/lsyncd.log’#this is the log file
}

to Execute the lsyncd Service

#lsyncd /etc/lsyncd/NAME.conf

if you need to execute LSYNCD through the network then you need to setup a no password ssh session from the master node to the slave node.

and then write a configuration file as shown below

sync{
default.rsync,
source=”/Source/Full/Path/”,
target=”192.168.1.2:/Destination/Full/path/”,
delay=0,#this is the time delay between the change and the sync trigger
rsyncOpts={“–delete”,”-az”,”-o”,”-g”},#normal Rsync Options
logfile = ‘/var/log/lsyncd.log’#this is the log file
}

Assume 192.168.1.2 the target Machine IP.

This is the only difference between the local and the network sync.

Regards,

Installing and Configuring MySQL Cluster

Introduction

This HOWTO is designed for a classic setup of two servers behind a load-balancer. The aim is to have true redundancy – either server can be unplugged and yet the site will remain up.

Notes:

You MUST have a third server as a management node but this can be shut down after the cluster starts. Also note that I do not recommend shutting down the management server (see the extra notes at the bottom of this document for more information). You can not run a MySQL Cluster with just two servers And have true redundancy.

Although it is possible to set the cluster up on two physical servers you WILL NOT GET the ability to “kill” one server and for the cluster to continue as normal. For this you need a third server running the management node.

we are going to talk about three servers:

node01.example.com 192.168.0.10

node02.example.com 192.168.0.20

node03.example.com 192.168.0.30

Servers node01 and node02 will be the two that end up “clustered”. This would be perfect for two servers behind a loadbalancer or using round robin DNS and is a good replacement for replication. Server node03 needs to have only minor changes made to it and does NOT require a MySQL install. It can be a low-end machine and can be carrying out other tasks.

Get the software:

For Generally Available (GA), supported versions of the software, download from

http://www.mysql.com/downloads/cluster/

Make sure that you select the correct platform – in this case, “Linux – Generic” and then the correct architecture (for LINUX this means x86 32 or 64 bit).

Note: Only use MySQL Server executables (mysqlds) that come with the MySQL Cluster installation.

STAGE1: Installation of Data and SQL nodes on node01 and node02

On each of the machines designated to host data or SQL nodes(in our case node01 and node02), perform the following steps as the system root user:

  1. create a new mysql user group, and then add a mysql user to this group:
    shell> groupadd mysqlshell> useradd -g mysql mysql

     

  2. Change location to the directory containing the downloaded file, unpack the archive, and create a symlink to the mysql directory named mysql. Note that the actual file and directory names vary according to the MySQL Cluster version number.
    shell> cd /var/tmpshell> tar -C /usr/local -xzvf mysql-cluster-gpl-7.1.5-linux-x86_64-glibc23.tar.gzshell> ln -s /usr/local/mysql-cluster-gpl-7.1.5-linux-i686-glibc23 /usr/local/mysql

    shell> export PATH=$PATH:/usr/local/mysql/bin

    shell> echo “export PATH=\$PATH:/usr/local/mysql/bin” >> /etc/bash.bashrc

  3. Change location to the mysql directory and run the supplied script for creating the system databases:
    shell> cd mysqlshell> ./scripts/mysql_install_db –user=mysql
  4. Set the necessary permissions for the MySQL server and data directories:
    shell> chown -R root .shell> chown -R mysql datashell> chgrp -R mysql .
  5. Copy the MySQL startup script to the appropriate directory, make it executable, and set it to start when the operating system is booted up:
    shell> cp support-files/mysql.server /etc/init.d/mysqlshell> chmod +x /etc/init.d/mysqlshell> update-rc.d mysql defaults

STAGE2: Installation of Management node on node03

Installation of the management node does not require the mysqld binary. Only the MySQL Cluster management server (ndb_mgmd) is required; I assume that you have placed mysql-cluster-gpl-7.1.5-linux-i686-glibc23.tar.gz in /var/tmp.

As system root perform the following steps to install ndb_mgmd and ndb_mgm on the Cluster management node host (node02):

  1. Change location to the /var/tmp directory, and extract the ndb_mgm and ndb_mgmd from the archive into a suitable directory such as /usr/local/bin:
    shell> cd /var/tmpshell> tar -zxvf mysql-cluster-gpl-7.1.5-linux-i686-glibc23.tar.gzshell> cd /usr/local/mysql-cluster-gpl-7.1.5-linux-i686-glibc23

    shell> cp bin/ndb_mgm* /usr/local/bin

  2. Change location to the directory into which you copied the files, and then make both of them executable:
    shell> cd /usr/local/binshell> chmod +x ndb_mgm*

STAGE3: Configuration of Management node

The first step in configuring the management node is to create the directory in which the configuration file can be found and then to create the file itself. For example (running as root):

shell> mkdir /var/lib/mysql-clustershell> cd /var/lib/mysql-clustershell> vi config.ini

For our setup, the config.ini file should read as follows:

[NDBD DEFAULT]
NoOfReplicas=2
DataDir=/var/lib/mysql-cluster
DataMemory=1G
IndexMemory=1G[MYSQLD DEFAULT]
[NDB_MGMD DEFAULT]
[TCP DEFAULT]

# 1 Managment Servers
[NDB_MGMD]
HostName=192.168.116.30 # IP/host of first NDB_MGMD-node
NodeId=3

# 2 Storage Engines
[NDBD]
HostName=192.168.116.10 # IP/host of first NDBD-node
NodeId=1
[NDBD]
HostName=192.168.116.20 # IP/host of second NDBD-node
NodeId=2

# 3 MySQL Clients
# Lave this blank to allow rapid changes of the mysql clients.

[mysqld]
NodeId=5
HostName=192.168.116.10
[mysqld]
NodeId=6

STAGE4: Configuration of Data and SQL nodes

The first step in configuring the management node is to create the directory in which the configuration file can be found and then to create the file itself. For example (running as root):

shell> vi /etc/my.cnf

Note :
We show vi being used here to create the file, but any text editor should work just as well.

For each data node and SQL node in our setup, my.cnf should look like this:

[client]
port = 3306
[mysqld]port = 3306

skip-external-locking
# Options for mysqld process:
ndbcluster # run NDB storage engine
ndb-connectstring=192.168.116.30 # location of management server

[mysql_cluster]
# Options for ndbd process:
ndb-connectstring=192.168.116.30 # location of management server

Important :
Once you have started a mysqld process with the NDBCLUSTER and ndb-connectstring parameters in the [mysqld] in the my.cnf file as shown previously, you cannot execute any CREATE TABLE or ALTER TABLE statements without having actually started the cluster. Otherwise, these statements will fail with an error.

STAGE4: Starting the MySQL Cluster

Starting the cluster is not very difficult after it has been configured. Each cluster node process must be started separately, and on the host where it resides. The management node should be started first, followed by the data nodes, and then finally by any SQL nodes:

  1. On the management host(node03), issue the following command from the system shell to start the management node process:
    shell> ndb_mgmd -f /var/lib/mysql-cluster/config.ini –configdir=/var/lib/mysql-clusetr
  2. On each of the Data/SQL node hosts, run these commands to start the ndbd and mysql server process:
    shell> /usr/local/mysql/bin/ndbdshell> /etc/init.d/mysql start

If all has gone well, and the cluster has been set up correctly, the cluster should now be operational. You can test this by invoking the ndb_mgm management node client. The output should look like that shown here:

node03:~# ndb_mgm– NDB Cluster — Management Client –ndb_mgm> SHOW

Connected to Management Server at: localhost:1186

Cluster Configuration

———————

[ndbd(NDB)] 2 node(s)

id=2 @192.168.0.10 (mysql-5.1.44 ndb-7.1.5, Nodegroup: 0, Master)

id=3 @192.168.0.20 (mysql-5.1.44 ndb-7.1.5, Nodegroup: 0)

[ndb_mgmd(MGM)] 1 node(s)

id=1 @192.168.0.30 (mysql-5.1.44 ndb-7.1.5)

[mysqld(API)] 2 node(s)

id=4 @192.168.0.10 (mysql-5.1.44 ndb-7.1.5)

id=5 @192.168.0.20 (mysql-5.1.44 ndb-7.1.5)

STAGE5: Testing the Setup

If you are OK to here it is time to test mysql. On either server node01 or node02 enter the following commands: Note that we have no root password yet:

shell> mysqlcreate database testdb;use test;

CREATE TABLE cluster_test (i INT) ENGINE=NDBCLUSTER;

INSERT INTO cluster_test (i) VALUES (1);

SELECT * FROM cluster_test;

You should see 1 row returned (with the value 1).

If this works, now go to the other server and run the same SELECT and see what you get. Insert from that host and go back to previous host and see if it works. If it works then congratulations!

Install Nginx/PHP-FPM on Fedora 17/16, CentOS/RHEL 6.2/5.8

PHP-FPM is easier to use and configure than FastCGI and some tests says that PHP-FPM is even much more efficient than FastCGI.

Nginx (engine x) is a robust, small and high performance http server, reverse proxy server and also mail proxy server.

PHP-FPM (FastCGI Process Manager) is an alternative PHP FastCGI implementation with some additional features useful for sites of any size, especially busier sites.

Install Nginx, PHP 5.4.4 and PHP-FPM on Fedora 17/16/15/14, CentOS 6.2/6.1/6/5.8, Red Hat (RHEL) 6.2/6.1/6/5.8

1. Change to root user.

sudo -i
## OR ##
su -

2. Install needed repositories

Fedora 17/16/15/14 Remi repository

## Remi Dependency on Fedora 17, 16, 15
rpm -Uvh http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-stable.noarch.rpm 
rpm -Uvh http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-stable.noarch.rpm

## Fedora 17 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-17.rpm

## Fedora 16 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-16.rpm

## Fedora 15 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-15.rpm

## Fedora 14 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-14.rpm

CentOS 6.2/6.1/6/5.8 and Red Hat (RHEL) 6.2/6.1//6/5.8 Remi repository

## Remi Dependency on CentOS 6 and Red Hat (RHEL) 6 ##
rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-7.noarch.rpm

## CentOS 6 and Red Hat (RHEL) 6 ##
rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-6.rpm

## Remi Dependency on CentOS 5 and Red Hat (RHEL) 5 ##
rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

## CentOS 5 and Red Hat (RHEL) 5 ## 
rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-5.rpm

CentOS 6.2/6.1/6/5.8 and Red Hat (RHEL) 6.2/6.1/6/5.8 Nginx repository

Create file /etc/yum.repos.d/nginx.repo and add following content to repo file:
CentOS

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1

Red Hat (RHEL)

[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/rhel/$releasever/$basearch/
gpgcheck=0
enabled=1

3. Install Nginx, PHP 5.4.4 and PHP-FPM

Fedora 17/16/15/14, CentOS 6.2/5.8 and Red Hat (RHEL) 6.2/5.8

yum --enablerepo=remi install nginx php php-fpm php-common

CentOS 6.2/5.8 and Red Hat (RHEL) 6.2/5.8

yum --enablerepo=remi,remi-test install nginx php php-fpm php-common

4. Install PHP 5.4.4 modules

  • APC (php-pecl-apc) – APC caches and optimizes PHP intermediate code
  • CLI (php-cli) – Command-line interface for PHP
  • PEAR (php-pear) – PHP Extension and Application Repository framework
  • PDO (php-pdo) – A database access abstraction module for PHP applications
  • MySQL (php-mysql) – A module for PHP applications that use MySQL databases
  • PostgreSQL (php-pgsql) – A PostgreSQL database module for PHP
  • MongoDB (php-pecl-mongo) – PHP MongoDB database driver
  • SQLite (php-sqlite) – Extension for the SQLite V2 Embeddable SQL Database Engine
  • Memcache (php-pecl-memcache) – Extension to work with the Memcached caching daemon
  • Memcached (php-pecl-memcached) – Extension to work with the Memcached caching daemon
  • GD (php-gd) – A module for PHP applications for using the gd graphics library
  • XML (php-xml) – A module for PHP applications which use XML
  • MBString (php-mbstring) – A module for PHP applications which need multi-byte string handling
  • MCrypt (php-mcrypt) – Standard PHP module provides mcrypt library support

Select what you need: APC, CLI, PEAR, PDO, MySQL, PostgreSQL, MongoDB, SQLite, Memcache, Memcached, GD, MBString, MCrypt, XML

Fedora 17/16/15/14

yum --enablerepo=remi install php-pecl-apc php-cli php-pear php-pdo php-mysql php-pgsql php-pecl-mongo php-sqlite php-pecl-memcache php-pecl-memcached php-gd php-mbstring php-mcrypt php-xml

CentOS 6.2/5.8 and Red Hat (RHEL) 6.2/5.8

yum --enablerepo=remi,remi-test install php-pecl-apc php-cli php-pear php-pdo php-mysql php-pgsql php-pecl-mongo php-sqlite php-pecl-memcache php-pecl-memcached php-gd php-mbstring php-mcrypt php-xml

5. Stop httpd (Apache) server, Start Nginx HTTP server and PHP-FPM (FastCGI Process Manager)

Stop httpd (Apache)

/etc/init.d/httpd stop
## OR ##
service httpd stop

Start Nginx

/etc/init.d/nginx start ## use restart after update
## OR ##
service nginx start ## use restart after update

Start PHP-FPM

/etc/init.d/php-fpm start ## use restart after update
## OR ##
service php-fpm start ## use restart after update

6. Autostart Nginx and PHP-FPM on boot, also prevent httpd (Apache) autostarting on boot

Prevent httpd (Apache) autostarting on boot

chkconfig httpd off

Autostart Nginx on boot

chkconfig --add nginx
chkconfig --levels 235 nginx on

Autostart PHP-FPM on boot

chkconfig --add php-fpm
chkconfig --levels 235 php-fpm on

7. Configure Nginx and PHP-FPM

Create directory layout for your site

I use here testsite.local site, but this could of course be your real site, like www.if-not-true-then-false.com.

## public_html directory and logs directory ##
mkdir -p /srv/www/testsite.local/public_html
mkdir /srv/www/testsite.local/logs
chown -R nginx:nginx /srv/www/testsite.local

Alternative setup to add logs under /var/log directory.

## public_html directory and logs directory ##
mkdir -p /srv/www/testsite.local/public_html
mkdir -p /var/log/nginx/testsite.local
chown -R nginx:nginx /srv/www/testsite.local
chown -R nginx:nginx /var/log/nginx

Create and configure virtual host directories under /etc/nginx

mkdir /etc/nginx/sites-available
mkdir /etc/nginx/sites-enabled

Add following lines to /etc/nginx/nginx.conf file, after “include /etc/nginx/conf.d/*.conf” line (inside http block).

## Load virtual host conf files. ##
include /etc/nginx/sites-enabled/*;

Create testsite.local virtual host file

Add following content to /etc/nginx/sites-available/testsite.local file. This is very basic virtual host config.

server {
    server_name testsite.local;
    access_log /srv/www/testsite.local/logs/access.log;
    error_log /srv/www/testsite.local/logs/error.log;
    root /srv/www/testsite.local/public_html;

    location / {
        index index.html index.htm index.php;
    }

    location ~ \.php$ {
        include /etc/nginx/fastcgi_params;
        fastcgi_pass  127.0.0.1:9000;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME /srv/www/testsite.local/public_html$fastcgi_script_name;
    }
}

Link your virtual host to /etc/nginx/sites-enabled

cd /etc/nginx/sites-enabled/
ln -s /etc/nginx/sites-available/testsite.local
service nginx restart

Add your testsite.local “domain” to /etc/hosts file

/etc/hosts file Nginx on same machine
127.0.0.1 … row should look like example following:

127.0.0.1               localhost.localdomain localhost testsite.local

And if you use another machine where you are running your Nginx server, then add it’s public IP as following:

10.0.2.19               wordpress

Note: This is just very simple basic configuration, but if you want configure and optimize Nginx and PHP-FPM more then check following guide, Nginx and PHP-FPM Configuration and Optimizing Tips and Tricks

8. Test your Nginx and PHP-FPM setup

Create /srv/www/testsite.local/public_html/index.php file with following content:

<?php 
    phpinfo();
?>

Enable Remote Connection to Nginx Web Server (Open Port 80 on Iptables Firewall)

1. Edit /etc/sysconfig/iptables file:

nano -w /etc/sysconfig/iptables

2. Add following line before COMMIT:

-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT

3. Restart Iptables Firewall:

service iptables restart
## OR ##
/etc/init.d/iptables restart

4. Test remote connection

Access following address, with your browser. http://your.domain/

Nginx and PHP-FPM Configuration and Optimizing Tips and Tricks

if you want to squeeze all the juice out of your VPS or web server / servers and do your maintenance work little bit easier, then this guide might be useful. These tips are based entirely on my own experience, so they may not be an absolute truth, and in some situations, a completely different configuration may work better. It’s also good to remember leave resources for another services also if you run example, MySQL, PostgreSQL, MongoDB, Mail server, Name server and/or SSH server on same machine.

And yes here we go…

Nginx Configuration and Optimizing Tips and Tricks

Nginx Tip 1. – Organize Nginx Configuration Files

Normally Nginx configuration files are located under /etc/nginx path.
One good way to organize configuration files is use Debian/Ubuntu Apache style setup:

## Main configuration file ##
/etc/nginx/nginx.conf

## Virtualhost configuration files on ##
/etc/nginx/sites-available/
/etc/nginx/sites-enabled/

## Other config files on (if needed) ##
/etc/nginx/conf.d/

Virtualhost files have 2 paths, because sites-available directory can contain any stuff, like test configs, just copied/created configs, old configs and so on. And sites-enabled contains only really enabled configurations, actually just only symbolic links to sites-available directory.

Remember add following includes at the end of your nginx.conf file:

## Load virtual host conf files. ##
include /etc/nginx/sites-enabled/*;

## Load another configs from conf.d/ ##
include /etc/nginx/conf.d/*;

Nginx Tip 2. – Determine Nginx worker_processes and worker_connections

Default setup is okay for worker_processes and worker_connections, but these values could be little bit optimized:
max_clients = worker_processes * worker_connections

Just Nginx basic setup can handle hundreds of concurrent connection:

worker_processes  1;
worker_connections  1024;

Normally 1000 concurrent connection / per one server is good, but sometimes other parts like disks on server might be slow, and it causes that the Nginx is locked on I/O operations. To avoid locking use example following setup: one worker_precess / per processor core, like:
Worker Processes

worker_processes [number of processor cores];

To check how many processor cores do you have, run following command:

cat /proc/cpuinfo |grep processor
processor	: 0
processor	: 1
processor	: 2
processor	: 3

So here is 4 cores and worker_processes final setup could be following:

worker_processes 4;

Worker Connections
Personally I stick with 1024 worker connections, because I don’t have any reason to raise this value. But if example 4096 connections per second is not enough then it’s possible to try to double this and set 2048 connections per process.

worker_processes final setup could be following:

worker_connections 1024;

I have seen some configurations where server admins are used too much Apache and think if I set Nginx worker_processes to 50 and worker_connections to 20000 then my server could handle all traffic once what we get monthly…but yes it’s not true. It’s just wasting of resources and might cause some serious problems…

Nginx Tip 3. – Hide Nginx Server Tokens / Hide Nginx version number

This is good for security reasons hide server tokens / hide Nginx version number, especially, if run some outdated version of Nginx. This is very easy to do just set server_tokens off under http/server/location section, like:

server_tokens off;

Nginx Tip 4. – Nginx Request / Upload Max Body Size (client_max_body_size)

If you want to allow users upload something or upload personally something over the HTTP then you should maybe increase post size. It can be done with client_max_body_size value which goes underhttp/server/location section. On default it’s 1 Mb, but it can be set example to 20 Mb and also increase buffer size with following configuration:

client_max_body_size 20m;
client_body_buffer_size 128k;

If you get following error, then you know that client_max_body_size is too low:
“Request Entity Too Large” (413)

Nginx Tip 5. – Nginx Cache Control for Static Files (Browser Cache Control Directives)

Browser caching is import if you want save resources and bandwith. It’s easy setup with Nginx, following is very basic setup where logging (access log and not found log) is turned off and expires headers are set to 360 days.

location ~* \.(jpg|jpeg|gif|png|css|js|ico|xml)$ {
    access_log        off;
    log_not_found     off;
    expires           360d;
}

If you want more complicated headers or some other expiration by filetypes then you could configure those separately.

Nginx Tip 6. – Nginx Pass PHP requests to PHP-FPM

Here you could use default tpc/ip stack or use directly Unix socket connection. You have to also setup PHP-FPM listen exactly same ip:port or unix socket (with Unix socket also socket permission have to be right). Default setup is use ip:port (127.0.0.1:9000) you could of course change ips and ports what PHP-FPM listens. Here is very basic configuration with Unix socket example commented out:

# Pass PHP scripts to PHP-FPM
location ~* \.php$ {
    try_files       $uri /index.php;
    fastcgi_index   index.php;
    fastcgi_pass    127.0.0.1:9000;
    #fastcgi_pass   unix:/var/run/php-fpm/php-fpm.sock;
    include         fastcgi_params;
    fastcgi_param   SCRIPT_FILENAME    $document_root$fastcgi_script_name;
    fastcgi_param   SCRIPT_NAME        $fastcgi_script_name;
}

It’s also possible to run PHP-FPM another server and Nginx another.

Nginx Tip 7. – Prevent (deny) Access to Hidden Files with Nginx

It’s very common that server root or other public directories have hidden files, which starts with dot (.) and normally those is not intended to site users. Public directories can contain version control files and directories, like.svn, some IDE properties files and .htaccess files. Following deny access and turn off logging for all hidden files.

location ~ /\. {
    access_log off;
    log_not_found off; 
    deny all;
}

PHP-FPM Configuration Tips and Tricks

PHP-FPM Tip 1. – PHP-FPM Configuration files

Normally PHP-FPM configuration files are located on /etc/php-fpm.conf file and /etc/php-fpm.d path. This is normally excellent start and all pool configs goes to /etc/php-fpm.d directory. You need to add following include line on your php-fpm.conf file:

include=/etc/php-fpm.d/*.conf

PHP-FPM Tip 2. – PHP-FPM Global Configuration Tweaks

Set up emergency_restart_thresholdemergency_restart_interval and process_control_timeout. Default values for these options are totally off, but I think it’s better use these options example like following:

emergency_restart_threshold 10
emergency_restart_interval 1m
process_control_timeout 10s

What this mean? So if 10 PHP-FPM child processes exit with SIGSEGV or SIGBUS within 1 minute then PHP-FPM restart automatically. This configuration also sets 10 seconds time limit for child processes to wait for a reaction on signals from master.

PHP-FPM Tip 3. – PHP-FPM Pools Configuration

With PHP-FPM it’s possible to use different pools for different sites and allocate resources very accurately and even use different users and groups for every pool. Following is just example configuration files structure for PHP-FPM pools for three different sites (or actually three different part of same site):

/etc/php-fpm.d/site.conf
/etc/php-fpm.d/blog.conf
/etc/php-fpm.d/forums.conf

Just example configurations for every pool:
/etc/php-fpm.d/site.conf

[site]
listen = 127.0.0.1:9000
user = site
group = site
request_slowlog_timeout = 5s
slowlog = /var/log/php-fpm/slowlog-site.log
listen.allowed_clients = 127.0.0.1
pm = dynamic
pm.max_children = 5
pm.start_servers = 3
pm.min_spare_servers = 2
pm.max_spare_servers = 4
pm.max_requests = 200
listen.backlog = -1
pm.status_path = /status
request_terminate_timeout = 120s
rlimit_files = 131072
rlimit_core = unlimited
catch_workers_output = yes
env[HOSTNAME] = $HOSTNAME
env[TMP] = /tmp
env[TMPDIR] = /tmp
env[TEMP] = /tmp

/etc/php-fpm.d/blog.conf

[blog]
listen = 127.0.0.1:9001
user = blog
group = blog
request_slowlog_timeout = 5s
slowlog = /var/log/php-fpm/slowlog-blog.log
listen.allowed_clients = 127.0.0.1
pm = dynamic
pm.max_children = 4
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
pm.max_requests = 200
listen.backlog = -1
pm.status_path = /status
request_terminate_timeout = 120s
rlimit_files = 131072
rlimit_core = unlimited
catch_workers_output = yes
env[HOSTNAME] = $HOSTNAME
env[TMP] = /tmp
env[TMPDIR] = /tmp
env[TEMP] = /tmp

/etc/php-fpm.d/forums.conf

[forums]
listen = 127.0.0.1:9002
user = forums
group = forums
request_slowlog_timeout = 5s
slowlog = /var/log/php-fpm/slowlog-forums.log
listen.allowed_clients = 127.0.0.1
pm = dynamic
pm.max_children = 10
pm.start_servers = 3
pm.min_spare_servers = 2
pm.max_spare_servers = 4
pm.max_requests = 400
listen.backlog = -1
pm.status_path = /status
request_terminate_timeout = 120s
rlimit_files = 131072
rlimit_core = unlimited
catch_workers_output = yes
env[HOSTNAME] = $HOSTNAME
env[TMP] = /tmp
env[TMPDIR] = /tmp
env[TEMP] = /tmp

So this is just example howto configure multiple different size pools.

PHP-FPM Tip 4. – PHP-FPM Pool Process Manager (pm) Configuration

Best way to use PHP-FPM process manager is use dynamic process management, so PHP-FPM processes are started only when needed. This is almost same style setup than Nginx worker_processes and worker_connections setup. So very high values does not mean necessarily anything good. Every process eat memory and of course if site have very high traffic and server lot’s of memory then higher values are right choise, but servers, like VPS (Virtual Private Servers) memory is normally limited to 256 Mb, 512 Mb, 1024 Mb. This low RAM is enough to handle even very high traffic (even dozens of requests per second), if it’s used wisely.

It’s good to test how many PHP-FPM processes a server could handle easily, first start Nginx and PHP-FPM and load some PHP pages, preferably all of the heaviest pages. Then check memory usage per PHP-FPM process example with Linux top or htop command. Let’s assume that the server has 512 Mb memory and 220 Mb could be used for PHP-FPM, every process use 24 Mb RAM (some huge content management system with plugins can easily use 20-40 Mb / per PHP page request or even more). Then simply calculate the server max_children value:
220 / 24 = 9.17

So good pm.max_children value is 9. This is based just quick average and later this could be something else when you see longer time memory usage / per process. After quick testing it’s much easier to setup pm.start_serversvalue, pm.min_spare_servers value and pm.max_spare_servers value.

Final example configuration could be following:

pm.max_children = 9
pm.start_servers = 3
pm.min_spare_servers = 2
pm.max_spare_servers = 4
pm.max_requests = 200

Max request per process is unlimited by default, but it’s good to set some low value, like 200 and avoid some memory issues. This style setup could handle large amount of requests, even if the numbers seems to be small.

Install MySQL on Linux

MySQL is a relational database management system (RDBMS) that runs as a server providing multi-user access to a number of databases. This is guide, howto install or upgrade MySQL Community Server latest and greatest version 5.5.25 on Fedora 17/16/15/14/13/12, CentOS 6.2/6.1/6/5.8 and Red Hat (RHEL) 6.2/6.1/6/5.8.

Note: If you are upgrading MySQL (from earlier version), then make sure that you backup (dump and copy) your database and configs. And remember run mysql_upgradecommand.

Install MySQL Database 5.5.25 on Fedora 17/16/15/14/13/12, CentOS 6.2/6.1/6/5.8, Red Hat (RHEL) 6.2/6.1/6/5.8

1. Change root user

su -
## OR ##
sudo -i

2. Install Remi repository

Fedora

Currently extra repos are not needed on Fedora 16.

## Remi Dependency on Fedora 17, 16, 15
rpm -Uvh http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-stable.noarch.rpm 
rpm -Uvh http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-stable.noarch.rpm

## Fedora 17 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-17.rpm

## Fedora 16 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-16.rpm

## Fedora 15 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-15.rpm

## Fedora 14 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-14.rpm

## Fedora 13 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-13.rpm

## Fedora 12 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-12.rpm

CentOS and Red Hat (RHEL)

## Remi Dependency on CentOS 6 and Red Hat (RHEL) 6 ##
rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-7.noarch.rpm

## CentOS 6 and Red Hat (RHEL) 6 ##
rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-6.rpm

## Remi Dependency on CentOS 5 and Red Hat (RHEL) 5 ##
rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

## CentOS 5 and Red Hat (RHEL) 5 ## 
rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-5.rpm

3. Check Available MySQL versions

Fedora 17, 16, 15, 14, 13, 12

yum --enablerepo=remi list mysql mysql-server

CentOS 6.2/6.1/6/5.8 and Red Hat (RHEL) 6.2/6.1/6/5.8

yum --enablerepo=remi,remi-test list mysql mysql-server

Output:

Loaded plugins: changelog, fastestmirror, presto, refresh-packagekit
...
remi                                                            | 3.0 kB     00:00     
remi/primary_db                                                 | 106 kB     00:00     
Available Packages
mysql.i686                               5.5.25-1.fc14.remi                        @remi
mysql-server.i686                        5.5.25-1.fc14.remi                        @remi

4. Update or Install MySQL 5.5.25

Fedora 17, 16, 15, 14, 13, 12

yum --enablerepo=remi install mysql mysql-server

CentOS 6.2/6.1/6/5.8 and Red Hat (RHEL) 6.2/6.1/6/5.8

yum --enablerepo=remi,remi-test install mysql mysql-server

5. Start MySQL server and autostart MySQL on boot

Fedora 17/16

systemctl start mysqld.service ## use restart after update

systemctl enable mysqld.service

Fedora 15/14/13/12/11, CentOS 6.2/6.1/6/5.8 and Red Hat (RHEL) 6.2/6.1/6/5.8

/etc/init.d/mysqld start ## use restart after update
## OR ##
service mysqld start ## use restart after update

chkconfig --levels 235 mysqld on

6. MySQL Secure Installation

  • Set (Change) root password
  • Remove anonymous users
  • Disallow root login remotely
  • Remove test database and access to it
  • Reload privilege tables

Start MySQL Secure Installation with following command

/usr/bin/mysql_secure_installation

Output:

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
      SERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!

In order to log into MySQL to secure it, we\'ll need the current
password for the root user.  If you\'ve just installed MySQL, and
you haven\'t set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none): 
OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MySQL
root user without the proper authorisation.

Set root password? [Y/n] Y
New password: 
Re-enter new password: 
Password updated successfully!
Reloading privilege tables..
 ... Success!

By default, a MySQL installation has an anonymous user, allowing anyone
to log into MySQL without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] Y
 ... Success!

Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] Y
 ... Success!

By default, MySQL comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n] Y
 - Dropping test database...
 ... Success!
 - Removing privileges on test database...
 ... Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n] Y
 ... Success!

Cleaning up...

All done!  If you\'ve completed all of the above steps, your MySQL
installation should now be secure.

Thanks for using MySQL!

Note: If you don’t want some reason, do a “MySQL Secure Installation” then at least it’s very important to change the root user’s password

mysqladmin -u root password [your_password_here]

## Example ##
mysqladmin -u root password myownsecrectpass

7. Connect to MySQL database (localhost) with password

mysql -u root -p

## OR ##
mysql -h localhost -u root -p

8. Create Database, Create MySQL User and Enable Remote Connections to MySQL Database

This example uses following parameters:

  • DB_NAME = webdb
  • USER_NAME = webdb_user
  • REMOTE_IP = 10.0.15.25
  • PASSWORD = password123
  • PERMISSIONS = ALL
## CREATE DATABASE ##
mysql> CREATE DATABASE webdb;

## CREATE USER ##
mysql> CREATE USER 'webdb_user'@'10.0.15.25' IDENTIFIED BY 'password123';

## GRANT PERMISSIONS ##
mysql> GRANT ALL ON webdb.* TO webdb_user@'10.0.15.25';

##  FLUSH PRIVILEGES, Tell the server TO reload the GRANT TABLES  ##
mysql> FLUSH PRIVILEGES;

Enable Remote Connection to MySQL Server –> Open MySQL Port (3306) on Iptables Firewall (as root user again)

1. Edit /etc/sysconfig/iptables file:

nano -w /etc/sysconfig/iptables

2. Add following line before COMMIT:

-A INPUT -m state --state NEW -m tcp -p tcp --dport 3306 -j ACCEPT

3. Restart Iptables Firewall:

service iptables restart
## OR ##
/etc/init.d/iptables restart

4. Test remote connection:

mysql -h dbserver_name_or_ip_address -u webdb_user -p webdb

Install phpMyAdmin 3.5.1 on Fedora 17/16, CentOS/RHEL 6.2/5.8

2.1 Change root user

su -
## OR ##
sudo -i

2.2 Install Remi repository

Fedora

## Remi Dependency on Fedora
rpm -Uvh http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-stable.noarch.rpm 
rpm -Uvh http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-stable.noarch.rpm

## Fedora 17 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-17.rpm

## Fedora 16 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-16.rpm

## Fedora 15 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-15.rpm

## Fedora 14 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-14.rpm

## Fedora 13 ##
rpm -Uvh http://rpms.famillecollet.com/remi-release-13.rpm

CentOS and Red Hat (RHEL)

## Remi Dependency on CentOS 6 and Red Hat (RHEL) 6 ##
rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-7.noarch.rpm

## CentOS 6 and Red Hat (RHEL) 6 ##
rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-6.rpm

## Remi Dependency on CentOS 5 and Red Hat (RHEL) 5 ##
rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

## CentOS 5 and Red Hat (RHEL) 5 ## 
rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-5.rpm

Note: You can also use original Fedora phpMyAdmin builds or EPEL packages.

2.3 Install phpMyAdmin

yum --enablerepo=remi install phpmyadmin

3. Setup phpMyAdmin with Apache or Nginx

[Apache]

3.1a Check phpMyAdmin Apache Virtual Host

When you install phpMyAdmin you got working Apache Virtual Host automatically, but if you want change it, then check /etc/httpd/conf.d/phpMyAdmin.conf file and modify it.

[Nginx]

3.1b Create phpMyAdmin Nginx Virtual Host and Other Setups

Create /etc/nginx/sites-available/phpMyAdmin file with following content:

server {
       listen   80;
       server_name pma;
       access_log /var/log/nginx/phpmyadmin/access.log;
       error_log /var/log/nginx/phpmyadmin/error.log;
       root /usr/share/phpMyAdmin;

       location / {
           index  index.php;
       }

       ## Images and static content is treated different
       location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|xml)$ {
           access_log        off;
           expires           360d;
       }

       location ~ /\.ht {
           deny  all;
       }

       location ~ /(libraries|setup/frames|setup/libs) {
           deny all;
           return 404;
       }

       location ~ \.php$ {
           include /etc/nginx/fastcgi_params;
           fastcgi_pass 127.0.0.1:9000;
           fastcgi_index index.php;
           fastcgi_param SCRIPT_FILENAME /usr/share/phpMyAdmin$fastcgi_script_name;
       }
}

This is Nginx phpMyAdmin virtual host configuration which works pretty much like original phpMyAdmin Apache configuration. I use here server_name pma you could freely change this real host name or some other.

Set pma pointing to localhost, add following to /etc/hosts file:

127.0.0.1  pma

Create needed directory and enable Nginx virtual host:

mkdir -p /var/log/nginx/phpmyadmin

cd /etc/nginx/sites-enabled
ln -s /etc/nginx/sites-available/phpMyAdmin

[Apache]

3.2a Restart Apache Web Server

/etc/init.d/httpd restart

[Nginx]

3.2b Restart Nginx Web Server

/etc/init.d/nginx restart

4. Use phpMyAdmin

[Apache]

4.1a Open phpMyAdmin with Apache Setup

Open browser and Goto following address http://localhost/phpmyadmin.

[Nginx]

4.1b Open phpMyAdmin with Nginx Setup

Open browser and Goto following address http://pma/.

4.2 phpMyAdmin Login Screen

Login with your MySQL username and password.

[Apache]

4.3a phpMyAdmin 3.4.10 Running on CentOS 6.2 with Apache

[Nginx]

4.3b phpMyAdmin 3.4.10 Running on CentOS 6.2 with Nginx