Graphite

By kire
0

Graphite is a highly scalable real-time graphing system. We use it to graph all sorts of metrics. Basically, you write an application that collects numeric time-series data that you are interested in graphing, and send it to Graphite’s processing backend, carbon, which stores the data in Graphite’s specialized database. The data can then be visualized through graphite’s web interfaces. This tutorial expects that you are using a RHEL/CentOS based server.

Requirements

    python2.4 or greater
    pycairo (with PNG backend support)
    mod_python
    django
    python-ldap (optional - needed for ldap-based webapp authentication)
    python-memcached (optional - needed for webapp caching, big performance boost)
    python-sqlite2 (optional - a django-supported database module is required)
    bitmap and bitmap-fonts required on some systems, notably Red Hat

Installation

Here I will go over all of the steps required to install the many components which make graphite work. Note that these instructions are intended for CentOS/RHEL based systems only.

Install EPEL

rpm -i http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm

Install rpms

yum -y install bitmap bitmap-fonts Django pycairo python-devel python-ldap python-memcached mod_wsgi python-sqlite2 glibc-devel gcc gcc-c++ git openssl-devel
python-zope-interface httpd memcached

Install twisted

cd /tmp
wget “http://twistedmatrix.com/Releases/Twisted/11.0/Twisted-11.0.0.tar.bz2″
tar -xjf Twisted-11.0.0.tar.bz2
cd Twisted-11.0.0
python setup.py install

Install graphite

cd /tmp
wget “http://launchpad.net/graphite/1.0/0.9.8/+download/graphite-web-0.9.8.tar.gz”
tar -zxf graphite-web-0.9.8.tar.gz
cd graphite-web-0.9.8
python setup.py install

Install whisper

cd /tmp
wget “http://launchpad.net/graphite/1.0/0.9.8/+download/whisper-0.9.8.tar.gz”
tar -zxf whisper-0.9.8.tar.gz
cd whisper-0.9.8
python setup.py install

Install carbon

cd /tmp
wget “http://launchpad.net/graphite/1.0/0.9.8/+download/carbon-0.9.8.tar.gz”
tar -zxf carbon-0.9.8.tar.gz
cd carbon-0.9.8
python setup.py install

Install Node-js

cd /tmp
git clone http://github.com/joyent/node.git
cd node
git checkout v0.4.9
./configure && make && make install

Install npm

cd /tmp
wget http://npmjs.org/install.sh
. ./install.sh
/usr/local/bin/npm install express

Install statsd

cd /tmp
git clone http://github.com/etsy/statsd.git
cp -R /tmp/statsd /opt/graphite

Configuration

Now, we have to configure apache to accept connections on port 8080 for graphite.

First, create /etc/httpd/conf.d/graphite.conf (just copy and paste all of this into a command line)

cat > /etc/httpd/conf.d/graphite.conf

Listen 8080
# You may need to manually edit this file to fit your needs.
# This configuration assumes the default installation prefix
# of /opt/graphite/, if you installed graphite somewhere else
# you will need to change all the occurances of /opt/graphite/
# in this file to your chosen install location.
<VirtualHost *:8080>
ServerName graphite

DocumentRoot “/opt/graphite/webapp”

# I’ve found that an equal number of processes & threads tends
# to show the best performance for Graphite (ymmv).
WSGIDaemonProcess graphite processes=5 threads=5 display-name=’%{GROUP}’ inactivity-timeout=120
WSGIProcessGroup graphite

# You will need to create this file! There is a graphite.wsgi.example
# file in this directory that you can safely use, just copy it to graphite.wgsi
WSGIScriptAlias / /opt/graphite/conf/graphite.wsgi

Alias /content/ /opt/graphite/webapp/content/
<Location “/content/”>
SetHandler None
</Location>

# NOTE: In order for the django admin site media to work you
# must change @DJANGO_ROOT@ to be the path to your django
# installation, which is probably something like:
# /usr/lib/python2.6/site-packages/django
Alias /media/ “/usr/lib/python2.4/site-packages/django/contrib/admin/media/”
<Location “/media/”>
SetHandler None
</Location>

# The graphite.wsgi file has to be accessible by apache. It won’t
# be visible to clients because of the DocumentRoot though.
<Directory /opt/graphite/conf/>
Order deny,allow
Allow from all
</Directory>

</VirtualHost>

Lastly, uncomment the following line in /etc/httpd/conf.d/wsgi.conf:

LoadModule wsgi_module modules/mod_wsgi.so
WSGISocketPrefix /var/run/wsgi

configure statsd

You need to create /opt/graphite/statsd/local.js, again you can paste this into a terminal to do so:

cat > /opt/graphite/statsd/local.js
{
graphitePort: 2003
, graphiteHost: “localhost”
, port: 8125
}

copy example configs

cd /opt/graphite/conf/
for i in graphite.wsgi carbon.conf storage-schemas.conf ; do cp $i.example $i ; done
cp /opt/graphite/webapp/graphite/local_settings.py.example /opt/graphite/webapp/graphite/local_settings.py

local_settings.py

Update local_settings.py with correct values for memcache timezone and DEBUG

TIME_ZONE = ‘America/Los_Angeles’
MEMCACHE_HOSTS = ['127.0.0.1:11211']
DEBUG = True

create database

python /opt/graphite/webapp/graphite/manage.py syncdb
sudo chown -R apache:apache /opt/graphite/storage/

Starting Services


start apache

/etc/init.d/httpd start

start memcached

/etc/init.d/memcached start

Start carbon

/usr/bin/python /opt/graphite/bin/carbon-cache.py start

start nodejs/statsd

cd /opt/graphite/statsd
nohup /usr/local/bin/node stats.js local.js &