This post originated from an RSS feed registered with Python Buzz by Carlos de la Guardia. Original Post: Let supervisor monitor your services Feed Title: I blog therefore I am Feed URL: http://blog.delaguardia.com.mx/feed.atom Feed Description: A space to put my thoughts into writing. Latest Python Buzz Posts Latest Python Buzz Posts by Carlos de la Guardia Latest Posts From I blog therefore I am

Supervisor is a Unix process monitoring system that can automatically start and monitor any number of services for you and restart them whenever they stop unexpectedly. It is written in Python, and is very extensible. It includes a command line client program and also a web interface for easy remote process management.

Supervisor can be extended directly with Python or using any other language via XML-RPC. Its event notification protocol makes programming new event listeners a very simple task.

I used Supervisor recently to set up a Plone installation and I heartily recommend it to anyone who wants to monitor server processes like those used by a Plone site.

My site was using two Zope instances with ZEO, the Pound load balancer and the Varnish cache.
Supervisor allows me to handle all of these via its remote web interface, from where I can start, stop or even check the log files of any process. Supervisor lets you assign a priority to every program that it controls, so that it is possible to start or stop all services with one command in a specific order. Here is a screen capture of the web interface:

Supervisor can be installed with easy_install, which makes it very convenient to use with any Python installation. Integrating it into a Plone site, where zc.buildout is the favorite deployment mechanism, is also very easy. Here's what I added to my buildout.cfg file to use it:

parts = supervisor

[supervisor]
recipe = repoze.recipe.egg
eggs = supervisor

In addition to that, supervisor needs a configuration file. I added an etc directory to my buildout with the following supervisord.conf file in it:

[inet_http_server]
port=127.0.0.1:9999

username=admin
password=admin

[supervisord]
logfile=%(here)s/../var/log/supervisord.log
logfile_maxbytes=50MB
logfile_backups=10 
loglevel=info
pidfile=%(here)s/../var/supervisord.pid
nodaemon=false

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=http://127.0.0.1:9999

[program:zeo]
command = %(here)s/../parts/zeoserver/bin/runzeo
priority = 10

[program:zope_instance1]
command = %(here)s/../parts/instance1/bin/runzope
priority = 20
redirect_stderr = true

[program:zope_instance2]
command = %(here)s/../parts/instance2/bin/runzope
priority = 20
redirect_stderr = true

[program:pound]
command = bin/pidproxy %(here)s/../parts/pound/var/pound.pid %(here)s/../bin/pound
priority = 40
redirect_stderr = true

[program:varnish]
command = %(here)s/../bin/varnish
priority = 60
redirect_stderr = true

Note that supervisor starts its subprocesses via fork/exec and thus they must not dameonize. That's why on the example config Zope and ZEO are started using runzope and runzeo rather than zopectl and zeoctl.

For programs which launch another process separately and control that via PID files, like MySQL or Pound in the example above, the pidproxy script can take care of this. Just give it the launch script and the PID file locations in the config and everything will work OK.

Supervisor is a great solution for process monitoring and this post just scratches the surface of what is possible with it. I hope I can post later about using its extension mechanism. In the meantime, I recommend you try it out or at least read the manual and see what it can do.