ll.sisyphus

Writing cron jobs with Python

Home › Python software › ll.sisyphusText · XIST · Python
Python softwarelist of projects

ll.sisyphus simplifies running Python stuff as cron jobs.

There will be no more than one sisyphus job of a certain name running at every given time. When the job is already running and a second is started, the second one will quit immediately if the first one hasn't exceeded its maximum allowed lifetime yet. If it has exceeded the allowed lifetime the first job will be killed and the second will start running.

In addition to that, logging features are provided.

To use this module, you must derive your own class from Job and implement the execute method.

The job announces its presence (and its process id) in a file that is stored in the ~/run directory. Logs will be created in the ~/log directory (This can be changes by deriving new subclasses and overwriting the appropriate class attribute).

There are three log files:

~/log/jobname_progress.log

The progress of one job execution is logged here. This log file will be truncated at the start of every execution, so you can be rather verbose. Use the logProgress method for writing to this log file.

~/log/jobname_loop.log

One log line may be appended to the log after every job execution. Call the method logLoop once at the end of execute for this.

~/log/jobname_error.log

Here exceptions happening during the execution of a job will be logged. This is done via the logError method with can be used for reporting other exception conditions apart from exceptions.

To execute a job, use the module level function execute.

Example

The following example illustrates the use of this module:

#!/usr/bin/env python

import os
import urllib
from ll import sisyphus

class Fetch(sisyphus.Job):
        "savely fetches an HTML file and saves it to a local file."

        def __init__(self):
                sisyphus.Job.__init__(self, 180, name="Fetch")
                self.url = "http://www.python.org/"
                self.tmpname = "Fetch_Tmp_%d.html" % os.getpid()
                self.officialname = "Python.html"

        def execute(self):
                self.logProgress("fetching data from %r" % self.url)
                data = urllib.urlopen(self.url).read()
                datasize = len(data)
                self.logProgress("writing file %r (%d bytes)" % (self.tmpname, datasize))
                open(self.tmpname, "wb").write(data)
                self.logProgress("renaming file %r to %r" % (self.tmpname, self.officialname))
                os.rename(self.tmpname, self.officialname)
                self.logLoop("cached %r as %r (%d bytes)" % (self.url, self.officialname, datasize))

if __name__=="__main__":
        sisyphus.execute(Fetch())

def _formattime​(timestamp):

Format timestamp into a string.

def _formattimedelta​(timedelta):

Format timedelta into a string.

class LogFile:

A log file. All lines written to the file will be prepended with a time stamp.

def __init__​(self, name, mode='a', buffering=True, encoding='iso-8859-1'):

Create a new log file (which will be opened on the first write). Arguments are:

name

The filename (either as a string or a ll.url.URL instance).

mode

The mode for opening the file (should be "w" or "a").

buffering

The buffering for the file (0 is unbuffered, 1 is line buffered, any other integer specifies the buffersize).

encoding

The encoding to use for the strings written to the file.

def __open​(self):

def write​(self, *texts):

Write texts to the log file.

class Job​(object):

A Job object executes a task once.

The job announces its presence (and its process id) in a file that is stored in the ~/run directory. Logs will be created in the ~/log directory (This can be changes by deriving new subclasses).

To use this class, derive your own class from it and overwrite the execute method.

def __init__​(self, maxruntime=0, name=None, raiseerrors=False, printkills=False):

Create a new job. Arguments are:

maxruntime: (integer)

The maximum allowed runtime in seconds for this job;

name: (string or None)

The name to be used for the log files. If None, the name of the class will be used;

raiseerrors: (bool)

should exceptions that occur during the excution of the job be raised (which results in a output to the terminal, or an email from the cron daemon);

printkills: (bool)

should the fact that a previous job was killed, be printed on stdout (resulting in a mail from the cron daemon)

def __writepid​(self):

Create the file containing the pid of the current process.

def __killpid​(self):

Delete the pid file.

def logLoop​(self, *texts):

Log the message texts to the loop and progress log. (The call to logLoop should be the last statement in the execute method.)

def logProgress​(self, *texts):

Log the message texts to the progress log.

def logErrorOnly​(self, *texts):

Log the error to the error log. texts may be strings or exception objects.

def logError​(self, error):

Log the error to the error log and the progress log. texts may be strings or exception objects.

def execute​(self):

Execute the job once. At the end of the job you should write one line to the loop log. Overwrite in subclasses.

def failed​(self):

Called when running the job generated an exception. Overwrite in subclasses, to e.g. rollback your database transactions.

def handleexecution​(self):

Handle executing the job including handling of duplicate or hanging jobs. This is the method to be called from the outside world.

def execute​(*jobs):

Execute several jobs.

Items in jobs are job objects, that will be executed sequentially.