Django: Cache any function

I wrote this decorator in Python a few days ago, posted it at djangosnippets.org, and just updated it to add better support for threading.

Basically, it’s the same as [font=monospace]cache_page[/font], except it can be applied to any function. I use it to periodically check on my del.icio.us bookmarks, and I plan on doing the same with my Flickr photos also.

def cache_function(length):
    """
    Caches a function, using the function itself as the key, and the return
    value as the value saved. It passes all arguments on to the function, as
    it should.
    
    The decorator itself takes a length argument, which is the number of
    seconds the cache will keep the result around.
    
    It will put in a MethodNotFinishedError in the cache while the function is
    processing. This should not matter in most cases, but if the app is using
    threads, you won't be able to get the previous value, and will need to
    wait until the function finishes. If this is not desired behavior, you can
    remove the first two lines after the ``else``.
    """
    def decorator(func):
        def inner_func(*args, **kwargs):
            from django.core.cache import cache
            
            value = cache.get(func)
            if cache.has_key(func):
                return value
            else:
                # This will set a temporary value while ``func`` is being
                # processed. When using threads, this is vital, as otherwise
                # the function can be called several times before it finishes
                # and is put into the cache.
                class MethodNotFinishedError(Exception): pass
                cache.set(func, MethodNotFinishedError(
                    'The function %s has not finished processing yet. This \
value will be replaced when it finishes.' % (func.__name__)
                ), length)
                result = func(*args, **kwargs)
                cache.set(func, result, length)
                return result
        return inner_func
    return decorator

The syntax is the exact same as [font=monospace]cache_page[/font], except it can be used by any function that is called by a view.

I also used Django’s signals to listen for when a page was requested, and I start a new update thread at the beginning of the request. The page will be returned in the original thread, while the del.icio.us/Flickr threads will either die because they’re still cached, or update the database with information from the APIs, taking as long as they need (up to a limit, of course).

The threading source is based on this great page: http://www.imalm.com/blog/2007/feb/10/using-django-signals-ping-sites-update/

import threading
import time

from django.core.signals import request_started
from django.dispatch import dispatcher

from nokrev.blog import api_calls

class APIThread(threading.Thread):
    """
    This thread is used to call APIs from api_calls.py. It listens to page
    signals, to be notified and called.
    """
    def __init__(self, api_call):
        threading.Thread.__init__(self)
        self.api_call = api_call
    
    def run(self):
            self.api_call()

def start_api_thread(sender, signal):
    APIThread(api_calls.delicious).start()
    # APIThread(flickr).start()

dispatcher.connect(start_api_thread, signal=request_started)

Hope you like it. :smiley:

Signals are pretty neat, in Django.