afurlan's blog

/^random (nerd|geek)? posts$/

tag: seo

Creating the urllist.txt with your sitemaps configuration

Tuesday, December 1, 2009 - Two comments

In Django, you can configure the sitemap of your website using the sitemaps framework and then use this configuration to generate the sitemap.xml. Notwithstanding, the Yahoo! used to support only the urllist.txt type of sitemap and, because of that, I still use to have both (urllist.txt and sitemap.xml) available on my websites.

But, once you have your sitemap configured, why not use it to generate the urllist.txt? I created a view which generates the urllist.txt based on your sitemap configuration, as follows:

# -*- coding: utf-8 -*-

from django.conf import settings
from django.http import HttpResponse
from django.contrib.sites.models import Site

def urllist_from_sitemaps(request, sitemaps):
        urllist = []
        protocol = getattr(settings, 'PROTOCOL', 'http')
        baseurl = u'%s://%s' % (protocol, Site.objects.get_current().domain)
        for cls in sitemaps.values():
                instance = object.__new__(cls)
                for item in instance.items():
                        urllist.append(baseurl + item.get_absolute_url())
        return HttpResponse(u'\n'.join(urllist), mimetype='text/plain')

And now, you just need to configure the urls.py and add the urllist.txt entry:

sitemaps = {
    'entries': EntrySitemap,
    'tags': TagSitemap,
    'archive': ArchiveSitemap,
}

urlpatterns = patterns('',
    ...
    (r'^sitemap.xml$', 'django.contrib.sitemaps.views.sitemap', {'sitemaps': sitemaps}),
    (r'^urllist.txt$', 'myproj.apps.myapp.views.urllist_from_sitemaps', {'sitemaps': sitemaps}),
    ...
)

And I think that is all for now...

Update 1: I submitted a patch to be added into the official Sitemaps framework of the Django. The patch is very different of the code in this post and, if you'd like to see it and its code, here it is.

As always: if you found some english bug, warn me and I'll be glad to fix it. :)