afurlan's blog

In Django, you can configure the sitemap of your website using the sitemaps framework and then use this configuration to generate the sitemap.xml. Notwithstanding, the Yahoo! used to support only the urllist.txt type of sitemap and, because of that, I still use to have both (urllist.txt and sitemap.xml) available on my websites.

But, once you have your sitemap configured, why not use it to generate the urllist.txt? I created a view which generates the urllist.txt based on your sitemap configuration, as follows:

# -*- coding: utf-8 -*-

from django.conf import settings
from django.http import HttpResponse
from django.contrib.sites.models import Site

def urllist_from_sitemaps(request, sitemaps):
        urllist = []
        protocol = getattr(settings, 'PROTOCOL', 'http')
        baseurl = u'%s://%s' % (protocol, Site.objects.get_current().domain)
        for cls in sitemaps.values():
                instance = object.__new__(cls)
                for item in instance.items():
                        urllist.append(baseurl + item.get_absolute_url())
        return HttpResponse(u'\n'.join(urllist), mimetype='text/plain')

And now, you just need to configure the urls.py and add the urllist.txt entry:

sitemaps = {
    'entries': EntrySitemap,
    'tags': TagSitemap,
    'archive': ArchiveSitemap,
}

urlpatterns = patterns('',
    ...
    (r'^sitemap.xml$', 'django.contrib.sitemaps.views.sitemap', {'sitemaps': sitemaps}),
    (r'^urllist.txt$', 'myproj.apps.myapp.views.urllist_from_sitemaps', {'sitemaps': sitemaps}),
    ...
)

And I think that is all for now...

Update 1: I submitted a patch to be added into the official Sitemaps framework of the Django. The patch is very different of the code in this post and, if you'd like to see it and its code, here it is.

As always: if you found some english bug, warn me and I'll be glad to fix it. :)

Some time ago I changed my default shell to use IPython instead of Bash. The IPython is a really great shell and IMHO has great advantages but for those who, like me, are completely addicted to the clear command, the IPython's clear is a huge disadvantage.

In IPython, the clear command clear varios data like input, output and directory histories but it doesn't clear the screen! You can take a look at the documentation of default function for the clear command below and see how it works:

afurlan@merlin:~$ from IPython.Extensions.clearcmd import clear_f
afurlan@merlin:~$ print clear_f.__doc__
 Clear various data (e.g. stored history data)

    %clear out - clear output history
    %clear in  - clear input history
    %clear shadow_compress - Compresses shadow history (to speed up ipython)
    %clear shadow_nuke - permanently erase all entries in shadow history
    %clear dhist - clear dir history

afurlan@merlin:~$

So I created a new function to extend the first one adding the option to clear the screen when no args were passed. If you'd like to do the same, add the following lines in your ~/.ipython/ipy_user_conf.py:

def new_clear_f(*args, **kwargs):
    ''' Extend the default clear function adding the option to
    clear the screen when no args were passed.'''
    if not args[1]:
            ip.system('clear')
    else:
            from IPython.Extensions.clearcmd import clear_f
            clear_f(*args, **kwargs)
            ip.expose_magic('clear', new_clear_f)
ip.expose_magic('clear', new_clear_f)

And now it all seems clear again. :)

As always: if you found some english bug, warn me and I'll be glad to fix it. :)

Some days ago Google released Go, its own open source programming language. Go is intended to be a simple, fast and safe programming language that promotes the users to write programs through concurrent and communicating lightweight processes. If want to know more about how to install and how to use Go, please take a look at its official site: http://golang.org.

Today I installed Go to play with it a little bit and (as I use to), after the world famous "hello world" code, I wrote a program to generate the fibonacci's sequence. Actually I wrote two versions of the same program, the first one is the most common fibonacci code:

package main

import "os"
import "fmt"
import "strconv"

func fib1(n int) int {
    if n < 0 { return 0; }
    if n < 2 { return n; }
    return fib1(n-2) + fib1(n-1);
}

func main() {
    if len(os.Args) < 2 {
            fmt.Printf("Usage: %s NUM\n", os.Args[0]);
            os.Exit(1);
    }
    n, _ := strconv.Atoi(os.Args[1]);
    fmt.Printf("%d\n", fib1(n));
}

And the second version is basically the same code but keeping a history of the known results:

package main

import "os"
import "fmt"
import "strconv"

func fib2(n int, hist *map[int]int) int {
    if n < 0 { return 0; }
    if n < 2 { return n; }
    _, found := (*hist)[n];
    if !found {
            (*hist)[n] = fib2(n-2, hist) + fib2(n-1, hist)
    }
    return (*hist)[n];
}

func main() {
    if len(os.Args) < 2 {
            fmt.Printf("Usage: %s NUM\n", os.Args[0]);
            os.Exit(1);
    }
    n, _ := strconv.Atoi(os.Args[1]);
    hist := make(map[int]int);
    fmt.Printf("%d\n", fib2(n, &hist));
}

Once you have finished these codes, it's always nice to see how different their execution time are:

afurlan@merlin:~$ time ./fib1 45
1134903170

real    0m43.979s
user    0m43.887s
sys     0m0.008s
afurlan@merlin:~$ time ./fib2 45
1134903170

real    0m0.004s
user    0m0.004s
sys     0m0.000s

Go is a very nice language, let's go to Go. :)

As always: if you found some english bug, warn me and I'll be glad to fix it. :)

If you are a shell addicted person and like Python, then the IPython was made just for you! The IPython provides a very usefull feature (that may looks a bit strange at first) of mixing Python and Shell commands in a easy and intuitive way. I don't pretend to say why you should use IPython instead of your current shell but I think that, if you know Python, you should prefer to write Python code instead of Bash code.

I'll assume that you already have the IPython installed but if you don't, take a look at IPython's documentation to see how to install it. I'm running Debian and my setup is focused in my machine, so it should work for the most of the others linux distros but if you are running a different system, I'm sorry for you you may have to adapt some commands and/or paths. :)

Configuring the IPython's prompt

The first time you run IPython, it displays a warning about the creation of your personal configuration directory. One of the most important parts of this warning is about the new configuration file: ~/.ipython/ipy_user_conf.py. So let's configure this file and the IPython's prompt to make it looks exactly like the Debian's prompt. Edit your configuration file and change the following settings as below:

...
import ipy_profile_sh
...
o.prompt_in1 = r'\C_Normal\u@\H:\Y2$ '
o.prompt_in2 = r'\C_Normal... '
...

The first option, import ipy_profile_sh, avoid you to have to escape all the Bash commands. The second and third options, o.prompt_in1 and o.prompt_in2, change the IPython's prompt. And now start the IPython as follow:

afurlan@merlin:~$ ipython -nosep -nobanner -noconfirm_exit
afurlan@merlin:~$
afurlan@merlin:~$ 
afurlan@merlin:~$ print 'hello python!'
hello python!
afurlan@merlin:~$ echo 'hello bash!'
hello bash!

Now you're able to run Python commands:

afurlan@merlin:~$ for line in open('domains.txt'):
              ...     print line
              ...    
              ...    
afurlan.org

blog.afurlan.org

And a mixed of Python and Bash commands:

afurlan@merlin:~$ for line in open('domains.txt'):
              ...     echo "$line"
              ...    
              ...    
afurlan.org

blog.afurlan.org

afurlan@merlin:~$ for line in open('domains.txt'):
              ...     echo "$line.strip()"
              ...    
              ...    
afurlan.org
blog.afurlan.org

Setting the IPython as your default shell

Actually I don't like to set the IPython as my default shell... I use to run screen so I configure it to open five IPython sessions by default but if you really like to set it as your default shell, you can add the following line at the end of your ~/.bashrc file:

...
exec ipython -nosep -nobanner -noconfirm_exit

If you're a screen user too, you can add the following lines at your ~/.screenrc:

...
# open 5 ipython sessions by default
screen -t p 1 ipython -nosep -nobanner -noconfirm_exit
screen -t p 2 ipython -nosep -nobanner -noconfirm_exit
screen -t p 3 ipython -nosep -nobanner -noconfirm_exit
screen -t p 4 ipython -nosep -nobanner -noconfirm_exit
screen -t p 5 ipython -nosep -nobanner -noconfirm_exit
...

Or, if you like, you can also get my current .screenrc file.

And, as always: if you found some english bug, warn me and I'll be glad to fix it. :)

Yep, I know, in the previous post I said that I'd try to write more often and in small chunks but looks like it's not working...

By the way, today I needed to find out the origin country of an hostname (actually a list of hostnames). As Debian has a GeoIP database available via aptitude, I could accomplish that using this database and the Python's GeoIP module.

First of all I installed the necessary packages

merlin:~# aptitude install geoip-database python-geoip

And then I wrote the following small script that configure the Python's GeoIP module to use the Debian's GeoIP database:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import re
import sys
import GeoIP

# oops, check the usage! :)
if len(sys.argv) != 2:
        print 'Usage: %s ADDRESS' % sys.argv[0]
        sys.exit(1)

# geoip database from "geoip-database" debian package
GEOIP_DATABASE = '/usr/share/GeoIP/GeoIP.dat'
geoip = GeoIP.open(GEOIP_DATABASE, GeoIP.GEOIP_STANDARD)

# get the country name based in the given input
if re.match('^[0-9]{1,3}(\.[0-9]{1,3}){3}$', sys.argv[1]):
        response = geoip.country_name_by_addr(sys.argv[1])
else:
        response = geoip.country_name_by_name(sys.argv[1])

# print the response if there is any response
if response:
        print response

Some tests to check if it is really working... and it looks like it is. :)

afurlan@merlin:~$ ./hostname_country.py afurlan.org
United States
afurlan@merlin:~$ ./hostname_country.py mandriva.com
France
afurlan@merlin:~$ ./hostname_country.py debian.com
Netherlands
afurlan@merlin:~$ ./hostname_country.py mps.com.br
Brazil
afurlan@merlin:~$ ./hostname_country.py some-nonexistent-domain.org
afurlan@merlin:~$

I think that's all...

Update: I changed the code based on the Bogdano's comment, thanks Bogdano. :)

And, please, never forget: if you found some english bug, warn me and I'll be glad to fix it. :)