Python generator to list decorator

Python generators can be evil because of the implicit state they create. How many time have you done this:

items = get_items()
print list(items)
f(items, x)

It might prevent confusion if generators are only used when one needs lazy iteration (e.g when reading from files of unbounded size)

However, avoiding yield can be work: creating a list, appending to it several times, and then returning it is an effort. This is probably the reason that I use generators when I don’t need to.

One workaround is to create a python generator-to-list decorator like so:

def listify(gen):
    "Convert a generator into a function which returns a list"
    def patched(*args, **kwargs):
        return list(gen(*args, **kwargs))
    return patched

@listify
def f(x):
     for i in range(x):
        yield "item" + str(i)

assert f(5) == "item0 item1 item2 item3 item4".split()

One can also define a ‘dictate’ decorator as follows:

from functools import wraps

def dictate(func):
    @wraps(func)
    def patched(*args, **kwargs):
        return dict(*func(*args, **kwargs))
    return patched

This turns a generator that yields pairs (a_i, b_i) into the dictionary {a_i: b_i}.

Automatically populating a gcal event with reminders

I find it irritating to have to set reminders for google events by hand, so in the theme of efficient-over-effective here is a the urls.py of a django project which will create a new event in google calendar with several reminders and redirect you to a page for editing its details. If you have a virtual server you could host this project there. You can then bookmark this url on the machines you use.

A little (if any) adaptation would make this work on google app engine (and might even allow you to login without hard coding a password).

Alternatives would be to write a greasemonkey script for google reader or write a ubiquity command that does this and redirects you to the page – but both these things seem more difficult to debug.

from django.conf.urls.defaults import patterns
from django.http import HttpResponseRedirect

import atom
import datetime
from gdata.calendar import CalendarEventEntry, When, Reminder
from gdata.calendar.service import CalendarService

def create_reminder(req):
    S = CalendarService()
    S.email = "EMAIL@ADDRESS"
    S.password = "password1"
    S.ProgrammaticLogin()

    event = CalendarEventEntry()

  
starttime is manditory - tomorrow seems as good a day as any


    def make_google_dt(dt):
      return dt.strftime('%Y-%m-%dT%H:%M:%S.000Z')

    now = datetime.datetime.now()
  
    start_datetime = now + datetime.timedelta(days=1)
    start_time = make_google_dt(start_datetime)
    end_time = make_google_dt(start_datetime + datetime.timedelta(minutes=30))

    occurence = When(start_time=start_time, end_time=end_time)
    occurence.reminder = [
        Reminder(hours=3, extension_attributes={'method':'sms'}),
        Reminder(days=2, extension_attributes={'method':'email'}), 

  
to feel happily surprised but not panic


        Reminder(days=7, extension_attributes={'method':'email'}),
        Reminder(days=14, extension_attributes={'method':'email'}),
        ]
    event.when.append(occurence)

    new_event = S.InsertEvent(event, '/calendar/feeds/default/private/full')
    edit_link = new_event.GetHtmlLink().href
    return HttpResponseRedirect(edit_link)

urlpatterns = patterns('',
    (r'', create_reminder)
)

Note that this is a slighlty evil – insofar as a GET request is having side effects… but POST commands are slightly troublesome to book mark, and adding a layer of indirection slightly defeats the purpose.

Technical notes

• Underneath the python library this is constructing and xml command and sending this to a google webservice.
• The python library does really support email reminders. It does however support adding arbitrary attributes to the xml element representing the reminder:

  Reminder(days=7, extension_attributes={'method':'email'})
  

  this add method=”email” to the reminder tag. (One can also set method=sms).

  This is not documented in the google python documentation. It is (I think) documented in the gdata plain xml documentation.

• For a working reference implementation in javascript using plain xml see the gtd tickler greasemonkey gmail plugin.

Useful references

Getting python argspecs

For the benefit of google:

def f(x, *args, **kwargs):
    return x
import inspect
print inspect.getargspec(f)

Human string comparison in python

NOTE: An alternative way to do this

I’ve found a possible alternative to this. If you only want to sort strings in your current locale you can use locale.strcoll, which is a cmp function in your current locale.

import locale
["one", "two", "three"].sort(locale.strcoll)
l

On my machine this still isn’t a very human friendly sort but depending on your LC_COLLATE environment variable it may be. This is very much less of a reinvent-the-wheel-because-buying-one-is-too-hard approach – though I suspect it might take one 4 times as long to get working…

Of course at times what you want your commadline to do isn’t what you want all other programs to do…

---

The string comparison in python does not order lists as one would expect:

>>> l.sort(weird_strcmp)
>>> l
['hello', '?', '}']
>>> l = ["1", "Hello", "abc", "?", "}"]
>>> l.sort()
>>> l
['1', '?', 'Hello', 'abc', '}']
>>> 

python sorts strings as sequences of bytes.

I couldn’t find any easy way to sort strings in a more friendly fashion (Though surely one must exist?!) So here is an quick implementation of one (use this code as you wish):

from itertools import chain

letters = list(chain(*zip(range(0x41, 0x41 + 26), 
               range(0x61, 0x61 + 26)))) # interspersed upper and lower

numbers = range(0x30, 0x30 + 10)

symbols = sorted(list(
        set(range(0x80)) - set(letters) - set(numbers)))

weird_order = list(chain(letters, numbers, symbols))

assert set(weird_order) == set(range(0x80)), "Didn't get every character"

def weird_strcmp(a, b):
    """Compare strings for lists for English humans who hate 
         unicode and aren't using python 3k any time soon."""

    a = map(ord, a)
    b = map(ord, b)

    for x in chain(a, b):
        assert x < 0x80, "%s is not in the ascii character range" % x
    
    a = map(weird_order.index, a)
    b = map(weird_order.index, b)
    return cmp(a, b)

>>> l.sort(weird_strcmp)
>>> l
['abc', 'Hello', '1', '?', '}']

Be aware that this isn’t really tested and may not order symbols how you think they should be. It, however, has the advantage of not taking you 15-30 minutes to write and debug. Say if you can think of any foibles and I’ll fix them.

Debugging Wifi on Ubuntu Linux

To actually get ubuntu to give you some logging when connecting to wifi (you know so that you can act according to some information rather than at random):

1. Become superuser
2. Run killall NetworkManager
3. Run NetworkManager –no-daemon

This will then give you output for every stage of the connection process.

Network manager is (I believe) the tool that handles the overall process of opening a wifi connection using different processes (iwlist, iwconfig, wpa_supplicant etc) to do the actual connection.

Notes

Messages of the form “supplication connection state change number -> number” are (i think) from wpa_supplicant. Someone who isn’t me should change NetworkManager so that these are written in english. For now note that the states mean the following:

0 – WPA is disconnected
1 – WPA is inactive (no enabled connections and wpa isn’t trying to connect
2 – WPA is scanning
3 – WPA is associating (a loose-form of connection)
4 – WPA is associated
5 – WPA 4-way hand shake
6 – WPA group handshake
7 – WPA completed

I was seeing a lot of 3 -> 0 state changes when I was debugging. [This was taken from the wpa_supplicant source code in src/common/defs.h]