AIPY SIG 05/03/2009: Statistics Versus Logic

On the Python Artificial Intelligence SIG, the topic of the week was statistics versus logic.

Graham Higgins dropped quite a few links:

[1] http://languagelog.ldc.upenn.edu/nll/?p=1374
[2] http://en.wikipedia.org/wiki/Conditional_entropy
[3] http://www.safarmer.com/fsw2.pdf
[4] http://www.cs.washington.edu/homes/rao/ScienceIndus.pdf
[5] http://homepages.inf.ed.ac.uk/s0450736/maxent_toolkit.html
[6] http://r.research.att.com/tools/
[7] http://www.amazon.com/Gods-Demons-Symbols-Ancient-Mesopotamia/dp/0292...
[8] http://www.prehistory.it/ftp/winn.htm
[9] http://www.flonnet.com/fl1801/18010730.htm
[10] http://blog.wired.com/wiredscience/2009/04/indusscript.html
[11] http://www.newscientist.com/article/dn17012-scholars-at-odds-over-mysterious-indus-script.html
[12] http://www.acl-ijcnlp-2009.org/main/acceptedfullpapers.html

Rick Thomas responded back with:

http://code.google.com/p/pebl-project/
http://code.google.com/p/python-dlp/
http://www.betaversion.org/~stefano/linotype/news/275/
http://ite.gmu.edu/~klaskey/papers/Costa_Laskey_MEBN_wo_Tears.pdf

AIPY Mailing List: http://groups.google.com/group/aipy?hl=en

Python Functional Programming Antipatterns: When Closures Can Be A Solution In Search of A Problem (PART 1)

One of the things I don't like about closures [1](via nested functions) is how they obscure intent in code. For example, if you just want to retain state why use a closure if you could just use a class? Sure a closure sounds cooler, but a class or a regular group of functions is often more flexible and readable than a closure.


Example 1: Simple Persistent State
Closure That Stores State: (1A)

In [39]: def outer():
   ....:     x = 1
   ....:     def inner():
   ....:         return x
   ....:     return inner
   ....: 

In [40]: func = outer()

In [41]: func
Out[41]: 

In [42]: func()
Out[42]: 1

Class That Stores State (1B)

In [46]: class State(object):
   ....:     def __init__(self):
   ....:         self.x = 1
   ....:     def func(self):
   ....:         return self.x
   ....:     
   ....:     

In [47]: func = State().func

In [48]: func()
Out[48]: 1

Score: +1 Class
Summary: If you simply want to persistent state, why use a closure that
has an odd signature, when you can simple use a class?


Example 2: Flat is better than nested, closures taken to an extreme with multiple nesting

Nested function w/ function that operates on state (2A)





In [18]: def way_outer():
   ....:     wo = 1
   ....:     def inner():
   ....:         i = 2
   ....:         def way_inner():
   ....:             return i + wo
   ....:         return way_inner
   ....:     return inner
   ....: 

In [19]: out = way
way_inner  way_outer  

In [19]: out = way_outer()

In [20]: out
Out[20]: 

In [21]: out()
Out[21]: 

In [22]: way_way_out = out()

In [23]: way_way_out()
Out[23]: 3

Class functions that operate on state (2B)




In [24]: class Addition(object):
   ....:     def __init__(self):
   ....:         self.x = 1
   ....:         self.y = 2
   ....:     def add(self):
   ....:         return self.x + self.y
   ....:     
   ....:     

In [25]: a = Addition().add

In [26]: a
Out[26]: [bound method Addition.add of __main__.Addition object at 0x78db70]

In [27]: a()
Out[27]: 3

Score: +2 Class
Summary: You might be thinking "duh" with this example, but yes, that is exactly the point! Why nest things if you don't have to? Especially because of the fact that if someone looks at your highly nested closure they will go WTF as they should. Why make things more complex then they need to be? Just because you can use closures doesn't mean you should or it makes your code intuitive and readable.

Example 3: Delaying Execution of a Function
Delayed execution of a function using closures (3A)

In [29]: def outer():
   ....:     def inner(x):
   ....:         return x + 1
   ....:     return inner
   ....: 

In [30]: func = outer()

In [31]: func(3)
Out[31]: 4

Delayed execution of a function using class: Example (3B)

In [1]: class DelayCall(object):
   ...:     def __init__(self, x):
   ...:         self.x = x
   ...:     def logic(self):
   ...:         return self.x + 1
   ...:     def delay(self):
   ...:         return self.logic
   ...:     
   ...:     

In [3]: d = DelayCall(3)

In [4]: func = d.delay()

In [5]: func()
Out[5]: 4

Delayed execution of a function that takes args using a lambda in a class: Example (3C)

In [19]: class LambdaDelayCall(object):
   ....:     def logic(self, x):
   ....:         return x + 1
   ....:     def delay(self):
   ....:         return (lambda x: self.logic(x))
   ....:     
   ....:     

In [20]: l = LambdaDelayCall()

In [21]: func = l.delay()

In [22]: func(5)
Out[22]: 6

Delayed execution of a function that takes args using a lambda in a class w/ __call__: Example (3D)

n [19]: class LambdaDelayCall(object):
   ....:     def logic(self, x):
   ....:         return x + 1
   ....:     def __call__(self):
   ....:         return (lambda x: self.logic(x))
   ....:     
   ....:     

In [37]: l = LambdaDelayCall()

In [40]: func = l()

In [41]: func(3)
Out[41]: 4

Score: +3 Class
Summary: While 3A is shorter, I think it is much less clear then 3B, 3C or 3D, the class examples. I think of a closure in terms of an outer function that needs to operate on an inner function, such as in the case of a decorator. Using a closure just to delay execution of a function doesn't seem right to me, and I feel like it obscures the code.

Conclusion:

Closures are useful when you want to modify the state of inner function and return, as in the case of decorators. Lambdas, from Learning Python 3rd edition, " are often used as a way to inline a function definition, or to defer execution of a piece of code.". While closures are useful and powerful, with power comes responsibility. [1] Nested functions aren't that clear, try a class or maybe you don't even need a closure?. Make sure you Python for good, not evil. Stay tuned for the next installment.

[1] Added note about "nested functions" 04/29/2009


References:
Bruce Eckel on Decorators
Zen of Python
Functools Partial
Dive Into Python: Lambda

Python Artificial Intelligence SIG Weekly Update: 04/26/2009

I am going to attempt to do weekly updates based on the newly formed Python AI SIG, http://groups.google.com/group/aipy . Sunday, New Zealand time, is going to be my day to summarize what is going on.

So far the hot topics are as follows:

1. Getting a code repository setup somewhere to share ideas.
2. Getting some domain setup so we can share and categorize what we learn. Jeff Rush mentioned possibly ai.python.org and using Sphinx. I kind of like that idea.
3. Filtered RSS reader: It seems like getting a bot to pre-filter RSS is low hanging fruit.
4. Continuous monitoring of the body: We are discussing the feasibility of continuous monitoring of the body, one gotcha so far is what device do we use?

An "Adaptable" Commandline Tool and Web Service Generator

I have been hacking on a Commandline tool and Web Service Generation framework called Adapt. Last night, Adam Shand, and I hacked on getting Phase 2 together, which is making a web service automatically serve out the same URLs as a commandline tool options, which are defined in a config file.

The not even Alpha quality code is here: http://bitbucket.org/noahgift/adapt/

The basic problems I am trying to solve is this:

1. Automatically convert existing Python command line tools into Web Services.
2. Create a plug and play WSGI application so I can talk to other WSGI goodies like LDAP auth, debugging, etc.
3. Create Yet Another Web Framework in Python (although it is pretty unconventional)
4. Automatically adapt existing shell scripts, aliases and Python scripts into a dynamically created command line tool.
5. Declarative mapping of actions via a simple config file.
6. Let people that are completely ignorant of Python, create command line tools and web services by running a tool.
7. Let people do BOTH, in one step.

Is this crazy...yes. But, it is also fun. Once I complete code coverage to 100% and move common code into a library, and make some "frameworkish" code, I will release it. Feel free to help if you think this sounds fun too.


I named it after my blog post "Adapt or Die". Because it was a direct outshoot of my philosophy on tools.

Python Artificial Intelligence Special Interest Group Mailing List

From the title of my blog,"Artificial Code", you can tell I am interested in Artificial Intelligence. I have noticed quite a few people in the Python community are interested in AI, so wondered if there was a mailing list, evidenced by viewing Raymond's AI talk at PyCon, and a recent blog post by Tennessee, and even Guido himself :). My google searches revealed there wasn't, so I started one:

http://groups.google.com/group/aipy

I think it would be fun to share ideas about artificial intelligence and then make real world implementations. We can also share how everything thinks that people interested in AI are crazy and that it is an unsolvable problem :) If you are interested in both AI and writing Python implementations, then join the list, and let's get to work.

Some of the things I am interested in are solving "low hanging fruit" problems that can be done this year, or sooner.

Short-Circuiting Python Module Lookup Gets 2066% Performance Improvement

Recently I ran nosetests and it took 26 seconds to run. On looking at an strace and it generated about 53K lines of output in strace on ubuntu.

After some help from a couple of co-workers, I noticed that I could get the following performance boost, note this is one a massive NFS installation, not my laptop...

Total Elapsed Time: 2066 % speed improvement
Lines of strace output: 3050| 1695 % reduction in calls to file system

This was accomplished through a series of tests:

#1 Hijack setuptools by explicitly crafting sys.path to include only the eggs it needs. If you don't use --multi-version it will traverse EVERY egg and put them in your sys.path.
#2 unset my Python path eliminating all of the places that were there by default
#3 write a bootstrap script that calls python -S "myscript.py" so site-packages don't get looked at.

I was pretty shocked at how big of an improvement this made. The strace output shrunk to about 3K lines, which I might even be smaller if I only looked at open files.

There are a few downsides to this, obviously flexibility, but I need the speed. Has anyone done anything similar to hijack Python import to speed up time to fire up an interpreter. My hunch is I can go much, much farther.

ReSTless a Restructured Text Preview Cocoa Application

(Note, sorry Planet Python for the previous, accidental cross post from my workout blog...)


One thing I had almost forgot to blog about, ReSTless
. One of my Atlanta area friends, Aaron Hilligass wrote a really sweet, and useful app to demo how to call Python from Cocoa in our book, Python For Unix and Linux System Administration. It turns to be a really, really useful application that I use all the time to preview Restructured text while I write it.

I have a link to the source code at the bottom of this post, as well as a link to the docutils source. One thing to watch out for, is to make sure you adjust the MyDocument.m source file to look at where you actually have rst2html.py. In my situation I used this location when I compiled:

    [task setLaunchPath:@"/usr/local/bin/rst2html.py"];

One of the other cool thing that it does is show you how to connect to a piped Python script. Nice work Aaron!









References:
Python For Unix and Linux System Administration Book
Source Code For Python For Unix and Linux System Administration
Latest Docutils Source Code