Planet SoC -

How to write good Kamaelia components and systems

JasonMBaker — Tue, 15 Jul 2008 22:05:58 +0200

I’d like to apologize for two things:

For not posting to my blog in so long.
For not continuing the distutils discussion. I’d kind of abandoned the idea for using distutils to distribute my app, but might end up using it to package my gateway, so stay tuned.

With that said, I think it’s time for me to share my views on the subject of what a good Kamaelia component is. Some of these are general rules of thumb of computer science, but they bear repeating here too. So here it goes. My rules for writing good Kamaelia components:

Keep it simple, avoid premature optimization, and take small steps. If you’re not totally new to programming, you’ve probably heard this a million times before. But a reminder never hurts.
Design your components to work well with others. You don’t know what someone else will want to use them for. For example, I’m using Ryan’s HTTPServer code to serve webpages via XMPP. I won’t get into the technical details of how that works here in my blog, but suffice it to say that I’m using Ryan’s code for something I’m for sure he didn’t imagine it would be used for. And it works pretty well. Make sure to keep in mind that rule 1 overrides this rule, though. Don’t convolute your code because you think someone else may want to use it later.
Don’t forget about shutdown handling. Let’s face it. Making your components shut down properly in Kamaelia/Axon is a pain. I think Michael and Matt will both agree with me on this. But don’t ignore it. That will only make the problem worse.
Beware the “one true” programming paradigm, including Kamaelia. Since I’ve been programming in Kamaelia, I’ve begun to wonder how I’ll ever transition back to “normal” programming. Indeed, Kamaelia is a fun and inventive way to program concurrently. But that doesn’t mean that it’s best suited for every situation. You may find that it’s better to just go with plain old structured or object oriented programming for some problems. And there’s nothing wrong with that. Personally, I use what I like to call a “wrap around” rule. That is to say, I always try to progam the solution I can wrap my head around the easiest. Chances are, I can get Kamaelia to do just about anything I want it to do. But if I find myself unable to fathom how I’ll do something in Kamaelia, I don’t do it in Kamaelia.
Be creative and have fun. I constantly am trying to think of new ways to do old problems. And sometimes it doesn’t work out so well. But the worst case scenario is that I learn a new way not to do things and have to start over. Best case scenario is that I find something innovative.
Be lazy. I won’t get too much into the specifics of this, but I see a lot of components written using a lot of unnecessary code. For example, when you check an inbox, how many times have you written the following lines of code:
```
while self.dataReady('some_inbox'):
    msg = self.recv('some_inbox')
    do_stuff_with_msg(msg)
```
You may not realize it, but Axon components now have a built in function Inbox that takes care of two of those three lines for you. Thus, those three lines can be written using a simple list comprehension:
```
[do_stuff_with_msg(msg) for msg in self.Inbox('inbox')]
```
In my opinion, that’s a lot more readable than the first version.

You may agree or disagree with the points I bring up. And that’s fine. But those are my views on what it takes to write good systems using Axon and Kamaelia.

Python distutils and you: Part I

JasonMBaker — Wed, 04 Jun 2008 07:21:40 +0200

In the past few days, I’ve grown a lot more familiar with python’s packaging options than I ever hoped to become. So, I thought I would share some of the things I learned in case anyone else runs across a need to use this in their GSoC projects (or any other project for that matter).
The setup script
This is the meat of python distutils. Surprisingly, there’s really not all to writing the setup.py script itself. Suppose I have a file called “foo.py” that I’d like to install on someone’s computer. Here’s what a setup.py script will look like: (examples shamelessly copied from the distutils documentation)

from distutils.core import setup

setup(name='foo',

      version='1.0',

      py_modules=['foo'],

That wasn’t so difficult was it? All you need now is to put foo.py in the same directory as setup.py and you’re good to go!
Packages
Now suppose that instead of a single module, we want to put a whole package into our distribution (a package being a directory containing an init.py). That’s where the packages keyword comes into play. Just pass a list of package names to the setup function.
So, let’s assume we have a directory ‘foo’ in the same directory as our setup.py script. This directory contains the files init.py, bar.py, and bar3.py. Here’s how we would do that:

from distutils.core import setup

setup(name='foo',

      version='1.0',

      packages=['foo'],

Now, suppose we want to get something we can distribute. That’s pretty easy. All you have to do is type

python setup.py sdist

That will create a directory called ‘dist’. If you look inside dist, you will see foo.tar.gz (or foo.zip for you windows users out there). Now all your user has to do is type in

sudo python setup.py install

and foo will go into the end user’s site-packages directory.
Scripts
Now this is all well and good if all we want to distribute is a library. But I’m packaging an application. Suppose I have a python file called ‘dofoo.py’ that that will import modules from foo when the end user runs it. Here’s how we would put that into the file:

from distutils.core import setup

setup(name='foo-program',

      version='1.0',

      packages=['foo'],
scripts=['dofoo.py'],

What does this do? Well first of all, the installer will automagically change the shebang line in dofoo.py to the end user’s python location if the first line of it begins with #! and contains the word python. Secondly, dofoo.py will be installed to /usr/bin unless the end user puts it elsewhere. Ideally, the key is to focus on what functionality something provides here rather than where it will go in the user’s file system since the user has ultimate control over what goes where.
Conclusion
Well, the part about writing the setup.py script ended up being longer than I had originally intended. There is still a lot more metadata that can go into setup.py. But hopefully that should be sufficient to give you a general idea of what distutils is all about. Next time I plan on talking a little bit more about the manifest and possibly some more distribution options.

Kamaelia Publish v. 0.0.1!

JasonMBaker — Sat, 31 May 2008 01:44:05 +0200

You may download the initial version of Kamaelia Publish here (Mac OS X only currently). Upon installing the package, you will find that Kamaelia Publish can serve basic WSGI webpages. You will get a demonstration of basic WSGI functions by going to http://127.0.0.1:8082/simple. I’ve gotten it running using MoinMoin, but for some reason POST methods don’t seem to work (thus you can’t edit pages).
And yes, I know the fact that the icon keeps bouncing up and down in the dock is annoying. I’ll have to find a way to fix it.

Gold Soundz

orphansandoligarchs — Fri, 23 May 2008 12:36:28 +0200

It’s been far too long since the last post (exams looming large over my otherwise peaceful existence), and given that my fellow SoCers have managed it I think it must be time for a status update. Firstly (and perhaps most importantly) we have sound! The screenshot below shows pygame drawing a bouncy XY pad, all created as a Kamaelia component. This is then outputting OSC messages which are read by PD and used to trigger and modulate percussive noise.

If you look closely at the PD patch you can begin to get an idea of how easy it is to work with OSC messages - the OSCroute objects make it simple to split up the different messages, giving you complete control over what is received and where it is used. So what does it sound like? Well, kinda chaotic and noisy and full of goodness - listen here.
Changing tack slightly, over at Create Digital Music there is an interesting interview with the guys from Future Audio Workshop talking about their rather lovely looking new synth Circle. There’s a lot of interesting stuff about interface design, and also a bit about its use of OSC. I think Gavin Burke, FAW’s co-founder, sums up the difference between OSC and MIDI nicely:
You’re not trying to think about MIDI and what MIDI channels you’re on and all that kind of stuff; it just keeps it nice and simple. [OSC] is where it’s heading. It’s where most people want to be; most people don’t want to [have to do MIDI mappings].
Read the whole thing though - it’s interesting stuff.

Miscelleny

JasonMBaker — Fri, 23 May 2008 04:51:16 +0200

I just thought I’d make a post about some of the miscellaneous goings on inside the Kamaelia project.

Davbo posts about multicore support. Although somewhat rambling (hey, he says so himself), he brings up some interesting points.
Lawouach makes a long and very informative post about XMPP, atompub, and microblogging. I’m working on reading the whole thing, I promise. I really need to learn more about XMPP and headstock so that when it comes time to do some stuff with it, I won’t be totally lost.
In other news, Lawouach was also forced to use Twitter. Poor guy.
You can also review the extremely interesting notes of our last meeting. I know. Try to contain your excitement.

For me, not too much has changed since my last post. I’m inching towards having better wsgi.errors support, and I think I’m almost there. I did also make an attempt at creating a better solution towards managing component shutdown in Kamaelia, but I think until some issues get worked out, it’s going to be best for testing components.

Kamaelia Multicore Support

Davbo — Fri, 23 May 2008 01:06:45 +0200

In response to a blog post over at Ye Olde Clue. I thought I’d write up my thoughts on this topic since I’m (hopefully) going to try to review this code and merge it into the core of Kamaelia. I say hopefully because it’s partly a matter of experience with subversion which at the minute I’m a few years behind other people on the Kamaelia project. Fortunately they’re all patient with me while I ask questions.
Okay, so, meanwhile back at the point: “Where should Multicore support sit in the Kamaelia tree?”
In my opinion the scope of what we’re calling “Multicore support”, is potentially more than just that. It potentially opens up avenues for things like distributed computing and many interesting things which require “Process Manipulation”. Of course I don’t see this happening in the near future but Kamaelia already has some very cool support for various network protocols. I think it would be reasonably trivial now we have 2 Processes working on different cores to stretch the distance of those cores to say the scale of a WAN. That sounds pretty damn cool to me, it may be a long way off or have little benefits. However with Axon I believe this kind of thing could be done in 1 line of code as opposed to several hundred. Yet again displaying how the Axon model makes concurrency so elegant and simple.
More immediate effects (pointed out by Matt on IRC) this component also applies to single core CPU’s with things like Hyperthreading, along with any other/further technologies implemented on this level of hardware.
Okay so I kinda rambled off-topic. Anyway; for things like this I think it’s a good idea to not box this into a label like “Multicore support” when we’re looking at another tool in Kamaelia arsenal
This is why I would rather see this code placed in “Kamaelia.Chassis.Process” rather than “Kamaelia.Chassis.Multicore”. Hopefully this makes sense.

What’s new

JasonMBaker — Wed, 21 May 2008 23:36:56 +0200

It’s been a little while since I’ve posted last, and there’s a lot that’s new, so I’ll enumerate the points:

The WSGI code is close to being complete. There are a couple of caveats though. First of all, there is no support for the write() callable. This is important for legacy app support as well as for streaming. Secondly, the wsgi.errors object still directs to stderr. There’s not necessarily anything wrong with this, but it’s certainly a good idea to have some kind of custom error log (see the next bullet point for more info on how I plan to achieve this). And lastly, my implementation of wsgi.input can be prone to DOS attacks. This is definitely something that needs to be fixed, but isn’t necessarily a first priority for me.
I’ve written a logging component for Kamaelia. It essentially uses a backplane to register itself to a service. The idea being that you won’t necessarily need a reference to any kind of object in order to post messages to a log. What I’m working on at the moment is an adapter for this component to act as a wsgi.errors object.
I’ve set up the basic structure of how I think the Descartes server will be. It’s far from being distribution ready, but it should be enough to give you the basic idea. For now, I’m placing some of the components (like the ServerCore) in my sketches directory as I’m not sure if any modifications I make to them will necessarily be a good fit for Kamaelia as a whole.

And that’s it for now. I think I may spend a little bit of time reading my start of program present, Beautiful Code.

URI vs URL

JasonMBaker — Wed, 14 May 2008 07:59:46 +0200

Ok, so this is one issue that’s been bugging me about HTTP. I keep hearing the acronyms URI and URL mentioned. I knew that URL wasn’t technically accurate, but I couldn’t ever find a good explanation of what the difference between the two are or why URI is more technically accurate. This is even after reading various explanations about the subject. Here’s what I’ve come up with:
URI
A URI is a name that identifies something globally. Admittedly, this explanation is a little bit vague, but then again the idea of a URI is kind of a vague concept. We’ll come back to this later, but I’ll give you a few examples of URIs:

```
http://www.coderspalace.com
```
```
http://www.coderspalace.com/index.php
```
```
file://usr/lib/python
```

URL
A URL is a special kind of URI. It gives you more precise instructions on where something is located. Thus, something like http://www.coderspalace.com/j_baker/ will tell you what computer a webpage is and will even narrow down where the webpage is located, but it won’t give you an exact location of the file like http://www.coderspalace.com/j_baker/index.php will.
So I suppose the next question is: who cares? The point is that nowadays you don’t really need the level of precision that a URL requires and haven’t for a long time. Try going to http://www.coderspalace.com/j_baker/ and http://www.coderspalace.com/j_baker/index.php and see if you get any difference between the two links. You won’t. This is because my webserver is smart enough to know that when you go to http://www.coderspalace.com/j_baker/ you really mean http://www.coderspalace.com/j_baker/index.php. Pretty cool, huh?

Kamaelia in DevChix

JasonMBaker — Wed, 14 May 2008 07:09:04 +0200

Kamaelia: The future of Python frameworks looks promising
Gloria from DevChix writes about Kamaelia. It is nice to see that the project is gaining some attention from others. Gloria does bring up one important point though: installing some of the dependencies for Kamaelia is a pain. Especially if you’re a newbie. Hell, sometimes even if you’re not a newbie.
I would like to point out that to me, this really seems like more of a python issue than a Kamaelia one. I mean, try installing Eric or SPE without some kind of package manager. You’ll quickly learn the meaning of the words “dependency hell.”
With that said, I think that regardless of the platform that Kamaelia is on, it does aim to be a programming methodology that is easy to use for beginners. And it does need to be able to overcome these problems if it ever intends to achieve that goal.
I was planning on making an “easy to use” package manager for installing WSGI software for end users to use. If I do it properly, I may be able to make this be a solution for the problem that Gloria mentions.

Google FriendConnect

JasonMBaker — Tue, 13 May 2008 18:31:10 +0200

Ars Technica writes about Google’s upcoming FriendConnect technology. From what I can see, it looks pretty awesome. It is essentially an extension of Google’s OpenSocial that allows site owners to embed authentication and various other social apps in their webpage. They give Ingrid Michaelson’s website as an example of what FriendConnect can do. Whatever the case, it looks pretty interesting. And more importantly, it looks like something I would want to put into Kamaelia Publish. I just signed up for a preview release of it. We’ll see if I get in.