Blog entries from Thousand Parsec SoC students and mentors

Ruby Oneliner: Benchmarking a string concordance

jphr — Thu, 06 Nov 2008 07:12:36 +0100

Just the other week in one of my university Comp. Sci. classes I was asked to use a supplied Linked List to create a Concordance from standard input (in C I might add). The problem wasn’t necessarily hard, in fact, it was simple enough some friends and I realized it was a great Ruby one-liner candidate; Sure enough this was the result after no more than a minute of jabbering:

hash = Hash.new(0); str.split.each { |m| hash[m] += 1}

Well thats all fine and dandy… A plain old Ruby one-liner. My friend Stef, however, suggested this close alternative:

hash = Hash.new(0); str.scan(/\w+/m) { |m| hash[m] += 1}

Whats different? Well Stef’s code uses a regex scan of a “m”ultiline string, then adds 1 to each match in the hash. His regex takes series of 1 or more “\w”ord characters to be a match. Whereas my code uses Ruby’s built-in “split” method to split on whitespace, then iterate over the resultant array.
This is how split works:

str = "My name is Ryan."
str.split #=> ["My","name","is","Ryan."]

For simple strings, like “My name is Ryan.”, Stef’s regex scan works almost identically. For this example we will ignore the fact that “\w” won’t match things like ‘-’, its not really that important at the moment.
As any good Computer Scientists our divergence of methods lead to a great argument… Which one was better? From my point of view, “split.each” is much more readable, clearly (without a regex) splits on whitespace, and is nearly as terse as the regex equivalent. From Stef’s point of view he A) didn’t have to use “each” and B) had more control over the split. We agreed to disagree, clearly each works best in different situations. Split is best for a simple split, but Scan is far more versatile.
Having put semantics aside we began wrestling over which one would be faster. We threw together this “Benchmark” script:

require 'benchmark'

str = "1 2 3 4-5 6 7 8-9"
Benchmark.bm do |bm|
bm.report("split: ") {10000.times do hash = Hash.new(0); str.split.each { |m| hash[m] += 1}; end  }
bm.report("scan: (\w+) ") { 10000.times do hash = Hash.new(0); str.scan(/\w+/m) { |m| hash[m] = 1} end  }
bm.report("scan: (\w+(-\w+)?) ") { 10000.times do hash = Hash.new(0); str.scan(/(\w+(-\w+)?)/m) { |m| hash[m] += 1} end  }
end

Here’s the result of those benchmarks on various Ruby versions:



Native environment tests - 1.8.7

Creating one hash and clear it: (hash.clear instead of hash = Hash.new(0))

user     system      total        real

split:            0.490000   0.150000   0.640000 (  0.656165)

scan: (\w+)       0.800000   0.180000   0.980000 (  1.003529)

scan: (w+(-w+)?)  1.390000   0.340000   1.730000 (  1.745792)

Creating a new hash every time:

user     system      total        real

split:            0.470000   0.140000   0.610000 (  0.643760)

scan: (\w+)       0.800000   0.180000   0.980000 (  0.989383)

scan: (w+(-w+)?)  1.170000   0.260000   1.430000 (  1.457280)

Variety tests by Stef Penner

mbp:rubinius stefan$ ruby -v

-> ruby 1.8.7 (2008-06-20 patchlevel 22) [i686-darwin9.3.0]

mbp:rubinius stefan$ macruby -v

-> MacRuby version 0.3 (ruby 1.9.0 2008-06-03) [universal-darwin9.0]

mbp:rubinius stefan$ jruby -v

-> ruby 1.8.6 (2008-06-22 rev 6555) [i386-jruby1.1.1]

mbp:rubinius stefan$ rbx -v

-> rubinius 0.9.0 (ruby 1.8.6 compatible) (8038487c4) (10/19/2008) [i686-apple-darwin9.5.0]

Variety tests by Stef Penner

$ rubinous regx.rb

user     system      total        real

split:           1.422384   0.000000   1.422384 (  1.422366)

scan: (\w+)      1.458300   0.000000   1.458300 (  1.458299)

scan: (w+(-w+)?) 2.127930   0.000000   2.127930 (  2.127929)

$ ruby regx.rb

user     system      total        real

split:           0.410000   0.140000   0.550000 (  0.559599)

scan: (\w+)      0.670000   0.180000   0.850000 (  0.862585)

scan: (w+(-w+)?) 0.990000   0.270000   1.260000 (  1.268065)

$ ruby1.9 regx.rb

user     system      total        real

split:           0.090000   0.000000   0.090000 (  0.096752)

scan: (\w+)      0.170000   0.000000   0.170000 (  0.164321)

scan: (w+(-w+)?) 0.280000   0.000000   0.280000 (  0.291374)

$ macruby regx.rb

user     system      total        real

split:           0.440000   0.030000   0.470000 (  0.490660)

scan: (\w+)      4.310000   0.050000   4.360000 (  4.449849)

scan: (w+(-w+)?) 4.380000   0.040000   4.420000 (  4.503897)

$ jruby regx.rb

user     system      total        real

split:           0.456000   0.000000   0.456000 (  0.456000)

scan: (\w+)      0.261000   0.000000   0.261000 (  0.260000)

scan: (w+(-w+)?) 0.369000   0.000000   0.369000 (  0.369000)

$jruby 1.1.3 regx.rb

user     system      total        real

split:           0.235000   0.000000   0.235000 ( 0.234993)

scan: (\w+)      0.228000   0.000000   0.228000 ( 0.228318)

scan: (w+(-w+)?) 0.329000   0.000000   0.329000 ( 0.328884)

Its rather interesting to see how each version of ruby compares, yes Rubinius is slower, but WOW, Ruby 1.9.1 takes only 16% the time 1.8.7 takes!

repurposing the blog

jphr — Thu, 09 Oct 2008 03:21:52 +0200

Well as GSoC has come to an end and my interests have moved on. I will be repurposing the blog as a more code/personal-centric blog (it is, THE, hammer of code). If your GSoC RSS keeps nagging you with my annoying posts give me a shout telling me where I’m getting to you from; I’ll remove myself from that feed aggregator.
As for me, I’ve come upon a new stage of my “developership” – I’ve been learning Ruby, Rails, CSS, HTML, and a bit of Javascript. (in addition to all the languages I’ve been learning in school: FP, C, Lisp and Prolog at present). As much as I didn’t expect I’ve been exhilarated by web development these last few weeks. I’ve been spending more and more time with my two good friends Burke and Stef over at Chromium 53; both of whom attend the University of Manitoba with me. They’ve been doing big things, and it has gotten me excited. I’m currently playing catch up with Ruby and Rails in hopes of working with those two fine gentlemen (and the way things are going they will need a few helping hands ).
Stay tuned as I may start posting little snips and pieces of code I write or find interesting.

TP Single Player Release

ezod — Mon, 06 Oct 2008 21:03:56 +0200

We’re planning to release a version of the Thousand Parsec tpclient-pywx client soon with the new single player wizard. Support for single player mode is also coming in tpserver-cpp and (hopefully) a couple of the AI clients. The target date for the first candidate is October 19.

I’ve significantly updated the Gentoo overlay for Thousand Parsec recently, stabilizing most of the release ebuilds and adding ebuilds that grab the latest Git versions. Among other things, this means you can grab the as-yet unreleased tpserver-cpp and tpadmin-cpp combo implementing the new administration protocol!

There And Back Again

ezod — Mon, 15 Sep 2008 04:48:51 +0200

After about a month’s break (vacation to BC, followed by a week buried in books, followed by ICDSC), I’m back to both school and Thousand Parsec development.

For the former, attending ICDSC couldn’t have come at a better time. I’m just in the process of starting new research, and some of the topics presented there have given me some good perspective on what’s important currently. Several researchers (like this guy) employed stereo camera nodes for various applications, lending some credence to my advocacy for them, and also providing some validation for my M.A.Sc. thesis and paper on calibration. A number of groups presented research employing vision graphs and agent-based stuff, both of which are on our menu currently. Finally, there was some interesting talk on the last day about middleware for DSC networks which piqued my interest.

For the latter, my config-branch (remote administration) changes have been merged in tpserver-cpp and family. I’m now going to improve the implementation of the single player back-end functionality in libtpclient-py. A major concern is the compatibility of the bash scripts and motivation for the component developers to actually create them, so I’m going to try to change it to some versatile Python process management code, and have the custom execution stuff live in the description XML file. Hopefully in the process I’ll be able to further centralize the work so that writing the extra XML is reasonably easy while still being able to handle a variety of possibilities. I’ll also spend some time beefing up the inline documentation throughout the client library.

Moving

jphr — Thu, 11 Sep 2008 02:23:00 +0200

I’ll be moving to a new host over the next few hours. If you happen to hit up the site before then be warned; I’ll be back soon, but I can’t guarantee things will be up and running right away.

A quick update, just to break the silence

jmingtan — Thu, 04 Sep 2008 17:23:00 +0200

I’ve posted a new Windows build, and am currently trying to get the client to work fully with the python-ogre deb on Ubuntu. As expected, getting python-ogre to work on Mac is quite a chore, I’m almost there, got everything compiled but running the demos gives me a “wrapper” error, and I’ll probably have to understand how the code generation in python-ogre works to figure it out.
Packaging issues aside, I’ve made a few updates to the client. The lines in the screen represent “Wormholes”, which are basically routes between star systems. Currently, only the Risk ruleset uses this feature, so lines won’t display anywhere else, such as on the demo1.thousandparsec.net server which usually runs Minisec.

I’ve also tweaked the “Information” window to show more useful information such as the planetary resources (I tested it with the “Armies” resource in Risk). In general, I’m trying to locate more crashes by testing the client with different systems and python-ogre builds. OgreAL has been a little problematic, and it’s one of the things which I am trying to get working with the linux python-ogre deb file.
My main concern about the client is the performance on older hardware. Mithro informed me that even the login screen runs at a crawl on his laptop, which is probably due to the large textures used in the skybox (1024x1024). I guess a solution to this would be to offer a lower-resolution skybox thats selectable from the options menu.
I’ve started my industrial attachment, so I’m pretty much limited to weekends. Nevertheless, I’ll be continuing to update the client, and when tp04 support has been added content management will be a lot more elegant, allowing “media” tags which mean no more hardcoded graphics. Another thing I’m looking forward to is more battlexml support in the servers. On my end, I also hope to include some shaders to give game objects a more realistic look, but there are still many things to work out before I have time for that.

P.S. If you zoom in on the pictures it kinda looks like those old VGA games.

From Dev Hobby

still alive?

jphr — Wed, 03 Sep 2008 07:06:18 +0200

Well, contrary to how my current weekly post-count must look, I am still alive. Where I last left off I had just finished up the official part of my work on thousand parsec. Since then I travelled a few times, to a family member’s cabin in Gimli, MB, and into Winnipeg while my wife did some professional development for work. All the while I was gaming to my hearts content; In my time I played Patapon, Lumines, Geometries Wars 2, Fable (in anticipation of Fable 2), and am presently playing Castle Crashers (which I’m surprised my sister actually enjoys playing with me). In addition I’ve begun extended support and polishing off TP-risk, particularly some bug fixes, and planning on a substantial fork to work out what I would call unresolvable issues.
Today I wrapped up some errands at the university before I head back to school on thursday. All I can say is I am very much sick and tired of being nickeled and dimed by univesities and the like treating their students like invalids; I’m a big **ing boy now, I should be able to use a credit card to pay for things, declare majors, not pay almost $120 for each book I purchase, and finally, I should be treated as if I were responsible for my own decisions. AND SCENE… (I’ll cut myself short here, it will get pretty incoherent if I continue)
As for classes I am taking this term; I’ll be taking:

Discrete Mathematics for Computer Science
Introduction to Scientific Computing
Introduction to Artificial Intelligence
Programming Practices (a course in C taught by my favorite professor M. Zapp)
Programming Language Concepts

I’m not sure how the load of the term will be, but if it isn’t that much (hopefully my GSoC experience has trained me well) I hope to spend some more time developing myself a decent game engine to start making games on, or perhaps learn Ruby or Python, as well as trying to make a dime or two on RentACoder. It is getting late though, I should be off to bed

still alive?

jphr — Wed, 03 Sep 2008 07:06:18 +0200

Discrete Mathematics for Computer Science
Introduction to Scientific Computing
Introduction to Artificial Intelligence
Programming Practices (a course in C taught by my favorite professor M. Zapp)
Programming Language Concepts

Weekly Report: August 21-September 2

JLafont — Wed, 03 Sep 2008 01:21:00 +0200

Wow. I’ve been way too busy this past week. Classes started and the workload managed to pile on way too fast. I still haven’t even gotten some of my books. Thankfully, most of Dronesec was already done for the deadline on August 18. I’ve managed to do a couple of things in the short amounts of free time I had but not nearly as much as I would like. I managed to get tpserver-py on windows to work which I hope to put on a full How-To article on the TP wiki tonight or tomorrow if time permits.

I used epydoc on tpserver-py and I found that I need to find a more standard format for the docstrings and to fill it out for a lot more documentation. I’ll be slowly working on that this week as my time permits. I do however think that finding a good standard form for the docstrings would be a good idea to make sure that at least most of the information needed is there.

Today officially marks the end of GSOC, but I think I’ll keep writing on this blog once in a while to help keep track of dronesec and what I’m doing with tpserver-py. I imagine the next couple of months will see a transition into the TP04 protocol which I am very excited about and hope it will be relatively painless :-D

Weekly Report August 13 - August 20

JLafont — Thu, 21 Aug 2008 00:04:00 +0200

Its the second to last week and most of the touch ups and documentation are on their way. Unfortunately, classes began this week and it has kept me much busier than I hoped. However, dronesec and tpserver-py are both coming along nicely and windows operability has been achieved!

This week I: 7a50714cf2217b0d2ea663f6ce0b9af7

The server is now operational on Windows! I haven’t fully tested it yet but so far the server will start and allow players to join. I already have a quick installation guide made for it but I hope to add a more fleshed out and comprehensive guide to the TP wiki.

I hope to continue doing more documentation and get tpserver-py ready for a release this week as well as another screencast.