Planet SoC -

If there was one thing you wish you had known before…

Sjors — Mon, 01 Sep 2008 04:54:48 +0200

One of the questions in the student evaluation of the Google Summer of Code reads:
If there was one thing you wish you had known before getting started in Summer of Code, what would it be?
It is a very typical evaluation question and we all sortof know what it means and how to answer it. However, if you insist on thinking about it - and this is very acceptable behavior in some circles - it is actually a very difficult question.
I tend to read this question as follows: if you could meet yourself in the past, what would you tell your past self?
Well, I would probably give myself the final git repository, plus an external hard disk with as much of the interesting new information on the present day Internet as possible.
What would my past self do with that information? He would probably decide to use the outcome of a couple of footy matches to make a decent living. But apart from that, he would pick another project. Not because my project is uninteresting, but I really enjoyed that part of the project that I worked on. Continuing to work on this particular code base is really interesting, but not as interesting as it was to build it in the first place.
So my conclusion is: the more advice that I give to my past self, the less intersting his project would become. This is not a real problem however, because the information would provide him with many new opportunities.
I have the same view on Sudoko puzzles. A friend of mine wrote a computer program, while he was drinking beer in the pub, that could solve a lot of these puzzles. Many fanatical puzzlers would never consider using such a program; it would take away the fun.
I completely disagree with them. Now that my friend has releaved the world of The Sudoko Problem, mankind can move on to solving new problems.
I do not understand why people take such pleasure in creating artificial problems and then solving them over and over again, when there is an astonishing abundance of problems already out there waiting to be solved.
Just to make an even bolder statement: anyone who spends even a minute a day solving problems that have already been solved, should feel really guilty about climate change, poverty, diseases, slow public transport and millions and millions of other problems. Well at least, I tend to look at my own behavior from that perspective. All that without losing the ability to enjoy live; that is the real tricky part.

Language filtering of diary entries!!

makghosh — Sat, 23 Aug 2008 22:06:57 +0200

This feature was originally proposed by Dan Karran as a comment in one of my earlier posts. Anyway, so here we have “show all / filter by my language” tags in the user diaries’ pages. By default, all entries are shown. We can click on ‘filter by my language’ so as filter the posts by the locale selected by the user in his preferences. We can also have feeds for these filtered entries. The screen-shot of current diary entries page,
Image #1: openstreetmap-language-filter-diary-entries
And for this we need to specify the language of each diary entry. So new entry page has an option or rather a dropdown menu from where we select the language of the entry. This preference, by default takes the value of user’s current locale and is saved in language column of diary_entries table. The screen-shot of ‘new diary entry’ is as follows,
Image #2: openstreetmap-new-diary-entry
Also put up your suggestions…

Language filtering of diary entries!!

makghosh — Sat, 23 Aug 2008 21:49:57 +0200

This feature was originally proposed by Dan Karran as a comment in one of my earlier posts. Anyway, so here we have “show all / filter by my language” tags in the user diaries’ pages. By default, all entries are shown. We can click on ‘filter by my language’ so as filter the posts by the locale selected by the user in his prefernces. We can also have feeds for these filtered entries. The screen-shot of current diary entries page,
Image #1: openstreetmap-language-filter-diary-entries
And for this we need to specify the language of each diary entry. So new entry page has an option or rather a dropdown menu from where we select the language of the entry. This preference, by default takes the value of user’s current locale and is saved in language column of diary_entries table. The screen-shot of new entry is as follows,
Image #2: openstreetmap-new-diary-entry
Also put up your suggestions…

Language filtering of diary entries!!

makghosh — Sat, 23 Aug 2008 21:49:57 +0200

This feature was originally proposed by Dan Karran as a comment in one of my earlier posts. Anyway, so here we have “show all / filter by my language” tags in the user diaries’ pages. By default, all entries are shown. We can click on ‘filter by my language’ so as filter the posts by the locale selected by the user in his prefernces. We can also have feeds for these filtered entries. The screen-shot of current diary entries page,
Image #1: openstreetmap-language-filter-diary-entries
And for this we need to specify the language of each diary entry. So new entry page has an option or rather a dropdown menu from where we select the language of the entry. This preference, by default takes the value of user’s current locale and is saved in language column of diary_entries table. The screen-shot of ‘new diary entry’ is as follows,
Image #2: openstreetmap-new-diary-entry
Also put up your suggestions…

Localization Statistics

makghosh — Sat, 23 Aug 2008 21:48:18 +0200

After translation updates, I did the l10n statistics. This basically give the translators an idea of number of pending/completed strings and percentage of localization done for all locales currently supported by the app. For this i created a Statistics table in database using migration. Now the statistics are updated when user clicks on the l10n link on his/her home page. The logic (in TranslateController#stats) is if the current language of the user is not present in the statistics table, it is added. Otherwise normally the values are updated with the help of globalize_translations table. The screen-shot of the l10n statistics,
Image #1: openstreetmap-l10n-statistics
Its’ this statistics that apears first when the translator clicks on l10n link in his/her home page. Another nifty feature is that, s/he can view the sort the l10n statistics in two ways, ‘by percentage’ or ‘by number of completed strings’ in descending order. Also, every locale has a link to the pending/completed strings page associated with the numbers.
Apart from this, i have kept this l10n statistics page (without those pending/completed strings link) visible to all users, just to motivate more of them into translation. The screenshot of user’s home page with l10n link (view rendered in bn-IN locale as you can see one string i translated for example),
Image #2: openstreetmap-user-home-bengali-partial
The difference between user/translator is done by tr_status column in user table,
0 = User
1 = Translator
2 = l10n Admin (proposed)
Todo: For now, the tr_status is updated manually. We have thought about a admin page for the l10n admin where s/he can add/remove translators, add/remove coordinators etc. Thoughts??
Please put up your comments or suggestions…

Localization Statistics

makghosh — Sat, 23 Aug 2008 21:48:18 +0200

After translation updates, I did the l10n statistics. This basically give the translators an idea of number of pending/completed strings and percentage of localization done for all locales currently supported by the app. For this i created a Statistics table in database using migration. Now the statistics are updated when user clicks on the l10n link on his/her home page. The logic (in TranslateController#stats) is if the current language of the user is not present in the statistics table, it is added. Otherwise normally the values are updated with the help of globalize_translations table. The screen-shot of the l10n statistics,
Image #1: openstreetmap-l10n-statistics
Its’ the ‘l10n statistics’ that apears first when the translator clicks on l10n link in his/her home page. Another nifty feature is that, s/he can view the sort the l10n statistics in two ways, ‘by percentage’ or ‘by number of completed strings’ in descending order. Also, every locale has a link to the pending/completed strings page associated with the numbers.
Apart from this, i have kept this l10n statistics page (without those pending/completed strings link) visible to all users, just to motivate more of them into translation. The screenshot of user’s home page with l10n link (view rendered in bn-IN locale as you can see one string i translated for example),
Image #2: openstreetmap-user-home-bengali-partial
The difference between user/translator is done by tr_status column in user table,
0 = User
1 = Translator
2 = l10n Admin (proposed)
Todo: For now, the tr_status is updated manually. We have thought about a admin page for the l10n admin where s/he can add/remove translators, add/remove coordinators etc. Thoughts??
Please put up your comments or suggestions…

Translation updates via RSS

makghosh — Sat, 23 Aug 2008 21:47:16 +0200

Its’ been a while since i last posted an update. So now, I need to open-discuss all my exploits for last 2-3 weeks. I owe lot of thanks to my mentor Mikel Maron for helping me out in testing different features, pointing out some important bugs, suggesting new features and how to code/implement them. Now, coming straight to the point, with help of globalize plugin in osm rails-port, we now have translation feeds. I mean now translators can now have rss feeds for their default language (say spanish (es-ES)), both for pending and completed strings. These are the strings generated or rather added in the globalize_translations table while different views are rendered in that locale.
We have the entire list of pending or completed strings for a particular locale (which is actually the language chosen in user settings page) paginated in groups of ten, which is in turn available as updates by rss feeds. We can simply move to “older strings / newer strings” accordingly. For doing these i needed to add translate_controller.rb, corresponding views and also updated the routes.rb as required. The screen-shot of the “l10n home” that we have in osm rails-port now,
Image #1: openstreetmap-l10n-home-pending-strings-with-rss
Also, each string has a link that opens up the translation interface where we can view the string and add/update the translation for that particular string. For this purpose, I have used a form, where we can type-in the translation for the string in the translators’ locale. When we add/update translation for a string, its’ updated in the database and if everything works fine, we are confirmed by a flash notice “Translation was updated successfully”. The next time we open the corresponding view, we find the string translated. Here’s the screenshot of the translation interface,
Image #2: openstreetmap-translation-interface
There are some languages which are common to multiple countries. So i had to update the views so that locale without any country tag can be added. I mean earlier, Spanish(es-ES) & German(de-DE) etc got to have the country tag. But now languages like Arabic(ar) common to multiple countries can also be added.
Todo: Now this web-based translation is actually meant to be handled by the ajax in-place-editor of click-to-globalize. Because it places the translatable strings in context, as an advantage. But the problems with CtG are partially fixed. More on this at a later post. So we need the in-place editor start working. Although, the existing translation interface will stay on, so that translators can update translation of any string on the go, if needed just by checking the feeds. They don’t have to search for the concerned view! Thoughts??

The end of the Summer - let there be Summer!

Sjors — Fri, 15 Aug 2008 14:52:51 +0200

Although there are still a couple of days left until the official Pencils Down date of the Summer of Code, I am now officially putting my pencil down because I need to catch a train to Adelaide tomorrow morning.
I guess this really marks the end of my student period; even though I graduated in June, this project allowed me to feel like a student just a little longer. Sniff, now I really have to enter the big scary adult world.
But first I will go on a trip for two weeks to see Adelaide, the Ghan train, Darwin and Kakadu National park. It will be a very culturally diverse trip; from what I have heard, Adelaide and Darwin are pretty much as different as it gets here in Australia.
Route profile demo by Lambertus
I am very happy to see that my application has found its way to an actual route planner website (see figure above)!
So what is next? Well, I will obviously have much less time to work on this project, so my highest priority will be to explain other people how to use and improve the application and how to install it on their own server. So don’t hesitate to mail me about that!
I have to keep this post short because I still have to pack some stuff and it is already late. But I do want to thank some people of course. Thanks Google for sponsoring me (and for creating all sorts of cool and useful tools for my project). Thanks OpenStreetMap community for selecting my project, your confidence in me and your support. And of course, thanks Artem for mentoring me during the project and for being a great and interesting person to talk to in general!
These are just thankyou’s, not goodbyes. So see you soon!

Weekly update route altitude profile

Sjors — Tue, 29 Jul 2008 05:29:52 +0200

It’s a bit of a boring title, but it actually has been an interesting week. Although I found myself highly distracted by some unrelated but fascinating things, I still managed to get quite a bit done.
The script that I used to download the SRTM data set and import it into a Postgres database can now deal with all continents and supports uploading a subset of a continent by means of a bounding box. I also put the md5 check sum of every tile in the source code.
Since the App Engine still has some issues, I have revived the Postgres version of my application. It is located at http://altitude-pg.sprovoost.nl and contains most of Europe as far east as Moscow and as far south as Cyprus. It runs on my home computer in The Netherlands, so please be nice to it. I use apache-mod-python for the formal demonstration website and apache-mod-wsgi combined with web.py for the altitude profile server. To make this as painless as possible, I have moved all App Engine and Postgress specific code to their own files and kept as much common functionality as possible in the main file. I can now run the development servers for both Apache and the App Engine from the same source code folder, at the same time.
I have requested more storage space on the App Engine and I am also considering a more efficient storage method. In stead of storing one altitude per record, I could store 100 altitudes per record and zip them. That would drastically reduce the total storage requirement, but at the cost of performance because I often need only about 2 out of these 100 altitudes.
I have also been a bit more active on their mailing list; it feels good to be able to answer peoples questions and at the same time it allows me to verify my own code and design. There are also some interesting albeit more philosophical discussions on the list.
I have signed and fulfilled a pledge to “spend an hour OpenStreetMapping features on Caribbean islands from Yahoo! aerial imagery and […] donate £10 to the OpenStreetMap Foundation but only if 60 other people will do the same.”. I felt like I could really use another jet-lag. The pledge is full, but who knows, if they can rally another 60 people there might be a second ticket?
Those of you who laboriously follow every commit to the OpenStreetMap subversion repository, may have noticed that I am still struggling with git-svn. I got really tired of fixing conflicts, so I unleashed the power of git-svn set-tree:

git-svn set-tree -i trunk 3cb585dca1d7fe10791312ca26125168506b61c1
git-svn set-tree -i trunk 07c9024f5ea4ce60f481b8089b61d4988e7588fa

Even the manual recommends against doing this, and you should make sure nobody else (like your mentor) has committed anything to subversion before you do this.
I find git-svn to be harder to use than it should be. I think it is trying to hard to properly translate between The Git Way and The Subversion Way. I just want the subversion repository to ’sort of’ track my git repository. I don’t care if it has to represent the history a bit different. Just keep the code up to date. I am looking forward to this command:

git-svn just-do-it

I really think Git would benefit the OpenStreetMap community, because it reflects the decentralized nature of OpenStreetMap. With Git, there is no such thing as a central repository. People can write any code they like without having to live in constant fear of breaking the trunk with their next commit. In stead, when they build something cool or useful, they will tell their friends to pull it in and experiment with it. The person who operates a production website will only pull pieces of code that he or she considers safe and useful enough.
But the reality is that many organizations rely on subversion at the moment and have excellent reasons for not risking their operations by making an instant jump to Git. So people are not going to adopt Git very quickly as long as it is so hard to sync with subversion. But lets wait for a while and see…
I am getting better and better at keeping my git repository synchronized with the osm subversion, but I would not recommend this strategy to others.
I created a project on Google Code Hosting project for the altitude profile. Not to host the code, not even for the wiki, but just to keep a list of issues. I realize I could have applied for a place on the OpenStreetMap Trac, but I want to use Google Code Hosting for my new project: Jobtorrent. This is also the reason most of the issues point to the Git source (I do point to subversion on the main page and the only reason I do not always point to both is that I am lazy). I will write more about Jobtorrent later; first I need to work on my Summer of Code project you know…
This list of issues should be good for continuity. Because my project does not interact with any OpenStreetMap code at the moment, I am probably the only one in the community who knows how the code works and what needs to be improved. That is a very low bus factor! (”tram factor” would be a better term in Melbourne) Now I really like the OpenStreetMap effort and I will certainly find ways to stay involved in the future, but it might be in a completely different project. Depending on circumstances, I should at least prepare for the possibility that the altitude profile project will be orphaned within a few months.
I use a personal organizing method inspired by the book Getting Things Done (David Allen) and that makes it very easy to transfer everything I am working on or thinking about to the Internet. So that is what I did.
The more difficult part is keeping it synchronized. David recommends that you never share your projects. That is, you should always keep your own lists and let nobody else touch them. Your lists must reflect what you want, or you will start to rebel against them and as a result mess up your system.
So in practice you will end up with a central list (e.g. the list of issues on Google Code) and your local copy of it. They will not be the same. There are a couple of things on my personal list that are not online (nothing ground braking, don’t worry) and my own priorities are not identical to the ones online. The online version reflects what is important for The Project, the offline version reflect what is important for me. At least in theory; as long as I am the only one working on it, it probably reflects my opinion a lot better than it ideally should.
Now I am pretty sure the average recruiter looking for a “true team player” does not like what I just said in the last paragraph.

User’s language preference added in osm rails-port

makghosh — Tue, 22 Jul 2008 11:43:56 +0200

In last couple of days, I added the user language preference in the osm rails-port. It is basically implemented by a drop-down menu in the settings page of the user, from where he can easily select & save the desired locale. When the selection is saved, it’s saved in the database where a locale column has been added to user table by migration (012_add_user_locale.rb). The drop-down menu is implemented in the view (account.rhtml) by a helper method called “select” as given,
select ("user", "locale", { "English(US)" => "en-US", "Bengali(IN)" => "bn-IN", "Hindi(IN)" => "hn-IN", "Spanish" => "es-ES" })
Correspondingly, the user’s model & controller has been updated. Here, i have used a hash within the select method for specifying languages, which can be easily replaced by a hash variable defined in config/environment.rb for convinience. Another approach maybe to find all the locale codes from globalize_translation table migrated by the globalize plugin using “collection_select” helper method. But, in the previous one we actually will have the flexibility to add only those languages we prefer (it may be based on availabity of translations or rather request for the language branch by an l10n team). This preference option is quite independant. But the locale-routing part is yet to be decided, regarding how to do that, because click-to-globalize also implements some sort of routing associated with locale_controller.rb. The screenshot of the user settings page:
User's language preference added.
More importantly, a sort of bug that came with the globalize plugin as i have posted earlier (error log) is now fixed. As it can seen in the log, while I tried to create a new user, <% error_messages_for 'user' %> is called in the app/view/user/new.rhtml. Now, globalize plugin overrides the “error_messages_for” helper method for it’s own requirements. So, as a matter of fact, the method “error_messages_for” tried to execute nil.errors? (from the error log) which is quite obvious, since the object user is created after rendering the view. I first tried to take care of this by a small change in active_record_helper.rb within the globalize plugin:
return "" if object_name.nil?
But it didn’t do much good. Later i found a better fix than that which actually worked flawlessly. Here’s the new active_record_helper.rb. So one of the two big problems i previously blogged with these plugins is solved. Since, globalize started working i could also test the views related to ‘user’. There were some sillymistakes for example, forgot to use to_s in @user.messages.size in the globalized view.
"You have %s new messages and".t(nil,@user.new_messages.size.to_s) + " %s old messages".t(nil,(@user.messages.size - @user.new_messages.size).to_s)
I have commited all these updates in the i18n branch. Also, apart from the inherent loopholes in click-to-globalize, we are also focussing on other areas that needs to addressed including translation notifications etc.