Planet SoC

Hi! Google Summer of Code 2008 just ended and I thought that it’d be good to make a new blog entry about where I got during the last days of GSoC and talk a bit about the overall experience.
Status update
The problem I was having with player movement last time really was just that, the player movement routine (executePlayerInput) was incomplete. Once I reverse engineered Operation Stealth’s version of it and modified ScummVM’s current implementation (r33872) of that function a bit I got the swimming working and I could get to the girl and free her:
Freeing the girl in ScummVM
I might as well tell a bit about what happens in the game after that:
The player character and the girl swim to the surface and are rescued by a boat:
Being rescued by boat
The story goes on and they meet The Movement for The Liberation of Santa Paragua for the first time:
Meeting the movement
Then they leave to infiltrate a palace:
Leaving for the palace
And enroute to the palace some masking bugs
Masking bugging while enroute to the palace
And then after a cutscene at the palace there’s another arcade sequence: A labyrinth.
The labyrinth
Okay, so far I’ve gotten to the second arcade sequence which is a labyrinth:
First labyrinth
But there are some showstopping problems here. The player can be controlled but the doors that should revolve when walked against don’t revolve and so the arcade sequence is not yet completable.
I played Operation Stealth using DOSBox to the labyrinth and used DOSBox’s debugger to look for the collision data page (And I found it! Yay!) and dumped it to a binary file. What I wanted to do was to compare the original’s collision data page to the ScummVM’s version in the labyrinth to see if there were any differences that could be breaking the gameplay. And voila, there were! Here are some pictures:
First labyrinth's collision data from DOSBox
First labyrinth's collision data from ScummVM
By the way the collision data page can currently be viewed in the Cine engine by pressing F11 (I added that in r33877).
I tried loading the collision data I dumped from DOSBox to Cine’s collision data page and yes, I got the doors to revolve! But alas, they didn’t yet work totally correctly, sometimes you could just simply walk through a door like it was thin air and sometimes a door would shake between two frames of animation a bit weirdly.
So it’s looking like at least there’s something wrong with how Cine currently manages the collision data or how it’s loaded. I checked the collision data loading routine loadCtOS and the data conversion routine gfxConvertSpriteToRaw that it uses. It looks like the gfxConvertSpriteToRaw is indeed correct, I reverse engineered a version of it myself from Operation Stealth’s disassembly, tried using my own version of it but it didn’t seem to make a difference at all. So that particular routine doesn’t look to be the culprit here.
Release testing related bugfixes
With the next stable version of ScummVM being prepared and release testing in progress I got some Future Wars related bug reports and did some bugfixing:

• Adlib sound related bugfix to fix a crash when teleporting to the swamp (r33935 trunk, r33937 0.12.0)
• Fix for overparanoid monks beating player unconscious (r33950 trunk, r33951 0.12.0)
• Fix for crashing when examing rubble above a manhole (r33967 trunk, r33966 0.12.0)

And with those fixes in place Future Wars’s English DOS floppy version should now be completable using the 0.12.0 branch.
Reported Future Wars bugs still unfixed
There are some regressions and bugs still left in the Cine engine though:

• Bug #2057656: FW: Assert during demo (regression)
• Bug #2057637: FW: Spaceship glitches in demo (regression)
• Bug #2057619: FW: Glitches in title display of demo (regression)
• Bug #2055912: FW: incrustSprite fails on savegame loading
• Bug #2055836: FW: Music is not restarted when loading a saved game
• Bug #2019344: FW: crash with Amiga Italian version (photocopy room)

What I learned during this summer
Ok, enough about the status update, here are some points I’d like to make about what I learned during this summer of code:

• A lot about reverse engineering - I feel a lot more competent in it now than I did before this summer - and in particular that IDA’s graph view is simply essential, I wouldn’t dream about going back to not using it when reverse engineering something. It’s just absolutely essential, no doubt about that.
• Tried Valgrind for the first time and it looks like a good tool to have around when fixing memory corruption problems. I recommend giving it a go if you have memory corruption problems in a software you’re developing!
• Tried DOSBox’s debugger for the first time and it looks also to be a useful piece of software, although I probably don’t know half of what can be done with it yet.
• That I’m not always as productive as I might like to, my effort changes with the tasks I’m doing. So knowing how to focus and get things done even though the task might not be to my current liking is something I haven’t yet truly mastered. I’ve heard that if you are completely present in a situation and totally focused on something then that something *becomes* interesting, whatever it may be. Now that’d be a lifesaving thing to know how to do - maybe some meditation (e.g. Zazen) would be in order?
• That I probably take my work too seriously at times: Getting something working felt great at times and not getting something working felt awful at other times. Or maybe it’s simply human nature to feel like that? Dunno. Anyway, I should probably know when to relax and let go, simply do and not measure my worth by what I get done but by the effort I put in regardless of the outcome (Yeah, quite the Gandhi talking :-)).

What’s so good about this reverse engineering thing anyway?
All in all this summer was a learning experience for me. What I found particularly interesting was the reverse engineering bit, taking a function in disassembly, looking at how big and intractable it looks and then starting on it, bit by bit, part by part making it more and more understandable until what you have in front of you is something you actually are able to understand. An example:
This is the graph of executePlayerInput function without any work done on it yet.
Here's how the function looks after quite a bit of abstracting done on it.
And here's the highest abstraction level graph that I made of the function.
It’s quite a bit of work, I tell you, but it’s quite riveting (There’s that focus thing again. I mean, just take a look at what Merriam-Webster says about the word riveting: “having the power to fix the attention”). Yeah!
What about the future?
I’m still committed to making Operation Stealth completable although it’ll be outside this GSoC. I’ll be fixing some of the Cine engine related bugs for the upcoming 0.12.0 release of ScummVM too.
Closing words
I’d like to congratulate everyone who made it to the end of Google Summer of Code 2008! It was an interesting ride, glad so many people with such diverse projects could get along for it. Thanks Google!
And last but not least thanks to my mentor Eugene for guiding me to a more productive direction in my work at times during this summer. His sane voice was the voice of practicality and, I imagine, experience.