Planet SoC -

Carcode GSOC

clsdaniel — Thu, 28 Aug 2008 02:02:00 +0200

I haven’t posted anything since Alpha 1 and currently we have Alpha 2 available, Beta 1 is around the corner (has been for quite a while).Google Summer of Code has ended, most features from my proposal where fulfilled, but I’m still working on it on my free time, altough now with school I don’t have as much time as in summer vacations but is free time anyways.I have updated the screenshots section from the carcode project page with latest work (beta 1):http://code.google.com/p/carcode/wiki/ScreenshotsCurrently I’m working on adding a few levels in order to have a Beta with something to work on, more levels will come and ofcourse, you can make suggestions, there is a wikipage on level design:http://code.google.com/p/carcode/wiki/LevelDesignFeel free to add and modify!Last but not so last, I’m Alive! and working on carcode! don’t worry :)

pyttk 0.2 released - August 25, 2008

gpolo — Mon, 25 Aug 2008 23:17:20 +0200

pyttk 0.2 has been released today. As before, you can pick its source at http://pypi.python.org/pypi/pyttk.

Besides bug fixes and some improvements in its usage, it finally brings a test suite! This is also the first release after GSoC finished.

Last week - August 11, 2008

gpolo — Mon, 11 Aug 2008 23:39:37 +0200

Hello again,

Today I’ve marked the project as complete, even though I haven’t touched its code since last wednesday (probably). The remaining goal was related to IDLE, for which I had a big patch that was sent to its maillist last monday and got no complaints (and no other feedback either) yet, and I’ve been using it here with all these changes without apparent problems. So, as I see, this GSoC project can be marked as complete now.

I hope to write more news here in the future, although it won’t be weekly news, and those will probably be related to updates in the ttk wrapper (changes caused by Tk 8.6, etc). Also, I promise you the news will have better titles/subtitles than the current ones.

Finally, thanks everyone :)

Wrapping it up

fredrikj — Thu, 07 Aug 2008 15:34:00 +0200

The final deadline of GSoC is approaching quickly. Fortunately, I’m almost finished with my project; the main part of the code was pushed in a few days ago, and in the coming days I only expect to fix bugs and write documentation.

Here are some examples of things that should now work with the hg version of SymPy:

Numerical differentiation and summation of generic infinite series is also on its way. The following works with my local copy:

>> Sum(1/n-log(1+1/n), (n, 1, oo)).evalf(50)
0.57721566490153286060651209008240243104215933593992>> Derivative(-gamma(x), x).evalf(50, subs={x:1})
0.57721566490153286060651209008240243104215933593992

The series is summed using the Euler-Maclaurin formula; in fact, the high-order derivatives are computed symbolically behind the scenes. (Because of the need for high-order derivatives, the Euler-Maclaurin formula makes much more sense in the context of a CAS than purely numerically.) This algorithm does have its drawbacks, so at some points pure extrapolation methods (which are already used for slowly convergent hypergeometric series) will be supported as well.

The use of an adaptive algorithm for evalf turns out to be especially useful for numerical differentiation, as one can just directly use a finite difference formula with a tiny step size, without worrying about cancellation error.

There is a lot more to be said about the details, but I’m leaving that for the to-be-written documentation.

PyPy 2.5-features - Starting to port the standard lib

brunogola — Wed, 06 Aug 2008 19:23:02 +0200

A little late I know.
This week I finished to fix all the failing tests and to implement PEP 328 (Absolute/Relative import).
Now it’s the final step for supporting Python2.5 on PyPy, to port the standard library.
I’m starting with it today and I really hope I can finish it all (including tests and everything else) until the end of next week.
Well, one more time, if anyone want to help me, test your python programs (specially the ones that use python2.5 specific features, like with statement, conditional expressions and new generator stuff) in PyPy-2.5-features (svn co http://codespeak.net/svn/pypy/2.5-features pypy-2.5-features).
How to test? Simple:
$ cd pypy-2.5-features $ ./bin/py.py your_python_program.py
Be careful and patient, it may take a little bit to things start happening
Of course you can translate PyPy using the translator if you want. Take a look at PyPy’s website for more information.

Week 15 - August 03, 2008

gpolo — Sun, 03 Aug 2008 19:18:16 +0200

Hey there,

Now this was a good week, yes. Everything I wanted to do regarding IDLE is done now, and except for the obvious bugs, and probably others that I will find, the GSoC project is complete. A screenshot can be found at http://code.google.com/p/python-ttk/wiki/Screenshots.

But.. while the week was great, I have to face the fact that the ttk wrapper won’t go into Python 2.6/3.0 since, I guess, it wasn’t considered important enough to be reviewed and accepted (the issue is still sitting alone at http://bugs.python.org/issue2983). So the ttk wrapper can be download at http://pypi.python.org/pypi/pyttk now.

That was it, good luck for everyone in their (suggested) last GSoC 2008 week.

Google Code Jam - Round 1

brunogola — Wed, 30 Jul 2008 16:53:40 +0200

Just to update my blog, let’s get ready to GCJ Round 2
Saturday 4pm UTC (it’s 1pm here in Brazil, right?!)!
Coming Soon: GSoC Status Update (I promise!!!)

Division: the sequel (with bonus material)

fredrikj — Mon, 28 Jul 2008 00:13:00 +0200

I have neglected to write a GSoC update for two weeks (where “two” was computed using round-to-floor), so let me first apologize for that.

In my last post, I wrote about how Newton’s method (coupled with a fast multiplication algorithm, such as Karatsuba multiplication) can be used for asymptotically fast division of long integers. I have now fixed up the code to compute a correctly rounded quotient. In fact, it now performs the full divmod operation, returning both the quotient and a correct remainder.

The trick is to perform an extra exact multiplication at the end to determine the remainder. By definition, the quotient r of p/q is correct if the remainder m = p - r·q satisfies 0 ≤ m < q. If this inequality does not hold, one need only perform an additional divmod operation (which can be performed using standard long division, since the error will almost certainly fit in a single limb) to correct both the quotient and the remainder.

The extra multiplication causes some slowdown, but it’s not too bad. The new idivmod function still breaks even with the builtin divmod somewhere around 1000-2000 digits and is 10 times faster at half a million digits (i.e. when dividing a million digit number by a half-million digit number):


>> timedivision(int, 2.0, 20)
      size     old time     new time       faster  correct
------------------------------------------------------------
        16     0.000008     0.000052     0.155080     True
        32     0.000005     0.000052     0.102151     True
        64     0.000008     0.000059     0.132075     True
       128     0.000013     0.000070     0.190476     True
       256     0.000035     0.000107     0.325521     True
       512     0.000126     0.000215     0.583658     True
      1024     0.000431     0.000532     0.810399     True
      2048     0.001855     0.001552     1.195104     True
      4096     0.007154     0.005050     1.416708     True
      8192     0.028505     0.015449     1.845033     True
     16384     0.111193     0.046938     2.368925     True
     32768     0.443435     0.142551     3.110706     True
     65536     1.778292     0.432412     4.112497     True
    131072     7.110184     1.305771     5.445200     True
    262144    28.596926     3.919399     7.296253     True
    524288   116.069764    11.804032     9.833061     True

As a bonus, a fast divmod also provides a fast way to convert long integers to decimal strings, by recursively splitting n in half using L, R = divmod(n, 10b/2) where b is the number of digits in n:

>> timestr(int, 20)
      size     old time     new time       faster  correct
------------------------------------------------------------
        16     0.000005     0.000013     0.413043     True
        32     0.000006     0.000009     0.588235     True
        64     0.000008     0.000012     0.674419     True
       128     0.000020     0.000023     0.865854     True
       256     0.000059     0.000133     0.442105     True
       512     0.000204     0.000333     0.613255     True
      1024     0.000895     0.001194     0.749708     True
      2048     0.003505     0.002252     1.556824     True
      4096     0.013645     0.006600     2.067429     True
      8192     0.052386     0.018334     2.857358     True
     16384     0.209164     0.052233     4.004412     True
     32768     0.834201     0.153238     5.443827     True
     65536     3.339629     0.450897     7.406639     True
    131072    13.372223     1.339044     9.986392     True
    262144    53.547894     3.998352    13.392491     True
    524288   214.847486    11.966933    17.953429     True

So printing an integer with half a million digits takes 12 seconds instead of 3.5 minutes. This can be quite a usability improvement.

The code can be found in this file: div.py.

I have now submitted a request for these algorithms to be implemented in the Python core. Since the pure Python implementation is very simple, I don’t think porting it to C would be all that difficult. By “request”, I mean that I might eventually do it myself, but there are many others who are much more familiar with both the Python codebase and the C language, and if any of those persons happens to have a few free hours, they could certainly do it both faster and better. If you are interested in helping out with this, please post to the issue tracker.

The division code is just one of several small projects I’ve been working on lately. Basically, I’ve found that there are some arithmetic functions that are needed very frequently in all kinds of mathematical code. These include:

Pure convenience functions like sign, product, …

Bit counting

Radix conversion

Integer square roots

Rational (and complex rational) arithmetic

Integer factorization, multiplicity tests, gcd, etc

Factorials, binomial coefficients, etc

Such utility functions are currently scattered throughout mpmath, SymPy and SympyCore codebases. Many of them are hack-ish, duplicates, and/or don’t always work/have very poor worst-case performance. Right now, I’m trying to collect them into a single module and optimizing / strain-hardening them.

The current result of this effort can be found here (be aware that it’s very much a work in progress). Among other things, it includes the mentioned fast division code, fast square root code based on my implementation from mpmath, much needed improvements to the nth root and integer factorization code I wrote for SymPy, plus the extremely fast multinomial coefficient code optimized by Pearu and myself for SympyCore (which, if I correctly recall the results of previous benchmarking, makes polynomial powering in SympyCore faster than Singular).

My plan is to split this off to a standalone project, as it could be useful to other people as such, but keeping it a single, self-contained .py file that is easy to include in mpmath and SymPy. There won’t be too much feature creep (hopefully); the advanced number theory functions in SymPy won’t move, nor will the floating-point functions from mpmath.

Finally, I have done a bit more work on the adaptive numerical evaluation code for SymPy. The main new features are support for converting approximate zeros to exact zeros (thereby preventing the algorithm from hanging when it encounters a nonsimplified zero expression, and overall prettifying output), and support for logarithms and powers/exponentials of complex numbers. See evalf.py and test_evalf.py. I’ve also done miscellaneous other work on SymPy, such as patching the existing evalf code to use mpmath and getting rid of the global precision.

Week 13 - July 20, 2008

gpolo — Sun, 20 Jul 2008 23:14:22 +0200

So I used this week to read some of the IDLE’s code and started adding support for tabs. While I have added a notebook, it barely works with all the functionality provided by IDLE, and, introduced several bugs at the same time. This week was also useful for me in the sense that now I know I underestimated the time necessary for adding this feature, but I believe it can be done in GSoC time and it is very likely I will be working on this till the end of GSoC, given the other parts of the project are done (or almost done).

Lastly, if you happen to be interested in getting this feature into IDLE you may want to help me testing my custom idlelib, or something like that, so I can get some more motivation :)

PyPy 2.5-features - Yet another status update

brunogola — Fri, 18 Jul 2008 05:27:29 +0200

Here comes another GSoC status update
Some finished tasks:

throw() method on generators
close() method…
faking 2.5 behavior for IMPORT_NAME and IMPORT_FROM opcodes
changing the default value for magic attribute of PyCode objects
changing the magic number that goes in .pyc files compiled by PyPy
fixing tests and more tests…

From those changes the only one I would like to comment is the change of both magic numbers.
First, they have different values and meanings in PyPy. The magic number for .pyc files (defined in pypy/modules/__builtin__/importing.py) is the number that identifies the bytecode “version”. It’s used to know if the interpreter should use the .pyc file or should recompile the .py. PyPy’s value is different from CPython’s one. As we are changing some opcodes (as IMPORT_NAME, mentioned before) this number had to be changed. The old value in PyPy was 1024 (or 1024 + 2 or 1024 + 4 or 1024 + 2 + 4, depending on some command line options), the new value is 1034 (or ….). We are just using the same policy CPython uses to change the value, add 10 to the old value.
Now the PyCode magic attribute. This value is the CPython magic number (the one explained above)., the old default value was the value from CPython 2.4 (62021). Some checks against this value are made through the code to decide if the bytecode should be interpreted one way or another. One example is the IMPORT_NAME opcode. In Python 2.4 IMPORT_NAME did not have the level parameter, this parameter is new in 2.5 because of the absolute import feature. So if the bytecode represented by a PyCode object is 2.4, when we visit a import statement we should not try to pop the level value from the stack (because it’s not there), but if it is 2.5 we should! So we check the magic value. The problem is, the default value was 2.4 but we changed the opcodes to behave like 2.5, so those checks were not working. The solution was to change the magic default value to 63231 (2.5c2 value I think), so now our bytecode interpreter is (almost?) compatible with 2.5 bytecode. And our compiler is generating 2.5 compatible bytecode as well.