Tuesday, May 27, 2014

Escaping the GIL with WebWorkers


The Global Interpreter Lock (GIL) prevents threads in Python from using all cores for CPU bound tasks, and using more threads can even make your program slower. These limitations of the GIL can be bypassed with the faking threading module in PythonJS. The fake threading module provides a function threading.start_webworker(f,args) that will run the function f with args in a new WebWorker. If you pass a list or dict as arguments to the worker function they will become shared between the parent and worker process (each append operation on a list will call self.postMessage to keep the parent and child in sync).

The following benchmark measures how many prime numbers can be found per-second. These tests were done with a dual core CPU, NodeJS 0.10.22, and workerjs. The dual core version of the script is 2X faster than the single core version when translated by PythonJS.

prime numbers per-second (single core)

https://github.com/PythonJS/PythonJS/blob/master/regtests/bench/webworker_single.py

prime numbers per-second (dual core)

https://github.com/PythonJS/PythonJS/blob/master/regtests/bench/webworker.py


sleep(T)

If you looked at the source code for the benchmarks above you might have been surprised to see the sleep function imported from the time module. I recently added experimental support for sleep to PythonJS in these commits: [1], [2]. This allows you to write code in a simpler synchronous style, and behind the scenes the code will be broken down into blocks and async callbacks generated.

python input

from time import sleep

def main():
 sleep(0.01)
 a = []
 sleep(0.1)
 a.append(1)
 sleep(0.1)
 a.append(2)

 TestError( len(a)==2 )

javascript output

main = function() {
  var a;
  __run__ = true;
    var __callback0 = function() {
    a = [];
    __run__ = true;
        var __callback1 = function() {
      a.append(1);
      __run__ = true;
            var __callback2 = function() {
        a.append(2);
        TestError("sleep.py",10,len(a)==2,"len(a)==2");
      }

      if (__run__) {
        setTimeout(__callback2, 100.0);
      } else {
        if (__continue__) {
          setTimeout(__callback3, 100.0);
        }
      }
    }

    if (__run__) {
      setTimeout(__callback1, 100.0);
    } else {
      if (__continue__) {
        setTimeout(__callback2, 100.0);
      }
    }
  }

  if (__run__) {
    setTimeout(__callback0, 10.0);
  } else {
    if (__continue__) {
      setTimeout(__callback1, 10.0);
    }
  }
}

Sunday, May 18, 2014

PythonJS now faster than CPython


pystone benchmark

PythonJS using the dart backend is 6x faster than CPython.

pystone.py

nbody benchmark

PythonJS using the fast javascript backend is faster than CPython and even 2x faster than PyPy.

nbody.py

richard's benchmark

PythonJS using the dart backend is 7x faster than CPython.

richards.py

float benchmark

PythonJS using the dart backend is 3x faster than CPython.

float.py


Micro-Benchmarks

recursive fibonacci micro-benchmark

PythonJS using the dart backend is 27x faster than CPython and 7x faster than PyPy.

recursive_fib.py

simple add loop micro-benchmark

PythonJS using any backend, is about 30x faster than CPython.

add.py


The benchmarks were run with node.js 0.10.22, pypy 1.9, and dart-sdk 1.0. To run them yourself: get the latest PythonJS source from github, and download and extract the dart-sdk to ~/dart-sdk-1.0. The benchmark eps graphs are written to "/tmp"

cd PythonJS/regtests
./run.py