Escaping the GIL with WebWorkers

The Global Interpreter Lock (GIL) prevents threads in Python from using all cores for CPU bound tasks, and using more threads can even make your program slower. These limitations of the GIL can be bypassed with the faking threading module in PythonJS. The fake threading module provides a function threading.start_webworker(f,args) that will run the function f with args in a new WebWorker. If you pass a list or dict as arguments to the worker function they will become shared between the parent and worker process (each append operation on a list will call self.postMessage to keep the parent and child in sync).

The following benchmark measures how many prime numbers can be found per-second. These tests were done with a dual core CPU, NodeJS 0.10.22, and workerjs. The dual core version of the script is 2X faster than the single core version when translated by PythonJS.

prime numbers per-second (single core)

https://github.com/PythonJS/PythonJS/blob/master/regtests/bench/webworker_single.py

prime numbers per-second (dual core)

https://github.com/PythonJS/PythonJS/blob/master/regtests/bench/webworker.py

---

sleep(T)

If you looked at the source code for the benchmarks above you might have been surprised to see the sleep function imported from the time module. I recently added experimental support for sleep to PythonJS in these commits: [1], [2]. This allows you to write code in a simpler synchronous style, and behind the scenes the code will be broken down into blocks and async callbacks generated.

python input

from time import sleep

def main():
 sleep(0.01)
 a = []
 sleep(0.1)
 a.append(1)
 sleep(0.1)
 a.append(2)

 TestError( len(a)==2 )

javascript output

main = function() {
  var a;
  __run__ = true;
    var __callback0 = function() {
    a = [];
    __run__ = true;
        var __callback1 = function() {
      a.append(1);
      __run__ = true;
            var __callback2 = function() {
        a.append(2);
        TestError("sleep.py",10,len(a)==2,"len(a)==2");
      }

      if (__run__) {
        setTimeout(__callback2, 100.0);
      } else {
        if (__continue__) {
          setTimeout(__callback3, 100.0);
        }
      }
    }

    if (__run__) {
      setTimeout(__callback1, 100.0);
    } else {
      if (__continue__) {
        setTimeout(__callback2, 100.0);
      }
    }
  }

  if (__run__) {
    setTimeout(__callback0, 10.0);
  } else {
    if (__continue__) {
      setTimeout(__callback1, 10.0);
    }
  }
}