Sunday, September 7, 2014

Python to Go


Golang is a fully typed language with simple and clean syntax. The Go compiler will throw an error if you made a typo or called a method on the wrong object. This means you need to worry less about runtime errors, and writting unit tests. Go programs are compiled to native executables, or can be compiled to JavaScript using GopherJS.

PythonJS has a new backend in development that translates to Go. See the documentation here. The Go backend supports translation of Python classes and list comprehensions to Go. The PythonJS syntax has also been extended to support Go features like: go-routines and channels.

Go/Python Meetup

The Barcelona Python users group is hosting a free meetup on Oct 13th, see the details here. I will be going and giving a talk about using PythonJS to translate to Go, and using GopherJS to translate it back to JavaScript.

Tuesday, July 22, 2014

new pythonjs syntax


PythonJS is being extended with optional static typing and new syntax. The translator pre-processes input source code and transforms it into an intermediate form that can be parsed by Python's ast module. This simplifies adding new syntax to PythonJS, see this commit that implements Exception Expressions.

a = d[ 'somekey' ] except KeyError: 'mydefault'

inline anonymous functions

I am now researching new syntax that can be added to PythonJS. After looking at RapydScript, I see that inline anonymous functions are a must have, and recently implemented them in PythonJS. Inline functions can be used in function calls as a named keyword argument, or in a dict literal. This solves the problem that Lambda functions can only be a single statement, which is often not that useful.


a.func(
 callback1=def (x,y,z):
  x += y
  return x - z,
 callback2= def (x, y):
  return x * y
)

b = {
 'cb1' : def (x):
  return x,
 'cb2' : def (y):
  return y
}

switch


switch a==b:
  case True:
    pass
  case False:
    pass
  default:
    pass

Go channels and select

def send_data( A:chan int, B:chan int, X:int, Y:int):
 while True:
  print('sending data..')
  A <- X
  B <- Y

def select_loop(A:chan int, B:chan int, W:chan int) -> int:
 print('starting select loop')
 y = 0
 while True:
  print('select loop:',y)
  select:
   case x = <- A:
    y += x
    W <- y
   case x = <- B:
    y += x
    W <- y
 print('end select loop', y)
 return y

def main():
 a = go.channel(int)
 b = go.channel(int)
 w = go.channel(int)

 go(
  select_loop(a,b, w)
 )


 go(
  send_data(a,b, 5, 10)
 )

 z = 0
 while z < 100:
  z = <- w
  print('main loop', z)

 print('end test')

Thursday, July 17, 2014

WebGL CSS3D Hybrid part2


Despite WebGL having been around since 2011, you probably do not run into many websites with 3D. And when you do, its likely a fullscreen game or demo, not a typical website. Traditional web technologies do not mix well with WebGL. Libraries like Voodoo.js try to bridge this gap by allowing you to insert 3D objects inside a normal webpage. Voodoo.js graphics are not anti-aliased when rendering with WebGL, the contrast with the rest of the smooth text and graphics on a webpage then becomes very harsh and downright ugly.

Threepy is an experimental library built on THREE.js that allows some HTML and CSS to be mixed together with WebGL rendering and post-processing FX. The library wraps many standard HTML element types with special hacks like: callbacks, catching events, and rendering to proxy clones. Not every combination of HTML and CSS is compatible with this method. The video below tests the basic UI elements: checkboxes, text input, select drop down lists, number input, color selector, tab menus, slider controls, images and videos. Works with GoogleChrome and NodeWebkit.

Monday, July 7, 2014

WebGL CSS3D Hybrid


Jerome Etienne has an interesting post where he combines CSS3D and WebGL, see his post here. However, his method is not compatible with Three.js post-processing shaders that is required to blur and perform other 2D effects on the 3D rendering.

The videos below show integration of WebGL and CSS3D with post-processing in THREE.js. The main hack do to this requires double rendering the DOM in two CSS3D layers. The first layer is rendered under the WebGL layer, and is a clone of the top CSS layer. The top CSS layer is rendered transparent over the WebGL layer and catches keyboard and mouse events. The WebGL layer is rendered with alpha blending. See the source code here.

dnd, iframes, video, collada

Update july10th, testing new hacks to integrate: iframes, html5 video, and collada 3D files by drag and drop. Starting new module dddom.py with simple API for creating 3D widgets mixing WebGL and CSS3D.

Tuesday, June 24, 2014

GLSL Classes


GLSL is not object oriented, there are no classes or method calls. GLSL4.0 has subroutine variables that can provide an object oriented solution, but this is not compatible with GLSL in WebGL.

PythonJS GPU Class

In the example below an array of MyObject is created and uploaded to the GPU, where it is iterated over. The method call s.mymethod(1.1, 2.2) translates to MyObject_mymethod(s, 1.1, 2.2) in the GLSL shader.

@gpu.object
class MyObject:
 @gpu.method
 float def subroutine(self, x,y):
  float x
  float y
  return x + y * self.attr2

 @gpu.method
 float def mymethod(self, x,y):
  float x
  float y
  if self.index == 0:
   return -20.5
  elif self.index == 0:
   return 0.6
  else:
   return self.subroutine(x,y) * self.attr1

 def __init__(self, a, b, i):
  self.attr1 = a
  self.attr2 = b
  self.index = int16(i)


class myclass:
 def run(self, w):
  self.array = [MyObject(1.1,1.2,x) for x in range(w)]

  @returns( array=64 )
  @gpu.main
  def gpufunc():
   struct* A = self.array
   float b = 0.0

   for s in iter(A):
    b += s.mymethod(1.1, 2.2)

   return b

  return gpufunc()

Sunday, June 22, 2014

GLSL Array of Arrays


WebGL shader code written in GLSL is based on the OpenGL-ES standard, see GLSL 1.2 spec pdf. GLSL 1.2 has no support for multidimensional arrays (array of arrays), and loops require a constant expression for iteration. These limitations make it very hard to write generic shader programs.

PythonJS shader translation provides a workaround, and supports one-level-deep array of arrays and iteration over them. The input array data and sizes can change at runtime because the shader is fully recompiled each call to its wrapper function. Attributes from the current scope in JavaScript can also be inlined into the shader. Read the syntax documentation here.

array of array example

class myclass:
 def __init__(self, s):
  self.s = s
 def my_method(self):
  return self.s

 def run(self, w, h):
  self.array = [ [x*y*0.5 for y in range(h)] for x in range(w) ]

  @returns( array=64 )
  @gpu.main
  def gpufunc():
   float* A = self.array
   float b = self.my_method()

   for subarray in A:
    for j in range( len(self.array[0]) ):
     b += subarray[j]
   return b

  return gpufunc()

GLSL output

The inner function gpufunc becomes main below. Inside gpufunc above, the assignment to A as a float pointer float* A = self.array triggers the wrapper code to unroll A into A_𝑛 and inline the values, '𝑛' is the length of the array.

 void main() {
 float A_0[4];
 A_0[0]=0.0;A_0[1]=0.0;A_0[2]=0.0;A_0[3]=0.0;float A_1[4];A_1[0]=0.0;A_1[1]=0.5;A_1[2]=1.0;A_1[3]=1.5;float A_2[4];A_2[0]=0.0;A_2[1]=1.0;A_2[2]=2.0;A_2[3]=3.0;float A_3[4];A_3[0]=0.0;A_3[1]=1.5;A_3[2]=3.0;A_3[3]=4.5;float A_4[4];A_4[0]=0.0;A_4[1]=2.0;A_4[2]=4.0;A_4[3]=6.0;float A_5[4];A_5[0]=0.0;A_5[1]=2.5;A_5[2]=5.0;A_5[3]=7.5;float A_6[4];A_6[0]=0.0;A_6[1]=3.0;A_6[2]=6.0;A_6[3]=9.0;float A_7[4];A_7[0]=0.0;A_7[1]=3.5;A_7[2]=7.0;A_7[3]=10.5;
 ...

the wrapper inlines the runtime value of float b = self.my_method(). Iteration over the list for subarray in A: is translated into a for loop that copies the data from A_𝑛 into the iterator target subarray.

    float b;
 b = 0.1;
 for (int _iter=0; _iter < 4; _iter++) {
  float subarray[4];
  if (_iter==0) { for (int _J=0; _J<4; _J++) {subarray[_J] = A_0[_J];} }
  if (_iter==1) { for (int _J=0; _J<4; _J++) {subarray[_J] = A_1[_J];} }
  if (_iter==2) { for (int _J=0; _J<4; _J++) {subarray[_J] = A_2[_J];} }
  if (_iter==3) { for (int _J=0; _J<4; _J++) {subarray[_J] = A_3[_J];} }
  for (int j=0; j < 4; j++) {
   b += subarray[j];
  }
 }
    out_float = b;
 }

array of structs example

To iterate over an array of structs, wrap the struct* with iter in a for loop. The struct below contains a float number 'num' and array of floats 'arr'. The nested loop iterates over the indices of the structs array 'arr'.

class myclass:
 def new_struct(self, g):
  return {
   'num' : g,
   'arr' : [0.1 for s in range(6)]
  }

 def run(self, w):
  self.array = [ self.new_struct( x ) for x in range(w) ]
  @returns( array=64 )
  @gpu.main
  def gpufunc():
   struct* A = self.array
   float b = 0.0
   for s in iter(A):
    b += s.num
    for i in range(len(s.arr)):
     b += s.arr[i]
   return b
  return gpufunc()

GLSL output

The assignment struct* A = self.array triggers the wrapper code to generate a struct typedef that is inserted into the shader header. self.array is inlined at runtime as 𝙎𝙩𝙧𝙪𝙘𝙩𝙉𝙖𝙢𝙚 A_𝑛 Before the struct is constructed the array attribute arr is assigned to the variable _arrA_𝑛. The for loop switches the iterator target s based on the loop index 𝑛.

void main( ) {
    float b;
 b=0.0;

 float _arrA_0[6];_arrA_0[0]=0.1;_arrA_0[1]=0.1;_arrA_0[2]=0.1;_arrA_0[3]=0.1;_arrA_0[4]=0.1;_arrA_0[5]=0.1;
 𝙎𝙩𝙧𝙪𝙘𝙩𝙉𝙖𝙢𝙚 A_0 = 𝙎𝙩𝙧𝙪𝙘𝙩𝙉𝙖𝙢𝙚(0.0,_arrA_0);
 ...
 𝙎𝙩𝙧𝙪𝙘𝙩𝙉𝙖𝙢𝙚 A_6 = 𝙎𝙩𝙧𝙪𝙘𝙩𝙉𝙖𝙢𝙚(6.0,_arrA_6);
  float _arrA_7[6];_arrA_7[0]=0.1;_arrA_7[1]=0.1;_arrA_7[2]=0.1;_arrA_7[3]=0.1;_arrA_7[4]=0.1;_arrA_7[5]=0.1;
 𝙎𝙩𝙧𝙪𝙘𝙩𝙉𝙖𝙢𝙚 A_7 = 𝙎𝙩𝙧𝙪𝙘𝙩𝙉𝙖𝙢𝙚(7.0,_arrA_7);

 for (int _iter=0; _iter < 8; _iter++) {
  𝙎𝙩𝙧𝙪𝙘𝙩𝙉𝙖𝙢𝙚 s;
  if (_iter==0) { s=A_0;}
  if (_iter==1) { s=A_1;}
  if (_iter==2) { s=A_2;}
  if (_iter==3) { s=A_3;}
  if (_iter==4) { s=A_4;}
  if (_iter==5) { s=A_5;}
  if (_iter==6) { s=A_6;}
  if (_iter==7) { s=A_7;}
  b += s.num;
  for (int i=0; i < 6; i++) {
      b += s.arr[i];
     }
    }
    out_float = b;
  }

Wednesday, June 18, 2014

PythonJS GPU Mandelbrot


PythonJS now supports translation of a limited subset of Python syntax into GPU code. This is done with a new GLSL backend using the WebCLGL library by Roberto Gonzalez. The above benchmark calculates the Mandelbrot set for a size of 512x512, CPython3 with NumPy takes over eight seconds to complete, PythonJS specialized version takes just 0.2 seconds.

source code: mandelbrot.py

40X faster is probably not the best that WebCLGL can do, this benchmark was performed with a low-end GPU. To run the benchmarks yourself, install Python2 and Python3, and extract the latest PyPy to your home directory, and install NumPy for each. Install the NodeWebkit NPM package. Then run these commands, and it will save the file /tmp/mandelbrot.py.eps:

cd
git clone https://github.com/3DRoberto/webclgl.git
git clone https://github.com/PythonJS/PythonJS.git
cd PythonJS/regtests
./run ./bench/mandelbrot.py