TIC4005 project 1 – Image processing: implementation and analysis

Abstract

Develop, and evaluate, a small set of original image processing functions, which run both on a GPU and CPU platform, in a browser. You will be doing this (only) in a subset of Javascript, using GPU.js (GPU accelerated Javascript https://gpu.rocks/). In the assessment of your program, marks will be given for clarity, reflection subtlety, complexity, and for your use of any specific techniques to improve the behavior of your program (beyond the obvious one-pixel-per-thread in the demo program given to you). Your code is likely to be no more than 150-1500 commented lines.

1 Project outline

You are to develop GPU.js-based implementations of some image processing functions, with well-commented source code. Your program/web page should demonstrate your image processing functions. Here are some possibilities – but it is up to you to do one (or more) of the following options:

1. Crossfading as

2.A pipeline where your main page allows you to select a few simple kernel filters such as seen in Lab01 and see the result of doing them in a particular sequence.

3.A heatmap a bit like (but in this case triggered from other measurable things in your video image).

4.A laser tracker with artifact removal, and image manipulation, based on the tracker.

5. Edge detection of various kinds, or low pass and high pass filtering and so on1

6. Image recognition perhaps a colored-ball finder and tracker (i.e. draws a box around a colored ball) or similar.

7. Your own idea. Your main program should allow the user to try out your kernel(s) on a video stream, displaying the result in a web browser, and allowing easy comparison with the CPU version, as in the sample demos from Lab01.

The sample programs have buttons to switch between CPU and GPU, to display the before-filtered image.

It also displays an approximate number of frames per second . It is perhaps best to use Chrome, as your

programs will be tested using it. Your program may look nicer, and have extra options, but this task is not about the look of your web page, it is about the parallelized components of your program.

You can grab the sources. The core part is that GPU.js supplies a restricted mechanism to define functions that compute a single float (32 bit) value (i.e. – a pixel). This mechanism is supplied options:

1.If you call the mechanism with the option “ mode: cpu”, you get a function that will run on a CPU. With “mode: gpu”, you get a function that runs multiple threads on the GPU (akernel).

2. Another option defines a target matrix into which results are to be placed. If you call the mechanism with the option “output: [800,600]”, you get a function that will iterate over all elements of an 800×600 matrix if run on the CPU. If run on the GPU, each thread operates on a single element (of the whole matrix), with coordinates (this.thread.x,this.thread.y).

3.A third option specifies if you are going to be treating the output matrix as an image, in which case you should set “graphical: true”. In the example given, the nameless function inside gpu.createKernel(function,options) pro- vides this mechanism. The function definition looks like Javascript but instead is a restricted subset of Javascript, limited by the WebGL transpiler mechanism used.

1.1 Properties of GPU.js

GPU.js was (originally) developed at NUS during Hack&Roll in 2017, and so there is a fair amount of local knowledge about it. However, these days it is maintained by a large group and is finding particular application in the AI community. The source is freely available from the website https://gpu.rocks/.
The subset of Javascript used by GPU.js has some restrictions that are a little confusing, beginning with the restriction that the data types in the kernel functions can only be float32 (a restriction imposed by the OpenGL shader architecture). In the example, float32 arrays are fed to the functionDef, and the output is a 2D float32 matrix, which we treat as an rgba image. Loop bounds must be fixed at compile time, so the constant option helps you import constant values into your GPU function. There is no recursion. (!)
… GPUs generally do not do recursion.

The functions you can call within a kernel, can only be ones written in the restricted subset allowed by GPU.js, and a mechanism is provided to attach these functions to the kernel, to help you structure your program.

1.2 Completion?

The specific function(s) you implement has been left undefined. In general, the more, or more complex, image manipulation functions you try, the better. In your project paper presentation, you should clearly state at the beginning which function(s) you implemented. You should comment on the speedup (or otherwise) you get for each function you decide to implement.

Whatever you pick, there is an almost endless list of possible things to do, or not do:

•Image processing filters, blurring, sharpening, highlighting, antialiasing, and so on.

•Did you do anything special for accuracy? Speedup? Workload reduction?
•Can you adjust the parameters to specify different methods of operation?

Your documentation should include a tabulation and brief discussion of your results, along with a discussion about any machine and language-specific aspects that may affect these results.