Welcome to CS 461 :) I'm glad you're here and I'm excited to show you some techniques you can use to build interactive demos, visualizations, animations or games. We will cover several topics in this course, ranging from geometric modeling, ray-tracing, rasterization and animation. By the end of the course, you will be able to:

1. develop your own ray tracer to render complex scenes,
2. display and manipulate three-dimensional models using rasterization (with WebGL),
3. animate three-dimensional objects and physical systems.

Those are the three main themes: ray tracing, rasterization (with WebGL) and animation. I will post notes (like the notes on this page) to our course website, which will also contain interactive demos within which you can modify some small code snippet to understand the concepts in that reading.

The Goal of Computer Graphics: Setting Pixel Colors

That's it, really. We're going to have a grid of pixels to represent a picture (we'll call this a frame) and our job is to assign the color of each pixel. Well, that's not quite the end of the story. We need to do this very quickly and, in some situations, we might be limited by how much memory we can use to do this.

Let's take a step back first. What's a pixel anyway? A pixel is a "PIcture ELement" and it's the main building block of the pictures you see in your everyday life, like the pictures you take on your phone.

Consider the 8-bit image of the earth at the top-left of this page (originally from here) in which we can see the individual pixels. How many total colors are used here? I count four: black for the background, white for the stars, green for land and blue for the ocean.

There are a few important things we need to consider when assigning pixel colors in an image:

1. What is the size of the image? In other words, how many pixels are there along the width and height of the picture? The more pixels you have, the more memory your image will take up, and the more work your computer will do to process your image.
2. How do we represent the color assigned to each pixel? In general, we will use a red-green-blue (RGB) triplet to represent the color. In some cases each channel (red, green or blue) will have an integer value from 0-255 but in others we will assign floating-point values between 0-1.
3. What is the coordinate system of the image, i.e. what are the directions of the x- (width) and y- (height) axes? We will always use a coordinate system in which x- goes from left to right. However, some graphics systems have the y-axis pointing downwards.

Check out the following demo. On the left, you should see a rotatable and zoomable 3d model of a cow (a common model called "Spot"). Each point on the surface of Spot is translated to a pixel and, again, our job is to assign a color to that pixel. Later in the course, we will study WebGL, which will consist of writing shaders to programatically assign these pixel colors. For now, the pixel color is set to grey (0.5, 0.5, 0.5). Actually, you might notice there is a light that makes some points look brighter, but we'll talk about this in a few weeks when we talk about shading. The base modelColor is grey. Exercise 1: try adjusting the RGB values of the modelColor.

Okay, let's make this more interesting and add a flannel pattern. Exercise 2: try typing the following mathematical expression given the coordinates on the surface of Spot (x, y and z). Feel free to change the 100.0 and also the 3d model using the dropdown below!

if (sin(100.0 * x) < 0.0 && sin(100.0 * y) < 0.0 && sin(100.0 * z) < 0.0)
    modelColor = vec3(.4, 0, 0);
else
    modelColor = vec3(0.1, 0.1, 0.1);

Also, what happens if you change one of the 0.0 in the conditional to 0?

Frame Rate

Let's investigate a bit further. Here is a really nice demo of a fluid simulation. Click and drag to simulate the fluid! What do you notice if you increase the "resolution" setting to 1024? 4096? 8192? Have a look at the FPS display at the top-left.

https://philipclaude.github.io/webgl-fluids-demo/

Please note: I did not write this really nice demo! I just modified some of the resolution settings. The original code can be found here.

So let's restate our goal. We want to set the color of every pixel in a frame at some appropriate framerate. The framerate is measured in frames per second and often abbreviated FPS. The human eye can typically see between 30-60 FPS. Anything below that and the animation/simulation/interaction will feel "off".

JavaScript

JavaScript may look familiar if you've programmed in Java, C or C++, in terms of the use of semi-colons, curly braces and the syntax of if-statements and for loops. Everything in JavaScript is an object (numbers, strings, classes, etc.). There are some amazing online resources for learning JavaScript - for an in-depth guide, please have a look at Eloquent JavaScript or w3schools.

There's often more than one way of doing something in JavaScript, and our use of the language will be quite basic. AirBnB's style guide is a good resource for learning the dos and don'ts of JavaScript (which we will mostly follow). You'll be able to pick up JavaScript along the way, but be sure to review things like:

• comments
• strings
• functions
• arrays
• iteration
• control structures
• blocks
• naming conventions: camelCase for naming objects (variables, functions) and PascalCase for class declarations.

To avoid bugs, I recommend the following:

• always terminate lines with semicolons (;) - please just do it!
• use === to test for equality instead of == (for example, 5 == '5' is True but 5 === '5' is False).
• always declare variables using either let or const (e.g. let x = 5;) instead of var (e.g. var x = 5;) or as x = 5;). Otherwise, you will be hoisting your variables to the global namespace, and may cause a lot of bugs. See here for more info and examples.

As you read through these lecture notes, I will embed mini exercises that will prompt you to either modify some JavaScript code, or even one of the shaders (written in GLSL), as in the demo above. However, especially when you work on your labs, you should be familiar with a few other ways to develop in JavaScript code. First (and probably the most convenient), you can use the JavaScript console in your browser (Chrome: View -> Developer -> JavaScript console, Firefox: Tools -> Web Developer -> Web Console, Safari: Develop -> Show JavaScript Console). When you open the "console", you will face a prompt which looks like >; You can use the console to create objects, perform calculations - just like you would with a Python interpreter.

Examples

Here are some examples of basic JavaScript code syntax. Please open the JavaScript console and directly type the examples below (you can copy-paste some of the longer examples). The comments below the expressions demonstrate what the output should be when printing the variable.

Variables and Printing

let x = 2; // declare a variable
x = 3; // change the value

const y = 8; // declare a const variable
// y = 9; // <-- ERROR!

// printing
console.log('x + y = ', x + y);  // method 1
console.log(`x + y = ${x + y}`); // method 2 with template literals

Arrays, Loops and Conditionals

let a = new Array(10); // an array with 10 elements
for (let i = 0; i < a.length; ++i) {
  if (i % 2) a[i] = i + 1;
  else a[i] = 0;
}
// a = [0, 2, 0, 4, 0, 6, 0, 8, 0, 10]

a.push(1); // add an eleventh element
// a = [0, 2, 0, 4, 0, 6, 0, 8, 0, 10, 1]

// ternary conditional operator
const b = (a.length % 5) ? 34 : 7;
// b = 7

// another way to create an array
let x = [];
x.push(5);
x.push(8);
// x = [5, 8]

Functions

There are different ways to declare functions. The first method below (regular function) is hoisted and is generally not recommended. AirBnB recommends either using arrow functions or named function expressions - the latter have an advantage over anonymous function expressions when examining the strack trace of a program (e.g. when debugging). Arrow functions are preferred when assigning callbacks and the function needs to be anonymous (as in the last example).

// regular function (hoisted)
function add1(x, y) {
  return x + y;
}

// anonymous function expression (not hoisted)
let add2 = function(x, y) {
  return x + y;
};

// named function expression (not hoisted)
let add3 = function addsTwoNumbers(x, y) {
  return x + y;
}

// arrow functions (not hoisted)
let add4 = (x, y) => {
  return x + y;
};

console.log(add1(1, 2)); // 3
console.log(add2(2, 3)); // 5
console.log(add3(3, 4)); // 7
console.log(add4(4, 5)); // 9

// setting the callback when a key is pressed for the web page
window.addEventListener('keydown', (event) => {
  console.log(event.key, ' was pressed!');
});
// then click on the main page to bring it into focus and press some keys!

Classes and Objects

class Pixel {
  constructor(r, g, b) {
    this.r = r;
    this.g = g;
    this.b = b;
  }

  scale(a) {
    this.r *= a;
    this.g *= a;
    this.b *= a;
  }
}

let p = new Pixel(0.5, 0.5, 0.5); // create a pixel object
// p.r = 0.5, p.y = 0.5, p.z = 0.5

p.scale(255);
// p.r = 127.5, p.y = 127.5, p.z = 127.5

The scale method was defined directly in the class definition. Alternatively, we can define methods outside of the class using prototypes (although it isn't recommended by AirBnB):

Pixel.prototype.set = function(r, g, b) {
  this.r = r;
  this.g = g;
  this.b = b;
}
p.set(1, 0, 0);
// p.r = 1, p.g = 0, p.b = 0

Also note that objects can be created directly using JSON:

const canvasInfo = {
  'id': 'myHtmlCanvas',
  'width': 600,
  'height': 400
};
console.log(canvasInfo.width); // 600

Linear Algebra

Concepts in computer graphics are often more elegantly expressed in terms of matrices and vectors. You will often need to transform geometric models defined in the physical world to our virtual world (and vice versa), relying on operations such as matrix multiplication, addition, inverse, etc. Luckily, there are several libraries that perform these operations for you - you should still review them, but just know that you should almost never write these yourself, since these libraries have been highly optimized for performance.

Please be sure to review the following topics:

• vectors: basis, linear independence, orthogonal, orthornormal, change of bases, inner (dot) product, norm, unit vector, addition, cross product of 3d vectors.
• matrices: addition, multiplication, identity, inverse, transpose, solving a system of equations.

We'll spend the next few weeks reviewing these topics as we build up our own ray tracer, but it's still a good idea to review your notes from MATH 200.

You can also see the MDN guide on Matrix math for the web. We will perform all these matrix and vector operations using the glMatrix library which is useful for performing the 2-, 3- and 4-dimensional vector and matrix operations we will do in this course.

Philosophy

Computer graphics can be frustrating to learn because it often requires a lot of pieces to work together simultaneously to get your final image. It can be difficult to navigate through the sea of boilerplate code necessary to get WebGL to do what you want, or to figure out why your ray tracer is rendering such weird-looking images. It can also be challenging to translate the pictures, equations and descriptions from a textbook into the code you need to write. Because of this, I have designed these readings to be as interactive as possible with the hopes that

if a picture is worth a thousand words, then an interactive demo is worth a thousand pictures.

Each lecture will elaborate upon certain concepts, and I have embedded a mini text editor alongside a canvas to investigate some of the concepts without worrying too much about the boilerplate code. You will need to understand how things fit into a complete program when you work on the assignments, but as you learn the material, make sure you try out some of the examples and suggested exercises by modifying the code in the embedded editors.

I used ace to create the text editors. Any time a change to the editor is detected, it runs a callback associated with a particular exercise, while catching errors and discarding the rest of the callback, if necessary. If you see a red box below the editor, please read the error message in detail (and look for the line number) - figuring out how to debug shader code will help you a lot in the labs. Although the exercises are intended to be short, and involve only a few changes at a time, press ctrl-S (or cmd-S) to save your changes and ctrl-M (or cmd-M) to reload them.

© Philip Claude Caplan, 2023 (Last updated: 2023-09-12)