Chapter 1: Pixels

Learning Objectives

By the end of this lecture, you will:

  • Identify the important characteristics used to describe an image,
  • Write basic JavaScript code,
  • Investigate the frame rate of a fluid simulation.
  • Represent meshes using two flattened (1d) arrays for vertices and elements.
  • Access vertices of elements in a triangle mesh.

Welcome to CS 461 :) I'm glad you're here and I'm excited to show you some techniques you can use to build interactive demos, visualizations, animations or games. We will cover several topics in this course, ranging from geometric modeling, ray-tracing, rasterization and animation. By the end of the course, you will be able to:

  1. develop your own ray tracer to render complex scenes,
  2. display and manipulate three-dimensional models using rasterization (with WebGL),
  3. animate three-dimensional objects and physical systems.

Those are the three main themes: ray tracing, rasterization (with WebGL) and animation. I will post notes (like the notes on this page) to our course website, which will also contain interactive demos within which you can modify some small code snippet to understand the concepts in that reading.

The Goal of Computer Graphics: Setting Pixel Colors

That's it, really. We're going to have a grid of pixels to represent a picture (we'll call this a frame) and our job is to assign the color of each pixel. Well, that's not quite the end of the story. We need to do this very quickly and, in some situations, we might be limited by how much memory we can use to do this.

Let's take a step back first. What's a pixel anyway? A pixel is a "PIcture ELement" and it's the main building block of the pictures you see in your everyday life, like the pictures you take on your phone.

Consider the 8-bit image of the earth at the top-left of this page (originally from here) in which we can see the individual pixels. How many total colors are used here? I count four: black for the background, white for the stars, green for land and blue for the ocean.

There are a few important things we need to consider when assigning pixel colors in an image:

  1. What is the size of the image? In other words, how many pixels are there along the width and height of the picture? The more pixels you have, the more memory your image will take up, and the more work your computer will do to process your image.
  2. How do we represent the color assigned to each pixel? In general, we will use a red-green-blue (RGB) triplet to represent the color. In some cases each channel (red, green or blue) will have an integer value from 0-255 but in others we will assign floating-point values between 0-1.
  3. What is the coordinate system of the image, i.e. what are the directions of the x- (width) and y- (height) axes? We will always use a coordinate system in which x- goes from left to right. However, some graphics systems have the y-axis pointing downwards.

Check out the following demo. On the left, you should see a rotatable and zoomable 3d model of a cow (a common model called "Spot"). Each point on the surface of Spot is translated to a pixel and, again, our job is to assign a color to that pixel. Later in the course, we will study WebGL, which will consist of writing shaders to programatically assign these pixel colors. For now, the pixel color is set to grey (0.5, 0.5, 0.5). Actually, you might notice there is a light that makes some points look brighter, but we'll talk about this in a few weeks when we talk about shading. The base modelColor is grey. Exercise 1: try adjusting the RGB values of the modelColor.

Okay, let's make this more interesting and add a flannel pattern. Exercise 2: try typing the following mathematical expression given the coordinates on the surface of Spot (x, y and z). Feel free to change the 100.0 and also the 3d model using the dropdown below!

if (sin(100.0 * x) < 0.0 && sin(100.0 * y) < 0.0 && sin(100.0 * z) < 0.0)
    modelColor = vec3(.4, 0, 0);
else
    modelColor = vec3(0.1, 0.1, 0.1);

Also, what happens if you change one of the 0.0 in the conditional to 0?

Frame Rate

Let's investigate a bit further. Here is a really nice demo of a fluid simulation. Click on the link to open the demo and then click/drag to simulate the fluid! What do you notice if you increase the "resolution" setting to 1024? 4096? 8192? Have a look at the FPS display at the top-left.

https://philipclaude.github.io/webgl-fluids-demo/

Please note: I did not write this really nice demo! I just modified some of the resolution settings. The original code can be found here.

So let's restate our goal. We want to set the color of every pixel in a frame at some appropriate framerate. The framerate is measured in frames per second and often abbreviated FPS. The human eye can typically see between 30-60 FPS. Anything below that and the animation/simulation/interaction will feel "off".

JavaScript

JavaScript may look familiar if you've programmed in Java, C or C++, in terms of the use of semi-colons, curly braces and the syntax of if-statements and for loops. Everything in JavaScript is an object (numbers, strings, classes, etc.). There are some amazing online resources for learning JavaScript - for an in-depth guide, please have a look at Eloquent JavaScript or w3schools.

There's often more than one way of doing something in JavaScript, and our use of the language will be quite basic. AirBnB's style guide is a good resource for learning the dos and don'ts of JavaScript (which we will mostly follow). You'll be able to pick up JavaScript along the way, but be sure to review things like:

To avoid bugs, I recommend the following:

As you read through these lecture notes, I will embed mini exercises that will prompt you to either modify some JavaScript code, or even one of the shaders (written in GLSL), as in the demo above. However, especially when you work on your labs, you should be familiar with a few other ways to develop in JavaScript code. First (and probably the most convenient), you can use the JavaScript console in your browser (Chrome: View -> Developer -> JavaScript console, Firefox: Tools -> Web Developer -> Web Console, Safari: Develop -> Show JavaScript Console). When you open the "console", you will face a prompt which looks like >; You can use the console to create objects, perform calculations - just like you would with a Python interpreter.

Examples

Here are some examples of basic JavaScript code syntax. Please open the JavaScript console and directly type the examples below (you can copy-paste some of the longer examples). The comments below the expressions demonstrate what the output should be when printing the variable.

Variables and Printing
let x = 2; // declare a variable
x = 3; // change the value

const y = 8; // declare a const variable
// y = 9; // <-- ERROR!

// printing
console.log('x + y = ', x + y);  // method 1
console.log(`x + y = ${x + y}`); // method 2 with template literals
Arrays, Loops and Conditionals
let a = new Array(10); // an array with 10 elements
for (let i = 0; i < a.length; ++i) {
  if (i % 2) a[i] = i + 1;
  else a[i] = 0;
}
// a = [0, 2, 0, 4, 0, 6, 0, 8, 0, 10]

a.push(1); // add an eleventh element
// a = [0, 2, 0, 4, 0, 6, 0, 8, 0, 10, 1]

// ternary conditional operator
const b = (a.length % 5) ? 34 : 7;
// b = 7

// another way to create an array
let x = [];
x.push(5);
x.push(8);
// x = [5, 8]
Functions

There are different ways to declare functions. The first method below (regular function) is hoisted and is generally not recommended. AirBnB recommends either using arrow functions or named function expressions - the latter have an advantage over anonymous function expressions when examining the strack trace of a program (e.g. when debugging). Arrow functions are preferred when assigning callbacks and the function needs to be anonymous (as in the last example).

// regular function (hoisted)
function add1(x, y) {
  return x + y;
}

// anonymous function expression (not hoisted)
let add2 = function(x, y) {
  return x + y;
};

// named function expression (not hoisted)
let add3 = function addsTwoNumbers(x, y) {
  return x + y;
}

// arrow functions (not hoisted)
let add4 = (x, y) => {
  return x + y;
};

console.log(add1(1, 2)); // 3
console.log(add2(2, 3)); // 5
console.log(add3(3, 4)); // 7
console.log(add4(4, 5)); // 9

// setting the callback when a key is pressed for the web page
window.addEventListener('keydown', (event) => {
  console.log(event.key, ' was pressed!');
});
// then click on the main page to bring it into focus and press some keys!
Classes and Objects
class Pixel {
  constructor(r, g, b) {
    this.r = r;
    this.g = g;
    this.b = b;
  }

  scale(a) {
    this.r *= a;
    this.g *= a;
    this.b *= a;
  }
}

let p = new Pixel(0.5, 0.5, 0.5); // create a pixel object
// p.r = 0.5, p.y = 0.5, p.z = 0.5

p.scale(255);
// p.r = 127.5, p.y = 127.5, p.z = 127.5

The scale method was defined directly in the class definition. Alternatively, we can define methods outside of the class using prototypes (although it isn't recommended by AirBnB):

Pixel.prototype.set = function(r, g, b) {
  this.r = r;
  this.g = g;
  this.b = b;
}
p.set(1, 0, 0);
// p.r = 1, p.g = 0, p.b = 0

Also note that objects can be created directly using JSON:

const canvasInfo = {
  'id': 'myHtmlCanvas',
  'width': 600,
  'height': 400
};
console.log(canvasInfo.width); // 600

Representing the geometry in our scenes.

How would you represent the geometries in the previous examples? Typically, these are represented using a "soup" of either triangles or quadrilaterals in which each triangle is represented by three 3d points and each quadrilaterial is represented by four 4d points. This is fine for most of the things we will do in our course, but there are more efficient ways that use the "connectivity" between points on the surface without duplicating these points.

For example, consider the mesh below with 5 triangles. How much memory does it take to store this mesh if we represent each triangle by three 3d vectors? Assume floating-point values are stored as Float32 (4 bytes).

Solution With 3 vec3s for each triangle, there are 15 vec3s in total. Each vec3 uses 4 x 3 = 12 bytes of memory, so the total memory this small mesh consumes is 15 x 12 = 180 bytes of memory.

The issue with our unconnected representation is that the vertices shared by multiple triangles are duplicated in memory. We can solve this problem by storing a single vertex in memory and have each triangle reference that vertex. For the example above, there are 6 vertices and 5 triangles. The 6 vertices consume 6 x 3 x 4 = 72 bytes of memory. The memory used to store the triangles depends on how we reference the vertices to define the triangles. Assuming a 32-bit integer is used, then we have 3 x 4 = 12 bytes per triangle, thus 12 x 5 = 60 bytes for all 5 triangles. The total memory used to store the mesh is then 72 + 60 = 132 bytes. The difference with the 180 bytes we had before might not seem like a big difference, but this difference is much greater for larger meshes (more triangles).

Note that for this tiny mesh, we could even have gotten away with using a single byte for the triangle vertex indices (as an unsigned integer with a range from 0 - 255), which would have consumed 5 x 3 = 15 bytes for the triangles and 87 bytes total for the vertices and triangles.

In addition to memory consumption, another advantage of the new proposed representation is that triangles are connected. If the shapes in a spider web weren't connected, it would just fall to the ground. For us, this idea of connection is important to devise algorithms to search through and modify meshes.

Mesh: geometry and topology.

We will represent our models using a mesh, which contains two core ingredients: the geometry and topology. The geometry of the mesh refers to the points of the model in 3d space, whereas the topology refers to how those points are connected to represent the model. In our course, we will always use triangle meshes, but it's good to know that other types of meshes exist, for example, quadrilateral or more general polygons for surface meshes. It's also possible to have volumetric elements like tetrahedra, hexahedra, prisms, pyramids or general polyhedra. The fundamental shape of a mesh is usually called an element.

We will describe the geometry using an array of vertices (sometimes called nodes). These will form the vertices of each triangle in the mesh. We will also assign a unique number to each vertex which corresponds to its location in our array of vertices. Here is an example of a two-triangle, four-vertex mesh.

Storing a mesh with 1d arrays using a stride.

Instead of storing vertices in a 2d array (maybe, where each row corresponds to a single vertex), we will store the coordinates in a 1d array. Since we are working with 3d coordinates, each chunk of 3 values in this array represents the coordinates of a particular vertex. In the example mesh above, the coordinates of the vertices can be represented in a 1d array called vertices like this:

vertices = [x0, y0, z0, x1, y1, z1, x2, y2, z2, x3, y3, z3];

Similarly, the two triangles in the mesh can be represented in a 1d array called triangles like this:

triangles = [3, 2, 1, 2, 3, 0];

To access the y-coordinate of the third vertex (y2), we would use:

y2 = vertices[3 * 2 + 1];

where the 3 represents the stride to take when accessing each vertex. We use a stride of 3 since there are 3 coordinates for each vertex. The + 1 accesses the y coordinate here, which would be + 0 for the x-component and + 2 for the z-component. We could also access all three components using the slice method:

p2 = vertices.slice(3 * 2, 3 * 2 + 3);

The variable p2 will be an array of length 3, which can also be interpreted as a vec3 in glMatrix (passing this variable to glMatrix vec3 functions will work).

To more generally access the d'th component of the i'th vertex, we should use:

vertices[3 * i + d]

Triangles should always be oriented with a counterclockwise (CCW) ordering, which is shown in the blue arrows in the image above. This is important when we shade our models and need the normal vector (which depends on this ordering) to be pointing out of the model. If you trace the order of the triangle vertices with your right hand, then your thumb will point in the direction of the normal. The first triangle references vertices 3, 2 and 1 and the second triangle references vertices 2, 3 and 0.

Similar to how we access the coordinates of a vertex, we can access the vertex indices of each triangle using a stride of 3 (since there are three vertices per triangle). For example, the third vertex of the second triangle (which is 0) would be accessed using:

triangles[3 * 1 + 2]

where the 3 again represents the stride, the 1 represents the triangle index and the + 2 is used to access the third vertex index. Similary to what we did before, we can access the j'th vertex of the k'th triangle using:

triangles[3 * k + j]

To then access the vertex coordinates in a particular triangle, we can combine what we just derived for vertices and triangles. We can access the d'th coordinate of the jth vertex in triangle k using:

vertices[3 * triangles[3 * k + j] + d]
© Philip Caplan, 2025