Oh no! Mike Wazowski has been banished to the Himalayas again. We need to find him with our (virtual) airplane, but unfortunately, the aircraft controls are frozen. Our first task is to fix the controls. The second task is to make the terrain look colorful. Finally, you will extend the flight simulator with a feature of your choosing!

The third objective of this lab relates to the idea of working within a larger codebase and figuring out what already exists and where you need to add code to implement some functionality and/or fix bugs. This is an important skill to develop as you continue in computer science. The code has been labeled with comments such as PART XY to identify where you should add code for certain parts.

Please start by opening up the repl for this lab. Mike has been placed at the highest point in the terrain, which is defined by a grid in which the height at each grid point is computed using something that might look familiar!

You might be able to see Mike (original model found here, which I simplified using MeshLab). The flight simulator UI allows you to overlay the cockpit controls and adjust the speed of the airplane. You can also adjust the terrain using the dropdown.

The arrow keys are supposed to maneuver the airplane by adjusting the pitch, roll and yaw of the aircraft. But these are currently broken.

The scene is rendered from the viewpoint of the cockpit, and the flight simulator works by adjusting the gaze and up directions at each time step in the simulation. In contrast to previous labs and exercises, the up direction changes with time. This means that the view-matrix ($\mathbf{M}_v$) changes with time. First have a look at the vertex shader defined in the vertexShaderSource at the top of simulator.js. Notice that the u_ViewMatrix and u_ProjectionMatrix have been hard-coded, which is why the controls are frozen.

There are two models in our simulation which are stored in this.objects (a map/dictionary) of the FlightSimulator class. These are written to the GPU in the write method of this class. At every time step, the animate method updates the position, gaze and up vector of the aircraft, subsequently calling the draw method. Note that the vertices defining Mike Wazowski have been directly transformed when the simulation loads, and the terrain does not need to be transformed, so there is no need for a model matrix. The model matrix is always the identity matrix in this lab.

Part 1: Calculate the View Matrix and Perspective Matrix in JavaScript and pass them (and use them) in the shader program.

We are assuming that the forward direction of the aircraft aligns exactly with the gaze direction. Therefore, whenever the arrow keys are pressed, the gaze is updated, as well as the up vector. Please calculate a "look at" point and calculate a view matrix using mat4.lookAt. Search for PART 1A, which is where this should be calculated. As mentioned earlier, this.up,this.gaze, and this.position are all updated at each step in the simulation. You will need these to calculate the view matrix.

Next, write this view matrix to a uniform and use it in the vertex shader (PART 1B). Please do the same for the perspective projection matrix (using mat4.perspective instead). Use a field-of-view of $\frac{\pi}{4}$ and note the aspect ratio is calculated for you in the aspectRatio variable of the draw method. Use a near plane of 0.001 and a far plane of 1000.

When this works, you should be able to navigate the airplane with the arrow keys.

Part 2: Calculate the Normal Matrix in JavaScript and pass it (and use it) in the shader program.

The normal vector at every vertex of the mesh (in world space) is calculated and saved in each model as an array called normals. Each mesh (in this.objects) has a normalBuffer which is created and written to in the FlightSimulator write method. Also note that the a_Normal attribute is enabled right after the program is created and that the normalBuffer of each mesh is associated with this attribute whenever a mesh is drawn.

Please search for PART 2A, then calculate and set the normal matrix in the shader program. We would like to transform normals into the frame of reference of the airplane (camera space). Remember the model matrix is always the identity matrix in this lab.

For PART 2B, declare (1) a uniform for the normal matrix and (2) an output varying for transformed normals. Then use this normal matrix to transform the incoming a_Normal attribute into camera space to compute the transformed normal (PART 2C), which will be assigned to the varying you declared in PART 2B.

For PART 2D, declare the incoming (varying) normal vector for the fragment shader, and then use it instead of the constant normal vector (n) that is current used in the shading algorithm (PART 2E).

When this works, the terrain should look shaded, but everything has the same base color.

Part 3: Calculate color at each vertex in the models (M status).

For PART 3A, please push color values to the colors array of each mesh object (a 1d array with a stride of 3). For Mike Wazowski, you can set any color you want. I would recommend starting by setting a constant color for Mike Wazowski, for example, (0.5, 0.7, 0.5). Although this is a constant color, we will still assign it to every vertex in the mesh. You can make this vertex-dependent later if you want.

For the terrain (PART 3A (continued)), please push color values according to the height of the vertex:

For PART 3B, add some code to the write method which creates a buffer for the colors, and buffer each mesh color data to this color buffer. Make sure to save the variable for this buffer in each mesh object (just like we already have for positionBuffer and normalBuffer). You will need this variable below.

For PART 3C, declare a new attribute in the vertex shader for the color of each vertex. Then enable this attribute in the constructor of the FlightSimulator (at PART 3C (continued)).

For PART 3D, add some code to the loop in the draw method to associate the previously created color buffer (from PART 3B) with the color attribute.

Finally, for PART 3E, declare a varying in the vertex shader and assign the incoming color attribute to it (PART 3E (continued)). Then use this color as the km in the fragment shader, instead of the constant color which is currently there.

When this works, the terrain should look more realistic.

Part 4: Choose your own adventure (E status).

For E status, please extend your flight simulator by adding at least 1 additional feature. The following are some suggestions for things you might want to implement but please feel free to be creative here and suggest your own extension:

• Add another model. I have added a mesh for Sulley in sulley.obj in case you want to use that one. Inspect the preprocessWazowski function in the utils.js file to see how Mike Wazowski is transformed. You might need to experiment with the scaling and the orientation of your model. You can also find your own model (see the links at the end of the Week 05 notes) - if you do, comment out any instances of usemtl in the .obj file with a # since the webgl-obj-loader might omit some triangles otherwise. Another idea would be to render an airplane directly in front of the view, like you would sometimes see in a third-person view of some games.
• Move the color calculation for the terrain to the fragment shader. This can be done with a single shader program but you'll need to signal (perhaps with a uniform) to the shader program whether you are currently rendering Mike Wazowski or the terrain. Or you can use two shader programs: one for Mike Wazowski and one for the terrain.
• Add some gameplay, like detecting if the airplane is too close to the terrain and display a message to pitch the airplane upwards. Or maybe even make this upwards pitch happen automatically if the plane gets too close to the terrain.
• Alternatively, instead of directly implementing a feature, please think about how you would implement the following feature and describe it in as much detail as possible (2-3 paragraphs): how would you implement a feature in which Mike Wazowski is walking around during the simulation? Please describe how you would set up a model matrix from a specific expression for Mike's location (you can make something up for this expression). Please elaborate on what else would need to change (shaders, the way the models is pre-processed) to implement this feature. The response to this question should be in a new file called part4.md in your repl.

Submission

The initial submission for the lab is due on Wednesday 11/08 at 11:59pm EDT. I will then provide feedback within 1 week so you can edit your submission.

When you and your partner are ready, please submit the assignment on replit. I will then make comments (directly on your repl) and enter your current grade status at the top of the index.html file.

Please also remember to submit your reflection for this week in this Google Form.

© Philip Claude Caplan, 2023 (Last updated: 2023-11-01)