We explore the Processing.js library, its use in Khan Academy, and my personal creations in Processing.js.
Background
Processing.js/ProcessingJS is a JavaScript library developed in 2008 meant to make the Processing programming language accessible to the web by transpiling any code in Processing.js to JavaScript. Processing is a free, open-source software built on top of Java and first developed in 2001 that allows developers to create interactive visualizations and animations using a simplified syntax and drawing API. The visual outputs are easy to generate and quite reactive, serving very well to gain practice in design, gaming, and art.
Despite Processing.js being a very simple way to write Processing for the web, this library is actually no longer actively maintained nor recommended; it has been superseded by the p5.js library which came out in 2013. Both are interpretations of Processing to the web, but p5.js has departed from trying to mimic original Processing syntax in order to better match JavaScript syntax. I chose to write only about Processing.js in this blog post in order to highlight the library through which I and many other young programmers originally learned to code. Many of the principles encountered whilst programming in Processing.js are perfectly translatable to the p5.js library up to some syntax modifications. It's for this reason that I wrote the remainder of this post with beginner/intermediate programmers in mind; but I think even advanced programmers can get something out of this, if not just inspiration.
For clarity, we are not discussing Processing 3+, the version of Processing released as early as 2015 meant to respond to browsers having removed the ability to run applets and which introduced new features like surface and settings(). Processing.js doesn't support anything new to Processing 3.
If one were to compare Processing.js to other similar options in its time, several key properties would have stood out:
Natively offers an HTML5 canvas element on which a user can draw pixel-based graphics (alternatively, one may use Scalable Vector Graphics (SVGs) for rendering).
Easy cross-platform & cross-browser capabilities as based on JavaScript.
Offers a simplified syntax and intuitive API.
Non-optimized performance, so not the best suited option for data visualization nor game development.
Limited 3D support (as compared to, say, Three.js), as primarily focuses on 2D graphics.
Khan Academy
KhanAcademy.org offers an environment to write in a language based on the Processing.js library with ah HTML5 canvas immediately visible to the side of the editor in order to visualize what is being drawn. Changes are instantaneous between the editor and the canvas, so developing through this site is a pleasant experience. Typically Processing.js is embedded within an HTML file as follows:
The resulting render would contain a canvas element which is interactive.
Example Processing.js Embedding
The key to the interactivity here is the use of mouseX and mouseY, which are quite easy ways to access the current coordinates of the user's mouse on the canvas.
In contrast, Khan Academy removes much of the overhead to begin writing in Processing.js. There is no setup required to make the canvas, only instructions for what to draw. Moreover, KA follows a slightly different formatting rule: there is no need nor way to specify output type for a function, so void draw() { ... }. Instead, we must use the alternative function constructor draw = function() { }.
Unfortunately, the platform does not offer a great deal of wiggle-room to withstand heavy computations, so it limits just how demanding of methods may be run. Some other small differences include:
KA assuming angles to be expressed in degrees, not radians,
KA offering a custom image/sticker library accessed via the image and getImage commands,
KA offering a Gaussian random generator through the Random class,
The keyword mousePressed in Processing.js is written mouseIsPressed on KA,
Transformations like translate and rotate can appear anywhere in a KA script; but Processing.js usually acts on them only in a draw loop,
KA offers a Program object which contains some of the metadata/methods that can refer to the whole program, such as with its method Program.reset to simply start the program from zero; it's not native to Processing.js.
KhanAcademy's Processing.js environment is the platform through which I initially learned to code, and I've created dozens of programs since. I will proceed in this blog post by highlighting some of these programs and identifying elements of Processing.js that made these programs possible.
My First Program: Draw Loop
The following script and accompanying output represent my first program on Khan Academy (and ever!). The code attempts to illustrate an oscillating car.
Example Processing.js Embedding
My first every program!
Leaving behind how poorly written the code may be deemed, the code exemplifies a straightforward use of the draw method offered by Processing.js: we simply state which colors to be used for fills or outlines (a.k.a. strokes), which geometric objects to be displayed, and however certain variables should change over time. In particular, the draw method is special as it is repeatedly called by the page according to a fixed frame rate, constantly redrawing its contents. Drawing is also possible outside of the draw loop, but if your canvas should update over time, most drawing commands will belong in the draw loop.
For the the remainder of the article, I'll present most of my programs in GIF form, because running all of the corresponding canvases would break all of our browsers :)
Games
Curve Ball: Perspective and Computer Opponent
Now I present my remake of a classic online game called Curve Ball. I call it Curve Ball 2.0. The originaly version allows a user to compete against a computer player in a game of 3D pong. The catch is that either player may apply spin to the ball in a similar fashion to ping-pong. My version allows for any number of balls to be served, calculating the physics of each separately.
One custom method featured in this game is to resize objects based on their relative distance from the camera, an effect called perspective or foreshortening. The following method takes applies a perspective change according to the object's depth in the scene.
Moreover, the spin is accomplished by keeping track of paddle speeds and setting the ball's lateral acceleration proportional to that speed once rebounded off either player's paddle. The opponent is controlled by having it hunt for the closest ball's current lateral position but having only a fixed speed at which it can traverse its lateral space to chase that ball.
Tom the Koala's Surfing Adventure: Math and Character Design
The point of this game is for the player to help Tom the Koala to surf past all of the unhealthy items in red and pursue healthy items in green using his limited vertical boost.
One key feature to this game is having a virtually non-repeating landscape. It was a primary goal of mine to make sure the wave on which Tom surfs to never appear to repeat. Using the OOP nature of Processing.js, I proceeded in two steps:
First, define a custom Sin class which can store the parameters of a sine function like its amplitude, period, and phase, as well as an individual function to evaluate that function at any argument, plus get its derivative at any argument as well.
You'll notice the use of the logical or ||, which behaves as follows in Processing.js:
A || B evaluates to A if A is not false, and B otherwise.
Effectively, if the argument z already has a non-None value at z['amp'], then this.amp will take on the value z.amp; otherwise, this.amp is set to a default value 1.
Second, define a custom Wave object which will represent a sum of many Sin functions.
Mathematically, a Wavewill necessarily be periodic, but that periodicity will be made so large as N increases, meaning the player will not notice the periodicity.
I also feel that the system for displaying objects throughout the game is sensible in its approach. Each draw iteration will loop through all visible items (the red or green objects) and call their display method. The prototype display method is shown below.
Despite us having used very specific numbers to draw each component of the items, the transformations like translate and rotate within the pushMatrix() ... popMatrix() block allow us to generalize these commands to any position/orientation at which the item may be.
Jogo da Onça: Only do the Hard Work Once
I was in Brazil writing a paper about the Ethnomathematical study of indigenous Brazilian mathematics when I encountered a game called the Jogo da Onça: the Game of the Jaguar. Having found no online ways to play this game, I decided to code it up myself. The game works a bit like checkers:
There are two teams: the cachorros (dogs) versus the onça (jaguar).
The teams alternate in turns. Each player can move to any node to which their player is directly connected. The dogs may only move one dog per turn.
The jaguar may hop over a dog in the same style as checkers in order to capture the dog.
The jaguar hopes to capture 5 dogs in order to win. The dogs aim to trap the jaguar by giving it nowhere to move for its next turn.
Take a look at the GIF to get a sense of the gameplay:
To implement this game is inherently slightly complicated:
The jaguar and dogs all have pre-defined initial positions.
The triangular portion of the board complicates programming the positions of the vertices.
We have to encode all of the connections between vertices.
We have to write a method in order to express not just how a player can hop to an adjacent vertex, but also for the jaguar to be able to jump a cachorro. These jumps don't just follow from the connectivity: they must also be across three colinear vertices.
We would like to highlight in blue the valid moves able to be taken by a player throughout the game, which requires the same types of routines as expressed earlier.
Most of these tasks require writing by-hand the valid coordinates or combinations of vertices involved. After having fixed an enumeration of the vertices, I resulted to storing all of the tedious connections in a few lists:
Notice how the connections list is filled with tuples of various lengths: the information stored here asks: "what are the indices of the vertices to which this vertex is connected?" For example, vertex 0 is connected to vertices 1, 5, and 6. The edge relation is reflexive, but I opted to write all of the connections for each vertex for simplicity in the code. The jumping list contains tuples of vertices from which it would be valid to jump from one to the next over a dog as the jaguar. The jumped list contains the corresponding "jumped" vertex. All of these pieces of information were necessary and sufficient to run the full dynamics of the game. Moreover, having these hand-written, I was able to write the rest of the code without any reference to specific indices or positions: the logic of the game could simply be my focus throughout the remainder of the program. For instance, the following method checks if a dog with index index_in_cachorros is allowed to move to an intended position intended_index:
Animations
Adjustable Roller Coaster: 3D Graphics in Processing?!
One of my major successes during my heyday on Khan Academy was when I published my Adjustable Roller Coaster. It was probably the first 3D program to obtain many upvotes on the community page, accruing 2000+ votes and hundreds of "spin-offs." It's even featured in the Khan Academy curriculum for its use of custom-made buttons!
I learned the basics behind 3D graphics and transformations through Peter Collingridge's great tutorial on 3D Graphics with Processing.js. The original methods for rotation may not be available through his page any longer, but here I provide a sample of one of my methods inspired by his:
Light Vectors and Solid 3D Faces: True 3D Graphics
While producing a few 3D programs, I usually resorted to translucent faces because I hadn't figured out how to handle drawing based on depth. While producing the following 3D environment called Light Vectors and Solid 3D Faces, I actually got around this problem by only using convex shapes: no need to sort faces by depth, all we needed to do was determine which ones were facing forward. This program demonstrates 4 basic (convex) shapes that I was able to fully flesh out into subclasses of a generic Object3D class.
This program features inheritance: each type of object like Cuboid, Sphere, Pillow, and Tetrahedron inherit from the more general Object3D class specified below:
Inheritance looks like this in the case of the subclass Cuboid:
Finally, instantiation of a Cuboid woud look like this:
The display of each object works in a straightforward way:
This algorithm wouldn't work if our objects weren't convex! Moreover, this program won't do well trying to display multiple objects in the same line-of-sight.
Depth Sorter
Another attempt I made to display 3D objects, Depth Sorter. The general approach was to store all items in a global object called Frame and to have the ability to sort all objects.
The problem is that the code is quite inefficient and naïve: it assumes that sorting by depth can be done by calculating the average depth of all the vertices in an object and sorting by those average depths. I don't want to harp on this program too much, but I would like to highlight a few methods:
I made use of a more general set of rotation functions like so:
Unfortunately, this way of performing rotations is much less efficient because sin and cos are evaluated at theta for each point in the scene rather than all at once. It would have been better to run these on an object-basis rather than vertex-basis.
We also have an example in this program of interacting with keys:
For Proccesing.js, this is a generally good approach: store the keys that are currently pressed or free, and make actions based on these booleans. This especially works when the event of pressing isn't what matters, but the fact that the key is currently pressed is what matters.
Roller Coaster 2.0: Mathematically-Sound Coaster
I created Roller Coaster 2.0 to be a more useful 3D graphics engine and to beat my previous Adjustable Roller Coaster program shown above. The mathematics and logic behind this program are a bit more advanced than those previous.
My general idea was simple:
Building a roller coaster track should not be about defining every face and vertex explicitly. Instead, our three degrees of freedom are path, tilt, and size.
A good roller coaster is a loop: so let us assume that the path of the coaster is described by a continuous function \(f: [0, 1] \to \R^3\) which satisfies \(f(0) = f(1)\).
A good roller coaster not only loops but returns in the same orientation in which it started. So let us assume that the angular tilt of the track is a continuous functio \(\alpha: [0,1] \to \R\) such that \(\alpha(1) - \alpha(0) \equiv 0 \pmod{2 \pi}\).
Finally, we may have that a piece of the track has a certain width which also behaves in a periodic way. So let's assume the size of the track is a continuous function \(R: [0,1] \to \R_{ > 0}\) such that \(R(0) = R(1)\).
With these three mathematical ingredients, it is easy to automatically generate a discretized version of the roller coaster track that they determine: simply define a class TrackElement to calculate the face of the track at a certain value of t:
To display the scene, simply fill an array faces with Face objects created for each TrackElement as well as any other objects like for the cart and person riding the coaster, and sort the objects by depth:
Processing.js Components
After creating a few programs which benefiting by offering user interactions with on-screen components like buttons and sliders, I decided that I (and other users of Processing.js) needed a better way to make use of UI components. The system of Processing.js Components that I developed worked very similarly to that of Java's JComponents, and included functionalities for drop-downs, text fields, sliders, buttons, check boxes, check box groups, popups, icons, hyperlinked text, and even speech bubble groups. Try interacting with the components below:
Example Processing.js Embedding
Creating Processing.js components for use in other programs that require user interaction. You can try interacting with the components here.
Each particular component is stored in a Pane object:
And each component class inherits from the parent class Component:
Perhaps a good example of the logic that occurs at the component-level is best exemplified in the DropDown.prototype.update method:
Each of the methods called throughout the routine runs some other logic or display instructions. Each component can be pretty complex to map out but works by a similar logic.
The draw loop in this program handles the order of operations:
The core code in this program can be used in many situations, and certainly can help a new programmer to have a logical UI ready for their specific purpose. I made use of these components many times in subsequent programs myself.
Language and Intelligence:
The purpose of my program Language and Intelligence was to demonstrate the entropy of languages. I was initially inspired by Brit Cruise's A Mathematical Theory of Communication, where he explains how Shannon entropy could be used to analyze the randomness of human language. I built a program which could perform high-order approximations to any language using some amount of example text from that language.
For example, in the following (nonsensical) text in English:
Meant balls it if up doubt small purse. Required his you put the outlived answered position. An pleasure exertion if believed provided to. All led out world these music while asked. Paid mind even sons does he door no. Attended overcame repeated it is perceive marianne in. In am think on style child of. Servants moreover in sensible he it ye possible. Tolerably earnestly middleton extremely distrusts she boy now not.
we may count all instances of each letter, all instances of pairs of letters, triples, etc. You may notice that there are three instances of the pair is, two instances of the, and zero instances of zsj. What is the use in doing this counting?
Above is a screenshot of a typical use case of my program: it expresses that, out of the whole sample text that it received, the probabilities of finding each one-letter continuation of the string mo You'll notice that, reasonably, mor, moo, and mon are the most likely continuations, while moj, mox, moq did not even register. Besides being a fun way to explore the normal patterns of the English language, there is a deeper amount of information we may glean from the observed frequencies.
Shannon Entropy is an information-theoretic quantity used to quantify the degree of randomness or disorder across a probability distribution. If we consider the probability distribution of all \(n\)-tuples of English letters appearing in their natural frequency in written English, we may approximate the Shannon Entropy of the whole language. Simply fix an \(n \geq 1\), compute the probabilities of each \(n\)-tuple of letters using some sample text, and plug in the probabilities into Shannon's equation
This program is able to do this for any \(n\), as well as a bit more: it can perform the analysis, display an interactive conditional probability chart for any starting sequence on what's the most likely way to continue the sequence by one character, generate random messages of text following the same probability distribution but possibly including fake yet realistic words of that language, as well as attempt to autocomplete someone typing in that language without cross-checking with a dictionary.
For instance, when performing an \(n = 5\)-tuple approximation to English, my program generates the random message:
laundrywomen he xpresystem piqued ed here quired sion on oved sides jecting estrated gality ollectionishment iuolo ition icize decker ention ntly suppose erative ament ystifyingly shrieval uni oxide ltirewise termined ho xpedient a pleasure bers exercise hoal yrius us nfatigued haps rse beri fied on dent am ing aid azine glass she tron roperture ed orff ooms crated oopint ticreat ary enthronism id omise icatus rargyrius competing ne cher she egic istical onesian ettsville those cotypic usly attendent ark
Some of these outputs are real English, others convincingly English-like, and others nonsensical. This shows the power of harnessing patterns even without having access any rules about the language. I must say that this program was made well before large language models, and well before I was conscious of any AI models, so this was my baby step into thinking about generative models.
I'm very proud of this program, and especially glad that Processing.js could make it so widely available on Khan Academy's Computer Science community.
Miscellaneous Animations
Here I've collected some extra animations that I wanted to at least show without going into much detail abou thow I did them. I'll let you think about it. You can always click on the links to the see the code and decipher my process, or try for yourself.
Aurora Borealis
I submitted this Aurora program as part of a presentation in my AP Environmental Science class in high school.
Abelian Sandpile
An Abelian Sandpile is a grid of values that resembles a sand pile in its dynamics: cells can only hold so much sand, passing on excess to their neighbors. The exact dynamics can be found here.
Recursive Art
I made Recursive Art Project this to demonstrate some art with recursion. There are more animations in the original program.
Fourier Circle Approximations
Fourier Circles can be used to approximate any curve in the plane. I tried my hand at coding the required algorithm to make it approximate a few shapes like a Heart!
Algorithms
Regression with Genetic Algorithms
I was able to approach a standard math problem of polynomial regression via Genetic Algorithms. The program cycles through multiple generations of polynomials which attempt to approximate a standard polynomial whose outputs we can access but whose exact generating formula we cannot. New generations represent lightly-mutated offspring of the fittest individuals from the previous generation. Fitness is measured by minimizing the measure of difference between the individual and the reference polynomial.
The key method to this is how the Organism class reproduces from two previous instances of the Organism class. There will be use of some hyper-parameters.
This actually works pretty well, and there are plenty more opportunities to explore genetic algorithms more useful than this.
Extending the Numeric System
I had the idea to push the numeric system of the language past its limits. This program: Extending the Positive Integers, aimed to provide the ability to perform mathematical operations on numbers much bigger than the typical constraints of Processing.js.
In Processing, int represents the datatype for integers. Integers can be as large as \(2^{31} - 1 = 2,147,483,647\) and as low as \(-2^{31} = -2,147,483,648\). They are stored as 32 bits of information. With the GIF as my witness, my program is able to compute with numbers larger than \(2^{80} = 1,208,925,819,614,629,174,706,176\), over 100 trillion times larger. What explains this difference in capabilities?
Rather than being stored as the primitive data types int, these numbers are stored as string data containing binary representations in order to perform many basic calculations. I wrote original algorithms for how to perform computations on binary representations stored as strings and then convert back into decimal. We store the data in an INT object as follows:
The program assumes non-negative integers. Addition works according to the two methods:
There are many other methods worked out in this program, like for subtraction, multiplication, division, exponentiation, tetration, modular arithmetic, comparison, etc. The hardest was to interpret a written mathematical expression from the user into an executable sequence of instructions. All of that is summarized in INT.interpret:
The idea behind this program was simple, it was just a complicated set of methods to implement to perform all of the conversions and operations that I needed from a useful calculator.
Appendix
There were many helper functions I made use of throughout my programs. Some of them had to do with the PVector class, which can store vectors in the form of objects with .x, .y, and .z properties storing information in the \(x\), \(y\), and \(z\) coordinates, respectively:
Others had to do with the generic Object class from which all objects implicitly inherit:
Others had to do with the Array class, in which every instance has a length property:
Conclusion
I hope you can glean from this large summary of my work using Processing.js on Khan Academy that Projecessing.js is a fantastic option for beginner/intermediate programmers to learn visual and CS skills that appear in more advanced programming languages and programming tasks.
Please let me know if you have any questions! You can always find my programs here complete with the source codes and outputs.
Subscribe to Sifter
Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.