Walker, Texas Gamer - Final Design

The whole concept for a game came to me this summer while learning Processing from Daniel Schiffman’s YouTube course called The Coding Train. My game uses the random Walker code that Schiffman wrote as a sort of chance element to decide the fate of each player. I originally wrote the the game in Processing (code here) but had to port it over to p5.js later down the line.

The porting process from Processing to p5 was difficult regarding the way each deal with objects and classes. The norms are quite different and as often times in code there are a few different ways to go about it. I ended up removing much of the class language and making discreet functions and variables to simplify things. Here is a link to my final p5 sketch.

I wanted to use the dice rolling as sort of an extra “real world” layer so that you could weight the Walker to go in a certain direction - an idea that was floated in Schiffman’s original examples. I liked the idea that one had some control over the randomness, sort of like Black Jack at a casino. (I am quite the optimist when it comes to my chances when gambling, for the worse sometimes…).

So to get the dice data into my sketch, I first tried to use machine learning and image classification and a web cam. I recently attended a talk that Google employee Irene Alvarado had recently given at ITP’s Machine Learning meet up. She showed us Google’s Teachable Machine Boilerplate which I saw might fit my purposes. I took that code and tried to train it to learn what each possible combination of dice looked like (code here), but I had no way to save the model that I had trained. It would lose all the data if I refreshed the browser. So I set up a meeting with ITP Faculty Yining Shi who encouraged me to check out ml5’s image classifier in p5. This ended up being a simpler option as Dan Schiffman has a great video on how to save a model once you’ve trained it.

Now, I was able to save and load the model with no issues, however… image classification was not the best option for recognizing dice as each dice does not significantly change the amount of pixels in relation to the whole picture. Image classification models work better when the photos are significantly different from one another, as opposed to a picture with only a handful of pixel difference, like a dice roll of 2 or 3…

Luckily, I found a p5 blob detection sketch online which, with a lot of help from Yining, I was able to adapt for my purposes. We tweaked the sketch to look only for black blobs and I made the game board with no black portions, which I discovered was necessary after much trial and error.

Here are some videos of me explaining how the game itself works and how I calibrated the blob detection.

A Walker, Texas Ranger-themed board game using blob detection in p5.js to detect a pair of dice. This was a final project for Intro To Computational Media at NYU's ITP masters program.

More work Donut debugging brings newfound skills

scripterror0.png

I spent some more time this week trying to debug the Donut. I began to think that the glitching problem was specific to my computer since it ran so smoothly on the classroom machine. So I thought I could just download the sketch.js file and edit that in Sublime Text and launch it locally on my computer, viewing it in Chrome. But I kept getting “Script Error. ( : line 0)”. After some googling, I found that the error was happening because I was trying to run the sketch both with HTTP elements and local elements off my hard drive.

For instance, I was trying to launch a song stored in the sketch’s assets folder online while running the index.html file from my local hard drive. Chrome will allow you to mix protocols “http://” and “file://” in that way which, as we discussed in class briefly, is a protection layer so that an external website cannot look onto your hard drive without you knowing it.

The solution I found is to Run A Local Server on your desktop so that Chrome can view the local files via the “http://” protocol instead of “file://”. So I downloaded the p5js sketch using File > Download. The package came as a ZIP file with all the javascript files like “sketch.js” along with the “index.html” file, libraries, and my “assets” folder, which had the background song file. Once unzipped, this folder would be where I hosted my files from.

To launch a local server using http://, I followed the instructions on the link above:

  1. Download and install Python.

  2. Use the Terminal to navigate to the folder I wanted to serve my files from. (To quickly find the path of the folder, I hit CMD + I to get info which will show the path on of the files. Then in Terminal, type “cd” and paste in the path. Hit return and you will currently be “in” the folder you want to make changes to.)

  3. Type the following command into Terminal: “python -m SimpleHTTPServer” which will open a server instance in that folder under the path “http://localhost:8000” with 8000 being the port name.

    When I did the step above, an old website came up that I had been working on years ago. It was clear that somewhere in the past I had managed to do this same thing, although I clearly had not retained the information. So I had to open the local host from another port. This was done by simply adding a port number at the end of the command: “python -m SimpleHTTPServer 7800”. These port numbers were recommended from the web.

  4. Next, I simply typed in “http://localhost:8000/” into Chrome and my canvas popped up with my Donut and it wasn’t glitching!!!!

    I was able to edit my sketch.js file using Sublime Text and the Save after each edit and refresh the locally hosted webpage in Chrome to see the updates. The only problem now was that there was no Console like in the p5!

When I was done editing the Donut, I simply uploaded the sketch.js text back into p5 and launched the file and… but was still glitching!! So this led me to believe that the issue is will p5 and my computer somehow and not that the sketch was too processing heavy. I was able to get the sketch to run without the “Script Error ( : line 0)” when I launched a incognito window, so perhaps it was a login issue.

I would use this technique later when working on my PCOMP mid-term because the sketch we built there also exhibited this glitching behavior. But this time, I made the console stream down to right side of the screen.

Local hosting with p5 sketch and Chrome console.

I wouldn’t recommend this technique for editing files unless you know what you’re doing because when I tried to add p5.Sound functionality from the Sound Library, I got all kinds of issues with how to link in that library. Unless you build the guts of the sketch using the p5 editor and add all the libraries and external files there and download in one package, you can get into a lot of extra mess trying to manage all that locally. But it was an educational journey!

Editing a p5js sketch is harder than it looks

I got lost in a coding rabbit hole this week. I was trying to edit the Donut p5js sketch to prep it to get serial input. After replacing one of the hand-made sliders with a DOM slider, I noticed the sketch had gotten glitchy, i.e. not running smoothly, at some point in its iterations.

So I tried troubleshooting by commenting out particular lines, taking the song and amplitude elements out, taking all the sliders away. Even with all the donuts out, the background still exhibited this glitchy behavior. I tried commenting out the frameRate() controls. I tried running smooth();. I restarted my computer. As a last resort, I even tried porting it over to Processing, but this proved to be unfruitful as I found out that it has no DOM elements, so none of my sliders would be of any use.

Right now, I would like to solve the problem of the glitchy-ness so I can eventually make this work interactive. I booked some office hours with Jasmine to try to sort it out.

Walker, Texas Gamer

I devised a game using the Walker model Dan Schiffmann gives in the Processing tutorials There are elaborate instructions here on my Github page. It’s basically a 4 player betting game to see which direction the Walker will go. There is some dice rolling built in to give the illusion of choice and skill :) I built it because I love to gamble and would love to play this with friends. Future upgrades include new graphics, character development for an emcee, and interactive functionality to read the dice and weight the dice rolls properly.

Hitting The Wall

I was really banging my head up against the wall in a lot of ways this week. I feel like my brain is not putting the pieces together in a cohesive and methodical way.

The Donut Maker

Donut Maker w Sprinkles //
fullscreen sketch and code editor

I was able to add a slider to one of my previous sketches and get it to do some cool stuff. When I wanted to make 7 different sliders, I was able to do so using a FOR loop. But I realized that I would need different variables for each of the 7 sliders’ output values, which I did not know how to do from inside a FOR loop. I had in mind to make a bunch of interesting parameters move, a lot of which I marked in my sketch, but was unable to get to.

Here is my sketch in Code Editor and full screen mode for the Donut Maker.

sound.play(); error - Click HERE to see the sketch.

However, the most curious thing was my inability to load an mp3 file. I added it to my assets, but even though I checked my syntax repeatedly, I was kept getting the following error message >>>


As for Quiz 3…

For the quiz, I came up on a few snags in the logic. Questions 1 and 2 were clear to me. For question 3 though, I was able to change the behavior so that if you click the mouse when you’re inside one of the rectangles, it changes the background of that area to red and keeps it red until you mouse over it again.

Then I tried to make the third rectangle turn red and stay red on hover if clicked, instead of turning blue on hover. If I clicked the mouse again, the red hover color would toggle back to blue. I successfully did this like so:

if (redToggle == true) {
        rect3ColorHover = red;
      } else if (redToggle == false) {
        rect3ColorHover = blue;
      }

So I then wanted to make it so I could toggle all the squares to red if I wanted so I did this:

 if (rect1Check == true && redToggle == true) {
    rect1ColorHover = red;
  } else {
    rect1ColorHover = green;
  }
  if (rect2Check == true && redToggle == true) {
    rect2ColorHover = red;
  } else {
    rect2ColorHover = yellow;
  }
  if (rect3Check == true && redToggle == true) {
    rect3ColorHover = red;
  } else {
    rect3ColorHover = blue;
  }

Then I thought that I wanted to switch each rectangle’s hover color to red, toggling each independently. I worked on this for a while, but realized I’d need a boolean expression for each rect and not one global redToggle / turnRed setup.

BUT!! Then I re-read the question and realized that this was not what Question 3 was asking, although I did get a lot out of trying the last bit. Question 3 says that you need to toggle the square red when you click the mouse and leave it that way, using the function mouseClicked() presumably, and then when you hover over it again, it turns back to the hover color. I had no idea how to go about this.

The questions following #3 were pretty straightforward to me, so I got most all of them all to work except for question #7 where I kept getting this:

https://editor.p5js.org/billythemusical/sketches/SyDERoHK7

https://editor.p5js.org/billythemusical/sketches/BkxWAsBYm

I finally peaked at someone else’s sketch and found that I wasn’t using the modulus quite right. I had already tried to add the two For loops “i” and “j” values and evaluate them (i + j) % 2 > 0 to see if they were positive or negative, it’s just that I didn’t draw a rectangle in each of the the if statements.

On The Partner Assignment

Elvin and I spent a long while trying to figure out how Timothy had made those googly eyes in his sketch. We went down a long rabbit hole of sin,cos,tan, and atan which was all honestly over my head. We tried to apply those concepts to a sketch Elvin had made, which was rather hard because when you are reading someone else’s code, it’s like looking into someone’s brain in a very specific way. It was very hard for me to get in line with what he was up to, especially since he was just experimenting incrementally the with a lot of what he came up with.

After about an hour of getting no where, we gave up and tried a new idea Elvin had sketched out in his notebook which was to instantiate 3 separate clusters of 5 dots and have them follow mouseX/Y, but stay inside an invisible boundary ellipse. If mouseX/Y got outside of the boundary ellipse, the clusters would still follow the mouse while staying within and at the edge of the boundary ellipse, as well as retaining the relative position of each individual dot in each cluster.

So we broke it up into sections, where Elvin was going to code the boundary ellipse and the governing rules for movement. He made a simple inner ellipse to test the values. His progress can be seen HERE. We had the idea to use the dist() function value, but we were only able to get so far with that, as we were still using constrain() as well. You will see the functionality is busted a bit.

I was trying to design the clusters, giving each instance of a cluster a randomized smattering of dots. But each time I tried, either making a Cluster class or a Dots class, I was unable to get a randomized pattern of dots for each instance of a cluster. HERE is where I got to as well as a few previous revisions (1 & 2).

Self Portrait 2 - Interactive

With some help, this self portrait became interactive - click on over to try for yourself (Allow access to the web cam if Chrome asks…) or see the code here…

So a pleasant situation led to this modification of my self portrait. I ran into Jiwon in the hallway who was having trouble with her combination lock. She wasn’t able to get into her locker so I helped her to get it right, and I thought - maybe she would be down to lend a hand on my Self Portrait!

I had the idea to make the brightness on my mac’s webcam raise the eye brows on my p5Js sketch. She is a coding whizz and showed me how to initialize a variable to call the web cam, analyze the RGB values of each pixel in the camera for brightness and convert that analysis into a single average value. I plugged that value onto the anchor points of my bezier eyebrow lines and with a little tweaking - voila!

So I went further and thought that I could get the eye lids, iris, and pupil to do the same, but was having some trouble, so a 2nd year student named ______ came over and used the constraint function to better scale the values, as they were getting way to large when the light was bright. And that’s how we arrived at the final product. You may have to enable your webcam to get the code to run and of course, everyone’s camera will be brighter along with the environment that you are in so FYI.

Here’s the LINK to the p5Js code.

Self Portrait

I already had started Schiffman's Coding Train, although I hadn't used the p5.js web editor yet.  I had been in Processing.  So I had a bit of a head start, but also was happy to (painstakingly) learn the arc(), curve(), and bezier() functions more precisely.  I also wasn't used to a bit of the syntax, but it wasn't much of a hassle.

I knew I wanted to draw my face with the glasses using opacity.  To pick my RGB colors, I used this website and copy/pasted from there.  There may be better sites but that was the first one on Google.. ( I wonder if there is a setup function or class you can load that would allow simple word input for colors?)

I wish I knew a better way to draw the shapes so that the relationships could be maintained if the drawing was scaled, but I could only really do so much with dividing width and height.  Also, I would want to know more about how Reneissance painters use mathematics like the Golden Ration to scale their drawings properly. 

To help with navigation, knowing where my x and y values were as I hovered over say, the rim of the glasses, I put the code into a draw() function and used print(mouseX, mouseY); with Auto-Refresh turned on.  This helped me estimate where to draw lines to and such so that they intersected as needed.

Also, I would // each new line of code so that Auto-Refresh would work until i was ready to preview each new line...

// arc(

But I realized that if I type the function and close the parenthesis that followed, the code would not make the drawing disappear...

arc();

One question I had was - how could I draw the teeth using sin() function? I had to eyeball the curve of the lips and it was a pain in the ass.  I also made a variable for the upper lip values which made things a tad easier. 

var y = height / 1.55 + 2;
  line(168, y, 168, 292);
  line(180, y, 180, 300);
  line(192, y, 192, 306);
  line(204, y, 204, 306);
  line(216, y, 216, 303);
  line(228, y, 228, 296);

I would like to know a more streamlined way of drawing so that the picture could scale if the canvas size changed.  One thing that would have helped was to know if there was a way to reflect something across the screen?

Towards the end, I had to cut myself off from being too obsessive as I was getting into minor tweaks to the eye brows and eye shapes to change the emotion.  Very subtle changes in values were significant in effect!

About computation, my interest

OurMachine.jpg

I'm interested in computation because it is a very powerful form of mathematics.  The fact that its principles can be found in nature in the form of Fibonacci numbers, Mandalbrot fractcals, and the Golden Ratio make me want to understand how to use computation to bring things to life.  I want to create living, evolving, and reactive programs that morph and change in engaging and transformative ways.  

When it comes to music, I think computation is easily applicable, as all harmony is simply mathematical relationships between vibrations.  But there is something interesting that happens when humans experience these relationships as sound, as it often produces an emotional response.  All of my music (at least the best bits) are born purely out of an emotional reaction to a situation that I find myself in.  Understanding computation will allow me to better shape these musical moments as well as add visual and interactive elements to the music - computation as another tool in the creative tool box.

I can imagine making a lot of visually reactive programs that take input from light or sound or drawings that constantly grow and change or are visually pleasing or stunning.  But I am also curious to learn what else I can do with computation, what others are doing, as it could send me down a path I could not have known was there!

I am interested in style-trasnfer like the Glooby project did on a video I saw recently.  Also MaxMSP, Super Collider, NIME topics, and Fabrication.  I'd like to make a digital recorder and some other audio gadgets that I have been needing but never saw anyone offer.

Processing Drawings

This is a set of drawings completed during the introductory courses of Dan Schiffman's Coding Train web series on YouTube.  They are all based in part on his initial code examples.  My code is below.