Week 7

Machine Learning, Noise Divination

Image-based Machine Learning

Having understood the value of the image as a signifier from last week's exercise, I wanted to know how this can be interpreted by machine learning. I chanced upon ml5.js, which makes machine learning accessible for beginners.

To understand how it works, I set out to create a simple system that detects if I was happy, sad or shocked.

Round 1 — Not Confident

Not super accurate. Got 2 out of 3 right(ish).

I thought that it might have been because the samples weren't the same amount. So for the next one, I tried a consistent sample size (50) for each class.

Round 2 — 80-95% Confident

Better, but still not super confident about it.

I then realised that it was only getting the answer right when my face was at certain positions on the screen. I figured it was because all my samples taken was of me in the same position. So, I decided to add more samples of the expression but in different positions.

Round 3 – 100% Confident

After I added more samples of my face in different positions, the machine was able to much more accurate predict the right facial expressions.

As I was taking samples of my face in different positions, I realised that this was how my iPhone set up my Face ID as well. This blew my mind, because understanding machine learning has always been so impossible for me, but as soon as I did this, it made so much sense. Finding Teachable Machine made the process so much more approachable and easy to understand. It was a good step towards understanding better how machine learning works, and what I could do with it in the future.

Farmer Reads the Stars of Code – Design Fiction Exercise

Noise Divination / Pixel Oracle

Now that I understood machine learning a little better, I wanted to try to create a simple system of divination with it. This is achieved in a few steps.

1. Inventing Chance within Code (Object)

2. Words from the Magic 8-Ball (Semantics)

3. Divination through Machine Learning (Code)

1. Inventing Chance with Code (Object)

For this experiment, I set myself the goal of creating a simple system of divination with bare imagery, as little external significance as possible. I wanted to experiment with how to create a skeleton for a divination system in the context of code.

I started with experiment different generations of abstract imagery using p5.js. I ultimately decided to go with just a 1000x1000px canvas (below), each assigned to a binary outcome, black or white. I felt like this embodied the true binary decision-making process of the computer.

(Out of curiousity, I wanted to know what the possibility of generating 2 of the same image was, so I asked ChatGPT to do it for me. It is...*drum roll*... 1 divided by 2/1,000,000, which ChatGPT tells me is a number with over 300,000 digits! I'm not good with math but I think that means basically 0.)

I liken this 1000x1000px canvas of noise to the stars before people learnt how to read the skies. Patterns inperceivable to the human eye, but legible to a more-than-human being, or at least it was at first.

1000x1000 Binary Decisions

So I generated 30 of those images to feed into the machine learning system (abitrary number), 10 for each outcome.

2. Words from the Magic 8-Ball (Semantics)

I then had to decide what to name each of the outcomes – some combination of Yes, Maybe/Try Again and No — but abstract. I scratched my head for a while, and referred to the list of divination objects that I had created for myself. I realised that I had to look for the one that was catered to the widest audience, because the semantics of the outcome would be catered the most ambiguously.

So, I turned the epitome of commercial divination — the Magic 8-Ball. I looked up the list of responses that they gave, and selected 3 that tickled my fancy.

Let me explain why I thought these were funny.

Yes: "Signs point to yes."
– What signs are they referring to? The magic 8-ball god? The CEO of Mattel?

Maybe: "Concentrate and ask again."
– Concentrate on what exactly?

No: "My sources say no."
– Again, what sources?

All 3 of these suggest a "source" for divination, yet keeps it ambiguous enough that it doesn't answer the question. This employs the Barnum Effect, which Geoffrey Cornelius describes:

"The diviner offers statements of an almost universal applicability that seem specific yet will be agreed upon by everyone; or, offers a general statement geared to the particular client’s situation, yet with no obvious falsifiable detail. The client fills in specific details, crediting the diviner with having a marvellous insight into their situation."

That is my hope for the Pixel Oracle (a name I just came up with), that the client assess their subconscious and fills in the gap regardless of how their question is answered.

3. Divination through Machine Learning (Code)

Now that I have my Pixel Oracle designed, I decided to put it to the test. Before that, I had to decide how to put an image in. I thought that, because the machine learned on black and white images, I should stick to a black and white image. So, I created what essentially was an image filter than converts the image into a 1000x1000px canvas of pixels.

I assumed the role of an ancient farmer who came across this magical oracle, and wants to know if he would have a bountiful harvest in the spring. So, he inputs a picture that he took last spring with a magical camera he also chanced upon.

Original Image

Converted into 1000x1000px Image

(I realise that this logic has its loopholes. i.e. the black pixels are generated based on a threshold of brightness, so presenting the same image darker, or brighter could affect the outcome. BUT, that would also be a result of how the camera decided to capture the image, which would function as a second generator of chance. UNLESS, the client manipulates the image themselves, which would function as a chicane, a human intervention.

BUT the image would still be subjected to the Pixel Oracle, that has been trained on an extremely random arrangements of pixels.)

The Pixel Oracle is sure of a bountiful harvest

It tells him, with 99% certainty, that "Signs point to yes". Regardless of his understanding the machine, he is very affirmed and grateful that he was blessed with food for the coming year.

He brings the Pixel Oracle to all his fellow farm friends and spreads the divine word of this new technology. It tells the people what their assigned fate is from birth. It tells people how to become wealthy. It becomes an all-knowing entity that people turn to in times of strife. It becomes the basis in which all of life is run... for a while. Then someone stumbles upon an all-knowing deck of cards...

I started with this experiment because it was an exercise for me to create a "new mode of divination" that did not involve any cultural significance. It provided me with a skeleton from which I can build new modes of divinations with, or to understand how to analyse existing modes of divination. Through this, I was able to understand better the roles that the object, input (image/text), code, and output (image/text), plays in a divination system.

It also evolved into a little design fiction exercise in which I crafted a narrative around my artefact (the Pixel Oracle), a tool which I intend to employ in communicating future artefacts.