Feedback from friends

I got Zarer and Alda to try the website, and got a few feedback points from them. The overall feedback was that the website was not intuitive enough, and there was too much text. The explanation text could come at a later part of the website, and the process of inputting the image and reading could be more descriptive (visually) of the process of noise divination.

Feedback from Andreas

When I showed Andreas the website, he questioned the accuracy of the model. We tested a few images, and most of them were showing "My Sources Say No..." as the answer. I wondered why this happened because all the images that the model was trained on were truly random. I decided to consult Blackbox AI.

Question: My machine learning model is trained on 30 images of 1000x1000px, each pixel is randomly assigned a black or white color, creating a truly random imgae. There are 3 classes and each class is trained on 10 images. However, I seem to get the result "my sources say no" more than the other two. Why could that be?

It suggested some possibilities:

Class Imbalance: Even though you trained on 10 images per class, there might be some subtle imbalance in the data that's causing the model to favor one class over others.

Random Chance: Given that your images are truly random, it's possible that by chance, the "my sources say no" class had patterns that were more distinctive or easier for the model to learn.

Model Overfitting: With only 30 images total, there's a high risk of overfitting. The model might have latched onto specific patterns in the "my sources say no" images that aren't actually meaningful.

Threshold Effects: The model might be more confident in its "my sources say no" predictions, leading to more frequent selection of this class when applying classification thresholds.

Input Processing: If there's any preprocessing of the input images before classification that differs from the training process, it could inadvertently favor one class.

Model Architecture: The specific architecture of your model might be more prone to detecting features present in the "my sources say no" images.

Random Initialization: The initial random weights of the model could have led to a bias towards one class that persisted through training.

Testing Data Distribution: If you're testing on a small number of images, you might be seeing a temporary bias in results that would even out with more tests.



Overall, I think this was a great exercise into initial user testing and getting feedback for the website. I was able to identify a few problems that I could work on from here.

A few changes I intend of making are:
1. Overall, rethink the user experience and the amount of text it takes for them to understand the concept. I want to experiment with the experiment as a physical interactive experience, maybe with a camera.
2. Test the model again to see what the problem is, and why one result is more common than another.