Surveillance Design Final Project
Process & Concept
When you read a book that is full of imagery words, What ís going on inside your mind? What images do you have in your head when you reading?
Here is my project entitled “Mapping the Imagination.”
So, my project can be translated as “A Visual Practice of Human Imagination and Image Recognition from Text Redaction with Photo Data on Google Search.” So, this is my zine. For me, it was a really meaningful experience making a zine by my hand. It was almost first analog design project after DT and so much fun.
Along with the question that the machine can really recognize the images based on human imagination, I have been particularly interested in these areas of Human Imagination, Cognitive Psychology, Photo Vis,Computer Image Recognition Technology, Artificial Intelligence, Linguistics. So, this project was also an experiment to explore the realm of human cognition and artificial image recognition.
Inspired by some zines that Melanie showed us in the class, in terms of the design process and the overall concept, I did the similar way of text redaction.
As you can see, I cut each word in the text out in square boxes and there are two kinds of colored lines of blue and pink. The texts are from a famous fiction written by Marcel Proust – “In Search of Lost Time.”
Swan’s Way – Chapter 1 – In Search of Lost Time
The reason I chose this text for redacting is this fiction is well known as the most imaginative literature that is full of sensibility in human history, and this is one of my favorite fiction.
One thing that is also critical for this project is the process to search for photo data on Google.
As you know, there are a massive amount of big data including photo data, and the algorithm of image recognition by computer is unprecedentedly evolving based on those data.
but as you can see, in the process of choosing Images that match with words is based on human imagination and it’s an arbitral process.
So in my opinion, even though the computer can recognize text and match with the photo quickly, there is still a room for human imagination that affects the choice in visual search. Through this visual practice, I wanted to illuminate the fact that human visual system and imagination is still a rich source of information. And modern surveillance system cannot invade a person’s mental privacy yet because the cognitive processes of individuals are based on each person’s unique experience in the life and creative imagination even though the artificial intelligence like Google “Alphago” are evolving and even developing the creative process on its own.
After the final presentation, I really wanted to develop this project more by figuring out the correlations between the words and my cognition process of choosing those images.
So, for the further development, I made a diagram in a form of the semantic neural network with D3.js and JSON data to clearly visualize the invisible cognitive process of the arbitral image choice of mine and to figure the correlation between the words that I chose out. The chunking of words was exactly representing the images that I chose. So, It reveals the fact that the choice of images is not just a randomize, quick choice but a sophisticated cognitive process of human image recognition.
(below) JSON dataset that I reversely extract words from images.
Many studies in Machine learning and Artificial Intelligence are trying to extract visual classifiers from the human visual system. There is a field of Artificial imagination (AIm), also called Synthetic imagination or machine imagination which is defined as the artificial simulation of human imagination by general or special purpose computers or artificial neural networks.
I wanted to visualize physical data in real time, this was a goal in Large System class.
Arduino + Plotly Data Visualization – a real time data visualization of Arduino humidity detector.
Data Visualization class of 2016
Interactive data mapping project with CartoDB, GoogleRefine, Excel
Data Mining Process
Collaborated with Tzi Wu and Xiaomeng Tang (Parsons Design & Technology MFA students), we visualized the whole amounts of historical UFO sightings index by using the data listed on Wikipedia. (UFO sighting wiki: https://en.wikipedia.org/wiki/List_of_reported_UFO_sightings)
We wanted to present a dataset that shows how people see and interpret unexplained phenomenon of daily life, which UFO and aliens could be the symbolic representatives that illustrate the idea. The final map shows bubble charts on the world map which represents the numbers of UFO sighting by each location. Also, viewers can see the entire features UFO sighting has been exponentially increasing as time goes by. Also, users can simply see the change of the amount of in UFO sightings by dragging a timeline bar in this interactive map.
Surveillance Design Week 4 – Invasive Looking Prototyping
For the concept of invasive looking, I would suggest a new desktop dashboard UI that shows our daily life data gathered by mobile devices and social networks.
Nowadays, everything is connected with our online identification and all the devices and social networks are gathering the daily data and activities in every seconds in case they can access to my account.
So, I imagine a dashboard that collects all the daily data and analyze them with algorithms and then visualize the data in a creative way to let us know the daily pattern at a glance.
To make a visual example of the dashboard, I gathered my actual daily data by a health tracking app, my scheduler and other social networks and then manipulated the data in Excel and Rstudio.
The prototype is a bit crude now, but I plan to develop this for a better version by finding some hidden patterns with the daily data that I’ve gathered. Also, I want to make an interactive version of it to connect those data in real time by using API and social network widgets.
For the context of surveillance design, It would be a surveillance system that is a kind of self-monitoring and self-regulation. The system can give some recommendations for an individual’s life based on the data. Also, by using those metadata, companies and social network services can figure a person’s hidden needs and behavioral pattern by analyzing the connectivity between each daily activity of an individual.
I used Rstudio to visualize the data of the nutrition facts in Starbucks menu.
[Practice in Rstudio]
PROCESS ON ILLUSTRATOR
After manipulating the data on Excel and Rstudio,
I loaded it into illustrator to make them more aesthetic.
I collected each image of the beverage to classify them into the chart.