My Diary Design Process Team Re. Flection Youngdong Kim (ykim2720) Nattinee Srikasumbun (nsri2842) Yoko Tomishima(ytom4097)
1. How the concept was built Initially we tried to combine 3 team member s ideas together but we found this to be very difficult so we decided to pick the strongest one which was one of one of our concept self reflection. In addition to the concept self reflection, we came up with the idea of using a book where each page shows time based media such as videos and sounds. We combined both concepts and came up with our final concept My Diary a book of emotions. My Diary allows the audience to remember past memories by self- reflecting through the user of emotions. In our book there are four color pages, each represent the different four emotions such as happiness, sadness, anger and fear. Webcam captures the color which is displayed on the left side of the page. From this a system has been programmed to project a responsive video on the right side of the page to match the color therefore when page is flipped the video display will change as well.
2. Concept development process Our tasks are divided into 4 parts such as a book making, Filming, editing sound and video, and Coding. 2-1. Book design From the initial concept we all agreed to make our book look like old, we designed it as an antique book. First we started with the book as we want it to look real as much as we want it to look old and magical. The process start with searching how to make old texture on paper on Youtube. We researched and found some tutorials that tell how to make a paper looks old. The following websites gave helpful information on how to make an old looking book: http://www.wikihow.com/make- Paper- Look- Old https://www.youtube.com/watch?v=qihuu7k7sca
We wanted to stick to authentic looking antique book, however we encountered difficulty in creating old looking paper. We failed to make the paper look old firstly because the paper size we used was too large so the colour was uneven and papers started to become warped which made it impossible to bind into a book. This warping would also have more problems as it would distort the image when the video is projected onto it. We eventually decided on printed grunge textures and colors on papers. The final paper we picked is A3 paper 300 gram thick and math giving the best outcome as the pages are not too thick not too thin and the colour are quite flat. This also keeps the paper from reflecting light that would affect, how the webcam will read the colour and when we display video it wouldn t have wierd light reflecting on it. When we collated the paper to make a book it was too thin for case binding, so we sticked fabric tapes to the pages to make it thicker. Since this is our first time to make a book, we carefully scaled everything and made a diagram to not to be wrong with the scale. Even though we prepared the middle part didn t have enough gap so we couldn t close the book. We thought up the following solution for this, to make a plastic book cover so that we didn t need to stick the page and book but we could insert the first page to the plastic cover. Also it is good when we stick to the table to make it stable.
Making process 1. First make a lot of coffee in a container and wait till it warm. 2. Then put paper on a tray and pour coffee onto the tray just a bit over the paper. Use brushes to make it uneven to make it look like older. 3. Put the tray in the oven. Bake it for around 1-2 mins. 4. Take the papers out from the oven and dry it for a while. We did testing the color of coffee by testing with some smaller paper till we get the color right, the outcome looked amazing so we started working on with the real paper. The end of this experiment is not what we though because the actual paper size we use is really big and the biggest tray we have was still not big enough so the outcome of after baked paper was quite disappointing. The paper we baked were not suitable to use for our project as the color was strangely uneven as the paper bigger than the tray so one side of paper sticking out when the other side is soaking with coffee. Also after the paper became wet and
dried it started warping weirdly, as this will cause problem to the binding and how the video will look on the paper. The best solution we came up with is we are going to print an old paper texture onto papers. Colour range testing template is printed on A4 paper and tested with the webcam to get the color number that work best with the webcam before final printing on the actual paper. The book template design printed out really well on A3 300 gram math paper. Then we started to binding the book with very thick paper board cut into 2 pieces of A3 size, old red paper texture to make the book cover, white thin fabric to hold between the cover thick paper and red paper wrapping, ribbon to hold each page corner before binding and glue. First we prepared the inside pages by glue ribbon onto one corner of each page and leave it to dry. then we put all the pages together then sawing the corner that has ribbon on together to make it stronger then wrapped up the corner with white fabric. We have finish up the binding following instruction from Youtube.
Book decoration The colour pages looked so plain so we decided to decorate it with real life objects such as dried leaves, feathers and other things that we can find in daily life the outcome is as shown in the picture above. 2-2. Filming
During development process, we had to change some of our plans because scheduling was hard. These difficulties arose due to the following: equipment, actor s availability, the weather and location availability. With locations we had further problems due to areas under construction or too popular therefore a lot of people where there. Initially we were planning to go somewhere suitable whether it s far or not but because of timeframe and limitations we chose our shooting locations somewhere around the City. Luckily, we sometimes found some nice locations by chance during shooting so we sometimes could record some good moments. 2-3. Editing - Video - Our concept is evoking feelings to audience so each page has to attract the audience and remind them of some kind of memories so we designed our videos to be emotional. We used motifs and colors to represent the emotions and we also added words / quotes that are related to the emotion. Some videos
have faster cuts and camera movement to express some feelings. Also we used color grading to make it look like retro & vintage. Rather than just placing three videos on a white page we made layouts for each video and designed it like an antique photo book. Animations and effects were added to make it more magical like Harry Potter s magic book. We made opening sequence for the cover, first and last page to show the audience this is an interactive book.
Using multiple types of softwares was difficult. We used programs such as Final Cut Pro X, After Effects and Premiere Pro for editing the sound. The reason we used so many different programs was because each software provided its own unique feature and we needed them to achieve what we wanted for the final outcome. We did encounter some problems by using so many programs. One of these problems was file size, while editing each individual video, the Final cut Pro X file became massive (around 600 GB). We had to consider alternative ways and decided to use After Effects and Premiere for the rest of editing. This turned out to be more suitable as After Effects and Premiere link each other and have more effects options. - Sound - We tried to use surround sound for audiences emersion. The problem was we could not use 5.1 channel sound editing with the knowledge we currently have. So we compromised that we were going to use 2 channel sound. Even though it was only 2 Channel we made the sound directional and no coming from the one side. Basically the background sound is the same from both speakers, but the effect sounds are controlled such as appearing one side or weakly appearing another sound for more effective and making
a feeling of the space. The sound sources are from freesound.org and sounddogs.com and we used two sound editing programs to edit the sound, which are Audacity and Soundbooth CS5. Sound design is intended to involve the audience feeling toward each page of emotion. We brainstormed for video selections for each emotion, which include the sound that matches with the picture the audience will see. This also includes the videos and other sound effects that would relate to that emotion and building upon the attachment to audiences memories. Happy The happy sound design that merry go round will be the main background music as it match with the place where we shoot our actor, the sound effect used in Happy emotion are intended to give the feeling of fresh, sunshine, warm, and relaxing.
Sad When we first design the sad sound the first thing that came in our mind is raining or water drop, slow wave of ocean moving and a sad violin is used as background music. The final sound we want it to feel lonely, empty, image water drop from rock on to the other rock near the sea.
Angry The sound design for angry used thunder and storm as background sound as we think thunder and storm represent anger. We wanted to create a story around it. The couple fighting and arguing are used along the sound design but we put it quite low, broken glass appearing in some parts and also the people fighting then silent use for danger feeling and push the audience to get more attached with the video they are watching.
Fear The Fear sound a quite creepy as we used a disturbing music box to as a background sound and the sound effect we used such as squeaking door, bat sound, crow sound, heard beating. We want the audience to feel dark and scared to remind them of their fear memories.
2-4. Coding Coding for this project, it was the first time we used the hue colour range control for detection. We also attempted to use saturation; unfortunately we could not use this method. The light caused the colour to become slightly saturated and it also caused some confusion to the computer. Hue range was really tricky to control. The colours we see and the colours being detected by web camera are totally different. To fix this detection method we decided to use RGB colour detection, and that is not used for a range. We used it to detect which colour is displayed by how close the value summed up of RGB for checking black or white. BUT there were serious 2 big problems: The first big problem was the accuracy. If we set the sum(rgb)ranges for distinguishing between black, white or colours, computer could not detect colours correctly. This was because distortions in the light, distance from ceiling and shadows caused different sum results. The solution for this problem was to set the area being captured, before we only captured 1 pixel. The capturing area (10x10pixels dimension) s values returned to be averaged. So even if some little shadows from the audience or if lighting s power or numbers changes, computer can detect very well. This improved the accuracy a lot. The second problem was that the computer could hardly distinguish between black and white. Also this depended on the light position. In some situations even when I was looking with my eyes white and black looked the same. After requesting help from Phil, provided advice to use brightness. As soon as we checked the brightness values difference, we found out the brightness gap between black and white. And also we put some decoration for the book. Black page s been decorated with some black texture on it (no more glossy). Now it is clear with a use of brightness as well. But still we need to set the values like sum and brightness depends on the location.
3. Prototyping The prototype that we made was for a demonstration about working with colours. At this time the book was not completed yet so for testing purposes we used colour papers instead. At that time, we didn t have any videos, so we used demo versions of video. The test was good enough, so we moved to the next stage, which was filming. After filming we tested the size of the video because we will project on the A3 size book, which is quite unusual shape and size for video editing. We had to make sure the dimension and aspect ratio was correct so we don t need to resize later. We made prototype for just testing videos because we didn t have sound and book ready yet. Next prototype that we made was the sound in the video. As we had a sound editing experience in the sound design class this section wasn't quite as difficult. Commonly sound is really important to allow people immerse their feelings in a film. Our documentaries feeling is the most important part of our project. We also tried to use the Foley sound and surround sound for our project. Fortunately the prototype of the sound was good enough to fit with the video. After making prototype sound, we evaluate it as we editing the video. Then finally we combined as final prototype, which sound and video are working with codes.
4.User testing From in- house exhibition, we noticed some findings. Upon the questionnaires we created to find out what extent our interaction succeeded, we noticed some findings. We interviewed some audiences after finish interacting. Most audience liked our interaction. Successfully evoked some feelings. Successfully reminded them some memories that is related to the emotion Actor s face expression and movement are very effective
Interaction Interaction was successful. Simple interaction didn t confuse audience and effectively affected to audience s feelings. Typical interaction pattern was like this 1.Audience comes 2.Think about what this is 3. Open the book 4. Flip the page 5. Watch the 4 videos 6. Leave Some people flipped few pages at the same time so they didn t realize that there are four pages in our book but most people tend to watch four videos and then they left. Some people tried experimentation. They flipped the page to go back the page before and see what will happen, but our book was designed to respond to the color so it worked well in that situation. Some other people watched videos for longer and few more times. Reactions from audiences were interesting. Some people are smiling, some people were very serious, and some were scared. They seemed like they were thinking something during interaction. From our interview, we want to say that our installation was successful because we successfully evoked some kind of feeling to audience. Depends on the audience the most effective page was different. But overall, our four videos and sounds, which represent emotions, worked well and remind them of some memories. Recommendation There was a recommendation from someone who came with Dr. Lian. She was surprised and said it was brilliant idea. And she commented there was a better way to distinguishing page with an interaction. In another similar project, they used sensor to be detected with attaching on the paper. Also our idea was good enough but the use under some light may not be surprisingly accurate. If we have a chance to improve, we are willing to find the way that she said of course. Here are some words from the audience ^^ So nice!!!! That s really cool guys!!!
It s really cool and I feel that each pages I can feel the emotion Yes I can feel the emotion, I really like the concept where I open the book and things are animated the idea is really great. It s like Harry Potter world It s great I love the design graphic and even the background layout is really detail and the way you made all three video like theme scenario are connected you can just related directly what sort of mood you are trying to express. Yea even without words it s pretty obvious, This is great I really like it and fear emotion page are working most for me like the depres the darkness and the happy page it has detail like the flower bright colours those two are the most obvious one it s great!! but the actor kind of scared me a bit It s like magic reminds me of harry potter it s really good, that s really great and my favorite one is the sad one because we are depressing haha, yes I feel attach with the emotion in each page especially the sad one and the anger one that s great I like it and the happy page is my favorite one it s very colourful, I feel happy when I look at the page I feel connected with the emotion, each page bring back difference memorie Lastly Over all we all really enjoyed this project My Diary. We are from different profession, but helped each other and managed the tasks very well. From the initial concept proposal we discussed a lot until everyone says ok. We researched a lot until we can get the quality we want. We always contact each other and if there is any problem, every team member gather up and discussed about that problems. We all made efforts to make it work. As a result, our interaction was quite successful. Many audiences liked our project. It was pleasure to see what our efforts come together. We are so happy with the outcome. Thanks for everyone who involved our project! Team Re.Flection