GUFFROY Gregory DUT Informatique N4 07 th November 2015 18 th January 2016 Internship Report Glove interaction system using Arduino and Processing Hachinohe National College of Technology IUT A département Informatique, Université de Lille 1 Tutors : Mr HOSOKAWA Yasushi Mr LEBEGUE Patrick And with help of : Mr TENMA Ryotaro Mr SAKAMOTO Kazuya GUFFROY Gregory Internship - Hachinohe 1/31
GUFFROY Gregory Internship - Hachinohe 2/31
Acknowledgments First, I would like to thank both Lille 1 university and Hachinohe National College of Technology staff who helped me during this internship. I would like to thank Mr BABA, the person in charge of international relations in Japan who explains me many things about the college and Mr Lebegue the person in charge of the internships abroad in France, who helped making this internship possible. Then, I would like to thank Mr Hosokawa, my internship project tutor who was always available to help me, and my two student tutors, Mr Tenma and Mr Sakamoto for helping me in daily life in Hachinohe Kosen. Finally, I would like to thank all the people I meet during this internship, teachers, students, staffs and strangers who helped me enjoyed this three months. GUFFROY Gregory Internship - Hachinohe 3/31
Abstract To graduate of the diplom I am preparing, I made a three months internship in Japan. With a given project, and with the help of my tutors, what I had learned during the DUT and the new skills learned was evaluated. I chose to make my internship in Japan because it was an opportunity to learn more about a different foreign culture and a different way of life while improving my english and japanese languages skills. My project is to make a glove using Arduino and Processing making it possible to modify an 3D object. The goal of this project was to be able, with its hand, to deform an 3D object like a ball or a cube. This project was made in 4 steps. The first part was to learn about Arduino, Processing and electronics for the glove. The second part was to write programs, with Arduino for recovering the data from bends sensors and with Processing for create a 3D model and to modify it with the data. Then I assembled the glove and I ended by binding the whole. Thanks to this internship in Hachinohe, I learned to work in autonomy, respecting imposed delays. That also helped me to analyze the problems to find the solutions most suitable. GUFFROY Gregory Internship - Hachinohe 4/31
Résumé Dans le cadre de la fin de mon DUT informatique, j ai effectué un stage de trois mois au Japon. Avec un sujet donné, et avec l aide de mon tuteur, ce que j ai appris durant ma formation et les nouvelles connaissances acquises au cours du projet ont été évaluées. J ai choisi de faire mon stage au Japon, car il s agissait d une opportunité d en apprendre plus sur une culture et un mode de vie différent tout en améliorant mon niveau en anglais et en japonais. Mon projet est de réaliser un gant utilisant Arduino et Processing permettant de modifier un objet 3D. Le but de ce projet était de pouvoir, avec sa main, déformer un objet 3D comme une balle ou un cube. Ce projet a été réalisé en quatre étapes. La première consistait en l apprentissage d Arduino, de Processing et de l électronique pour le gant. Ensuite, j ai commencé à écrire les programmes, celui récupérant les données des bend sensor et celui qui créait le modèle 3D et qui le modifier avec ses données. Ensuite, j ai assemblé le gant pour finir par lier le tout. Grâce à ce stage à Hachinohe, j ai pu apprendre à travailler en autonomie, tout en respectant les délais imposés. Cela m a aussi aidé à analyser les problèmes pour trouver les solutions qui étaient les plus appropriées. GUFFROY Gregory Internship - Hachinohe 5/31
Table des matières Acknowledgments... 3 Abstract... 4 Résumé... 5 Introduction... 8 Internship presentation... 9 I. Environment... 9 1. Aomori Prefecture... 9 2. Hachinohe city... 9 3. Hachinohe Kosen... 9 4. The Laboratory... 9 II. Internship Project... 10 1. Arduino... 10 2. Processing... 10 The project... 11 I. Learning Basics... 11 1. Interact with Arduino : LEDs, Toggles switch, button IR... 11 2. Interact with Processing: basics examples... 11 II. Programming parts... 11 1. Recover value from a bend sensor using Arduino... 11 2. Create 3D model using processing... 12 III. Electronics part... 13 1. The glove... 13 III. Making the programs communicate... 14 1. Arduino side... 14 2. Processing side... 14 Side Ameliorations... 14 I. The glove... 14 II. The 3D model... 15 Conclusion... 15 Appendix... 16 Appendix 1 : Map of Japan... 16 Appendix 2 : Aomori Prefecture... 17 Appendix 3 : School System Of Japan... 17 Appendix 4 : Mr Hosokawa s laboratory... 18 Appendix 5 : Arduino Uno... 18 GUFFROY Gregory Internship - Hachinohe 6/31
Appendix 6 : Arduino Interface... 19 Appendix 7 Processing Interface... 20 Appendix 8 : Toggles switch, button IR... 21 Appendix 9 : Arduino program... 22 Appendix 10 : Processing program... 22 Appendix 11 : the glove... 26 Appendix 12 : IFC club... 27 Appendix 13 : Animation in a children school... 28 Appendix 14 : France Presentation at Hacci (Hachinohe Portal Museum)... 28 Appendix 14 : Mathias, Takeshi and me in Asakusa street... 29 References... 30 Glossary... 31 GUFFROY Gregory Internship - Hachinohe 7/31
Introduction As a second year student at the Computer sciences department of Lille 1 s IUT-A, I have to finish my studies by an internship. I chose to make my internship in Japan because it was an opportunity to learn more about a different foreign culture and a different way of life while improving my english and japanese languages skills. This internship was held from 1 st November 2015 until 18 th January 2016 to Hachinohe National College of Technology in Japan. Not Having never used electronics before this project, it was a first for me and I had to learn the basics of making a device with Arduino. How to build a devise included which who can recover folded values of finger and use these values in another program installed on a computer? Firstly, I will introduce the Hachinohe National College of Technology, its environment, and a detailed presentation of the project. In a second part, I will explain the analyse and the development of the project. Finally, I will conclude by this internship benefits, both educative and human. GUFFROY Gregory Internship - Hachinohe 8/31
Internship presentation I. Environment 1. Aomori Prefecture Aomori prefecture is the northern part of the Tohoku region. The capital of Aomori prefecture is Aomori city. Aomori stands for blue forest. As blue is often used in Japanese to define green things like traffic lights or plants. Thus the true meaning is green forest. Aomori prefecture has a total area of 9606.26 km2 and 1 373 thousands of inhabitants. It is famous for its big forest and for its apples. 2. Hachinohe city Hachinohe is the largest city is the east part of Aomori prefecture. Hachinohe is an important industrial city and has important fishing and industrial ports. The symbol of Hachinohe is a wooden horse with a gold saddle, the Yawata-uma. 3. Hachinohe Kosen First, I will explain what is a kosen and then I will present Hachinohe kosen. A kosen is equivalent to high school and the beginning of unniversity in France. It s a five years formation whitch can be followed by a two years advanced course. The kosen can be accessed by 15 years old student and at the and of the 5 years course, the student can choose whether to go work or to go to the advanced course. Hachinohe National College of Technology was created around 1962, there are four departements, Mechanical Engineering (M), Electrical and Computer Engineering (E), Chemical and Biological Engineering (C), Civil and Environmental Engineering (Z), Every department is distinguished by a letter, and every level by a number. During this internship, I join the E5 class. 4. The Laboratory For graduate, Every fifth grade and advanced course students have to create a research project. So students choose a project depends on their speciality and join the laboratory corresponding. Every teacher have a research specialist on their laboratory. I worked in Hosokawa s laboratory, with 10 others students. For my internship, I decide to work with my own laptop GUFFROY Gregory Internship - Hachinohe 9/31
II. Internship Project The aim of my internship was to make a device, able to recover values from bends sensors. Then I had to developp programs which use this feature. 1. Arduino Arduino is an open-source software and hardware created in 2005 in Italy. It was developed to teach electronics and programming. The IDE and the way a program run is based on Processing which is based on Wiring. The language used with Arduino is based on C language but has some changes. Arduino uses physical computing. It means using hardware and being able to interact with physical interactive systems. In an arduino program, you have to fill at least two methods : setup() and loop(). The first one is made to set up all the parameters and to make first preparation. When setup() instructions are done, the loop() is played repeatedly as long as the Arduino is switched on. It was not too hard to learn Arduino as it is not a complicated language. With Arduino interface, you do not need to compile it yourself and it is sent to Arduino device easily. 2. Processing Processing is a programming language founded in 2001. It is based on Java but its main use is for graphical purposes. Consequently, doing graphical interfaces in Procesing is far more easy as it is in Java. At first, Processing was used to teach programming basics in a graphical way, in order to be more easily understood. But now Processing is used in professional projects.getting used to the Processing IDE or the Processing language wasn't too difficult because I already knew Java. There are slight differences between Java and Processing. The main one is that the main class is composed of two methods : setup() and draw(). Setup() is called when an instance of the class is created. It is both a main and a constructor. The second method, draw(), is a method that loops as long as the program is running. The other particularity is that the global variable of the main class don't have visibility levels : they are always public, even if declared as private. The same applies to the methods. GUFFROY Gregory Internship - Hachinohe 10/31
The project I. Learning Basics 1. Interact with Arduino : LEDs, Toggles switch, button IR Not having used Arduino that only once during my studies, during the first days of my internship, I had to learn how to use it again. First step, in order to know if Arduino were connected to my laptop, I used some example program to make the leds blink and I read many books. This step finished, Mr Hosokawa gave me a program and asked me to adapt it. This program consisted of the fact of recovering values of ToggleSwitch, a button and a Infra Red sensor and to display them in the terminal of Arduino. Once finished, I had understood how to function Arduino which is really easy to take in hand for beginners. 2. Interact with Processing: basics examples Not having never used Processing during my studies, I have once again during the first weeks learn how to use it. Like for Arduino, I then started to test different programs example available on the web in order to understand how Processing work. Then after having understood how Processing work, the bases of his programation language, I read books on the bases of 3D with Processing, because the creation of 3D model with Processing was one of the bases of my internship. After several days, I finally understood how really work Processing. II. Programming parts 1. Recover value from a bend sensor using Arduino The first step is to define the ports with which, one will work. After you declaring two integer variable, «Sensor» and «degrees». With the first variable, you read and recovering the voltage from the voltage divider and with the second variable, you convert the voltage reading. For that you use map() function who use 5 parameters. GUFFROY Gregory Internship - Hachinohe 11/31
The first parameter is the «sensor» you want to convert, the first two numbers are the sensor values for straight (360) and bent (500), and that second two numbers are the degree readings we ll map that to (0 to 100 degrees). And after you print out the result. 2. Create 3D model using processing Create 3D model with processing is really simple because in Processing, you can found basic 3D model like box or Sphere but for my internship, I shoose to create 3D box model by myself because If in Processing you can make 3D very easily, it s more difficult if you want to modify it, for this reason I make my proper 3D box model. So you have to shoose what do you want like model? Me I shoose a box. First, like you want to create a 3D form, you have to specify it in setup() function like that. The first two numbers are the size of the window and P3D is that you want draw 3D shape. After, for make a 3D box model, you have to create a function box() and in this function you can create a complex forms and for that I used beginshape(parameters), parameters is what types of shapes you want like QUADS for me, and endshapes() for stop recording. Between this functions, I constructed my shapes by connecting a series of vertices with vertex() function. GUFFROY Gregory Internship - Hachinohe 12/31
Vertex example After that, for draw my box, I create a function with some specification in the draw() function. III. Electronics part 1. The glove Before to make the glove, I thinked what I want to do with this glove? The respond was modify x, y, z axes for deform my 3D model. So, for make the glove, I used different component : - A glove - Arduino uno - Breadboard - 3 Bend sensors - 3 resistors So, for modify x axe, I put a bend sensor on the thumbs, for y axe I put a bend sensor on the forefinger and for the z axe I put the bend sensor on the third finger. And I put Arduino Uno and the Breadboard on the back of the hand like that. GUFFROY Gregory Internship - Hachinohe 13/31
III. Making the programs communicate 1. Arduino side To send the data to the computer, we need to establish the communication. The communication is established in the setup() method. The communication runs at 9600bauds, which is the usual communication speed The easiest way to send the data is to send a String but for my program, I decide to send int value directly. 2. Processing side With processing, to communicate with other devices, the program must import the processing.serial package. First, we have to find on which USB port, the Arduino device is connected, after we have just need to detect a serial communication,receive the data and tell the program to use them. Side Ameliorations I. The glove The glove that I have fact is simple and can only modify X axes, Y axes and Z axes. To improve the glove, It would be necessary to place bend sensors on all fingers, between fingers and on the palm of the hand in order to provide to recognize and determine the exact shape of the hand. GUFFROY Gregory Internship - Hachinohe 14/31
With the exact shape of the hand, deforme 3D model will be then much more realistic. II. The 3D model The 3D model that I have fact remain simple, I can only modify the size of X, Y, Z axis, and transform the basic squarre into more or less a rectangle. To improve it, it would be necessary to provide to modify 3 axes but also all the others to be able to deform the basic square into something who is not a rectangle. And for that, it is necessary to be able to collect the exact form of the hand. Conclusion This internship in Japan was an opportunity to practice what I learned during the two years of my study in Lille university. All along the realization of my project, I learned a lot of news skills with Arduino and Processing with the help of my teacher and also many things about Japan and life with the help of Hachinohe student. I was able during this internship to develop a functional device using Arduino and Processing and in the end it give a good result. I had still points that could be improved like the number of bend sensors on the glove but I am proud of what I created during this three months. Finally, I want to thank again every teachers, staffs, and students who helped me during this internship. Without them, I would not have enjoyed this time as much as I did. GUFFROY Gregory Internship - Hachinohe 15/31
Appendix Appendix 1 : Map of Japan GUFFROY Gregory Internship - Hachinohe 16/31
Appendix 2 : Aomori Prefecture Appendix 3 : School System Of Japan GUFFROY Gregory Internship - Hachinohe 17/31
Appendix 4 : Mr Hosokawa s laboratory Appendix 5 : Arduino Uno GUFFROY Gregory Internship - Hachinohe 18/31
Appendix 6 : Arduino Interface GUFFROY Gregory Internship - Hachinohe 19/31
Appendix 7 Processing Interface GUFFROY Gregory Internship - Hachinohe 20/31
Appendix 8 : Toggles switch, button IR GUFFROY Gregory Internship - Hachinohe 21/31
Appendix 9 : Arduino program #define sensorpin1 A0 #define sensorpin2 A1 #define sensorpin3 A2 void setup() { // initialize serial communications Serial.begin(9600); void loop() { int sensor1, degrees1, sensor2, degrees2, sensor3, degrees3; // read the voltage from the voltage divider sensor1 = analogread(sensorpin1); // convert the voltage reading degrees1 = map(sensor1, 360, 500, 0, 100); sensor2 = analogread(sensorpin2); degrees2 = map(sensor2, 490, 545, 0, 100); sensor3 = analogread(sensorpin3); degrees3 = map(sensor3, 465, 520, 0, 100); // Send the result Serial.write(degrees1); Serial.write(degrees2); Serial.write(degrees3); // pause before taking the next reading delay(300); Appendix 10 : Processing program import processing.serial.*; Serial myport; int valeurxr,valeuryr,valeurzr; int Longeur = 400; int largeur = 400; int hauteur = 400; int valeurxf = 0; int valeuryf = 0; int valeurzf = 0; GUFFROY Gregory Internship - Hachinohe 22/31
int tmpvaleurx,tmpvaleury,tmpvaleurz; IntList tabx = new IntList(); IntList taby = new IntList(); IntList tabz = new IntList(); int volume = Longeur*largeur*hauteur; void setup(){ size(800, 800, P3D); String portname = Serial.list()[0]; //change the 0 to a 1 or 2 etc. to match your port myport = new Serial(this,portName, 9600); void draw() { while (myport.available() > 0) {// If data is available read it and store it valeurxr = myport.read(); valeuryr = myport.read(); valeurzr = myport.read(); //ON VERIFIE SI LES VALEUR SONT INFERIEUR A 0 OU SUPERIEUR A 100 ET ON LES MODIDIES SI BESOIN if(valeurxr > 100 ){ valeurxr = 100; else if(valeurxr == -1){ valeurxr = tmpvaleurx; if(valeuryr > 100){ valeuryr = 100; else if(valeuryr == -1){ valeuryr = tmpvaleury; if(valeurzr > 100){ valeurzr = 100; else if(valeurzr == -1){ valeurzr = tmpvaleurz; //ON CREER UN TABLEAU DE INT[10], ON ADDITIONNE TOUTE LES VALEURS ET ON LA DIVISE PAR 10 POUR STABILISER LES DONNEES RECUES //POUR X if(tabx.size() == 10){ tabx.remove(0); tabx.append(valeurxr); for(int i=0;i<tabx.size();i++){ valeurxf = valeurxf + tabx.get(i); valeurxf = valeurxf / 10; else{ tabx.append(valeurxr); GUFFROY Gregory Internship - Hachinohe 23/31
//POUR Y if(taby.size() == 10){ taby.remove(0); taby.append(valeuryr); for(int i=0;i<taby.size();i++){ valeuryf = valeuryf + taby.get(i); valeuryf = valeuryf / 10; else{ taby.append(valeuryr); //POUR Z if(tabz.size() == 10){ tabz.remove(0); tabz.append(valeurzr); for(int i=0;i<tabz.size();i++){ valeurzf = valeurzf + tabz.get(i); valeurzf = valeurzf / 10; else{ tabz.append(valeurzr); //ON DESSINE LE CARRE dessiner(longeur-valeurxf,largeur-valeuryf,hauteur-valeurzf); //ON AFFICHE LES RESULTATS print("tabx = "); println(tabx); print("taby = "); println(taby); print("tabz = "); println(tabz); print("x = "); print(valeurxf); print(" Y = "); print(valeuryf); print(" Z = "); println(valeurzf); //ON REMET A 0 CHAQUE VALEUR FINAL valeurxf = 0; valeuryf = 0; valeurzf = 0; //ON SAUVEGARDE LA DERNIERE VALEUR DE CHAQUE DONNEES RECUES POUR LES TESTS tmpvaleurx = valeurxr; tmpvaleury = valeuryr; tmpvaleurz = valeurzr; GUFFROY Gregory Internship - Hachinohe 24/31
void dessiner(float x,float y, float z){ background(0); camera(mousex, height/2, (height/2) / tan(pi/6), width/2, height/2, 0, 0, 1, 0); translate(width/2, height/2, -100); stroke(255); nofill(); mybox(x,y,z); void mybox(float size) { mybox(size, size, size); void mybox(float w, float h, float d) { float x1 = -w/2f; float x2 = w/2f; float y1 = -h/2f; float y2 = h/2f; float z1 = -d/2f; float z2 = d/2f; beginshape(quads); // front normal(0, 0, 1); vertex(x1, y1, z1); vertex(x2, y1, z1); vertex(x2, y2, z1); vertex(x1, y2, z1); // right normal(1, 0, 0); vertex(x2, y1, z1); vertex(x2, y1, z2); vertex(x2, y2, z2); vertex(x2, y2, z1); // back normal(0, 0, -1); vertex(x2, y1, z2); vertex(x1, y1, z2); vertex(x1, y2, z2); vertex(x2, y2, z2); // left normal(-1, 0, 0); vertex(x1, y1, z2); vertex(x1, y1, z1); vertex(x1, y2, z1); vertex(x1, y2, z2); // top normal(0, 1, 0); vertex(x1, y1, z2); vertex(x2, y1, z2); GUFFROY Gregory Internship - Hachinohe 25/31
vertex(x2, y1, z1); vertex(x1, y1, z1); // bottom normal(0, -1, 0); vertex(x1, y2, z1); vertex(x2, y2, z1); vertex(x2, y2, z2); vertex(x1, y2, z2); endshape(); Appendix 11 : the glove GUFFROY Gregory Internship - Hachinohe 26/31
Appendix 12 : IFC club GUFFROY Gregory Internship - Hachinohe 27/31
Appendix 13 : Animation in a children school Appendix 14 : France Presentation at Hacci (Hachinohe Portal Museum) GUFFROY Gregory Internship - Hachinohe 28/31
Appendix 14 : Mathias, Takeshi and me in Asakusa street GUFFROY Gregory Internship - Hachinohe 29/31
References Hachinohe and Aomori Processing Arduino - https://fr.wikipedia.org/wiki/aomori - https://fr.wikipedia.org/wiki/hachinohe - https://processing.org/ - Processing a programming handbook for visual designers and artists, Casey Reas and Ben Fry - https://www.arduino.cc/en/tutorial/homepage - Programing Arduino getting started with sketches, Simon Monk - Practical Arduino cool projects for open source hardware, Jonathan Oxer and Hugh Blemings - Beginning Arduino, Michael Mcroberts GUFFROY Gregory Internship - Hachinohe 30/31
Glossary - Arduino : An open-source open-hardware project which was developed in Italy. - Baud: Synonym of pulse per second. In telecommunications, it corresponds to the number of symbols in a second. Each symbol can represent one or several bits of data. - C : Programming language released in 1972. - DUT : French two-year technical formation in university. D stands for Diplôme (Diplom), U for «Universitaire» (University's) and T for «Technologique»(Technologique). - LED : stands for Light-Emitting Diode GUFFROY Gregory Internship - Hachinohe 31/31