ESE 350 HEXAWall v 2.0 Michelle Adjangba Omari Maxwell

Transcription

1 ESE 350 HEXAWall v 2.0 Michelle Adjangba Omari Maxwell

2 Abstract This project is a continuation from the HEXA interactive wall display done in ESE 350 last spring. Professor Mangharam wants us to take this project a step further. We will first work on the image processing aspect of the project. Using the Microsoft Kinect, we will be able to recognize various gestures to interact with each hexagon tile. The base gesture is a push that will then cause a ripple effect in the surrounding hexagons. The master s students worked on improving the mounts from last year to minimize current in the system. Now we have a task of improving the flow of the servos. Right now the motion is rigid so we need to come up with a way to make a smoother effect. We will then have to implement a couple patterns such as the tiles moving as a person walks by. Once these primary goals are accomplished we will look into new mechanical features to be implemented (some form of depth/3d visual) and last will be aesthetics. Currently it is just acrylic mounted to plywood. For the Kinect aspect of the project, with the help of Michelle (our TA) we will refine what constitutes a gesture and develop some kind of priority system in the event that multiple people are within the connect field of vision. We will be using the RaspberryPI as our microcontroller. We will also work with the robotics masters students to discuss what further mechanical implementations would be feasible or not given power constraints. Introduction People constantly look for wall decor in order to excite whatever bland surroundings they have. To this effect engineers have been developing interactive displays for an improved viewing experience. HEXAWall was initially inspired by one engineer who set out to accomplish this task (insert link). The Hexi Wall had over 60 vacuum molded panels that could move in the X and Y directions. Using motion tracking, the wall was able to create some very cool visual effects and add to the overall aesthetic of the room. We aim to accomplish a similar task with HEXAWall albeit to a smaller scale. We have 10 panels capable of moving in both X and Y directions as well as a Z feature resembling flower petals to give the wall a feeling of depth. HEXAWall uses an XBox 360 Kinect camera to track the user s movements and output coordinate information to our RaspberryPi microcontroller. The Pi then uses this input to talk to the servo motors drive which in turn drive each of the 30 different servo motors (3/panels; X servo, Y servo, Z servo).

3 Project Materials: Hardware Components: 1. Clear/Black Acrylic Sheets 2. 3D printed servo horns 3. Plastic hinges 4. Standard position servos bolts (½ and 1 ) and nuts (a lot) standoffs (½ and 1 ) 7. Nylon fishing wire 8. Aluminum jewelry crimps 9. Tension Springs Electronic Components: 1. RaspberryPI microcontroller 2. Three Adafruit 16 Channel 12 bit PWM/Servo Driver I2C interface PCA Xbox Kinect 4. Polulo Standard Position Servos 5. Three 5V,2A power adapters 6. Ethernet Cable Software Components: 1. OpenNi Kinect Library 2. PCA9685 library System Architecture Hardware We can break down the different subsystems to X motion, Y motion, Z motion, and electrical and describe the setups. The X motion and Y motion servos are simply the stock servo horns attached to an acrylic arm to drive motion in each direction. The Z motion servo is a little more complex. There is a custom 3D printed servo horn which pulls on a series of fishing lines. Their is one master thread attached to the horn and 6 threads connected to each leaf of the panel.

4 For our electrical subsystem we have a series of PWM servo drivers that are daisy chained together. The master servo driver is controlled by the RaspberryPi. We also have three 5V/2A power sources that are wired in parallel to allow us to source enough current and voltage for the 30 servos. Software The software starts from the Kinect to detect human gestures and then sends that information to the RaspberryPi IP so the code on the Pi can use the data to drive the PWM for each servo. Design Software Kinect Software We set up the Kinect to view the wall from its side where the wall is on the right side of the camera and the user is on the left side. To calibrate the position, we would turn on the petal closing mode and push until one flower closed and then adjust the kinect position to suit. With this orientation, when a user moves horizontally on the wall, that affects the depth reading on the Kinect s camera. This means that what the user sees as the x axis, the Kinect sees as the z axis. Similarly, when a user steps closer to the wall or push in and out of the wall (z axis), the Kinect reads that as the x axis. The OpenNi has push gestures as part of their API, so once a push gesture is recognized (a clear push in and pull out motion) the user s hand position is sent to the function called sendpacket. In sendpacket, there is another function call which sends a string of position information and the raspberrypi s ip address to sendpacketpi. This is the function that sends the information to the pi. If the pi is changed, you find this line and change the ip address. The path to the code is server/openni_nite_installer Linux /OpenNI Bin Dev Linux x64 v /sam ples/niusertracker/scenedrawer.cpp.

5 RaspberryPi Software To compile the code we ran gcc main_pi.cpp cordinates.cpp PCA9685.cpp pthread o main (pardon the lack of an o in the word coordinates). In order to run the code we type./main in the pi and./sampleusertracer 2 (for push mode). When we enter main, we initialize the positions of the panels and the pwm drivers. We then send a packet to the Kinect saying that we are in main. Once there is a recognized push gesture, it sends the info in a string format back to the pi. There is a variable called data which shows the mode. Since data is 2, it goes to handle the push. we make pthreads because we are going to be running operations on multiple servos at the same time. In pthread_create, that is where we call the function to start figuring out the actuation. It takes the name of the mode, this is mode2 (pulse out), there is also mode4 (close pushed flower), mode3 (random petal flutters), mode11 (pulse out with flutters), etc.

6 After calling the mode, we set the petal variable. If petal = 0 then we want x/y actuation, petal = 1 then we we want a flutter/ slight closing movement, if petal = 2, we want a fully closed petal. In this example where we have the x/y movement, we use functions in the coordinates.cpp, take the argument, set the pushpoint, set the angles they need to move in to actuate away from the push point, set the zones depending on the first set of surrounding panels, and the second set of surrounding panels, and reinitialize the panel objects so that have those altered attributes. After setting that up, we set the delay based on the zones. We changed the actuation delay of the zones to change exponentially vs linearly in order to give a more pulsing effect. We then create another pthread to call actuate panels which will deal with the servo movement. There are three possible functions to use to control the panels, xyshoot, flowerclose, and flowerpull (corresponding to petal being 0, 1, or 2 respectively.) In these functions we changed the nature of the servo rotations. Gaussian servo movement We wanted to servo to have a flowing feel to it, an acceleration, slow down, and accelerate back. While researching we landed on Gaussian functions. This image was what we wanted:

7 I made a separate python code to print the possible output values of the Gaussian function within x to +x range as a test. With this function, I had to change the mu value, the sigma value, and x value. In a while loop, I set a start x value then increase the value by.1 and had a delay of.01. When we ran the code we saw a promising difference between linear and Gaussian. Similar to the last method, we gradually decrease the delta x and the delay time. Our C++ code implementation looks like this: After some trial and error after connecting our panels, we came to set the mu value to 0, sigma to 0.5, and the x increments to 0.07 for horizontal/vertical movements and flower pulling x increments to The gaussian function returns each position that the servo has to be in (gx or gy for gaussian x/y). Depending on which direction the servo is moving, the gaussian function has to be shifted to set a starting point and then change from there. Each gx or gy is sent to function called setservopulse which takes the servo s channel on the motor driver and the desired pulse. Depending on the actuation setting, either the closing/fluttering petal pwm drivers will change or the horizontal or vertical servos on driver 1 or 2. SetPWM is a function in the PCA9685.cpp file so it manipulates the data for the drivers.

8 Design Mechanical One of the main design challenges was creating a housing for all 3 servo motors such that none of them had interference issues with each other. It was also a requirement that the mounts be able to withstand any torques that the servos would apply. To satisfy this requirement no feature on the mount was less than 0.25 thick and each section of the frame was reinforced by several aluminum standoffs to maintain structural integrity. In order to obtain our closing flower feature, we needed to find a way to pull six petals at the same time. Our first design was similar to an umbrella open/close mechanism. There was a pole coming out of the center with a collar on it and a pulley on the top. A servo would pull on the string that would then raise or lower the collar and then raise or lower the petals attached to the collar. That idea did not work out as well as we planned and it also was not aesthetically pleasing. So instead we pulled nylon string through a center hole, connected those six string to one main nylon string which was then pulled by the z axis servo. In order to ensure a strong connection, we used crimps to connect the loops of nylon. The z servo rotated about an inch so the nylon strings were pulled an inch to close the flower. To lessen the force on the servo, we made the starting position of the flower slightly closed.

9 GiventheadditionofmoreacrylicandanextraservofortheZ axis,tokeepthe panelsuprightwhentheservosarenotactuatingweusetensionspringstosupportthe weight.weinitiallythoughtthateachpetalwouldneedtobespringloadedsuchthat whenthez servowasoffthepetalscouldstayinanopenposition.uponmountingthe plastichingeswerealizedthattheywerespringyandcouldsupporttheweightofthe petalswithouttheneedforsprings.

10 FiguresoftheY axisandz axisservohousing,springsetupandhingesetup FigureofexplodedSolidWorksassembly Design Electrical MotorDriverswithPi

11 ThethreeAdafruitServoDriverswerethemostessentialcomponentsofourproject.The firsttwoservodriverscontrolthepwmofthexandymovementsofthepanelsandweaddeda thirddrivertocontroltheservosthatclosethepetals.inordertodifferentiatebetweenthe3,we havetosolderthea(x)platesinthetoprightofthedriver. Themotordriverthatcontrolsxandyofpanels0through4wasnotsoldered.Onthedriver

12 controlling x and y or panels 5 through 9, we soldered A0. Finally for the driver that controlled the z (closing) of panels 0 through 9, we soldered A1. We checked the detected drivers by running the command i2c detect y 0, which showed meaning all three drivers were identified when connected. Connection The servo drivers have a chainable design using a master slave system. Our master driver was x40, the one that was not soldered. The VCC is the digital supply for the IC (3.3V). We used pin 1 to supply 3.3V from the pi to the master. Pin 3 is SDA and pin 5 is SCL, which is used to share the i2c data line and clock in serial between the pi and the connected drivers. In addition, we connected ground (GPIO pin 6) to the master but not directly. We needed to have a common ground between the raspberrypi, master driver, and the power sources that will be supplying the voltage for the servos (will be discussed below). Once all the connections were made on the left of the board, we used female to female wires to connect the pins on the right of the driver to the 41 (soldered A0) driver, followed by the 42 (soldered A1) driver. Common Ground/ V+ Each driver has a V+ and Ground connection which power the servos. At first we used one one 5V/2A power adapter when we started the project with only thin panels with x and y movements. As we started to add heavier mechanical components and run 10 more servos, we noticed we were not supplying enough current. Unfortunately we were not in a good position to connect our project to a power source so we had to improvise. We hooked up three 5V/2A power adapters and put them in parallel. This way, each driver will supply 5V to each of the servos and increase the max current from 2A to 6A.

13 Thetwobrownwiresonrow18arethegroundsoftheRaspberryPiandmaster driver.thatgroundisthethenconnectedtothe3gndsoftheeachdriver(left column),whichisalsoconnectedtothe3groundsofthepoweradapters(wiresin row28).ontherightsidearethe3positiveendsofthepoweradapters(redwiresin row30),whicharethenconnectedtothev+ofthe3drivers(right+ column). Topreventoverheating,wesoldereda2200uFcapacitortothedriver. Asfortheconnectionsoftheservos,femaleheadersconnectedthePWM,V+,GNDof eachservoinadifferentcolumn.thepca9685librarythencontrolswhichchanneltoturnonto whichpwm. Testing/Evaluation Atourdeclarationofthisprojectwewantedtohaveahighlevelinteractivewalldisplay withlightsandotherfeature.however,ourtestingoftheprojectwasverydelayeddueto circumstancesnotinourcontrol(i.elasercuttersbeingdown,thekinectnotworking)andwe werenotabletotestasmuchasdesired.inthefinalpushwewereabletosuccessfully

14 implement the open/close feature and have the panels react to the outputs of the kinect camera in a several different patterns. Conclusion/Next Steps Despite all of the setbacks, this proved to be a very rewarding experience. The satisfaction of completing the assembly with over 1600 different parts(nuts and bolts included) by hand and then having it move how we want it to was amazing. We learned a lot about image processing, I2C protocols, servo motors, SolidWorks and mechanical assembly. The project is very scalable in terms of size and scope. The only limitations are really physical space and current draw. If we had more time we would have liked to add lights and more effects for the wall panels. I think it will be very easy for future teams to pick up where we left off. Github : v2 Demo : Blog :