Frmative Evaluatin f GeeGuides: Educatinal Technlgy t Enhance Art Explratin Prepared by Clleen F. Manning Senir Research Assciate Gdman Research Grup, Inc. Submitted t GeeGuides LLC March 2005
EXECUTIVE SUMMARY In Nvember 2004, GeeGuides cntracted with Gdman Research Grup, Inc. (GRG) t cnduct frmative evaluatin f tw f their educatinal technlgy prducts: 1) the sayart prgram, designed t take place n a cmputer in a classrm r at hme, and t intrduce children t the language f art, and 2) the seeart prgram, delivered n handheld devices (GeeGuides) in museums, and designed t facilitate children s experience f artwrk. The frmative evaluatin was designed t answer the fllwing questins: Hw appealing are the GeeGuides prgrams t the target audience? Hw d children use the GeeGuides prgrams? In what ways d the prgrams enhance children s art explratin, appreciatin, and educatin? Hw culd the GeeGuides prgrams be imprved t better meet the needs f the target audience? This Executive Summary prvides a cncise explanatin f the evaluatin methds, descriptin f the key findings, and presentatin f the recmmendatins based n the findings. We refer the reader t the full reprt fr a cmprehensive discussin f the findings and recmmendatins. METHOD The study tk place between December 2004 and February 2005 at three museums: the Jack S. Blantn Museum f Art at the University f Texas at Austin, the Vancuver Art Gallery (British Clumbia), and the Seattle Art Museum. A ttal f 213 children participated in the evaluatin. The majrity f children were frm schl grups wh participated in sayart and seeart, while the remainder f the sample visited the museum with their families and participated in seeart nly. The average age f participants was 10 years, a slight majrity was girls, and fur ut f five had experience visiting art museums. Data cllectin included bservatins, and written surveys r interviews with children in classrms (sayart) and museums (seeart). KEY FINDINGS Appeal f prgrams: Bth sayart and seeart prgrams were very appealing t children: 67% f sayart users and 86% f seeart users fund the prgram either fun r really fun. Appeal f activities: Students were mst engaged by activities that allwed them creative cntrl r expressin and gave them smething in return, and were less interested in r disappinted by activities that gave them n feedback. Children liked the Mvement mdule and activities best. Mst children fund the GeeGuides characters appealing.
Parents pinins f GeeGuides: Parents indicated their supprt fr seeart, with 88% indicating they wuld rather g thrugh anther museum with a GeeGuide than withut ne. Parents indicated a willingness t pay a maximum fee f abut $5.00, and 74% said they wuld cnsider renting a GeeGuide fr themselves. Parents cnfirmed their children s enjyment f GeeGuides, and believed the tl helped their children enjy and learn frm the museum experience. Parent pinin was split regarding the GeeGuide s helpfulness t parent-child interactins in the museum; abut ne-third thught it helped, ne-third were neutral, and ne-third thught it hindered interactin. Use f activities: Children fllwed mst f the suggestins made by the sayart and seeart prgrams. In the museum, sme children tried all the activities fr a particular mdule, while thers spent mre time n fewer activities. In bth schl and family grups, children ften selected a painting and began watching the accmpanying mvie befre they arrived at the artwrk. Children became t tired and restless standing during the GeeGuides activities. They were mre cmfrtable sitting n the flr r a bench, althugh this smetimes resulted in their nt facing the art wrk. In general, children r families that were physically remved frm the artwrk appeared mre passive and less engaged with the artwrk. Mst children were thughtful abut the activities; hwever, sme were mre interested in explring the capability f the technlgy, suggesting they were cnstructing meaning abut the technlgy rather than the art. Understanding f cncepts: Children demnstrated gd recall and understanding f the cncepts cvered in sayart and seeart, including artists use f lines t guide the eye and express emtins, that pig bladders, shells, and bttles were used t stre paint prir t the inventin f paint tubes, and the difference between stillness and mvement in a wrk f art. After using GeeGuides, children demnstrated aesthetic, narrative, lgical/quantitative, and fundatinal respnses t the artwrk they viewed. In particular, they used the fundamental principles f art featured n the GeeGuide t rganize their descriptins f paintings they viewed. Navigatin: Bth in classrms and museums, children ften sught apprval r directin frm adults befre cntinuing, and ccasinally needed prmpting t g n. On the laptp, sme children were cnfused abut what t d after hearing the welcme, and did nt intuitively gravitate tward the left-hand menu. Mving frm the Intr t the mdules als was cnfusing fr sme students. Sme clicked n the images f the paintings, and sme cycled back t the beginning f the Intr. Once started, mst students fund the prgram easy t use, and used it in its entirety. Small grups usually
explred each mdule and did each activity in a given mdule, smetimes in rder and ther times at randm. On the GeeGuide, students were uncertain whether there was an rder t the activities that fllwed each mvie; they had a little bit f difficulty initially fllwing the directins, and exiting mdules r activities; and they ften pressed t many buttns r pressed them prematurely. All f these situatins imprved with practice. Children preferred t navigate the GeeGuide using a stylus rather than tapping with their fingers. Explring the artwrk: The main cncern frm the first rund f testing at the Blantn Museum f Art that children did nt lk up frm the GeeGuide t the artwrk appeared t have been reasnably reslved in the latest versin f the GeeGuide. The balance f their time was still spent fcused n the GeeGuide, which cncerned at least ne f the museum educatrs wh bserved the testing. In the classrm, sayart users initially skipped explring the artwrk and prceeded directly t the mvie. When they did explre the artwrk, they fund the zm in ptin mst appealing. The limited explratin f the artwrk reduced discussin amng students abut it. Interactin: Bth sayart and seeart supprted a medium level f interactin amng schl children, and between children and parents with respect t the GeeGuide. Students psed questins alud t their grups, helped each ther, and negtiated their navigatin tgether. C-participants shared infrmatin abut their prgress thrugh the prgram verall, and thrugh specific activities, questined each ther, helped each ther, lked at what was n each thers screens, shwed each ther their screens, and pinted t and tapped n each thers screens. In the museum, they were mre likely t interact ver the GeeGuide than the paintings n the walls. In the museum, sme parents used the GeeGuide as a springbard fr asking their children questins abut the artwrk, r making suggestins, smetimes cnnecting the seeart experience t their children s lives. In additin, children asked fr and parents ffered clarificatin abut GeeGuides infrmatin. Smetimes, parents attempted t mve their children thrugh the experience mre quickly than the children desired. Suggestins: Children had similar suggestins fr imprving sayart and seeart, including mre cntent, mre creative activities, and larger-scale mvies. With regard t the GeeGuides themselves, children wanted less talking and an attached stylus.
RECOMMENDATIONS sayart Orient the students t what they will d in sayart: see and explre paintings, and d related activities. Include visual examples. Use a cnsistent term fr items n the left-hand menu, and label them accrdingly. If the terms art r artwrk are used, menu items shuld be accmpanied by the same images featured in the mvie. Prvide a visual link between the instructins at the end f the mvie and the left-hand menu. Give students feedback during activities, t fulfill their expectatins and enhance the interactivity f the experience. Allw students t alter their wrk during activities, t enable them t build n the meaning they have made frm an initial interactin. Add specific instructins fr students t explre the art, including bjectives fr ding s. Limit what students are able t d with the technlgy, t fcus them n understanding and applying the principles f art. Fully develp the menu, artwrk, and museum and artwrk infrmatin parts f the screen, then cnsider the size f the mvies. seeart Shrten the lk at the painting segments r embed suggestins t lk at the painting int the activities. Attach the stylus t the GeeGuide. Ensure that thse facilitating children s experiences with GeeGuides understand that the tl is fr use in the presence f artwrk. Take int accunt bench lcatin relative t featured artwrk when selecting the artwrk with partnering museums. Priritize develpment f materials t supprt teachers in engaging their students in mre meaningful discussins abut the art wrk and accmpanying GeeGuides cntent.
NEXT STEPS GeeGuides has a clear missin t enhance children s explratin f art and the frmative evaluatin supprts the cmpany s initial directin. Bth sayart and seeart were implemented successfully in their classrm and museum settings, respectively. Children, parents, and teachers fund the prgrams appealing, and children had the types f aesthetic, narrative, lgical/quantitative, and fundatinal respnses t the artwrk that GeeGuides intended. The evaluatin des pint t sme areas fr imprvement, as nted in the recmmendatins. Perhaps the mst imprtant next step fr GeeGuides is t wrk frm their missin t further develp their gals and bjectives fr their prgrams. They have already made sme prgress in this area, having drafted their desired effects fr children wh use their prgrams. They are nw in a psitin t develp specific and measurable bjectives, based n the gals. Frm the bjectives will fllw indicatrs (measurable evidence) that can be used t assess the effectiveness f their prgrams. They will then be in a gd psitin t cnduct an utcmes study, a study in which bserved effects can be attributed t GeeGuides.
Gdman Research Grup, Inc. 955 Massachusetts Avenue, Suite 201 Cambridge, Massachusetts 02139 Tel: (617) 491-7033 Fax: (617) 864-2399 inf@grginc.cm www.grginc.cm 2005 Gdman Research Grup, Inc.