Data Collection: Christmas Bird Count Counting Started: 1899

Size: px

Start display at page:

Download "Data Collection: Christmas Bird Count Counting Started: 1899"

Anis Sims
5 years ago
Views:

2 Data Collection: Christmas Bird Count Counting Started: 1899 Idea Competition: Nicolas Appert s Food Canning Competition started: 1795 Awards Won:

5 5

7 E.g. Sorting Algorithms: Many sorting algorithms available, Bubble Sort, Quick Sort, Merge Sort But, what would it be if we want to sort the trustworthiness of websites or the attractiveness of celebrities?» 3 categories of applications: Perceptual Skills: e.g. audio/video classifications Cognitive Skills: e.g. comprehensive planning and reasoning. Language Skills: e.g. natural language translation. 7

8 Crowds can be leveraged as:» Information Collectors.» Human Processors.» Content Contributors.» Idea Creators. 8

9 » Information Contributors.» Human Processors.» Idea Contributors. Mobile Sensing Crumbling Infrastructure Haiti Earthquake Photo Classification Expert Citizens on Open Foam Simulation Shelters For all Competition 9

10 » How much time do we want to allocate?» How accurate do we want our results to be?» How big is our budget to incentivize the workers? 10

11 » According to American Society of Civil Engineers (ASCE), the general condition of civil infrastructure in the US is in a worrisome situation.» When evaluating the overall condition in 2009, the ASCE issued an alarming score of D as the GPA of America s infrastructure condition.» In 2013, the GPA of America s infrastructure condition is D +. 11

13 I-35W Bridge in Minneapolis, Minnesota suddenly collapsed during the evening rush hour, in

14 » The amount of infrastructure in countries that covers a broad area, such as that in the US, overwhelmingly exceeds the number of inspectors.» A more pervasive, efficient, and operable sensing mechanism empowering national level coverage is needed. 14

15 GPS Tagging Digital Devices Yahoo API Address Translation Web Portal Google API Photo Aggregation Google Global Map 15

16 » Hand-held digital devices have pervasive coverage.» As a basic personal communication tool, cell phones have a relatively reliable power supply.» Digital device carriers assistance could be leveraged to achieve sophisticated application functionalities. 16

17 Minneapolis Portland 17

18 18

19 » Contributions: Established the cyber-infrastructure. Demonstrated the concept of citizen sensing. 10-module citizen sensing framework. 19

20 20

21 21

22 22

23 Two approaches:» Behavioral Analysis: Subjects behaviors in the process of performing tasks. e.g. average tagging time.» Output Analysis: Patterns embedded in subjects answers. e.g. consecutive appearances of Cannot Determine. 23

24 Freeloader: Average tagging time less than 13 sec. 24

25 Freeloader: Long Cannot Determine Sequences 25

26 » Before Data Cleaning > 9318 user-image pairs (Classifications) 204 users» After Data Cleaning: > 6186 user-image pairs (Classifications) 202 users 26

27 27

28 » One Subject has One Vote. All subjects have an equal weight.» The option that most subjects agreed upon is the crowd consensus.» Comparing the crowd consensus with the ground truth to evaluate the crowd performance. 28

29 29

30 30

31 31

32 32

33 » Contributions: Web portal architecture. 3 Data Cleansing Strategies. 4 Result Retrieval Algorithms. User motivation analysis. (Data Support for sociologists) 33

34 34

35 35

36 36

37 37

38 38

39 Established the front-end web portal. Demonstrated the concept of expert-citizen Engineering. Lessons learned: High-end users require high-performance computing facilities. Innovative ways need to be developed to evaluate high-skilled tasks. (An ongoing experiment on Mechanical Turk) The platform needs to be prepared to take burst requests. 39

40 40

41 41

42 42

43 43

44 Established the cyber infrastructure. Accomplished the platform Integration. 3 Directions for future research: The efficiency of open competition model. Competition level between participants. Investigate the potential opportunities for collaboration and networking among participants. 44

45 » Case Study I: Crumbling Infrastructure Photo Submission Crowds can be leveraged as information collectors.» Case Study II: Haiti Earthquake Photo Tagging project Algorithms for Data Cleansing, and Results Retrieval.» Case Study III: Expert Citizen Engineering System designs for high-skilled citizens.» Case Study IV: Crumbling Infrastructure Photo Effective design processes for far-reaching and large-scale innovative contests.

46 46

47 47

48 » A whole spectrum of citizen engineers with variable backgrounds and expertise.» Our preliminary research focuses on low-end average citizen engineers and high-end expert engineers.» How about the ones in between? How to shepherd junior engineers to become experienced and thus be able to fulfill complicated tasks? 48

49 » What is the optimal number of workers per job? Reliability vs. Efficiency» Teamwork: currently, most current platforms, including our photo tagging system, do not support teamwork. How can we group users to achieve higher productivity? 49

50 50

» "Citizen Engineering: Evolving OSS Practices to Engineering Design and Analysis", 8th International Conference on Open Source Systems, Hammamet, Tunisia 10-13 September, 2012» "Haiti Earthquake

51 » "Citizen Engineering: Evolving OSS Practices to Engineering Design and Analysis", 8th International Conference on Open Source Systems, Hammamet, Tunisia September, 2012» "Haiti Earthquake Photo Tagging: Lessons on Crowdsourcing In-Depth Image Classifications", Seventh International Conference on Digital Information Management (ICDIM 2012), August 22-24, 2012, Macau, China» "Crowdsourcing Highly Trustworthy Results",45th Hawaii International Conference on System Science (HICSS), 2012» "Expert-Citizen Engineering: 'Crowdsourcing' Skilled Citizens" 2011 International Conference on Social Computing and its Applications (SCA2011) December 2011, Sydney Australia» Timothy W. Schoenharl, Zhi Zhai, Ryan McCune, Alec Pawling, Gregory R. Madey: Design and implementation of an agent-based simulation for emergency response and crisis management. SpringSim 2009, San Diego, CA, March 2009» Francis Chen, Zhi Zhai and Greg Madey, "Dynamic Adaptive Disaster Simulation: Developing a Predictive Model of Emergency Behavior Using Cell Phones and GIS Data", SpringSim 2011/ADS 2011, Boston, April

52 » Academic advisor: Prof. Greg Madey» Committee Members: Prof. Brian Blake Prof. David Hachen Prof. Douglas Thain Prof. Tracy Kijewski-Correa» Collaborators: Prof. Ahsan Kareem Dr. Zack Kertcher Dr. Jenny Vaydich Ms. Ellen Childs Mr. Peter Sempolinski Mr. Andrew Weber» Sponsor: National Science Foundation 52

53 53

54 Human-Computer Interaction Sociology Data Mining Artificial Intelligence Crowdsourcing Psychology Statistics Database Communication Studies 54

Open Sourcing the Design of Civil Infrastructure (OSD-CI): A Paradigm Shift. T. Kijewski-Correa, M. ASCE 1

Open Sourcing the Design of Civil Infrastructure (OSD-CI): A Paradigm Shift T. Kijewski-Correa, M. ASCE 1 1 DYNAMO Laboratory, Department of Civil Engineering and Geological Sciences, University of Notre