• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Collective Inteligence Part I
 

Collective Inteligence Part I

on

  • 2,205 views

Collective Intelligence

Collective Intelligence

Statistics

Views

Total Views
2,205
Views on SlideShare
1,846
Embed Views
359

Actions

Likes
2
Downloads
59
Comments
1

3 Embeds 359

http://www.public.asu.edu 304
http://www.techgig.com 54
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • In Wikipedia, thousands of contributors from across the world have collectively created the world’s largest encyclopedia, with articles of remarkably high quality. Wikipedia has been developed with almost no centralized control. Anyone who wants to can change almost anything, and decisions about what changes to keep are made by a loose consensus of those who care. What’s more, the people who do all this work don’t even get paid; they’re volunteers.
  • Google, for instance, takes the judgments made by millions of people as they create links to Web pages and harnesses that collective knowledge of the entire Web to produce amazingly intelligent answers to the questions we type into the Google search bar.
  • http://en.wikipedia.org/wiki/CAPTCHACAPTCHA is vulnerable to a relay attack that uses humans to solve the puzzles. One approach involves relaying the puzzles to a group of human operators who can solve CAPTCHAs. In this scheme, a computer fills out a form and when it reaches a CAPTCHA, it gives the CAPTCHA to the human operator to solve.
  • Yahoo!Answers is another example.
  • In Threadless, anyone who wants to can design a T-shirt, submit that design to a weekly contest, and vote for their favorite designs. From the entries receiving the most votes, the company selects winning designs, puts them into production, and gives prizes and royalties to the winning designers. In this way, the company harnesses the collective intelligence of a community of over 500,000 people to design and select T-shirts
  • Turtles, Termites, and Traffic Jams - Explorations in Massively Parallel Microworlds
  • More examples on collective intelligence can be seen in games.
  • http://en.wikipedia.org/wiki/Collective_intelligence
  • Allowing others to share ideas and bid for franchising
  • This principle has been controversial with the question being “Should there be a law against the distribution of intellectual property?”
  • Online talent scouts pay off – USA Today April 1, 2010 Money Section B by John Swartzhttp://www.usatoday.com/money/industries/technology/2010-04-01-crowdsourcing01_ST_N.htm
  • http://en.wikipedia.org/wiki/File:CI_types1s.jpg
  • http://en.wikipedia.org/wiki/Crowd_wisdom

Collective Inteligence Part I Collective Inteligence Part I Presentation Transcript

  • DATA MINING AND MACHINE LEARNING IN A NUTSHELL COLLECTIVE INTELLIGENCE PART I Mohammad-Ali Abbasi http://www.public.asu.edu/~mabbasi2/ SCHOOL OF COMPUTING, INFORMATICS, AND DECISION SYSTEMS ENGINEERING ARIZONA STATE UNIVERSITY Arizona State University http://dmml.asu.edu/Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 1
  • About Collective Intelligence • Definition of collective intelligence – Examples happening around us • What constitutes collective intelligence – Groups, number of members, variety, etc. • How can one improve collective intelligence – What are necessary conditions to achieve CI – A case in data mining and machine learning? • What can one do with collective intelligence in the age of social media – Opportunities for Data Mining Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 2
  • Definitions for Collective intelligence • Wikipedia – Collective intelligence is a shared or group intelligence that emerges from the collaboration and competition of many individuals • MIT Center for CI – Groups of individuals doing things collectively that seem intelligent • Toby Segaran in Programming CI – Combining the behavior, preferences, or ideas of a group of people to create novel insights • Unknown – Collective intelligence is any intelligence that arises from - or is a capacity or characteristic of - groups and other collective living systems Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 3
  • Examples of collective intelligence - Wikipedia • Wikipedia • Thousands of contributors from across the world have collectively created the world’s largest encyclopedia • with almost no centralized control Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 4
  • Examples of collective intelligence - PageRank • PageRank Algorithm used by Google Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 5
  • Examples of collective intelligence - CAPTCHA • CAPTCHA – Completely Automated Public Turing test to tell Computers and Humans Apart – A reverse Turing test (machine to human instead of human to machine) • A service that helps to digitize books, newspapers and old time radio shows – About 200 million CAPTCHAs are solved by humans around the world every day – More than 150,000 hours of work each day Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 6
  • Vark.com 1. Send a question 2. Aardvark finds the perfect person to answer 3. Get their response Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 7
  • Kasparov vs. the World • Kasparov v. the World was a chess match held in 1999, when world champion Gary Kasparov played against “the World,” with the World’s moves determined by majority vote over the Internet of anyone who wanted to participate. Kasparov eventually won, but he said it was the hardest game he ever played Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 8
  • Examples of collective intelligence - Threadless • Threadless.com • In Threadless, anyone who wants to can design a T- shirt, submit that design to a weekly contest, and vote for their favorite designs • the company harnesses the collective intelligence of a community of over 500,000 people to design and select T-shirts Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 9
  • Examples of collective intelligence –Google ImageLabeler • It is a feature, in the form of a game, of Google Image Search that allows the user to label random images to help improve the quality of Googles image search results Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 10
  • Examples of collective intelligence – Ant Societies • Ant societies exhibit more intelligence than any other animal except for humans, if we measure intelligence in terms of technology. Ant societies are able to do agriculture, in fact, in several different forms of agriculture. Some ant societies keep livestock of various forms, for example, some ants keep and care for aphids for "milking”; Leaf cutters care for fungi and carry leaves to feed the fungi. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 11
  • Examples of collective intelligence - Games • Games such as WorldCraft, The Sims, Halo or Second Life are designed to be more non- linear and depend on collective intelligence for expansion. • This way of sharing is gradually evolving and influencing the mindset of the current and future generations. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 12
  • Principals of Collective Intelligence • Collective intelligence is of mass collaboration. In order for collective intelligence to emerge, four principles exist to promote creativity: – Openness – Peering – Sharing and – Acting globally Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 15
  • Openness • Traditionally, people and companies are naturally reluctant to share ideas and intellectual property because these resources provide the edge over competitors. • However, in time, openness is promoted when people and companies began to loosen hold over these resources as they reap more benefits in doing so. • Openness enables products to gain significant improvement and scrutiny through transparent collaboration. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 16
  • Peering • A form of horizontal organization with the capacity to create information technology and physical products. • One example is the ‘opening up’ of the Linux program where users are free to modify and develop it provided that they made it available for others. • Participants in this form of collective intelligence may have different motivations for contributing, but the results achieved are for the improvement of a product or service. • “Peering succeeds because it leverages self-organization – a style of production that works more effectively than hierarchical management for certain tasks.” Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 17
  • Sharing • Research has shown that more and more companies have started to share some, while maintaining some degree of control over others, like potential and critical patent rights. • This is because companies have realized that by limiting all their intellectual property, they are shutting out all possible opportunities. • Sharing some has allowed them to expand their market and bring out products faster. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 18
  • Acting Globally • The advancement in communication technology has prompted the rise of global companies, or e- Commerce that has allowed individuals to set up businesses at low to almost no overhead costs. • The influence of the Internet is widespread, therefore a globally integrated company would have no geographical boundaries but have global connections, allowing them to gain access to new markets, ideas and technology. • Therefore it is important for firms to get updated and remain globally competitive or they will face a declining rate of clients. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 19
  • Types of Collective Intelligence Arizona State UniversityData Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 20
  • Elements of Collective Intelligence • Staffing – Who is performing the task? • Incentives – Why are they doing it? • Goal – What is being accomplished? • Structure, process – How is it being done? Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 21
  • Elements of Collective Intelligence • Who? – Hierarchy – Crowd • Why? – Money – Love – Glory • What? – Create – Decide • Who – Collection – Collaboration Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 22
  • Mapping the collective intelligence elements forWikipedia Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 23
  • Issues with Crowd Wisdom • Questions – Why can the crowd be smarter than any individual in the crowd? – Is it guaranteed? If not, what are the conditions under which the crowd can make best decisions? – How can one gauge the reliability of crowd wisdom? Is crowd wisdom valid, trustworthy, and verifiable? – How to find a crowd, its leader/influencer/average opinion? – How is each member influenced by others? Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 24
  • Collective Intelligence and Societies • The main base of all kinds of CI’s is society • CI in traditional societies – Families, companies, countries, and armies are all groups of individuals doing things collectively that, at least sometimes, seem intelligent • CI in Web based societies- Social Networking sites – Internet and specially Web 2.0 applications provide a platform for communications and building societies Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 25
  • Collective Intelligence and the Internet • Web 2.0 • Social Computing Arizona State UniversityData Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 26
  • Web Impacts on CI • The ability of new media to easily store and retrieve information, predominantly through databases and the Internet, allows it to be shared without difficulty. • Thus, through interaction with new media, knowledge easily passes between sources resulting in another form of collective intelligence Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 27
  • WEB 2.0 and Many Variants Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 28
  • Elements of WEB 2.0 Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 29
  • Web 2.0: Evolution Towards a Read/Write Platform Web 1.0 Web 2.0 (1993-2003) (2003- beyond) Pretty much HTML pages viewed through a Web pages, plus a lot of other “content” shared browser over the web, with more interactivity; more like an application than a “page” “Read” Mode “Write” & Contribute “Page” Primary Unit of “Post / record” content “static” State “dynamic” Web browser Viewed through… Browsers, RSS Readers, anything “Client Server” Architecture “Web Services” Web Coders Content Created by… Everyone “geeks” Domain of… “mass amatuerization” Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 30
  • CI in Social Media • Crowd members assign different weights to individual inputs on the basis of their relationship with the people who provided them and then make individual decisions – Blogosphere – Facebook – YouTube – Epinions.com – Amazon – eBay – Digg Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 31
  • Blogging is the Most Recognized Example of Web 2.0 Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 32
  • Blogging is the Most Recognized Example of Web 2.0 Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 33
  • Wikipedia is a Collaborative Dictionary Being Edited in Real-time by Anyone Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 34
  • Alive At ASU Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 35
  • WEB 2.0 Technologies • APIs • RSS (Really Simple Syndication) – Content Syndication • Web Services – Open Data • AJAX (Asynchronous Javascript and XML) • CSS (Cascading Style Sheets) – Content with Style Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 36
  • WEB 2.0, Summing Up• Web 2.0 hard to define, but very far from just hype – Culmination of a number of Web trends• Importance of Open Data – Allows communities to assemble unique tailored applications• Importance of Users – Seek and create network effects• Browser as Application Platform – Huge potential for new kinds of Web applications Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 37
  • ProgrammingCollective Intelligence Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 38
  • Crawl the web Arizona State UniversityData Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 39
  • Spiders (Robots/Bots/Crawlers) • Start with a comprehensive set of root URL’s from which to start the search. • Follow all links on these pages recursively to find additional pages. • Index all novel found pages in an inverted index as they are encountered. • May allow users to directly submit pages to be indexed (and crawled from). Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 40
  • Search Strategies Breadth-first Search Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 41
  • Search Strategies (cont) Depth-first Search Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 42
  • Search Strategy Trade-Off’s • Breadth-first explores uniformly outward from the root page but requires memory of all nodes on the previous level (exponential in depth). Standard spidering method. • Depth-first requires memory of only depth times branching-factor (linear in depth) but gets “lost” pursuing a single thread. • Both strategies implementable using a queue of links (URL’s). Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 43
  • Spidering Algorithm • Initialize queue (Q) with initial set of known URL’s. • Until Q empty or page or time limit exhausted: – Pop URL, L, from front of Q. – If L is not to an HTML page (.gif, .jpeg, .ps, .pdf, .ppt…) • continue loop. – If already visited L, continue loop. – Download page, P, for L. – If cannot download P (e.g. 404 error, robot excluded) • continue loop. – Index P (e.g. add to inverted index or store cached copy). – Parse P to obtain list of new links N. – Append N to the end of Q. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 44
  • Keeping Spidered Pages Up to Date • Web is very dynamic: many new pages, updated pages, deleted pages, etc. • Periodically check spidered pages for updates and deletions: – Just look at header info (e.g. META tags on last update) to determine if page has changed, only reload entire page if needed. • Track how often each page is updated and preferentially return to pages which are historically more dynamic. • Preferentially update pages that are accessed more often to optimize freshness of more popular pages. Arizona State University Data Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 45
  • Mohammad-Ali Abbasi (Ali), Ali, is a Ph.D student at Data Mining and Machine Learning Lab, Arizona State University. His research interests include Data Mining, Machine Learning, Social Computing, and Social Media Behavior Analysis. http://www.public.asu.edu/~mabbasi2/ Arizona State UniversityData Mining and Machine Learning Lab Data Mining and Machine Learning- in a nutshell Collective Intelligence 46