SlideShare a Scribd company logo
1 of 22
1
The Pain of Complexity
Dmitry Zinoviev
Suffolk University, Boston
2
“Complexity” is one of those fuzzy concepts
that everyone seems to understand
differently. Other examples: sustainability,
resilience, success...
What Is Complexity?
3
Let's Ask the Mechanical Turk!
● Question: “Enter three nouns
(please, nouns only!) that, to
the best of your understanding,
are associated with complexity.
These words could be
synonyms, descriptors,
attributes of complexity, actions
or any other related words.”
● Ask 100 mTurk workers, pay a
nominal fee for an answer
● Reject non-nouns; stem* and
aggregate nouns
*We tried both “Porter” and “Lancaster” stemmers; “Porter” is less
aggressive, it produced more stems (“motifs”); some manual tweaking
was needed
4
Direct Survey Results
● The stems that were mentioned at least twice (in the order of
decreasing frequency):
– Most frequently mentioned (23+ times): COMPLIC*, DIFFICULTI*,
INTRICACI*
– Less frequently mentioned (2–9 times): PROBLEM*, ELABOR*,
ENTANGL*, PUZZL*, MULTIPL*, COMPLEX*, COMPUT*,
CONVOLUT*, HARD*, RAMIF*, SOPHIST*, CHALLENG*,
HUMAN*, INVOLV*, LIFE*, POLIT*, RIDDL*, SCIENC*,
COMPOSIT*, CONFUS*, DEPTH*, DESIGN*, DIVERS*,
LABYRINTH*, LANGUAG*, MATH*, MAZE*, OBSCUR*,
PHYSIC*, QUALITI*, RELATIONSHIP*, SPACE*,
STRUCTURE*, TEST*, TRICKI*, TROUBL*, VARIABL*,
VARIETI*
– The highlighted stems bear strong emotional load—though, no pain
yet!
5
Use Indirect Data
● Don't tell me who you are. “Tell me who your friends are, and I will tell
you who you are.”
● Instead of asking people what they think “complexity” is, let's collect
abundant implicit markers off the Internet.
6
Indirect Data: Possible Sources
● EBSCO keywords or subject tags (subscription and harvesting
software required; we haz it, but the results may be too academic!)
● Blogging sites like LiveJournal (LJ)
– LJ has individual blogs and community blogs (like forums)
– Both types of blogs can and do have user- or moderator-declared
interests
– The interests are usually carefully chosen to reflect the
blogger's/community's online identity
– Free access, easy-to-write harvester
7
Raw Data
● All LiveJournal communities that list at least one of the following
interests:
– “complexity”
– “complex systems”
– “complexity theory”
– “complexity theories”
● Found 59 communities, such as mixedtype, abstractthought,
ivygreenfanclub, pdmi_logic, investors, gifted_teens, etc.
● Many more individual bloggers—we ignored them, their interest lists
are way too broad!
● Selected 374 most frequently declared interests (the corpus)
8
Example: “Ecologists” community
● Community ecologists.livejournal.com
● 150 self-declared (by the moderator(s)) interests
● Includes “complexity,” but also many other keywords; are
they relevant? Let's assume they are.
9
Term Vector Model
● Treat communities as documents and interests as words
● Calculate generalized similarity between words (and between
communities—but why?) using Kovacs (2010) algorithm:
– Two words are similar if they belong to similar documents
– Two documents are similar if the consist of similar words
● Remark: generalized similarity, as defined by Kovacs, is essentially
Pearson correlation in an oblique Cartesian coordinate space
●
For any two words W and V from the corpus, -1 ≤ d(W,V) ≤ 1 is the
similarity between the words; for any two communities C and D, -1 ≤
d(C,D) ≤ 1 is the similarity between the communities (a by-product,
not needed)
10
Network Construction (1)
● Treat words as nodes in a
network
● Treat similarities as edges if
they are above T0
=0.65
(“slicing”); the choice of T0
is
tricky and arbitrary:
– If T0
is too high, the
network is complete
and not interesting
– If T0
is too low, the
network is sparse and
disconnected
● Gephi it!
11
“Complexity Dragon” (a.k.a. the
Mindscape of Complexity)
12
Modular Structure
Modularity=0.56
(not great, but
still visible)
psychology, chaos,
science, mathematics,
complexity theory, ...
philosophy, life,
imagination, self-expression,
knowledge, ...
complexity, honesty, music, simplicity,
love, poetry, creativity,
empathy, humor, hate, ...
writing, art, books,
romance, drawing, ...
13
What Is the Meaning of Modules?
● We could just stare at the
modules and try to come up
with a suitable name—
● —or we could use
crowdsourcing through Amazon
Mechanical Turk again!
Fragment of the “fireball”
14
Let's Ask the Mechanical Turk!
● Question: “Describe the
following group of 50* words
with a single most suitable word
or a two-word or three-word
phrase: ...”
*There are only 36 terms in the lower
left module.
● Ask 100 mTurk workers, pay a
nominal fee for an answer
● Reject non-nouns; stem and
aggregate nouns
● The nouns are module
descriptors (“motifs”)
15
Network Construction (2)
● Treat original modules as documents and motifs as words
● Calculate generalized similarity
● Do slicing
● Build a network of stems
● Gephi it!
16
The Three Shades of Complexity
● The new network has a simple,
but not trivial, structure, and is
highly modular
● The Three Shades of
Complexity:
– Science/Technology
– Society/Mind
– Creativity/Emotions
● Connections:
– via education
– via life/humanities
There are lines here, they
are just too thin...
17
Semantic Spectrography
mTurk for motif
extraction and clustering
Term source for
mindscape construction
Analysis term
selection
18
Spectrography vs Direct Survey
Direct Survey: 37
Spectrography: 382
4 9 4
104 111 113 37
24
19
Comparison Summary
● Positive:
– Spectrography is much more detailed: it catches 41% of direct
survey terms; direct survey catches only 4.5% of spectrography
terms
– Spectrography reveals structure
● Negative:
– Spectrography requires a source (or sources) of terms
20
Successfully Used Elsewhere
● D. Zinoviev, D. Stefanescu, L. Swenson, and G. Fireman, “Semantic
Networks of Interests in Online NSSI* Communities,” in Proc.
Workshop “Words and Networks,” Evanston, IL, June 2012,
published online (also submitted to Social Networks in 2013)
● D. Zinoviev and Z. Zhu, “Conceptual Structure of Sustainability:
Social and Scholarly Perspectives,” Sunbelt XXXIV Social Networks
Conference, St.-Pete Beach, FL, February 2014 (also submitted to
Social Networks in 2014)
*Non-Suicidal Self Injury—a common and
epidemically spreading activity among adolescents
and young adults
21
But Where Is the Pain?
● SE corner of the emotional module full of NSSI-specific terminology
● Communities: artificial_joy, humans_being (sic), the_addicted
● Complexity and creativity as attributes of human nature
22
Thank-You!

More Related Content

Similar to The Pain of Complexity

Social Networks & Memes
Social Networks & MemesSocial Networks & Memes
Social Networks & Memesmorganjbruce
 
SC01_IntroductionSC-Unit-I.ppt
SC01_IntroductionSC-Unit-I.pptSC01_IntroductionSC-Unit-I.ppt
SC01_IntroductionSC-Unit-I.pptRamya Nellutla
 
The Science Of Social Networks
The Science Of Social NetworksThe Science Of Social Networks
The Science Of Social NetworksEhren Foss
 
Core Methods In Educational Data Mining
Core Methods In Educational Data MiningCore Methods In Educational Data Mining
Core Methods In Educational Data Miningebelani
 
NLP and personality analysis
NLP and personality analysisNLP and personality analysis
NLP and personality analysisDavid Rostcheck
 
Three networks walk into a bar...
Three networks walk into a bar...Three networks walk into a bar...
Three networks walk into a bar...Clay Spinuzzi
 
Using construction grammar in conversational systems
Using construction grammar in conversational systemsUsing construction grammar in conversational systems
Using construction grammar in conversational systemsCJ Jenkins
 
M1. sem web & ontology introd
M1. sem web & ontology introdM1. sem web & ontology introd
M1. sem web & ontology introdMichele Missikoff
 
01 Introduction to Networks Methods and Measures
01 Introduction to Networks Methods and Measures01 Introduction to Networks Methods and Measures
01 Introduction to Networks Methods and Measuresdnac
 
01 Introduction to Networks Methods and Measures (2016)
01 Introduction to Networks Methods and Measures (2016)01 Introduction to Networks Methods and Measures (2016)
01 Introduction to Networks Methods and Measures (2016)Duke Network Analysis Center
 
Objective Fiction, i-semantics keynote
Objective Fiction, i-semantics keynoteObjective Fiction, i-semantics keynote
Objective Fiction, i-semantics keynoteAldo Gangemi
 
20111123 mwa2011-marc smith
20111123 mwa2011-marc smith20111123 mwa2011-marc smith
20111123 mwa2011-marc smithMarc Smith
 

Similar to The Pain of Complexity (20)

Mesoscale Structures in Networks - Mason A. Porter
Mesoscale Structures in Networks - Mason A. PorterMesoscale Structures in Networks - Mason A. Porter
Mesoscale Structures in Networks - Mason A. Porter
 
Social Networks & Memes
Social Networks & MemesSocial Networks & Memes
Social Networks & Memes
 
Social Complexity
Social ComplexitySocial Complexity
Social Complexity
 
SC01_IntroductionSC-Unit-I.ppt
SC01_IntroductionSC-Unit-I.pptSC01_IntroductionSC-Unit-I.ppt
SC01_IntroductionSC-Unit-I.ppt
 
How to map networks
How to map networks How to map networks
How to map networks
 
How to map networks
How to map networks How to map networks
How to map networks
 
Scits 2014
Scits 2014Scits 2014
Scits 2014
 
The P4 of Networkacy
The P4 of NetworkacyThe P4 of Networkacy
The P4 of Networkacy
 
The Science Of Social Networks
The Science Of Social NetworksThe Science Of Social Networks
The Science Of Social Networks
 
Core Methods In Educational Data Mining
Core Methods In Educational Data MiningCore Methods In Educational Data Mining
Core Methods In Educational Data Mining
 
NLP and personality analysis
NLP and personality analysisNLP and personality analysis
NLP and personality analysis
 
Three networks walk into a bar...
Three networks walk into a bar...Three networks walk into a bar...
Three networks walk into a bar...
 
Using construction grammar in conversational systems
Using construction grammar in conversational systemsUsing construction grammar in conversational systems
Using construction grammar in conversational systems
 
M1. sem web & ontology introd
M1. sem web & ontology introdM1. sem web & ontology introd
M1. sem web & ontology introd
 
Applied Science - Engineering Systems
Applied Science - Engineering SystemsApplied Science - Engineering Systems
Applied Science - Engineering Systems
 
01 Introduction to Networks Methods and Measures
01 Introduction to Networks Methods and Measures01 Introduction to Networks Methods and Measures
01 Introduction to Networks Methods and Measures
 
01 Introduction to Networks Methods and Measures (2016)
01 Introduction to Networks Methods and Measures (2016)01 Introduction to Networks Methods and Measures (2016)
01 Introduction to Networks Methods and Measures (2016)
 
Objective Fiction, i-semantics keynote
Objective Fiction, i-semantics keynoteObjective Fiction, i-semantics keynote
Objective Fiction, i-semantics keynote
 
20111123 mwa2011-marc smith
20111123 mwa2011-marc smith20111123 mwa2011-marc smith
20111123 mwa2011-marc smith
 
chatcptkk.ppt
chatcptkk.pptchatcptkk.ppt
chatcptkk.ppt
 

More from Dmitry Zinoviev

Machine Learning Basics for Dummies (no math!)
Machine Learning Basics for Dummies (no math!)Machine Learning Basics for Dummies (no math!)
Machine Learning Basics for Dummies (no math!)Dmitry Zinoviev
 
WHat is star discourse in post-Soviet film journals?
WHat is star discourse in post-Soviet film journals?WHat is star discourse in post-Soviet film journals?
WHat is star discourse in post-Soviet film journals?Dmitry Zinoviev
 
The “Musk” Effect at Twitter
The “Musk” Effect at TwitterThe “Musk” Effect at Twitter
The “Musk” Effect at TwitterDmitry Zinoviev
 
Are Twitter Networks of Regional Entrepreneurs Gendered?
Are Twitter Networks of Regional Entrepreneurs Gendered?Are Twitter Networks of Regional Entrepreneurs Gendered?
Are Twitter Networks of Regional Entrepreneurs Gendered?Dmitry Zinoviev
 
Using Complex Network Analysis for Periodization
Using Complex Network Analysis for PeriodizationUsing Complex Network Analysis for Periodization
Using Complex Network Analysis for PeriodizationDmitry Zinoviev
 
Text analysis of The Book Club Play
Text analysis of The Book Club PlayText analysis of The Book Club Play
Text analysis of The Book Club PlayDmitry Zinoviev
 
Exploring the History of Mental Stigma
Exploring the History of Mental StigmaExploring the History of Mental Stigma
Exploring the History of Mental StigmaDmitry Zinoviev
 
Roles and Words in a massive NSSI-Related Interaction Network
Roles and Words in a massive NSSI-Related Interaction NetworkRoles and Words in a massive NSSI-Related Interaction Network
Roles and Words in a massive NSSI-Related Interaction NetworkDmitry Zinoviev
 
“A Quaint and Curious Volume of Forgotten Lore,” or an Exercise in Digital Hu...
“A Quaint and Curious Volume of Forgotten Lore,” or an Exercise in Digital Hu...“A Quaint and Curious Volume of Forgotten Lore,” or an Exercise in Digital Hu...
“A Quaint and Curious Volume of Forgotten Lore,” or an Exercise in Digital Hu...Dmitry Zinoviev
 
Network analysis of the 2016 USA presidential campaign tweets
Network analysis of the 2016 USA presidential campaign tweetsNetwork analysis of the 2016 USA presidential campaign tweets
Network analysis of the 2016 USA presidential campaign tweetsDmitry Zinoviev
 
Network Analysis of The Shining
Network Analysis of The ShiningNetwork Analysis of The Shining
Network Analysis of The ShiningDmitry Zinoviev
 
The Lord of the Ring. A Network Analysis
The Lord of the Ring. A Network AnalysisThe Lord of the Ring. A Network Analysis
The Lord of the Ring. A Network AnalysisDmitry Zinoviev
 
DaVinci Code. Network Analysis
DaVinci Code. Network AnalysisDaVinci Code. Network Analysis
DaVinci Code. Network AnalysisDmitry Zinoviev
 
Soviet Popular Music Landscape: Community Structure and Success Predictors
Soviet Popular Music Landscape: Community Structure and Success PredictorsSoviet Popular Music Landscape: Community Structure and Success Predictors
Soviet Popular Music Landscape: Community Structure and Success PredictorsDmitry Zinoviev
 
C for Java programmers (part 2)
C for Java programmers (part 2)C for Java programmers (part 2)
C for Java programmers (part 2)Dmitry Zinoviev
 

More from Dmitry Zinoviev (20)

Machine Learning Basics for Dummies (no math!)
Machine Learning Basics for Dummies (no math!)Machine Learning Basics for Dummies (no math!)
Machine Learning Basics for Dummies (no math!)
 
WHat is star discourse in post-Soviet film journals?
WHat is star discourse in post-Soviet film journals?WHat is star discourse in post-Soviet film journals?
WHat is star discourse in post-Soviet film journals?
 
The “Musk” Effect at Twitter
The “Musk” Effect at TwitterThe “Musk” Effect at Twitter
The “Musk” Effect at Twitter
 
Are Twitter Networks of Regional Entrepreneurs Gendered?
Are Twitter Networks of Regional Entrepreneurs Gendered?Are Twitter Networks of Regional Entrepreneurs Gendered?
Are Twitter Networks of Regional Entrepreneurs Gendered?
 
Using Complex Network Analysis for Periodization
Using Complex Network Analysis for PeriodizationUsing Complex Network Analysis for Periodization
Using Complex Network Analysis for Periodization
 
Algorithms
AlgorithmsAlgorithms
Algorithms
 
Text analysis of The Book Club Play
Text analysis of The Book Club PlayText analysis of The Book Club Play
Text analysis of The Book Club Play
 
Exploring the History of Mental Stigma
Exploring the History of Mental StigmaExploring the History of Mental Stigma
Exploring the History of Mental Stigma
 
Roles and Words in a massive NSSI-Related Interaction Network
Roles and Words in a massive NSSI-Related Interaction NetworkRoles and Words in a massive NSSI-Related Interaction Network
Roles and Words in a massive NSSI-Related Interaction Network
 
“A Quaint and Curious Volume of Forgotten Lore,” or an Exercise in Digital Hu...
“A Quaint and Curious Volume of Forgotten Lore,” or an Exercise in Digital Hu...“A Quaint and Curious Volume of Forgotten Lore,” or an Exercise in Digital Hu...
“A Quaint and Curious Volume of Forgotten Lore,” or an Exercise in Digital Hu...
 
Network analysis of the 2016 USA presidential campaign tweets
Network analysis of the 2016 USA presidential campaign tweetsNetwork analysis of the 2016 USA presidential campaign tweets
Network analysis of the 2016 USA presidential campaign tweets
 
Network Analysis of The Shining
Network Analysis of The ShiningNetwork Analysis of The Shining
Network Analysis of The Shining
 
The Lord of the Ring. A Network Analysis
The Lord of the Ring. A Network AnalysisThe Lord of the Ring. A Network Analysis
The Lord of the Ring. A Network Analysis
 
Pickling and CSV
Pickling and CSVPickling and CSV
Pickling and CSV
 
Python overview
Python overviewPython overview
Python overview
 
Welcome to CS310!
Welcome to CS310!Welcome to CS310!
Welcome to CS310!
 
Programming languages
Programming languagesProgramming languages
Programming languages
 
DaVinci Code. Network Analysis
DaVinci Code. Network AnalysisDaVinci Code. Network Analysis
DaVinci Code. Network Analysis
 
Soviet Popular Music Landscape: Community Structure and Success Predictors
Soviet Popular Music Landscape: Community Structure and Success PredictorsSoviet Popular Music Landscape: Community Structure and Success Predictors
Soviet Popular Music Landscape: Community Structure and Success Predictors
 
C for Java programmers (part 2)
C for Java programmers (part 2)C for Java programmers (part 2)
C for Java programmers (part 2)
 

Recently uploaded

Cyanide resistant respiration pathway.pptx
Cyanide resistant respiration pathway.pptxCyanide resistant respiration pathway.pptx
Cyanide resistant respiration pathway.pptxSilpa
 
Thyroid Physiology_Dr.E. Muralinath_ Associate Professor
Thyroid Physiology_Dr.E. Muralinath_ Associate ProfessorThyroid Physiology_Dr.E. Muralinath_ Associate Professor
Thyroid Physiology_Dr.E. Muralinath_ Associate Professormuralinath2
 
Digital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptxDigital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptxMohamedFarag457087
 
GBSN - Microbiology (Unit 3)Defense Mechanism of the body
GBSN - Microbiology (Unit 3)Defense Mechanism of the body GBSN - Microbiology (Unit 3)Defense Mechanism of the body
GBSN - Microbiology (Unit 3)Defense Mechanism of the body Areesha Ahmad
 
module for grade 9 for distance learning
module for grade 9 for distance learningmodule for grade 9 for distance learning
module for grade 9 for distance learninglevieagacer
 
Porella : features, morphology, anatomy, reproduction etc.
Porella : features, morphology, anatomy, reproduction etc.Porella : features, morphology, anatomy, reproduction etc.
Porella : features, morphology, anatomy, reproduction etc.Silpa
 
Molecular markers- RFLP, RAPD, AFLP, SNP etc.
Molecular markers- RFLP, RAPD, AFLP, SNP etc.Molecular markers- RFLP, RAPD, AFLP, SNP etc.
Molecular markers- RFLP, RAPD, AFLP, SNP etc.Silpa
 
TransientOffsetin14CAftertheCarringtonEventRecordedbyPolarTreeRings
TransientOffsetin14CAftertheCarringtonEventRecordedbyPolarTreeRingsTransientOffsetin14CAftertheCarringtonEventRecordedbyPolarTreeRings
TransientOffsetin14CAftertheCarringtonEventRecordedbyPolarTreeRingsSérgio Sacani
 
Proteomics: types, protein profiling steps etc.
Proteomics: types, protein profiling steps etc.Proteomics: types, protein profiling steps etc.
Proteomics: types, protein profiling steps etc.Silpa
 
POGONATUM : morphology, anatomy, reproduction etc.
POGONATUM : morphology, anatomy, reproduction etc.POGONATUM : morphology, anatomy, reproduction etc.
POGONATUM : morphology, anatomy, reproduction etc.Silpa
 
Selaginella: features, morphology ,anatomy and reproduction.
Selaginella: features, morphology ,anatomy and reproduction.Selaginella: features, morphology ,anatomy and reproduction.
Selaginella: features, morphology ,anatomy and reproduction.Silpa
 
biology HL practice questions IB BIOLOGY
biology HL practice questions IB BIOLOGYbiology HL practice questions IB BIOLOGY
biology HL practice questions IB BIOLOGY1301aanya
 
Phenolics: types, biosynthesis and functions.
Phenolics: types, biosynthesis and functions.Phenolics: types, biosynthesis and functions.
Phenolics: types, biosynthesis and functions.Silpa
 
Reboulia: features, anatomy, morphology etc.
Reboulia: features, anatomy, morphology etc.Reboulia: features, anatomy, morphology etc.
Reboulia: features, anatomy, morphology etc.Silpa
 
The Mariana Trench remarkable geological features on Earth.pptx
The Mariana Trench remarkable geological features on Earth.pptxThe Mariana Trench remarkable geological features on Earth.pptx
The Mariana Trench remarkable geological features on Earth.pptxseri bangash
 
Gwalior ❤CALL GIRL 84099*07087 ❤CALL GIRLS IN Gwalior ESCORT SERVICE❤CALL GIRL
Gwalior ❤CALL GIRL 84099*07087 ❤CALL GIRLS IN Gwalior ESCORT SERVICE❤CALL GIRLGwalior ❤CALL GIRL 84099*07087 ❤CALL GIRLS IN Gwalior ESCORT SERVICE❤CALL GIRL
Gwalior ❤CALL GIRL 84099*07087 ❤CALL GIRLS IN Gwalior ESCORT SERVICE❤CALL GIRLkantirani197
 
Human genetics..........................pptx
Human genetics..........................pptxHuman genetics..........................pptx
Human genetics..........................pptxSilpa
 
Genetics and epigenetics of ADHD and comorbid conditions
Genetics and epigenetics of ADHD and comorbid conditionsGenetics and epigenetics of ADHD and comorbid conditions
Genetics and epigenetics of ADHD and comorbid conditionsbassianu17
 
FAIRSpectra - Enabling the FAIRification of Spectroscopy and Spectrometry
FAIRSpectra - Enabling the FAIRification of Spectroscopy and SpectrometryFAIRSpectra - Enabling the FAIRification of Spectroscopy and Spectrometry
FAIRSpectra - Enabling the FAIRification of Spectroscopy and SpectrometryAlex Henderson
 
Role of AI in seed science Predictive modelling and Beyond.pptx
Role of AI in seed science  Predictive modelling and  Beyond.pptxRole of AI in seed science  Predictive modelling and  Beyond.pptx
Role of AI in seed science Predictive modelling and Beyond.pptxArvind Kumar
 

Recently uploaded (20)

Cyanide resistant respiration pathway.pptx
Cyanide resistant respiration pathway.pptxCyanide resistant respiration pathway.pptx
Cyanide resistant respiration pathway.pptx
 
Thyroid Physiology_Dr.E. Muralinath_ Associate Professor
Thyroid Physiology_Dr.E. Muralinath_ Associate ProfessorThyroid Physiology_Dr.E. Muralinath_ Associate Professor
Thyroid Physiology_Dr.E. Muralinath_ Associate Professor
 
Digital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptxDigital Dentistry.Digital Dentistryvv.pptx
Digital Dentistry.Digital Dentistryvv.pptx
 
GBSN - Microbiology (Unit 3)Defense Mechanism of the body
GBSN - Microbiology (Unit 3)Defense Mechanism of the body GBSN - Microbiology (Unit 3)Defense Mechanism of the body
GBSN - Microbiology (Unit 3)Defense Mechanism of the body
 
module for grade 9 for distance learning
module for grade 9 for distance learningmodule for grade 9 for distance learning
module for grade 9 for distance learning
 
Porella : features, morphology, anatomy, reproduction etc.
Porella : features, morphology, anatomy, reproduction etc.Porella : features, morphology, anatomy, reproduction etc.
Porella : features, morphology, anatomy, reproduction etc.
 
Molecular markers- RFLP, RAPD, AFLP, SNP etc.
Molecular markers- RFLP, RAPD, AFLP, SNP etc.Molecular markers- RFLP, RAPD, AFLP, SNP etc.
Molecular markers- RFLP, RAPD, AFLP, SNP etc.
 
TransientOffsetin14CAftertheCarringtonEventRecordedbyPolarTreeRings
TransientOffsetin14CAftertheCarringtonEventRecordedbyPolarTreeRingsTransientOffsetin14CAftertheCarringtonEventRecordedbyPolarTreeRings
TransientOffsetin14CAftertheCarringtonEventRecordedbyPolarTreeRings
 
Proteomics: types, protein profiling steps etc.
Proteomics: types, protein profiling steps etc.Proteomics: types, protein profiling steps etc.
Proteomics: types, protein profiling steps etc.
 
POGONATUM : morphology, anatomy, reproduction etc.
POGONATUM : morphology, anatomy, reproduction etc.POGONATUM : morphology, anatomy, reproduction etc.
POGONATUM : morphology, anatomy, reproduction etc.
 
Selaginella: features, morphology ,anatomy and reproduction.
Selaginella: features, morphology ,anatomy and reproduction.Selaginella: features, morphology ,anatomy and reproduction.
Selaginella: features, morphology ,anatomy and reproduction.
 
biology HL practice questions IB BIOLOGY
biology HL practice questions IB BIOLOGYbiology HL practice questions IB BIOLOGY
biology HL practice questions IB BIOLOGY
 
Phenolics: types, biosynthesis and functions.
Phenolics: types, biosynthesis and functions.Phenolics: types, biosynthesis and functions.
Phenolics: types, biosynthesis and functions.
 
Reboulia: features, anatomy, morphology etc.
Reboulia: features, anatomy, morphology etc.Reboulia: features, anatomy, morphology etc.
Reboulia: features, anatomy, morphology etc.
 
The Mariana Trench remarkable geological features on Earth.pptx
The Mariana Trench remarkable geological features on Earth.pptxThe Mariana Trench remarkable geological features on Earth.pptx
The Mariana Trench remarkable geological features on Earth.pptx
 
Gwalior ❤CALL GIRL 84099*07087 ❤CALL GIRLS IN Gwalior ESCORT SERVICE❤CALL GIRL
Gwalior ❤CALL GIRL 84099*07087 ❤CALL GIRLS IN Gwalior ESCORT SERVICE❤CALL GIRLGwalior ❤CALL GIRL 84099*07087 ❤CALL GIRLS IN Gwalior ESCORT SERVICE❤CALL GIRL
Gwalior ❤CALL GIRL 84099*07087 ❤CALL GIRLS IN Gwalior ESCORT SERVICE❤CALL GIRL
 
Human genetics..........................pptx
Human genetics..........................pptxHuman genetics..........................pptx
Human genetics..........................pptx
 
Genetics and epigenetics of ADHD and comorbid conditions
Genetics and epigenetics of ADHD and comorbid conditionsGenetics and epigenetics of ADHD and comorbid conditions
Genetics and epigenetics of ADHD and comorbid conditions
 
FAIRSpectra - Enabling the FAIRification of Spectroscopy and Spectrometry
FAIRSpectra - Enabling the FAIRification of Spectroscopy and SpectrometryFAIRSpectra - Enabling the FAIRification of Spectroscopy and Spectrometry
FAIRSpectra - Enabling the FAIRification of Spectroscopy and Spectrometry
 
Role of AI in seed science Predictive modelling and Beyond.pptx
Role of AI in seed science  Predictive modelling and  Beyond.pptxRole of AI in seed science  Predictive modelling and  Beyond.pptx
Role of AI in seed science Predictive modelling and Beyond.pptx
 

The Pain of Complexity

  • 1. 1 The Pain of Complexity Dmitry Zinoviev Suffolk University, Boston
  • 2. 2 “Complexity” is one of those fuzzy concepts that everyone seems to understand differently. Other examples: sustainability, resilience, success... What Is Complexity?
  • 3. 3 Let's Ask the Mechanical Turk! ● Question: “Enter three nouns (please, nouns only!) that, to the best of your understanding, are associated with complexity. These words could be synonyms, descriptors, attributes of complexity, actions or any other related words.” ● Ask 100 mTurk workers, pay a nominal fee for an answer ● Reject non-nouns; stem* and aggregate nouns *We tried both “Porter” and “Lancaster” stemmers; “Porter” is less aggressive, it produced more stems (“motifs”); some manual tweaking was needed
  • 4. 4 Direct Survey Results ● The stems that were mentioned at least twice (in the order of decreasing frequency): – Most frequently mentioned (23+ times): COMPLIC*, DIFFICULTI*, INTRICACI* – Less frequently mentioned (2–9 times): PROBLEM*, ELABOR*, ENTANGL*, PUZZL*, MULTIPL*, COMPLEX*, COMPUT*, CONVOLUT*, HARD*, RAMIF*, SOPHIST*, CHALLENG*, HUMAN*, INVOLV*, LIFE*, POLIT*, RIDDL*, SCIENC*, COMPOSIT*, CONFUS*, DEPTH*, DESIGN*, DIVERS*, LABYRINTH*, LANGUAG*, MATH*, MAZE*, OBSCUR*, PHYSIC*, QUALITI*, RELATIONSHIP*, SPACE*, STRUCTURE*, TEST*, TRICKI*, TROUBL*, VARIABL*, VARIETI* – The highlighted stems bear strong emotional load—though, no pain yet!
  • 5. 5 Use Indirect Data ● Don't tell me who you are. “Tell me who your friends are, and I will tell you who you are.” ● Instead of asking people what they think “complexity” is, let's collect abundant implicit markers off the Internet.
  • 6. 6 Indirect Data: Possible Sources ● EBSCO keywords or subject tags (subscription and harvesting software required; we haz it, but the results may be too academic!) ● Blogging sites like LiveJournal (LJ) – LJ has individual blogs and community blogs (like forums) – Both types of blogs can and do have user- or moderator-declared interests – The interests are usually carefully chosen to reflect the blogger's/community's online identity – Free access, easy-to-write harvester
  • 7. 7 Raw Data ● All LiveJournal communities that list at least one of the following interests: – “complexity” – “complex systems” – “complexity theory” – “complexity theories” ● Found 59 communities, such as mixedtype, abstractthought, ivygreenfanclub, pdmi_logic, investors, gifted_teens, etc. ● Many more individual bloggers—we ignored them, their interest lists are way too broad! ● Selected 374 most frequently declared interests (the corpus)
  • 8. 8 Example: “Ecologists” community ● Community ecologists.livejournal.com ● 150 self-declared (by the moderator(s)) interests ● Includes “complexity,” but also many other keywords; are they relevant? Let's assume they are.
  • 9. 9 Term Vector Model ● Treat communities as documents and interests as words ● Calculate generalized similarity between words (and between communities—but why?) using Kovacs (2010) algorithm: – Two words are similar if they belong to similar documents – Two documents are similar if the consist of similar words ● Remark: generalized similarity, as defined by Kovacs, is essentially Pearson correlation in an oblique Cartesian coordinate space ● For any two words W and V from the corpus, -1 ≤ d(W,V) ≤ 1 is the similarity between the words; for any two communities C and D, -1 ≤ d(C,D) ≤ 1 is the similarity between the communities (a by-product, not needed)
  • 10. 10 Network Construction (1) ● Treat words as nodes in a network ● Treat similarities as edges if they are above T0 =0.65 (“slicing”); the choice of T0 is tricky and arbitrary: – If T0 is too high, the network is complete and not interesting – If T0 is too low, the network is sparse and disconnected ● Gephi it!
  • 11. 11 “Complexity Dragon” (a.k.a. the Mindscape of Complexity)
  • 12. 12 Modular Structure Modularity=0.56 (not great, but still visible) psychology, chaos, science, mathematics, complexity theory, ... philosophy, life, imagination, self-expression, knowledge, ... complexity, honesty, music, simplicity, love, poetry, creativity, empathy, humor, hate, ... writing, art, books, romance, drawing, ...
  • 13. 13 What Is the Meaning of Modules? ● We could just stare at the modules and try to come up with a suitable name— ● —or we could use crowdsourcing through Amazon Mechanical Turk again! Fragment of the “fireball”
  • 14. 14 Let's Ask the Mechanical Turk! ● Question: “Describe the following group of 50* words with a single most suitable word or a two-word or three-word phrase: ...” *There are only 36 terms in the lower left module. ● Ask 100 mTurk workers, pay a nominal fee for an answer ● Reject non-nouns; stem and aggregate nouns ● The nouns are module descriptors (“motifs”)
  • 15. 15 Network Construction (2) ● Treat original modules as documents and motifs as words ● Calculate generalized similarity ● Do slicing ● Build a network of stems ● Gephi it!
  • 16. 16 The Three Shades of Complexity ● The new network has a simple, but not trivial, structure, and is highly modular ● The Three Shades of Complexity: – Science/Technology – Society/Mind – Creativity/Emotions ● Connections: – via education – via life/humanities There are lines here, they are just too thin...
  • 17. 17 Semantic Spectrography mTurk for motif extraction and clustering Term source for mindscape construction Analysis term selection
  • 18. 18 Spectrography vs Direct Survey Direct Survey: 37 Spectrography: 382 4 9 4 104 111 113 37 24
  • 19. 19 Comparison Summary ● Positive: – Spectrography is much more detailed: it catches 41% of direct survey terms; direct survey catches only 4.5% of spectrography terms – Spectrography reveals structure ● Negative: – Spectrography requires a source (or sources) of terms
  • 20. 20 Successfully Used Elsewhere ● D. Zinoviev, D. Stefanescu, L. Swenson, and G. Fireman, “Semantic Networks of Interests in Online NSSI* Communities,” in Proc. Workshop “Words and Networks,” Evanston, IL, June 2012, published online (also submitted to Social Networks in 2013) ● D. Zinoviev and Z. Zhu, “Conceptual Structure of Sustainability: Social and Scholarly Perspectives,” Sunbelt XXXIV Social Networks Conference, St.-Pete Beach, FL, February 2014 (also submitted to Social Networks in 2014) *Non-Suicidal Self Injury—a common and epidemically spreading activity among adolescents and young adults
  • 21. 21 But Where Is the Pain? ● SE corner of the emotional module full of NSSI-specific terminology ● Communities: artificial_joy, humans_being (sic), the_addicted ● Complexity and creativity as attributes of human nature