A poster from TPDL 2013

1. LargeIscaleIcitationImatching
usingIApacheIHadoop
MateuszCFedoryszak9C%ominikaCTkaczykCandCŁukaszCæolikowski
WhatCisCcitationCmatching[ WhatCisCHadoop[
WeCareCtheCpartCofCUoNnSysA
PairwiseCsimilarity
Results
PerformanceCtests
IntermezzoGCNuthorCindexing
Pawlak9CZdzisławCx"vbJZ2CSRoughCsetsS2CInternationalCJournalCofCParallelCProgrammingC""Cx4ZGC53"–54–2
Pawlak 9 Zdzisław x "vbJ Z 2
author other author other year other other
222
222
DuringRcitationRmatchingRtheRlinksRfromRbibliographicRentriesRtoRreferencedR
publicationsRareRcreatedORSuchRlinksRareRindicatorsRofRtopicalRsimilarityRbetweenR
linkedRtextsARareRusedRinRassessingRtheRimpactRofRtheRreferencedRdocumentRandR
improveRnavigationRinRtheRuserRinterfacesRofRdigitalRlibrariesOR
FirstRofRallARmetadataRfieldsRareRextractedRfromRaRcitationRstringRinRaRprocessRcalledR
citationRparsing8RitRisRdoneRusingRCRF)basedRparserRsuppliedRwithRCERMINER
)RaRmetadataRandRcontentRextractionRtoolRzTkaczykRetRalOARVWHVARDAS2O
OurRcitationRmatchingRalgorithmRisRimplementedRasRtheRpartRofRCoAnSysARanRopenRsourceRframeworkRforRminingRscientificRpublicationsORItRisR
developedRatRtheRCentreRforROpenRScienceRzCeON2RinRtheRInterdisciplinaryRCentreRforRMathematicalRandRComputationalRModellingRzICM2AR
UniversityRofRWarsawO
ApacheRHadoopRisRanRopenRsourceRimplementationRofRMapReduceRparadigmAR
whichRallowsRbigRdataRtoRbeRpartitionedRandRprocessedRinRsmallRportionsO
ThenARthoseRpiecesRofRinformationRareRusedRtoRassessRsimilarityRbetweenRtwoR
citationsRorRbetweenRaRcitationRandRdocumentRmetadataRbyRmeansRofRfuzzyR
similarityRmetricsRwhichRareRcombinedRbyRSVMO
GivenRaRpairwiseRsimilaritiesARcitationsRareRclusteredRusingRaRsimpleRsingle)linkR
algorithmORWeRhaveRtestedRtheRclusteringRcorrectnessRonRCORA)refRdatasetRandR
comparedRtheRresultsRwithRthoseRachievedRbyRJointRMLNRmethodRzPoonRandR
DomingosARVWW7ARAI2O
)RAuthor similarityR)RheaviestRmatchingRofRtokensARanRinstanceRofRassignmentR
RRproblem
)RJournal similarityR)RlongestRcommonRsubsequenceRofRcharacters
)RTitle similarityR)RaRfractionRofRcommonRtrigramsRzcharacter)based2
)RPages similarityR)RaRfractionRofRcommonRtokens
)RYear equality
HadoopisedCcitationCmatching
ToRlimitRtheRnumberRofRpairwiseRcomparisonsARweRhaveRusedRaRheuristicRbasedRonR
authorRindexingORToRbeRlessRvulnerableRtoRspellingRerrorsRweRhaveRimplementedRanR
indexRsupportingRretrievalRofRtokensRwithReditRdistanceRlessRthanRorRequalRHO
InsteadRofRputtingRasRaRkeyRanRexactRwordRwARweRputRallRtheRrotationsRofRw$RzwhereR$R
isRaRcharacterRnotRpresentRinRanRalphabet2ORInRaRsimilarRmannerARtoRretrieveRaRwordR
fromRtheRindexARweRcreateRallRitsRrotationsRandRforReachRrotationRrRofRlengthRnARallRtheR
keysRofRlengthR≤ nRthatRmatchRatRleastRn-1RfirstRlettersRofRrRandRkeysRofRlengthR≤ n+1R
thatRmatchRfirstRnRlettersRareRreturnedO
NuthorCindexCbuilding
NctualCmatching
docId2"
docId2J
docId2"
doc2" doc2J
docId2J docId2J docId2J
token2" token2J token25
%ocuments
%ocumentCI%sCand
extractedCtokens
%ocumentCI%s
groupedCbyCtokens
Rotations
forCeachCtoken
docId2" docId2"
token2" token2J
token2" token2J token25
docId2"
docId2J
docId2"
docId2J
rot2J2" rot2J2J rot2J25
docId2"
docId2J
docId2"
docId2J
%ocumentCretrieval
TokenCextraction
Grouping
RotationsCgeneration
Persisting
doc2"
doc2J
doc25
ref2J2"
ref2J2J
ref2J25
ref2J2J
ref2J2J
ref2J2J
cand2J2J2"
cand2J2J2J
cand2J2J25
ref2J2J cand2J2J25
ref2x2y cand2x2y2x
ref2x2y cand2x2y2z
%ocument
reading
ReferenceCstring
extraction
HeuristicCmatching
Selecting
theCbestCmatching
Storing
resolvedCreferences
%ocuments References
ReferencesCwith
candidateCdocuments
æestCmatchingCpairs
Map Reduce
Nll .G..G46 "5 J
.G..G3– "5 .
5G."G5b 634 .
JG4"G.. vv– "
VisitCusCatChttps://github.com/CeON/CoAnSys/
Map ReduceSort
ICshuffle
References
["]CI2CNewton9CPhilosophiæCnaturalis222
[J]CN2CUopernicus9C%eCrevolutionibus222
ID Title Author
Uopernicus"3 %eCrevolutionibus222
ΕὐκλείδηςΣτοιχεῖα""
InterdisciplinaryCUentreCforCMathematicalCandCUomputationalCModelling9CUniversityCofCWarsaw
{m.fedoryszak, d.tkaczyk, l.bolikowski}@icm.edu.pl
WeRhaveRevaluatedRefficiencyRofRourRsolutionRusingRPMCROpenRAccessRSubsetR
documentRsetORItRconsistsRofRoverRJLWRthousandRdocumentsRcontainingRHVRmillionR
citationsORTheRbenchmarkRwasRperformedRonRourRHadoopRclusterRwhichRconsistsRofR
fourRʹfatʹRslaveRnodesAReachRcontainingRJqRCPURcoresO
–42bJR 6J24.R 66245R 6"2v4R 6b2".R
v42J"R v624"R v32vbR v42v.R v62..R
v52v"R v52.–R v6235R v32b.R v325.R
v324–R v42J5R v–2"vR v4255R v42–5R
clusterCrecall
pairwiseCprecision
pairwiseCrecall
pairwiseCF"
Nverage
Phase
TimeCspent
[hrsGminsGsecs]
IndexCbuilding
Matching
UitationCextraction
HeuristicCmatching
SelectingCtheCbestCmatch
TaskCNo2
Fold. Fold" FoldJ JointCMLN