本スライドは、弊社の梅本により弊社内の技術勉強会で使用されたものです。
近年注目を集めるアーキテクチャーである「Transformer」の解説スライドとなっております。
"Arithmer Seminar" is weekly held, where professionals from within and outside our company give lectures on their respective expertise.
The slides are made by the lecturer from outside our company, and shared here with his/her permission.
Arithmer株式会社は東京大学大学院数理科学研究科発の数学の会社です。私達は現代数学を応用して、様々な分野のソリューションに、新しい高度AIシステムを導入しています。AIをいかに上手に使って仕事を効率化するか、そして人々の役に立つ結果を生み出すのか、それを考えるのが私たちの仕事です。
Arithmer began at the University of Tokyo Graduate School of Mathematical Sciences. Today, our research of modern mathematics and AI systems has the capability of providing solutions when dealing with tough complex issues. At Arithmer we believe it is our job to realize the functions of AI through improving work efficiency and producing more useful results for society.
本スライドは、弊社の梅本により弊社内の技術勉強会で使用されたものです。
近年注目を集めるアーキテクチャーである「Transformer」の解説スライドとなっております。
"Arithmer Seminar" is weekly held, where professionals from within and outside our company give lectures on their respective expertise.
The slides are made by the lecturer from outside our company, and shared here with his/her permission.
Arithmer株式会社は東京大学大学院数理科学研究科発の数学の会社です。私達は現代数学を応用して、様々な分野のソリューションに、新しい高度AIシステムを導入しています。AIをいかに上手に使って仕事を効率化するか、そして人々の役に立つ結果を生み出すのか、それを考えるのが私たちの仕事です。
Arithmer began at the University of Tokyo Graduate School of Mathematical Sciences. Today, our research of modern mathematics and AI systems has the capability of providing solutions when dealing with tough complex issues. At Arithmer we believe it is our job to realize the functions of AI through improving work efficiency and producing more useful results for society.
33. データのロード
• PLDA形式のデータを読み込む
a 2 is 1 character 1
a 2 is 1 b 1 character 1 after 1
class MyPipe extends Pipe{ static InstanceList load(String fileName) {
@Override ArrayList<Pipe> pipeList = new ArrayList<Pipe>();
public Instance pipe(Instance inst) { pipeList.add(new MyPipe());
String data = (String)inst.getData(); pipeList.add(new TokenSequence2FeatureSequence());
String array[] = data.split("¥¥s+"); InstanceList list =
TokenSequence ret = new TokenSequence(); new InstanceList(new SerialPipes(pipeList));
for(int i = 0 ; i < array.length ; i += 2){ CsvIterator it = new CsvIterator(fileName, "(.*)",1, 0,0);
String word = array[i]; list.addThruPipe(it);
int freq = Integer.parseInt(array[i + 1]); return list;
for(int f = 0 ; f < freq; ++f){ }
ret.add(new Token(word));
}
}
inst.setData(ret);
return inst;
}
}
36. トピックの代表的単語の抽出
• printTopWordsを使う
0 0.1847 algorithm learning function gradient convergence parameter error iteration vector
1 0.03452 map dominance ocular development pattern mapping organizing kohonen eye
2 0.01327 hint return data cost market stock prediction load subscriber
3 0.71807 case term result form consider general defined order paper
4 0.02225 face images recognition image faces representation hand video facial
5 0.42392 values line order point number high step result factor
6 0.01545 disparity gamma game play player partition games board operator
7 0.09096 local point region surface contour segment data field path
8 0.04591 prediction series error network predict training road predictor committee
9 0.12844 vector matrix linear space component dimensional point data transformation
...
38. 参考文献
• [Asuncion+ 2009] On smoothing and inference for topic models, UAI
• [Blei+ 2003] Latent Dirichlet allocation, JMLR
• [Griffiths and Steyvers 2004] Finding scientific topics, PNAS
• [Newman+ 2007] Distributed inference for latent Dirichlet allocation, NIPS
• [Smola and Narayanamurthy 2010] An architecture for parallel topic models, VLDB
• [Steyvers and Griffiths 2007] Probabilistic topic models, In Handbook of Latent
Semantic Analysis
• [Teh+ 2007] A collapsed variational Bayesian inference algorithm for latent
Dirichlet allocation, NIPS
• [Wallach+ 2009] Rethinking LDA: Why Priors Matter, NIPS
• [Wang+ 2009] PLDA: Parallel Latent Dirichlet Allocation for Large-scale Applications,
AAIM
• [Wilson and Chew 2010] Term Weighting Schemes for Latent Dirichlet Allocation,
ACL
• [Yan+ 2009] Parallel Inference for Latent Dirichlet Allocation on Graphics
Processing Units, NIPS
• [Yao+ 2009] Efficient methods for topic model inference on streaming document
collections, SIGKDD
39. 参考文献2
• [Bao and Chang 2010] AdHeat: an influence-based
diffusion model for propagating hints to match ads
• [Chen+ 2009] Collaborative filtering for Orkut
communities : discovery of user latent behavior
• [Lau+ 2010] Best topic word selection for topic
labelling, Colling
• [Phan+ 2008] Learning to classify short and sparse text
& web with hidden topics from large-scale data
collections
• [Wei and Croft 2006] LDA-based document models for
ad-hoc retrieval, SIGIR