SlideShare a Scribd company logo
DEEP LEARNING JP
[DL Papers]
言語以外でのTransformerのまとめ
(ViT, Perceiver, Frozen Pretrained Transformer etc)
発表者:岩澤有祐
http://deeplearning.jp/
発表概要
• 言語で主に使われてきたSelf Attention(Transformer)が,様々な領域
で使われつつある
• 画像における利用
– SASA(NeurIPS2019),SANs(CPVR2020),ViT(ICLR2021)など
– ViTの後継論文としてDeIT,T2Tなど
• より汎用なモジュールとしての可能性についての論文も
– Perceiver:様々なモダリティで汎用に使えるモデル
– Frozen Pretrained Transformer:言語 -> ほかモダリティへの転移
• 主に言語以外でのTransformerの活用についてのまとめ 2
目次
• 前提知識:Self Attention
• 画像におけるSelf Attention
• モダリティに汎用な(Self) Attention
– Perceiver
– Frozen Pretrained Transformer
3
前提知識:SELF ATTENTION
4
前提の前提:単語埋め込み
5
入力 “Attention is all you need”
Embedding
size 𝐷
Token size 𝐿
= 𝑋𝑇
Self Attention
6
𝑄 = 𝐾 = 𝑉 = 𝑋の場合がSelf Attention
行列サイズの可視化
7
Multi Head Self Attention
8
各ヘッドの処理(h個)
• Q, K, Vを適当な重みWで埋め込む
• Wは(トークンサイズ,ヘッドサイズ)の重み
全体をまとめる処理
• Concat + Linear
• 通常は入力と同じサイズになるようにする
Positional Encoding
9
• Self AttentionはPermutation Invariant
(位置情報が入らない)
• 明示的に埋め込んだものがPE
• PEの次元は入力と同じ
"Visual Guide toTransformer Neural Networks - (Episode 1)
Position Embeddings" (Youtube)より抜粋
Positional Encoding(縦が位置,横が埋め込みの長さ)
10
"Transformer Architecture:The Positional Encoding" (Blog)より抜粋
Transformer Encoder
11
Residual Connection + Layer Norm
• 入力を足す(MHSAの出力は入力と同じ大きさ)
• LayerNormなどによる正規化
(Position-wise) Feed Forward
• 各トークンごとにFeed Forwardに入力
• FFN(x) = σ(xW1 _ b1)W2 + b2
• W1の次元は(トークン長,隠れ層サイズ)
• Σは活性化関数(GELUなどがよく使われる)
• 最終的な出力は入力と同じサイズ
Multi Head Self Attention
• 前述の通り
補足(余談)
• Feed Forward層は何をしているのか?
– Key-Valueのニューラルメモリ機構になっている
– “Transformer Feed-Forward Layers Are Key-Value Memories”, 2020
(arXiv)
• ResidualとFeed Forwardは必須
– “Attention is Not All You Need: Pure Attention Loses Rank Doubly
Exponentially with Depth”, 2021 (arXiv)
• 細かい部分は重要か?(活性化関数とか)
– “Do Transformer Modifications Transfer Across Implementations and
Applications?” 2021 (arXiv)
12
本題1:画像におけるSELF ATTENTION
13
画像におけるSelf Attentionの大別
14
“Bottleneck Transformers for Visual Recognition”より抜粋
ConvとAttentionを
組み合わせ
基本的に
Attentionのみ
本資料のメイン
画像における単語(トークン)とは?
15
テキストの場合 画像の場合
• 単語1つが1単位として文章を表す • 画像の場合は???
??
主な方法1:ピクセルをトークンとみなす
16
• 全体を見ると計算量が爆発
Local Attention
17
“Stand-Alone Self Attention in Vision Models”より抜粋
• 𝐾, 𝑉を空間的に
近い部分のみに
• 計算量を削減
• SASAやSANsなど
畳み込みとLocal Attention
18
畳み込み Local Attention
観測値に依存しない重みをかける 観測値に依存した重みをかける
※理論的な関係は”On the Relationship between Self-Attention and Convolution Layers”など
(相対的PEを使う場合任意の畳込みをSelfAttentionは近似出来る)
その他の計算量削減方法:Axial Attention [Wang+2020]
19
Self Attentionを縦方向と横方向に分けることで計算量削減
主な方法2:パッチをトークンとみなす
20
ViT
iGPT
※図はそれぞれの論文より抜粋
ViTの全体像
21
ViTの補足
• PEには1次元の学習可能なパラメータを利用
– その他に2次元の学習可能パラメータ,相対的な座標を使う,
使わないなどを検証(Appendix D参照)
• クラス分類はTransformer出力後のCLS tokenを利用する
– CLS tokenの初期値も学習可能パラメータ
– iGPTは最終層を空間方向に平均化して入力
• 若干Transformerの構造が元と違う
– 具体的にはNormalizationの位置
22
Self Attention vs. CNN
24
“A Survey on Visual Transformer”より抜粋
超巨大データを使うと畳み込みを凌駕
ここまでのまとめ
• 画像におけるPure Attention Baseな研究について説明
• 特に最近提案されたViTは大量データ大量パラメータだと
畳み込みを凌駕することも
– 基本的にはパッチで分けてTransformerに突っ込む
• データ効率を上げる研究もいくつかある
– DeiT[Hugo+ 2021],Tokens-to-Tokens [Li+ 2021]
• なおこの辺の手法はViT-pytorchという個人のリポで再現実装有
– 正しく検証できているのかは不明 25
モダリティに汎用な(SELF) ATTENTION
26
書誌情報
• “Perceiver: General Perception with Iterative Attention”
• 著者:Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol
Vinyals, Joao Carreira
• 所属:DeepMind
• 投稿日:2021/03/04 (arXiv)
• 概要
– 様々なモダリティに汎用に使えるモデル,Perceiverの提案
– データ構造を仮定せずに高次元の入力をなるべくそのまま扱う
– 画像,音声,動画,点群などのデータでSoTAに近しい性能を達成
– (詳細があまり書いてないので若干解釈入ってます)
27
お気持ち
• 畳み込みのような構造を仮定することは正しいのか?
• なるべく自由度を上げてデータが自分自身を語るようにするべき
なのか?(本論文の立場)
• 構造を仮定する具体的な問題:ドメインごとに異なるアーキテク
チャを設定しなければならない
– 手間がかかる.データを集めればOKという問題にしたい.
28
提案:Perceiver (=Transformer + Cross Attention)
29
Transformerは超柔軟だが,高次元(長い系列)の扱いに難
=> 潜在コードとのCross Attentionを導入(あとはTransformer)
新しい部分 既存 繰り返し
Cross Attention
30
前のページの図の拡大 • Latent Arrayは,適当なサイズの学
習可能なパラメタ
• Byte Arrayは入力(画像など)
• N <<< M
– 例えば224×224の場合M>50000
– Nはハイパラ(実験では512など
• O(NN) => O(MN)
– 実際は繰り返すのでかなり軽量化
全体像(再掲)
31
• Cross Attention -> 潜在変数上でのTransformerを繰り返し
• パラメータ共有しても良い(C.F. Universal Transformer)
新しい部分 既存 繰り返し
Perceiverの補足(推測)
• ViTは固定グリッドで圧縮,Perceiverは類似度で圧縮
• Latent Arrayは1024 * 512とか
• Byte ArrayとLatent Arrayはサイズが違うので内積取れない
– 多分MHSAと同様適当なサイズに次元変換(=線形層1層かます)
• Byte ArrayのPositional EncodingはSinusoidをConcat
– 普通のTransformerはAdd
– 任意の数のPEを用意出来る(次元揃える必要ないので)
• Latent ArrayのPEは1次元の学習可能パラメタ(多分)
32
実験:扱っているデータ
33
汎用にPerceiverが利用出来ることを検証
(ほぼ変更なしでSoTAと同等程度であることを確認)
実験設定:ImageNet
• 224×224にクロップしてRandAugmentでデータ拡張
– なので全くドメイン知識使ってないわけではない
• PEはクロップした画像を[-1, 1]としてつける
– クロップする前でやると過学習したとのこと
• 最適化はLamb Optimizer + Scheduling
– “Large Batch Optimization for Deep Learning: Training BERT in 76 minutes”
• (Cross Attention + Lattent×6)×8 (合計44Mのパラメタ)
– Cross Attentionは1ヘッドのみ
– Latent Arrayは1024個(それぞれ512次元)
– 最初のCross Attention + Latent以外は重み共有
34
結果:ImageNet
35
• 青がデータ構造をほぼ仮定してない手法
• Transformerは64×64にリサイズした画像を利用
• ResNet-50やViT(ImageNetで学習)と同程度
構造仮定なし
同じPE
元論文
結果:Permuted ImageNet
36
• ImageNetをPermutationしたものでテスト
– Fixedは全データ共通のPermutation
– Randomは全データで違うPermutation
• Perceiverは構造が無い入力でもうまくいく(ほかはダメ)
AudioSetとPoint Cloud
37
AudioSet Point Cloud
• 音声は61,440次元,動画は65,536次元
• PerceiverがSoTA
• データ拡張も音声特有のものは未使用
• 2000個の座標が入力
• 構造仮定なしでは提案が
良い
Perceiverまとめ
• 柔軟なTransformerで高次元入力をうまく扱う方法の提案
– Self Attentionではなく潜在変数とのCross Attentionを使う
• 様々なモダリティでSoTAに近い性能
• GANsformerでも似た方法が取られている
• ViT系の実装と同じ人が実装を公開している
– 更に余談としてこの人はGLOM [Hinton, 2021]も実装を公開
– 動いているのかは不明
38
PRETRAINED TRANSFORMERS AS
UNIVERSAL COMPUTATION ENGINE
39
書誌情報
• “Pretrained Transformers as Universal Computation Engines”
• 著者:Kevin Lu, Aditya Grover, Pieter Abbeel, Igor Mordatch
• 所属:UC Berkeley, FAIR, Google Brain
• 投稿日:2021/03/09 (arXiv)
• 概要
– 言語で訓練したTransformerは別のモダリティにも転移可能(!!!?)
– 実験論文
40
Frozen Pretrained Transformer(FPT)
41
• 言語でTransformer全体を訓練(構造はGPT2を利用)
• Self AttentionとFeed Forwardを固定
• 入力層の埋め込み,位置埋め込み,出力層,Layer Normは再学習
• 別のモダリティに転移して性能を検証
検証している後続タスク
• Bit Memory
• Bit XOR
• List Ops([MAX 4 3 2 1] -> 4)
• MNIST(画像 -> カテゴリ)
• CIRFAR-10(画像 -> カテゴリ)
• CIFAR-10 LRA(画像をグレースケールにしてFlatten)
• Remote homology detection(アミノ酸配列 -> 折りたたみ構造)
42
実験1:FPT vs. Full vs. LSTM
43
• FPTでFull Transformer(全学習)と同程度の性能が出る
• (というものの,そんなに精度が高くない気はする.
実験2:言語以外のモダリティで事前学習
44
• Randomは単に初期化した状態
• BitはBit Memoryタスクで事前学習
• ViTはViTの重みを利用
• 総合的には言語がよい(のでは)という主張(例えばHomologyでVitは▲
実験3:Transformer vs LSTM
45
• どちらもランダムに初期化したものの比較
• Transformerが大幅に良い
実験8:初期化の工夫
46
• 学習済みとランダムだと様々な統計量が大きく異なる
• 統計量だけ合わせて初期化するとよくなるか?
• Yes(だがそれだけでもない.
実験7:モデルサイズに応じて性能上がるか
47
• 大きいと良い
• (実験はもっと大量にあるので興味ある方は論文参照
FPTまとめ・感想
• Transformerが学習しているのは,関数というより処理
• どういう処理をするべきかは,抽象的な空間では同じ
(なのかもしれない...)
• 汎化にはグラフ構造が必要という話とも関係するかもしれない
• 割と小規模なデータでしか試してないので,もっと大きい
規模だと変わる気もする
• Global Workspaceとの関係が書かれてたがよくわからなかった
48
あまり話せなかった話題(文献末尾につけてます)
• GANsformer
– 画像分類だけでなく,画像生成などでも使われている
– GANsformer自体はPure Attentionではない
• DeiT,Tokens-to-Tokens
– ViTの改良手法
– 主にデータ効率が良くなる
• GLOM
– ViT+Top-Down Attention + Consensus (- Multi Head)
– 画像をパース木で表すようなモデル
49
全体まとめ・感想
• 言語以外でのTransformerの活用が進んでいる
– 画像:ViTなど
– 汎用:Perceiver,Frozen Pretrained Transformerなど
• なるべく仮定をへらす方向に進むのはおそらくそうなる
• Transformerは結局何をしているのか
– 単なる関数と捉えると見誤るような気もする
• 若干解釈が入ってます(誤りあったらごめんなさい m(_ _)m
50
主な文献等:Self Attention, Transformer一般
• “Attention is All You Need”, NeurIPS2017
• “Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with
Depth”, 2021 (arXiv)
• “Do Transformer Modifications Transfer Across Implementations and Applications?”
2021 (arXiv)
• “Transformer Architecture: The Positional Encoding”, Blog
• “Visual Guide to Transformer Neural Networks - (Episode 1) Position Embeddings”,
Youtube
51
主な文献等:画像に関するSelf Attention
Pure Attention系
(SASA) “Stand-Alone Self-Attention in Vision Models”, NeurIPS2019
(SANs) “Exploring Self-attention for Image Recognition”, CPVR2020
(axial attention) “Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation”, ECCV2020
(iGPT) “Generative Pretraining From Pixels”, ICML2020
(ViT) “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale”. ICLR2021 (DeiT) “Training
data-efficient image transformers & distillation through attention”, arXiv, 2021
(T2T) “Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet”, arXiv, 2021
(Survey) “A Survey on Visual Transformer”, arXiv, 2021
それ以外
(GANsformer) “Generative Adversarial Transformers”, arXiv, 2021
(Bottleneck Transformer) “Bottleneck Transformers for Visual Recognition”, arXiv, 2021
(GLOM) “How to represent part-whole hierarchies in a neural network”, arXiv, 2021
52
参考:ViT系の実装をガリガリしている人
53
※ ちょこちょこ細かいところは違うっぽい? https://github.com/lucidrains より

More Related Content

What's hot

Transformerを雰囲気で理解する
Transformerを雰囲気で理解するTransformerを雰囲気で理解する
Transformerを雰囲気で理解する
AtsukiYamaguchi1
 
Layer Normalization@NIPS+読み会・関西
Layer Normalization@NIPS+読み会・関西Layer Normalization@NIPS+読み会・関西
Layer Normalization@NIPS+読み会・関西
Keigo Nishida
 
[DL輪読会]Vision Transformer with Deformable Attention (Deformable Attention Tra...
[DL輪読会]Vision Transformer with Deformable Attention (Deformable Attention Tra...[DL輪読会]Vision Transformer with Deformable Attention (Deformable Attention Tra...
[DL輪読会]Vision Transformer with Deformable Attention (Deformable Attention Tra...
Deep Learning JP
 
backbone としての timm 入門
backbone としての timm 入門backbone としての timm 入門
backbone としての timm 入門
Takuji Tahara
 
【DL輪読会】Efficiently Modeling Long Sequences with Structured State Spaces
【DL輪読会】Efficiently Modeling Long Sequences with Structured State Spaces【DL輪読会】Efficiently Modeling Long Sequences with Structured State Spaces
【DL輪読会】Efficiently Modeling Long Sequences with Structured State Spaces
Deep Learning JP
 
[DL輪読会]MetaFormer is Actually What You Need for Vision
[DL輪読会]MetaFormer is Actually What You Need for Vision[DL輪読会]MetaFormer is Actually What You Need for Vision
[DL輪読会]MetaFormer is Actually What You Need for Vision
Deep Learning JP
 
画像生成・生成モデル メタサーベイ
画像生成・生成モデル メタサーベイ画像生成・生成モデル メタサーベイ
画像生成・生成モデル メタサーベイ
cvpaper. challenge
 
【DL輪読会】Scaling Laws for Neural Language Models
【DL輪読会】Scaling Laws for Neural Language Models【DL輪読会】Scaling Laws for Neural Language Models
【DL輪読会】Scaling Laws for Neural Language Models
Deep Learning JP
 
深層生成モデルと世界モデル
深層生成モデルと世界モデル深層生成モデルと世界モデル
深層生成モデルと世界モデル
Masahiro Suzuki
 
【DL輪読会】Flamingo: a Visual Language Model for Few-Shot Learning 画像×言語の大規模基盤モ...
【DL輪読会】Flamingo: a Visual Language Model for Few-Shot Learning   画像×言語の大規模基盤モ...【DL輪読会】Flamingo: a Visual Language Model for Few-Shot Learning   画像×言語の大規模基盤モ...
【DL輪読会】Flamingo: a Visual Language Model for Few-Shot Learning 画像×言語の大規模基盤モ...
Deep Learning JP
 
SSII2019OS: 深層学習にかかる時間を短くしてみませんか? ~分散学習の勧め~
SSII2019OS: 深層学習にかかる時間を短くしてみませんか? ~分散学習の勧め~SSII2019OS: 深層学習にかかる時間を短くしてみませんか? ~分散学習の勧め~
SSII2019OS: 深層学習にかかる時間を短くしてみませんか? ~分散学習の勧め~
SSII
 
【メタサーベイ】基盤モデル / Foundation Models
【メタサーベイ】基盤モデル / Foundation Models【メタサーベイ】基盤モデル / Foundation Models
【メタサーベイ】基盤モデル / Foundation Models
cvpaper. challenge
 
最近のDeep Learning (NLP) 界隈におけるAttention事情
最近のDeep Learning (NLP) 界隈におけるAttention事情最近のDeep Learning (NLP) 界隈におけるAttention事情
最近のDeep Learning (NLP) 界隈におけるAttention事情
Yuta Kikuchi
 
最適輸送入門
最適輸送入門最適輸送入門
最適輸送入門
joisino
 
SSII2022 [SS1] ニューラル3D表現の最新動向〜 ニューラルネットでなんでも表せる?? 〜​
SSII2022 [SS1] ニューラル3D表現の最新動向〜 ニューラルネットでなんでも表せる?? 〜​SSII2022 [SS1] ニューラル3D表現の最新動向〜 ニューラルネットでなんでも表せる?? 〜​
SSII2022 [SS1] ニューラル3D表現の最新動向〜 ニューラルネットでなんでも表せる?? 〜​
SSII
 
モデル高速化百選
モデル高速化百選モデル高速化百選
モデル高速化百選
Yusuke Uchida
 
最適輸送の計算アルゴリズムの研究動向
最適輸送の計算アルゴリズムの研究動向最適輸送の計算アルゴリズムの研究動向
最適輸送の計算アルゴリズムの研究動向
ohken
 
Transformerを多層にする際の勾配消失問題と解決法について
Transformerを多層にする際の勾配消失問題と解決法についてTransformerを多層にする際の勾配消失問題と解決法について
Transformerを多層にする際の勾配消失問題と解決法について
Sho Takase
 
SSII2019企画: 点群深層学習の研究動向
SSII2019企画: 点群深層学習の研究動向SSII2019企画: 点群深層学習の研究動向
SSII2019企画: 点群深層学習の研究動向
SSII
 
全力解説!Transformer
全力解説!Transformer全力解説!Transformer
全力解説!Transformer
Arithmer Inc.
 

What's hot (20)

Transformerを雰囲気で理解する
Transformerを雰囲気で理解するTransformerを雰囲気で理解する
Transformerを雰囲気で理解する
 
Layer Normalization@NIPS+読み会・関西
Layer Normalization@NIPS+読み会・関西Layer Normalization@NIPS+読み会・関西
Layer Normalization@NIPS+読み会・関西
 
[DL輪読会]Vision Transformer with Deformable Attention (Deformable Attention Tra...
[DL輪読会]Vision Transformer with Deformable Attention (Deformable Attention Tra...[DL輪読会]Vision Transformer with Deformable Attention (Deformable Attention Tra...
[DL輪読会]Vision Transformer with Deformable Attention (Deformable Attention Tra...
 
backbone としての timm 入門
backbone としての timm 入門backbone としての timm 入門
backbone としての timm 入門
 
【DL輪読会】Efficiently Modeling Long Sequences with Structured State Spaces
【DL輪読会】Efficiently Modeling Long Sequences with Structured State Spaces【DL輪読会】Efficiently Modeling Long Sequences with Structured State Spaces
【DL輪読会】Efficiently Modeling Long Sequences with Structured State Spaces
 
[DL輪読会]MetaFormer is Actually What You Need for Vision
[DL輪読会]MetaFormer is Actually What You Need for Vision[DL輪読会]MetaFormer is Actually What You Need for Vision
[DL輪読会]MetaFormer is Actually What You Need for Vision
 
画像生成・生成モデル メタサーベイ
画像生成・生成モデル メタサーベイ画像生成・生成モデル メタサーベイ
画像生成・生成モデル メタサーベイ
 
【DL輪読会】Scaling Laws for Neural Language Models
【DL輪読会】Scaling Laws for Neural Language Models【DL輪読会】Scaling Laws for Neural Language Models
【DL輪読会】Scaling Laws for Neural Language Models
 
深層生成モデルと世界モデル
深層生成モデルと世界モデル深層生成モデルと世界モデル
深層生成モデルと世界モデル
 
【DL輪読会】Flamingo: a Visual Language Model for Few-Shot Learning 画像×言語の大規模基盤モ...
【DL輪読会】Flamingo: a Visual Language Model for Few-Shot Learning   画像×言語の大規模基盤モ...【DL輪読会】Flamingo: a Visual Language Model for Few-Shot Learning   画像×言語の大規模基盤モ...
【DL輪読会】Flamingo: a Visual Language Model for Few-Shot Learning 画像×言語の大規模基盤モ...
 
SSII2019OS: 深層学習にかかる時間を短くしてみませんか? ~分散学習の勧め~
SSII2019OS: 深層学習にかかる時間を短くしてみませんか? ~分散学習の勧め~SSII2019OS: 深層学習にかかる時間を短くしてみませんか? ~分散学習の勧め~
SSII2019OS: 深層学習にかかる時間を短くしてみませんか? ~分散学習の勧め~
 
【メタサーベイ】基盤モデル / Foundation Models
【メタサーベイ】基盤モデル / Foundation Models【メタサーベイ】基盤モデル / Foundation Models
【メタサーベイ】基盤モデル / Foundation Models
 
最近のDeep Learning (NLP) 界隈におけるAttention事情
最近のDeep Learning (NLP) 界隈におけるAttention事情最近のDeep Learning (NLP) 界隈におけるAttention事情
最近のDeep Learning (NLP) 界隈におけるAttention事情
 
最適輸送入門
最適輸送入門最適輸送入門
最適輸送入門
 
SSII2022 [SS1] ニューラル3D表現の最新動向〜 ニューラルネットでなんでも表せる?? 〜​
SSII2022 [SS1] ニューラル3D表現の最新動向〜 ニューラルネットでなんでも表せる?? 〜​SSII2022 [SS1] ニューラル3D表現の最新動向〜 ニューラルネットでなんでも表せる?? 〜​
SSII2022 [SS1] ニューラル3D表現の最新動向〜 ニューラルネットでなんでも表せる?? 〜​
 
モデル高速化百選
モデル高速化百選モデル高速化百選
モデル高速化百選
 
最適輸送の計算アルゴリズムの研究動向
最適輸送の計算アルゴリズムの研究動向最適輸送の計算アルゴリズムの研究動向
最適輸送の計算アルゴリズムの研究動向
 
Transformerを多層にする際の勾配消失問題と解決法について
Transformerを多層にする際の勾配消失問題と解決法についてTransformerを多層にする際の勾配消失問題と解決法について
Transformerを多層にする際の勾配消失問題と解決法について
 
SSII2019企画: 点群深層学習の研究動向
SSII2019企画: 点群深層学習の研究動向SSII2019企画: 点群深層学習の研究動向
SSII2019企画: 点群深層学習の研究動向
 
全力解説!Transformer
全力解説!Transformer全力解説!Transformer
全力解説!Transformer
 

Similar to 【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)

The influence of "Distributed platforms" on #devops
The influence of "Distributed platforms" on #devopsThe influence of "Distributed platforms" on #devops
The influence of "Distributed platforms" on #devops
Kris Buytaert
 
BERT- Pre-training of Deep Bidirectional Transformers for Language Understand...
BERT- Pre-training of Deep Bidirectional Transformers for Language Understand...BERT- Pre-training of Deep Bidirectional Transformers for Language Understand...
BERT- Pre-training of Deep Bidirectional Transformers for Language Understand...
Kyuri Kim
 
Deep learning for real life applications
Deep learning for real life applicationsDeep learning for real life applications
Deep learning for real life applications
Anas Arram, Ph.D
 
Paulking dlp
Paulking dlpPaulking dlp
Paulking dlp
d0nn9n
 
Building a Neural Machine Translation System From Scratch
Building a Neural Machine Translation System From ScratchBuilding a Neural Machine Translation System From Scratch
Building a Neural Machine Translation System From Scratch
Natasha Latysheva
 
Transformer 動向調査 in 画像認識
Transformer 動向調査 in 画像認識Transformer 動向調査 in 画像認識
Transformer 動向調査 in 画像認識
Kazuki Maeno
 
Platform-independent static binary code analysis using a meta-assembly language
Platform-independent static binary code analysis using a meta-assembly languagePlatform-independent static binary code analysis using a meta-assembly language
Platform-independent static binary code analysis using a meta-assembly language
zynamics GmbH
 
The Joy of SciPy
The Joy of SciPyThe Joy of SciPy
The Joy of SciPy
kammeyer
 
Learning to Translate with Joey NMT
Learning to Translate with Joey NMTLearning to Translate with Joey NMT
Learning to Translate with Joey NMT
Julia Kreutzer
 
Foundation Models in Recommender Systems
Foundation Models in Recommender SystemsFoundation Models in Recommender Systems
Foundation Models in Recommender Systems
Anoop Deoras
 
Dmk audioviz
Dmk audiovizDmk audioviz
Dmk audioviz
Dan Kaminsky
 
Keras: A versatile modeling layer for deep learning
Keras: A versatile modeling layer for deep learningKeras: A versatile modeling layer for deep learning
Keras: A versatile modeling layer for deep learning
Dr. Ananth Krishnamoorthy
 
Smalltalk Debug Lives in the Matrix
Smalltalk Debug Lives in the MatrixSmalltalk Debug Lives in the Matrix
Smalltalk Debug Lives in the Matrix
ESUG
 
Deep Learning for Machine Translation
Deep Learning for Machine TranslationDeep Learning for Machine Translation
Deep Learning for Machine Translation
Matīss ‎‎‎‎‎‎‎  
 
Moby is killing your devops efforts
Moby is killing your devops effortsMoby is killing your devops efforts
Moby is killing your devops efforts
Kris Buytaert
 
Dynamic Language Practices
Dynamic Language PracticesDynamic Language Practices
Dynamic Language Practices
Paul King
 
Challenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in JavaChallenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in Java
lucenerevolution
 
VAEs for multimodal disentanglement
VAEs for multimodal disentanglementVAEs for multimodal disentanglement
VAEs for multimodal disentanglement
Antonio Tejero de Pablos
 
Deep Learning in NLP (BERT, ERNIE and REFORMER)
Deep Learning in NLP (BERT, ERNIE and REFORMER)Deep Learning in NLP (BERT, ERNIE and REFORMER)
Deep Learning in NLP (BERT, ERNIE and REFORMER)
Biswajit Biswas
 
Deep Learning & NLP: Graphs to the Rescue!
Deep Learning & NLP: Graphs to the Rescue!Deep Learning & NLP: Graphs to the Rescue!
Deep Learning & NLP: Graphs to the Rescue!
Roelof Pieters
 

Similar to 【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc) (20)

The influence of "Distributed platforms" on #devops
The influence of "Distributed platforms" on #devopsThe influence of "Distributed platforms" on #devops
The influence of "Distributed platforms" on #devops
 
BERT- Pre-training of Deep Bidirectional Transformers for Language Understand...
BERT- Pre-training of Deep Bidirectional Transformers for Language Understand...BERT- Pre-training of Deep Bidirectional Transformers for Language Understand...
BERT- Pre-training of Deep Bidirectional Transformers for Language Understand...
 
Deep learning for real life applications
Deep learning for real life applicationsDeep learning for real life applications
Deep learning for real life applications
 
Paulking dlp
Paulking dlpPaulking dlp
Paulking dlp
 
Building a Neural Machine Translation System From Scratch
Building a Neural Machine Translation System From ScratchBuilding a Neural Machine Translation System From Scratch
Building a Neural Machine Translation System From Scratch
 
Transformer 動向調査 in 画像認識
Transformer 動向調査 in 画像認識Transformer 動向調査 in 画像認識
Transformer 動向調査 in 画像認識
 
Platform-independent static binary code analysis using a meta-assembly language
Platform-independent static binary code analysis using a meta-assembly languagePlatform-independent static binary code analysis using a meta-assembly language
Platform-independent static binary code analysis using a meta-assembly language
 
The Joy of SciPy
The Joy of SciPyThe Joy of SciPy
The Joy of SciPy
 
Learning to Translate with Joey NMT
Learning to Translate with Joey NMTLearning to Translate with Joey NMT
Learning to Translate with Joey NMT
 
Foundation Models in Recommender Systems
Foundation Models in Recommender SystemsFoundation Models in Recommender Systems
Foundation Models in Recommender Systems
 
Dmk audioviz
Dmk audiovizDmk audioviz
Dmk audioviz
 
Keras: A versatile modeling layer for deep learning
Keras: A versatile modeling layer for deep learningKeras: A versatile modeling layer for deep learning
Keras: A versatile modeling layer for deep learning
 
Smalltalk Debug Lives in the Matrix
Smalltalk Debug Lives in the MatrixSmalltalk Debug Lives in the Matrix
Smalltalk Debug Lives in the Matrix
 
Deep Learning for Machine Translation
Deep Learning for Machine TranslationDeep Learning for Machine Translation
Deep Learning for Machine Translation
 
Moby is killing your devops efforts
Moby is killing your devops effortsMoby is killing your devops efforts
Moby is killing your devops efforts
 
Dynamic Language Practices
Dynamic Language PracticesDynamic Language Practices
Dynamic Language Practices
 
Challenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in JavaChallenges in Maintaining a High Performance Search Engine Written in Java
Challenges in Maintaining a High Performance Search Engine Written in Java
 
VAEs for multimodal disentanglement
VAEs for multimodal disentanglementVAEs for multimodal disentanglement
VAEs for multimodal disentanglement
 
Deep Learning in NLP (BERT, ERNIE and REFORMER)
Deep Learning in NLP (BERT, ERNIE and REFORMER)Deep Learning in NLP (BERT, ERNIE and REFORMER)
Deep Learning in NLP (BERT, ERNIE and REFORMER)
 
Deep Learning & NLP: Graphs to the Rescue!
Deep Learning & NLP: Graphs to the Rescue!Deep Learning & NLP: Graphs to the Rescue!
Deep Learning & NLP: Graphs to the Rescue!
 

More from Deep Learning JP

【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
Deep Learning JP
 
【DL輪読会】事前学習用データセットについて
【DL輪読会】事前学習用データセットについて【DL輪読会】事前学習用データセットについて
【DL輪読会】事前学習用データセットについて
Deep Learning JP
 
【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...
【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...
【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...
Deep Learning JP
 
【DL輪読会】Zero-Shot Dual-Lens Super-Resolution
【DL輪読会】Zero-Shot Dual-Lens Super-Resolution【DL輪読会】Zero-Shot Dual-Lens Super-Resolution
【DL輪読会】Zero-Shot Dual-Lens Super-Resolution
Deep Learning JP
 
【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv
【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv
【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv
Deep Learning JP
 
【DL輪読会】マルチモーダル LLM
【DL輪読会】マルチモーダル LLM【DL輪読会】マルチモーダル LLM
【DL輪読会】マルチモーダル LLM
Deep Learning JP
 
【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...
 【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo... 【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...
【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...
Deep Learning JP
 
【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition
【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition
【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition
Deep Learning JP
 
【DL輪読会】Can Neural Network Memorization Be Localized?
【DL輪読会】Can Neural Network Memorization Be Localized?【DL輪読会】Can Neural Network Memorization Be Localized?
【DL輪読会】Can Neural Network Memorization Be Localized?
Deep Learning JP
 
【DL輪読会】Hopfield network 関連研究について
【DL輪読会】Hopfield network 関連研究について【DL輪読会】Hopfield network 関連研究について
【DL輪読会】Hopfield network 関連研究について
Deep Learning JP
 
【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )
【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )
【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )
Deep Learning JP
 
【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...
【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...
【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...
Deep Learning JP
 
【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"
【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"
【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"
Deep Learning JP
 
【DL輪読会】"Language Instructed Reinforcement Learning for Human-AI Coordination "
【DL輪読会】"Language Instructed Reinforcement Learning  for Human-AI Coordination "【DL輪読会】"Language Instructed Reinforcement Learning  for Human-AI Coordination "
【DL輪読会】"Language Instructed Reinforcement Learning for Human-AI Coordination "
Deep Learning JP
 
【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models
【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models
【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models
Deep Learning JP
 
【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"
【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"
【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"
Deep Learning JP
 
【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...
【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...
【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...
Deep Learning JP
 
【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...
【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...
【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...
Deep Learning JP
 
【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...
【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...
【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...
Deep Learning JP
 
【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...
【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...
【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...
Deep Learning JP
 

More from Deep Learning JP (20)

【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
【DL輪読会】AdaptDiffuser: Diffusion Models as Adaptive Self-evolving Planners
 
【DL輪読会】事前学習用データセットについて
【DL輪読会】事前学習用データセットについて【DL輪読会】事前学習用データセットについて
【DL輪読会】事前学習用データセットについて
 
【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...
【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...
【DL輪読会】 "Learning to render novel views from wide-baseline stereo pairs." CVP...
 
【DL輪読会】Zero-Shot Dual-Lens Super-Resolution
【DL輪読会】Zero-Shot Dual-Lens Super-Resolution【DL輪読会】Zero-Shot Dual-Lens Super-Resolution
【DL輪読会】Zero-Shot Dual-Lens Super-Resolution
 
【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv
【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv
【DL輪読会】BloombergGPT: A Large Language Model for Finance arxiv
 
【DL輪読会】マルチモーダル LLM
【DL輪読会】マルチモーダル LLM【DL輪読会】マルチモーダル LLM
【DL輪読会】マルチモーダル LLM
 
【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...
 【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo... 【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...
【 DL輪読会】ToolLLM: Facilitating Large Language Models to Master 16000+ Real-wo...
 
【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition
【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition
【DL輪読会】AnyLoc: Towards Universal Visual Place Recognition
 
【DL輪読会】Can Neural Network Memorization Be Localized?
【DL輪読会】Can Neural Network Memorization Be Localized?【DL輪読会】Can Neural Network Memorization Be Localized?
【DL輪読会】Can Neural Network Memorization Be Localized?
 
【DL輪読会】Hopfield network 関連研究について
【DL輪読会】Hopfield network 関連研究について【DL輪読会】Hopfield network 関連研究について
【DL輪読会】Hopfield network 関連研究について
 
【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )
【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )
【DL輪読会】SimPer: Simple self-supervised learning of periodic targets( ICLR 2023 )
 
【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...
【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...
【DL輪読会】RLCD: Reinforcement Learning from Contrast Distillation for Language M...
 
【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"
【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"
【DL輪読会】"Secrets of RLHF in Large Language Models Part I: PPO"
 
【DL輪読会】"Language Instructed Reinforcement Learning for Human-AI Coordination "
【DL輪読会】"Language Instructed Reinforcement Learning  for Human-AI Coordination "【DL輪読会】"Language Instructed Reinforcement Learning  for Human-AI Coordination "
【DL輪読会】"Language Instructed Reinforcement Learning for Human-AI Coordination "
 
【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models
【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models
【DL輪読会】Llama 2: Open Foundation and Fine-Tuned Chat Models
 
【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"
【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"
【DL輪読会】"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware"
 
【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...
【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...
【DL輪読会】Parameter is Not All You Need:Starting from Non-Parametric Networks fo...
 
【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...
【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...
【DL輪読会】Drag Your GAN: Interactive Point-based Manipulation on the Generative ...
 
【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...
【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...
【DL輪読会】Self-Supervised Learning from Images with a Joint-Embedding Predictive...
 
【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...
【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...
【DL輪読会】Towards Understanding Ensemble, Knowledge Distillation and Self-Distil...
 

Recently uploaded

20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website
Pixlogix Infotech
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
mikeeftimakis1
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Neo4j
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
danishmna97
 
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIEnchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Vladimir Iglovikov, Ph.D.
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
Ana-Maria Mihalceanu
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Zilliz
 

Recently uploaded (20)

20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website20 Comprehensive Checklist of Designing and Developing a Website
20 Comprehensive Checklist of Designing and Developing a Website
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
Introduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - CybersecurityIntroduction to CHERI technology - Cybersecurity
Introduction to CHERI technology - Cybersecurity
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
How to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptxHow to Get CNIC Information System with Paksim Ga.pptx
How to Get CNIC Information System with Paksim Ga.pptx
 
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIEnchancing adoption of Open Source Libraries. A case study on Albumentations.AI
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AI
 
Monitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR EventsMonitoring Java Application Security with JDK Tools and JFR Events
Monitoring Java Application Security with JDK Tools and JFR Events
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...Building RAG with self-deployed Milvus vector database and Snowpark Container...
Building RAG with self-deployed Milvus vector database and Snowpark Container...
 

【DL輪読会】言語以外でのTransformerのまとめ (ViT, Perceiver, Frozen Pretrained Transformer etc)

Editor's Notes

  1. Beyond Reward Based End-to-End RL: Representation Learning and Dataset Optimization Perspective