Matsuo Lab at the University of Tokyo specializes in artificial intelligence and web engineering research. It has over 180 members including researchers, staff, students, and teaching assistants. The lab conducts fundamental research on deep learning and world models. It also provides education programs in areas like data science, deep learning, and entrepreneurship that have had over 7,500 attendees. Additionally, the lab helps nurture startups from its research and provides entrepreneurship education.

1. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO
Introduction to Matsuo Lab
December 2022
Photography, video
recording and disclosure
to third parties without
permissions are strictly
prohibited.

2. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO
1. About Us

3. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 3
Matsuo Lab, Graduate School of Engineering, The University of Tokyo
• Matsuo Lab belongs to the Graduate School of Engineering at The University of Tokyo and
specialize in Artificial Intelligence (AI) and Web Engineering research
• As a team-oriented organization, the lab aims for its members and the organization to
generate world-leading and high-level results
• Comprised of 100* members between staff and students
Researchers
(about 10
people)
Staff
(About 30
people)
Assigned
students
(About 40
people)
TAs
(About 20
people)
Prof. Matsuo
Board members
Masters and doctoral courses: Technology Management for
Innovation, Graduate School of Engineering
Department: Systems Innovation Program, Faculty of
Engineering
Matsuo Lab startups
And more
＋
14 PhDs.,
43 Master’s
31 Bachelor’s
88 graduates
(As of July 2021)
Areas of specialization
✓ AI
R&D on cutting-edge technology
(Deep Learning)
Application to image recognition
and robotics
✓ Web engineering
Analysis of social media, big data,
etc.
Design and operation of web
services
*Approximately 180 members when engineer interns are included
(As of August 2022)

4. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 4
Representative Officer
Prof. Yutaka Matsuo
1997 Graduated from the Faculty of Engineering at The University of Tokyo with
a Bachelor’s degree in Information and Communication Engineering
2002 Completed a doctoral program and earned a doctorate in engineering at
the Graduate School of Engineering at The University of Tokyo
Became a researcher with the National Institute of Advanced Industrial
Science and Technology (AIST)
From Oct. 2005. Visiting Scholar, Stanford University
From Oct. 2007 Associate Professor, Institute of Engineering Innovation / Center for
Structuring of Knowledge / Department of Technology Management for
Innovation, Graduate School of Engineering, The University of Tokyo
From 2014. Joint Representative and Project Associate Professor, Chair for Global
Consumer Intelligence, Department of Technology Management for
Innovation, Graduate School of Engineering, The University of Tokyo
2012–14 Editor-in-Chief, Transactions of the Japanese Society for Artificial
Intelligence, then Chair of Ethics Committee (present post)
June 2017 Founder and Director, Japan Deep Learning Association (JDLA)
From April 2019 Professor of Research Into Artifacts, Center for Engineering (RACE) /
Department of Technology Management for Innovation, Graduate School
of Engineering, The University of Tokyo
From June 2019 Concurrent Outside Director, SoftBank Group Corp.
From Oct. 2021 Member, Council of New Form of Capitalism Realization
Japan Deep Learning Association

5. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 5
Prof. Matsuo’s Curriculum Vitae
• Prof. Matsuo’s desire to “solve the mystery of intelligence” and his awareness of the issues he
faced during his time as a vising researcher at Stanford University became the basis for the
current organizational management
Completed
doctoral course
Joined AIST
Professor,
The University
of Tokyo
(current post)
2002–
Visiting scholar,
Stanford University
2005–2007
Associate
Professor,
The University
of Tokyo
2007–
✓ Prof. Matsuo observed up-
close the rise of Google and
Facebook in Silicon Valley
✓ Google and Facebook became
the world leaders in academic
research
✓ Realization that “A world
where academia and
industry collaborate to
create value is coming” and
“In that world, Japan
cannot compete”
✓ Prof. Matsuo took a number of actions
(launching his own startup company,
etc.)
✓ To nurture the next generation, Prof.
Matsuo decided to stop receiving
research funds from the government
✓ Prof. Matsuo made every effort to keep
managing laboratory while engaging in
joint research with multiple companies
✓ In addition, he made continuous effort to
foster startup companies.
✓ Prof. Matsuo Conducted
research on Web Engineering
and AI, driven by his
intellectual curiosity to
“solve the mysteries of
intelligence.”

6. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 6
Vision
Changing the world by nurturing pioneers:
Matsuo Lab create an ecosystem that foster “chains reactions of change.”
Matsuo Lab is a laboratory that nurture pioneers.
We continue to advance world-leading research and the generated results contribute to the accumulation of knowledge.
By enhancing the fundamental research, we believe that the quality and quantity of education would increase.
by providing valuable learning opportunity to more students.
Those students who studied in the laboratory would play an active role in joint researches with various companies or
start their own ventures. They become “pioneers” who lead into the future and foster innovation.
Success’ experience and knowledge flows back to research and education.
A change generates another, driving further change to people and society. Changes to the world get faster and more
dramatic, with increased momentum.
Matsuo Lab strives to build an ecosystem where such a “chain reaction of change” is started by nurturing “pioneers” and
creating an ecosystem that will change the world.

7. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 7
Building an Ecosystem
• Matsuo lab is aiming to build an ecosystem where the results of research are not kept within academia,
but widely shared in the form of startups and services, so that the benefits of those economic activities
recirculate and promote further research, in The University of Tokyo and the Hongo area.
Joint research
(Implementation)
Fundamental research
Advanced education
Startups
(eventually becoming
big companies)
Expertise, resources, etc.
for success to be returned
to the academia.
Return the know-how and
resources for success to the
university
Nurturing of
technological
“seeds” and let
them in society
Provide advanced education base on
fundamental research
Provide practical learning
opportunities through classroom
lectures, OJT and participation in
joint researches
Create a spiral of innovation

8. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO
2. Projects

9. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 9
Overview
3.
Social Impact
1.
Fundamental
Research
2.
Education
4.
Incubation
R&D focused on Deep Learning (World
Models) aiming to achieve smarter machines
and explain the principles of knowledge
In collaboration with private companies,
promotion of DX in industry through joint
researches on Deep Learning
Development and provision
of human resource
development programs for
students and adults (not
limited to students and
faculty members of The
University of Tokyo)
Nurture the creation
of startups from
universities and
laboratories, and
provide
entrepreneurship
education

10. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 10
Overview
3.
Social Impact
1.
Fundamental
Research
2.
Education
4.
Incubation
R&D focused on Deep Learning (World
Models) aiming to achieve smarter machines
and explain the principles of knowledge
In collaboration with private companies,
promotion of DX in industry through joint
researches on Deep Learning
Development and provision
of human resource
development programs for
students and adults (not
limited to students and
faculty members of The
University of Tokyo)
Nurture the creation
of startups from
universities and
laboratories, and
provide
entrepreneurship
education

11. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 11
Fundamental Research | Team Mission
Matsuo Lab is conducting cutting-edge R&D
in the field of Deep learning, especially
World Models, with the ambition of
“realizing intelligence in engineering”

12. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 12
Fundamental Research | World Models
• The key to future Deep Learning development is technology to simulate the real world, the machine equivalent of
human imagination.
• The world’s leading AI companies and are already conducting ongoing research on this topic.
Sources: https://deepmind.com/blog/article/neural-scene-representation-and-rendering, https://worldmodels.github.io
Technology Overview Examples
Humans can use imagination to compensate for gaps in information
and to posit future conditions.
Ex. Imagining future from the
current state
Similarly, the key to the future development of AI is to efficiently learn
“common sense” from experiences, and to be able to “imagine” the
future. The core technology is World Models.
CRASH!!
Ex. Looking at a part of an
object and imagining the whole
picture
Leading companies and laboratories like Google focus on World Models
research
Example from DeepMind (Google): Reconstructing an entire
object or group of objects from a series of limited views
The object of this game is to avoid the bullets. The
system’s ability to avoid being hit is improved by
incorporating an efficient mechanism to imagine
the future
“Neural scene representation and rendering”, S. A. Eslami, et al., Science,
360(6394):1204–1210, 2018.
Based on images from
three different
perspectives, AI
reconstructs the 3D view.
Example from Google Brain: Efficient prediction of future events
“Recurrent world models facilitate policy evolution”. D. Ha, J. Schmidhuber,
NeurIPS 2018, pp. 2455–2467, 2018.

13. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 13
Fundamental Research | Results and Case Studies (Theses)
• Matsuo Lab identify trends in papers in advanced areas such as large-scale models, NLP or multimodal technologies,
with a focus on World Models.
• In AY 2021-22, a total of 8 Matsuo Lab papers were selected for presentation at major conferences, including
NeurIPS2021 (spotlight) and ICLR2022 (spotlight).
Matsuo Lab has assembled a team in response to the increase on World Models research
Theses adopted for presentation (partial list) *Please refer here for the latest information
✓ “PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretraining”. The 2022 Conference of the North American
Chapter of the Association for Computational Linguistics (NAACL 2022)
✓ Robustifying Vision Transformer without Retraining from Scratch by Test-Time Class-Conditional Feature Alignment”, the 31st
International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence (IJCAI-ECAI 2022)
✓ “Generalized Decision Transformer for Offline Hindsight Information Matching”, International Conference on Learning Representations
(ICLR2022 , Spotlight)
✓ “Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization”, Advances in Neural Information Processing Systems
2021 (NeurIPS2021, Spotlight)
✓ “Co-Adaptation of Algorithmic and Implementational Innovations in Inference-based Deep Reinforcement Learning”, Advances in Neural
Information Processing Systems 2021 (NeurIPS2021)
✓ Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learning”, International
Conference on Machine Learning 2021 (ICML2021)
…and
others

14. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 14
Fundamental Research | Results and Case Studies (Robotics)
• Matsuo Lab aims to create positive impact by combining World Models with robotics.
• In AY 2021-22, the fundamental research team worked on a cleanup robot (runner up at WRS 2021) and
creating a test bed for soft objects manipulation, among others.

15. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 15
Overview
3.
Social Impact
1.
Fundamental
Research
2.
Education
4.
Incubation
R&D focused on Deep Learning (World
Models) aiming to achieve smarter machines
and explain the principles of knowledge
In collaboration with private companies,
promotion of DX in industry through joint
researches on Deep Learning
Development and provision
of human resource
development programs for
students and adults (not
limited to students and
faculty members of The
University of Tokyo)
Nurture the creation of
startups from
universities and
laboratories, and
provide
entrepreneurship
education

16. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 16
Education | Team Mission
We bring the knowledge gained from
fundamental research back to the society
through lectures.
By providing opportunity for as many people
as possible to learn from cutting-edge
knowledge and promoting social
implementation, we believe better future can
be created.

17. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 17
Education | Overview of Courses
Web Engineering and
Business Models
Web工学
Fundamental Projects
Web Engineering
Chair for Global Consumer
Intelligence
(Cultivation of data scientists)
Data Science
Business Models
Web工学
Deep Learning
Basic courses
Deep Learning
Web工学
Entrepreneurship
• In AY 2022-23, Matsuo lab plans to offer classes in 14 programs, covering a wide array of topics from fundamental technology to social
implementation.
• Examples
• “Web engineering” provide lecture on a fundamental technology for user experience design
• ”Data Science” and “AI Management” focus data-science technology and its applications in business and management
• “Deep Learning Foundations” and “World Models” covering topics from the basis of Deep Learning to cutting-edge knowledge
• “Entrepreneurship” offers ideas on starting a technology-based company
Intensive courses
Spring seminar
(Image recognition)
Summer school
（Financial trading and machine learning,
Deep Generative Model,
Natural Language Processing)
Entrepreneurship
Education Design Donation
Course
Invitation to Deep Tech
Entrepreneurs
Visionary startups
Chair for AI Business
Transformation
Chair for World Models Seminar on data-driven
entrepreneurship
Seminar on data-driven
business proposition

18. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 18
Education | Results (Attendance)
• From April 2014 to February 2021, a total of over 7,500 students and working adults
attended the courses.
• In AY 2022-23 alone, attendance is expected to surpass 4,000, raising cumulative
attendance to over 10,000.
0
500
1000
1500
2000
2500
2014 2015 2016 2017 2018 2019 2020 2021
To
approx.
4,000
3,200
2022

19. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 19
Education | (Reference) Metaverse School of Engineering
• In September 2022, The University of Tokyo’s Graduate School of Engineering, Faculty of Engineering took the lead in
establishing and opening Metaverse School of Engineering
• This Department uses digital technology to provide a forum for education in the engineering field. Its aim is to realize a
society in which everyone, from junior-high and high-school students to working adults, can acquire the latest information
and practical engineering skills to realize their dreams.
Note: For details, please visit About the Metaverse School of Engineering (https://www.t.u-tokyo.ac.jp/meta-school)
(Metaverse School of Engineering Executive Office).
Programs
1. Junior Engineering Educational Program
✓ Intended mainly for: junior-high and high-school students, parents and
teachers
✓ Partners with industry and universities to convey the excitement of
engineering and information science to a young audience
✓ Includes classes introducing engineering studies at universities and careers
available after graduation, hands-on learning in product development and
other topics, lab tours, etc.
2. Re-Skilling Engineering Educational Program
✓ Intended mainly for: Working adults and students
✓ Online program covering the latest engineering topics and information,
including AI, entrepreneurship, next-generation communications, etc., for
working adults and students who want to brush-up and re-skill their area of
specialty
Note: In addition to the above initiatives, the Metaverse School of Engineering is
launching a general information website about careers in engineering and
providing career information tailored to female engineers, for whom role models
in the field are still few and aiming to accelerate the promotion of diversity in the
engineering field.

20. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 20
Overview
3.
Social Impact
1.
Fundamental
Research
2.
Education
4.
Incubation
R&D focused on Deep Learning (World
Models) aiming to achieve smarter machines
and explain the principles of knowledge
In collaboration with private companies,
promotion of DX in industry through joint
researches on Deep Learning
Development and provision
of human resource
development programs for
students and adults (not
limited to students and
faculty members of The
University of Tokyo)
Nurture the creation
of startups from
universities and
laboratories, and
provide
entrepreneurship
education

21. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 21
Social Impact | Team Mission
In collaboration with private companies,
Matsuo lab conduct Deep Learning
researches and generate academic results

22. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 22
Social Impact | Results and Case Studies
• Matsuo Lab is pursuing the application of research in a wide variety of topics, industries or types of
data, such as self-driving cars, object detection, image processing, behavior analysis or forecasting
Theme Industry Project Overview
Image
analysis
Healthcare
Development of image diagnosis algorithms to assist in the
diagnosis of major dementias such as Alzheimer’s disease, by
detecting micro-hemorrhages in MRI scans
Behavior
analysis
Manufacturing
(Parts)
Detection and visualization of people’s movement in the factory
to analyze the cause of defective product rates and devise the
transfer of skilled workers’ expertise
Forecasting
Manufacturing
(Chemicals)
Early detection of abnormalities in chemical plants and
identification of their causes
AY 2021 result excerpts
(Photos are for illustrative
purposes only.)

23. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 23
Overview
3.
Social Impact
1.
Fundamental
Research
2.
Education
4.
Incubation
R&D focused on Deep Learning (World Models)
aiming to achieve smarter machines and
explain the principles of knowledge
In collaboration with private companies,
promotion of DX in industry through joint
researches on Deep Learning
Development and provision
of human resource
development programs for
students and adults (not
limited to students and
faculty members of The
University of Tokyo)
Nurture the creation
of startups from
universities and
laboratories, and
provide
entrepreneurship
education

24. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 24
Incubation | Team Mission
Matsuo lab nurture and support
entrepreneurs with the goal of producing
100 startups a year from the lab to
promote DX in various industries, to return
cutting-edge technology to society and
create a new industrial ecosystem.

25. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 25
Incubation | Startups Launched by Matsuo Lab
• Since 2012, 13 start-ups (including 2 listed companies) have been launched by the graduates of Matsu Lab
• In the AY 2021-22, 4 companies will be launched by students who completed the Kigyo Quest.
* Please refer to the next page.
Listed
Startups launched in the the 2021–22 academic year

26. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 26
Incubation | Kigyo Quest
• Kigyo Quest is a program that abstracts the success model of startups launched at Matsuo Lab and condenses the
know-how to increase the probability of success
• Launched in the summer of 2021, by April 2022, there are 3 companies registered and currently preparing to launch
1st Stage Getting Your Weapons (Education)
✓ Attend certified Kigyo Quest classes to learn the basics of your technology (data science/deep learning).
2nd Stage Getting Stronger Through Practice (Application)
✓ Join a company as a certified AI engineering intern and gain experience in development, project
management and pitching proposals to clients as on-the-job training.
✓ Practice to hone the skills and business strengths that will serve you well.
3rd Stage Forming Your Party, Starting Your Journey
✓ Find partners and set foot into the real business world.
✓ Gain experience in making proposals to clients and acquire the
knowledge and skills necessary to start up a company
New entrants
Veteran engineer
Entrepreneur in the making
START

27. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO
3. Organization

28. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 28
Organizational Chart
Fundamental Research,
Development and Information
Systems
✓ Responsible for fundamental R&D in deep
learning and world models
✓ Responsible for IT system management,
including high-performance computing in
support of R&D
Education
✓ Responsible for curriculum
development and management of
machine learning, Deep Learning,
Web Engineering, and other courses
Social implementation
✓ Responsible for proposals for joint
research projects to companies and
their management
Think Tank
✓ Under the direct supervision of Prof.
Matsuo, drafts policies and proposals
to companies based on knowledge
gained from the laboratory activities
Incubation
✓ Nurtures and supports student
entrepreneurs
✓ Plans and operates
entrepreneurship education
Back Office
✓ Acts as an intermediary between the
laboratory and university administration.
✓ Responsible for back-office functions such
as accounting, personnel and general
affairs.

29. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO
4. Working Environment

30. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 30
Basic Information
✓ Comprehensive social insurance
(unemployment, workers’
compensation, healthcare, social
services, pension)
✓ Commuting allowance, etc.
✓ Five-day work week (Saturday
and Sunday off)
✓ Golden Week, summer leave,
year-end/New Year leave,
annual paid leave,
bereavement leave,
maternity/paternity leave,
childcare leave, etc.
✓ Remote work possible
(during the COVID-19
pandemic)
✓ Annual renewal of employment
contracts based on the University’s
system
✓ Renewal of employment contracts
based on results, work attitude,
etc.
✓ Access to the University
library and thesis archives,
comprehensive research
equipment including
computing environment
✓ Supply of PCs according to
custom specifications
✓ Purchase of additional
equipment like robot arms,
etc.
✓ Unlimited purchase of
books and publications
✓ Opportunities for travel
to overseas for
seminars and training,
conferences, etc.
University of Tokyo Employment Regulations for Designated, Fixed-Term Faculty and Staff
University of Tokyo Employment Regulations for Designated, Fixed-Term Faculty and Staff on Reduced Hours

31. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO 31
Seminars
• A wide range of self-study seminars are offered, mainly for those who have completed lectures provided
by Matsuo Lab.
• Provide an environment where both members and non-members of Matsuo Lab can inspire and improve
each other’s understanding of the topics of the discussion
Examples of seminars
1. Deep Learning reading group
Weekly reading sessions of papers related to cutting-edge research
trends in Deep Learning
2. Deep-Learning hacks
Weekly sessions for sharing tips on the implementations of the latest
papers, troubleshooting, etc. This seminar focus on improving Deep
Learning implementation skills
3. TRAIL (robotics group)
The Tokyo Robot And Intelligence Lab (TRAIL) conducts R&D activities
centered on robot learning, with the aim of realizing intelligence in the
real world. The team also participate in robot contests.
4. Reading-in-turn sessions on finance
Reading sessions on books and publications on the intersection of finance
and Deep Learning.
Activities other than the above include occasional lectures by invited guest
researchers, etc.

32. ©︎MATSUO LAB, THE UNIVERSITY OF TOKYO