Reinforcement Learning Tutorial for Minecraft

•

0 likes•306 views

- 프로젝트명 : 광산캐기 - 발표 제목 : Reinforcement Learning Tutorial for Minecraft - 발표자: 이영무 - 네이버 - 내용 요약 : 강화학습을 이용해서 마인크래프트를 학습시켜보면서 마주쳤던 문제들을 공유하고 조금 더 쉽게 마인크래프트를 할 수 있도록 만들어진 환경을 소개합니다. 이런 환경으로 학습해 본 경험을 공유하여 마인크래프트를 학습하는 환경을 쉽게 접하는데 도움이 되었으면 합니다.

Engineering

목차
1. 팀원 소개
2. Malmo
a. Introduction
b. launchClient
c. Missions
d. Examples
3. 서버에서 말모 실행
4. Malmo 단점
5. MarLo 소개
6. MarLo Tutorial

팀원 소개
● 이영무
○ 네이버 인턴중
○ Email : zzing0907@gmail.com
● 양기창
○ 숭실대학교 재학중
○ Email : ykcha9@gmail.com

Malmo: Introduction
링크 : https://github.com/Microsoft/malmo

Malmo: Launch Client
링크 → https://github.com/Microsoft/malmo/blob/master/doc/build_linux.md

Malmo: Launch Client
설치 가이드 → https://github.com/reinforcement-learning-kr/minecraft_rl_tutorial/wiki

Malmo: Launch Client
Dependency 설치 후에
launchClient.sh 실행시키면
client가 실행이 된다.

Malmo: Missions
● 미션파일을 수정 가능
○ 커스텀 미션
○ 새로운 태스크
● 수행할 수 있는 태스크가 많지는 않음
○ 어떤 블록을 터치
○ 어떤 아이템을 획득
● 리워드를 주는 방식이 커스터마이징이 어
려움

Malmo: Sample Missions
총 11개의 샘플 미션이 있음(xml 파일)
추가적으로 Example에 몇가지 미션이
더 있음

Malmo: Examples
● 많은 예제들이 있음
● 대부분이 룰 기반으로 환경을 품
● 딥러닝 예제는 들어있지 않음
● 강화학습 예제는 tabular-q-learning 하나 뿐임
● 그래도 있을 건 다 있다!
(ex. 환경의 속도를 변화, dynamic한 환경 구성 등)

서버에서 말모 실행
● 도커를 이용해서
서버에서 실행할 수 있음
● Dockerfile 제공
(https://github.com/reinforce
ment-learning-
kr/minecraft_rl_tutorial/blob/
master/docker/Dockerfile)
● Docker guide 제공

Malmo의 단점
코드가 복잡하다!
설정할 것들이 많음
강화학습 코드에 집중
X

Malmo의 단점
python코드에서 커맨드를 보내는 사
이에도 환경은 계속해서 돌아감
모델이 inference하는 속도가 느리다
면
학습에 크게 영향을 미침

Malmo의 단점
● 환경이 불안정한 경우가 많다.
○ 학습을 하다가 연결이 종료되는 경우 (윈도우)
○ 에피소드 종료 후 이미 client가 연결되어 있다면서 실행되지 않는 경우
● 컴퓨터의 성능에 따라 결과가 달라짐
○ 렌더링 속도에 따라 결과가 달라짐
○ 서버에서는 점수가 잘 나오지만 로컬에서 점수가 안나오는 경우도 있음
● 보기 편한 Document가 없다.

MarLo
Crowd AI 에서 Malmo를 이용한
challenge 개최
멀티 에이전트를 위한 challenge

MarLo Challenge
MarLo 2018 challenge
Coming Soon !

MarLo Code
깃헙이 있으나 마스터 브랜치는
아직 개발중인..
(에러 발생)
install 브랜치 클론해서 설치하
면
사용 가능한 버전을 받을 수 있
음

MarLo Code
코드가 훨씬 간단해졌다!
OpenAI gym 프래임워크를 이용해서 코
드가 간결해짐

Featured

Product Design Trends in 2024 | Teenage Engineerings

Pixeldarts

Mental health has been in the news quite a bit lately. Dozens of U.S. states are currently suing Meta for contributing to the youth mental health crisis by inserting addictive features into their products, while the U.S. Surgeon General is touring the nation to bring awareness to the growing epidemic of loneliness and isolation. The country has endured periods of low national morale, such as in the 1970s when high inflation and the energy crisis worsened public sentiment following the Vietnam War. The current mood, however, feels different. Gallup recently reported that national mental health is at an all-time low, with few bright spots to lift spirits. To better understand how Americans are feeling and their attitudes towards mental health in general, ThinkNow conducted a nationally representative quantitative survey of 1,500 respondents and found some interesting differences among ethnic, age and gender groups. Technology For example, 52% agree that technology and social media have a negative impact on mental health, but when broken out by race, 61% of Whites felt technology had a negative effect, and only 48% of Hispanics thought it did. While technology has helped us keep in touch with friends and family in faraway places, it appears to have degraded our ability to connect in person. Staying connected online is a double-edged sword since the same news feed that brings us pictures of the grandkids and fluffy kittens also feeds us news about the wars in Israel and Ukraine, the dysfunction in Washington, the latest mass shooting and the climate crisis. Hispanics may have a built-in defense against the isolation technology breeds, owing to their large, multigenerational households, strong social support systems, and tendency to use social media to stay connected with relatives abroad. Age and Gender When asked how individuals rate their mental health, men rate it higher than women by 11 percentage points, and Baby Boomers rank it highest at 83%, saying it’s good or excellent vs. 57% of Gen Z saying the same. Gen Z spends the most amount of time on social media, so the notion that social media negatively affects mental health appears to be correlated. Unfortunately, Gen Z is also the generation that’s least comfortable discussing mental health concerns with healthcare professionals. Only 40% of them state they’re comfortable discussing their issues with a professional compared to 60% of Millennials and 65% of Boomers. Race Affects Attitudes As seen in previous research conducted by ThinkNow, Asian Americans lag other groups when it comes to awareness of mental health issues. Twenty-four percent of Asian Americans believe that having a mental health issue is a sign of weakness compared to the 16% average for all groups. Asians are also considerably less likely to be aware of mental health services in their communities (42% vs. 55%) and most likely to seek out information on social media (51% vs. 35%).

How Race, Age and Gender Shape Attitudes Towards Mental Health

ThinkNow

AI Trends in Creative Operations 2024 by Artwork Flow.pdf

marketingartwork

Skeleton Culture Code

Skeleton Technologies

PEPSICO Presentation to CAGNY Conference Feb 2024

Neil Kimberley

Content Methodology: A Best Practices Report (Webinar)

contently

How to Prepare For a Successful Job Search for 2024

Albert Qian

A report by thenetworkone and Kurio. The contributing experts and agencies are (in an alphabetical order): Sylwia Rytel, Social Media Supervisor, 180heartbeats + JUNG v MATT (PL), Sharlene Jenner, Vice President - Director of Engagement Strategy, Abelson Taylor (USA), Alex Casanovas, Digital Director, Atrevia (ES), Dora Beilin, Senior Social Strategist, Barrett Hoffher (USA), Min Seo, Campaign Director, Brand New Agency (KR), Deshé M. Gully, Associate Strategist, Day One Agency (USA), Francesca Trevisan, Strategist, Different (IT), Trevor Crossman, CX and Digital Transformation Director; Olivia Hussey, Strategic Planner; Simi Srinarula, Social Media Manager, The Hallway (AUS), James Hebbert, Managing Director, Hylink (CN / UK), Mundy Álvarez, Planning Director; Pedro Rojas, Social Media Manager; Pancho González, CCO, Inbrax (CH), Oana Oprea, Head of Digital Planning, Jam Session Agency (RO), Amy Bottrill, Social Account Director, Launch (UK), Gaby Arriaga, Founder, Leonardo1452 (MX), Shantesh S Row, Creative Director, Liwa (UAE), Rajesh Mehta, Chief Strategy Officer; Dhruv Gaur, Digital Planning Lead; Leonie Mergulhao, Account Supervisor - Social Media & PR, Medulla (IN), Aurelija Plioplytė, Head of Digital & Social, Not Perfect (LI), Daiana Khaidargaliyeva, Account Manager, Osaka Labs (UK / USA), Stefanie Söhnchen, Vice President Digital, PIABO Communications (DE), Elisabeth Winiartati, Managing Consultant, Head of Global Integrated Communications; Lydia Aprina, Account Manager, Integrated Marketing and Communications; Nita Prabowo, Account Manager, Integrated Marketing and Communications; Okhi, Web Developer, PNTR Group (ID), Kei Obusan, Insights Director; Daffi Ranandi, Insights Manager, Radarr (SG), Gautam Reghunath, Co-founder & CEO, Talented (IN), Donagh Humphreys, Head of Social and Digital Innovation, THINKHOUSE (IRE), Sarah Yim, Strategy Director, Zulu Alpha Kilo (CA).

Social Media Marketing Trends 2024 // The Global Indie Insights

Kurio // The Social Media Age(ncy)

The search marketing landscape is evolving rapidly with new technologies, and professionals, like you, rely on innovative paid search strategies to meet changing demands. It’s important that you’re ready to implement new strategies in 2024. Check this out and learn the top trends in paid search advertising that are expected to gain traction, so you can drive higher ROI more efficiently in 2024. You’ll learn: - The latest trends in AI and automation, and what this means for an evolving paid search ecosystem. - New developments in privacy and data regulation. - Emerging ad formats that are expected to make an impact next year. Watch Sreekant Lanka from iQuanti and Irina Klein from OneMain Financial as they dive into the future of paid search and explore the trends, strategies, and technologies that will shape the search marketing landscape. If you’re looking to assess your paid search strategy and design an industry-aligned plan for 2024, then this webinar is for you.

Trends In Paid Search: Navigating The Digital Landscape In 2024

Search Engine Journal

From their humble beginnings in 1984, TED has grown into the world’s most powerful amplifier for speakers and thought-leaders to share their ideas. They have over 2,400 filmed talks (not including the 30,000+ TEDx videos) freely available online, and have hosted over 17,500 events around the world. With over one billion views in a year, it’s no wonder that so many speakers are looking to TED for ideas on how to share their message more effectively. The article “5 Public-Speaking Tips TED Gives Its Speakers”, by Carmine Gallo for Forbes, gives speakers five practical ways to connect with their audience, and effectively share their ideas on stage. Whether you are gearing up to get on a TED stage yourself, or just want to master the skills that so many of their speakers possess, these tips and quotes from Chris Anderson, the TED Talks Curator, will encourage you to make the most impactful impression on your audience. See the full article and more summaries like this on SpeakerHub here: https://speakerhub.com/blog/5-presentation-tips-ted-gives-its-speakers See the original article on Forbes here: http://www.forbes.com/forbes/welcome/?toURL=http://www.forbes.com/sites/carminegallo/2016/05/06/5-public-speaking-tips-ted-gives-its-speakers/&refURL=&referrer=#5c07a8221d9b

5 Public speaking tips from TED - Visualized summary

SpeakerHub

Everyone is in agreement that ChatGPT (and other generative AI tools) will shape the future of work. Yet there is little consensus on exactly how, when, and to what extent this technology will change our world. Businesses that extract maximum value from ChatGPT will use it as a collaborative tool for everything from brainstorming to technical maintenance. For individuals, now is the time to pinpoint the skills the future professional will need to thrive in the AI age. Check out this presentation to understand what ChatGPT is, how it will shape the future of work, and how you can prepare to take advantage.

ChatGPT and the Future of Work - Clark Boyd

Clark Boyd

Getting into the tech field. what next

Tessa Mero

Google's Just Not That Into You: Understanding Core Updates & Search Intent

Lily Ray

How to have difficult conversations

Rajiv Jayarajah, MAppComm, ACC

Introduction to Data Science

Christy Abraham Joy

Time Management & Productivity - Best Practices

Vit Horky

The six step guide to practical project management If you think managing projects is too difficult, think again. We’ve stripped back project management processes to the basics – to make it quicker and easier, without sacrificing the vital ingredients for success. “If you’re looking for some real-world guidance, then The Six Step Guide to Practical Project Management will help.” Dr Andrew Makar, Tactical Project Management

The six step guide to practical project management

MindGenius

Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...

RachelPearson36

During this webinar, Anand Bagmar demonstrates how AI tools such as ChatGPT can be applied to various stages of the software development life cycle (SDLC) using an eCommerce application case study. Find the on-demand recording and more info at https://applitools.info/b59 Key takeaways: • Learn how to use ChatGPT to add AI power to your testing and test automation • Understand the limitations of the technology and where human expertise is crucial • Gain insight into different AI-based tools • Adopt AI-based tools to stay relevant and optimize work for developers and testers * ChatGPT and OpenAI belong to OpenAI, L.L.C.

Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...

Applitools

12 Ways to Increase Your Influence at Work

GetSmarter

Featured (20)

Product Design Trends in 2024 | Teenage Engineerings

How Race, Age and Gender Shape Attitudes Towards Mental Health

AI Trends in Creative Operations 2024 by Artwork Flow.pdf

Skeleton Culture Code

PEPSICO Presentation to CAGNY Conference Feb 2024

Content Methodology: A Best Practices Report (Webinar)

How to Prepare For a Successful Job Search for 2024

Social Media Marketing Trends 2024 // The Global Indie Insights

Trends In Paid Search: Navigating The Digital Landscape In 2024

5 Public speaking tips from TED - Visualized summary

ChatGPT and the Future of Work - Clark Boyd

Getting into the tech field. what next

Google's Just Not That Into You: Understanding Core Updates & Search Intent

How to have difficult conversations

Introduction to Data Science

Time Management & Productivity - Best Practices

The six step guide to practical project management

Beginners Guide to TikTok for Search - Rachel Pearson - We are Tilt __ Bright...

Unlocking the Power of ChatGPT and AI in Testing - A Real-World Look, present...

12 Ways to Increase Your Influence at Work

Reinforcement Learning Tutorial for Minecraft

1. Reinforcement Learning Tutorial for Minecraft

2. 목차 1. 팀원 소개 2. Malmo a. Introduction b. launchClient c. Missions d. Examples 3. 서버에서 말모 실행 4. Malmo 단점 5. MarLo 소개 6. MarLo Tutorial

3. 팀원 소개 ● 이영무 ○ 네이버 인턴중 ○ Email : zzing0907@gmail.com ● 양기창 ○ 숭실대학교 재학중 ○ Email : ykcha9@gmail.com

4. 마인크래프트

5. Malmo: Introduction 링크 : https://github.com/Microsoft/malmo

6. Malmo: Launch Client 링크 → https://github.com/Microsoft/malmo/blob/master/doc/build_linux.md

7. Malmo: Launch Client 설치 가이드 → https://github.com/reinforcement-learning-kr/minecraft_rl_tutorial/wiki

8. Malmo: Launch Client Dependency 설치 후에 launchClient.sh 실행시키면 client가 실행이 된다.

9. Malmo: Launch Client

10. Malmo: Missions ● 미션파일을 수정 가능 ○ 커스텀 미션 ○ 새로운 태스크 ● 수행할 수 있는 태스크가 많지는 않음 ○ 어떤 블록을 터치 ○ 어떤 아이템을 획득 ● 리워드를 주는 방식이 커스터마이징이 어 려움

11. Malmo: Sample Missions 총 11개의 샘플 미션이 있음(xml 파일) 추가적으로 Example에 몇가지 미션이 더 있음

12. Malmo: Examples ● 많은 예제들이 있음 ● 대부분이 룰 기반으로 환경을 품 ● 딥러닝 예제는 들어있지 않음 ● 강화학습 예제는 tabular-q-learning 하나 뿐임 ● 그래도 있을 건 다 있다! (ex. 환경의 속도를 변화, dynamic한 환경 구성 등)

13. Malmo: Examples

14. Malmo: Examples

15. 서버에서 말모 실행 ● 도커를 이용해서 서버에서 실행할 수 있음 ● Dockerfile 제공 (https://github.com/reinforce ment-learning- kr/minecraft_rl_tutorial/blob/ master/docker/Dockerfile) ● Docker guide 제공

16. Malmo의 단점 코드가 복잡하다! 설정할 것들이 많음 강화학습 코드에 집중 X

17. Malmo의 단점 python코드에서 커맨드를 보내는 사 이에도 환경은 계속해서 돌아감 모델이 inference하는 속도가 느리다 면 학습에 크게 영향을 미침

18. Malmo의 단점 ● 환경이 불안정한 경우가 많다. ○ 학습을 하다가 연결이 종료되는 경우 (윈도우) ○ 에피소드 종료 후 이미 client가 연결되어 있다면서 실행되지 않는 경우 ● 컴퓨터의 성능에 따라 결과가 달라짐 ○ 렌더링 속도에 따라 결과가 달라짐 ○ 서버에서는 점수가 잘 나오지만 로컬에서 점수가 안나오는 경우도 있음 ● 보기 편한 Document가 없다.

19. MarLo Crowd AI 에서 Malmo를 이용한 challenge 개최 멀티 에이전트를 위한 challenge

20. MarLo Challenge

21. MarLo Challenge MarLo 2018 challenge Coming Soon !

22. MarLo Code 깃헙이 있으나 마스터 브랜치는 아직 개발중인.. (에러 발생) install 브랜치 클론해서 설치하 면 사용 가능한 버전을 받을 수 있 음

23. MarLo Code 코드가 훨씬 간단해졌다! OpenAI gym 프래임워크를 이용해서 코 드가 간결해짐

24. MarLo Code

25. MarLo Tutorial - Eating

26. MarLo Tutorial - CatchTheMob

27. 앞으로의 방향성

28. Q&A