Adam Streck - Reinforcement Learning in Unity. Teach Your Monsters - Codemotion Berlin 2018

•Download as PPTX, PDF•

0 likes•32 views

With the advent of deep learning many of the tasks in computer science that have been deemed impossible suddenly became only a few clicks away. One of the approaches made available is reinforcement learning - a method for solving problems by establishing an action-reward scheme. Combined with the power and availability of the general-purpose game engines, anyone with a rudimentary knowledge of the topic can create and train their virtual creatures. In this talk we will use this power to solve one of the most frustratingly difficult (according to the internet) games of our era.

Technology

Reinforcement Learning in Unity
Adam Streck
Berlin | November 20 - 21, 2018

@AdamStreck
2000 2010 2018
GAMES DEV
RESEARCH - MODELING OF BIOLOGICAL SYSTEMS

@AdamStreck
2000 2010 2018
GAMES DEV
RESEARCH - MODELING OF BIOLOGICAL SYSTEMS
VIRTUAL REALITY DEV

@AdamStreck
2000 2010 2018
GAMES DEV
RESEARCH - MODELING OF BIOLOGICAL SYSTEMS
VIRTUAL REALITY DEV
RESEARCH - VR PSYCHOLOGY/NEUROLOGY (@DZNE)

WHAT IS THE MEANING OF THIS?
• ultra-popular game engine
• a big bulk of libraries glued together
UNITY

WHAT IS THE MEANING OF THIS?
• ultra-popular game engine
• a big bulk of libraries glued together
• conditioning (positive/negative)
• built on top of (deep) neural networks
UNITY
(DEEP) REINFORCEMENT LEARNING

HISTORY (in a very few words)
CREATURES (1996)
©RockPaperShotgun

HISTORY (in a very few words)
CREATURES (1996) BLACK&WHITE (2001)
©ReviewAnyGame©RockPaperShotgun

HISTORY (in a very few words)
CREATURES (1996) BLACK&WHITE (2001)
©ReviewAnyGame©RockPaperShotgun
...DARK AGES?
©iStock

HISTORY (in a very few words)
CREATURES (1996) BLACK&WHITE (2001)
FAST FORWARD
(DEEP LEARNING HYPE)
©ReviewAnyGame©RockPaperShotgun
...DARK AGES?
©iStock

HISTORY (in a very few words)
CREATURES (1996) BLACK&WHITE (2001)
ALPHA GO (2016)
FAST FORWARD
(DEEP LEARNING HYPE)
©DeepMind
©ReviewAnyGame©RockPaperShotgun
...DARK AGES?
©iStock

WHAT IS …?
*https://en.wikipedia.org/wiki/Q-learning

WHAT IS …?
*https://en.wikipedia.org/wiki/Q-learning
©DZone

HANDLE WITH CARE
OpenAI - Faulty Reward Functions in the
Wild

• ML-Agents: unity3d.com/machine-learning
SAUCE

• ML-Agents: unity3d.com/machine-learning
• OpenAI Gym: gym.openai.com
SAUCE

• ML-Agents: unity3d.com/machine-learning
• OpenAI Gym: gym.openai.com
• IEEE Conference on Games: ieee-cog.org
SAUCE

• ML-Agents: unity3d.com/machine-learning
• OpenAI Gym: gym.openai.com
• IEEE Conference on Games: ieee-cog.org
(Formerly IEEE CIG 2005-2018 ieee-cig.org)
SAUCE

• ML-Agents: unity3d.com/machine-learning
• OpenAI Gym: gym.openai.com
• IEEE Conference on Games: ieee-cog.org
(Formerly IEEE CIG 2005-2018 ieee-cig.org)
• AI Blunders: arxiv.org/abs/1803.03453
SAUCE

Many recent advances in deep reinforcement and artificial intelligence learning have stemmed from video games. In this session, we'll explore a brief history of this relationship, looking specifically at how Unity is pushing the boundaries of AI research with the Obstacle Tower Challenge. We'll also show how Unity is leveraging cutting-edge research to solve gaming's biggest challenges with the Unity Machine Learning Agents Toolkit, one of the most popular open-source toolkits for deep learning. Speaker: Matthew Crosby - Imperial College London

Simon Wardley at SpringOne Platform 2017

VMware Tanzu

Demystifying Machine Learning - How to give your business superpowers.

10x Nation

Semantic Web: In Quest for the Next Generation Killer Apps

Jie Bao

Economies of Scaling Software

Joshua Long

NISO Plus: Artificial Intelligence and Machine Learning 101

National Information Standards Organization (NISO)

I should really sync audio to these slides since most of the information is not on the slides. Open source development has reached a stable and remarkable maturity. With services like SourceForge and Google Code for hosting projects, the Open Source Initiative to vet and curate Open Source licenses, and organizations like the FSF and Apache where like-minded developers can work together to build sustainable and open communities around Open Source projects, and the support of hundreds of thousands of developers and major corporations alike, the success of open source is firmly established. Yet when we turn our attention away from open source and instead to specifications and standards for the open web, much of this infrastructure doesn't yet exist. Formal standards bodies may enforce interoperability, but they don't always guarantee that a standard is freely implementable by everyone or that the development community is open to all potential contributors. As software development is increasingly centered on protocols and formats instead of simply source code, many newer initiatives, like Microformats, OpenID, OAuth and OpenSocial, have had to each invest time and money reinventing the legal and organizational infrastructure required to ensure that the specifications they create are open and their communities are healthy and run in meritocratic fashions. Isn't there a better way? What can we learn from the open source movement that will help us create open specifications for the web? The newly created Open Web Foundation is tackling this exact question by borrowing heavily from the proven model established by the Apache Foundation. This talk discusses the Open Web Foundation's progress so far, our goals for the future, and how you can get involved.

What is AI, Machine Learning, Neural Networks, Deep Learning and Data Science

Som Shahapurkar

Drools and jBPM 6 Overview

Mark Proctor

KubeMonkey를 통한 Chaos Engineering 실전 운영하기 - 윤석찬 (AWS 테크에반젤리스트)

Channy Yun

카오스 엔지니어링(Chaos Engineering)이란 프로덕션 서비스의 각종 장애 조건을 견딜 수 있는 시스템의 신뢰성을 확보하기 위해 분산 시스템을 실험 하고 배우는 분야입니다. 즉, 개발자들이 현실 세계에서 발견되는 시스템 장애를 미리 탐지하여 복원성 높은 아키텍처를 구성하는 방법을 공유합니다.클라우드 컴퓨팅의 발전과 데브옵스 방법론을 기반으로 자동화를 통해 좀 더 쉽게 개발자들이 직접 분산 시스템을 통제된 환경에서 실험을 하는 동안 나오는 결과를 관찰함으로써 현실에서 실제 행동 방법을 배울 수 있습니다. 본 세션에서는 카오스 엔지니어링의 기본 개념과 함께 Kubernetes용 Chaos Tool인 KubeMonkey를 통해 무작위로 클러스터의 포드를 삭제하여 장애 복구 서비스 아키텍처를 검증하는 방법을 설명합니다.

Recommender Systems at Scale

Eoin Hurrell, PhD

Deep Learning in the Real World

Lukas Biewald

Just the basics_strata_2013Ken Mwai

ORIGINLAB 2018 FOR PERSOLAL USE.pptx

THE CREATORS ACADEMY

Big data and AI presentation slides

CloudxLab

Evolution of AI - Why is my computer still so dumb?

Olivia Klose

Everyone is talking about artificial intelligence nowadays. It seems to be the golden solution to every problem: image processing, speech recognition, conversations, pattern recognition for predictive maintenance. But what is under the hood and is it really that new? In this session, we will go through a couple of questions on AI: how did it start? What is AI winter? Why does my computer struggle with talking to me? Are there alternatives to AI? The session will be garnished with examples and obviously code.

AI and Interactive Narrative

Mirjam Eladhari

AI and Interactive Narrative in 2019

Mirjam Eladhari

34th.余凯.机器学习进展及语音图像中的应用

komunling

Sustainable software institute Collaboration workshop

Yannick Wurm

Openlaw - A Project of Alphalaw

Sunghyun Kim

The real value of open source: ROI and beyond

Jeffrey McGuire

Developer's Introduction to Machine Learning

Christopher Mohritz

Fuzz-testing: A hacker's approach to making your code more secure | Pascal Ze...

Codemotion

Pompili - From hero to_zero: The FatalNoise neverending story

Codemotion

Similar to Adam Streck - Reinforcement Learning in Unity. Teach Your Monsters - Codemotion Berlin 2018

Building the iRODS Consortium

All Things Open

Convolutional neural network in practice

남주 김

Real Life InfoSec - Tel Aviv UniversityMichael Shalyt

Learning from Apache to create Open Specifications

David Recordon

What is AI, Machine Learning, Neural Networks, Deep Learning and Data Science

Som Shahapurkar

Drools and jBPM 6 Overview

Mark Proctor

KubeMonkey를 통한 Chaos Engineering 실전 운영하기 - 윤석찬 (AWS 테크에반젤리스트)

Channy Yun

Recommender Systems at Scale

Eoin Hurrell, PhD

Deep Learning in the Real World

Lukas Biewald

Just the basics_strata_2013Ken Mwai

ORIGINLAB 2018 FOR PERSOLAL USE.pptx

THE CREATORS ACADEMY

Big data and AI presentation slides

CloudxLab

Evolution of AI - Why is my computer still so dumb?

Olivia Klose

AI and Interactive Narrative

Mirjam Eladhari

AI and Interactive Narrative in 2019

Mirjam Eladhari

34th.余凯.机器学习进展及语音图像中的应用

komunling

Sustainable software institute Collaboration workshop

Yannick Wurm

Openlaw - A Project of Alphalaw

Sunghyun Kim

The real value of open source: ROI and beyond

Jeffrey McGuire

Developer's Introduction to Machine Learning

Christopher Mohritz

Similar to Adam Streck - Reinforcement Learning in Unity. Teach Your Monsters - Codemotion Berlin 2018 (20)

Building the iRODS Consortium

Convolutional neural network in practice

Real Life InfoSec - Tel Aviv University

Learning from Apache to create Open Specifications

What is AI, Machine Learning, Neural Networks, Deep Learning and Data Science

Drools and jBPM 6 Overview

KubeMonkey를 통한 Chaos Engineering 실전 운영하기 - 윤석찬 (AWS 테크에반젤리스트)

Recommender Systems at Scale

Deep Learning in the Real World

Just the basics_strata_2013

ORIGINLAB 2018 FOR PERSOLAL USE.pptx

Big data and AI presentation slides

Evolution of AI - Why is my computer still so dumb?

AI and Interactive Narrative

AI and Interactive Narrative in 2019

34th.余凯.机器学习进展及语音图像中的应用

Sustainable software institute Collaboration workshop

Openlaw - A Project of Alphalaw

The real value of open source: ROI and beyond

Developer's Introduction to Machine Learning

More from Codemotion

Fuzz-testing: A hacker's approach to making your code more secure | Pascal Ze...

Codemotion

Pompili - From hero to_zero: The FatalNoise neverending story

Codemotion

Pastore - Commodore 65 - La storia

Codemotion

Pennisi - Essere Richard Altwasser

Codemotion

Rivivere l'ebbrezza di progettare un vecchio computer o una consolle da bar è oggi possibile sfruttando le FPGA, ovvero logiche programmabili che consentono a chiunque di progettare il proprio hardware o di ricrearne uno del passato. In questa sessione si racconta come dal reverse engineering dell'hardware di vecchie glorie come il Commodore 64 e lo ZX Spectrum sia stato possibile farle rivivere attraverso tecnologie oggi alla portata di tutti.

Michel Schudel - Let's build a blockchain... in 40 minutes! - Codemotion Amst...

Codemotion

There's a lot of talk about blockchain, but how does the technology behind it actually work? For developers, getting some hands-on experience is the fastest way to get familiair with new technologies. So let's build a blockchain, then! In this session, we're going to build one in plain old Java, and have it working in 40 minutes. We'll cover key concepts of a blockchain: transactions, blocks, mining, proof-of-work, and reaching consensus in the blockchain network. After this session, you'll have a better understanding of core aspects of blockchain technology.

Richard Süselbeck - Building your own ride share app - Codemotion Amsterdam 2019

Codemotion

When was the last time you were truly lost? Thanks to the maps and location technology in our phones, a whole generation has now grown up in a world where getting lost is truly a thing of the past. Location technology goes far beyond maps in the palm of our hand, however. In this talk, we will explore how a ridesharing app works. How do we discover our destination?How do we find the closest driver? How do we display this information on a map? How do we find the best route?To answer these questions,we will be learning about a variety of location APIs, including Maps, Positioning, Geocoding etc.

Eward Driehuis - What we learned from 20.000 attacks - Codemotion Amsterdam 2019

Codemotion

Eward Driehuis, SecureLink's research chief, will guide you through the bumpy ride we call the cyber threat landscape. As the industry has over a decade of experience of dealing with increasingly sophisticated attacks, you might be surprised to hear more attacks slip through the cracks than ever. From analyzing 20.000 of them in 2018, backed by a quarter of a million security events and over ten trillion data points, Eward will outline why this happens, how attacks are changing, and why it doesn't matter how neatly or securely you code.

Francesco Baldassarri - Deliver Data at Scale - Codemotion Amsterdam 2019 -

Codemotion

IoT revolution is ended. Thanks to hardware improvement, building an intelligent ecosystem is easier than never before for both startups and large-scale enterprises. The real challenge is now to connect, process, store and analyze data: in the cloud, but also, at the edge. We’ll give a quick look on frameworks that aggregate dispersed devices data into a single global optimized system allowing to improve operational efficiency, to predict maintenance, to track asset in real-time, to secure cloud-connected devices and much more.

Martin Förtsch, Thomas Endres - Stereoscopic Style Transfer AI - Codemotion A...

Codemotion

What if Virtual Reality glasses could transform your environment into a three-dimensional work of art in realtime in the style of a painting from Van Gogh? One of the many interesting developments in the field of Deep Learning is the so called "Style Transfer". It describes a possibility to create a patchwork (or pastiche) from two images. While one of these images defines the the artistic style of the result picture, the other one is used for extracting the image content. A team from TNG Technology Consulting managed to build an AI showcase using OpenCV and Tensorflow to realize such goggles.

Melanie Rieback, Klaus Kursawe - Blockchain Security: Melting the "Silver Bul...

Codemotion

Blockchain (and Cryptocurrency) is an evolution of 20-year old research from scientists like Chaum, Lamport, and Castro & Liskov. Due to the current hype, it's hard to distinguish beneficial aspects of the technology from a desire for a "silver bullet" for device security, verifiable logistics, or "saving democracy". The problem: blockchain introduces new security challenges - and blind adoption without understanding reduces overall security. In this talk, Melanie Rieback and Klaus Kursawe explain the pitfalls and limits of blockchain, so you can avoid making your applications LESS secure.

Angelo van der Sijpt - How well do you know your network stack? - Codemotion ...

Codemotion

Networking is a core part of computing in the digital world we inhabit. But, how well do you know how it works? Do you understand all the moving parts of the OSI stack inside your computer, and how the network is actually put together? How can this ever work? This guided safari of layers, standards, protocols, and happenstance will bring us close to the copper wire, and up through the layers of CDMA/CD, ARP, routing and HTTP. We will make a few excursions through patchworks that still work forty years later, and cleverly designed mechanisms that show that simplicity is the only way to last.

Lars Wolff - Performance Testing for DevOps in the Cloud - Codemotion Amsterd...

Codemotion

Performance tests are not only an important instrument for understanding a system and its runtime environment. It is also essential in order to check stability and scalability – non-functional requirements that might be decisive for success. But won't my cloud hosting service scale for me as long as I can afford it? Yes, but… It only operates and scales resources. It won't automatically make your system fast, stable and scalable. This talk shows how such and comparable questions can be clarified with performance tests and how DevOps teams benefit from regular test practise.

Sascha Wolter - Conversational AI Demystified - Codemotion Amsterdam 2019

Codemotion

Sascha will demonstrate the opportunities and challenges of Conversational AI learned from the practice. Both Technology and User Experience will be covered introducing a process finding micro-moments, writing happy paths, gathering intents, designing the conversational flow, and finally publishing on almost all channels including Voice Services and Chatbots. Valuable for enterprises, developers, and designers. All live on stage in just minutes and with almost no code.

Michele Tonutti - Scaling is caring - Codemotion Amsterdam 2019

Codemotion

A key challenge we face at Pacmed is quickly calibrating and deploying our tools for clinical decision support in different hospitals, where data formats may vary greatly. Using Intensive Care Units as a case study, I’ll delve into our scalable Python pipeline, which leverages Pandas’ split-apply-combine approach to perform complex feature engineering and automatic quality checks on large time-varying data, e.g. vital signs. I’ll show how we use the resulting flexible and interpretable dataframes to quickly (re)train our models to predict mortality, discharge, and medical complications.

Pat Hermens - From 100 to 1,000+ deployments a day - Codemotion Amsterdam 2019

Codemotion

Coolblue is a proud Dutch company, with a large internal development department; one that truly takes CI/CD to heart. Empowerment through automation is at the heart of these development teams, and with more than 1000 deployments a day, we think it's working out quite well. In this session, Pat Hermens (a Development Managers) will step you through what enables us to move so quickly, which tools we use, and most importantly, the mindset that is required to enable development teams to deliver at such a rapid pace.

James Birnie - Using Many Worlds of Compute Power with Quantum - Codemotion A...

Codemotion

Quantum computers can use all of the possible pathways generated by quantum decisions to solve problems that will forever remain intractable to classical compute power. As the mega players vie for quantum supremacy and Rigetti announces its $1M "quantum advantage" prize, we live in exciting times. IBM-Q and Microsoft Q# are two ways you can learn to program quantum computers so that you're ready when the quantum revolution comes. I'll demonstrate some quantum solutions to problems that will forever be out of reach of classical, including organic chemistry and large number factorisation.

Don Goodman-Wilson - Chinese food, motor scooters, and open source developmen...

Codemotion

Chinese food exploded across America in the early 20th century, rapidly adapting to local tastes while also spreading like wildfire. How was it able to spread so fast? The GY6 is a family of scooter engines that has achieved near total ubiquity in Europe. It is reliable and cheap to manufacture, and it's made in factories across China. How are these factories able to remain afloat? Chinese-American food and the GY6 are both riveting studies in product-market fit, and both are the product of a distributed open source-like development model. What lessons can we learn for open source software?

Pieter Omvlee - The story behind Sketch - Codemotion Amsterdam 2019

Codemotion

Dave Farley - Taking Back “Software Engineering” - Codemotion Amsterdam 2019

Codemotion

Would you fly in a plane designed by a craftsman or would you prefer your aircraft to be designed by engineers? We are learning that science and empiricism works in software development, maybe now is the time to redefine what “Software Engineering” really means. Software isn't bridge-building, it is not car or aircraft development either, but then neither is Chemical Engineering. Engineering is different in different disciplines. Maybe it is time for us to begin thinking about retrieving the term "Software Engineering" maybe it is time to define what our "Engineering" discipline should be.

Joshua Hoffman - Should the CTO be Coding? - Codemotion Amsterdam 2019

Codemotion

What is the job of a CTO and how does it change as a startup grows in size and scale? As a CTO, where should you spend your focus? As an engineer aspiring to be a CTO, what skills should you pursue? In this inspiring and personal talk, I describe my journey from early Red Hat engineer to CTO at Bloomon. I will share my view on what it means to be a CTO, and ultimately answer the question: Should the CTO be coding?

More from Codemotion (20)