Photo-realistic Single Image Super-resolution using a Generative Adversarial ...Hansol Kang
* Ledig, Christian, et al. "Photo-realistic single image super-resolution using a generative adversarial network." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
Co Speaker: Cheryl Biswas
Talk Description:
How about this: a blue team talk given by red teamers. But here’s our rationale - your best defence right now is a strategic offence. The rules of the game have changed and we need to get defence up to speed.
We’ll show you what the key elements are in a good defence strategy; what you can and need to be using to full advantage. We’ll talk about the new “buzzwords” and how they apply: visibility; patterns; big data. There’s a whole lotta data to wrangle, and you aren’t seeing the whole picture if you aren’t doing things right. Threat intel is about getting the big picture as it applies to you. You’ll learn the importance of context and prioritization so that you can manipulate intel feeds to do your bidding. And then we’ll take things further and talk about hunting the adversary, using an update on proven methodologies.
We’ll show you how to understand your data, correlate threats and pin point attacks. Attendees will leave with a new understanding of the resources they have on hand, and how to leverage those into an Adaptive Proactive Defense Strategy.
Photo-realistic Single Image Super-resolution using a Generative Adversarial ...Hansol Kang
* Ledig, Christian, et al. "Photo-realistic single image super-resolution using a generative adversarial network." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
Co Speaker: Cheryl Biswas
Talk Description:
How about this: a blue team talk given by red teamers. But here’s our rationale - your best defence right now is a strategic offence. The rules of the game have changed and we need to get defence up to speed.
We’ll show you what the key elements are in a good defence strategy; what you can and need to be using to full advantage. We’ll talk about the new “buzzwords” and how they apply: visibility; patterns; big data. There’s a whole lotta data to wrangle, and you aren’t seeing the whole picture if you aren’t doing things right. Threat intel is about getting the big picture as it applies to you. You’ll learn the importance of context and prioritization so that you can manipulate intel feeds to do your bidding. And then we’ll take things further and talk about hunting the adversary, using an update on proven methodologies.
We’ll show you how to understand your data, correlate threats and pin point attacks. Attendees will leave with a new understanding of the resources they have on hand, and how to leverage those into an Adaptive Proactive Defense Strategy.
A grand challenge of AI has fallen - a decade earlier than "experts" predicted. But should we care?
What made AlphaGo, the AI built by DeepMind, so unique?
Dive into AlphaGo's system of deep learning, evaluation, and search algorithms that combined to defeat the reigning Go world champion, and draw your own conclusions.
Tim Riser presented an analysis of "Mastering the Game of Go with Deep Neural Networks & Tree Search", a paper by Google DeepMind to the Boston/Cambridge chapter of Papers We Love, a computer science discussion group on June 28, 2016.
Slides for a short lecture (~1 hr) on the foundations of the Alpha Go model developed by Google. Intended for people with little technical background, but with basic familiarity with Deep Learning.
Jose Selvi - Side-Channels Uncovered [rootedvlc2018]RootedCON
En los últimos años, el término "side-channel" a pasado de ser un concepto únicamente conocido en el sector de hardware hacking a ser un término popular dentro de la industria debido a las vulnerabilidades que se han ido publicando. CRIME, BREACH o FIESTA son claros ejemplos de vulnerabilidades que explotan un side-channel en TLS. Más recientemente, también hemos visto vulnerabilidades empleando este mismo concepto en procesadores, como Spectre o Meltdown.
En esta charla, repasaremos el concepto de "side-channel" y haremos un repaso por las diferentes vulnerabilidades que se han ido publicando a lo largo de estos últimos años, explicando en que consisten y que limitaciones tienen.
Machine Learning Exposed - James Weaver - Codemotion Amsterdam 2017Codemotion
The term "machine learning" is increasingly bandied about in corporate settings and cocktail parties, but what is it, really? In this session we'll answer that question, providing an approachable overview of machine learning concepts, technologies, and use cases. We'll then take a deeper dive into machine learning topics such as supervised learning, unsupervised learning, and deep learning. We'll also survey various machine learning APIs and platforms. You'll be the hit of your next party when you're able to express the near-magical inner-workings of artificial neural networks!
In this session, we will be discussing major outages that happened in major enterprises. We will be analyzing the actual thread dumps, heap dumps, GC logs, and other artifacts captured at the time of the problem. After this session, troubleshooting CPU spikes, OutOfMemoryError, response time degradations, network connectivity issues, application unresponsiveness may not stump you.
Accelerating Incident Response To Production OutagesTier1 app
In this webinar, following topics were discussed
1) Production outages that happened in major enterprises in their JVM applications.
2) Analyzing the actual thread dumps, heap dumps, GC logs, and other artifacts captured at the time of the problem.
Unsupervised Computer Vision: The Current State of the ArtTJ Torres
This presentation was originally given at a styling research presentation at Stitch Fix, where I talk about some of the recent progress in the field of unsupervised deep learning methods for image analysis. It includes descriptions of Variational Autoencoders (VAE), Generative Adversarial Networks (GAN), their hybrid (VAE/GAN), Generative Moment Matching Networks (GMMN), and Adversarial Autoencoders.
최근 IT업계에서는 딥러닝에 대한 큰 활약이 이슈화 되고 있습니다.
이와 같은 트렌드에 따라가고 싶지만 선형대수 등의 수학적 배경지식 습득부터 시작하여, 딥러닝의 원리와 주요 개념들을 이해 후에 실험을 시도하기에는 많은 시간과 노력이 필요합니다.
그러나 기존의 유용한 딥러닝 오픈소스를 활용한다면 어렵지 않게 딥러닝을 맛볼 수 있습니다.
본 발표는 수학적인 설명을 최대한 배제하고, 오픈소스 툴인 theano, pylearn2를 활용한 예제에 대해 설명하려고 합니다. 추가로 필요할 코드들도 소개하려고 합니다.
그리고 word2vec 를 활용하여, 자연어 처리에 딥러닝을 적용하는 사례를 다루려고 합니다.
주제가 학문적이고 이론적이기 때문에 발표가 부담되지만, 최대한 개념적으로 설명하여 실험을 쉽게 따라 할 수 있도록 돕고자 합니다.
오픈소스 툴들의 문서화가 잘 되어있지만, 저 또한 처음 접했을 때는 어려움이 있었기 때문에 딥러닝을 시작해보려는 분들에게 도움이 될 듯합니다.
컴퓨터가 딥러닝하는 동안 틈틈이 이론공부 하시면 좋겠네요.
Atari Game State Representation using Convolutional Neural Networksjohnstamford
I recently gave a talk to some MSc Machine Learning students at De Montfort University about the project I did for my MSc. The work included looking at feature extraction from game screens using the Arcade Learning Environment and Convolutional Neural Networks (CNN).
The work was planned to investigate if the costly nature Q-Learning could be offset by the use of a trained system using 'expert' data. The system uses the same technology as used by Deepmind in their 2013 paper.
What I learned about IoT Security ... and why it's so hard!Christoph Engelbert
Smart devices taking over our living rooms, our bed rooms, and, in general, our life. It has never been more important to build secure devices, but most companies seem to fail, and they fail hard. We (only) build systems for farms and barns, and still, I wanted security for Cow-stumers.
Building a mostly secure system is fairly simple. There is a good set of low-hanging fruits. Building a really locked down system is tough, though. Much harder than expected. Here is what I learned.
Deep Learning in Python with Tensorflow for FinanceBen Ball
Speaker: Ben Ball
Abstract: Python is becoming the de facto standard for many machine learning applications. We have been using Python with deep learning and other ML techniques, with a focus in prediction and exploitation in transactional markets. I am presenting one of our implementations (A dueling double DQN - a class of Reinforcement Learning algorithm) in Python using TensorFlow, along with information and background around the class of deep learning algorithm, and the application to financial markets we have employed. Attendees will learn how to implement a DQN using Tensorflow, and how to design a system for deep learning for solving a wide range of problems. The code will be available on github for attendees.
Bio: Ben is a believer in making a career out of what you love. He is inspired by the joining of excellent technology and research and likes building software that is easy to use, but does amazing things. His work has spanned 15 years, with a dual focus in AI Software Engineering and Algorithmic Trading. He is currently working as the CTO of http://prediction-machines.com
Video of the presentation:
https://engineers.sg/video/deep-learning-with-python-in-finance-singapore-python-user-group--1875
Descripción de la plática:
CoreML es el puente entre iOS y Machine Learning, pero con ciertas limitantes. Explotaremos el potencial que tiene CoreML, responderemos las siguientes preguntas:
- ¿Cómo podríamos ir más allá de los límites de CoreML?
- ¿Cómo puede nuestra App aprender de la experiencia?
生成式對抗網路 (Generative Adversarial Network, GAN) 顯然是深度學習領域的下一個熱點,Yann LeCun 說這是機器學習領域這十年來最有趣的想法 (the most interesting idea in the last 10 years in ML),又說這是有史以來最酷的東西 (the coolest thing since sliced bread)。生成式對抗網路解決了什麼樣的問題呢?在機器學習領域,回歸 (regression) 和分類 (classification) 這兩項任務的解法人們已經不再陌生,但是如何讓機器更進一步創造出有結構的複雜物件 (例如:圖片、文句) 仍是一大挑戰。用生成式對抗網路,機器已經可以畫出以假亂真的人臉,也可以根據一段敘述文字,自己畫出對應的圖案,甚至還可以畫出二次元人物頭像 (左邊的動畫人物頭像就是機器自己生成的)。本課程希望能帶大家認識生成式對抗網路這個深度學習最前沿的技術。
There are certain Java APIs that we use in our everyday programming. However, we may not be aware of their notorious performance side effects. In this session, we are going to discuss a few common Java APIs and their performance impact on your application.
A grand challenge of AI has fallen - a decade earlier than "experts" predicted. But should we care?
What made AlphaGo, the AI built by DeepMind, so unique?
Dive into AlphaGo's system of deep learning, evaluation, and search algorithms that combined to defeat the reigning Go world champion, and draw your own conclusions.
Tim Riser presented an analysis of "Mastering the Game of Go with Deep Neural Networks & Tree Search", a paper by Google DeepMind to the Boston/Cambridge chapter of Papers We Love, a computer science discussion group on June 28, 2016.
Slides for a short lecture (~1 hr) on the foundations of the Alpha Go model developed by Google. Intended for people with little technical background, but with basic familiarity with Deep Learning.
Jose Selvi - Side-Channels Uncovered [rootedvlc2018]RootedCON
En los últimos años, el término "side-channel" a pasado de ser un concepto únicamente conocido en el sector de hardware hacking a ser un término popular dentro de la industria debido a las vulnerabilidades que se han ido publicando. CRIME, BREACH o FIESTA son claros ejemplos de vulnerabilidades que explotan un side-channel en TLS. Más recientemente, también hemos visto vulnerabilidades empleando este mismo concepto en procesadores, como Spectre o Meltdown.
En esta charla, repasaremos el concepto de "side-channel" y haremos un repaso por las diferentes vulnerabilidades que se han ido publicando a lo largo de estos últimos años, explicando en que consisten y que limitaciones tienen.
Machine Learning Exposed - James Weaver - Codemotion Amsterdam 2017Codemotion
The term "machine learning" is increasingly bandied about in corporate settings and cocktail parties, but what is it, really? In this session we'll answer that question, providing an approachable overview of machine learning concepts, technologies, and use cases. We'll then take a deeper dive into machine learning topics such as supervised learning, unsupervised learning, and deep learning. We'll also survey various machine learning APIs and platforms. You'll be the hit of your next party when you're able to express the near-magical inner-workings of artificial neural networks!
In this session, we will be discussing major outages that happened in major enterprises. We will be analyzing the actual thread dumps, heap dumps, GC logs, and other artifacts captured at the time of the problem. After this session, troubleshooting CPU spikes, OutOfMemoryError, response time degradations, network connectivity issues, application unresponsiveness may not stump you.
Accelerating Incident Response To Production OutagesTier1 app
In this webinar, following topics were discussed
1) Production outages that happened in major enterprises in their JVM applications.
2) Analyzing the actual thread dumps, heap dumps, GC logs, and other artifacts captured at the time of the problem.
Unsupervised Computer Vision: The Current State of the ArtTJ Torres
This presentation was originally given at a styling research presentation at Stitch Fix, where I talk about some of the recent progress in the field of unsupervised deep learning methods for image analysis. It includes descriptions of Variational Autoencoders (VAE), Generative Adversarial Networks (GAN), their hybrid (VAE/GAN), Generative Moment Matching Networks (GMMN), and Adversarial Autoencoders.
최근 IT업계에서는 딥러닝에 대한 큰 활약이 이슈화 되고 있습니다.
이와 같은 트렌드에 따라가고 싶지만 선형대수 등의 수학적 배경지식 습득부터 시작하여, 딥러닝의 원리와 주요 개념들을 이해 후에 실험을 시도하기에는 많은 시간과 노력이 필요합니다.
그러나 기존의 유용한 딥러닝 오픈소스를 활용한다면 어렵지 않게 딥러닝을 맛볼 수 있습니다.
본 발표는 수학적인 설명을 최대한 배제하고, 오픈소스 툴인 theano, pylearn2를 활용한 예제에 대해 설명하려고 합니다. 추가로 필요할 코드들도 소개하려고 합니다.
그리고 word2vec 를 활용하여, 자연어 처리에 딥러닝을 적용하는 사례를 다루려고 합니다.
주제가 학문적이고 이론적이기 때문에 발표가 부담되지만, 최대한 개념적으로 설명하여 실험을 쉽게 따라 할 수 있도록 돕고자 합니다.
오픈소스 툴들의 문서화가 잘 되어있지만, 저 또한 처음 접했을 때는 어려움이 있었기 때문에 딥러닝을 시작해보려는 분들에게 도움이 될 듯합니다.
컴퓨터가 딥러닝하는 동안 틈틈이 이론공부 하시면 좋겠네요.
Atari Game State Representation using Convolutional Neural Networksjohnstamford
I recently gave a talk to some MSc Machine Learning students at De Montfort University about the project I did for my MSc. The work included looking at feature extraction from game screens using the Arcade Learning Environment and Convolutional Neural Networks (CNN).
The work was planned to investigate if the costly nature Q-Learning could be offset by the use of a trained system using 'expert' data. The system uses the same technology as used by Deepmind in their 2013 paper.
What I learned about IoT Security ... and why it's so hard!Christoph Engelbert
Smart devices taking over our living rooms, our bed rooms, and, in general, our life. It has never been more important to build secure devices, but most companies seem to fail, and they fail hard. We (only) build systems for farms and barns, and still, I wanted security for Cow-stumers.
Building a mostly secure system is fairly simple. There is a good set of low-hanging fruits. Building a really locked down system is tough, though. Much harder than expected. Here is what I learned.
Deep Learning in Python with Tensorflow for FinanceBen Ball
Speaker: Ben Ball
Abstract: Python is becoming the de facto standard for many machine learning applications. We have been using Python with deep learning and other ML techniques, with a focus in prediction and exploitation in transactional markets. I am presenting one of our implementations (A dueling double DQN - a class of Reinforcement Learning algorithm) in Python using TensorFlow, along with information and background around the class of deep learning algorithm, and the application to financial markets we have employed. Attendees will learn how to implement a DQN using Tensorflow, and how to design a system for deep learning for solving a wide range of problems. The code will be available on github for attendees.
Bio: Ben is a believer in making a career out of what you love. He is inspired by the joining of excellent technology and research and likes building software that is easy to use, but does amazing things. His work has spanned 15 years, with a dual focus in AI Software Engineering and Algorithmic Trading. He is currently working as the CTO of http://prediction-machines.com
Video of the presentation:
https://engineers.sg/video/deep-learning-with-python-in-finance-singapore-python-user-group--1875
Descripción de la plática:
CoreML es el puente entre iOS y Machine Learning, pero con ciertas limitantes. Explotaremos el potencial que tiene CoreML, responderemos las siguientes preguntas:
- ¿Cómo podríamos ir más allá de los límites de CoreML?
- ¿Cómo puede nuestra App aprender de la experiencia?
生成式對抗網路 (Generative Adversarial Network, GAN) 顯然是深度學習領域的下一個熱點,Yann LeCun 說這是機器學習領域這十年來最有趣的想法 (the most interesting idea in the last 10 years in ML),又說這是有史以來最酷的東西 (the coolest thing since sliced bread)。生成式對抗網路解決了什麼樣的問題呢?在機器學習領域,回歸 (regression) 和分類 (classification) 這兩項任務的解法人們已經不再陌生,但是如何讓機器更進一步創造出有結構的複雜物件 (例如:圖片、文句) 仍是一大挑戰。用生成式對抗網路,機器已經可以畫出以假亂真的人臉,也可以根據一段敘述文字,自己畫出對應的圖案,甚至還可以畫出二次元人物頭像 (左邊的動畫人物頭像就是機器自己生成的)。本課程希望能帶大家認識生成式對抗網路這個深度學習最前沿的技術。
There are certain Java APIs that we use in our everyday programming. However, we may not be aware of their notorious performance side effects. In this session, we are going to discuss a few common Java APIs and their performance impact on your application.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteGoogle
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-pilot-review/
AI Pilot Review: Key Features
✅Deploy AI expert bots in Any Niche With Just A Click
✅With one keyword, generate complete funnels, websites, landing pages, and more.
✅More than 85 AI features are included in the AI pilot.
✅No setup or configuration; use your voice (like Siri) to do whatever you want.
✅You Can Use AI Pilot To Create your version of AI Pilot And Charge People For It…
✅ZERO Manual Work With AI Pilot. Never write, Design, Or Code Again.
✅ZERO Limits On Features Or Usages
✅Use Our AI-powered Traffic To Get Hundreds Of Customers
✅No Complicated Setup: Get Up And Running In 2 Minutes
✅99.99% Up-Time Guaranteed
✅30 Days Money-Back Guarantee
✅ZERO Upfront Cost
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfJay Das
With the advent of artificial intelligence or AI tools, project management processes are undergoing a transformative shift. By using tools like ChatGPT, and Bard organizations can empower their leaders and managers to plan, execute, and monitor projects more effectively.
A Comprehensive Look at Generative AI in Retail App Testing.pdfkalichargn70th171
Traditional software testing methods are being challenged in retail, where customer expectations and technological advancements continually shape the landscape. Enter generative AI—a transformative subset of artificial intelligence technologies poised to revolutionize software testing.
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
How Recreation Management Software Can Streamline Your Operations.pptxwottaspaceseo
Recreation management software streamlines operations by automating key tasks such as scheduling, registration, and payment processing, reducing manual workload and errors. It provides centralized management of facilities, classes, and events, ensuring efficient resource allocation and facility usage. The software offers user-friendly online portals for easy access to bookings and program information, enhancing customer experience. Real-time reporting and data analytics deliver insights into attendance and preferences, aiding in strategic decision-making. Additionally, effective communication tools keep participants and staff informed with timely updates. Overall, recreation management software enhances efficiency, improves service delivery, and boosts customer satisfaction.
top nidhi software solution freedownloadvrstrong314
This presentation emphasizes the importance of data security and legal compliance for Nidhi companies in India. It highlights how online Nidhi software solutions, like Vector Nidhi Software, offer advanced features tailored to these needs. Key aspects include encryption, access controls, and audit trails to ensure data security. The software complies with regulatory guidelines from the MCA and RBI and adheres to Nidhi Rules, 2014. With customizable, user-friendly interfaces and real-time features, these Nidhi software solutions enhance efficiency, support growth, and provide exceptional member services. The presentation concludes with contact information for further inquiries.
Cyaniclab : Software Development Agency Portfolio.pdfCyanic lab
CyanicLab, an offshore custom software development company based in Sweden,India, Finland, is your go-to partner for startup development and innovative web design solutions. Our expert team specializes in crafting cutting-edge software tailored to meet the unique needs of startups and established enterprises alike. From conceptualization to execution, we offer comprehensive services including web and mobile app development, UI/UX design, and ongoing software maintenance. Ready to elevate your business? Contact CyanicLab today and let us propel your vision to success with our top-notch IT solutions.
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTier1 app
Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Unleash Unlimited Potential with One-Time Purchase
BoxLang is more than just a language; it's a community. By choosing a Visionary License, you're not just investing in your success, you're actively contributing to the ongoing development and support of BoxLang.
6. 6
Atari Breakout in OpenAI Gym
import gym
env = gym.make("ALE/Breakout-v5", render_mode="human")
state, info = env.reset()
for index in range(1000):
action = env.action_space.sample() # action by random or policy
state, reward, terminated, truncated, info = env.step(action)
if terminated or truncated:
state, info = env.reset()
env.close()
https://www.gymlibrary.dev/
https://gymnasium.farama.org/
7. 7
State/Action/Reward in Atari Breakout
State:
●
(210, 160, 3) - image
Action:
●
0 - NO OP
●
1 - FIRE
●
2 - RIGHT
●
3 - LEFT
Reward:
●
Red - 7 points
●
Orange - 7 points
●
Yellow - 4 points
●
Green - 4 points
●
Aqua - 1 point
●
Blue - 1 point
https://www.gymlibrary.dev/
https://gymnasium.farama.org/
8. 8
From One Game to All The Games in Atari
https://www.deepmind.com/blog/agent57-outperforming-the-human-atari-benchmark
9. 9
A Journey to Artificial General Intelligence
https://www.assemblyai.com/blog/reinforcement-learning-with-deep-q-learning-explained/
https://www.deepmind.com/blog/agent57-outperforming-the-human-atari-benchmark
DQN/2015
R2D2/2019
NGU/2019
Agent57/2020
10. 10
OpenAI Gym Taxi-v3 : State/Action/Reward
State:
●
Number of Variable : 1
●
Range of Variable : [1, 500]
●
25 taxi positions x 5 passenger positions x 4 destination locations
Action:
●
0 : move south
●
1 : move north
●
2 : move east
●
3 : move west
●
4 : pickup passenger
●
5 : drop off passenger
Reward:
●
+20 : delivering passenger
●
-10 : pickup/dropoff illegally
●
-1 : per step unless other rewards is triggered
https://www.gymlibrary.dev/environments/toy_text/taxi/
14. 14
Deep Q Network (DQN) Architecture (1/2)
Ref : Human-level control through deep reinforcement learning
15. 15
Deep Q Network (DQN) Architecture (2/2)
Ref : Massively Parallel Methods for Deep Reinforcement Learning
16. 16
Deep Q Learning (with experience replay and dual networks)
1. initialize replay memory
5. store transition in replay memory
6. get batch from replay memory
2. initialize main network
3. initialize target network
4. epsilon greedy policy from main network
7. calculate error between two networks
8. synchronize two networks
Ref : Human-level control through deep reinforcement learning
17. 17
Deep Q Network (DQN) on Breakout
Artificial Intelligence and the Future - Demis Hassabis/DeepMind
https://youtu.be/zYII3AOSgo8?t=2236
18. 18
Deep Q Network (DQN) Benchmark
Ref : Human-level control through deep reinforcement learning
19. 19
Four Tough Games in Atari
Pitfall Solaris Skiing Montezuma’s Revenge
Problems : long-term credit assignment and exploitation/exploration tradeoff
Solutions : intrinsic motivation, meta-controller, short-term/episodic memory, distributed agents, etc.
https://www.deepmind.com/blog/agent57-outperforming-the-human-atari-benchmark
24. 24
Reinforcement Learning at DeepMind
https://analyticsindiamag.com/all-hail-the-king-of-reinforcement-learning-deepmind/
25. 25
Mastering Go at DeepMind
https://analyticsindiamag.com/all-hail-the-king-of-reinforcement-learning-deepmind/
26. 26
A Journey to Artificial General Intelligence
https://www.deepmind.com/blog/muzero-mastering-go-chess-shogi-and-atari-without-rules
https://www.youtube.com/watch?v=lVMgxtm5L-U
28. 28
AlphaGo Fan/Lee/Master
●
European Go Champion Fan Hui — 5:0
●
South Korean professional Go player Lee Sedol — 4:1
●
Online games with players from China/Korea/Japan — 60:0
●
Chinese professional Go player Ke Jie — 3:0
https://www.youtube.com/watch?v=lVMgxtm5L-U
https://www.youtube.com/watch?v=LX8Knl0g0LE
29. 29
AlphaGo Fan/Lee/Master
●
European Go Champion Fan Hui — 5:0
●
South Korean professional Go player Lee Sedol — 4:1
●
Online games with players from China/Korea/Japan — 60:0
●
Chinese professional Go player Ke Jie — 3:0
https://www.youtube.com/watch?v=lVMgxtm5L-U
https://www.youtube.com/watch?v=WXuK6gekU1Y
30. 30
AlphaGo Inputs and Policy/Value Networks
https://www.slideshare.net/ckmarkohchang/alphago-in-depth
31. 31
AlphaGo Monte Carlo Tree Search
https://www.slideshare.net/ckmarkohchang/alphago-in-depth
33. 33
AlphaZero Network for Chess
Ref: Acquisition of Chess Knowledge in AlphaZero
AlphaGo
• Two networks: policy network and value network
• Conv/ReLu-based layer structure
AlphaZero
• One network with two heads: policy and value
• ResNet-based layer structure
34. 34
AlphaGo Zero Performance Benchmark
https://thirdeyedata.ai/how-to-build-your-own-alphazero-ai-using-python-and-keras/
35. 35
MuZero Training Process
h: representation
f: prediction
g: dynamics
Ref: Mastering Atari, Go, chess and shogi by planning with a learned model