Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Steve Omohundro, profile picture
Uploaded bySteve Omohundro
PPTX, PDF4,470 views

The Future of AI is Generative not Discriminative 5/26/2021

The document discusses the evolution of artificial intelligence towards generative models, emphasizing their potential over discriminative models. It highlights technological advancements and the importance of generative AI in driving innovation and creativity in various fields. The text outlines the capabilities of generative models such as GANs and VAE, and their role in future AI developments including empathy and co-creation.

Technology◦
Downloaded 182 times
The Future of AI is Generative (not Discriminative) Steve Omohundro
z Observed Variables: Hidden (Latent) Variables: Discriminative Models Generative Models z x x Bayesian: p(z|x) p(z) and p(x|z)
The Future of AI is Generative 1. The deep learning revolution has been discriminative 2. We now have brain-level hardware but not software 3. Generative 1.0: GANs, VAEs, GPT-3, etc. 4. Generative NG: Causality, Theory of Mind, Reasoning 5. AI Empathy and Human/AI co-creation 6. Will be the driver for innovative AI business and society
2012 AlexNet: The Deep Learning Revolution! https://paperswithcode.com/paper/imagenet-classification-with-deep https://www.wired.com/2013/03/google-hinton/ Google purchased for $44 million! https://www.wired.com/story/secret-auction-race-ai-supremacy-google-microsoft-baidu/
GPU Hardware and Big ImageNet Data • NVIDIA GeForce GTX 580 • 1.6 TeraFLOPS, $499.99 https://www.techpowerup.com/gpu-specs/geforce-gtx-580.c270 ImageNet LSVRC-2010 1.2 million images https://dl.acm.org/doi/10.1145/3065386
2013 DeepMind DQN Plays 7 Atari Games https://towardsdatascience.com/dqn-part-1-vanilla-deep-q-networks-6eb4a00febfb https://arxiv.org/abs/1312.5602 Google purchased for $600 million! https://www.cnbc.com/2020/12/17/deepmind-lost-649-million-and-alphabet-waived-a-1point5-billion-debt-.html
PwC: AI will create $15.7 trillion/year by 2030 https://www.pwc.com/gx/en/issues/data-and-analytics/publications/artificial-intelligence-study.html
NVIDIA DGX A100 5 PetaFLOPS $199,000 https://www.nvidia.com/en-us/data-center/dgx-a100/
Human Brain: 1 PetaFLOP https://www.openphilanthropy.org/brain-computation-report
Problems with Discriminative Models… • Need lots of training data • Have adversarial examples • Don’t generalize beyond the training set • Inscrutable representations • Knowledge is hard to transfer • But can be applied by anyone, anywhere, to anything!
Generative Models 1.0 • Bayesian Networks • Generative Adversarial Networks (GAN) • Variational Autoencoders (VAE) • Autoregressive Models • Normalizing Flows • SurVAE Networks • …
NVIDIA Stylegan2 https://github.com/NVlabs/stylegan2
Transformer+VQGAN High-res Image Synthesis https://compvis.github.io/taming-transformers/
GPT-3 Poetry https://humanise.ai/blog/ai-writes-poetry/
DALL-E Generates Images from Text https://openai.com/blog/dall-e/
More DALL-E: https://openai.com/blog/dall-e/
Generative Models: The Next Generation • Causality • Simulation-based • Generative process models reality • Semantic and explainable • Composable • Transferable • Metalearning • Creative Synthesis
Will more red or yellow blocks fall if bumped? https://www.pnas.org/content/110/45/18327
Human Cognitive Core 1. Objects and mechanical interactions 2. Agents and goal-directed actions 3. Sets and numbers 4. Geometric relationships Infants, Adults, Cultures, Primates - Spelke and Kinzler https://www.today.com/shop/best-gifts-babies-our- 2017-holiday-gift-guide-t118481 https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467- 7687.2007.00569.x
Google Dreamer V2 https://github.com/danijar/dreamerv2
Probabilistic Programming • E.g. Pyro built on PyTorch • Simulate probabilistic agents • Probabilistic Inference for modeling and reasoning • Program synthesis (e.g. DreamCoder) to generate new programs https://arxiv.org/abs/2006.08381v1 https://arxiv.org/abs/1604.00289
PPL Theory of Mind – AI Empathy • Reasoning about reasoning • Nested inference • Inverse Reinforcement Learning • AI Teachers – model student • AI Counselors – model dysfunction • AI Co-Creators – model intent https://www.sciencedirect.com/science/article/abs/pii/S1389041713000387 https://www.psychalive.org/empathy-can-help-us-right-now/
Generative Models for AI Innovation • Generative AI models will drive rapid innovation • Shift investment and business • Shift social and entertainment • Require generative models to handle rapid change • Generative models can empathize and co-create • A glorious human future much bigger than current projections! https://www.surfertoday.com/environment/sunrise-and- sunset-interesting-facts-about-the-golden-hour