NextCollab Hallucinations 202311280 v1.pptx
•Download as PPTX, PDF•
0 likes•23 views
URL: https://www.nextcolabs.io/decodinghallucination URL: http://pasky.or.cz/dev/brmson/papers-watson/p1877-spangler.pdf URL: https://youtu.be/ARf0WyFau0A?si=66kbwPRbP2hlW1ro
Recommended
Recommended
More Related Content
Similar to NextCollab Hallucinations 202311280 v1.pptx
Similar to NextCollab Hallucinations 202311280 v1.pptx (20)
More from ISSIP
More from ISSIP (16)
Recently uploaded
Recently uploaded (20)
NextCollab Hallucinations 202311280 v1.pptx
- 2. Jim Spohrer is a Silicon Valley-based Advisor to industry, academia, governments, startups and non-profits on topics of AI upskilling, innovation strategy, and win- win service in the AI era. Most recently with a consulting team working for a top 10 market cap global company, he contributed to a strategic plan for a globally connected AI Academy for achieving rapid, nation-scale upskilling with AI. With the US National Academy of Engineering, he co-led a 2022 workshop on “Service Systems Engineering in the Era of Human-Centered AI” to improve well-being. Jim is a retired IBM Executive since July 2021, and previously directed IBM’s open- source Artificial Intelligence developer ecosystem effort, was CTO IBM Venture Capital Group, co-founded IBM Almaden Service Research, and led IBM Global University Programs. In the 1990’s at Apple Computer, as a Distinguished Engineer Scientist and Technologist, he was executive lead on next generation learning platforms. In the 1970’s, after his MIT BS in Physics, he developed speech recognition systems at Verbex (Exxon) before receiving his Yale PhD in Computer Science/AI. In 1989, prior to joining Apple, he was a visiting scholar at the University of Rome, La Sapienza advising doctoral students working on AI and Education dissertations. With over ninety publications and nine patents, he received the Christopher Lovelock Career Contributions to the Service Discipline award, Gummesson Service Research award, Vargo and Lusch Service-Dominant Logic award, Daniel Berg Service Systems award, and a PICMET Fellow for advancing service science. Jim was elected and previously served as Linux Foundation AI & Data Technical Advisory Board Chairperson and ONNX Steering Committee Member (2020-2021). Today, he is a UIDP Senior Fellow for contributions to industry-university collaborations, and a member of the Board of Directors of the International Society of Service Innovation (ISSIP) and ServCollab. Jim Spohrer, Advisor Retired Industry Executive (Apple, IBM) UIDP Senior Fellow Board of Directors, ServCollab Board of Directors, ISSIP.org Changemaker Priorities 1. Service Innovation 2. Upskilling with AI 3. Future Universities 4. Geothermal Energy 5. Poverty Reduction 6. Regional Development Competitive Parity Technologies 1. AI & Robotics 2. Digital Twins 3. Open Source 4. AR/VR/XR 5. Geothermal 6. Learning Platforms
- 3. Icons of AI Progress • 1955-1956: Dartmouth Workshop organized by: • Two early career faculty • John McCarthy (Dartmouth, later Stanford) • Marvin Minsky (MIT) • Two senior industry scientists • Claude Shannon (Bell Labs) • Nathan Rochester (IBM) • 1997: Deep Blue (IBM) - Chess • 2011: Watson Jeopardy! (IBM) • 2016: AlphaGo (Google DeepMinds) • 2017: All you need is attention (Google) - Transformers • Attention heads (working memory) to predict what comes next • 2018: AlphaFold (Google DeepMinds) • 2020: Language models are few-shot learners (OpenAI) • 2022: DALL-E 2 & ChapGPT (OpenAI) • 2022: Constitutional AI (Anthropic) – “Behave yourself!” • 2023: New Bing+ (Microsoft) & GPT-4 (OpenAI) • 2023: ?Q* (OpenAI) - to reduce hallucinations (generation stage errors) 11/30/2023 pohrer
- 5. MORE SPOHRER USE CASES: https://service-science.info/archives/6521 Generate Architectures Impressive, but imperfect
- 7. 1960 1980 2000 2020 2040 2060 2080 Time
- 8. 1960 1980 2000 2020 2040 2060 2080 $1,000,000,000,000 (Trillion) $1,000,000 (Million) $1,000,000,000 (Billion) $1,000 (Thousand) $1 Cost
- 9. 1960 1980 2000 2020 2040 2060 2080 $1,000,000,000,000 (Trillion) $1,000,000 (Million) $1,000,000,000 (Billion) $1,000 (Thousand) $1 Cost of Computation (Diagonals)
- 10. 1960 1980 2000 2020 2040 2060 2080 $1,000,000,000,000 (Trillion) $1,000,000 (Million) $1,000,000,000 (Billion) $1,000 (Thousand) $1 Cost of Computation (Diagonals) Note: Adjust Kilo and Mega scales slightly to fit data better (early days – more cost – learning curve).
- 11. 1960 1980 2000 2020 2040 2060 2080 $1,000,000,000,000 (Trillion) $1,000,000 (Million) $1,000,000,000 (Billion) $1,000 (Thousand) $1 GDP/Employee Trend Estimating Knowledge Worker Productivity Based on USA Historical Data Year Value 1960 $10K 1980 $33K 2000 $78K 2020. $151K 2023 $169K Cost of computation goes down by 1000x every 20 years (left to right diagonals), driving knowledge worker productivity up.
- 12. 2September 2018 / © 2018 IBM Corporation Petaflops = 1,000,000,000,000,000 or a million billion = 10 ** 15 Megaflops = 1,000,000 = million = 10 ** 6 Gigaflops = 1,000,000,000 = billion = 10 ** 9 One of the AI Super Computers in the World, = 13 MegaWatts of Power (HOT!)
- 13. 3September 2018 / © 2018 IBM Corporation Exascale = 1,000,000,000,000,000,000 or a billion billion = 10 ** 18 Megaflops = 1,000,000 = million = 10 ** 6 Gigaflops = 1,000,000,000 = billion = 10 ** 9 Human Brain = 20 Watts (COOL!)
- 14. “NorthPole Chip” Far less energy.
- 15. Impressive, but imperfect Physical realm: Energy Social realms: Digital property theft
- 16. Optimistic Realistic Knowing Doing How to keep up with accelerating change? Follow a diverse collection of people… make up dimensions meaningful to you! Sadly for me… my brain is biased into thinking I can understand older, white, males the best… maybe AI can help overcome!
Editor's Notes
- Here is the explanation that kids and everyone I spoke with understands. Explaining Generative AI to Nearly Anyone Have you ever used a calculator to add up numbers? Generative AI is like a calculator - you ask it to please do something - and you will always get an answer, usually pretty quickly. However, depending on the question you ask, the answer is only correct one day a week.. Four days a week the answer is not correct, but made up - and may sound very creative or very convincing. Two days a week, the calculator says - sorry I cannot help you - again, depending on what you ask the calculator to do. What would you do with a calculator like this - only giving a correct answer some of the time? Some people would throw it away. However, some people are very happy to use it to make-up creative answers to hard questions - even if it only helps them some of the time. Especially for pictures, stories, poems, and art work, like this one picture of a monkey and a parrot - some people find these kind of calculators helpful. I asked a generative AI calculator to do something for me once - I asked for a picture, and asked in this funny way: "Please create an image of a library, and in the library is monkey using a typewriter with a stochastic parrot dictating to the monkey." It made a pretty funny picture, wouldn't you say? Also, generative AI calculators are good at making up answers very, very fast... ...and sometimes they are right, but most of the time they are either creatively wrong or don't give an answer. Sadly, some people think the calculator is really smart - because it can give a correct answer to a hard question, some of the time - and do it really, really fast. The calculator has fooled them into thinking it is really, really smart - when in fact, it is not smart. If it was smart, it would not make so many mistakes. So that works most of the time. However, if they do not know what a calculator is - I was talking with a 3 year the other day whose parent works for tech company in Silicon Valley, and had asked me to explain AI to his daughter - I basically just substitute "a magic genie" for the calculator, and talk about a genie who gives you your wishes, but makes a lot of mistakes - so you have to be careful what you wish for. To explain why it is "so good" sometimes, I have to explain N-Gram Statistics - which is a bit harder, but if they do not understand N-grams, I just say: How does the generative AI calculator work? Use your imagination to imagine a computer keyboard/or typewriter that had whole words and whole sentences on millions of keys, so when you push a key at random, you get something that makes sense. The more examples you show this magic keyboard over time, the more keys the keyboard gets - billions or even trillions of keys, and the better the keyboard gets at putting a big key in the middle that says "PUSH ME NEXT PLEASE". Using this magic keyboard/typewriter even a monkey or a parrot could sometimes create pretty amazing things. Inside a computer words, pictures, videos are just represented as 0’s and 1’s – as big big numbers. To understand this we can play the game called 20 questions, which can be used to identify anything by asking enough yes and no questions. Are you thinking about an animal? Yes or No? Yes, Does the animal have a furry tale? Yes, Does the animal like to ear bananas? Yes. Is the animal a monkey? Not sure if this is helpful, but thought I would share it.
- What is really going on in this progress? Starting at the 2017 paper introducing the transformer architecture. The big insights are once you have a general purpose learning architecture (with enough scale), you can then get by with a transformer architecture (attention heads as working memory to predict what comes next) – as you get better at predicting what comes next (with enough scale), you being to see emergence (the ability to get new-ish capabilities) by appropriate prompt engineering (few-shot learning) – his happens where the scale model has developed a good compressed version of some pattern of reality that is a good predictor. Getting these compressed models of reality that are good predictors under certain conditions is both an efficiency and a trap. Efficiency is the world is stable. Trap is the world is changing rapidly, and the compressed version is no longer valid. Predicting what comes next is a kind of un-supervised learning – what comes next is a property of the data/world and does not require labeling. Once you have the predictor working well, and the compressed models of the data/world working well, then the emergence happens for newish-capabilities few-shot learning with appropriate prompts. This is like learning by being told. It has to be rewarded as many times as possible, when it makes use of the ”value statements in the constitution” – reinforncing the in-group bias. Because LLMs have all the data (lots and lots of data) from different groups with difference values/biases/belief systems, it is important to get the addition of a constitution and lots of rewards, or else …. Or else, bad behaviour, hallucinations, and other strange phenomena will occur. Is this where schizophrenia comes from in people? A clash of belief systems, and what should get rewarded. Looking for truth, but without the flexibility to adapth. Without the anchor principles that can make life a peaceful journey. Perhaps. 2020 – Few shot learners implies emergence. Emergence just means ”good enough compressed models of reality/diverse data at scale” that a short prompt can bias the prediction of what comes next. 2022 – Constitutional AI implies value systems/belief systems matter for socially acceptable behavior. A learner must be conditioned/repeatedly rewarded for pro-social behaviors. DALL-E URL: https://openai.com/product/dall-e-2 ChatGPT URL: https://chat.openai.com GPT-4 URL: https://openai.com/research/gpt-4 AlphaFold: URL https://en.wikipedia.org/wiki/AlphaFold Microsoft Bing: URL: https://en.wikipedia.org/wiki/Microsoft_Bing URL: https://en.wikipedia.org/wiki/History_of_artificial_intelligence URL: http://www.businessinsider.com/infographic-ai-effect-on-economy-2017-8 Today’s infographic comes from the Extraordinary Future 2017, a new conference in Vancouver, BC that focuses on emerging technologies such as AI, autonomous vehicles, fintech, and block http://extraordinaryfuture.com/e/extraordinary-future-2017-71chain tech. Nathaniel Rochester: In 1948, Rochester moved to IBM where he designed the IBM 701, the first general purpose, mass-produced computer. He wrote the first symbolic assembler, which allowed programs to be written in short, readable commands rather than pure numbers or punch codes. BiblioV2017 Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin (2017) Attention Is All You Need. URL: https://arxiv.org/abs/1706.03762v5 Quotes: "The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data."; BiblioB2020 Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, Dario Amodei (2020) Language Models are Few-Shot Learners. URL: https://arxiv.org/abs/2005.14165v4 Quotes: "Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting. For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model. GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic. At the same time, we also identify some datasets where GPT-3's few-shot learning still struggles, as well as some datasets where GPT-3 faces methodological issues related to training on large web corpora. Finally, we find that GPT-3 can generate samples of news articles which human evaluators have difficulty distinguishing from articles written by humans. We discuss broader societal impacts of this finding and of GPT-3 in general."; BiblioB2022 Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, Jared Kaplan (2022) Constitutional AI: Harmlessness from AI Feedback URL: https://arxiv.org/abs/2212.08073 Quotes: "As AI systems become more capable, we would like to enlist their help to supervise other AIs. We experiment with methods for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs. The only human oversight is provided through a list of rules or principles, and so we refer to the method as 'Constitutional AI'. The process involves both a supervised learning and a reinforcement learning phase. In the supervised phase we sample from an initial model, then generate self-critiques and revisions, and then finetune the original model on revised responses. In the RL phase, we sample from the finetuned model, use a model to evaluate which of the two samples is better, and then train a preference model from this dataset of AI preferences. We then train with RL using the preference model as the reward signal, i.e. we use 'RL from AI Feedback' (RLAIF). As a result we are able to train a harmless but non-evasive AI assistant that engages with harmful queries by explaining its objections to them. Both the SL and RL methods can leverage chain-of-thought style reasoning to improve the human-judged performance and transparency of AI decision making. These methods make it possible to control AI behavior more precisely and with far fewer human labels.";
- URL: http://www-formal.stanford.edu/jmc/history/dartmouth/dartmouth.html URL: https://en.wikipedia.org/wiki/Nathaniel_Rochester_(computer_scientist) BiblioM1955 McCarthy J, Minsky ML, Rochester N, Shannon CE (1955) A proposal for a summer workshop on Artificial Intelligence. URL: http://jmc.stanford.edu/articles/dartmouth/dartmouth.pdf Quotes: "We propose that a 2 month, 10 man study of artificial intelligence be carried out during the summer of 1956 at Dartmouth College in Hanover, New Hampshire. The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer. The following are some aspects of the artificial intelligence problem: (1) Automatic Computers, (2) How Can a Computer be Programmed to Use a Language, (3) Neuron Nets, (4) Theory of the Size of a Calculation, (5) Self_Improvement, (6) Abstraction, (7) Randomness and Creativity, (8) "; "Estimated Expenses 6 salaries of 1200 - $7200 2 salaries of 700 - 1400 8 traveling and rent expenses averaging - 2400 Secretarial and organizational expense - 850 Additional traveling expenses - 600 Contingencies - 550 Total - $13,500"; BiblioO2023 OpenAI (2023) GPT-4 Technical Report. Via_OpenAI_SM_JCS. URL: https://cdn.openai.com/papers/gpt-4.pdf Quotes: "Abstract - We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-world scenarios, GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers. GPT-4 is a Transformer- based model pre-trained to predict the next token in a document. The post-training alignment process results in improved performance on measures of factuality and adherence to desired behavior. A core component of this project was developing infrastructure and optimization methods that behave predictably across a wide range of scales. This allowed us to accurately predict some aspects of GPT-4’s performance based on models trained with no more than 1/1,000th the compute of GPT-4.";
- Here is the explanation that kids and everyone I spoke with understands. Explaining Generative AI to Nearly Anyone Have you ever used a calculator to add up numbers? Generative AI is like a calculator - you ask it to please do something - and you will always get an answer, usually pretty quickly. However, depending on the question you ask, the answer is only correct one day a week.. Four days a week the answer is not correct, but made up - and may sound very creative or very convincing. Two days a week, the calculator says - sorry I cannot help you - again, depending on what you ask the calculator to do. What would you do with a calculator like this - only giving a correct answer some of the time? Some people would throw it away. However, some people are very happy to use it to make-up creative answers to hard questions - even if it only helps them some of the time. Especially for pictures, stories, poems, and art work, like this one picture of a monkey and a parrot - some people find these kind of calculators helpful. I asked a generative AI calculator to do something for me once - I asked for a picture, and asked in this funny way: "Please create an image of a library, and in the library is monkey using a typewriter with a stochastic parrot dictating to the monkey." It made a pretty funny picture, wouldn't you say? Also, generative AI calculators are good at making up answers very, very fast... ...and sometimes they are right, but most of the time they are either creatively wrong or don't give an answer. Sadly, some people think the calculator is really smart - because it can give a correct answer to a hard question, some of the time - and do it really, really fast. The calculator has fooled them into thinking it is really, really smart - when in fact, it is not smart. If it was smart, it would not make so many mistakes. So that works most of the time. However, if they do not know what a calculator is - I was talking with a 3 year the other day whose parent works for tech company in Silicon Valley, and had asked me to explain AI to his daughter - I basically just substitute "a magic genie" for the calculator, and talk about a genie who gives you your wishes, but makes a lot of mistakes - so you have to be careful what you wish for. To explain why it is "so good" sometimes, I have to explain N-Gram Statistics - which is a bit harder, but if they do not understand N-grams, I just say: How does the generative AI calculator work? Use your imagination to imagine a computer keyboard/or typewriter that had whole words and whole sentences on millions of keys, so when you push a key at random, you get something that makes sense. The more examples you show this magic keyboard over time, the more keys the keyboard gets - billions or even trillions of keys, and the better the keyboard gets at putting a big key in the middle that says "PUSH ME NEXT PLEASE". Using this magic keyboard/typewriter even a monkey or a parrot could sometimes create pretty amazing things. Inside a computer words, pictures, videos are just represented as 0’s and 1’s – as big big numbers. To understand this we can play the game called 20 questions, which can be used to identify anything by asking enough yes and no questions. Are you thinking about an animal? Yes or No? Yes, Does the animal have a furry tale? Yes, Does the animal like to ear bananas? Yes. Is the animal a monkey? Not sure if this is helpful, but thought I would share it.
- BiblioR2018 Rouse WB, Spohrer JC (2018) Automating versus augmenting intelligence. Journal of Enterprise Transformation, 8:1-2, 1-21, DOI: 10.1080/19488289.2018.1424059. URL: https://service-science.info/wp-content/uploads/2018/08/Rouse-Spohrer-Automating-Versus-Augmenting-Intelligence-12-21-17-copy.pdf Quotes: "Abstract: This article addresses the prospects for automating intelligence versus augmenting human intelligence. The evolution of artificial intelligence (AI) is summarized, including contemporary AI and the new capabilities now possible. Functional requirements to augment human intelligence are outlined. An overall architecture is presented for providing this functionality, including how it will make deep learning explainable to decision makers. Three case studies are addressed, including driverless cars, medical diagnosis, and insurance underwriting. Paths to transformation in these domains are discussed. Prospects for innovation are considered in terms of what we can now do, what we surely will be able to do soon, and what we are unlikely to ever be able to do."; BiblioS2017 Imagination Challenge: Quantify and graph cost of digital workers and GDP per employee USA from 1960-2080. Service-Science.Info Blog Post. URL: https://service-science.info/archives/4741 Quotes: "Imagination challenge: Consider quantifying and graphing the decreasing cost of digital workers due to Moore’s Law, and increasing GDP/Employees USA from 1960 to 2080 (projected). A narrow digital worker will cost about a million dollars by 2025, and require a petascale computational system. The same digital worker will cost about a thousand dollars by 2045, and about $1 by 2065."; If you are an entrepreneur or CEO you are excited about the drop in cost of digital workers, because that means GDP per employee will continue its exponential increase. Source: http://service-science.info/archives/4741
- To understand in part why this is so, take a look at the fastest super-computer in the world… IBM helped build it for Oakridge National Labs, and it can do 200 million billion instructiosn per second using 13 Megawatts of power.
- Compate that to the human brain which can by many estimates perform a billion billion instructions per second (5x more than Summit) on a mere 20 watts, over 100,000 times less power.
- Others Lukasz_Kaiser - https://www.linkedin.com/in/lukaszkaiser/ AnatasiInTech - https://www.youtube.com/@AnastasiInTech AI Explained - https://www.youtube.com/@aiexplained-official Ross_Dawson - https://www.linkedin.com/in/futuristkeynotespeaker/ Andrew Ng - https://www.linkedin.com/in/andrewyng/ Higher Bar – avoid the hype and understand potential harms Substack: Arvind Narayanan & Sayash Kapoor - AI Snake Oil (Princeton) Substack: Gary Markus (NYU) Facebook: Ernest Davis (NYU) LinkedIn & Twitter: Stephen Wolfram Blog: Irving Wladawsky-Berger (MIT, retired IBM) Practical AI Upskilling Advice – benefits, which prompts to explore and why? Substack: Ethan Mollick (U Penn Wharton) Tracking AI Capabilities – (FOMO) which tools to try? The Neuron Daily (email AI newsletter - Purrfect): Noah Edelman & Pete Huang LinkedIn & Website: Terri Griffith (Simon Frasier) YouTube: AI Explained ArXiv publications from Google, Deepmind, Microsoft, OpenAI, Facebook/Meta, IBM, etc. Website: PapersWithCode/SOTA (and GiTHub – tracking stars on projects) Broader topics and implications (overly optimistic?) YouTube: Alan D. Thomas (Australia) (super optimistic) See his interview with Harvey Castro MD (here) – also books, and visionary uses for personalized medicine (personalized communications) YouTube: Lex Friedman (MIT) (super knowledgeable guests interviewed) Substack: Lee Nackman (retired IBM) (Win-Win Democracy and AI topics – balanced – well researched) YouTube: Kartik Gada (The ATOM) – accelerating change [accelerating change perspective – details beyond Kurzweil & Altman) Substack – Ethan Mollick: https://oneusefulthing.substack.com/p/using-ai-to-make-teaching-easier Email – The Neuron Daily – Noah Edelman & Pete Huang https://www.theneurondaily.com/p/ai-deepfakes BiblioN2023 Narayanan A, Kapoor S (2023) Evaluating LLMs is a minefield: Annotated slides from a recent talk. Sayash and Arvind from AI Snake Oil <aisnakeoil@substack.com> Wed, Oct 4, 2023 at 7:40 AM. ARVIND NARAYANAN AND SAYASH KAPOOR. OCT 4 Via_Substack. URL: https://www.aisnakeoil.com/p/evaluating-llms-is-a-minefield Quotes: "We have released annotated slides for a talk titled Evaluating LLMs is a minefield. We show that current ways of evaluating chatbots and large language models don't work well, especially for questions about their societal impact. There are no quick fixes, and research is needed to improve evaluation methods."; BiblioM2023 Marcus G (2023) Seven Lies in Four Sentences. Gary Marcus on AI. Via_Substack. URL: https://garymarcus.substack.com/p/seven-lies-in-four-sentencesQuotes: "Earlier today I learned that 2 billion people are eligible to vote in elections in 2024, in scores of elections around the globe. Tyler Cowen tried to argue yesterday in his Bloomberg column that misinformation doesn’t matter. Anybody remember Brexit?"; BiblioT2023 Thompson AD (2023) AI + medicine - with Harvey Castro MD (GPT-4, Med-PaLM 2, Carbon Health, Ambience, 311 ChatGPT call). Via_Harvey_Casto. URL: https://youtu.be/jTmkiGjrgpA Quotes: "14,365 views Jul 5, 2023 The Memo: https://lifearchitect.ai/memo/ Annotated Med-Palm 2 paper: https://lifearchitect.ai/report-card/ Harvey: https://www.harveycastromd.info/ https://www.amazon.com/stores/Harvey-... https://www.linkedin.com/in/harveycas... https://twitter.com/harveycastromd https://www.instagram.com/harveycastr... 00:00 Start! 07:23 AI by age 12:17 The Gap 14:47 Models (ChatGPT, GPT-4, Med-PaLM 2) 23:26 Use cases including non-emergency calls 35:28 Medicine vs self-driving cars 46:56 Harvey's favorite AI use case 52:56 AI as a medical partner Dr Alan D. Thompson is a world expert in artificial intelligence (AI), specialising in the augmentation of human intelligence, and advancing the evolution of ‘integrated AI’. Alan’s applied AI research and visualisations are featured across major international media, including citations in the University of Oxford’s debate on AI Ethics in December 2021. https://lifearchitect.ai/ ";