SlideShare a Scribd company logo
1 of 40
Future of Generative AI
https://abacus.ai/
Table of contents
01
04
02
05
03
06
Understanding
Generative AI
Evolution AI Now
Future of Generative AI Problem of Gen-AI Conclusion
Generative AI ?
1. Generative AI creates new content (images, text,
music) based on learned patterns.
2. It learns from vast examples and can produce
original, unseen works.
3. Capable of blending learned elements to generate
unique outputs.
4. Can produce customized creations based on
specific prompts.
5. Improves and refines its output over time with
more data and feedback.
At the core of Generative AI, we basically find two main types of models
GAN(Generative Adversarial Network):
1. Experts in generating Image from both text as
well as image input
2. Has 2 neural network(Generator and
Discriminator)
3. Generator generates “fake” content from input
vectors
4. Discriminator distinguishes between real sample
from domain and fake image generated by
generator
1. Deep Learning model that rely on self-attention
mechanism to process sequential data
2. Unlike traditional models, transformers can
handle data in parallel reducing training time and
improving efficiency
3. Backbone of today’s LLM models(GPT, Gemini,
BERT)
Transformers
Some tweets regarding Generative AI
AI Progression in
the Past
Growth in AI models
Time Series from Regression to Large
Language Models
1. Regression – Came in 1805. It generally has 10 to 60 inputs with similar weights to optimize
2. ARIMA(Auto Regressive Integrated Moving Average) – Came in 1970s. Inputs are past time and average
3. Backpropagation – basic neural net – Came in 1970, usually fully connected layers which can range from
500 to 500k weights
4. RNN (Recurrent Neural Networks) – Came in 1980 – Output is fed as input. Didn’t work well in time series.
“Vanishing Gradient” problem
5. LSTM(Long Short Term Memory) – came in 1997 – added “memory state”. May have 20M weights
6. Attention Networks – Came in 2017 (transformers)
7. BERT(Bidirectional Transformers) – Came in 2018(~100 m weights)
8. GPT 3.5 – Launched in 2022(175B parameters). Launched by OpenAI
9. Llama model – Launched in Feb 2023. It has 3 variants(7B, 13B, 70B weights). Came as a competitor for
OpenAI by Meta
10. GPT 4.0 – Launched in March 2023(1.76T parameters). Launched by openAI
11. Gemini model – Launched in December 2023(1.5T parameters). It has 3 variants(Gemini Nano, Pro and
Ultra). Updated version of BARD model of Google
Lots of Evolution in LLMs in short time
https://arxiv.org/pdf/2304.13712.pdf
The Expansion of
Generative AI in
today’s
Landscape
https://marketoonist.com/2023/01/ai-tidal-wave.html
Few Leading LLM models
OpenAI models
What are Transformers?
What are Transformers?
1. Boom on generative AI started with
transformers
2. Family of deep learning neural network
architecture(2017)
3. Architecture contains
. Word2Vec Embeddings
. Positional Encoding
. Self Attention
. Feed Forward Neural Network
4. Primary Application: Translation
GPT(Generative Pretrained Transformers)
GPT is a pretrained model build for OpenAI
for natural language Processing
Generative refers to the term of the model
to generate natural language.
Pretrained has 2 phases
1. Pretraining: The model is trained over
large corpus of data to predict next
word in a sentence
2. Fine-Tuning: Once the model is
trained it can be fine tuned to perform
specific task with supervised learning
GPT3 Training Data and Parameters
https://arxiv.org/pdf/2005.14165.pdf
GPT3 Effectiveness
https://arxiv.org/pdf/2005.14165.pdf
GPT3 through RLHF(Reinforcement Learning through Human Feedback)
1. RLHF refers to making the model learn from human feedback without the need of labelled data.
2. Due to the training data being scraped from internet might contain inappropriate or false information, it
must be aligned using RLHF to make it user appropriate
https://huyenchip.com/2023/05/02/rlhf.html
Llama 2 by Meta
Open Source – even for commercial
use !!
Model sizes, in Billions of weights
1. 70B, 34B(not released), 13B,
7B
2. both regular versions and chat
versions.
Llama model also used RLHF
technique
https://arxiv.org/pdf/2307.09288.pdf
Llama Details
Comparison with
ChatGPT 3.0(by OpenAI)
PaLM, in the Bison size,
Falcon 40B
Vicuna by UC Berkeley
MPT 7B
https://arxiv.org/pdf/2307.09288.pdf
Diffusion models
Diffusion models are another form of Generative models which works by adding noise to the images in the training
data by a process called forward diffusion process and then reversing the process to recover the original image
using reverse diffusion.
Forward Process(Diffusion): The model starts with data samples and gradually adds noise to these samples over a
series of steps until the data is completely transformed into random noise
Reverse Process(Denoising): The model learns to reverse the diffusion process by starting with the noise and
gradually removing it across many steps to reconstruct the original data samples. This is the main process where
model’s generative capabilities come into play as it learns to transform noise to high quality images
Stable Diffusion, Dalle, Midjourney are few examples of diffusion models
Stable Diffusion
Stable Diffusion is a text-to-image
state of the art model. It has four
important key components
1. Diffusion Probabilistic Model
2. U-Net Architecture
3. Latent Text Encoding
4. Classifier-Free Guidance
https://github.com/Stability-AI/stablediffusion
Dalle by OpenAI
MidJourney
SORA model by OpenAI
https://openai.com/sora
Future of
Generative AI
https://marketoonist.com/2023/06/ai-and-productivity.html
Maybe AI will help you work. But more likely, you’ll be working for AI
Talking to AI might
be the most
important skill of the
century
The Atlantic
Will AI Take my Job ?
Probably Not.
But new Job would be created.
But !!!
Jobs like data entry, customer support, content creation might be replaced by AI.
Task Category Most Likely to be Replaced by
Generative AI
Data Entry and Analysis Data input and transcription
Customer Support Basic support via chatbots
Content Creation Basic copywriting and image
generation
Design and Art Simple graphic and pattern design
Programming and Development Code generation for repetitive tasks
GPT5 ?
1. GPT-5 is expected to continue improving on
the logical reasoning and broader knowledge
bases demonstrated by GPT-4.
2. Anticipated enhancements include a
significant increase in parameter size,
enriching its processing and output
capabilities.
3. It might introduce advanced multimodal
functionalities, potentially incorporating video,
for a more integrated AI experience.
4. The release timeline remains speculative,
following OpenAI's historical pattern of
delivering gradual, impactful updates
https://www.datacamp.com/blog/everything-we-know-about-gpt-5
Impacts on Economy
Mandeep Singh, Senior Technology
Analyst at Bloomberg Intelligence and lead
author of the report said, “The world is
poised to see an explosion of growth in
the generative AI sector over the next
ten years that promises to
fundamentally change the way the
technology sector operates. The
technology is set to become an
increasingly essential part of IT
spending, ad spending, and
cybersecurity as it develops.”
https://synthedia.substack.com/p/generative-ai-to-reach-13-trillion
Risks associated with Generative AI
https://marketoonist.com/2023/06/impact-of-chatgpt.html
Things to keep in mind
Generating harmful content
Bias
Fake news and disinformation
“hallucination” – faking things
Privacy and data protection
The unpredictability
Conclusion
The future of Generative AI promises to be transformative, building on its current
capabilities of producing highly original and customized content across various media.
It is expected to further advance logical reasoning and integrate multimodal functions,
which may include handling video content, potentially revolutionizing how AI is
experienced. This evolution will likely influence IT, advertising, and cybersecurity
spending significantly, creating new job roles while automating others. With continued
innovation, Generative AI is poised to become an even more essential tool in the
technology landscape, necessitating a critical understanding of AI interaction as a
pivotal skill set for future generations.
References
1. Chip, H. (2023, May 2). RLHF. Retrieved from https://huyenchip.com/2023/05/02/rlhf.html
2. Touvron, H., Almahairi, A., ..., Scialom, T. (2023). Llama2: Open Foundation and Fine-Tuned Chat
Models. arXiv:2307.09288. Retrieved from https://arxiv.org/pdf/2307.09288.pdf
3. UCD Professional Academy. (n.d.). How will AI play a part in HR in 2024 and beyond? Retrieved
from https://www.ucd.ie/professionalacademy/resources/how-will-ai-play-a-part-in-hr-in-2024-and-
beyond/
4. DataCamp. (n.d.). Everything we know about GPT-5. Retrieved from
https://www.datacamp.com/blog/everything-we-know-about-gpt-5
5. Quillen, R. (n.d.). Large Language Models (LLM) - Immediate Future. Retrieved from
https://www.linkedin.com/pulse/large-language-models-llm-immediate-future-reinhold-quillen/
6. Statista. (n.d.). Artificial Intelligence - worldwide. Retrieved from
https://www.statista.com/outlook/tmo/artificial-intelligence/generative-ai/worldwide
7. Synthedia. (n.d.). Generative AI to reach $1.3 trillion. Retrieved from
https://synthedia.substack.com/p/generative-ai-to-reach-13-trillion
8. Touvron, H., Lavril, T., ..., Izacard, G. (2023). LLaMA: Open and Efficient Foundation Language
Models. arXiv:2302.13971. Retrieved from https://arxiv.org/pdf/2302.13971.pdf
9. Ouyang, L., Zhang, C., ..., Agarwal, S. (2022). Training language models to follow instructions with
human feedback. arXiv:2203.02155. Retrieved from https://arxiv.org/pdf/2203.02155.pdf
10. Gates, B. [Bill Gates]. (n.d.). Episode 6: Sam Altman. [Video]. YouTube. Retrieved from
https://www.youtube.com/watch?v=PkXELH6Y2lM
Thank you

More Related Content

What's hot

Blue Technologies : Blue Brain & Blue Eyes
Blue Technologies : Blue Brain & Blue EyesBlue Technologies : Blue Brain & Blue Eyes
Blue Technologies : Blue Brain & Blue Eyes
Mittal Patel
 

What's hot (20)

Introduction to ai (artificial intelligence)
Introduction to ai (artificial intelligence)Introduction to ai (artificial intelligence)
Introduction to ai (artificial intelligence)
 
The Blue Brain Technology
The Blue Brain TechnologyThe Blue Brain Technology
The Blue Brain Technology
 
Artificial Intelligence
Artificial IntelligenceArtificial Intelligence
Artificial Intelligence
 
PPT on Artificial Intelligence(A.I.)
PPT on Artificial Intelligence(A.I.) PPT on Artificial Intelligence(A.I.)
PPT on Artificial Intelligence(A.I.)
 
Blue Technologies : Blue Brain & Blue Eyes
Blue Technologies : Blue Brain & Blue EyesBlue Technologies : Blue Brain & Blue Eyes
Blue Technologies : Blue Brain & Blue Eyes
 
GENERATIVE AI, THE FUTURE OF PRODUCTIVITY
GENERATIVE AI, THE FUTURE OF PRODUCTIVITYGENERATIVE AI, THE FUTURE OF PRODUCTIVITY
GENERATIVE AI, THE FUTURE OF PRODUCTIVITY
 
Cavalry Ventures | Deep Dive: Generative AI
Cavalry Ventures | Deep Dive: Generative AICavalry Ventures | Deep Dive: Generative AI
Cavalry Ventures | Deep Dive: Generative AI
 
AI: Myth vs Reality (November 2017)
AI: Myth vs Reality (November 2017)AI: Myth vs Reality (November 2017)
AI: Myth vs Reality (November 2017)
 
Deep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial IntelligenceDeep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial Intelligence
 
Artificial Intelligence
Artificial Intelligence Artificial Intelligence
Artificial Intelligence
 
Using the power of Generative AI at scale
Using the power of Generative AI at scaleUsing the power of Generative AI at scale
Using the power of Generative AI at scale
 
ppt Artificial intelligence .pptx
ppt Artificial intelligence .pptxppt Artificial intelligence .pptx
ppt Artificial intelligence .pptx
 
ChatGPT OpenAI Primer for Business
ChatGPT OpenAI Primer for BusinessChatGPT OpenAI Primer for Business
ChatGPT OpenAI Primer for Business
 
Artificial intelligence
Artificial intelligenceArtificial intelligence
Artificial intelligence
 
chatgpt.pptx
chatgpt.pptxchatgpt.pptx
chatgpt.pptx
 
The bright and dark side of AI
The bright and dark side of AIThe bright and dark side of AI
The bright and dark side of AI
 
Power Point Presentation on Artificial Intelligence and Cool Current Projects...
Power Point Presentation on Artificial Intelligence and Cool Current Projects...Power Point Presentation on Artificial Intelligence and Cool Current Projects...
Power Point Presentation on Artificial Intelligence and Cool Current Projects...
 
AI.pptx
AI.pptxAI.pptx
AI.pptx
 
Journey of Generative AI
Journey of Generative AIJourney of Generative AI
Journey of Generative AI
 
Responsible Generative AI
Responsible Generative AIResponsible Generative AI
Responsible Generative AI
 

Similar to Past, Present and Future of Generative AI

AI and Blockchain with Reference to IPRs-Advocate_Prity_Khastgir.pdf
AI and Blockchain with Reference to IPRs-Advocate_Prity_Khastgir.pdfAI and Blockchain with Reference to IPRs-Advocate_Prity_Khastgir.pdf
AI and Blockchain with Reference to IPRs-Advocate_Prity_Khastgir.pdf
Tech Corp International Strategist
 

Similar to Past, Present and Future of Generative AI (20)

How to use Generative AI to make app testing easy.pdf
How to use Generative AI to make app testing easy.pdfHow to use Generative AI to make app testing easy.pdf
How to use Generative AI to make app testing easy.pdf
 
leewayhertz.com-How to build a generative AI solution From prototyping to pro...
leewayhertz.com-How to build a generative AI solution From prototyping to pro...leewayhertz.com-How to build a generative AI solution From prototyping to pro...
leewayhertz.com-How to build a generative AI solution From prototyping to pro...
 
Generative AI: A Comprehensive Tech Stack Breakdown
Generative AI: A Comprehensive Tech Stack BreakdownGenerative AI: A Comprehensive Tech Stack Breakdown
Generative AI: A Comprehensive Tech Stack Breakdown
 
Inspector Gadget 2023 - CalCPA.pdf
Inspector Gadget 2023 - CalCPA.pdfInspector Gadget 2023 - CalCPA.pdf
Inspector Gadget 2023 - CalCPA.pdf
 
How to build a generative AI solution.pdf
How to build a generative AI solution.pdfHow to build a generative AI solution.pdf
How to build a generative AI solution.pdf
 
Ai open powermeetupmarch25th
Ai open powermeetupmarch25thAi open powermeetupmarch25th
Ai open powermeetupmarch25th
 
2024 Build Generative AI for Non-Profits
2024 Build Generative AI for Non-Profits2024 Build Generative AI for Non-Profits
2024 Build Generative AI for Non-Profits
 
AI and Blockchain with Reference to IPRs-Advocate_Prity_Khastgir.pdf
AI and Blockchain with Reference to IPRs-Advocate_Prity_Khastgir.pdfAI and Blockchain with Reference to IPRs-Advocate_Prity_Khastgir.pdf
AI and Blockchain with Reference to IPRs-Advocate_Prity_Khastgir.pdf
 
Top Artificial Intelligence Tools & Frameworks in 2023.pdf
Top Artificial Intelligence Tools & Frameworks in 2023.pdfTop Artificial Intelligence Tools & Frameworks in 2023.pdf
Top Artificial Intelligence Tools & Frameworks in 2023.pdf
 
Intel 20180608 v2
Intel 20180608 v2Intel 20180608 v2
Intel 20180608 v2
 
A Research Paper on HUMAN MACHINE CONVERSATION USING CHATBOT
A Research Paper on HUMAN MACHINE CONVERSATION USING CHATBOTA Research Paper on HUMAN MACHINE CONVERSATION USING CHATBOT
A Research Paper on HUMAN MACHINE CONVERSATION USING CHATBOT
 
Generative-AI-Exploring-beyond-the-horizons-possibilities-of-AI-WP.pdf
Generative-AI-Exploring-beyond-the-horizons-possibilities-of-AI-WP.pdfGenerative-AI-Exploring-beyond-the-horizons-possibilities-of-AI-WP.pdf
Generative-AI-Exploring-beyond-the-horizons-possibilities-of-AI-WP.pdf
 
Unleashing the Power of Generative AI.pdf
Unleashing the Power of Generative AI.pdfUnleashing the Power of Generative AI.pdf
Unleashing the Power of Generative AI.pdf
 
How Mistral AI raised €105m with no pitch deck or product
How Mistral AI raised €105m with no pitch deck or productHow Mistral AI raised €105m with no pitch deck or product
How Mistral AI raised €105m with no pitch deck or product
 
Mistral AI Strategic Memo.pdf
Mistral AI Strategic Memo.pdfMistral AI Strategic Memo.pdf
Mistral AI Strategic Memo.pdf
 
IRJET - Fake News Detection using Machine Learning
IRJET -  	  Fake News Detection using Machine LearningIRJET -  	  Fake News Detection using Machine Learning
IRJET - Fake News Detection using Machine Learning
 
Wall Street Mastermind Sector Spotlight - Technology (October 2023).pdf
Wall Street Mastermind Sector Spotlight - Technology (October 2023).pdfWall Street Mastermind Sector Spotlight - Technology (October 2023).pdf
Wall Street Mastermind Sector Spotlight - Technology (October 2023).pdf
 
Generative Artificial Intelligence and Data Privacy: A Primer
Generative Artificial Intelligence and Data Privacy: A Primer Generative Artificial Intelligence and Data Privacy: A Primer
Generative Artificial Intelligence and Data Privacy: A Primer
 
Unleashing the Power of Generative AI.pdf
Unleashing the Power of Generative AI.pdfUnleashing the Power of Generative AI.pdf
Unleashing the Power of Generative AI.pdf
 
20240507 QFM013 Machine Intelligence Reading List April 2024.pdf
20240507 QFM013 Machine Intelligence Reading List April 2024.pdf20240507 QFM013 Machine Intelligence Reading List April 2024.pdf
20240507 QFM013 Machine Intelligence Reading List April 2024.pdf
 

Recently uploaded

Online crime reporting system project.pdf
Online crime reporting system project.pdfOnline crime reporting system project.pdf
Online crime reporting system project.pdf
Kamal Acharya
 
Complex plane, Modulus, Argument, Graphical representation of a complex numbe...
Complex plane, Modulus, Argument, Graphical representation of a complex numbe...Complex plane, Modulus, Argument, Graphical representation of a complex numbe...
Complex plane, Modulus, Argument, Graphical representation of a complex numbe...
MohammadAliNayeem
 
ALCOHOL PRODUCTION- Beer Brewing Process.pdf
ALCOHOL PRODUCTION- Beer Brewing Process.pdfALCOHOL PRODUCTION- Beer Brewing Process.pdf
ALCOHOL PRODUCTION- Beer Brewing Process.pdf
Madan Karki
 

Recently uploaded (20)

RM&IPR M5 notes.pdfResearch Methodolgy & Intellectual Property Rights Series 5
RM&IPR M5 notes.pdfResearch Methodolgy & Intellectual Property Rights Series 5RM&IPR M5 notes.pdfResearch Methodolgy & Intellectual Property Rights Series 5
RM&IPR M5 notes.pdfResearch Methodolgy & Intellectual Property Rights Series 5
 
Lesson no16 application of Induction Generator in Wind.ppsx
Lesson no16 application of Induction Generator in Wind.ppsxLesson no16 application of Induction Generator in Wind.ppsx
Lesson no16 application of Induction Generator in Wind.ppsx
 
Insurance management system project report.pdf
Insurance management system project report.pdfInsurance management system project report.pdf
Insurance management system project report.pdf
 
Theory for How to calculation capacitor bank
Theory for How to calculation capacitor bankTheory for How to calculation capacitor bank
Theory for How to calculation capacitor bank
 
Software Engineering - Modelling Concepts + Class Modelling + Building the An...
Software Engineering - Modelling Concepts + Class Modelling + Building the An...Software Engineering - Modelling Concepts + Class Modelling + Building the An...
Software Engineering - Modelling Concepts + Class Modelling + Building the An...
 
Online book store management system project.pdf
Online book store management system project.pdfOnline book store management system project.pdf
Online book store management system project.pdf
 
Multivibrator and its types defination and usges.pptx
Multivibrator and its types defination and usges.pptxMultivibrator and its types defination and usges.pptx
Multivibrator and its types defination and usges.pptx
 
Filters for Electromagnetic Compatibility Applications
Filters for Electromagnetic Compatibility ApplicationsFilters for Electromagnetic Compatibility Applications
Filters for Electromagnetic Compatibility Applications
 
SLIDESHARE PPT-DECISION MAKING METHODS.pptx
SLIDESHARE PPT-DECISION MAKING METHODS.pptxSLIDESHARE PPT-DECISION MAKING METHODS.pptx
SLIDESHARE PPT-DECISION MAKING METHODS.pptx
 
Online crime reporting system project.pdf
Online crime reporting system project.pdfOnline crime reporting system project.pdf
Online crime reporting system project.pdf
 
Interfacing Analog to Digital Data Converters ee3404.pdf
Interfacing Analog to Digital Data Converters ee3404.pdfInterfacing Analog to Digital Data Converters ee3404.pdf
Interfacing Analog to Digital Data Converters ee3404.pdf
 
"United Nations Park" Site Visit Report.
"United Nations Park" Site  Visit Report."United Nations Park" Site  Visit Report.
"United Nations Park" Site Visit Report.
 
Introduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AIIntroduction to Artificial Intelligence and History of AI
Introduction to Artificial Intelligence and History of AI
 
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdfInvolute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
Involute of a circle,Square, pentagon,HexagonInvolute_Engineering Drawing.pdf
 
BURGER ORDERING SYSYTEM PROJECT REPORT..pdf
BURGER ORDERING SYSYTEM PROJECT REPORT..pdfBURGER ORDERING SYSYTEM PROJECT REPORT..pdf
BURGER ORDERING SYSYTEM PROJECT REPORT..pdf
 
E-Commerce Shopping using MERN Stack where different modules are present
E-Commerce Shopping using MERN Stack where different modules are presentE-Commerce Shopping using MERN Stack where different modules are present
E-Commerce Shopping using MERN Stack where different modules are present
 
BRAKING SYSTEM IN INDIAN RAILWAY AutoCAD DRAWING
BRAKING SYSTEM IN INDIAN RAILWAY AutoCAD DRAWINGBRAKING SYSTEM IN INDIAN RAILWAY AutoCAD DRAWING
BRAKING SYSTEM IN INDIAN RAILWAY AutoCAD DRAWING
 
Complex plane, Modulus, Argument, Graphical representation of a complex numbe...
Complex plane, Modulus, Argument, Graphical representation of a complex numbe...Complex plane, Modulus, Argument, Graphical representation of a complex numbe...
Complex plane, Modulus, Argument, Graphical representation of a complex numbe...
 
Introduction to Heat Exchangers: Principle, Types and Applications
Introduction to Heat Exchangers: Principle, Types and ApplicationsIntroduction to Heat Exchangers: Principle, Types and Applications
Introduction to Heat Exchangers: Principle, Types and Applications
 
ALCOHOL PRODUCTION- Beer Brewing Process.pdf
ALCOHOL PRODUCTION- Beer Brewing Process.pdfALCOHOL PRODUCTION- Beer Brewing Process.pdf
ALCOHOL PRODUCTION- Beer Brewing Process.pdf
 

Past, Present and Future of Generative AI

  • 1. Future of Generative AI https://abacus.ai/
  • 2. Table of contents 01 04 02 05 03 06 Understanding Generative AI Evolution AI Now Future of Generative AI Problem of Gen-AI Conclusion
  • 3. Generative AI ? 1. Generative AI creates new content (images, text, music) based on learned patterns. 2. It learns from vast examples and can produce original, unseen works. 3. Capable of blending learned elements to generate unique outputs. 4. Can produce customized creations based on specific prompts. 5. Improves and refines its output over time with more data and feedback.
  • 4. At the core of Generative AI, we basically find two main types of models GAN(Generative Adversarial Network): 1. Experts in generating Image from both text as well as image input 2. Has 2 neural network(Generator and Discriminator) 3. Generator generates “fake” content from input vectors 4. Discriminator distinguishes between real sample from domain and fake image generated by generator
  • 5. 1. Deep Learning model that rely on self-attention mechanism to process sequential data 2. Unlike traditional models, transformers can handle data in parallel reducing training time and improving efficiency 3. Backbone of today’s LLM models(GPT, Gemini, BERT) Transformers
  • 6. Some tweets regarding Generative AI
  • 8. Growth in AI models
  • 9. Time Series from Regression to Large Language Models 1. Regression – Came in 1805. It generally has 10 to 60 inputs with similar weights to optimize 2. ARIMA(Auto Regressive Integrated Moving Average) – Came in 1970s. Inputs are past time and average 3. Backpropagation – basic neural net – Came in 1970, usually fully connected layers which can range from 500 to 500k weights 4. RNN (Recurrent Neural Networks) – Came in 1980 – Output is fed as input. Didn’t work well in time series. “Vanishing Gradient” problem 5. LSTM(Long Short Term Memory) – came in 1997 – added “memory state”. May have 20M weights 6. Attention Networks – Came in 2017 (transformers) 7. BERT(Bidirectional Transformers) – Came in 2018(~100 m weights) 8. GPT 3.5 – Launched in 2022(175B parameters). Launched by OpenAI 9. Llama model – Launched in Feb 2023. It has 3 variants(7B, 13B, 70B weights). Came as a competitor for OpenAI by Meta 10. GPT 4.0 – Launched in March 2023(1.76T parameters). Launched by openAI 11. Gemini model – Launched in December 2023(1.5T parameters). It has 3 variants(Gemini Nano, Pro and Ultra). Updated version of BARD model of Google
  • 10. Lots of Evolution in LLMs in short time https://arxiv.org/pdf/2304.13712.pdf
  • 11. The Expansion of Generative AI in today’s Landscape
  • 13. Few Leading LLM models OpenAI models
  • 15. What are Transformers? 1. Boom on generative AI started with transformers 2. Family of deep learning neural network architecture(2017) 3. Architecture contains . Word2Vec Embeddings . Positional Encoding . Self Attention . Feed Forward Neural Network 4. Primary Application: Translation
  • 16. GPT(Generative Pretrained Transformers) GPT is a pretrained model build for OpenAI for natural language Processing Generative refers to the term of the model to generate natural language. Pretrained has 2 phases 1. Pretraining: The model is trained over large corpus of data to predict next word in a sentence 2. Fine-Tuning: Once the model is trained it can be fine tuned to perform specific task with supervised learning
  • 17. GPT3 Training Data and Parameters https://arxiv.org/pdf/2005.14165.pdf
  • 19. GPT3 through RLHF(Reinforcement Learning through Human Feedback) 1. RLHF refers to making the model learn from human feedback without the need of labelled data. 2. Due to the training data being scraped from internet might contain inappropriate or false information, it must be aligned using RLHF to make it user appropriate https://huyenchip.com/2023/05/02/rlhf.html
  • 20. Llama 2 by Meta Open Source – even for commercial use !! Model sizes, in Billions of weights 1. 70B, 34B(not released), 13B, 7B 2. both regular versions and chat versions. Llama model also used RLHF technique https://arxiv.org/pdf/2307.09288.pdf
  • 21. Llama Details Comparison with ChatGPT 3.0(by OpenAI) PaLM, in the Bison size, Falcon 40B Vicuna by UC Berkeley MPT 7B https://arxiv.org/pdf/2307.09288.pdf
  • 22. Diffusion models Diffusion models are another form of Generative models which works by adding noise to the images in the training data by a process called forward diffusion process and then reversing the process to recover the original image using reverse diffusion. Forward Process(Diffusion): The model starts with data samples and gradually adds noise to these samples over a series of steps until the data is completely transformed into random noise Reverse Process(Denoising): The model learns to reverse the diffusion process by starting with the noise and gradually removing it across many steps to reconstruct the original data samples. This is the main process where model’s generative capabilities come into play as it learns to transform noise to high quality images Stable Diffusion, Dalle, Midjourney are few examples of diffusion models
  • 23. Stable Diffusion Stable Diffusion is a text-to-image state of the art model. It has four important key components 1. Diffusion Probabilistic Model 2. U-Net Architecture 3. Latent Text Encoding 4. Classifier-Free Guidance https://github.com/Stability-AI/stablediffusion
  • 26. SORA model by OpenAI https://openai.com/sora
  • 28. https://marketoonist.com/2023/06/ai-and-productivity.html Maybe AI will help you work. But more likely, you’ll be working for AI
  • 29. Talking to AI might be the most important skill of the century The Atlantic
  • 30. Will AI Take my Job ? Probably Not. But new Job would be created.
  • 31. But !!! Jobs like data entry, customer support, content creation might be replaced by AI. Task Category Most Likely to be Replaced by Generative AI Data Entry and Analysis Data input and transcription Customer Support Basic support via chatbots Content Creation Basic copywriting and image generation Design and Art Simple graphic and pattern design Programming and Development Code generation for repetitive tasks
  • 32. GPT5 ? 1. GPT-5 is expected to continue improving on the logical reasoning and broader knowledge bases demonstrated by GPT-4. 2. Anticipated enhancements include a significant increase in parameter size, enriching its processing and output capabilities. 3. It might introduce advanced multimodal functionalities, potentially incorporating video, for a more integrated AI experience. 4. The release timeline remains speculative, following OpenAI's historical pattern of delivering gradual, impactful updates https://www.datacamp.com/blog/everything-we-know-about-gpt-5
  • 33. Impacts on Economy Mandeep Singh, Senior Technology Analyst at Bloomberg Intelligence and lead author of the report said, “The world is poised to see an explosion of growth in the generative AI sector over the next ten years that promises to fundamentally change the way the technology sector operates. The technology is set to become an increasingly essential part of IT spending, ad spending, and cybersecurity as it develops.” https://synthedia.substack.com/p/generative-ai-to-reach-13-trillion
  • 34. Risks associated with Generative AI
  • 36.
  • 37. Things to keep in mind Generating harmful content Bias Fake news and disinformation “hallucination” – faking things Privacy and data protection The unpredictability
  • 38. Conclusion The future of Generative AI promises to be transformative, building on its current capabilities of producing highly original and customized content across various media. It is expected to further advance logical reasoning and integrate multimodal functions, which may include handling video content, potentially revolutionizing how AI is experienced. This evolution will likely influence IT, advertising, and cybersecurity spending significantly, creating new job roles while automating others. With continued innovation, Generative AI is poised to become an even more essential tool in the technology landscape, necessitating a critical understanding of AI interaction as a pivotal skill set for future generations.
  • 39. References 1. Chip, H. (2023, May 2). RLHF. Retrieved from https://huyenchip.com/2023/05/02/rlhf.html 2. Touvron, H., Almahairi, A., ..., Scialom, T. (2023). Llama2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288. Retrieved from https://arxiv.org/pdf/2307.09288.pdf 3. UCD Professional Academy. (n.d.). How will AI play a part in HR in 2024 and beyond? Retrieved from https://www.ucd.ie/professionalacademy/resources/how-will-ai-play-a-part-in-hr-in-2024-and- beyond/ 4. DataCamp. (n.d.). Everything we know about GPT-5. Retrieved from https://www.datacamp.com/blog/everything-we-know-about-gpt-5 5. Quillen, R. (n.d.). Large Language Models (LLM) - Immediate Future. Retrieved from https://www.linkedin.com/pulse/large-language-models-llm-immediate-future-reinhold-quillen/ 6. Statista. (n.d.). Artificial Intelligence - worldwide. Retrieved from https://www.statista.com/outlook/tmo/artificial-intelligence/generative-ai/worldwide 7. Synthedia. (n.d.). Generative AI to reach $1.3 trillion. Retrieved from https://synthedia.substack.com/p/generative-ai-to-reach-13-trillion 8. Touvron, H., Lavril, T., ..., Izacard, G. (2023). LLaMA: Open and Efficient Foundation Language Models. arXiv:2302.13971. Retrieved from https://arxiv.org/pdf/2302.13971.pdf 9. Ouyang, L., Zhang, C., ..., Agarwal, S. (2022). Training language models to follow instructions with human feedback. arXiv:2203.02155. Retrieved from https://arxiv.org/pdf/2203.02155.pdf 10. Gates, B. [Bill Gates]. (n.d.). Episode 6: Sam Altman. [Video]. YouTube. Retrieved from https://www.youtube.com/watch?v=PkXELH6Y2lM