This presentation presents an overview of the challenges and opportunities of generative artificial intelligence in Web3. It includes a brief research history of generative AI as well as some of its immediate applications in Web3.
AI and ML Series - Introduction to Generative AI and LLMs - Session 1DianaGray10
Session 1
👉This first session will cover an introduction to Generative AI & harnessing the power of large language models. The following topics will be discussed:
Introduction to Generative AI & harnessing the power of large language models.
What’s generative AI & what’s LLM.
How are we using it in our document understanding & communication mining models?
How to develop a trustworthy and unbiased AI model using LLM & GenAI.
Personal Intelligent Assistant
Speakers:
📌George Roth - AI Evangelist at UiPath
📌Sharon Palawandram - Senior Machine Learning Consultant @ Ashling Partners & UiPath MVP
📌Russel Alfeche - Technology Leader RPA @qBotica & UiPath MVP
GENERATIVE AI, THE FUTURE OF PRODUCTIVITYAndre Muscat
Discuss the impact and opportunity of using Generative AI to support your development and creative teams
* Explore business challenges in content creation
* Cost-per-unit of different types of content
* Use AI to reduce cost-per-unit
* New partnerships being formed that will have a material impact on the way we search and engage with content
Part 4 of a 9 Part Research Series named "What matters in AI" published on www.andremuscat.com
This presentation presents an overview of the challenges and opportunities of generative artificial intelligence in Web3. It includes a brief research history of generative AI as well as some of its immediate applications in Web3.
AI and ML Series - Introduction to Generative AI and LLMs - Session 1DianaGray10
Session 1
👉This first session will cover an introduction to Generative AI & harnessing the power of large language models. The following topics will be discussed:
Introduction to Generative AI & harnessing the power of large language models.
What’s generative AI & what’s LLM.
How are we using it in our document understanding & communication mining models?
How to develop a trustworthy and unbiased AI model using LLM & GenAI.
Personal Intelligent Assistant
Speakers:
📌George Roth - AI Evangelist at UiPath
📌Sharon Palawandram - Senior Machine Learning Consultant @ Ashling Partners & UiPath MVP
📌Russel Alfeche - Technology Leader RPA @qBotica & UiPath MVP
GENERATIVE AI, THE FUTURE OF PRODUCTIVITYAndre Muscat
Discuss the impact and opportunity of using Generative AI to support your development and creative teams
* Explore business challenges in content creation
* Cost-per-unit of different types of content
* Use AI to reduce cost-per-unit
* New partnerships being formed that will have a material impact on the way we search and engage with content
Part 4 of a 9 Part Research Series named "What matters in AI" published on www.andremuscat.com
Generative AI models, such as ChatGPT and Stable Diffusion, can create new and original content like text, images, video, audio, or other data from simple prompts, as well as handle complex dialogs and reason about problems with or without images. These models are disrupting traditional technologies, from search and content creation to automation and problem solving, and are fundamentally shaping the future user interface to computing devices. Generative AI can apply broadly across industries, providing significant enhancements for utility, productivity, and entertainment. As generative AI adoption grows at record-setting speeds and computing demands increase, on-device and hybrid processing are more important than ever. Just like traditional computing evolved from mainframes to today’s mix of cloud and edge devices, AI processing will be distributed between them for AI to scale and reach its full potential.
In this presentation you’ll learn about:
- Why on-device AI is key
- Full-stack AI optimizations to make on-device AI possible and efficient
- Advanced techniques like quantization, distillation, and speculative decoding
- How generative AI models can be run on device and examples of some running now
- Qualcomm Technologies’ role in scaling on-device generative AI
Episode 2: The LLM / GPT / AI Prompt / Data Engineer RoadmapAnant Corporation
In this episode we'll discuss the different flavors of prompt engineering in the LLM/GPT space. According to your skill level you should be able to pick up at any of the following:
Leveling up with GPT
1: Use ChatGPT / GPT Powered Apps
2: Become a Prompt Engineer on ChatGPT/GPT
3: Use GPT API with NoCode Automation, App Builders
4: Create Workflows to Automate Tasks with NoCode
5: Use GPT API with Code, make your own APIs
6: Create Workflows to Automate Tasks with Code
7: Use GPT API with your Data / a Framework
8: Use GPT API with your Data / a Framework to Make your own APIs
9: Create Workflows to Automate Tasks with your Data /a Framework
10: Use Another LLM API other than GPT (Cohere, HuggingFace)
11: Use open source LLM models on your computer
12: Finetune / Build your own models
Series: Using AI / ChatGPT at Work - GPT Automation
Are you a small business owner or web developer interested in leveraging the power of GPT (Generative Pretrained Transformer) technology to enhance your business processes?
If so, Join us for a series of events focused on using GPT in business. Whether you're a small business owner or a web developer, you'll learn how to leverage GPT to improve your workflow and provide better services to your customers.
Chat GPT 4 can pass the American state bar exam, but before you go expecting to see robot lawyers taking over the courtroom, hold your horses cowboys – we're not quite there yet. That being said, AI is becoming increasingly more human-like, and as a VC we need to start thinking about how this new wave of technology is going to affect the way we build and run businesses. What do we need to do differently? How can we make sure that our investment strategies are reflecting these changes? It's a brave new world out there, and we’ve got to keep the big picture in mind!
Sharing here with you what we at Cavalry Ventures found out during our Generative AI deep dive.
This talk overviews my background as a female data scientist, introduces many types of generative AI, discusses potential use cases, highlights the need for representation in generative AI, and showcases a few tools that currently exist.
AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platfor...DianaGray10
📣 AI plays a crucial role in the UiPath Business Automation Platform. In this session you will learn about how the UiPath Business Automation Platform is well-suited for AI, the use of LLM and integrations you can use. Topics include the following:
Introductions.
AI powered automations overview.
Discover why the UiPath Business Automation Platform is well-suited for AI.
LLM + Automation framework and integrations with LangChain.
Generative AI Automation Patterns Demonstration.
👨🏽🤝👨🏻 Speakers:
Dhruv Patel, Senior Sales Solution Architect @UiPath
Russel Alfeche, Technology Leader, RPA @qBotica and UiPath MVP
An Introduction to Generative AI - May 18, 2023CoriFaklaris1
For this plenary talk at the Charlotte AI Institute for Smarter Learning, Dr. Cori Faklaris introduces her fellow college educators to the exciting world of generative AI tools. She gives a high-level overview of the generative AI landscape and how these tools use machine learning algorithms to generate creative content such as music, art, and text. She then shares some examples of generative AI tools and demonstrate how she has used some of these tools to enhance teaching and learning in the classroom and to boost her productivity in other areas of academic life.
The Future of AI is Generative not Discriminative 5/26/2021Steve Omohundro
The deep learning AI revolution has been sweeping the world for a decade now. Deep neural nets are routinely used for tasks like translation, fraud detection, and image classification. PwC estimates that they will create $15.7 trillion/year of value by 2030. But most current networks are "discriminative" in that they directly map inputs to predictions. This type of model requires lots of training examples, doesn't generalize well outside of its training set, creates inscrutable representations, is subject to adversarial examples, and makes knowledge transfer difficult. People, in contrast, can learn from just a few examples, generalize far beyond their experience, and can easily transfer and reuse knowledge. In recent years, new kinds of "generative" AI models have begun to exhibit these desirable human characteristics. They represent the causal generative processes by which the data is created and can be compositional, compact, and directly interpretable. Generative AI systems that assist people can model their needs and desires and interact with empathy. Their adaptability to changing circumstances will likely be required by rapidly changing AI-driven business and social systems. Generative AI will be the engine of future AI innovation.
Regulating Generative AI - LLMOps pipelines with TransparencyDebmalya Biswas
The growing adoption of Gen AI, esp. LLMs, has re-ignited the discussion around AI Regulations — to ensure that AI/ML systems are responsibly trained and deployed. Unfortunately, this effort is complicated by multiple governmental organizations and regulatory bodies releasing their own guidelines and policies with little to no agreement on the definition of terms.
Rather than trying to understand and regulate all types of AI, we recommend a different (and practical) approach in this talk based on AI Transparency —
to transparently outline the capabilities of the AI system based on its training methodology and set realistic expectations with respect to what it can (and cannot) do.
We outline LLMOps architecture patterns and show how the proposed approach can be integrated at different stages of the LLMOps pipeline capturing the model's capabilities. In addition, the AI system provider also specifies scenarios where (they believe that) the system can make mistakes, and recommends a ‘safe’ approach with guardrails for those scenarios.
Large Language Models, No-Code, and Responsible AI - Trends in Applied NLP in...David Talby
An April 2023 presentation to the AMIA working group on natural language processing. The talk focuses on three current trends in NLP and how they apply in healthcare: Large language models, No-code, and Responsible AI.
Leveraging Generative AI & Best practicesDianaGray10
In this event we will cover:
- What is Generative AI and how it is being for future of work.
- Best practices for developing and deploying generative AI based models in productions.
- Future of Generative AI, how generative AI is expected to evolve in the coming years.
In this session, you'll get all the answers about how ChatGPT and other GPT-X models can be applied to your current or future project. First, we'll put in order all the terms – OpenAI, GPT-3, ChatGPT, Codex, Dall-E, etc., and explain why Microsoft and Azure are often mentioned in this context. Then, we'll go through the main capabilities of the Azure OpenAI and respective usecases that might inspire you to either optimize your product or build a completely new one.
В рамках C/C++/Embedded місяця у GlobalLogic нещодавно відбувся Online TechTalk "Patterns in Embedded SW Design"
Спікер розібрав паттерни на кісточки: від поняття до практичного використання з прикладом проєктного коду.
У доповіді спеціаліст розглянув:
- Поняття патернів у програмному забезпеченні з акцентом на Embedded розробку.
- Основні переваги використання патернів.
- Класифікацію патернів для Embedded напрямку, деякі з них було розглянуто.
- Було наведено приклад використання на прикладі проєктного коду.
Деталі та відео заходу: https://bit.ly/3DaKx7t
Generative AI models, such as ChatGPT and Stable Diffusion, can create new and original content like text, images, video, audio, or other data from simple prompts, as well as handle complex dialogs and reason about problems with or without images. These models are disrupting traditional technologies, from search and content creation to automation and problem solving, and are fundamentally shaping the future user interface to computing devices. Generative AI can apply broadly across industries, providing significant enhancements for utility, productivity, and entertainment. As generative AI adoption grows at record-setting speeds and computing demands increase, on-device and hybrid processing are more important than ever. Just like traditional computing evolved from mainframes to today’s mix of cloud and edge devices, AI processing will be distributed between them for AI to scale and reach its full potential.
In this presentation you’ll learn about:
- Why on-device AI is key
- Full-stack AI optimizations to make on-device AI possible and efficient
- Advanced techniques like quantization, distillation, and speculative decoding
- How generative AI models can be run on device and examples of some running now
- Qualcomm Technologies’ role in scaling on-device generative AI
Episode 2: The LLM / GPT / AI Prompt / Data Engineer RoadmapAnant Corporation
In this episode we'll discuss the different flavors of prompt engineering in the LLM/GPT space. According to your skill level you should be able to pick up at any of the following:
Leveling up with GPT
1: Use ChatGPT / GPT Powered Apps
2: Become a Prompt Engineer on ChatGPT/GPT
3: Use GPT API with NoCode Automation, App Builders
4: Create Workflows to Automate Tasks with NoCode
5: Use GPT API with Code, make your own APIs
6: Create Workflows to Automate Tasks with Code
7: Use GPT API with your Data / a Framework
8: Use GPT API with your Data / a Framework to Make your own APIs
9: Create Workflows to Automate Tasks with your Data /a Framework
10: Use Another LLM API other than GPT (Cohere, HuggingFace)
11: Use open source LLM models on your computer
12: Finetune / Build your own models
Series: Using AI / ChatGPT at Work - GPT Automation
Are you a small business owner or web developer interested in leveraging the power of GPT (Generative Pretrained Transformer) technology to enhance your business processes?
If so, Join us for a series of events focused on using GPT in business. Whether you're a small business owner or a web developer, you'll learn how to leverage GPT to improve your workflow and provide better services to your customers.
Chat GPT 4 can pass the American state bar exam, but before you go expecting to see robot lawyers taking over the courtroom, hold your horses cowboys – we're not quite there yet. That being said, AI is becoming increasingly more human-like, and as a VC we need to start thinking about how this new wave of technology is going to affect the way we build and run businesses. What do we need to do differently? How can we make sure that our investment strategies are reflecting these changes? It's a brave new world out there, and we’ve got to keep the big picture in mind!
Sharing here with you what we at Cavalry Ventures found out during our Generative AI deep dive.
This talk overviews my background as a female data scientist, introduces many types of generative AI, discusses potential use cases, highlights the need for representation in generative AI, and showcases a few tools that currently exist.
AI and ML Series - Leveraging Generative AI and LLMs Using the UiPath Platfor...DianaGray10
📣 AI plays a crucial role in the UiPath Business Automation Platform. In this session you will learn about how the UiPath Business Automation Platform is well-suited for AI, the use of LLM and integrations you can use. Topics include the following:
Introductions.
AI powered automations overview.
Discover why the UiPath Business Automation Platform is well-suited for AI.
LLM + Automation framework and integrations with LangChain.
Generative AI Automation Patterns Demonstration.
👨🏽🤝👨🏻 Speakers:
Dhruv Patel, Senior Sales Solution Architect @UiPath
Russel Alfeche, Technology Leader, RPA @qBotica and UiPath MVP
An Introduction to Generative AI - May 18, 2023CoriFaklaris1
For this plenary talk at the Charlotte AI Institute for Smarter Learning, Dr. Cori Faklaris introduces her fellow college educators to the exciting world of generative AI tools. She gives a high-level overview of the generative AI landscape and how these tools use machine learning algorithms to generate creative content such as music, art, and text. She then shares some examples of generative AI tools and demonstrate how she has used some of these tools to enhance teaching and learning in the classroom and to boost her productivity in other areas of academic life.
The Future of AI is Generative not Discriminative 5/26/2021Steve Omohundro
The deep learning AI revolution has been sweeping the world for a decade now. Deep neural nets are routinely used for tasks like translation, fraud detection, and image classification. PwC estimates that they will create $15.7 trillion/year of value by 2030. But most current networks are "discriminative" in that they directly map inputs to predictions. This type of model requires lots of training examples, doesn't generalize well outside of its training set, creates inscrutable representations, is subject to adversarial examples, and makes knowledge transfer difficult. People, in contrast, can learn from just a few examples, generalize far beyond their experience, and can easily transfer and reuse knowledge. In recent years, new kinds of "generative" AI models have begun to exhibit these desirable human characteristics. They represent the causal generative processes by which the data is created and can be compositional, compact, and directly interpretable. Generative AI systems that assist people can model their needs and desires and interact with empathy. Their adaptability to changing circumstances will likely be required by rapidly changing AI-driven business and social systems. Generative AI will be the engine of future AI innovation.
Regulating Generative AI - LLMOps pipelines with TransparencyDebmalya Biswas
The growing adoption of Gen AI, esp. LLMs, has re-ignited the discussion around AI Regulations — to ensure that AI/ML systems are responsibly trained and deployed. Unfortunately, this effort is complicated by multiple governmental organizations and regulatory bodies releasing their own guidelines and policies with little to no agreement on the definition of terms.
Rather than trying to understand and regulate all types of AI, we recommend a different (and practical) approach in this talk based on AI Transparency —
to transparently outline the capabilities of the AI system based on its training methodology and set realistic expectations with respect to what it can (and cannot) do.
We outline LLMOps architecture patterns and show how the proposed approach can be integrated at different stages of the LLMOps pipeline capturing the model's capabilities. In addition, the AI system provider also specifies scenarios where (they believe that) the system can make mistakes, and recommends a ‘safe’ approach with guardrails for those scenarios.
Large Language Models, No-Code, and Responsible AI - Trends in Applied NLP in...David Talby
An April 2023 presentation to the AMIA working group on natural language processing. The talk focuses on three current trends in NLP and how they apply in healthcare: Large language models, No-code, and Responsible AI.
Leveraging Generative AI & Best practicesDianaGray10
In this event we will cover:
- What is Generative AI and how it is being for future of work.
- Best practices for developing and deploying generative AI based models in productions.
- Future of Generative AI, how generative AI is expected to evolve in the coming years.
In this session, you'll get all the answers about how ChatGPT and other GPT-X models can be applied to your current or future project. First, we'll put in order all the terms – OpenAI, GPT-3, ChatGPT, Codex, Dall-E, etc., and explain why Microsoft and Azure are often mentioned in this context. Then, we'll go through the main capabilities of the Azure OpenAI and respective usecases that might inspire you to either optimize your product or build a completely new one.
В рамках C/C++/Embedded місяця у GlobalLogic нещодавно відбувся Online TechTalk "Patterns in Embedded SW Design"
Спікер розібрав паттерни на кісточки: від поняття до практичного використання з прикладом проєктного коду.
У доповіді спеціаліст розглянув:
- Поняття патернів у програмному забезпеченні з акцентом на Embedded розробку.
- Основні переваги використання патернів.
- Класифікацію патернів для Embedded напрямку, деякі з них було розглянуто.
- Було наведено приклад використання на прикладі проєктного коду.
Деталі та відео заходу: https://bit.ly/3DaKx7t
Smart speakers and messaging apps are increasing in popularity due to their convenience, intuitive usage, and new user experiences. Companies are racing to develop voice interfaces and AI technologies in order to utilize smart speakers and messaging platforms as a sales channel.
We are introducing Grid Genie, a completely open source, multi-device, conversational commerce platform. With this technology, we can help you engineer a powerful platform to combine your Alexa skills, Google actions, and Facebook Messenger into a single seamless experience.
Smart speakers and messaging apps are increasing in popularity due to their convenience, intuitive usage, and new user experiences. Companies are racing to develop voice interfaces and AI technologies in order to utilize smart speakers and messaging platforms as a sales channel.
We are introducing Grid Genie, a completely open source, multi-device, conversational commerce platform. With this technology, we can help you engineer a powerful platform to combine your Alexa skills, Google actions, and Facebook Messenger into a single seamless experience.
Application of Foundation Model for Autonomous DrivingYu Huang
Since DARPA’s Grand Challenges (rural) in 2004/05 and Urban Challenges in 2007, autonomous driving has been the most active field of AI applications. Recently powered by large language models (LLMs), chat systems, such as chatGPT and PaLM, emerge and rapidly become a promising direction to achieve artificial general intelligence (AGI) in natural language processing (NLP). There comes a natural thinking that we could employ these abilities to reformulate autonomous driving. By combining LLM with foundation models, it is possible to utilize the human knowledge, commonsense and reasoning to rebuild autonomous driving systems from the current long-tailed AI dilemma. In this paper, we investigate the techniques of foundation models and LLMs applied for autonomous driving, categorized as simulation, world model, data annotation and planning or E2E solutions etc.
Big Data in the Cloud: How the RISElab Enables Computers to Make Intelligent ...Amazon Web Services
Scientists, developers, and other technologists from many different industries are taking advantage of Amazon Web Services to perform big data workloads from analytics to using data lakes for better decision making to meet the challenges of the increasing volume, variety, and velocity of digital information. This session will feature UCB's RISELab (Real time Intelligent Secure Execution), a new lab recently created at UCB to enable computers to make intelligent, real-time decisions. You will hear how they are building on their earlier success with AMPLab to enable applications to interact intelligently and securely with their environment in real time, wherever computing decisions need to interact with the world. From cybersecurity to coordinating fleets of self-driving cars and drones to earthquake warning systems, you will come away with insight on how they are using AWS to develop and experiment with the systems for important research. Learn More: https://aws.amazon.com/government-education/
Perception.JS - A Framework for Context Acquisition Processing and PresentationSupun Dissanayake
Perception.js is a framework I have developed for my final research project for my Masters in Computer Science at University of Moratuwa. My research focused on developing a framework that will enable JavaScript developers to write context-awareness applications by enabling them to integrate various devices, gather data from those devices, specify rules for inferencing, and to respond to contextual changes.
Lessons learned moving from .NET to JavaScript, and from small startup to large enterprise. Presented at Node in the Wild meetup, Brightcove, Boston - 2014-12-03.
Following on from the success of last year, this annual event for London's architect community will have architectural innovation as a theme this year, and particularly CQRS. At the DDD eXchange we will feature leading thinkers and architects who will share their experience and Eric Evans is the programme lead.
Similar to Foundation Models in Recommender Systems (20)
Saudi Arabia stands as a titan in the global energy landscape, renowned for its abundant oil and gas resources. It's the largest exporter of petroleum and holds some of the world's most significant reserves. Let's delve into the top 10 oil and gas projects shaping Saudi Arabia's energy future in 2024.
Forklift Classes Overview by Intella PartsIntella Parts
Discover the different forklift classes and their specific applications. Learn how to choose the right forklift for your needs to ensure safety, efficiency, and compliance in your operations.
For more technical information, visit our website https://intellaparts.com
Harnessing WebAssembly for Real-time Stateless Streaming PipelinesChristina Lin
Traditionally, dealing with real-time data pipelines has involved significant overhead, even for straightforward tasks like data transformation or masking. However, in this talk, we’ll venture into the dynamic realm of WebAssembly (WASM) and discover how it can revolutionize the creation of stateless streaming pipelines within a Kafka (Redpanda) broker. These pipelines are adept at managing low-latency, high-data-volume scenarios.
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)MdTanvirMahtab2
This presentation is about the working procedure of Shahjalal Fertilizer Company Limited (SFCL). A Govt. owned Company of Bangladesh Chemical Industries Corporation under Ministry of Industries.
Final project report on grocery store management system..pdfKamal Acharya
In today’s fast-changing business environment, it’s extremely important to be able to respond to client needs in the most effective and timely manner. If your customers wish to see your business online and have instant access to your products or services.
Online Grocery Store is an e-commerce website, which retails various grocery products. This project allows viewing various products available enables registered users to purchase desired products instantly using Paytm, UPI payment processor (Instant Pay) and also can place order by using Cash on Delivery (Pay Later) option. This project provides an easy access to Administrators and Managers to view orders placed using Pay Later and Instant Pay options.
In order to develop an e-commerce website, a number of Technologies must be studied and understood. These include multi-tiered architecture, server and client-side scripting techniques, implementation technologies, programming language (such as PHP, HTML, CSS, JavaScript) and MySQL relational databases. This is a project with the objective to develop a basic website where a consumer is provided with a shopping cart website and also to know about the technologies used to develop such a website.
This document will discuss each of the underlying technologies to create and implement an e- commerce website.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
Water billing management system project report.pdfKamal Acharya
Our project entitled “Water Billing Management System” aims is to generate Water bill with all the charges and penalty. Manual system that is employed is extremely laborious and quite inadequate. It only makes the process more difficult and hard.
The aim of our project is to develop a system that is meant to partially computerize the work performed in the Water Board like generating monthly Water bill, record of consuming unit of water, store record of the customer and previous unpaid record.
We used HTML/PHP as front end and MYSQL as back end for developing our project. HTML is primarily a visual design environment. We can create a android application by designing the form and that make up the user interface. Adding android application code to the form and the objects such as buttons and text boxes on them and adding any required support code in additional modular.
MySQL is free open source database that facilitates the effective management of the databases by connecting them to the software. It is a stable ,reliable and the powerful solution with the advanced features and advantages which are as follows: Data Security.MySQL is free open source database that facilitates the effective management of the databases by connecting them to the software.
Welcome to WIPAC Monthly the magazine brought to you by the LinkedIn Group Water Industry Process Automation & Control.
In this month's edition, along with this month's industry news to celebrate the 13 years since the group was created we have articles including
A case study of the used of Advanced Process Control at the Wastewater Treatment works at Lleida in Spain
A look back on an article on smart wastewater networks in order to see how the industry has measured up in the interim around the adoption of Digital Transformation in the Water Industry.
Immunizing Image Classifiers Against Localized Adversary Attacksgerogepatton
This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks
(CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We
introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations.
When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves
the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach
using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10
and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing
accuracy improvements over previous techniques. The results indicate that the combination of the volumetric
input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating
adversary training.
We have compiled the most important slides from each speaker's presentation. This year’s compilation, available for free, captures the key insights and contributions shared during the DfMAy 2024 conference.
RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Hori...
Foundation Models in Recommender Systems
1. Zero Shot Recommenders,
LLMs and Prompt Engineering
PRS Workshop, Net
fl
ix, 2023
June 9th, 2023
Hao Ding (haodin haoding2019 ) and Anoop Deoras (adeoras )
AWS AI, Amazon
1
Towards Building Foundation Models in Recommender Systems
2. Our Mission at AWS
Put Machine Learning in the Hands of Every Developer
2
3. The AWS ML Stack
Broadest and Most Complete Set of ML Capabilities
GenAI
NEW
Bedrock
CodeWhisperer
3
4. Amazon Personalize
Who are we in a nutshell ?
• Customers can elevate the user experience with ML-powered personalization
• We cater to many thousands of customers from many diverse domains
• Such as: Retail, News and Media, Video on Demand, Travel and Hospitality, ..
• We provide recommendations that respond in real-time to changing user behavior
• In short, we provide the concierge service for all things personalization
4
6. Customer Obsessed Science
Applied Research at AWS AI
• Constantly innovating on behalf of the customers
• Amazon fundamentally believes that scienti
fi
c innovation is essential to being the most customer-
centric company in the world
• Science at Amazon enables new customer experiences, addresses existing customer pain points,
complements engineering and product disciplines.
6
7. 3 Anchors for the Discussion Today
ColdStart, Foundation Models in RecSys and LLMs
• Cold Start Problems in Recommender Systems
• Foundation Models in Recommender Systems
• Role Large Language Models (LLMs) can play in Recommender Systems
7
8. 3 Cold Start Problems in Recommender System
• Cold Users: Users during inference are unseen during training and model needs to generalize
• Cold Items: New items get introduced to catalogue
• Cold Domains: Target data available only for inference. No Models can be built.
• Less extreme case: Domains with very little training data / less frequent training cadence
• Performance of RecSys relies heavily on the amount of training data available
8
9. Foundation Models in Recommender Systems
Why should we talk about them ?
• De
fi
nition of a Foundation Model: A model trained on broad data that can be adapted to a wide range of
downstream tasks.
• Why Foundation Models in RecSys? Two main selling points:
• They encode “world knowledge”, thus complementary to models on domain’s behavioral data
• LLM Foundation Models’ interactive nature can potentially help with explaining away the recommendations
9
10. Two Approaches for Building Foundation Models
RecSys from Other Domains, Large Language Models
• We will talk about 2 research e
ff
ort
• ZeroShot Learning: Can we leverage the knowledge in one domain to kick start a
recommendation in a completely di
ff
erent domain
• ZeroShot Inference: We will further assume that we have no source domain to rely on. How can
we kick start a recommendation with large language models
10
12. The Status-Quo
Collaborative Filtering, Item IDs and their Embeddings
• Current RecSys models learn item ID embeddings through interactions
• Item ID Embeddings are parameters of your neural network and we learn them via BackProp
• These embeddings are indexed by categorical domain speci
fi
c item ID
• These are transductional and not generalizable to unseen items
12
13. Concept of Universal Item Embeddings
Collaborative Filtering, Item IDs and their Embeddings
• The idea behind universal item embeddings is to tap into item’s content information.
• e.g. Natural Language product description / movie synopsis etc
• Strong NLP models are used to obtain continuous universal item representations
• Universal user representations can then be built on top of these universal item representations.
13
14. Introducing ZESRec [1]
Zero Shot Recommender System
[1] “Zero Shot Recommender Systems”, Hao Ding, Anoop Deoras, Yuyang Wang, Hao Wang. ICLR Workshop 2022
• ZESRec learns the universal item embeddings based on domain-agnostic generic features — text;
• ZESRec adopts sequential recommenders which generates the universal user embeddings
14
15. We want to ask 2 questions about ZESRec
Relevance, Lead Time
• How relevant are ZESRec recommendations compared to a fully trained systems ?
• How much in domain data is needed to outperform ZESRec
• How much is the lead time ?
15
16. High Level Approach
ZESRec Training
SEQ
SEQ
SEQ
… User Universal
Embedding
1-Layer NN
Pretrained BERT
Model
X
1-Layer NN
Pretrained BERT
Model
…
0.36
0.29
…
0.09
0.02
Prediction
Score
Item Universal
Embedding
Pretrained BERT
Model
1-Layer NN
Item Universal
Embedding
Pretrained BERT
Model
1-Layer NN
Item Universal
Embedding
Item Universal
Embedding
Item Universal
Embedding
…
…
Latent Item
Offset Vector
+
Latent Item
Offset Vector
+
Latent Item
Offset Vector
+
Latent Item
Offset Vector
… Latent Item
Offset Vector
+
+
Latent User
Offset Vector
16
17. High Level Approach
ZESRec Inference
SEQ
SEQ
SEQ
… User Universal
Embedding
1-Layer NN
Pretrained BERT
Model
X
1-Layer NN
Pretrained BERT
Model
…
0.36
0.29
…
0.09
0.02
Prediction
Score
Item Universal
Embedding
Pretrained BERT
Model
1-Layer NN
Item Universal
Embedding
Pretrained BERT
Model
1-Layer NN
Item Universal
Embedding
Item Universal
Embedding
Item Universal
Embedding
…
…
17
19. Results
How long before In-Domain Model Takes over ?
19
10K 10K
5K
5K
2.5K 2.5K
0
0
Number of Interactions Number of Interactions
0.04
0.02
0
0.04
0.02
0
0.06
0.08
Recall@20 Recall@20
MIND dataset
Amazon dataset
21. From ZeroShot Learning to ZeroShot Inference
Task and Limitations
• Now lets imagine we don’t have the luxury of even having any source domain RecSys
• How realistic this assumption is ? Answer: Quite Realistic (startups, new business lines ..)
• What can we do ?
• There is no learning part left for ZeroShot Learning
• We need to resort to ZeroShot Inference
21
22. LLM Foundation Models to the rescue
Can we kick start recommendations using Large Language Models ?
• Pre-trained language models such as BERT and GPT learn general text representations
• They encode “world knowledge”
• Question we want to ask: Can we leverage these powerful LLMs as recommender systems
• Use prompts to reformulate session based recommendation task
22
23. Introducing LMRecSys[3]
Converting user’s interaction history into a text inquiry — Prompts
science fiction film directed by Peter Weir. The screenplay by Andrew Nicole was
adapted from Nicole’s 1997 novel of the same name. The film tells the story of
Truman Burbank, a man who is unwittingly placed in a televised reality show that
broadcasts every aspect of his life without his knowledge.
A user watched Jaws, Saving Private Ryan, The Good, the Bad, and the Ugly, Run Lola
Run, Goldfinger. Now the user may want to watch something funny and light-hearted
comfort him after having seen some horrors.
Knowledge
Reasoning
J1-Jumbo
Large Pre-trained Language
Model
(178B Parameters)
Bolded texts are generated by the
model.
A user watched Jaws, Saving Private Ryan, The Good, the Bad, and the Ugly, Run Lola Run, Goldfinger.
Now the user may want to watch __ __ __
p(d(xt)| f([d(x1), . . . , d(xt−1)]))
Item 372 Item 168 Item 413 Item 77 Item 952
p(xt |x1, . . . , xt−1)
Item 1
Item 2
Item N
…
Recommended Item
Token 1
Token 2
Token V
…
Token 1
Token 2
Token V
…
Token 1
Token 2
Token V
…
Item 1
Item 2
Item N
…
Recommended Item
Predicted Token Distributions from Language Models
Enable zero-shot recommendation
Improve data efficiency
Goal
GRU4Rec
Traditional Recommender System
LMRecSys
PLMs as Recommender System
[3] “Language Models as Recommender Systems: Evaluations and Limitations”,
Yuhui Zhang, Hao Ding, Zeren Shui, Yifei Ma, James Zou, Anoop Deoras, Hao Wang. NeurIPS Workshop 2021
23
24. Generation OR Multi-Token Inference
Answering the question of how to be faithful to one’s catalogue
• Sequence of item ID can be mapped to a long prompt
• How do we obtain ranked list of next item recommendation ?
• Generation of free form text — Need to be careful with Hallucination
• Probability Assignment on available catalogue
24
25. A Few Open Questions
Linguistic & Seq. Length Biases, Scales of LM and Creative Prompts
• Multi-Token Inference: Length normalization is important. Recommendations highly sensitive to
inference methods.
• Linguistic Biases Disentanglement: Item names need not be
fl
uent English.
• Scales of Language Models: Model size has signi
fi
cant impact on performance and latency
• Prompt Engineering: Its important to design the right prompts
25
27. The world after ChatGPT
Unleashing the immense power of Large Language Models
27
28. Recent Advances in Merging LLMs with RecSys
FineTuning an LLM
M6-Rec[5]:
P5[4]: designed a text to text
fi
ne-tuning
paradigm based on the pre-trained T5.
[4] “Recommendation as language processing (rlp): A uni
fi
ed pretrain, personalized prompt & predict paradigm (p5)”,
Geng Shijie et.al.. RecSys 2022
[5] “M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems”,
Zeyu Cui et.al.. ArXiv 2022
28
29. Recent Advances in Merging LLMs with RecSys
Inference with LLM
[6] "Zero-Shot Next-Item Recommendation using Large Pretrained Language Models." Wang, Lei, and Ee-Peng Lim. ArXiv 2023.
[7] “Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System”, Yunfan Gao et.al. ArXiv 2023
Zeyu Cui et.al.. ArXiv 2022
• NIR [6], Chat-REC[7] and [8] propose to directly recommend using LLMs — Inference only.
• Most e
ff
ort spent around “Prompt Engineering”
• Optimal encoding of user context in the prompts
• “Out of Vocabulary” problems solved using techniques such as candidate pools, text-matching
• Mixed success. Still a long way to go.
[8] “Is ChatGPT a Good Recommender? A Preliminary Study ”, Junling Liu et.al. ArXiv 2023
30. Concluding Remarks
• With the goal of building foundation models in RecSys, our e
ff
orts have been made in two directions:
• Extract Knowledge from data in similar domains
• Use Generic World Knowledge
• We believe, the ultimate path is the hybrid of both: ZESRec + LMRecSys
30