SlideShare a Scribd company logo
1 of 36
Download to read offline
Key projects in
AI, ML and
Generative AI
February 29
2024
This is a summary of my twelve years hands-on towards achieving the required experience, and skills in AI, ML and
Generative AI technologies. This includes key partner trainings from Google® and Microsoft®
Key projects in AI /ML and Generative AI
Project
No.
Project Name Project Summary / Learning
1 Simple Prompt Design
using Vertex AI
Zero –shot prompting, and Context based Chat in Vertex AI, using text-bison@001 model
2 Sentiment and Text
Analysis
Lab to query the LLM using Vertex AI for sentiment analysis and keyword extraction from a Japanese file
3 Image Classification Lab to train Vertex ML Pipeline to classify images, here flowers; used flower dataset with 1,000 images to
train the model. Model accuracy was about 97%.
4 Video classification Video classification using AutoML training , use of evaluation like confusion matrix for precision measurement
5 Improving data
quality
This notebook introduced a few concepts to improve data quality. We resolved missing values, converted the
Date feature column to a datetime format, renamed feature columns, removed a value from a feature
column, and created one-hot encoding features for feeding to ML algorithm. Pandas, dataframes, seaborn
packages used. Key ones are listed below:
a) Handling Categorical Columns
The feature column "lightduty" is categorical and has a "Yes/No" choice. We cannot feed values like
this into a machine learning model. We need to convert the binary answers from strings of yes/no to
integers of 1/0. There are various methods to achieve this. We will use the "apply" method with a
lambda expression. Pandas. apply() takes a function and applies it to all values of a Pandas series
b) Machine learning algorithms expect input vectors and not categorical features. Specifically, they
cannot handle text or string values. Thus, it is often useful to transform categorical features into
vectors.
One transformation method is to create dummy variables for our categorical features. Dummy
variables are a set of binary (0 or 1) variables that each represent a single class from a categorical
feature. We simply encode the categorical variable as a one-hot vector, i.e. a vector where only one
element is non-zero, or hot. With one-hot encoding, a categorical feature becomes an array whose
size is the number of possible choices for that feature.
6 Exploratory data
analysis using
BigQuery
EDA using linear regression using Python and Scikit-Learn, heatmaps for predicting US house value US and
taxi fare estimation
7 Using the BigQuery
ML Hyperparameter
Tuning to Improve
Model Performance
This lab introduces data analysts to BigQuery ML. BigQuery ML enable users to create and execute machine
learning models in BigQuery using SQL queries. In this lab, you use the “tlc_yellow_trips_2018 sample table”
to create a model that predicts the tip for a taxi ride. You will see a ~40% performance (r2_score)
improvement with hyperparameter tuning
8 TensorFlow Dataset
API
Linear Regression Model uses data from a`tf.data.Dataset`, and we will learn how to implement stochastic
gradient descent using loss function, and gradient. Then we write production input pipelines with feature
engineering (batching, shuffling, etc.).
9 Classifying Structured
Data using Keras
In this notebook, you learn how to classify structured data (e.g. tabular data in a CSV). You will use Keras to
define the model, and preprocessing layers as a bridge to map from columns in a CSV to features used to
Preprocessing Layers train the model. Use of numpy, pandas, sklearn libraries.
The Keras preprocessing layers API allows you to build Keras-native input processing pipelines. You will use
preprocessing layers to demonstrate the feature preprocessing code.
1. Normalization- Feature-wise normalization of the data.
2. CategoryEncoding - Category encoding layer.
3. StringLookup - Maps strings from a vocabulary to integer indices.
4. IntegerLookup- Maps integers from a vocabulary to integer indices.
10 Keras Sequential API
on Vertex AI Platform
In this lab, we'll see how to build a simple deep neural network model using the Keras Sequential API and
feature columns. Once we have trained our model, we will deploy it using Vertex AI and see how to call our
model for online prediction.
We will use feature columns to connect our raw data to our Keras DNN model. Feature columns make it easy
to perform common types of feature engineering on your raw data. For example, you can one-hot encode
categorical data, create feature crosses, embeddings and more.
11 Build a DNN using the
Keras Functional API
In this notebook, we will build a Keras DNN to predict the fare amount for NYC taxi cab rides
1. Review how to read in CSV file data using tf.data.
2. Specify input, hidden, and output layers in the DNN architecture.
3. Review and visualize the final DNN shape.
4. Train the model locally and visualize the loss curves.
5. Deploy and predict with the model using Cloud AI Platform.
12 Performing Basic
Feature Engineering
in BQML
In this lab, we utilize feature engineering to improve the prediction of the fare amount for a taxi ride in New
York City. We will use BigQuery ML to build a taxifare prediction model, using feature engineering to
improve and create a final model.
In this Notebook we set up the environment, create the project dataset, create a feature engineering table,
create and evaluate a baseline model, extract temporal features, perform a feature cross on temporal
features, and evaluate model performance throughout the process.
In this lab, you:
• Create SQL statements to evaluate the model
• Extract temporal features
• Perform a feature cross on temporal features
Allow the model to learn traffic patterns by creating a new feature that combines the time of day and day of
week (this is called a [feature cross]. Create the SQL statement to feature cross the dayofweek and
hourofday using the CONCAT function
Few of the evaluation metrics:
1. mean_absolute_error
2. mean_squared_error
3. mean_squared_log_error
4. median_absolute_error
5. r2_score
RMSE results
13 Performing Basic
Feature Engineering
in Keras
In this lab, we utilize feature engineering to improve the prediction of housing prices using a Keras
Sequential Model.
1. Create an input pipeline using tf.data.
2. Engineer features to create categorical, crossed, and numerical feature columns.
It is very important for numerical variables to get scaled before they are "fed" into the neural network. Here
we use min-max scaling. We cannot feed strings directly to a model. Instead, we must first map them to
numeric values. The categorical vocabulary columns provide a way to represent strings as a one-hot vector.
Often, you don't want to feed a number directly into the model, but instead split its value into different
categories based on numerical ranges. Consider our raw data that represents a homes' age. Instead of
representing the house age as a numeric column, we could split the home age into several buckets using a
bucketized column. Combining features into a single feature, better known as feature crosses, enables a
model to learn separate weights for each combination of features.
14 Exploring and
Creating an
Ecommerce Analytics
Pipeline with Cloud
Dataprep v1.5
Cloud Dataprep® by Trifacta® is an intelligent data service for visually exploring, cleaning, and preparing
structured and unstructured data for analysis. In this lab we will explore the Cloud Dataprep UI to build an
ecommerce transformation pipeline that will run at a scheduled interval and output results back into
BigQuery.
The dataset we will be using is an ecommerce dataset that has millions of Google® Analytics records for the
Google® Merchandise Store loaded into BigQuery
In this lab, you learn how to perform these tasks:
 Connect BigQuery datasets to Cloud Dataprep
 Explore dataset quality with Cloud Dataprep
 Create a data transformation pipeline with Cloud Dataprep
 Schedule transformation jobs outputs to BigQuery
Analytics Pipeline
Recipe Book with rules
Results with duplicate rows removed
15 ML pipelines with
Vertex Pipelines
In this lab, you learn how to create and run ML pipelines with Vertex Pipelines.
Objectives
• Use the Kubeflow Pipelines SDK to build scalable ML pipelines.
• Create and run a 3-step intro pipeline that takes text input.
Few of the components:
• Kubeflow Pipelines: This is the SDK we'll be using to build our pipeline. Vertex Pipelines supports
running pipelines built with either Kubeflow Pipelines or TFX.
• Google® Cloud Pipeline Components: This library provides pre-built components that make it easier
to interact with Vertex AI services from your pipeline steps.
We'll create a pipeline that prints out a sentence using two outputs: a product name and an emoji
description. This pipeline will consist of three components:
1. product_name: This component will take a product name (or any noun you want really) as input, and
return that string as output
2. emoji: This component will take the text description of an emoji and convert it to an emoji. For
example, the text code for * is "sparkles". This component uses an emoji library to show you how to
manage external dependencies in your pipeline
3. build_sentence: This final component will consume the output of the previous two to build a sentence
that uses the emoji. For example, the resulting output might be "Vertex Pipelines is *"
16 Advanced
Visualizations with
TensorFlow Data
Validation
This lab illustrates how TensorFlow Data Validation (TFDV) can be used to investigate and visualize your
dataset. That includes looking at descriptive statistics, inferring a schema, checking for and fixing anomalies,
and checking for drift and skew in our dataset
First we'll use `tfdv.generate_statistics_from_csv` to compute statistics for our training data.
TFDV can compute descriptive statistics that provide a quick overview of the data in terms of the features
that are present and the shapes of their value distributions. Now let's use [`tfdv.infer_schema`] to create a
schema for our data.
Does our evaluation dataset match the schema from our training dataset? This is especially important for
categorical features, where we want to identify the range of acceptable values.
Drift detection is supported for categorical features and between consecutive spans of data (i.e., between
span N and span N+1), such as between different days of training data. We express drift in terms of [L-
infinity distance], and you can set the threshold distance so that you receive warnings when the drift is
higher than is acceptable.
Adding skew and drift comparators to visualize and make corrections. Few of the uses are:
1. Validating new data for inference to make sure that we haven't suddenly started receiving bad
features
2. Validating new data for inference to make sure that our model has trained on that part of the decision
surface
3. Validating our data after we've transformed it and done feature engineering (probably using
[TensorFlow Transform] to make sure we haven't done something wrong
17 Distributed Training
with Keras
We learn:
1. How to define distribution strategy and set input pipeline.
2. How to create the Keras model.
3. How to define the callbacks.
4. How to train and evaluate the model.
The tf.distribute.Strategy API provides an abstraction for distributing your training across multiple processing
units. The goal is to allow users to enable distributed training using existing models and training code, with
minimal changes.
This lab uses the tf.distribute.MirroredStrategy, which does in-graph replication with synchronous training on
many GPUs on one machine. Essentially, it copies all of the model's variables to each processor. Then, it uses
all-reduce to combine the gradients from all processors and applies the combined value to all copies of the
model. MirroredStrategy is one of several distribution strategies available in TensorFlow core.
The callbacks used here are:
1. TensorBoard: This callback writes a log for TensorBoard which allows you to visualize the graphs.
2. Model Checkpoint: This callback saves the model after every epoch.
3. Learning Rate Scheduler: Using this callback, you can schedule the learning rate to change after
every epoch/batch.
18 TPU Speed Data
Pipelines
TPUs are very fast, and the stream of training data must keep up with their training speed. In this lab, you
will learn how to load data from Cloud Storage with the tf.data.Dataset API to feed your TPU
You will learn:
• To use the tf.data.Dataset API to load training data.
• To use TFRecord format to load training data efficiently from Cloud Storage.
19 Detecting Labels,
Faces, and Landmarks
in Images with the
Cloud Vision API
The Cloud Vision API lets you understand the content of an image by encapsulating powerful machine
learning models in a simple REST API.
In this lab, you send images to the Vision API and see it detect objects, faces, and landmarks
In this lab, you learn how to perform the following tasks:
 Create a Vision API request and call the API with curl.
 Use the label, face, and landmark detection methods of the vision API
20 Classifying Images
using Dropout and
Batchnorm Layer
In this lab, you learn how to build a neural network to classify the tf-flowers dataset using dropout and
batchnorm layer.
Learning objectives
• Define Helper Functions.
• Apply dropout and batchnorm layer
Dropouts are the regularization technique that is used to prevent overfitting in the model. Batch
normalization is a layer that allows every layer of the network to do learning more independently. The layer
is added to the sequential model to standardize the input or the outputs. Add a dropout and batchnorm layer
after each of the hidden layers.
21 Classifying Images
with Transfer
Learning
In this lab, you learn how to build a neural network to classify the tf-flowers (5 flowers) dataset by using a
pre-trained image embedding. You load a pre-trained model which is trained on very large, general-purpose
datasets and transfer that knowledge to the actual dataset that you want to classify. This means you use a
pre-trained model instead of the Flattened layer as your first layer
You learn how to apply data augmentation in two ways:
• Understand how to set up preprocessing in order to convert image type and resize the image to the
desired size.
• Understand how to implement transfer learning with MobileNet.
Pre-trained models are models that are trained on large datasets and made available to be used as a way to
create embeddings. For example, the [MobileNet model] is a model with 1-4 million parameters that was
trained on the [ImageNet (ILSVRC) dataset] which consists of millions of images corresponding to hundreds
of categories that were scraped from the web. The resulting embedding therefore has the ability to efficiently
compress the information found in a wide variety of images. As long as the images you want to classify are
similar in nature to the ones that MobileNet was trained on, the embeddings from MobileNet should give a
great pre-trained embedding that you can use as a starting point to train a model on your smaller tf-flowers
(5 flowers) dataset. A pre-trained MobileNet is available on TensorFlow Hub and you can easily load it as a
Keras layer by passing in the URL to the trained model.
22 Text classification
using reusable
embeddings
In this lab, you implement text models to recognize the probable source (GitHub, Tech-Crunch, or The New-
York Times) of titles present in the title dataset, which are created in the respective labs.
Learning objectives
In this lab, you learn how to:
• Use pre-trained TF Hub text modules to generate sentence vectors.
• Incorporate a pre-trained TF-Hub module into a Keras model.
• Deploy and use a text model on CAIP
In this lab, we will use pre-trained [TF-Hub embeddings modules for English] for the first layer of our
models. One immediate advantage of doing so is that the TF-Hub embedding module will take care for us of
processing the raw text. This also means that our model will be able to consume text directly instead of
sequences of integers representing the words. However, as before, we still need to preprocess the labels into
one-hot-encoded vectors
We will first try a word embedding pre-trained using a [Neural Probabilistic Language Model]. TF-Hub has a
50-dimensional one called [nnlm-en-dim50-with-normalization], which also normalizes the vectors produced.
Once loaded from its URL, the TF-hub module can be used as a normal Keras layer in a sequential or
functional model. Since we have enough data to fine-tune the parameters of the pre-trained embedding
itself, we will set `trainable=True` in the `KerasLayer` that loads the pre-trained embedding
Then we will try a word embedding obtained using [Swivel], an algorithm that essentially factorizes word co-
occurrence matrices to create the words embeddings. TF-Hub hosts the pretrained [gnews-swivel-20dim-
with-oov], 20-dimensional Swivel module.
Swivel trains faster but achieves lower validation accuracy, and requires more epochs to train on.
23 RNN Encoder Decoder
for Translation
In this notebook, you will use encoder-decoder architecture to create a text translation function.
In this lab, you will:
• Create a tf.data.Dataset for a seq2seq problem.
• Train an encoder-decoder model in Keras for a translation task.
• Save the encoder and the decoder as separate model.
• Learn how to piece together the trained encoder and decoder into a translation function
 Learn how to use the BLUE score to evaluate a translation model
We will start by creating train and eval datasets (using the `tf.data.Dataset` API) that are typical for seq2seq
problems. Then we will use the Keras functional API to train an RNN encoder-decoder model, which will save
as two separate models, the encoder and decoder model. Using these two separate pieces we will implement
the translation function. At last, we'll benchmark our results using the industry standard BLEU score.
The `utils_preproc.preprocess_sentence()` method does the following:
1. Converts sentence to lower case
2. Adds a space between punctuation and words
3. Replaces tokens that aren't a-z or punctuation with space
4. Adds `<start>` and `<end>` tokens
The `utils_preproc.tokenize()` method does the following:
1. Splits each sentence into a token list
2. Maps each token to an integer
3. Pads to length of longest sentence
It returns an instance of a [Keras Tokenizer] containing the token-integer mapping along with the integerized
sentences
We use an encoder-decoder architecture, however we embed our words into a latent space prior to feeding
them into the RNN.
Next we implement the encoder network with Keras functional API. It will
 start with an `Input` layer that will consume the source language integerized sentences
 then feed them to an `Embedding` layer of `EMBEDDING_DIM` dimensions
 which in turn will pass the embeddings to a `GRU` recurrent layer with `HIDDEN_UNITS`
The output of the encoder will be the `encoder_outputs` and the `encoder_state`.
Next, implement the decoder network, which is very similar to the encoder network.
It will
 start with an `Input` layer that will consume the source language integerized sentences
 then feed that input to an `Embedding` layer of `EMBEDDING_DIM` dimensions
 which in turn will pass the embeddings to a `GRU` recurrent layer with `HIDDEN_UNITS`
**Important:** The main difference with the encoder, is that the recurrent `GRU` layer will take as input
not only the decoder input embeddings, but also the `encoder_state` as outputted by the encoder above.
This is where the two networks are linked!
The output of the encoder will be the `decoder_outputs` and the `decoder_state`
The last part of the encoder-decoder architecture is a softmax `Dense` layer that will create the next word
probability vector or next word `predictions` from the `decoder_output'
We can't just use model.predict(), because we don't know all the inputs we used during training. We only
know the encoder_input (source language) but not the decoder_input (target language), which is what we
want to predict (i.e., the translation of the source language)!
We do however know the first token of the decoder input, which is the `<start>` token. So using this plus
the state of the encoder RNN, we can predict the next token. We will then use that token to be the second
token of decoder input, and continue like this until we predict the `<end>` token, or we reach some defined
max length.
So, the strategy now is to split our trained network into two independent Keras models:
1. an encoder model with signature `encoder_inputs -> encoder_state`
2. a decoder model with signature `[decoder_inputs, decoder_state_input] -> [predictions,
decoder_state]`
Given that input, the decoder will produce the first word of the translation, by sampling from the
`predictions` vector (for simplicity, our sampling strategy here will be to take the next word to be the one
whose index has the maximum probability in the `predictions` vector) along with a new state vector, the
`decoder_state`.
At this point, we can feed again to the decoder the predicted first word and as well as the new
`decoder_state` to predict the translation second word.
This process can be continued until the decoder produces the token `<stop>`.
Now that we have a separate encoder and a separate decoder, implement a translation function, to which we
will give the generic name of `decode_sequences` (to stress that this procedure is general to all seq2seq
problems).
`decode_sequences` will take as input
 `input_seqs` which is the integerized source language sentence tensor that the encoder can consume
 `output_tokenizer` which is the target languague tokenizer we will need to extract back words from
predicted word integers
 * `max_decode_length` which is the length after which we stop decoding if the `<stop>` token has
not been predicted
Many attempts have been made to develop a better metric for natural language evaluation. The most
popular currently is Bilingual Evaluation Understudy (BLEU).
- It is quick and inexpensive to calculate.
- It allows flexibility for the ordering of words and phrases.
- It is easy to understand.
- It is language independent.
- It correlates highly with human evaluation.
- It has been widely adopted.
The score is from 0 to 1, where 1 is an exact match.
It works by counting matching n-grams between the machine and reference texts, regardless of order. BLUE-
4 counts matching n grams from 1-4 (1-gram, 2-gram, 3-gram and 4-gram). It is common to report both
BLUE-1 and BLUE-4
24 Hybrid
Recommendations
with the MovieLens
Dataset
The matrix factorization approach does not use any information about users or movies beyond what is
available from the ratings matrix. However, we will often have user information (such as the city they live,
their annual income, their annual expenditure, etc.) and we will almost always have more information about
the products in our catalog. How do we incorporate this information in our recommendation model?
The answer lies in recognizing that the user factors and product factors that result from the matrix
factorization approach end up being a concise representation of the information about users and products
available from the ratings matrix. We can concatenate this information with other information we have
available and train a regression model to predict the rating
In this lab, you will:
 Know how to extract user and product factors from a BigQuery Matrix Factorizarion Model
 Know how to format inputs for a BigQuery Hybrid Recommendation Model
25 TFX on Cloud AI
Platform Pipelines
In this lab, you use utilize the following tools and services to deploy and run a TFX pipeline on Google®
Cloud that automates the development and deployment of a TensorFlow 2.3 WideDeep Classifier to predict
forest cover from cartographic data. In this notebook, you will work with the [Covertype Data Set and use
TFX to analyze, understand, and pre-process the dataset and train, analyze, validate, and deploy a multi-
class classification model to predict the type of forest cover from cartographic features
Learning objectives
1. Develop a high level understanding of TFX pipeline components.
2. Learn how to use a TFX Interactive Context for prototype development of TFX pipelines.
3. Work with the Tensorflow Data Validation (TFDV) library to check and analyze input data.
4. Utilize the Tensorflow Transform (TFT) library for scalable data preprocessing and feature
transformations.
5. Employ the Tensorflow Model Analysis (TFMA) library for model evaluation.
26 Continuous Training
Pipelines with Cloud
Composer
In this lab you will learn how to write an Airflow DAG for continuous training and deploy the DAG within a
Cloud Composer environment. You will also learn how to explore and monitor your DAG runs using the
Apache Airflow webserver.
Objectives
In this lab, you will learn to perform the following tasks:
 Provision a Cloud Composer environment.
 Deploy an Apache Airflow Dialog.
 Monitor a continuous training pipeline in the Airflow webserver.
 Explore Airflow logs using Cloud Operations.
27 Image Captioning
with Visual Attention
Image captioning models take an image as input, and output text. Ideally, we want the output of the model
to accurately describe the events/things in the image, similar to a caption a human might provide. In order to
generate text, we will build an encoder-decoder model, where the encoder output embedding of an input
image, and the decoder output text from the image embedding
## Learning Objectives
1. Learn how to create an image captioning model
2. Learn how to train and predict a text generation model.
We will use the TensorFlow datasets capability to read the [COCO captions]
(https://www.tensorflow.org/datasets/catalog/coco_captions) dataset. This version contains images,
bounding boxes, labels, and captions from COCO 2014, split into the subsets defined by Karpathy and Li
(2015)
You will transform the text captions into integer sequences using the [TextVectorization] layer
Now let's design an image captioning model. It consists of an image encoder, followed by a caption decoder.
The image encoder model is very simple. It extracts features through a pre-trained model and passes them
to a fully connected layer. The caption decoder incorporates an attention mechanism that focuses on
different parts of the input image. The decoder uses attention to selectively focus on parts of the input
sequence. The attention takes a sequence of vectors as input for each example and returns an "attention"
vector for each example
The decoder's job is to generate predictions for the next output token.
1. The decoder receives current word tokens as a batch.
2. It embeds the word tokens to `ATTENTION_DIM` dimension.
3. GRU layer keeps track of the word embeddings, and returns GRU outputs and states.
4. Bahdanau-style attention attends over the encoder's output feature by using GRU outputs as a query.
5. The attention outputs and GRU outputs are added (skip connection), and normalized in a layer
normalization layer.
6. It generates logit predictions for the next token based on the GRU output.
We can define all the steps in Keras Functional API, but please note that here we instantiate layers that have
trainable parameters so that we reuse the layers and the weights in inference phase.
The predict step is different from the training, since we need to keep track of the GRU state during the
caption generation, and pass a predicted word to the decoder as an input at the next time step. In order to
do so, let's define another model for prediction while using the trained weights, so that it can keep and
update the GRU state during the caption generation.
Results:
28 Text generation using
RNN
This tutorial demonstrates how to generate text using a character-based RNN. You will work with a dataset
of Shakespeare's writing from Andrej Karpathy's [The Unreasonable Effectiveness of Recurrent Neural
Networks].Given a sequence of characters from this data ("Shakespear"), train a model to predict the next
character in the sequence ("e"). Longer sequences of text can be generated by calling the model repeatedly.
Build a model with the following layers
1. `tf.keras.layers.Embedding`: The input layer. A trainable lookup table that will map each character-ID
to a vector with `embedding_dim` dimensions;
2. `tf.keras.layers.GRU`: A type of RNN with size `units=rnn_units` (You can also use an LSTM layer
here.)
3. `tf.keras.layers.Dense`: The output layer, with `vocab_size` outputs. It outputs one logit for each
character in the vocabulary. These are the log-likelihood of each character according to the model.
For each character the model looks up the embedding, runs the GRU one timestep with the embedding as
input, and applies the dense layer to generate logits predicting the log-likelihood of the next character. At
this point the problem can be treated as a standard classification problem. Given the previous RNN state, and
the input this time step, predict the class of the next character.
The simplest way to generate text with this model is to run it in a loop, and keep track of the model's internal
state as you execute it.
Each time you call the model you pass in some text and an internal state. The model returns a prediction for
the next character and its new state. Pass the prediction and state back in to continue generating text.
Sample result:
29 Classify text with
BERT
Learning Objectives
1. Learn how to load a pre-trained BERT model from TensorFlow Hub
2. Learn how to build your own model by combining with a classifier
3. Learn how to train a BERT model by fine-tuning
4. Learn how to save your trained model and use it
5. Learn how to evaluate a text classification model
This lab will show you how to fine-tune BERT to perform sentiment analysis on a dataset of plain-text IMDB
movie reviews. In addition to training a model, you will learn how to preprocess text into an appropriate
format.
30 Decision making using
Dialogflow CX
generators
In this lab, you learn how to perform the following tasks:
 Use Dialogflow AI Agent to create a chat app and add unstructured data to a data store
 Use knowledge handlers to allow end-users to have conversations with a virtual agent about the
content added to a data store.
 Configure a generator text prompt and make it contextual by using built-in generator prompt
placeholders.
 Mark words as generator prompt placeholders and later associate them with session parameters in
fulfillment to use their values during execution.
 Configure a generator to handle responses that involve knowledge from a large textual dataset and
context from the current conversation.
 Generate a formal email using generators
 Test your agent and simulate customer questions that trigger generated responses
The Vertex AI Conversation feature creates a special Dialogflow agent, called a data store agent.
With this feature, you provide a website URL, structured data or unstructured data (data stores), then
Google® parses your content and creates a virtual agent that is powered by data stores and large language
models. Your customers and end users can then have conversations with the agent and ask questions about
the content.
The generator feature is a Dialogflow CX feature that allows developers to use Google's latest generative
large language models (LLMs) and custom prompts to generate agent responses at runtime
The generative fallback feature uses Google's latest generative large language models (LLMs) to generate
virtual agent responses when end-user input does not match an intent or parameter for form filling. The
feature can be configured with a text prompt that instructs the LLM how to respond. You can use a
predefined text prompt or add your own prompts. You can enable generative fallback on no-match event
handlers used in flows, pages, or during parameter filling.
Chat Reponse
31 Bard API /Gemini Features of Bard and Gemini models
Greet with name and time
Marketing Blog
Summarizing Article
Generate creative prompt
32 Microsoft® Copilot
Studio BOT using
Power Platform
GitHub Copilot CODE Suggestions
Bing Chat
Disclaimer:
We have sourced the content from various courses and partner trainings. All details, references are for educational purposes only
• Intel®
is trademark of the Intel®
Corporation. Microsoft®
and Microsoft Azure®
are trademarks of the Microsoft®
group of companies. Framer®
is
the trademark of Framer®
Incorporation.
• All logos, trademarks and brand names belong to the respective owners as specified. We have no intention to infringe any copyrights or alter
related permissions set by the owners. Please refer to source websites for any further details. This is for educational and information purpose
only.
For more details, contact:
Bhadale IT Pvt. Ltd; Email: vijaymohire@gmail.com

More Related Content

Similar to Key projects in AI, ML and Generative AI

Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...Databricks
 
Unlocking the Potential of AI in Spring.pdf
Unlocking the Potential of AI in Spring.pdfUnlocking the Potential of AI in Spring.pdf
Unlocking the Potential of AI in Spring.pdfInexture Solutions
 
Practical data science
Practical data sciencePractical data science
Practical data scienceDing Li
 
Introduction to Machine Learning with SciKit-Learn
Introduction to Machine Learning with SciKit-LearnIntroduction to Machine Learning with SciKit-Learn
Introduction to Machine Learning with SciKit-LearnBenjamin Bengfort
 
.NET Portfolio
.NET Portfolio.NET Portfolio
.NET Portfoliomwillmer
 
Bhadale Group of Companies - digital projects
Bhadale Group of Companies - digital projectsBhadale Group of Companies - digital projects
Bhadale Group of Companies - digital projectsVijayananda Mohire
 
Simplifying the Creation of Machine Learning Workflow Pipelines for IoT Appli...
Simplifying the Creation of Machine Learning Workflow Pipelines for IoT Appli...Simplifying the Creation of Machine Learning Workflow Pipelines for IoT Appli...
Simplifying the Creation of Machine Learning Workflow Pipelines for IoT Appli...ScyllaDB
 
(Py)testing the Limits of Machine Learning
(Py)testing the Limits of Machine Learning(Py)testing the Limits of Machine Learning
(Py)testing the Limits of Machine LearningRebecca Bilbro
 
Visualizing Model Selection with Scikit-Yellowbrick: An Introduction to Devel...
Visualizing Model Selection with Scikit-Yellowbrick: An Introduction to Devel...Visualizing Model Selection with Scikit-Yellowbrick: An Introduction to Devel...
Visualizing Model Selection with Scikit-Yellowbrick: An Introduction to Devel...Benjamin Bengfort
 
49.INS2065.Computer Based Technologies.TA.NguyenDucAnh.pdf
49.INS2065.Computer Based Technologies.TA.NguyenDucAnh.pdf49.INS2065.Computer Based Technologies.TA.NguyenDucAnh.pdf
49.INS2065.Computer Based Technologies.TA.NguyenDucAnh.pdfcNguyn506241
 
PL SQLDay Machine Learning- Hands on ML.NET.pptx
PL SQLDay Machine Learning- Hands on ML.NET.pptxPL SQLDay Machine Learning- Hands on ML.NET.pptx
PL SQLDay Machine Learning- Hands on ML.NET.pptxLuis Beltran
 
Christine_Straub - ML Engineer.pdf
Christine_Straub - ML Engineer.pdfChristine_Straub - ML Engineer.pdf
Christine_Straub - ML Engineer.pdfChristine Straub
 
Augmenting Machine Learning with Databricks Labs AutoML Toolkit
Augmenting Machine Learning with Databricks Labs AutoML ToolkitAugmenting Machine Learning with Databricks Labs AutoML Toolkit
Augmenting Machine Learning with Databricks Labs AutoML ToolkitDatabricks
 
House price prediction
House price predictionHouse price prediction
House price predictionSabahBegum
 
Infrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload DeploymentInfrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload DeploymentDatabricks
 
Vertex AI: Pipelines for your MLOps workflows
Vertex AI: Pipelines for your MLOps workflowsVertex AI: Pipelines for your MLOps workflows
Vertex AI: Pipelines for your MLOps workflowsMárton Kodok
 
Adopting AnswerModules ModuleSuite
Adopting AnswerModules ModuleSuiteAdopting AnswerModules ModuleSuite
Adopting AnswerModules ModuleSuiteAnswerModules
 
Productionizing Machine Learning - Bigdata meetup 5-06-2019
Productionizing Machine Learning - Bigdata meetup 5-06-2019Productionizing Machine Learning - Bigdata meetup 5-06-2019
Productionizing Machine Learning - Bigdata meetup 5-06-2019Iulian Pintoiu
 
Machine Learning Pipelines - Joseph Bradley - Databricks
Machine Learning Pipelines - Joseph Bradley - DatabricksMachine Learning Pipelines - Joseph Bradley - Databricks
Machine Learning Pipelines - Joseph Bradley - DatabricksSpark Summit
 

Similar to Key projects in AI, ML and Generative AI (20)

Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
Analytics Zoo: Building Analytics and AI Pipeline for Apache Spark and BigDL ...
 
Unlocking the Potential of AI in Spring.pdf
Unlocking the Potential of AI in Spring.pdfUnlocking the Potential of AI in Spring.pdf
Unlocking the Potential of AI in Spring.pdf
 
Practical data science
Practical data sciencePractical data science
Practical data science
 
Introduction to Machine Learning with SciKit-Learn
Introduction to Machine Learning with SciKit-LearnIntroduction to Machine Learning with SciKit-Learn
Introduction to Machine Learning with SciKit-Learn
 
.NET Portfolio
.NET Portfolio.NET Portfolio
.NET Portfolio
 
Bhadale Group of Companies - digital projects
Bhadale Group of Companies - digital projectsBhadale Group of Companies - digital projects
Bhadale Group of Companies - digital projects
 
Simplifying the Creation of Machine Learning Workflow Pipelines for IoT Appli...
Simplifying the Creation of Machine Learning Workflow Pipelines for IoT Appli...Simplifying the Creation of Machine Learning Workflow Pipelines for IoT Appli...
Simplifying the Creation of Machine Learning Workflow Pipelines for IoT Appli...
 
(Py)testing the Limits of Machine Learning
(Py)testing the Limits of Machine Learning(Py)testing the Limits of Machine Learning
(Py)testing the Limits of Machine Learning
 
Visualizing Model Selection with Scikit-Yellowbrick: An Introduction to Devel...
Visualizing Model Selection with Scikit-Yellowbrick: An Introduction to Devel...Visualizing Model Selection with Scikit-Yellowbrick: An Introduction to Devel...
Visualizing Model Selection with Scikit-Yellowbrick: An Introduction to Devel...
 
49.INS2065.Computer Based Technologies.TA.NguyenDucAnh.pdf
49.INS2065.Computer Based Technologies.TA.NguyenDucAnh.pdf49.INS2065.Computer Based Technologies.TA.NguyenDucAnh.pdf
49.INS2065.Computer Based Technologies.TA.NguyenDucAnh.pdf
 
PL SQLDay Machine Learning- Hands on ML.NET.pptx
PL SQLDay Machine Learning- Hands on ML.NET.pptxPL SQLDay Machine Learning- Hands on ML.NET.pptx
PL SQLDay Machine Learning- Hands on ML.NET.pptx
 
Christine_Straub - ML Engineer.pdf
Christine_Straub - ML Engineer.pdfChristine_Straub - ML Engineer.pdf
Christine_Straub - ML Engineer.pdf
 
Vedic Calculator
Vedic CalculatorVedic Calculator
Vedic Calculator
 
Augmenting Machine Learning with Databricks Labs AutoML Toolkit
Augmenting Machine Learning with Databricks Labs AutoML ToolkitAugmenting Machine Learning with Databricks Labs AutoML Toolkit
Augmenting Machine Learning with Databricks Labs AutoML Toolkit
 
House price prediction
House price predictionHouse price prediction
House price prediction
 
Infrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload DeploymentInfrastructure Agnostic Machine Learning Workload Deployment
Infrastructure Agnostic Machine Learning Workload Deployment
 
Vertex AI: Pipelines for your MLOps workflows
Vertex AI: Pipelines for your MLOps workflowsVertex AI: Pipelines for your MLOps workflows
Vertex AI: Pipelines for your MLOps workflows
 
Adopting AnswerModules ModuleSuite
Adopting AnswerModules ModuleSuiteAdopting AnswerModules ModuleSuite
Adopting AnswerModules ModuleSuite
 
Productionizing Machine Learning - Bigdata meetup 5-06-2019
Productionizing Machine Learning - Bigdata meetup 5-06-2019Productionizing Machine Learning - Bigdata meetup 5-06-2019
Productionizing Machine Learning - Bigdata meetup 5-06-2019
 
Machine Learning Pipelines - Joseph Bradley - Databricks
Machine Learning Pipelines - Joseph Bradley - DatabricksMachine Learning Pipelines - Joseph Bradley - Databricks
Machine Learning Pipelines - Joseph Bradley - Databricks
 

More from Vijayananda Mohire

Certificate- Peer Review of Book Chapter on ML
Certificate- Peer Review of Book Chapter on MLCertificate- Peer Review of Book Chapter on ML
Certificate- Peer Review of Book Chapter on MLVijayananda Mohire
 
Bhadale IT Hub-Multi Cloud and Multi QAI
Bhadale IT Hub-Multi Cloud and Multi QAIBhadale IT Hub-Multi Cloud and Multi QAI
Bhadale IT Hub-Multi Cloud and Multi QAIVijayananda Mohire
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIVijayananda Mohire
 
Azure Quantum Workspace for developing Q# based quantum circuits
Azure Quantum Workspace for developing Q# based quantum circuitsAzure Quantum Workspace for developing Q# based quantum circuits
Azure Quantum Workspace for developing Q# based quantum circuitsVijayananda Mohire
 
My Journey towards Artificial Intelligence
My Journey towards Artificial IntelligenceMy Journey towards Artificial Intelligence
My Journey towards Artificial IntelligenceVijayananda Mohire
 
Bhadale IT Cloud Solutions for Agriculture
Bhadale IT Cloud Solutions for AgricultureBhadale IT Cloud Solutions for Agriculture
Bhadale IT Cloud Solutions for AgricultureVijayananda Mohire
 
Bhadale IT Cloud Solutions for Agriculture
Bhadale IT Cloud Solutions for AgricultureBhadale IT Cloud Solutions for Agriculture
Bhadale IT Cloud Solutions for AgricultureVijayananda Mohire
 
Bhadale IT Intel and Azure Cloud Offerings
Bhadale IT Intel and Azure Cloud OfferingsBhadale IT Intel and Azure Cloud Offerings
Bhadale IT Intel and Azure Cloud OfferingsVijayananda Mohire
 
Practical ChatGPT From Use Cases to Prompt Engineering & Ethical Implications
Practical ChatGPT From Use Cases to Prompt Engineering & Ethical ImplicationsPractical ChatGPT From Use Cases to Prompt Engineering & Ethical Implications
Practical ChatGPT From Use Cases to Prompt Engineering & Ethical ImplicationsVijayananda Mohire
 
Cloud Infrastructure - Partner Delivery Accelerator (APAC)
Cloud Infrastructure - Partner Delivery Accelerator (APAC)Cloud Infrastructure - Partner Delivery Accelerator (APAC)
Cloud Infrastructure - Partner Delivery Accelerator (APAC)Vijayananda Mohire
 
Red Hat Sales Specialist - Red Hat Enterprise Linux
Red Hat Sales Specialist - Red Hat Enterprise LinuxRed Hat Sales Specialist - Red Hat Enterprise Linux
Red Hat Sales Specialist - Red Hat Enterprise LinuxVijayananda Mohire
 
Generative AI Business Transformation
Generative AI Business TransformationGenerative AI Business Transformation
Generative AI Business TransformationVijayananda Mohire
 
Microsoft Learn Transcript Jan 2024- vijaymohire
Microsoft Learn Transcript Jan 2024- vijaymohireMicrosoft Learn Transcript Jan 2024- vijaymohire
Microsoft Learn Transcript Jan 2024- vijaymohireVijayananda Mohire
 
Bhadale Group of Companies -Futuristic Products Brief-Ver 1.0
Bhadale Group of Companies -Futuristic Products Brief-Ver 1.0Bhadale Group of Companies -Futuristic Products Brief-Ver 1.0
Bhadale Group of Companies -Futuristic Products Brief-Ver 1.0Vijayananda Mohire
 
Intel Partnership - Gold Member - 2024
Intel Partnership - Gold Member - 2024Intel Partnership - Gold Member - 2024
Intel Partnership - Gold Member - 2024Vijayananda Mohire
 
Bhadale Group of Companies -Defense Academy Ver. 2.0
Bhadale Group of Companies -Defense Academy Ver. 2.0Bhadale Group of Companies -Defense Academy Ver. 2.0
Bhadale Group of Companies -Defense Academy Ver. 2.0Vijayananda Mohire
 
Develop Generative AI solutions with Azure OpenAI Service
Develop Generative AI solutions with Azure OpenAI ServiceDevelop Generative AI solutions with Azure OpenAI Service
Develop Generative AI solutions with Azure OpenAI ServiceVijayananda Mohire
 

More from Vijayananda Mohire (20)

Certificate- Peer Review of Book Chapter on ML
Certificate- Peer Review of Book Chapter on MLCertificate- Peer Review of Book Chapter on ML
Certificate- Peer Review of Book Chapter on ML
 
Bhadale IT Hub-Multi Cloud and Multi QAI
Bhadale IT Hub-Multi Cloud and Multi QAIBhadale IT Hub-Multi Cloud and Multi QAI
Bhadale IT Hub-Multi Cloud and Multi QAI
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAI
 
Azure Quantum Workspace for developing Q# based quantum circuits
Azure Quantum Workspace for developing Q# based quantum circuitsAzure Quantum Workspace for developing Q# based quantum circuits
Azure Quantum Workspace for developing Q# based quantum circuits
 
My Journey towards Artificial Intelligence
My Journey towards Artificial IntelligenceMy Journey towards Artificial Intelligence
My Journey towards Artificial Intelligence
 
Bhadale IT Cloud Solutions for Agriculture
Bhadale IT Cloud Solutions for AgricultureBhadale IT Cloud Solutions for Agriculture
Bhadale IT Cloud Solutions for Agriculture
 
Bhadale IT Cloud Solutions for Agriculture
Bhadale IT Cloud Solutions for AgricultureBhadale IT Cloud Solutions for Agriculture
Bhadale IT Cloud Solutions for Agriculture
 
Bhadale IT Intel and Azure Cloud Offerings
Bhadale IT Intel and Azure Cloud OfferingsBhadale IT Intel and Azure Cloud Offerings
Bhadale IT Intel and Azure Cloud Offerings
 
GitHub Copilot-vijaymohire
GitHub Copilot-vijaymohireGitHub Copilot-vijaymohire
GitHub Copilot-vijaymohire
 
Practical ChatGPT From Use Cases to Prompt Engineering & Ethical Implications
Practical ChatGPT From Use Cases to Prompt Engineering & Ethical ImplicationsPractical ChatGPT From Use Cases to Prompt Engineering & Ethical Implications
Practical ChatGPT From Use Cases to Prompt Engineering & Ethical Implications
 
Cloud Infrastructure - Partner Delivery Accelerator (APAC)
Cloud Infrastructure - Partner Delivery Accelerator (APAC)Cloud Infrastructure - Partner Delivery Accelerator (APAC)
Cloud Infrastructure - Partner Delivery Accelerator (APAC)
 
Red Hat Sales Specialist - Red Hat Enterprise Linux
Red Hat Sales Specialist - Red Hat Enterprise LinuxRed Hat Sales Specialist - Red Hat Enterprise Linux
Red Hat Sales Specialist - Red Hat Enterprise Linux
 
RedHat_Transcript_Jan_2024
RedHat_Transcript_Jan_2024RedHat_Transcript_Jan_2024
RedHat_Transcript_Jan_2024
 
Generative AI Business Transformation
Generative AI Business TransformationGenerative AI Business Transformation
Generative AI Business Transformation
 
Microsoft Learn Transcript Jan 2024- vijaymohire
Microsoft Learn Transcript Jan 2024- vijaymohireMicrosoft Learn Transcript Jan 2024- vijaymohire
Microsoft Learn Transcript Jan 2024- vijaymohire
 
Bhadale Group of Companies -Futuristic Products Brief-Ver 1.0
Bhadale Group of Companies -Futuristic Products Brief-Ver 1.0Bhadale Group of Companies -Futuristic Products Brief-Ver 1.0
Bhadale Group of Companies -Futuristic Products Brief-Ver 1.0
 
Intel Partnership - Gold Member - 2024
Intel Partnership - Gold Member - 2024Intel Partnership - Gold Member - 2024
Intel Partnership - Gold Member - 2024
 
Bhadale Group of Companies -Defense Academy Ver. 2.0
Bhadale Group of Companies -Defense Academy Ver. 2.0Bhadale Group of Companies -Defense Academy Ver. 2.0
Bhadale Group of Companies -Defense Academy Ver. 2.0
 
Develop Generative AI solutions with Azure OpenAI Service
Develop Generative AI solutions with Azure OpenAI ServiceDevelop Generative AI solutions with Azure OpenAI Service
Develop Generative AI solutions with Azure OpenAI Service
 
IEEE Member Certificate-2024
IEEE Member Certificate-2024IEEE Member Certificate-2024
IEEE Member Certificate-2024
 

Recently uploaded

A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
Which standard is best for your content?
Which standard is best for your content?Which standard is best for your content?
Which standard is best for your content?Rustici Software
 
Introduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptxIntroduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptxmprakaash5
 
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024BookNet Canada
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneUiPathCommunity
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...Karmanjay Verma
 
Bitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactiveBitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactivestartupro
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Dublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxDublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxKunal Gupta
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...BookNet Canada
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfAarwolf Industries LLC
 
Transport in Open Pits______SM_MI10415MI
Transport in Open Pits______SM_MI10415MITransport in Open Pits______SM_MI10415MI
Transport in Open Pits______SM_MI10415MIRomil Mishra
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...BookNet Canada
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfROWELL MARQUINA
 
Software Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey HightowerSoftware Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey HightowerAnchore
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentMahmoud Rabie
 

Recently uploaded (20)

A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
Which standard is best for your content?
Which standard is best for your content?Which standard is best for your content?
Which standard is best for your content?
 
Introduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptxIntroduction-to-Wazuh-and-its-integration.pptx
Introduction-to-Wazuh-and-its-integration.pptx
 
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
Green paths: Learning from publishers’ sustainability journeys - Tech Forum 2024
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
WomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyoneWomenInAutomation2024: AI and Automation for eveyone
WomenInAutomation2024: AI and Automation for eveyone
 
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...React JS; all concepts. Contains React Features, JSX, functional & Class comp...
React JS; all concepts. Contains React Features, JSX, functional & Class comp...
 
Bitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactiveBitdefender-CSG-Report-creat7534-interactive
Bitdefender-CSG-Report-creat7534-interactive
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Dublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptxDublin_mulesoft_meetup_API_specifications.pptx
Dublin_mulesoft_meetup_API_specifications.pptx
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
Transcript: New from BookNet Canada for 2024: BNC SalesData and LibraryData -...
 
Landscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdfLandscape Catalogue 2024 Australia-1.pdf
Landscape Catalogue 2024 Australia-1.pdf
 
Transport in Open Pits______SM_MI10415MI
Transport in Open Pits______SM_MI10415MITransport in Open Pits______SM_MI10415MI
Transport in Open Pits______SM_MI10415MI
 
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
Transcript: Green paths: Learning from publishers’ sustainability journeys - ...
 
QMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdfQMMS Lesson 2 - Using MS Excel Formula.pdf
QMMS Lesson 2 - Using MS Excel Formula.pdf
 
Software Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey HightowerSoftware Security in the Real World w/Kelsey Hightower
Software Security in the Real World w/Kelsey Hightower
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
Digital Tools & AI in Career Development
Digital Tools & AI in Career DevelopmentDigital Tools & AI in Career Development
Digital Tools & AI in Career Development
 

Key projects in AI, ML and Generative AI

  • 1. Key projects in AI, ML and Generative AI February 29 2024 This is a summary of my twelve years hands-on towards achieving the required experience, and skills in AI, ML and Generative AI technologies. This includes key partner trainings from Google® and Microsoft®
  • 2. Key projects in AI /ML and Generative AI Project No. Project Name Project Summary / Learning 1 Simple Prompt Design using Vertex AI Zero –shot prompting, and Context based Chat in Vertex AI, using text-bison@001 model 2 Sentiment and Text Analysis Lab to query the LLM using Vertex AI for sentiment analysis and keyword extraction from a Japanese file
  • 3. 3 Image Classification Lab to train Vertex ML Pipeline to classify images, here flowers; used flower dataset with 1,000 images to train the model. Model accuracy was about 97%. 4 Video classification Video classification using AutoML training , use of evaluation like confusion matrix for precision measurement
  • 4. 5 Improving data quality This notebook introduced a few concepts to improve data quality. We resolved missing values, converted the Date feature column to a datetime format, renamed feature columns, removed a value from a feature column, and created one-hot encoding features for feeding to ML algorithm. Pandas, dataframes, seaborn packages used. Key ones are listed below: a) Handling Categorical Columns The feature column "lightduty" is categorical and has a "Yes/No" choice. We cannot feed values like this into a machine learning model. We need to convert the binary answers from strings of yes/no to integers of 1/0. There are various methods to achieve this. We will use the "apply" method with a lambda expression. Pandas. apply() takes a function and applies it to all values of a Pandas series b) Machine learning algorithms expect input vectors and not categorical features. Specifically, they cannot handle text or string values. Thus, it is often useful to transform categorical features into vectors. One transformation method is to create dummy variables for our categorical features. Dummy variables are a set of binary (0 or 1) variables that each represent a single class from a categorical feature. We simply encode the categorical variable as a one-hot vector, i.e. a vector where only one element is non-zero, or hot. With one-hot encoding, a categorical feature becomes an array whose
  • 5. size is the number of possible choices for that feature. 6 Exploratory data analysis using BigQuery EDA using linear regression using Python and Scikit-Learn, heatmaps for predicting US house value US and taxi fare estimation 7 Using the BigQuery ML Hyperparameter Tuning to Improve Model Performance This lab introduces data analysts to BigQuery ML. BigQuery ML enable users to create and execute machine learning models in BigQuery using SQL queries. In this lab, you use the “tlc_yellow_trips_2018 sample table” to create a model that predicts the tip for a taxi ride. You will see a ~40% performance (r2_score) improvement with hyperparameter tuning 8 TensorFlow Dataset API Linear Regression Model uses data from a`tf.data.Dataset`, and we will learn how to implement stochastic gradient descent using loss function, and gradient. Then we write production input pipelines with feature engineering (batching, shuffling, etc.). 9 Classifying Structured Data using Keras In this notebook, you learn how to classify structured data (e.g. tabular data in a CSV). You will use Keras to define the model, and preprocessing layers as a bridge to map from columns in a CSV to features used to
  • 6. Preprocessing Layers train the model. Use of numpy, pandas, sklearn libraries. The Keras preprocessing layers API allows you to build Keras-native input processing pipelines. You will use preprocessing layers to demonstrate the feature preprocessing code. 1. Normalization- Feature-wise normalization of the data. 2. CategoryEncoding - Category encoding layer. 3. StringLookup - Maps strings from a vocabulary to integer indices. 4. IntegerLookup- Maps integers from a vocabulary to integer indices.
  • 7. 10 Keras Sequential API on Vertex AI Platform In this lab, we'll see how to build a simple deep neural network model using the Keras Sequential API and feature columns. Once we have trained our model, we will deploy it using Vertex AI and see how to call our model for online prediction. We will use feature columns to connect our raw data to our Keras DNN model. Feature columns make it easy to perform common types of feature engineering on your raw data. For example, you can one-hot encode categorical data, create feature crosses, embeddings and more. 11 Build a DNN using the Keras Functional API In this notebook, we will build a Keras DNN to predict the fare amount for NYC taxi cab rides 1. Review how to read in CSV file data using tf.data. 2. Specify input, hidden, and output layers in the DNN architecture. 3. Review and visualize the final DNN shape. 4. Train the model locally and visualize the loss curves.
  • 8. 5. Deploy and predict with the model using Cloud AI Platform. 12 Performing Basic Feature Engineering in BQML In this lab, we utilize feature engineering to improve the prediction of the fare amount for a taxi ride in New York City. We will use BigQuery ML to build a taxifare prediction model, using feature engineering to improve and create a final model. In this Notebook we set up the environment, create the project dataset, create a feature engineering table, create and evaluate a baseline model, extract temporal features, perform a feature cross on temporal features, and evaluate model performance throughout the process. In this lab, you: • Create SQL statements to evaluate the model • Extract temporal features • Perform a feature cross on temporal features Allow the model to learn traffic patterns by creating a new feature that combines the time of day and day of week (this is called a [feature cross]. Create the SQL statement to feature cross the dayofweek and hourofday using the CONCAT function Few of the evaluation metrics: 1. mean_absolute_error 2. mean_squared_error 3. mean_squared_log_error 4. median_absolute_error
  • 9. 5. r2_score RMSE results 13 Performing Basic Feature Engineering in Keras In this lab, we utilize feature engineering to improve the prediction of housing prices using a Keras Sequential Model. 1. Create an input pipeline using tf.data. 2. Engineer features to create categorical, crossed, and numerical feature columns. It is very important for numerical variables to get scaled before they are "fed" into the neural network. Here we use min-max scaling. We cannot feed strings directly to a model. Instead, we must first map them to numeric values. The categorical vocabulary columns provide a way to represent strings as a one-hot vector. Often, you don't want to feed a number directly into the model, but instead split its value into different categories based on numerical ranges. Consider our raw data that represents a homes' age. Instead of representing the house age as a numeric column, we could split the home age into several buckets using a bucketized column. Combining features into a single feature, better known as feature crosses, enables a model to learn separate weights for each combination of features. 14 Exploring and Creating an Ecommerce Analytics Pipeline with Cloud Dataprep v1.5 Cloud Dataprep® by Trifacta® is an intelligent data service for visually exploring, cleaning, and preparing structured and unstructured data for analysis. In this lab we will explore the Cloud Dataprep UI to build an ecommerce transformation pipeline that will run at a scheduled interval and output results back into BigQuery. The dataset we will be using is an ecommerce dataset that has millions of Google® Analytics records for the Google® Merchandise Store loaded into BigQuery In this lab, you learn how to perform these tasks:  Connect BigQuery datasets to Cloud Dataprep  Explore dataset quality with Cloud Dataprep
  • 10.  Create a data transformation pipeline with Cloud Dataprep  Schedule transformation jobs outputs to BigQuery Analytics Pipeline Recipe Book with rules
  • 11. Results with duplicate rows removed
  • 12. 15 ML pipelines with Vertex Pipelines In this lab, you learn how to create and run ML pipelines with Vertex Pipelines. Objectives • Use the Kubeflow Pipelines SDK to build scalable ML pipelines. • Create and run a 3-step intro pipeline that takes text input. Few of the components: • Kubeflow Pipelines: This is the SDK we'll be using to build our pipeline. Vertex Pipelines supports running pipelines built with either Kubeflow Pipelines or TFX. • Google® Cloud Pipeline Components: This library provides pre-built components that make it easier to interact with Vertex AI services from your pipeline steps. We'll create a pipeline that prints out a sentence using two outputs: a product name and an emoji description. This pipeline will consist of three components: 1. product_name: This component will take a product name (or any noun you want really) as input, and return that string as output 2. emoji: This component will take the text description of an emoji and convert it to an emoji. For example, the text code for * is "sparkles". This component uses an emoji library to show you how to manage external dependencies in your pipeline 3. build_sentence: This final component will consume the output of the previous two to build a sentence that uses the emoji. For example, the resulting output might be "Vertex Pipelines is *"
  • 13. 16 Advanced Visualizations with TensorFlow Data Validation This lab illustrates how TensorFlow Data Validation (TFDV) can be used to investigate and visualize your dataset. That includes looking at descriptive statistics, inferring a schema, checking for and fixing anomalies, and checking for drift and skew in our dataset First we'll use `tfdv.generate_statistics_from_csv` to compute statistics for our training data. TFDV can compute descriptive statistics that provide a quick overview of the data in terms of the features that are present and the shapes of their value distributions. Now let's use [`tfdv.infer_schema`] to create a schema for our data. Does our evaluation dataset match the schema from our training dataset? This is especially important for categorical features, where we want to identify the range of acceptable values. Drift detection is supported for categorical features and between consecutive spans of data (i.e., between span N and span N+1), such as between different days of training data. We express drift in terms of [L- infinity distance], and you can set the threshold distance so that you receive warnings when the drift is higher than is acceptable. Adding skew and drift comparators to visualize and make corrections. Few of the uses are: 1. Validating new data for inference to make sure that we haven't suddenly started receiving bad features
  • 14. 2. Validating new data for inference to make sure that our model has trained on that part of the decision surface 3. Validating our data after we've transformed it and done feature engineering (probably using [TensorFlow Transform] to make sure we haven't done something wrong 17 Distributed Training with Keras We learn: 1. How to define distribution strategy and set input pipeline. 2. How to create the Keras model. 3. How to define the callbacks. 4. How to train and evaluate the model. The tf.distribute.Strategy API provides an abstraction for distributing your training across multiple processing units. The goal is to allow users to enable distributed training using existing models and training code, with minimal changes. This lab uses the tf.distribute.MirroredStrategy, which does in-graph replication with synchronous training on many GPUs on one machine. Essentially, it copies all of the model's variables to each processor. Then, it uses all-reduce to combine the gradients from all processors and applies the combined value to all copies of the model. MirroredStrategy is one of several distribution strategies available in TensorFlow core. The callbacks used here are: 1. TensorBoard: This callback writes a log for TensorBoard which allows you to visualize the graphs. 2. Model Checkpoint: This callback saves the model after every epoch. 3. Learning Rate Scheduler: Using this callback, you can schedule the learning rate to change after every epoch/batch. 18 TPU Speed Data Pipelines TPUs are very fast, and the stream of training data must keep up with their training speed. In this lab, you will learn how to load data from Cloud Storage with the tf.data.Dataset API to feed your TPU You will learn: • To use the tf.data.Dataset API to load training data. • To use TFRecord format to load training data efficiently from Cloud Storage. 19 Detecting Labels, Faces, and Landmarks in Images with the Cloud Vision API The Cloud Vision API lets you understand the content of an image by encapsulating powerful machine learning models in a simple REST API. In this lab, you send images to the Vision API and see it detect objects, faces, and landmarks
  • 15. In this lab, you learn how to perform the following tasks:  Create a Vision API request and call the API with curl.  Use the label, face, and landmark detection methods of the vision API 20 Classifying Images using Dropout and Batchnorm Layer In this lab, you learn how to build a neural network to classify the tf-flowers dataset using dropout and batchnorm layer. Learning objectives • Define Helper Functions. • Apply dropout and batchnorm layer Dropouts are the regularization technique that is used to prevent overfitting in the model. Batch normalization is a layer that allows every layer of the network to do learning more independently. The layer
  • 16. is added to the sequential model to standardize the input or the outputs. Add a dropout and batchnorm layer after each of the hidden layers. 21 Classifying Images with Transfer Learning In this lab, you learn how to build a neural network to classify the tf-flowers (5 flowers) dataset by using a pre-trained image embedding. You load a pre-trained model which is trained on very large, general-purpose datasets and transfer that knowledge to the actual dataset that you want to classify. This means you use a pre-trained model instead of the Flattened layer as your first layer
  • 17. You learn how to apply data augmentation in two ways: • Understand how to set up preprocessing in order to convert image type and resize the image to the desired size. • Understand how to implement transfer learning with MobileNet. Pre-trained models are models that are trained on large datasets and made available to be used as a way to create embeddings. For example, the [MobileNet model] is a model with 1-4 million parameters that was trained on the [ImageNet (ILSVRC) dataset] which consists of millions of images corresponding to hundreds of categories that were scraped from the web. The resulting embedding therefore has the ability to efficiently compress the information found in a wide variety of images. As long as the images you want to classify are similar in nature to the ones that MobileNet was trained on, the embeddings from MobileNet should give a great pre-trained embedding that you can use as a starting point to train a model on your smaller tf-flowers (5 flowers) dataset. A pre-trained MobileNet is available on TensorFlow Hub and you can easily load it as a Keras layer by passing in the URL to the trained model.
  • 18. 22 Text classification using reusable embeddings In this lab, you implement text models to recognize the probable source (GitHub, Tech-Crunch, or The New- York Times) of titles present in the title dataset, which are created in the respective labs. Learning objectives In this lab, you learn how to: • Use pre-trained TF Hub text modules to generate sentence vectors. • Incorporate a pre-trained TF-Hub module into a Keras model. • Deploy and use a text model on CAIP In this lab, we will use pre-trained [TF-Hub embeddings modules for English] for the first layer of our models. One immediate advantage of doing so is that the TF-Hub embedding module will take care for us of processing the raw text. This also means that our model will be able to consume text directly instead of sequences of integers representing the words. However, as before, we still need to preprocess the labels into one-hot-encoded vectors We will first try a word embedding pre-trained using a [Neural Probabilistic Language Model]. TF-Hub has a 50-dimensional one called [nnlm-en-dim50-with-normalization], which also normalizes the vectors produced. Once loaded from its URL, the TF-hub module can be used as a normal Keras layer in a sequential or functional model. Since we have enough data to fine-tune the parameters of the pre-trained embedding itself, we will set `trainable=True` in the `KerasLayer` that loads the pre-trained embedding
  • 19. Then we will try a word embedding obtained using [Swivel], an algorithm that essentially factorizes word co- occurrence matrices to create the words embeddings. TF-Hub hosts the pretrained [gnews-swivel-20dim- with-oov], 20-dimensional Swivel module. Swivel trains faster but achieves lower validation accuracy, and requires more epochs to train on. 23 RNN Encoder Decoder for Translation In this notebook, you will use encoder-decoder architecture to create a text translation function. In this lab, you will: • Create a tf.data.Dataset for a seq2seq problem. • Train an encoder-decoder model in Keras for a translation task. • Save the encoder and the decoder as separate model. • Learn how to piece together the trained encoder and decoder into a translation function  Learn how to use the BLUE score to evaluate a translation model We will start by creating train and eval datasets (using the `tf.data.Dataset` API) that are typical for seq2seq problems. Then we will use the Keras functional API to train an RNN encoder-decoder model, which will save as two separate models, the encoder and decoder model. Using these two separate pieces we will implement the translation function. At last, we'll benchmark our results using the industry standard BLEU score. The `utils_preproc.preprocess_sentence()` method does the following: 1. Converts sentence to lower case 2. Adds a space between punctuation and words 3. Replaces tokens that aren't a-z or punctuation with space 4. Adds `<start>` and `<end>` tokens The `utils_preproc.tokenize()` method does the following: 1. Splits each sentence into a token list 2. Maps each token to an integer 3. Pads to length of longest sentence It returns an instance of a [Keras Tokenizer] containing the token-integer mapping along with the integerized sentences We use an encoder-decoder architecture, however we embed our words into a latent space prior to feeding them into the RNN. Next we implement the encoder network with Keras functional API. It will  start with an `Input` layer that will consume the source language integerized sentences
  • 20.  then feed them to an `Embedding` layer of `EMBEDDING_DIM` dimensions  which in turn will pass the embeddings to a `GRU` recurrent layer with `HIDDEN_UNITS` The output of the encoder will be the `encoder_outputs` and the `encoder_state`. Next, implement the decoder network, which is very similar to the encoder network. It will  start with an `Input` layer that will consume the source language integerized sentences  then feed that input to an `Embedding` layer of `EMBEDDING_DIM` dimensions  which in turn will pass the embeddings to a `GRU` recurrent layer with `HIDDEN_UNITS` **Important:** The main difference with the encoder, is that the recurrent `GRU` layer will take as input not only the decoder input embeddings, but also the `encoder_state` as outputted by the encoder above. This is where the two networks are linked! The output of the encoder will be the `decoder_outputs` and the `decoder_state` The last part of the encoder-decoder architecture is a softmax `Dense` layer that will create the next word probability vector or next word `predictions` from the `decoder_output'
  • 21. We can't just use model.predict(), because we don't know all the inputs we used during training. We only know the encoder_input (source language) but not the decoder_input (target language), which is what we want to predict (i.e., the translation of the source language)! We do however know the first token of the decoder input, which is the `<start>` token. So using this plus the state of the encoder RNN, we can predict the next token. We will then use that token to be the second token of decoder input, and continue like this until we predict the `<end>` token, or we reach some defined max length. So, the strategy now is to split our trained network into two independent Keras models: 1. an encoder model with signature `encoder_inputs -> encoder_state` 2. a decoder model with signature `[decoder_inputs, decoder_state_input] -> [predictions, decoder_state]` Given that input, the decoder will produce the first word of the translation, by sampling from the `predictions` vector (for simplicity, our sampling strategy here will be to take the next word to be the one
  • 22. whose index has the maximum probability in the `predictions` vector) along with a new state vector, the `decoder_state`. At this point, we can feed again to the decoder the predicted first word and as well as the new `decoder_state` to predict the translation second word. This process can be continued until the decoder produces the token `<stop>`. Now that we have a separate encoder and a separate decoder, implement a translation function, to which we will give the generic name of `decode_sequences` (to stress that this procedure is general to all seq2seq problems). `decode_sequences` will take as input  `input_seqs` which is the integerized source language sentence tensor that the encoder can consume  `output_tokenizer` which is the target languague tokenizer we will need to extract back words from predicted word integers  * `max_decode_length` which is the length after which we stop decoding if the `<stop>` token has not been predicted Many attempts have been made to develop a better metric for natural language evaluation. The most popular currently is Bilingual Evaluation Understudy (BLEU). - It is quick and inexpensive to calculate. - It allows flexibility for the ordering of words and phrases. - It is easy to understand. - It is language independent. - It correlates highly with human evaluation. - It has been widely adopted. The score is from 0 to 1, where 1 is an exact match.
  • 23. It works by counting matching n-grams between the machine and reference texts, regardless of order. BLUE- 4 counts matching n grams from 1-4 (1-gram, 2-gram, 3-gram and 4-gram). It is common to report both BLUE-1 and BLUE-4 24 Hybrid Recommendations with the MovieLens Dataset The matrix factorization approach does not use any information about users or movies beyond what is available from the ratings matrix. However, we will often have user information (such as the city they live, their annual income, their annual expenditure, etc.) and we will almost always have more information about the products in our catalog. How do we incorporate this information in our recommendation model? The answer lies in recognizing that the user factors and product factors that result from the matrix factorization approach end up being a concise representation of the information about users and products available from the ratings matrix. We can concatenate this information with other information we have available and train a regression model to predict the rating In this lab, you will:  Know how to extract user and product factors from a BigQuery Matrix Factorizarion Model
  • 24.  Know how to format inputs for a BigQuery Hybrid Recommendation Model 25 TFX on Cloud AI Platform Pipelines In this lab, you use utilize the following tools and services to deploy and run a TFX pipeline on Google® Cloud that automates the development and deployment of a TensorFlow 2.3 WideDeep Classifier to predict forest cover from cartographic data. In this notebook, you will work with the [Covertype Data Set and use TFX to analyze, understand, and pre-process the dataset and train, analyze, validate, and deploy a multi- class classification model to predict the type of forest cover from cartographic features Learning objectives 1. Develop a high level understanding of TFX pipeline components. 2. Learn how to use a TFX Interactive Context for prototype development of TFX pipelines. 3. Work with the Tensorflow Data Validation (TFDV) library to check and analyze input data. 4. Utilize the Tensorflow Transform (TFT) library for scalable data preprocessing and feature transformations. 5. Employ the Tensorflow Model Analysis (TFMA) library for model evaluation.
  • 25. 26 Continuous Training Pipelines with Cloud Composer In this lab you will learn how to write an Airflow DAG for continuous training and deploy the DAG within a Cloud Composer environment. You will also learn how to explore and monitor your DAG runs using the Apache Airflow webserver. Objectives In this lab, you will learn to perform the following tasks:  Provision a Cloud Composer environment.  Deploy an Apache Airflow Dialog.  Monitor a continuous training pipeline in the Airflow webserver.  Explore Airflow logs using Cloud Operations.
  • 26. 27 Image Captioning with Visual Attention Image captioning models take an image as input, and output text. Ideally, we want the output of the model to accurately describe the events/things in the image, similar to a caption a human might provide. In order to generate text, we will build an encoder-decoder model, where the encoder output embedding of an input image, and the decoder output text from the image embedding ## Learning Objectives 1. Learn how to create an image captioning model 2. Learn how to train and predict a text generation model. We will use the TensorFlow datasets capability to read the [COCO captions] (https://www.tensorflow.org/datasets/catalog/coco_captions) dataset. This version contains images, bounding boxes, labels, and captions from COCO 2014, split into the subsets defined by Karpathy and Li (2015) You will transform the text captions into integer sequences using the [TextVectorization] layer Now let's design an image captioning model. It consists of an image encoder, followed by a caption decoder. The image encoder model is very simple. It extracts features through a pre-trained model and passes them to a fully connected layer. The caption decoder incorporates an attention mechanism that focuses on different parts of the input image. The decoder uses attention to selectively focus on parts of the input sequence. The attention takes a sequence of vectors as input for each example and returns an "attention" vector for each example
  • 27. The decoder's job is to generate predictions for the next output token. 1. The decoder receives current word tokens as a batch. 2. It embeds the word tokens to `ATTENTION_DIM` dimension. 3. GRU layer keeps track of the word embeddings, and returns GRU outputs and states. 4. Bahdanau-style attention attends over the encoder's output feature by using GRU outputs as a query. 5. The attention outputs and GRU outputs are added (skip connection), and normalized in a layer normalization layer. 6. It generates logit predictions for the next token based on the GRU output. We can define all the steps in Keras Functional API, but please note that here we instantiate layers that have trainable parameters so that we reuse the layers and the weights in inference phase. The predict step is different from the training, since we need to keep track of the GRU state during the caption generation, and pass a predicted word to the decoder as an input at the next time step. In order to do so, let's define another model for prediction while using the trained weights, so that it can keep and update the GRU state during the caption generation. Results:
  • 28. 28 Text generation using RNN This tutorial demonstrates how to generate text using a character-based RNN. You will work with a dataset of Shakespeare's writing from Andrej Karpathy's [The Unreasonable Effectiveness of Recurrent Neural Networks].Given a sequence of characters from this data ("Shakespear"), train a model to predict the next character in the sequence ("e"). Longer sequences of text can be generated by calling the model repeatedly. Build a model with the following layers 1. `tf.keras.layers.Embedding`: The input layer. A trainable lookup table that will map each character-ID to a vector with `embedding_dim` dimensions; 2. `tf.keras.layers.GRU`: A type of RNN with size `units=rnn_units` (You can also use an LSTM layer here.) 3. `tf.keras.layers.Dense`: The output layer, with `vocab_size` outputs. It outputs one logit for each character in the vocabulary. These are the log-likelihood of each character according to the model. For each character the model looks up the embedding, runs the GRU one timestep with the embedding as input, and applies the dense layer to generate logits predicting the log-likelihood of the next character. At this point the problem can be treated as a standard classification problem. Given the previous RNN state, and the input this time step, predict the class of the next character. The simplest way to generate text with this model is to run it in a loop, and keep track of the model's internal state as you execute it. Each time you call the model you pass in some text and an internal state. The model returns a prediction for the next character and its new state. Pass the prediction and state back in to continue generating text. Sample result:
  • 29. 29 Classify text with BERT Learning Objectives 1. Learn how to load a pre-trained BERT model from TensorFlow Hub 2. Learn how to build your own model by combining with a classifier 3. Learn how to train a BERT model by fine-tuning 4. Learn how to save your trained model and use it 5. Learn how to evaluate a text classification model This lab will show you how to fine-tune BERT to perform sentiment analysis on a dataset of plain-text IMDB movie reviews. In addition to training a model, you will learn how to preprocess text into an appropriate format. 30 Decision making using Dialogflow CX generators In this lab, you learn how to perform the following tasks:  Use Dialogflow AI Agent to create a chat app and add unstructured data to a data store  Use knowledge handlers to allow end-users to have conversations with a virtual agent about the content added to a data store.  Configure a generator text prompt and make it contextual by using built-in generator prompt placeholders.  Mark words as generator prompt placeholders and later associate them with session parameters in fulfillment to use their values during execution.  Configure a generator to handle responses that involve knowledge from a large textual dataset and context from the current conversation.  Generate a formal email using generators  Test your agent and simulate customer questions that trigger generated responses The Vertex AI Conversation feature creates a special Dialogflow agent, called a data store agent.
  • 30. With this feature, you provide a website URL, structured data or unstructured data (data stores), then Google® parses your content and creates a virtual agent that is powered by data stores and large language models. Your customers and end users can then have conversations with the agent and ask questions about the content. The generator feature is a Dialogflow CX feature that allows developers to use Google's latest generative large language models (LLMs) and custom prompts to generate agent responses at runtime The generative fallback feature uses Google's latest generative large language models (LLMs) to generate virtual agent responses when end-user input does not match an intent or parameter for form filling. The feature can be configured with a text prompt that instructs the LLM how to respond. You can use a predefined text prompt or add your own prompts. You can enable generative fallback on no-match event handlers used in flows, pages, or during parameter filling. Chat Reponse
  • 31. 31 Bard API /Gemini Features of Bard and Gemini models Greet with name and time
  • 34. Generate creative prompt 32 Microsoft® Copilot Studio BOT using Power Platform GitHub Copilot CODE Suggestions
  • 36. Disclaimer: We have sourced the content from various courses and partner trainings. All details, references are for educational purposes only • Intel® is trademark of the Intel® Corporation. Microsoft® and Microsoft Azure® are trademarks of the Microsoft® group of companies. Framer® is the trademark of Framer® Incorporation. • All logos, trademarks and brand names belong to the respective owners as specified. We have no intention to infringe any copyrights or alter related permissions set by the owners. Please refer to source websites for any further details. This is for educational and information purpose only. For more details, contact: Bhadale IT Pvt. Ltd; Email: vijaymohire@gmail.com