The document discusses different types of regression models and the k-nearest neighbors algorithm. It describes linear, multiple, and non-linear regression models and how each relates independent and dependent variables. It also discusses stepwise regression modeling as a technique to add variables to a model. The document then summarizes applications of k-NN including data preprocessing, recommendation engines, and its advantages of being easy to implement and adapt to new data, though it has disadvantages of not scaling well and being prone to overfitting.

1. Advance regression
model
&
K neighbour
algorithm
Submitted by
M.Neya shree
M.sc cs

2. What Is a Regression Model?
A regression model provides a function that
describes the relationship between one or more
independent variables and a response, dependent, or target
variable.
For example, the relationship between height and weight
may be described by a linear regression model.

3. Types of Regression
1. Linear
A linear regression is a model where the relationship between
inputs and outputs is a straight line. This is the easiest to
conceptualize and even observe in the real world. Even when a
relationship isn’t very linear, our brains try to see the pattern
and attach a rudimentary linear model to that relationship.
One example may be around the number of responses to a
marketing campaign. If we send 1,000 emails, we may get five
responses. If this relationship can be modeled using a linear
regression, we would expect to get ten responses when we
send 2,000 emails. Your chart may vary, but the general idea is
that we associate a predictor and a target, and we assume a
relationship between the two.

4. 2. Multiple
Multiple regression indicates that there are more than one input
variables that may affect the outcome, or target variable. For
our email campaign example, you may include an additional
variable with the number of emails sent in the last month.
By looking at both input variables, a clearer picture starts to
emerge about what drives users to respond to a campaign and
how to optimize email timing and frequency. While
conceptualizing the model becomes more complex with more
inputs, the relationship may continue to be linear.
For these models, it is important to understand exactly what
effect each input has and how they combine to produce the
final target variable results.

5. 3. Non-Linear
A linear model may work for some parts of the marketing
example above. However, we know that as we keep increasing
the number of emails in a particular campaign, the number of
responses starts to decline vs the number of emails sent. To
model this, we need a non-linear regression model.
This model will initially show a positive relationship between
number of emails and the response, but as the number of
emails increases, the model will flatten out and become almost
constant. The important takeaway here is that it is important to
understand when a model could potentially be non-linear. If you
make assumptions based on a linear model, you could get
results that are very different than expectations.

6. 4. Stepwise Regression Modeling
While the other items we have talked about until now are specific
types of models, stepwise regression is more of a technique. If a
model involves many potential inputs, the analyst may start with the
most directly correlated input variable to build a model. Once that is
accomplished, the next step is to make the model more accurate.
To do that, additional input variables can be added to the model one
at a time in order of significance of the results. Using our email
marketing example, the initial model may be based on just the number
of emails sent. Then we would add something like the average age of
the email recipient. After that, we would add the average number of
emails each recipient has received from us. Each additional variable
would add a small amount of additional accuracy to the model.

8. Applications of k-NN in machine learning
The k-NN algorithm has been utilized within a variety of
applications, largely within classification. Some of these use cases
include:
- Data preprocessing: Datasets frequently have missing values,
but the KNN algorithm can estimate for those values in a process
known as missing data imputation.
- Recommendation Engines: Using clickstream data from
websites, the KNN algorithm has been used to provide automatic
recommendations to users on additional content. This research
(link resides outside of ibm.com) shows that the a user is assigned
to a particular group, and based on that group’s user behavior,
they are given a recommendation. However, given the scaling
issues with KNN, this approach may not be optimal for larger
datasets.

9. Advantages
- Easy to implement: Given the algorithm’s simplicity and
accuracy, it is one of the first classifiers that a new data
scientist will learn.
- Adapts easily: As new training samples are added, the
algorithm adjusts to account for any new data since all training
data is stored into memory.
- Few hyperparameters: KNN only requires a k value and a
distance metric, which is low when compared to other machine
learning algorithms.

10. Disadvantages
● Does not scale well
● Curse of dimensionality
● Prone to overfitting

11. Thank you