The document discusses weighted least squares ratio (WLSR) methods for M-estimators as an alternative to ordinary least squares (OLS). It proposes using WLSR to fit initial regression models and calculate residuals, then applying weighting functions in an iteratively reweighted least squares ratio approach. A simulation study compares WLS and WLSR methods under different conditions, finding WLSR often outperforms WLS apart from with one weighting function. The document concludes WLSR may provide better results than WLS for M-estimation when outliers are present.

1. The Weighted Least Squares Ratio (WLSR) Method
to M-Estimators
SAI Computing Conference, July 2016, London, United Kingdom
e-mail: muraty@jforce.com.tr
MURAT YAZICI, M.Sc.
Sr. Data Scientist & Researcher

2. Murat YAZICI, M.Sc.
Contents
 Introduction: The Regression Model and Its Usage Areas
 What is the problems while setting up a regression model?
 The Least Square Ratio (LSR) Method and Its Benefits
 The M-Estimators and The Proposed WLSR Method
 A Simulation Study of The LSR vs. The OLS Approach to M-Estimators
 Conclusion and Future Work
 Who is JForce IT Company?
 References
 Thanks…
SAI Computing Conference, July 2016, London, United Kingdom

3. Murat YAZICI, M.Sc.
Introduction: The Regression Model
SAI Computing Conference, July 2016, London, United Kingdom

4. Murat YAZICI, M.Sc.
Introduction: Its Usage Areas
Some of usage areas of The Regression Analysis;
 Economy
 Finance
 Business
 Law
 Meteorology
 Medicine
 Biology
 Chemistry
 Engineering
 Education
 Sports
 History
 Sociology
 Psychology
SAI Computing Conference, July 2016, London, United Kingdom

5. Murat YAZICI, M.Sc.
What is the problems while setting up a regression model?
While setting up a linear regression model, Ordinary Least Square (OLS) Method is generally used.
The OLS Method: One of the biggest problems of the OLS
method is that it could not successfully
estimate coefficients in case of outliers
and/or extrem values.
SAI Computing Conference, July 2016, London, United Kingdom

6. Murat YAZICI, M.Sc.
The Least Square Ratio (LSR) Method and Its Benefits
The OLS method aims to estimate observed values with zero error: we can indicate this goal by ,
or . Hence, the ordinary least squares approach satisfies this aim by finding the regression
parameters minimizing the sum of . From the error definition , it is clear that, the size
of error does not depend on the size of . For example, consider estimating 100 as 200 and 1,000 as
1,100: we get the same error −100. However, another point of view says that for the first estimation
there is 100% error, but only 10% for second.*
The Least Square Ratio (LSR) starts with the same goal as in OLS. However, it proceeds by dividing
through by and so is obtained under an assumption of . Hence, it is obvious that,
equations and are raised by basic mathematical operations. This final
equation is taken into account as the origin of the LSR which minimizes the sum of .*
* O. Akbilgic, E.D. Akinci, A Novel Regression Approach: Least Squares Ratio, Communications in Statistics - Theory and Methods 38:9 (2009) 1539-1545.
SAI Computing Conference, July 2016, London, United Kingdom

7. Murat YAZICI, M.Sc.
The Least Square Ratio (LSR) Method and Its Benefits
The LSR Method: One of the biggest problems of the OLS
method is that it could not successfully
estimate coefficients in case of outliers
and/or extrem values.
SAI Computing Conference, July 2016, London, United Kingdom

8. Murat YAZICI, M.Sc.
The M-Estimators and The Proposed WLSR Method
M-estimation was first proposed by Huber (1964, 1973, 2004). M-estimation for regression is a relatively
straightforward extension of M-estimation for location. It represents one of the first attemps at a
compromise between the efficiency of the least squares estimators and the resistance of the LAV
estimators, both of which can be seen as special cases of M-estimation. In simplest terms, the M-
estimator minimizes some function of the residuals. As in the case of M-estimation location, the
robustness of the estimator is determined by the choice of weight function [3].
SAI Computing Conference, July 2016, London, United Kingdom
OLS LSR

9. Murat YAZICI, M.Sc.
The M-Estimators and The Proposed WLSR Method
SAI Computing Conference, July 2016, London, United Kingdom
Huber’s
weighting function:
Andrew’s
weight functions:
Tukey’s
weighting function:
Ramsay’s
weighting function:

10. Murat YAZICI, M.Sc.
The M-Estimators and The Proposed WLSR Method
SAI Computing Conference, July 2016, London, United Kingdom

11. Murat YAZICI, M.Sc.
A Simulation Study of The LSR vs. The OLS Approach to M-Estimators
SAI Computing Conference, July 2016, London, United Kingdom
The simulation study appraises linear multiple regression analysis by using two independent variables as
follows. WLS and WLSR methods are checked against according to the MAE of and the MAE of .
In the simulation process, the independent variables x1 and x2 are randomly generated from a normal
distribution with are equal to 1, so Thus, the
regression model becomes as follows:
Finally, errors are randomly generated as Gaussian white noise with variance . Therefore, the
dependent variable has a normal distribution with mean 201 and variance 200 + .

12. Murat YAZICI, M.Sc.
A Simulation Study of The LSR vs. The OLS Approach to M-Estimators
SAI Computing Conference, July 2016, London, United Kingdom
The simulations were performed by R by using various sample sizes and error variances. During
calculation of m-estimators, OLS and LSR methods were used to fit initial regression model; initial
residuals were found, and they were scaled by MAD; a chosen weight function was applied to obtain
preliminary weights. The preliminary weights were used in iteratively reweighted least squares and
iteratively reweighted least squares ratio methods to obtain regression parameters; secondary residuals
were found during the first iteration. In the second and other iterations, the residuals were scaled by
Huber proposal 2 until the best model was found. The following criteria were used to obtain the final
estimates;
where refers to the number of iterations; indicates a
very small positive number. In this study, took the
value of 0.0001.

13. Murat YAZICI, M.Sc.
A Simulation Study of The LSR vs. The OLS Approach to M-Estimators
SAI Computing Conference, July 2016, London, United Kingdom

14. Murat YAZICI, M.Sc.
Conclusion and Future Work
In this study, it is shown which method (WLS and WLSR) gives better results in M-estimation
according to statistics values of the mean absolute errors (MAE) of the estimated regression
parameters and dependent value through a simulation study using various sample sizes and error
variances. It was studied on Huber, Tukey, Andrew and Ramsay’s weighting functions in this paper.
Based on the simulation results, WLSR Method outperforms than WLS Method in case of the
presence of outliers and increased error variance apart from Andrew’s weighting function. For future
work, other weighting functions in the literature can be examined for which method gives better
results.
SAI Computing Conference, July 2016, London, United Kingdom

15. Murat YAZICI, M.Sc.
About JForce IT Company
 JFORCE is founded in 2003 with a Focus in Insurance, Banking and IBM solutions.
 Acting as a software house and system integrator JForce is delivering innovative
solutions with state of the art technologies.
 With a team of 40 and over 85 Certifications JForce is one of the biggest solution
providers of IBM in Turkey.
 Key Technology focus is in Systems&Middleware : Application Servers, Integration,
Business Process Management, Business Rules Management, Complex Events ,
Statistical Modelling
SAI Computing Conference, July 2016, London, United Kingdom

16. Murat YAZICI, M.Sc.
JFORCE Key Solution Areas for Insurance
JForce is serving Insurance Market with
• Core Insurance Solutions (NTT Data Partnership)
• Claims Process Automation & Analytic Dashboards
• Fraud & Leakage Management
• Underwriting Automation / Contract Management
• Dynamic Pricing
• Telematics and Related Mobile Applications
• Predictive Customer Intellegence, Realtime Marketing and Event Management
• Service Integration
• Provision Automation and
• Online Pharmacy Automation Solutions.
SAI Computing Conference, July 2016, London, United Kingdom

17. Murat YAZICI, M.Sc.
Our Global Partnerships
International Conference for Engineering and Technology, May 2016, Toronto

18. Murat YAZICI, M.Sc.
Awards & Recognitions
2003 IBM Best Project of The Year
2004 IBM Best System i Partner
2004 INDEX Best Performing Partner
2005 IBM Best System i Partner
2006 IBM Best System i Partner
2006 ORACLE Partner Network
2007 IBM Best System i Partner
2008 IBM TÜRK 70. YIL ÖZEL ÖDÜLÜ
2009 IBM Best Performance Websphere
2009 Best Performance – Power i
2009 VISION SOLUTIONS Quota Achievement
2012 IBM Best Project of The Year
2013 Best Performing DB2 Partner
2014 IBM Most Competitive Project of The Year
2015 IBM Technical Accelence Award in 3 Categories
SAI Computing Conference, July 2016, London, United Kingdom

19. Murat YAZICI, M.Sc.
Some of Our References
SAI Computing Conference, July 2016, London, United Kingdom

20. Murat YAZICI, M.Sc.
Contact Us
www.jforce.com.tr
Göztepe Mh. Göksuevleri Sit. Sardunya Sk.
B212B 34810 Anadoluhisari / Istanbul, Turkey
Phone: 0090 216 668 0290
Fax: +90 216 668 02 95
E-mail: info@jforce.com.tr
SAI Computing Conference, July 2016, London, United Kingdom

21. Murat YAZICI, M.Sc.
References
1. R. Muthukrishnan, M. Radha, “M-estimators in regression models,” Journal of Mathematics Research, 2010, vol.
2, no. 4, pp. 23-27.
2. O. Akbilgic, E. D. Akinci, “A novel regression approach: Least squares ratio. Communications in Statistics - Theory
and Methods, 2009, vol. 38, no. 9, pp.1539-1545.
3. R. Andersen, Modern Methods for Robust Regression. Sage Publications, 2007.
4. A. Ali, M. F. Quadir, “A modified M-estimator for the detection of outliers,” Pakistan Journal of Statistics and
Operation Research, 2005, vol.1, no. 1, pp. 49-64.
5. D. C. Hoaglin, F. Mosteller, and J. W. Tukey, Understanding robust and exploratory data analysis. John Wiley and
Sons, New York, 1983.
SAI Computing Conference, July 2016, London, United Kingdom

22. Thank you!..
MURAT YAZICI, M.Sc.
Sr. Data Scientist & Researcher
LinkedIn: https://tr.linkedin.com/in/muraty1
E-mail: muraty@jforce.com.tr
Phone: 0090 539 601 6854