ANOVA andLinear Regression
Analysis of Variance(ANOVA)
BUS B272 Unit 1Analysis of VarianceThe Analysis of Variance (ANOVA) is a procedure that tests to determine whether differences exist between two or more populations.The techniques analyzes the variance of the data to determine whether we can infer that the populations differ.
	One way (Single-factor) analysis of varianceANOVA assumptionsF  test for difference among k  meansBUS B272 Unit 1Topics
BUS B272 Unit 1General Experimental SettingInvestigator controls one or more independent variablesCalled treatments or factorsEach treatment contains two or more levels (or categories/classifications)Observe effects on dependent variableResponse to different levels of independent variableExperimental design: the plan used to test hypothesis
BUS B272 Unit 1Completely Randomized DesignExperimental units (subjects) are assigned randomly to treatmentsSubjects are assumed homogeneousOnly one factor or independent variableWith two or more treatment levelsAnalyzed byOne-way analysis of variance (one-way ANOVA)
BUS B272 Unit 1Randomized Design Example
BUS B272 Unit 1One-way Analysis of Variance F  TestEvaluate the difference among the mean responses of 2 or more (k) populationse.g. : Several types of tires, oven 	temperature settings, different types 	of marketing strategies
BUS B272 Unit 1Samples are randomly and independently drawn
This condition must be met
Populations are normally distributed
F  test is robust to moderate departure from normality
Populations have equal variancesAssumptions of ANOVA
BUS B272 Unit 1Hypotheses of One-Way ANOVAAll population means are equal No treatment effect (no variation in means among groups)At least one population mean is different (others may be the same!) There is treatment effect Does not mean that all population means are different
BUS B272 Unit 1One-way ANOVA (No Treatment Effect)The Null Hypothesis is True
BUS B272 Unit 1One-way ANOVA (Treatment Effect Present)The Null Hypothesis is NOT True
BUS B272 Unit 1One-way ANOVA(Partition of Total Variation)Total Variation SS(Total)Variation Due to Treatment   SSTVariation Due to Random Sampling   SSE+=
BUS B272 Unit 1ANOVA set-up
BUS B272 Unit 1Total Variation     : the i-th observation in group j     : the number of observations in group jn   : the total number of observations in all groupsk   :  the number of groupsthe overall or grand mean
BUS B272 Unit 1Total Variation(continued)
BUS B272 Unit 1Among-Treatments VariationVariation Due to Differences Among Groups
BUS B272 Unit 1Among-Treatments Variation(continued)
BUS B272 Unit 1Summing the variation within each treatment and then adding over all treatments.Within-Treatment Variation
BUS B272 Unit 1Within-Treatment Variation(continued)
BUS B272 Unit 1Within-Treatment Variation(continued)If more than 2 groups, use F test.
For 2 groups, use t-test.  F test is more limited.For k = 2, this is the pooled-variance in the t-test.
BUS B272 Unit 1One-way ANOVAF  Test StatisticTest statistic:MST is mean squares among or between variancesMSE is mean squares within or error variancesDegrees of freedom:
BUS B272 Unit 1One-way ANOVA Summary Table
BUS B272 Unit 1Features of One-way ANOVA F StatisticThe F statistic is the ratio of the among estimate of variance and the within estimate of variance.The ratio must always be positive df1 = k -1 will typically be smalldf2 = n - k  will typically be largeThe ratio should be closed to 1 if the null is true.
BUS B272 Unit 1One-way ANOVA F  Test ExampleAs production manager, you want to see if three filling machines have different mean filling times.  You assign 15 similarly trained and experienced workers, five per machine, to the machines.  At the 0.05 significance level, is there a difference in mean filling times?Machine1Machine2Machine3	25.40	      23.40	      20.00	26.31	      21.80	      22.20	24.10	      23.50	      19.75	23.74	      22.75	      20.60	25.10	      21.60	      20.40
BUS B272 Unit 1One-way ANOVA Example: Scatter DiagramMachine1Machine2Machine3	25.40	      23.40	      20.00	26.31	      21.80	      22.20	24.10	      23.50	      19.75	23.74	      22.75	      20.60	25.10	      21.60	      20.40Time in Seconds272625242322212019•••••••••••••••
BUS B272 Unit 1Machine 1Machine 2Machine 3	25.40	     23.40	      20.00	26.31	     21.80	      22.20	24.10	     23.50	      19.75	23.74	     22.75	      20.60	25.10	     21.60	      20.40One-way ANOVA Example Computations
BUS B272 Unit 1
BUS B272 Unit 1Summary TableMST/MSE=25.6023-1=247.164023.582015-3=1211.05320.921115-1=1458.2172
BUS B272 Unit 1 = 0.05F0One-way ANOVA Example SolutionCritical Value(s):H0: 1 = 2 = 3H1: Not all the means are equalTest Statistic: 3.89df1= 2      df2 = 12Reject H0 at  = 0.05There is evidence to believe that at least one  i  differs from the rest.
BUS B272 Unit 1Computer ApplicationTo obtain the Microsoft Excel computer output in the previous page, first enter the data into c columns in an Excel file, then follow the commands:	Tools/ Data Analysis/ Anova: Single Factor
BUS B272 Unit 1Computer Output using Data Analysis of Excel
Exercise 1The manager of a large department store wants to test if the average size of customer transactions differs with four types of payment: Visa card, company card, cash or cheque. If there are differences in the average customer transaction size among the four types of payment, the manager will further investigate which types of payment will give rise to higher transaction volumes and hence he will design an appropriate promotional programme. A random sample of 54 customer transactions using various types of payment was drawn during the past two months. With reference to sampled data, the sample statistics are obtained as follows:BUS B272 Unit 1Test if differences of average customer transaction size exist among the four types of payment at a 0.05 level of significance.
Exercise 1BUS B272 Unit 1One factor is involved, i.e. the type of payment. Under this factor, there are k = 4 treatments (or factor levels) which represent the four types of payment: Visa card, company card, cash and cheque. The experimental units are customer transactions.
Exercise 1Since the test statistic of 39.16 is greater than the critical value of 2.80, reject H0. At 0.05 level of significance, there is evidence to reveal that the average customer transaction sizes are significantly different among the four types of payment. BUS B272 Unit 1
Can ANOVA be replaced by t-Test?t-Test : any difference between two population means μ1 and μ2Multiple t-tests are required for more than two population meansConducting multiple tests increases the probability of making Type I errors. 	E.g. compare 6 population means, if use ANOVA with significant level 5%, there will be a 5% chance we reject the null hypothesis when it is true. 	If we use t-test, we need to perform 15 tests and if same 5% significant level is set, the chance of a Type I error will be		1 – (1 - 0.05)15 = 0.54BUS B272 Unit 1
Linear Regression
BUS B272 Unit 1Linear RegressionOrigin of regressionDetermining the simple linear regression equationAssessing the fitness of the model Correlation analysisEstimation and prediction Assumptions of regression and correlation
BUS B272 Unit 1Origin of Regression“Regression," from a Latin root meaning "going back," is a series of statistical methods used in studying the relationship between two variables and were first employed by Francis Galton in 1877. Galton was interested in studying the relationship between a father’s height and the son’ s height. Making use of the “regression” method, he found that son’s height regress to the overall mean and the method is then called “regression”.
BUS B272 Unit 1Linear Regression AnalysisLinear Regression analysis is used primarily to model and describe linear relationship and provide prediction among variables Predicts the value of a dependent (response) variable based on the value of at least one independent (explanatory) variableExpress statistically the effect of the independent variables on the dependent variable
BUS B272 Unit 1Types of Regression ModelsPositive Linear RelationshipRelationship NOT LinearNegative Linear RelationshipNo Relationship
BUS B272 Unit 1Simple Linear Regression ModelThe relationship between two variables, sayX and Y,  is described by a linear function.The change of the variable Y, (called dependent or response variable) is associated with the change in the other variable X(called independent or explanatory variable). Explore the dependency of Y on X.
(4, 5)(2, 2.5)(3, 2.5)(1, 2)Why Regression?The larger the sum of squares, the poor the estimate.X1234Y22.52.55BUS B272 Unit 1
BUS B272 Unit 1Linear RelationshipWe wish to study whether there is any association between two quantitative variables, sayX and YIf ‘Y tends to increase as X increases’ If ‘Y tends to decrease as X increases’	If the corresponding magnitude of increase or decrease follows a specific proportion, the relationship identified is said to be a linear one.–  apositive relationship–  anegative relationship
BUS B272 Unit 1Scatter DiagramA scatter diagram is a graph plotted for all X-Y pairs of the sample data.By viewing a scatter diagram, one can determine whether a relationship exists between the two variables. It can also suggest the likely mathematical form of that relationship that allow one to judge initially and intuitively whether or not there exists a linear relationship between the two variables involved.
BUS B272 Unit 1ExampleThe level of air pollution at Kwun Tong and the total number of consultations relating to respiratory diseases in a public clinic in the area were recorded during a specific time period on 14 randomly selected days.
BUS B272 Unit 1Population Linear RegressionPopulation regression line is a straight line that describes the dependence of the average value (conditional mean) of one variable on the other Random ErrorPopulation SlopeCoefficient Population Y  intercept Dependent (Response) VariablePopulationRegressionLine  (conditional mean)Independent (Explanatory) Variable
BUS B272 Unit 1Population Linear Regression(continued)Random Error (vertical discrepancies or residual for point i )Y(Observed Value of Y) =(Conditional Mean)XObserved Value of Y
BUS B272 Unit 1Least Squares MethodThe line fitted by least squares is the one that makes the sum of squares of all those vertical discrepancies (residuals) as small as possible, i.e. minimum of which is the sum of squared residuals.
BUS B272 Unit 1Sample Y  interceptResidualSample regression line is formed by the point estimates of     and     , i.e.,     and    .  It provides an estimate of the population regression line as well as a predicted value of YSample Linear RegressionSamplecoefficient of slopeSample regression line (Fitted regression line or predicted value)
BUS B272 Unit 1Sample Linear Regression(continued)and      are obtained by finding the specific values of       and      that minimizes the sum of the squared residuals
BUS B272 Unit 1Coefficients of Sample Linear RegressionFor
BUS B272 Unit 1Interpretation of the Slope and the Interceptis the average value of Y when the value of X  is zero.		 measures the change in the average value of Y as a result of a one-unit change in X.
BUS B272 Unit 1(continued)is the estimated average value of Y when the value of X  is zero.	  	 is the estimated change in the average value of Y as a result of one-unit change in X.Interpretation of the Slope and the Intercept
BUS B272 Unit 1Example 1 : Simple Linear RegressionSuppose that you want to examine the linear dependency of the annual sales among seven stores on their size in square footage. Sample data for seven stores were obtained. Find the equation of the straight line that fits the data best.Annual Store	   Square 	 Sales		     Feet	($1000)   1           1,726	  3,681   2           1,542	  3,395   3	     2,816	  6,653   4	     5,555	  9,543   5	     1,292	  3,318   6	     2,208	  5,563   7	     1,313	  3,760
BUS B272 Unit 1Example 1 : Scatter DiagramExcel Output
BUS B272 Unit 1Computation of Regression Coefficient	                     Annual           Square     SalesStore   Feet      ($1000) XY   1       1,726      3,681	  	   2       1,542      3,395	  	   3	  2,816      6,653	   4	  5,555      9,543	   5	  1,292      3,318	  	   6	  2,208      5,563	   7	  1,313      3,760	  	 2,979,076 2,377,764 7,929,85630,858,025 1,669,264 4,875,264 1,723,96913,549,76111,526,02544,262,40991,068,84911,009,12430,946,96914,137,600  6,353,406   5,235,09018,734,84853,011,365  4,286,85612,283,104  4,936,88016,45235,913104,841,54952,413,218216,500,737
BUS B272 Unit 1Computation of Regression Coefficient
BUS B272 Unit 1Example 1 : Equation for the Sample 	Regression LineYi = 1636.415 +1.487Xi
BUS B272 Unit 1Example 1 : Interpretation of Results The slope of 1.487 means that for each increase of one unit in X, we predict the average of Y to increase by an estimated 1.487 units.The model estimates that for each increase of one square foot in the size of the store, the expected annual sales are predicted to increase by $1487.
BUS B272 Unit 1Predicting Annual Sales Based on Square Footage Suppose that we would like to use the fitted model to predict the average annual sales for a store with 4,000 square feet.
BUS B272 Unit 1Interpolation versus ExtrapolationFor using regression line for prediction purpose, it is not appropriate to make predictions beyond the relevant range (in the previous example: (1,292, 5,555)) of the independent variable.That is, we may interpolate within the relevant range of X  values, but we SHOULD NOT extrapolate beyond the range of X values. For example, it is not appropriate to predict the average annual sales for a store with 7,000 square feet since it is beyond the range of X  values, i.e., (1,292, 5,555).
BUS B272 Unit 1Causal Relationship?In general, when there is a relationship identified between X and Y using regression analysis, we usually would say that ‘X is associated with Y’ instead of saying ‘X causes Y’.We cannot claim that two variables are related by cause and effect just because there is a statistical relationship between the two. In fact, you cannot infer a causal relationship from statistics alone.
BUS B272 Unit 1For example, the price of dog food and houses, may well be positively correlated over time. When you collect data concerning the price of dog food and the price of houses over time, you might end up with an inference that they have a positive relationship, but can you conclude that an increase in the price of dog food would directly cause the price of houses to increase too? It might be that an inflationary force is influencing both and hence they can be seen to move in the same general direction over time.
BUS B272 Unit 1Computer ApplicationImport the data into two adjacent columns in an Excel file and then click Tools/Data Analysis/ Regression(See page 624-5 for detail description).
BUS B272 Unit 1Example 1: Computer Output
BUS B272 Unit 1Exercise 2Consider the example about the level of air pollution at Kwun Tong and the total number of consultations that relate to respiratory diseases in a public clinic in the area. The corresponding data were given as follows:
BUS B272 Unit 1Exercise 1(a)	Determine the sample regression line to predict 	the number of consultations by the level of 	pollution.(b)	Interpret the coefficients.Solution:
BUS B272 Unit 1Exercise 1For      , each additional increase in pollution level, the number of consultations increases, on average by 0.456701074. No meaningful interpretation for       can be made, as the range of x does not include zero.
BUS B272 Unit 1Assessing the simple linear regression modelFrom time to time, after we have set up a linear regression model, we wish to assess the fitness of the model. That is, we wish to find out how well the model fit to the given data. For a good fit, the data as a whole should be quite close to the regression line and the independent variable can thus be used to predict the value of the dependent variable with high accuracy. To examine how well the independent variable predicts the dependent variable, we need to develop several measures of variation.
BUS B272 Unit 1Total Sample VariabilityUnexplained Variability=Explained Variability+Measure of Variation: The Sum of SquaresSS(Total)         =SSR            +           SSE
BUS B272 Unit 1Measure of Variation: The Sum of SquaresSS(Total) = total sum of squares Measures the variation of the Yi values around their mean YSSR = regression sum of squares Explained variation attributable to the relationship between X and YSSE = error sum of squares Variation attributable to factors other than the relationship between X and Y  (Unexplained variation)(continued)
BUS B272 Unit 1Measure of Variation: The Sum of Squares_SS(Total) = (Yi  – Y )2(continued)YYiSSE=(Yi - Yi)2__SSR = (Yi - Y)2_YXXi
BUS B272 Unit 1
BUS B272 Unit 1Standard Error of EstimateThe standard deviation of the variation of observations around the regression line.
The smallest value that        can assume is 0, which occurs when SSE = 0, that is, when all the points fall on the regression line. Thus, when      is small, the fit is excellent, and the linear regression model is likely to be an effective analytical and forecasting tool.When      is large, the regression model is a poor one, it is of little value to be used.BUS B272 Unit 1Standard Error of Estimate
BUS B272 Unit 1The Coefficient of Determination (r 2  or R 2 )By themselves, SSR, SSE  and SS(Total) provide little that can be directly interpreted.  A simple ratio of SSR and SS(Total) provides a measure of the usefulness of the regression equation.Measures the proportion of variation in Y  that is explained by the independent variable X  in the regression model
BUS B272 Unit 1Coefficients of Determination (r 2)r2 = 1YYr2 = 1^Y = b + bXi01i^Y = b + bXi01iXXr2 = 0r2 = 0.8YY^^Y = b + bXY = b + bXi01ii01iXX
BUS B272 Unit 1Coefficient of CorrelationCoefficient of correlation is used to measure strength of association (linear relationship) between two numerical variables)Only concerned with strength of the relationshipNo causal effect is implied
BUS B272 Unit 1(continued)Population correlation coefficient is denoted by  (Rho).Sample correlation coefficient is denoted by r . It is an estimate of   and is used to measure the strength of the linear relationship in the sample observations.Coefficient of Correlation
BUS B272 Unit 1Coefficient of Correlation
BUS B272 Unit 1Sample of Observations from Various r  ValuesYYYXXXr = –1r = –0.6r = 0YYXXr = 0.6r = 1
BUS B272 Unit 1Features of r and rUnit freeRange between –1 and 1The closer to –1, the stronger the negative linear relationshipThe closer to 1, the stronger the positive linear relationshipThe closer to 0, the weaker the linear relationship
BUS B272 Unit 1There is also a more systematic way to assess model fitness, i.e., to perform a hypothesis testing on the slope of the regression line.Inference about the SlopeIf the two variables involved are not at all linearly related, one could observe from the scatter diagram shown on the right that the slope of the regression line will be zero.
BUS B272 Unit 1Hence, we can determine whether a significant relationship between the variables X  and Y exists by testing whether 	(the true slope) is equal to zero.Inference about the Slope(There is no linear relationship)(There is a linear relationship)If       is rejected, there is evidence to believe that a linear relationship exists between X  and Y.
BUS B272 Unit 1The standard error of the slopeThe estimated standard error of     .
BUS B272 Unit 1Inference about the Slope: t  Testt  test for a population slopeIs there a linear dependency of Y on X ?Null and alternative hypothesesH0:  1 = 0	(no linear dependency)H1:  1 0	(linear dependency)Test statistic:
BUS B272 Unit 1Example: Store SalesData for Seven Stores:Estimated Regression Equation:Annual Store	   Square 	 Sales		     Feet	($000)   1           1,726	  3,681   2           1,542	  3,395   3	     2,816	  6,653   4	     5,555	  9,543   5	     1,292	  3,318   6	     2,208	  5,563   7	     1,313	  3,760	Yi = 1636.415 +1.487XiThe slope of this model is 1.487. Is square footage of the store affecting its annual sales?
H0: 1 = 0          0.05H1: 1 0          df7 - 2  = 5Test Statistic: BUS B272 Unit 1
BUS B272 Unit 1Inferences about the Slope: t  Test ExampleRejectReject0.0250.02502.5706-2.5706Decision:Conclusion:Critical Value(s):Reject H0At 5% level of significance, there is evidence to reveal that square footage is associated with annual sales.
BUS B272 Unit 1(No linear relationship)(A linear relationship)(No positive linear relationship)(A positive linear relationship)(No negative linear relationship)(A negative linear relationship)Inferences about the Slope
BUS B272 Unit 1Exercise 3	Consider the data of Exercise 2 about the level of air pollution at Kwun Tong and the total number of consultations that relate to respiratory diseases in a public clinic in the area. Test at the 5% level of significance to determine whether level of air pollution and the total number of consultations are positively linearly related.
BUS B272 Unit 1Solution:0.05;   df14 - 2  = 12
BUS B272 Unit 1Exercise 3
BUS B272 Unit 1Computer OutputFor two-tailed test
BUS B272 Unit 1Exercise 3Decision:Conclusion:Reject H0Critical Value(s):Reject H0At 5% level of significance, there is evidence to believe that level of air pollution and total number of consultations are positively linearly related.0.0501.7823
BUS B272 Unit 1You have seen how can we assess the model fitness. If the model fits satisfactorily, we can use it to forecast and estimate values of the dependent variable. We can obtain a point prediction of Y with a given value of X  using the linear regression line.Confidence interval about the particular value of Y  or the average of Y  for a given value of X  can also be computed if desired.Estimation of Mean Values
BUS B272 Unit 1Estimation of Mean ValuesConfidence interval estimate for             :The mean of Y given a particular  Size of interval varies according to distance away from mean,    Standard error of the estimatet value from table with df = n - 2
BUS B272 Unit 1Prediction of Individual ValuesPrediction interval for individual response Yi at a particular Addition of one increases width of interval from that for the mean of Y
BUS B272 Unit 1Interval Estimates for Different Values of XConfidence Interval for the mean of YPrediction Interval for a individual YiYYi = b0 + b1XiXY given X
BUS B272 Unit 1Example: Stores SalesData for seven stores:Predict the annual sales for a store with 2000 square feet.Annual Store	   Square 	 Sales		     Feet	($000)   1           1,726	  3,681   2           1,542	  3,395   3	     2,816	  6,653   4	     5,555	  9,543   5	     1,292	  3,318   6	     2,208	  5,563   7	     1,313	  3,760	Regression Model Obtained:Yi = 1636.415 +1.487Xi
Estimation of Mean Values: ExampleConfidence Interval Estimate forFind the 95% confidence interval for the average annual sales for a 2,000 square-foot store.Predicted Sales Yi = 1636.415 +1.487Xi = 4609.68 ($000)tn-2 = t5 = 2.571X = 2350.29BUS B272 Unit 1
Prediction Interval for Y : ExamplePrediction Interval for Individual YFind the 95% prediction interval                                           for the annual sales of a 2,000 square-foot storePredicted Sales Yi = 1636.415 +1.487Xi = 4609.68 ($000)tn-2 = t5 = 2.571X = 2350.29BUS B272 Unit 1
BUS B272 Unit 1Computer ApplicationCommands:Tools/ Data Analysis Plus/ Prediction Interval.
BUS B272 Unit 1Computer Output
BUS B272 Unit 1Linear Regression Assumptions1.  NormalityY values are normally distributed for each XProbability distribution of error is normal2.	Homoscedasticity (Constant Variance)3.	Independence of Errors
BUS B272 Unit 1Y  values are normally distributed around the regression line.
 For each X value, the “spread” or variance around the regression line is the same.Variation of Errors around the Regression Linef(e)YX2X1XSample Regression Line.
Multiple Regression
BUS B272 Unit 1IntroductionExtension of the simple linear regression model to allow for any fixed number of independent variables. That is, the number of independent variables could be more than one.
BUS B272 Unit 1Multiple Linear RegressionTo make use of computer printout to Assess the modelHow well it fits the dataIs it usefulAre any required conditions violated?Employ the modelInterpreting the coefficientsPredictions using the prediction equationEstimating the expected value of the dependent variable
BUS B272 Unit 1Allow for k independent variables to potentially be related to the dependent variabley = b0 + b1x1+ b2x2 + …+ bkxk + eRegressionCoefficientsRandom error variableDependent variableIndependent variablesModel and Required Conditions
Multiple Regression for k = 2, Graphical DemonstrationX1The simple linear regression modelallows for one independent variable, “x”for y = b0 + b1x + eyy = b0 + b1x1 + b2x2y = b0 + b1x1 + b2x2y = b0 + b1x1 + b2x2y = b0 + b1x1 + b2x2y = b0 + b1x1 + b2x2y = b0 + b1x1 + b2x2y = b0 + b1x1 + b2x2The multiple linear regression modelallows for more than one independent variable.Y = b0 + b1x1 + b2x2  + eX2BUS B272 Unit 1
BUS B272 Unit 1The errore is normally distributed.The mean is equal to zero and the standard deviation is constant (se)for all values of y. The errors are independent.Required conditions for the error variable
BUS B272 Unit 1Estimating the Coefficients andAssessing the ModelThe procedure used to perform multiple regression analysis:Obtain the model coefficients and statistics using a statistical software.
Assess the model fitness using statistics obtained from the sample.
If the model assessment indicates good fit to the data, use it to interpret the coefficients and generate predictions.BUS B272 Unit 1Estimating the Coefficients and Assessing the Model, ExampleExample 18.1 Keller: Where to locate a new motor inn?La Quinta Motor Inns is planning to build new inns.Management wishes to predict which sites are likely to be profitable.Several areas where predictors of profitability (operating margin) can be identified are:CompetitionMarket awarenessDemand generatorsDemographicsPhysical qualityLa Quinta defines profitable inns as those with an operating margin in excess of 50% and unprofitable ones with margins of less than 30%.
Estimating the Coefficients and Assessing the Model, ExamplePhysicalProfitabilityMargin (%)Market awarenessCompetitionCustomersCommunityNumberOfficespaceIncomeDistanceNearestEnrollmentMedianhouseholdincome of nearbyarea (in $thousands)Number of hotels/motelsrooms within 3 miles from the siteEnrollemnt in nearby university or college (in thousands)Distance to the downtowncore (in miles)Number of miles to closest competitionOffice space in nearby communityBUS B272 Unit 1
BUS B272 Unit 1Estimating the Coefficients and Assessing the Model, ExampleData were collected from randomly selected 100 inns that belong to La Quinta, and ran for the following suggested model:Margin = b0 + b1Rooms + b2Nearest + b3Office + 	b4College + b5Income + b6DisttwnXm18-01
BUS B272 Unit 1Regression Analysis, Excel OutputMargin = 38.14 - 0.0076Number +1.65Nearest+ 0.020Office Space +0.21Enrollment+ 0.41Income - 0.23DistanceThis is the sample regression equation (sometimes called the prediction equation)
BUS B272 Unit 1Model AssessmentThe model is assessed using two tools:The coefficient of determinationThe F -test of the analysis of varianceThe standard error of estimates participates in building the above tools.
BUS B272 Unit 1Standard Error of EstimateThe standard deviation of the error is estimated by the Standard Error of Estimate:The magnitude of seis judged by comparing it to
BUS B272 Unit 1From the printout, se = 5.51 Calculating the mean value of y, we haveIt seems se is not particularly small. Question:Can we conclude the model does not fit the data well? Standard Error of Estimate
BUS B272 Unit 1Coefficient of DeterminationThe definition is:From the printout,  r 2 = 0.525152.51% of the variation in operating margin is explained by the six independent variables. 47.49% remains unexplained.
BUS B272 Unit 1Testing the Validity of the ModelFor testing the validity of the model, the following question is asked:	Is there at least one independent variable linearly related to the dependent variable? To answer the question we test the hypothesisH0: b1 = b2 = … = bk = 0H1: At least one bi is not equal to zero.If at least one bi is not equal to zero, the model has some validity or usefulness.
BUS B272 Unit 1Testing the Validity of the La Quinta Inns Regression ModelThe hypotheses are tested by an ANOVA procedure ( the Excel output)MSR / MSEk      =n–k–1 =   n-1  = SSRMSR=SSR / kSSEMSE=SSE / (n-k-1)
BUS B272 Unit 1Testing the Validity of the La Quinta Inns Regression Model	[Total variation in y] SS(Total) = SSR + SSE. 	Large F  results from a large SSR. That implies much of the variation in y can be explained by the regression model; the model is useful, and thus, the null hypothesis should be rejected.  Therefore, the rejection region is:F > Fa, k, n – k – 1while the test statistic is:
BUS B272 Unit 1Testing the Validity of the La Quinta Inns Regression ModelFa, k, n-k-1 = F0.05,6,100-6 -1 = 2.17F = 17.14 > 2.17Conclusion:  There is sufficient evidence to reject the null hypothesis in favor of the alternative hypothesis.  At least one of the bi is not equal to zero. Thus, at least one independent variable is linearly related to y.   This linear regression model is valid.Also, the p-value (Significance F) = 0.0000; Reject the null hypothesis.

Bus b272 f unit 1

  • 1.
  • 2.
  • 3.
    BUS B272 Unit1Analysis of VarianceThe Analysis of Variance (ANOVA) is a procedure that tests to determine whether differences exist between two or more populations.The techniques analyzes the variance of the data to determine whether we can infer that the populations differ.
  • 4.
    One way (Single-factor)analysis of varianceANOVA assumptionsF test for difference among k meansBUS B272 Unit 1Topics
  • 5.
    BUS B272 Unit1General Experimental SettingInvestigator controls one or more independent variablesCalled treatments or factorsEach treatment contains two or more levels (or categories/classifications)Observe effects on dependent variableResponse to different levels of independent variableExperimental design: the plan used to test hypothesis
  • 6.
    BUS B272 Unit1Completely Randomized DesignExperimental units (subjects) are assigned randomly to treatmentsSubjects are assumed homogeneousOnly one factor or independent variableWith two or more treatment levelsAnalyzed byOne-way analysis of variance (one-way ANOVA)
  • 7.
    BUS B272 Unit1Randomized Design Example
  • 8.
    BUS B272 Unit1One-way Analysis of Variance F TestEvaluate the difference among the mean responses of 2 or more (k) populationse.g. : Several types of tires, oven temperature settings, different types of marketing strategies
  • 9.
    BUS B272 Unit1Samples are randomly and independently drawn
  • 10.
  • 11.
  • 12.
    F testis robust to moderate departure from normality
  • 13.
    Populations have equalvariancesAssumptions of ANOVA
  • 14.
    BUS B272 Unit1Hypotheses of One-Way ANOVAAll population means are equal No treatment effect (no variation in means among groups)At least one population mean is different (others may be the same!) There is treatment effect Does not mean that all population means are different
  • 15.
    BUS B272 Unit1One-way ANOVA (No Treatment Effect)The Null Hypothesis is True
  • 16.
    BUS B272 Unit1One-way ANOVA (Treatment Effect Present)The Null Hypothesis is NOT True
  • 17.
    BUS B272 Unit1One-way ANOVA(Partition of Total Variation)Total Variation SS(Total)Variation Due to Treatment SSTVariation Due to Random Sampling SSE+=
  • 18.
    BUS B272 Unit1ANOVA set-up
  • 19.
    BUS B272 Unit1Total Variation : the i-th observation in group j : the number of observations in group jn : the total number of observations in all groupsk : the number of groupsthe overall or grand mean
  • 20.
    BUS B272 Unit1Total Variation(continued)
  • 21.
    BUS B272 Unit1Among-Treatments VariationVariation Due to Differences Among Groups
  • 22.
    BUS B272 Unit1Among-Treatments Variation(continued)
  • 23.
    BUS B272 Unit1Summing the variation within each treatment and then adding over all treatments.Within-Treatment Variation
  • 24.
    BUS B272 Unit1Within-Treatment Variation(continued)
  • 25.
    BUS B272 Unit1Within-Treatment Variation(continued)If more than 2 groups, use F test.
  • 26.
    For 2 groups,use t-test. F test is more limited.For k = 2, this is the pooled-variance in the t-test.
  • 27.
    BUS B272 Unit1One-way ANOVAF Test StatisticTest statistic:MST is mean squares among or between variancesMSE is mean squares within or error variancesDegrees of freedom:
  • 28.
    BUS B272 Unit1One-way ANOVA Summary Table
  • 29.
    BUS B272 Unit1Features of One-way ANOVA F StatisticThe F statistic is the ratio of the among estimate of variance and the within estimate of variance.The ratio must always be positive df1 = k -1 will typically be smalldf2 = n - k will typically be largeThe ratio should be closed to 1 if the null is true.
  • 30.
    BUS B272 Unit1One-way ANOVA F Test ExampleAs production manager, you want to see if three filling machines have different mean filling times. You assign 15 similarly trained and experienced workers, five per machine, to the machines. At the 0.05 significance level, is there a difference in mean filling times?Machine1Machine2Machine3 25.40 23.40 20.00 26.31 21.80 22.20 24.10 23.50 19.75 23.74 22.75 20.60 25.10 21.60 20.40
  • 31.
    BUS B272 Unit1One-way ANOVA Example: Scatter DiagramMachine1Machine2Machine3 25.40 23.40 20.00 26.31 21.80 22.20 24.10 23.50 19.75 23.74 22.75 20.60 25.10 21.60 20.40Time in Seconds272625242322212019•••••••••••••••
  • 32.
    BUS B272 Unit1Machine 1Machine 2Machine 3 25.40 23.40 20.00 26.31 21.80 22.20 24.10 23.50 19.75 23.74 22.75 20.60 25.10 21.60 20.40One-way ANOVA Example Computations
  • 33.
  • 34.
    BUS B272 Unit1Summary TableMST/MSE=25.6023-1=247.164023.582015-3=1211.05320.921115-1=1458.2172
  • 35.
    BUS B272 Unit1 = 0.05F0One-way ANOVA Example SolutionCritical Value(s):H0: 1 = 2 = 3H1: Not all the means are equalTest Statistic: 3.89df1= 2 df2 = 12Reject H0 at  = 0.05There is evidence to believe that at least one  i differs from the rest.
  • 36.
    BUS B272 Unit1Computer ApplicationTo obtain the Microsoft Excel computer output in the previous page, first enter the data into c columns in an Excel file, then follow the commands: Tools/ Data Analysis/ Anova: Single Factor
  • 37.
    BUS B272 Unit1Computer Output using Data Analysis of Excel
  • 38.
    Exercise 1The managerof a large department store wants to test if the average size of customer transactions differs with four types of payment: Visa card, company card, cash or cheque. If there are differences in the average customer transaction size among the four types of payment, the manager will further investigate which types of payment will give rise to higher transaction volumes and hence he will design an appropriate promotional programme. A random sample of 54 customer transactions using various types of payment was drawn during the past two months. With reference to sampled data, the sample statistics are obtained as follows:BUS B272 Unit 1Test if differences of average customer transaction size exist among the four types of payment at a 0.05 level of significance.
  • 39.
    Exercise 1BUS B272Unit 1One factor is involved, i.e. the type of payment. Under this factor, there are k = 4 treatments (or factor levels) which represent the four types of payment: Visa card, company card, cash and cheque. The experimental units are customer transactions.
  • 40.
    Exercise 1Since thetest statistic of 39.16 is greater than the critical value of 2.80, reject H0. At 0.05 level of significance, there is evidence to reveal that the average customer transaction sizes are significantly different among the four types of payment. BUS B272 Unit 1
  • 41.
    Can ANOVA bereplaced by t-Test?t-Test : any difference between two population means μ1 and μ2Multiple t-tests are required for more than two population meansConducting multiple tests increases the probability of making Type I errors. E.g. compare 6 population means, if use ANOVA with significant level 5%, there will be a 5% chance we reject the null hypothesis when it is true. If we use t-test, we need to perform 15 tests and if same 5% significant level is set, the chance of a Type I error will be 1 – (1 - 0.05)15 = 0.54BUS B272 Unit 1
  • 42.
  • 43.
    BUS B272 Unit1Linear RegressionOrigin of regressionDetermining the simple linear regression equationAssessing the fitness of the model Correlation analysisEstimation and prediction Assumptions of regression and correlation
  • 44.
    BUS B272 Unit1Origin of Regression“Regression," from a Latin root meaning "going back," is a series of statistical methods used in studying the relationship between two variables and were first employed by Francis Galton in 1877. Galton was interested in studying the relationship between a father’s height and the son’ s height. Making use of the “regression” method, he found that son’s height regress to the overall mean and the method is then called “regression”.
  • 45.
    BUS B272 Unit1Linear Regression AnalysisLinear Regression analysis is used primarily to model and describe linear relationship and provide prediction among variables Predicts the value of a dependent (response) variable based on the value of at least one independent (explanatory) variableExpress statistically the effect of the independent variables on the dependent variable
  • 46.
    BUS B272 Unit1Types of Regression ModelsPositive Linear RelationshipRelationship NOT LinearNegative Linear RelationshipNo Relationship
  • 47.
    BUS B272 Unit1Simple Linear Regression ModelThe relationship between two variables, sayX and Y, is described by a linear function.The change of the variable Y, (called dependent or response variable) is associated with the change in the other variable X(called independent or explanatory variable). Explore the dependency of Y on X.
  • 48.
    (4, 5)(2, 2.5)(3,2.5)(1, 2)Why Regression?The larger the sum of squares, the poor the estimate.X1234Y22.52.55BUS B272 Unit 1
  • 49.
    BUS B272 Unit1Linear RelationshipWe wish to study whether there is any association between two quantitative variables, sayX and YIf ‘Y tends to increase as X increases’ If ‘Y tends to decrease as X increases’ If the corresponding magnitude of increase or decrease follows a specific proportion, the relationship identified is said to be a linear one.– apositive relationship– anegative relationship
  • 50.
    BUS B272 Unit1Scatter DiagramA scatter diagram is a graph plotted for all X-Y pairs of the sample data.By viewing a scatter diagram, one can determine whether a relationship exists between the two variables. It can also suggest the likely mathematical form of that relationship that allow one to judge initially and intuitively whether or not there exists a linear relationship between the two variables involved.
  • 51.
    BUS B272 Unit1ExampleThe level of air pollution at Kwun Tong and the total number of consultations relating to respiratory diseases in a public clinic in the area were recorded during a specific time period on 14 randomly selected days.
  • 52.
    BUS B272 Unit1Population Linear RegressionPopulation regression line is a straight line that describes the dependence of the average value (conditional mean) of one variable on the other Random ErrorPopulation SlopeCoefficient Population Y intercept Dependent (Response) VariablePopulationRegressionLine (conditional mean)Independent (Explanatory) Variable
  • 53.
    BUS B272 Unit1Population Linear Regression(continued)Random Error (vertical discrepancies or residual for point i )Y(Observed Value of Y) =(Conditional Mean)XObserved Value of Y
  • 54.
    BUS B272 Unit1Least Squares MethodThe line fitted by least squares is the one that makes the sum of squares of all those vertical discrepancies (residuals) as small as possible, i.e. minimum of which is the sum of squared residuals.
  • 55.
    BUS B272 Unit1Sample Y interceptResidualSample regression line is formed by the point estimates of and , i.e., and . It provides an estimate of the population regression line as well as a predicted value of YSample Linear RegressionSamplecoefficient of slopeSample regression line (Fitted regression line or predicted value)
  • 56.
    BUS B272 Unit1Sample Linear Regression(continued)and are obtained by finding the specific values of and that minimizes the sum of the squared residuals
  • 57.
    BUS B272 Unit1Coefficients of Sample Linear RegressionFor
  • 58.
    BUS B272 Unit1Interpretation of the Slope and the Interceptis the average value of Y when the value of X is zero. measures the change in the average value of Y as a result of a one-unit change in X.
  • 59.
    BUS B272 Unit1(continued)is the estimated average value of Y when the value of X is zero. is the estimated change in the average value of Y as a result of one-unit change in X.Interpretation of the Slope and the Intercept
  • 60.
    BUS B272 Unit1Example 1 : Simple Linear RegressionSuppose that you want to examine the linear dependency of the annual sales among seven stores on their size in square footage. Sample data for seven stores were obtained. Find the equation of the straight line that fits the data best.Annual Store Square Sales Feet ($1000) 1 1,726 3,681 2 1,542 3,395 3 2,816 6,653 4 5,555 9,543 5 1,292 3,318 6 2,208 5,563 7 1,313 3,760
  • 61.
    BUS B272 Unit1Example 1 : Scatter DiagramExcel Output
  • 62.
    BUS B272 Unit1Computation of Regression Coefficient Annual Square SalesStore Feet ($1000) XY 1 1,726 3,681 2 1,542 3,395 3 2,816 6,653 4 5,555 9,543 5 1,292 3,318 6 2,208 5,563 7 1,313 3,760 2,979,076 2,377,764 7,929,85630,858,025 1,669,264 4,875,264 1,723,96913,549,76111,526,02544,262,40991,068,84911,009,12430,946,96914,137,600 6,353,406 5,235,09018,734,84853,011,365 4,286,85612,283,104 4,936,88016,45235,913104,841,54952,413,218216,500,737
  • 63.
    BUS B272 Unit1Computation of Regression Coefficient
  • 64.
    BUS B272 Unit1Example 1 : Equation for the Sample Regression LineYi = 1636.415 +1.487Xi
  • 65.
    BUS B272 Unit1Example 1 : Interpretation of Results The slope of 1.487 means that for each increase of one unit in X, we predict the average of Y to increase by an estimated 1.487 units.The model estimates that for each increase of one square foot in the size of the store, the expected annual sales are predicted to increase by $1487.
  • 66.
    BUS B272 Unit1Predicting Annual Sales Based on Square Footage Suppose that we would like to use the fitted model to predict the average annual sales for a store with 4,000 square feet.
  • 67.
    BUS B272 Unit1Interpolation versus ExtrapolationFor using regression line for prediction purpose, it is not appropriate to make predictions beyond the relevant range (in the previous example: (1,292, 5,555)) of the independent variable.That is, we may interpolate within the relevant range of X values, but we SHOULD NOT extrapolate beyond the range of X values. For example, it is not appropriate to predict the average annual sales for a store with 7,000 square feet since it is beyond the range of X values, i.e., (1,292, 5,555).
  • 68.
    BUS B272 Unit1Causal Relationship?In general, when there is a relationship identified between X and Y using regression analysis, we usually would say that ‘X is associated with Y’ instead of saying ‘X causes Y’.We cannot claim that two variables are related by cause and effect just because there is a statistical relationship between the two. In fact, you cannot infer a causal relationship from statistics alone.
  • 69.
    BUS B272 Unit1For example, the price of dog food and houses, may well be positively correlated over time. When you collect data concerning the price of dog food and the price of houses over time, you might end up with an inference that they have a positive relationship, but can you conclude that an increase in the price of dog food would directly cause the price of houses to increase too? It might be that an inflationary force is influencing both and hence they can be seen to move in the same general direction over time.
  • 70.
    BUS B272 Unit1Computer ApplicationImport the data into two adjacent columns in an Excel file and then click Tools/Data Analysis/ Regression(See page 624-5 for detail description).
  • 71.
    BUS B272 Unit1Example 1: Computer Output
  • 72.
    BUS B272 Unit1Exercise 2Consider the example about the level of air pollution at Kwun Tong and the total number of consultations that relate to respiratory diseases in a public clinic in the area. The corresponding data were given as follows:
  • 73.
    BUS B272 Unit1Exercise 1(a) Determine the sample regression line to predict the number of consultations by the level of pollution.(b) Interpret the coefficients.Solution:
  • 74.
    BUS B272 Unit1Exercise 1For , each additional increase in pollution level, the number of consultations increases, on average by 0.456701074. No meaningful interpretation for can be made, as the range of x does not include zero.
  • 75.
    BUS B272 Unit1Assessing the simple linear regression modelFrom time to time, after we have set up a linear regression model, we wish to assess the fitness of the model. That is, we wish to find out how well the model fit to the given data. For a good fit, the data as a whole should be quite close to the regression line and the independent variable can thus be used to predict the value of the dependent variable with high accuracy. To examine how well the independent variable predicts the dependent variable, we need to develop several measures of variation.
  • 76.
    BUS B272 Unit1Total Sample VariabilityUnexplained Variability=Explained Variability+Measure of Variation: The Sum of SquaresSS(Total) =SSR + SSE
  • 77.
    BUS B272 Unit1Measure of Variation: The Sum of SquaresSS(Total) = total sum of squares Measures the variation of the Yi values around their mean YSSR = regression sum of squares Explained variation attributable to the relationship between X and YSSE = error sum of squares Variation attributable to factors other than the relationship between X and Y (Unexplained variation)(continued)
  • 78.
    BUS B272 Unit1Measure of Variation: The Sum of Squares_SS(Total) = (Yi – Y )2(continued)YYiSSE=(Yi - Yi)2__SSR = (Yi - Y)2_YXXi
  • 79.
  • 80.
    BUS B272 Unit1Standard Error of EstimateThe standard deviation of the variation of observations around the regression line.
  • 81.
    The smallest valuethat can assume is 0, which occurs when SSE = 0, that is, when all the points fall on the regression line. Thus, when is small, the fit is excellent, and the linear regression model is likely to be an effective analytical and forecasting tool.When is large, the regression model is a poor one, it is of little value to be used.BUS B272 Unit 1Standard Error of Estimate
  • 82.
    BUS B272 Unit1The Coefficient of Determination (r 2 or R 2 )By themselves, SSR, SSE and SS(Total) provide little that can be directly interpreted. A simple ratio of SSR and SS(Total) provides a measure of the usefulness of the regression equation.Measures the proportion of variation in Y that is explained by the independent variable X in the regression model
  • 83.
    BUS B272 Unit1Coefficients of Determination (r 2)r2 = 1YYr2 = 1^Y = b + bXi01i^Y = b + bXi01iXXr2 = 0r2 = 0.8YY^^Y = b + bXY = b + bXi01ii01iXX
  • 84.
    BUS B272 Unit1Coefficient of CorrelationCoefficient of correlation is used to measure strength of association (linear relationship) between two numerical variables)Only concerned with strength of the relationshipNo causal effect is implied
  • 85.
    BUS B272 Unit1(continued)Population correlation coefficient is denoted by  (Rho).Sample correlation coefficient is denoted by r . It is an estimate of  and is used to measure the strength of the linear relationship in the sample observations.Coefficient of Correlation
  • 86.
    BUS B272 Unit1Coefficient of Correlation
  • 87.
    BUS B272 Unit1Sample of Observations from Various r ValuesYYYXXXr = –1r = –0.6r = 0YYXXr = 0.6r = 1
  • 88.
    BUS B272 Unit1Features of r and rUnit freeRange between –1 and 1The closer to –1, the stronger the negative linear relationshipThe closer to 1, the stronger the positive linear relationshipThe closer to 0, the weaker the linear relationship
  • 89.
    BUS B272 Unit1There is also a more systematic way to assess model fitness, i.e., to perform a hypothesis testing on the slope of the regression line.Inference about the SlopeIf the two variables involved are not at all linearly related, one could observe from the scatter diagram shown on the right that the slope of the regression line will be zero.
  • 90.
    BUS B272 Unit1Hence, we can determine whether a significant relationship between the variables X and Y exists by testing whether (the true slope) is equal to zero.Inference about the Slope(There is no linear relationship)(There is a linear relationship)If is rejected, there is evidence to believe that a linear relationship exists between X and Y.
  • 91.
    BUS B272 Unit1The standard error of the slopeThe estimated standard error of .
  • 92.
    BUS B272 Unit1Inference about the Slope: t Testt test for a population slopeIs there a linear dependency of Y on X ?Null and alternative hypothesesH0: 1 = 0 (no linear dependency)H1: 1 0 (linear dependency)Test statistic:
  • 93.
    BUS B272 Unit1Example: Store SalesData for Seven Stores:Estimated Regression Equation:Annual Store Square Sales Feet ($000) 1 1,726 3,681 2 1,542 3,395 3 2,816 6,653 4 5,555 9,543 5 1,292 3,318 6 2,208 5,563 7 1,313 3,760 Yi = 1636.415 +1.487XiThe slope of this model is 1.487. Is square footage of the store affecting its annual sales?
  • 94.
    H0: 1 =0 0.05H1: 1 0 df7 - 2 = 5Test Statistic: BUS B272 Unit 1
  • 95.
    BUS B272 Unit1Inferences about the Slope: t Test ExampleRejectReject0.0250.02502.5706-2.5706Decision:Conclusion:Critical Value(s):Reject H0At 5% level of significance, there is evidence to reveal that square footage is associated with annual sales.
  • 96.
    BUS B272 Unit1(No linear relationship)(A linear relationship)(No positive linear relationship)(A positive linear relationship)(No negative linear relationship)(A negative linear relationship)Inferences about the Slope
  • 97.
    BUS B272 Unit1Exercise 3 Consider the data of Exercise 2 about the level of air pollution at Kwun Tong and the total number of consultations that relate to respiratory diseases in a public clinic in the area. Test at the 5% level of significance to determine whether level of air pollution and the total number of consultations are positively linearly related.
  • 98.
    BUS B272 Unit1Solution:0.05; df14 - 2 = 12
  • 99.
    BUS B272 Unit1Exercise 3
  • 100.
    BUS B272 Unit1Computer OutputFor two-tailed test
  • 101.
    BUS B272 Unit1Exercise 3Decision:Conclusion:Reject H0Critical Value(s):Reject H0At 5% level of significance, there is evidence to believe that level of air pollution and total number of consultations are positively linearly related.0.0501.7823
  • 102.
    BUS B272 Unit1You have seen how can we assess the model fitness. If the model fits satisfactorily, we can use it to forecast and estimate values of the dependent variable. We can obtain a point prediction of Y with a given value of X using the linear regression line.Confidence interval about the particular value of Y or the average of Y for a given value of X can also be computed if desired.Estimation of Mean Values
  • 103.
    BUS B272 Unit1Estimation of Mean ValuesConfidence interval estimate for :The mean of Y given a particular Size of interval varies according to distance away from mean, Standard error of the estimatet value from table with df = n - 2
  • 104.
    BUS B272 Unit1Prediction of Individual ValuesPrediction interval for individual response Yi at a particular Addition of one increases width of interval from that for the mean of Y
  • 105.
    BUS B272 Unit1Interval Estimates for Different Values of XConfidence Interval for the mean of YPrediction Interval for a individual YiYYi = b0 + b1XiXY given X
  • 106.
    BUS B272 Unit1Example: Stores SalesData for seven stores:Predict the annual sales for a store with 2000 square feet.Annual Store Square Sales Feet ($000) 1 1,726 3,681 2 1,542 3,395 3 2,816 6,653 4 5,555 9,543 5 1,292 3,318 6 2,208 5,563 7 1,313 3,760 Regression Model Obtained:Yi = 1636.415 +1.487Xi
  • 107.
    Estimation of MeanValues: ExampleConfidence Interval Estimate forFind the 95% confidence interval for the average annual sales for a 2,000 square-foot store.Predicted Sales Yi = 1636.415 +1.487Xi = 4609.68 ($000)tn-2 = t5 = 2.571X = 2350.29BUS B272 Unit 1
  • 108.
    Prediction Interval forY : ExamplePrediction Interval for Individual YFind the 95% prediction interval for the annual sales of a 2,000 square-foot storePredicted Sales Yi = 1636.415 +1.487Xi = 4609.68 ($000)tn-2 = t5 = 2.571X = 2350.29BUS B272 Unit 1
  • 109.
    BUS B272 Unit1Computer ApplicationCommands:Tools/ Data Analysis Plus/ Prediction Interval.
  • 110.
    BUS B272 Unit1Computer Output
  • 111.
    BUS B272 Unit1Linear Regression Assumptions1. NormalityY values are normally distributed for each XProbability distribution of error is normal2. Homoscedasticity (Constant Variance)3. Independence of Errors
  • 112.
    BUS B272 Unit1Y values are normally distributed around the regression line.
  • 113.
    For eachX value, the “spread” or variance around the regression line is the same.Variation of Errors around the Regression Linef(e)YX2X1XSample Regression Line.
  • 114.
  • 115.
    BUS B272 Unit1IntroductionExtension of the simple linear regression model to allow for any fixed number of independent variables. That is, the number of independent variables could be more than one.
  • 116.
    BUS B272 Unit1Multiple Linear RegressionTo make use of computer printout to Assess the modelHow well it fits the dataIs it usefulAre any required conditions violated?Employ the modelInterpreting the coefficientsPredictions using the prediction equationEstimating the expected value of the dependent variable
  • 117.
    BUS B272 Unit1Allow for k independent variables to potentially be related to the dependent variabley = b0 + b1x1+ b2x2 + …+ bkxk + eRegressionCoefficientsRandom error variableDependent variableIndependent variablesModel and Required Conditions
  • 118.
    Multiple Regression fork = 2, Graphical DemonstrationX1The simple linear regression modelallows for one independent variable, “x”for y = b0 + b1x + eyy = b0 + b1x1 + b2x2y = b0 + b1x1 + b2x2y = b0 + b1x1 + b2x2y = b0 + b1x1 + b2x2y = b0 + b1x1 + b2x2y = b0 + b1x1 + b2x2y = b0 + b1x1 + b2x2The multiple linear regression modelallows for more than one independent variable.Y = b0 + b1x1 + b2x2 + eX2BUS B272 Unit 1
  • 119.
    BUS B272 Unit1The errore is normally distributed.The mean is equal to zero and the standard deviation is constant (se)for all values of y. The errors are independent.Required conditions for the error variable
  • 120.
    BUS B272 Unit1Estimating the Coefficients andAssessing the ModelThe procedure used to perform multiple regression analysis:Obtain the model coefficients and statistics using a statistical software.
  • 121.
    Assess the modelfitness using statistics obtained from the sample.
  • 122.
    If the modelassessment indicates good fit to the data, use it to interpret the coefficients and generate predictions.BUS B272 Unit 1Estimating the Coefficients and Assessing the Model, ExampleExample 18.1 Keller: Where to locate a new motor inn?La Quinta Motor Inns is planning to build new inns.Management wishes to predict which sites are likely to be profitable.Several areas where predictors of profitability (operating margin) can be identified are:CompetitionMarket awarenessDemand generatorsDemographicsPhysical qualityLa Quinta defines profitable inns as those with an operating margin in excess of 50% and unprofitable ones with margins of less than 30%.
  • 123.
    Estimating the Coefficientsand Assessing the Model, ExamplePhysicalProfitabilityMargin (%)Market awarenessCompetitionCustomersCommunityNumberOfficespaceIncomeDistanceNearestEnrollmentMedianhouseholdincome of nearbyarea (in $thousands)Number of hotels/motelsrooms within 3 miles from the siteEnrollemnt in nearby university or college (in thousands)Distance to the downtowncore (in miles)Number of miles to closest competitionOffice space in nearby communityBUS B272 Unit 1
  • 124.
    BUS B272 Unit1Estimating the Coefficients and Assessing the Model, ExampleData were collected from randomly selected 100 inns that belong to La Quinta, and ran for the following suggested model:Margin = b0 + b1Rooms + b2Nearest + b3Office + b4College + b5Income + b6DisttwnXm18-01
  • 125.
    BUS B272 Unit1Regression Analysis, Excel OutputMargin = 38.14 - 0.0076Number +1.65Nearest+ 0.020Office Space +0.21Enrollment+ 0.41Income - 0.23DistanceThis is the sample regression equation (sometimes called the prediction equation)
  • 126.
    BUS B272 Unit1Model AssessmentThe model is assessed using two tools:The coefficient of determinationThe F -test of the analysis of varianceThe standard error of estimates participates in building the above tools.
  • 127.
    BUS B272 Unit1Standard Error of EstimateThe standard deviation of the error is estimated by the Standard Error of Estimate:The magnitude of seis judged by comparing it to
  • 128.
    BUS B272 Unit1From the printout, se = 5.51 Calculating the mean value of y, we haveIt seems se is not particularly small. Question:Can we conclude the model does not fit the data well? Standard Error of Estimate
  • 129.
    BUS B272 Unit1Coefficient of DeterminationThe definition is:From the printout, r 2 = 0.525152.51% of the variation in operating margin is explained by the six independent variables. 47.49% remains unexplained.
  • 130.
    BUS B272 Unit1Testing the Validity of the ModelFor testing the validity of the model, the following question is asked: Is there at least one independent variable linearly related to the dependent variable? To answer the question we test the hypothesisH0: b1 = b2 = … = bk = 0H1: At least one bi is not equal to zero.If at least one bi is not equal to zero, the model has some validity or usefulness.
  • 131.
    BUS B272 Unit1Testing the Validity of the La Quinta Inns Regression ModelThe hypotheses are tested by an ANOVA procedure ( the Excel output)MSR / MSEk =n–k–1 = n-1 = SSRMSR=SSR / kSSEMSE=SSE / (n-k-1)
  • 132.
    BUS B272 Unit1Testing the Validity of the La Quinta Inns Regression Model [Total variation in y] SS(Total) = SSR + SSE. Large F results from a large SSR. That implies much of the variation in y can be explained by the regression model; the model is useful, and thus, the null hypothesis should be rejected. Therefore, the rejection region is:F > Fa, k, n – k – 1while the test statistic is:
  • 133.
    BUS B272 Unit1Testing the Validity of the La Quinta Inns Regression ModelFa, k, n-k-1 = F0.05,6,100-6 -1 = 2.17F = 17.14 > 2.17Conclusion: There is sufficient evidence to reject the null hypothesis in favor of the alternative hypothesis. At least one of the bi is not equal to zero. Thus, at least one independent variable is linearly related to y. This linear regression model is valid.Also, the p-value (Significance F) = 0.0000; Reject the null hypothesis.
  • 134.
    BUS B272 Unit1Interpreting the Coefficientsb0 = 38.14. This is the intercept, the value of y when all the variables take the value zero. Since the data range of all the independent variables do not cover the value zero, do not interpret the intercept. b1 = – 0.0076. In this model, for each additional room within 3 mile of the La Quinta inn, the operating margin decreases on average by 0.0076% (assuming the other variables are held constant).
  • 135.
    BUS B272 Unit1Interpreting the Coefficientsb2 = 1.65. In this model, for each additional mile that the nearest competitor is to a La Quinta inn, the operating margin increases on average by 1.65% when the other variables are held constant. b3 = 0.020.For each additional 1000 sq-ft of office space, the operating margin will increase on average by 0.02% when the other variables are held constant. b4 = 0.21. For each additional thousand students the operating margin increases on average by 0.21% when the other variables are held constant.
  • 136.
    BUS B272 Unit1Interpreting the Coefficientsb5 = 0.41. For additional $1000 increase in median household income, the operating margin increases on average by 0.41%, when the other variables remain constant.b6 = -0.23. For each additional mile to the downtown center, the operating margin decreases on average by 0.23% when the other variables are held constant.
  • 137.
    BUS B272 Unit1Testing the CoefficientsThe hypothesis for each bi isExcel printoutTest statistic:H0: bi= 0H1: bi¹ 0d.f. = n - k -1
  • 138.
    BUS B272 Unit1Using the Linear Regression EquationThe model can be used for making predictions byProducing prediction interval estimate for the particular value of y, for a given set of values of xi.Producing a confidence interval estimate for the expected value of y, for a given set of values of xi.The model can be used to learn about relationships between the independent variables xi, and the dependent variable y, by interpreting the coefficients bi
  • 139.
    BUS B272 Unit1La Quinta Inns, PredictionsXm18-01Predict the average operating margin of an inn at a site with the following characteristics:3815 rooms within 3 miles,Closet competitor 0.9 miles away,476,000 sq-ft of office space,24,500 college students,$35,000 median household income,11.2 miles away from downtown center.MARGIN = 38.14 - 0.0076(3815)+1.65(0.9) + 0.020(476) +0.21(24.5) + 0.41(35) - 0.23(11.2) = 37.1%
  • 140.
    BUS B272 Unit1La Quinta Inns, PredictionsInterval estimates by Excel (Data Analysis Plus)It is predicted, with 95% confidence that the operating margin will lie between 25.4% and 48.8%.It is estimated the average operating margin of all sites that fit this category falls within 33% and 41.2%.Both of them suggested that the given site would not be profitable (less than 50%).