In simple terms, regression analysis is a statistical method of examining the relationship between two or more variables. To understand the relationship between two variables it is important to understand the types of variables. Variables are of two types-
- Dependent Variable– These are the variables that change with changes in other variables. These are the variables which are usually tested for dependence. The dependence of these variables is tested on other variables
- Independent Variable– These are the variables that do not change by any changes in other variables. As the name signifies, these are independent of other variables. Independent variable is also called explanatory variable
Example– If one wants to analyze the dependence of heart rate on three factors say age, height, and weight. In this case, the heart rate is the dependent variable since its dependence is being tested on the other three variables. The other three variables that are age, height and weight are independent variables. This is because these three variables will not change and are impacting the heart rate. Any changes in these three variables will change the heart rate (dependent variable) but vice versa is not possible. If the age of a person is increased, then his/her heart rate will change. But if the heart rate is changed then it will not change the age of the person.
In other words, the variable(s) being affected is the dependent variable and the variable(s) effecting are the independent variables. Thus, it can be stated that regression analysis examines the influence of one or more independent variables on a dependent variable(s).
Apart from establishing the relationship between variables, regression analysis is also a type of predictive modeling technique. In other words, regression analysis can be used to predict the value of the dependent variable(s) as well. Thus, in regression analysis, independent variables are also known as the predictor variable and the dependent variable is called a target variable. For predicting the values, a model of the relationship is hypothesized, and estimates of the parameter values are used to develop an estimated regression equation. Various tests are then employed to determine if the model is satisfactory. If the model is deemed satisfactory, the estimated regression equation can be used to predict the value of the dependent variable given values for the independent variables.
Depending on the number of variables, regression analysis can be categorized further-
- Simple Linear Regression– When there is one dependent and one independent variable
- Multiple Linear Regression– When there are one dependent variable and multiple independent variables
- Multivariate Regression– When there are multiple dependent variables and multiple independent variables
Though there are many types of regression analysis, at their core they all examine the influence of one or more independent variables on a dependent variable(s).
Some of the commonly used regression analysis which will be discussed in this series of discussion have been enlisted as follows-
- Linear Regression
- Multivariate regression
- Logistic regression
- Ordinal regression
- Poisson Regression
- Lasso Regression
- Principal Component Regression (PCR)
- Partial Least Square (PLS) Regression
For any queries, comments or additional information required on the above discussion, contact us please drop a mail to info@helpinproject.com