Thinking like an entrepreneur

What comes to your mind when you hear the word “Entrepreneurship”? If you had asked me the same question a year ago I would have said, it is an act of bringing your ideas into existence or an act of…

独家优惠奖金 100% 高达 1 BTC + 180 免费旋转

Predictive Modeling for Regression

course note for Data Mining

1. a method describe relationship between X and Y

1. Linear regression model: linear in the parameters
2. Minimize sum of squares of (residual) error

1. Removing outliers
2. Fitting non-linear functions(more complex)
3. Feature and target scaling (ex. standardization)
4. Dealing with highly-correlated features

1. Scaling by min-max will scale the original values within [0,1]
2. x = (x-min)/(max-min)

1. range too big
2. scale x by sublinear function (ex. log(x), sqrt(x))

1.y = 1000*x1*x2 → log(y)

1.in practice, matrix are hard and complex to compute
→ viewed as parameter fitting
2. learning rate small in early stage, late bigger

Simplifying Models by Regularization
1. Regularization is the trick to add extra terms to the objective function to favor models with smaller coefficients
ex. Ridge regression, LASSO regression

1. smaller w is preferred
2. The closed form to optimize w is

3. can also use gradient descent

1. simple
2. efficient: divide-and-conquer greedy strategy
3. no assumption of function forms
4. unreliable estimates due to smaller samples at lower levels of the tree
5. sensitive to minor changes to training data
6. not so smooth regression functions