Get different ridge regression weight using deducted formula

Vote count: 0

I used numpy to compute the weight for ridge regression. Two formula is used, the original optimal weight for ridge regression.

The original formula:

The deducted formula using SVD on X:

Given , I get

Now. when I use numpy to compute weight using these two formulas, I should expect same result, but I get different. The first formula's code yields same result as Sklearn.linear_model.Ridge, but the second does not. I cannot find where I did wrong.

Here is the code:

 X = np.random.rand(50, 13)
 y = np.random.rand(50)
 lamb = 100
 first_ridg_coef = np.linalg.inv(np.diag([lamb for i in xrange(X.shape[1])]) + X.T.dot(X)).dot(X.T).dot(y)
 >>[ 0.04117069  0.04430925  0.05702583  0.0656194   0.04440966  0.03160402
     0.04342164  0.05363837  0.04332074  0.04664736  0.04564837  0.04077096
     0.02984096]
 U, e, V = np.linalg.svd(X, full_matrices = false)
 E = np.diag(e)
 second_ridg_coef = V.dot(np.linalg.inv(E.dot(E) + np.diag([lamb for i in xrange(X.shape[1])]))).dot(E).dot(U.T).dot(y)
 >>[ 0.03425105,  0.05637386, -0.02538696,  0.05337022, -0.06876592,
    0.03368448, -0.04782126,  0.02734215, -0.02576422,  0.07095561,
    0.01912719, -0.06375412, -0.02771472]

asked 2 mins ago

Finix

Let's block ads! (Why?)

Get different ridge regression weight using deducted formula