【逻辑回归LR算法】 机器学习公式推导计算+详细过程
- Logistic回归是”广义线性模型“，用于解决分类问题。是在线性回归的基础上加入了非线性映射sigmoid函数
线性回归公式
hθ(x)=θ0x0+θ1x1+θ2x2+...+θnxn h_\theta(x) = \theta_0x_0 + \theta_1x_1 + \theta_2x_2 + ... + \theta_nx_n hθ(x)=θ0x0+θ1x1+θ2x2+...+θnxn
线性回归向量形式
hθ(x)=θTx h_\theta(x) = \theta^Tx hθ(x)=θTx
sigmoid函数
g(z)=11+e−z g(z) = \frac{1}{1 + e^{-z} } g(z)=1+e−z1
- 其中， z=hθ(x)=θTx z = h_\theta(x) = \theta^Tx z=hθ(x)=θTx
g(z)=11+e−θTx g(z) = \frac{1}{1 + e^{-\theta^Tx } } g(z)=1+e−θTx1
逻辑回归损失函数
J(θ)=−1m∑i=0m[y(i)loghθ(x(i))+(1−y(i))log(1−hθ(x(i)))] J(\theta) = -\frac{1}{m} \sum_{i=0} ^ m [y^{(i)}log{h_{\theta} (x ^ {(i)})}+ (1 - y ^ {(i)})log{(1 -h_{\theta} (x ^ {(i)}))}] J(θ)=−m1i=0∑m[y(i)loghθ(x(i))+(1−y(i))log(1−hθ(x(i)))]
梯度下降更新θ\thetaθ
θj:=θj−α∂∂θjJ(θ) \theta_j := \theta_j - \alpha \frac{\partial}{\partial \theta_j} J (\theta) θj:=θj−α∂θj∂J(θ)
sigmoid函数求导
g(x)=11+e−x g(x) = \frac{1}{1 + e^{-x } } g(x)=1+e−x1
g′(x)=g(x)(1−g(x))(5.8) g'(x) = g(x)(1 - g(x)) \quad (5.8)g′(x)=g(x)(1−g(x))(5.8)
求偏导数推导
∂J(θ)∂θj=−1m∑i=0m[y(i)1hθ(x(i))∗∂hθ(x(i))∂θj−(1−y(i))∗11−hθ(x(i))∗∂hθ(x(i))∂θj] \frac{\partial J (\theta)}{\partial \theta_j} = -\frac{1}{m} \sum_{i=0} ^ m [y ^ {(i)} \frac{1}{h_\theta(x^{(i)})} * \frac{\partial h_\theta (x^{(i)})}{\partial \theta_j} - (1 - y ^ {(i)}) *\frac{1}{1 - h_\theta(x^{(i)})} * \frac{\partial h_\theta (x^{(i)})}{\partial \theta_j}] ∂θj∂J(θ)=−m1i=0∑m[y(i)hθ(x(i))1∗∂θj∂hθ(x(i))−(1−y(i))∗1−hθ(x(i))1∗∂θj∂hθ(x(i))]
=−1m∑i=0m[y(i)1g(θTx(i))−(1−y(i))11−g(θTx(i))]∗∂g(θTx(i))∂θj = -\frac{1}{m} \sum_{i=0} ^ m [y ^ {(i)} \frac{1}{g(\theta^Tx ^ {(i)})} - (1 - y ^ {(i)}) \frac{1}{1 -g(\theta^Tx ^ {(i)})}] * \frac{\partial g(\theta^Tx ^ {(i)})}{\partial \theta_j} =−m1i=0∑m[y(i)g(θTx(i))1−(1−y(i))1−g(θTx(i))1]∗∂θj∂g(θTx(i))
=−1m∑i=0m[y(i)1g(θTx(i))−(1−y(i))11−g(θTx(i))]∗g(θTx(i))(1−g(θTx(i))xj(i) = -\frac{1}{m} \sum_{i=0} ^ m [y ^ {(i)} \frac{1}{g(\theta^Tx ^ {(i)})} - (1 - y ^ {(i)}) \frac{1}{1 -g(\theta^Tx ^ {(i)})}] * g(\theta^Tx ^ {(i)})(1 - g(\theta^Tx ^ {(i)}) x^{(i)}_j =−m1i=0∑m[y(i)g(θTx(i))1−(1−y(i))1−g(θTx(i))1]∗g(θTx(i))(1−g(θTx(i))xj(i)
=−1m∑i=0m[y(i)(1−g(θTx(i))−(1−y(i))g(θTx(i))]xj(i) = -\frac{1}{m} \sum_{i=0} ^ m [y ^ {(i)} (1 - g(\theta^Tx ^ {(i)}) - (1 - y ^ {(i)})g(\theta^Tx ^ {(i)})] x^{(i)}_j =−m1i=0∑m[y(i)(1−g(θTx(i))−(1−y(i))g(θTx(i))]xj(i)
=−1m∑i=0m(y(i)−g(θTx(i)))xj(i) = -\frac{1}{m} \sum_{i=0} ^ m (y ^ {(i)} - g(\theta^Tx ^ {(i)})) x^{(i)}_j=−m1i=0∑m(y(i)−g(θTx(i)))xj(i)
=1m∑i=0m（hθ(x(i))−y(i)）xj(i) = \frac{1}{m} \sum_{i=0} ^ m（h_\theta(x^{(i)}) - y^{(i)}） x^{(i)}_j =m1i=0∑m（hθ(x(i))−y(i)）xj(i)
欢迎大家交流学习，任何问题都可以留言
北京人形旗下天工造物具身智能开源社区，聚焦具身天工与慧思开物两大平台
更多推荐
0
0- 0
已为社区贡献3条内容
所有评论(0)