多元函数的极值点与求解

Relative Extreme Values

设 $f(x,y)$ 是双变量函数，点 $(x_0,y_0)$ 被称为极值点，当且仅当 $f(x_0,y_0)\ge (\le) f(x,y)$ for all $(x,y)$ in some neighborhood of $(x_0,y_0)$．

当 $(x_0,y_0)$ 为 $f(x,y)$ 的极值点时，有 $f_x(x_0,y_0)=f_y(x_0,y_0)=0$．满足偏导数为 $0$ 的点称为 critical point．

Saddle Point, Extreme Point

令 $P(x_0,y_0)$ 为 $f(x,y)$ 的关键点，且 $A=f_{xx}(x_0,y_0), B=f_{xy}(x_0,y_0),C=f_{yy}(x_0,y_0)$，对于 $H=AC-B^2$

• 如果 $H\gt 0\texttt{ and }A\lt 0$，那么 $P$ 为极大值点 (Relative Maximum)
• 如果 $H\gt 0\texttt{ and }A\gt 0$，那么 $P$ 为极小值点 (Relative Minimum)
• 如果 $H\lt 0$，那么 $P$ 是鞍点 (Saddle Point)
• 否则 $H=0$，无法判断．

拉格朗日乘数法找极值点

沃趣，这不是最优化理论里面讲到的符号优化方法吗（

Consider constraints $g(x,y,z)=0$, we just need to minimize the following objective function, so that the constraints are satisfied.

$$f(x,y,z)-\lambda g(x,y,z)$$

Consider finding the partial derivatives for $x,y,z,\lambda$ so as to find the critical points.

$$\begin{aligned} f_x &=\lambda g_x \\ f_y &=\lambda g_y \\ f_z &=\lambda g_z \\ g &= 0 \end{aligned}$$

Newton-Raphson Method to Solve Equations

沃趣，迭代法！

经典的牛顿迭代法是这样做单变量的迭代：

$$x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}$$

This method can be extended to multi-variable cases. Consider the following eqⁿs

$$\begin{cases} f(x,y)=0 \\ g(x,y)=0 \end{cases}$$

假设其真实解为 $(a,b)$ 我们初始的估计为 $(x_0,y_0)$，那么我们可以不妨设

$$\begin{aligned} x_0+h&=a\\ y_0+k&=b \end{aligned}$$

那么，让两个方程的左式都在 $(x_0,y_0)$ 处进行泰勒展开：

$$\begin{cases} 0=f(x_0,y_0)+h f_x(x_0,y_0)+k f_y(x_0,y_0)\\ 0=g(x_0,y_0)+h g_x(x_0,y_0)+k g_y(x_0,y_0)\\ \end{cases}$$

于是，我们可以解出 $h_0,k_0$ 的具体数值，再代入 $x_1=x_0+h_0,y_1=y_0+k_0$ 进行多轮迭代．