matrix calculus

[Math] Hessian : Summary

이 문서는 Numeartor Layout Convention 을 사용함. Hessian : Summary 2nd order derivative of multivariate functoin. 여기서 multivariate function은 입력은 vector, 출력은 scalar Hessian matrix는 다음과 같음. $$\begin{aligned}H[f](\textbf{x})=H(\textbf{x})&=\left(J\left[\nabla f(\textbf{x})\right]\right)^T\\ &= \begin{bmatrix}\dfrac{\partial^2 f}{\partial x_1^2} (\textbf{x})& \dfrac{\partial^2 f}{\partial {x_1} \partial{x_2..

[Math] Commonly used Vector derivatives.

많이 사용되는 vector 도함수들을 정리함. Numeartor Layout 과 Denominator Layout을 구분하여 살펴야 함. $$f(\textbf{x})$$ $$\frac{\partial f(\textbf{x})}{\partial \textbf{x}}$$ Convention $$f(x)$$ $$\frac{df(x)}{dx}$$ $$\textbf{x}^T \textbf{b}$$ $$\textbf{b}^T$$ Numerator Layout $$bx$$ $$b$$ $$\textbf{b}^T \textbf{x}$$ $$\textbf{b}^T$$ Numerator Layout $$bx$$ $$b$$ $\textbf{b}\cdot \textbf{x}$ or $\textbf{x}\cdot \textbf{..