[DL] Softsign : tanh의 유사품

hyperbolic tangent (=tanh)와 유사한 함수.

tanh 대신 activation function으로 사용되는 경우도 있음.

$$\text{softsign}(x)=\frac{x}{1+|x|}$$

softsign의 derivative는 다음과 같음.

$$\dfrac{d}{dx}\text{softsign}(x)=\dfrac{1}{\left(1+|x|\right)^2}$$

---

차트 비교

---

관련소스

import numpy as np
import matplotlib.pyplot as plt
 
x = np.linspace(-20.,20., 100)
softsign = x/ (1.+np.abs(x))
logistic = 1/ (1+np.exp(-x))
tanh = np.tanh(x)
 
fig,ax = plt.subplots(1,1)
ax.plot(x,softsign, label='soft sign')
ax.plot(x,logistic, label='logistic')
ax.plot(x,tanh, label='tanh')
ax.legend()
ax.grid()

---

참고자료들

2023.08.13 - [Computer] - [DL] Hyperbolic Tangent Function (tanh)

[DL] Hyperbolic Tangent Function (tanh)

2022.12.28 - [.../Math] - [Math] Sigmoid function

[Math] Sigmoid function

https://gist.github.com/dsaint31x/9de184e851b49f3f3232b0498648bc0c

math_softsign.ipynb