Bent Identity

Bent Identity activation function.

This activation function provides a smooth approximation of the identity function. It introduces non-linearity while preserving the identity mapping for large inputs.

activations_plus.BentIdentity.forward(self, x: Tensor) → Tensor

Compute a custom transformation of the input tensor.

Perform a combination of squaring, square rooting, and addition. The result is adjusted and normalized using specific constants.

Parameters:

x (torch.Tensor) – A tensor of numeric values on which the custom transformation is performed. The tensor should consist of real-valued numbers.

Returns:

A tensor containing the transformed values after applying the custom operation to the input tensor.

Return type:

torch.Tensor

Reference Paper: Bent Identity Activation Function

Mathematical Explanation:

The Bent Identity activation function is defined as:

\[f(x) = \frac{\sqrt{x^2 + 1} - 1}{2} + x\]

This introduces a slight non-linearity for negative inputs.

Example Usage:

import torch
from activations_plus.bent_identity import BentIdentity

activation = BentIdentity()
x = torch.tensor([-3.0, 0.0, 3.0])
y = activation(x)
print("Bent Identity Output:", y)