tf.extra_losses - GitLite

KO | EN

gitlite — search

#javascript #python #react #ai #typescript #go #deep-learning #nodejs #macos #java #awesome #pytorch #mcp #ios

tf.extra_losses

(★ 104)

Large-Margin Softmax Loss, Angular Softmax Loss, Additive Margin Softmax, ArcFaceLoss And FocalLoss In Tensorflow

File Explorer

Download Latest Version (.zip)

.gitignore
check_op.py
CMakeLists.txt
common.cc
common.h
l_softmax_grad_op.cc
l_softmax_grad_op.cu
l_softmax_op.cc
l_softmax_op.cu
l_softmax_op.h
LICENSE
py_loss.py
README.md
test_op.py
work_sharder.h

# Installation Guide

1. Get the code

git clone https://github.com/HiKapok/tf.extra_losses

Downloads the entire project code from GitHub to your computer.

cd tf.extra_losses

Moves into the project folder you just downloaded.

2. CMake

Medium Recommended

Prerequisites

Git Needed to download the project code from GitHub.
CMake The tool used to generate build configuration.
C/C++ 컴파일러 Windows needs Visual Studio (Community edition, free), macOS needs Xcode Command Line Tools, Linux needs the gcc/g++ package.

cd build && cmake ..

Moves into the project folder you just downloaded.

✅ Check that an executable was created inside the build folder, then run it directly (e.g. ./build/app_name).

Pulled directly from this repo's README.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Large-Margin Softmax Loss, Angular Softmax Loss, Additive Margin Softmax, ArcFaceLoss And FocalLoss In Tensorflow

This repository contains core codes of the reimplementation of the following papers in TensorFlow:

- [Large-Margin Softmax Loss for Convolutional Neural Networks](https://arxiv.org/abs/1612.02295)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://arxiv.org/abs/1704.08063)
- [Additive Margin Softmax for Face Verification](https://arxiv.org/abs/1801.05599) or [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://arxiv.org/abs/1801.09414)
- [ArcFace: Additive Angular Margin Loss for Deep Face Recognition](https://arxiv.org/abs/1801.07698)
- [Focal Loss for Dense Object Detection](https://arxiv.org/abs/1708.02002)

If your goal is to reproduce the results in the original paper, please use the official codes:

- [Large Margin Softmax Loss in ICML 2016](https://github.com/wy1iu/LargeMargin_Softmax_Loss)
- [Angular Softmax Loss in CVPR 2017](https://github.com/wy1iu/sphereface)
- [Additive Margin Softmax](https://github.com/happynear/AMSoftmax)
- [ArcFace: Additive Angular Margin Loss](https://github.com/deepinsight/insightface)
- [Focal Loss in ICCV 2017](https://github.com/facebookresearch/Detectron)

## ##

For using these Ops on your own machine:

- copy the header file "cuda\_config.h" from "your\_python\_path/site-packages/external/local\_config\_cuda/cuda/cuda/cuda\_config.h" to "your\_python\_path/site-packages/tensorflow/include/tensorflow/stream\_executor/cuda/cuda\_config.h".

- run the following script:

```sh
mkdir build
cd build && cmake ..
make
```

- run "test\_op.py" and check the numeric errors to test your install
- follow the below codes snippet to integrate this Op into your own code:
	- For Large Margin Softmax Loss:

```python
	op_module = tf.load_op_library(so_lib_path)
	large_margin_softmax = op_module.large_margin_softmax

@ops.RegisterGradient("LargeMarginSoftmax")
	def _large_margin_softmax_grad(op, grad, _):
	  '''The gradients for `LargeMarginSoftmax`.
	  '''
	  inputs_features = op.inputs[0]
	  inputs_weights = op.inputs[1]
	  inputs_labels = op.inputs[2]
	  cur_lambda = op.outputs[1]
	  margin_order = op.get_attr('margin_order')

grads = op_module.large_margin_softmax_grad(inputs_features, inputs_weights, inputs_labels, grad, cur_lambda[0], margin_order)
	  return [grads[0], grads[1], None, None]

var_weights = tf.Variable(initial_value, trainable=True, name='lsoftmax_weights')
	result = large_margin_softmax(features, var_weights, labels, global_step, 4, 1000., 0.000025, 35., 0.)
	loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(labels=labels, logits=result[0]))
	```

- For Angular Softmax Loss:

```python
	op_module = tf.load_op_library(so_lib_path)
	angular_softmax = op_module.angular_softmax

@ops.RegisterGradient("AngularSoftmax")
	def _angular_softmax_grad(op, grad, _):
	  '''The gradients for `AngularSoftmax`.
	  '''
	  inputs_features = op.inputs[0]
	  inputs_weights = op.inputs[1]
	  inputs_labels = op.inputs[2]
	  cur_lambda = op.outputs[1]
	  margin_order = op.get_attr('margin_order')

grads = op_module.angular_softmax_grad(inputs_features, inputs_weights, inputs_labels, grad, cur_lambda[0], margin_order)
	  return [grads[0], grads[1], None, None]

var_weights = tf.Variable(initial_value, trainable=True, name='asoftmax_weights')
	normed_var_weights = tf.nn.l2_normalize(var_weights, 1, 1e-10, name='weights_normed')
	result = angular_softmax(features, normed_var_weights, labels, global_step, 4, 1000., 0.000025, 35., 0.)
	loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(labels=labels, logits=result[0]))
	```
	- For others just refer to this [script](https://github.com/HiKapok/tf.extra_losses/blob/master/py_loss.py).

All the codes was tested under TensorFlow 1.6, Python 3.5, Ubuntu 16.04 with CUDA 8.0. The outputs of these Ops in C++ had been compared with the original caffe codes' outputs, and the bias could be ignored. The gradients of this Op had been checked using [tf.test.compute\_gradient\_error](https://www.tensorflow.org/api_docs/python/tf/test/compute_gradient_error) and [tf.test.compute\_gradient](https://www.tensorflow.org/api_docs/python/tf/test/compute_gradient). While the others are implemented following the official implementation in Python Ops.

If you encountered some linkage problem when generating or loading *.so, you are highly recommended to read this section in the [official tourial](https://www.tensorflow.org/extend/adding_an_op#compile_the_op_using_your_system_compiler_tensorflow_binary_installation) to make sure you were using the same C++ ABI version.

Any contributions to this repo is welcomed.

## ##
MIT License