Published at May 11, 2023
2 min read
Integral Human Pose Regression
Heatmap based are currently the most used method for Pose Estimation. However, despite its good performance, there are some issues sush as non-differentiable post-processing or quantization error... This post propose a simple Integral operation for avoiding those issues as it is differrentiable, effiecient,...
Heatmap Issues
Operation
Given heat map H(K) for K th joint. Known that each location in the heatmap represent the probability of the location being the joint.
The final joint location coordinate is
In a simple term. It find the location with maximum value in the heatmap.
Issues
- The taking-maximum operation is not differentiable -> cant be a component of learning -> prevents end-to-end traning.
- Quantization Error - lower resolution than that of input image (Because of Pooling step) and it is limited by the quantization factor.
- Large Heatmap can be used -> cost of extra storage, computation.
Integral Regression
Modifies the Taking maximum operation to taking-expectation. So the joint is estimated as the integration of all locations in the heatmap, weighted by their probabilities (normalized from likelihoods). The Integral regression (Soft-argmax) is parameter free, only transforms the pose representation from heatmap to joint. So it does not affect other algorithm and can be combined with any of them (or any tasks in the image below).
Advantages over Heatmap based
- Enable End-to-end.
- Output is continuous and up to arbitrary localization accuracy (solve the quantization problem).
Operation
Modify the Max operation to take the expectation
Notes
- Quantization Erro: Using discrete set for represent a continuous values.