SBA (Sparse Bundle Adjustment)

class sba.core.SBA(viewpoint_indices, point_indices, do_check_args=True)

The constructor takes two arguments: viewpoint_indices and point_indices, that represent visibility of 3D points in each viewpoint.

In general, not all 3D points can be observed from all viewpoints. Some points cannot be observed because of occlusion, motion blur, etc.

We consider an example that we have four 3D points \(\{\mathbf{p}_0, \mathbf{p}_2, \mathbf{p}_3, \mathbf{p}_4\}\) that are observed from three cameras under the condition that:

  • All 3D points can be observed from the zeroth viewpoint.
  • \(\{\mathbf{p}_0, \mathbf{p}_2, \mathbf{p}_3\}\) can be observed from the first viewpoint.
  • \(\{\mathbf{p}_1, \mathbf{p}_2\}\) can be observed from the second viewpoint.

Then, viewpoint_indices and point_indices should be the following:

viewpoint_indices = [0, 0, 0, 0, 1, 1, 1, 2, 2]
    point_indices = [0, 1, 2, 3, 0, 2, 3, 1, 2]
Args:
viewpoint_indices (list of ints), size n_keypoints:
Array of viewpoint indices.
point_indices (list of ints), size n_keypoints:
Array of point indices.
weights (np.ndarray), shape (n_keypoints, 2, 2):
Weight matrices
do_check_args (bool, optional):
SBA.compute checks if given arguments are satisfying the condition that the approximated Hessian \(J^{\top} J\) is invertible.
This can be disabled by setting check_args=False.
compute(x_true, x_pred, A, B, weights=None, mu=0.0)

Calculate a Gauss-Newton update. Elements of the arguments correspond to argument arrays of the constructor. For example, if the index arrays are like below,

viewpoint_indices = [0, 0, 0, 0, 1, 1, 1, 2, 2]
point_indices     = [0, 1, 2, 3, 0, 2, 3, 1, 2]

then x_true should be

\[\mathbf{x}_{true} = \begin{bmatrix} \mathbf{x}_{00} & \mathbf{x}_{01} & \mathbf{x}_{02} & \mathbf{x}_{03} & \mathbf{x}_{10} & \mathbf{x}_{12} & \mathbf{x}_{13} & \mathbf{x}_{21} & \mathbf{x}_{22} \end{bmatrix}^{\top}.\]

Other arguments also should follow this manner.

Args:
x_true (np.ndarray), shape (n_keypoints, 2):
Observed 2D keypoints of shape.
x_pred (np.ndarray), shape (n_keypoints, 2):
2D keypoints predicted by a projection function
A (np.ndarray), shape (n_keypoints, 2, n_pose_params):
Jacobian with respect to pose parameters.
Each block A[index] represents a jacobian of x_pred[index] with respect to the corresponding pose parameter.
B (np.ndarray), shape (n_keypoints, 2, 3):
Jacobian with respect to 3D points.
Each block B[index] represents a jacobian of x_pred[index] with respect to a 3D point coordinate.
weights (np.ndarray), shape (n_keypoints, 2, 2):
Weights for Gauss-Newton
Each weights[index] has to be symmetric
mu (float), mu >= 0:
Damping factor in the Levenberg-Marquardt method
Returns:
(tuple):
delta_a (np.ndarray), shape (n_viewpoints, n_pose_params):
Update of pose parameters.
delta_b (np.ndarray), shape (n_points, n_point_params):
Update of 3D points.

Indices and tables