Instrumentation & Measurement Magazine 24-3 - 28

A Modern Solution for an Old
Calibration Problem
Shuwei Qiu, Miaomiao Wang, and Mehrdad R. Kermani

C

ameras endow a robot with a sense of vision to
see the world. The data acquired from a camera is
originally expressed in the camera coordinate system. However, a robot manipulator only accepts the data
represented in the robot coordinate system. Under such
circumstances, robotic applications that employ cameras necessitate the requirement of converting the camera-acquired
data into the robot coordinate system. Since the robot coordinate system is often attached to the base of the robot, the
relationship between the camera's and the robot's frames
(commonly described by a homogeneous transformation
matrix) is composed of two parts: the transformation matrix
between the base frame and the end-effector frame of the robot manipulator; and the transformation matrix between the
end-effector frame and the camera frame. As the first transformation matrix can be attained from robot kinematics, the
problem of representing the camera-acquired data in the robot
coordinate system boils down to the estimation of the transformation matrix between the robot's end-effector frame and
the camera frame. This estimation problem is widely known
as the hand-eye calibration problem since the end-effector and
the camera are commonly regarded as the hand and the eye of
a robot, respectively. The hand-eye calibration problem plays
an important role in robotic applications as it enables the use
of cameras in such applications. This problem also emerges in
other applications such as visual servoing, 3D scanning systems, and other sensor calibrations.
A camera is essentially a projective device that maps
the 3D world into a 2D image [1]. This means only 2D data
can be obtained from a single camera and the depth information is lost. However, most robotic applications require
3D data and depth information to reconstruct 3D models
of the target objects for further manipulations. To recover
the depth information and obtain 3D data, stereo cameras
are frequently employed in robotic applications. A stereo
camera is a type of camera system that consists of two cameras to mimic human binocular vision and attain 3D data.
This particular feature of stereo cameras can be used as a
means of obtaining a solution for the hand-eye calibration

28	

problem. By utilizing the 3D points acquired from a stereo
camera, the hand-eye calibration problem can be essentially
formulated as an estimation problem of the transformation
matrix between two sets of 3D points (also known as the
point set matching problem). Through this point set matching formulation, the accuracy of the hand-eye calibration
can be improved as will be demonstrated in our experimental results.
In this paper, we review some of the classic methods proposed in the literature for the hand-eye calibration problem
as well as those for point set matching. We also explain how
the problem of hand-eye calibration with the employment
of stereo cameras can be formulated and solved as a point
set matching problem to gain better calibration accuracy. In
the end, we demonstrate that the methods based on point set
matching formulation outperform the conventional hand-eye
calibration algorithms.

Hand-Eye Calibration: AX = XB
formulation
A camera can be either attached to the robot end-effector (commonly referred to as eye-in-hand configuration) or placed as
an independent device away from the robot end-effector (commonly referred to as eye-to-hand configuration). The focus of
this article is on the eye-in-hand configuration.
AX = XB is the conventional formulation of the hand-eye
calibration problem with the eye-in-hand configuration. Fig.
1 presents a typical set-up to solve AX = XB, in that Ai (i = 1,2)
represents the homogeneous transformation matrix between
the base of the robot and the robot end-effector (e.g., a gripper)
at two different poses, Bi (i = 1,2) represents the homogeneous
transformation matrix between the camera's frame and the
world's frame affixed to a calibration tool (such as a checkerboard pattern) at these two poses, and X is the unknown
homogeneous transformation matrix between the end-effector
frame and the camera frame. In this context, one can express
these relations as:
	

A1XB1 A2 XB2  A21  A1X   XB2 B11	(1)

IEEE Instrumentation & Measurement Magazine	
1094-6969/21/$25.00©2021IEEE

May 2021
Instrumentation & Measurement Magazine 24-3

Table of Contents for the Digital Edition of Instrumentation & Measurement Magazine 24-3
