 Research Paper
 Open Access
 Published:
Camera calibration with coplanar conics: a unified explanation and ambiguity analysis
IPSJ Transactions on Computer Vision and Applications volume 10, Article number: 14 (2018)
Abstract
In this paper, we propose a twostep method to give a unified explanation of camera calibration with two coplanar conics. Various kinds of conicsbased patterns in which often two parameters are unknown have been studied in previous literatures. The key in such algorithms is to adopt different strategies to compute the worldtoimage projective transformation (also called 2D homography). In the first step of our method, we show that two unknown parameters can always be computed in general cases by utilizing the underlying constraints on all parameters through the projective transformation (mathematically called projective invariants). The accompanied ambiguity problem is that the solutions of the unknown parameters are multiple. In the second step, the four intersection points (real or complex) of two totally known conics are utilized to compute the homography. The ambiguity in this step arises from the point correspondence problem. This results in multiple possibilities of correspondences followed by the ambiguous homographies. After analyzing the reasons of the two kinds of ambiguities, we apply the Centre Circle constraint to completely remove them. Finally, the experiments are shown to validate the proposed technique.
Introduction
Conic as an important image primitive has been studied very well in the early 1990s [1–3]. Many major problems in computer vision, such as reconstruction, motion estimation, and pose determination, can be solved with two coplanar conics. For instance, Forsyth et al. use projective invariants of coplanar conic pairs to recognize curved planar objects [4]. Rothwell et al. use four intersection points of two conics to obtain homography which in fact results in the solution of relative motion and pose [5]. However, the above works all assume that the conics’ parameters are known which is not always available in practice. If the a priori knowledge about the object conics is scarce or imprecise, there is no method to handle these cases. More importantly, the methods of computing homography in [1, 5] are not suitable for calibration because the proposed posterior rules of removing the correspondence ambiguity are only effective when the intrinsic parameters of the camera are given.
In the 2000s, people explored various 2D conics patterns, such as concentric circles, confocal conics, coplanar circles, PrincipalAxes Aligned (PAA) conics, conics with a common axis of symmetry, enclosing ellipses, and degenerate conics with double complex contact to calibrate a camera [6–13]. The core step in such algorithms is to determine the homography between the model plane and its image. But it is hard to extend their methods as they only find one special pattern. Moreover, due to lack of the natural link between different conditions, there is no explanation for the confusing question why this pattern is valid while others are not.
In recent years, Zhao [14] proposed a novel method of the 2D Euclidean structure recovery from the conic feature correspondences. The conic features are transformed from the homogeneous coordinates to the lifted coordinates to represent the geometric objects without considering the conic dual to the absolute points. Wang et al. [15] propose an algorithm which is efficient and easy to estimate the pose of camera based on the conic correspondences from world plane to image plane system. The above method needs more than two conics to work. However, it is not common to see many conics in the nature scene.
In fact, the key of camera calibration based on two coplanar conics is how to obtain homography using partial information of the conics in the model plane. In other words, what is the minimal condition used to compute the homography for two coplanar conics? Based on this idea, we propose a twostep method to give a unified solution of conicsbased camera calibration. The proposed method not only easily explains all existing conics patterns, but also finds other possibilities. The first step is to get the unknown parameters by using geometric invariance. In this step, we discuss which form of projective invariants should be chosen to construct the equations on the unknown parameters followed by the algebraic and geometric explanation of ambiguous solutions. This ambiguity problem results in the uncertain parameters of the conics. The second step is to compute homography by utilizing four intersection points of two conics and their correspondences in the image, no matter whether they are real, complex, or partially complex. This correspondence ambiguity will lead to multiple possibilities of correspondences and the ambiguous homographies. To obtain the only correct solution, these two kinds of ambiguities must be removed. Thereby, owing to the Centre Circle constraint [16, 17] which provides the geometrical explanation of the worldtoimage homography in perspective transformation, we set a cuboid bound for the camera center [18] to judge whether the computed homography is correct. Finally, the experiments with real and simulated data verify the correctness of the proposed method.
The contributions of this paper could be concluded as follows:

A unified explanation to the problem of calibration with coplanar conics is proposed.

We show how to use projective invariants to compute unknown parameters of conics.

We analyze the correspondence ambiguity of four intersection points of conics.

We apply the Centre Circle constraint to remove the above two ambiguities.
The paper is organized as follows. Section 2 includes some preliminaries. Section 3 gives the problem statements. Section 4 describes different invariant forms of conics and the algebraic and geometric reasons of ambiguous solutions. Section 5 first classifies the reasons of the pointscorrespondence ambiguity into three aspects. Then, several constraints are adopted to seriously reduce the number of the ambiguous correspondences. The degenerate situations are also discussed in this section. Section 6 provides our experimental results with simulated and real data.
Preliminaries
Basic equations
Suppose the object plane is located on Z=0 plane in the world coordinate system. The worldtoimage homography matrix H is given by
where (u_{0},v_{0}) are the pixel coordinates of the principal point and f_{u} and f_{v} are the scale factors in the image’s u and v axes. r_{1}, r_{2} are the first two vectors of rotation matrix R. t denotes the translation vector between the world coordinate system and the camera coordinate system. K is called the intrinsic parameter matrix and (R, t) are called the extrinsic parameters.
Two constraints on K provided by the homography are:
where h_{1}, h_{2} are the first two columns of H. It is well known in [19–21] that given the correct correspondences of four coplanar points, the 2D homography H can be solved.
Projective invariants related to two coplanar conics
Invariant theory as an important technique has been investigated in previous literatures [2, 4, 22, 23]. There are mainly three forms of invariants used by researchers as follows:
1) The form of trace. For two coplanar conics C_{1} and C_{2} (here, determinants of the matrices must be 1 through normalization), two scalar invariants [4, 22] are given by
2) The form of the generalized eigenvalues. The generalized eigenvalues denoted by λ_{1},λ_{2},λ_{3} of two conics are projective invariants as a set, up to a scale factor. So two ratios from three generalized eigenvalues are the scalar invariants [4, 10]. One simple choice is:
3) The form of real invariants. Gros and Quan [23] utilized a generalization method to obtain two real invariants of two conics in which the matrices of the conics can be multiplied by a scalar.
Note that Eq. 4 is closely related to Eq. 6. Detailed distinction in application is shown in Section 4.1 and more specific discussion is stated in [23].
The Centre Circle constraint and the cuboid bound for the optical center
Gurdjos et al. [16] proposed a Centre Circle theory to explain the constraints imposed from a 2D homography on the optical center. When a planar figure is the central projection of another planar figure, the center of projection then lies on a spatial circle (called Centre Circle) which is intersected by a sphere S (called Centre Sphere) and a plane perpendicular to this intersection (called Centre Plane). The two equations determining the position of the Centre Circle can be deduced from Eqs. (2) and (3) respectively.
Owing to the Centre Circle constraint described above, Cai [18] builds a geometric criterion to judge the computed homography’s correctness, i.e., the Centre Circle must appear near the camera center. This criterion actually includes two basic physical meanings: the principal point should be near the image center and the focal length should be limited by the prior knowledge of the lens. Here, judging whether the camera centers lie in the geometric cuboid bound can be divided into two steps. First, judge whether the Centre Line intersected by the Centre Plane and the image plane passes through the given square region. Second, if the first condition suffices, judge whether the focal length varying with the changes in principal point is in the given range (see Fig. 1). Thus, for a cuboid bound, there are two controlling parameters: the one denoting the radius of the square controls the varying scope of the principal point and the other one denoting the height of the cuboid controls the varying range of the focal length.
Problem statements
Assume that the correspondences between two coplanar conics and their images are given. The basic equations about two conics and 2D homography are expressed by
where H denotes the homography. C_{1} and C_{2} are the matrices of the two conics in the model plane. Symbol \(\tilde {\mathbf {X}}\) denotes the image of the object X in the model plane. The symbol “ ≡” denotes the equivalence up to a scale factor.
Note that in the previous works [1, 5], C_{1} and C_{2} are totally known. Thus, there are 10 constraints on 8 unknowns in the process of computing homography and 10 constraints on 6 unknowns in pose estimation. By counting unknowns, if C_{1} and C_{2} are partially known, we can easily obtain the minimal condition of computing homography, i.e., C_{1} and C_{2} have two unknown parameters and the equations provide 10 constraints on 10 unknowns. Although the equations can be easily written, it is impossible to directly solve them due to the complicated nonlinearity.
Let us reconsider this problem by observing Fig. 2. After abandoning a global scale factor for simplicity, two general coplanar conics C_{1} and C_{2} can be denoted by six meaningful parameters: the eccentricities e_{1} and e_{2}, the displacements between the centers of two conics d_{x} and d_{y}, the relative rotation θ, and the relative scale factor k between two conics. In another way, C_{1} and C_{2} (that are two ellipses here for simplicity) with six parameters are expressed by the following equations:
According to the number of prior conditions imposed on two base conics, we classify two coplanar conics into six classes which are two general conics, two PrincipalAxis Parallel (PAP) conics (i.e., a=0), two Center Coaxial (CC) conics (i.e., f=0), two One PrincipalAxis Superposed (OPAS) conics (i.e., a=0 and f=0), two concentric conics (i.e., e=0 and f=0), and two PrincipalAxes Aligned (PAA) conics (i.e., a=0, e=0, and f=0). For most of the classes, we find that the quadrangle formed by four intersection points has different shapes and degrees of freedom (DOF) (see Fig. 3 and the proof is provided in Appendix). Such a classification not only gives us many geometrical explanations, but also provides some conveniences in further computing.
Based on the above analysis, we can propose an important argument about the number of the unknowns and given constraints as follows.
Proposition 1
A necessary and sufficient condition of obtaining the worldtoimage homography up to a transformation for two coplanar conics is that four effective constraints on six parameters are given.
Proof
(⇒) Assume four effective constraints on the above six parameters are given (the simplest form is to give exact values of four parameters; the term “effective” means that the four constraints must be independent and different from the two constraints provided by geometric invariance), we can use two scalar invariants of two conics to establish two equations (be nonlinear in most cases) of two unknown parameters. By solving them, we can get all parameters. Thus, the four intersection points can be obtained followed by the computation of homography. (⇐) Assuming the homography up to a similarity transformation is given, we can compute backprojected points in the model plane. Note that the quadrangle formed by four intersection points of C_{1} and C_{2} actually has four DOF. Since the four independent DOF imposed on the quadrangle are invariant under the similarity transformation, we can obtain four equations in six unknowns, i.e., four effective constraints on six parameters. □
To further describe these four effective constraints, we can divide them into two parts: conics class constraints and additional constraints. Conics class constraints refer to the constraints imposed on the conics class mentioned above. Additional constraints refer to the constraints a pair of coplanar conics still have after removing conics class constraints. Moreover, the number of additional constraints should be equal to the DOF of the quadrangle formed by four common points. Under the minimal conditions, the number of constraints should satisfy the following equation:
Proposition 1 and the above equation actually answer the confusing question why one specific conics pattern can be used to get the homography. For example, any two coplanar circles belong to OPAS conics which have two DOF. Thereby, we have two conics class constraints and two additional constraints (i.e., their eccentricities) which satisfy the above equation about the minimal condition of computing the homography. Two coplanar circles belonging to OPAS conics have two unknown parameters, the displacement d_{x} and the scale factor k. As these two unknowns can be computed by using projective invariants, all parameters and the trapezoid formed by four common points will be obtained. Adding the projective invariants, (9) for coplanar circles can be expressed by
Now, we discuss confocal conics [9] which belong to PAA conics. As PAA conics already has three class constraints and one DOF, it needs to be given one effective and additional constraint to compute the homography. Therefore, the additional constraint for confocal conics is that two conics are confocal. Ying and Zha [7] also propose a condition of computing homography for PAA conics: giving the eccentricity of any one conic. This constraint could also be considered as a specially additional constraint.
For concentric circles, the analysis is a little complicated. Because of belonging to PAA conics, concentric circles belong to the degenerate conics system with double contact which cannot be explained by (9). On the conics system with double contact, interested readers could see [13] for the detailed properties. Here, we only give the conclusion: any two degenerate conics with double contact except concentric circles will result in a oneparameter family of homographies with an uncertain rotation parameter. For concentric circles, although the computed homography also has one unknown rotation parameter, the image of circular points is not influenced as the unknown variable is just in a similarity transformation under which the true homography is transformed into the computed one.
Although this question has been explained in principle, there still exist two problems that need to be solved. One is how to choose the suitable form of invariants to solve two unknown parameters and remove the ambiguous solutions and the other is after calculating four intersection points, how to automatically determine their correspondences in the image to compute the correct homography, especially for the case with complex intersections. These two significant problems will be demonstrated in the next two sections.
The ambiguous solutions of unknown parameters
In this section, we focus on how to use the projective invariants to compute two unknown parameters of a pair of coplanar conics. To our knowledge, this idea is first implemented by Gurdjos [10] to compute two unknown parameters for two coplanar circles. Note that coplanar circles belong to OPAS conics, whose configuration is not general as their two parameters are equal to zero. Because of the complexity of two general conics, some new problems arise and Gurdjos’s method could not handle them. Thus, we revisit this problem which actually includes three subquestions as follows:

Geometric invariants involving a pair of conics have various forms. Which form should be chosen to establish the equations of the unknown parameters?

Why ambiguous solutions occur?

How to remove the wrong solutions?
Gros and Quan [23] reveal that all the mentioned invariant forms in Section 2.2 are related to each other. They also imply that the complexities of the four invariant forms are different. Owing to that work, the first two questions can be answered. The removing method based on the reasonability of the intrinsic parameters is depicted in the last subsection.
Computing unknown parameters using the real invariant form
For the general conics C_{1} and C_{2} in (8), it is impossible to obtain their cross ratios and the generalized eigenvalues by using Matlab’s symbolic computation. Moreover, the invariant form of the trace requires normalizing the determinants of two matrices, which will result in the fractional power functions needing to be further changed. Therefore, we finally choose the real invariants to compute the two unknown parameters. After some simple calculations, we have
For different combinations of unknowns, the polynomial functions have different degrees and solutions. For example, if a, b, e, f are given, the two polynomials functions are expressed by:
where \(\square \) denotes a real number. Substituting \((\square cd + \square d + \square c + \square)\) in (13) to (14), we have
It is obvious that (13) is a polynomial function with 2 degree and (15)’s degree is 3. Therefore, the maximum number of solutions of the unknowns c and d is 6. Do the similar manipulations and all results about degrees of the polynomials (DoP), and the maximum number of solutions (MNoS) are shown in Table 1.
Here, we do not pay more attention to obtain the exact number of solutions of the polynomial equations for each combination of two unknowns, because we experimentally find that Matlab can directly solve these polynomial equations in two variables, which allow us not to care about the solving details. Notably, when some parameters equal to zero (corresponding to the five nongeneral classes mentioned in Section 3), the above polynomial equations will be further simplified and the number of solutions becomes smaller.
Reasons of the invariant ambiguity
It is easy to verify that the projective invariance is only a necessary condition, not a sufficient condition for a pair of plane conics. In other words, different pairs of coplanar conics may have the same projective invariants that will result in the ambiguity of solving unknown parameters. Now, we will explain the underlying reasons of the ambiguous solutions in geometry and algebra.
Consider the projective property that two general conics can be transformed into two PAA conics under a projective transformation (the direct conclusion in [24, pp.157]). The equations of two PAA conics in the model plane up to a global scale factor are:
The generalized eigenvalues of two PAA conics are:
There is also a twofold ambiguity caused by the confusion of λ_{1} and λ_{2}. As pointed out in [23], two independent invariants, λ_{1}/λ_{3} and λ_{2}/λ_{3}, can be interpreted geometrically by the above two cross ratios. Therefore these two kinds of invariant forms arise from the uncertainty of correspondence in geometry.
The real invariants are related to the invariants of the generalized eigenvalues by the following equations ([23, pp.23]):
Note that in the above expressions, λ_{1} and λ_{2} can be interchanged, which implies the real invariants conceal the geometrical ambiguity. Moreover, the real invariants also bring some new ambiguous solutions because of the increase of polynomial complexity. Therefore, the ambiguity produced in solving real invariant equations can be seen as a purely algebraic problem.
Ambiguity removal
Because of the invariant ambiguity, the conics whose configuration is wrong could be projected to the image of the correct conics. Thus, it is impossible to remove them in theory. In other words, for one image of two partially unknown conics, some ambiguous solutions are projectively correct. However, the principle of removing the invariant ambiguity is the same as the principle of removing the correspondence ambiguity described in Section 2.3. Given a homography, the Centre Circle which the camera center should lie in can be used to evaluate the reasonability of the homography.
Computation of homography between totally known conics and their images
After obtaining all parameters of the pair of conics, the four intersection points of two conics on the model plane and their projections can be computed. But consequently, the problem of how to find the correct correspondences arises. For real intersection points, people may manually select the correct correspondences in the image as the popular calibration tool does [25]. For complex intersection points without physical position, it is impossible to directly find their correspondences in the image.
Rothwell et al. [5] point out that there are 24 ways to match 4 image points to 4 object points. For real intersection points and complex intersection points, the possibilities can be reduced from 24 to 4 and 8 respectively. Then, they used several posterior rules to eliminate this ambiguity. The similar argument is given in [1]. However, these methods are only effective for estimating the pose as the intrinsic parameters must be given. If we use N images of object to calibrate the camera, there will be 8^{N} combinations to compute the intrinsic parameters. It is impossible to verify them in terms of such posterior rules. Although [6, 10] propose some methods to distinguish the circular points from the four complex intersections of two circles, it is still far away from the final solution to the general case.
In this section, we pay more attention to analyze the reasons of correspondence ambiguity and present the possibilities of correspondences (PoC) in theory for different cases of conics. In this way, the number of PoC we must deal with decreases obviously. Then, based on the Centre Circle constraint (as described in Section 2.3) which provides the geometrical explanation of the worldtoimage homography in perspective transformation, we set a cuboid bound for the optical center to remove wrong correspondences, no matter whether the intersections are complex or real.
Possibilities of correspondences for different cases
Two coplanar conics with four distinguished intersection points could be divided into four cases: (1) quasienclosing conics with four complex intersections, (2) quasiseparate conics with four complex intersections, (3) intersecting conics with four real intersections, and (4) intersecting conics with two complex and two real intersections. Here, the terms “quasienclosing” and “quasiseparate” arising from the geometric properties of two conics will be explained later. We start our analysis from the cases with four complex intersections (including (1) and (2) cases above) and the other two cases will be demonstrated last. Denote these four intersection points by A, B, C, D and their correct image points by \(\tilde {A}\), \(\tilde {B}\), \(\tilde {C}\), \(\tilde {D}\). The PoC for four complex points mapping to image points are given by [5]
where either {A,B} or {C,D} is a pair of conjugate complex points. The above possibilities are obtained by only using the projective property that all complex conjugate pairs should be projected to conjugate pairs, i.e., the real lines which the complex points lie in still correspond to real lines. Given all possibilities of the numbers 1–8 from left to right for analyzing convenience, we find that the generation of these eight possibilities can be concluded by three different states as follows.

The correspondences of real lines. If the correspondences of two real lines in the quadrangle are clear, we can distinguish the PoC 1–4 from the PoC 5–8.

The adjacency of four complex intersections. Assuming that there indeed exists the adjacent relationship between the complex points no matter their order is clockwise or anticlockwise, the PoC 1,4,5,8 can be distinguished from the PoC 2,3,6,7 if this adjacency is known.

The order of points around the conic. The PoC 1,3,5,7 and 2,4,6,8 can be respectively treated as the clockwise order and the anticlockwise order.
Next, we will show how to utilize these three states to further reduce the number of PoC.
The adjacency of four complex points
Here, we will show that the adjacency of four complex intersection points can be found. Let us turn back to observe the geometric relationships of the separate conics and the enclosing conics both of which have four complex intersections (see Fig. 4). The quadrangle ABCD has two real lines AB and CD. Two complex lines AD and BC intersect at the real vertex Z of the selfpolar triangle XYZ. The complex lines BD and AC intersect at the real vertex Y. Assume the two conics are in visible halfspace segmented by the principal plane through the camera center parallel to the image plane (e.g., two ellipses). Thus, their four complex intersections also should be in visible halfspace and the possible points lying in invisible halfspace are X and Z.^{Footnote 1} Because the real point Y always lies inside one conic, the line AC and BD intersect at Y both in the world plane and in the image. This implies that A should be adjacent to D, not to C. Therefore, as the position of the vertex Y intersected by two complex conjugate lines is projectively invariant, we obtain the adjacency of four complex intersections. Consequently, the number of PoC for these two cases is reduced to 4.
The correspondences of two real lines
The correspondence problem of two real lines is previously investigated in [6, 10]. Specifically, they propose to divide two coplanar circles with four complex intersections into two cases: the separate case and the enclosing case. Note that the terms “separate” and “enclosing” are visually suitable for a pair of circles or ellipses since they are closed and finite in Euclidean geometry. However, to the visually infinite conics, some adjustment should be given. Thus, we slightly extend the previous works to define the term “quasiseparate” and “quasienclosing” to cover all conics cases with four complex.
Definition 1
For two coplanar conics C_{1} and C_{2} with four distinguished complex intersections, let the two real lines consisted of the two pairs of complex conjugate points be l_{1} and l_{2}. We name

(i)
C_{1} and C_{2} are quasiseparate if and only if they lie on the adjacent regions divided by l_{1} and l_{2}.

(ii)
C_{1} and C_{2} are quasienclosing if and only if they lie on the same region or the opposite regions divided by l_{1} and l_{2}.
Figure 5 gives some examples of the quasiseparate and the quasienclosing conics. For example, Fig. 5e depicts an ellipse and a half hyperbola. Their four imaginary intersections lie on two real lines, only one of which separates two conics. Thus, the correspondences of two real lines can be distinguished. The correct PoC only ranges from 1 to 4 or 5 to 8. Furthermore, as Wu proves for a pair of circles in [6], the above relationships are invariant under a quasiaffine transformation [26]. In other words, the real line between two conics can be distinguished for the quasiseparate case. Thus, the number of PoC for the quasiseparate conics is reduced to 2 (see Table 2).
After analyzing the cases with four complex intersections, it is not difficult for us to obtain the PoC for the other two cases. For the case with two real and two complex intersections, as both the adjacency of four points and the real line passing through two real points can be distinguished, the correspondences only have two possibilities (see Table 3). For the real case, it is easy to verify that the PoC B, C, D, A and D, A, B, C will result in the wrong correspondences of the selfpolar triangle XYZ followed by the wrong reprojection image (see Fig. 4) Therefore, the number of PoC for the real case is finally reduced to 2.
The order of four points and geometric symmetry
The notion of adjacency or order is not existing in projective geometry as any complex point does not have the physical position. Thus, the order of four complex points cannot be distinguished and the PoC that arose from the order must be considered in general cases. Tables 2 and 3 together show the PoC of four intersection points in four general conics cases. Now, we will reveal the relationship between PoC and the geometric symmetry of the quadrangle ABCD.
The influence of centrosymmetric to PoC is obvious. When the quadrangle ABCD is centrosymmetric, the PoC ABCD and CDAB will generate the same homographies no matter the intersections are complex or real. As a result, we can pick up 2 PoC out of 4 to compute the homography.
When the quadrangle is axisymmetric, such as the trapezoid formed by two OPAS conics (see Fig. 2), the situation is a little complicated. Axisymmetry which lets the PoC ABCD and BADC generate the same homography can only influence the three cases without four real intersections. For the fourrealintersecting case, there is still 2 PoC needed to be considered.
The rectangle formed by PAA conics is both centrosymmetric and axisymmetric. Thus, for two enclosing PAA conics, all 4 PoC will generate the same homographies. It is also worth noting that, for a geometrically symmetric quadrangle, the number of the PoC is reduced at the cost of the ambiguous extrinsic parameters.
Summary
After analyzing the three states which influence the PoC, the correspondence ambiguity of four intersections could be concluded in three points as follows:

The adjacency of four intersections for all cases of two coplanar conics can always be distinguished.

The correspondences of real lines can be distinguished for the quasiseparate case and the tworealintersecting case.

For the geometrically symmetric conics, the number of PoC often can be further reduced.
Compared to the previous works, the number of the PoC we must deal with in theory declines obviously. For the residual PoC, we adopt the two steps described in Section 2.3 to judge whether the Centre Circle corresponding to one computed homography intersects with the cuboid bound around the optical center. Our experiments will verify this method can remove the invariant ambiguity and the correspondence ambiguity simultaneously.
The pseudo code of whole calibration method for two coplanar conics is illustrated in Algorithm 1.
Degenerate situations
After discussing the general case of two conics, we now analyze in which conditions the degenerate cases will occur. There are four types of degenerate conics systems which are the simplecontact system, the threepoint contact system, the system with double contact, and the system with fourpoint contact [24, pp.158160]. As the degenerate situations rarely happen in practice, we only give the conclusion about whether the degenerate conics can be used to compute a homography.
For the simplecontact conics system (see Fig. 6a), two common tangents l_{1} and l_{2} are coincident. But four points intersected by two conics and the other common tangents l_{3} and l_{4} respectively can be always obtained. In the similar way, for the threepoint contact system (see Fig. 6b), two common points and two points on the tangent l_{4} can be obtained. Thus, for the above two degenerate cases, the complete homography can be computed.
For the system with double contact (see Fig. 6c), only three points including two common points and the point intersected by the tangent l_{3} and l_{4} can be distinguished from the object plane and its image. The undetermined thing is the correspondence of another general point P of one conic which has one DOF. As a result, a oneparameter family of homographies will map this degenerate pattern to its image (interested readers could see the detailed derivation in [13]). Similarly, for the system with fourpoint contact, any line t passing through the common point has one DOF. The points M and N as the intersections of conics and t have one common unknown. Subsequently, the other two points intersected by the common tangent and the tangents passing through M and N still have one unknown. As a result, an incomplete homography with one parameter can be obtained.
Experiments
Computer simulations
In our experiments, the simulated camera has the following intrinsic parameters: f_{u} = 1200, f_{v} = 1190, u_{0} = 500, and v_{0} = 400. The image resolution is 1024 × 768. The model plane is composed of a hyperbola and six ellipses as shown in Fig. 7, which actually covers all the cases in Table 4. C1, C2, C3, and C4 are four concentric ellipses centered at (0,0). The centers of C5, C6, and C7 are (80,0), (90,20), and (− 120,20) respectively. The radii of all conics are not listed here as the combination relationship between two of them is clear in Fig. 7. Project the pattern (approximately 500 points evenly extracted from each ellipse) to the simulated image planes at several different positions. The conic fitting algorithm presented in [27] is used to obtain the equations of the projected conics.
Performance w.r.t the invariant ambiguity We first pick up several pairs of conics to compute the unknown parameters. The results of the number of their solutions are shown in Table 5. Note that all the numbers of the solutions are equal or less than the MNoS shown in Table 1.
For the general conics pair {C_{1},C_{6}}, the solutions of {c,d} are shown in Table 6. After removing the last two complex solutions, the conics corresponding to the other four solutions of unknown parameters are depicted in Fig. 8. The first solution is correct. Both the third and the fourth solutions do not satisfy the perspective relationship. The second one which we have to deal with is meaningfully wrong. We set the region parameter of the principal point and the range parameter of the focal length 50 and 20% respectively. Using these two different solutions to compute the homographies, there is only one correct Centre Circle passing through the cuboid. Therefore, the correct unknowns, PoC, and homography can be obtained. Detailed results are shown in Fig. 9 and Table 7.
Performance w.r.t the correspondence ambiguity Assume the conics are totally known. For all the PoC shown in Tables 2 and 3, the number of the corresponding homographies satisfying the cuboid constraint is shown in Table 8.
The above results verify the correctness of Table 4 and our propositions. For instance, five pairs of conics {C_{2},C_{3}}, {C_{6},C_{7}}, {C_{1},C_{2}}, {C_{2},C_{6}}, and {C_{1},C_{7}} are quasienclosing and belong to PAA, OPAS, concentric, PAP, and general classes of conics respectively. In the original eight PoC of four complex points, only the projections of the conics from the residual four PoC in Table 7 are correct. The example of the pair of conics {C_{1},C_{7}} is shown in Fig. 10. Then, we apply the cuboid constraint to verify the residual four PoC. For the geometrically asymmetrical conics {C_{2},C_{6}} and {C_{1},C_{7}}, only one homography from the correct PoC satisfies the cuboid constraint. For the geometrically symmetrical conics {C_{2},C_{3}}, {C_{6},C_{7}}, and {C_{1},C_{2}}. The homographies obtained from two or four PoC of four complex intersections are same and their Centre Circles all pass through the cuboid at the cost of the ambiguous extrinsic parameters. Thus, we can choose any one PoC to compute the correct homography.
Performance w.r.t the noise level The Gaussian noise with mean 0 and standard deviation ranging from 0.3 to 1.5 is added to the projected image points. We perform 100 times independent experiments to solve camera calibration with three pairs of conics { C_{1},C_{6}}, { C_{2},C_{5}}, and { C_{3},C_{4}}. The averaged results are shown in Fig. 11. Note that the largest pair of conics { C_{2},C_{5}} has the least errors, which implies that the larger the size of conics, the better the antinoise performance.
Real images
Complete camera calibration The first real experiment is run to compute the complete intrinsic parameters using the proposed algorithm with the following setup:

Print a model plane with two general ellipses and a parabola as shown in Fig. 12. The left ellipse (denoted by E_{1}) can be obtained by rotating the ellipse x^{2}/16+y^{2}/9=1 30° anticlockwise. The equation of the right ellipse (denoted by E_{2}) is (x−8)^{2}/1+(y−6)^{2}/4=1. The equation of the parabola (denoted by P_{1}) is \(y5=\frac {3}{7}\left (x3\right)^{2}\).

Use a CCD camera (Point Grey FL208S2MC) with 4mm lens (uTron FV0420) to take 12 photos of the model plane. The image resolution is 1024×768.

Use Canny operator to detect the edge and Fitzgibbon’s conic fitting method [27] to obtain the conics.
First, we compute the unknown translation parameters d_{x} and d_{y} for two conics pairs { E_{1},E_{2}} and { E_{1},P_{1}} respectively. Here, four priori conditions that include the eccentricities of two conics, the relative rotation, and the relative scale factor between two conics are given. The two pairs of real solutions for two conics pairs are listed in Table 9. Note that the first group of solution is correct while the second one is wrong. The mean value of every solution is very close to its theoretical value.
We then applied the cuboid constraint with the size parameters 50 and 20% to remove the two kinds of ambiguities in the process of computing homography. For each solution of the unknown parameters of the conics pair { E_{1},E_{2}}, the computed Centre Lines from four PoC are shown in Fig. 13. Note that for each case, only two Centre Lines pass through the given square region of the principal point. Table 10 further shows the ranges of the focal length computed from the above two PoC. Finally, only one PoC obtained from the correct parameters satisfies the cuboid constraint.
After obtaining the homography for each image, the intrinsic parameters are computed. To verify the accuracy of our method, we also use the Camera Calibration Toolbox [25] to calibrate the same camera. In addition, to robustly extract the chessboard corner, we refer to the literature [28] which shows the better performance under severe distortion or illumination. The results are shown in Table 11. The metric rectification result of the depicted image is shown in Fig. 12b.
Focal length calibration and AR The second experiment focuses on a specific AR application in practice.
There is a round box on a table. Put a printed ellipse on anyplace of this table (see Fig. 14). Thus, an ellipse and a coplanar circle form PAP conics (shown in Fig. 3) on the table plane. The semimajor axis and the semiminor axis of the ellipse is 4 cm and 2 cm, respectively. The radius of the bottom circle of the box is 3.25 cm. With these four known parameters, we can compute the unknown translations between the centers of two conics, followed by the homography between the table plane and one image. Three images captured by an iPhone 8 are shown in Fig. 14. Note that the ellipse positions in three images are different. We still use the above method to extract the conics (colored in pink and cyan respectively) in the image. Assuming the principal point is given on the image center, here we only calibrate the focal length using the computed homography from each image individually. This is because the slight offset of the principal point to image center has very little impact on the results of focal length calibration and AR effects. The calibration results from three images and the Camera Calibration Toolbox are shown in Table 12. After obtaining the intrinsic parameters, the pose of every ellipse can be estimated and a virtual teapot with its local coordinate system are drawn. Constructing a moving AR marker with an ellipse shows the practical use of the proposed method, even if under the case that the camera has a zoom lens.
Conclusions
In this paper, a twostep method is developed to give a unified explanation of camera calibration based on coplanar conics. We also display a particular description of the invariant and correspondence ambiguity problems existing in the proposed method. Based on the analysis of the solutions of residual ambiguity in various cases, the wrong homographies can be removed by setting a cuboid bound for the camera center for each image and we can directly obtain the unambiguous calibration results. Furthermore, as the natural link between various conics patterns has been established, readers could get a full comprehension of the conicsbased calibration.
Appendix: Proof of Fig. 3
In this appendix, we use analytic geometry knowledge to verify the shape of the quadrangle formed by four intersection points of two conics for every case shown in Fig. 3.
See Fig. 15. C_{1} and C_{2} are two PAP conics whose axes are parallel. A, B, C, and D are four intersection points. Now, we will prove D is on the circumcircle defined by the points A, B, and C.
The isogonal conjugate of the circumconic C_{1} with respect to the triangle ABC is the line l_{1}. Meanwhile, the isogonal conjugate of C_{2} with respect to the triangle ABC is the line l_{2}. For a given triangle, the direction of the isogonal conjugate line of a circumconic is only related to the direction of the axes of the circumconic. Therefore, l_{1} and l_{2} are parallel. The isogonal conjugate of the point D as a common point of C_{1} and C_{2} will be the point at infinity intersected by l_{1} and l_{2}. Moreover, as the isogonal conjugate of the circumcircle with respect to the triangle ABC is the line at infinity, D must be on the circumcircle.
For OPAS conics, as both of them are axisymmetric, the four intersections must be axisymmetric. Thus, the quadrangle formed by these four intersections can only be a trapezoid. In the similar way, two concentric conics are centrosymmetric, so are their intersections. Thus, the quadrangle can only be a parallelogram. For PAA conics, the quadrangle should be axisymmetric and centrosymmetric simultaneously. Thus, it is a rectangle.
Notes
 1.
This is an experimental conclusion and cannot be proved in strict mathematics
References
 1
Ma S (1993) Conicsbased stereo, motion estimation, and pose determination. Int J Comput Vis (IJCV) 10(1):7–25.
 2
Mundy JL, Zisserman A (1992) Geometric invariance in computer vision. MIT Press, Europe.
 3
Quan L (1996) Conic reconstruction and correspondence from two views. IEEE Trans Pattern Anal Mach Intell (PAMI) 18(2):151–160.
 4
Forsyth D, Mundy JL, Zisserman A, Coelho C, Heller A, Rothwell C (1991) Invariant descriptors for 3D object recognition and pose. IEEE Trans Pattern Anal Mach Intell (PAMI) 13(10):971–991.
 5
Rothwell C, Zisserman A, Marinos C, Forsyth DA, Mundy JL (1992) Relative motion and pose from arbitrary plane curves. Image Vis Comput (IVC) 10(4):250–262.
 6
Wu Y, Li X, Wu F, Hu Z (2006) Coplanar circles, quasiaffine invariance and calibration. Image Vis Comput (IVC) 24(4):319–326.
 7
Ying X, Zha H (2007) Camera calibration using principalaxes aligned conics. Asisan Conference on Computer Vision (ACCV).
 8
Kim JS, Gurdjos P, Kweon IS (2005) Geometric and algebraic constraints of projected concentric circles and their applications to camera calibration. IEEE Trans Pattern Anal Mach Intell (PAMI) 27(4):637–642.
 9
Gurdjos P, Kim JS, Kweon IS (2006) Euclidean structure from confocal conics: theory and application to camera calibration. in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, New York.
 10
Gurdjos P, Sturm P, Wu Y (2006) Euclidean structure from n≥2 parallel circles: theory and algorithms. European Conference on Computer Vision (ECCV). Springer, Graz.
 11
Zhao Z (2011) Conics with a common axis of symmetry: properties and applications to camera calibration:1079–1084. TwentySecond International Joint Conference on Artificial Intelligence (IJCAI). IJCAI/AAAI, Barcelona, Catalonia.
 12
Cai S, Zhao Z, Huang L, Liu Y (2013) Camera calibration with enclosing ellipses by an extended application of generalized eigenvalue decomposition. Mach Vis Appl 24(3):513–520.
 13
Cai S, Wang C, Yan Y, Liu Y (2013) Analysis of the pencil of conics with double complex contact and its application to camera calibration. J Shanghai Jiaotong Univ (Science) 18(1):1–6.
 14
Zhao Z (2014) Euclidean structure from conic feature correspondences In: International Conference on Pattern Recognition, 4010–4014.. IEEE, Stockholm.
 15
Wang W, Zhang X, Qi Y, Cheng W (2018) An algorithm of pose estimation based on conic correspondences In: IEEE International Conference on Robotics and Biomimetics, 314–319.. IEEE, Macau.
 16
Gurdjos P, Crouzil A, Payrissat R (2002) Another way of looking at planebased calibration: the centre circle constraint. Eur Conf Comput Vis (ECCV). Springer, Copenhagen.pp 252–266.
 17
Poncelet JV (1862) Applications D¡ Analyse et de G¡äeometrie  Trait¡äe des Propri¡äet¡äes Projectives des Figures. Tome I. Imprimerie de MalletBachelier, Paris.
 18
Cai S, Huang L, Liu Y (2012) Automatically obtaining the correspondences of 4 coplanar points for an uncalibrated camera. Appl Opt 51(22):5369–5376.
 19
Hartley R, Zisserman A (2003) Multiple view geometry in computer vision, Second Ed. Cambridge Univ. Press, New York.
 20
Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell (PAMI) 22(11):1330–1334.
 21
Sturm PF, Maybank SJ (1999) On planebased camera calibration: a general algorithm, singularities, applications In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR).. IEEE, Fort Collins.
 22
Forsyth D, Mundy JL, Zisserman A, Brown CM (1990) Invariance  a new framework for vision. International Conference on Computer Vision (ICCV). IEEE, Osaka.
 23
Gros P, Quan L (1992) Technique report: projective invariants for vision.
 24
Semple J, Kneebone G (1952) Algebraic projective geometry. Oxford Univ. Press, Oxford.
 25
Bouguet J (2011) Camera Calibration Toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/index.html.
 26
Hartley R (1998) Chirality. Int J Comput Vis (IJCV) 26(1):41–61.
 27
Fitzgibbon AW, Pilu M, Fisher RB (1999) Direct leastsquares fitting of ellipses. IEEE Trans Pattern Anal Mach Intell (PAMI) 21(5):476–480.
 28
Placht S, Fürsattel P, Mengue EA, Hofmann H, Schaller C, Balda M, Angelopoulou E (2014) Rochade: robust checkerboard advanced detection for camera calibration In: European Conference on Computer Vision, 766–779.. Springer, Cham.
Funding
This work was partially supported by NSFC Program 61472075 and 61703092.
Author information
Affiliations
Contributions
SC proposed the ideas, deduced the formula, and was a major contributor in writing the manuscript. ZW did the experiment and complemented the manuscript. Both authors read and approved the final manuscript.
Corresponding author
Correspondence to Shen Cai.
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Received
Accepted
Published
DOI
Keywords
 Homography
 Conics
 Projective invariants
 Ambiguity problems
 Centre Circle constraint