Integration Framework

A 2-level framework is employed for integrating lower-level and higher-level vision features. Given the isotropic feature vectors ${\bf X}_{\cal S}$ and ${\bf X}_{\cal H}$ and anisotropic feature ${\bf X}_{\cal T}$ extracted from a query image, and ${\bf X}_{{\cal S}_j}$, ${\bf X}_{{\cal H}_j}$ and ${\bf X}_{{\cal T}_j}$ extracted from the $j^{th}$ image in the database, the first level of the framework maps the feature vectors to a discriminant value within each of the 3 categories, structure, histogram and texture. The respective mappings $\Phi_{\cal S}$: ${\Re}^{N_{\cal S}} \to {\Re}$, $\Phi_{\cal H}$: ${\Re}^{N_{\cal H}} \to {\Re}$ and $\Phi_{\cal T}$: ${\Re}^{N_{\cal T}} \to {\Re}$, where $N_{\cal S} = 3$, $N_{\cal H} = 512$ and $N_{\cal T} = 48$, are selected as $\ell_2$ norms: $\Phi_{\cal S}({\bf X}_{{\cal S}_j}, {\bf X}_{\cal S}) = \vert\vert{\bf X}_{{\cal S}_j} - {\bf X}_{\cal S}\vert\vert$, $\Phi_{\cal H}({\bf X}_{{\cal H}_j}, {\bf X}_{\cal H}) = \vert\vert{\bf X}_{{\cal H}_j} - {\bf X}_{\cal H}\vert\vert$ and $\Phi_{\cal T}({\bf X}_{{\cal T}_j}, {\bf X}_{\cal T}) = \vert\vert{\bf X}_{{\cal T}_j} - {\bf X}_{\cal T}\vert\vert$.

At the second level a supra discriminant is generated by utilizing the mapping $\Psi_{\cal SHT}$: ${\Re}^3 \times {\Re}^3 \to {\Re}$ that is given as:

$$\begin{array}{ll} \Psi_{\cal SHT}({\bf X}_{{\cal S}_j},{\bf X}_{{... ...{{\cal T}_j},{\bf X}_{\cal S}, {\bf X}_{\cal H}, {\bf X}_{\cal T})& \end{array}$$ (16)

where ${\cal W}^t = (w_1, w_2, w_3)^t$ is a weight vector such that $\sum_{i=1}^3 w_i = 1$, $\Psi_{\cal SHT} \in [0,1]$ and $\Phi_{\cal SHT}:\Re^3 \times \Re^3 \to \Re^3$, such that $\Phi_{\cal SHT} \in [0,1] \times [0,1] \times [0,1]$, is given as:

$$\begin{array}{ll} \Phi_{\cal SHT}({\bf X}_{{\cal S}_j},{\bf X}_{{... ...),{\hat{\Phi}}_{\cal T}({\bf X}_{{\cal T}_j},{\bf X}_{\cal T}))^t & \end{array}$$ (17)

where

$${\hat{\Phi}}_{\cal S}({\bf X}_{{\cal S}_j},{\bf X}_{\cal S}) = \f... ...{\bf X}_{\cal S})}{\max_j \Phi_{\cal S}({\bf X}_{{\cal S}_j},{\bf X}_{\cal S})}$$

$${\hat{\Phi}}_{\cal H}({\bf X}_{{\cal H}_j},{\bf X}_{\cal H}) = \f... ...{\bf X}_{\cal H})}{\max_j \Phi_{\cal H}({\bf X}_{{\cal H}_j},{\bf X}_{\cal H})}$$

$${\hat{\Phi}}_{\cal T}({\bf X}_{{\cal T}_j},{\bf X}_{\cal T}) = \f... ...{\bf X}_{\cal T})}{\max_j \Phi_{\cal T}({\bf X}_{{\cal T}_j},{\bf X}_{\cal T})}$$ (18)

The above normalizations ensure that ${\hat{\Phi}}_{\cal S} \in [0,1]$, ${\hat{\Phi}}_{\cal H} \in [0,1]$ and ${\hat{\Phi}}_{\cal T} \in [0,1]$ for properly constructing $\Phi_{\cal SHT}$. The index $\hat{i}$ of the image most similar to a given query image is then given as:

$$\hat{i} = \arg \min_i \: \Psi_{\cal SHT}({\bf X}_{{\cal S}_i... ...al T}_i},{\bf X}_{\cal S}, {\bf X}_{\cal H}, {\bf X}_{\cal T})$$ (19)

The next most similar image is retrieved by removing the $i^{th}$ image from the database and utilizing equation 19 again. The process is repeated for retrieving any number of images most similar to a given query image.

Figure 4: Retrieval by image query (Databases #1 & #2): Flower, leaves and grass.

The above integration framework has the following advantages over a simple concatenation of vectors ${\bf X}_{{\cal S}_j}$, ${\bf X}_{{\cal H}_j}$ and ${\bf X}_{{\cal T}_j}$. First, the different lengths of these three vectors preclude the proper construction of a concatenated vector that is equally sensitive to all of its components. The 3-dimensional vector output by $\Phi_{\cal SHT}$ is equally sensitive to all of its three 1-dimensional components. Second, the size of the corresponding weight vector for the concatenated vector will be large, making the selection of proper weights difficult and unfeasible. Third, in our proposed integration, weights are assigned at the module level, i.e., structure, histogram and texture, whereas weights in a concatenated vector are assigned at the vector component level without particular regard to the modular structure of the system. The weight vector plays an important role in controlling the content of images retrieved. For a given image query, different weights can be assigned to structure, histogram and texture according to user specification to control the images retrieved.