# Image Retrieval with Relational Semantic Indexing Color and Gray Images Abstract-Due to the development of digital technology large number of image is available in web and personal database and it take more time to classify and organize them. In AIA assigns label to image content with this image is automatically classified and desired image can be retrieved. Image retrieval is the one of the growing research area. To retrieve image Text and content based methods used. In recent research focus on annotation based retrieval. Image annotation represents assigning keywords to image based on its contents and it use machine learning techniques. Using image content with more relevant keywords leads fast indexing and retrieval of image from large collection of image database. Many techniques have been proposed for the last decades and it gives some improvement in retrieval performance. In this proposed work # Introduction utomatic Image retrieval is the one of the growing research area. The aim of the image retrieval is to search or find similar images from databases. Large number of image is available in web and personal database due to the development of digital technology. Advances in image acquisition and storage technology have led to tremendous growth in very large and detailed image databases. These images, if analyzed, can reveal useful information to the human users. Image mining deals with the extraction of implicit knowledge, image data relationship, or other patterns not explicitly stored in the images. Image mining handles with the hidden knowledge extraction, image data association and additional patterns which are not clearly accumulated in the images. The traditional image retrieval approaches separated into two categories text based retrieval and content based retrieval. Text based image retrieval similar to document retrieval. It needs manual annotation. More expensive and time consuming. To overcome this content based image retrieval technique was used. It takes image low level features such as color, texture and shape. Retrieval performance is increased but accuracy is yet refined. Everything not derived from image features. There is a semantic gap. Smeulders define the semantic gap as "lack of coincidence between the information that one can extract from the visual data and the interpretation that the same data have for a user in a given situation" [18]. The aim of annotation based retrieval system is to bridging the semantic gap between image low level features and high level semantics. Semantic description of image content is associated with image. It will enable the user to retrieve the image with keywords or queries. Nowadays annotation based retrieval is an interesting research area. It can be defined as Assigning keywords to the image based on its contents. Keyword describes the visual content of the image. Using image content with more relevant keywords leads to fast indexing. In the proposed image retrieval system used to identify and retrieve the image. In training images previsiously annotated with keywords and with usage of machine learning techniques test image is automatically annotated with keywords. Image annotation models can be classified into two types. Namely Probabilistic model Discriminative model. # II. # Literature Survey Much research effort has been focused on content based and text based retrieval during recent years, resulting in remarkable achievements [1], [2] Among others, automatic image annotation technology, which associates images with labels or tags, has received much research interest [3]. This technique enables conversion of image retrieval into text matching Duygulu [19] proposed the translation model to treat AIA as a process of translation from a set of blob tokens, obtained by clustering image regions, to a set of keywords. Image annotation thus brings several benefits in image retrieval, such as high efficiency and accuracy in the field of Research area such as multimedia and computer vision, pattern recognition. Many research has been done in these area. Yang's proposed learning paradigm [1] and Carneiro a generative paradigm [2]. Latent Dirichlet Allocation (CorrLDA) [3] considers associations through a latent topic space in a generatively learned model. Carneiro [20] proposed supervised multi-class labeling (SML), which utilizes optimal principle of minimum probability of error and treats annotation as a multi-class classification problem. A semi automatic framework for image annotation using Locality Sensitive Hashing improved the searching combination of keywords. With the application of LSH, though search effectiveness was improved, search time increased with different combination of image annotation [6]. Texture based approaches are efficient in dealing with complex background with dissimilar textual structure to the text regions. But the computational complexity restricts its applications in large databases. Han et al [6] proposed a system prototype for multimedia data mining capable of performing data summarization, comparison, classification, association mining and clustering. Metzler proposed a graph representation of joint queries [6], and cross-language LSI [7], offer means for linking the word-image occurrences, but still do not perform as well as the non-parametric mode. Even though improved results have been reported by annotation based retrieval techniques it lacks a comparison with simple baseline measures across diverse image datasets. In the absence of such a comparison, it is hard to understand the gains and justify the need for complex models and training processes as required by most of the current annotation methods. Alnihoud has proposed a novel approach for CBIR using Fuzzy Color Histogram (FCH) and subtractive fuzzy clustering algorithm Similarity mining into a joint framework [7]. Yi Yang Propose a new inductive algorithm for image annotation by integrating label correlation mining. Recent researchers have shown that the designing of manual tags are often insignificant and not reliable. Moreover, as many users select the most general and ambiguous tags for minimizing their involvement while selected more appropriate words, tags are considered to be noisy. To provide solution to this problem, tag completion in [12] filled the missing tags in an automatic manner and also corrected noisy tags for the images provided as input resulting in significant improvement but tag completion based on compressed sensing and matrix completion remain unaddressed. A novel method was presented for graph indexing for better image retrieval using hyper graph which improved the scalability of images being retrieved. A structural learning algorithm was instantiated for implementing large-scale image classification. But the results, accuracy obtained for large scale classification was not precise. [14] The rest of this paper is organized as follows. In Section 3, introduce image Pre-processing techniques section describes edge detections, Section 4,5 feature extraction and RSI based annotation algorithm associated with linear quad tree construction is presented for image annotation framework. Section 5, 6 provides an experimental evaluation with the detailed section 7 gives Result Analysis finally section 8 concludes with remarks. # III. Framework for Proposed System The Four main tasks of the proposed system are: 1. Texture Feature Extraction (Haralick and Tamura). 2. Shape Feature Extraction (moment invariant) 3. Translating keywords and Classification. 4. Using Euclidean Distance Method Similarity of the query image and database images are retrieved. # a) Outline and Pre-processing The sizes of the Input images are verified by its pixel range as 256*256. Image with noise leads to low accuracy of image retrieval. So smoothing filter [16]technique used for noise removal. By using edge detection techniques edge of the images is identified image is spilited into 4*4 blocks. Features of all blocks can be derived. Semantic tag is automatically assigned by comparing training image feature vector. System works gray and binary image. 4 Haralick and 2 Tamura texture features and 7 Hu's moment invariant features extracted. The extracted features from the query image and database images are compared using the Euclidean distance. For binary image segmented using threshold mean segmentation. The converted binary image contains all the essential information about the location and outline of the objects for segmenting the images with minimal time. The purpose of thresholding is to extract those pixels from some image which represent an object (either text or other line image data such as graphs, maps). Though the information is binary the pixels represent a range of intensities. Thus the objective of binarization is to mark pixels that belong to true foreground regions with a single intensity and background regions with different intensities. With the threshold mean value, the objects position and shape is identified. Madirakshi Das proposed segmentation approach to foreground segment detection. It is based on elimination of background. This is accomplished by combining a color-based background detection step with refinement of the segmentation using edge information. [18].In # Feature Extraction Like color, the texture is a powerful feature for image search and retrieval applications. Texture descriptor provides measures of the properties such as smoothness, coarseness, and regularity. V. # Classification and Keyword Translation The extracted features are provided as the input to the classification process in the image annotation framework. With the features being extracted, the RSI technique uses the minimum distance classification that contains different types of class centres such as, ?? ?? = 1,2 ? ??. Relational indexing handles different combination of keywords with (x, y) pixel points for effectual quality of search. The combinatorial interpretation in RSI technique chooses 'n' semantic keywords and result with potentially high search result on larger database. This proposed model trained with small set of manually annotated images. This training set class used to assign a keyword to one or more classes. Using RSI technique the unknown image annotated by word translation model. For efficient image indexing and retrieval proposed system used RSI with quad tree. The main goal of Relational Semantic Indexing technique in the image annotation framework is to reduce the search time for different combinations of semantic keyword. The RSI technique also works to improve the quality of search by developing an entity relationship model. # VI. # Evaluation and Measurement Proposed system use translation model to translate semantic keyword from training image to test image. The retrieval performance is measured with precision and recall. It is the ratio of the number of relevant images retrieved to the total number of irrelevant and relevant images retrieved. Recall is the total number of relevant images that exist. Proposed system uses the mean of precisions and recalls for a given query set. # No. relevant images retrieved Total No images retrieved Recall = No. relevant images retrieved Total No. relevant images in the collection In otherwise precision evaluates the proportion of relevant images in the retrieval results, while recall calculates that of the relevant images in all of the relevant images contained in the database. # VII. Experiment and Result Analysis Image Annotation Framework based on the Relational Semantic Indexing (RSI) technique performs the experimental work using MATLAB coding. RSI indexing is experimented using INIRA holidays dataset.RSI is compared against the Search based retrieval Inductive Algorithm for Image Annotation (IAIA) and Semi automatic framework with Locality Sensitive Hashing (LSH). In order to evaluate the proposed system INIRA Holidays database is used. In the database 800 images were taken for experiment. The data set contains 10 classes. 500 images for training and 300 images for testing are used for the experimental study. The Image Annotation framework based on the Relational Semantic Index (RSI) technique is compared against the existing inductive algorithm for image annotation [1] and Locality Sensitive Hashing [2] and Learning based retrieval algorithm. The simulation results using MATLAB are compared and analyzed Table1, 2 provides a few statistics on precision and recall with respect to 800 test images for experimental purpose. Consequently, the results obtained using proposed RSI is compared with the existing IAIA [1] and LSH [2].Figure 3,4 provides comparison chart for precision and recall respectively. It is observed that the search time using RSI is better as an entity relationship is developed to identify the relationships with different combination of semantic keywords by minimizing the search time. It is also observed that the performance of search time are affected using the existing IAIA [1] and LSH [2]. This improves the search time using RSI by 6 -23 % when compared to IAIA. In addition, using RSI, the keyword retrieves all combination of objects from classified class and reduces the search time using the relational semantic indexing by 16 -34 % when compared to LSH. The simulation shows that the recall rate is higher using the proposed RSI with Quad tree than when compared to the existing inductive algorithm for image annotation (IAIA) [1] and Locality Sensitive Hashing (LSH) This is because of the application of relational database for indexing different combination of semantic keywords which in turn increases the recall ratio in an optimal manner based on the number of test images by 2 -8 % when compared to IAIA [13]. In a similar manner, with the application of linear quad tree, the two dimensional space is efficiently partitioned into four quadrant form that increases the recall ratio by 7 -13 % when compared to LSH. But comparatively, the precision rate is higher using the RSI technique. This is because with the application of semantic keyword addition on image annotation framework therefore increases the precision rate by 7 -10% when compared to IAIA. RSI technique fetches the user result with higher precision rate by 14 -20 % when compared to LSH [13]. # VIII. # Conclusion With the increasing need of multimedia applications over the Internet, the importance of image mining and image retrieval has also increased [17]. Currently, many new schemes are proposed in the field of Image Mining and Retrieval. The application of relational semantic indexing dramatically encapsulates the function for varying combination of keywords. In these experiments, RSI with quad tree framework has several advantages, such as efficiency and accuracy. 1![Fig. 1: input image after segmentation](image-2.png "Fig. 1 :") 2![Figure 2: No. of test images versus precision](image-3.png "Figure 2 :") 1No. of testRecall (%)Images (MB)RSIIAIALSH500675660600685861700675961800686262900696363100070646511006864661200566567Query Image/Keyword-sea * Content-based multimedia information retrieval: State of the art and challenges MSLew NSebe CDjeraba RJain ACM Trans. Multimedia Comput., Commun. Appl 2 1 Feb. 2006 * Region-based image annotation using asymmetrical support vector machine-based multiple-instance learning CYang MDong JHua Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition the IEEE International Conference on Computer Vision and Pattern Recognition 2006 * Vasconcelos, N Supervised learning of semantic classes for image annotation and retrieval GCarneiro ABChan PJMoreno IEEE Transactions on Pattern Analysis and Machine Intelligence 2007 * The Randomized Approximating Graph Algorithm for Image Annotation Refinement Problem YohanJin KibumJin BLatifurkhan Prabhakaran IEEE Computer Society on Computer Vision and Pattern Recognition Jan 2008 * Image annotation within the context of personal photo collections using hierarchical event and scene models LCao JLuo HSKautz TSHuang IEEE Trans. Multimedia 11 2 Feb. 2009 * Perceptual segmentation: Combining image segmentation with object tagging RBergman HNachlieli IEEE Trans. Image Process 20 6 Jun. 2011 * Content-Based Image Retrieval System Based on Self Organizing Map, Fuzzy Color Histogram and Subtractive Fuzzy Clustering JAlnihoud the International Arab Journal of Information Technology 9 5 2012 * Web and Personal Image Annotation by Mining Label Correlation With Relaxed Visual Graph Embedding YiYang FeiWu FeipingNie YuetingHeng Tao Shen AlexanderGZhuang Hauptmann IEEE Transactions On Image Processing 21 3 March * Label-specific training set construction from web resource for image annotation JinhuiTang ChunxiaShuicheng Yan Zhao RameshTat-Seng Chua Jain Signal Processing 2012 Elsevier Journal * Image Retrieval Using Combination of Texture and Shape Features NPuviarasan .RDr ABhavani Vasanthi International Journal of Advanced Research in Computer and Communication Engineering 3 March 2014 * Training inter-related classifiers for automatic image classification and annotation PeixiangDong Kuizhimei Nanningzheng JianpingHaolei Fan Pattern Recognition Oct 2013 Elsevier * A Survey on Automatic Annotation and Annotation Based Image Retrieval HimaliChaudhari DDProf Patil International Journal of Computer Science and Information Technologies 5 2 2014 * A New Content Based Image Retrieval System by HOG of Wavelet Sub Bands AnnaDr S. VinodSaro Vijendran Kumar International Journal of Signal Processing 8 4 2015 Image Processing and Pattern Recognition * Image Processing Analysis and machine Vision by Milan Sonka, Vaclav Hlavac and Roger Boyer Overview of Image Retrieval Techniques, International Conference on Recent Trends in Computing and Communication 17 ICRTCC 2015 * Cochin College of Engineering & Technology 4 1 June 2015 * Content-based image retrieval at the end of the early years AW MSmeulders MWorring AGupta RJain IEEE Trans. Pattern Anal. Mach. Intell 22 12 2000 * Object recognition as machine translation: learning a lexicon for a fixed image vocabulary PDuygulu KBarnard NFreitas DForsyth Proc. ECCV ECCV 2002 * Supervised learning of semantic classes for image annotation and retrieval GCarneiro AChan PMoreno NVasconcelos IEEE Trans. PAMI 29 3 2007 * Content Based Image Retrieval using Color and Texture Manimala Singha* KHemachandran Dept Silchar India of Computer Science, Assam University * Statistical Texture Measures Computed from Gray Level Coocurrence Matrices Fritz Albregtsen * Textural Features for Image Classification RMHaralick KShanmugam IDinstein IEEE Trans. on Systems, Man and Cybernetics 3 1973