A State-of-the art Review: A survey on Multimedia Tagging Techniques

Authors

  • Kirubai Dhanaraj  Research Department of Computer Science Bishop Heber College, Tiruchirappalli, India
  • Rajkumar Kannan  Research Department of Computer Science Bishop Heber College, Tiruchirappalli, India

Keywords:

Image annotation, video annotation, automatic approach, social annotation, collaborative annotation, crowdsourcing

Abstract

Social media prevails in every walk of our life. Multimedia shared through social networks has tremendously increased the need for efficient retrieval methods and expects more accuracy in terms of annotating an existing image or video. Retrieval methods and annotation techniques are two sides in the development of an efficient multimedia retrieval system. Annotating the image and video is a challenging task. Collaborative user annotations can be incorporated into multimedia to increase the efficiency and accuracy in the retrieval methods. Collaborative user annotations are useful for two reasons: (i) Multi-label annotation for a multimedia is possible with less time-consuming even for large-scale image corpus (ii) Correlation between images and videos build a multi-class label propagation without much human effort and in reduced cost. There are many areas of research the collaborative annotations are incorporated with small modification in the existing machine learning algorithms. This survey paper presents the state-of-the-art annotation techniques for multimedia in the new era.

References

  1. L. S. Kennedy, S.-F. Chang, I. V. Kozintsev, To search or to label? Predicting the performance of search-based automatic image classifiers, in:Proc. of ACM MIR, Santa Barbara, CA, USA, 2006, pp. 249–258.
  2. B. Sigurbj¨ornsson, R. van Zwol, Flickr tag recommendation based on collective knowledge, in: Proc. Of WWW, Beijing, China, 2008, pp. 327–336.
  3. X. Li, T. Uricchio, L. Ballan, M. Bertini, C. G. M. Snoek, A. Del Bimbo, Socializing the semantic gap: A comparative survey on image tag assignment, refinement and retrieval, arXiv preprint arXiv:1503.08248 (2015).
  4. Kirubai Dhanaraj, Rajkumar Kannan, Harnessing the Social Annotations for Tag Refinement in Cultural Multimedia, IJSRCEIT, 2018, pp. 1802-1808.
  5. Emily Moxley, TaoMei, B. S. Manjunath, Video Annotation Through Search and Graph Reinforcement Mining, Published in IEEE Transaction on Multimedia Vol.12, No.3 April 2010 pp 184 – 193.
  6. L. Ballan, M. Bertini, T. Uricchio, A. Del Bimbo, Data-driven approaches for social image and video tagging, Multimedia Tools and Applications 74 (2015) 1443–1468.
  7. Y. Yang, Y. Yang, Z. Huang, H. T. Shen, Tag localization with spatial correlations and joint group sparsity, in: Proc. of CVPR, Providence, RI, USA, 2011, pp. 881–888.
  8. X. Cao, X.Wei, Y. Han, Y. Yang, N. Sebe, A. Hauptmann, Unified dictionary learning and region tagging with hierarchical sparse representation, Computer Vision and Image Understanding 117 (2013) 934–946. 
  9. Xiangyu Chen, Yadong Mu, Shuicheng Yan, and Tat-Seng Chua, Efficient large-scale Image Annotation by Probabilistic Collaborative Multi-Label Propagation, ACM MM 2010
  10. R. Kannan, G. Ghinea, S. Swaninathan, Salient region detection using patch level and region level image abstractions, 2015, IEEE, Signal Processing Letters 22(6), pp 686-690.
  11. G. Ghinea, R. Kannan, S. Kannaiyan, Gradient-Oriented based PCA subspace for novel face recognition, IEEE Access 2014, pp 914-920.
  12. H. Li, L. Yi, Y. Guan, H. Zhang, DUT-WEBV: A benchmark dataset for performance evaluation of tag localization for web video, in: Proc. Of MMM, Huangshan, China, 2013, pp. 305–315.
  13. J. Song, Y. Yang, Z. Huang, H. T. Shen, J. Luo, Effective multiple feature hashing for large-scale near-duplicate video retrieval, IEEE Transactions on Multimedia 15 (2013) 1997–2008.
  14. Sophia Swamiraj, Rajkumar Kannan, Twitter based stock recommendations using SVM and Ant Colony Optimization Methods. Advances in Natural and Applied Sciences, 2017, 11(9): pp 306-313.
  15. H. Li, L. Yi, B. Liu, Y. Wang, Localizing relevant frames in web videos using topic model and relevance filtering, Machine Vision and Applications 25 (2014) 1661–1670.
  16. Krassimira Ivanova, Peter Stanchev, Evgeniya Velikova, Keon Vanhoof, Benoi Depaire, Rajkumar Kannan, Iliya Mitov, Features for Art Painting Classification Based on Vector Quantization of MPEG-7 Descriptors, published in Springer-Verlag, 2011, ICDEM 2010, LNCS 6411, pp 146-153.
  17. Jinhui Tang, Richang Hong, Shuicheng Yan, Tat-Seng Chua, Guo-Jun Qi, Ramesh Jain, Image Annotation by kNN-Sparse Graph-based label propagation over Noisily-tagged web images, published in ACM Transaction on Intelligent Systems and Technology, Vol. 1, No. 1, September 2010, pp-111-126
  18. Carl Vondrick, Donald Patterson, Deva Ramana, Efficiently Scaling up CrowdSourced Video Annotation, Springer, International Journal on Computer Vision, September 2012
  19. R. Di Salvo, D. Giordano, I. Kavasidis, A Crowdsourcing Approach to support Video Annotation, ACM VIGTA Conference, July 2013.
  20. Kyra Schlining, SusanVon Thun, Linda Kuhnz, Brian Schlining, Lonny Lundsten, Nancy Jacob senstout, Lori Chaney, Judith Connor, Debris in the deep: Using a 22-year video annotation database to survey marine litter in monterey, published in Elsevier, Journal on SciVerse ScienceDirect on January 2013.
  21. Mohammad Soleymani, Martha Larson, Crowdsourcing for affective annotation of video: Development of a viewer-reported boredom corpus, published in the proceeding of SIGIR 2010 workshop on Crowdsourcing for Search Evaluation, July 2010.
  22. Jinhui Tang, Qiang Chen, Meng Wang, Shuicheng Yan, Tat-Seng Chua, Ramesh Jain, Towards Optimizing Human Labeling for Interactive Image Tagging, Published in ACM Transaction on Multimedia Computing Communication and Applications Vol.9, No.4, August 2013.
  23. Elaheh Momeni, Clarie Cardie, Myle Ott, Properties, Predictions, and Prevalence of Useful User-generated Comments for Descriptive Annotation of Social Media Objects, published in Association for the Advancement of Artificial Intelligence, 2013.
  24. Vivian Genaro Motti, Dave Raggett, Quill: A Collaborative Design Assistant for Cross Platform Web Application User Interfaces, Published in ACM WWW 2013 Companion, May 2013, pp 3-5.
  25. Stefan Siersdorfer, Jose San Pedro, Mark Sanderson, Automatic Video Tagging Using Content Redundancy, Published in ACM SIGIR July 2013.
  26. Jinhui Tang, Haojie Li, Guo-Jun, Tat-Seng Chua, Image Annotation by Graph-based Inference with Integrated Multiple / Single Instance Representations, published in IEEE Transaction on Multimedia Vol.12, No.2, February 2011, pp 131-141
  27. Jianping Fan, Yi Shen, Chunlei Yang, Ning Zhou, Structured Max-margin Learning for Inter-related Classifier Training and Multilabel Image Annotation, published in IEEE Transaction on Image Processing Vol.20, No.3, March 2011 pp 837- 854.
  28. Xiangyang Xue, Hangzai Luo, Jianping Fan, Structured Max-margin Learning for Multi-label Image Annotation, published in ACM CIVR July 2010, pp 82-88
  29. Changhu Wang, Feng Jing, Lei Zhang, Hong-Jiang Zhang, Image Annotation Refinement using Random Walk with Restarts, performed at Microsoft Research Asia, published in ACM MM October 2006
  30. Jan C. Van Gemert, Cees G. M. Snoek, Cor J. Veenam, Arnold W. M. Smeulders, Jan-Mark Geusebroek, Comparing Compact codebooks for Visual Categorization, Published in Elsevier Journal on Computer Vision and Image Understanding 114, 2010, pp 450 – 462. 
  31. Lei Wu, Linjin Yang, Nenghai Yu, Xian-Sheng Hua, Learning to Tag, Published in ACM WWW April 2009, pp 361 -370
  32. Jinhui Tang, Richang Hong, Shuicheng Yan, Tat-Seng Chua, Guo-Jun Qi, Published in ACM Conference on Multimedia, October 2009 pp- 223-232 
  33. Jose San Pedro, Tom Yeh, Nuria Oliver, Leveraging user comments for aesthetic aware image search reranking, published in ACM WWW April 2012, pp 439 – 448.
  34. Marco Bertini, Alberto Del Bimbo, Carlo Tornial, Automatic Annotation and Semantic Retrieval of Video Sequences using Multimedia Ontologies, published in ACM MM, October 2006, pp 679- 682
  35. Dong Liu, Xian-Sheng Hua, Meng-Wang, Hong Jiang Zhang, Boost search relevance for tag-based social image retrieval, published in IEEE on published in ICME 2009, pp-1636-1639.
  36. Xirong Li, Cees G. M. Snoek, Marcel Worring, Dennis Koelma, Arnold W. M. Smeulders, Bootstrapping Visual Categorization with Relevant Negatives, published in IEEE Transaction on Multimedia, Vol 15, No. 4, June 2013, pp 933 -945.
  37.  Jianping Fan, Yuli Gao, Hangzai Luo, Multi-level annotation of Natural Scenes using Dominant Image Components and Semantic Concepts, published in ACM MM 04, October 2004, pp 540-547
  38. Xirong Li, Cees G. M. Snoek, Marcel Worring, Arnold W. M. Smeulders, Harvesting Social Images for Bi-concept Search, published in IEEE Transaction on Multimedia, Vol.14, August 2012, pp 1091- 1104.
  39. Jinhui Tang, Shuicheng Yan, Chunxia Zhao, Tat-Seng Chua, Ramesh Jain, Label-specific training set construction from web resource for image annotation, published in Elsevier Journal on Signal Processing , 2012.
  40. Tahu Kuribayashi, Yasuhito Asano, Masatoshi Yoshikawa, Ranking Method specialized for content description of classical music, published in ACM WWW 2013 companion, May 2013.
  41. Guangyu Zhu, Shuicheng Yan, Yi Ma, Image Tag Refinement Towards Low-Rank, Content-Tag Prior and Error Sparsity, MM’10 October 25-29, 2010, pp: 461-470
  42. Yang Yang, Zheng-Jun Zha, Heng Tao Shen, Tat-Seng Chua, Robust Semantic Video Indexing by Harvesting Web Images, Published in Springer-Verlag MMM 2013, Part-I, LNCS 7732, pp 70-80, 2013.
  43. Samuel Huron, Petra Isenberg, Jean Daniel Fekete, PolemicTweet: Video Annotation and Analysis through Tagged Tweets, published in Proceedings of the IFIP TC13 conference on Human-computer Interaction (INTERACT), version 1, April 2013.
  44. Xin Jin, Andrew Gallagher, Liangliang Cao, Jiebo Luo, Jiawei Han, The Wisdom of Social Multimedia: Using Flickr for Prediction and Forecast, published in ACM MM October 2010.
  45. Isaak Kavasidis, Simone Palazzo, Robert Di Salvo, Daniela Giordana, Concetto Spampinato, An Innovative web-based collaborative platform for Video annotation, Published in Springer on Multimedia Tools and Applications, March 2013.
  46. Joho Kim, Phu Nguyen, Sarah Weir, Philip J. Guo, Robert C. Miller, Krzysztof Z. Gajos, Crowdsourcing step-by-step Information Extraction to Enhance existing How-to Videos, submitted to ACM CHI 2014.
  47. Mahashweta Das, Gautam Das, Vegelis Hristidis, Leveraging Collaborative Tagging for Web Item Design, Published in ACM KKD, August 2011.
  48. A. Makadia, V. Pavlovic, S. Kumar, A new baseline for image annotation, in: Proc. of ECCV, Marseille, France, 2008, pp. 316-329.
  49. Cees G. M. Snoek, Bauke Freiburg, Johan Oomen, Roeland Ordelman, Crowd sourcing Rock n' Roll Multimedia Retrieval, ACM Conference on Multimedia, october 2010.
  50. Okasana Yakhnenko, Vasant Honavar, Annotating images and image objects using a hierarchical Dirichlet process model, published in ACM MDM / KDD August 2008.

International Journal of Scientific Research in Science and Technology

Downloads

Published

2018-04-30

Issue

Volume 4, Issue 5, March-April-2018

Section

Research Articles

License

Copyright (c) IJSRST

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

[1]
Kirubai Dhanaraj, Rajkumar Kannan, " A State-of-the art Review: A survey on Multimedia Tagging Techniques, International Journal of Scientific Research in Science and Technology(IJSRST), Online ISSN : 2395-602X, Print ISSN : 2395-6011, Volume 4, Issue 5, pp.377-386, March-April-2018.