Development of Naïve technique for Removing Fences, Reflections and Raindrops from image Pattern Recognition Techniques

Authors

  • Arvind Singh  Research Scholar,Sheat College of Engineering, Varanasi, Uttar Pradesh, India
  • Shailesh Kumar Singh  Sheat College of Engineering, Varanasi, Uttar Pradesh, India

Keywords:

Deep Convolutional Neural Network, Fence Removal, Flow Estimation, Dense Optical Flow

Abstract

Mobile, Smartphone, Tablets, and other devices with built-in cameras are now widely available at affordable prices, allowing regular people to capture and share life's most important moments with the world. However, a novice photographer may not be pleased with the results after posting them online. It's possible that the photographer's subject of interest has been blocked off or walled off in some way. The authors write, "We provide a learning-based technique for removing unwanted barriers from a rapid succession of photographs obtained by a moving camera, such as window reflections, fence occlusions, or rainfall. It takes into account the relative velocity of the foreground and background. Inaccuracies in the flow estimate and brittle assumptions like brightness consistency may be accounted for thanks to the learning-based layer reconstruction. We show how well training on synthetic data generalises to real-world photographs. Our research into many challenging reflection and fence removal scenarios has shown encouraging results, demonstrating the efficacy of the proposed approach.

References

  1. Jean-Baptiste Alayrac, Joao Carreira, and Andrew Zisserman. The visual centrifuge: Model-free layered video representations. In CVPR, 2019.
  2. Nikolaos Arvanitopoulos, Radhakrishna Achanta, and Sabine Susstrunk. Single image reflection suppression. In CVPR, 2017.
  3. Efrat Be’Ery and Arie Yeredor. Blind separation of superimposed shifted images using parameterized joint diagonalization. TIP, 17(3):340–353, 2008. 1
  4. Sean Bell, Kavita Bala, and Noah Snavely. Intrinsic images in the wild. ACM TOG, 33(4):159, 2014.
  5. Yuhua Chen, Cordelia Schmid, and Cristian Sminchisescu. Self-supervised learning with geometric constraints in the monocular video: Connecting flow, depth, and camera. In ICCV, 2019.
  6. Chen Du, Byeongkeun Kang, Zheng Xu, Ji Dai, and Truong Nguyen. Accurate and efficient video de-fencing using convolutional neural networks and temporal information. In ICME, 2018.
  7. Elmar Eisemann and Fre´do Durand. Flash photography enhancement via intrinsic relighting. ACM TOG, 23(3):673– 678, 2004.
  8. Qingnan Fan, Jiaolong Yang, Gang Hua, Baoquan Chen, and David Wipf. A deep generic architecture for single image reflection removal and image smoothing. In ICCV, 2017.
  9. Kun Gai, Zhenwei Shi, and Changshui Zhang. Blind separation of superimposed images with unknown motions. In CVPR, 2009.
  10. Kun Gai, Zhenwei Shi, and Changshui Zhang. Blind separation of superimposed moving images using image statistics. TPAMI, 34(1):19–32, 2011.
  11. Roger Grosse, Micah K Johnson, Edward H Adelson, and William T Freeman. Ground truth dataset and baseline evaluations for intrinsic image algorithms. In ICCV, 2009.
  12. Xiaojie Guo, Xiaochun Cao, and Yi Ma. Robust separation of reflection from multiple images. In CVPR, 2014.
  13. Jia-Bin Huang, Sing Bing Kang, Narendra Ahuja, and Johannes Kopf. Temporally coherent completion of dynamic video. ACM TOG, 35(6):196, 2016.
  14. Shachar Ilan and Ariel Shamir. A survey on data-driven video completion. Computer Graphics Forum, 34(6):60–85, 2015.
  15. Junho Jeon, Sunghyun Cho, Xin Tong, and Seungyong Lee. Intrinsic image decomposition using structure-texture separation and surface normals. In ECCV, 2014.
  16. Meiguang Jin, Sabine Su¨sstrunk, and Paolo Favaro. Learning to see through reflections. In ICCP, 2018.
  17. Sankaraganesh Jonna, Krishna K Nakka, and Rajiv R Sahay. Deep learning-based fence segmentation and removal from an image using a video sequence. In ECCV, 2016.
  18. Sankaraganesh Jonna, Sukla Satapathy, and Rajiv R Sahay. Stereo image de-fencing using smartphones. In ICASSP, 2017. 2Zdenek Kalal, Krystian Mikolajczyk, and Jiri Matas. Tracking-learning-detection. TPAMI, 34(7):1409–1422, 2011.
  19. Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
  20. Yu Li and Michael S Brown. Exploiting reflection change for automatic reflection removal, in ICCV, 2013.
  21. Yu Li and Michael S Brown. Single image layer separation using relative smoothness. In CVPR, 2014.
  22. Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chandraker. Inverse rendering for complex indoor scenes: Shape, spatially-varying lighting, and svbrdf from a single image. In CVPR, 2020.
  23. Ce Liu, Jenny Yuen, Antonio Torralba, Josef Sivic, and William T Freeman. Sift flow: Dense correspondence across different scenes, in ECCV, 2008.
  24. Yadong Mu, Wei Liu, and Shuicheng Yan. Video de-fencing. IEEE Transactions on Circuits and Systems for Video Technology, 24(7):1111–1121, 2013.
  25. Ajay Nandoriya, Mohamed Elgharib, Changil Kim, Mohamed Hefeeda, and Wojciech Matusik. Video reflection removal through Spatio-temporal optimization. In ICCV, 2017.
  26. Minwoo Park, Kyle Brocklehurst, Robert T Collins, and Yanxi Liu. Image de-fencing revisited in ACCV, 2010.
  27. Abhijith Punnappurath and Michael S Brown. Reflection removal using a dual-pixel sensor. In CVPR, 2019.
  28. Soumyadip Sengupta, Jinwei Gu, Kihwan Kim, Guilin Liu, David W Jacobs, and Jan Kautz. Neural inverse rendering of an indoor scene from a single image. In ICCV, 2019.
  29. YiChang Shih, Dilip Krishnan, Fredo Durand, and William T Freeman. Reflection removal using ghosting cues. In CVPR, 2015.
  30. Sudipta N Sinha, Johannes Kopf, Michael Goesele, Daniel Scharstein, and Richard Szeliski. Image-based rendering for scenes with reflections. ACM TOG, 31(4):100–1, 2012.
  31. Deqing Sun, Xiaodong Yang, Ming-Yu Liu, and Jan Kautz. Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume. In CVPR, 2018.
  32. Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei A Efros, and Moritz Hardt. Test-time training for out-of-distribution generalization. arXiv:1909.13231, 2019.
  33. Richard Szeliski, Shai Avidan, and P Anandan. Layer extraction from multiple images containing reflections and transparency. In CVPR, 2000.
  34. Renjie Wan, Boxin Shi, Ling-Yu Duan, Ah-Hwee Tan, and Alex C Kot. Benchmarking single-image reflection removal algorithms. In ICCV, 2017. 6
  35. Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al. Image quality assessment: from error visibility to structural similarity. TIP, 13(4):600–612, 2004.
  36. Kaixuan Wei, Jiaolong Yang, Ying Fu, David Wipf, and Hua Huang. Single image reflection removal exploiting misaligned training data and network enhancements. In CVPR, 2019.
  37. Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, and William T Freeman. Video enhancement with the task-oriented flow. IJCV, 127(8):1106–1125, 2019.
  38. Tianfan Xue, Michael Rubinstein, Ce Liu, and William T Freeman. A computational approach for obstruction-free photography. ACM TOG, 34(4):79, 2015.
  39. Jie Yang, Dong Gong, Lingqiao Liu, and Qinfeng Shi. Seeing deeply and bidirectionally: A deep learning approach for single image reflection removal. In ECCV, 2018.
  40. Renjiao Yi, Jue Wang, and Ping Tan. Automatic fence segmentation in videos of dynamic scenes. In CVPR, 2016.

International Journal of Scientific Research in Science and Technology

Downloads

Published

2022-11-04

Issue

Volume 9, Issue 6, November-December-2022

Section

Research Articles

License

Copyright (c) IJSRST

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

[1]
Arvind Singh, Shailesh Kumar Singh, " Development of Naïve technique for Removing Fences, Reflections and Raindrops from image Pattern Recognition Techniques, International Journal of Scientific Research in Science and Technology(IJSRST), Online ISSN : 2395-602X, Print ISSN : 2395-6011, Volume 9, Issue 6, pp.48-55, November-December-2022.