Zwiększenie rozdzielczości obrazów termowizyjnych metodą sieci neuronowych głębokiego uczenia

pol Article in Polish DOI: 10.14313/PAR_241/31

send Piotr Więcek , Dominik Sankowski Politechnika Łódzka, Instytut Informatyki Stosowanej

Download Article

Streszczenie

W pracy przedstawiono nowy algorytm zwiększenia rozdzielczości obrazów termowizyjnych. W tym celu zintegrowano sieć resztkową z modułem współdzielonego filtru z podpróbkowaniem obrazu KSAC (ang. Kernel-Sharing Atrous Convolution). Uzyskano znaczne skrócenie czasu działania algorytmu przy zachowaniu dużej dokładności. Sieć neuronową zrealizowano w środowisku PyTorch. Przedstawiono wyniki działania proponowanej nowej metody zwiększenia rozdzielczości obrazów termowizyjnych o wymiarach 32 × 24, 160 × 120 i 640 × 480 dla skali 2–6.

Słowa kluczowe

głębokie uczenie maszynowe, obraz termograficzny, obraz termowizyjny, PyTorch, resztkowe sieci neuronowe, superrozdzielczość

Increasing of Thermal Images Resolution Using Deep Learning Neural Networks

Abstract

The article presents a new algorithm for increasing the resolution of thermal images. For this purpose, the residual network was integrated with the Kernel-Sharing Atrous Convolution (KSAC) image sub-sampling module. A significant reduction in the algorithm’s complexity and shortening the execution time while maintaining high accuracy were achieved. The neural network has been implemented in the PyTorch environment. The results of the proposed new method of increasing the resolution of thermal images with sizes 32 × 24, 160 × 120 and 640 × 480 for scales up to 6 are presented.

Keywords

Deep Learning, PyTorch, residual neural networks, super-resolution, thermographic image

Bibliography

  1. Agustsson E., Timofte R., NTIRE 2017 challenge on single image super-resolution: Dataset and study, 2017, [ https://data.vision.ee.ethz.ch/cvl/DIV2K ], DOI: 10.1109/CVPRW.2017.150.
  2. Bengio Y., Simard P., Frasconi P., Learning long-term dependencies with gradient descent is difficult. “IEEE Transactions on Neural Networks”, Vol. 5, No. 2, 1994, 157–166, DOI: 10.1109/72.279181.
  3. Duchon C.E., Lanczos filtering in one and two dimensions, “Journal of Applied Meteorology and Climatology”, Vol. 8, Bo. 18, 1979, 1016–1022, DOI: 10.1175/1520-0450(1979)018<1016:LFIOAT>2.0.CO;2.
  4. Glorot X., Bengio Y., Understanding the difficulty of training deep feedforward neural networks, Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, 2010.
  5. Gu J., Wang Z., Kuen J., Ma L., Shahroudy A., Shuai B., Liu X.W.T., Wang G., Recent advances in convolutional neural networks, Pattern Recognition, Elsevier, 2017.
  6. Huang Y., Wang Q., Jia W., He X., See more than once – kernel-sharing atrous convolution for semantic segmentation, arXiv:1908.09443 [cs.CV], 16 November, 2019.
  7. Ioffe S., Szegedy C., Batch normalization: Accelerating deep network training by reducing internal covariate shift, Proceedings of the 32nd International Conference on Machine Learning, 2015, Lille, France.
  8. Kim J., Lee J.-K., Lee K.-M., Accurate image super-resolution using very deep convolutional networks, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1646–1654, Munich, September 8–14, 2016, DOI: 10.1109/CVPR.2016.182.
  9. Kim J.-H., Lee J.-S., Deep residual network with enhanced upscaling module for super-resolution, 15th European Conference on Computer Vision, 800–808, Salt-Lake City, June 18–22, 2018.
  10. Kincaid D., Cheney W., Analiza numeryczna, Wydawnictwo Naukowe Techniczne, 2007, ISBN 83-204-3078-X.
  11. Li J., Fang F., Mei K., Zhang G., Multi-scale residual network for image super-resolution, 15thEuropean Conference on Computer Vision, Munich, September 8–14, 2018.
  12. Lim B., Son S., Kim H., Nah S., Lee K.M., Enhanced deep residual networks for single image super-resolution, The IEEE Conference on Computer Vision and Pattern Recognition Workshops, July 21–26, 2017, Honolulu.
  13. Mandanici E., Tavasci L., Corsini F., Gandolfi S., A multi-image super-resolution algorithm applied to thermal imagery. “Applied Geomatics”, Vol. 11, No. 3, 2019, 215–228, DOI: 10.1007/s12518-019-00253-y.
  14. Shi W., Caballero1 J., Huszar F., Totz J., Aitken R.B.A.P., Rueckert D., Wang Z., Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, [In:] IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, 1974– 1883, 2016, DOI: 10.1109/CVPR.2016.207.
  15. Szegedy C., Ioffe S., Vanhoucke V., Alemi A., Inception v4, inception-ResNet and the impact of residua connections on learning, arXiv:1602.07261, 2016.
  16. Wang Z., Chen J., Hoi S.C.H., Deep learning for image super-resolution: A survey, arXiv:1902.06068 [cs.CV], 8 Feb. 2020, 1, 2020.
  17. Więcek P., Sankowski D., Low-cost, low-resolution IR system with super-resolution interpolation of thermal images for industrial applications, “Measurement Automation Monitoring”, Vol. 64, No. 4, 2018, 108–111.
  18. Więcek P., Wybrane algorytmy ilościowej analizy obrazów rozkładu temperatury wsadów w procesach nagrzewania indukcyjnego, Rozprawa doktorska, Instytut Informatyki Stosowanej, Politechnika Łódzka, 2021.
  19. Wu X.-S., Cai Y., Techniques of optical microscan in infrared imaging system, “Journal of Infrared and Millimeter Waves”, Vol. 26, No. 1, 2007, 10–14.
  20. Zhang Y., Li K., Li K., Wang L., Zhong B., Fu Y., Image super-resolution using very deep residual channel attention networks, 15th European Conference on Computer Vision, Munich, September 8–14, 2018.
  21. Zieliński T., Cyfrowe przetwarzanie sygnałów. Od teorii do zastosowań. Wydawnictwa Komunikacji i Łączności WKŁ, 2007.
  22. Flirone, Flir Systems, 2020, [ www.flir.com/products/flir-one-pro ].
  23. Melexis. Melexis Inspired Engineering, 2019, [ www.melexis.com/en/product///mlx90640/far-infrared-thermal-sensor-array ].
  24. Optris, Optris, 2020, [ www.optris.global/thermal-imager-optris-pi160 ].
  25. PyTorch v1.3, Pytorch, 2020, [ https://pytorch.org ].
  26. Pytorch-Flopscounter v0.6, 2020, [ https://github.com/sovrasov/flops-counter.pytorch ]
  27. Pytorch-forwardhook, PyTorch, 2020, [ https://pytorch.org/docs/stable/nn.html ].