컴퓨터비전, 머신러닝, 딥러닝을 이용한 의료영상분석 11. Medical image Enhancement (3) SRCNN, GAN, SRGAN, Enhancement Metric

본 내용은 Edwith의 컴퓨터비전, 머신러닝, 딥러닝을 이용한 의료영상분석을 요약 정리한 내용으로 DGIST 박상현 교수님과 Edwith, STAR-MOOC에 그 저작권이 있음을 미리 공지합니다.

11-1 URL : https://www.edwith.org/medical-20200327/lecture/63183

11-2 URL : https://www.edwith.org/medical-20200327/lecture/63184

11-3 URL : https://www.edwith.org/medical-20200327/lecture/63185

11-4 URL : https://www.edwith.org/medical-20200327/lecture/63186

11-5 URL : https://www.edwith.org/medical-20200327/lecture/63187

11-6 URL : https://www.edwith.org/medical-20200327/lecture/63188

11-7 URL : https://www.edwith.org/medical-20200327/lecture/63189

11-1 SRCNN

Super-Resolution using CNN

이전 우리는 Dictionary를 활용한 방법을 통해 minimize $\alpha$를 찾는 문제를 알 수 있었음

Feature를 뽑아 둘 간의 관계를 확인

이를 바탕으로 Mapping, average

Representation -> Non-linear mapping -> Reconstruction

SRCNN은 최초로 Super-Resolution에 Convolutional Neural Network(CNN)을 통한 과정을 진행함

SRCNN : "Image Super-Resolution Using Deep Convolutional Networks", ECCV 2014, TPAMI 2015

http://mmlab.ie.cuhk.edu.hk/projects/SRCNN.html

Learning a Deep Convolutional Network for Image Super-Resolution

Image Super-Resolution Using Deep Convolutional Networks Chao Dong, Chen Change Loy, Kaiming He, Xiaoou Tang Department of Informaiton Engineering, The Chinese University of Hong Kong Microsoft Research Abstract We propose a deep learning method for single

mmlab.ie.cuhk.edu.hk

End-to-End Super-Resolution

Bicubic 혹은 Bilinear를 통한 Interpolation 이후 CNN을 통과시킴

11-2. Upsampling strategy

SRCNN의 경우 Interpolation한 이미지 Input 크기가 output 크기와 같다.

- Upsample을 했다는 것은 정보의 손실이 가능하며 computational cost 또한 크다.

이 때문에 본 저자들의 후속 논문으로 LR의 conv layer를 통과시키는 방법과 더불어 Deconv를 통한 Upsample을 제안. 대표적인 예 Transposed Convolution, Pixel-Shuffle

FSRCNN : "Accelerating the Super-Resolution Convolutional Neural Network", ECCV 2016

http://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html

Accelerating the Super-Resolution Convolutional Neural Network

Accelerating the Super-Resolution Convolutional Neural Network Chao Dong, Chen Change Loy, Xiaoou Tang Department of Information Engineering, The Chinese University of Hong Kong dongchao@sensetime.com, {ccloy, xtang}@ie.cuhk.edu.hk Abstract As a success

mmlab.ie.cuhk.edu.hk

Sub-Pixel CNN : "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network", CVPR 2016

https://openaccess.thecvf.com/content_cvpr_2016/papers/Shi_Real-Time_Single_Image_CVPR_2016_paper.pdf

11-3. Deep networks for super resolution

Deep CNN for Super Resolution

초기 Interpolation 이후 다수의 CNN을 사용 20 layers - residual

SRCNN 학습시 오래 걸린다. learning rate가 작으면 수렴 속도가 느리고 너무 크면 발산

이를 보완하고자 gradient clipping 방법 제안

VDSR : "Accurate Image Super-Resolution Using Very Deep Convolutional Networks", CVPR 2016

https://ieeexplore.ieee.org/document/7780551

Accurate Image Super-Resolution Using Very Deep Convolutional Networks

We present a highly accurate single-image superresolution (SR) method. Our method uses a very deep convolutional network inspired by VGG-net used for ImageNet classification [19]. We find increasing our network depth shows a significant improvement in accu

ieeexplore.ieee.org

https://arxiv.org/abs/1511.04587

Accurate Image Super-Resolution Using Very Deep Convolutional Networks

We present a highly accurate single-image super-resolution (SR) method. Our method uses a very deep convolutional network inspired by VGG-net used for ImageNet classification \cite{simonyan2015very}. We find increasing our network depth shows a significant

arxiv.org

Gradient Clipping

Gradient가 클 때 alpha(learning rate)도 크다면 너무 큰 범주를 왔다갔다하기 때문에 threshold를 기점으로 clipping을 진행함.

SRResNet, SR DenseNet, ....

Laplician Pyramid Network

이외 Super-Resolution Deep Networks

11-4. Generative Adversarial Network

Generative Adversarial Networks, NIPS 2014

https://arxiv.org/abs/1406.2661v1

Generative Adversarial Networks

We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that

arxiv.org

Pix2Pix

Image-to-Image Translation with Conditional Adversarial Networks, CVPR 2017

https://arxiv.org/abs/1611.07004

Image-to-Image Translation with Conditional Adversarial Networks

We investigate conditional adversarial networks as a general-purpose solution to image-to-image translation problems. These networks not only learn the mapping from input image to output image, but also learn a loss function to train this mapping. This mak

arxiv.org

CycleGAN

Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks, ICCV 2017

https://arxiv.org/abs/1703.10593

Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

Image-to-image translation is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image using a training set of aligned image pairs. However, for many tasks, paired training data will not be a

arxiv.org

11-5. SRGAN

C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al. "Photo-realistic single image super-resolution using a generative adversarial network.", 2017 CVPR

https://arxiv.org/abs/1609.04802

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Despite the breakthroughs in accuracy and speed of single image super-resolution using faster and deeper convolutional neural networks, one central problem remains largely unsolved: how do we recover the finer texture details when we super-resolve at large

arxiv.org