Announcements	Digital Image Processing	Quick Links
26 October 2015 Advanced subjects next sem: EE61008 - Medical Image Analysis CS60052 - Adv. DIP & Comp. Vis. EC60502 - Pat. Recog. & Image Understand. 24 October 2015 Term Project presentation schedules on fb/TutoringGroup Events. 15 October 2015 Enter your preferences for Term Project presentation. Scroll down this page to fill in the form.	EE60062 Subject Type: Elective | LTP: 3-1-0 | Credits: 4 Location: NC243, Nalanda Lecture Hall Complex, IIT Kharagpur Time: Slot A / Mon (7:30 AM - 9:30 AM) + Tue (11:30 AM - 12:30 PM) + Wed (10:30 AM - 11:30 AM) Instructor: Dr. Debdoot Sheet Tutoring: N240, SIP Lab, Electrical Engg. Abhijit Guha Roy, Tue (5:30 - 6:30 PM) + Thu (5:30 - 6:30 PM) Rachana Sathish, Mon (5:30 - 6:30 PM) Kanithi Praveen Kumar, Wed (5:30 - 6:30 PM)	Linear Algebra - Gilbert Strang Digital Signal Processing - Alan V. Oppenheim Design and Analysis of Algorithms - Dana Moshkovitz and Bruce Tidor Introduction To MATLAB Programming - Yossi Farjoun Computational Methods of Scientific Programming - Thomas Herring and Chris Hill Tutoring Group on Facebook

Why this subject?

Contents

Digital image processing (DIP) refers to the field of processing of images handled in a digital format using a digital computer. The field had a major impact on our day to day lives, across smartphone apps, visual media, advertisements, infotainment, gaming, medical imaging, vehicular and diving technology, navigation, survellance, etc. in the current age and has been imbibed into the common place beyond our pondering. DIP is the foundation of a larger field known as machine vision, which is valued industrially as a $9.5 Billion market by 2020. If you are looking forward to a career in imaging technology, smart cameras and smartphones, image sensors, photography and videography, digtal multimedia, visualization, augmented reality, gaming, automotive and navigation system, this a foundation subject you should definitely opt for. Text books: [1]. M. Sonka, V. Hlavac and R. Boyle, Image Processing, Analysis, and Machine Vision, Cengage Learning, 2008. [2]. R. C. Gonzalez and R. E. Woods, Digital Image Processing, Pearson, 2009.

Image formation and digital imaging Pixels, colors, image formats, display Histograms, numerical calculus, fuzzy sets Convolution, correlation, Fourier transform Coordinate transformation and interpolation Image enhancement Noise models and image restoration Image segmentation Mathematical morphology Texture and wavelets Object representation and description HDR and EDF photography Object detection and tracking

Module I: Introductory Concepts

1. Image formation	Refresher of geometric optics, pin-hole and lens-based image formation, real-world to camera to image plan coordinate system, lens aberation and distortions, radiometric concents of image formation.	[1]. 3.4
2. Digital imaging sensors	Photosensitivity of semiconductors, digital monochromatic camera, color sensor.	[1]. 2.5 [2]. 2.3
3. Mathematical representations and properties	Image representation in analog domain as continuous function, image digitization, resolution, sampling and quantization, storage and retrieval, metric and topological properties, relationships between pixels, adjacency, connectivity, distance.	[1]. 2.1 - 2.3.1 [2]. 2.4 - 2.5
4. Colors in images	Physics of color, color perception by human vision, color representations, color spaces and transformation, indexed (paletted) color representations, color reproduction in electronic and printed systems.	[1]. 2.5 [2]. 6.1 - 6.3

Module II: Analytical Foundation for Digital Image Processing

1. Image histogram and probability density	Histogram of a grayscale and color image, probability density from histogram, kernel density estimation.	[1]. 2.3.2 [2]. 3.3
2. Fuzzy set theory	Preliminaries and definition, computing a fuzzy histogram, some basic operations.	[2]. 3.8
3. Numerical calculus	Refresher of numerical differentiation and integration.
4. Convolution and correlation	Convolution as a mathematical operation from 1D to 2D, correlation, spatial point operations and solutions via convolution and correlation, kernels and their significance.	[1]. 3.1.2 [2]. 3.4.2 - 3.4.4
5. Functional mapping and look-up tables	Point operators and intersity transformation via functional mapping and look-up tables.	[1]. 5.1.2 [2]. 3.2
6. Fourier transform	Images as linear systems, from 1D to 2D Fourier transform, sampling and discrete Fourier transform (DFT), properties and zero-centering of the 2D-DFT, frequency in images.	[1]. 3.2.1 - 3.2.5 [2]. 4.2 - 4.6
7. Image interpolation and coordinate space conversion	Image interpolation, nearest neighbor rule, bilinear rule, polar-to-cartesian scan conversion and vice-versa.	[1]. 5.2.2 [2]. 2.4.4, 2.6.5

Module III: Image Enhancement

1. Grayscale intensity transformation	Image negation, log- and power-law transform, gamma transformation and correction, piece-wise linear transform.	[1]. 5.1 [2]. 3.2
2. Global histogram processing	Histogram equalization, matching to specifications, image contrast and histogram statistics, modality of histogram.	[1]. 5.1 [2]. 3.3
3. Local histogram processing	Locally adaptive histogram equalization (LAHE) and Contrast-limited adaptive histogram equalization (CLAHE).	DOI: 10.1109/ VBC.1990.109340
4. Dynamic and brightness preserving histogram processing	Bi-histogram equalization (BHE), brightness preserving dynamic histogram equalization (BPDHE) and faster implementation via. processing in fuzzy sets.	DOI: 10.1109/ TCE.2010.5681130
5. Color equalization	Vignetting correction, color constancy and auto white-balancing (AWB).	DOI: 10.1109/ ISCE.2005.1502356

Module IV: Noise Models and Image Restoration in Spatial Domain

1. Noise models and properties	Additive and multiplicative noise, spatial and frequency properties, stochastic models of noise and uncertainty, the Gaussian, Poisson, Rayleigh, Rician, Nakagami, Gumbel, etc. distributions, periodic and aperiodic noise.	[1]. 2.3.6 [2]. 5.1 - 5.2
2. Estimation of noise properties	Signal to noise ratio (SNR), root mean squared (RMS) value, mean squared error (MSE).	[1]. 2.3.6 [2]. 5.1 - 5.2
3. Image quality assessment	Universal image quality index, structural similarity, gradient structural similarity metric.	DOI: 10.1109/ ICSMB.2010.5735353
4. Spatial linear and non-linear filtering via convolution	Mean (average) filter, weighted average filter, sharpening filters, first order (gradient) and second order derivative (Laplacian) filtering, unsharp masking and high-boost filtering.	[1]. 5.3.1 [2]. 3.5 - 3.7
5. Adaptive and non-linear spatial filtering	Median, Frost's, homogeneous mask area, maximim local homogeneity filtering.	DOI: 10.1109/ ICSMB.2010.5735353
6. Vector and order statistics filtering	Alpha trimmed mean filter, general vector order statistics filter, sigma and Lee's filter.	[2]. 5.3.2
7. Adaptive image restoration filtering	Bilateral, adaptive bilateral filter, aggressive region growing, diffusion, non-local means filtering, inverse filtering, minimum mean-square error filtering, geometric mean filter.	[1]. 5.4 [2]. 5.3.1, 5.3.3, 5.6 - 5.10

Module V: Image Restoration in Frequency Domain

1. High-frequency and periodic noise reduction	Ideal low-pass filter, Butterworth filter, Gaussian weighted filter, band-pass and band-reject filters, notch and optimum notch filter.	[1]. 5.3.8 [2]. 4.8, 4.10, 5.4
2. Image enhancement and restoration in frequency domain	Ideal high-pass filter, Butterworth high-pass filter, inverse Gaussian weighted filter, Laplacian filter, unsharp masking, high-boost filtering, homomorphic filtering.	[1]. 5.3.8 [2]. 4.9

Module VI: Image Segmentation

1. Intensity based segmentation	Binary segmentation with fixed and dynamic threshold selection, histogram partitioning approach, iterative threshold selection, intensity statistics based approach (Otsu's method), information theory based threshold selection.	[1]. 6.1 [2]. 10.3
2. Multiclass segmentation and evaluation	Dynamic histogram partitioning, constained optimization for multi-threshold selection, multi-variable thresholding.	[1]. 6.5 [2]. 10.3.6 - 10.3.8
3. Color image segmentation	Mean-shift clustering, nearest neighbor search, skin color segmentation, optimum statistical classifiers, neural network models.	[1]. 7.1, Ch 9 [2]. 6.7, 12.2 DOI: 10.1109/ 76.767122
4. Region growing, split and merge	Region growing approach, region splitting and mergin techniques for segmentation.	[1]. 6.3 [2]. 10.4 - 10.5
5. Edge detection	Sobel, Prewitt and consistent gradient operators for edge detection, edge linking and Canny's approach, dynamic programming, Hough transform.	[1]. 6.2 [2]. 10.2
6. Active contours and graph cuts for image segmentation	Image representation as a graph, max-flow min-cut algorithm.	[1]. 7.2 - 7.7

Module VII: Mathematical Morphology

1. Preliminaries	Binary morphology, grayscale operatiors and analogy with order-statistics.	[1]. 13.1 - 13.2 [2]. 9.1, 9.6
2. Fundamental and some basic operators	Erosion, dilation, opening, closing.	[1]. 13.3 - 13.4 [2]. 9.2 - 9.3
3. Morphological processing algorithms	Hit and miss transform, boundary extraction, region filling, thickening and thinning, skeleton extraction, prunning, connected component extraction and labeling, morphological watersheds.	[1]. 13.5 - 13.7 [2]. 9.4 - 9.5

Module VIII: Textures and Wavelets

1. Co-occurrence matrices	Basics of textures in optical and non-optical images, graylevel co-occurrence matrices (GLCM), fuzzy co-occurrence matrices.	[1]. 15.1 [2]. 11.3.3
2. Texture features from Fourier transform	Image spectrum, frequency components, periodic vs. aperiodic texture appearance models.	[2]. 11.3.3
3. Kernel transforms for texture analysis	Gabor functions, Laws' masks, local binary patterns (LBP).	[2]. 7.3 DOI: 10.1109/ TPAMI.2002.1017623
4. Image pyramids and sub-bands	Image decomposition, image pyramid, sub-band decomposition, Harr wavelets.	[2]. 7.1
5. Multiresolution expansion of functions	Series expansions and the scaling functions.	[2]. 7.2
6. Wavelet transform	1D and 2D wavelet transform, fast wavelet transform, Hadamard and Daubechis' wavelets.	[1]. 3.2.7 [2]. 7.4 - 7.6

Module IX: Object Representation and Description

1. Shape boundary descriptors	Chain codes, polygon approximation, shape signatures, boundary segments, skeleton, shape number, Fourier descriptors and statistical moments.	[1]. 8.2 [2]. 11.1 - 11.2
2. Region descriptors	Topological descriptors, moments of 2D functions, principal components and the Karhunen-Loeve (KL) transform and the principal component analysis (PCA).	[1]. 8.3 [2]. 11.3 - 11.5

Module X: Advanced Techniques for Image Processing and Machine Vision

1. HDR imaging	High dynamic-range (HDR) imaging and photography through fusion of low dynamic-range images or exposure bracketed sequences.	DOI: 10.1109/ PG.2007.17 Code | Images
2. EDF photography	Extended depth-of-field (EDF) photography through fusion of limited depth-of-field images.	DOI: 10.1109/ TIP.2012.2231087 Code
3. Object detection and tracking	Background modeling, kernel based tracking, object path analysis.	[1]. 16.5

Learning Resources

Conferences	Int. Conf. Image Processing (ICIP), Computer Vision and Pattern Recognition (CVPR), Int. Conf. Comput. Vis. (ICCV), Eur. Conf. Comput. Vis. (ECCV), Asian Conf. Comput. Vis. (ACCV), ACM SIGGRAPH, ACM SIGGRAPH-Asia, Indian Conf. Vis., Graph., Image Process. (ICVGIP), Nat. Conf. Vis., Pat. Recog., Image Process., Graph. (NCVPRIPG)
Journals	IEEE Trans. Image Process. (TIP), IEEE Trans. Pat. Anal., Mach. Intell. (T-PAMI), Int. J. Comput. Vis. (IJCV), IET Image Process. (IET-IP), ACM Trans. Graphics (TOG), IEEE Trans. Consumer Electron. (TCE), IEEE Trans. Circ., Sys., Video Tech. (TCSVT)
Matlab Image Processing Toolbox	Comprehensive set of reference-standard algorithms, functions, and apps for image processing, analysis, visualization, and algorithm development in Matlab environment.
Open CV	Open source computer vision library.
MeVisLab	Medical image processing, analysis and visualization SDK.
Python(x,y)	Python (2.7) based numerical computations, data analysis and data visualization, Qt graphical user interfaces and Spyder interactive scientific development environment. Can be of great use to develop image analytics tools and integrate then with MeVis projects or Android apps.
Android Studio and SDK	Developer studio, IDE and SDK if you are interested to develop Android apps as part of your mini-project.
Tortoise SVN	Version control utility for Windows implemented using the standard SVN. Helpful for managing (big / multiple file) medical software development projects.
MikTeX	Up-to-date implementation of TeX/LaTeX and related programs for Windows (all current variants). Wiki for writing Hello world! using LaTeX. Template for submitting 1-Page report on the mini-project.
Grand Challenges in Biomedical Image Analysis	Hub for Mini-Project ideas. Collection of the grand challenges in medical image analysis hosted at ISBI, MICCAI and other conferences and symposia. Good source for you to find a challenging research problem or to look for a revenue potenial product buildup.

Tutorials and Assignments

Module II-III

Download practice problem sheet. Due on 31 Aug. 2015 at 7:30 AM in class.

Term Projects

Schedule preference for Presentation. Template for submitting 1-Page Paper on the Term Project. Template for presentation.

Pedagogical Objectives

The aim is to teach students fundamentals of digital image processing. It would be beneficial to students opting for specialization in machine vision, graphics and animation, media design, medical imaging instrument design, medical imaging, medical visualization, robotics, augmented reality and gaming, who can use the gained skills in order to develop newer technological innovations and regularize their high-throughput translation and usage.

On completion of the course, a student would be able to: 1. Explain and discuss the scientific principles of digital image formation, energy interaction as basis of image sensing, inter-relation between image appearance and objects in real world, quantitative approaches, multi-scale and multi-resolution relation, graphics, visualization and augmented reality. 2. Demonstrate the ability of analyzing images using their foundations of linear algebra, graph based and machine learning based approaches. 3. Design and develop new techniques for improving machine vision based workflow with incorporation of digital image processing and analysis in regular industrial and other usage.