Summer Internship At IIT Jammu
Once a wise turtle quoted “You often meet your destiny on the road you take to avoid it.” This statement in a nutshell quite aptly describes my research internship which I did in IIT Jammu. Now before proceeding to further, a little introduction from, myself; I am Asfahan Shah, a third-year undergraduate student majoring in computer science in Bennett University, India. Now with this little bit of background knowledge, let’s get to the meat of the matter and answer some pertinent questions, what was the internship about? what did I do in internship? and more importantly how does the above wisdom correlate with my summer internship?
Now the answers to first two questions in quite simple and will be properly explained in below sections of the blog but what about the last question. Well, you see IIT Jammu wasn’t my first choice for research internship and it was not like that I wasn’t able to get internship in other institutes of India. In fact, I got good research internship opportunities but due to circumstances whose discussion is beyond the scope of the blog, I landed in IIT Jammu. Now you can imagine I might be quite miserable; I could have gone to other place but due to unfortunate circumstances wasn’t able to. But that’s where the above quote describes my journey and as such in hindsight, I have to say that going to IIT Jammu for summer internship was one of the best decisions in my life. There will be several questions in your head. But let’s digress and move to the details of the internship because I am sure that after these below sections all your queries will be satisfied.
So, what was the internship about you may ask.
The topic of my internship was “Image quality assessment using deep learning techniques” under the guidance of Dr. Vinit Jakhetiya. What exactly is image quality assessment? In simple terms it is a way to assess the quality of images by assigning a score to each of the images present. This can be done using both conventional methods and deep architectures. In this internship I applied vision transformers which is a deep learning architecture.
Now you might say what is a vison transformer or more pertinently what in the heck is a transformer.
Transformer is a deep learning architecture that is most commonly used in NLP. Normally a transformer has two parts:
1) Encoder
2) Decoder.
Now you might say, what exactly are these two parts:
Well, encoder consists of several smaller block and each block consists of a multi headed self-attention layer and a feed forward network.
Similar is the case of decoder where each smaller block consists of a masked multiheaded self-attention layer, encoder decoder multi headed attention layer and a feed forward network.
Author “what is attention?” I hear you say
Attention mechanism is based upon human cognitive attention. Attention tries to increase some input portions while reducing the other parts. Thus, enables us to focuses more on small but important parts. To calculate attention, we need three vectors Query(Q), Key(K) and Value.
Now one question may arise how is this transformer stuff related to image quality assessment?
Well now comes the concept of a vision transformer.
Vision transformer splits an image into various patches. These patches are then linearized and a positional encoding is placed. These patches are then forwarded to a transformer encoder. The output of which is goes through a classifier such as MLP to get the result.
So why we have to split the images into patches why not just put the whole image? I hear you say.
Well, the reason behind is, if input is whole image, then during attention (self) phase, each pixel will have to be associated with the other. Thus, making the operation very costly and not viable for real world cases.
Now coming to what I actually build in internship with tools I discussed above.
I build a model based on concepts like vision transformer and CNN for image quality assessment on data set RealSRQ**.
**Q. Jiang et al., “Single Image Super-Resolution Quality Assessment: A Real- World Dataset, Subjective Studies, and an Objective Metric,” in IEEE Transactions on Image Processing, vol. 31, pp. 2279–2294, 2022, doi: 10.1109/TIP.2022.3154588.
Well now what?
The work is still currently going on. I cannot say much about it right now but here is a little bit sneak peek:
With that It concludes my blog of summer internship in IIT Jammu, where I not only learnt but also got exposure to new technologies. Truly a good and refreshing experience.
