Shehan Munasinghe

I am a first-year Master’s student at MBZUAI, UAE, currently pursuing an MSc (research-based) in Computer Vision, under the supervision of Dr Salman Khan at the the Intelligent Visual Analytics Lab (IVAL).

I completed my undergrad at the Department of Electronic and Telecommunication Engineering, University of Moratuwa, Sri Lanka. There I worked on dynamic object detection, tracking and trajectory prediction for autonomous driving, advised by Dr Peshala Jayasekara and Dr Ranga Rodrigo.

My research interests are video understanding and large multimodal models.

Email  /  CV  /  LinkedIn  /  Google Scholar  /  Twitter  /  GitHub

Publications

PG-Video-LLaVA: Pixel Grounding Large Video-Language Models

Extends recent advances in images-based LLMs to video understanding by incorporating audio transcripts to support video context understanding, while introducing a baseline framework and benchmark for conversation-based spatial grounding.

Class-Aware Attention for Multimodal Trajectory Prediction

Presents a novel model architecture for multimodal trajectory prediction in autonomous driving, that takes the physical properties of the target and surrounding vehicles into account through a weighted attention module. Achieved the highest results out of the models which use rasterized maps to encode environment information in the NuScenes benchmark.

A Novel Transfer Learning-Based Approach for Screening Pre-existing Heart Diseases Using Synchronized ECG Signals and Heart Sounds

Introduces a novel Dual-Convolutional Neural Network based approach which uses transfer learning to tackle the problem of having limited amounts of simultaneous PCG and ECG data that is publicly available.

Other Projects

Multi-Sensor Based Dynamic Object Detection, Tracking & Trajectory Prediction for Self-Driving

Worked on developing machine learning models for detection, tracking and trajectory prediction of dynamic agents in autonomous-driving, and integrating them with Robot Operating System (ROS).

Experience

MBZUAI (Mohamed Bin Zayed University of Artificial Intelligence), Abu Dhabi, UAE

Conducted research on explainability and visualization methods for multimodal and vision transformer models.

Creative Software, Sri Lanka

Implemented solutions for semantic segmentation and object detection in corrosion and industrial object identification. Pioneered the creation of synthetic data through the Unity 3D game engine for enhanced model training.

Education

MBZUAI (Mohamed Bin Zayed University of Artificial Intelligence), Abu Dhabi, UAE

University of Moratuwa, Sri Lanka

GPA : 3.96/4.2 (First Class)
Dean’s List : Semester 1,2,3,4,6,7,8

Design from Jon Barron and Leo Keselman