As the potential for autonomous vehicles to be integrated on a large scale into modern traffic systems continues to grow, ensuring safe navigation in dynamic environments is crucial for smooth integration. To guarantee safety and prevent collisions, autonomous vehicles must be capable of accurately predicting the trajectories of surrounding traffic agents. Over the past decade, significant efforts from both academia and industry have been dedicated to designing solutions for precise trajectory forecasting. These efforts have produced a diverse range of approaches, raising questions about the differences between these methods and whether trajectory prediction challenges have been fully addressed. This paper reviews a substantial portion of recent trajectory prediction methods proposing a taxonomy to classify existing solutions. A general overview of the prediction pipeline is also provided, covering input and output modalities, modeling features, and prediction paradigms existing in the literature. In addition, the paper discusses active research areas within trajectory prediction, addresses the posed research questions, and highlights the remaining research gaps and challenges.

Nadya Abdel Madjid, Abdulrahman Ahmad, Murad Mebrahtu, Yousef Babaa, Abdelmoamen Nasser, Sumbal Malik, Bilal Hassan, Naoufel Werghi, Jorge Dias, Majid Khonji (2025). “Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions.” Information Fusion (IF). preprint.

This paper addresses limitations in 3D tracking-by-detection methods, particularly in identifying legitimate trajectories and reducing state estimation drift in Kalman filters. Existing methods often use threshold-based filtering for detection scores, which can fail for distant and occluded objects, leading to false positives. To tackle this, we propose a novel track validity mechanism and multi-stage observational gating process, significantly reducing ghost tracks and enhancing tracking performance. Our method achieves a 29.47% improvement in Multi-Object Tracking Accuracy (MOTA) on the KITTI validation dataset with the Second detector. Additionally, a refined Kalman filter term reduces localization noise, improving higher-order tracking accuracy (HOTA) by 4.8%. The online framework, RobMOT, outperforms state-of-the-art methods across multiple detectors, with HOTA improvements of up to 3.92% on the KITTI testing dataset and 8.7% on the validation dataset, while achieving low identity switch scores. RobMOT excels in challenging scenarios, tracking distant objects and prolonged occlusions, with a 1.77% MOTA improvement on the Waymo Open dataset, and operates at a remarkable 3221 FPS on a single CPU, proving its efficiency for real-time multi-object tracking.

Mohamed Nagy, Naoufel Werghi, Bilal Hassan, Jorge Dias, Majid Khonji (2025). RobMOT: Enhancing 3D Multi-Object Tracking Through Observational Noise and State Estimation Drift Mitigation in LiDAR Point Clouds. IEEE Transactions on Intelligent Transportation Systems (T-ITS). preprint

Risk-bounded motion planning for autonomous driving in dynamic environments presents significant research challenges. Ensuring continuous navigation towards a destination while making real-time decisions is a nonconvex problem. This paper presents a graph-based local planning method constrained by user-specific driving preference, represented as a risk-bound criterion for motion planning. First, we propose a lattice graph construction method that adheres to the vehicle’s curvature constraints. Then, we formulate the trajectory planning problem as an integer-linear programming task, addressed by our novel risk-bounded and prediction-aware constrained shortest path. Our solution accounts for both static and dynamic obstacles in urban settings, adhering to traffic regulations. At the core of our approach is a conservative spatiotemporal risk assessment mechanism, which evaluates collisions considering the uncertain delay from speed control of the ego vehicle and predicted trajectories of dynamic obstacles. We implemented our solution using the CARLA simulator and the ROS2 platform, within a comprehensive framework encompassing global planning, local planning, and vehicle control. The effectiveness of our approach is demonstrated through notable collision avoidance, improved path-tracking, and enhanced risk-bounded planning capabilities.

Abdulrahman Hamdy, Majid Khonji, Khaled Elbassioni, Jorge Dias, Ameena Al Sumaiti (2025). “Online Risk-Bounded Graph-Based Local Planning for Autonomous Driving with Theoretical Guarantees.” IEEE International Conference on Robotics and Automation (ICRA). Atlanta, US.

Bilal Hassan, Nadya Abdel Madjid, Fatima Kashwani, Mohamad Alansari, Majid Khonji, Jorge Dias (2024). “A Transformer-Based Framework for Vision-Centric Autonomous Navigation in Off-Road Environments.” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Abu Dhabi. UAE.

Fatima Kashwani, Bilal Hassan, Peng-Yong Kong, Majid Khonji, Jorge Dias (2024). “Evaluation of Predictive Display for Teleoperated Driving using CARLA Simulator.” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Abu Dhabi. UAE.

Please participate in our exciting challenge focused on detecting road agents, events, atomic activity recognition by identifying labels (region_start -> region_end, agent).

We’ve expanded our ROAD++ dataset to include ROAD-Waymo, ROAD-UAE, and TACO allowing participants to explore novel scenarios and domain generalization using videos from three countries: the UK, US, and UAE.

Important links:
ROAD agent/event Dataset GitHub repo: https://lnkd.in/eGCWcRCC
ROAD Baseline GitHub repo: https://lnkd.in/eRsbXHPk
Activity Recognition GitHub repo: https://lnkd.in/ejmk2_NR
Eval AI challenge: https://lnkd.in/ewBbpr8z

Important Dates:
·       Registration open: June 03 2024
·       Training and validation data: June 03 2024
·       Test date: July 15 2024
·       Submission deadline: Aug 25 2024
·       Results: Aug 31 2024

Fabio Cuzzolin Salman Khan Reza Javanmard Alitappeh Eleonora Giunchiglia Izzeddin Teeti Mihaela Catalina Stoian Andrew Bradley Gurkirt singh Jorge Dias Nadya Abdel Madjid Majid Khonji Bilal Hassan Hank Kung Yi-Hsuan Tsai Yi-Ting Chen

Navigating autonomous vehicles efficiently across unstructured and off-road terrains remains a formidable challenge, often requiring intricate mapping or multi-step pipelines. However, these conventional approaches struggle to adapt to dynamic environments. This paper presents TerrainSense, an end-to-end framework that overcomes these limitations. By utilizing a transformers, TerrainSense detects lane semantics and topology from camera images, enabling mapless path planning without the reliance on highly detailed maps. The efficacy of TerrainSense was rigorously assessed on six diverse datasets, evaluating its efficacy in detection, segmentation, and path prediction using various metrics. Notably, it outperforms the other state-of-the-art methods by 9.32\% in precisely predicting the path with 18.28\% faster inference time.

Bilal Hassan, Arjun Sharma, Nadya Abdel Madjid, Majid Khonji, Jorge Dias. “TerrainSense: Vision-Driven Mapless Navigation for Unstructured Off-Road Environments. ICRA 2024.” Yokohama. Japan.

1. “Transformer-Based Multi-Modal Probabilistic Pedestrian Prediction for Risk-Aware Autonomous Vehicle Navigation”
   • Authors: Murad Mebrahtu, Awet Araia, Abiel Ghebreslasie, Jorge Dias, and Majid Khonji.
2. “Multi-Target Tracker for Low Light Vision”
   • Authors: Nadya Abdel Madjid, Arjun Sharma, Bilal Hassan, Naoufel Werghi, Jorge Dias, and Majid Khonji.
3. “Collision-Free Path Generation for Teleoperation of Unmanned Vehicles”
   • Authors: Fatima Kashwani, Bilal Hassan, Majid Khonji, and Jorge Dias.

For more details, visit the conference website: IEEE ICAR 2023.

The Khalifa University’s Autonomous Vehicle Lab (AV-Lab) team was led by Dr. Majid Khonji, Assistant Professor of Electrical and Computer Science, and included the following team members:

• Riyadh Alkharrat, Research Assistant
• Arjun Sharma, Research Associate
• Murad Mebrahtu, Research Engineer

The fixed-horizon constrained Markov Decision Process (C-MDP) is a well-known model for planning in stochastic environments under operating constraints. Chance-Constrained MDP (CC-MDP) is a variant that allows bounding the probability of constraint violation, which is desired in many safety-critical applications. CC-MDP can also model a class of MDPs, called Stochastic Shortest Path (SSP), under dead-ends, where there is a trade-off between the probability-to-goal and cost-to-goal. This work studies the structure of (C)C-MDP, particularly an important variant that involves local transition. In this variant, the state reachability exhibits a certain degree of locality and independence from the remaining states. More precisely, the number of states, at a given time, that share some reachable future states is always constant. (C)C-MDP under local transition is NP-Hard even for a planning horizon of two. In this work, we propose a fully polynomial-time approximation scheme for (C)C-MDP that computes (near) optimal deterministic policies. Such an algorithm is among the best approximation algorithm attainable in theory and gives insights into the approximability of constrained MDP and its variants.

Majid Khonji (2023). “A Fully Polynomial Time Approximation Scheme for Constrained MDPs under Local Transitions.” IEEE Conference on Decision and Control (CDC), Singapore.