Existing systems for searching and analyzing traffic surveillance footage often rely on predefined event detection methods and lack the ability to interact with users through natural language. The main goal of this thesis is to create such system by utilizing multimodal large language models (MLLMs) and related technologies, namely multimodal embedding models, both of which have seen rapid advancements in recent years. The system is designed to support multiple models of both types, enabling flexible integration and comparison. Furthermore, the available models are benchmarked specifically in the domain of traffic footage to evaluate their performance and suitability for practical deployment.
- thesis.pdf - The main text of the thesis (technical report).
- /app - The application (the implemented system itself). The folder contains its own README.md.
- /model_benchmarks - The performed benchmarks of the multimodal embedding models and the MLLMs. The folder contains its own README.md.
- /text - The Latex source code of the thesis' text, including figures etc.
- /excel_at_fit - Materials used to present the work at the Excel@FIT conference (poster, short paper, video, preview).