I am trying to finetune the model on my ecg dataset. while creating the setup, I am facing a problem of not having projector.bin file, which is required to connect vision encoder with language model. extract_mm_projector.py file is creating a bin file, but when used that file in script, it raises error of having empty weights and keys. what is probable reason for the issue and how to solve this?
I am trying to finetune the model on my ecg dataset. while creating the setup, I am facing a problem of not having projector.bin file, which is required to connect vision encoder with language model. extract_mm_projector.py file is creating a bin file, but when used that file in script, it raises error of having empty weights and keys. what is probable reason for the issue and how to solve this?