I found out it only happens when I try to use H264 (NVENC), regular H264 it runs just fine despite the framerate.
I've been trying to update my graphics drive but the H264 (NVENC) just ends up disappearing from the dropdown list.
Update: I managed to get the option back to the drop down but it still has the same problem for NVENC