There is no big difference if you encode game videos for YouTube or short films with your DSLR (at least not from the encoding point of view). In fact there are no "best" settings, its depending on your use case. Also what does "best" mean? "best encoding speed", "best quality", smallest file size?
NVENC h264 can be very fast to encode, but that depends on your GPU and CPU. With the RTX generation the resulting quality should be on par with x264.
Only use bitrate based strategies when you aim for a specific file size (i.e. to fit a film on a DVD, USB stick, ...), Otherwise I'd recommend Constant Quantizer (NVENC) or CFR (x264).
Both NVENC and x264 can produce lossless output. Or nearly lossless. It is just a question which one is creating the smaller file size.
Just play with the Constant Quantizer (NVENC) value and CFR (x264). The lesser each value the better the quality (and the larger the file size).
