Memory leak?

Graphics card memory doesn't seem to be released after encoding with VoukoderPro. Up to and including v0.7.2.6.

I have GPU globally disabled in Vegas 20 (because it slows rendering down - it's a Vegas thing!).

Typically I'm running 2 instances of Vegas in parallel, and rendering in both simultaneously using NVENC through VoukoderPro. My first pair of VoPro renders usually use about 3 GB of VRAM. When the renders end, the memory is not released. If I run a second set of renders, now usage is up to 6 GB. If I run a third set of renders, now usage is 9 GB (paged, my current card is 8 GB). And so on.

This behavior has not been observed with other render methods, including Voukoder 13.0.2. These other methods release the video memory as soon as the render is stopped/finished.

The memory is released if I restart Vegas.

I'm afraid it isn't fixed in 0.7.2.7. Maybe improved, but still an issue.

Sequence of actions starting and stopping the same render job, with VRAM usage (8 GB card):

• Windows idle: 0.4 GB
• Loaded 2 instances of Vegas 20: 1.8 GB
• During 1st render: 3.6 GB
• After 1st render: 3.0 GB
• During 2nd render: 4.8 GB
• After 2nd render: 4.1 GB
• During 3rd render: 5.9 GB
• After 3rd render: 5.3 GB
• During 4th render: 7.1 GB
• After 4th render: 6.4 GB
• During 5th render: 7.8 GB (paging)
• After 5th render: 7.2 GB (possibly paging)
• During 6th render: 7.8 GB (page 10 GB)
• After 6th render: 7.2 GB (paging)
• During 7th render: 7.7 GB (page 12.x GB)
• After 7th render: 7.3 GB (page 11.3 GB)

Screenshot taken partway through the test run:

No change in version 0.7.2.8. Tried just a single render, and the problem exists there too.

I tried to see if there was an upper limit to the memory usage. Eventually VoukoderPro does refuse to run, displays error "Unable to start VoukoderPro: Undefined error!" Screenshot below is taken at this point, after starting and stopping the same render many times. As you can see, the total video memory usage has grown to over 35 GB.

The last few lines of the log file at the time of the crash are:

2023-08-29 17:39:39 (trace)    [FFmpeg:0] Loaded Nvenc version 12.1
2023-08-29 17:39:39 (trace)    [FFmpeg:0] Nvenc initialized successfully
2023-08-29 17:39:39 (trace)    [FFmpeg:0] Lookahead enabled: depth 20, scenecut enabled, B-adapt enabled.
2023-08-29 17:39:39 (trace)    [FFmpeg:0] Calling dl_fn->cuda_dl->cuCtxPushCurrent(ctx->cu_context)
2023-08-29 17:39:39 (trace)    [FFmpeg:0] Calling dl_fn->cuda_dl->cuCtxPopCurrent(&dummy)
2023-08-29 17:39:39 (trace)    [FFmpeg:0] InitializeEncoder failed: generic error (20): EncodeAPI Internal Error.
2023-08-29 17:39:39 (trace)    [FFmpeg:0] Calling dl_fn->cuda_dl->cuCtxPushCurrent(ctx->cu_context)
2023-08-29 17:39:39 (trace)    [FFmpeg:0] Calling dl_fn->cuda_dl->cuCtxPopCurrent(&dummy)
2023-08-29 17:39:39 (trace)    [FFmpeg:0] Nvenc unloaded
2023-08-29 17:39:39 (error)    [EncoderNode.cpp:260] Opening the encoder plugin failed with error code: -1313558101
2023-08-29 17:39:39 (error)    [Router.cpp:274] Open phase failed: -1313558101
2023-08-29 17:39:39 (info)    [VoukoderPro.cpp:502] Unable to start VoukoderPro: Undefined error!

And here's an example scene that causes the issue:

temp v0.1.scene.zip

Zitat von Vouk

I was not aware people run parallel instance of it

In a multi-socket server/workstation system, it's normal to have multiple jobs running, each confined to its own NUMA node. Likewise if you're running multiple hardware encoders, each with different render projects. This is part of why VoukoderPro is such a big deal.

To be clear, the memory leak issue is occurring even with a single render in a single instance of Vegas. Simply starting and stopping the same render over and over will do this, as in Post #5 above. Leak still present in version 0.7.2.9.

That's odd. I'm using Win 10, Nvidia drivers 536.99 patched with keylase, and Vegas 20 build 411. Pretty vanilla system.

Are you testing using the Scene from post #5, above?

I'm still testing, trying to pin down exactly what is causing the issue.

What I'm seeing so far is that when the following chain is used, there is no memory leak: Video input -> NVENC encoder (either h.264 or HEVC).

works.scene.zip

However, when the following chain is used, the memory leak exists: Video input -> CUDA upload -> NVENC encoder (either h.264 or HEVC).

doesn't work.scene.zip

No apparent memory leaks if I go Upload -> [various CUDA filters] -> Download -> [software encoder]. Some of the filters just plain didn't work (FFmpeg error), but that could be operator error.

The only other possibly-related unusual behavior I've noticed so far is with the CUDA Thumbnail filter. Maybe not a leak, but something is getting left behind in GPU memory, then cleared at the start of each render:

Load Vegas, 1.3 GB GPU memory.

During 1st render with Thumbnail filter: 1.8 GB

After 1st render: 1.8 GB

Immediately after starting 2nd render: briefly drops back to 1.3 GB

During 2nd render: 1.8 GB

After 2nd render: 1.8 GB

thumbnail filter.scene.zip

Yes, I have been using YUV 420 8-bit templates exclusively for this.

I tried installing a fresh unpatched driver (537.13) and this made no difference. The patch only relates to how many things you can do at once on a non-Quadro card, so shouldn't have anything to do with anything.

I tried enabling GPU globally in Vegas (which is the default), and this made no difference either.

Every time I start a render, the process allocates ~0.6 GB of VRAM. After the render, ~0.2 GB is released, and the remaining ~0.4 GB stays occupied.

I'll run through the different CUDA filters again when I get time . . . but as I said, that was probably operator error. I was just quickly looking for something CUDA-based that wasn't an encoder. Never checked the output at all, just some of them started to encode and others wouldn't. Chroma Key was one that wouldn't work at all.

Using "doesn't work.scene" from above (post #11). Single encode, single card, single instance of Vegas 20, stock drivers, VoPro 0.7.2.9.

7 runs using normal graph speed in Task Manager:

30 runs using slow graph speed in Task Manager:

It does not occur during Scene Test.

Initial GPU memory used: 0.2 GB

Memory during test: 0.5 GB

Memory after test: 0.2 GB

Memory after 30 consecutive tests: 0.2 GB

Note regarding Vegas: No matter how much GPU memory is used by Vegas/VoPro, all of it is released on exit from Vegas.

Update for version 0.7.4:

• Memory leak does not occur on any of the 3 cards in my system when using simple Video Source -> NVENC Encode scenes.
• Memory leak *does* occur using more complicated Scenes involving Video Source -> CUDA Upload -> NVENC Encode. These Scenes are only possible to test on GPU0, due to the current bug involving duplicate "gpu" assertions in the FFmpeg command.

PS, If you want a quick and dirty way to add more cards to your system for multi-GPU testing, you could grab a couple x1 riser cables and burned-up cards from a starving miner. 😋 The performance won't be great because of the PCIe x1 bandwidth bottleneck, but it'd be good enough for testing.

No memory leaks observed using FFmpeg command line. Tested all 3 GPUs individually.

The following is for GPU1. To change to a different GPU, just change the target of hwupload_cuda. Using AVI file input, which is the closest to what VoPro is doing (no hardware decoding):

ffmpeg -y -probesize 42M -analyzeduration 10 -i "d:\temp\input.avi" -filter_complex:a "[0:a]asplit [asplit1][asplit2]" -filter_complex:v "[0:v]hwupload_cuda=1, split [split1][split2], [split1]scale_cuda=1920:1080:interp_algo=4:format=yuv420p:force_original_aspect_ratio=1 [split1scaled],[split2]scale_cuda=1280:720:interp_algo=4:format=yuv420p:force_original_aspect_ratio=1 [split2scaled]" -map "[asplit1]" -c:a ac3 -b:a 96k -map "[split1scaled]" -c:v h264_nvenc -2pass 0 -b:v 2500k -maxrate 5000k -bufsize 5000k -bluray-compat 1 -coder 1 -cq 0 -g 48 -level 4 -preset:v p7 -profile:v high -rc:v vbr -rc-lookahead 20 -tune:v hq "output_1080v4.9.mp4" -map "[asplit2]" -c:a aac -profile:a aac_main -b:a 96k -map "[split2scaled]" -c:v h264_nvenc -2pass 0 -b_ref_mode:v middle -preset:v p7 -profile:v 2 -qp 30 -rc 0 -rc-lookahead 20 -tune:v hq "output_720v2.1.mp4"