The importance of making "digital intermediate" files

[spoRv]

When I run out of HDD space, usually I don't do the so-called "digital intermediate" (read: lossless version of the project that will be converted in a lossy version); this let you save A LOT of space, but if something will go wrong (read: you discover some wrong frames, blackouts etc.) it will take A LOT more time to do a unprocessed->processing->encoding task, instead of lossless-processed->encoding...


Plus, if you have many processing filters, it's a wise choice to do several temporary intermediates, because it could save a lot of time...

Example (purely hypotetical, but reflects actual experience):

source->color regrade+denoise+upscale->encoding to final lossy version = 13 hours

source->color regrade->lossless intermediate 1 = 1 hour
lossless intermediate 1->denoise->lossless intermediate 2 = 2 hours
lossless intermediate 2->upscale->lossless intermediate 3 = 3 hours
lossless intermediate 3->encoding to final lossy version = 2 hours

a LOT of HDD space, that will be regained once the project is ready, but also a lot of spared time, in particular when you use a "vintage" PC like I do... of course, in newer PCs if you spare few minutes it doesn't worth the hassle...

What is your experience?

I'm not sure about the prices in Italy, but in the USA, HDD prices are fairly reasonable, so I don't mind using lossless digital intermediates for some of my projects.

They act kind of like bookmarks in case I want to go back and do something over. Very convenient and flexible.

[spoRv]

Discovered right now two things:

- converting a YUY2 lossless capture (HuffYUV) to a YV12 lossless file (Lagarith), including crops and trimmings, does not speed up the process chain
- converting a YV12 lossy file (AVC inside MKV) to a YV12 lossless file (Lagarith), untouched, does not speed up the process chain if the former is indexed (this time I used FFvideosource, but I bet it's the same with MPEG-2 and DGIndexed files)

So, apart intermediate steps path, that is always worthwhile, processing one or more sources, lossy, AND indexed, with basically just one heavy avisynth filter, would take the same time as using lossless versions of those file(s); more, you would spend some time for the lossless conversion, so I think it's faster to use just the indexed lossy files as sources.

Opinions?

[zoidberg]

This may all change when you upgrade your PC spoRV...

A lot of it is down to whether the codec supports multiple threads and what is being encoded/decoded. I do know that using UT Video as a 'final' (for x264 encoding) is a lot quicker than lags or huff (but takes more space). I've heard that magicYUV is pretty fast too but haven't checked it out yet.

[spoRv]

I used both lagarith and MagicYUV, and speed is comparable; don't know if UTvideo is much faster but lagarith is multithreading.

Speaking about new PC... just wait some minutes for my new thread about this... and DO NOT forget The Matrix laserdisc capture!

[jerryshadoe]

It depends on the software and workflow you are using.
I do most, but not all, of my video editing with Sony Vegas which has no problems with indexing any kind of lossless or lossy stream I throw at it, EXCEPT avc-encoded video - regardless of container and, if I start splitting the video to apply different filters for different portions - vegas goes stupid and start showing me incorrect frames (even during rendering)... In that case, the lossless file is the way for me to go...