Hello guest, if you like this forum, why don't you register? https://forum.fanres.com/member.php?action=register (December 14, 2021) x

The importance of making "digital intermediate" files

5 Replies, 5458 Views

When I run out of HDD space, usually I don't do the so-called "digital intermediate" (read: lossless version of the project that will be converted in a lossy version); this let you save A LOT of space, but if something will go wrong (read: you discover some wrong frames, blackouts etc.) it will take A LOT more time to do a unprocessed->processing->encoding task, instead of lossless-processed->encoding...

Plus, if you have many processing filters, it's a wise choice to do several temporary intermediates, because it could save a lot of time...

Example (purely hypotetical, but reflects actual experience):

source->color regrade+denoise+upscale->encoding to final lossy version = 13 hours

source->color regrade->lossless intermediate 1 = 1 hour
lossless intermediate 1->denoise->lossless intermediate 2 = 2 hours
lossless intermediate 2->upscale->lossless intermediate 3 = 3 hours
lossless intermediate 3->encoding to final lossy version = 2 hours

a LOT of HDD space, that will be regained once the project is ready, but also a lot of spared time, in particular when you use a "vintage" PC like I do... of course, in newer PCs if you spare few minutes it doesn't worth the hassle...

What is your experience?
Sadly my projects are lost due to an HDD crash... Sad
Fundamental Collection | Vimeo channel | My blog
I'm not sure about the prices in Italy, but in the USA, HDD prices are fairly reasonable, so I don't mind using lossless digital intermediates for some of my projects.

They act kind of like bookmarks in case I want to go back and do something over. Very convenient and flexible.
Discovered right now two things:

- converting a YUY2 lossless capture (HuffYUV) to a YV12 lossless file (Lagarith), including crops and trimmings, does not speed up the process chain
- converting a YV12 lossy file (AVC inside MKV) to a YV12 lossless file (Lagarith), untouched, does not speed up the process chain if the former is indexed (this time I used FFvideosource, but I bet it's the same with MPEG-2 and DGIndexed files)

So, apart intermediate steps path, that is always worthwhile, processing one or more sources, lossy, AND indexed, with basically just one heavy avisynth filter, would take the same time as using lossless versions of those file(s); more, you would spend some time for the lossless conversion, so I think it's faster to use just the indexed lossy files as sources.

Sadly my projects are lost due to an HDD crash... Sad
Fundamental Collection | Vimeo channel | My blog
This may all change when you upgrade your PC spoRV... Big Grin

A lot of it is down to whether the codec supports multiple threads and what is being encoded/decoded. I do know that using UT Video as a 'final' (for x264 encoding) is a lot quicker than lags or huff (but takes more space). I've heard that magicYUV is pretty fast too but haven't checked it out yet.
I used both lagarith and MagicYUV, and speed is comparable; don't know if UTvideo is much faster but lagarith is multithreading.

Speaking about new PC... Big Grin Big Grin Big Grin just wait some minutes for my new thread about this... and DO NOT forget The Matrix laserdisc capture! Wink
Sadly my projects are lost due to an HDD crash... Sad
Fundamental Collection | Vimeo channel | My blog
It depends on the software and workflow you are using.
I do most, but not all, of my video editing with Sony Vegas which has no problems with indexing any kind of lossless or lossy stream I throw at it, EXCEPT avc-encoded video - regardless of container and, if I start splitting the video to apply different filters for different portions - vegas goes stupid and start showing me incorrect frames (even during rendering)... In that case, the lossless file is the way for me to go...

Possibly Related Threads…
Thread Author Replies Views Last Post
  [Help] Dolby Surround 2.0 Matrixed files Upmix CSchmidlapp 67 27,754 2020-08-06, 11:24 PM
Last Post: allldu
  Any way to watch VUDU mp4 files in MKV format? Jetrell Fo 2 3,395 2017-12-18, 10:57 PM
Last Post: Jetrell Fo
  [SOLVED] From Premiere to uncompressed AVI intermediate Evit 27 15,005 2017-01-21, 04:53 AM
Last Post: PDB
  Bluray playlists - making composite cuts Colek 5 4,695 2016-02-23, 12:52 AM
Last Post: scotlandman
  Need help joining files by muxing. DoomBot 3 4,944 2015-05-26, 05:52 AM
Last Post: DoomBot

Users browsing this thread: 1 Guest(s)