best format to publish raw video?

Started by favvomannen, August 08, 2017, 04:08:01 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

favvomannen

Paul, so what i can take from this is that the colour degradation from 14 to 8 bit via working in a 8 bit environment in ae is p
retty amazing. my problem was never that i thought i had bad colors. in fact im 110% satisfied  of the color acr gives me. my question in the topic refered more to compression issues of avi to .mp4.

having worked with many 8bit cameras where sky gets banding from the slightest change to the image. the 14bit raw files in a 8bit workspace never get banding in aftereffects and typical 8bit problems like breaking during cc. WHY IS THAT? :o


reddeercity

It how you are adjusting the camera raw in ACR  and the compression used .
AVI is not a good codec to use in a raw workflow , as normally it's rgb 24bit(8bit per channel RGB --millions of colors) default windows
Unless you are using Blackmagic codec r210 (10bit YUV 4:2:2 lossless compressed --billions of colors)  or AJA codec's  r10k (10bit RGB Uncompressed)
The other problem with AVI also they very chunky 1500 Mb/s and up  :( where as QT Codec e.g. ProRes are far less chunky for the same file 260Mb/s )
I test a avi codec blackmagic 10bit v210 & windows umcompressed avi @8bit and there's no differents in size but in quality there is (billion of colors to millions of color)

M28-1750_1.avi    BlackMagic 10bit
Format                                   : AVI
Format/Info                              : Audio Video Interleave
Format profile                           : OpenDML
File size                                : 5.78 GiB
Duration                                 : 27s 736ms
Overall bit rate                         : 1 790 Mbps
Recorded date                            : 2017-08-14T00:39:30.00683-06:00
Writing application                      : Adobe After Effects CS6 (Windows)

Video
ID                                       : 0
Format                                   : r210
Codec ID                                 : r210
Duration                                 : 27s 736ms
Bit rate                                 : 1 790 Mbps
Width                                    : 2 144 pixels
Height                                   : 1 072 pixels
Display aspect ratio                     : 2.000
Frame rate                               : 23.976 fps
Bits/(Pixel*Frame)                       : 32.478
Time code of first frame                 : 00:00:00:00 / 00:00:00:00
Time code source                         : Adobe tc_A / Adobe tc_O
Stream size                              : 5.78 GiB (100%)


M28-1750.avi  Windows Uncompressed 8bit
Format                                   : AVI
Format/Info                              : Audio Video Interleave
Format profile                           : OpenDML
File size                                : 5.69 GiB
Duration                                 : 27s 736ms
Overall bit rate                         : 1 763 Mbps
Recorded date                            : 2017-08-14T00:26:33.00123-06:00
Writing application                      : Adobe After Effects CS6 (Windows)

Video
ID                                       : 0
Format                                   : RGBA
Codec ID                                 : 0x00000000
Codec ID/Info                            : Basic Windows bitmap format. 1, 4 and 8 bpp versions are palettised. 16, 24 and 32bpp contain raw RGB samples
Duration                                 : 27s 736ms
Bit rate                                 : 1 763 Mbps
Width                                    : 2 144 pixels
Height                                   : 1 072 pixels
Display aspect ratio                     : 2.000
Frame rate                               : 23.976 fps
Bit depth                                : 8 bits
Bits/(Pixel*Frame)                       : 32.000
Time code of first frame                 : 00:00:00:00 / 00:00:00:00
Time code source                         : Adobe tc_A / Adobe tc_O
Stream size                              : 5.69 GiB (100%)


and for the same quality but with less data rate QT ProRes4444 (10bit windows version FFmpeg)

M28-1750_2.mov
Format                                   : MPEG-4
Format profile                           : QuickTime
Codec ID                                 : qt 
File size                                : 742 MiB
Duration                                 : 27s 736ms
Overall bit rate mode                    : Variable
Overall bit rate                         : 225 Mbps
Encoded date                             : UTC 2017-08-14 07:52:24
Tagged date                              : UTC 2017-08-14 07:01:01
Writing library                          : Apple QuickTime
©TIM                                     : 00:00:00:00
©TSC                                     : 2997
©TSZ                                     : 125

Video
ID                                       : 1
Format                                   : ProRes
Format version                           : Version 0
Format profile                           : 4444
Codec ID                                 : ap4h
Duration                                 : 27s 736ms
Bit rate mode                            : Variable
Bit rate                                 : 225 Mbps
Width                                    : 2 144 pixels
Clean aperture width                     : 2 144 pixels
Height                                   : 1 072 pixels
Clean aperture height                    : 1 072 pixels
Display aspect ratio                     : 2.000
Clean aperture display aspect ratio      : 2.000
Frame rate mode                          : Constant
Frame rate                               : 23.976 fps
Chroma subsampling                       : 4:4:4
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 4.075
Stream size                              : 742 MiB (100%)
Writing library                          : Mrzn
Language                                 : English
Encoded date                             : UTC 2017-08-14 07:52:24
Tagged date                              : UTC 2017-08-14 08:01:01
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Other
ID                                       : 2
Type                                     : Time code
Format                                   : QuickTime TC
Duration                                 : 27s 736ms
Time code of first frame                 : 00:00:00:00
Time code, striped                       : Yes
Language                                 : English
Encoded date                             : UTC 2017-08-14 08:01:01
Tagged date                              : UTC 2017-08-14 08:01:01

The original MLV files was 1.8GB and 8bit AVI blow it up to 5.7GB & 10bit 5.8GB were the ProRes is only 742MB !

Post your XMP file from ACR , that would help see where the problem is or even upload your test mlv file and I can check in my raw workflow with A.E.acr & MLVProducer
and then I'll report back.

favvomannen

yeah i do most colorcorecting in acr. so i just do minor cc in ae. cool, so my avi workflow is not degrading the images, but i agree, a bit clunky editing with uncompressed avi, (as big as the mlv files themselves or even bigger).

im not used to  prores files, and i thought uncompressed avi had better quality than prores, albleit very large files?


anyway,its when i edit these files in premiere pro, that the 200mb/s output mp4 file have a big degradation compared to the uncompressed .avi files used for editing. reedecity,  you seem to have loads of insights, is there a way around degradation from the avi to .mp4? or is it just that way with compression?

(this was my initial question)

1. mlv to dngs
2. import dngs to acr
3. basic cc in acr
4. 8bit composition, minor cc in ae
5. export to uncompressed .avi file
6. edit these files in premiere pro

7. export to 200mbps .mp4 file (here is where i see degradation and quality loss)

Kharak

Take a look at the difference between 8 Bit and 16/32 Bit in Project settings. Film some clouds at Dusk and watch the posterization.

You can even Ctrl+Z and Ctrl+Y the 8 Bit to 16 Bit change, back and forth to see the difference.

Even though your end product will be 8 bit, while you are pulling your colours in 8 Bit you are working with 256 Shades of Gray, you are smashing your colour information before putting it in to a final product. 14 Bit has some 16000 Shades of gray, setting the workspace to 16 bit "dithers" those 16000 to some 20k shades of gray (not sure about this number). You are effectively reducing your color information about 62 times and any fine adjustment wont be very fine, even if you do little adjustments.


And MP4 will always destroy your footage. There is no avoiding it. So will any Internet based platform.

But! Uploading a 10 bit Prores or DNxHD file to Vimeo or Youtube will leave their servers doing the compression giving you a much better final output than ruining your footage with MP4 to then let Vimeo/Youtube compress it again further degrading the image.

@DeafEyeJedi

I work in 16 bit, because Dark Energy plugin does not function properly in 32 bit and for Favvomannen not doing a lot of CC in AE, 32 bit is overkill.
once you go raw you never go back

bpv5P

Quote from: Kharak on August 14, 2017, 09:37:01 PM
Take a look at the difference between 8 Bit and 16/32 Bit in Project settings. Film some clouds at Dusk and watch the posterization.

I'm not advocating to everyone work in 8bit. I'm saying to work in 16bit. The difference between 16bit and 32bit is absolutely minimal and the power necessary to
compute 32bit is much bigger.

Quote
I work in 16 bit, because Dark Energy plugin does not function properly in 32 bit and for Favvomannen not doing a lot of CC in AE, 32 bit is overkill.

It's not even overkill, it's just that there's no difference at all.



@Andy600
I'll be waiting for your tests. This discussion will be ad infinitum if we keep replying each other that way...
Quote
That 0.2% 'shift' in the red spectrum you talk of may actually be the logo of a huge corporation in an advertisement across multiple media outlets.
Crazy world... remember me of Brazil (1985):
https://www.youtube.com/watch?v=K9gO01pyv24

favvomannen

ok so you all know im pretty new at this, does prores contain audio?. when uploading to youtube or vimeo  it sounded like a great idea letting the only compression be vimeo.

but what about audio? or does prores contain audio?


bpv5P

Quote from: favvomannen on August 15, 2017, 07:20:53 AM
does prores contain audio?

Yes.

Quote
when uploading to youtube or vimeo  it sounded like a great idea letting the only compression be vimeo.

No exactly. Each of these streaming websites have their own guideline for encoding for a purpose. Uploading lossless content to these websites is mindless. Not just it will take a longer time to upload, but you'll also waste your own bandwidth and overload their servers.
From what I remember, some of these sites, such as youtube, do not recompress if you follow exactly what is on their guideline, just the DASH content needs to be encoded, for adaptative resolution. Facebook, as another example, do not recompress photographs that's under 100Kb. I don't know their protocol for video, though.

Kharak

@Bpvp52, you obviously have something to prove to yourself regarding cinelog c. For us working with CinemaDNG, cinelog-c workflow gives the highest quality and as a bonus, makes the output flicker free from acr. If you prefer log-c prores from MLVproducer, super. Is the Log-c from mlvp accurate enough for transforming to slog2? Red-log gamma? I doubt it, but let me know. With Cinelog-C you get a standard that can match all the other major DCP/Logs.

I think i misunderstood what this thread is about, i thought it was about delivering the best quality for publishing.

Uploading for youtube guidelines, i dont know if they dont touch it again then, but that will be some blocky ugly looking stuff you are uploading.I personally put a lot of work and detail in to my work and I try to fight to preserve the highest quality throughout and could not care less if Youtube's/vimeo servers are strained because i upload DNxHD, besides I pay for the upload on Vimeo. If your internet cant handle the bandwith, upgrade.
once you go raw you never go back

bpv5P

Quote from: Kharak on August 15, 2017, 09:57:42 AM
@Bpvp52, you obviously have something to prove to yourself regarding cinelog c.

I do? No, I don't.
There's no proof, any tests (that I'm aware of), showing that Cinelog-C is better than free alternatives.
I do not claim MLVProducer is more precise and I'm not affiliated with any people or product. I just can't see how it's different from AlexaLog, since the author itself said Cinelog-C is based on it. Note that Cinelog-C is paid, while AlexaLog is not. I don't want people spreading false information just to sell it's product here.
This place should be a community without astroturfing and without marketing people trying to fool others. I'm not against selling anything here, just say it explicitly.
I'm not saying Andy600 is one of those people (yet). What I'm asking for (on the other thread) is to prove it. And, as I have said in this thread, I'll be one of the folks (like you) that advocate for Cinelog-C if it really proves to have perceptible advantages. I have no problem admitting I was wrong and apologize.

Quote
i dont know if they dont touch it again

I don't too.

Quote
but that will be some blocky ugly looking stuff you are uploading.

Not exactly. 10MB/s bitrate for 1080p is enough if you use the latest x.264 with psy (psy-rdo and psy-trellis) optimization. If they don't re-encode the material, and you're using grain in your footage, you can possibly get better results than letting vimeo/youtube encode (because you can tune for grain). Besides that, they will already brutally compress the footage, it will be "blocky ugly".

Quote
I personally put a lot of work and detail in to my work and I try to fight to preserve the highest quality throughout

I do too, although I can't do most of it the way I want, because my clients want the material ready as fast as possible.
But I'll not do a workflow that instead of during 6 hours to process will take the double of the time, just because of a 0,02% improvement in color. Or, I'll not pay $50 in a DCP that has no proofs that it works better than free alternatives.

Quote
If your internet cant handle the bandwith, upgrade.

Like everyone has money to, or everyone lives in Romania, right?

Kharak

What is your output codec from MLVProducer in Log-c?

For me its not the 0.2% colour difference or possibly more or maybe none for that matter. I personally prefer the ACR route because I think it is, as of this writing moment, the highest quality output possible for MLV e.g. Most DR and the sharpest most "true" representation of what I capture in camera.

My workflow:

Mlv - (as of mow) Danne's Windows batch script - import to AE (ACR) - Apply Cinelog-c, increase signal strenght 1.5 ev - In AE: Upscale to 2K, Noise Reduction and detail enhancement with Neat Video and Dark Energy with in-house noise profiles to every shot, low light or not - Export Log intermediates in Cineform - Edit and grade in resolve.

It takes time for sure, rendering intermediates for a project in AE takes anything between 24 - 48 hours + the NR processing, so about 4 days before i can hand it to my editor (which usually is me). But i think of it like working with Film, only Film would take weeks sometimes months before anyone saw the final image.

I rarely do fast turn arounds, I mostly work in Narrative. When I do, I make sure the client knows I will not deliver in top quality. I render from MLV to prores, used to be via mlrawviewer, but since some months ago prores export wont work for me. But have not done fast turn around for a year so will see what i do next time if ever again.

I've found Andy to be extremely helpful with any technical questions or problems needing solving. Not just sales speeches.

And i can assure you Youtube will compress 10MB/s bitrate, there are not many people with that kind of download speed. I think vimeo recommends 35Mb/s. (Note: Mb Megabit, MB MegaByte)
once you go raw you never go back

bpv5P

Quote from: Kharak on August 15, 2017, 02:05:24 PM
What is your output codec from MLVProducer in Log-c?

DNxHD when I need good quality. H.264 when the time is short.


If you permit me, some comments on your workflow:
Quote
increase signal strenght 1.5 ev

Using a curve slightly flat in highlights gives me better results than linear exposure raise...

Quote
- In AE: Upscale to 2K, Noise Reduction and detail enhancement with Neat Video and Dark Energy with in-house noise profiles to every shot, low light or not - Export Log intermediates in Cineform - Edit and grade in resolve.

Upscale should be one of the last processes, if you want better quality. Noise reduction will be less effective if you do upscale first, because of interpolation artifacts. Also, noise reduction needs to be before sharpening/deconvolution. I personally do in this other (although I don't upscale my footages):

Input > Log conv. in 16bit workspace > Noise reduction > Upscale > Color Grading > Sharpen > Grain

Also, color grading with LUTs is generally a good practice.

Quote
about 4 days before i can hand it to my editor (which usually is me).

I personally need to work with constrained time, so 4 days to just pre-process is too much. But your workflow is good :^)

Quote
I've found Andy to be extremely helpful with any technical questions or problems needing solving. Not just sales speeches.

Yes, I remember Andy600 before even MLV was developed. Actually even from the time of Tragic Lantern 600D hacks.
It's not personal our discussion here. But there's no proof about Cinelog-C.

Kharak

I will try your upscale method.

But I did a lot of testing, basically a year before i settled for this method. I call it AIM (Alexa Imperia Method). Alexa, because I am in love with its image and I try always to emulate it, Imperia, because anything I find pride in I call Imperia, like my Legendary Pasta sauce 'Pasta Imperia'™© ;). Method, well you know.

If I recall correctly, i got better results with the upscaling before the noise reduction, basically increasing the Canvas for Neat Video and Dark Energy to work with. Bicubic upscaling ofcourse. Forgot to mention adding texture with Dark Energy Matter right after the NR. Two Adjustment layers, 1. NV. 2. DE Anti-Matter, DE Matter.

The NV with DE is what hogs the performance down. If only one of them were applied it would cut the rendering time down to less than half. But I am a Quality whore, a blessing and a curse.

The 3x3 Bayer of the 5DIII Raw holds a lot more sharpness to it than most people think, if done right.

Also forgot to mention in AE i also apply a 3rd. Adjustment layer for Opencolor IO for transforming Cinelog-C to Log-C or any other log if required. Often Slog 123 for a production company I often work with.
once you go raw you never go back

favvomannen

do you use nv and dark energy noice reducer even for bright lit scenes? noise only for dark scenes right?

has anyone any example on a clip with dark energy texture vs same clip whout texture. im not so fond of grain added by filmconvert. real filmgrain is much more scene dependant and organic.

sry for being a tad ot, but all this talk about dark energy plugin got me curious.


DeafEyeJedi

Quote from: favvomannen on August 15, 2017, 09:34:56 PM
...im not so fond of grain added by filmconvert. real filmgrain is much more scene dependant and organic.

The caveat here w FilmConvert is the fact that 'film grain' is by default @ 100% (not sure why tho) so perhaps the trick is to try starting from 0% and upward to your own taste.

Also there are reasons why it's rather more ideal to work in 32-bit float for those that are either curious or oppose to it. Looking forward to @Andy600's next major update w his products in Cinelog DCP which should finally pull the trigger for those who haven't purchased already.
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

PaulHarwood856

Neat video has an option to use a percentage of the original footage noise and Neat Video's noise reduction. It's called Mix with Original. Raw video from Canon DSLRs have filmic grain, so I really like this method. I do noise reduction at the end in Premiere Pro with ProRes 4444 XQ transcoded clips with Cinelog-C.

hyalinejim

Quote from: bpv5P on August 15, 2017, 01:00:30 PM

There's no proof, any tests (that I'm aware of), showing that Cinelog-C is better than free alternatives.
I do not claim MLVProducer is more precise and I'm not affiliated with any people or product. I just can't see how it's different from AlexaLog, since the author itself said Cinelog-C is based on it. Note that Cinelog-C is paid, while AlexaLog is not. I don't want people spreading false information just to sell it's product here.
This place should be a community without astroturfing and without marketing people trying to fool others. I'm not against selling anything here, just say it explicitly.
I'm not saying Andy600 is one of those people (yet). What I'm asking for (on the other thread) is to prove it. And, as I have said in this thread, I'll be one of the folks (like you) that advocate for Cinelog-C if it really proves to have perceptible advantages. I have no problem admitting I was wrong and apologize.

Well, why not buy it and try it?

Athough we may not have posted the results of the tests, there are reasons why numerous ML users for years now have been advising that Cinelog through ACR offers the best image quality available of all the post workflow options.

These have to do with dynamic range preservation, noise suppression and reproduction of pleasing colour.

But why should we test this for you? If you're not willng to cough up the money to do your own tests, then please stop banging on about it.

Andy600

Thanks for your endorsements guys. I appreciate your support for Cinelog but we have pulled this post far away from @favvomannen's original question so I ask everyone to keep the discussion on topic from here onwards.

I will present the MLVP vs Cinelog DCP test results in the Cinelog thread when ready.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com