best format to publish raw video?

[favvomannen]

whenever i export from raw dngs to .mp4 (dual vbr) 150mbps- 200mbps via aftereffects or  premiere pro i can still see the quality degradation.

is there just the way it is?

[PaulHarwood856]

Hey favvomannen,

     Are you on Mac or Windows? I recommend extracting your DNGs (Cinema DNG Lossless is best) and bring into After Effects. Use the Smart Import 2 Script for After Effects (found here on the forum), and this will load all of your clips and sync the audio files. I recommend converting to a flat profile in Adobe Camera Raw (Cinelog-C is the best I've used - you can purchase online for $50), and convert all the sequences into ProRes 4444 XQ with Alpha if on Mac, or GoPro Cineform RGBA 12 Bit if on Windows. Then grade and edit in Premiere Pro. This is called transcoding, and will be easier to edit on your computer and solves a lot of headaches with Adobe dynamic link. I hope this helps, and please let me know if you have any questions.

- Paul

[bpv5P]

Do you want to publish or archive?
1- Publishing = lossy codec
2- Archiving = lossless codec

Good practices:
1- H.264. Alternatively, use VP9 (very slow) or H.265 (not supported for streaming)
2- Lagarith Lossless

For lossy, you should not use the high bitrate you're using, since the algorithm is not meant to be used that way. For reference on bitrate settings, see this youtube page:
https://support.google.com/youtube/answer/1722171?hl=en

For better results, you can encode with Lagarith, then use Handbrake:
https://handbrake.fr/

You can also use a general compression tool for long term archiving, like .7z (LZMA2):
http://7zip.org

As for ProRes or Cineform, as @PaulHarwood856 suggested, you can also them too, but they are proprietary, and ProRes cannot be used outside of Mac OSX platforms.
Another suggestion is to use DNxHR instead of Lagarith, but it's not trully lossless.

[favvomannen]

Neither publish nor archive. Want to give a client a usb with a high quality mp4 for them to play on pc or to tv.

200mbps mp4 dual pass from is a bit irritatingly far away from cinemadngs in terms of quality.

1. will filmgrain help when compressing to mp4 . (keeping the organic feel of raw)?

2. will upscaling to 4k before compressing to +200mbps help maby, or both? (this ive heard is good for publishing on youtube). 4k bitrate higher online.

any other tips?


PS PAUL hartwood. im on win10, i actually have bought  cinelog c, havnt gotten into it yet, but i really like the natural colour of acr with a slight saturation gain for a start. really magical canon colours, i place dng folders in a 8bit sequence in aftereffects then export as a totally uncompressed AVI file straight from ae.

im guessing its alot like a prores 444 file?

then edit in premiere. but the mp4 leaves some to be desired. its good but i still think it could be better. im sitting 2 meters from a 75 inch tv when editing an watching content, maby thats why i see the degradation so well. hmm.

[bpv5P]

1. will filmgrain help when compressing to mp4 . (keeping the organic feel of raw)?

No, it will actually increase the size of your file.

2. will upscaling to 4k before compressing to +200mbps help maby, or both? (this ive heard is good for publishing on youtube). 4k bitrate higher online.

Absolutelly no. Where did you heard it?

any other tips?

Export on Lagarith. Import on Handbrake Nightly build. Encode using 2-pass 30MB/s (no more). Enable Sharpen filter and deblocking in filter tab.
This will produce a smaller file size and the loss of quality will be unperceptible for basically any person (assuming your footage is 1080p - if it's 2-4K, you need higher bitrate; 50MB/s is enough).
Your assumption that a 200MB/s H.264 encoded video is loosing quality is probably placebo (or nocebo).

Also, no need to buy Cinelog-C. The MLVProducer with Alexa-Log does basically de same, and the author could not provide evidence that his DCP is better than free alexa-log implementation.

[PaulHarwood856]

Hello favvomannen,

    I recommend transcoding with Cinelog-C to GoPro Cineform 12 bit before editing in a timeline. That way when the finsl project is exported to mp4 there is less degredstionnin quality.

Hello bpv5P,

     I've used MLVProducer to extract Cinema DNGs (no compression) for one project. It was good, and I still recommend taking Cinema DNGs from MLVProducer and bringing into Adobe Camera Raw through After Effects. Andy600 gave you an explanation about Cinelog-C. I am glad I purchased it, and have no regrets.

- Paul

[favvomannen]

ahh. ive shoulda done that. i like that tip.

can i just aswell make a 16bit sequence in ae and import the dng to that sequence? (to suit my workflow)?  then render to .avi? it should be as good as cineformgopro to edit with?

aftereffects avi is uncompressed an  keeps the bitrate?

[favvomannen]

No, it will actually increase the size of your file.Absolutelly no. Where did you heard it?Export on Lagarith. Import on Handbrake Nightly build. Encode using 2-pass 30MB/s (no more). Enable Sharpen filter and deblocking in filter tab.
This will produce a smaller file size and the loss of quality will be unperceptible for basically any person (assuming your footage is 1080p - if it's 2-4K, you need higher bitrate; 50MB/s is enough).
Your assumption that a 200MB/s H.264 encoded video is loosing quality is probably placebo (or nocebo).

Also, no need to buy Cinelog-C. The MLVProducer with Alexa-Log does basically de same, and the author could not provide evidence that his DCP is better than free alexa-log implementation.

thx, i have my work cut out for me learning handbrake and all these new things. thx for all tips very much.

1. its not placebo, but im sitting very close to a 75inch tv while editing, im pixelpeeping.

2. my "thought" is that 4k downscaled to 1080p is scientifically prooven to increase quality, therore i thought that upscaling to 4k, render in 4k would gain me more bitrate per second  while watching it downscaled on my tv, thus more total quality per second.     

(i will try this and post results)

(but im theorizing here)

[Andy600]

Also, no need to buy Cinelog-C. The MLVProducer with Alexa-Log does basically de same, and the author could not provide evidence that his DCP is better than free alexa-log implementation.

@bpv5P - When have I ever said I 'could not' provide evidence? 

As I said in a previous post, I am happy to publish comparisons but these things take time to do properly and I need to slot it in around my other work and commitments. I will publish comparative test results of Log-C produced in MLVProducer vs Log-C derived using Cinelog DCPs asap so you can see for yourself which is 'better'.

[Kharak]

As soon as you import your footage as 8 bit in AE, the damage is already done.

Set your Project Settings to 16 bit.

[favvomannen]

but is the damage already done really? i never get 8bit banding in sky importing it into a 8bit sequence. colours are also amazing. making me wonder? anyways i dont have a hdr screen.

[DeafEyeJedi]

...Set your Project Settings to 16 bit.

Instead of the recommended 32-bit Float?

[favvomannen]

again i dont get any color banding or other typical 8bit problems when importing dngs to a 8bit timeline in ae. in fact the color is pristine when exported to uncompressed avi.

do i really degrade footage?

i see the degradation after .mp4 export though.

[bpv5P]

I still recommend taking Cinema DNGs from MLVProducer and bringing into Adobe Camera Raw through After Effects.

Why would you do that? CDNG is raw, it will take no effect.

Andy600 gave you an explanation about Cinelog-C. I am glad I purchased it, and have no regrets.

No, he did not. No evidence.

I will publish comparative test results of Log-C produced in MLVProducer vs Log-C derived using Cinelog DCPs asap so you can see for yourself which is 'better'.

Would be really good to see it. Also, public, unaltered DNG's, so we can replicate your test...

[bpv5P]

Instead of the recommended 32-bit Float?

The DNG produced by MLV is only 14bit. 32-bit float is only for HDR images, like the ones produced on HDRMerge or similar software.

[PaulHarwood856]

Hey bpv5P,

     I'm simply giving my suggestions to favvomannen. Your workflow might be different. I personally find it easier to edit (I can still bring back highlights like raw CDNG) a log profile (Cinelog-C is what I use) in a friendly codec. GoPro Cineform RGBA 12 Bit in After Effects will work great for favvomannen. And I've read online working in 32 bit float is beneficial vs 16 bit. Again everyone has their own workflow.

- Paul

Hey favvomannen,

     I would recommend for future projects working in After Effects in 32 bit float. I think you will really like the workflow of GoPro Cineform in Cinelog-C. It's friendly to edit with, you can pull back highlights, and Adobe works well with it. I hope this helps you out, and please let me know if you have any other questions.

- Paul

[bpv5P]

And I've read online working in 32 bit float is beneficial vs 16 bit. Again everyone has their own workflow.

Ok, I don't want to be mean, but this is not religion or philosophy. 32bit, in this specific case, has no value, and this is not a "difference", you're just using it the wrong way.
You can keep doing it, though, I'm not stopping you from anything, I'm just bringing this to your attention.

[Danne]

32bit-float environment is recommended for certain reasons. You can find some of them here:
http://wolfcrow.com/blog/the-advantages-of-working-in-32-bit-float/

[bpv5P]

32bit-float environment is recommended for certain reasons. You can find some of them here:
http://wolfcrow.com/blog/the-advantages-of-working-in-32-bit-float/

Ironically, the article itself agrees with me. Here:

None of this requires anything more than 12-bit maximum. The average professional requirement is 10-bit (or even 8-bit).

The article talks about VFX, since we're talking about 14bit image, I'll ignore that.
Be here we go:

1. Greater precision in calculations and color operations
2. A lot more colors to choose from, and results which are visible on a high-end monitor
3. Allowance for a greater tonal range, which helps in giving footage a better highlight or shadow roll-off, for example, that mimics how tones behave in the real world

You're already working with Red.709. Your color is already limited. 32bit will not allow your to have "a lot more colors to choose". Also, your monitor is already limited too, unless you have professional HDR-prepared monitors and, even with that, most people watch videos on normal monitors.
It will also not give you "better highlight or shadow", because your footage is in 14bit, not an 32bit HDR.

True compatibility with wide gamut color spaces, including the CIE XYZ space for DCI

Cinema works with Rec.709 or new Rec.2020. No one works with ProPhoto RGB.

Here's the best:

Hopefully, in the future, the entire pipeline – from camera to screen – will be in 32-bit (or even 64-bit)!

64-bit? Really? Are you doing astro-cinematography? No? Then don't.

[Andy600]

I think there may be some confusion here about image bit depths, workspace bit depths and Adobe specific architecture.

While the DNG files are 14/12/10bit, ACR can export in 8 or 16bit and processes in half-float space. If you set ACR to 8bits you will always get a lossy image and likely some posterization. This may only become apparent when you start grading and pushing the image around. Always set ACR to 16bits when dealing with raw images and 16bit TIFFs. The setting is accessible when launching ACR from Photoshop or Bridge and is persistent.

Even though your image may be developed and stored or transferred in 8 or 16bits, setting After Effects workspace to 32bit float is important for the operation of some plugins. It is also essential for VFX and avoids clipping the signal path.

Yes, 32bit containers are used for HDR images but, unless I'm missing something, that is not what is being discussed here. Incidentally a 16bitf .EXR is perfectly capable of holding extreme HDR imagery with lossless precision over 30 stops (i.e. 10^9 - well beyond what any camera is capable of and beyond any HDR images you are likely to make) and much more economically than 32bit files but all this is overkill unless you work in VFX and use it for interchange when sharing between artists and studios i.e. an ACES pipeline.

You can also store HDR in lower bit depths through log encoding and done properly you will maintain a relationship with linear light

Why would you do that? CDNG is raw, it will take no effect.

Maybe because he want's to develop the image in ACR!?

No, he did not. No evidence.

Not in this topic but there is plenty of info in the Cinelog topic. As for 'evidence' see below.

Would be really good to see it. Also, public, unaltered DNG's, so we can replicate your test...

Yes, and the source MLVs (I don't think MLVProducer can develop DNG image sequences!?) and of course detailed process information. That's what I mean by doing it properly 


Honestly, with your level of interest I'm beginning to wonder if you are secretly working for us using reverse psychology

[bpv5P]

Honestly, with your level of interest I'm beginning to wonder if you are secretly working for us using reverse psychology

I'm not interested in your product specifically or doing the "devils advocate" here, I just don't want people to spread false information. Remember that people will read this in future, through findings in search engines.
But, if you show us the results of your comparison, and it really has noticiable better DR and color than Alexa-Log, I'll for myself advocate for Cinelog-C and say "I was wrong about your product" here.

[Andy600]

Cinema works with Rec.709 or new Rec.2020. No one works with ProPhoto RGB.

Oh lord  @bpv5P maybe you should step back and do some research before posting such nonsense.

Cinema is not and never has used Rec709 or Rec2020 colorspaces. These are ITU HDTV and UHDTV broadcast standards. The standard Cinema colorspace currently in use is DCI-P3. ProPhoto is not used for broadcast or Cinema but photographers work in ProPhoto all the time.

The article may be about VFX but most of it applies to raw workflows because, in both, you are dealing with scene-referred linear imagery. Even when a raw image is developed, viewed and processed in a display space, maintaining a mathematical connection to linear light (a fundamental of physics), especially where color and apparent realism is is important, is still very desirable.

You're already working with Red.709. Your color is already limited. 32bit will not allow your to have "a lot more colors to choose". Also, your monitor is already limited too, unless you have professional HDR-prepared monitors and, even with that, most people watch videos on normal monitors.
It will also not give you "better highlight or shadow", because your footage is in 14bit, not an 32bit HDR.

This is only relative to images that have been rendered in Rec709. Raw images do not have this limitation and this is why you set your working space bit depth as high as possible to retain all the color and dynamic range of the material. This also applies to properly encoded, perceptually lossless log material.

[Andy600]

I'm not interested in your product specifically or doing the "devils advocate" here, I just don't want people to spread false information. Remember that people will read this in future, through findings in search engines.

Of course. Which is why I try to remain factual, stand by what I say and acknowledge if/when I am wrong.

But, if you show us the results of your comparison, and it really has noticiable better DR and color than Alexa-Log, I'll for myself advocate for Cinelog-C and say "I was wrong about your product" here.

and that I have no problem with

DR is only a one part of it. You should also pay attention to color

[bpv5P]

Hi Andy600.
First, let me assume my error: in fact, cinema does not use Rec.709 as the working space, but for the broadcast. My fault.

But, here we go:

The standard Cinema colorspace currently in use is DCI-P3.

Maybe high-end cinema ("US-American film industry", as defined by technicolor), yes.
ACR does not uses it by default.



ACR needs you to define it for yourself and most people don't do it.

https://helpx.adobe.com/after-effects/using/color-management.html

A quote from the adobe page above:

For best results, when working with 8-bpc color, match the working color space to the output color space. If you are rendering to more than one output color space, you should set the project color depth to 16 bpc or 32 bpc, at least for rendering for final output. The working color space should match the output color space that has the largest gamut. For example, if you plan to output to Adobe RGB and sRGB, then use Adobe RGB as your working color space, because Adobe RGB has a larger gamut and can therefore represent more saturated colors. To preserve over-range values, work in 32-bpc color for its high dynamic range.

Suggestions for working color space choices:

    SDTV NTSC or SDTV PAL is a good choice if you're making a movie for standard-definition broadcast television, including standard-definition DVD.

    HDTV (Rec. 709) is a good choice if you're making a movie for high-definition television. This color space uses the same primaries as sRGB, but it has a larger gamut, so it makes a good working space for many kinds of work.

    ProPhoto RGB with a linear tone response curve (gamma of 1.0) is a good choice for digital cinema work.

    sRGB IEC61966-2.1 is a good choice if you're making a movie for the Web, especially cartoons.

So, this page shows a least some people work on ProPhoto RGB (although I would also not advocate it), you've said "ProPhoto is not used for broadcast or Cinema".

Now to the main point. The same page above says that you "To preserve over-range values, work in 32-bpc color for its high dynamic range.". I do not agree with that. Adobe is probably talking about VFX, and our point here is digital camera RAW files.

This is only relative to images that have been rendered in Rec709. Raw images do not have this limitation and this is why you set your working space bit depth as high as possible to retain all the color and dynamic range of the material.

Most people watch in Rec.709. Displays used today are not capable of reproducing wide-gamut.
The concept of maintaining DR through bit depth seems totally bullshit for me. If you have configured a working color space, you've already limited that (unless you're working with tangential wide gamut, and that's not the case since you've said DCI-P3). Bit-depth precision may calculate more precisely the colors (although it would not be perceptible), but it cannot preserve more luma DR, it's already limited to 14bit, your greys is already there, you can't create this information from nothing.


tl;dr you're talking about an idealistic POV about workflow, but people watch our content on realistic devices, that are not HDR, not precise and most people will give zero shit about a 0,2% shift in your red spectrum on their 4-inches Iphone screens, because most people can't even perceive this. Even on 4k displays you can feel it. I know because I've tested.
You all want to work with 32-float, wide-gamut, galactic-fucking-computer-expensive-workflow, go ahead, I'm not stopping you from anything. But I'll not follow this.

Related meme
]

P.S: Another quote, this time from:
https://forums.creativecow.net/thread/2/1026156

RED footage is interpreted as Rec 709 in After Effects by default. All transformations to/from other color spaces will be done with that in mind unless changed manually.

[Andy600]

Hi Andy600.
First, let me assume my error: in fact, cinema does not use Rec.709 as the working space, but for the broadcast. My fault.

Thanks for acknowledging your error.

Just to be clear, my reply below is not intended to mock you or belittle you in any way. I simply have to correct some of your misunderstandings as some of your opinions that may be perceived by some readers to be based in fact are incorrect.

But, here we go:
Maybe high-end cinema ("US-American film industry", as defined by technicolor), yes.

DCI-P3 (by Digital Cinema Initiatives) is the standardized colorspace of digital cinema projectors and was published by SMPTE. If you are delivering media for projection in a cinema (usually as a DCP - Digital Cinema Package in XYZ colorspace) it is transformed to DCI-P3 on projection. This is the universally accepted standard around the world. Technicolor don't really come into this other than their DI dept. adhering to the standard.

You can convert Rec709 to XYZ or DCI-P3 but you will have already clipped colors.

ACR does not uses it by default.

I never said it did. ACR is a raw photo developer. I (Cinelog) retask the app to do something it was not specifically designed for.

ACR needs you to define it for yourself and most people don't do it.

This is only when using ACR with Photoshop. The colorspace setting has no affect when dynamically linking ACR with After Effects (and then onto PPro if desired) which is one of the many reasons Cinelog exists

https://helpx.adobe.com/after-effects/using/color-management.html

A quote from the adobe page above:

So, this page shows a least some people work on ProPhoto RGB (although I would also not advocate it), you've said "ProPhoto is not used for broadcast or Cinema".

Adobe state that Linear gamma/ProPhoto is a 'good choice' for digital cinema work and in the absence of anything else I would agree simply because ProPhoto has a significantly larger gamut than Rec709 (i.e. less clipping of colors).

Color clipping also occurs in ProPhoto and this is another reason for Cinelog - because Cinelog-C's scene-referred log transfer and virtual primaries keep the color information in-gamut for the transfer to After Effects and can effectively store 13.5+ f-stops in as little as 10bits. The transfer boundaries are too small for a true linear transfer which is why Adobe state 'linear gamma'.

The term 'Linear' is a much abused term. Linear (to light) and Linear (gamma) are not the same thing.

Now to the main point. The same page above says that you "To preserve over-range values, work in 32-bpc color for its high dynamic range.". I do not agree with that. Adobe is probably talking about VFX, and our point here is digital camera RAW files.

Again you are assuming anything HDR has to be VFX related. This is not true.

Any digital camera from the past 10 years that produces raw images can capture HDR images relative to most current displays (with the exception of maybe a Dolby Pulsar). Your own camera can likely take images of 10+ f-stops and in Rec709 this is HDR because the data can not be represented linearly within that colorspace.

There are also many transient pixel-level operations in plugins that can produce over-range values - these would be clipped in an 8 or 16bit environment.

Regardless of this misunderstanding. What applies to VFX workflows generally also applies to raw image workflows because of the physical properties of light.

Technically speaking you can work in 8bits if your deliverable is 8bits, the same colorspace and any plugins or pixel manipulations in the app are handled in an internal float space but you will always irretrievably lose information if your source material is 10/12/14bit raw. Obviously, a 14bit raw image will fit in a 16bit workspace but can still clip with some operations so switching to a 32bit float space is desirable to avoid any data loss and important if you are targeting other colorspaces (Adobe even say this).

Most people watch in Rec.709. Displays used today are not capable of reproducing wide-gamut.

Just because most viewers may watch in Rec709 or sRGB does not mean that you, the cinematographer, DIT, colorist, editor or software engineer should not aim to maintain data integrity, color fidelity and dynamic range throughout the entire process until output. There are plenty of monitors, TVs and even smart phone screens that now support wider gamuts (including DCI) and uptake is accelerating. HDRTV is currently in a standards battle but it will eventually become the norm and I suspect sooner than expected.

By maintaining scene-referred masters, either raw, .EXR (ACES) or economical log you can future-proof your videos and take advantage of new HDRTV standards and displays.

The concept of maintaining DR through bit depth seems totally bullshit for me. If you have configured a working color space, you've already limited that (unless you're working with tangential wide gamut, and that's not the case since you've said DCI-P3). Bit-depth precision may calculate more precisely the colors (although it would not be perceptible), but it cannot preserve more luma DR, it's already limited to 14bit, your greys is already there, you can't create this information from nothing.

What may seem like bullshit to you is the bread and butter of any reputable VFX house and colorists who have a basic understanding of color science.

Defining a gamut does not necessarily restrict dynamic range! Scene-referred colorspaces (ACES 2065-1, Log-C, Cinelog-C, SLog3/S-gamut3 etc) maintain the relationship to linear light (and thus maintain quantifiable linear dynamic range) and have linear RGB values that extend well beyond Rec709, sRGB, ProPhoto, Adobe 1998, DCI-P3 and other display-referred colorspaces.

It's not about preserving more, it's about preserving what is there and, where log colorspaces are concerned, preserving what is there while compressing the signal in a visually lossless manner for lower storage costs - with the convenient benefit of a film-like response to light and simple mathematical inversion to a linear-light representation of the data i.e. identical or close to identical pixel values of the debayered, white balance raw image.

A colorspace is the only way our eyes can make any visual sense of the captured data and so the data must be brought into a colorspace (preferably a bigger colorspace than it will ever be displayed in) in the least destructive way then transformed to a display space in order to reproduce the image as faithfully as possible in terms of perceptual color appearance and contrast, relative to the device (and it's colorspace) used to display the image. In other words there is no getting away from colorspaces.

Your eyes have a colorspace and the idea is that digitally captured data should be transformed into a colorspace resembling what your eyes see (to the best of your display's capabilities). To be clear, it's not your particular eyes but the eyes of a set of observers who participated when the standard CIE models were derived. There will always be differences between how each person sees the same image or color for a multitude of physiological and technical reasons but we use CIE standards as a foundation in color science (for instance, as a model of how the eye perceives light (CIE-1931) or a connecting space i.e. CIE-XYZ etc). This at least tries to maintain some predictability and uniformity to everything we do in the digital cinema, video and photographic arts.

DR (Dynamic range) is only one part of this process. Obviously you cannot reproduce 15 stops of DR on a display that can at best display only 6-7 stops but you can manipulate the data in such a way as to map the higher dynamic range to something that looks perceptually correct to most viewers - this should be the very last thing you do in the processing pipeline because once that mapping happens there is no way to get the extended data back.

tl;dr you're talking about an idealistic POV about workflow, but people watch our content on realistic devices, that are not HDR, not precise and most people will give zero shit about a 0,2% shift in your red spectrum on their 4-inches Iphone screens, because most people can't even perceive this. Even on 4k displays you can feel it. I know because I've tested.
You all want to work with 32-float, wide-gamut, galactic-fucking-computer-expensive-workflow, go ahead, I'm not stopping you from anything. But I'll not follow this.

Related meme
]

That is quite a statement.

I am talking about a workflow that has been developed through years of research and is in use by anyone who actually cares about their work regardless of whether it is for a tentpole Hollywood blockbuster or a Youtube video of your cats, seen on an phone screen or an iMax screen.

That 0.2% 'shift' in the red spectrum you talk of may actually be the logo of a huge corporation in an advertisement across multiple media outlets. You may not see it on that phone but what happens when the ad is viewed on something better, something calibrated like a cinema screen? - also, there are already millions of wide gamut 4k devices in the hands of the public. The shift becomes more and more apparent the better these device become and somehow these tiny, inconceivable differences start to matter.


There is a great deal of psychology involved with color perception and accurately translating color in a controlled, calibrated environment leads to less error down the line but that is a whole other topic.

P.S: Another quote, this time from:
https://forums.creativecow.net/thread/2/1026156

Apart from that article being from 2012 and related to After Effects CS5 I don't understand the relevance of you highlighting it.

RED footage may have been brought into AE in Rec709 but a) the colorspace can be changed (hell, you can even convert it to Cinelog-C if you want ) b) assuming the workspace is 32bit float the RED footage is held in an unclipped float space so no color is lost and c) most RED users and colorists use RED's own tools to produce intermediates anyway.

[favvomannen]

Paul, so what i can take from this is that the colour degradation from 14 to 8 bit via working in a 8 bit environment in ae is p
retty amazing. my problem was never that i thought i had bad colors. in fact im 110% satisfied  of the color acr gives me. my question in the topic refered more to compression issues of avi to .mp4.

having worked with many 8bit cameras where sky gets banding from the slightest change to the image. the 14bit raw files in a 8bit workspace never get banding in aftereffects and typical 8bit problems like breaking during cc. WHY IS THAT?

[reddeercity]

It how you are adjusting the camera raw in ACR  and the compression used .
AVI is not a good codec to use in a raw workflow , as normally it's rgb 24bit(8bit per channel RGB --millions of colors) default windows
Unless you are using Blackmagic codec r210 (10bit YUV 4:2:2 lossless compressed --billions of colors)  or AJA codec's  r10k (10bit RGB Uncompressed)
The other problem with AVI also they very chunky 1500 Mb/s and up  where as QT Codec e.g. ProRes are far less chunky for the same file 260Mb/s )
I test a avi codec blackmagic 10bit v210 & windows umcompressed avi @8bit and there's no differents in size but in quality there is (billion of colors to millions of color)

Code Select Expand

M28-1750_1.avi    BlackMagic 10bit
Format                                   : AVI
Format/Info                              : Audio Video Interleave
Format profile                           : OpenDML
File size                                : 5.78 GiB
Duration                                 : 27s 736ms
Overall bit rate                         : 1 790 Mbps
Recorded date                            : 2017-08-14T00:39:30.00683-06:00
Writing application                      : Adobe After Effects CS6 (Windows)

Video
ID                                       : 0
Format                                   : r210
Codec ID                                 : r210
Duration                                 : 27s 736ms
Bit rate                                 : 1 790 Mbps
Width                                    : 2 144 pixels
Height                                   : 1 072 pixels
Display aspect ratio                     : 2.000
Frame rate                               : 23.976 fps
Bits/(Pixel*Frame)                       : 32.478
Time code of first frame                 : 00:00:00:00 / 00:00:00:00
Time code source                         : Adobe tc_A / Adobe tc_O
Stream size                              : 5.78 GiB (100%)


Code Select Expand

M28-1750.avi  Windows Uncompressed 8bit
Format                                   : AVI
Format/Info                              : Audio Video Interleave
Format profile                           : OpenDML
File size                                : 5.69 GiB
Duration                                 : 27s 736ms
Overall bit rate                         : 1 763 Mbps
Recorded date                            : 2017-08-14T00:26:33.00123-06:00
Writing application                      : Adobe After Effects CS6 (Windows)

Video
ID                                       : 0
Format                                   : RGBA
Codec ID                                 : 0x00000000
Codec ID/Info                            : Basic Windows bitmap format. 1, 4 and 8 bpp versions are palettised. 16, 24 and 32bpp contain raw RGB samples
Duration                                 : 27s 736ms
Bit rate                                 : 1 763 Mbps
Width                                    : 2 144 pixels
Height                                   : 1 072 pixels
Display aspect ratio                     : 2.000
Frame rate                               : 23.976 fps
Bit depth                                : 8 bits
Bits/(Pixel*Frame)                       : 32.000
Time code of first frame                 : 00:00:00:00 / 00:00:00:00
Time code source                         : Adobe tc_A / Adobe tc_O
Stream size                              : 5.69 GiB (100%)

and for the same quality but with less data rate QT ProRes4444 (10bit windows version FFmpeg)

Code Select Expand

M28-1750_2.mov
Format                                   : MPEG-4
Format profile                           : QuickTime
Codec ID                                 : qt 
File size                                : 742 MiB
Duration                                 : 27s 736ms
Overall bit rate mode                    : Variable
Overall bit rate                         : 225 Mbps
Encoded date                             : UTC 2017-08-14 07:52:24
Tagged date                              : UTC 2017-08-14 07:01:01
Writing library                          : Apple QuickTime
©TIM                                     : 00:00:00:00
©TSC                                     : 2997
©TSZ                                     : 125

Video
ID                                       : 1
Format                                   : ProRes
Format version                           : Version 0
Format profile                           : 4444
Codec ID                                 : ap4h
Duration                                 : 27s 736ms
Bit rate mode                            : Variable
Bit rate                                 : 225 Mbps
Width                                    : 2 144 pixels
Clean aperture width                     : 2 144 pixels
Height                                   : 1 072 pixels
Clean aperture height                    : 1 072 pixels
Display aspect ratio                     : 2.000
Clean aperture display aspect ratio      : 2.000
Frame rate mode                          : Constant
Frame rate                               : 23.976 fps
Chroma subsampling                       : 4:4:4
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 4.075
Stream size                              : 742 MiB (100%)
Writing library                          : Mrzn
Language                                 : English
Encoded date                             : UTC 2017-08-14 07:52:24
Tagged date                              : UTC 2017-08-14 08:01:01
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Other
ID                                       : 2
Type                                     : Time code
Format                                   : QuickTime TC
Duration                                 : 27s 736ms
Time code of first frame                 : 00:00:00:00
Time code, striped                       : Yes
Language                                 : English
Encoded date                             : UTC 2017-08-14 08:01:01
Tagged date                              : UTC 2017-08-14 08:01:01

The original MLV files was 1.8GB and 8bit AVI blow it up to 5.7GB & 10bit 5.8GB were the ProRes is only 742MB !

Post your XMP file from ACR , that would help see where the problem is or even upload your test mlv file and I can check in my raw workflow with A.E.acr & MLVProducer
and then I'll report back.

[favvomannen]

yeah i do most colorcorecting in acr. so i just do minor cc in ae. cool, so my avi workflow is not degrading the images, but i agree, a bit clunky editing with uncompressed avi, (as big as the mlv files themselves or even bigger).

im not used to  prores files, and i thought uncompressed avi had better quality than prores, albleit very large files?


anyway,its when i edit these files in premiere pro, that the 200mb/s output mp4 file have a big degradation compared to the uncompressed .avi files used for editing. reedecity,  you seem to have loads of insights, is there a way around degradation from the avi to .mp4? or is it just that way with compression?

(this was my initial question)

1. mlv to dngs
2. import dngs to acr
3. basic cc in acr
4. 8bit composition, minor cc in ae
5. export to uncompressed .avi file
6. edit these files in premiere pro

7. export to 200mbps .mp4 file (here is where i see degradation and quality loss)

[Kharak]

Take a look at the difference between 8 Bit and 16/32 Bit in Project settings. Film some clouds at Dusk and watch the posterization.

You can even Ctrl+Z and Ctrl+Y the 8 Bit to 16 Bit change, back and forth to see the difference.

Even though your end product will be 8 bit, while you are pulling your colours in 8 Bit you are working with 256 Shades of Gray, you are smashing your colour information before putting it in to a final product. 14 Bit has some 16000 Shades of gray, setting the workspace to 16 bit "dithers" those 16000 to some 20k shades of gray (not sure about this number). You are effectively reducing your color information about 62 times and any fine adjustment wont be very fine, even if you do little adjustments.


And MP4 will always destroy your footage. There is no avoiding it. So will any Internet based platform.

But! Uploading a 10 bit Prores or DNxHD file to Vimeo or Youtube will leave their servers doing the compression giving you a much better final output than ruining your footage with MP4 to then let Vimeo/Youtube compress it again further degrading the image.

@DeafEyeJedi

I work in 16 bit, because Dark Energy plugin does not function properly in 32 bit and for Favvomannen not doing a lot of CC in AE, 32 bit is overkill.

[bpv5P]

Take a look at the difference between 8 Bit and 16/32 Bit in Project settings. Film some clouds at Dusk and watch the posterization.

I'm not advocating to everyone work in 8bit. I'm saying to work in 16bit. The difference between 16bit and 32bit is absolutely minimal and the power necessary to
compute 32bit is much bigger.

I work in 16 bit, because Dark Energy plugin does not function properly in 32 bit and for Favvomannen not doing a lot of CC in AE, 32 bit is overkill.

It's not even overkill, it's just that there's no difference at all.



@Andy600
I'll be waiting for your tests. This discussion will be ad infinitum if we keep replying each other that way...

That 0.2% 'shift' in the red spectrum you talk of may actually be the logo of a huge corporation in an advertisement across multiple media outlets.

Crazy world... remember me of Brazil (1985):
https://www.youtube.com/watch?v=K9gO01pyv24

[favvomannen]

ok so you all know im pretty new at this, does prores contain audio?. when uploading to youtube or vimeo  it sounded like a great idea letting the only compression be vimeo.

but what about audio? or does prores contain audio?

[bpv5P]

does prores contain audio?

Yes.

when uploading to youtube or vimeo  it sounded like a great idea letting the only compression be vimeo.

No exactly. Each of these streaming websites have their own guideline for encoding for a purpose. Uploading lossless content to these websites is mindless. Not just it will take a longer time to upload, but you'll also waste your own bandwidth and overload their servers.
From what I remember, some of these sites, such as youtube, do not recompress if you follow exactly what is on their guideline, just the DASH content needs to be encoded, for adaptative resolution. Facebook, as another example, do not recompress photographs that's under 100Kb. I don't know their protocol for video, though.

[Kharak]

@Bpvp52, you obviously have something to prove to yourself regarding cinelog c. For us working with CinemaDNG, cinelog-c workflow gives the highest quality and as a bonus, makes the output flicker free from acr. If you prefer log-c prores from MLVproducer, super. Is the Log-c from mlvp accurate enough for transforming to slog2? Red-log gamma? I doubt it, but let me know. With Cinelog-C you get a standard that can match all the other major DCP/Logs.

I think i misunderstood what this thread is about, i thought it was about delivering the best quality for publishing.

Uploading for youtube guidelines, i dont know if they dont touch it again then, but that will be some blocky ugly looking stuff you are uploading.I personally put a lot of work and detail in to my work and I try to fight to preserve the highest quality throughout and could not care less if Youtube's/vimeo servers are strained because i upload DNxHD, besides I pay for the upload on Vimeo. If your internet cant handle the bandwith, upgrade.

[bpv5P]

@Bpvp52, you obviously have something to prove to yourself regarding cinelog c.

I do? No, I don't.
There's no proof, any tests (that I'm aware of), showing that Cinelog-C is better than free alternatives.
I do not claim MLVProducer is more precise and I'm not affiliated with any people or product. I just can't see how it's different from AlexaLog, since the author itself said Cinelog-C is based on it. Note that Cinelog-C is paid, while AlexaLog is not. I don't want people spreading false information just to sell it's product here.
This place should be a community without astroturfing and without marketing people trying to fool others. I'm not against selling anything here, just say it explicitly.
I'm not saying Andy600 is one of those people (yet). What I'm asking for (on the other thread) is to prove it. And, as I have said in this thread, I'll be one of the folks (like you) that advocate for Cinelog-C if it really proves to have perceptible advantages. I have no problem admitting I was wrong and apologize.

i dont know if they dont touch it again

I don't too.

but that will be some blocky ugly looking stuff you are uploading.

Not exactly. 10MB/s bitrate for 1080p is enough if you use the latest x.264 with psy (psy-rdo and psy-trellis) optimization. If they don't re-encode the material, and you're using grain in your footage, you can possibly get better results than letting vimeo/youtube encode (because you can tune for grain). Besides that, they will already brutally compress the footage, it will be "blocky ugly".

I personally put a lot of work and detail in to my work and I try to fight to preserve the highest quality throughout

I do too, although I can't do most of it the way I want, because my clients want the material ready as fast as possible.
But I'll not do a workflow that instead of during 6 hours to process will take the double of the time, just because of a 0,02% improvement in color. Or, I'll not pay $50 in a DCP that has no proofs that it works better than free alternatives.

If your internet cant handle the bandwith, upgrade.

Like everyone has money to, or everyone lives in Romania, right?

[Kharak]

What is your output codec from MLVProducer in Log-c?

For me its not the 0.2% colour difference or possibly more or maybe none for that matter. I personally prefer the ACR route because I think it is, as of this writing moment, the highest quality output possible for MLV e.g. Most DR and the sharpest most "true" representation of what I capture in camera.

My workflow:

Mlv - (as of mow) Danne's Windows batch script - import to AE (ACR) - Apply Cinelog-c, increase signal strenght 1.5 ev - In AE: Upscale to 2K, Noise Reduction and detail enhancement with Neat Video and Dark Energy with in-house noise profiles to every shot, low light or not - Export Log intermediates in Cineform - Edit and grade in resolve.

It takes time for sure, rendering intermediates for a project in AE takes anything between 24 - 48 hours + the NR processing, so about 4 days before i can hand it to my editor (which usually is me). But i think of it like working with Film, only Film would take weeks sometimes months before anyone saw the final image.

I rarely do fast turn arounds, I mostly work in Narrative. When I do, I make sure the client knows I will not deliver in top quality. I render from MLV to prores, used to be via mlrawviewer, but since some months ago prores export wont work for me. But have not done fast turn around for a year so will see what i do next time if ever again.

I've found Andy to be extremely helpful with any technical questions or problems needing solving. Not just sales speeches.

And i can assure you Youtube will compress 10MB/s bitrate, there are not many people with that kind of download speed. I think vimeo recommends 35Mb/s. (Note: Mb Megabit, MB MegaByte)

[bpv5P]

What is your output codec from MLVProducer in Log-c?

DNxHD when I need good quality. H.264 when the time is short.


If you permit me, some comments on your workflow:

increase signal strenght 1.5 ev

Using a curve slightly flat in highlights gives me better results than linear exposure raise...

- In AE: Upscale to 2K, Noise Reduction and detail enhancement with Neat Video and Dark Energy with in-house noise profiles to every shot, low light or not - Export Log intermediates in Cineform - Edit and grade in resolve.

Upscale should be one of the last processes, if you want better quality. Noise reduction will be less effective if you do upscale first, because of interpolation artifacts. Also, noise reduction needs to be before sharpening/deconvolution. I personally do in this other (although I don't upscale my footages):

Input > Log conv. in 16bit workspace > Noise reduction > Upscale > Color Grading > Sharpen > Grain

Also, color grading with LUTs is generally a good practice.

about 4 days before i can hand it to my editor (which usually is me).

I personally need to work with constrained time, so 4 days to just pre-process is too much. But your workflow is good :^)

I've found Andy to be extremely helpful with any technical questions or problems needing solving. Not just sales speeches.

Yes, I remember Andy600 before even MLV was developed. Actually even from the time of Tragic Lantern 600D hacks.
It's not personal our discussion here. But there's no proof about Cinelog-C.

[Kharak]

I will try your upscale method.

But I did a lot of testing, basically a year before i settled for this method. I call it AIM (Alexa Imperia Method). Alexa, because I am in love with its image and I try always to emulate it, Imperia, because anything I find pride in I call Imperia, like my Legendary Pasta sauce 'Pasta Imperia'™© . Method, well you know.

If I recall correctly, i got better results with the upscaling before the noise reduction, basically increasing the Canvas for Neat Video and Dark Energy to work with. Bicubic upscaling ofcourse. Forgot to mention adding texture with Dark Energy Matter right after the NR. Two Adjustment layers, 1. NV. 2. DE Anti-Matter, DE Matter.

The NV with DE is what hogs the performance down. If only one of them were applied it would cut the rendering time down to less than half. But I am a Quality whore, a blessing and a curse.

The 3x3 Bayer of the 5DIII Raw holds a lot more sharpness to it than most people think, if done right.

Also forgot to mention in AE i also apply a 3rd. Adjustment layer for Opencolor IO for transforming Cinelog-C to Log-C or any other log if required. Often Slog 123 for a production company I often work with.

[favvomannen]

do you use nv and dark energy noice reducer even for bright lit scenes? noise only for dark scenes right?

has anyone any example on a clip with dark energy texture vs same clip whout texture. im not so fond of grain added by filmconvert. real filmgrain is much more scene dependant and organic.

sry for being a tad ot, but all this talk about dark energy plugin got me curious.

[DeafEyeJedi]

...im not so fond of grain added by filmconvert. real filmgrain is much more scene dependant and organic.

The caveat here w FilmConvert is the fact that 'film grain' is by default @ 100% (not sure why tho) so perhaps the trick is to try starting from 0% and upward to your own taste.

Also there are reasons why it's rather more ideal to work in 32-bit float for those that are either curious or oppose to it. Looking forward to @Andy600's next major update w his products in Cinelog DCP which should finally pull the trigger for those who haven't purchased already.

[PaulHarwood856]

Neat video has an option to use a percentage of the original footage noise and Neat Video's noise reduction. It's called Mix with Original. Raw video from Canon DSLRs have filmic grain, so I really like this method. I do noise reduction at the end in Premiere Pro with ProRes 4444 XQ transcoded clips with Cinelog-C.

[hyalinejim]

There's no proof, any tests (that I'm aware of), showing that Cinelog-C is better than free alternatives.
I do not claim MLVProducer is more precise and I'm not affiliated with any people or product. I just can't see how it's different from AlexaLog, since the author itself said Cinelog-C is based on it. Note that Cinelog-C is paid, while AlexaLog is not. I don't want people spreading false information just to sell it's product here.
This place should be a community without astroturfing and without marketing people trying to fool others. I'm not against selling anything here, just say it explicitly.
I'm not saying Andy600 is one of those people (yet). What I'm asking for (on the other thread) is to prove it. And, as I have said in this thread, I'll be one of the folks (like you) that advocate for Cinelog-C if it really proves to have perceptible advantages. I have no problem admitting I was wrong and apologize.

Well, why not buy it and try it?

Athough we may not have posted the results of the tests, there are reasons why numerous ML users for years now have been advising that Cinelog through ACR offers the best image quality available of all the post workflow options.

These have to do with dynamic range preservation, noise suppression and reproduction of pleasing colour.

But why should we test this for you? If you're not willng to cough up the money to do your own tests, then please stop banging on about it.

[Andy600]

Thanks for your endorsements guys. I appreciate your support for Cinelog but we have pulled this post far away from @favvomannen's original question so I ask everyone to keep the discussion on topic from here onwards.

I will present the MLVP vs Cinelog DCP test results in the Cinelog thread when ready.