Basic DR question about MLVProducer and log formats vs sRGB

Started by skrk, December 24, 2016, 09:40:54 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

skrk

Hi - I am doing some simple processing of frames from raw video taken with ML.

Currently I use MLVProducer to remove the focus pixels and convert all the frames to 16bit TIFF, sRGB.

I process these with custom code: this code converts the 16bit integer TIFFs into floating point, does math, and then writes them out as 16bit integer TIFFs again. The noise floor is a big issue (long story: low-light exposures, so most frames are very dark). I do lots of work to remove the noise, deal with it, work around it, etc. (FPN noise removal, adjustment curve, etc.)

Everything works fine, but I want to make sure I'm doing as much as I can in terms of the noise floor.

It occurred to me that I might be throwing away dynamic range by converting the RAW video to sRGB -- is this correct? Does this happen when MLVProducer converts RAW video to sRGB, or is that compression (or non-compression) just controlled by the sliders? In other words: is a (small) dynamic range built in to the definition of the sRGB color space? (My impression is that Rec.709 does have DR built in to it?)

If DR is being compressed, would it be smarter for me to instead convert the frames to some kind of log format? Then when I convert to floating point I could reverse the gamma curve and process as normal without having thrown away dynamic range?

My main concern is that when I convert to sRGB, I'm squashing the dynamic range and thereby combining the noise floor with the lowest end of my image data. Am I crazy?

Any help is appreciated, thanks!
100D -- ML v1.0.1

whyfy

I'm kinda in the same process as you; and I do think you're throwing away dynamic range when converting to sRGB.
Personally I think you should use a flatter colour space like log preset or one of the Slogs?

kontrakatze

Things are a little complicated regarding colorspace and I'm not absolutely sure if I'm right:


It depends on your goal:
If you convert your footage to sRGB Tiff and want to grade for Rec2020, you will definitely loose information right before you start grading. In this case it would be much better to go for ProPhoto RGB as it is almost the same as Rec2020 (or even better/wider) and there wouldn't be to much conversions or mappings going on.
But if you grade for Rec709 sRGB would be fine and almost congruent. Yes you would loose information, but these informations wouldn't be displayed anyway.

Why not staying RAW using cDNGs? This way you could stay away from conversions up to the last moment. Work with the full RAW information, display it using Rec2020 and if necessary, convert to Rec709 using LUTs. And if you grade for computer screens (Vimeo, Youtube and the like), in my opinion you go along better with Rec2020. But it's all a question of taste, too.

Merry X-mas,
kontrakatze

skrk

Thanks for the replies! -- @kontrakatze, I'm aware that the sRGB primaries are pretty close to Rec.709 and that the gamut is more or less the same. I'm not worried about color information, I'm worried about the gamma (or maybe "transfer function" is the correct term?) (And I don't use a DNG for various reasons: i don't own Adobe software, and i'm not (yet) sure how to process it in Python, but maybe I should look at that!)

I know that sRGB defines a gamma curve, i'm just not clear on how that relates to the full dynamic range of a RAW frame. My impression is that Rec.709, and possibly also sRGB, squashes a lot of the dynamic range into a tighter range -- that's the whole reason for log formats, as I understand it. Further, my understanding is that sRGB especially squashes the under/overrun tighter than the middle, which makes sense.

Here's an illustration of what I'm visualizing might be happening:



This link seems to confirm @whyfy and my suspicion about DR compression: https://blogarithms.com/2012/01/09/wasting-dynamic-range/comment-page-1/

...it shows that using TIFF or JPG or any other non-DNG (/non-HDR) format, in any color space (ProPhoto, sRGB, whatever) will reduce dynamic range when coming from Lightroom, so I guess it's a pretty safe bet that MLVProducer will do the same.

My eventual end format is sRGB and/or Rec.709, but the reason I'm caring about this is that my processing needs to keep the useful image data out of the noise floor as much as possible. If the DR compression is combining the noise floor with the low-end of the image data, I'd like to avoid that. I don't know if the above diagram of the noise vs. the image data is accurate -- maybe most/all of the noise is already overlapping the image data and the DR compression won't affect the SNR -- but I want to avoid it if it's possible.

It's a hard thing to test because I don't (yet) have the ability to reverse any of the log curves in software. I could try to figure that out, but I don't want to put in the effort if there's no potential gain from it.

I ran a test where I exported the same frame to 16bit TIFF-sRGB and 16bit TIFF-slog and compared... after pulling the images up in GIMP and crudely tweaking the curves/levels of the slog version, it didn't seem like there was any more detail present in the low end, or any less noise... so I'm still left wondering if this is worth pursuing. But if the above diagram is accurate, then it would be...

Maybe I'll look in to setting up a DNG workflow... RAW formats intimidate me (in terms of writing code to process them). :-)
100D -- ML v1.0.1

Kharak

you seem to go a long way to avoid RAW. I will recommend that you try out RAW properly, the full power in your hands.

There is free software like RAWTherapee http://rawtherapee.com

Now read my signature ;)
once you go raw you never go back

Danne

sRGB is a a friendly gamma curve and dng files coming from canon cams are display referred looking like rec709. If you export to highres 16bit I don,t see any problems with the gamma info. You reduce gamut to sRGB but on the other hand when will you ever need more color?
The bigger issue is with highlights which all clips to display referred 1.0(dng spec). What you want is the linear scene referred stuff which delivers the full dynamic range of your dng file. Then you want to do your exports. Now with acr you can use a linear dcp profile which will expand dr considerably(check link http://www.magiclantern.fm/forum/index.php?topic=13512.msg172443#msg172443).
You can also use dcraw and work around the setting -H 2 to bring back a lot of dr into your output.
Ok, and now let,s wait for Andy600 to give us the real color science "slap" and set all of this straight  :P.

skrk

Thanks, both! --

In case it wasn't clear from above, I love RAW and use it all the time; I'm not talking about picture development here, I'm talking about custom code that I've written that I want to make as effective as possible. Python seems to have the rawpy and rawkit modules, both of which look good, and which seem like they would enable me to develop the raw video frames into linear-gamma sRGB arrays that I could then process... but I'm a bit lost on then changing the RAW/linear gamma into sRGB when I save the sRGB tiff at the end. I can figure it out, I just don't want to spend a week doing research if there's no point to begin with.

My code is calculating based on values in the current frame and previous frames... and the nature of the calculation is that low-level noise builds up over time. Currently I have to crush the low end of each frame of the histogram with an adjustment curve before processing. It works fine, but I sacrifice some details in the shadows' falloff, etc. I got to wondering if I could keep my image out of the noise floor by using a smarter transfer function.

This is why I'm concerned about throwing away DR -- I don't really know how the noise distribution looks in the RAW histogram (that's something I will explore next) but if the conversion to sRGB is combing the noise floor with the low-end of the image, that would be a place I could improve my process.
100D -- ML v1.0.1


DeafEyeJedi

5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

kontrakatze

I'm not sure if what you are talking about is a matter colorspace, gamut or gamma in itself, meaning the definitions of sRGB or ProPhoto and its gamma curves. In my opinion, it is the way you convert or transform from RAW to whatever you want to. What do you do with the given information? Do you compress information? Do you use a hard knee, soft knee, or do you just crush blacks?
Your graph implies some kind of compression which to me appears to be a kind of lift (soft knee, 5% lift?). Of course this would increase the noise as the noisefloor gets lifted and it doesn't matter to what you convert to afterwards, the noise it there.

But anyway, my experience with noise reduction is, you might get rid of some pesky pixel, but f the noise was implied on the sensor level the possible information got masked out by grainy colored pixel - you will never get this information back.

skrk

Thanks again, everyone --

I do want to post my code at some point, but right now it's not doing anything related to what I'm asking about in this thread, so I don't think it will help anything :-) -- right now it just processes 16bit sRGB tiffs, and I'm just wondering if I have an opportunity to make it better, and I think that depends on how RAWs are developed:

QuoteYour graph implies some kind of compression which to me appears to be a kind of lift (soft knee, 5% lift?). Of course this would increase the noise as the noisefloor gets lifted and it doesn't matter to what you convert to afterwards, the noise it there.

That graph doesn't indicate what I'm doing, it was intended to depict what my imagination is wondering about MLVProducer (and others): everything I've read tells me that translating to sRGB (or any other non-RAW color space) is going to compress the dynamic range. I'm just not clear on the specifics.

Maybe this will illustrate my question better. Which of the the following three methods (if any!) roughly approximates what happens to overall brightness values as a RAW is converted to sRGB -- A, B, or C?:

UPDATE: D'OH - NOPE. This graph is using the wrong gamma curve (the reverse transformation) -- IGNORE.



So my question is just about how MLVProducer (and presumably all the other RAW->sRGB converters in the world) convert the large DR of RAW into sRGB.

If either method A or B are correct, then I will wonder if the noise floor in the low end of the RAW range is being compressed into the low end of my image data, and I'll be interested to investigate further (e.g. to see if there is even noise down there that matters in the first place, or if it's already overlapping my image data anyway). If method C is correct, then I don't need to worry about it.

This link (wikipedia) specifies the 3x3 matrix and the gamma curve, and it seems to imply that method "A" above is roughly correct, but I don't know linear algebra and so I can't guess what that matrix is doing to the brightness.

And if I'm just off in crazy land on all this, then tell me to go back to school and I'll stop wasting your collective time. :-)
100D -- ML v1.0.1

skrk

D'oh -- just realized I had the sRGB gamma curve backwards in my last gif, above! That was the reverse transformation, per wikipedia.

The proper gamma curve (sRGB = 1.055*x^(1/2.4) - 0.055) makes it feel like my concern is unfounded: wherever the RAW noise lands (if it lands at all), and however the gamma curve is applied to the linear values, the lowest-end noise will only be separated further from the image data (since slope > 1), afaict.

So I'm learning a bit more about matrices and it sounds like the DR of the RAW form is maybe only squashed or not depending on how sliders are set in MLVProducer, and that if you develop a RAW without setting a white/black point you are essentially containing the whole DR of a RAW image in the DR of sRGB, and the DR will then be limited by the output device's DR?

Anyway, I'm going to play with some sliders in MLVProducer and make some histograms in python and see if this all lines up.
100D -- ML v1.0.1

skrk

Huh, well: there does seem to be noise down in the low levels, below the image data, as I suspected. If I export to a linear-gamma TIFF I can see it down there. If I manually crush the lowest levels to zero, that noise goes away.

I can see that setting the black point in MLVProducer (which defaults to 2044 for my camera) does not simply clip the levels below 2044 to black; it does seem to "squash" them into the low end. (To test this I exported a series of linear tiffs at black point gradually increasing from 0 to 2044 and watched the histograms as the noise hump is gradually squashed closer to and eventually in to the image data.)

I assume there are good reasons for that, as most of raw development is still above my head. When I take a linear tiff exported with blackpoint set to 0, I can manually crush that low-level noise to zero, but when I then try to manually scale the blackpoint and apply an sRGB gamma curve, the results are a disaster. That low level noise may be gone, and the overall brightness of the image is good, but tons of new noise is present. Certainly not an improvement. :-)

I have no idea why that happens, but clearly there is a lot of extra magic going on that escapes me. So for now I'm gonna have to go back to school on this stuff and not worry about that noise.

I do think that I've learned that my complicated notions of how the dynamic range of raw images is fit into the dynamic range of sRGB were misguided. Apparently sRGB 0 maps to XYZ Y=0, and sRGB 1 to XYZ Y=1 -- meaning, apparently you just linearly scale the DR of RAW into sRGB (plus whatever adjustments you make while developing, black point, etc) and the output device imposes whatever DR limitations it imposes... I assumed something much fancier was going on. Lots to learn. I'm wondering now if the link I posted before that claims that any TIFF or JPG has less dynamic range than a DNG/RAW/CR2/etc is actually correct: it doesn't sound like there is a DR defined in the sRGB standard, surprising as that is to me: I would have expected the black and white points to be referenced to some output illumination strength.

Thanks to those who helped.
100D -- ML v1.0.1

TrEK

Tell me please... why if i export from MlRawViewer_1_4_3 to DNG_SEQUENCE... i can import it to Adobe Premiere...



But if i export in \MLV Converter 1.9.2 ... i cannot import :






WHY ?

maxotics

I haven't read this thread very closely, so apologies if I miss the point.

1. sRGB generally uses (edit thanks Walter) a 24-bit color space that represents around 16 millions colors, which is the MAXIMUM of what we can see and which our equipment can display.  EVERYTHING ends up in a similar bit-depth, adobe RGB, rec709 etc.  All viewable/practical dynamic range is represented by 8-bit channel/24-bit full color data range.     

2. RAW is a NON-VISUAL per pixel measurement of light intensity, usually in 14-bits, or 16,343 values that falls on a camera sensor.  Each pixel has a color filter placed above it that is used to measure color.  Each pixel can only see one color.  Neighboring pixel color values are necessary to create a full-color value.  This creates a huge barrier to understanding.  (8-bit color in sRGB video is a misnomer because it really means 8 bits per channel, when it is really a 24-bit full color value). 

When you see the washed-out footage of RAW video you are seeing A LIE

RAW data can create 4 trillion colors, or dynamic range BEYOND which we can see on our displays, or with our pupils fixed at one size.  Therefore, RAW DATA CANNOT BE SEEN.  What you see is a visual proxy.  RAW data could be shown as hyper saturated and then all grading would show going from hyper-saturated to normal saturation.  The way RAW video is displayed is completely arbitrary.  The videos one sees where the person shows the washed-out RAW footage and their grade is amateur hour (sorry, I know it's fun).  A "flat-profile" in 8bit video removes saturation (color) for more brightness detail.  You can add color back in, but not as well as if you had it in the first place.  "Grading" RAW, again, RAW doesn't have any look, gray, washed out, or otherwise.

RAW data must be understood scientifically as the readings of silicon wafers which we must put in visual space, just like x-rays, ultra-violet and other near infrared frequencies.  Yes, you can no more "see" RAW video than you can see x-rays.

I wish I had the time to write a better post here. 

Anyway, exporting DNGs into Premiere doesn't make sense to me because MLVProducer will create a nice sRGB data set so you don't need to process the RAW files in Premiere. If you are going to use DNGs, Photoshop has better tools for RAW to RGB.  Keep in mind a benefit of processing RAW on the computer, vs camera, is you have access to more powerful de-bayering algorithms (the programs that borrow color values from neighboring pixels).  The camera might use bicubic for example to save power, where you can run aMaze on your PC, which does a better job.

As for noise at the floor.  You have to pick a range of data in RAW to create a real-life sRGB color space.  "Noise" is subjective.  Converting RAW to something usable is about subjectively representing 42-bits of data (which can't be fully seen) into 24 bits of visual data within our biological dynamic range. 

Another way to think of it is that in sRGB you could record someone talking in the park and they look great, but the sky is washed out, as it would be in real life (unless we looks up at the sky and our pupils contracted).  Later, you discover there were UFOs in the sky.  You can't recover them in the sRGB footage because the sky is washed out (again, just as it would be in the sky if you tried to look at them in real life and you had just gone to the eye doctor, who gave you something so your pupils couldn't contract).  If you had RAW data, you could re-translate the data, "cast" the sky data into an image where you could see the UFO.  Of course, the same algorithm would darken out your original subject.

As for s-logs, they are 8-bit KLUDGES that move the sensitivity of light around in a 24-bit space, AT THE EXPENSE OF COLOR in the place they borrowed data from.  Most of the stuff about s-logs and color profiles I see on the Internet is complete cow-dung when it comes to improving dynamic-range.  It looks like magic.  You try it yourself and you'll see it's just snake oil.  S-log has a place, but a VERY LIMITED one, like shooting in a bright sunny day.  Again, it does NOT increase absolute dynamic range. 

So to answer the question here.  If you shoot RAW, don't overthink it.  Use MLVProducer (or MLRawViewer, etc.) to get a good 24-bit translation and get on with your life.  If you want higher DR in your 8-bit camera s-log is only a LIMITED solution for LIMITED situations which you better understand.  If you've seen great 8-bit video from a Canon C100 or Sony FS5 say, then you'll just have to get one ;)  They do in-camera DR translation better than consumer cameras but it has more to do with having s-log.  That is, they don't need log profiles to give more natural looking video than consumer cameras.




Walter Schulz

Quote from: maxotics on January 29, 2017, 05:58:18 PM
1. sRGB is a 24-bit color space

I have a problem with this statement. Color spaces like sRGB are IMO not defined in bit depths.

maxotics

Hi Walter, yes, of course you're right.  There are  many errors in what I wrote (but my aim was to explain away some misconceptions greater than the difference between color spaces and bit depth), feel free to explain it better! I found that the more precise I tried to become, the more difficult to get my point across. I'd love to hear a better explanation of how RAW differs from whatever you want to call it ;)