Does RAW actually increase dynamic range or does it just add bit depth?

Started by CyJackX, March 20, 2014, 07:06:40 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

CyJackX

I'm assuming the dynamic range (as in the ratio of the highest signal to the lowest signal) would still be the same with a RAW or a non-raw frame?  Does RAW enable the sensor to pick up a wider range of data?  There's certainly more latitude in post to "stretch" the values you acquire with more bit depth, but I have been wondering if plain RAW video from a Mark III vs the h264 improves on the dynamic range.  This has come up because I've been looking at things like the BMPCC which is supposed to have ~13 stops DR.

tupp

Quote from: CyJackX on March 20, 2014, 07:06:40 PM
I'm assuming the dynamic range (as in the ratio of the highest signal to the lowest signal) would still be the same with a RAW or a non-raw frame?  Does RAW enable the sensor to pick up a wider range of data?

Dynamic range and bit depth are two very different and independent properties.  As you said, dynamic range is the ratio (of amplitude) between the highest and lowest signal (above the noise floor) in both analog and digital systems.  Bit depth applies only to digital systems, and bit depth is number of digital level increments within the highest and lowest signal (including the noise floor).

The capture dynamic range of the sensor itself remains unchanged regardless of how its signal is processed after it is captured.  However, it seems clear that the h264 (and jpeg) processing inside the Canon cameras reduces the "capturable" dynamic range of the entire system to less than that of the sensor.


Quote from: CyJackXThere's certainly more latitude in post to "stretch" the values you acquire with more bit depth, but I have been wondering if plain RAW video from a Mark III vs the h264 improves on the dynamic range.

Not sure what is meant by "to 'stretch' the values you acquire with more bit depth," but it seems clear that the use of raw data from Canon sensors will capture a greater dynamic range of values in the subject than using the Canon h264 (or jpeg) processed image.

Pelican

EOS 7D Mark II, EOS 7D, EOS 5, EOS 100 + lenses (10mm to 300mm), 600EX, 550EX, YN600EX x 3
EOScard, EOS DSLR firmwares, ARMu, NiControl, etc.: http://pel.hu/down

chmee

no, finally they're aren't independent. in terms of "datadepth" you need some bits to store linear digital signals, as they come from a visual sensor. the main aspect is, yes, you could get the whole dynamic range by compressing into this byte-field (as its in h.264) - but it looks ugly, if you like to tweak it a little, horrible banding is the most common artifact. try some picturepresets like flaat_12 - you can use them only, if you're on point while recording, if not, you re lost.. (and in most situations you have to tweak 12ELV to look natural)

(btw. its a everlasting discussion, how generally to define lower and upper ends of a usable signal. i made tests with an older 5d(photo) and yes there are more than 11 ELV Range, but the lowest lowlights are not usable, because the noise so much stronger than the useable picturesignal. DxO's saying it's about 11.1 ELV, not a bad result for a 12bit-ad-converter..

regards chmee
[size=2]phreekz * blog * twitter[/size]

tupp

Quote from: chmee on March 20, 2014, 09:01:51 PM
no, finally they're aren't independent.

Yes.  Dynamic range and bit depth are two very different and independent properties.

A camera can have a lot of dynamic range, but little bit depth.  By the same token, a camera can have little dynamic range and a lot of bit depth.  Furthermore, a camera can have a lot of dynamic range and a lot of bit depth, or it can have little dynamic range and little bit depth.  Dynamic range and bit depth are independent.

There are zillions of real life examples, but one need not look any further than right here at ML -- dual ISO.  Dual ISO gives a phenomenal increase the the capture dynamic range, yet the bit depth doesn't change.  Dynamic range and bit depth are two different, independent properties.


Quote from: chmeein terms of "datadepth" you need some bits to store linear digital signals,  as they come from a visual sensor.

Not sure what is meant by "datadepth."  However, yes there must be at least two shades in digital systems to establish a dynamic range.


Quote from: chmeethe main aspect is, yes, you could get the whole dynamic range by compressing into this byte-field (as its in h.264) - but it looks ugly, if you like to tweak it a little, horrible banding is the most common artifact. try some picturepresets like flaat_12 - you can use them only, if you're on point while recording, if not, you re lost.. (and in most situations you have to tweak 12ELV to look natural)

I am not referring to how "ugly" an image looks -- I am referring to what dynamic range and bit rate  are.

By the way, banding has a lot to do with color depth, which is also different than bit depth.


Quote from: chmee(btw. its a everlasting discussion, how generally to define lower and upper ends of a usable signal. i made tests with an older 5d(photo) and yes there are more than 12 ELV Range, but the lowest lowlights are not usable, because the noise so much stronger than the useable picturesignal

Well, dynamic range is a pretty "cut & dry" technical term as it relates to analog and digital systems, and noise is factored into dynamic range.  A lot of people (including me) also confuse dynamic range with signal-to-noise ratio, but those two properties are more closely related to each other than are dynamic range and bit depth.

Wish I had more time to explain some things, but I gotta run.

kgv5

Here is an explanation, more or less:

https://vimeo.com/52269416

According to the author, there are two types of dynamic ranges: percieved DR and available DR. The first one is what we can see immidiatelly (so including DualISO), second one comes from the image manipulation in post - shadow and highlight recovery (and here comes the bit depth).
www.pilotmovies.pl   5D Mark III, 6D, 550D

chmee

@tupp
are you workin any scientistic kind? because i'm a little bit confused about some of your words..

QuoteA camera can have a lot of dynamic range, but little bit depth.
no. are you talking about security cams without affection to good picture and with pre/postprocessing? the result is no longer a high dynamic range-picture
QuoteBy the same token, a camera can have little dynamic range and a lot of bit depth.
"CAN"! - picturesensors giving linear data, means, to save and sample them accordingly, you need a "linear" system, its quite good described with the bitdepth. 1bit for 1ELV (>1bit) - theoretical meaning 14bit is able to sample/save nearly 14ELV of visual data accordingly without loss/artifacts. that doesnt mean, if theres a 14bit ad-converter, the source side (here: sensor) must deliver the full range.
QuoteDual ISO gives a phenomenal increase the the capture dynamic range, yet the bit depth doesn't change.
lol. no. DualISO is changing the ampification per sensel-line. both different amplificated "pictures" are limited in their dynamicrange, one's losing data in the lowlights, the other loses values in full-well-sensor-area (highlights). they're kind of preprocessed. but you said "zillions of real life examples". fine.
Quotethere must be at least two shades in digital systems to establish a dynamic range.
a switch with two states has no dynamic range. how do you describe dynamicrange between on and off? you need a "lowest" measurable value (not zero!). leads to at least three states (or you said: two shades). means: [three states] 0, 0.5 and 1. whats the dynamicrange of that? other example: 4 states (2bit): 0, 0.33, 0.66, 1 (=1/0.33).. (16384 states) 14bit -> 1:16383 -> ln(16383)/ln(2) = 13.99 ELV
Quote..color depth, which is also different than bit depth.
why? "color" is just an electromagnetic property. if you measure three separate frequencies by its amount (here: photons), you just measure the dynamicrange in finite frequencies.. (please dont talk about the simplifaction from graphic-cards-world)

the [btw] was to describe, theres no "one holy grail" about measuring dynamicrange. some like the way, DxO does, other doesnt.

regards chmee
[size=2]phreekz * blog * twitter[/size]

Audionut

DR describes the difference between the highest recorded signal, and the noise floor.
Bit depth, in a nut shell, describes the resolution (accuracy), of the digital system.
With increased accuracy, you can capture increased DR.

They are separate components, but they are intertwined.

Quote from: chmee on March 21, 2014, 12:06:09 AM
the [btw] was to describe, theres no "one holy grail" about measuring dynamicrange. some like the way, DxO does, other doesnt.

Well, you have the engineering definition of DR, which is not subject to subjective opinion.  But of course, in terms of the noise level in the shadows, that you, I, or anyone else may find acceptable, it's not an accurate measure.


On topic, the sensor is a fixed component, and not effected by the downstream process.
The H.264 capture system is only 8 bits.  So you lose accuracy in the shadows (lower DR).

chmee

Quote from: Audionut on March 21, 2014, 02:06:00 AM
They are separate components, but they are intertwined.
intertwined. didnt knew this word.
QuoteOn topic, the sensor is a fixed component, and not effected by the downstream process.
The H.264 capture system is only 8 bits.  So you lose accuracy in the shadows (lower DR).
because theres the picture-style in between (as postprocessing-unit), you can preserve the impression of dynamicrange, the typical HDR->LDR-way.. (do not forget the gamma-curve, its not linear). where are the 10 or 12bit displays for the common user?
[size=2]phreekz * blog * twitter[/size]

Audionut


tupp

Quote from: chmee on March 21, 2014, 12:06:09 AMare you workin any scientistic kind? because i'm a little bit confused about some of your words..

I do not understand your question here.


Quote from: chmeeno. are you talking about security cams without affection to good picture and with pre/postprocessing?

A camera can have a lot of dynamic range, but little bit depth.

I am not singling-out security cameras, although there are certainly some that have increased DR and an 8-bit depth.

However, there are plenty of other examples, too.  For instance, many cameras give a choice of bit depths -- yet the DR remains the same regardless of the chosen bit depth.  So, at the lowest bit-depth setting, the camera has its greatest dynamic range, but has it's lowest bit depth.

Dynamic range and bit depth are two independent properties.


Quote from: chmee"CAN"! - picturesensors giving linear data, means, to save and sample them accordingly, you need a "linear" system,

As I am sure you know, sensors give analog data, and that info can be mapped to digital values in ways other than "linear."

Regardless, given a linear mapping method, a camera can still have little dynamic range and a great bit depth.  Take any early 16-bit camera and compare its capture DR to that of a current 16-bit camera -- the newer cameras will generally capture a greater dynamic range.  So, the early cameras have little capture DR but great bit-depth.


Quote from: chmeeits quite good described with the bitdepth. 1bit for 1ELV (>1bit) - theoretical meaning 14bit is able to sample/save nearly 14ELV of visual data accordingly without loss/artifacts. that doesnt mean, if theres a 14bit ad-converter, the source side (here: sensor) must deliver the full range.

I am not sure what the point is here, but the value associated with each incremental step in bit-depth in a linear system is not a universal constant.  That value can (and often does) vary from system to system, camera to camera.  That variation is a precise reflection of why DR and bit-depth are different.


Quote from: chmeelol. no. DualISO is changing the ampification per sensel-line. both different amplificated "pictures" are limited in their dynamicrange, one's losing data in the lowlights, the other loses values in full-well-sensor-area (highlights). they're kind of preprocessed.

Thanks for the explanation of Dual ISO, but I already understood the basics of how it works.

As I said, Dual ISO gives a phenomenal increase to the capture dynamic range, yet the bit depth doesn't change.  That fact is absolutely true.

However, Dual ISO does not change the dynamic range of each pixel in the sensor (I never claimed that it does).  In addition, with Dual ISO, one is sacrificing resolution and color depth along the vertical axis for the increase in capture DR.  I would also guess that Dual ISO makes the images more vulnerable to aliasing/moire.

By the way, a few years ago Panavision was working on its "Dynamax" HDR sensor, which used extra pixel groups of differing exposures, somewhat similar to Dual ISO.  "Dynamax" is now the name of Panavision's sensor division (don't know what became of the HDR sensor).


Quote from: chmeebut you said "zillions of real life examples". fine.

As I said, there are countless examples in which dynamic range and bit-depth are independent.  Dual ISO is only one.

The situation that I gave above of setting a camera to its lowest and highest bit depth while yielding the same DR is another case.  There are many cameras that have such capability, hence there are many such examples of how bit-depth is independent from DR.

The above scenario of an older camera with 16-bit depth and low capture DR compared to the capture DR of today's 16-bit cameras is yet another case of how DR and bit-depth are independent.  There are many old and new cameras to compare, thus there a numerous examples demonstrating the independence between DR and bit-depth.

We haven't even touched on the differing dynamic ranges of digital systems and digital displays that have identical bit depths.

Again, we need look no further than ML for another example in which capture DR is independent from bit-depth -- HDR video.  With HDR video, the DR increases, yet the bit-depth remains unchanged.  Of course, we are not increasing the actual dynamic range of the sensor, and we sacrifice longer shutter speeds and suffer motion artifacts in exchange for the DR increase.

These general examples (along with the specific ML examples) are fairly clear, so I don't see the need to site particular digital cameras/systems to prove the point.  However, I will, if you like.


Quote from: chmeea switch with two states has no dynamic range.

Two shades can have dynamic range.


Quote from: chmeehow do you describe dynamicrange between on and off? you need a "lowest" measurable value (not zero!)

Who said that the lower shade is "off/zero?"

However, zero is a given in any system, so, you are right, it should be considered as a third shade.


Quote from: chmee[/b]. leads to at least three states (or you said: two shades). means: [three states] 0, 0.5 and 1. whats the dynamicrange of that? other example: 4 states (2bit): 0, 0.33, 0.66, 1 (=1/0.33).. (16384 states) 14bit -> 1:16383 -> ln(16383)/ln(2) = 13.99 ELV

There is no way to quantify the specific dynamic range of any of those systems, unless the noise level is known.  On the other hand, it seems that these examples demonstrate how dynamic range can relatively remain the same while the bit depth is changed.


Quote from: chmee"color" is just an electromagnetic property. if you measure three separate frequencies by its amount (here: photons), you just measure the dynamicrange in finite frequencies.. (please dont talk about the simplifaction from graphic-cards-world)

Well, if you take color on  the "photon" level at the subject, you are talking about infinite frequencies.

However, my statement to which you responded involved the fact that color depth and bit depth are two different properties, so we are primarily referring to digital imaging systems.  For the sake of simplification, let's address RGB digital imaging systems, and disregard alpha channels, Bayer interpolations or any obscure digital color systems.  Let's also stick to simple raw imaging and not involve compression, dithering or any other such modifications.

Color depth in digital systems is a result of two primary factors:  resolution and bit-depth.  Here is the basic mathematical relationship:
Color Depth = (Resolution x Bit Depth)3

So, a small increase in resolution can yield a huge increase in color depth.  Of course, an increase in bit-depth also boosts the color depth.

That's pretty much it.

chmee

if its played, does it have the dynamic range? we see all the day movies, wheres used (H)DR while recording, then squeezed/processed into a LDR-file. by definition, they dont have the DR. so i said for simplification of the question (in the first entry): it looks like great DR in a 8bit-File, but they dont have (example: flaat_12)

@tupp now i have to go, answering later.
[size=2]phreekz * blog * twitter[/size]

tupp

Quote from: Audionut on March 21, 2014, 02:06:00 AMDR describes the difference between the highest recorded signal, and the noise floor.

Basically agree.  Might word it a little differently.


Quote from: AudionutBit depth, in a nut shell, describes the resolution (accuracy), of the digital system.

I would be with you a lot more if it was worded, "Bit depth describes the 'resolution-per-channel' of the digital system."  Really, bit-depth is the number of possible level increments per channel in a digital system.


Quote from: AudionutWith increased accuracy, you can capture increased DR.  They are separate components, but they are intertwined.

I am afraid that we differ strongly here.  Dynamic range and bit depth are two different, independent properties.  Increasing the bit depth does not yield an increased dynamic range (in most practical cases).


chmee

Quote from: tupp on March 21, 2014, 10:41:13 AM
I am afraid that we differ strongly here.  Dynamic range and bit depth are two different, independent properties.  Increasing the bit depth does not yield an increased dynamic range (in most practical cases).
*!* i'm with you on this part. BUT practically all producers combine a sensor with a a/d-converter, that is able to at least preserves the needed accuracy. but the inversion (low bitdepth with high dynamic range) is wrong.

every 8bit-picture of hdr-data (as it is in a dslr) is just a simplification, based on processing. every example you'd show is just a reinterpretation, it was a high dynamic range scene (your brain helps you to say that). if i'd show this 8bitHDR on a projector, the dynamic range of lowest to highest value wouldnt ever reach the old dynamic range of the original data, no matter if 500$ or 50.000$ projector.. did you've seen ever a high dynamic range picture? me not. everytime it was a nonlinear interpretation. Raw-Converter with preview? Lightroom fi? reinterpretation!) beside logarithmic/gamma there is no nonlinear mapping, that is interesting to us. as i said before, visual sensors are linear. four times the amount of photons, four times the read-out-value .

[older entry] no need to discuss about it, but this:
Quote
Quotehow do you describe dynamicrange .. you need a "lowest" measurable value (not zero!)

[three states][1,5bit lol] 0, 0.5 and 1. whats the dynamicrange of that?
other example: 4 states (2bit): 0, 0.33, 0.66, 1 (=1/0.33)..
(16384 states) 14bit -> 1:16383 -> ln(16383)/ln(2) = 13.99 ELV
On the other hand, it seems that these examples demonstrate how dynamic range can relatively remain the same while the bit depth is changed.
[and]
Color Depth = (Resolution x Bit Depth)³
ok. i'm out.

@CyJackX
as i said, the difference is mostly in postprocessing. if you do some, fi 14-bit raw gives you a lot more versatility, where you got quite no space for adjustements with 8bit-data. finally you're ending in a 8bit compressed visualization. do your test with flaat_12 or other s- or c-curved presets. after that, you will recognise, its not nearly the same.

regards chmee


[size=2]phreekz * blog * twitter[/size]

tupp

Quote from: chmee on March 21, 2014, 03:00:44 PMpractically all producers combine a sensor with a a/d-converter, that is able to at least preserves the needed accuracy. but the inversion (low bitdepth with high dynamic range) is wrong.

Sensors are often sold without the A/D convertor.

Don't know what is meant by, "(low bitdepth with high dynamic range) is wrong."  Cameras exist which have a higher dynamic range (yet lower bit depth) than other cameras featuring higher bit depth.  In addition, a camera can allow one to choose different bit depths while its dynamic range remains unchanged -- so one can have a lower bit depth with a higher dynamic range.


Quote from: chmeeevery 8bit-picture of hdr-data (as it is in a dslr) is just a simplification, based on processing. every example you'd show is just a reinterpretation, it was a high dynamic range scene (your brain helps you to say that).

Heck, every time one takes the signals from a sensor and digitally maps them to a chosen bit depth, one is "interpreting."  However, such "interpretation" would not change the capture range of the sensor nor necessarily change the dynamic range of it's analog output.

So, one can map a sensor's signals to a variety of bit, depths but that mapping doesn't change the dynamic range (hopefully) -- bit depth and dynamic range are two different properties.


Quote from: chmeeif i'd show this 8bitHDR on a projector, the dynamic range of lowest to highest value wouldnt ever reach the old dynamic range of the original data, no matter if 500$ or 50.000$ projector..

Give me almost any projector that has contrast adjustment and let me choose two shades from a grey scale in the scene.  I will then create a greater relative amplitude range between the two projected shades, versus the two shades in the scene.


Quote from: chmeedid you've seen ever a high dynamic range picture?

Yes.  I have seen analog and digital images that have a higher dynamic range than other images.


Quote from: chmeeme not. everytime it was a nonlinear interpretation. Raw-Converter with preview? Lightroom fi? reinterpretation!) beside logarithmic/gamma there is no nonlinear mapping, that is interesting to us.

Not sure if Lightroom and similar programs change the actual dynamic range of the signal or if such programs just change the contrast/curve within the given dynamic range of the signal.

Furthermore, merely remapping gamma/curves in Lightroom (and similar programs) does not affect the bit depth per se -- the bit depth remains unchanged until it is intentionally reduced (usually when exported).  So, one is merely remapping tones within the given bit depth.

Also, I do not presume to declare which type of non-linear mapping is interesting to us, but the ones you mentioned are in wide use probably for practical reasons.


Quote from: chmeeas i said before, visual sensors are linear. four times the amount of photons, four times the read-out-value .

That's true for the most part.  Response curves are not perfect, and they breakdown at the extremes.

Not sure what is the point.


Quote from: chmee[older entry] no need to discuss about it, but this:On the other hand, it seems that these examples demonstrate how dynamic range can relatively remain the same while the bit depth is changed.
[and]
Color Depth = (Resolution x Bit Depth)³

ok. i'm out.

Were you not demonstrating mapping different bit depths with the same sensor?  Does the sensor's capture range and dynamic range change just because the output is mapped with more or less bit depth increments?  It seems to me that you were demonstrating my point.

In regards to the color depth formula (for RGB systems), you can dismiss it, but it still remains a fact.