Author Topic: How to check if a .DNG is really raw data  (Read 4965 times)

Milk and Coffee

  • Member
  • ***
  • Posts: 121
How to check if a .DNG is really raw data
« on: June 02, 2020, 09:51:21 PM »
I've received a .DNG sequence from a coworker, and the files are incredibly small (like 200kb-ish a frame.) My suspicion is that they exported a .JPG sequence, and then converted the .JPG's to .DNG (which apparently you can do that???)

How can I check if a .DNG is really raw data? Can I open in a text editor? Is there a utility that makes it easy?

Thanks all!
Canon 5D Mark II, Mac/OSX

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8659
Re: How to check if a .DNG is really raw data
« Reply #1 on: June 02, 2020, 09:56:56 PM »
Let me suggest a non-technical approach: Ask your coworker to deliver unaltered raw footage instead of DNG.

Milk and Coffee

  • Member
  • ***
  • Posts: 121
Re: How to check if a .DNG is really raw data
« Reply #2 on: June 02, 2020, 10:06:41 PM »
Thanks Walter. I already did that. Can you provide a technical approach?
Canon 5D Mark II, Mac/OSX

names_are_hard

  • Developer
  • Hero Member
  • *****
  • Posts: 716
  • Dev: 200D, 750D, 850D, 7D2
Re: How to check if a .DNG is really raw data
« Reply #3 on: June 02, 2020, 10:12:19 PM »
What OS?  If Linux or Mac, run "file" over the file.  Test with a known JPG and DNG, the output should be consistent and different.

yourboylloyd

  • Contributor
  • Senior
  • *****
  • Posts: 323
Re: How to check if a .DNG is really raw data
« Reply #4 on: June 02, 2020, 11:17:07 PM »
Just open it in Adobe camera Raw (maybe phortoshop, lightroom, or after effects) and try to color grade it. Push the highlights all the way down and shadows all the way up. A real RAW image would still hold up. An 8bit image would turn grey
Join the ML discord! https://discord.gg/H7h6rfq

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8659
Re: How to check if a .DNG is really raw data
« Reply #5 on: June 02, 2020, 11:51:35 PM »
@yourboylloyd: Can you provide some samples/screenshots showing the difference?
What happens if they messed up and converted raw to 8 bit dng or converted to 8 bit somewhere else in post and stored it as 16 bit dng after?
He gave not the slightest hint about pp...

Forensic with this kind of info?

mothaibaphoto

  • Senior
  • ****
  • Posts: 393
  • pesky kid
Re: How to check if a .DNG is really raw data
« Reply #6 on: June 03, 2020, 04:23:38 AM »
https://www.rawdigger.com/download
You can see absolute pixel values of every channel.
No need to buy/activate for that.

Luther

  • Senior
  • ****
  • Posts: 323
Re: How to check if a .DNG is really raw data
« Reply #7 on: June 03, 2020, 04:24:45 AM »
Use Rawtherapee. On the demosaicing menu, choose "bayer". If it is a raw image, you will see the bayer pattern (mozaic of RGGB). This is the easiest way.
But for the images to be 200kb, someone probably made lossy compression on those files.

Milk and Coffee

  • Member
  • ***
  • Posts: 121
Re: How to check if a .DNG is really raw data
« Reply #8 on: June 03, 2020, 06:52:55 AM »
Thanks all! I’ll try a few of those suggestions! Is there no way to open in a text editor Or something and view the code that would reveal it to be raw data?
Canon 5D Mark II, Mac/OSX

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7701
Re: How to check if a .DNG is really raw data
« Reply #9 on: June 03, 2020, 07:37:31 AM »
Use exiftool or what has already been recommended.
Dng can be lossless raw or even some lossy jpg turned into dng. Too bad you don't take the time to share samples.

Milk and Coffee

  • Member
  • ***
  • Posts: 121
Re: How to check if a .DNG is really raw data
« Reply #10 on: June 28, 2020, 08:24:38 PM »
Use exiftool or what has already been recommended.
Dng can be lossless raw or even some lossy jpg turned into dng. Too bad you don't take the time to share samples.

Sorry about the late reply! Here is a frame from that sequence!
https://www.mediafire.com/view/ve4pxi7gbd1wtra/M24-2054_1_2020-06-24_0001_C0000_000155.dng/file
Canon 5D Mark II, Mac/OSX

names_are_hard

  • Developer
  • Hero Member
  • *****
  • Posts: 716
  • Dev: 200D, 750D, 850D, 7D2
Re: How to check if a .DNG is really raw data
« Reply #11 on: June 28, 2020, 10:14:50 PM »
Here's the output from running "file":
M24-2054_1_2020-06-24_0001_C0000_000155.dng: TIFF image data, little-endian, direntries=43, height=144, bps=530, compression=JPEG, PhotometricIntepretation=YCbCr, orientation=upper-left, width=256

So I say that's JPEG inside TIFF.  Would need to see another suspected good frame to be sure the other kinds of DNG get detected sensibly as well.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7701
Re: How to check if a .DNG is really raw data
« Reply #12 on: June 28, 2020, 10:57:29 PM »
Code: [Select]
Image Width                     : 1880
Image Height                    : 1056
Bits Per Sample                 : 8 8 8
Compression                     : Lossy JPEG
Looks quite good despite compression. What was it filmed with from the beginning?

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2303
Re: How to check if a .DNG is really raw data
« Reply #13 on: June 29, 2020, 12:47:13 AM »
Interesting , Looks like sRaw compression on 5d2
Code: [Select]
Active Area                     : 0 0 1056 1880
Y Cb Cr Coefficients            : 0.299 0.587 0.114
Y Cb Cr Sub Sampling            : YCbCr4:4:4 (1 1)
Y Cb Cr Positioning             : Co-sited

This a great read & explains the compression in detail Subsampling.pdf

from the above pdf file


Code: [Select]
History Parameters  : converted from image/jpeg to image/dng, saved to new locationDidn't know you could do that

Milk and Coffee

  • Member
  • ***
  • Posts: 121
Re: How to check if a .DNG is really raw data
« Reply #14 on: June 29, 2020, 02:11:15 AM »
So I asked my coworker what his original workflow was. Originally, I asked him to deliver a DNG sequence. He exported a JPG sequence with adjustments from ACR because he felt the DNG's looked "dark." (Probably because the DNG's are previewed in linear gamma on mac?) He then batch converted the adjusted JPGs back to DNG. (He obviously doesn't know the benefits of RAW)

I hate that DNGs can hold raster images. Now ill be skeptical if I'm receiving RAW data or not. :-\

Anyway, after some talking I asked for the original DNG sequence. Below is the original RAW DNG from Canon 5D2. (File size is larger, but still 740kb.)
https://www.mediafire.com/view/lzsxznzy1aq8fjy/M24-2054_1_2020-06-24_0001_C0000_000155.dng/file

Canon 5D Mark II, Mac/OSX

names_are_hard

  • Developer
  • Hero Member
  • *****
  • Posts: 716
  • Dev: 200D, 750D, 850D, 7D2
Re: How to check if a .DNG is really raw data
« Reply #15 on: June 29, 2020, 02:44:50 AM »
Output from file for both:

M24-2054_1_2020-06-24_0001_C0000_000155.dng: TIFF image data, little-endian, direntries=43, height=144, bps=530, compression=JPEG, PhotometricIntepretation=YCbCr, orientation=upper-left, width=256
M24-2054_1_2020-06-24_0001_C0000_000155(2).dng: TIFF image data, little-endian, direntries=41, height=1056, bps=10, compression=JPEG, PhotometricIntepretation=(unknown=0x8023), manufacturer=Canon, model=Canon EOS 5D Mark II, orientation=upper-left, width=1880

It thinks they're both JPEG which feels weird.  That made me check with exiftool, which says one is "Lossy JPEG" and one is "JPEG".  Possibly jpeg2000?  I'd forgotten, but that does have a lossless mode.

Milk and Coffee

  • Member
  • ***
  • Posts: 121
Re: How to check if a .DNG is really raw data
« Reply #16 on: June 30, 2020, 12:40:34 AM »
That is weird! Doesn't ML use a JPEG compression method for making lossless RAW? Is MLV App using it too for lossless DNG? I see that when I open both in ACR, the "JPEG DNG" doesn't show Adobe's RAW profiles in the drop down. Whereas, the second DNG I posted (should be real RAW) DOES have Adobe's RAW profiles available.
Canon 5D Mark II, Mac/OSX

names_are_hard

  • Developer
  • Hero Member
  • *****
  • Posts: 716
  • Dev: 200D, 750D, 850D, 7D2
Re: How to check if a .DNG is really raw data
« Reply #17 on: June 30, 2020, 02:52:01 AM »
Don't know.  DNG uses TIFF/EP, and TIFF/EP supports lossless JPEG, lossy JPEG and other lossless formats.  I couldn't be bothered spending the time to manually analyse the sample file to dig deeper.  It's an open format so if you want to, go ahead :)

masc

  • Contributor
  • Hero Member
  • *****
  • Posts: 2104
Re: How to check if a .DNG is really raw data
« Reply #18 on: June 30, 2020, 09:05:03 AM »
Doesn't ML use a JPEG compression method for making lossless RAW? Is MLV App using it too for lossless DNG?
MLVApp uses LJ92 library for lossless footage.
5D3.113 | EOSM.202

Milk and Coffee

  • Member
  • ***
  • Posts: 121
Re: How to check if a .DNG is really raw data
« Reply #19 on: July 01, 2020, 02:18:34 AM »
Thank you masc! Then I would suspect that’s why exif is showing JPEG for both.
Canon 5D Mark II, Mac/OSX

names_are_hard

  • Developer
  • Hero Member
  • *****
  • Posts: 716
  • Dev: 200D, 750D, 850D, 7D2