Reading 14 bit RAW

Started by ilia3101, March 24, 2017, 09:27:09 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Danne

I think metadata is present in later builds. Not exactly sure. If not you can always do the mlvfs route.
Check line 660.
https://bitbucket.org/dmilligan/mlvfs/src/24ebdf591dba0a0431cf1fb674b3a8fcd2730b62/mlvfs/dng.c?at=master&fileviewer=file-view-default

ilia3101

Ah ok, thanks for pointing me to that.

Will try to remember for next time: always look at MLVFS source before asking :D

So it turns out there is no accurate way to detect mv720 :(

This approach will fail in the case of something like: non mv720 footage at 1:2.35 aspect

Maybe I'll include FPS in the equation and allow user to override it.

dmilligan

MLVFS does not check the recorded resolution, it checks the resolution of the video mode (in raw_info struct) only way for this to be wrong is if one can adjust the line skipping parameters for the video mode such as in the crop_rec and 4K branch. That is why there was new metadata added in these branches for precise specification of crop and resolution parameters: http://www.magiclantern.fm/forum/index.php?topic=17021.msg181639;topicseen#msg181639

ilia3101

:o RAWC looks useful.

Anyway normal MLV headers...

So the RAW image's (unstretched) pixel height = frame_headers->rawi_hdr.raw_info.active_area.y2 - frame_headers->rawi_hdr.raw_info.active_area.y1;
Is the value of frame_headers->rawi_hdr.yRes different?
Or are the values equal in all cases?

Basically trying to find out if there's a true/false way of being sure if the clip needs unsqueezing, or does it just have to be guessed from aspect ratio and other known info?

Danne

Not sure what you mean. Either you run by checking aspect ratio info according to mlvfs and end up with:
if(aspect_ratio > 2.0 && rawH <= 720)
or you probably need to check into the later metadata structure(link provided from dmilligan).
The case where mlvfs calculations will be wrong is when crop_rec module is set. The module will tell the exact same aspect ratio as squeezed footage only the footage isn´t squeezed. In switch user have to choose manually. At least for the eosm. Havn´t really gotten to other cams yet :).

Danne

I have a painfully slow connection and shortening MLV files doesn´t keep RAWC metadata so I´m uploading full MLV samples. Meanwhile. Here is metadata from the two samples coming from crop_rec_4k branch. Every crop mode has its own metadata representation. Check RAWC at the bottom. Perfect metadata.

*Sample files at the bottom

mv720(squeezed)
MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: '/Users/dan/Desktop/MLV_files/test_Ilia/mv720.MLV'
   - Verbose messages
   - Verify file structure
   - Dump all block information
File /Users/dan/Desktop/MLV_files/test_Ilia/mv720.MLV opened
File /Users/dan/Desktop/MLV_files/test_Ilia/mv720.M00 not existing.
Processing...
File Header (MLVI)
    Size        : 0x00000034
    Ver         : v2.0
    GUID        : 13170075225100483085
    FPS         : 59.940000
    File        : 0 / 0
    Frames Video: 209
    Frames Audio: 0
    Class Video : 0x00000001
    Class Audio : 0x00000000
Block: RAWI
  Offset: 0x00000034
    Size: 180
    Time: 1.106000 ms
    Res:  1920x672
    raw_info:
      api_version      0x00000001
      height           692
      width            2080
      pitch            3640
      frame_size       0x00266F60
      bits_per_pixel   14
      black_level      2047
      white_level      16200
      active_area.y1   20
      active_area.x1   146
      active_area.y2   692
      active_area.x2   2078
      exposure_bias    0, 0
      cfa_pattern      0x02010100
      calibration_ill  1
Block: RAWC
  Offset: 0x000000e8
    Size: 32
    Time: 1.119000 ms
    raw_capture_info:
      sensor res      5760x3840
      sensor crop     1.00 (Full frame)
      sampling        5x3 (bin 5 lines, bin 3 columns)




mv720(not squeezed)
MLV Dumper v1.0
-----------------

Mode of operation:
   - Input MLV file: '/Users/dan/Desktop/MLV_files/test_Ilia/crop_rec.MLV'
   - Verbose messages
   - Verify file structure
   - Dump all block information
File /Users/dan/Desktop/MLV_files/test_Ilia/crop_rec.MLV opened
File /Users/dan/Desktop/MLV_files/test_Ilia/crop_rec.M00 not existing.
Processing...
File Header (MLVI)
    Size        : 0x00000034
    Ver         : v2.0
    GUID        : 13157316559061766588
    FPS         : 59.940000
    File        : 0 / 0
    Frames Video: 144
    Frames Audio: 0
    Class Video : 0x00000001
    Class Audio : 0x00000000
Block: RAWI
  Offset: 0x00000034
    Size: 180
    Time: 0.842000 ms
    Res:  1920x632
    raw_info:
      api_version      0x00000001
      height           692
      width            2080
      pitch            3640
      frame_size       0x00266F60
      bits_per_pixel   14
      black_level      2047
      white_level      16200
      active_area.y1   60
      active_area.x1   146
      active_area.y2   692
      active_area.x2   2078
      exposure_bias    0, 0
      cfa_pattern      0x02010100
      calibration_ill  1
Block: RAWC
  Offset: 0x000000e8
    Size: 32
    Time: 0.858000 ms
    raw_capture_info:
      sensor res      5760x3840
      sensor crop     1.00 (Full frame)
      sampling        1x1 (read every line, read every column)



Uploaded*
mv720 sample
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mv720.MLV


Crop rec sample
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec.MLV


Files are compressed in post so you might wanna decompress the MLV files:
mlv_dump -d -o output_mv720.MLV mv720.MLV

dmilligan

Quote from: Ilia3101 on July 24, 2017, 10:54:22 AM
Is the value of frame_headers->rawi_hdr.yRes different?
Yes. The xRes and yRes are the recorded resolution (user selectable). The raw_info stuff is constant per video mode, and has nothing to do with the actual recorded resolution.

ilia3101

Quote from: dmilligan on July 24, 2017, 01:29:42 PM
The raw_info stuff is constant per video mode, and has nothing to do with the actual recorded resolution.
Aha! I got it now - my interpretation: it's basically like the maximum available sensor resolution in that mode... I think I see how it works now.

@Danne thank you so much for uploading those files! They are really useful to experiment, especially the RAWC files, don't know where else I could have found unusual ones like that. Thanx ;D
And the headers you printed in mlv_dump seem to confirm what dmilligan said, which is nice :)

...downloading the files right now

EDIT:
Are the files losslessly compressed? I get this kind of pattern when decoding both:



(no unsqueezing applied btw)

EDIT 2: Actually read your post thoroughly, sorry missed it before... never mind :-X

Danne

No problem.
Filmed 14bit uncompressed then compressed in post to be able to upload. Mlv_dump for president :).
Just decompress after downloading.
Tell me if you want the other modes from crop rec mode.

BTW. @g3gg0, @a1ex.  Shortening mlv files doesn't include RAWC metadata in the shortened file.

g3gg0

Quote from: Danne on July 24, 2017, 05:58:41 PM
BTW. @g3gg0, @a1ex.  Shortening mlv files doesn't include RAWC metadata in the shortened file.

thanks! will fix it.
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

ilia3101

@Danne, (Having trouble decompressing :() I was unable to use mlv_dump to decompress it with what you showed: mlv_dump -d -o output_mv720.MLV mv720.MLV
The output was the same as input, still had the compression. I tried compiling mlv_dump.c from modules/mlv_rec in a recent 10-12bit branch clone, that version still did not work.
I think I need a specific new version of mlv_dump? If so... which one? Should I try mlv_dump on steroids?

Danne

Yes, steroid should work.
Decompression function by the way is probably in crop_rec_4k branch.
@g3gg3. Thanks, looking forward to see RAWC added.

DeafEyeJedi

Quote from: g3gg0 on July 24, 2017, 06:48:23 PM
thanks! will fix it.


Looking forward to it @g3gg0 and much thanks to @Danne for pushing @Ilia3101's project ahead. :D
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

ilia3101

Ok, I'm having another go at implementing camera matrices. When I tried before, my results were all purple, but now I think about it, I realise, the matrix probably transformed the footage to XYZ with some kind of white point adjustment, so when I did the same WB multiplication on that XYZ colour that I do on raw space, it would increase the blue and red too much, which the matrix had already done.

So, does anyone know what temperature/white point the result of matrices 1 and 2 is? Is it 6500k/D65? What is the difference between 1 and 2? I looked at DNG spec, not really sure which one I need, but I know they both convert to raw to XYZ (once inverted)

Also, about this useful thing:
struct cam_matrices {
    char * camera;
    int32_t ColorMatrix1[18];
    int32_t ColorMatrix2[18];
    int32_t ForwardMatrix1[18];
    int32_t ForwardMatrix2[18];
};

Are ForwardMatrix1 and ForwardMatrix2 the same as ColorMatrix1 and ColorMatrix2, but inverted?

I'm almost there, matrix infrastructure in the app is there, just needs to be set up correctly.

(and thanks everyone)

ilia3101

My suspicions are confirmed!

I've mostly figured it out now.

This is the result of using inverted colourmatrix2 and no white balance adjustment:



I know this gross looking crap as XYZ (or SLog :P), but what I don't know is: what is the white balance temperature of this? The DNG spec holds no information other than matrix 1 and 2 will create different temperature images.

Can anyone tell me? I promise this is the last question :P

Also: which function can give me XYZ white balance multipliers from MLVFS dng.c?

dmilligan

DNG only has RGB multipliers for white balance (AsShotNeutral). MLVFS has some functions borrowed from ufraw for computing RGB multipliers from Kelvin, and they are not perfect.

ilia3101

Thanks, I think I've found a solution for multipliers now.

Right now my white balance matrix for raw space is created through this process:
(debayered and black level corrected)raw space {1,0,0, 0,1,0, 0,0,1} -> convert to xyz space (using colourmatrix2) -> convert to cone/LMS space(using CIECAM02) -> WB multiply -> back to xyz (inverted CIECAM02) -> convert to RGB (using matrix from dng.c) -> done

As far as I can tell from all I've learnt from (mostly)g3gg0 and others, this is correct... but it looks horrible.

Red tones don't exist anymore:




The pretty image I showed before looks like this:



As you can see, awful apple photos on the right is even doing a better job - look there is no red highlights in the flower in my app :'(

Is my order correct?

Danne

@Andy600 would know all there is to know about those matrices.
Also check dcraw sources. Only uses one color matrix.
Unified mlv_dump uses that one matrix as well.

g3gg0

@Ilia3101:
dont know if they already do some saturation correction etc. tried loading in e.g. lightroom or daktable without any correction applied?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: [email protected]
ONLY donate for things we have done, not for things you expect!

ilia3101

I tried other apps, all showed much nicer tones than MLV app with matrices, for now i've disabled them and put the whole matrix thing on the back burner for a little more time. I was looking at old posts in this thread, and realised there's still a little more to white balance than what i was doing...

Quote from: g3gg0 on April 30, 2017, 02:51:30 PM
/* RAW --> RAW-WB --> XYZ --> Kelvin-WB --> XYZ --> (s)RGB --> RGB-WB */
CamToRgbMatrix = WhiteBalanceMatrix * RGBToXYZMatrix.Inverse() * xyzKelvinWb * XYZToCamMatrix.Inverse() * WhiteBalanceMatrixRaw
with:
Matrix xyzKelvinWb = coneDomain.Inverse() * xyzScale * coneDomain;
What's the RAW-WB stage? why is there two white balance steps: RAW-WB and Kelvin-WB?