Author Topic: 12-bit (and 10-bit) RAW video development discussion  (Read 764382 times)

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1741
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1025 on: January 08, 2017, 04:04:10 PM »
@Geggo
I did use the same mlv_dump as in your link.
The build from 25th december for osx

When I use that build on 12 bit mlv's, I'll get 14 bit dng's  ???
Which is nice, as in you can view them normal in finder, but the downside is that file size is also increased  :P

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1026 on: January 08, 2017, 05:23:40 PM »
@Danne
mlv_dump --dng --black-fix=204 -o %Input%.raw %Input%.mlv
I suspect is something to do with ACR.  If I use exiftool then it works..  I was hoping to eliminate the exiftool step.
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7296
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1027 on: January 08, 2017, 05:31:01 PM »
Tried putting --black fix command before the --dng?
Are you on mac? Could you upload a dng?

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7296
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1029 on: January 08, 2017, 05:55:49 PM »
I don,t think you are doing it right. Try something like this. It absolutely works over here.
Code: [Select]
mlv_dump --black-fix=2048 --dng My.MLV

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1030 on: January 08, 2017, 06:04:21 PM »
I don't remember the 50D ever recording in 10x but it does now  ???

So it joins the 5D2 and 7D as cameras that shouldn't be working but are, sometimes, at certain settings? Does the 50D have zoom mode? Zoom in the LiveView to 5x or 10x and record video? I believe that's the setting that's working on the 5D2 but I'm not sure.

Those use EVF_STATE being DIGIC4. All other DIGIC4's using LVSTATE due to the missing EVF_STATE are the ones that are not working. a1ex has the state diagrams published on his bitbucket repo and allocating the buffer is not the problem

So the 60D, 600D, 1100D use EVF_STATE while the 5D2, 7D, 50D, 500D, 550D use LVSTATE and are missing the EVF_STATE? Learning how the 600D was resolved #define DEFAULT_RAW_BUFFER MEM(MEM(0x51FC)) isn't going to help with the 550D even though lv_raw_dump starts out very similar on both cameras? I keep getting lost and not knowing how the 600D address was discovered I'm clueless when it comes to trying to figure out these platforms. After @dmilligan explained how he traced lv_raw_dump on the EOSM figuring it out on the 700D and 650D was relatively easy. (Thanks to @eNnvi for explaining dmilligan's explaination  :D )

https://bitbucket.org/hudson/magic-lantern/pull-requests/781/650d-and-700d-enable/diff
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1031 on: January 08, 2017, 06:05:50 PM »
@Danne  No Go!
As I said it seems to be something wrong with ACR..  If I use the exiftool then it works.
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3183
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1032 on: January 08, 2017, 06:11:43 PM »
When I use that build on 12 bit mlv's, I'll get 14 bit dng's  ???
vs.
Input file is 12 bit mlv, output file is a single frame 14 bit mlv

do we talk about the MLV -> MLV(averaged) script that produces 14 bit files
or about DNG conversion which is 14 bit only yet?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3183
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1033 on: January 08, 2017, 06:13:44 PM »
@Danne  No Go!
As I said it seems to be something wrong with ACR..  If I use the exiftool then it works.

is it better with '--no-fixcp --no-stripes' ?
Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7296
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1034 on: January 08, 2017, 06:25:09 PM »
@Renatophoto.
This command can,t work which you put up.
Code: [Select]
mlv_dump --dng --black-fix=204 -o %Input%.raw %Input%.mlvYou can,t even create a new MLV with another black level. In case it would have worked it would look like this.
Code: [Select]
mlv_dump --black-fix=2048 -o Output.MLV Input.MLV
It only works when spitting out dng files. Try exactly what I put out here.
Code: [Select]
mlv_dump --dng --black-fix=2048 INPUT.MLV
If still not working upload part of you mlv file

@Levas
Quote
Input file is 12 bit mlv, output file is a single frame 14 bit mlv
I thought this was to be expected, especially since g3gg0 updated code so all bits(10/12/14) would work with the 14bit code in mlv_dump?



RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1035 on: January 08, 2017, 06:45:03 PM »
Danne: I tried your suggestion and it does not work.
g3gg0: I tried but no luck either.

I think it something related to my system and ACR.  I will use exiftool to do the black level correction until it is fixed directly on the mlv_rec.mo
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1741
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1036 on: January 08, 2017, 06:46:57 PM »
@Geggo

MLV conversion to dng's gives 14bit dng's even with 12 bit mlv's.
So 12 bit MLV gives 14 bit dng frames
Doesn't matter if average is used or not, always end up with 14 bit dng's

The MLV_dump build I used before, 20 November 2016 build, gave 12 bit dng's or 10 bit or 14 bit, depending whatever bit depth the mlv file was recorded.


g3gg0

  • Developer
  • Hero Member
  • *****
  • Posts: 3183
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1037 on: January 08, 2017, 08:17:10 PM »
so MLV averaging produces the correct format, right?
i am asking because you reported the generated MLV has 14 bits although it should have 12 bits.

DNGs are *always* 14 bit in the branches i know. maybe you used mlv_dump builds from other sources.
do not know about the pending PR if it would produce 12bpp DNGs


Help us with datasheets - Help us with register dumps
magic lantern: 1Magic9991E1eWbGvrsx186GovYCXFbppY, server expenses: paypal@g3gg0.de
ONLY donate for things we have done, not for things you expect!

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1038 on: January 08, 2017, 10:52:07 PM »
@A1ex. r2 and r4 looks good. Bottom three not so.

Agreed!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1039 on: January 09, 2017, 04:46:42 PM »
3X resolution testing  of latest dfort:  magiclantern-raw_video_10bit_12bit.2017Jan05.5D3123

In liveview (zoom 3x) the ML Grey Scale preview flickers when selecting aspect ratios beyond 16:9 i.e 1.85:1, 2:1, etc

Using cinnema ratios like 2.35:1 In liveview (zoom 3x) in ML Grey Scale preview the areas above and below the video, which should be black, have images (lines here) on the right half of the screen.  see below:



With big resolutions I get some bad frames:

3008x1320:


3520x1320:


Best Regards,
Renato
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1040 on: January 09, 2017, 05:56:46 PM »
3X resolution testing  of latest dfort:  magiclantern-raw_video_10bit_12bit.2017Jan05.5D3123

Also ML Grey Scale turns to Canon preview while recording..
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

Levas

  • Contributor
  • Hero Member
  • *****
  • Posts: 1741
  • 6d - Nightly build user
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1041 on: January 09, 2017, 06:23:30 PM »
@Geggo
Quote
so MLV averaging produces the correct format, right?

Yes, the resulting average MLV has the right format, the average MLV is 12 bit, when input MLV is 12 bit.

Only thing new to me was that when extracting dng's, it gives 14 bit dng's no matter what the input is, 10/12 or 14 bit.
I was used to a MLV_dump which gave dng's in the same bit depth as the recording was.



budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1042 on: January 10, 2017, 07:15:47 AM »
I made a test with a card Tone 14/12/10 bits
5d Mark III 1.1.3
Build January 5

I hope this can be usefull for the comunity.

Full info into the video.


MitchLally

  • New to the forum
  • *
  • Posts: 31
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1043 on: January 10, 2017, 11:50:23 AM »
I made a test with a card Tone 14/12/10 bits
5d Mark III 1.1.3
Build January 5

I hope this can be usefull for the comunity.

Full info into the video.


Is this before or after the black level fix posted by A1ex above for 10 bit files. It is supposed to remove the purple colour cast in 10 bit.

budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1044 on: January 10, 2017, 04:38:09 PM »
I didn't apply any correction, no black level 2048 in 5d Mark III.
But if @a1ex need a new test I can make it again!

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7296
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1045 on: January 10, 2017, 04:48:18 PM »
Here is the latest version of mlv_dump with g3gg0 latest commits. It,s vital to specify black level.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump

Source: https://bitbucket.org/hudson/magic-lantern/commits/75f61f30b6e2a6fb1a136d538b8fe190919024a9?at=unified

Specify like this if working with a 5D mark III
mlv_dump --black-fix=2048 --dng your.MLV
Correction:
mlv_dump --black-fix=2040 --dng your.MLV

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1046 on: January 10, 2017, 04:50:58 PM »
... 2040 ;)

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7296
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1047 on: January 10, 2017, 04:55:12 PM »
Thanks, corrected.
WHat is the black level number for 12bit? The same as for 10bit?

Tracerman

  • New to the forum
  • *
  • Posts: 4
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1048 on: January 10, 2017, 05:51:10 PM »
Hi,

I've tested 10bit and 12bit mlv_rec on the 550D : magiclantern-raw_video_10bit_12bit.2017Jan05.550D109

10 bit recording is indeed eagerly awaited on this platform, because 1280x544(2.35) at 23.98 fps makes about 20MBps
which is the maximum rate I can get from it (SandDisk 64Gb ExtremePro) so its pretty cool :)
big thanks to the ml devs, it gives a new life to this (ok rather old) camera but with (it seems) a lot of users still ...

I tested it in this configuration, recording goes OK (green light!) but when I try to read back it's not quite OK yet
(I've tried with mlv_play on the camera (with raw_twk loaded), with mlvfs on Linux and mlvproducer on Win7 with the same result) :


In 10 bit mode : it seems that 1 frame out of 2 is in fact the 1st recorded frame, and the other has problems on the top third of the frame.

In 12 bit mode : every frame is split vertically in 2 and offset in time and/or vertically (I can post another video if you need).

14 bit mode is all OK but recording stops after a few seconds (buffer full).

If I can be of some help in testing just let me know, having at least 10bit mlv_rec working would be awesome !

Thanks !


budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1049 on: January 10, 2017, 06:24:15 PM »
@a1ex @danne

Yes it works! 5d3.
No color pink/magenta and blue. Tested 10/12/14 bits

./mlv_dump --black-fix=2040 --dng M10-0135.MLV








image upload no limit