Author Topic: 12-bit (and 10-bit) RAW video development discussion  (Read 373818 times)

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 2788
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1625 on: March 12, 2018, 09:43:26 PM »
Worked with andy kh on the 70D and came up with this:

70D: 4B328000 - 4CFFFFFC (repeatable; tested mv1080, x5)



mv1080


5x zoom

5D3.* 7D.* 700D.115 EOSM.* EOSM2.103 500D.112

reddeercity

  • Hero Member
  • *****
  • Posts: 1570
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1626 on: March 16, 2018, 06:28:00 AM »
started to grade the experimental 12bit @1856x1044 1:1 from my 5D2 very nice , I don't what to post a lot of images here (I make post somewhere else)
so I'll post links to the orginal dng's & the export from CS6 A.E. also the graded frame exports for FCPX with
film convert pro , I used the RED MX camera source profile (closest to raw footage) .
By the way it's just my ugly old face -- so you being warned  :P
M11-0006_002688.dng  no-grade.png  full grade_black backgound   full grade_white backgound

Edit:shot at 70mm (24-70f2.8L canon) F4 200 ISO @ 1/53th second (closest I can get to 1/48) and no preprocessing with mlv_dump option

reddeercity

  • Hero Member
  • *****
  • Posts: 1570
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1627 on: March 21, 2018, 03:41:34 AM »
the images I posted are a little soft so I found a really sharp in focus frame 12bit for 5D2 1:1
Link below to the dng from mlvfs (quick mount win7) so it's 16bit & tif from A.E+ACR ,
when I got more time I'll post a true 12bit for mlv_dump .
M11-0006_006342.dng
M11-0006_06342.tif
I basically just normalize the .dng in to legal color space from a.e.

Edit: What I find amazing is the absence of any aliasing and moiré at least I don't see any
the other thing I wonder about is the native bit depth from the sensor , is it really 14bit ? or a up-sample 12bit?
If I read it right the digic 3 is native 12bit sooo very interesting , just a thought  ;D

reddeercity

  • Hero Member
  • *****
  • Posts: 1570
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1628 on: March 26, 2018, 08:16:35 AM »
Posted my 12bit experimental & digic4 development update video on 5D MarkII thread if anyone interested to see
12bit 1856x1044+audio 1:1  ;D
https://www.magiclantern.fm/forum/index.php?topic=11205.msg198906#msg198906

IDA_ML

  • Senior
  • ****
  • Posts: 320
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1629 on: March 26, 2018, 02:29:00 PM »
Nice to see that you are doing some pretty good progress on the Digic 4 cameras, Reddeercity.  And you seem to be pretty close to turning our fantastic Digic 4 cameras into real ML RAW video shooting beasts.  Keep up the good work!

I have a question.  How do you get rid of the ugly moire and aliasing issue in the 12bit 1856x1044+audio 1:1 mode? 

TrEK

  • New to the forum
  • *
  • Posts: 15
  • Canon 5d mk2 , CF Sandisk Extreme PRO 64gb 160MB/s
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1630 on: March 26, 2018, 09:54:31 PM »
Yea I know , I still convinced that it will work , I did some test the other day with the 10_12bit_broken_3xcrop_mode_2017Dec10.5D2212.zip
That I complied with the change I did (mainly the write channel ) ( the one that a1ex said not to use but works in 1:1)
I what to see what problem or crashes would happen in hopes of maybe seeing something that can help resolve this problem .
So with mlv_lite  without hdmi connected there was no issue crashes etc. ...  but with hdmi in standby ,
I would get the flicker with pink  or out of sequence image(see below) when I start recording raw video the flicker stops and liveview is good to record from hdmi .
I can't say the same as  full mlv + audio I got a memory overflow  ??? didn't matter if hdmi was connected or not .
printed a message on screen "mlv_rec.c line 161" has to do with the SRM memory
maybe audio pushed it over the edge not sure .
 

Next I'm going to be looking at big LVState diagram , I did look them over a year ago but really didn't understand that much now I hope I can make head or tales out of it


hello!
i have such trouble too
what you did with it ?>

TrEK

  • New to the forum
  • *
  • Posts: 15
  • Canon 5d mk2 , CF Sandisk Extreme PRO 64gb 160MB/s
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1631 on: March 26, 2018, 09:56:09 PM »
Nice to see that you are doing some pretty good progress on the Digic 4 cameras, Reddeercity.  And you seem to be pretty close to turning our fantastic Digic 4 cameras into real ML RAW video shooting beasts.  Keep up the good work!

I have a question.  How do you get rid of the ugly moire and aliasing issue in the 12bit 1856x1044+audio 1:1 mode?
where i can download CORRECT 10/12 ML for Canon 5dm2 ?

reddeercity

  • Hero Member
  • *****
  • Posts: 1570
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1632 on: March 28, 2018, 05:34:00 AM »
where i can download CORRECT 10/12 ML for Canon 5dm2 ?
search it , it's not hard to find  ;)
hint ,maybe on this page , clue : has something to do with "broken 3xcrop_mode" and you may have or not quoted it in a post  ::)

reddeercity

  • Hero Member
  • *****
  • Posts: 1570
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1633 on: March 28, 2018, 07:20:15 AM »
Nice to see that you are doing some pretty good progress on the Digic 4 cameras, Reddeercity.  And you seem to be pretty close to turning our fantastic Digic 4 cameras into real ML RAW video shooting beasts.  Keep up the good work!
It was not just me @dfort had a lot to do with it not to mention a1ex , g3gg0 and other's .
I have a question.  How do you get rid of the ugly moirĂ© and aliasing issue in the 12bit 1856x1044+audio 1:1 mode?
read here , tried to explain it in this post
https://www.magiclantern.fm/forum/index.php?topic=11205.msg198964#msg198964

TrEK

  • New to the forum
  • *
  • Posts: 15
  • Canon 5d mk2 , CF Sandisk Extreme PRO 64gb 160MB/s
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1634 on: March 28, 2018, 08:58:08 PM »
search it , it's not hard to find  ;)
hint ,maybe on this page , clue : has something to do with "broken 3xcrop_mode" and you may have or not quoted it in a post  ::)

sorry but i dont understand

i have raw_video_10bit_12bit_LVState-wip.2018Feb04.5D2212 but its bot work correctly

reddeercity

  • Hero Member
  • *****
  • Posts: 1570
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1635 on: March 29, 2018, 06:04:56 AM »
No that's not it , you already posted it in quote from me .
The reason I'm being a little difficult is I want you search for it , Because it could be dangerous , I have accepted the risk that it could fry the camera or corrupt  the firmware .

Look thought your recent posts (this page)

check post #1623
Quote
As we know the write channel  I'm using (0x2)has a very small amount of data being written to as a1ex said
Quote
What does all this mean , not sure but it is usable , but then again it could blowup and you get to kept all the parts  :P :P :P
If anyone want's to use it for a project you must test test test & hope doesn't it lockup (more then likely it would be ok) but Great Caution must be Observed
I'm serious about this , no joke !

If all else fails look in my bitbucket downloads

reddeercity

  • Hero Member
  • *****
  • Posts: 1570
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1636 on: April 01, 2018, 06:51:28 AM »
Happy Easter everyone !! :))

Found some time now to get back working on digic4 issue ,
as promised I extracted a few true 12bit frames from my 5D2 on the experimentally 12bit 1:1 mode with the latest mlv_dump on the download page .
These are from the same mlv 30GB file I made . I uploaded the same frames as before but in true 12bit so everyone can compare to the mlvfs 16bit extraction I made before.
Exiftool info , yea it' only 2.8MB pre frame , where as the 14bit is around 3.8MB
Code: [Select]
ExifTool Version Number         : 10.45
File Name                       : M11-0006_002688.dng
Directory                       : C:/Newfolder
File Size                       : 2.8 MB
File Modification Date/Time     : 2018:03:31 22:05:13-06:00
File Access Date/Time           : 2018:03:31 22:41:30-06:00
File Creation Date/Time         : 2018:03:31 22:41:30-06:00
File Permissions                : rw-rw-rw-
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Subfile Type                    : Full-resolution Image
Image Width                     : 1856
Image Height                    : 1044
Bits Per Sample                 : 12
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Fill Order                      : Normal
Make                            : Canon
Camera Model Name               : Canon EOS 5D Mark II
Strip Offsets                   : 1198
Orientation                     : Horizontal (normal)
Samples Per Pixel               : 1
Rows Per Strip                  : 1044
Strip Byte Counts               : 2906496
Planar Configuration            : Chunky
Software                        : MLV_DUMP CDNG
Modify Date                     : 2018:03:11 00:08:36
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 0 1 1 2
Exposure Time                   : 1/53
F Number                        : 4.0
ISO                             : 200
Sensitivity Type                : ISO Speed
Exif Version                    : 0230
Subject Distance                : 172 m
Focal Length                    : 70.0 mm
Focal Plane X Resolution        : 1283.070596
Focal Plane Y Resolution        : 1302.713987
Focal Plane Resolution Unit     : inches
Lens Model                      : EF24-70mm f/2.8L USM
DNG Version                     : 1.4.0.0
Unique Camera Model             : Canon EOS 5D Mark II
Black Level                     : 448
White Level                     : 4050
Default Scale                   : 1 1
Default Crop Origin             : 0 0
Default Crop Size               : 1856 1044
Color Matrix 1                  : 0.5309 -0.0229 -0.0336 -0.6241 1.3265 0.3337 -
0.0817 0.1215 0.6664
Color Matrix 2                  : 0.4716 0.0603 -0.083 -0.7798 1.5474 0.248 -0.1
496 0.1937 0.6651
As Shot Neutral                 : 0.5017448176 1 0.5119116727
Baseline Exposure               : 0
Camera Serial Number            : ********
Calibration Illuminant 1        : Standard Light A
Calibration Illuminant 2        : D65
Active Area                     : 0 0 1044 1856
Forward Matrix 1                : 0.8924 -0.1041 0.176 0.4351 0.6621 -0.0972 0.0
505 -0.1562 0.9308
Forward Matrix 2                : 0.8924 -0.1041 0.176 0.4351 0.6621 -0.0972 0.0
505 -0.1562 0.9308
Time Codes                      : 00:01:52.00
Frame Rate                      : 23.976
Reel Name                       : M11-0006.mlv
Baseline Exposure Offset        : 0
Aperture                        : 4.0
CFA Pattern                     : [Red,Green][Green,Blue]
Image Size                      : 1856x1044
Megapixels                      : 1.9
Scale Factor To 35 mm Equivalent: 1.0
Shutter Speed                   : 1/53
Circle Of Confusion             : 0.029 mm
Depth Of Field                  : inf (33.77 m - inf)
Field Of View                   : 28.0 deg
Focal Length                    : 70.0 mm (35 mm equivalent: 72.1 mm)
Hyperfocal Distance             : 42.00 m
Light Value                     : 8.7

12bit_M11-0006_006342.dng  12bit_M11-0006_002688.dng &  corrupted   12bit_M11-0006_000001.dng , the only one I could find .

reddeercity

  • Hero Member
  • *****
  • Posts: 1570
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1637 on: April 02, 2018, 03:16:24 AM »
Further test of the 10-12bit with broken 3xCrop_mode , run some 10bit tests and no problems recording up to 1856x1248 @23.976 1:1 liveview FHD (not 3xcrop_mode)
Looks like it's continuous or very close to it , I stop it after 1200 frames . There is one consisted factor across 10 ,12 & 14bit on this version of bit reduction is every clip has the second frame corrupted no matter what bit depth , yes even 14bit after that all frames are good . I should clarify the broken 3xCrop_mode  ,will it's not really broken you can still use it to get accurate focus just when you record raw video in all bit depths every other frame is frozen with movement in the other alternating frames .

reddeercity

  • Hero Member
  • *****
  • Posts: 1570
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1638 on: April 06, 2018, 03:50:08 AM »
trying some different write channels on the D4/5d2 (10-12bit)
Code: [Select]
0x03 , 0x04 , 0x05 , 0x10 , 0x12 , 0x13 , 0x16All either error out , (raw detect error) or have corruption in liveview
Still the only one that works so far is
Code: [Select]
0x02hoping to stumble across a channel that 3xcrop works with FHD .

reddeercity

  • Hero Member
  • *****
  • Posts: 1570
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1639 on: April 07, 2018, 06:08:17 AM »
I see there's being downloads of the 10-12bit broken 3xcrop_mode with 0x02 write channel (which is not totally safe)




Can these people give some feed back please , I like to know about crashes/lockups and anything else of interest .
how is the 10bit in all video modes ? same for 12bit ? plus how it react to HDMI , FPS change , Over Ride Lock on or off , 25p , 30p ,
W/B changes , Picture Style liveview changes/adjustments .
What ever brick it , don't need to report about (digital ISO freezes the cam.) when adjusting ,
I know about it (work around ) turn mlv_rec module off reboot then adjust to pure ISO (100,200 etc. ..) .
 

jpegmasterjesse

  • New to the forum
  • *
  • Posts: 37
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1640 on: April 08, 2018, 04:06:21 AM »
I'm using it for 12-bit & dual-iso and haven't had any serious problems beyond the ones you mentioned.  I also haven't tested it in that many conditions yet - I'll try HDMI soon and report back.

reddeercity

  • Hero Member
  • *****
  • Posts: 1570
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1641 on: April 09, 2018, 05:37:14 AM »
Thanks , that sound good.
I'm using it for 12-bit & dual-iso
Never thought of trying with Dual ISO video -- you do mean video right ?

jpegmasterjesse

  • New to the forum
  • *
  • Posts: 37
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1642 on: April 09, 2018, 06:03:40 AM »
Yes - and the MLVapp developers have made it work pretty flawlessly. I can do dual-iso 12bit with darkframe subtraction to get some pretty nice frames.

I know someone questioned why I'd pair 12bit with dual iso as 12bit loses some dynamic range, but I find that to be an ideal combination, especially with continuous shooting being an option at 12.

reddeercity

  • Hero Member
  • *****
  • Posts: 1570
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1643 on: April 09, 2018, 07:42:44 AM »
Great to hear ,
Yes , I agree -- the reduce bit depth to 12bit gives cleaner shadow while retaining the hi-lights be on my vision that's recoverable in MLVProducer(the best mlv post app yet) & A.E. ACR
and of course artifact are almost all eliminated if not all ! I find with reduce bits the line skipping seem to be a non issue , this may sound strange ,
I have gut feeling that 14bit=4 trillion shades of color  is just to many shades for canon pixel binding & line skipping on 5D2
where I think 12bit=68 billion shades of colors , sort of dither the  color or soften it . That's one reason why in grading apps
I always reduce contrast & saturation heavily  (can always be adding later in the NLE) .

jai554

  • New to the forum
  • *
  • Posts: 16
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1644 on: April 14, 2018, 07:35:49 PM »
Hi,
I tried to copy the MLV rec module into the latest 60d ML firmware 1.1.2 test build
The module fail to load
Is there anything else need to be copied?

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 2788
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1645 on: April 14, 2018, 09:17:05 PM »
60D.112 just got ported last night to the unified branch. The changes need to be merged into raw_video_10bit_12bit and compiled on that branch. This is something that we'll get around to eventually but first let's see if passes some basic tests.

You're more than welcome to clone my repository and merge branches. Better yet, get the 60D working on the crop_rec_4k branch along with the sd_uhs module. Wheeeeeee!
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103 500D.112

TrEK

  • New to the forum
  • *
  • Posts: 15
  • Canon 5d mk2 , CF Sandisk Extreme PRO 64gb 160MB/s
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1646 on: April 15, 2018, 04:42:25 PM »
Yes - and the MLVapp developers have made it work pretty flawlessly. I can do dual-iso 12bit with darkframe subtraction to get some pretty nice frames.

I know someone questioned why I'd pair 12bit with dual iso as 12bit loses some dynamic range, but I find that to be an ideal combination, especially with continuous shooting being an option at 12.

how you use DUAL ISO in video recording ? and how you edit it on postproduction ?
MLVFS will be good for it ?

jpegmasterjesse

  • New to the forum
  • *
  • Posts: 37
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1647 on: April 15, 2018, 07:04:12 PM »
This: https://drive.google.com/open?id=19ZgqOpGWgs0af6HF0DFAqVh7ZYdacsbg

It's an old module that was being tested on the 5D2.  I don't have any documentation on who created it.

You can replace the old dual-iso module in your modules folder on the card with this one.  No need to reinstall ML. Just drag and replace.

You will probably have varying degrees of luck with different programs in trying to convert and view these easily.  I highly recommend MLVApp.

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 2788
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1648 on: April 15, 2018, 09:21:39 PM »
Why are we talking about Dual ISO on the 12-bit (and 10-bit) RAW video development discussion?

Let's continue discussing it here: Video - dual iso for 5D2
5D3.* 7D.* 700D.115 EOSM.* EOSM2.103 500D.112