Author Topic: 12-bit (and 10-bit) RAW video development discussion  (Read 749797 times)

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7252
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1050 on: January 10, 2017, 06:45:43 PM »
Could you upload the dng files? And name them 10/12/14-bit so one can differ them.
Shouldn,t 14-bit have 2048 black level? 10 and 12 can,t be both 2040 or what am I missing?

budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1051 on: January 10, 2017, 06:56:02 PM »
@Danne

I applied the same 2040 to 10/12/14 bits.

https://www.dropbox.com/sh/26id73senhicchy/AADDKTM9PefdK6rY0n7AhIa3a?dl=0


off topic: resolve studio (pay version) has a command called Noise Reduction. I don't know if this is usefull:

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1052 on: January 10, 2017, 08:07:47 PM »
I need a 14-bit DNG sample from every single camera model, from a recent build (not older than 2017Jan05), either from experimental nightly or from dfort (doesn't matter). It must have some overexposed highlights and should be extracted with:
Code: [Select]
mlv_dump --dng --no-fixcp --no-stripes input.MLV

Please post them in this thread: http://www.magiclantern.fm/forum/index.php?topic=11899

In particular, I expect surprises on 6D and 1100D, but I'd like to check all models.

(sorry for the large font, but I'd like to merge this in the main builds before 5D3.123, 100D, 70D, and I need these samples to make sure everything is fine)

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3410
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1053 on: January 10, 2017, 09:51:38 PM »
Some more crash logs (Error 70) recently with @dfort's latest 5D3.123 (2017Jan05) and this usually happens after previewing files through mlv_play.mo so not sure if they are related or even have anything to do with raw_twk.mo?

https://mega.nz/#F!vhdRET6S!rIZA_Ko_BMjxQGbwIm-d4w
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1054 on: January 10, 2017, 10:29:49 PM »
@dfort:
Testing 7D.203 with :  magiclantern-raw_video_10bit_12bit.2017Jan05.7D203

14 bit works well..

12bit: I get damaged frame on every other frame, see below:


10bit: I get damaged frame on every other frame, see below:
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

D_Odell

  • Member
  • ***
  • Posts: 187
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1055 on: January 11, 2017, 12:04:53 AM »
@a1ex @danne

Yes it works! 5d3.
No color pink/magenta and blue. Tested 10/12/14 bits

./mlv_dump --black-fix=2040 --dng M10-0135.MLV








image upload no limit


Looking at your tests I get increased blue in 12 and 10-bit. It gos higher and higher per bit. Look below:



It's really easy to fix in post per se, but for your info the blue/magenta is still there.
5D3 (OLPF removed) :: 1.1.3 :: Canon FD L Serie

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3751
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1056 on: January 11, 2017, 01:58:49 AM »
Testing 7D.203

Right, that's expected. I believe that is also happening on the 5D2 but crop (zoom) mode is working on that camera. Does the 7D have that feature? Press the magnifying glass button to 5x or 10x and record video. Which feature is that on the features list?

That isn't the solution to  but it is interesting that it works on the 5D2. Maybe it also works on the 7D?

5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

Frank7D

  • Senior
  • ****
  • Posts: 457
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1057 on: January 11, 2017, 05:29:14 AM »
dfort, if you mean regular classic 5x zoom crop mode, then yes the 7D has that feature and 10 and 12 bit have no corruption issues when using that mode.

budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1058 on: January 11, 2017, 07:18:21 AM »
@D_Odell

Thanks for the comparation and design. But I see a big mistake (it's my house and I know how the light goes over the table, white and stanley, and less behind that line and on the floor).

If you see carefully, horizontally (the same design like your version but not vertical) you can see is not a increment of blue or magenta. There is a difference in the noise between 10 and 14. But no incresed the blue and pink.

Other way to comprobe this is zoom in at 1000%. Something not usefull in the reality. Remember video has "movement" between pixels.

If you want to try the horizontally design, comparation only over the Card Tone, could be interesting and more accurate.

 ;)

Edit: the light behind the tone card (kitcheen) is Fluorescent (blue and pink), and over the card, other room, its tungsten 3200K.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7252
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1059 on: January 11, 2017, 07:37:35 AM »
Could you post an example using black level 2048 on all three picures Budafilm?

budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1060 on: January 11, 2017, 07:53:29 AM »
@Danne

Reply #1050 to you!

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7252
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1061 on: January 11, 2017, 09:16:22 AM »
I,d say run all footage with black level 2048(5D mark III). A1ex seems to agree, at least counting from yesterday https://bitbucket.org/hudson/magic-lantern/commits/e3ef8c75cf5de2e4018098d21ae6d1d3bcd183d5
The 14-bit file is slightly different. White balance, color depth?

Order: 10/12/14





VanGogh7

  • New to the forum
  • *
  • Posts: 14
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1062 on: January 11, 2017, 10:07:25 AM »
I see at the  top of the pict. in the blue band a good visible difference between the 10,12,14 bit. 14bit has more details, see at the reflections in it.

Ottoga

  • Member
  • ***
  • Posts: 203
  • The Ox is Slow but the Earth is Patient
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1063 on: January 11, 2017, 12:18:30 PM »
@DFORT

10/12bit Testing

Camera 7D
ML Build: magiclantern-raw_video_10bit_12bit.2017Jan02.7D203 (DFort's)

Modules Loaded:   Fileman, mlv_rec, mlv_play, mlv_snd, pic_view & raw_twk                           

DNG's Extracted using mlv_dump (07/01/2017 build), parameters used --dng --cs2x2
Post processing/editing of extracted dngs - none.


Results of 10bit test using mlv_rec

Frame 000000 looks good with no colour casts, hot/cold pixels etc.
Frame 000001 looks good with no colour casts, hot/cold pixels etc.
Frame 000002 Has the top 1/3 of the image distorted and barely discernable as an image.
                       The bottom  2/3rds of the image look good.
Frame 000003 Same as frames 000000 and 000001.
frame 000004 Same as frame 000002

All subsequent  pairs of frames i.e: 000005&000006, 000007&000008 etc. behave the same as frames 000003&000004.

What is actualy happening here?  from what I can see:

1) All of the frames that have the top 1/3 distorted are a combination of the actual image that should be within that frame and a static image that is constantly repeating and overwriting the actual image. Having shuffled through the frames I believe that it is the distorted part of the frame that is the actual image for the respective frame. The undistorted part is likely from frame 000000.

2) Apart from frame 000000, all of the frames that are distortion free is actually a static image repeating itself. Likely from frame 000000.



Results of 12bit test using mlv_rec

Frame 000000 has the top 1 cm of the image displaced a few pixels to the right but is otherwise OK.
Frame 000001 has the top 1cm corrupted with the balance of the image looking normal
Frame 000002 No Corruption and the image looks normal.
Frame 000003 On first glance looks normal but upon closer inspection it is in fact corrupted. It is in fact a blended image with the top 1 cm of the image being
                       from one image and the balance from another (statc image).
Frame 000004 No Corruption and the image looks normal.
Frame 000005 Like frame 000003 it is a blended image with the top 1 cm of the image being from one image and the balance
                       from another (static image).

All subsequent  pairs of frames i.e: 000006&000007, 000008&000009 etc. behave the same as frames 000004&000005.

What is actually happening with the 12bt mlv? It is similar but different to what is happening with the the 10bit mlv.  For the 12bit mlv, the first two frames 000000&000001 are corrupt. From there on in every odd numbered frame is corrupt (a static repeating image) whilst every even numbered frame is ok. I proved this by:

   - Deleting all of the odd numbered dng frames.
   - Imported the even numbered dng frames in PowerDirector (corrupt frame 000000 is included)
   - Exported to an 1920 x 1080p mpeg4 video.

File Links

The below links are extracts from the original 10 and 12bit mlv files and contain 24 frames each. Also I have add ed link to the 12 bit video generated from PowerDirector.

10bit mlv extract - https://www.dropbox.com/s/hx61s68c4mcjdxq/7d_10bit_extract.mlv?dl=0

12bit mlv extract - https://www.dropbox.com/s/kp31jkj4vsun6t9/7d_12bit_extract.mlv?dl=0

MPEG4 from 12bit (even numbered frames) - https://www.dropbox.com/s/zg4hpbiqxlponuk/7D_14bit_test.mp4?dl=0 


Hopefully this helps the devs get a step closer to resolving the 10/12bit raw for the 7D. 
EOS 7D.203, EFS 55-250mm, EF 75-300 III, Tamron 16-300 DiII VC PZD Macro, SpeedLite 580EX II.

nikfreak

  • Developer
  • Hero Member
  • *****
  • Posts: 1197
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1064 on: January 11, 2017, 12:35:14 PM »
@ottoga:
7D is only confirmed to produce "good" results with 10/12 bit when using crop/zoom mode. Otherwise you will get corrupted frames
70D.112 & 100D.101

Deadcode

  • Freshman
  • **
  • Posts: 61
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1065 on: January 11, 2017, 12:41:00 PM »
Probably a beginner thought, im not even at novice lvl in coding, but it's like a synchronisation slip/clock error happens when you chop 2/4 bits from the incoming stream while recording 10/12 bit with 50D/7D/5D2

dmilligan

  • Developer
  • Hero Member
  • *****
  • Posts: 3218
  • 60Da / 1100D / EOSM
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1066 on: January 11, 2017, 01:36:46 PM »
a1ex has already explained it: http://www.magiclantern.fm/forum/index.php?topic=5601.msg174181#msg174181

If your camera has the "raw slurp" method implemented, everything will work. If it does not, then the solution is to implement it. That requires some reverse engineering. If slurp is not implemented on your camera, the fact that you get any good frames at all is just luck.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1067 on: January 11, 2017, 01:38:49 PM »
I,d say run all footage with black level 2048(5D mark III). A1ex seems to agree, at least counting from yesterday https://bitbucket.org/hudson/magic-lantern/commits/e3ef8c75cf5de2e4018098d21ae6d1d3bcd183d5

Only applies to mlv_dump builds containing that change (such as the one from http://builds.magiclantern.fm/utilities.html ).

Older mlv_dump builds would require 2048 for 14 bpp, 2046 for 12 bpp and 2040 for 10 bpp.

GutterPump

  • Member
  • ***
  • Posts: 185
  • 5DIII/5DII/550D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1068 on: January 11, 2017, 03:23:52 PM »
@A1ex

Did you saw my post about this issue in blown highlight when using mlv_dump for fix verticals stripes ?

It is a known issue ? Or maybe it is me who has not understood something.

When using MLVFS all is ok about stripes fix, even for blown highlight.

Danne

  • Developer
  • Hero Member
  • *****
  • Posts: 7252
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1069 on: January 11, 2017, 04:02:33 PM »

budafilms

  • Hero Member
  • *****
  • Posts: 711
  • 5D Mark a1ex
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1070 on: January 11, 2017, 07:28:39 PM »
@Danne @A1ex and Raw Devs.

Samples of 5d3 build Jan6 in 10/12/14 bits with MLV_DUMP with black fix at 2040, 2046, 2048 in all bit dephs.

Inside the folder I'd included MLV_DUMP and MLV.MO for check versions.

Let's go foward!

https://www.dropbox.com/sh/ruu0sclnxb7xk6m/AAAYRuRuLjlSAjz9lmOsL8dCa?dl=0

 :P

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1071 on: January 11, 2017, 08:45:13 PM »
@dfort
Further 7D testing (croped) magiclantern-raw_video_10bit_12bit.2017Jan05.7D203
Croped mode generates proper dngs at 10/12/14 bits.
LiveView during recording (at 5x) goes to Canon View but sometimes locks up (no motion) in GrayScale.
10x zoom does not work.  It can be made to work if in LiveView at 5X yo enter ML menue and then return to LiveView.  Pressing zoom again will bring 10x, and then pressing again will go back to 1X.
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

Markus

  • Senior
  • ****
  • Posts: 338
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1072 on: January 12, 2017, 06:37:15 PM »
Is there a working 5d3 123 version with mlv 10,12,14bit and crop rec version available to try? I downloaded the one from experiments but it only had Mlv-Lite module for raw.

RenatoPhoto

  • Moderators
  • Hero Member
  • *****
  • Posts: 1509
  • 5DM3 / 7D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1073 on: January 12, 2017, 07:08:05 PM »
mlv_rec not available yet, only with raw_rec available here: https://builds.magiclantern.fm/experiments.html
http://www.pululahuahostal.com  |  EF 300 f/4, EF 100-400 L, EF 180 L, EF-S 10-22, Samyang 14mm, Sigma 28mm EX DG, Sigma 8mm 1:3.5 EX DG, EF 50mm 1:1.8 II, EF 1.4X II, Kenko C-AF 2X

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1074 on: January 13, 2017, 12:48:39 AM »
Black level issues should be fixed now; no more trickery should be needed with latest builds.