12-bit (and 10-bit) RAW video development discussion

[Danne]

Could you upload the dng files? And name them 10/12/14-bit so one can differ them.
Shouldn,t 14-bit have 2048 black level? 10 and 12 can,t be both 2040 or what am I missing?

[budafilms]

@Danne

I applied the same 2040 to 10/12/14 bits.

https://www.dropbox.com/sh/26id73senhicchy/AADDKTM9PefdK6rY0n7AhIa3a?dl=0


off topic: resolve studio (pay version) has a command called Noise Reduction. I don't know if this is usefull:

[a1ex]

I need a 14-bit DNG sample from every single camera model, from a recent build (not older than 2017Jan05), either from experimental nightly or from dfort (doesn't matter). It must have some overexposed highlights and should be extracted with:

Code: [Select]

mlv_dump --dng --no-fixcp --no-stripes input.MLV

Please post them in this thread: http://www.magiclantern.fm/forum/index.php?topic=11899

In particular, I expect surprises on 6D and 1100D, but I'd like to check all models.

(sorry for the large font, but I'd like to merge this in the main builds before 5D3.123, 100D, 70D, and I need these samples to make sure everything is fine)

[DeafEyeJedi]

Some more crash logs (Error 70) recently with @dfort's latest 5D3.123 (2017Jan05) and this usually happens after previewing files through mlv_play.mo so not sure if they are related or even have anything to do with raw_twk.mo?

https://mega.nz/#F!vhdRET6S!rIZA_Ko_BMjxQGbwIm-d4w

[RenatoPhoto]

@dfort:
Testing 7D.203 with :  magiclantern-raw_video_10bit_12bit.2017Jan05.7D203

14 bit works well..

12bit: I get damaged frame on every other frame, see below:


10bit: I get damaged frame on every other frame, see below:

[D_Odell]

@a1ex @danne

Yes it works! 5d3.
No color pink/magenta and blue. Tested 10/12/14 bits

./mlv_dump --black-fix=2040 --dng M10-0135.MLV








image upload no limit

Looking at your tests I get increased blue in 12 and 10-bit. It gos higher and higher per bit. Look below:



It's really easy to fix in post per se, but for your info the blue/magenta is still there.

[dfort]

Testing 7D.203

Right, that's expected. I believe that is also happening on the 5D2 but crop (zoom) mode is working on that camera. Does the 7D have that feature? Press the magnifying glass button to 5x or 10x and record video. Which feature is that on the features list?

That isn't the solution to  but it is interesting that it works on the 5D2. Maybe it also works on the 7D?

[Frank7D]

dfort, if you mean regular classic 5x zoom crop mode, then yes the 7D has that feature and 10 and 12 bit have no corruption issues when using that mode.

[budafilms]

@D_Odell

Thanks for the comparation and design. But I see a big mistake (it's my house and I know how the light goes over the table, white and stanley, and less behind that line and on the floor).

If you see carefully, horizontally (the same design like your version but not vertical) you can see is not a increment of blue or magenta. There is a difference in the noise between 10 and 14. But no incresed the blue and pink.

Other way to comprobe this is zoom in at 1000%. Something not usefull in the reality. Remember video has "movement" between pixels.

If you want to try the horizontally design, comparation only over the Card Tone, could be interesting and more accurate.

 

Edit: the light behind the tone card (kitcheen) is Fluorescent (blue and pink), and over the card, other room, its tungsten 3200K.

[Danne]

Could you post an example using black level 2048 on all three picures Budafilm?

[budafilms]

@Danne

Reply #1050 to you!

[Danne]

I,d say run all footage with black level 2048(5D mark III). A1ex seems to agree, at least counting from yesterday https://bitbucket.org/hudson/magic-lantern/commits/e3ef8c75cf5de2e4018098d21ae6d1d3bcd183d5
The 14-bit file is slightly different. White balance, color depth?

Order: 10/12/14

[VanGogh7]

I see at the  top of the pict. in the blue band a good visible difference between the 10,12,14 bit. 14bit has more details, see at the reflections in it.

[Ottoga]

@DFORT

10/12bit Testing

Camera 7D
ML Build: magiclantern-raw_video_10bit_12bit.2017Jan02.7D203 (DFort's)

Modules Loaded:   Fileman, mlv_rec, mlv_play, mlv_snd, pic_view & raw_twk                           

DNG's Extracted using mlv_dump (07/01/2017 build), parameters used --dng --cs2x2
Post processing/editing of extracted dngs - none.


Results of 10bit test using mlv_rec

Frame 000000 looks good with no colour casts, hot/cold pixels etc.
Frame 000001 looks good with no colour casts, hot/cold pixels etc.
Frame 000002 Has the top 1/3 of the image distorted and barely discernable as an image.
                       The bottom  2/3rds of the image look good.
Frame 000003 Same as frames 000000 and 000001.
frame 000004 Same as frame 000002

All subsequent  pairs of frames i.e: 000005&000006, 000007&000008 etc. behave the same as frames 000003&000004.

What is actualy happening here?  from what I can see:

1) All of the frames that have the top 1/3 distorted are a combination of the actual image that should be within that frame and a static image that is constantly repeating and overwriting the actual image. Having shuffled through the frames I believe that it is the distorted part of the frame that is the actual image for the respective frame. The undistorted part is likely from frame 000000.

2) Apart from frame 000000, all of the frames that are distortion free is actually a static image repeating itself. Likely from frame 000000.



Results of 12bit test using mlv_rec

Frame 000000 has the top 1 cm of the image displaced a few pixels to the right but is otherwise OK.
Frame 000001 has the top 1cm corrupted with the balance of the image looking normal
Frame 000002 No Corruption and the image looks normal.
Frame 000003 On first glance looks normal but upon closer inspection it is in fact corrupted. It is in fact a blended image with the top 1 cm of the image being
                       from one image and the balance from another (statc image).
Frame 000004 No Corruption and the image looks normal.
Frame 000005 Like frame 000003 it is a blended image with the top 1 cm of the image being from one image and the balance
                       from another (static image).

All subsequent  pairs of frames i.e: 000006&000007, 000008&000009 etc. behave the same as frames 000004&000005.

What is actually happening with the 12bt mlv? It is similar but different to what is happening with the the 10bit mlv.  For the 12bit mlv, the first two frames 000000&000001 are corrupt. From there on in every odd numbered frame is corrupt (a static repeating image) whilst every even numbered frame is ok. I proved this by:

   - Deleting all of the odd numbered dng frames.
   - Imported the even numbered dng frames in PowerDirector (corrupt frame 000000 is included)
   - Exported to an 1920 x 1080p mpeg4 video.

File Links

The below links are extracts from the original 10 and 12bit mlv files and contain 24 frames each. Also I have add ed link to the 12 bit video generated from PowerDirector.

10bit mlv extract - https://www.dropbox.com/s/hx61s68c4mcjdxq/7d_10bit_extract.mlv?dl=0

12bit mlv extract - https://www.dropbox.com/s/kp31jkj4vsun6t9/7d_12bit_extract.mlv?dl=0

MPEG4 from 12bit (even numbered frames) - https://www.dropbox.com/s/zg4hpbiqxlponuk/7D_14bit_test.mp4?dl=0 


Hopefully this helps the devs get a step closer to resolving the 10/12bit raw for the 7D. 

[nikfreak]

@ottoga:
7D is only confirmed to produce "good" results with 10/12 bit when using crop/zoom mode. Otherwise you will get corrupted frames

[Deadcode]

Probably a beginner thought, im not even at novice lvl in coding, but it's like a synchronisation slip/clock error happens when you chop 2/4 bits from the incoming stream while recording 10/12 bit with 50D/7D/5D2

[dmilligan]

a1ex has already explained it: http://www.magiclantern.fm/forum/index.php?topic=5601.msg174181#msg174181

If your camera has the "raw slurp" method implemented, everything will work. If it does not, then the solution is to implement it. That requires some reverse engineering. If slurp is not implemented on your camera, the fact that you get any good frames at all is just luck.

[a1ex]

I,d say run all footage with black level 2048(5D mark III). A1ex seems to agree, at least counting from yesterday https://bitbucket.org/hudson/magic-lantern/commits/e3ef8c75cf5de2e4018098d21ae6d1d3bcd183d5

Only applies to mlv_dump builds containing that change (such as the one from http://builds.magiclantern.fm/utilities.html ).

Older mlv_dump builds would require 2048 for 14 bpp, 2046 for 12 bpp and 2040 for 10 bpp.

[GutterPump]

@A1ex

Did you saw my post about this issue in blown highlight when using mlv_dump for fix verticals stripes ?

It is a known issue ? Or maybe it is me who has not understood something.

When using MLVFS all is ok about stripes fix, even for blown highlight.

[Danne]

Here is the latest mlv_dump for mac.
https://bitbucket.org/Dannephoto/magic-lantern/downloads/mlv_dump

[budafilms]

@Danne @A1ex and Raw Devs.

Samples of 5d3 build Jan6 in 10/12/14 bits with MLV_DUMP with black fix at 2040, 2046, 2048 in all bit dephs.

Inside the folder I'd included MLV_DUMP and MLV.MO for check versions.

Let's go foward!

https://www.dropbox.com/sh/ruu0sclnxb7xk6m/AAAYRuRuLjlSAjz9lmOsL8dCa?dl=0

 

[RenatoPhoto]

@dfort
Further 7D testing (croped) magiclantern-raw_video_10bit_12bit.2017Jan05.7D203
Croped mode generates proper dngs at 10/12/14 bits.
LiveView during recording (at 5x) goes to Canon View but sometimes locks up (no motion) in GrayScale.
10x zoom does not work.  It can be made to work if in LiveView at 5X yo enter ML menue and then return to LiveView.  Pressing zoom again will bring 10x, and then pressing again will go back to 1X.

[Markus]

Is there a working 5d3 123 version with mlv 10,12,14bit and crop rec version available to try? I downloaded the one from experiments but it only had Mlv-Lite module for raw.

[RenatoPhoto]

mlv_rec not available yet, only with raw_rec available here: https://builds.magiclantern.fm/experiments.html

[a1ex]

Black level issues should be fixed now; no more trickery should be needed with latest builds.