12-bit (and 10-bit) RAW video development discussion

[Nikolas]

Thanks to developers for latest build for 1100d with 10/12bit raw video (2017-01-20)! Compared with the previous version (december 2016), record of raw video works very stable. 10/12 bit really is supported. in the previous build had a lot of colored dots, now they have disappeared.

But in the latest build, strong processing gives vertical banding (pattern noise?). Like i said, in the previous version were only colored dots, vertical banding been absent. Image quality with hard processing is very important for me (I use raw video for astrophotography).

I tries to use different combinations of recording settings and to use subtraction dark frames, but it not work.

Dear developers, Is it possible to fix in the next build? Or how i can make a correction of these bands?

[Danne]

Interesting.
How are you subtracting darkframes? Can you upload a darkframe and a clip with banding?

[Greg]

Code Select Expand

#ifdef CONFIG_80D_101
#define DEFAULT_RAW_BUFFER MEM(0x66B0 + 0x54)
#endif

[Sganzerla]

@a1ex
@g3gg0
@GutterPump
@chmee
@RenatoPhoto

Regarding my post [b#1081]]#1081[/b] I finally (after some struggle) learnt how to process .MLV with mlv_dump and could see that there is no color cast between 14bit and 10bit files using this software.

Thanks for all of your input!

[dfort]

The Bit depth in raw_rec and mlv_rec are slightly different:





Note that mlv_rec says "bpp" for bits per pixel.

In addition, the up/down buttons go the opposite direction in raw_rec so I made a small change and put in a pull request.





Hope you like the small improvements and hope this doesn't break anything--tested and working fine over here.

[LEVISDAVIS]

Canon 50D_01-14-2017 109 Firmware Test

All Processed w/ CMD and MLVFS:

NOTES:
     
             RAW REC 1.1 works in 14-Bit Full Sensor Readout and Canon Preview.

             MLV 2.0 works in 14-bit Full Sensor and 5X Zoom w/Canon Preview. However, it is super close to working in 12-bit.



- - - - - - - - - - - - - - - RAW RECORD 1.1 LITE - - - - - - - - - - - - - - -

10-Bit - Full Sensor Readout - - RAW RECORD 1.1 - - Canon Preview Only - Hacked / B&W Freeze Image on LCD - -
   RESULTS: Repeating Frame; every other frame. Top 1/3 of the image is an inverted data sector / bad write.

10-Bit 5X Zoom - RAW RECORD 1.1 - - Canon Preview Only - Hacked / B&W Freeze Image on LCD - -
   RESULTS: Purple Frames. Repeating Frame; every other frame. Top 1/3 of the image is separated from bottom 2/3. Bottom 2/3 features a "wiggly zoom-in" distortion appearing in a timed pattern.


- - - - - - - - - - - - - - - RAW RECORD 1.1 LITE

** Best Results w/ 5X Zoom:

12-Bit Full Sensor Readout - - RAW RECORD 1.1 - - Canon Preview Only - Hacked / B&W Freeze Image on LCD - -
   RESULTS: Repeating Frame; every other frame. Top 1/6 of the image is an inverted data sector / bad write (still repeats image).

12-Bit 5X Zoom - RAW RECORD 1.1 - - Canon Preview Only - Hacked / B&W do not work - -
   RESULTS: Purple Frames. Repeating Frame; every other frame. Top 1/3 of the image is separated from bottom 2/3.


- - - - - - - - - - - - - - - RAW RECORD 1.1 LITE

14-Bit Full Sensor Readout - - RAW RECORD 1.1 - - Canon Preview Only - Hacked / B&W Freeze Image on LCD - -
   RESULTS: Full Sensor Readout is great.

14-Bit 5X Zoom - - RAW RECORD 1.1 - - Canon Preview Only - Hacked / B&W Freeze Image on LCD - -
   RESULTS:  Purple Frames. Repeating Frame; every other frame. Top 1/3 of the image is separated from bottom 2/3. Bottom 2/3 features a "wiggly zoom-in" distortion appearing in a timed pattern.



- - - - - - - - - - - - - - - MLV 2.0 - - - - - - - - - - - - - - -


14-Bit Full Sensor Readout - - MLV 2.0 - - Canon Preview Only - Hacked / B&W Freeze Image on LCD - -
   RESULTS: Full Sensor Readout is great.

14-Bit 5X Zoom - - MLV 2.0 - - Canon Preview Only - Hacked / B&W Freeze Image on LCD - -
   RESULTS: 5X Zoom Results Record Great. However, FRAMING IS OFF.


- - - - - - - - - - - - - - - MLV 2.0

12-Bit Full Sensor Readout - - MLV 2.0 - - Canon Preview Only - Hacked / B&W Freeze Image on LCD - -
   RESULTS: Top 1/8 of the image is a repeating inverted data sector / bad write every other frame.

12-Bit 5X Zoom - MLV 2.0 - - Canon Preview Only - Hacked / B&W do not work - -
   RESULTS: Top 1/3 of frame appears to be separated from both 2/3 and repeating every other frame. NO PURPLE FRAMES.


- - - - - - - - - - - - - - - MLV 2.0

10-Bit Full Sensor Readout - - MLV 2.0 - - Canon Preview Only - Hacked / B&W Freeze Image on LCD - -
   RESULTS: Repeating Frame; every other frame. Top 1/3 of the image is an inverted data sector / bad write.

10-Bit 5X Zoom - MLV 2.0 - - Canon Preview Only - Hacked / B&W do not work - -
   RESULTS: Top 1/3 of frame appears to be separated from both 2/3. NO PURPLE FRAMES.

[a1ex]

Looks like it is worth repeating that the following cameras need to get the CONFIG_EDMAC_RAW_SLURP treatment before it is worth doing any further tests with them.

5D2.212	-	Digic 4
7D.203	-	Dual Digic 4
50D.109	-	Digic 4
500D.111	-	Digic 4
550D.109	-	Digic 4

[Lars Steenhoff]

For any tests done I think its worth mentioning wich memory card is used to isolate any card speed issues.

[a1ex]

... for any *speed* tests (hint, hint)

[LEVISDAVIS]

The internal temp reading on the 50D is reading, "Calm and collected."

Komputerbay 128 1066x, by the way.

[bobbyshots2]

Iv been testing the  10-12 bit on the 550d (t2i)
I get good results on the 12 bit using mlv 2.0
on the 10 bit the top part of the frame is jittery or "repeating" as i see someother user posting

[hjfilmspeed]

I am super impressed by the Jan 13 experimental build the Incorporates the 10bit and crop mode. 10bit is a dream to work with. If we could just get a better preview for high resolution. But 10 bit has been working flawlessly for me. So many possibilities here for higher fps and resolution bravo.

[Tracerman]

@bobbyshots2 what build did you use ?

[Walter Schulz]

Looks like it is worth repeating that the following cameras need to get the CONFIG_EDMAC_RAW_SLURP treatment before it is worth doing any further tests with them.

5D2.212   -   Digic 4
7D.203   -   Dual Digic 4
50D.109   -   Digic 4
500D.111   -   Digic 4
550D.109   -   Digic 4

Sorry for fullquote. You highlighted 50D. Work is done for other cams?

[a1ex]

Nope.

[bobbyshots2]

I am using the download from the link that reddeercity posted previously.
the build say 01-12-2017
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads

* i notice now that there is a new date 01-14-2017 that i will have to try.

[Walter Schulz]

@a1ex and dfort: 7D here but no clue. Not that fluent in code. Directions, please!

[a1ex]

Unsuccessful on 5D2 after trying for a couple of evenings...

For new models, it's pretty straightforward if one looks at the commits. The 1200D is probably the only new model not yet ported. Old models have a very different codebase for LiveView (though the hardware is likely to work in the same way).

[dfort]

@a1ex and dfort: 7D here but no clue. Not that fluent in code. Directions, please!

Me neither! Fluent in code that is. I helped a bit with the 700D but only because it is very similar to the EOSM which dmilligan previously figured out. Those older models are proving very difficult to crack. There's an 1100D build on the Experiments download page but I haven't seen any reports on how well it is working. Though looking that camera probably won't get you very far with the 7D or 5D2 because it is a slightly newer model.

[reddeercity]

Unsuccessful on 5D2 after trying for a couple of evenings...
Old models have a very different codebase for LiveView (though the hardware is likely to work in the same way).

Sounds encouraging 

At the risk of not knowing anything about liveview code (over my head) is there something my 5d2 or myself can help with ?
I can compile , I understand some of the code but I get easily confuse at times but willing to try . 

[reddeercity]

Ok doing some research on the liveview problem on the 5D2 in 1:1
Come across wiki Register_Map found Liveview Registers stuff .

0xC0F140C0 does luma scaling
0xC0F140C4 does chroma scaling
0xC0F140C8 sets the bmp overlay mode
0xC0F140E8 [s32] pixels to skip - causes distorted image
0xC0F140F0 [s32] pixels to skip - causes distorted image
0xC0F37014 LV raw type (see lv_af_raw, lv_set_raw) on DIGIC5 (DIGIC4 uses PACK32_ISEL)
0xC0F371FC unpacking mode (see PACK16_MODE) for the input module that feeds image data to lossless encoder

0xC0F37014 LV raw type seems to point to  "PACK32_ISEL" for digic4 (5d2) (from ML code I seen this is where digic5 cam gets configured with "LV raw type" if I'm not mistaken

0xC0F08090 - PACK32 Control register
0xC0F08094 - PACK32_MODE
0xC0F08114 - PACK32_ISEL (0-15)

Edit: Sorry posted wrong address 
I see the variable (0-15) now
nice to know what those are .

How's can this be configured ?

Like this for the "PACK16_MODE" ?

0xC0F08120 - PACK16 Control register
0xC0F08124 - PACK16_MODE (mask 0x131)
             From ProcessPathForFurikake (raw_twk):
             -------x --xx----
                  111 (0x130) = 16-bit output
                  011 (0x030) = 14-bit output
                  010 (0x020) = 14-bit output
                  001 (0x010) = 12-bit output
                  000 (0x000) = 10-bit output

I hope this is on the right track , What do the developers think ?
If this is correct how would I processed from here.

[a1ex]

0xC0F08114 - PACK32_ISEL (0-15)

Just look it up in raw.c and earlier in this thread

[Markus]

When recording on these new experimental builds the raw histogram switches over to the standard histogram on record. Have I missed some simple setting or is it i bug?

5d3 113 crop + 10-14bit experimental version.

[a1ex]

All raw overlays are currently hardcoded for 14-bit data. See this post for more details.

[ShootMeAlready]

raw_twk.mo issue with 70D

For 10 bit raw, if you want to have liveview playback working
you need to copy raw_twk.mo to your ML->modules folder on your card.
*** It seems to work for one pass, but 2nd clip it hangs your camera ***
*** Looks like that its not initializing properly for 2nd clip ***

here is a link for the raw_twk.mo I used.
https://www.dropbox.com/sh/arb43xa6svvjljr/AACyi51IrJfMGs3ZQ8EC_6hOa?dl=0

The modules that need to be enabled for it to work include
file_man
mlv_play
mlv_rec
mlv_send
raw_rec
raw_twk