Author Topic: 12-bit (and 10-bit) RAW video development discussion  (Read 470984 times)

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1857
  • Have you tried turning it off and on again?
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #725 on: December 12, 2016, 06:48:33 PM »
I've disassembled the 50D's FW and found 4 instances where 'lv_raw_dump' is mentioned but after that I'm lost  ??? - yes I did read @dmilligan's post (http://www.magiclantern.fm/forum/index.php?topic=5601.msg175969#msg175969) but I still don't know what I'm looking at.
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #726 on: December 12, 2016, 07:34:44 PM »
...found 4 instances where 'lv_raw_dump' is mentioned but after that I'm lost...

Yeah, I found the same in my EOSM and 700D disassembly. I had to look for the address that dmilligan used on the EOSM and find the equivalent in the 700D. In art that would be like tracing a master's drawing but hey, it worked.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

eNnvi

  • New to the forum
  • *
  • Posts: 47
  • 700D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #727 on: December 12, 2016, 07:42:44 PM »
there are 4 because 2 are real lv_raw_dump, the other 2 shoulde be lv_raw_dump2

anyway the same instances of lv_raw_dump should take you to the same spot (address jump)

from here it's harder :D

there should be a sub that actually loads the address of the raw buffer, you need to find that! how? well, read the lv_raw_dump function (the bl from the string you found) and check when the return pointer is used for reading or writing something

D_Odell

  • Member
  • ***
  • Posts: 187
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #728 on: December 12, 2016, 08:10:03 PM »
D_Odell. Are you using raw_twk? When in crop mode this will surely happen.
Hi Danne! No not at that time, when used it was completely green..
5D3 (OLPF removed) :: 1.1.3 :: Canon FD L Serie

Rewind

  • Senior
  • ****
  • Posts: 356
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #729 on: December 12, 2016, 09:05:49 PM »
To activate RAW_DEBUG_TYPE, define it here:
Got it. Tested, and at least 0x10 PREFERRED_RAW_TYPE works like we used to (650D, raw_rec, both 10 and 12 bit). No tearing, no artifacting, no shifting (even PDR works). Absolutely usable for shooting already. Thank you dfort!

The pull request is quite an interesting read
Indeed. I'll try other raw types as well. By the way, when using mlv_rec instead of raw_rec, i got some tearing in 10 bit mode: every other frame's top third is slightly shifted (5 or 6 pix) to the top.

Are you planning on putting up a pull request for this
Shame on me, but i spent some years in different kind of activity, and i just forget everything about Git, Bitbucket, branching etc. Actually coding is way more understandable :) I'll try to refresh and join later. As for 650D's CONFIG_EDMAC_RAW_SLURP, it may be easier for you to just add the proper DEFAULT_RAW_BUFFER MEM(0x25B00 + 0x3C) in your commit. Other changes are basically the same as yours for 700D obviously.
Or I can share my test build for other 650D users if this make sense.

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12241
  • Maintenance mode
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #730 on: December 12, 2016, 09:15:13 PM »
From this analysis, PREFERRED_RAW_TYPE 0x10 appears to be the best choice for all DIGIC 5 and 6 cameras, and likely 0x5 for all DIGIC 4.

The EDMAC channel should probably be 0x12 for all DIGIC 5 models (unless used by something else).

Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1857
  • Have you tried turning it off and on again?
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #731 on: December 12, 2016, 09:31:01 PM »
...from here it's harder :D

there should be a sub that actually loads the address of the raw buffer, you need to find that! how? well, read the lv_raw_dump function (the bl from the string you found) and check when the return pointer is used for reading or writing something

Ok, but I don't understand the last bit  :-\

the bl is loc_ff064830 but what do I do with it? How do I check?

Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #732 on: December 12, 2016, 11:36:12 PM »
...when using mlv_rec instead of raw_rec, i got some tearing in 10 bit mode: every other frame's top third is slightly shifted (5 or 6 pix) to the top...

Yeah, noticed that too, raw_rec (MLV Lite) is working better at this time.

...it may be easier for you to just add the proper DEFAULT_RAW_BUFFER MEM(0x25B00 + 0x3C) in your commit. Other changes are basically the same as yours for 700D obviously.
Or I can share my test build for other 650D users if this make sense.

I updated my pull request so it includes the 650D. @Rewind -- are you on bitbucket? There are a few Rewinds there so I'm not sure if one of those is you. In any case, I'd appreciate if you check it out and make sure I didn't leave out anything important.

I made a new set of test builds mainly for 650D and 700D users but found out that the 60D and 600D has CONFIG_EDMAC_RAW_SLURP defined and heard reports that the 7D was also working so I went ahead and built all the platforms. This time I didn't zip them all up one bundle so we can check up on the number of downloads per platform. Also took down the older builds to avoid confusion.

https://bitbucket.org/daniel_fort/magic-lantern/downloads
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

kamranjon

  • New to the forum
  • *
  • Posts: 37
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #733 on: December 13, 2016, 01:33:12 AM »
Just tested this new build on 7D - it works! - the live view in some cases works as well which is awesome - what i had to do was: i turn global draw/liveview on, and then i turn it off, and when i hit record i am able to see what i am shooting! If i don't do that first it doesn't seem to work, it just freezes like it had previously.

LEVISDAVIS

  • Member
  • ***
  • Posts: 134
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #734 on: December 13, 2016, 07:38:18 AM »
Canon 50D to MLVFS Workflow w/ 12-12-16 Build:

Module Report: RAW Lite

1. 10-bit and 12-bit display a repeating first frame; every other frame.

2. 5X features "pink frames" in 10-bit, 12-bit, and 14-bit; every frame.

3. 14-bit works great; aside from 5X zoom.

Levi S. Davis

Danne

  • Contributor
  • Hero Member
  • *****
  • Posts: 5640
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #735 on: December 13, 2016, 09:03:40 AM »
Just tested 500D, 550D, 600D and the 7D with dfort latest compilations. (raw_rec, mlv_lite)

500D , raw10bit, 12bit working, earthquake picture, scrambled lines on top(Check LEVISDAVIS description)
550D, raw10bit, 12bit working, earthquake picture, scrambled lines on top(Check LEVISDAVIS description)
600D, raw10bit, 12bit works flawlessly! raw slurp enabled already?
7D, raw10bit, 12bit working, earthquake picture, scrambled lines on top(Check LEVISDAVIS description)

timbytheriver

  • Member
  • ***
  • Posts: 246
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #736 on: December 13, 2016, 12:10:27 PM »
Will the 650D/700D port work on the 750D? Or is that a different kettle entirely… ?   :-\
5D3 1.1.3
5D2 2.1.2

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 6762
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #737 on: December 13, 2016, 12:22:59 PM »
How to saddle a horse not born yet?
Photogs and videographers: Assist in proof reading upcoming in-camera help!. Your input is wanted and needed!

timbytheriver

  • Member
  • ***
  • Posts: 246
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #738 on: December 13, 2016, 12:26:52 PM »
Thanks Walter. What's delaying the birth? Testers? Major technical differences? I'm genuinely interested.
5D3 1.1.3
5D2 2.1.2

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 6762
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #739 on: December 13, 2016, 12:46:09 PM »
As always: Lack of people with skills (Assembler and C for embedded devices, preferable ARM architecture) and spare time at hand. It will take (estimated) some hundred hours - partly mind numbing - to port ML to a new platform. And this is not just a new platform but a different processor generation, too.
And such a person will be asked to maintain the cam and its ML port. If there is a new Canon firmware and it is decided to port ML for this new firmware: Yupp, reverse engineer it a second time (should be easier, but PITA, 2nd edition).
See threads for 750D, 760D and 80D. 7D2 will have to wait.
Photogs and videographers: Assist in proof reading upcoming in-camera help!. Your input is wanted and needed!

timbytheriver

  • Member
  • ***
  • Posts: 246
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #740 on: December 13, 2016, 12:49:19 PM »
Thanks for the comprehensive reply Walter.
5D3 1.1.3
5D2 2.1.2

eNnvi

  • New to the forum
  • *
  • Posts: 47
  • 700D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #741 on: December 13, 2016, 12:51:18 PM »
First i didn't understand why you post it here as the thread is about something else.
Second you Can get your cam, study assembly, C coding, arm architetture by your own and help porting the code speeding up a lot the porting process

timbytheriver

  • Member
  • ***
  • Posts: 246
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #742 on: December 13, 2016, 12:55:06 PM »
@ eNnvi Walter just explained the exact same thing to me – but with tact.
5D3 1.1.3
5D2 2.1.2

eNnvi

  • New to the forum
  • *
  • Posts: 47
  • 700D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #743 on: December 13, 2016, 03:21:00 PM »
Sorry i didn't mean to be rude but that's what was told me sometime ago (for another project) and i tried and studied and now i Can at least understand the concept and the c code behind ml but i'm still learning a lot about assembly (even if i studied it at University). There's no limit about you but about how much you want to help and learn :)

timbytheriver

  • Member
  • ***
  • Posts: 246
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #744 on: December 13, 2016, 04:23:57 PM »
@eNnvi Thanks. No offence taken. :)
5D3 1.1.3
5D2 2.1.2

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3715
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #745 on: December 13, 2016, 05:15:14 PM »
500D , raw10bit, 12bit working, earthquake picture, scrambled lines on top(Check LEVISDAVIS description)
550D, raw10bit, 12bit working, earthquake picture, scrambled lines on top(Check LEVISDAVIS description)
600D, raw10bit, 12bit works flawlessly! raw slurp enabled already?
7D, raw10bit, 12bit working, earthquake picture, scrambled lines on top(Check LEVISDAVIS description)

This makes sense because the 600D is the only camera on the list that has CONFIG_EDMAC_RAW_SLURP defined. I just went through all the platforms and checked the ones that are "slurping":

1100D.105No. Last Successful Build (2015-05-03 00:25) What's up with that?
500D.111No
50D.109No but @Andy600 is working on it.
550D.109No
5D2.212No but @reddeercity is reporting some success.
5D3.113Yes
600D.102Yes
60D.111Yes
650D.104Yes with pending pull request (test build posted)
6D.116No
700D.114Yes with pending pull request (test build posted)
7D.203No but maybe @nikfreak is working on it?
EOSM.202Yes
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 M50.102

eNnvi

  • New to the forum
  • *
  • Posts: 47
  • 700D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #746 on: December 13, 2016, 05:26:07 PM »
Working on 6D and 7D

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3171
  • 5D3 / M1 / 7D / 70D / SL1 / M2
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #747 on: December 14, 2016, 01:34:47 AM »
Thanks @dfort for the latest updates.

@EOSM users:

Are you guys able to get mlv_play.mo to work w 10/12-bit files after each recording takes? I notice sometimes I have to either turn off camera and back on to get mlv_play.mo to work properly otherwise I'll get a black screen stuck on LCD liveview.

Btw it's nice to be able to see LiveView while recording. Also confirmed to work in crop_mode.mo but not in 5x zoom mode.


More test to come later with 5D3 & 7D.

Perhaps I'll wait until @eNnvi provides a new build for the 7D.
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101 • EOSM2.*

saulbass

  • Freshman
  • **
  • Posts: 85
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #748 on: December 14, 2016, 01:49:47 AM »
@dfort - thanks for the recent 650D update - (magiclantern-raw_video_10bit_12bit.2016Dec12.650D104.zip).

10bit RAW 1280 x 720 works continuously - only issue are the focus pixels and some image tearing but I’m not using my fastest SD card.

With the MLV sound added to this the video is very choppy.


arrinkiiii

  • Hero Member
  • *****
  • Posts: 1101
  • ML ADDICTED 7D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #749 on: December 14, 2016, 02:54:11 AM »
Hi and thanks for ALL  :)

I test the new version for the 7D and i can record continuos and with the camera screen (LiveView) good (canon preview), but in camera with mlv_play.mo i just see pink/flashy image. In computer, after use MPV player i see this pink and good image blinking very fast.

Cheers