Author Topic: 12-bit (and 10-bit) RAW video development discussion  (Read 840561 times)

timbytheriver

  • Senior
  • ****
  • Posts: 476
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1725 on: October 05, 2018, 11:30:23 AM »
With @a1ex's 10/12bit build working so well on 5D2, is there a reason why we can't get a standard 1080 x 1920 24p preset available on the 5D2? I may well have missed something obvious…  :o
Thanks.


5D3 1.1.3
5D2 2.1.2

zcream

  • Freshman
  • **
  • Posts: 52
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1726 on: October 05, 2018, 12:46:20 PM »
Thanks for the 50d check-in. I don't have access to my 50d ATM, could someone kindly test on the 50d..

Sent from my Lenovo TB-8703F using Tapatalk


Andy600

  • Contributor
  • Hero Member
  • *****
  • Posts: 1863
  • Have you tried turning it off and on again?
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1727 on: October 05, 2018, 03:39:12 PM »
Thanks for the 50d check-in. I don't have access to my 50d ATM, could someone kindly test on the 50d..

Sent from my Lenovo TB-8703F using Tapatalk


https://bitbucket.org/hudson/magic-lantern/commits/db4ee396f4a261d33688b528c39447682f871a07#general-comments
Colorist working with Davinci Resolve, Baselight, Nuke, After Effects & Premier Pro. Occasional Sunday afternoon DOP. Developer of Cinelog-C Colorspace Management and LUTs - www.cinelogdcp.com

andrew_dotdot

  • New to the forum
  • *
  • Posts: 10
  • 5D3, 600D (Fuji X-T3)
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1728 on: October 10, 2018, 02:49:05 PM »
Sorry, didn't notice it was missing; previous feedback appears to reference mlv_rec, so I've assumed it was OK.

Fixed, and also included the 50D (mostly untested, except for the low-level notes from aprofiti).

The error message about sound recording is still outdated; feel free to suggest something better.

The above post made me hopeful that the new latest 70D package might also now include the sound module, which has always been missing in this branch. Is there a recommended version of the sound module to use for the 70D, or is the sound definitely not working on this camera with this branch?

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 12564
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1729 on: October 10, 2018, 04:20:28 PM »
Um... I really need a way to fail the build if certain modules are not included. Currently, if some module throws a compile error, the build system assumes it's simply not compatible with the current camera and removes it, with a warning message buried somewhere in the logs (i.e. not visible on the front page).

I don't want to manually "whitelist" modules either, so... here's an idea: fail the build whenever a module that used to work, now fails to compile.

Rough draft of what I think it might work (not tested):

- there should be a platform-specific (!) list of required modules (different wording welcome)
- when running "make zip" from scratch, all modules that happen to build successfully should be marked as "required" (i.e. added to that list)
- "make clean" should not erase that list (that's the whole point - to detect modules that worked in previous builds)
- there should be some way to erase that list (possibly with a short help when the build fails because of some module in that list)
- if some required module would fail to build, fail the entire build
- if some new module is successfully built when running "make zip", auto-add it to the list of required modules
- partial builds (make install ML_MODULES="foo bar baz") should keep working; I use these often, as it compiles much faster that way
- for example: platform/70D.112/Makefile.modules.required (or maybe modules/Makefile.modules.required.70D.112) might contain:
Code: [Select]
ML_MODULES_REQUIRED ?= \
mlv_lite \
mlv_play \
mlv_rec \
mlv_snd \
        ... (autogenerated list)

Who's going to help? May require a bit of Makefile hacking (something I'm not good at); otherwise, a Python script might do the job, too. You may come up with a different workflow, too, as long as it does the job.

Build updated. For cameras other than 70D, it's exactly the same as before.

dariSSight

  • Member
  • ***
  • Posts: 163
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1730 on: October 22, 2018, 11:04:42 PM »
@TBar15 , What are talking about  ? Everything you need is in the build .
https://builds.magiclantern.fm/experiments.html

Just follow the really sample instructions that's being postedHow is this not straight forward ?
I'm testing out firmware a lot you guys are doing, I'm wondering are anyone doing professional work with magiclantern-raw_video_10bit_12bit.2018Oct10.5D2212.zip? I didn't get the September 17 version, but  magiclantern-raw_video_10bit_12bit.2018Oct10.5D2212.zip looks stable. Please let me know what you guys think. THANKS FOR ANY RESPONSE
Canon 5D Mark II

banertop

  • Freshman
  • **
  • Posts: 65
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1731 on: November 15, 2018, 10:54:09 PM »
Hi guys,

please, how can i record 2880x1080p with 5d2? Is this possible with 10bit, 12bit build from experimental page (witch i try but no success)?,

or, it is another build? Witch one?

Tnx, and sorry if i asked something that was already asked before....sometimes, it is really hard to find information on forum, there are on different topics.....

I am shooting with 5d2 after a while, so getting in shape again>))

tnx for the help

Walter Schulz

  • Contributor
  • Hero Member
  • *****
  • Posts: 8606

banertop

  • Freshman
  • **
  • Posts: 65
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1733 on: November 16, 2018, 12:10:28 AM »
i will.

thank you very much

reddeercity

  • Contributor
  • Hero Member
  • *****
  • Posts: 2303
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1734 on: November 16, 2018, 06:28:27 AM »
Did you look here 3K/UHD 5D2 Raw development and Other Digic IV Cams
It really not that hard to find thing here if you use the search engine , by the way always check the first post of any thread for possible downloads . 

banertop

  • Freshman
  • **
  • Posts: 65
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1735 on: November 16, 2018, 08:34:41 PM »
yes, i see it...

it is more clear now.....

i bought 5d ii (again) after seeing your YT video about 2880p progress....it is really amazing,,,thanks a lot

manudom

  • Just arrived
  • *
  • Posts: 1
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1736 on: December 04, 2018, 10:04:05 PM »
It does not work on my 5D3. I tested with 10 bit in crop x5 mode and I only get extremely noisy frames.

domo94

  • Member
  • ***
  • Posts: 131
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1737 on: February 18, 2019, 04:32:00 PM »
How's development on 7d 10/12 bit?

I'm running 2.0.6 but I am going back to 2.0.3 for more testing purposes and all.

dfort

  • Guest
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1738 on: February 18, 2019, 09:18:20 PM »
What needs to happen on the 7D and other Digic 4 cameras is to find a few stubs like aprofiti did on the 50D. That's a first step. I tried but couldn't find one of them, can't remember which one at the moment.

domo94

  • Member
  • ***
  • Posts: 131
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1739 on: February 20, 2019, 05:29:19 PM »
& just how can a normy like me do so?

Or is that beyond my capability without knowing the code process and all?

aprofiti

  • Contributor
  • Member
  • *****
  • Posts: 194
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1740 on: February 20, 2019, 06:24:36 PM »
What needs to happen on the 7D and other Digic 4 cameras is to find a few stubs like aprofiti did on the 50D. That's a first step. I tried but couldn't find one of them, can't remember which one at the moment.
the correct stubs should start with 0xFF9 instead of 0xFF1. That means, you should patch some address that Canon code is going to execute, not a mirrored copy.

Make sure the hook is actually executing (add some printf, for example).

@dfort Can you try these ones?
Code: [Select]
NSTUB(0xFF27E674, StartImagePass_x1_SetEDmac)
NSTUB(0xFF27F444, StartImagePass_x5_SetEDmac) 

First one It's a bit different but function called looks likely the same.
Looking into the disassembly I can't find a mirror for both stubs in the 0xFF9 range or similar like on 50D

dfort

  • Guest
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1741 on: February 20, 2019, 07:54:34 PM »
@aprofiti - Thanks for the tips. Couldn't get very far. There's something in the mysterious 7D Master code that is causing a compile error on the raw_video_10bit_12bit_LVState branch.

Code: [Select]
[ VERSION  ]   ../../platform/7D_MASTER.203/version.bin
[ CC       ]   master_boot.o
master_boot.c: In function 'isrlog':
master_boot.c:173:5: error: implicit declaration of function 'clean_d_cache' [-Werror=implicit-function-declaration]
     clean_d_cache();
     ^
cc1: some warnings being treated as errors
make[1]: *** [master_boot.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [all] Error 2

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
12-bit (and 10-bit) RAW video development discussion
« Reply #1742 on: February 20, 2019, 08:02:35 PM »
That's real bummer @dfort... Thanks for sharing @aprofiti!

Regardless I bet you this only brought us closer to perfection. Indeed it was well worth the effort. :-X

Perhaps this whole mysterious thing is benefiting all the 7D bodies itself in the long run after all. Ha.

In terms of wear and tear. It's sitting like a lifeless puppy ready to burst out ‘n grow w ML yet once again!
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

aprofiti

  • Contributor
  • Member
  • *****
  • Posts: 194
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1743 on: February 20, 2019, 09:48:24 PM »
I replaced with _clean_d_cache() and found the reference; error is probably due to some refactor like this commit.
Tried also with sync_caches() and I had to change to _sync_caches()

But it doesn't compile for 7D (no problems for 50D without any changes needed):
Code: [Select]
Using ~/gcc-arm-none-eabi-5_4-2016q3/bin/arm-none-eabi-gcc (preferred).
[ VERSION  ]   ../../platform/7D_MASTER.203/version.bin
[ CC       ]   master_boot.o
[ LD       ]   magiclantern
gdb.o: In function `gdb_unarm_bkpt':
gdb.c:(.text+0x2ec): undefined reference to `unpatch_memory'
gdb.o: In function `gdb_arm_bkpt':
gdb.c:(.text+0x380): undefined reference to `patch_instruction'
make: *** [magiclantern] Error 1
It strange because they are defined in patch.c and his header is already included in gdb.c...

@a1ex maybe is there some makefile/external reference/header to update for 7D?

IDA_ML

  • Hero Member
  • *****
  • Posts: 1014
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1744 on: February 21, 2019, 07:43:40 AM »
It's sitting like a lifeless puppy ready to burst out ‘n grow w ML yet once again!

This same hope makes me keep my 7D too.  I still use it with the legendary Dec. 1-st 2016 Build by Dfort and every time I see this unique beautiful cinematic look of the video coming out of it, I keep saying to myself - no, I will keep this good old boy.  Maybe, some day it will receive the ML functionality it deserves.  In my country, we have a saying: "The hope dies last" ...

A1ex,

You state so often on the forum that you are here to help.  The 7D is one example where your help would be invaluable.  You can hardly imagine how many people you would make happy if you could help Dfort, Aprofiti and other skilled people implement 4k crop recording to the 7D.  I know, this is low-priority for you but sometimes, just a few helpful hints in the right direction can create miracles!  The miracle happened already when you helped Reddeercity revive the 5D2.

domo94

  • Member
  • ***
  • Posts: 131
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1745 on: February 21, 2019, 09:26:55 AM »
Dec 1st 2016 Build? *EDIT* - it's Dec 3

Looks like I'll be using that one from now on until further notice!

Yes, if the 7D gets some love on the development side, I will gladly be your guinea pig all day.

I help but NOT throw away my 7d or switch out.

I wish the low light was great, but I hardly ever shoot low light, so it's pointless for me.

I don't care for 4k as much as I do 10-12 bit raw recording. I get more footage time out of my CF Card :D

dfort

  • Guest
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1746 on: February 21, 2019, 05:19:23 PM »
I replaced with _clean_d_cache() and found the reference; error is probably due to some refactor like this commit.

Same here on platform/7D_MASTER.203/master_boot.c.

But it doesn't compile for 7D (no problems for 50D without any changes needed):

Right, it compiles fine on all the other cameras. The problem is in patchmgr so it should be fixed on that branch and re-merged into raw_video_10bit_12bit_LVState. Ran hg bisect on it:

Code: [Select]
The first bad revision is:
changeset:   16796:ac33688aae07
branch:      patchmgr
parent:      16445:e7a7eadbe2a0
parent:      16512:97b73e7a781e
user:        alex@thinkpad
date:        Wed Jan 24 00:04:28 2018 +0100
summary:     Merged unified into patchmgr

Looks like I've been here before:

https://bitbucket.org/hudson/magic-lantern/commits/ac33688aae076a3c54c3de9f064be54e28e9e324

dfort

  • Guest
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1747 on: February 21, 2019, 06:59:05 PM »
A little rough around the edges but something to start playing around with on the 7D:

raw_video_10bit_12bit_LVState_7D_experiments

Test build on my downloads page.

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3411
  • 5D3 | M1 | 7D | 70D | SL1 | M2 | 50D
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1748 on: February 21, 2019, 07:41:08 PM »
Thanks for this @dfort! Finally downgraded my 7D from 2.0.6 back to thee good ole' 2.0.3 and will definitely give these rough edges a little test run.

*EDIT*

Just a friendly reminder to those who may find themselves in a similar situation whereas the MLV files are corrupted while shooting in 2.5K (x5 zoom) while the trick in here is to set Preview to 'REALTIME' while 'AUTO/FRAMING/FREEZE' won't do you any good. Just reminded myself of this upon playing w this build.  ;D

Same goes if you press half-shutter to show 'FRAMING' which will often show a frozen preview from last recorded clip. Must press Canon menu in/out to reset this. Sometimes forcing to turn off/on cam if that doesn't help. Indeed a bit hacky but definitely well worth the effort!

Obviously this is all temporarily solutions for now. Have fun and remember to utilize ETTR as much as possible for best color-noise reduction especially for a 7D (can pull down -2.00 more or less in MLV App) and let's try to kill as many bugs as possible while we're at it!

*EDIT2*

Also noticed a similar pattern with how if one were to press half-shutter to check preview while recording it can cause a corruption (pinkish effect) within MLV take. Just for a slight second (during the moment of pressing) as I also suspect this phenomenon happens while utilizing EOSM 4.5K stuff. Need to investigate this more!  :o
5D3.113 | 5D3.123 | EOSM.203 | 7D.203 | 70D.112 | 100D.101 | EOSM2.* | 50D.109

domo94

  • Member
  • ***
  • Posts: 131
Re: 12-bit (and 10-bit) RAW video development discussion
« Reply #1749 on: February 21, 2019, 08:29:24 PM »
Good to see a new update on the 10/12 bit build.

I guess I'll try it out.

I can't do 5x zoom for normal use, so I don't consider it. It's not worth it for me, personally, to do so.
I'd rather 900p 14 bit raw footage with my 7D to make magic.