3K/UHD 5D2 Raw development and Other Digic IV Cams

[waza57]

I think yes , from waza comments there more then just the crop_rec code that was changed .

Here , in "NOTES for developers: " You can find where i change the code other than in crop_rec module:

https://www.magiclantern.fm/forum/index.php?topic=19336.msg205624#msg205624

Sorry, always without more time to help....

[Jackeatley]

Hey 50D users I've started to port 50D to crop_rec , first step was to code a simple preset in ADTG_GUI.mo where a1ex first tested the 5d3 code.
Reasons why I put the 50D crop_rec in adtg_gui is to make sure it work or course & a simple coding task for me ! Since I very weak on that I
being teaching myself (with the help of the internet  ) c code syntax , I'm slowing making good process , much to learn but it's worth it in the end 

I can report working as expected with my 50d, have only experienced one freeze when the buttons wouldn't respond and the card led was solid, some reduced responsiveness but I expect that's due to the first version.

i'm getting around 7 seconds of 10bit onto my Komputerbay 1050x 128gb card.  Image looks pretty much as i'd expect, very impressed.

Thanks for all the hard work, lots of fun revisiting the 50d.

[york824]

I have been using this build for a few months now. Really enjoying it. Can't say thank you enough for your work.
It's also a learning process for me as I am really new to video. One small question though: I saw that there is an app called DSLR Controller on Google Play Store that seems to allow Android phones to control Canon DSLRs. My smartphone has a much better screen so I am wondering if it is compatible with Magic Lantern. I did some Google search on this but most information was from 2013 or 2014 so I am not sure how it is today.

[jpegmasterjesse]

You can expect problems using DSLR Controller and ML. Especially crop mode. In standard Raw recording you MAY be able to expect RAW video to work ok.

[zcream]

Looking forward to 1x1 2784x1160 on 50d. That is about the same as micro 43 crop due to lower Res sensor. I think FHD will be limited to 1600x1000 (approx) in 3x3 binning mode sadly.

Sent from my Lenovo TB-8703F using Tapatalk

[zcream]

No we need the higher write speed , compressed raw would give us (at .5 compression , which I read would the average)
from http://rawcalculator.netlify.com/calculator_desktop
basic 10bit just out of reach of cf/5d2-d4 cams

Code: [Select]

Resolution:2880x1080 10bit
 Data rate:88.9MiB/s
 Frame size:3.7MiB
 Crop factor:1.96x crop
 Field of view:20.8°
 RECORD TIME: 5 minutes 43 seconds per 32GB card.
14bit compressed raw at .5 compression

Code: [Select]

Data rate:62.2MiB/s
So this would be the max if lossless worked on 5d2 , better crop factor

Code: [Select]

Resolution:3200x1080
Data rate:69.1MiB/s
Frame size:2.9MiB
Crop factor:1.76x crop
Field of view:23.1°
 
16x9 2880 with increased height & compressed raw , still out reach of the current CF D4 bandwidth (75-80MB/s)
need faster interface --  overclock the cf bus driver or find away to run cards at udm7 , theoretically @ umd7 the write speed should be 120-133MB/s

Code: [Select]

Resolution:2880x1556
Data rate:89.7MiB/s
Frame size:3.7MiB
Crop factor:1.96x crop
Field of view:20.8°
 Max rez with compressed raw with increased height at the current cf bandwidth (75MB/s)

Code: [Select]

Resolution:2880x1200
Data rate:69.1MiB/s
Frame size:2.9MiB
Crop factor:1.96x crop
Field of view:20.8°
I prefer for more horizontal rez , thou there nothing wrong with increasing the height to the max .

This is what I'm hoping to reach as a goal for raw video , it could happen only with 2 things
compressed raw (.5) & faster UMD 6 ->7 & or overclock the cf interface on the write side

Code: [Select]

Resolution:3840x1556
Data rate:119.5MiB/s
Frame size:5MiB
Crop factor:1.47x crop
Field of view:27.5°
Now , I basic all this on continuous recording for raw video  @ 24p
If some user what to use the full width at lower frame rate with increase height at maybe 10 of 12 fps
for hyperlays etc... will that's great , so there's really should be no limit on resolutions just
limitations on the presets in the crop_rec.mo .

At least now I can say there's no more aliasing & moiré patterns 

I know this was an old post, but is there a possibility of overclock of CF interface? Something similar to sd_uhs for Eos m?

Sent from my Lenovo TB-8703F using Tapatalk

[reddeercity]

I know this was an old post, but is there a possibility of overclock of CF interface? Something similar to sd_uhs for Eos m?

Short answer , Yes . read links below
https://www.magiclantern.fm/forum/index.php?topic=19336.msg207053#msg207053
https://www.magiclantern.fm/forum/index.php?topic=19336.msg207054#msg207054
https://www.magiclantern.fm/forum/index.php?topic=19336.msg207128#msg207128
https://www.magiclantern.fm/forum/index.php?topic=19336.msg207348#msg207348
But  I haven had time to go back ,
Lossless Compression is closer to be do-able , I just have a few error to fix read this post
https://www.magiclantern.fm/forum/index.php?topic=18443.msg198487#msg198487
there again I haven had time to go back 
hoping to get there soon , if you like you can jump in and help , you will need to do some reading , ok a lot of reading 

[york824]

You can expect problems using DSLR Controller and ML. Especially crop mode. In standard Raw recording you MAY be able to expect RAW video to work ok.

Thanks for your reply.

[reddeercity]

doing some UDMA clocking on the CF Card interface with DIGIC POKE , I'm able to push the write speed to a average of 75.5MB/s  (I'm using a will used Lexar 1066x 32GB from 2014)
normally it's around 68MB/s , so peak write speed that I saw was around 80MB/s + . Seems to be a 10% increase ,
I don't think I have the right reg's yet or I need more reg's , I can change the reg's for write speed to as low as 5.5MB/s then I increase it slowly to 12 , 15, 20, 25, 33, 48, 60, 68MB/s to 75.8MB/s with different reg's valves .


So far

Code: [Select]

0x101seem to be the fastest at the moment  , I'll have to start working in the HDParm branch to see if I can clock it higher
I have the branch cloned just need to compile for 5d2 and investigate what can be done to increase write speed .

Edit: My test resolution for this was 2144x1078 @ 29.66 FPS in 14bit raw ( default ML 3x crop_mode)
so around 7-8 seconds .

[Luther]

Tested now the 50D build. Thanks to the @AF-OFF tip about "GB Allow" and "Preview Auto", I'm not getting pink frames. With other configs there is half-screen glitchs.

Using KomputerBay 64gb 1000x:
- 8 seconds on 10bit 3744x1080px
- 7 seconds on 14bit 2160x1080px
- 16 seconds on 12bit 2160x1080px
- More than 1min on 10bit 2160x1080px

This is awesome!

Don't know if that's expected or not, but it fails to load mlv_play.mo:

Code: [Select]

tcc: error: undefined symbol 'lossless_decompress_raw'
   [E] failed to link modules
UILock: 00000000 -> 41000000 => 00000000

Will it be possible to have live preview? And the 'anamorphic mode' from Danne's EOS M would be possible, eventually? The crop factor is too big for what I need, unfortunatelly
Thanks for all the work @reddeercity and everyone that contributed!

[reddeercity]

Don't know if that's expected or not, but it fails to load mlv_play.mo:

error message hints to the problem
mlv_play is no supported right now in crop_rec d4 branch , until I finish implementing lossless compression  (almost there) , been jumping around between different 5d2 & 50d
crop_rec problems . Trying to workout the bugs for crop_rec module on 50D ,(cmos hooks , are different for 50d then the 5d2 or 5d3) 50D uses "cmos3 & cmos4 for height & width
respectively , where all other cams to my knowledge uses cmos1 & cmos2 . So I need to change the code to use cmos 3&4 hooks (just opened up a big can of worms  )

Will it be possible to have live preview? And the 'anamorphic mode' ?

Should be possible , but not as good as the d5/5d3/EOSm , d4 cam 50d & 5d2 don't have as much processing power , so real time liveview will be limited
to the center crop of the resolution e.g. 3744x1080 =>1920x1080 b/w which gets resize for the lcd sceen to 720x480.
but the 1x3 Anamorphic just like the 5d2 has a de-squeezes function for liveview already in ML display features (2:1) close , should be (3:1)
see here https://www.magiclantern.fm/forum/index.php?topic=19336.msg213010#msg213010

[reddeercity]

Still having trouble with adtg hooks for d4/5d2 in crop_rec (needed for 1x3 anamorphic) there just don't work .
this is what I have

Code: [Select]

   
              /* ADTG2/4[0x1000] = 0: read every line */
              /* ADTG2/4[0x100C] = 2: vertical binning factor = 3 */ 
              if (is_5D2)
              {             
              adtg_new[2] = (struct adtg_new) {6, 0x1000, 5};
              adtg_new[2] = (struct adtg_new) {0, 0x100c, 2};
              }
              break;where "0x1000" default valve is "5" 0x5 (3xcrop_mode) and "6" 0x6 the line skipping mode that it's get changed to ,
and "0x100c" is the pinning , default valve is "2" 0x2 -> "0" 0x0
and add "1000" &  "100c" to the adtg_hooks

Code: [Select]

patch_hook_function(ADTG_WRITE, MEM_ADTG_WRITE, &adtg_hook, "crop_rec: ADTG[1000,100c,8000,8806,] parameters hook");
It I put the 5d2 1x3 preset in to the adtg_gui.mo like the 50D 3.7k preset I can get one of the adtg reg to load "adtg[12] 0x1000" but not "adtg[12] 0x100c"
this what I did to get it to work , commented out the first line and then it loaded the first case 0x1000.

Code: [Select]

//else if (regs[reg].dst == 2)        /* ADTG 12 */
    {
        switch (regs[reg].reg)
        {
            case 0x1000: return 0x6;      /* it's 5 in zoom mode and 6 in 1080p; this also overrides ADTG4 */
            case 0x100c: return 0x0;
            case 0x8806: return 0x6088; /* without this, you get some weird artifacts; this should only go to ADTG2, not 4 */
So I'm kind of lost , @a1ex I know you are not really working much here on the forum , but do you have any ideas where I should start to look to fix this 
Any help would be appreciated .

[Luther]

see here https://www.magiclantern.fm/forum/index.php?topic=19336.msg213010#msg213010

Wow. That seems good.
Got some pink frames while testing again today with 50D, but nothing too bad. It's working nicely with in 2560x1072px 10-bit.

[Luther]

One frame from the tests. Quite nice, considering the lens I'm using is 50 years old Also, the iso is 200, there's some noise. Processed with MLVApp.


Click for full-res.

[DeafEyeJedi]

Beautifully rendered @Luther and the contrast looks amazing especially coming from a 50 year old glass...

[Volumetrik]

I think I may be doing something wrong with the  crop_rec-4k3x1_2784x1160-1x1_FHD48p3x3_11.59pm-5D2-eXperimental.2019Jul15.5D2212  build. I can't get the preview or the recorded video to be centered.

Following instructions on the first post : 1080p30 --> crop_rec & mlv_rec only --> 10bit & 1:1 --> x5 zoom --> 3.5k preset --> 2784x1160 --> liveview and record.

Here is a normal photo of the scene. Full frame 3:2 :



And here is the framing I get out of the settings mentioned :



And here are these two pictures overlaid to illustrate the difference in framing.



Here is a picture of what I see when recording :



Am I doing something wrong ?

[Luther]

Beautifully rendered @Luther and the contrast looks amazing especially coming from a 50 year old glass...

Thanks. Some pixel pipping using mlvapp always helps the low contrast of the lens :p
Maybe I will do a video test, got inspired by what @york824 did.

[reddeercity]

Updating my source  code &  comments
https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/
8eae4f2 & 2ebc7c7
add the 50D raw_slurp for extended rez & 10/12bit raw in crop_rec
still have issue with 50d crop_rec compiling right now , but the rest of ml compiles ok .
hope to the rest of the 5d2 code updated by the weekend 

[reddeercity]

I think there a way to get 7D (1st ver.) in to crop_rec for extended resolution in 5xZoom .
By way of "Rom Hack" this by-passes the raw_slurp in the EDmac to push the vertical
that how waza57 first got the crop_rec to work on the 5d2 at 2880x1080 . Now the 5d2 doesn't need this any more as we found how to get raw_slurp working .
 See this commit e7bfe64 for detail (currently it's disable for 5d2 but wouldn't take long to enable it for 7D)

Code: [Select]

//static uint32_t RAWROMHEIGHT_WRITE = 0;
//static uint32_t RAWROMWIDTH_WRITE = 0;
//static uint32_t MEM_RAWROMHEIGHT = 0;
//static uint32_t MEM_RAWROMWIDTH = 0; 
etc. .....
It may be possible , but don't have a 7d to test on
an then it would have to be added to the crop_rec code -- thou I don't have the time but if someone with some coding ability & 7D wants to
get involved I can help push it alone .

[reddeercity]

Updated the first page with the 50D adtg_gui crop_rec experiment 3744x1080 @ 23.976 fps
https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476

[Galterius]

Hey !
I have been trying to get 4k anamorphic and 48p to work for a few days, but it keeps showing me weird flickering black and white liveview, which shows on the mlv file when opening it in mlv app, the weird thing is that I can see that its enabled (it shows 4064x590 when using 4k anamorphic, and 48p crop_rec seems to load correctly). So if you have any ideas, please help me ^^
Thanks a lot for your awesome work !!

[stokis]

It would be great to have 4k anamorphic working without bugs. I would say 4k anamorphic is second most  important thing after losless compression for 5d mkii. At least for me.

[reddeercity]

Hey !
I have been trying to get 4k anamorphic and 48p to work for a few days, but it keeps showing me weird flickering black and white liveview, which shows on the mlv file when opening it in mlv app, the weird thing is that I can see that its enabled (it shows 4064x590 when using 4k anamorphic, and 48p crop_rec seems to load correctly). So if you have any ideas, please help me ^^
Thanks a lot for your awesome work !!

did you do this ?
https://www.magiclantern.fm/forum/index.php?topic=19336.msg218753#msg218753

[Igor_Braun]

@stokis, absolutely agree. Even though my 5d2 stopped working (i've tried to dissasemble it, but made a mistake.... so, I can't participate en testings).

[reddeercity]

I would say 4k anamorphic is second most  important thing after losless compression for 5d mkii. At least for me.

I disagree , 5k Anamorphic in x5 Zoom is the most important thing to me (adtg hook problem), Lossless compression is way down on my priority list .
This is my priorities :Get 50D in to the crop_rec code at the same level as the 5D2 (including Anamorphic)
5D2:  5k Anamorphic in x5 Zoom (works in the adtg_gui module) adtg hook write problem in crop_rec code
CF Card interface bus Overclocking write speed (looking for at least 100MB/s write speed , 30% increase to be useful)
Lossless compression for 5d2 & 50d
And at the bottom ....
7D with limited "rom hack" extended  horizontal  resolutions (maybe 3k)

Edit: FYI 50D maximum resolution @ 24fps ( with a good A.R.)  in crop_mode 3200x1254 @ 24.011 fps = to 100.7 Mpixel/s ( Mega Pixels per second)
Sensor speed on the 50D is 28.8 MPixel/s x 4 channel readout = 115.2MPixel/s
Still have a vertical offset problem , only 1087 vertical pixels are visible out of 1254 (the other 167 lines are black no image data)
may need to do a dm_spy log for cr2/full res liveview height to see what regs are being applied .