3K/UHD 5D2 Raw development and Other Digic IV Cams

[nikfreak]

title says it all: "..and other digic IV cams"  and I bet the 7D will fit 3K continuous recording too
As far as I understand the LCD doesn't freeze now any more while recording 10/12bit @danne??

[Danne]

title says it all: "..and other digic IV cams"  and I bet the 7D will fit 3K continuous recording too
As far as I understand the LCD doesn't freeze now any more while recording 10/12bit @danne??

No freezing, no corruption, only happy, healthy MLV files 

*update. mv1080p mode that is

[Walter Schulz]

Try 720p60 in Canon menu with crop mode ... Frozen LCD here ...

[reddeercity]

I'll ask nicely one more time , please stay ON TOPIC !!!
If that can not happen then I will remove this thread all
Together , as I started this thread and all information will be
Lost .

[dfort]

This is for UHD/3.5k development.

Right, like I said before:

There are lots of parts to this.

You can continue trying to increase the image size but you won't be able to pass all that data to the card unless you reduce the bit depth and compress it.

This seemed like the best place to discuss the features needed to get Digic IV cameras recording at the highest possible resolution but since you feel I'm off topic I'll move the 10bit/12bit discussion somewhere else.

[reddeercity]

Thanks, I agree there many parts & compression will be needed
At some point but not right now . I what to focus on max resolution
& custom buffers . I just don't what HD resolution 10-12bit discuss here
It better on the 10-12bit thread

[dfort]

Just as a suggestion, you probably should get the 5D2 working with the "basic" CROP_PRESET_3x3_1X before attempting the more advanced stuff. I helped with that on the EOSM, 700D, 100D and work is in progress on the 650D and 6D.

Whatever path you take to get to higher resolution on these older Digic IV cameras, best of luck!

[Audionut]

Hats off to you reddeercity.  You've gone from a 'typical' user to someone willing to dip their toes in the water.  I was excited to see you start this thread, but so busy at the time that I never got around to jumping in and sending some praises your way.  The funny thing about coding for ML is that no one really knows what they are doing.  Some people can for sure code better then others, and some people have more knowledge regarding some things then others, but it's all trial and error to begin with.

My advice would be to let the conversation flow, even when it begins to go a little off-topic.  Even in the off topic discussion, you're likely to find pieces of the specific part of the puzzle you are trying to solve, and sometimes it's just best to move onto another area of the puzzle, and then come back to the original area of the puzzle after you've had some success elsewhere.  I also prefer to split discussions that become large and deserve their own thread, rather then one or two posts.  So if the off-topic drags on for to long, send a pm and I'll happily split it.  And try to remember that we're all working towards the same end goal.  dfort isn't here to ruin the party, take all of the accolades or all of the attention, he seems to have some good amount of spare time atm, and he also seems to be dumping all of that time into ML.  This is a good thing for the project, and in my eyes, the project always comes first.  Take advantage of his time and knowledge, bug him about coding stuff.  Bug him for knowledge.

And the guys with all the know how need to remember that things that might seem easy, or common sense, generally aren't so easy for us plebes.  And sometimes we're happy just to plod along at our own slow pace, despite the common sense.  If that's frustrating for you, well, what goes around comes around.   

I get email notification about PM and forum reports, and always check emails.  I can find time if I'm really needed.  I'm here to help.  If you need to vent, pm me, make it obvious you need to vent.  Then vent.

And lastly, for the sake of clarity, deleted threads only disappear from a users point of view, mod+ have access.  Keep that in mind if you accidentally delete something, it's not gone, just pm me with the details.

[ItsMeLenny]

Was anything happening with lossless compressed raw on 550D?
And I can't find the main threads on compressed raw anymore, was it ever applicable to anything less than 14bit?

[Walter Schulz]

And I can't find the main threads on compressed raw anymore

Top of page -> Downloads -> Downloads (pulldown menu) -> Experiments -> 4K raw video recording; lossless compression -> Forum discussion

[reddeercity]

@a1ex when you found the 3072 vertical resolution for the preset in this code from adtg_gui.c-1121

Code: [Select]

static int res3k_reg(int reg)
{
    if (regs[reg].dst == 0xC0F0)
    {
        switch (regs[reg].reg)
        {
            case 0x6804:                /* C0F06804 - raw resolution */
                return 0x52801AB;       /* from 0x528011B -> 3072px in raw_rec */
        }
    }In my code I chanced "0x60804"(digic v) to "0x6084"(digic iv) before I test it l was hopping to find the right size for the 3k preset as you have "0x52801AB"
was this found thought testing different registers or for investigating Log files ?
I wondering if this is the reason I'm lock to resolution 3584x1068 .

[a1ex]

I've used trial and error at first, then figured out the math.

From raw.c, raw_lv_get_resolution: you have 0xC0F06800/4 <-> 0xC0F06084/8.

Vertical units (high half of the 32-bit value) are pixels; horizontal units are probably number of column groups (some columns are probably read out in parallel, so you can't get an arbitrary number of columns; it has to be multiple of column_factor = 2 on 5D2 and 8 on 5D3).

For the full-res LiveView, I tried to mimic the photo mode configuration. That didn't work out of the box, so I've tried to extend the resolution in small steps, until hitting some barrier (such as getting invalid image data or black bars past some resolution); then I've tweaked other registers that seemed to be size-related, until breaking these barriers. The option to diff video modes from adtg_gui was very helpful; so was the option to lock the modified registers, effectively attempting to turn one video mode into another; then, from these modified registers, I've tried to find a minimal subset that does the job (figuring out the meaning of these registers in the process). That's the general workflow; here's a step by step example.

Math for the above value:
- default raw buffer size on 5D3: 2080x1318 (see this post, in particular, this file)
- in 1080p (the video mode I've started from), the values of these two registers are 0x10017 and 0x528011b => 2080x1319;
  had to subtract 1 to address an issue with invalid frames; explained here.
- out of these 2080 pixels, 146 are skipped from the left (optical black) and 2 from the right; that leaves 1932 active pixels;
  rounded to multiples of 8 to work on whole raw_pixblock's => 152+8 skipped, 1920 left
- (0x1AB - 0x17) * 8 - 160 = 3072.

[reddeercity]

Didn't read the previous post correctly
sorry typo , It should had been 0x6804 for the source .
Thank you , That helps clear up things 

[reddeercity]

I think I may have proof now that my custom buffer is correct for Full Frame (5792x3804 -- includes out of bound area) in Video Mode see screen shot below



I did some logging with one of my memspy build a few months back with some screen shots , never really noticed it before until now.
in the red box you see resolution changed -- in the yellow box is the buffer size 2040x1267 -> in the green box 5792x3804 is the sensor size including out of bound area .
this buffer size (2040x1267) is what I use to get 3584x1068 in video mode with the my really , really ugly hack of the 5d3 3k preset .
I guess I can assume the size is correct and this here seem to back it up , 

[dariSSight]

@reddeercity,
I've set up my Compiling System via dfort Mac tutorial, also clone dfort and I've test to see if my building Build work and every thing look like its work. Only thing I change was in raw.c was
#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0x1000)
#endif

to

#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0xF000)
#endif

by the way is that correct

I'm going to test it now, but what can I do to help out or get to your latest stable build?

by the way when I put code below in Terminal it gave me 2 build

cd Magic-LanternCloneRepository/platform/5D2.212
make zip
cp *.zip ~/Desktop
make clean

[reddeercity]

Came across some Ti (Texas instruments) chip info from a screen shot , it's the raw pipe with internal 10 & 12 bit
this is helping me understand what going on , for digic iv UHD

[reddeercity]

Reading my decompile rom in Microsoft  Visual Studio , very nice to use 
For some info found this

Code: [Select]

ff0322b4 > LimitRawSize [%d]
So I guess there must be some limits for processing image , would be nice to know what it is

[reddeercity]

Some liveview raw stuff from rom I think

Code: [Select]

ff034b80 Enable AE Sequence = %1d
ff034b9c Enable AF Sequence = %1d
ff034bb8 Output AF RAW Data = %1d
ff034bd4 Output AE RAW Data = %1d
ff034bf0 Output DEFM Raw    = %1d
ff034c0c Output RSHD Raw    = %1dso when the af & ae sequence are enabled there are output to more then one regs

[waza57]

I try to increase the size of 5D2 and i have some success without using adtg_gui.mo.
As soon as possible I hope come back  to post all my work on the 5D2.
For the instant I have a problem for increase height.

For Digic register like  C0f06084/88  i use EngDrvOutLV(digic_register, new_val) and it' s OK . The new value is correctly apply.

But i need to tweak C0F0713C/50 too for increase height , so i tried different solutions but the value of digic register don't change :

• EngDrvOutLV(C0F0713C, new_val);
• patch_memory(C0F0713C, oldValue, newValue, "comment");
• cache_fake(0xff186ee8, InstructNew, TYPE_DCACHE) (or ICACHE);            // *for this....
• patch_instruction(0xff186ee8, instructOri, InstructNew, "comment");      // ...and this I found in ROM

* in ROM1.BIN.dis (5D2)
   With C0F0713C for example:

Code: [Select]

ff186ee4: e51f0064 ldr r0, [pc, #-100] ; ff186e88: (00007d9c)
ff186ee8: e59f11c8 ldr r1, [pc, #456] ; ff1870b8: (c0f0713c)
..................
ff986ee4: e51f0064 ldr r0, [pc, #-100] ; ff986e88: (00007d9c)
ff986ee8: e59f11c8 ldr r1, [pc, #456] ; ff9870b8: (c0f0713c)

So i believe understand that  instructOri = e59f11c8 and try to increase it with InstructNew= e59f11dc
With this change I hope to transform  ldr    r1, [pc, #456]         in        ldr      r1, [pc, #476]   but this don't  works.

It is possible with adtg_gui.mo but there is too much troubles , please someone could explain me how to write in this registers (c0f0713c/50) ?

[reddeercity]

Hey waza57 , cool 
can you share some more details ? looks interesting -- can you save a dng?
I can't answer your question about "write in this registers"
I did all my stuff with the iso_research branch , which included 3k preset with crop_rec (1:1)
I didn't have any problems or issue with adtg_gui.mo , what was the problem , enabling changed registers valves ?

Found a tool to help with find fps head timer numbers , thou it have more uses then that
eostimergen  Thanks @g3gg0

EOSTimerGen is a tool for Magic Lantern users that want to
 a) over/undercrank the camera (to get higher or lower frame rates)
 b) fine tune their frame rate to overcome production tolerances
 c) synchronize two or more cameras to have nearly the same rate

You can do that with frame over ride but nice to do it in software to check it .

[waza57]

reddeercity, Thanks for that but it seems timer B don't increase height like with other models.

I tried a lot of things but nothing works.

Perhaps ADTG [8xxx] but unfortunately I don't succeed to see it with adtg-gui module in 5D2.

An example :


how to delete mint account


effective width: 2840
effective height: 1082

http://www.adapteo.fr/images/M17-0638_000000.dng

[ilia3101]

OMG 😍😍😍!!! waza57 your the person that made video dual ISO on 5D2 almols work rigtht??!

Can you record MLV yet with this confuguration? If not, how close is it??? And how could I help? The goal would be to put it in a single menu option in a normal build, how much work is that? I would love to help if possible. Or test it  (as every noob always says)

I'm so excited!!! 2.8K—that's Alexa sensor resolution (it must be a sign)

[reddeercity]

reddeercity, Thanks for that but it seems timer B don't increase height like with other models.
I tried a lot of things but nothing works.
Perhaps ADTG [8xxx] but unfortunately I don't succeed to see it with adtg-gui module in 5D2.

try maybe ADTG1 [105f]N , or the ADTG2 -- one is for 3xcrop mode & the other is for 1:1 FHD
I've had some success increasing a little height
I most I've been able to get is 1068(h) with the (w) 3584
I see the frame rate is 23.614 with exiftool , this maybe the issue -- the other test I've seen on digic4 have frame rate at 12fps
Have you tried to slow the frame rate down ? Did you try with frame override enabled or disabled ?

Very interesting @waza57 -- can you share your code yet ? I would like very much to look at it , if you don't want to publicly display it pm me 

I've being going with just modifying CMOS 1 & 2 registers with the Crop_Record Module loaded , it seem to put the sensor in 1:1 pixel mode centered
then enable the 5d3 3K preset ( the reg's are very close to 5d2 and it kind of works , I have a timer problem now) 
Did you get this in 3xCrop mode or 1:1FHD ?
Good job 

[reddeercity]

@ waza57 great ! you got your code up crop_rec_4k_5D2 -- I see you are extending the 3xCrop windows , interesting then for sure look at the those  ADTG1 [105f]N , &  ADTG2 in the adtg_gui.mo
I played around with then in 3xcrop_mode .
If this so , then the old  10bit without raw_slurp will work in 3xCrop_mode with that extended resolution .
I coming from it thought 1:1 FHD with the crop_record.mo -- it put the sensor into every line , by passes the line skipping ,
I'll have a closer look at the code  , maybe we can have a special "4k crop_record.mo " for digic4   

[ilia3101]

@reddeercity have you tested the 2.8k yet??? I really wanna see some nice samples of 5D mark ii 2.8k!!! It will give me motivation to fix my broken 5D2

I'm truly excited to see if anything finally happens with this camera.