Magic Lantern Forum

Experimental builds (WIP) => crop_rec and derived builds => Topic started by: reddeercity on April 06, 2017, 12:22:27 AM

Title: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 06, 2017, 12:22:27 AM
Full width 5.6k 16x9 A.R. , 4K Anamorphic 3x1, 1x3,  UHD 3K Plus 10-12bit Full HD ,
this is the latest build as things process I'll keep updating this post

See below short video on how to enable 48fps 3x3 preset

October 03 2020
Experimental build:MLV_Play.Mo and Raw_Twk.Mo Updated for Crop_Rec.Mo 4k3x1-3k1x1-2.8k1x1-48pfhd3x3
5D2_mlv_play+raw_twk_10-3-2020_8.18pm.zip (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/5D2_mlv_playraw_twk_10-3-2020_8.18pm.zip)

(https://i.ibb.co/ggbjTJt/5d2-mlv-play-10-3-20-small.png) (https://imgbb.com/)

Fixed problems with raw_twk & mlv_play modules on my D4/5D2 4k Crop_Rec branch .
Tested it in 3x3(1880x1258) , crop_rec 3.5k centered preset(2784x1160) & 3x1 4k anamorphic (4096x590)
there's all view ok in camera , except for 3x1 4k there noise/snow on the bottom half of the screen .
(the anamorphic part of the code is not vertically stretching the image need to look further in the code) but still viewable .
Make sure you load both module's in the zip file , that's it . when you what to view a file just press the > play button 


September 3 2020
Experimental build:50D eXperimental-crop-rec_3.7k_24fps
magiclantern-eXperimental-crop-rec_3.7k-1x1.11.30pm.2020Aug16.50D109.zip (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/magiclantern-eXperimental-crop-rec_3.7k-1x1.11.30pm.2020Aug16.50D109.zip)
Here is a stable Crop_Rec build for the 50D , so no more adtg_gui crop_rec needed.
There only one preset at the moment , as these's problem with cmos & adtg hooks which i'm working on .
Loads just like the 5D2 crop_rec module , the 50D Crop_Rec resolution is 3760x1080 @ 23.976fps
50D related info for crop_rec https://www.magiclantern.fm/forum/index.php?topic=19336.msg229944#msg229944

July 20 2020
Experimental build:Crop_Rec.Mo Updated for 4k3x1-3k1x1-2.8k1x1-48pfhd3x3
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2-4k1x3-3k1x1-2.8k1x1-48pfhd3x3_7-10-2020_8.42pm_crop_rec.zip
New Google Drive updated download link
5D2-4k3x1-3k1x1-2.8k1x1-48pfhd3x3_7-10-2020_8.42pm_crop_rec.zip (https://drive.google.com/file/d/)

New Updated Bitbucket Down Load Link(8-27-2020)
5D2-4k1x3-3k1x1-2.8k1x1-48pfhd3x3_7-10-2020_8.42pm_crop_rec.zip (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/5D2-4k1x3-3k1x1-2.8k1x1-48pfhd3x3_7-10-2020_8.42pm_crop_rec.zip)

Updated the Crop_Rec Module for multiplies of 3xcrop_mode/5xZoom presets .
Now there is 3k 1:1 (3016x1080 23.98fps) & Centered 3.5k 1:1 (2784x1160 24fps) in the same build plus 4k(3x1) & FHD 3x3 48p (1880x774)
With the new updated mlv_rec. module you get 8 more pixel in 3k & 24 more pixel in 48p 3x3 when use together .
Short Video tutorial on how to enable the new crop_rec presets in the latest update - (shot my 50D in 1x3 (1376x1742 @24fps ->4128x1742 from mlv. app)  ;)



July 5 2020
Experimental build:MLV_Rec.Mo Updated for 1880 (3x3) 2152(5xZoom)
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip
New Google Drive Download Link
1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip (https://drive.google.com/file/d/)

New Updated Bitbucket Down Load Link(8-27-2020)
1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip)

In Canon default  resolution 3x3 you gain 24 more pixels (1856->1880) & in 3xcrop_mode(5xZoom) there a gain of 8 pixels (2144->2152)
Will load on top of the Crop_Rec eXperimental builds , did not test with the nightly builds
(https://i.ibb.co/JvnTghP/VRAM103-small.png) (https://imgbb.com/) (https://i.ibb.co/0cWP49X/VRAM104-5x-Zoom-small.png) (https://imgbb.com/)


Jan 5 2020
Experimental build:5632x3124 @ 3.8fps
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip
New Google Drive download link
crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip (https://drive.google.com/file/d/)

New Updated Bitbucket Down Load Link(8-27-2020)
crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip)

This a 1:1 pixel 5632x3124 @ 3.8fps , Very eXperimental you need at least a 1066x CFcard for this .
Read the discussion in these posts
https://www.magiclantern.fm/forum/index.php?topic=19336.msg223937#msg223937
https://www.magiclantern.fm/forum/index.php?topic=19336.msg224036#msg224036

Update: Feb 10 2020             New video sample of the updated 16x9 1:1 5632x3124 3.8fps centered preset , 6min @ 3.8fps -> 1 min 24fps in a hyper lapse



Here is a sample from the 5.6k preset (5632x2356 4.22fps) **note** this the first draft of this preset , I have updated since.
Shot this Jan-3-2020 , at sunset (4:30pm mst) , use my ef24-70mm 2.8L canon lens @ 24mm
F8 , ISO 100 , 1/5th second and because it was a sunset I had to use a Variable ND filter and stop it down 4 stops
I processed the 10bit cdng's thought Adobe After Effect CS6 with ACR ,I didn't spend much time on grading , just the basic .
I capture 7:00 min's continuously to a 32GB Lexar 1066x CF Card ,
I set the A.E. project to 24fps and that made the time condensed to about 1:15 seconds from 7:00 min's .
Watch it @ 5K (2880p) for the best viewing .
 



Dec 31 2019
Experimental build: 4k(4096) 3x1 Anamorphic ,  2784x1160 1x1 @ 24.00fps & FHD 3x3 (1856x774) @ 48p
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip
New Google Drive Download link
crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip (https://drive.google.com/file/d/)

New Updated Bitbucket Down Load Link(8-27-2020)
crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip)

In proved 4k-3x1 HDMI preview at 480p & changed frame rate to 24.00fps , reconfigured 2784x1160 for true 24.00 fps & realtime preview,
(1856x774  2:40 A.R. @ 48.028fps) write speed is 82MB/s so close to continuous
reduced the rolling shutter in 48p from 29.7ms to 17.8ms  :D 

August 21 2019
Experimental build: Dual ISO Video Module
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dual_iso_video-updates_8-21-2019.zip
New Google Download download link
5d2_dual_iso_video_updated_21-8-2020 (https://drive.google.com/file/d/)

New Updated Bitbucket Down Load Link(8-27-2020)
5d2_dual_iso_video_updated_21-8-2020.rar (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/5d2_dual_iso_video_updated_21-8-2020.rar)

Works with my crop_rec builds (not too sure about the nightly builds didn't test them) at 3x3 FHD(1856x1248) & 5xZoom(2144x1074) (3x crop_mode) @ 10bit raw ,
I didn't check it with the Crop_Rec Preset , so if you do try it with Crop_Rec it may freeze you cam and may need I battery pull .

Oct. 22 2019
Experimental build: 50D ADTG_GUI Module Crop_Rec experiment 3744x1080 @ 23.976 fps
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/adtg-gui_crop_rec-3.7k-50D-eXperimental.2019Oct01.50D109.zip
New Google Download Link
adtg-gui_crop_rec-3.7k-50D-eXperimental.2019Oct01.50D109.zip (https://drive.google.com/file/d/)

New Updated Bitbucket Down Load Link(8-27-2020)
adtg-gui_crop_rec-3.7k-50D-eXperimental.2019Oct01.50D109.zip (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/adtg-gui_crop_rec-3.7k-50D-eXperimental.2019Oct01.50D109.zip)

Hey 50D users I've started to port 50D to crop_rec , first step was to code a simple preset in ADTG_GUI.mo where a1ex first tested the 5d3 code.
Reasons why I put the 50D crop_rec in adtg_gui is to make sure it work or course .

How to get started:
Load mlv_rec.mo & adtg_gui.mo
Enable adtg_gui go to the advance menu and enable ENGIO Registers

(https://i.ibb.co/VvfN2g6/VRAM0-small.png) (https://imgbb.com/)

Press half shutter to exit to liveview then press the play button (make sure to have a cr2 there to view ,this refreshes liveview & enables the other 1500 reg's)
go back to the advance tab in the adtg_gui.mo and now enable the 3k 1:1 50D preset

(https://i.ibb.co/48Wwqzk/VRAM2-small.png) (https://imgbb.com/)

Press half shutter to exit to liveview , then you may have to cycle thought the 5xzoom button ( but you should only have to press it once)
Make sure you set the shutter to as close to 1/48th of second as you can , I was able to set it to 1/49th and when the preset is enabled it will
go to 1/48th of a second  for 23.9876 fps @ 3744x1080

(https://i.ibb.co/6XXtmBW/VRAM3-small.png) (https://imgbb.com/)

Don't worry about the line on the right had side , there will not be in the raw image (this is the same thing that happen with the 5D2 , still working on a fix )
To make sure everything gets loaded , you can check the memory patches in the debug menu you should have 7 like this

(https://i.ibb.co/xJFSvcv/VRAM4-small.png) (https://imgbb.com/)

On a Lexar 1066x 64GB CF card was getting around 10 second in 10bit .

This is very Bleeding Edge Stuff , so expect lockups , cam liveview freezes , etc. .... if it brakes you get to keep both halves  :P



New Update: November 14/2019
3008x1080 @ 23.98 fps Updated Corrected Horizontal Centering
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-3k_Updated_Center_4.20pm-5D2-eXperimental.2019Nov14.5D2212.zip
New Google Drive Download Link
magiclantern-crop_rec-3k_Updated_Center_4.20pm-5D2-eXperimental.2019Nov14.5D2212.zip (https://drive.google.com/file/d/)

New Updated Bitbucket Down Load Link(8-27-2020)
magiclantern-crop_rec-3k_Updated_Center_4.20pm-5D2-eXperimental.2019Nov14.5D2212.zip (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/magiclantern-crop_rec-3k_Updated_Center_4.20pm-5D2-eXperimental.2019Nov14.5D2212.zip)

Details lower down on this post

*******************************************************************************************************************

So now you have stable 10-12bit in 1:1 Full HD with 3k crop_rec , more usable now .
To come full hd 48fps or maybe 50 & Full resolution 5.6k @4 fps  to start with.
Enjoy


Short Tutrial for Setting up Crop_Rec Module (3.5k 1:1 Preset) on the 5D Mark ii

For best results start with 30 fps ntsc in the canon Liveview menu
(https://image.ibb.co/gXcHYA/VRAM6-Small.png) (https://imgbb.com/)

Load crop_rec  & mlv_rec , mlv_snd can be load but reduces the amount of frames recorded .
(https://image.ibb.co/jXKT6V/VRAM7-small.png) (https://imgbb.com/)

Next setup Liveview preview , This is the most important part without this set correctly it will not work !
(https://image.ibb.co/gKpLmV/VRAM8-small.png) (https://imgbb.com/)
Liveveiw needs to be set to ML Gray Scale
Set bit depth to 10bit & aspect ratio to 1:1

Next go in to the crop_rec menu select 3.5k preset .
Note ! this only work when you enter 5x zoom/3x crop_mode as per the hi-lighted texts says
(https://image.ibb.co/dTBuDA/VRAM9-small.png) (https://imgbb.com/)

Once to go to 3x crop_mode you may need to go back to 1:1 then back to crop_mode  to get
2880x1080 sometimes , this is what you should see in the Movie tab once successful
(https://image.ibb.co/mgmSYA/VRAM10-small.png) (https://imgbb.com/)
Note the Red hi-light box the Frame Rate now says 23.976
Now you see the "crop_rec" is enabled & the resolution is now 2880x1080

Now go to the Raw Movie Tab you should see the following
(https://image.ibb.co/d2OGtA/VRAM11-small.png) (https://imgbb.com/)
You can choice resolutions less then 2880 if you wish , and you need to have ML Grey Scale selected

In Liveview this is what it should look like
(https://image.ibb.co/nimQmV/VRAM13-small.png) (https://imgbb.com/)
Notice the yellow circle "5x zoom"
Green hi-lighted area "Frame Rate" 23.976
The Red Hi-lighted area "crop_rec 1:1 3.5k preset "

For full color High Res preview press the half shutter
(https://image.ibb.co/fUo20q/VRAM14-small.png) (https://imgbb.com/)

Known Limitations
Once recording started you will have a Frozen B/W Liveview .
At this point it's unknown how to fix it yet , but Liveview return after recording stops.
In the current state you should see about 20 to 30 second of 10bit video.


New Update:

July 20 2020
Experimental build:Crop_Rec.Mo Updated for 4k3x1-3k1x1-2.8k1x1-48pfhd3x3
5D2-4k3x1-3k1x1-2.8k1x1-48pfhd3x3_7-10-2020_8.42pm_crop_rec.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2-4k1x3-3k1x1-2.8k1x1-48pfhd3x3_7-10-2020_8.42pm_crop_rec.zip)
Updated the Crop_Rec Module for multiplies of 3xcrop_mode/5xZoom presets .
Now there is 3k 1:1 (3016x1080 23.98fps) & Centered 3.5k 1:1 (2784x1160 24fps) in the same build plus 4k(3x1) & FHD 3x3 48p (1880x774)
With the new updated mlv_rec. module you get 8 more pixel in 3k & 24 more pixel in 48p 3x3 when use together .
Short Video tutorial on how to enable the new crop_rec presets in the latest update - (shot my 50D in 1x3 (1376x1742 @24fps ->4128x1742 from mlv. app)  ;)




July 5 2020
Experimental build:MLV_Rec.Mo Updated for 1880 (3x3) 2152(5xZoom)
1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip)

In Canon default  resolution 3x3 you gain 24 more pixels (1856->1880) & in 3xcrop_mode(5xZoom) there a gain of 8 pixels (2144->2152)
Will load on top of the Crop_Rec eXperimental builds , did not test with the nightly builds
(https://i.ibb.co/JvnTghP/VRAM103-small.png) (https://imgbb.com/) (https://i.ibb.co/0cWP49X/VRAM104-5x-Zoom-small.png) (https://imgbb.com/)

Jan 5 2020
Experimental build:5632x3124 @ 3.8fps
crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip)]
This a 1:1 pixel 5632x3124 @ 3.8fps , Very eXperimental you need at least a 1066x CFcard for this .
Read the discussion in these posts
https://www.magiclantern.fm/forum/index.php?topic=19336.msg223937#msg223937
https://www.magiclantern.fm/forum/index.php?topic=19336.msg224036#msg224036

Dec 31 2019
Experimental build: 4k(4096) 3x1 Anamorphic ,  2784x1160 1x1 @ 24.00fps & FHD 3x3 (1856x774) @ 48p
crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip)
In proved 4k-3x1 HDMI preview at 480p & changed frame rate to 24.00fps , reconfigured 2784x1160 for true 24.00 fps & realtime preview,
(1856x774  2:40 A.R. @ 48.028fps) write speed is 82MB/s so close to continuous
reduced the rolling shutter in 48p from 29.7ms to 17.8ms  :D

                                                                                                                     Short video on how to enable 48fps preset



                                                                                   Short test with 48fps in FHD (1856x688) then slowed down to 23.976fps & 12fps , shot @ 1/91th second , 400ISO lens @ f8


Same as before , it may lockup , freeze , etc. ... you guy know the routine
48p can be hard to enable , what I did was first put the cam in 1:1 (3x crop_mode)
enable 4k Anamorphic for a few second then switch to "FHD 48p" press the play but twice .
It will say in the left corner 1:1 but it's really 3x3 FHD , need to change that to say 3x3 .
FYI about 48p & shutter speed , to get correct shutter speed for 48p you should be double
your frames rate e.g. 24fps = 1/48th , 48fps = 1/96th of second ideally so the best I can get is 1/91th of second
to get that I set shutter speed in 3x3 without crop_rec enabled to 1/41th or 1/40th of a second then enable FHD 48p
should be 1/91th of a second , this give very good motion blur for slow-mo shots down to 12fps

August 21 2019
Experimental build: Dual ISO Video Module
dual_iso_video_5D2.mo (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dual_iso_video-updates_8-21-2019.zip)
Works with my crop_rec builds (not too sure about the nightly builds didn't test them) at 3x3 FHD(1856x1248) & 5xZoom(2144x1074) (3x crop_mode) @ 10bit raw ,
I didn't check it with the Crop_Rec Preset , so if you do try it with Crop_Rec it may freeze you cam and may need I battery pull .
Best to start out at 100 ISO with the recovery ISO at 800 , so 100/800 ISO , I wouldn't recommend a recovery ISO higher then 1600 ISO (200/1600 ISO Max.)
Anything beyond  that is just noise -- remember 5D2 has only 5 native analog ISO's (100 , 200 , 400 , 800 , 1600) the rest is just digital push or pulled ISO (e.g. 160 ISO = 100 ISO digital pushed)
The Zip file is just the dual_iso.mo (module) not the full build , if you think you need the full build with dual_iso.mo let me know , but it should work on top of
my builds on the first page , in theory it should work with the 5D2 10-12bit  magic lantern experiments


Oct. 22 2019
Experimental build: 50D ADTG_GUI Module Crop_Rec experiment 3744x1080 @ 23.976 fps
adtg-gui_crop_rec-3.7k-50D-eXperimental.2019Oct01.50D109.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/adtg-gui_crop_rec-3.7k-50D-eXperimental.2019Oct01.50D109.zip)
Hey 50D users I've started to port 50D to crop_rec , first step was to code a simple preset in ADTG_GUI.mo where a1ex first tested the 5d3 code.
Reasons why I put the 50D crop_rec in adtg_gui is to make sure it work or course .

How to get started:
Load mlv_rec.mo & adtg_gui.mo
Enable adtg_gui go to the advance menu and enable ENGIO Registers

(https://i.ibb.co/VvfN2g6/VRAM0-small.png) (https://imgbb.com/)

Press half shutter to exit to liveview then press the play button (make sure to have a cr2 there to view ,this refreshes liveview & enables the other 1500 reg's)
go back to the advance tab in the adtg_gui.mo and now enable the 3k 1:1 50D preset

(https://i.ibb.co/48Wwqzk/VRAM2-small.png) (https://imgbb.com/)

Press half shutter to exit to liveview , then you may have to cycle thought the 5xzoom button ( but you should only have to press it once)
Make sure you set the shutter to as close to 1/48th of second as you can , I was able to set it to 1/49th and when the preset is enabled it will
go to 1/48th of a second  for 23.9876 fps @ 3744x1080

(https://i.ibb.co/6XXtmBW/VRAM3-small.png) (https://imgbb.com/)

Don't worry about the line on the right had side , there will not be in the raw image (this is the same thing that happen with the 5D2 , still working on a fix )
To make sure everything gets loaded , you can check the memory patches in the debug menu you should have 7 like this

(https://i.ibb.co/xJFSvcv/VRAM4-small.png) (https://imgbb.com/)

On a Lexar 1066x 64GB CF card was getting around 10 second in 10bit .

This is very Bleeding Edge Stuff , so expect lockups , cam liveview freezes , etc. .... if it brakes you get to keep both halves  :P



New Update: November 14/2019
3008x1080 @ 23.98 fps Updated Corrected Horizontal Centering
magiclantern-crop_rec-3k_Updated_Center_4.20pm-5D2-eXperimental.2019Nov14.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-3k_Updated_Center_4.20pm-5D2-eXperimental.2019Nov14.5D2212.zip)
Updated the cmos[2] Horizontal offset to correct centering offset ,  used full res CR2 (5613x3744) for referencing center as this post explained here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg222262#msg222262)

****************************************************************************************************************************************************

Ok by now a lot  if not all ml user know about crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView  (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182052#msg182052)
Yes as it says I'm looking in to 3K/UHD on the 5d2 cam and being that most of the digic iv cam are similar
My hope is it will help on other cams like 7D , 50D , etc.... in reality only cam with CF Card have the potential
of recording 3K/UHD other SD cams maybe in short busts or reduce frame rate but I think the 5d2 and 7D should be capably
of 3K or close to it . That's my target , right now 10bit 2144x1076 @ 23.976p is a reality at about 60-63 MB/s Write speed
with a crop factor of 2.61 . Some may ask if this is possible on cams other then the 5d3 , will the whole discovery came from a
500d experiment (http://www.magiclantern.fm/forum/index.php?topic=10111.msg123909#msg123909) from Greg , at least that's my understanding . To my knowledge a1ex reproduce the experiment on 5d3 digic v
which ended up code in the adtg_gui module (https://bitbucket.org/hudson/magic-lantern/commits/50d8f06018)  from iso_research branch . That's what I'm basing my development on and hints that a1ex gave my here (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182149#msg182149) ,
so the first step was to compile a adtg_gui module from the iso_research branch.
Of which I had many problem , thank to dfort for helping me with this issue the thread can read here (http://www.magiclantern.fm/forum/index.php?topic=19307.msg182371;topicseen#msg182371). Once I load the module I can now get in to the cmos H & V.
I loaded the 5d3 3K registry preset in 3x crop mode and start play round .
I what so far I have discovered with the vertical registry I'm able to move the crop window from the left most leading edge to the right leading edge in 46 increments.
Stating at cmos[2] @ 0x10c <--0x40E
0x1c
0x2c
0x3c
0x4c
0x5c
0x6c
0x7c
0x8c
0x9c
0xac
0xbc
0xcc
0xec
0xfc
0x10c
0x11c
0x12c
0x13c
0x14c
0x15c
0x16c
0x17c
0x18c
0x19c
0x1ac
0x1bc
0x1cc
0x1dc
0x1ec
0x1fc
0x20c
0x21c
0x22c
0x23c
0x24c
0x25c
0x26c
0x27c
0x28c
0x29c
0x2ac
0x2bc
0x2cc
0x2dc
0x2ec


When I have any new developments  or share any info I'll post here and anyone that's wishes to help or build on my work Please join in
looking at you @Ilia3101  ::)   ;D



Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on April 06, 2017, 12:45:30 AM
I'll be standing by you with all of my support for this. Also own a 7D and a coworker of mine owns a 5D2 whom is more than willingly to loan to me for to test along. Thanks for doing this @reddeercity!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on April 06, 2017, 03:25:04 AM
Juicy!
I will do whatever is useful. If you need me to try hundreds of values on registers I can do that. Will be back in a couple days and ready to do stuff(excited to look at the source code :o and the Gr3g resolution hack)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 06, 2017, 05:42:57 AM
@ DeafEyeJedi good to hear.

@ Ilia3101 sound good , if you go to my bitbucket_downloads (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/) there's a atdg_gui build for 5d2 that I'm using ,
get that and load the adtg_gui , raw_rec & file_manger modules keep it simple . In the submenu of adtg_gui in Advance , toggle on the digic registers
then at the bottom of the menu you will see 1:1 3K crop mode  (5d3) load that preset and go back to the root of adtg_gui and you will see 2 Green icon by cmos[1] Vertical  cmos[2]Horizontal
beware you will get lockups & liveview freeze , just pull the battery and restart etc.... . I use frame over ride @ 23.976p . Start reading the code here (https://bitbucket.org/hudson/magic-lantern/src/50d8f06018dc52f72f04265432dd83106e495e85/modules/adtg_gui/adtg_gui.c?fileviewer=file-view-default#adtg_gui.c-144) for hints & info on the registers plus look at this  post (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182149#msg182149) for more help.
From what I understand it's a combination of registers & video timers according to a1ex.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on April 06, 2017, 07:02:11 AM
Thank you. I've played around with the atgdgooi branch before, to be honest it scares me a little, as I suspect I've done damage to the sensor (I think it increased the fixed Patten noise) by playing with it. I also discovered 'green_rec' along with 'magenta_rec' and 'vertical_bar_rec'.
Will try out the 3k presets when I get back. The goal is to find the right values of those registers for the other cameras(5D2) to increase resolution in the same way? Or would it be controlled by other registers on other cameras? I guess this what we need to find to get it working. I will be reading the wiki for some homework ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 06, 2017, 08:35:46 AM
Quote from: Ilia3101 on April 06, 2017, 07:02:11 AM
The goal is to find the right values of those registers for the other cameras(5D2) to increase resolution in the same way?
more or less , but first 5d2 then others .

Something interesting
cmos[1]0xaac <--0xeac
cmos[2]0x404 <--0x10E
looks like almost full view compared to the frame on the right which was not modified
the left all squeezed together .  I was using a EF24-70f2.8L at 24mm so in 3x crop mode that's 24mmx2.61=62.64mm
so I would say the left frame is very close to 24mm

(https://c1.staticflickr.com/3/2858/33055712593_8481a985f8_b.jpg) (https://flic.kr/p/Sn2aHc)
full view crop mode (https://flic.kr/p/Sn2aHc) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

Exported Jpeg from MLVProducer , I pushed the exposure to show detail.

(https://c1.staticflickr.com/3/2852/33055724173_5696406a53_b.jpg) (https://flic.kr/p/Sn2e9R)
M05-2224_00001 (https://flic.kr/p/Sn2e9R) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
Non modified registers cmos cdng 2K
https://www.dropbox.com/s/hzds2gbijp4tlzv/M05-2336_00000.dng?dl=0
Modified cmos[1] & cmos[2] cdng
https://www.dropbox.com/s/ng1zbz3xy8auopp/M05-2224_00000.dng?dl=0

It's a start  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Deadcode on April 06, 2017, 08:49:58 AM
Cheers! I hope you make it!

Is it possible to implement the lossless compression with older Digic 4 cameras?
With my calculations, the highest continuous resolution in crop mode might be 2736x1164 at 23,976 fps, with 2,1x crop ratio. That requires 128MB/s write speed and with the "usual" 58% compression is just about 74MB/s. That's around the max stable write speed with the 5D2 in crop mode.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on April 06, 2017, 08:59:32 AM
@Deadcode The compression is definitely possible in the 5D2 as it also has those features in the firmware that are being utilised on the 5D3 right now, but I think it takes an a1ex level of digic programming skills to do. So possibly may happen... or not. You can never predict.
@reddeercity squished crop mode would be pretty good, looks like there's a big black area on the right of that frame though. I bet getting rid of that pink corruption will be the hardest part.
Great start though!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 07, 2017, 02:44:24 AM
Yea I think compressed raw will work , I did a test merge of the compressed_raw branch in to unified source code and have a little problem with src/raw.c merge  that I hadn't had time to look in too fully but no crashes on the build .

Quote from: Ilia3101 on April 06, 2017, 08:59:32 AM
I bet getting rid of that pink corruption will be the hardest part.
No , shouldn't be too big of a problem , I can clean up the corruption by changing addresses . Main thing I'm looking for is the full View with the right aspect ratio right now .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: goldenchild9to5 on April 07, 2017, 08:05:44 AM
@reddeercity Great work..
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on April 07, 2017, 08:11:05 AM
Some of the values are actually bit fields. My old notes on 5D2 registers:


CMOS[0]: 0x203 at ISO 100
---- ------xx default 11, setting to 00 results in dark image with very low stdev
---- ---xxx-- ISO, values from 000 to 100, ascending (ISO 100-1600)
---- xxx----- second ISO field (as in dual ISO), disabled by default (see next bit)
---x -------- enables second ISO field (default 0)
--x- -------- enables vertical OB clamping maybe (setting it to 0 results in severe horizontal banding that looks like random walk)
-x-- -------- compresses the image horizontally (left side squashed, right side black)

CMOS[4]: 0x242 at iso 100...800 and 0x244 at iso 1600
---x -------- looks like some vertical dual ISO
-x-- -------- seems to cleanup 0.2 or 0.25 stops of shadow noise
No effect observed on the other bits.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on April 07, 2017, 01:45:18 PM
@reddeercity Can't wait, but what is the atdg_gui build file for? So far the only way to use 10bit or your possible build is to push the magnify but for 5X to put in crop mode and you have to have a lens like 24-70mm 2.8L lens to view?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 07, 2017, 11:09:29 PM
Quote from: dariSSight on April 07, 2017, 01:45:18 PM
@reddeercity Can't wait, but what is the atdg_gui build file for?
For developing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 07, 2017, 11:19:31 PM
Quote from: a1ex on April 07, 2017, 08:11:05 AM
Some of the values are actually bit fields. My old notes on 5D2 registers:


CMOS[0]: 0x203 at ISO 100
-x-- -------- compresses the image horizontally (left side squashed, right side black)

Thanks  :) , interesting that's on cmos[2] 0x404 reproduces this .
I found more on Greg's work last night read thought the CMOS/ADTG/Digic register investigation on ISO thread , and it seems
I may have been adjusting the wrong registers (cmos 1&2) it seems he was adjusting cmos 5 &6 for digic iv cams
I'll give that a try .
No ISO related , seems it has more in common to 5D3
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 08, 2017, 09:51:39 AM
Some more update/process , Getting cleaner image now with some surprise's  :D

(https://c1.staticflickr.com/4/3952/33776580001_4c3b4d137a_b.jpg) (https://flic.kr/p/TsHNtB)
M07-2145_00001 (https://flic.kr/p/TsHNtB) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr   5643x1076


(https://c1.staticflickr.com/3/2819/33776587241_a225ae18b8_b.jpg) (https://flic.kr/p/TsHQCr)
M07-2149_00157 (https://flic.kr/p/TsHQCr) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr    5643x1076

Ok I had to do a little post work to get it to the right Aspect Ratio   ::), I still getting the compressed image but it's seem to be all there.
I crop off the black area then just increased the horizontal by 300% and left the vertical unchanged

(https://c1.staticflickr.com/4/3727/33062768594_ef793a51fb_z.jpg) (https://flic.kr/p/SnDkdo)
M07-2145_00001 (https://flic.kr/p/SnDkdo) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

(https://c1.staticflickr.com/4/3816/33062774114_e01881b6ae_z.jpg) (https://flic.kr/p/SnDmRy)
M07-2149_00157 (https://flic.kr/p/SnDmRy) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

Screenshots from crop mode , showing the capture area that's what recorded for reference only

(https://c1.staticflickr.com/3/2930/33062795444_28a1527301_b.jpg) (https://flic.kr/p/SnDtcj)
5D2 Liveview (https://flic.kr/p/SnDtcj) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

I'm having problem with the FPS timer A , it doesn't want to go below 610 , I have no problem with timer B in fact I had it up to 3000 that help clean up the frame
it seems timer A is lock and I think that what stopping me for getting the full width capture not compressed to 2K frame , I had at about 17 fps to get a clean image

Or do I have to play with the Head Timers? I know there's three Can't find any info on them , I believes I need to get timer A to 455 or in that range and timer B about 2200 .

Cheers
Edit: This is frame with FHD 1856x1044 I used on my registers crop full view test , for reference only
(https://c1.staticflickr.com/3/2889/33777229671_62d61665f6_z.jpg) (https://flic.kr/p/TsM8AP)
M07-2337_00001 (https://flic.kr/p/TsM8AP) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on April 08, 2017, 11:09:44 AM
Good work. Does the black area go away if you move the crop window by the right amount? Yay now we have the 5D3 3x1 binning from crop rec!
Edit: technically the black bar can't go away unless you set the resolution to < 1881. Also that's insane spectrum ratio at 5.25:1
@a1ex genius. Are the registers 12 bit? That's weird ::)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 09, 2017, 07:28:34 AM
I'm getting more stability now ,
the registers I'm using is:
ADTG1 [105f]N-----0xa41
CMOS[1]------------0x2c7
CMOS[2]------------0x404

Exact FPS
FPS timer A --------610 (can't  lower it for some reason)
FPS timer B -------2000
main clock---------24.00 Mhz
Actual FPS---------19.672

Image is very clean now , no artifacts.
but image is still compressed to the left , I believe that because of FPS timer A is not be able to lowered from 610
I think there's something I'm missing , I feel I'm getting very close now If I can figure out the video timer problem .

Edit:

ADTG1 [105f]N-----0xab56
CMOS[1]------------0x2c7
CMOS[2]------------0x404
CMOS[5]------------0xc05

New setting that open up all registers , before I couldn't figure out way I didn't see the registers at where documented in other posts.
By accident I started to play around with CMOS[5] and BOOM there is all these Registers  :o
there go on forever . Now I access more RAW H&V and video timer stuff.


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on April 10, 2017, 04:48:41 PM
@reddeercity I've now tried the iso-research branch and setting registers, your settings didn't work too well for me, I managed to get the squish working with cmos[0], but black level kept going up and down causing it to all look ugly, setting cmos[1] and others to values you recommend didn't help much other than locking up the camera. Basically i've not had any success.
But one more thing: when the correct registers/settings to unlock resolution are found, will it automatically be detected in the raw_rec or mlv module so the higher resolution can be set straight away?(I mean: does it auto detect max resolution?)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: nikfreak on April 10, 2017, 04:57:54 PM
once the sweet spot(s) for 5D2 are found you could for e.g. adjust/activate your new "crop mode / resolution" via ML menu for the moment. Check a1ex's 2016 implementation (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182149#msg182149)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 11, 2017, 02:29:16 AM
@Ilia3101 ,when selecting registers I find it's more temperamental the warmer the sensor gets . There is a certain  order you have to do and of course the video timers have to adjusted or it will not work not matter what you do. 1st set to low light , video timer A ( low as it can go) , video timer B (2000 and up) works in the adtg_gui  advance menu enable digic registers then apply the 1:1 3k preset (which sets cmos[1] & cmos[2] the I adjust cmos[2] first for width then cmos[1] for height next is ADTG1[105f]N (x5 & x10 zoom shutter blanking), move the frame up & down by either by shorting or lengthen shutter duration, to fix pink frame you may have to find a different registers in cmos[1] oh yea also enable cmos[5] helps with pink hi-lights you may also need to play with frame over ride a bit , seems that the frame is clean around 18-20 fps not sure why. Also It looks like I may have got the 1:1 Crop_record working on the 5D2 so you don't have to be in zoom mode for it to work  ;D  As it say 1:1  but at 1856 not 5616 but I can move the capture area cross the sensor just like 3x crop ,  plus in that mode I was able to record a H264 in 1:1when I turn off raw video but the problem I having and you know there's always a problem  ::) I can only record 1856 width(center crop) but full height which is compressed vertically , once I stretch it out it's 1865x3744 .

1:1 Crop_recording with 3K preset , This is Not 3x Crop mode but 1:1 , like I said with compressed vertically . 1856x1044

(https://c1.staticflickr.com/3/2943/33579296740_8b26e7bcfd_z.jpg) (https://flic.kr/p/TahF35)
M10-021100001 (https://flic.kr/p/TahF35) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr 

After I resized the vertical by about 258% I get 1856x2694 close to full height of the sensor but at 16x9
 
(https://c1.staticflickr.com/3/2908/33151750183_9c03de8ab8_z.jpg) (https://flic.kr/p/SvvokH)
1856x2694-M10-021100057 (https://flic.kr/p/SvvokH) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

1:1 crop_record H264 from the 5D2 , turn off the raw video and hit record  :) , Maybe Highly Compressed h264 3K or 4K  :P :P
Just click on the image and ficker will link to the file on page. or watch off dropbox see link below

(https://c1.staticflickr.com/4/3871/33579323710_ef0f247671.jpg) (https://flic.kr/p/TahP45)
MVI_8653 (https://flic.kr/p/TahP45) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

So how do I know if it's truly 1:1 , will I don't really only thing that tells me is the fact there no moiré or aliasing pattern
look at the the fine hole on my MacPro even in h264 I can't tell , thou the lens I use was 24mm so no lens correction
which give the distortion width at the top .

H264 1:1 crop_record MVI_8653.MOV (https://www.dropbox.com/s/06mqxxa40aqkupf/MVI_8653.MOV?dl=0) source file link from my dropbox if any one what to download 45Mb

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 11, 2017, 09:01:21 AM
Doing some more quick tests in 1:1 LiveView with the following setting
adtg1[1061] 0x449
cmos[1] 0x3e0
cmos[2] 0x104
cmos[5] 0xc21

23.976 from 24
Video Timer A 584 (FT +12)
Video Timer B 3422 (FT +1672)
Actual frame rate 12.00

crop_record 1:1 (Not 100% sure it working but seems to setup a lot easier and centered
1:1 3K preset that's it .

3000x1250
(https://c1.staticflickr.com/3/2872/33839219381_57645a82f5_c.jpg) (https://flic.kr/p/TyfQYV)
M10_3000x1250 (https://flic.kr/p/TyfQYV) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr


3000x1250
(https://c1.staticflickr.com/4/3933/33928144306_f1aabc196d_c.jpg) (https://flic.kr/p/TG7Bgh)
M10_2_3000x1250 (https://flic.kr/p/TG7Bgh) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

There's a little trick I did to get this to work , I post tomorrow on what the special thing is  8)
The hint is in the frame size   ;) 

ProRes files of the 3K test  , not big  about 100Mb each
M10_1.mov (https://www.dropbox.com/s/0475byz3iw23csi/M10_1.mov?dl=0)
M10_2.mov (https://www.dropbox.com/s/1onvbkm14l0indn/M10_2.mov?dl=0)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on April 11, 2017, 09:27:40 AM
I see the hint: 1250 ;) Maybe you got something like the vertical compression in the post before but sideways? just a guess...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: goldenchild9to5 on April 12, 2017, 01:00:39 AM
@reddeercity Great job  :) Now I wanna buy a 5D2 as a back-up to my 5D3. 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Greg on April 12, 2017, 02:51:07 AM
This will not work on old cameras (500D, 5D2 etc) until you find a way to change the EDMAC size.

Old cameras do not support #ifdef CONFIG_EDMAC_RAW_SLURP
https://bitbucket.org/hudson/magic-lantern/src/11f405b62b312294d705e8e7bb8688b820bb9eea/src/raw.c?at=crop_rec_4k&fileviewer=file-view-default#raw.c-429

Maybe patch here :
LiveViewMg:000929e0:00:00: *** ConnectWriteEDmac(0x10, 0x0), from ff1f7134
LiveViewMg:000929e0:00:00: *** SetEDmac(0x10, 0x463390a4, 0xff44e5c8, 0x20000000), from ff1f7164


Or try "#define CONFIG_ALLOCATE_RAW_LV_BUFFER"
https://bitbucket.org/hudson/magic-lantern/src/11f405b62b312294d705e8e7bb8688b820bb9eea/src/raw.c?at=crop_rec_4k&fileviewer=file-view-default#raw.c-126
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 12, 2017, 03:05:24 AM
Thanks for the info Greg I looking for something like this
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 12, 2017, 04:22:01 AM
So would EDMAC#5 cofo4500
address:43e28a4
State : 1
Flags : 20000040
Size : B (4046 x1127
Connection : write=0x0    read=0xff

found that in "Show EDMAC" in the debug menu
Look like the right frame size if that's what the 4046x1127 is ?

Or I'm I miss understanding this .

Looking at this #crop_rec_4k-raw.c-479 (https://bitbucket.org/hudson/magic-lantern/src/11f405b62b312294d705e8e7bb8688b820bb9eea/src/raw.c?at=crop_rec_4k&fileviewer=file-view-default#raw.c-479) , so if I replace "EDMAC+8" with "EDMAC+5 ,  will that work or I'm I off base here ?
   /* autodetect raw size from EDMAC */
    uint32_t lv_raw_height = shamem_read(RAW_LV_EDMAC+4);
    uint32_t lv_raw_size = shamem_read(RAW_LV_EDMAC+8);
    if (!lv_raw_size) return 0;

    int pitch = lv_raw_size & 0xFFFF;
    *width = pitch * 8 / 14;
   
    /* 5D2 uses lv_raw_size >> 16, 5D3 uses lv_raw_height, so this hopefully covers both cases */
    *height = MAX((lv_raw_height & 0xFFFF) + 1, ((lv_raw_size >> 16) & 0xFFFF) + 1);
    return 1;
#endif


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Greg on April 12, 2017, 06:40:09 PM
4046/14*8 = 2312
So the size is 2312 x 1127

500D "LV_Initialize %s" :
int a1 = 0x1D78 + 0x2600;
*(a1 + 208) = 1664;
*(a1 + 210) = 660;

It looks like width and height raw, but who knows.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 13, 2017, 08:47:54 AM
That's Greg  :)
Now the Crazy Part  :o

Some how either with dumb luck or divine intervention ::) I manage to get 3.5K in 1:1  @1.6 crop  8) 8) 8)

Check it out , thou I couldn't record any image just yet had a small error in edmac-memcpy.c line 80

(https://c1.staticflickr.com/4/3944/33622003720_5fcb760cb6_o.jpg) (https://flic.kr/p/Te4ykj)
5D Mark ii experimental 3.5K Raw Video (https://flic.kr/p/Te4ykj) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

The write requirements for that size was reported about 198MB/s @30fps 14bit :o
If and when I get if working fully then compressed raw is next.

The files I modified are raw.c , edmac-memcpy.c
in raw.c I add this

#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (2040*1267)
#endif


also in raw.c  changed to 0xC0F04500 from  0xC0F04508

#if defined(CONFIG_5D2) || defined(CONFIG_50D)
#define RAW_LV_EDMAC 0xC0F04500
#endif



in edmac-memcpy.c change the r/w channel to 0xff & 0x0 from read 0x19 & write 0x03
If may be a problem as I have a write issue , need to look further in to this of course.

#if defined(CONFIG_5D2) || defined(CONFIG_50D)
uint32_t edmac_read_chan = 0xff;
uint32_t edmac_write_chan = 0x0;


That's a good step in the right direction and the start many buggy problem to come
But Hey I'm so much closer now thanks to all  :D
Poor mans 1DC  :P :P :P
more to come

Edit:FYI to get this build I clone the source ,  updated with iso-research branch then modified the files 
Title: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on April 13, 2017, 09:20:07 AM
Really incredible stuff and I am loving this progress so far, David!

'Poor man's 1DC' -- very well said!

Also after some thoughts in my head recently I remember helping @nikfreak out w 70D/100D merge awhile ago by doing this...

https://vimeo.com/197958207

Guess it's basically like a cheat sheet by clicking on 'don't click me' then afterwards you go into EDMAC which will show a screen full of registers addresses after while in each mode (mv1080, mv720, 5x zoom crop, etc) and it seemed to help him out.

Maybe you already are aware of this protocol but felt it was worth sharing just in case.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: nikfreak on April 13, 2017, 09:49:03 AM
I can recommend to undo all code snippets from above. Yor best bet is to try to get near 3k width (http://www.magiclantern.fm/forum/index.php?topic=19336.msg182706#msg182706) with adtg / fps overrides
Don't change the EDMAC channels. They have proven to work for years and there's find_free_edmac_channels() if you really need to add more.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on April 13, 2017, 10:58:32 AM
@reddeercity Wow, difficult to believe thats happening :o :o :o I was a little pessimistic about getting rid of that pink corruption, but you got rid of it, so I guess you'll be able to fix the record error ;D I can't be much help developing this stuff, as I don't understand how the the Magic Lantern code even works and I'm not a lot of use at programming in general. I'm always ready to test tho :D :D

@nikfreak What is the level of risk in doing this kind of stuff, corrupted image or corrupted ROM?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: nikfreak on April 13, 2017, 11:19:01 AM
Low risk of a brick if you know what you do. You got to keep all pieces if it breaks ofc.
You can get raw recording errors, corrupted frames or wrong recording sizes etc.

http://www.magiclantern.fm/forum/index.php?topic=19336.msg182847#msg182847

follow the links to Greg's PoC and a1ex's 3k implementation.(Digic5) They did it just with modifying register(s). The difference between Digic5 and Digic4 is:

Ofc you must find apropriate value for 3k width for 5D2. Don't think that 5D3's will match 5D2.

Check this code snippet (https://bitbucket.org/hudson/magic-lantern/src/50d8f06018dc52f72f04265432dd83106e495e85/modules/adtg_gui/adtg_gui.c?fileviewer=file-view-default#adtg_gui.c-126)

I guess Canon did some calculation themself befor implementing zoom/crop modes for FHD H264 30FPS recording. Now you can calculate yourself how much you can extend the resolution in zoom/crop mode (x*y*fps) for recording with 24fps. Maybe 15-20% could be fine but you might have to extend the buffer, too...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Greg on April 14, 2017, 08:59:02 PM
500D  LV zoom 5x - "StartPass_x5 CrawAddr:%lx / Kind:%d"
SetEDmac(0x10, *(0x16604 + 808), 0xFF44F0B0, 0x20000000);

ROM:FF44F0B0
ROM:FF44F0C8        DCD 0xE1C, 0      //‭3612‬
ROM:FF44F0D0        DCD 0x35D, 0, 0   //861


5D2  LV zoom 5x - "StartImagePass_x5 CrawAddr : %lx / KindOfCraw : %d"
SetEDmac(0x5, *(0x13ECC + 804), 0xFFCAA1D4, 0x20000000);

ROM:FFCAA1D4
ROM:FFCAA1EC        DCD 0xFCE, 0       //‭4046
ROM:FFCAA1F4        DCD 0x467, 0, 0  ‭  //1127‬



This code will increase EDMAC size, but the buffer will be overflowed - it's dangerous.
static void run_test()
{
    static uint32_t craw_5x_edmac_h = 0xFF44F0D0;
    patch_memory(craw_5x_edmac_h, 861, 861 + YRES_DELTA, "lv_edmac_h"); // add height
}


Now we should use our own buffer.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 15, 2017, 04:03:14 AM
Thanks for the code but I'm not really sure how implement it  , being read the source code raw.c & edmac_mencpy.c
for clues still not sure , can you point me in the right direction please.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ItsMeLenny on April 15, 2017, 08:08:14 AM
I'm keen to get this going on the 550D.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Greg on April 16, 2017, 02:27:06 PM
I increased the height by 140px (841px to 981px). More does not work.
I do not know why, some problem with memory or edmac.

To do this you need to change (500D):
CMOS[5] //vertical offset
ADTG[2]1172 -> old_val + YRES_DELTA (nrzi)
ADTG[2]1173 -> old_val + YRES_DELTA (nrzi)
ADTG[2]1178 -> old_val + YRES_DELTA (nrzi)
ADTG[2]1179 -> old_val + YRES_DELTA (nrzi)
ADTG[2]1214 -> old_val + YRES_DELTA (nrzi)
ADTG[2]120A -> old_val + YRES_DELTA (nrzi)
C0F0713C -> 0x370 + YRES_DELTA
C0F07150 -> 0x388 + YRES_DELTA
C0F06088 //RAW last line|column
C0F06008 -> 0xaf90af9   //photo mode value
C0F0600C -> 0xaf90af9  //photo mode value
C0F06010 -> 0xaf9    //photo mode value
C0F06014 -> 0xc83   //photo mode value
patch EDMAC size
patch buffer
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Greg on April 16, 2017, 09:33:28 PM
Buffer patched :

void* buf = malloc(2064 * 2000 / 8 * 14);
MEM(0x16604 + 808) = buf;


(https://s16.postimg.org/y4mp9zyph/500d-lv.jpg)

It still works a bit randomly, some frames have standard height...

So if someone finds a solution, it will work.
I do not plan further research.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 17, 2017, 01:29:51 AM
@Greg Cool thanks , I'll keep going .
Just a couple of questions.
Was this test with the 3x Live view code & edmac size buffer patch   or just buffer patch ?

I still haven't tried the code you posted , need to find where I should add this code too .



Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Greg on April 18, 2017, 10:59:26 PM
It was in live view 5x zoom. Both buffer patch and edmac size.

The buffer should not be needed :

silent.c
/* Reprogram the raw EDMAC to output the data in our buffer (ptr) */
raw_lv_redirect_edmac(sp_frames[next_slot % sp_buffer_count]);
sp_num_frames++;


raw.c
/* to be called from vsync hooks */
void FAST raw_lv_redirect_edmac(void* ptr)
{
    #ifdef CONFIG_EDMAC_RAW_SLURP
    redirected_raw_buffer = (void*) CACHEABLE(ptr);
    #else
    MEM(RAW_LV_EDMAC) = (intptr_t) CACHEABLE(ptr);
    #endif
}


So ML uses its own buffer, which should have the correct size after patching EDMAC.

If you do not solve this problem, we will announce the end of digic IV support. :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on April 18, 2017, 11:55:34 PM
The custom buffer is required, as ML redirects it only while recording.

In the crop_rec_4k branch I've used a SRM buffer (which can accommodate a full-res 14-bit picture). This needs to be passed to SetEDmac as the first argument (instead of Canon's default buffer).

The buffer redirection without CONFIG_EDMAC_RAW_SLURP is fragile: it relies on lucky timing. It's best refactored somehow, but cache patching is also ugly...

It's quite difficult to get these patches right, but I'm making progress on a tool that should help understanding the LiveView internals a lot better.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 19, 2017, 07:24:39 AM
@Greg thanks for the info
Quote from: Greg on April 18, 2017, 10:59:26 PM
If you do not solve this problem, we will announce the end of digic IV support. :D
Yea  I had thoughts , but if there a will there way  :D

Quote from: a1ex on April 18, 2017, 11:55:34 PM
The custom buffer is required, as ML redirects it only while recording.
So when I use the custom RAW_LV_BUFFER_ALLOC_SIZE (2040*1267) , RAW_LV_EDMAC  0xC0F04508 & r/w channel to 0xff & 0x0 and got 3584x1068 (got all this from the "show edmac"  in the debug menu in 1:1)
(I forgot to mention in  Reply #27 where I got 3.5k , I enabled adtg_gui with digic registers and applied the 3k 5d3 preset)
So this happen by chance to have access to that lv buffer size ? Oh , I also had the Raw video frame size at the 3584 preset , thou I could not adjust the size or aspect ratio
it was lock to that size . Just trying to understand better .

Edit: A thought just came to mind , so if there was a higher preset  , lets say 4096 etc. .... instead of 3584 would it go to the highest possible size ? or is this limited to lv buffer size for that  frame size and I had frame override turned off with canon menu frame rate set to 30 ntsc

Quote from: a1ex on April 18, 2017, 11:55:34 PM
....... but I'm doing progress on a tool that should help understanding the LiveView internals a lot better.
Yes  :) That great to hear @a1ex
There's still hope for digic iv
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on April 19, 2017, 07:48:04 AM
Just FYI, I had an attempt (https://bitbucket.org/hudson/magic-lantern/commits/1ee967909f18d06f8af21d6f244fd1062fb59cfd) to find out the size of Canon's LiveView raw buffer on 5D3.

Result (not committed): it's not allocated using "traditional" malloc, but its address (and probably its size too) are hardcoded, near 0xFF16B318 (1.2.3). The size reported by the memory "allocation" routine was 0x1cae000 (30MB - enough for 10-bit full-res LiveView), but when trying to use that, I've noticed the bottom of the frame being overwritten.

So, I don't know the size of the default LV RAW buffer. On 1100D (a camera with only 128MB RAM), Canon's default buffer is overwritten (by other Canon code) even at small LiveView resolutions, so the safest way is to assume nothing about it...

Most models appears to handle x5 zoom fine though, but that's not a guarantee other tasks are not using this memory. Might even be the cause for some kind of frame corruptions...

(memory management on these cameras is quite strange, with many hardcoded (http://www.magiclantern.fm/forum/index.php?topic=5071.0) areas, fragmentation, and descriptions found in strings not always matching the reality)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 20, 2017, 09:05:27 AM
Thanks help a lot to understand , since I had a little luck with that custom lv buffer I used in post#27 I thought I would try and investigate further and see if any variable change EDMAC#5 in 1:1 & 3x crop . So I notice that in photo live view there is no secondly lv buffer for 5x zoom(3x crop) (could be that it's in 1:1 pixel mode so not needed I guess) but in movie mode there is and by switching Raw photo mode in the canon menu from raw to small raw1 & small raw2 the address changes as follows:
EDMAC Address 103e28a4  (3578x1267 1:1)(4046x1127 3xcrop) raw(5616x3744) (Movie mode)
EDMAC Address 103e28a4  (3578x1267 1:1)(4046x1127 3xcrop) small raw1(3861x2574)(Movie mode)
EDMAC Address 43e28a4   (3578x1267 1:1)(4046x1127 3xcrop) small raw2(2784x1856)(Movie mode)

EDMAC Address 103e28a4  (3578x1267 1:1)(No buffer change) (5x zoom) (3xcrop) raw (5616x3744) (Photo mode)
EDMAC Address 103e28a4  (3578x1267 1:1)(No buffer change) (5x zoom) (3xcrop) small raw1(3861x2574)(Photo mode)
EDMAC Address 43e28a4   (3578x1267 1:1)(No buffer change) (5x zoom) (3xcrop) small raw2(2784x1856)(Photo mode)


Not sure if this help , look interesting -- from this I'm concluding that 3X crop has a larger lv buffer maybe because in 1:1 pixel instead of line skipping and pixel binding
for FHD , just guess there. Next I think I'll try that large buffer and see if I can hard code the edmac address unless ML check for available addresses (no sure) , specially with the small raw2 and check the results .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on April 20, 2017, 09:19:00 AM
Uncompressed 14-bit data takes 14 bits regardless of what pixel binning mode is used. Should be obvious...

The addresses are allocated dynamically, so of course, changing the camera settings (or sometimes, just starting in a different mode) will give different numbers.

Photo quality shouldn't be relevant (it might change slighly the amount of available RAM, or how it's divided between SRM and shoot buffers), but you will have to use some memory allocation API to get a memory buffer.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 21, 2017, 04:50:17 AM
Ok ,understood .
Quote from: a1ex on April 20, 2017, 09:19:00 AM
..... you will have to use some memory allocation API to get a memory buffer.
Maybe this will help ? did a search for "memory allocation API "and got this , go figure developer ARMS api , could be a wealth of information , assuming I can understand it  ::)
https://developer.arm.com/docs/0030/latest/modules/memory-allocation

Edit: something interesting to look at schematic diagrams for 5d2 I think , takes me back to my early auto mechanic days when computers started (mid '80's ) to control basic engine functions
which of course was very buggy  :P
https://photo-parts.com.ua/parts/Datasheets/KCD-A210-BACH8SchematicDiagrams_ENG.pdf
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 21, 2017, 07:00:17 AM
Some interesting info from the datasheet
Quote32-bit input capture register : 4 channels
https://photo-parts.com.ua/parts/Datasheets/TMP19A43FDXBGTMP19A43FDXBG.pdf

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 21, 2017, 08:23:05 AM
I think I found register memory map  :D
http://www.ti.com/lit/ds/symlink/tms320dm368.pdf
page 50 of 208
Table 3-1. System Module Register Memory Map

Table 3-1. System Module Register Memory Map
Hex address Register           Acronym            DESCRIPTION(1)
0x01C4 0000                    PINMUX0            Pin Mux 0 (Video In) Pin Mux Register
0x01C4 0004                    PINMUX1            Pin Mux 1 (Video Out) Pin Mux Register
0x01C4 0008                    PINMUX2            Pin Mux 2 (AEMIF) Pin Mux Register
0x01C4 000C                    PINMUX3            Pin Mux 3 (GIO/Misc) Pin Mux Register
0x01C4 0010                    PINMUX4            Pin Mux 4 (Misc) Pin Mux Register
0x01C4 0014                    BOOTCFG           Boot Configuration
0x01C4 0018                   ARM_INTMUX      Multiplexing Control for Interrupts
0x01C4 001C                  EDMA_EVTMUX     Multiplexing Control for EDMA Events
0x01C4 0020                  DDR_SLEW           DDR Slew Rate
0x01C4 0024                  UHPICTL               UHPI Control
0x01C4 0028                  DEVICE_ID           Device ID
0x01C4 002C                  VDAC_CONFIG      Video DAC Configuration
0x01C4 0030                  TIMER64_CTL       Timer64 Input Control
0x01C4 0034                  USB_PHY_CTL      USB PHY Control
0x01C4 0038                  MISC                   Miscellaneous Control
0x01C4 003C                  MSTPRI0             Master Priorities Register 0
0x01C4 0040                  MSTPRI1             Master Priorities Register 1
0x01C4 0044                  VPSS_CLK_CTL    VPSS Clock Mux Control
0x01C4 0048                 PERI_CLKCTL        Peripheral Clock Control
0x01C4 004C                 DEEPSLEEP           DEEPSLEEP Control
0x01C4 0050 -                                          Reserved
0x01C4 0054                 DEBOUNCE0         Debounce for GIO0 Input
0x01C4 0058                 DEBOUNCE1         Debounce for GIO1 Input
0x01C4 005C                 DEBOUNCE2         Debounce for GIO2 Input
0x01C4 0060                 DEBOUNCE3         Debounce for GIO3 Input
0x01C4 0064                 DEBOUNCE4         Debounce for GIO4 Input
0x01C4 0068                 DEBOUNCE5         Debounce for GIO5 Input
0x01C4 006C                DEBOUNCE6          Debounce for GIO6 Input
0x01C4 0070                DEBOUNCE7          Debounce for GIO7 Input
0x01C4 0074                VTPIOCR               VTP IO Control
0x01C4 0078                PUPDCTL0 IO         cell pullup/down on/off control #0
0x01C4 007C                PUPDCTL1 IO         cell pullup/down on/off control #1
0x01C4 0080                HDVICPBT              HDVICP Boot Register
0x01C4 0084                PLL1_CONFIG         PLL1 Configuration Register
0x01C4 0088                PLL2_CONFIG         PLL2 Configuration Register


http://www.ti.com/lit/ug/sprufg5a/sprufg5a.pdf
More Registers (4.1 Memory Map) Page 25 Table 6. Memory Map & Page 27 Table 7. ARM Configuration Bus Access to Peripherals
http://www.ti.com/lit/ds/symlink/tms320dm368.pdf
page 85 Table 6-9. PRTC Interface (PRTCIF) Registers
page 86 Table 6-10. Power Management and Real Time Clock Subsystem (PRTCSS) Registers
http://www.ti.com/lit/ds/symlink/tms320dm368.pdf
page 91 Table 6-16. EDMA Channel Synchronization Events
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on April 21, 2017, 08:32:44 AM
Quote from: reddeercity on April 21, 2017, 07:00:17 AM
Some interesting info from the datasheet https://photo-parts.com.ua/parts/Datasheets/TMP19A43FDXBGTMP19A43FDXBG.pdf

That's the MPU (http://www.magiclantern.fm/forum/index.php?topic=17596.0).

Quote from: reddeercity on April 21, 2017, 08:23:05 AM
http://www.ti.com/lit/ds/symlink/tms320dm368.pdf

That's probably a chip similar to the one running JPCORE (http://www.magiclantern.fm/forum/index.php?topic=18443.msg177082#msg177082), but we could not confirm this hypothesis (http://www.magiclantern.fm/forum/index.php?topic=18443.msg177264#msg177264).

For memory allocation API (https://en.wikipedia.org/wiki/Application_programming_interface), see http://www.magiclantern.fm/forum/index.php?topic=8358.0 (and http://www.magiclantern.fm/forum/index.php?topic=5071.0 for some more low-level info).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 23, 2017, 08:42:33 AM
going thought the branches I came across "new-lv-buffer-detections"  :o
as this is my issue atm , so I complied it are run it enabled the mem_spy.cfg & mem_chk along with raw video
not too sure how to use it correctly . mem_spy was running in the background but I was hoping it would make a log file but did not.
I was able to run the stub test in the "burn in test" with mem_chk and it did generate a log file , not knowing how long it would run I aborted after a few minute
it below here
[Pass] strlen("abc") => 0x3
[Pass] strlen("qwertyuiop") => 0xa
[Pass] strlen("") => 0x0
[Pass] strcpy(msg, "hi there") => 0x17b068
[Pass] msg => 'hi there'
[Pass] snprintf(a, sizeof(a), "foo") => 0x3
[Pass] snprintf(b, sizeof(b), "foo") => 0x3
[Pass] strcmp(a, b) => 0x0
[Pass] snprintf(a, sizeof(a), "bar") => 0x3
[Pass] snprintf(b, sizeof(b), "baz") => 0x3
[Pass] strcmp(a, b) => 0xfffffff8
[Pass] snprintf(a, sizeof(a), "Display") => 0x7
[Pass] snprintf(b, sizeof(b), "Defishing") => 0x9
[Pass] strcmp(a, b) => 0x4
[Pass] snprintf(buf, 3, "%d", 1234) => 0x2
[Pass] buf => '12'
[Pass] memcpy(foo, bar, 6) => 0x17b000
[Pass] foo => 'asdfghuiop'
[Pass] memset(bar, '*', 5) => 0x17afe0
[Pass] bar => '*****hjkl;'
       bzero32(bar + 5, 5)
[Pass] bar => '****'
       t0 = *(uint32_t*)0xC0242014 => 0xccf82
       msleep(250)
       t1 = *(uint32_t*)0xC0242014 => 0x61e8c
[FAIL] ABS(mod(t1-t0, 1048576)/1000 - 250) => 0x168
       LoadCalendarFromRTC( &now )
       s0 = now.tm_sec => 0x5
       Date/time: 2017/04/23 00:05:0ü&,¸ ]&,½ l,CK<&²Ã
–(¹`UrV‹x&'°ËX'¢½"k":‹%R­JD&ÿ½´j"'K'¿Ëu%}©<L¢ ¨'²Ã w$Hªô^ÂÓ ì',½Ìv(mÇh‡BþŒP)¸J7%E¸ u'‹ÌØ(ÇKv& ½€wbÒ ¤&ÂØÒÓ2yßBÍс3ó8       m0 = MALLOC_FREE_MEMORY => 0x203c8
[Pass] p = _malloc(50*1024) => 0x104930
[Pass] CACHEABLE(p) => 0x104930
       m1 = MALLOC_FREE_MEMORY => 0x13bd8
       _free(p)
       m2 = MALLOC_FREE_MEMORY => 0x203d8
[Pass] ABS((m0-m1) - 50*1024) => 0x10
[Pass] ABS(m0-m2) => 0x10
       m0 = GetFreeMemForAllocateMemory() => 0x3e55e8
[Pass] p = _AllocateMemory(256*1024) => 0x972f94
[Pass] CACHEABLE(p) => 0x972f94
       m1 = GetFreeMemForAllocateMemory() => 0x3a5574
       -_FreeMemory(p)
       m2 = GetFreeMemForAllocateMemory() => 0x3e5580
[Pass] ABS((m0-m1) - 256*1024) => 0x74
[Pass] ABS(m0-m2) => 0x68
       m01 = MALLOC_FREE_MEMORY => 0x203d8
       m02 = GetFreeMemForAllocateMemory() => 0x3e5520
[Pass] p = _alloc_dma_memory(256*1024) => 0x409d0270
[Pass] UNCACHEABLE(p) => 0x409d0270
[Pass] CACHEABLE(p) => 0x9d0270
[Pass] UNCACHEABLE(CACHEABLE(p)) => 0x409d0270
       _free_dma_memory(p)
[Pass] p = (void*)_shoot_malloc(24*1024*1024) => 0x4e5d00e8
[Pass] UNCACHEABLE(p) => 0x4e5d00e8
       _shoot_free(p)
       m11 = MALLOC_FREE_MEMORY => 0x203d8
       m12 = GetFreeMemForAllocateMemory() => 0x3e6634
[Pass] ABS(m01-m11) => 0x0
[FAIL] ABS(m02-m12) => 0x1114
[FAIL] p = (void*)_shoot_malloc(24*1024*1024) => 0x0
[FAIL] UNCACHEABLE(p) => 0x40000000
[FAIL] ABS(m02-m12) => 0x101fc
4ÐD-òûÏÅ+fðع¤ŽØ*"åN¡)×,,ŽâÍ@(ÒË z'vÂ<gÎ ¼%¢·KT$4­üE"¸Jô#B§Ê;#J <1"߉˜"Rœ‰'"©—by‰ ¡oÇÂo(À¡ýF!pÀp ¿ÇpÇÁpÃÑDApÅoLÄ¡ ÇáoGÃp<ÅaGLq‡Ä pÊ ‡`ÁqGÇq¬Ë‡ QqÌp¤Ì¡ÇÀqÏq¸Ì±GŒQq‡Ì q´ÉAÇhqpÇÈq"ɱX1qGÆ qdɇ|ÁpÉpÀÈ¡G˜1p‡ÊqÈËA‡¼QqÏqρGqrÇÐ)r@Õ¡(‡xtÛ9rœÛñ[Pass] suite = shoot_malloc_suite_contig(0) => 0x86b91c
[Pass] suite->signature => 'MemSuite'
[Pass] suite->num_chunks => 0x1
[Pass] suite->size => 0x1cf8000
[Pass] chunk = GetFirstChunkFromSuite(suite) => 0x86b940
[Pass] chunk->signature => 'MemChunk'
[Pass] chunk->size => 0x1cf8000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x42004070
[Pass] UNCACHEABLE(p) => 0x42004070
       shoot_free_suite(suite); suite = 0; chunk = 0;
[Pass] suite = shoot_malloc_suite(64*1024*1024) => 0x9b8758
[Pass] suite->signature => 'MemSuite'
[Pass] suite->num_chunks => 0x3
[Pass] suite->size => 0x4000000
[Pass] chunk = GetFirstChunkFromSuite(suite) => 0x9b877c
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x1cf8000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x42004070
[Pass] UNCACHEABLE(p) => 0x42004070
       chunk = GetNextMemoryChunk(suite, chunk) => 0x9b87f4
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3724000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x565d00e4
[Pass] UNCACHEABLE(p) => 0x565d00e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0x9b886c
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x4000000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x525d00e4
[Pass] UNCACHEABLE(p) => 0x525d00e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0x0
[Pass] total => 0x4000000
       shoot_free_suite(suite); suite = 0; chunk = 0;
[FAIL] suite = shoot_malloc_suite(0) => 0x0
[FAIL] suite->signature => '‡'
[FAIL] suite->num_chunks => 0xe59ff014
t*†H½.½°ü"¶ˆ+²...H­-†ŒXþÒÍH0,BˆÞ-ֈظ'– -ŽHú,ǃ¨•ÒX˜,"‰ˆÛ1÷'hüÊH˜/ãŽH +´ŒÔ$ÉÄ/‹Èâ/Ã"LgŠ"A¤Á bB#¾¼É"VL€@³¦In.ˆÔu'ÔØ#"z‡-"œx wÀ quÇöHu¼'[GœuÇúEu˜ùEpAtGúTtöQ=ÇÑsí:v¨þLÇä!u‡þ Zu<õ1IÇ8¡tÇî6sÌîaF‡Ôs‡ì4s íá2‡ ±rGè;t ðAFÇh¢t‡  UuûA[X "w$#¤}ìVú‡L'ƒH™)@‡àÌ,Hð-,‡H¡(8,X`
HÔ&²ƒÈ,­'¸
$6£Ê¬@¿µF$,,‹àE¶Ê9<Ò ¬QC}É`3"/ðŽýRáH,0C‰ /‹,,ê±HÈ-"ŠÖ.¼Œæ2͐0òŒï1è'#ÃÉ|1ˆî,•‰³k€)r,ÇW!³uDöÑF‡1sÇç@s0äAE‡$ Rv&¹...üÆB–ˆ -âˆHÑ.ÈŒðòÊÈì/cˆ
0ûŽ ÃíȨ/R‹ÈÓ.¸‹¸ãÒåˆ"0ãI$9œ«˜+ô ,,F¤»KfD³·T44{Ë@D"º pG÷ÀTsì °D3´
Þ6t˜,úBÈHˆ("GC Žuðïá6Œs‡ïRvÜâžG#R{Ç*"¯y BfÇÈÑu‡õSu
ò‡Ü#r€Hƒ.£˜có'Ê\?"ºÌ€MžÓx %YÍ|OäÎ̯DQ¶ÅCpÊ63š ,$Š ›wˆ(,†‡)ŒìÏ@I\2ƒ¡Š˜AÜ¿‡"㠐PÕÜNfY{ìˆìÕCÏÔ`¦ö4c×ûœXö8Ég¦¢i–é‡ÖF hFЋgI éb†ûfæÿyg'Ù,Ö=ñgŠg'AtV#ÐagcfþxoF+ÐqgÆqfþ$CæЈcÆõ
^0ë˜Ë5¾ÎX\ÅìNÈ\·ì€Íµ¸h\•êµZ...å(•>ÎÌWâNrVßl`µë¼UÅávUÜÌ@žlSÕÙÍ*S•Ùu...
°QµÖÍQlÕP ...;Í°NôÎLÊJËÇx'4B ,F´¾KBB±·´/K\>Ó°Ëò@E¸|ŠôÈÍa— "†‹TR1wÉný‰cÔ‰ƒä......7L >銥;¥­8ÚÓ ?"² ?9³°õC! `>±ÊÚ=ê¬4Å"»Š <3«Š¼;¯«|¶¤Êt;cªŠ¹;¬«È¸3¤Š€;"§J¶: ªô³cªÊ$;é
¨9¦ä—ãrÊÐ8"¥Êˆ8h¥Ìa
:Ó¥
¡9s¦<›ƒhJD9S¥Ê‡8W¥ "3e
9ó¥
™9l§À ³zJ:㨊¬9—¨ø£C›ŠL:3¨Š¤:†¨€ "Šü9"§Šš9h§l"CgJ 9¦
‹8h¥èŠã]
8¦
8p¥À‰cŠÌ8c¦Š'8f¦¼‰SLŠ¸7s£Š}60 ¬bcŠT5£‰N6ô£,Š³œ
´<ä²
DÕÇôŠÔûŒ°L4×ÍãN"Ù$Ýd¤ÌM"×MÝMƒ×TÙŒÍ,,M"ØMÐLØÌÓ¤˜ÍÄMtØ
ØM˜×Ñ"c¼L"ÙÍÓLy×tÏ´ƒÈLtÖMÇLƒÖlÁTnÍxKՍ¿KEÔä¯4/ͬJdԍ§J+Ïp´üŒxHÄÍ ~HàÌ}ÄÃŒ4G,,ËÌnF±Êäe"Ì(FÆLTC6ÁÐ'4ó‹°Aƒ½‹ý>ž¶¸×#@ `<3°J¤9½¨sãVŠ(6s¢IJ3ô›`£' H0•Ií.>Í"ßHˆ+R‹ª)§ˆÀ‹,l (...È{&E,üf'pGÀpÔÀ¡û† !p‡Á
o¾¡
±pGÁpì¾AG$1pÇÄpøÄ¡Ç$1qǽpÁÑG$qqÃr8Á‡8±rÃ(rdÂ'%h1sÃ/r,ÅÑ)‡PÑq‡Ã'rhÁÑ)‡8'rGÄ-r4Ʊ*‡ParÇÄ&rLÃ%‡`¡qÂq@Á‡$!r‡À#rHÂ1‡tsÅ*rDÅÁ;ÇxasGÉBsTÃ!<,,QtGÌRuÇñe‡œáuÇÌav¸È1\G|ñuÈUu È1EÇœtÅ/rhÇÑ(G°'tËVyÓa¹ÇÌá~Hå6‰¼öÁÐHÄrI X™äÒËÉ !¢œ !ᝀÌIŒ!B‰!âœ|"·‰!ÒšI m"Œ
úHL RH áp rú!'" $#½¡lEB...Š8%R®Ka& ±pb20‹Ô&² l&B³¸nâC‹'bµ u'Iµq"j 0'²¶ z'o¸|zbË"'·K~'²·Ü~ò›‹ü'2¹ (¼»ˆ¸Kh(òº‹†(Á»D...R¼‹€(R½‹Š(î¾ÀŠ'æ‹,)r¿L'(Áä–â h)RÁÌ–))ÂX"'3ŒŒ)RÄL)\Äl 'XŒä)bÅÌŸ*iÇ £', ˜*rÈŒ§*œÉD¨R£Ìh*,É̪*ÀÌ,,®bÃÌä*rÌ ³+ÛÍ ³,èL8+bÍ̵+éвRç̼+²ÎŒÂ+ÿÑ,,·R0,ÂÑͼ+3ÔôÄ\
D,BÕMÇ,\ÖtËUÍ-R×ÍË,'ØìÏR¥M(-òٍÐ-¼Ü4ÙÒ×
"-òۍÓ-àÝ"×,ëÍø-Þ
Ü-üàüáBøÍ(.'àÎç.'Ý ÌM¨+2Ð̤(¼<jRúJˆ$ò¢I* `Š,,ô±Jˆ€ñ,Èé:ŠìÿPIX!B¡J*"ž®ø7rVK0%ÒÅ       SetGUIRequestMode(0); msleep(1000);
[FAIL] CURRENT_DIALOG_MAYBE => 0x26
[FAIL] display_idle() => 0x0

There some memory address stuff but not LV , so I should re-run the stub test and see  if there any LV stuff ?
is there any info I can read on how to use the mem_spy for the raw lv buffer or is this a red herring
thanks in advance.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 24, 2017, 05:15:56 AM
Ok run the full stubs test with "new-lv-buffer-detections" branch results below
[Pass] strlen("abc") => 0x3
[Pass] strlen("qwertyuiop") => 0xa
[Pass] strlen("") => 0x0
[Pass] strcpy(msg, "hi there") => 0x177048
[Pass] msg => 'hi there'
[Pass] snprintf(a, sizeof(a), "foo") => 0x3
[Pass] snprintf(b, sizeof(b), "foo") => 0x3
[Pass] strcmp(a, b) => 0x0
[Pass] snprintf(a, sizeof(a), "bar") => 0x3
[Pass] snprintf(b, sizeof(b), "baz") => 0x3
[Pass] strcmp(a, b) => 0xfffffff8
[Pass] snprintf(a, sizeof(a), "Display") => 0x7
[Pass] snprintf(b, sizeof(b), "Defishing") => 0x9
[Pass] strcmp(a, b) => 0x4
[Pass] snprintf(buf, 3, "%d", 1234) => 0x2
[Pass] buf => '12'
[Pass] memcpy(foo, bar, 6) => 0x176fe0
[Pass] foo => 'asdfghuiop'
[Pass] memset(bar, '*', 5) => 0x176fc0
[Pass] bar => '*****hjkl;'
       bzero32(bar + 5, 5)
[Pass] bar => '****'
       t0 = *(uint32_t*)0xC0242014 => 0x63161
       msleep(250)
       t1 = *(uint32_t*)0xC0242014 => 0xa1441
[Pass] ABS(mod(t1-t0, 1048576)/1000 - 250) => 0x4
       LoadCalendarFromRTC( &now )
       s0 = now.tm_sec => 0x15
       Date/time: 2017/04/23 18:34:21
       msleep(1500)
       LoadCalendarFromRTC( &now )
       s1 = now.tm_sec => 0x17
[Pass] mod(s1-s0, 60) => 0x2
[Pass] mod(s1-s0, 60) => 0x2
       m0 = MALLOC_FREE_MEMORY => 0x203d8
[Pass] p = _malloc(50*1024) => 0x104930
[Pass] CACHEABLE(p) => 0x104930
       m1 = MALLOC_FREE_MEMORY => 0x13bd8
       _free(p)
       m2 = MALLOC_FREE_MEMORY => 0x203d8
[Pass] ABS((m0-m1) - 50*1024) => 0x0
[Pass] ABS(m0-m2) => 0x0
       m0 = GetFreeMemForAllocateMemory() => 0x3eb7fc
[Pass] p = _AllocateMemory(256*1024) => 0x9670fc
[Pass] CACHEABLE(p) => 0x9670fc
       m1 = GetFreeMemForAllocateMemory() => 0x3abaa0
       -_FreeMemory(p)
       m2 = GetFreeMemForAllocateMemory() => 0x3ebaac
[Pass] ABS((m0-m1) - 256*1024) => 0x2a4
[Pass] ABS(m0-m2) => 0x2b0
       m01 = MALLOC_FREE_MEMORY => 0x203d8
       m02 = GetFreeMemForAllocateMemory() => 0x3eba4c
[Pass] p = _alloc_dma_memory(256*1024) => 0x4096713c
[Pass] UNCACHEABLE(p) => 0x4096713c
[Pass] CACHEABLE(p) => 0x96713c
[Pass] UNCACHEABLE(CACHEABLE(p)) => 0x4096713c
       _free_dma_memory(p)
[Pass] p = (void*)_shoot_malloc(24*1024*1024) => 0x42004074
[Pass] UNCACHEABLE(p) => 0x42004074
       _shoot_free(p)
       m11 = MALLOC_FREE_MEMORY => 0x203d8
       m12 = GetFreeMemForAllocateMemory() => 0x3ebaac
[Pass] ABS(m01-m11) => 0x0
[Pass] ABS(m02-m12) => 0x60
[Pass] suite = shoot_malloc_suite_contig(24*1024*1024) => 0xa6e5e4
[Pass] suite->signature => 'MemSuite'
[Pass] suite->num_chunks => 0x1
[Pass] suite->size => 0x1800000
[Pass] chunk = GetFirstChunkFromSuite(suite) => 0xa6e608
[Pass] chunk->signature => 'MemChunk'
[Pass] chunk->size => 0x1800000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x42004070
[Pass] UNCACHEABLE(p) => 0x42004070
       shoot_free_suite(suite); suite = 0; chunk = 0;
[Pass] suite = shoot_malloc_suite_contig(0) => 0xa7a038
[Pass] suite->signature => 'MemSuite'
[Pass] suite->num_chunks => 0x1
[Pass] suite->size => 0x1cf8000
[Pass] chunk = GetFirstChunkFromSuite(suite) => 0xa7a05c
[Pass] chunk->signature => 'MemChunk'
[Pass] chunk->size => 0x1cf8000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x42004070
[Pass] UNCACHEABLE(p) => 0x42004070
       shoot_free_suite(suite); suite = 0; chunk = 0;
[Pass] suite = shoot_malloc_suite(64*1024*1024) => 0xa7a69c
[Pass] suite->signature => 'MemSuite'
[Pass] suite->num_chunks => 0x3
[Pass] suite->size => 0x4000000
[Pass] chunk = GetFirstChunkFromSuite(suite) => 0xa7a6c0
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x1cf8000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x42004070
[Pass] UNCACHEABLE(p) => 0x42004070
       chunk = GetNextMemoryChunk(suite, chunk) => 0xa7a738
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3724000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x5a5d00e4
[Pass] UNCACHEABLE(p) => 0x5a5d00e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xa7a7b0
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x4000000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x565d00e4
[Pass] UNCACHEABLE(p) => 0x565d00e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0x0
[Pass] total => 0x4000000
       shoot_free_suite(suite); suite = 0; chunk = 0;
[Pass] suite = shoot_malloc_suite(0) => 0xa89580
[Pass] suite->signature => 'MemSuite'
[Pass] suite->num_chunks => 0x7
[Pass] suite->size => 0xb700000
[Pass] chunk = GetFirstChunkFromSuite(suite) => 0xa895a4
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x1cf8000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x42004070
[Pass] UNCACHEABLE(p) => 0x42004070
       chunk = GetNextMemoryChunk(suite, chunk) => 0xa8961c
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x3724000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x5a5d00e4
[Pass] UNCACHEABLE(p) => 0x5a5d00e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xa89694
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x5150000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x565d00e4
[Pass] UNCACHEABLE(p) => 0x565d00e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xa8970c
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x6b7c000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x525d00e4
[Pass] UNCACHEABLE(p) => 0x525d00e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xa89784
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x85a8000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4e5d00e4
[Pass] UNCACHEABLE(p) => 0x4e5d00e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xa897fc
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0x9fd4000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x4a5d00e4
[Pass] UNCACHEABLE(p) => 0x4a5d00e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0xa89874
[Pass] chunk->signature => 'MemChunk'
[Pass] total += chunk->size => 0xb700000
[Pass] p = GetMemoryAddressOfMemoryChunk(chunk) => 0x465d00e4
[Pass] UNCACHEABLE(p) => 0x465d00e4
       chunk = GetNextMemoryChunk(suite, chunk) => 0x0
[Pass] total => 0xb700000
       shoot_free_suite(suite); suite = 0; chunk = 0;
       EngDrvOut(LCD_Palette[0], 0x1234)
[Pass] shamem_read(LCD_Palette[0]) => 0x1234
       call("TurnOnDisplay")
[Pass] DISPLAY_IS_ON => 0x1
       call("TurnOffDisplay")
[Pass] DISPLAY_IS_ON => 0x0
       call("TurnOnDisplay")
[Pass] DISPLAY_IS_ON => 0x1
       SetGUIRequestMode(1); msleep(1000);
[Pass] CURRENT_DIALOG_MAYBE => 0x1
       SetGUIRequestMode(2); msleep(1000);
[Pass] CURRENT_DIALOG_MAYBE => 0x2
       SetGUIRequestMode(0); msleep(1000);
[Pass] CURRENT_DIALOG_MAYBE => 0x0
[Pass] display_idle() => 0x1
       GUI_Control(BGMT_PLAY, 0, 0, 0); msleep(500);
[Pass] PLAY_MODE => 0x1
[Pass] MENU_MODE => 0x0
       GUI_Control(BGMT_MENU, 0, 0, 0); msleep(500);
[Pass] MENU_MODE => 0x1
[Pass] PLAY_MODE => 0x0
[Pass] MEM(dialog->type) => 0x414944
       GUI_Control(BGMT_MENU, 0, 0, 0); msleep(500);
[Pass] MENU_MODE => 0x0
[Pass] PLAY_MODE => 0x0
       task_create("test", 0x1c, 0x1000, test_task, 0) => 0x818400b4
[Pass] test_task_created => 0x1
[Pass] get_task_name_from_id(get_current_task()) => 'run_test'
[Pass] mq = mq ? mq : (void*)msg_queue_create("test", 5) => 0x81940086
[Pass] msg_queue_post(mq, 0x1234567) => 0x0
[Pass] msg_queue_receive(mq, (struct event **) &m, 500) => 0x0
[Pass] m => 0x1234567
[Pass] msg_queue_receive(mq, (struct event **) &m, 500) => 0x9
[Pass] sem = sem ? sem : create_named_semaphore("test", 1) => 0x81e6015a
[Pass] take_semaphore(sem, 500) => 0x0
[Pass] take_semaphore(sem, 500) => 0x9
[Pass] give_semaphore(sem) => 0x0
[Pass] take_semaphore(sem, 500) => 0x0
[Pass] give_semaphore(sem) => 0x0
[Pass] rlock = rlock ? rlock : CreateRecursiveLock(0) => 0x82360098
[Pass] AcquireRecursiveLock(rlock, 500) => 0x0
[Pass] AcquireRecursiveLock(rlock, 500) => 0x0
[Pass] ReleaseRecursiveLock(rlock) => 0x0
[Pass] ReleaseRecursiveLock(rlock) => 0x0
[Pass] ReleaseRecursiveLock(rlock) => 0xf
[Pass] f = FIO_CreateFileEx("test.dat") => 0x4
[Pass] FIO_WriteFile(f, (void*)ROMBASEADDR, 0x10000) => 0x10000
[Pass] FIO_WriteFile(f, (void*)ROMBASEADDR, 0x10000) => 0x10000
       FIO_CloseFile(f)
[Pass] FIO_GetFileSize("test.dat", &size) => 0x0
[Pass] size => 0x20000
[Pass] p = alloc_dma_memory(0x20000) => 0x409648dc
[Pass] f = FIO_Open("test.dat", O_RDONLY | O_SYNC) => 0x4
[Pass] FIO_ReadFile(f, p, 0x20000) => 0x20000
       FIO_CloseFile(f)
       free_dma_memory(p)
[Pass] count => 0x3a98
[Pass] FIO_RemoveFile("test.dat") => 0x0
       SW1(1,100)
[Pass] HALFSHUTTER_PRESSED => 0x1
       SW1(0,100)
[Pass] HALFSHUTTER_PRESSED => 0x0


lots of address but doesn't look like any LV buffer address etc. .... unless someone smarter can help .

(https://c1.staticflickr.com/3/2935/33390807224_e20c28e94c.jpg) (https://flic.kr/p/SSCBEQ)
Screenshot EDMAC (https://flic.kr/p/SSCBEQ) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
Screenshot of the edmac  and I notice that edmac#5 is now active (green) (3570x1267) that the buffer size I used with very limited success to get 3584x1068 in post #27
When I did update to this branch I notice it was on the older ml core even thou I cloned a fresh copy of the source.
on the newer ml source code that channel is not active (yellow) so I have to wonder why ? could this be a issue , have to investigate further
# Magic Lantern v2.3.NEXT.2017Apr23.5D2212 (09dcd0475cb3+ (new-lv-buffer-detection))
# Built on 2017-04-23 05:47:00 UTC by ml@ml-pc
# Configuration saved on 2017/04/23 18:44:11
beta.warn = 23


So I'm not sure it these stubs are still useful .

Next I complied the "allocate-raw-lv-buffer" and that one seem to use the newer ml core source .
# Magic Lantern Nightly.2017Apr24.5D2212 (5d168ad4e8c9+ (allocate-raw-lv-buffer))
# Built on 2017-04-24 01:16:11 UTC by ml@ml-pc
# Configuration saved on 2017/04/23 19:47:45
beta.warn = 23


(https://c1.staticflickr.com/3/2885/34232210575_4191fe7346.jpg) (https://flic.kr/p/U9Z2za)
Screenshot of Lv with MPU_Dump running (https://flic.kr/p/U9Z2za) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

So I found this very interesting as I pushed the 5x Zoom button I notice that the Live view resolution is not what I thought  & have read to be  ???
In 1:1 a surprising 2040x1268  and in 3x crop (5x Zoom) 2312x1128 , Really !
I was under the impression that full Liveview was only 1880x1250  and 3x crop was 2152x1078.
Wondering out loud , could be a result of the edmac channel #5 being active (maybe  larger buffer size)
More mysteries to solve  :D
I have a MPU-EEP , Ram & Rom dumps not sure how to use them yet , there may be nothing useful either .

On a lighter note I never notice the False Color Overlays , there where on by default in the  "Magic Lantern v2.3.NEXT" version
If it wasn't for that i would have never notice them  8)

(https://c1.staticflickr.com/5/4156/33390807244_93b069408b.jpg) (https://flic.kr/p/SSCBFb)
False Color Overlays  (https://flic.kr/p/SSCBFb) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on April 24, 2017, 09:06:56 AM
@reddeercity I've been watch your God like devotion to further improving 5D2 Thanks a million. This looks promising so when can we mere mortals get to test it?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on April 24, 2017, 09:42:14 AM
[emoji2]
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Greg on April 24, 2017, 02:15:04 PM
Quote from: reddeercity on April 24, 2017, 05:15:56 AM
In 1:1 a surprising 2040x1268  and in 3x crop (5x Zoom) 2312x1128 , Really !
I was under the impression that full Liveview was only 1880x1250  and 3x crop was 2152x1078.

This is probably a resolution with black borders.

raw.c
#ifdef CONFIG_5D2
skip_top        = zoom ?   52 : 18;
skip_left       = 160;
#endif
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: PressureFM on April 24, 2017, 06:04:24 PM
How can people with 5D2 help?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 25, 2017, 05:09:21 AM
@PressureFM  thanks , the more the merrier .
Currently the issue is the customs Raw LV buffer , I've been trying to spy the memory to find the LV raw buffer
and looking thought my 5d2 decompiled rom dump for addresses , a1ex gave me some clues at the links
http://www.magiclantern.fm/forum/index.php?topic=19336.msg183374#msg183374
http://www.magiclantern.fm/forum/index.php?topic=19336.msg183382#msg183382

I found the registers needed to modify frame size with adtg_gui but the problem is the LV buffer
I did have some success with my  RAW_LV_BUFFER_ALLOC_SIZE (2040*1267) as per post #27
The reason I used that was it was the largest buffer on the edmac & it didn't change when switch to Photo Liveveiw see post #49
Edmac#5 buffer size is what I used , now I think there's I little dilemma here , when I updated to "new-lv-buffer-detections" branch and complied
I ended up with the older core "Magic Lantern v2.3.NEXT" , reason I mention this edmac#5 is Green (active) where in the nightly builds compiles
on newer code edmac Channel #5 is Yellow , which I think is a error  or all this could mean nothing , just thought I would mention it .
So Yea LV raw buffer needs to be solved & I thing that will also help to get 10 & 12bit problem in 1:1 solved also

Link to my bitbucket downloads for those memspy builds I talked about .
magiclantern-Nightly.memspy.lv.raw.buffer.extended.resoultion.2017Apr24.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-Nightly.memspy.lv.raw.buffer.extended.resoultion.2017Apr24.5D2212.zip)
magiclantern-v2.3.NEXT.Memspy.2017Apr23.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-v2.3.NEXT.Memspy.2017Apr23.5D2212.zip)
You may just want to compile your own copy of adtg_gui from the ISO-research branch or you can download a copy from my bitbucket download.
So any suggestion are very welcome  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 25, 2017, 07:02:24 AM

ff036750 lv_rshd_raw
ff0365f4 lv_raw_dump
ff036704 lv_save_raw
ff036724 lv_continous_frame
ff036760 lv_rec
ff03676c lv_output_device
ff036604 lv_continous_frame_save
ff036784 lv_hd

some lv raw stuff from my decompiled rom dump , I think the ff036750 lv_rshd_raw is what I'm after
FF036520: e59f1224  ldr r1, [pc, #548] ; 0xff03674c: pointer to 0xff8347f8 ⬁
FF036524: e28f0f89  add r0, pc, #548 ; *'lv_rshd_raw'


dropbox link to FF036524.htm (https://www.dropbox.com/s/8xvupbnn6oa5vb7/0xFF036000%20___%200xFF037FFF_htm%23_FF036524.htm?dl=0) where I found the raw liveview from the decompiled rom
Title: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on April 25, 2017, 10:40:27 AM
This is all nothing short of spectacular and wonderful progress so far. Just confirmed w my co-worker to bring his 5D2 to work tmw for me to test out the memspy build.

Thanks @reddeercity just for being you, as always!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on April 26, 2017, 01:52:41 AM
Quote from: reddeercity on April 25, 2017, 05:09:21 AM
@PressureFM  thanks , the more the merrier .
Currently the issue is the customs Raw LV buffer , I've been trying to spy the memory to find the LV raw buffer
and looking thought my 5d2 decompiled rom dump for addresses , a1ex gave me some clues at the links
http://www.magiclantern.fm/forum/index.php?topic=19336.msg183374#msg183374
http://www.magiclantern.fm/forum/index.php?topic=19336.msg183382#msg183382

I found the registers needed to modify frame size with adtg_gui but the problem is the LV buffer
I did have some success with my  RAW_LV_BUFFER_ALLOC_SIZE (2040*1267) as per post #27
The reason I used that was it was the largest buffer on the edmac & it didn't change when switch to Photo Liveveiw see post #49
Edmac#5 buffer size is what I used , now I think there's I little dilemma here , when I updated to "new-lv-buffer-detections" branch and complied
I ended up with the older core "Magic Lantern v2.3.NEXT" , reason I mention this edmac#5 is Green (active) where in the nightly builds compiles
on newer code edmac Channel #5 is Yellow , which I think is a error  or all this could mean nothing , just thought I would mention it .
So Yea LV raw buffer needs to be solved & I thing that will also help to get 10 & 12bit problem in 1:1 solved also

Link to my bitbucket downloads for those memspy builds I talked about .
magiclantern-Nightly.memspy.lv.raw.buffer.extended.resoultion.2017Apr24.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-Nightly.memspy.lv.raw.buffer.extended.resoultion.2017Apr24.5D2212.zip)
magiclantern-v2.3.NEXT.Memspy.2017Apr23.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-v2.3.NEXT.Memspy.2017Apr23.5D2212.zip)
You may just want to compile your own copy of adtg_gui from the ISO-research branch or you can download a copy from my bitbucket download.
So any suggestion are very welcome  :D

Forgive me for my ignorance but what do I do with the downloads to us this build?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 03, 2017, 08:41:09 AM
From the live view log , from DeafEyeJedi
Some very interesting information about liveveiw specially the "tomSetRawJpgMode (Type = 0x4)"
Thou I don't understand it all give more info to work from that I don't have before
thanks to a1ex  ;)

line 1760 --- 51BF1>      TOMgr:ffa21790:41:05:  tomSetRawJpgMode (Type = 0x4)
line 2163 --- 6A4E4>    Startup:ff8e5f88:98:03: GetImageTrimming X(2976)=0(2414, 2416), Y(1928)=0(1552, 1548)


line 2293 --- 72089> LiveViewMg:ff8e6108:98:02: Origin2 X=911, Y=518 W:1122 C:2976

line 2294 --- 721BA> LiveViewMg:ff8de8e0:9b:03: PROP_MOVIE_PARAM 0 1

line 2295 --- 722B0> LiveViewMg:ff8dd52c:9b:16: PROP_LIVE_VIEW_VIEWTYPE_SELECT 0->2

line 2296 --- 722F3> LiveViewMg:ff8dcdb0:9b:03: ViewType:2

line 4927 --- 38AAA>    Startup:ff867658:82:03: InitializeDisplayDeviceController (PUB)

start at line 4975

3BA12> GuiMainTas:ff892834:85:03: AllocateVramCBR pAddress=43f80000
3BA90> GuiMainTas:ff86503c:00:03: [BmpDDev] CreatePhysicalVram (PUB)
3C401> GuiMainTas:ff864db0:00:03: [BmpDDev] DisplayPhysicalScreen (PUB) 0x43F00008
3C49C> GuiMainTas:ff9a77d0:00:01: [CLKSAVER] ��ClockSave In��
3C4D9> GuiMainTas:ff864be8:82:02:  SelectParameterToBmp (PRI)
3C502> GuiMainTas:ff864a40:82:02:  SetParameterToBitmapDisplayDevice (PRI)
3C526> GuiMainTas:ff864d3c:82:02: EnableBitmapVBufferForPlayBackAndWaiting (PUB)



3D012> GuiMainTas:ff83f068:36:05:  RegisterAdapterStatusCallback
3DB69> GuiMainTas:ffa84814:04:02: Partial memcpy start (x,y,w,h)=( 0, 0, 960, 540 )


4039D> GuiMainTas:ffa84844:04:02: Partial memcpy end
403D0> GuiMainTas:ff9c17ac:83:03: GuiStartGraphics (PUB)
404CE> GuiMainTas:ff82ebd4:80:03: SRM_AllocateMemoryResourceForImgVram 131 3
40582>     RscMgr:ff8b6ea8:80:03: srmAllocateImgVram 131 3
405BD>     RscMgr:ff8b6ec8:80:03: ImgVram Before -1 -1 -1
405F8>     RscMgr:ff8b6fa4:80:03: ImgVram After 131 131 131
4067A> GuiMainTas:ffa41e6c:18:03: (PUB) StartImagePlayer(mode=0) 1203
406BE> GuiMainTas:ffa42000:18:01: mode:0 dec:0x0 rot:0x0 2nd:0x0
406E6> GuiMainTas:ffa4532c:18:03: (PUB) SetVisibleImageVramOffset 4064
40712> GuiMainTas:ffa42304:18:03: (PUB) SetCopyVramMode(mode=0) 1411
4073B> GuiMainTas:ff9c15b0:83:03: GuiGraphicsNormalMode (PUB)


40760> GuiMainTas:ffa4522c:18:03: (PUB) SetImageWorkMemory 1:0x40d00000, 2:0x41700000
40799> GuiMainTas:ffa45274:18:03: (PUB) SetImageWorkMaxPixel W:2200,H:1872,0x5e4340
407CD> GuiMainTas:ffa44db8:18:03: (PUB) SetEffectiveSizeOfJpeg(w:7488 h:4992) 2995
407F8> GuiMainTas:ffa44dd0:18:01: SetEffectiveSizeOfJpeg w:7488 h:4992
40824> GuiMainTas:ffa423f8:18:03: (PUB) SetYuvColorParameter 1447
4089A> GuiMainTas:ff9dc868:83:03: IDLEHandler INITIALIZE_CONTROLLER
408CD> GuiMainTas:ff9dd028:83:03: IDLEHandler GOT_TOP_OF_CONTROL
408F1> GuiMainTas:ff9c3ffc:83:03: GuiClearImage
40914> GuiMainTas:ffa420fc:18:03: (PUB) SetVramInformation(w:1920 h:1080)
4094C> GuiMainTas:ffa421cc:18:03: 1920x1080 BaseVram:1920, VramYuv:1
40977> GuiMainTas:ffa421cc:18:03: 1920x1080 BaseVram:1920, VramYuv:1
409A0> GuiMainTas:ffa42210:18:01: fLcd 0


409C8> GuiMainTas:ffa423b8:18:03: SetImageVramParameter x:0 y:0 w:1920 h:1080/* Aspect:0*/
409FE> GuiMainTas:ffa44c3c:18:03: (PUB) SyncroAllClearImagePlayWorkVramWithoutEngine 2953
40A27> GuiMainTas:ffa4161c:18:03: GetVramNumber 619
40A4B> GuiMainTas:ffa4161c:18:03: GetVramNumber 619
416A5> **INT-0Ah*:00095c38:00:00: >>> INT-Ah Timer ff81027c(0)
416E5> **INT-0Ah*:0000057c:00:00: <<< INT-Ah done
43DB3> **INT-0Ah*:00095c38:00:00: >>> INT-Ah Timer ff81027c(0)
43DDC> **INT-0Ah*:0000057c:00:00: <<< INT-Ah done
45389> GuiMainTas:ffa420fc:18:03: (PUB) SetVramInformation(w:720 h:480)
453B5> GuiMainTas:ffa421cc:18:03: 720x480 BaseVram:720, VramYuv:1
453DD> GuiMainTas:ffa421cc:18:03: 720x480 BaseVram:720, VramYuv:1
45406> GuiMainTas:ffa42210:18:01: fLcd 0


4542B> GuiMainTas:ffa423b8:18:03: SetImageVramParameter x:0 y:0 w:720 h:480/* Aspect:0*/
45459> GuiMainTas:ffa4161c:18:03: GetVramNumber 619
45483> GuiMainTas:ff863840:82:02: GetVramSize (PUB)
454BD> GuiMainTas:ff86413c:82:03: EnableImagePhysicalScreenParameter
4556B> GuiMainTas:ff9a77d0:00:01: [CLKSAVER] ��ClockSave In��
455AC> GuiMainTas:ff8638c4:82:01: ImgDDev SelectParameter DispType=0
455D1> GuiMainTas:ff863840:82:02: GetVramSize (PUB)
455FC> GuiMainTas:ff8640d4:82:02: EnableImageVBufferForPlayBackAndWait (PUB)
45688> GuiMainTas:ffa4161c:18:03: GetVramNumber 619
456BD> GuiMainTas:ffa4161c:18:03: GetVramNumber 619
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ItsMeLenny on May 14, 2017, 04:17:36 AM
Can somebody explain in laymans terms what needs to happen to get this to 550D.
And also what exactly is this. Is this for the different bit depths, or for the compressed raw, or for 1:1 pixels.
I'm never entirely sure.
In addition to that, is it compressed raw being referred to as "lossless".
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 15, 2017, 04:48:33 AM
Quote from: ItsMeLenny on May 14, 2017, 04:17:36 AM
Can somebody explain in laymans terms what needs to happen to get this to 550D.

Quote from: a1ex on April 18, 2017, 11:55:34 PM
The custom buffer is required, as ML redirects it only while recording.
In the crop_rec_4k branch I've used a SRM buffer (which can accommodate a full-res 14-bit picture).
This needs to be passed to SetEDmac as the first argument (instead of Canon's default buffer).
The buffer redirection without CONFIG_EDMAC_RAW_SLURP is fragile:
it relies on lucky timing. It's best refactored somehow, but cache patching is also ugly...
Quote from: ItsMeLenny on May 14, 2017, 04:17:36 AM
.... is it compressed raw being referred to as "lossless".
Yes
------------------------------------------------------------------------------------------------------------------------

I keep running in to walls .
(http://orig15.deviantart.net/8f4f/f/2010/294/a/a/man_vs_wall___s_by_0vonix0-d318uuv.gif)
What needed now is someone with advanced knowledge of digic4 Liveview buffers , that would be a main developer like a1ex
this is getting above my knowledge level right now.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on May 15, 2017, 07:13:53 AM
Ha ha. That's how I feel too. Been trying this on a couple of Digic V cameras but keep running into walls too.

Maybe put it aside and try something else for a while.


Sent from my iPad using Tapatalk
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Teamsleepkid on May 15, 2017, 10:30:59 AM
I want it so bad...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on May 15, 2017, 11:36:13 AM
For 550D, the interesting parts are:
- for video: the lossless compression
- for timelapse: the full-res LiveView

The first one was covered in detail in "the two in two out discussion", with partial success on 500D (only working outside LiveView, and output not decoded yet). Until the first issue (which wasn't present on 5D3) is solved, porting it probably doesn't make much sense, as the two models are very similar, and you'll likely run into the same problem. Newer models (60D and newer) are probably much easier to port, as their LiveView implementation is a lot closer to 5D3's.

The second one (full-res LiveView) requires adtg_gui (iso-research branch), but to explain in detail how it works, I need to try it myself on an old-gen camera, to have an idea of similarities and differences. The most promising attempts, at the time of writing, are on 700D (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183286#msg183286) and 5D2 (this thread).

However, increasing the resolution beyond what you get in regular LiveView requires allocating a new buffer. On recent cameras, it's easy (just enable some definitions and tell how much you need - see how it's done on 1100D and 5D3). On old generation cameras, it's hard and probably requires patching Canon code.

Other than that, it's just a matter of patching some registers (see the crop_rec source code for the various presets).

Quote from: reddeercity on May 15, 2017, 04:48:33 AM
What needed now is someone with advanced knowledge of digic4 Liveview buffers

Unfortunately, my level of understanding is not there yet, but I've made some small progress emulating the LiveView in QEMU. On 500D, it appears to work for a few frames before locking up, so it's not yet published :D

However, both the 550D and 5D2 (and other models) now have the GUI emulation working (you can navigate Canon menu in the emulator), so the first step for LiveView is to get a set of MPU spells that covers... entering LiveView. This is easy - use the dm-spy-experiments branch (or the startup-log builds (https://builds.magiclantern.fm/jenkins/view/Experiments/job/startup-log/)), enter LiveView while the LED is blinking (that is, while debug messages are recorded) and then run the extract_init_spells.py (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/eos/mpu_spells/) script on the log file. This alone is enough for some LiveView bitmap overlays on 500D (Greg's screenshot (http://www.magiclantern.fm/forum/index.php?topic=2864.msg179867#msg179867) - didn't try on other models). Progressing from there is much harder. I can share the current patch if there is interest, but it's "too much of a hack" at this stage. I recommend starting with easier things instead.

Currently, adtg_gui and the QEMU LV trick are low-hanging fruits. Creating debugmsg.gdb files for camera models that don't have one, or adding more entries to an existing one, is also very easy (as they mostly contain stubs (http://www.magiclantern.fm/forum/index.php?topic=12177.0)).

Porting the FRSP emulation to QEMU ranges from extremely easy to hard, depending on model. For some models, you only need to supply a reference full-res DNG and to add the camera model to the test suite - in other words, confirming it's working. Others may require some minor fiddling, others may require a deeper level of understanding. To get started, look up FRSP or FA_CaptureTestImage on the QEMU test suite (https://builds.magiclantern.fm/jenkins/job/QEMU-tests/) (logs, screenshots). At the time of writing, this is working on 60D, 1200D, and also 5D3 1.1.3 (where my test image has a hot pixel in the OB area, which is the reason for not passing the test). Models with GUI emulation are probably easier to adapt (just a guess, as I didn't try).

Playing with custom logging (http://www.magiclantern.fm/forum/index.php?topic=2388.msg113154#msg113154) on the dm-spy branch is also easy once you get some courage, and can be done on both a real camera or QEMU. Understanding the logs is harder. LiveView is *very* complex, so it makes sense understanding smaller bits first (such as logging task names as they are started, or following a button event).

Debugging the lossless compression routine in QEMU should also be accessible. You'll need to run both Canon's original routine and your modified one with "-d io" and compare the JPCORE and related I/O activity - see Greg's examples (http://www.magiclantern.fm/forum/index.php?topic=18443.msg184283#msg184283).

Emulating a CR2 picture taking (until getting a valid CR2 on the SD/CF card image) should be also doable, and a whole lot easier than emulating LiveView. Maybe I should try that first.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on May 15, 2017, 01:00:59 PM
Quote from: a1ex on May 15, 2017, 11:36:13 AM
use the dm-spy-experiments branch (or the startup-log builds (https://builds.magiclantern.fm/jenkins/view/Experiments/job/startup-log/)), enter LiveView while the LED is blinking (that is, while debug messages are recorded) and then run the extract_init_spells.py (https://bitbucket.org/hudson/magic-lantern/src/qemu/contrib/qemu/eos/mpu_spells/) script on the log file.
Does this still need to be done for 5D2?
just in case: https://www.dropbox.com/s/84hknim2jrhd5ww/copypasta.txt?dl=0 (https://www.dropbox.com/s/84hknim2jrhd5ww/copypasta.txt?dl=0)
The script needed some changes tho, this didn't work:

cam_dir = [d for d in os.listdir(ml_dir)
             if d.split(".")[0] == camera_model
             and os.path.isfile(os.path.join(ml_dir, d, "gui.h"))
          ][0]

just manually set it to "5D2.212", also default ML directory was wrong in my case.
^^ Maybe that helps anyone else doing it.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on May 15, 2017, 01:19:55 PM
Right now, none of the spells from the repository include LiveView activity (it was easier to understand how it works without it). I've only tried the LiveView trick on 500D (which I find easier to understand). So, if you want to play around, you can give it a try; however, once you get past the initial GUI (or maybe earlier), you *will* "run into walls". I'm currently in the middle of one such wall :P

An initial goal would be to get both the menu navigation and the LiveView trick in the same set of spells - that didn't work in my experiment, but it's probably easy to fix. Either perform both menu navigation, play mode switch and LiveView in the same logging session to re-create the spell set from scratch, or figure out which bits are specific to LiveView and add them to current spell set.

BTW, the script is meant to be run from the QEMU installation directory (outside the ML tree). That's probably the reason you've got the path error.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 16, 2017, 07:35:50 AM
Thanks a1ex , I was hoping for something like this  :D gives me more meat to chew on.
Quote from: a1ex on May 15, 2017, 11:36:13 AM
This alone is enough for some LiveView bitmap overlays on 500D (Greg's screenshot (http://www.magiclantern.fm/forum/index.php?topic=2864.msg179867#msg179867) - didn't try on other models).
Progressing from there is much harder. I can share the current patch if there is interest
I would be interested in at least looking at the code .
Great there's still a change this can be ported  8)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ItsMeLenny on June 12, 2017, 02:17:53 PM
Any progress on this? Seems to have stopped still for a month.
Has the way to crop record been discovered yet?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 19, 2017, 06:56:49 AM
Quote from: ItsMeLenny on June 12, 2017, 02:17:53 PM
Any progress on this? Seems to have stopped still for a month.
No.
Reason , there is no interest in any Digic IV (4) development The forum is being driven by Digic V (5) and up cameras , in particular 5D3.
If you think I'm kidding just look at all the posts over the last mouth . In fact even 10 or 12bit development is dead for Digic IV cam's
other then the experimental builds I have on my bitbucket download (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/)  , no one else really care and that a trouble shame ! :(
As I don't see anyone else participating here , I can no longer justify developing this thread .

Quote from: ItsMeLenny on June 12, 2017, 02:17:53 PM
Has the way to crop record been discovered yet?
It's very close (http://www.magiclantern.fm/forum/index.php?topic=19336.msg183098#msg183098) , the only issue is having a custom image buffer with the right address .
I already know which registers that need to modified but with out the correct buffer/memory address it stop there .

Even thou I would really like to see this for the 5d2 , I want this for all digic 4 cams ( Wishful thinking)
In the near future if I see activity here then I my continue further , until such time I'll just make improvement's for myself .
Sorry
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ItsMeLenny on June 19, 2017, 10:42:53 AM
I'm very keen for it for 550D. Hopefully to have something that can record 1080p 10bit.
Yeah the discussions on the forums do seem to be around 5d3, but I still think there's interest for Digic IV, they're just not very talkative.
And I think it's also partly due to it working on Digic 5, so people are using it. Where as it doesnt work on 4.
I can imagine Digic 4s becoming popular in discussion if there were things like 2K/4K 10bit and all that hoohar.

Let me know if I can do anything.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Ottoga on June 19, 2017, 02:06:48 PM
+1 here with a 7D. Can't code but can test.

I'm sure that there are plenty of lurkers waiting for something exciting to happen that they can assist with.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: loknar on June 19, 2017, 08:54:30 PM
@reddeercity
It's not like we wouldn't like new builds, but most of us are not programmers, so we do some test, and that's it. I for one would love 10-bit RAW working on 550D, but I have same problems as others, all frames are corrupted 1/3 of screen is shifted and frozen. Since this has been reported, i'm afraid that's all I can do.

(I was also hoping in mv1080 on EOS M, but all my questions in appropriate threads have been ignored, so i kinda gave up)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on June 20, 2017, 01:22:56 AM
Quote from: loknar on June 19, 2017, 08:54:30 PM
(I was also hoping in mv1080 on EOS M, but all my questions in appropriate threads have been ignored, so i kinda gave up)

What do you mean? EOS M can do mv1080 if you turn on H.264 proxy. It is rather buggy and experimental but hey.

http://www.magiclantern.fm/forum/index.php?topic=16608.msg179969#msg179969
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Tullen on June 20, 2017, 04:25:40 PM
I really want to get this going, especially for the 50D. As soon as I have fished my thesis I hope to be able to help. Even would like to start a Magic Lantern club in Stockholm. That would be in September though. Please dont give up. I am sure that more people would like to help. I am willing to put in time for this for sure.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: loknar on June 20, 2017, 08:59:34 PM
Quote from: dfort on June 20, 2017, 01:22:56 AM
What do you mean? EOS M can do mv1080 if you turn on H.264 proxy. It is rather buggy and experimental but hey.

http://www.magiclantern.fm/forum/index.php?topic=16608.msg179969#msg179969

I mean I wasn't able to replicate it, I wasn't able to get same nightly and with my version of ML although i've been able to capture whole frames, all were in pink and after correction of black and white levels it has been extremely noisy to the point of unusability. Same processing of test mlv of Koks the cat lead to excellent results, just my camera didn't deliver same input. When I asked about what build i should use, it stayed unanswered  ( http://www.magiclantern.fm/forum/index.php?topic=9741.msg183234#msg183234 ) (not complaining, just stating). I've got in bookmarks your bitbucket page, but you wrote that your version was really experimental, and also i'd have to update firmware to 2.03, and i thought since 2.02 is more mainstream, that development would go faster there.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 26, 2017, 06:08:01 AM
Ok --So I decided to forge ahead after urging from some forum members and help from a1ex.
So to get up to speed read the posts below to those members who are following the development , should help explain where I am in all this.
http://www.magiclantern.fm/forum/index.php?topic=19386.msg186209#msg186209
http://www.magiclantern.fm/forum/index.php?topic=19386.msg186213#msg186213
Basically I trying to get back to the 3584x1068 raw buffer I had working here (http://www.magiclantern.fm/forum/index.php?topic=19336.msg183098#msg183098)
Stay tuned  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ItsMeLenny on June 26, 2017, 02:12:00 PM
Good to hear! I think many people are ready to step up if you need a hand.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: nikki on June 26, 2017, 09:11:55 PM
let us know I have my 5d2 ready and charged
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Tullen on June 27, 2017, 01:08:43 PM
Quote from: reddeercity on June 26, 2017, 06:08:01 AM
Ok --So I decided to forge ahead after urging from some forum members and help from a1ex.
So to get up to speed read the posts below to those members who are following the development , should help explain where I am in all this.
http://www.magiclantern.fm/forum/index.php?topic=19386.msg186209#msg186209
http://www.magiclantern.fm/forum/index.php?topic=19386.msg186213#msg186213
Basically I trying to get back to the 3584x1068 raw buffer I had working here (http://www.magiclantern.fm/forum/index.php?topic=19336.msg183098#msg183098)
Stay tuned  :)

Great to hear. As I said, I will be helping as much as I can from September onward.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: scotophorus on June 28, 2017, 12:17:07 AM
Quote from: reddeercity on June 26, 2017, 06:08:01 AM
Ok --So I decided to forge ahead after urging from some forum members and help from a1ex.
So to get up to speed read the posts below to those members who are following the development , should help explain where I am in all this.
http://www.magiclantern.fm/forum/index.php?topic=19386.msg186209#msg186209
http://www.magiclantern.fm/forum/index.php?topic=19386.msg186213#msg186213
Basically I trying to get back to the 3584x1068 raw buffer I had working here (http://www.magiclantern.fm/forum/index.php?topic=19336.msg183098#msg183098)
Stay tuned  :)

This is great news. Big thanks to you, Alex, and all the other geniuses for pushing forward the development. I'm ready to test and help in anything you need.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 28, 2017, 06:41:46 AM
Quote from: Tullen on June 27, 2017, 01:08:43 PM
Great to hear. As I said, I will be helping as much as I can from September onward.
Sounds good , In the mean time I would suggest familiarize yourself with this thread , 10-12bit thread and of course the  iso-research (https://bitbucket.org/hudson/magic-lantern/src/bd8e89d9dccf2d1541bb449efd06bb450cdce153/?at=iso-research) source code. (That's where I started from)
You will need a developing environment -- I use a VM (VirtualBox) on Window7 Pro with a preconfigured magic lantern clone of Ubuntu (http://www.magiclantern.fm/forum/index.php?topic=18259.msg174934#msg174934) ready to compile ML . Download links for clone of Ubuntu  here (http://www.magiclantern.fm/forum/index.php?topic=7579.msg134989#msg134989)
You can run the VM under Mac or Windows just get the correct version for your OS , if you feel adventurous you can setup your own environment @dfort has make some tutorial here (http://www.magiclantern.fm/forum/index.php?topic=16012.msg155422#msg155422) and here (http://www.magiclantern.fm/forum/index.php?topic=15894.msg154435#msg154435)
You need to be very comfortable with compiling , merging , update source with "HG" etc. ..... I work with a VM because it's faster & easier to get up and running in less then 30 mins .

 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: whitelight on July 10, 2017, 12:53:18 AM
Hello, I just want to thank all the developers for the work done, I wish this will one day be ported to 7D, old camera but still doing an amazing job when used with ML.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: flofifull on July 10, 2017, 11:53:08 AM
Quote from: whitelight on July 10, 2017, 12:53:18 AM
Hello, I just want to thank all the developers for the work done, I wish this will one day be ported to 7D, old camera but still doing an amazing job when used with ML.

+1
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 17, 2017, 04:12:43 AM
5d2 users --could I please get a screenshot of the edmac in video mode like in this post here (http://www.magiclantern.fm/forum/index.php?topic=19336.msg183621#msg183621) I what to compare to my 5d2 , in fact it wouldn't hurt to see other digic iv cams also for reference , Thanks  :) 

Edit:
You can use the latest nightly , but for the 5d2 users could you do 2 screenshot , 1 with the latest nightly and the other with the adtg_gui build from my bitbucket download  here (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-adtg_gui.2017Apr04.5D2212.zip)  please
screen shoot feature in the debug menu , you have 10's to get to the right window.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Tullen on July 17, 2017, 01:31:52 PM
Quote from: reddeercity on June 28, 2017, 06:41:46 AM
Sounds good , In the mean time I would suggest familiarize yourself with this thread , 10-12bit thread and of course the  iso-research (https://bitbucket.org/hudson/magic-lantern/src/bd8e89d9dccf2d1541bb449efd06bb450cdce153/?at=iso-research) source code. (That's where I started from)
You will need a developing environment -- I use a VM (VirtualBox) on Window7 Pro with a preconfigured magic lantern clone of Ubuntu (http://www.magiclantern.fm/forum/index.php?topic=18259.msg174934#msg174934) ready to compile ML . Download links for clone of Ubuntu  here (http://www.magiclantern.fm/forum/index.php?topic=7579.msg134989#msg134989)
You can run the VM under Mac or Windows just get the correct version for your OS , if you feel adventurous you can setup your own environment @dfort has make some tutorial here (http://www.magiclantern.fm/forum/index.php?topic=16012.msg155422#msg155422) and here (http://www.magiclantern.fm/forum/index.php?topic=15894.msg154435#msg154435)
You need to be very comfortable with compiling , merging , update source with "HG" etc. ..... I work with a VM because it's faster & easier to get up and running in less then 30 mins .

Thanks for the recommendations, hopefully will make me helpful a bit faster.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: obiyan19 on July 18, 2017, 01:01:54 AM
Quote from: reddeercity on July 17, 2017, 04:12:43 AMfor the 5d2 users could you do 2 screenshot , 1 with the latest nightly and the other with the adtg_gui

just because i read this tpic with interest, done with my 5d2 an 24-105 mounted on (& CF sandisk extreme 32 GB 120 mb/s udma 7)

the first image done with nightly

(http://thumb.ibb.co/j7sgEa/screenshot_ML5_D2_nightly_18_07_2017.jpg) (http://ibb.co/j7sgEa)


and the second with adtg_gui.

(http://thumb.ibb.co/cwOSnv/screenshot_ML5_D2_adtg_gui.jpg) (http://ibb.co/cwOSnv)

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 18, 2017, 06:10:36 AM
Thanks @obiyan19 for the screenshots
nightly_18_07_2017
(http://image.ibb.co/ha68AF/screenshot_ML5_D2_nightly_18_07_2017.jpg) (http://imgbb.com/)

adtg_gui_2017_Apr04
(http://image.ibb.co/dQJicv/screenshot_ML5_D2_adtg_gui.jpg) (http://imgbb.com/)

Screenshots from @honza123 , thanks
Nightly_2017_Jul03
(http://image.ibb.co/dUxZHv/VRAM_ML_Nightly_2017_Jul03_5_D2212.jpg) (http://imgbb.com/)

adtg_gui_2017_Apr04
(http://image.ibb.co/iAaX4a/VRAM_adtg_gui_2017_Apr04_5_D2212.jpg) (http://imgbb.com/)

Good thanks , this lines up with my 5d2 --I wanted to see the buffers from different 5d2 cams and to make sure I didn't corrupt
my ram , all good now to processed to the next level .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ErwinH on July 18, 2017, 03:15:46 PM
Here is a screenshot of the 550D running Nightly.2017Jul16.550D109 unified.
 
(http://thumb.ibb.co/khneHv/550d.png) (http://ibb.co/khneHv)



Hope it helps.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 19, 2017, 07:37:30 AM
Thanks , yes all info is useful  ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on July 20, 2017, 08:28:57 PM
Hello, I joined just to post this encouragement for you to pursue this for 550D...
I'd LOVE to be able to use my 550D as a b-camera for filming 10 bit 24P raw 1080p!!!

Please be assured that your efforts will be appreciated and will keep many good cameras working for years to come!

I'm starting a new film this fall, and this would be excellent!

Please make it happen!
Thanks.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 28, 2017, 08:58:48 AM
I can now replicate the 3.5k resolution on my 5D2 but it's extremely fragile (I suspect the buffer I'm using & or how it's coded for liveview is little unstable)

(https://farm5.staticflickr.com/4310/36173282776_15a884200a.jpg) (https://flic.kr/p/X7vwHf)
5D2 3.5k eXperimental  (https://flic.kr/p/X7vwHf) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

(https://farm5.staticflickr.com/4295/35409866793_ec5be132c1.jpg) (https://flic.kr/p/VX3PJH)
LV-002_Liveview Raw Buffer Dump 5D2 3.5k eXperimental  (https://flic.kr/p/VX3PJH) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

(https://farm5.staticflickr.com/4305/35409898703_0754f41c19.jpg) (https://flic.kr/p/VX3ZdT)
Screen Shot 5D2 3.5k eXperimental with ML B/W Preview (https://flic.kr/p/VX3ZdT) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

(https://farm5.staticflickr.com/4330/36079669101_f28ce32216.jpg) (https://flic.kr/p/WYeJCn)
5D2 3.5k eXperimental Memory Patches (https://flic.kr/p/WYeJCn) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr


(https://farm5.staticflickr.com/4294/35409821883_a834515a3a.jpg) (https://flic.kr/p/VX3Aop)
5D2 3.5k eXperimental Memory Patches Error (https://flic.kr/p/VX3Aop) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

(https://farm5.staticflickr.com/4313/35379315564_21629218d4.jpg) (https://flic.kr/p/VUmeVo)
5D2 3.5k eXperimental Free Memory (https://flic.kr/p/VUmeVo) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

(https://farm5.staticflickr.com/4325/36214860345_3a08406c80.jpg) (https://flic.kr/p/XbbCgT)
5D2 3.5k eXperimental Edmac ScreenShot (https://flic.kr/p/XbbCgT) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

I uploaded my adtg.log file to my google drive for those who are following along , I still need to analyzes the information for more developing but feel free to download it .
https://drive.google.com/file/d/0BxNY4d4Aa7KuQUNtM2V4UnY1UEU/view?usp=sharing

Still lots to understand as everyone can see , I could not change resolution or aspect ratio seem to be lock to 3584x1068 .
That could be because of the preset from the 3K 5d3 1:1 in the adtg_gui.mo with ISO research branch --not sure thou.
So to recap I loaded adtg_gui.mo , crop_rec.mo , raw_rec.mo(not mlv_lite) on  change set bd8e89d9dccf+ (iso-research) that I complied locally on my VM
1st: load 1:1 3x crop_rec setting
2nd: with the adtg_gui.mo enabled I activated the Digic register then the 3k 5d3 1:1 preset
then adjust the image with the cmos 1 & 2 .

I modified  raw.c , edmac-memcpy.c
in raw.c I add this

#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (2040*1267)
#endif

also in raw.c  changed to 0xC0F04500 from  0xC0F04508

#if defined(CONFIG_5D2) || defined(CONFIG_50D)
#define RAW_LV_EDMAC 0xC0F04500
#endif

in edmac-memcpy.c change the r/w channel to 0xff & 0x0 from read 0x19 & write 0x03

#if defined(CONFIG_5D2) || defined(CONFIG_50D)
uint32_t edmac_read_chan = 0xff;
uint32_t edmac_write_chan = 0x0;


From the Liveview dump I can see that there is something there , seem to have the right size on the screen
Of course more reverse engineering needed , but there is some progress .  :)


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: nikfreak on July 28, 2017, 12:59:59 PM
May I just chime in and sum it up?


So frozen preview or not. i would say 1st goal would be to replicate what Greg/a1ex did AND be carefully with whatever you are trying out above this post  :o
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 29, 2017, 07:10:26 AM
Thanks for the links , gives me some ideas  ;)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: tonij on July 30, 2017, 02:48:55 AM
Quote from: whitelight on July 10, 2017, 12:53:18 AM
Hello, I just want to thank all the developers for the work done, I wish this will one day be ported to 7D, old camera but still doing an amazing job when used with ML.

+2

I don't post and there are probably many others like me but I find myself following along discussions like these.
All I see is page after page of green matrix symbols and sit here in awe at the ones who are able to stop bullets mid flight  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on August 23, 2017, 03:16:31 PM
@reddeercity

Just wanted to bump this because it looks like it was progressing nicely and suddenly stopped. Believe me I know how frustrating it is trying to get new features working. A while back I was trying to get lossless compression on the EOSM and 700D and had to put it aside. A while later ErwinH picked it up on the 700D and figured out the missing pieces. Following his lead I was able to get the EOSM working.

What made this possible was sharing the code on bitbucket. Checking up on your bitbucket account I see that you forked the ML repository a few times, the last activity was back in March. A lot has happened since then. If you could keep your development current in bitbucket maybe some of us could offer suggestions.

There are several good articles for working with bitbucket and Mercurial (hg) and if you have any questions there are ML forum members willing to help out.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on August 25, 2017, 10:00:54 PM
Quote from: dfort on August 23, 2017, 03:16:31 PMIf you could keep your development current in bitbucket maybe some of us could offer suggestions.
+1
Please do that, I'm wanting to test out what you're doing, but the verbal 'patches' are just too cumbersome to keep up with.

Also what exactly is a 'liveview buffer'? How is it different from just a piece of memory?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: tecgen on August 29, 2017, 02:44:50 PM
+1

Could you please share the source code? Maybe you could simply push all your changes as one commit. We can compare the sources against the master/trunk within BitBucket afterwards.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 03, 2017, 02:06:21 AM
Sorry guys I haven't had time to responded to the posts , had one of my main servers crashed (lost the raid & the backup  :( ) so I had to rebuild from scratch which took some time.
@dfort ,Ilia3101, tecgen
I really don't know how share my code on bitbucket , I work locally in my VM of Ubuntu which is very comfortable for me to develop in.
I tried once to set it up but could not figure it out , so I just kept working locally .
I did post my source code in my Bitbucket downloads , but all my changes are very simple I posted all my code changes and I thought I explained it in other posts.
I know this is not the way every one works , but for me it's the easiest way , so unless some one can tell me in very simple terms how you use/share my code on bitbucket correctly
with out  pointing to badly written tutorial I would gladly comply. This may slow down development , until then sorry I'll have to keep what I'm doing .

I'll try to explain what I have done and why . Here the Source Code (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magic-lantern_compressed_raw.zip) I been working from , it's has not been modified with my code .

I guess I should post a declaimer first -- I'm not a full time coder/hacker I know how thing work , I read code and understand perfectly just don't know the proper syntax when writing c code
I have to see example and do a lot of reading to make sure I'm doing the syntax correctly.


For this to work there needs to be a customs raw lv buffer , other then the canon which is for the default 1:1 (1880x1250)
So I followed a  tip (https://bitbucket.org/hudson/magic-lantern/src/11f405b62b312294d705e8e7bb8688b820bb9eea/src/raw.c?at=crop_rec_4k&fileviewer=file-view-default#raw.c-125)  from a1ex about the 1100D in the 4K branch and I use the largest buffer I could find in the Edmac debug menu
I use the 3x crop mode buffer because it was big , I use this (http://www.magiclantern.fm/forum/index.php?topic=19336.msg183023#msg183023) from Greg to help get the buffer but it unstable , could be because I did not disable the Canon default liveview buffer when I tested my code
In RAW.C  I added the new LV buffer
#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (2040*1267)
#endif

With out the buffer it will not work
Quote from: a1ex on April 18, 2017, 11:55:34 PM
The custom buffer is required, as ML redirects it only while recording.
In the crop_rec_4k branch I've used a SRM buffer (which can accommodate a full-res 14-bit picture).
This needs to be passed to SetEDmac as the first argument (instead of Canon's default buffer).
The buffer redirection without CONFIG_EDMAC_RAW_SLURP is fragile: it relies on lucky timing. It's best refactored somehow, but cache patching is also ugly...
Also in RAW.C I change the memory addressed for Liveveiw default was 0xC0f04508 to 0xC0f04500
Reason was it's the Photo Liveview buffer address and I thought logically it should have what I need to get to 3.5k and I was right to a certain degree .

#if defined(CONFIG_5D2) || defined(CONFIG_50D)
#define RAW_LV_EDMAC 0xC0F04500
#endif

In edmac-memcpy.c I also changed the default R/W channels , default Read 0x19 & Write 0x03
Reason , it seem to been the r/w for the larger buffer I'm using , but this my not be the case as I could can not save a dng in the liveview buffer dump just the YUV 422 dumps
as per post #90 http://www.magiclantern.fm/forum/index.php?topic=19336.msg187766#msg187766
#if defined(CONFIG_5D2) || defined(CONFIG_50D)
uint32_t edmac_read_chan = 0xff;
uint32_t edmac_write_chan = 0x0;


That's It for code modifications the rest is done with  adtg_gui.mo with ISO research branch , by enabling 5D3 1:1 3k preset and adjusting the cmos registers .
See post #90 for details , not sure if there a better way to implement the custom buffer "RAW_LV_BUFFER_ALLOC_SIZE " this need more work to understand
I would say this would be the first thing to focus on -- that about it for now if you or any one have question about what I have done just ask I do my best to answer .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on September 03, 2017, 11:50:10 PM
Quote from: reddeercity on September 03, 2017, 02:06:21 AMSorry guys I haven't had time to responded to the posts , had one of my main servers crashed (lost the raid & the backup  :( ) so I had to rebuild from scratch which took some time.
Oh no :(  Hope you didn't lose any important footage.

Quote from: reddeercity on September 03, 2017, 02:06:21 AMI know how thing work , I read code and understand perfectly just don't know the proper syntax when writing c code
Exact opposite here, know C, but have no idea how Magic Lantern or Canon works :D

Quote from: reddeercity on September 03, 2017, 02:06:21 AMI really don't know how share my code on bitbucket
Ok, well even in that case, you can still just upload a zip file with your modified code.
I think that is still better than describing what code you added to what files, as I don't know what part/line of the file you add stuff to, as that could matter in some cases, if you uploaded it I wouldn't have to worry about that as much.

But thanks, I will have another go and see if the same thing works on my camera tomorrow.

When you have the ~3500x1000px buffer showing in the menu, is the camera in a state where it can record a small test clip at that resolution?
Or would it just be noise and garbage? If so, what would the steps to getting a clean image be from that point onwards?

Also what do developers like a1ex think of reddeercity's work?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 05, 2017, 07:33:19 AM
@ Ilia3101 here you go Source Code with modification magic-lantern-adtg-gui-4k-test_source_code.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magic-lantern-adtg-gui-4k-test_source_code.zip)
In  edmac-memcpy.c Line 16 & 17 modified
In raw.c Line 78 , 116-118
in raw_rec.mo line 94 & 95 add resolution frame size 3840 & 4096 .
"Note" this is only working with raw_rec.mo not mlv_rec.mo
Quote from: Ilia3101 on September 03, 2017, 11:50:10 PM
Exact opposite here, know C, but have no idea how Magic Lantern or Canon works :D
That good to know , that will help a lot  :D

Quote from: Ilia3101 on September 03, 2017, 11:50:10 PM
When you have the ~3500x1000px buffer showing in the menu, is the camera in a state where it can record a small test clip at that resolution?
Or would it just be noise and garbage? If so, what would the steps to getting a clean image be from that point onwards?
Can't save a dng yet , tried a image dump I can save the liveview422 see  post #90 (http://www.magiclantern.fm/forum/index.php?topic=19336.msg187766#msg187766)  here is the original LV dump 422 image  LV-002.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-002.422) in Black & White ML perview  I guess I should correct my self it did save a DNG file but it was "0" bytes and there was a HD Liveview dump image HD-002.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/HD-002.422) which was normal looking .
I think to clean it up it could be as easy as fps A & B timing  or even the  Cmos  register , once there a clean LV image with ML B/W preview I think it may save a dng dump , there a raw record error when I try to save .raw file (Yes I'm still using raw 1st version not mlv lite , it's very basic just record raw once I have something stable I'll move to MLV Lite)

Quote from: Ilia3101 on September 03, 2017, 11:50:10 PM
Also what do developers like a1ex think of reddeercity's work?
http://www.magiclantern.fm/forum/index.php?topic=19336.msg187776#msg187776
http://www.magiclantern.fm/forum/index.php?topic=19336.msg184661#msg184661
http://www.magiclantern.fm/forum/index.php?topic=19336.msg183023#msg183023
http://www.magiclantern.fm/forum/index.php?topic=19336.msg183200#msg183200
http://www.magiclantern.fm/forum/index.php?topic=19336.msg183374#msg183374
http://www.magiclantern.fm/forum/index.php?topic=19336.msg183461#msg183461
I guess I could go on & on , if you are asking is this a realistic development or even possible  , my belief is it is possible.
Or else I don't think I would have got so many tips from a1ex and nikfreak and other developer .

@ Ilia3101 I will also PM you with the test 3.5k build I was using to do my test  that I posted in #90 , I don't what to post it publicly so it get out in the wild.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on September 05, 2017, 05:12:34 PM
Quote from: reddeercity on September 05, 2017, 07:33:19 AM@ Ilia3101 here you go Source Code with modification magic-lantern-adtg-gui-4k-test_source_code.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magic-lantern-adtg-gui-4k-test_source_code.zip)
Thanks a lot, I'll see if I can come up with any useful changes.

Quote from: reddeercity on September 05, 2017, 07:33:19 AM
Can't save a dng yet , tried a image dump I can save the liveview422 see  post #90 (http://www.magiclantern.fm/forum/index.php?topic=19336.msg187766#msg187766)  here is the original LV dump 422 image  LV-002.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-002.422) in Black & White ML perview  I guess I should correct my self it did save a DNG file but it was "0" bytes and there was a HD Liveview dump image HD-002.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/HD-002.422) which was normal looking .
I think to clean it up it could be as easy as fps A & B timing  or even the  Cmos  register , once there a clean LV image with ML B/W preview I think it may save a dng dump , there a raw record error when I try to save .raw file (Yes I'm still using raw 1st version not mlv lite , it's very basic just record raw once I have something stable I'll move to MLV Lite)
I have an idea why the preview is all noise: The 3K preset adjusts the preview parameters as well as raw parameters, so image framing is correct on 5D3 at the 3K res(is this true?), so maybe it ruins the preview on 5D2.
I'll try and find where the presets are in the code and comment out the changes in it that are irrelevant, maybe the image will be clear then.

Quote from: reddeercity on September 05, 2017, 07:33:19 AM@ Ilia3101 I will also PM you with the test 3.5k build I was using to do my test  that I posted in #90 , I don't what to post it publicly so it get out in the wild.
Thx
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 06, 2017, 07:53:04 AM
A small correction I did not use the 3x crop buffer in raw.c #define RAW_LV_BUFFER_ALLOC_SIZE (2040*1267) it's the 1:1 buffer , just look at post#90 in the Edmac screen shot on EDM#5 , address "43e28a4" 3570x1267 (3570/14*8 = 2040) . You could try the 3x crop buffer which is 4046 x 1127 so (4046/14*8 = 2312)
QuoteI have an idea why the preview is all noise: The 3K preset adjusts the preview parameters as well as raw parameters, so image framing is correct on 5D3 at the 3K res(is this true?), so maybe it ruins the preview on 5D2.
I'll try and find where the presets are in the code and comment out the changes in it that are irrelevant, maybe the image will be clear then.
Look at this for help adtg_gui.c-1127 (https://bitbucket.org/hudson/magic-lantern/src/bd8e89d9dccf2d1541bb449efd06bb450cdce153/modules/adtg_gui/adtg_gui.c?at=iso-research&fileviewer=file-view-default#adtg_gui.c-1127) and also this adtg_gui.c-1343 (https://bitbucket.org/hudson/magic-lantern/src/bd8e89d9dccf2d1541bb449efd06bb450cdce153/modules/adtg_gui/adtg_gui.c?at=iso-research&fileviewer=file-view-default#adtg_gui.c-1343)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 07, 2017, 05:13:57 AM
Found some helpful information for "CONFIG_EDMAC_RAW_SLURP"  treatment on 5D2 , as many times I read over this code I never see this  :o
I think this is a better way then "RAW_LV_BUFFER_ALLOC_SIZE" but not sure if this works on digic 4 cam's  so far it works with digic5 ,
I'll give it try , this is the link to the code starts at line 407 allocate-raw-lv-buffer#Lsrc/raw.cT407 (https://bitbucket.org/hudson/magic-lantern/commits/1a2a91eb31a38ad0a3b118f14b64229e8823ecc1?at=allocate-raw-lv-buffer#Lsrc/raw.cT407)

408  * How to find buffer dimensions for CONFIG_EDMAC_RAW_SLURP:
409  * - Go to LV and use lv_save_raw
410  * - Check the RAW_LV_EDMAC debug info
411  *  - Suppose it reports W: 0xA3A H: 0x3C7 (taken from 1100D in LV mode)
412  * - EDMAC W is the number of bytes per line
413  *   - The W resolutions is computed as: W * 8 / 14 (raw buffer is 14 bits per pixel)
414  *   - Thus 0xA3A  8 / 14 -> 1496 pixels
415  * - EDMAC H is the number of "jumps"
416  *   - The H resolutions is H + 1 -> 0x3C8 -> 968 pixels


Edit: This will have to be add to "platform / 5D2.212 /  internals.h" /** this method bypasses Canon's lv_save_raw and slurps the raw data directly from connection #0 */
#define CONFIG_EDMAC_RAW_SLURP
not to sure about "connection #0" thou -- this maybe a wild goose chase
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on September 07, 2017, 09:45:04 AM
Haven't really managed to get anywhere ::)
I tried changing the presets, separating the 3k one out in to 2 parts, but didn't really help.

I suspect I might have the same exact 'soft brick' you complained about before, as I had tried out your changes a while ago and seen results, now I'm not seeing anything.

Every time I try and do anything with magic lantern I always slow down eventually and give up for a bit, then I try again and the cycle repeats :(

Interesting about the SLURP thing though.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on September 07, 2017, 09:53:11 AM
Quote from: reddeercity on September 07, 2017, 05:13:57 AM
not to sure about "connection #0" thou -- this maybe a wild goose chase

Connections are probably hardcoded (or configured in a way we don't currently understand) to various image processing modules. So far, all the D4 and D5 models get the LiveView RAW data from connection 0. Connection 6 and 7 are pass-through (whatever you transfer there using a Read EDMAC channel will be copied on the other side, to a Write EDMAC channel configured for the same connection).

See this diagram (from http://www.magiclantern.fm/forum/index.php?topic=18315.msg188630#msg188630 )

(http://a1ex.magiclantern.fm/bleeding-edge/capture/5D3-playback.gif)

A read EDMAC channel will read the data from RAM and send it to some image processing module.

A write EDMAC channel will get the data from some image processing module and will write it into RAM.

The data can be read via some input module (such as DSUNPACK, ADUNPACK, UNPACK24, or others - possibly unnamed), where you can configure the input bit depth. In this case, the input stream can be 10-bit, 12-bit, 14-bit or 16-bit, configured using DSUNPACK_MODE / ADUNPACK_MODE / UNPACK24_MODE / 0xC0F371FC (http://www.magiclantern.fm/forum/index.php?topic=18443.msg181620#msg181620) / etc. In this case, the image processing module that actually does the work (e.g. JPCORE) probably receives normalized data.

A similar process happens on the output side, where a PACK module is used (PACK16, PACK32). Remember the PACK32_ISEL and PACK32_MODE (http://www.magiclantern.fm/forum/index.php?topic=5601.msg174177#msg174177):
- PACK32_ISEL probably means "wire the input of PACK32 module to whatever other image processing module that outputs Bayer data, in various places in the pipeline (http://www.magiclantern.fm/forum/index.php?topic=18393.0)";
- PACK32_MODE configures the output bit depth of whatever image data arrives to the PACK32 module (10/12/14/16).

Currently, the uncompressed bit depth selection is done in raw_lv_request_bpp (raw.c, crop_rec_4k branch).

Experiments on the above can be made on existing code that's known to work (raw_twk for digic 4/5, EekoAddRawPath for digic 5), or on FA_MaxSelectTestImage / FA_SubtractTestImage (low-hanging fruit for understanding the image processing modules).

One interesting note from the crop_rec_4k thread, where I've implemented the 10/12-bit lossless compression by darkening the input raw data (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183106#msg183106) (so the input and output would still be 14-bit, but there will be fewer levels actually used by the image - as many as a 12-bit or a 10-bit image). For lossless compression, the entropy of a 10/12-bit stream is similar (maybe identical?) to the entropy of a 14-bit stream with each value shifted by 4 or 2 bytes (integer division by 16 or by 4).

How does that work?

raw.c:raw_lv_request_digital_gain:
- lv_raw_gain is written to SHAD_GAIN_REGISTER
- RAW_TYPE_REGISTER is set to 0x12 (DEFCORRE) (http://www.magiclantern.fm/forum/index.php?topic=18393.0)
- this image happens to be scaled by digital ISO gain and is not affected by bad pixels.

When the digital gain is not set, RAW_TYPE_REGISTER is set to CCD (probably the first stage where the raw data gets in the digital domain).

For a better understanding, set RAW_TYPE_REGISTER to DEFCORRE (0x4 on digic 4) without overriding SHAD_GAIN, and notice what happens at ISO 320 vs 400. Repeat for RAW_TYPE_REGISTER set to CCD. Then start overriding SHAD_GAIN with any values you want, even something like this:

if (get_halfshutter_pressed())
{
    EngDrvOut(SHAD_GAIN_REGISTER, rand());
}



Now look at the problems that appeared from this change (the 10/12-bit lossless implementation):

- First of all, the "slurp" EDMAC channel (the one that writes the raw data into memory) had to be configured for exact resolution; the autodetection from raw_lv_get_resolution (0xC0F0680x/0xC0F0608x) is not exact - it's often off by one on the vertical direction, although the exact reason is unknown. Being off by one on 5D3 resulted in the raw data being correct only on every other frame (although I don't really understand why that happens). The issue was only with RAW_TYPE_REGISTER set to DEFCORRE, but it all worked fine when it was set to CCD.

- Next, take a look at this bug (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183246#msg183246): in a video mode with increased vertical resolution, the darkening works only on the top side, on an area equal to default Canon resolution in that video mode. That means, we have to reconfigure some more registers (http://www.magiclantern.fm/forum/index.php?topic=19300.msg183181#msg183181) - probably in the image processing pipeline, all the way from CCD (sic) to DEFCORRE. Which ones? I don't know - couldn't find them in adtg_gui. I hope to find them by emulating the LiveView in QEMU, but that's going to be a really long journey.

That's why, for now, the 10/12-bit lossless compression only works in video modes with unmodified resolution (plain 1080p, plain 720p and 5x zoom).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 08, 2017, 05:51:43 AM
@Ilia3101 -- baby steps  ;D just pick one thing to focus on and learn as much as you can , look closer at what the code is doing in the 3k 5d3 preset , there a lot of clues there even thou it for the 5d3 the 5d2 share some similarities in this respect
it may not look like there much progress right now but we are laying out the frame work for more then 3.5k video this will also lead to FHD 10/12bit and even in time compressed raw .

@a1ex thanks for the lesson , things are getting clearer .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Wayne H on September 08, 2017, 07:18:20 AM
@Alex that is an excellent and highly detailed explanation on how you Magic Lantern magicians are squeezing 3.5K and 4K out of the 5d3,

I was wondering a while back why the 12bit and 11-8bit lossless options were only available on the 3.5K mode, and not in the other UHD and 4K modes, but after doing some magic lantern reading, i quickly understood that the 3.5K mode works differently to all the other high resolution modes,

Makes me appreciate just how much work you guys do in your own free time, LIVEVIEW is incredibly complex. (it took Canon 6 months just to get the 5d3 to spit out clean and mirrored HDMI via a firmware update) AMAZING WORK GUYS.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 03, 2017, 07:23:33 AM
Ok I'm going to try and get "CONFIG_EDMAC_RAW_SLURP" for 5D2 working ,  from this allocate-raw-lv-buffer#Lsrc/raw.cT407 (https://bitbucket.org/hudson/magic-lantern/commits/1a2a91eb31a38ad0a3b118f14b64229e8823ecc1?at=allocate-raw-lv-buffer#Lsrc/raw.cT407) tell how to go about it , but I still don't understand .  
408  * How to find buffer dimensions for CONFIG_EDMAC_RAW_SLURP:
409  *   - Go to LV and use lv_save_raw
410  *   - Check the RAW_LV_EDMAC debug info
411  *   - Suppose it reports W: 0xA3A H: 0x3C7 (taken from 1100D in LV mode)
412  *   - EDMAC W is the number of bytes per line
413  *   - The W resolutions is computed as: W * 8 / 14 (raw buffer is 14 bits per pixel)
414  *   - Thus 0xA3A  8 / 14 -> 1496 pixels
415  *   - EDMAC H is the number of "jumps"
416  *   - The H resolutions is H + 1 -> 0x3C8 -> 968 pixels


from line 411 -- W: 0xA3A H: 0x3C7  how do I figure out what these compute to , I must be missing something here . Also how do I implement "lv_save_raw" from line "409" there must be some code I missed . I remember some where it was posted but not sure where , @dfort didn't you go though this with the eosm ? I can't seem to find it , if not I have to do more searching .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on October 05, 2017, 01:43:07 AM
Quote from: reddeercity on October 03, 2017, 07:23:33 AM
@dfort didn't you go though this with the eosm ? I can't seem to find it , if not I have to do more searching .

I can't find it either. This is the closest that I could come up with: http://www.magiclantern.fm/forum/index.php?topic=5601.msg179673#msg179673

Maybe write something in dubug.c calling lv_save_raw and print it on the screen or console? There's a hint in lv_rec.c:

    /* this causes the function to hang!? */
    if(data.options.rawMode)
    {
        //~ bmp_printf( FONT(FONT_MED, COLOR_WHITE, COLOR_BLACK), 30, 20 * yPos++, "Make sure you ran call('lv_save_raw')");
        call("lv_save_raw", 1);
        msleep(200);
        raw_update_params();
    }


The rest of it makes sense to me:

0xA3A (hex) = 2618 (decimal)
2618 *  8 / 14 -> 1496 pixels
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 05, 2017, 06:21:21 AM
Quote from: dfort on October 05, 2017, 01:43:07 AM
I can't find it either. This is the closest that I could come up with: http://www.magiclantern.fm/forum/index.php?topic=5601.msg179673#msg179673

Maybe write something in dubug.c calling lv_save_raw and print it on the screen or console? There's a hint in lv_rec.c:

    /* this causes the function to hang!? */
    if(data.options.rawMode)
    {
        //~ bmp_printf( FONT(FONT_MED, COLOR_WHITE, COLOR_BLACK), 30, 20 * yPos++, "Make sure you ran call('lv_save_raw')");
        call("lv_save_raw", 1);
        msleep(200);
        raw_update_params();
    }

@dfort thanks , that will help a lot

Quote from: dfort on October 05, 2017, 01:43:07 AM

The rest of it makes sense to me:

0xA3A (hex) = 2618 (decimal)
2618 *  8 / 14 -> 1496 pixels
:-[ I never thought hex , I light bulb just came on as soon as I that.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on October 05, 2017, 07:50:01 AM
It might help if you read how dmilligan found the addresses for the EOSM starting around here:

https://www.magiclantern.fm/forum/index.php?topic=5601.msg175969#msg175969

Yeah, it is confusing and doesn't translate across platforms but it might get you a step closer.

There's also several methods of finding some of this stuff. It appears that in the code snippet you posted it is showing, "How to find buffer dimensions for CONFIG_EDMAC_RAW_SLURP" using the 1100D as an example. The size is 1496 x 968. Isn't that the size of the full raw buffer?

Another way of getting the full raw buffer is to set the camera's movie settings to the mode you want (say 1920x1080/24 in the Canon menu) and while the camera is in movie mode shoot a simple silent still DNG. Take that DNG and run it through dcraw using the -4 and -E options. The output file (in .pgm format) will be the size of the full raw buffer. That's what I had to do when working out the focus pixel maps for cameras that show focus pixels in raw video.

Quote from: a1ex on January 13, 2016, 11:42:53 PM
To get the entire raw buffer from a DNG, including OB area: dcraw -4 -E
To get the active area only: dcraw -4 -D

That's the second time today that I quoted this tip. OB means out of bounds. There's actually an area that gets cropped out when viewing the DNG of the full raw buffer and using that -E option will show you the entire raw buffer.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 30, 2017, 04:00:53 AM
@dfort thanks , I'll give that a go
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 30, 2017, 05:34:24 AM
So some more tests -- with these latest attempt I  see some progress in  Liveview  @  3584x1068  (http://www.magiclantern.fm/forum/index.php?topic=19336.msg187766#msg187766)

(http://preview.ibb.co/mjngfm/VRAM4.png) (http://ibb.co/m7vZ0m) (http://preview.ibb.co/jVihmR/VRAM5.png) (http://ibb.co/c2Brfm)

The first image with the pink noise bar across the top of liveview was with a half shutter press and the second was with Zebras & Focus Peck enable -- see image below

(http://preview.ibb.co/g8ef6R/VRAM6.png) (http://ibb.co/ifEDRR)

So this leading me to believe the there is either a vertical resolution out of sync or fps timers .
I guess I should say what my configuration was  ;D I'm still using my custom build with modified Liveview buffer  (http://www.magiclantern.fm/forum/index.php?topic=19336.msg189550#msg189550) as noted in other post here on this thread .
Camera setting are real basic , Preview I use "ML B/W preview for liveview , frame override enable to 23.976 . I only loaded adtg_gui.mo , crop_rec.mo , raw_rec.mo (I use .raw format for testing)

I notice something very interesting , on first boot I enable adtg_gui.mo & the rest -- mess with  cmos regs etc. ... when I got the image all messed up I did a reboot to clear cmos registers and that's when it happen  :o it automatically boot to my custom buffer , now I was back to 3584x1068 -- the strange thing adtg_gui.mo was not enable just crop_rec 1:1 , there was no memory patches . That what the screen shots are from , the non enabled cmos registers , just crop record . The code from crop_record comes from the 5D3 and I have not change it at all , I should mention before I rebooted and adtg_gui.mo enabled I had use the 5d3 3K preset so unless I rebooted to quickly and those cmos registers so how stayed in memory I can't seeing it but you never know. So as far as I know all this is done with the custom liveview buffer I'm using

#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (2040*1267)
#endif

Plus the other changes I made to the code here (http://www.magiclantern.fm/forum/index.php?topic=19336.msg189550#msg189550)
Setting:
crop_rec.cfg
# Config file for module crop_rec (CROP_REC.MO)crop.preset = 1
magic.cfg
# Magic Lantern Nightly.2017Apr14.5D2212 (bd8e89d9dccf+ (iso-research))
# Built on 2017-04-14 03:42:23 UTC by ml@ml-pc
# Configuration saved on 2017/10/29 19:17:40
beta.warn = 29
menu.first = -3
movie.log = 1
rec.notify = 0
enable-liveview = 2
fps.override.idx = 11
battery.drain.rate.rev = 69
hist.log = 0
spotmeter.draw = 0
zebra.draw = 0
disp.mode.x = 149


raw_rec.cfg
# Config file for module raw_rec (RAW_REC.MO)
raw.video.enabled = 1
raw.res.x = 20
raw.dolly = 1
raw.preview = 2
raw.warm.up = 2


I also tested this with HDMI output connected my atomos ninja hdmi hard drive recorder , uploaded to my bitbucket downloads link below
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/00150_H264.mp4
there's some shearing etc. ... but did not crash the camera , one thing I notice thou was the ml scopes where blank no feed back for the image .
at the end of the video I try to record raw video but got a "Raw error" this was expected .
But in the next file after I cycled the hdmi connection I then had the ML scopes came back see it in the link below
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/00153_H264.mp4

So I can also take a CR2 photo in this configuration and it all right no problem below is a short video that I take with my iPhone with me pressing the shutter fully to get a cr2 photo link below also
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/IMG_2273.MOV
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/_MG_8668.CR2

I notice as I panned the camera left to right the preview that was ml b/w the distortion would move or pulse and of course with half shutter I get color distortion , but it's not a static frozen image







Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 30, 2017, 06:04:18 AM
I notice there one line of pixel you can see here the LV-004.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-004.422) image
Liveview dump
(http://preview.ibb.co/mQPYt6/LV_004.jpg) (http://ibb.co/ktpzY6)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 07, 2017, 06:11:25 AM
Going over ARM Configuration Bus Access to Peripherals (http://www.ti.com/lit/ds/symlink/tms320dm368.pdf) which I been led to believe to be in the 5d2 and all digic 4

(http://thumb.ibb.co/mvdusb/Arms_1.jpg) (http://ibb.co/mvdusb)

Look at the red hi-lighted box , that looks like the resizer registers I have been adjusting in my earlier tests I posted with adtg_gui.mo 
the IPIPE Registers are interesting DM365 IPIPE register set (http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/100/t/36802)
Quote....These include for example register offset ranges 0x44-0x1CC and 0x1F0-0x228.
According to the code these address ranges contain registers for OTF Defect Pixel Correction, Lens Shading Correction, Noise Filtering and CFA interpolation. ....
will that seem to be dslr stuff  :o  , need to dig deeper .


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 07, 2017, 06:27:02 AM
Could this be the Sensor Interface (ISIF) – RAW Data Processing Flow for canon digic 4 or all eos ?

(http://thumb.ibb.co/nfKg5w/cmos2raw_flow.jpg) (http://ibb.co/nfKg5w)

http://www.ti.com/lit/ug/sprufg8c/sprufg8c.pdf

I think this my be a very importance pdf with registers that could map out to what is needed for development


Edit: interesting , Image Pipe Interface with dark frame subtraction ?
(http://thumb.ibb.co/gqiVyG/cmos2raw_flow_1.jpg) (http://ibb.co/gqiVyG)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ItsMeLenny on November 08, 2017, 10:23:15 AM
I have nothing valuable to add I just wanted you to know I get so excited when there's a new reply in this thread.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on November 09, 2017, 07:02:56 PM
Quote from: ItsMeLenny on November 08, 2017, 10:23:15 AM
I have nothing valuable to add I just wanted you to know I get so excited when there's a new reply in this thread.

+1  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on November 09, 2017, 07:22:33 PM
Quote from: ItsMeLenny on November 08, 2017, 10:23:15 AM
I have nothing valuable to add I just wanted you to know I get so excited when there's a new reply in this thread.

Agreed!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: domo94 on November 29, 2017, 09:37:52 PM
I am coming from a 550d thread.

To help with further development with 550d, and hopefully to get this software ported with Canon 7D to get the most consistent and solid 10bit 1080p recording I need!

How can I help?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on November 30, 2017, 07:12:14 AM
There are lots of parts to this.

There's the 10bit/12bit (http://www.magiclantern.fm/forum/index.php?topic=5601.0) part that apparently requires either CONFIG_EDMAC_RAW_SLURP and/or CONFIG_RAW_LIVEVIEW but that can't be right because if you notice that there is an experimental build for the 1100D that supports 10bit/12bit using a branch where RAW_SLURP isn't defined on that camera. However, if RAW_SLURP is required that has been cracked on the 1100D in the allocate-raw-lv-buffer branch.

Then there's lossless compression where it seems that we found all the stubs for the 7D (http://www.magiclantern.fm/forum/index.php?topic=18443.msg193534#msg193534) but haven't figured out how to get it working properly yet.

Finally, there's the crop_rec module (http://www.magiclantern.fm/forum/index.php?topic=17021.0). Several cameras are working with the 3x3 mode which allows shooting nice quality 60fps raw video without that terrible line skipping that shows lots of aliasing artifacts in the 1280x720p mode. However, to get to the juicy stuff that includes 4k raw video will require a lot more work that so far is only supported on the 5D3.

Attacking this all at once is overwhelming, believe me I tried. I'd suggest taking one of these topics and start reading from the beginning like a novel to see how these puzzles were eventually solved.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 01, 2017, 06:48:59 AM
Actually it's really ISO_Research Branch with crop_record that's it . Here (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magic-lantern-adtg-gui-4k-test_source_code.zip)  is my source code I'm working from 
If you read my thread from the start it's all explained.
10/12bit & lossless compression is not part of this right now so has no bearing on Digic IV UHD/3.5K raw video development ,
Thou I will implement 10bit (as this work on 5D2) after I can get a unscrambled stable image  (very close now)

Digic IV cams work with RAW_LV_BUFFER_ALLOC_SIZE  as long as you find the right buffer e.g. 5D2 #define RAW_LV_BUFFER_ALLOC_SIZE (2040*1267)
I would suspect 550d would be the same but for 7D it may not because of the dual cpu there are more difficulty

Quote from: domo94 on November 29, 2017, 09:37:52 PM
I am coming from a 550d thread , To help with further development with 550d
Thanks , look here (http://www.magiclantern.fm/forum/index.php?topic=19336.msg183277#msg183277) Greg was helping with 500d with some promising result but stop.
So search Greg's posts on my thread & on the forum he has some very good info to share e.g. 500D, trying to increase the resolution of the video raw (crop mode).  (http://www.magiclantern.fm/forum/index.php?topic=10111.msg123909#msg123909)
In fact from his initial experiment's a1ex started the whole 4K/UHD for 5D3  crop_rec on steroids: 3K, 4K, 1080p48, full-resolution LiveView  (http://www.magiclantern.fm/forum/index.php?topic=19300.msg182052#msg182052) he credited in the first post .

If you can compile the source code (you need a working environment I recommended a prebuild VM of Ubuntu  Quick Guide and Solutions for VirtualBox  (http://www.magiclantern.fm/forum/index.php?topic=7579.msg134989#msg134989) that the one I use)
can be ran on Windows or MAC , It's all setup nothing to be configured --install VirtualBox for your OS and start working.
The other useful thing is to decompile your ROM Dump files look here (http://www.magiclantern.fm/forum/index.php?topic=12177.msg117735#msg117735) for help in disassembling.

Quote from: domo94 on November 29, 2017, 09:37:52 PM
...... ported with Canon 7D to get the most consistent and solid 10bit 1080p recording I need!
This is off topic but here you go  ;)
Will there is 10bit 1080p already with the 7D with the experimental 10/12bit build , but only works in 3x cropmode ,
can be downloaded from my bitbucket account 10bit_12bit_raw_2016Dec01.7D203.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-10bit_12bit_raw_twk_crop_rec.2016Dec01.7D203.zip)
1:1 liveview produces corrupted image every other frame usually the top 1/3 , that's why it's not in the main build area and only works in cropmode with audio by the way  :)
 


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 01, 2017, 09:59:02 AM
Yes, I've been following this topic and I am able to compile Magic Lantern and disassemble ROMs. In fact I compiled that 7D build you pointed to on your last post exactly one year ago. I took it down because 10bit/12bit wasn't supported on the 7D (or 5D2, 50D, 500D, 550D).

According to the experiments download page (https://builds.magiclantern.fm/experiments.html) "Only models with CONFIG_EDMAC_RAW_SLURP are compiled." However, I thought it was worth noting that there is a 10bit/12bit build for the 1100D available for download. It is being compiled on the raw_video_10bit_12bit branch and if you look through the code you'll see that CONFIG_EDMAC_RAW_SLURP isn't defined for the 1100D in that branch.

Quote from: reddeercity on December 01, 2017, 06:48:59 AM
Digic IV cams work with RAW_LV_BUFFER_ALLOC_SIZE  as long as you find the right buffer e.g. 5D2 #define RAW_LV_BUFFER_ALLOC_SIZE (2040*1267)

You defined RAW_LV_BUFFER_ALLOC_SIZE in raw.c but you didn't define CONFIG_EDMAC_RAW_SLURP in internals.h. You are working on the iso-research branch but for this experiment you will probably have better luck using the allocate-raw-lv-buffer branch. Note that neither iso-research or allocate-raw-lv-buffer has the 10bit/12bit code so you'll need to merge those changes with the raw_video_10bit_12bit branch. [EDIT - Yeah, you're doing all that.] For everything you're trying to do your goal should be to eventually get it working in the crop_rec_4k branch. Good luck!

Note: It would really help if you would update your Bitbucket repository (https://bitbucket.org/reddeercity/) instead of posting a link to your local repository.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 02, 2017, 08:11:15 AM
Let me backpedal a bit. This thing is rather complicated and hard to grasp all at once.

Instead of blabbing away let me offer a bit of code to find something that might help. A while back I was searching for the SRM_BUFFER_SIZE on some cameras that I didn't have access to so I made a custom build to print it out on the LCD screen. This is something that you will probably need as a1ex noted in this post (http://www.magiclantern.fm/forum/index.php?topic=19336.msg183374#msg183374).

src/exmem.c  (based on the unified branch)
static void srm_malloc_cbr(void** dst_ptr, void* raw_buffer, uint32_t raw_buffer_size)
{
    if (!srm_buffer_size)
    {
        /* we can't tell how much to allocate; the allocator tells us */
        srm_buffer_size = raw_buffer_size;
    }
    else
    {
        /* it should tell us the same thing every time */
        ASSERT(srm_buffer_size == raw_buffer_size);
    }
   
    /* return the newly allocated buffer in the output variable */
    *dst_ptr = raw_buffer;
   
    /* announce it's done */
    give_semaphore(srm_alloc_sem);
   
    bmp_printf(FONT_LARGE, 50, 300, "SRM_BUFFER_SIZE 0x%x", srm_buffer_size);
}


The only change is that last line in order to print the SRM_BUFFER_SIZE on the screen when shooting a simple silent DNG, like this on the 7D:

(https://farm5.staticflickr.com/4552/38744482622_fa7c746eb2.jpg) (https://flic.kr/p/222HAR5)

So this can be added to the consts.h file (malloc section):

#define SRM_BUFFER_SIZE 0x1F80000   /* print it from srm_malloc_cbr */

@tecgen found it for the 550D (http://tecgen) - 0x1F68000 - I should add that he found it on his own.

He said he was going to try to find it on the 50D and 5D2 but I haven't seen any updates from on this. So how about we look for these? I put up builds for these cameras on my downloads page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/) so we can document the SRM_BUFFER_SIZE. If anyone wants to look for it on a camera that isn't listed I'd be glad to post a build or it can be compiled from the unified-find-SRM_BUFFER_SIZE branch (https://bitbucket.org/daniel_fort/magic-lantern/src/3418e399402f28ef7ab6930284a0d8f2fd39b0e3/?at=unified-find-SRM_BUFFER_SIZE) in my repository.

I also did a screenshot of edmac on the 7D and see something very different from the 5D2. Maybe the channels we want are running on the master processor so they aren't visible on the slave? I don't know.

(https://farm5.staticflickr.com/4569/38747827772_fa536d6542.jpg) (https://flic.kr/p/2231Kf5)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Andy600 on December 02, 2017, 10:38:37 AM
@dfort - Just ran your build on the 50D and the SRM buffer size shows as 0x1b14000

useful bit of code :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on December 02, 2017, 06:49:30 PM
(http://image.ibb.co/nb2Tzb/VRAM0.png)

I can finally 'help' with getting 3k on the 5D2! thx dfort

0x25d0000

37MB buffer?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 04, 2017, 06:55:12 AM
From adtg_gui.c-1121 (https://bitbucket.org/hudson/magic-lantern/src/50d8f06018dc52f72f04265432dd83106e495e85/modules/adtg_gui/adtg_gui.c?fileviewer=file-view-default#adtg_gui.c-1121) This is the 3K 5d3 preset I use on my 5D2

static int res3k_reg(int reg)
{
    if (regs[reg].dst == 0xC0F0)
    {
        switch (regs[reg].reg)
        {
            case 0x6804:                /* C0F06804 - raw resolution */
                return 0x52801AB;       /* from 0x528011B -> 3072px in raw_rec */

with buffer
#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (2040*1267)
#endif

that end up to be 3584x1068 .
A hint from nikfreak from here (http://www.magiclantern.fm/forum/index.php?topic=19336.msg183107#msg183107) I see where I my have a problem , just dawn on me
that the registers are a little different 
•C0F06804 (Digic5)
•C0F06084 / C0F06088 (Digic4)

Dam Dyslexia  >:(

As seen in this piece of code adtg_gui.c-126 (https://bitbucket.org/hudson/magic-lantern/src/50d8f06018dc52f72f04265432dd83106e495e85/modules/adtg_gui/adtg_gui.c?fileviewer=file-view-default#adtg_gui.c-126) full res liveview uses C0F06084
   * 5D2 photo:
     * 0xC0F06088.hi - 0xC0F06084.hi is the vertical resolution (3804)
     * 0xC0F06084.hi shifts the image in one-line increments, so it must be the first scanned line
     * (0xC0F06088.lo - 0xC0F06084.lo) * 2 is the horizontal resolution (5792)


So it look like I need to make a few changes to my source code to take that in account .
Stay tuned ;)


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on December 04, 2017, 07:13:28 AM
This is all smoking hot... keep it up guys!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 04, 2017, 07:24:51 AM
Something extra to add , I thing I know way I'm lock to 3584x1068 , I see that cmos[1] is set to 0x802 and if
I convert the hex to decimal so 0x802 = 2050 *14 /8 = 3587.5 -- my custom buffer 2040x1267 , so I my change the buffer to 2050 instead of 2040 and see what happens
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: IDA_ML on December 04, 2017, 05:27:54 PM
A very hot topic, indeed!  I keep checking for new posts at least twice a day.  Am a right by thinking that there might be some hope for the good old 7D too?  Excellent work, guys!  I keep my thumbs pressed!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 04, 2017, 06:55:41 PM
Checking it twice a day? Really?

Of course it would be much easier if we all upgraded our older cameras for something not so old but not too new either.

Anyway--I did some playing around with the 7D over the weekend. One of the things I wanted to see was all the changes that @reddeercity actually did so I found which changeset he started with and managed to put it into a branch in my repository. This (https://bitbucket.org/daniel_fort/magic-lantern/commits/deee398f9f00c674b791cee7d922e0327a571e6a) should provide a clearer picture of what is going on. I also made a dummy pull request of the stuff I worked on which was trying to get CONFIG_EDMAC_RAW_SLURP working. I used the allocate-raw-lv-buffer branch because it was pointed out somewhere to look at how the 1100D was done done on this branch but it didn't compile! Turns out that it still had a problem in the code that was resolved a while back. BTW, the 1100D is an EVF_STATE camera while the 50D, 5D2, 7D and other cameras we're looking at are LVState cameras. None of the LVState cameras have CONFIG_EDMAC_RAW_SLURP working--yet. Anyway, here's my work in progress:

https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/15/searching-for-config_edmac_raw_slurp-on/diff

There are lots of pieces to this puzzle and it would be great to get at least some of the answers we found cataloged in some way. David (a.k.a. reddeercity) how about modifying your first post and adding the SRM_BUFFER_SIZE for the various cameras that we have this resolved? At some point down the line we'll need this and having them in one place instead of scattered throughout various posts would be much easier to work with.

Another thing worth sharing are these screenshots I did on the 7D.

First the default:
(https://farm5.staticflickr.com/4555/37936637285_b425e1546a.jpg) (https://flic.kr/p/ZNkbzc)

Next, what happens after recording a bit of raw video again at the default raw video settings:
(https://farm5.staticflickr.com/4556/38106830844_5c0fb2db6b.jpg) (https://flic.kr/p/214ntbU)

Of significance is what happens with channels 6 and 10. Note this in the code:

edmac-memcpy.c
#elif defined(CONFIG_7D)
uint32_t edmac_read_chan = 0x0A;  /*Read 0x19 0x0D 0x0B 0x0A(82MB/S)*/
uint32_t edmac_write_chan = 0x06; /* Write 0x5 0x6 0x4 (LV) */


Channel 6 is the write channel and 10 (0x0A hex) is the read channel.

Now let's put the camera in zoom mode, max out the image size and record a clip--albeit a very short clip.

(https://farm5.staticflickr.com/4580/37936638545_1952fba3c7.jpg) (https://flic.kr/p/ZNkbWV)

(https://farm5.staticflickr.com/4529/38106832204_245b7a9bda.jpg) (https://flic.kr/p/214ntAm)

Pretty cool, eh? (Speaking Canadian.)

Not sure what channel 16 is for but it is interesting that it is the largest and the address didn't change.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 05, 2017, 01:30:52 AM
 :) Yea , 2 more things -- wear a toque get a beer and I will make you a honorary Canadian  ;D
(https://upload.wikimedia.org/wikipedia/en/thumb/2/28/Bob_and_Doug_McKenzie.jpg/275px-Bob_and_Doug_McKenzie.jpg)

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 05, 2017, 02:06:55 AM
Ok now down to business -- That large buffer is what I would focus on from channel #16 , That's how I choose the 5d2 buffer on channel #5 in the Edmac.
If you navigate  to channel #16 on the 7D I would expect to see a different address then the default in raw_rec.c
#if defined(CONFIG_500D) || defined(CONFIG_550D) || defined(CONFIG_7D)
#define RAW_LV_EDMAC 0xC0F26008
#endif

That's why I changed define RAW_LV_EDMAC 0xC0F04508 to define RAW_LV_EDMAC 0xC0F04500
Something interesting -- if you take Edmac#16 (on 7D) buffer "4410x1225"
4410/14*8 = 2520
So this could be your custom buffer 2520 x 1127 1225-- this is how I came up with my buffer.

Quote from: dfort on December 04, 2017, 06:55:41 PM
....... at least some of the answers we found cataloged in some way. David (a.k.a. reddeercity) how about modifying your first post and adding the SRM_BUFFER_SIZE for the various cameras that we have this resolved?

Yes I will update the first post with screen shots and buffers information on all digic iv that are being actively be worked on ,
this means any one following this thread with a digic iv cam that don't see your camera on the first post will need to be involved.
At this point the camera's with CF Cards have priority 5D2 , 7D , 50D -- then the 550d & 500d but not limited in that order 

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 05, 2017, 03:05:09 AM
Also I use the crop_rec.mo  to get the 3.5k . I enable the 1:1 preset which I sure give me every line etc. ...
  switch (crop_preset)
        {
            /* 1:1 (3x) */
            case CROP_PRESET_3X:
                /* start/stop scanning line, very large increments */
                /* note: these are two values, 6 bit each, trial and error */
                cmos_new[1] = (is_720p())
                    ? PACK12(14,10)     /* 720p,  almost centered */
                    : PACK12(11,11);    /* 1080p, almost centered */
               
                cmos_new[2] = 0x10E;    /* read every column, centered crop */
                cmos_new[6] = 0x170;    /* pink highlights without this */
                break;
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: IDA_ML on December 05, 2017, 12:28:13 PM
Quote from: dfort on December 04, 2017, 06:55:41 PM
Checking it twice a day? Really?

Of course it would be much easier if we all upgraded our older cameras for something not so old but not too new either.

O, absolutely!  Of course, you are right, Dfort, it would be much better if we upgraded our cameras to something newer but do we really have that choice?  The only alternative and better model to the 7D that supports Magic Lantern is the 5D3 but how many people on this forum can afford this body and the lenses for it?  I also own the 100D which is a fantastic little ML camera, perfectly suited for travel, especially after the latest ML developments, but it deffinitely is not appropriate for serious work, considering the short battery life, the limited vertical resolution, lack of VAF filter and especially the very annoying "hick-ups" (random jumps throughout the video) in the 5x-magnification mode which makes the camera unuseable in that most important for serious practical video shooting mode.

It may be interesting for you to know that this is not only my opinion.  I have discussed the issue with several  friends and colleagues of mine who use their 7Ds, 5D2s and even 50Ds on a daily bases and they all share these same thoughts.  That is why, the excitement about the latest developments concerning these old Digic IV cameras among the video shooting community is tremendous!  I am sure, Mr. Reddeercity who obviosly loves his 5DMk2 so much, understands what I mean :-))).

And finally, a moral issue that I would like to share here.  You say, upgrading to newer models. But why, if the old models continue working so well, are built like tanks, are extremely reliable and in many practical situations outperform the newer ones?  Why would I want to sell my trusted 7D that I payed so much money for back in 2011 if it still works and looks like brand new and in fact, I have not used up even 50% of its shutter life?  How much does it cost to recycle a 7D and how much energy would it take?  Wouldn't we all make a friendly gesture to our Mother Nature if we use our cameras more consciously and do not upgrade them so often?  In fact, I think that Magic Lantern fulfills a mission in that respect - by giving new life to older cameras it helps protecting our environment.  Another reason why I like it so much and why I am so gratefull to all our developers - the real missionaries!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 05, 2017, 05:11:24 PM
@IDA_ML - I'm not arguing with you, just hinting that it would be easier if we all had 5D3's. My first ML capable camera was an EOSM. I got it because thought it could do almost everything the other ML capable cameras could do. I does, depending on your definition of almost.

Back on topic--

Quote from: reddeercity on December 05, 2017, 02:06:55 AM
Something interesting -- if you take Edmac#16 (on 7D) buffer "4410x1225"
4410/14*8 = 2520
So this could be your custom buffer 2520 x 1127 -- this is how I came up with my buffer.

How did you get 1127 from 1225?

What I'm looking for is how to translate this from the 1100D to the 7D and other cameras that use LVState.

#ifdef CONFIG_1100D
#define RAW_LV_BUFFER_ALLOC_SIZE (3906*968)
#endif


Note that originally it was coded like this:

#define RAW_LV_BUFFER_ALLOC_SIZE (3906*766)

"Blame" shows that G. Condello (a.k.a. @nanomad) wrote that line but I searched all over and couldn't find out how he got the "3906" part. The "968" part is spelled out in this old comment you already posted:

408  * How to find buffer dimensions for CONFIG_EDMAC_RAW_SLURP:
409  *   - Go to LV and use lv_save_raw
410  *   - Check the RAW_LV_EDMAC debug info
411  *   - Suppose it reports W: 0xA3A H: 0x3C7 (taken from 1100D in LV mode)
412  *   - EDMAC W is the number of bytes per line
413  *   - The W resolutions is computed as: W * 8 / 14 (raw buffer is 14 bits per pixel)
414  *   - Thus 0xA3A  8 / 14 -> 1496 pixels
415  *   - EDMAC H is the number of "jumps"
416  *   - The H resolutions is H + 1 -> 0x3C8 -> 968 pixels


So is W width and H height of one of the buffers? Maybe but "W: 0xA3A H: 0x3C7" would translate to 2618x967. Note that the 1100D specifications show a maximum video resolution of "720p HD video at 25 or 30 fps" so maybe, just maybe, that's a reason why it didn't even need CONFIG_EDMAC_RAW_SLURP for it to work in the raw_video_10bit_12bit branch.

I also went back into the history of the source code and couldn't figure out when these "W:" and "H:" values were reported in hex but maybe (and this is just an uneducated guess) the values for the 7D should be taken from the maximum size of channel 16:

#ifdef CONFIG_7D
#define RAW_LV_BUFFER_ALLOC_SIZE (4410*1226)
#endif


Note the +1 on the "H:" value as noted in that old comment we are referencing.

This seems logical because the 7D can record higher resolution than the 1100D. Then again so many things in ML aren't what they seem so I might be totally off base on these assumptions.

By the way, your calculation of the width of the "custom buffer" matches up with the zoom mode raw buffer. I shot silent stills in various modes and this is what exiftool is showing on the DNG files:

mv640
Default Crop Size               : 1736 694
Active Area                     : 26 256 720 1992

mv720
Default Crop Size               : 1736 694
Active Area                     : 26 256 720 1992

mv1080
Default Crop Size               : 1736 1156
Active Area                     : 26 256 1182 1992

zoom
Default Crop Size               : 2520 1200
Active Area                     : 26 0 1226 2520

FRSP
Default Crop Size               : 5202 3466
Active Area                     : 50 158 3516 5360


Note the height of the zoom full raw buffer? Back to my question, "How did you get 1127 from 1225?" Would you agree that it should be 1226? That would mean that your "custom buffer" is really just the zoom mode buffer.

Exercise for other Digic IV cameras: Shoot silent DNG's at various video modes. Just change the Canon movie settings and shoot the silent stills with the camera in movie mode. Don't forget to do one for zoom (a.k.a. 5x crop mode). If anyone wants to try this but hasn't mastered exiftool, post a link to your DNG's and we'll check them out for you.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on December 05, 2017, 07:21:20 PM
RAW_LV_BUFFER_ALLOC_SIZE answered here (http://www.magiclantern.fm/forum/index.php?topic=18443.msg188572#msg188572). You need to allocate space for how much raw image data *you* plan to save from connection #0 (you are reconfiguring the LiveView resolution, after all). For experiments, you could always allocate one SRM buffer, which is the largest contiguous chunk you can get from Canon's memory allocators - a 14-bit full-res uncompressed*) image won't be larger than that. This is done on 5D3 for full-res LV (crop_rec_4k).

*) "14-bit uncompressed" is important; it can get larger if you request 16-bit data, or after lossless compression (yes, a compressed image *can* be larger than the input image). Currently there's no known way to store such image in a contiguous chunk of memory. For lossless, it usually works, because - in practice - most of the time, the compressed image ends up smaller. Still, you also have to handle the case where compressed is larger than uncompressed (otherwise you'll overwrite other stuff in memory, sooner than you may think).

1100D *has* CONFIG_EDMAC_RAW_SLURP (check the build log from the Experiments page). Don't remember anyone reporting back for this model, so I don't know whether the 10/12-bit build works or not (or whether to commit the allocate-raw-lv-buffer merge, for that matter).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 05, 2017, 08:52:36 PM
Thanks for clearing things up though at the same time a few things got muddied up.

Quote from: a1ex on December 05, 2017, 07:21:20 PM
1100D *has* CONFIG_EDMAC_RAW_SLURP (check the build log from the Experiments page). Don't remember anyone reporting back for this model, so I don't know whether the 10/12-bit build works or not (or whether to commit the allocate-raw-lv-buffer merge, for that matter).

So this whole thing about using the 1100D as an example might just turn out to be fool's gold? I could have sworn that raw_video_10bit_12bit 1100D/internals.h (https://bitbucket.org/hudson/magic-lantern/src/870cedc16e2d5ad35852cc8c7b04fd95ea020c45/platform/1100D.105/internals.h?at=raw_video_10bit_12bit&fileviewer=file-view-default) *doesn't* have CONFIG_EDMAC_RAW_SLURP defined while allocate-raw-lv-buffer 1100D/internals.h (https://bitbucket.org/hudson/magic-lantern/src/5d168ad4e8c9f6876ad33383eadf628218854c91/platform/1100D.105/internals.h?at=allocate-raw-lv-buffer&fileviewer=file-view-default) *does*. Looked at it again and I still don't see it. I also looked all over the build log (https://builds.magiclantern.fm/jenkins/job/raw_video_10bit_12bit/26/consoleText) and that didn't didn't turn up any nuggets.

However, this turned on a light:

Quote from: a1ex on August 15, 2017, 08:15:19 PM
RAW_LV_BUFFER_ALLOC_SIZE is how much we need for a full-res image (max W * max H * 14/8).

Looking at the exiftool output for the active area of a full-res silent image, 5360 * 3516 * 14/8 = 32980080 or 0x1F73C70 hex. Hum, sounds familiar:

#define SRM_BUFFER_SIZE 0x1F80000   /* print it from srm_malloc_cbr */

So does that mean something like this should work?

raw.c
#ifdef LV_STATE
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE)
#endif


Haven't tried it and my syntax might be out of whack but that's the idea.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on December 05, 2017, 09:11:02 PM
Another tip: each autoexec.bin has human-readable metadata - open it with nano and check the header. That tells you exactly where it was built from (and whether there were any uncommitted changes).

SRM_BUFFER_SIZE would work if you allocate with srm_malloc_suite. With the malloc wrappers, there will be a small overhead from the memory backend, so... just reduce the size by 0x100 bytes or so. Didn't try, but this should be generic for any model that's going to implement full-res LiveView.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 05, 2017, 10:01:46 PM
QuoteHow did you get 1127 from 1225?
Sorry , mistake I meant 2520x1225 . But it seems I may be wrong after reading a1ex posts need to investigate more
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 05, 2017, 11:25:17 PM
Don't forget about this:

416  *   - The H resolutions is H + 1

That means it should be 2520x1226 and that works but we can go much further than that.

Quote from: a1ex on December 05, 2017, 09:11:02 PM
SRM_BUFFER_SIZE... just reduce the size by 0x100 bytes or so.

Did some experimenting and this is working:

#ifdef CONFIG_7D
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0x1000)
#endif


The generic "#ifdef LV_STATE" didn't work and that's the least amount of shaving off of SRM_BUFFER_SIZE that worked.

Let me qualify how well this allocat-lv-raw-buffer experiment is currently going. Up until now I was just checking DNG frames extracted from RAW and MLV files and they were looking good so I thought I'd render a video file and it was doing the old shake rattle and roll. In other words the DNG images looked clean but it appeared to be changing the capture area randomly. I backed out all the changes and guess what, it was still shaking so whatever I'm doing didn't cause this and didn't make it any worse.

So is CONFIG_EDMAC_RAW_SLURP working? Maybe, I don't know. Looks like the thing to do is to switch over to the raw_video_10bit_12bit branch and see what happens.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 06, 2017, 12:25:27 AM
Cool , I'll play with this on 5d2 and see what happens
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 06, 2017, 02:24:49 AM
Try out the code on the 5D2 , no problem every thing work normal in 1:1 & crop mode -- recorded 2 short .raw file 

#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0x1000)
#endif

At least no ill effects , complied it on the iso_research branch I'm using , but did not load adtg_gui yet to see if it changes anything.

Edit: going though the DM-Spy Log from the 5D2
E05D3> GuiMainTas:ffa4522c:18:03: (PUB) SetImageWorkMemory 1:0x40d00000, 2:0x41700000
E060A> GuiMainTas:ffa45274:18:03: (PUB) SetImageWorkMaxPixel W:2200,H:1872,0x5e4340
E0640> GuiMainTas:ffa44db8:18:03: (PUB) SetEffectiveSizeOfJpeg(w:7488 h:4992) 2995
E066D> GuiMainTas:ffa44dd0:18:01: SetEffectiveSizeOfJpeg w:7488 h:4992


So can I assume that 0x5e4340 is 2200x1872 ?

I'm giving "RAW_SLURP" another try -- that why I was thing 0x5e4340 may work for that like this ?
#ifdef CONFIG_5D2
#define DEFAULT_RAW_BUFFER MEM(MEM(0x5e4340))
#endif


I add #define CONFIG_EDMAC_RAW_SLURP
in to platform/5D2.212/internals.h
did I miss anything ?

I'll update later tonight.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 06, 2017, 04:37:33 AM
Houston, we have a problem. This is using a nightly build:

[EDIT] Nothing to see here -- discovered it was a post production problem. Camera is fine.

--snip--

Pressing on, I uploaded allocate-raw-lv-buffer branch builds to my downloads page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/). Included are builds for the 7D, 5D2, 50D, 550D and even the 1100D. Yeah, I know, not very interesting. It just seems to be an old branch that doesn't even have mlv_lite but it works quite differently on these cameras.

@reddeercity - I have no idea what you are doing but interested in how your experiment turns out. This is what I did with the allocate-raw-lv-buffer branch:

https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/15/searching-for-config_edmac_raw_slurp-on/diff

[EDIT] Got it working! 10bit/12bit on the 7D. I've got to do some more cleanup on the code because it is messing up the Digic V cameras but thought I'd put out the good news. Since there are so many requests from 7D users I uploaded an early build (https://bitbucket.org/daniel_fort/magic-lantern/downloads/) if anyone wants to be a volunteer guinea pig.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on December 06, 2017, 10:14:00 AM
You got it working!!??? Maybe getting it for the 5D2 is similar. How did you do it??

I see your commit has CONFIG_EDMAC_RAW_SLURP defined for 5D2, could it be working yet?

Surely it can't be that far if you found it for 7d already, the difficult camera :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 06, 2017, 10:14:15 AM
dfort with the magic fingers. 7D working with 10/12 bits, tested just now:
10bit sample
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M06-1005_samples_10bit_7D.zip
12bit sample
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M06-1006_samples_12bit_7D.zip
14bit sample
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M06-1007_samples_14bit_7D.zip



Any chance to get a build for testing on 550D?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Andy600 on December 06, 2017, 10:48:12 AM
@dfort and for the 50D if you can.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: IDA_ML on December 06, 2017, 12:42:31 PM
Dfort, you fully deserve your "Hero Member" status!  I can't wait to test.  Thank you so much!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 06, 2017, 02:59:34 PM
Ok can we please stay on topic here , any 10-12bit 7d should be discuss
either on the 7d or 10-12bit thread no here . This is for UHD/3.5k development.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 06, 2017, 03:44:35 PM
Ok can we please stay on topic here , any 10-12bit 7d should be discuss
either on the 7d or 10-12bit thread no here . This is for UHD/3.5k development.

I would rather welcome the guy(dfort) who potentially could hold the key to the raw_slurp code snippet for the 5D mark II. But hey, that´s just me.
Keep up the good work everyone. 10bit on 7D is golden.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: nikfreak on December 06, 2017, 04:33:37 PM
title says it all: "..and other digic IV cams"  and I bet the 7D will fit 3K continuous recording too :P
As far as I understand the LCD doesn't freeze now any more while recording 10/12bit @danne??
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 06, 2017, 04:59:28 PM
Quote from: nikfreak on December 06, 2017, 04:33:37 PM
title says it all: "..and other digic IV cams"  and I bet the 7D will fit 3K continuous recording too :P
As far as I understand the LCD doesn't freeze now any more while recording 10/12bit @danne??

No freezing, no corruption, only happy, healthy MLV files  :D

*update. mv1080p mode that is
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on December 06, 2017, 05:43:48 PM
Try 720p60 in Canon menu with crop mode ... Frozen LCD here ...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 06, 2017, 06:39:28 PM
I'll ask nicely one more time , please stay ON TOPIC !!!
If that can not happen then I will remove this thread all
Together , as I started this thread and all information will be
Lost .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 06, 2017, 07:05:28 PM
Quote from: reddeercity on December 06, 2017, 02:59:34 PM
This is for UHD/3.5k development.

Right, like I said before:

Quote from: dfort on November 30, 2017, 07:12:14 AM
There are lots of parts to this.

You can continue trying to increase the image size but you won't be able to pass all that data to the card unless you reduce the bit depth and compress it.

This seemed like the best place to discuss the features needed to get Digic IV cameras recording at the highest possible resolution but since you feel I'm off topic I'll move the 10bit/12bit discussion somewhere else.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 06, 2017, 08:21:50 PM
Thanks, I agree there many parts & compression will be needed
At some point but not right now . I what to focus on max resolution
& custom buffers . I just don't what HD resolution 10-12bit discuss here
It better on the 10-12bit thread
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 06, 2017, 09:09:21 PM
Just as a suggestion, you probably should get the 5D2 working with the "basic" CROP_PRESET_3x3_1X before attempting the more advanced stuff. I helped with that on the EOSM, 700D, 100D and work is in progress on the 650D and 6D.

Whatever path you take to get to higher resolution on these older Digic IV cameras, best of luck!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Audionut on December 07, 2017, 01:32:37 PM
Hats off to you reddeercity.  You've gone from a 'typical' user to someone willing to dip their toes in the water.  I was excited to see you start this thread, but so busy at the time that I never got around to jumping in and sending some praises your way.  The funny thing about coding for ML is that no one really knows what they are doing.  Some people can for sure code better then others, and some people have more knowledge regarding some things then others, but it's all trial and error to begin with.

My advice would be to let the conversation flow, even when it begins to go a little off-topic.  Even in the off topic discussion, you're likely to find pieces of the specific part of the puzzle you are trying to solve, and sometimes it's just best to move onto another area of the puzzle, and then come back to the original area of the puzzle after you've had some success elsewhere.  I also prefer to split discussions that become large and deserve their own thread, rather then one or two posts.  So if the off-topic drags on for to long, send a pm and I'll happily split it.  And try to remember that we're all working towards the same end goal.  dfort isn't here to ruin the party, take all of the accolades or all of the attention, he seems to have some good amount of spare time atm, and he also seems to be dumping all of that time into ML.  This is a good thing for the project, and in my eyes, the project always comes first.  Take advantage of his time and knowledge, bug him about coding stuff.  Bug him for knowledge.

And the guys with all the know how need to remember that things that might seem easy, or common sense, generally aren't so easy for us plebes. (https://www.urbandictionary.com/define.php?term=plebe)  And sometimes we're happy just to plod along at our own slow pace, despite the common sense.  If that's frustrating for you, well, what goes around comes around.   :P

I get email notification about PM and forum reports, and always check emails.  I can find time if I'm really needed.  I'm here to help.  If you need to vent, pm me, make it obvious you need to vent.  Then vent.

And lastly, for the sake of clarity, deleted threads only disappear from a users point of view, mod+ have access.  Keep that in mind if you accidentally delete something, it's not gone, just pm me with the details.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ItsMeLenny on January 07, 2018, 02:16:51 AM
Was anything happening with lossless compressed raw on 550D?
And I can't find the main threads on compressed raw anymore, was it ever applicable to anything less than 14bit?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on January 07, 2018, 02:35:49 AM
Quote from: ItsMeLenny on January 07, 2018, 02:16:51 AM
And I can't find the main threads on compressed raw anymore

Top of page -> Downloads -> Downloads (pulldown menu) -> Experiments -> 4K raw video recording; lossless compression -> Forum discussion
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 26, 2018, 06:06:26 AM
@a1ex when you found the 3072 vertical resolution for the preset in this code from adtg_gui.c-1121 (https://bitbucket.org/hudson/magic-lantern/src/50d8f06018dc52f72f04265432dd83106e495e85/modules/adtg_gui/adtg_gui.c?fileviewer=file-view-default#adtg_gui.c-1121)
static int res3k_reg(int reg)
{
    if (regs[reg].dst == 0xC0F0)
    {
        switch (regs[reg].reg)
        {
            case 0x6804:                /* C0F06804 - raw resolution */
                return 0x52801AB;       /* from 0x528011B -> 3072px in raw_rec */
        }
    }

In my code I chanced "0x60804"(digic v) to "0x6084"(digic iv) before I test it l was hopping to find the right size for the 3k preset as you have "0x52801AB"
was this found thought testing different registers or for investigating Log files ?
I wondering if this is the reason I'm lock to resolution 3584x1068 .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on January 26, 2018, 08:51:51 AM
I've used trial and error at first, then figured out the math.

From raw.c, raw_lv_get_resolution: you have 0xC0F06800/4 <-> 0xC0F06084/8.

Vertical units (high half of the 32-bit value) are pixels; horizontal units are probably number of column groups (some columns are probably read out in parallel, so you can't get an arbitrary number of columns; it has to be multiple of column_factor = 2 on 5D2 and 8 on 5D3).

For the full-res LiveView, I tried to mimic the photo mode configuration. That didn't work out of the box, so I've tried to extend the resolution in small steps, until hitting some barrier (such as getting invalid image data or black bars past some resolution); then I've tweaked other registers that seemed to be size-related, until breaking these barriers. The option to diff video modes from adtg_gui was very helpful; so was the option to lock the modified registers, effectively attempting to turn one video mode into another; then, from these modified registers, I've tried to find a minimal subset that does the job (figuring out the meaning of these registers in the process). That's the general workflow; here's a step by step example (http://www.magiclantern.fm/forum/index.php?topic=10111.msg191218#msg191218).

Math for the above value:
- default raw buffer size on 5D3: 2080x1318 (see this post (http://www.magiclantern.fm/forum/index.php?topic=16054.msg195213;topicseen#msg195213), in particular, this file (http://a1ex.magiclantern.fm/bleeding-edge/raw/raw_res.txt))
- in 1080p (the video mode I've started from), the values of these two registers are 0x10017 and 0x528011b => 2080x1319;
  had to subtract 1 to address an issue with invalid frames; explained here (https://bitbucket.org/hudson/magic-lantern/pull-requests/897).
- out of these 2080 pixels, 146 are skipped from the left (optical black) and 2 from the right; that leaves 1932 active pixels;
  rounded to multiples of 8 to work on whole raw_pixblock's => 152+8 skipped, 1920 left
- (0x1AB - 0x17) * 8 - 160 = 3072.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 26, 2018, 10:10:45 AM
Didn't read the previous post correctly
sorry typo , It should had been 0x6804 for the source .
Thank you , That helps clear up things  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 27, 2018, 06:23:13 AM
I think I may have proof now that my custom buffer is correct for Full Frame (5792x3804 -- includes out of bound area) in Video Mode see screen shot below

(https://preview.ibb.co/mtzyOb/VRAM11_full_live_view_buffer.png) (https://ibb.co/hQeyOb)

I did some logging with one of my memspy build a few months back with some screen shots , never really noticed it before until now.
in the red box you see resolution changed -- in the yellow box is the buffer size 2040x1267 -> in the green box 5792x3804 is the sensor size including out of bound area .
this buffer size (2040x1267) is what I use to get 3584x1068 in video mode with the my really , really ugly hack of the 5d3 3k preset .
I guess I can assume the size is correct and this here (http://a1ex.magiclantern.fm/bleeding-edge/raw/raw_res.txt) seem to back it up , 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on January 27, 2018, 07:57:30 AM
@reddeercity,
I've set up my Compiling System via dfort Mac tutorial, also clone dfort and I've test to see if my building Build work and every thing look like its work. Only thing I change was in raw.c was
#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0x1000)
#endif

to

#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0xF000)
#endif

by the way is that correct

I'm going to test it now, but what can I do to help out or get to your latest stable build?

by the way when I put code below in Terminal it gave me 2 build

cd Magic-LanternCloneRepository/platform/5D2.212
make zip
cp *.zip ~/Desktop
make clean
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 03, 2018, 05:51:58 AM
Came across some Ti (Texas instruments) chip info from a screen shot , it's the raw pipe with internal 10 & 12 bit
this is helping me understand what going on , for digic iv UHD

(https://thumb.ibb.co/nHhUER/cmos2raw_flow.jpg) (https://ibb.co/nHhUER)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 10, 2018, 03:33:49 AM
Reading my decompile rom in Microsoft  Visual Studio , very nice to use  :)
For some info found this
ff0322b4 > LimitRawSize [%d]
So I guess there must be some limits for processing image , would be nice to know what it is
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 10, 2018, 03:41:34 AM
Some liveview raw stuff from rom I think
ff034b80 Enable AE Sequence = %1d
ff034b9c Enable AF Sequence = %1d
ff034bb8 Output AF RAW Data = %1d
ff034bd4 Output AE RAW Data = %1d
ff034bf0 Output DEFM Raw    = %1d
ff034c0c Output RSHD Raw    = %1d

so when the af & ae sequence are enabled there are output to more then one regs
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on March 22, 2018, 12:02:29 PM
I try to increase the size of 5D2 and i have some success without using adtg_gui.mo.
As soon as possible I hope come back  to post all my work on the 5D2.
For the instant I have a problem for increase height.

For Digic register like  C0f06084/88  i use EngDrvOutLV(digic_register, new_val) and it' s OK . The new value is correctly apply.

But i need to tweak C0F0713C/50 too for increase height , so i tried different solutions but the value of digic register don't change :

* in ROM1.BIN.dis (5D2)
   With C0F0713C for example:


ff186ee4: e51f0064 ldr r0, [pc, #-100] ; ff186e88: (00007d9c)
ff186ee8: e59f11c8 ldr r1, [pc, #456] ; ff1870b8: (c0f0713c)
..................
ff986ee4: e51f0064 ldr r0, [pc, #-100] ; ff986e88: (00007d9c)
ff986ee8: e59f11c8 ldr r1, [pc, #456] ; ff9870b8: (c0f0713c)


So i believe understand that  instructOri = e59f11c8 and try to increase it with InstructNew= e59f11dc
With this change I hope to transform  ldr    r1, [pc, #456]         in        ldr      r1, [pc, #476]   but this don't  works.

It is possible with adtg_gui.mo but there is too much troubles , please someone could explain me how to write in this registers (c0f0713c/50) ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 05, 2018, 10:28:35 PM
Hey waza57 , cool  :)
can you share some more details ? looks interesting -- can you save a dng?
I can't answer your question about "write in this registers"
I did all my stuff with the iso_research branch , which included 3k preset with crop_rec (1:1)
I didn't have any problems or issue with adtg_gui.mo , what was the problem , enabling changed registers valves ?

Found a tool to help with find fps head timer numbers , thou it have more uses then that
eostimergen (http://www.g3gg0.de/wordpress/projects/eostimergen/)  :)) Thanks @g3gg0

QuoteEOSTimerGen is a tool for Magic Lantern users that want to
a) over/undercrank the camera (to get higher or lower frame rates)
b) fine tune their frame rate to overcome production tolerances
c) synchronize two or more cameras to have nearly the same rate


(https://image.ibb.co/eZeAxH/EOS_Video_Time_Gen.png) (https://imgbb.com/)

You can do that with frame over ride but nice to do it in software to check it .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on April 16, 2018, 09:44:44 PM
reddeercity, Thanks for that but it seems timer B don't increase height like with other models.

I tried a lot of things but nothing works.

Perhaps ADTG [8xxx] but unfortunately I don't succeed to see it with adtg-gui module in 5D2.

An example :
(https://thumb.ibb.co/b79O2S/M17_0638_00000.png) (https://ibb.co/b79O2S)

how to delete mint account (https://deleteacc.com/mint)


effective width: 2840
effective height: 1082

http://www.adapteo.fr/images/M17-0638_000000.dng (http://www.adapteo.fr/images/M17-0638_000000.dng)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on April 16, 2018, 10:40:16 PM
OMG 😍😍😍!!! waza57 your the person that made video dual ISO on 5D2 almols work rigtht??!

Can you record MLV yet with this confuguration? If not, how close is it??? And how could I help? The goal would be to put it in a single menu option in a normal build, how much work is that? I would love to help if possible. Or test it  ;) ;) ;) ;) ;) (as every noob always says)

I'm so excited!!! 2.8K—that's Alexa sensor resolution (it must be a sign)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 17, 2018, 12:47:43 AM
Quote from: waza57 on April 16, 2018, 09:44:44 PM
reddeercity, Thanks for that but it seems timer B don't increase height like with other models.
I tried a lot of things but nothing works.
Perhaps ADTG [8xxx] but unfortunately I don't succeed to see it with adtg-gui module in 5D2.
try maybe ADTG1 [105f]N , or the ADTG2 -- one is for 3xcrop mode & the other is for 1:1 FHD
I've had some success increasing a little height
I most I've been able to get is 1068(h) with the (w) 3584
I see the frame rate is 23.614 with exiftool , this maybe the issue -- the other test I've seen on digic4 have frame rate at 12fps
Have you tried to slow the frame rate down ? Did you try with frame override enabled or disabled ?

Very interesting @waza57 -- can you share your code yet ? I would like very much to look at it , if you don't want to publicly display it pm me  :)

I've being going with just modifying CMOS 1 & 2 registers with the Crop_Record Module loaded , it seem to put the sensor in 1:1 pixel mode centered
then enable the 5d3 3K preset ( the reg's are very close to 5d2 and it kind of works , I have a timer problem now) 
Did you get this in 3xCrop mode or 1:1FHD ?
Good job  :))

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 20, 2018, 06:36:49 AM
@ waza57 great ! you got your code up crop_rec_4k_5D2  (https://bitbucket.org/waza57/magic-lantern/commits/f73a13b72e7fb4e7757ab6547a7685d6dcbc5216) -- I see you are extending the 3xCrop windows , interesting then for sure look at the those  ADTG1 [105f]N , &  ADTG2 in the adtg_gui.mo
I played around with then in 3xcrop_mode .
If this so , then the old  10bit without raw_slurp will work in 3xCrop_mode with that extended resolution .
I coming from it thought 1:1 FHD with the crop_record.mo -- it put the sensor into every line , by passes the line skipping ,
I'll have a closer look at the code  , maybe we can have a special "4k crop_record.mo " for digic4  ;) 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on April 23, 2018, 11:22:30 PM
@reddeercity have you tested the 2.8k yet??? I really wanna see some nice samples of 5D mark ii 2.8k!!! It will give me motivation to fix my broken 5D2

I'm truly excited to see if anything finally happens with this camera.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 24, 2018, 06:53:59 AM
@Ilia3101 no not yet , but I'll give it try soon.
I'm actually trying to find out why HDMI is broken in the nightly builds (unified) no one has reported it :(
and as far as I can tell it's been broken since the "New Core" (I use the Old Core for production work I must have
HDMI for my production workflow) , so I'm undecided to which way to go . I can implement 10-12bit
and my experiments in UHD plus waza57 code in to the "Old Core" I'm starting to see too many restriction with the new core as
it must be compatible with Digic 5 Camera (5D3) and I'm not to sure about the Digic's 4 (5D2) life expectedly or support .
I know it sound discouraging , but I have to wonder if all this time & effort is really worth it after all .
I can deal with a limited 1:1 resolution but not HDMI , that the deal breaker .
So I need to recess this thread in the direction it goes .
Sorry to be off topic a bit .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 25, 2018, 07:02:22 AM
Quote from: waza57 on April 16, 2018, 09:44:44 PM
reddeercity, Thanks for that but it seems timer B don't increase height like with other models.
I tried a lot of things but nothing works.
Perhaps ADTG [8xxx] but unfortunately I don't succeed to see it with adtg-gui module in 5D2.
An example :
(https://thumb.ibb.co/b79O2S/M17_0638_00000.png) (https://ibb.co/b79O2S)
effective width: 2840
effective height: 1082
Being trying to reproduce this from @waza57 but no go , I clone crop_rec_4k_5d2 branch and update to comment f73a13b (https://bitbucket.org/waza57/magic-lantern/commits/09dfad1ad108fbe480077aab0bf39f418cc2cda2) -- I can load crop_crop.mo but it's disable , the mlv_lite and adtg_gui.mo didn't make .
So can you tell me how you got this please ? was this on mlv_lite or mlv_rec ? I see in the "Digic Poke" where you code is , can you share the setting in there ?
In the mean time I'll play with more , maybe by chance i can find the right setting to reproduce this  :D
Edit: your working in this right ?
(https://thumb.ibb.co/fufTmx/VRAM0.png) (https://ibb.co/fufTmx)
Edit:2 it seem I'm still missing some code in raw.c 9a0b318 (https://bitbucket.org/waza57/magic-lantern/commits/9a0b318953a02cc12adc09e44253c5772abeb939) plus some comment's I missed , I thought the f73a13b (https://bitbucket.org/waza57/magic-lantern/commits/f73a13b72e7fb4e7757ab6547a7685d6dcbc5216) comment was the consolidation of the code from waza57 , I'll continue tomorrow


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 26, 2018, 08:03:18 AM
I'm getting some error compiling now in raw.c , I'm not sure why
../../src/raw.c: In function 'raw_lv_get_resolution':
../../src/raw.c:682:5: error: invalid storage class for function 'raw_update_params_work'
int raw_update_params_work()


int raw_update_params_work()
anything to do with "invalid storage class for function 'raw_update" etc. .....
Can some one please help explain this error .
I'm working off waza57 "crop_rec_5d2"
https://bitbucket.org/waza57/magic-lantern/commits/9a0b318953a02cc12adc09e44253c5772abeb939#Lsrc/raw.cF678T678
Edit: Here my source code I'm working off I uploaded to my bitbucket downloads
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2018-4-26-crop_rec_5d2.zip
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 03, 2018, 07:53:47 AM
I put aside "waza57 crop_rec_5d2" code on hold and when back my code and my some improvements (It's gets better)  :)
in raw.c I added
#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0xF000)
#end

In adtg_gui.c I changed 0x6800 -->0x6084/8

static int res3k_reg(int reg)
{
    if (regs[reg].dst == 0xC0F0)
    {
        switch (regs[reg].reg)
        {
            case 0x6084/8:                /* C0F06084/8 - raw resolution */
                return 0x52801AB;       /* from 0x528011B -> 3072px in raw_rec */
        }
    }

and added in mlv_lite.c the extended resolutions
3840, 4096


(https://thumb.ibb.co/itnjpS/VRAM1.png) (https://ibb.co/itnjpS)

(https://thumb.ibb.co/b44aw7/VRAM5.png) (https://ibb.co/b44aw7)
 
(https://thumb.ibb.co/d4fyG7/VRAM4.png) (https://ibb.co/d4fyG7)

I was able to extend the horizontal to1076 from 1044 , not much but a start  :)
I did dump the image buffer this what I'm getting , (please remember I did not do any timer adjustment yet)
The HD Dump
(https://image.ibb.co/nkzEpS/HD_000_Small.png) (https://imgbb.com/)
The LV Dump
(https://image.ibb.co/b51c9S/LV_001_Small.png) (https://imgbb.com/)
The second HD Dump after trying to move the image down to center
(https://image.ibb.co/jSa4pS/HD_001_Small.png) (https://imgbb.com/)
Playing around with liveview preview
(https://image.ibb.co/dw5yG7/LV_000_small.png) (https://imgbb.com/)
Screen shot -- I didn't do any timer adjustment yet
(https://image.ibb.co/deiJG7/VRAM3_Small.png) (https://imgbb.com/)

Here the build I compiled (the latest updated source + ISO Research)
# Magic Lantern Nightly.2018May03.5D2212 (f683753bc4e3+ (iso-research))
# Built on 2018-05-03 03:54:46 UTC by ml@ml-pc
# Configuration saved on 2018/05/02 22:40:24
beta.warn = 2
menu.first = -4
movie.log = 1
rec.notify = 0
enable-liveview = 2
fps.preset = 1
fps.override.idx = 32
fps.override = 1
battery.drain.rate.rev = 59
hist.log = 0
spotmeter.draw = 0
clear.preview = 4
zebra.draw = 0
disp.mode.x = 149


So What I did was load the adtg_gui.mo ,  mlv_lite.mo  &  Crop_Rec.mo with crop_rec I put the cam in to 1:1 3x mode so every pixel & line (by pass line skipping)
then loaded the 3k 5d3 preset but I changed the regs as noted above .
It changed CMOS[1]0xc00 --> 0x2cb & CMOS[2] 0x40e --> 0x10e
I'm able to switch between all res's up to 4096 (I added 3840 & 4096) yes I can set it to 3520 also , before I was lock to just 3584 and thou the cmos[2]
I was able to change the  horizontal by 32 lines , (not much but a start)

I really just simplified the code after learning what I read in the 10-12bit thread  ;D
I think this is the right path , though the registers just like the 5d3 only make sense being there's so closely related -- brother from a different mother  :D
more to come , next is to clean the raw image up and start recording a image only if for a few second until the buffer is full then I need to finish the lossless compressed raw
code in silent .c
The last thing is the content of MENUS.CFG
0 3F77F0FE 3D41FF14 Overlay\Zebras
0 3F783078 3D556563 Overlay\Spotmeter
0 409A2D5E 3E935DBF Movie\FPS override
0 3FF3E2C6 3CA1335E Movie\Crop mode
0 409BA922 3F3448D6 Movie\RAW video
0 3F711935 3B573CCE Movie\Movie Tweaks
0 3FFA4E97 3E70270A Display\Clear overlays
0 407B0F0F 3F208C48 Display\Anamorphic
0 3FFB325A 3EF96A01 Debug\Memory patches
0 3FF8BDA4 3EF33695 Debug\Show console
0 409C1067 3F4B1C91 Debug\ADTG Registers
0 3F7308B9 3BE6B0C1 Debug\Lens info
0 40BC1075 40007E74 Debug\Screenshot - 10s
0 3FFA92E5 3EDB7170 Debug\Dump image buffers
0 3F7346F1 3BFDC26F Debug\Don't click me!
0 3F7E7825 3F10815C Debug\Free Memory
0 3F7156EE 3B6CC2E3 Movie Tweaks\Movie Logging
0 3F7194B7 3B8237FD Movie Tweaks\REC/STBY notify
0 3F71D290 3B8F3D97 Movie Tweaks\Force LiveView
0 3F72CA91 3BD1B7F5 FPS override\Desired FPS
0 3F728C79 3BBEA73B FPS override\Optimize for
0 3F787002 3D6ABC54 Histogram\Scaling
0 3F75B891 3CA48BF9 Advanced\DIGIC Registers
0 3F78EF46 3D8E03DC Advanced\Log Registers Now
0 3FF9750E 3EAE7395 Advanced\1:1 3K crop mode (5D3)
0 4079DAD8 3FA5BBBF RAW video\Resolution
0 3FF5D867 3D364505 RAW video\Aspect ratio
0 3FFF9DD2 3FDE2613 RAW video\Preview
0 3F747EF9 3C4C5756 RAW video\Advanced...
0 3F7EB94A 3F1EF4B2 Free Memory\Allocated RAM
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 04, 2018, 08:12:50 AM
some progress cleaning up Lv @ 4096x1044

Image dump of Liveview in "framing preview"
(https://image.ibb.co/mxi5yn/LV_003_small.png) (https://imgbb.com/)
adjustments CMOS[1]  with ADTG, 0x106E, Shutter blanking for LiveView 1x
(https://image.ibb.co/kv0qW7/LV_002_small.png) (https://imgbb.com/)
Screen capture Liveview
(https://image.ibb.co/he6fW7/VRAM9_small.png) (https://imgbb.com/)

Even thou it in 4096x1044 It seem it retaining the 1856 buffer & just mirroring  it
maybe my size is not right but with SRM buffer my understanding there should enough 32mb per frame , right ?
RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0xF000)
I thing this what keeping me from saving a DNG on dumps , it does export a DNG but it's empty
I'll continue tomorrow .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 05, 2018, 06:00:11 AM
Still haven't got the raw image right yet but I can increase the vertical to 1250 , so that's 4096x1250 .
It's was a matter of just changing Aspect Ratio to 1:1 or 3:2 .
(https://image.ibb.co/hVPXeS/VRAM12_Small.png) (https://imgbb.com/)
Image dump
(https://image.ibb.co/j21pR7/LV_005_Small.png) (https://imgbb.com/)

Edit: Gave a go at 3x crop mode to see if I could get something there got a interesting result
(https://image.ibb.co/ciEeB7/VRAM16_Small_3xcrop.png) (https://imgbb.com/)
Just played around with c0f0 0x6084 & c0f0 0x6088 reg's and when to the change the res in the raw video menu .
I don't think there any there yet but interesting

Edit:2 I thing I have to rework the buffer or I need to fully implement raw_slurp  with SRM memory
I can easily put the sensor in to 1:1 without the 3k preset in adtg_gui.mo
cmos[1] 0xc00 --> 0xbc0  (height)
cmos[2] 0x40e --> 0x10e (width)

With that I choose up to 4096x1250 , I got that by just switching in to 3x crop_mode and reading what the reg's at cmos 1&2  are.
plus I think I found the reg's for c0f0 0x6084 & 0x6088 in 3x crop_mode
c0f0 0x6084 --> 0x10036 (1:1 FHD)
c0f0 0x6084 --> 0x30036 (3x crop_mode)
c0f0 0x6088 --> 0x4f40432 (1:1 FHD)
c0f0 0x6088 --> 0x46a04ba (3x crop_mode)

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 06, 2018, 06:26:27 AM
Having some trouble get the custom buffer to compile for the higher res redirect using srm memory  as per a1ex  talks said here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg183374#msg183374)  I found the comment  crop_rec_4k#Lsrc/raw.cT149 (https://bitbucket.org/hudson/magic-lantern/commits/7ad3754768b26a106d057550ca359089ba837fcf?at=crop_rec_4k#Lsrc/raw.cT149)  and I finished implmenting the rest of raw_slurp for srm memory as per what a1ex has done for the 5d3 for 4k . So my error is this
src/raw.c:578:39: error: 'DEFAULT_RAW_BUFFER'...........
see below -- from the terminal window
../../src/raw.c:161:0: warning: "RAW_LV_BUFFER_ALLOC_SIZE" redefined [enabled by default]
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0x1000)
^
../../src/raw.c:147:0: note: this is the location of the previous definition
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0xF000)
^
../../src/raw.c: In function 'raw_lv_realloc_buffer':
../../src/raw.c:578:39: error: 'DEFAULT_RAW_BUFFER' undeclared (first use in this function)
         if (raw_lv_buffer != (void *) DEFAULT_RAW_BUFFER)
                                       ^
../../src/raw.c:578:39: note: each undeclared identifier is reported only once for each function it appears in
make: *** [raw.o] Error 1

if you have trouble following what posted from the raw.c I uploaded the whole file to my bitbucket downloads folder
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/adtg_gui_4k_raw_slurp_redierct_buffer-raw.c
here  part of the code, I hope someone can help me understand the problem
I added more code then needed  so everyone can get the idea of with happening

  71- /*********************** Camera-specific constants ****************************/
72-
73- #ifdef CONFIG_EDMAC_RAW_SLURP
74- /* undefine so we don't use it by mistake */
75- #undef RAW_LV_EDMAC
76-
77- /* hardcode Canon's raw buffer directly */
78- /* you can find it from lv_raw_dump, arg1 passed to dump_file:
79-  *
  80- * raw_buffer = get_raw_buffer()
  81- * sprintf_maybe(filename, '%08lx.mm1', raw_buffer)
  82-  * ...
  83-  * dump_file(filename, raw_buffer, 7*something...)
  84- */
..................
146- #ifdef CONFIG_5D2
147- #define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0xF000)
148- #define DEFAULT_RAW_BUFFER_SIZE (9*1024*1024)     (I added this as I thought it be big enough , not too sure thou -- is this my problem  not sure how to find it out )
149- #endif
150-
151- #ifndef DEFAULT_RAW_BUFFER_SIZE
152- /* todo: figure out how much Canon code allocates for their LV RAW buffer - how? */
153- #warning FIXME: using dummy DEFAULT_RAW_BUFFER_SIZE
154- #define DEFAULT_RAW_BUFFER_SIZE (9*1024*1024)
155- #endif
156-
157- /* for higher resolutions we'll allocate a new buffer, as needed */
158- /* all cameras using CONFIG_EDMAC_RAW_SLURP should be able to handle this */
159- /* SRM_BUFFER_SIZE matches the full-res image size, as 14-bit uncompressed (actually a bit larger, but not much) */
160- #define CONFIG_ALLOCATE_RAW_LV_BUFFER
161- #define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0x1000)
..........................

549- #ifdef CONFIG_EDMAC_RAW_SLURP
550-
551- /* requires raw_sem */
552- static void raw_lv_free_buffer()
553- {
554-   printf("Freeing LV raw buffer %x.\n", raw_lv_buffer);
555-   if(raw_allocated_lv_buffer) {
556-        free(raw_allocated_lv_buffer);
557-        raw_allocated_lv_buffer = 0;
558-   }
559-   raw_lv_buffer = 0;
560-   raw_lv_buffer_size = 0;
561- }
562-
563- /* requires raw_sem */
564- static void raw_lv_realloc_buffer()
565- {
566-   int width, height;
567-  int ok = raw_lv_get_resolution(&width, &height);
568-  if (!ok)
569-  {
570-      ASSERT(0);
  571-     return;
572-   }
573-
574-    int required_size = width * height * 14/8;
575-    if (DEFAULT_RAW_BUFFER_SIZE >= required_size)
576-   {
577-      /* no need for a larger buffer */
578-      if (raw_lv_buffer != (void *) DEFAULT_RAW_BUFFER)     ----- (this where the error is  not sure to resolve it )
579-      {
580-          printf("Default raw buffer OK for %dx%d (%s)", width, height, format_memory_size(required_size));
581-
582-          if (raw_lv_buffer && raw_lv_buffer == raw_allocated_lv_buffer)
583-          {
584-             printf(" - back to default.\n");
  585-            raw_lv_free_buffer();
  586-         }
  587-         else if (raw_lv_buffer)
  588-         {
  589-               printf(": %x -> %x\n", raw_lv_buffer, DEFAULT_RAW_BUFFER);
   590-        }
  591-         else
  592-         {
  593-              printf(".\n");
  594-           }
   595-       }
   596-
   597-      raw_lv_buffer = (void *) DEFAULT_RAW_BUFFER;
   598-      raw_lv_buffer_size = DEFAULT_RAW_BUFFER_SIZE;
   599-      return;
   600-   }
   601-
   602-    if (raw_lv_buffer_size >= required_size)
   603-    {
   604-          /* no need for a larger buffer */
    605-       return;
    606-    }
    607-
    608-   printf("Default raw buffer too small (%s", format_memory_size(raw_lv_buffer_size));
    609-   printf(", need %dx%d %s) - reallocating.\n", width, height, format_memory_size(required_size));
    610-
    611-   if (raw_lv_buffer && raw_lv_buffer != (void *) DEFAULT_RAW_BUFFER)
    612-   {
    613-       ASSERT(0);
    614-        return;
     615-   }
    616-
    617- #ifdef CONFIG_ALLOCATE_RAW_LV_BUFFER
    618-         raw_allocated_lv_buffer = fio_malloc(RAW_LV_BUFFER_ALLOC_SIZE);
    619-         raw_lv_buffer = raw_allocated_lv_buffer;
    620-         raw_lv_buffer_size = RAW_LV_BUFFER_ALLOC_SIZE;
    621-         return;
622- #endif /* CONFIG_ALLOCATE_RAW_LV_BUFFER */
623-
624-          /* you should enable CONFIG_ALLOCATE_RAW_LV_BUFFER
625-           * or find some other way to reserve memory for the RAW LV buffer */
626-          ASSERT(0);
627- }
628-
629- #endif  /* CONFIG_EDMAC_RAW_SLURP */
630- #endif /* CONFIG_RAW_LIVEVIEW */
631-
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 09, 2018, 01:00:11 AM
It look like I have problems with the redirect buffer
crop_rec_4k#Lsrc/raw.c-614 (https://bitbucket.org/hudson/magic-lantern/commits/7ad3754768b26a106d057550ca359089ba837fcf?at=crop_rec_4k#Lsrc/raw.cF616T614)
if (raw_lv_buffer != (void *) DEFAULT_RAW_BUFFER)
I added
#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0x1000)
#define DEFAULT_RAW_BUFFER_SIZE (9*1024*1024).
#endif

Is there a way to find the "DEFAULT_RAW_BUFFER_SIZE" ?
I just copy the default values from
crop_rec_4k#Lsrc/raw.cT146 (https://bitbucket.org/hudson/magic-lantern/commits/7ad3754768b26a106d057550ca359089ba837fcf?at=crop_rec_4k#Lsrc/raw.cF154T146)
#define DEFAULT_RAW_BUFFER_SIZE (9*1024*1024)
#endif

but if I'm using "SRM_Buffer" do I really need the "Default_Raw_Buffer_Size"
this is confusing me , I did find the "working buffer size" in one of my logs but seem too small
1726C> GuiMainTas:ffa4522c:18:03: (PUB) SetImageWorkMemory 1:0x40d00000, 2:0x41700000
172A7> GuiMainTas:ffa45274:18:03: (PUB) SetImageWorkMaxPixel W:2200,H:1872,0x5e4340

as it for 2200x1872 pixels.

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 10, 2018, 06:31:43 AM
Will every think works as far as getting in to 1:1 pixel mode in FHD and ml can select any resolution from 1856 to 4096 up to 1:1 Aspect Ratio 
and it will be able (theoretical specking) to capture up full width 5634 . Very clever software , Magic Lantern !
I starting to see the genius of the code -- the whole infrastructure is very flexible , once I followed a1ex's comments , it's all very clear now :))  ,
the 5d2 works so very close to 5d3 but with I small difference really , e.g. , c0f0 0x6800 0x6804 (d5/5d3) c0f0 0x6088 0x6084 (d4/5d2)
Indeed the 5d3's has the same cmos [2] reg's (0x10E) ,  is the same as the 5d2 cmos[2] reg's (0x10E) for 1:1 pixel FHD and the vertical offset cmos[1]  is a little different
but close enough to get started on cleaning up the offset .
The height limitations is equal to the height of the image buffer , in my case it's still using either the full HD buffer (1880x1250) in 1:1 or 3x crop 2144x1076 and that's the new kettle of fish Has to be solved .
I reproduce every thing in 3xcrop_mode and can get up to 4096x1076 , but the height is lock to the 3x buffer , that why I can get 1250 vertical in 1:1 because the image buffer size ,
the default full raw buffer size need to be found so the redirect buffer code infrastructure can be used with SRM Memory like all the rest of the cam or almost all .
If that can't work then a custom redirect image buffer for d4 cam will need to be written , maybe similar to what I was working on  before raw_slurp
#ifdef CONFIG_5D2
#define RAW_LV_BUFFER_ALLOC_SIZE (2040*1267)
#endif

that's equal to the full image buffer full image buffer (https://www.magiclantern.fm/forum/index.php?topic=19336.msg196437#msg196437) on 5d2 .

I have a request from any d4 cam users (50d ,550d etc. ..) but I prefer 50d or 7d , I'm interested in if we can put others d4 cams in to 1:1 FHD 4k mode
you need ADTG_GUI.mo from the ISO-Research branch , the best way is to compile with the latest source +the iso-research branch .
some small mod's to adtg_gui.c  adtg_gui.c-1127 (https://bitbucket.org/hudson/magic-lantern/src/50d8f06018dc52f72f04265432dd83106e495e85/modules/adtg_gui/adtg_gui.c?fileviewer=file-view-default#adtg_gui.c-1127) change 0x6804 to 0x6084/8 & C0F06804 to C0F06084/8
and in mlv_lite.c add 3840 & 4096 to the list of resolutions that sure do it. If you can't compile of has a problem let me know here and I can provide a build.
What I'm looking for is the cmos[1] , cmos[2] & c0f0 0x6084 & c0f0 0x6088 you'll need to enable digic reg's and put the cam in to 3x crop_mode and record the setting at those reg's
c0f0 0x6084 should be something like 0x30036 in 3x crop mode & 0x10036 in 1:1 mode at least that what the 5d2 says & cmos[2] should be 0x10e in 3x crop 1:1 is 0x40e same thing
that the 5d2 info so if other cam report the same then we can port UHD or at least a true crop record of the sensor in 1:1 maybe 2.5k so who knows.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 14, 2018, 06:13:55 AM
Will I got the redirect buffer to compile in raw.c -- stupid mistake in the "#ifdef CONFIG_5D2" forgot to close the #ifdef with "#endif" :(
any how as I said before I implemented "raw_slurp" for "srm memory" for the redirect buffer . link to the raw.c file below if any one wants to follow along closer
redirect_buffer_4k_res_iso-research_branch_raw.c (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/redirect%20_buffer_4k_res_iso-research_branch_raw.c)
added
#ifdef CONFIG_5D2
/* for higher resolutions we'll allocate a new buffer, as needed */
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0x1000)
#define DEFAULT_RAW_BUFFER (4096*4096)
#endif

not sure if it working yet , just start to experimenting  , I don't think the "default_raw_buffer" is correct I just pick the biggest one there was I saw
in the EDMAC 4096*4096 . I did try 3574*1268 but when I enable raw vide(mlv_lite) all there was "0" or 0x0 for size so I changed to 4096*4096 and raw vide work again at
least before I enabled the adtg_gui.mo . Needed to be reworked more , I wanted at least at this point to be able too compile and it does so fine tuning  buffer sizes
and or finding the correct size is next .

the redirect buffer
/* requires raw_sem */
static void raw_lv_realloc_buffer()
{
    int width, height;
    int ok = raw_lv_get_resolution(&width, &height);
    if (!ok)
    {
        ASSERT(0);
        return;
    }

    int required_size = width * height * 14/8;
    if (DEFAULT_RAW_BUFFER_SIZE >= required_size)
    {
        /* no need for a larger buffer */
        if (raw_lv_buffer != (void *) DEFAULT_RAW_BUFFER)
        {
            printf("Default raw buffer OK for %dx%d (%s)", width, height, format_memory_size(required_size));

            if (raw_lv_buffer && raw_lv_buffer == raw_allocated_lv_buffer)
            {
                printf(" - back to default.\n");
                raw_lv_free_buffer();
            }
            else if (raw_lv_buffer)
            {
                printf(": %x -> %x\n", raw_lv_buffer, DEFAULT_RAW_BUFFER);
            }
            else
            {
                printf(".\n");
            }
        }

        raw_lv_buffer = (void *) DEFAULT_RAW_BUFFER;
        raw_lv_buffer_size = DEFAULT_RAW_BUFFER_SIZE;
        return;
    }

    if (raw_lv_buffer_size >= required_size)
    {
        /* no need for a larger buffer */
        return;
    }

    printf("Default raw buffer too small (%s", format_memory_size(raw_lv_buffer_size));
    printf(", need %dx%d %s) - reallocating.\n", width, height, format_memory_size(required_size));

    if (raw_lv_buffer && raw_lv_buffer != (void *) DEFAULT_RAW_BUFFER)
    {
        ASSERT(0);
        return;
    }

#ifdef CONFIG_ALLOCATE_RAW_LV_BUFFER
    raw_allocated_lv_buffer = fio_malloc(RAW_LV_BUFFER_ALLOC_SIZE);
    raw_lv_buffer = raw_allocated_lv_buffer;
    raw_lv_buffer_size = RAW_LV_BUFFER_ALLOC_SIZE;
    return;
#endif /* CONFIG_ALLOCATE_RAW_LV_BUFFER */

    /* you should enable CONFIG_ALLOCATE_RAW_LV_BUFFER
     * or find some other way to reserve memory for the RAW LV buffer */
    ASSERT(0);
}


In the edmac-memcpy.c -- added for raw_slurp (I just took the code from the  here (https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/15/searching-for-config_edmac_raw_slurp-on/diff)
/** this method bypasses Canon's lv_save_raw and slurps the raw data directly from connection #0 */
#ifdef CONFIG_EDMAC_RAW_SLURP
   ......
#elif defined(EVF_STATE)
uint32_t raw_write_chan = 0x12; /* 60D and newer, including all DIGIC V */
#elif defined(CONFIG_5D2)
uint32_t raw_write_chan = 0x02; /* testing 0x01 and 0x02 */
#endif


platform/5D2.212/consts.h
#define SRM_BUFFER_SIZE 0x25D0000   /* print it from srm_malloc_cbr */

platform/5D2.212/internals.h
/** this method bypasses Canon's lv_save_raw and slurps the raw data directly from connection #0 */
#define CONFIG_EDMAC_RAW_SLURP
#define CONFIG_ALLOCATE_RAW_LV_BUFFER

adtg_gui.c I changed c0f0 0x6804 to 6084/8
static int res3k_reg(int reg)
{
    if (regs[reg].dst == 0xC0F0)
    {
        switch (regs[reg].reg)
        {
            case 0x6084/8:                /* C0F06084/8 - raw resolution */
                return 0x52801AB;       /* from 0x528011B -> 3072px in raw_rec */
        }
    }

    return 0;
}

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 17, 2018, 01:50:19 AM
another piece of the puzzle of the redirect buffer
I just added
/* for higher resolutions we'll allocate a new buffer, as needed */
/* all cameras using CONFIG_EDMAC_RAW_SLURP should be able to handle this */
/* SRM_BUFFER_SIZE matches the full-res image size, as 14-bit uncompressed (actually a bit larger, but not much) */
#define CONFIG_ALLOCATE_RAW_LV_BUFFER
#define RAW_LV_BUFFER_ALLOC_SIZE (SRM_BUFFER_SIZE - 0x1000)

I updated the raw.c and it's on my bitbucket downloads I renamed it to adtg_gui_redirect_4k_buffer_raw.c (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/adtg_gui_redirect_4k_buffer_raw.c) for those who are following my progress .

I also found out how to find the "default_raw_buffer" -- compile the allocate-raw-lv-buffer branch with these comments from a1ex e799e89 (https://bitbucket.org/hudson/magic-lantern/commits/e799e89f67877825c6a4dc2868f87bed1196432b)
QuoteRaw backend -- attempt to autodetect Canon's raw buffer size
Usage:
- compile with CONFIG_MARK_UNUSED_MEMORY_AT_STARTUP (config-defines.h)
- open the console (Debug menu) and enable something that uses LiveView RAW features (raw video, raw histogram etc)
- test by starting the camera in all video modes (photo, 1080p, 720p, crop, x5 etc)
- take a screenshot or write down the console messages

This something all Digic 4 cam user can do easily and would help with other features beside 4k/UHD
It you do please just post result on this thread .
looking for :
5D2 -- (I should be able to get this one , but it won't hurt to post other results)
50D
7D
550d
600d (? maybe )
did I miss any ? for digic4
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 18, 2018, 08:20:22 AM
some progress in 3x crop_mode , did a image dump LV-005.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-005.422) LV-004.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-004.422) , starting to get the image to move to the left and push out the ghost image .
you can see the black border shrinking , tried to export the 422 image but the "YUV 422 Convertor" didn't recognizes the resolution but it could be view in the app.
most notable things I'm adjusting is the "A" &"B" Timers & the 3x shutter blanking & cmos[4]
screen shots of app with 422 dumps
(https://image.ibb.co/i88JAd/3xcrop_mode_3200x1076.png) (https://imgbb.com/)

(https://image.ibb.co/iNtqOy/3xcrop_mode_3200x1076_adjustments.png) (https://imgbb.com/)
yea some how I enabled dual iso with cmos [4]
And one last thing the A & B timers make it more stable , seems timer "B" with higher numbers (greater then 1910<) keeps the image in extended resolutions
so it's not only totally depended on just cmos's reg's , timers have a great deal to do with it also .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 15, 2018, 07:17:50 AM
I got QEMU up & running finally and i'm looking for some other d4 cam rom's to investigate .
mainly looking for 50D plus T3i (600D) T2i(550d) , T1i (500D)
I have 7D 203 (firmware)   rom . Just PM a link to the roms  , don't post  links here
Thanks. :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: histor on July 06, 2018, 10:04:24 PM
Just a strange bug in the broken3x-12bit build - I've seen it twice with different cards. It destroys the \ML\modules\ folder (or does something wrong to the filesystem). Finally it fails with "Can't find 5D2_212.sym"
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on July 06, 2018, 10:09:26 PM
5D2 handling takes special care. Never! remove card without waiting some seconds for write operations to complete. Yes, cam will access card *after* opening card compartment lid!
Are you sure you took that into consideration?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: histor on July 06, 2018, 10:24:54 PM
The card was in all the time. As well as battery. It's something build specific, I guess.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 07, 2018, 07:12:38 AM
Quote from: histor on July 06, 2018, 10:04:24 PM
Just a strange bug in the broken3x-12bit build - I've seen it twice with different cards. It destroys the \ML\modules\ folder (or does something wrong to the filesystem). Finally it fails with "Can't find 5D2_212.sym"
Thanks for the feed back on the bleeding edge build (not for the light hearted) 
What @Walter said still hold true , even if you open the cf card door to fast while the red light still blinking (writing file for ml) you can get wipe out files or corruption
Yes ,  I've seen this happen before on just the plain nightly builds , but i find it's usually a user error . Pressing  buttons/switch's to fast or too many times , etc. ....
But to really know I would need more info , like all the cam setting , any error messages , etc. ...... , If you clear the setting file in ML on CF card does it still happen ?
If you put the card in your reader , are all the ML files still there ? after ml can't find the .sys file ?
What CF cards are you using ? (needs to be @ least 1000x)

To be clear , this build has limited feature support . That means only basic functions , e.g. all manual adjustment not Auto feature.(frame over ride is OK to use )
Cam must set to manual M mode and Only use pure analog ISO's (100,200,400,800) not digital ISO's (160,320,500,600,etc. ...)
You must turn off(disable) mlv_rec and or mlv_lite before you adjust ISO's or it will crash/lockup .

One more thought , after you get the "Can't find 5D2_212.sym" turn cam off and pull the battery out and try a reboot from cold.
Does it work now ? I've sometimes seen ML miss loading files like "crop marks" or module file , I just pull battery out and then reboot
all is fine usually after that .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: histor on July 07, 2018, 02:03:08 PM
I've packed logs and settings. http://disk.karelia.pro/crfYi80/broken3x-12bit-bug.zip
In short - files from the build stay unaltered but all modules vanished. That's what cart reader sees. While previous crash I saw some alien characters instead of the filenames. Windows can do nothing with them but format.  The second time I've just copied the Modules\ back – and everything works with the same settings.
What camera sees when  the error occurs:  the message about 5D2_212.sym and no Global draw overlays. You can shoot, can't call ML menus. It's interesting, what was before that, but I can't roll my mind back.
There was no intensive card writing, the cards are OK (SanDisk Extreme Compact  SDCFXSB-128G-G46 and smaller SDCFXSB-064G-G46). No video recording, only lazy taking pictures in Live view. ISO1000 is a bad setting, but it doesn't stop taking stills or working live view. Another bad setting is leaving mlv_lite, mlv_rec and mlv_snd enabled together. That's a cruel way to get 10bit raw with sound. And that's not a sound of explosion : ) Is there any legal way to do it?
In fact 10-12bit is a fascinating feature for 5d2 and I'm watching  every reddeercity's  post with great attention. I'm not afraid of multiple crashes now, I have to accommodate with the limitations - to avoid them in the field.
Maybe that's not very useful information, it's hard to trace it from actual user error to the final result. My only question is: why ML rewrites modules on shutdown? and what for?
To end with something good – I'm actually charmed with ML for a decade.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on July 07, 2018, 02:22:54 PM
Can you find a way to reproduce the issue? Can you do the same without entering LiveView? Can you narrow it down to some particular module?

Can you get the same issue with regular builds?

ML does not rewrite modules on shutdown; it just saves its settings. However, if there's a bug that causes some sort of memory corruption, its side effects can be just about anything. I'm unable to find other explanation for this behavior.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: IDA_ML on July 07, 2018, 02:30:36 PM
Quote from: histor on July 07, 2018, 02:03:08 PM
In fact 10-12bit is a fascinating feature for 5d2 and I'm watching  every reddeercity's  post with great attention.

So am I.  I have the 7D and the footage that it provides at 2520x1200 resolution and 10 and 12 bits is absolutely gorgeous.  It has this very exciting film look in it that, in my opinion, only the 7D has.  Don't know about 5DMkII.  I do hope that Reddeercity, with the help of other developers,  succeeds with his most valuable work in porting 2,5, 3 and 4k-crop recording to the 7D.  I keep my thumbs pressed and watch this development with great anxiety!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 12, 2018, 09:48:21 AM
Being spending time working new angles to get UHD cleanly , I thought I would try and reproduce waza57  (https://www.magiclantern.fm/forum/index.php?topic=19336.msg198809#msg198809) experiment in 3x crop mode which he got 2880(V)
-- to date I still can't reproduce his work (I need his variables for reg's 6084 & 6088) . In any case I pushed on ,
I being experimenting with the dm-spy build (has a menu option to select reg's and change them with half shutter button)
-- so find the reg you what to change e.g. (c0f06084 , etc. ....)
(https://image.ibb.co/muotkp/VRAM3_small.png) (https://imgbb.com/)

Play around with 6084 & 6088 , I noticed that in 3x crop if I change the reg 6084 from 30036(default) to 30037 or 30038 I can chose extended rez's e'g' "3520"
but not 30036 thou the image is distorted at reg 30038
(https://image.ibb.co/dzAg5p/VRAM6_small.png)3520x1076 (https://imgbb.com/)

I still can't save a dng/mlv at this point though I can dump the Vram and get images from Liveview.
Things took a turn for the better when decided to check out the reg's in 10x zoom . I noticed that most the reg's are
the same as 3x crop as far as 6084 & 6088 are concerned but
10x reg's from dm-spy
[REG] @@@@@@@@@@@@ Start ADTG[1]
[REG] ADTG:[0x105f0301]
[REG] ADTG:[0x10610301]
[REG] ############ Start CMOS
[REG] CMOS:[0x20b]
[REG] CMOS:[0x1e6a]
[REG] CMOS:[0x210e]
[REG] CMOS:[0x3005]
[REG] CMOS:[0x4242]
[REG] CMOS:[0x5c01]
[REG] @@@@@@@@@@@@ Start ADTG[2]
[REG] ADTG:[0x14e00a]
[REG] ADTG:[0x150609]
[REG] @@@@@@@@@@@@ Start ADTG[3]
[REG] ADTG:[0x9011c]
[REG] ADTG:[0xb811c]


3x crop (5x zoom)
Start ADTG[1]
[REG] ADTG:[0x105f0301]
[REG] CMOS:[0x20b]
[REG] CMOS:[0x1e6a]
[REG] CMOS:[0x210e]
[REG] CMOS:[0x3005]
[REG] CMOS:[0x4242]
[REG] CMOS:[0x5c01]
Start ADTG[2]
[REG] ADTG:[0x14e00a]
[REG] ADTG:[0x150609]
Start ADTG[3]
[REG] ADTG:[0x93d1a]
[REG] ADTG:[0xbbd1a]


the differences is at "ADTG[3]"
0x9011c & 0xb811c (10x)
0x93d1a & 0xbbd1a (3x)
here the 10x3.5k_dm_spy_dm-0006.log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/10x3.5k_dm_spy_dm-0006.log) & 10x3.5k_digic11.LOG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/10x3.5k_digic11.LOG) , why I'm I calling this 3.5k LOG's ?
Will I can chose up to 3520 (V) in MLV_lite  :) right out of the box without any modification's to any reg's and I get a clean Image from vram for Liveview (still no dng , "raw error")
10x zoom mode Liveview dump , I selected 3520 in mlv_lite

Original LV-000.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-000.422) at 3520x1076

(https://image.ibb.co/mnrtJ9/LV_000_small.png) (https://imgbb.com/)

Original LV-001.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-001.422)

(https://image.ibb.co/mnrtJ9/LV_000_small.png) (https://imgbb.com/)

So I know that 10x zoom is disabled in raw.c if it's enabled can it record the 10x zoom Liveview ?
I think at 10x zoom is really  1:1 pixel , if not how could I select 3520 ? I notice in the dm-log that there is
2 different zoom mode for raw zoom mode "1" (3xcrop) & zoom mode "2" (10x zoom)
part of the log the deal with 10x image process
63A82> LiveViewMg:ffa09310:99:02: StartImagePass_x10
63AFC> LiveViewMg:00096224:00:00: *** StartEDmac(0x12, 0x2), from ffa093e8
63B2E> LiveViewMg:00096790:00:00:     addr 1b07800, ptr 1b07800, size
63B9F> LiveViewMg:00096224:00:00: *** ConnectWriteEDmac(0x5, 0x0), from ffa09418
63BFB> LiveViewMg:00096224:00:00: *** RegisterEDmacCompleteCBR(0x5, 0xffa08fa0 "WriteEDmacCrawCompleteCBR_x10", 0x0), from ffa09428
63C56> LiveViewMg:00096224:00:00: *** StartEDmac(0x5, 0x1), from ffa0944c
63C81> LiveViewMg:00096790:00:00:     addr 43e28a4, ptr 43e28a4, size
63CBE> LiveViewMg:ffa0946c:99:02: StartImagePass_x10 CrawAddr : 443e28a4 / KindOfCraw : 0
63D65> LiveViewMg:00096224:00:00: *** StartEDmac(0x8, 0x1), from ffa096b4
63D97> LiveViewMg:00096790:00:00:     addr 44afb4, ptr 44afb4, size
63DD9> LiveViewMg:ffa09df4:99:02: StartIntermediatePassMagnify Addr : 0x50000080
63E3E> LiveViewMg:00096224:00:00: *** StartEDmac(0x0, 0x0), from ffa09e18
63E66> LiveViewMg:00096790:00:00:     addr 10000080, ptr 10000080, size
63ED1> LiveViewMg:00096224:00:00: *** ConnectWriteEDmac(0x11, 0x19), from ffa0b5f4
63F04> LiveViewMg:ffa0b63c:99:02: StartQuarkYuvPass 0x5C578400
63F5C> LiveViewMg:00096224:00:00: *** StartEDmac(0x11, 0x2), from ffa0b660
63F82> LiveViewMg:00096790:00:00:     addr 1c578400, ptr 1c578400, size
6422E> LightMeasu:00096224:00:00: *** SetHPTimerAfter(0x61a8, 0xff86727c, 0xff86727c, 0x0), from ff867310
65BF4> **INT-6Ah*:00096840:00:00: *** TryPostEvent(LiveViewMgr, 0x2, 0x0, 0x0), from ff8d8828
65CBA> **INT-6Dh*:ff9a407c:00:02: [ENG] WriteDMACInterrupt(5)(0x2)(0)
65D01> **INT-6Dh*:00000558:99:02: WriteEDmacCrawCompleteCBR_x10
65D30> LiveViewMg:00096098:00:00: *** LVState: (2) --2--> (3)          ff8e17a8 (x=6bac4c z=0 t=0)
65D73> LiveViewMg:ff8e17c4:98:02: lvVDInterrupt
65DE1> LiveViewMg:00096224:00:00: *** register_interrupt("HEAD3", 0xd9, 0xff986e2c, 0x0), from ff986ee0
65E59> LiveViewMg:00096224:00:00: *** register_interrupt("HEAD4", 0xe0, 0xff986f4c, 0x0), from ff986f98
65EA9> LiveViewMg:ff8da96c:98:02: GetWbIntegFrameForWb 0
65EDD> LiveViewMg:ff8e8fb4:08:01: lvcaeGetWbIntegFrame(sync)
65F02> LiveViewMg:ff8e8fcc:08:01: WBIntegFrame(WB:0x152a40,FST:0x1529dc)
65F79> LiveViewMg:ffa0adfc:99:02: SetWbFstPasParameter
65FAA> LiveViewMg:ffa0b00c:99:02: SetWbIntegParameter
66012> LiveViewMg:ffa06df8:98:02: ReleaseEngineResource Res:4, Free:fffffffc Count:0
66065> LiveViewMg:ff8e1b48:98:02: DispGain:1024, ISO:89
66091> LiveViewMg:ff8e1b70:98:02: WB 1982 1024 1024 2016(4037)
66110> LiveViewMg:ff8e1718:98:02: setHorizontalIrcutData(po0=100)
66198> LiveViewMg:ff863840:82:02: GetVramSize (PUB)
661E8> LiveViewMg:ff867ea8:82:03: SetPBForLV ZoomMode=2, LVType::Info=0, LVType::DispType=0
6621D> LiveViewMg:ff867ed4:82:02: SetPBForLV (3066)
662D1> LiveViewMg:ff9a77d0:00:01: [CLKSAVER] ��ClockSave Out��
6631E> LiveViewMg:ff8638c4:82:01: ImgDDev SelectParameter DispType=0
66349> LiveViewMg:ff863840:82:02: GetVramSize (PUB)
6637C> LiveViewMg:ff8637e4:82:01: ImgDDev Reg c0f140e0 01b07800
663AE> LiveViewMg:ff8637e4:82:01: ImgDDev Reg c0f140e4 01b07da0
663E1> LiveViewMg:ff86449c:82:02: AsyncEnableImageVBufferForPlayBackAndWait (PUB)
66409> LiveViewMg:ff863fb4:82:01:  RequestNotifyBlank (PRI)


This what I find very interesting
661E8> LiveViewMg:ff867ea8:82:03: SetPBForLV ZoomMode=2, LVType::Info=0, LVType::DispType=0
6621D> LiveViewMg:ff867ed4:82:02: SetPBForLV (3066)

ZoomMode=2 so can I assume this is still 14bit raw but out a different channel .
I wonder if SetPBForLV (3066) is the Zoom window for capture ?
as a little side project I'll try to enable 10x zoom recording and see if this work out of the box  :)
That would be nice to just have to push the zoom button twice (10xzoom) to get UHD , this could change everything I've being work on for the last year.
Is this a "secret" special bug that canon dev. forgot to close ?  :o in the same lines as the 50d video that wasn't supposes to be active , here's hoping  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 13, 2018, 08:18:42 AM
Same more info on the above post -- I forgot to add the A & B timers info
Desired FPS -------23.976(from30)
Optimize for --------- Low Light
Shutter range --------1/24-1/109
FPS timer A -----------610(FT-28)
FPS timer B -----------1630(FT-10)
Main Clock ------------24.00MHz
Actual FPS ------------24.137


I just check a plain ml build and In 10x zoom you can't select anything over 2k so it must be the FPS timer .
It was a kind of a guess factor , I remembered something that a1ex said about A timers need to be low like (540) can't go below 610
and b timer need to be around 1600-1700 I think , so I play around to get it close and it appears to work in 10x zoom
(https://image.ibb.co/eidCVp/VRAM7_small.png) (https://imgbb.com/)

Also I think 3x crop_mode is a resample of FHD and there's some pixel binding going on , I took 3 screen shots , first FHD then 3x cropmode (5xzoom) and last 10x zoom .
1:1 FHD (line skipping)
(https://image.ibb.co/njUNwU/VRAM9_small.png) (https://imgbb.com/)
3x crop_mode (5x zoom)
(https://image.ibb.co/cNqZAp/VRAM10_small.png) (https://imgbb.com/)
10x zoom
(https://image.ibb.co/c6MSwU/VRAM11_small.png) (https://imgbb.com/)
I still believe 10x is truly 1:1 of the 5616x3744 with some cropping in movie mode and I think the back lcd sceen is mapped to the sensor in 1:1 so just a 720x480 of the 5.6k image
it I use the digital dolly I can scroll to the top of the image  same as  full res image but with a little cropping maybe 16x9 ? not sure.
I do not get the same results if I use the dolly in 3x crop , I'll post a example of this later on this week .

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 13, 2018, 08:48:42 AM
One more quick little post , while in 10x zoom I did a image dump at 2880 & 3520 (V)
and it seem that the height is being increased in 2880 not sure why ,
maybe the A & B timers are close in tune with 2880 then 3520 (V) .

2880(V)(https://image.ibb.co/bEOXD9/LV_003_small.png) (https://imgbb.com/)3520(V)(https://image.ibb.co/ncAjRU/LV_004_small.png) (https://imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 15, 2018, 08:27:05 AM
More research in to 10x Zoom to understand if this mode is really 1:1of the sensor or just a resample :
Adjusting the A & B timers are giving my some very interesting results ,
I started with 29 from 30 fps in "low jello" mode , A-timers = 610 (FT-2018) was not changed , I started B-timer at 2333 (FT+ 977)
and increase to 2575 (FT+1219) . In mlv_lite (raw video) set resolution to 3520 (V)

A-timer 610 (FT-2018) B-timer 2333 (FT+977)
(https://image.ibb.co/e3K1T9/LV_031_small.png) (https://imgbb.com/)

A-timer610(FT-2018)-B-timer2400(FT+1044)
(https://image.ibb.co/gnQN1U/LV_032_small.png) (https://imgbb.com/)

A-timer610(FT-2018) B-timer2450(FT+1094)  :D
(https://image.ibb.co/kVn21U/LV_033_small.png) (https://imgbb.com/)

A-timer610(FT-2018) B-timer2500(FT+1144)
(https://image.ibb.co/ngUt89/LV_034_small.png) (https://imgbb.com/)

A-timer610(FT-2018) B-timer2525(FT+1169)  :D
(https://image.ibb.co/iOeoap/LV_035_small.png) (https://imgbb.com/)

A-timer610(FT-2018) Btimer2550(FT+1194)
(https://image.ibb.co/emVGT9/LV_037_small.png) (https://imgbb.com/)

A-timer610(FT-2018) B-timer2575(FT+1219)
(https://image.ibb.co/c01RBU/LV_038_small.png) (https://imgbb.com/)

A-timer610(FT-2018) B-timer2581(FT+1225)
(https://image.ibb.co/hHTCkp/LV_039_small.png) (https://imgbb.com/)

Now do I have the full res 5.6K image ? I need to apply this with the adtg_gui module so I can adjust the cmos reg's
I still don't know what size the buffer is in 10x Zoom .

This is what the default Liveview Image dump in 10x zoom looks like , I use a plain nightly ML build

(https://image.ibb.co/bOvYQp/LV_001_small.png) (https://imgbb.com/)

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 16, 2018, 07:58:08 AM
Continue on to research 10x zoom with ADTG_GUI "ISO_Resarch_Branch"
finding the same results with Frame timers A & B as with dm-spy .
Started with 23.976 from 30 fps
default  A & B timers for 10x zoom
A-timer 610(-22)
B-timer 1640 (x1.28)


A-timer 610(FT-2018)
B-timer 1715 (FT+75)

Here the LV-005.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-005.422) for those timer , getting full res Liveveiw just like the above post .

Loaded the Edmac.mo and it seems that 10x zoom using the same buffer as 3x crop
@a1ex , the math for "B"timer works 1640(base) +75 = 1715 , but for "A" timer 610 seem to be unchanged even though I have (-2018) so with the math that's "-1408" is this being
applied ? do I have to modify them though the reg's (head timers) ? to get 540
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on August 16, 2018, 12:27:31 PM
Quote from: reddeercity on August 16, 2018, 07:58:08 AM
@a1ex , the math for "B"timer works 1640(base) +75 = 1715 , but for "A" timer 610 seem to be unchanged even though I have (-2018)


That happens when it hits the lower limit for timer A, as declared in fps-engio.c. That is, the value sent to hardware won't be below the declared limit, no matter what you dial in the UI.

To be able to dial a lower value, edit that limit (FPS_TIMER_A_MIN).

If timer A is too small, the side effects begin to be visible on the right side (usually noise instead of image data). If you keep reducing it, the image will break. The ideal value is the smallest value that still gives clean image (with correct image data on the right border).

QuoteI think at 10x zoom is really  1:1 pixel

So is the x5 zoom. The only difference between these modes is how the preview is rendered. In both x5 and x10 modes, the captured raw image is identical. So, I see no reason why anything would work better in x10, rather than in x5.

For the UI, I prefer to leave the x10 mode untouched, for focusing.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 19, 2018, 08:48:39 AM
Will a little good & bad news , first the good -- I figured out what reg's & timers I need for extended resolutions as seen in the previous post's with the Liveview_dumps I've posted .
Now the bad -- the problem I'm having is I'm lock in to the frame buffer (LiveviewBuffer) of either 1880x1248 or 2144x1076 (3x crop_mode) I can't extent the buffer ,
so I'm getting a doubling of the same frame buffer as seen in 4096x1248 here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg201004#msg201004) & here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg201033#msg201033) . I still can't extend the 3x crop window to 3k like @waza57 did (If you are out there waza57 please can you comment on your experiment ) But I have been able to extent 1:1 (FHD) to 2144x1076 from 1880x1248 . Thou it uses the frame resolutions of 2.1k it's still has the 1880x1248 frame buffer , it just overlaps part the same frame buffer to fill the frame resolution , I guess that has something to do with the edmac .

Liveview_dump 422 , frame res. is 2144 (V) in 1:1 FHD and is filled with the 1856 (V) Liveview buffer , see the black bar then part of the next same frame
(https://image.ibb.co/bEzKqz/LV_022_small.png) (https://imgbb.com/)

I guess I I'm stuck now at this point , been going around in circles and unless I can figure out the liveview buffer (This may be beyond my ability  :-[ )
a1ex is there any hint for this ? I'm lost on the Liveview buffer , or I'm I missing a step . I would think I should be able to save single dng even in extended res.

Quote from: a1ex on August 16, 2018, 12:27:31 PM
For the UI, I prefer to leave the x10 mode untouched, for focusing.
Ok , just looking to see if there is any usable information that can be used
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 20, 2018, 09:30:18 AM
Switch back to "ISO_Research_Branch" getting much better results , thou the buffer is still a problem.
Finding "CMOS[0]" can change the Liveview to full width of the sensor from a center crop.
Here are the setting I found to be the most stable and cleanest image while there's still lots of room for improvement .

What I did to get the results that I'm posting -- Load "iso research branch" build (I added 4096 (V) & 3840 (V) to mlv_lite.c  for extended resolution .
Loaded
adtg_gui.mo
crop_rec.mo
edmac.mo **(only for info)**
mlv_lite.mo

enabled "crop_rec.mo" , set to 1:1
enabled  "adtg_gui.mo" , toggled on "digic reg's" then enabled the 3K 5d3 preset(I changed the reg's to work with 5d2 ,6808-->6088 , 6804 -->6084)
that it , and trial & error got me to this point .
ADTG[0x1061] 0x249 -->3ca      **(1:1 frame Blanking)**
CMOS[0] 0x20b -->0xe1b
CMOS[1] 0xc00 -->0x10
CMOS[2] 0x40e -->0x10c


Here's the Liveview image dump for those setting -- frame resolution set to 4096x1250

(https://image.ibb.co/cLb3kz/LV_031_small.png) (https://imgbb.com/)

And the HD image dump for the above Liveview dump

(https://image.ibb.co/i6dfyK/HD_031_small.png) (https://imgbb.com/)


when CMOS[0] is disabled (no over riding reg's) the image turn back to center crop .
ADTG[0x1061] 0x249 -->3ca      **(1:1 frame Blanking)**
CMOS[1] 0xc00 -->0x10
CMOS[2] 0x40e -->0x10c


Liveview image dump -- frame resolution set to 4096x1250

(https://image.ibb.co/hhmEse/LV_029_small.png) (https://imgbb.com/)

HD image dump -- frame resolution as set to 4096x1250

(https://image.ibb.co/eJdYkz/HD_029_small.png) (https://imgbb.com/)

So any one with a d4 (digic4) cam 7d , 50d etc. ... can follow along -- the process should be very similar

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 25, 2018, 09:41:13 AM
I'm starting to smell success !
Made go great process today with "crop_rec_4k_5D2" branch from "wasa57" in 3x crop_mode
I was able to get the vertical to 2304 (V) , max is 2144(V) thou I couldn't save a dng at that rez ,
but I was about to save a 2252x1075 dng after ml crop it , so really it's 2416x1127 the un-crop size or the "Raw Size"
Active Area  : 52 160 1127 2416

Here the Original dng from the Image_dump 2256x1075-RAW-000.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2256x1075-RAW-000.DNG)

(https://image.ibb.co/h7xVA9/2256x1075_RAW_000_small.png) (https://imgbb.com/)

here is also 2256x1075-RAW-000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2256x1075-RAW-000.png) from the same image dump

ExifTool Version Number         : 10.45
File Name                       : RAW-000.DNG
Directory                       : C:/dcraw
File Size                       : 4.6 MB
File Modification Date/Time     : 2018:08:24 22:18:22-06:00
File Access Date/Time           : 2018:08:11 23:06:24-06:00
File Creation Date/Time         : 2018:08:11 23:06:24-06:00
File Permissions                : rw-rw-rw-
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Image Description               :
Make                            : Canon
Camera Model Name               : Canon EOS 5D Mark II
Orientation                     : Horizontal (normal)
Software                        : Magic Lantern
Modify Date                     :
Artist                          :
Subfile Type                    : Full-resolution Image
Image Width                     : 2416
Image Height                    : 1127
Bits Per Sample                 : 14
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Strip Offsets                   : 33792
Samples Per Pixel               : 1
Rows Per Strip                  : 1127
Strip Byte Counts               : 4764956
X Resolution                    : 180
Y Resolution                    : 180
Planar Configuration            : Chunky
Resolution Unit                 : inches
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 0 1 1 2
Black Level                     : 1278
White Level                     : 16200
Default Crop Origin             : 0 0
Default Crop Size               : 2256 1075
Active Area                     : 52 160 1127 2416
Opcode List 1                   : (Binary data 28 bytes, use -b option to extrac
t)
Copyright                       :
Exposure Time                   : 0
F Number                        : 0
Exposure Program                : Not Defined
ISO                             : 0
Exif Version                    : 0221
Date/Time Original              :
Exposure Compensation           : 0
Max Aperture Value              : 1.0
Metering Mode                   : Unknown
Flash                           : No Flash
Focal Length                    : 0.0 mm
Sub Sec Time                    :
Sub Sec Time Original           :
Focal Length In 35mm Format     : 0 mm
TIFF-EP Standard ID             : 1 0 0 0
Serial Number                   :
Lens Model                      :
DNG Version                     : 1.3.0.0
DNG Backward Version            : 1.3.0.0
Unique Camera Model             : Canon EOS 5D Mark II
Color Matrix 1                  : 0.4716 0.0603 -0.083 -0.7798 1.5474 0.248 -0.1
496 0.1937 0.6651
Analog Balance                  : 1 1 1
As Shot Neutral                 : 0.473635 1 0.624
Baseline Exposure               : undef
Baseline Noise                  : 1
Baseline Sharpness              : 1.333333333
Linear Response Limit           : 1
Calibration Illuminant 1        : D65
Frame Rate                      : 25
CFA Pattern                     : [Red,Green][Green,Blue]
Image Size                      : 2416x1127
Megapixels                      : 2.7
Shutter Speed                   : 0


Just adjusting with digic poke in the debug menu ,
c0f06084 - 0x30002
c0f06088 - 0x469ff77

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: honza123 on August 25, 2018, 10:28:30 AM
Great !
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 29, 2018, 05:51:05 AM
Now that a1ex fixed adtg_gui.c in iso-resaerch branch I can proceed to get the full raw buffer for full raw width
Here's (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/) (magiclantern-iso-research-4k-D4.2018Aug29.5D2212.zip) the Up dated build I'm using , did some quick adjustments to 6084 & 6088 . Got a error printed on the screen

Default raw buffer too small (38MB , need 2148x16377 59MB - reallocating
[BKT] giving up
black 2/5 stdev too large (47077/100 ref 15535/100


(https://image.ibb.co/d6n1a9/IMG_2620_small.png) (https://imgbb.com/)

Will at lest I know I'm working off a 38MB buffer -- do I need a 59MB buffer ?
@a1ex , what would  the buffer size the 5D3 uses for 4k extended res ? does it need to be 59MB

Here are the 6084 & 6088 settings , I first enable the digic reg's , exited to liveview press the play button reviewed a cr2 photo
re-entered adtg_gui.mo enabled the 3k preset for crop rec -- adjusted reg's 6084 & 6088

(https://image.ibb.co/d6KToU/IMG_2625_small.png) (https://imgbb.com/)

Shot a short 25 second 720p video on my iPhone , showing the error being printed on the screen with some other info
https://drive.google.com/file/d/129ONClwQUoZrQeJ9tg0moK7YnIeRdzKm/view?usp=sharing

the setting for 6088 was just a random number to see if the changes do really take effect , and they do

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on August 30, 2018, 07:55:13 AM
59MB is way more than what you can allocate contiguously on these cameras; that number comes from the value dialed to C0F06088:

0x3FF90433 -> split in two -> 0x3FF9 and 0x433.
0x1 -> split in two -> 0x0 and 0x1 (better don't touch this one; leave it at 0x10036).
Difference: 0x3FF8 and 0x3FD.
The latter is X resolution divided by 2 (number of columns read out at once). That's how you've got 2148.
The former is Y resolution, but higher bits might be unused (hardware does not interpreted them, i.e. it works in the same way whether these bits are 1 or 0). However, ML interprets them.
So, 0x3FF8 = 16376. ML requests memory for 2148x16377 (looks like in your source, it adds 1). The hardware delivers 0xFF8 (4088) or 0x7F8 (2040); likely the former.

In your previous example, 0x469 - 3 = 1126; not sure how to interpret 0xff77 - 2 to get close to 2256...

In any case, I suggest adding small quantities to either the low halfword (for X) or to high one (for Y).

As you increase X, you may have to increase FPS timer A by the same amount (1 step = 2 pixels on 5D2), and possibly other registers that use similar values. For example, on 700D, timer A must be written to 0xC0F06008, 0xC0F0600C, 0xC0F06010, 0xC0F06824, 0xC0F06828, 0xC0F0682C and 0xC0F06830; the last 4 must be in sync and if you don't increment them, you will get just black bars.

As you increase Y, you may have to increase FPS timer B and PowerSaveTiming registers (https://www.magiclantern.fm/forum/index.php?topic=19300.msg202783#msg202783).

Other registers can be figured out by comparing x5 (1:1 crop) with plain 1080p (that's pretty much all you've got on 5D2). On 700D, comparing the regular 1080p with the 1:1 crop mode from Canon firmware was very helpful.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 30, 2018, 08:06:51 AM
Thanks @a1ex that's exactly what I was looking for , makes sense now .
I was blinding adjusting , now I have more organized approached thanks.
 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 30, 2018, 08:41:27 AM
From 6088 "default" 4f40432 in 1:1 FHD
4f4 =1268 (V or X)  432 = 1074 x 2 = 2148 (H or Y)
I tried 5f40432 to see what happens
5f4 = 1524 (V)  432 = 1074 x 2 =2148 (H)
increased vertically by 256 pixels if I did the math right .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 31, 2018, 08:47:07 AM
Looking up now ,  :)
I'm taken baby steps , able to increase the height now with
c0f06088 , A & B timers , target height is 2160 -- I'm at 1760
cmos[2]  40E --> 10E
c0f06088
4f40432 ->6f40432

Low Jello , 180d
A timers  --> 792
B timer --> 1750 (around there)


(https://image.ibb.co/jEuRf9/VRAM2_small_6088.png) (https://imgbb.com/)

(https://image.ibb.co/d9QwDU/VRAM1_small.png) (https://imgbb.com/)

disregard the 3200 , I haven't adjust the lower half yet (432) i did quickly set the lower half to 632 to the first half of 6f4 and got it to lock to 2880x1760 & 3072x1760
but it lock up , needs more reg's to make it stable , I'm slowly finding the reg that correspond to the 5d3/D5 camera.
Can't save a dng yet , thou liveview dumps look promising .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on August 31, 2018, 07:48:13 PM
Made a couple experiments on 5D2, too. Initial notes:

- black level depends on raw type (either 1024 or 1792; need to sit down to find it)
- pink preview still stubborn
- CONFIG_EDMAC_RAW_SLURP kinda works, but gives corrupted frames
- timing for raw_lv_vsync appears to be correct
- calling raw_lv_vsync from the same place where Canon code would call it (StartImagePass x1/x5 CrawAddr / KindOfCraw), did not help
- patching the SetEDmac call from the above routine appears to work:


static void raw_lv_setedmac_patch(uint32_t* regs, uint32_t* stack, uint32_t pc)
{
    /* R0: EDMAC channel */
    /* R1: output buffer */
    /* R2: EDMAC info (geometry) */
    /* R3: flags */

    int width, height;
    int ok = raw_lv_get_resolution(&width, &height);
    if (ok)
    {
        if (redirected_raw_buffer)
        {
            /* optionally update output buffer */
            regs[1] = redirected_raw_buffer;
        }

        /* update EDMAC image size */
        int pitch = width * raw_info.bits_per_pixel / 8;
        static struct edmac_info dst_edmac_info;
        dst_edmac_info.xb = pitch;
        dst_edmac_info.yb = height - 1;
        regs[2] = (uint32_t) &dst_edmac_info;

        /* we can override this here */
        EngDrvOut(RAW_TYPE_REGISTER, lv_raw_type);
    }
}

static void raw_init()
{
    ...

    /* SetEDmac from StartImagePass_x1/x5 CrawAddr / KindOfCraw */
    /* FIXME: why it fails when the hook is placed on the BL instruction?! */
    patch_hook_function(0xFFA08B20, MEM(0xFFA08B20), raw_lv_setedmac_patch, "RAW LV x5");
    patch_hook_function(0xFFA08008, MEM(0xFFA08008), raw_lv_setedmac_patch, "RAW LV x1");
}


Still need to figure out why our code called from the same place doesn't work well, but the patched one does.

Now, adtg_gui stuff. FPS timer A has a small quirk - one unit means 4 pixels on this camera, but one unit of C0F06088 means 2 pixels. So, the easiest experiment is:
- enable ADTG registers and (in the advanced tab) ENGIO registers
- go to x5 zoom, enable mlv_rec's grayscale preview
- override FPS timer A (C0F06008) from 0x2770277 to 0x2770377 (increase X by 0x100)
- this change lets you increase raw resolution register (C0F06088) by 0x200, i.e. from 0x46A04BA to 0x46A06BA
- this gives 3168 pixels horizontally in mlv_rec: (0x6BA - 0x36)*2 = 3336, minus 160px black bar = 3176, then 3168 is multiple of 32.

Next is adjusting CMOS registers for centering the image horizontally - reply #10. Will edit later.




Part 2. Let's try to get full width LiveView.

Target values: the ones from photo mode.

              x1     x5 centered        photo    description
C0F06084:   0x10036     0x30036        0x10037   start row/column
C0F06088: 0x4F40432   0x46A04BA      0xEDD0B87   stop row/column (column: 1 unit = 2 pixels)
C0F06008: 0x23B023B   0x2770277      0x5DB05DB   FPS timer A (1 unit = 4 pixels horizontally)
C0F06014:     0x6D5       0x4FF          0xEDC   FPS timer B (1 unit = 1 pixel vertically)
CMOS[1] :     0xC00       0xE6A          0xC00   vertical start/stop
CMOS[2] :     0x40E       0x10E            0x8   horizontal pos & binning mode
CMOS[3] :       0x5         0x5            0x7   ???
CMOS[4] :     0x244       0x244          0x244   ???
CMOS[5] :       0x1        0x1           0xC05   ???



CMOS[2]:
xx-- -------- binning mode (00: read every column, 01: bin every 3 columns, 10 and 11: bin even more columns?!)
--xx xxxx---- horizontal position (0: start from left, 0x2a: thin stripe of valid data, 0x2b: no more valid data)
---- ----xxxx ??? (0x8 in photo mode, 0xE in LiveView, double highlights with 0x8 in LV)


For full width LiveView:
CMOS[2] = 0x00E
C0F06008: 0x27705DB (only the lowest half appears to matter on this camera)
C0F06084: 0x30037 (copied lowest half from photo mode)
C0F06088: 0x46A0B87 (copied lowest half from photo mode)
=> 5632x1074 in mlv_rec, 12.5 FPS.




Now, to increase height.
C0F06014: 0xEDC (copied from photo mode) => 4.2 FPS, clean image, so far so good.
C0F06088: 0x56A0B87 (increase Y by 0x100). Image breaks. Undo the change and look for other registers that might be related to timer B (close to 0x4FF)
First candidates: HEAD timers. Attempting to change them results in camera lockup. Back to the drawing board.

Some success starting from vanilla x5 and adjusting just the height:
- change C0F07150 from 0x49C to 0x59C and pray it won't lock up
- change C0F0713C from 0x476 to 0x576 and pray it won't lock up
- change C0F06088 from 0x46A04BA to 0x56A04BA => mlv_rec shows 2144x1330, but image is not good on the bottom half
- try to change CMOS[1] => E69 looks better, but image stutters; F6A looks even better, also with stutter, still some image data missing
- look for powersave timing registers: ADTG1[1047] from 0x550 to 0x650 appears to do the trick. maybe not needed? go to 1x and back to x5 to refresh.
=> 2144x1330 in mlv_rec with clean preview, still with some stutter with ML grayscale preview, but looks fine with Canon preview.

Let's try some more:
- C0F07150: 0x69C
- C0F0713C: 0x676
- C0F06088 to 0x66A04BA
=> 2144x1586 apparently working just fine.

- C0F07150: 0x79C
- C0F0713C: 0x776
- C0F06088 to 0x76A04BA
=> 2144x1842 apparently working just fine.

- CMOS[1]: 0xC00
- C0F07150: 0x79C
- C0F0713C: 0x776
- C0F06088 to 0x76A04BA
=> 2144x2354 apparently working just fine.

Battery empty, will continue tomorrow.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 01, 2018, 04:42:34 AM
Holy Cow !!!  :o ,  it's not my birthday & I know it's not 'Xmas , just look outside to make sure it wasn't snowing  :D
5D2 & D4 users  a1ex has given use a amazing gift ! the proof that 4k is a real possibility on 5d2 or at least 3k  8)

This is incredible , thanks @a1ex -- so I guess I should try and reproduce some of your experiments (by the way I like your experiments way better then mind)
I knew about these one 's but anyway lock up cam
C0F07150:
C0F0713C

"CONFIG_EDMAC_RAW_SLURP" sound very encouraging

Anyhow here we go , let see if I can save a good dng in extended rez.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 01, 2018, 06:22:20 AM
Quoteeasiest experiment
Ok I can reproduce , got even an dng to export thought the image dump -- thou it's all distorted

Default Crop Size               : 3176 1075
Active Area                     : 52 160 1127 3336


dng from image dump converted to png so I can post it
(https://image.ibb.co/dapqAK/RAW_001_small.png) (https://imgbb.com/)

Liveview .422 image
(https://image.ibb.co/kJ8xPe/LV_007_small.png) (https://imgbb.com/)

Here the original 3176x1075 RAW-001.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-001.DNG) & LV-007.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-007.422) image .

ExifTool Version Number         : 10.45
File Name                       : RAW-001.DNG
Directory                       : C:/dcraw
File Size                       : 6.3 MB
File Modification Date/Time     : 2018:08:31 21:37:14-06:00
File Access Date/Time           : 2018:08:31 21:52:05-06:00
File Creation Date/Time         : 2018:08:31 21:52:05-06:00
File Permissions                : rw-rw-rw-
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Make                            : Canon
Camera Model Name               : Canon EOS 5D Mark II
Orientation                     : Horizontal (normal)
Software                        : Magic Lantern
Subfile Type                    : Full-resolution Image
Image Width                     : 3336
Image Height                    : 1127
Bits Per Sample                 : 14
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Strip Offsets                   : 33792
Samples Per Pixel               : 1
Rows Per Strip                  : 1127
Strip Byte Counts               : 6579426
X Resolution                    : 180
Y Resolution                    : 180
Planar Configuration            : Chunky
Resolution Unit                 : inches
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 0 1 1 2
Black Level                     : 8833
Default Crop Size               : 3176 1075
Active Area                     : 52 160 1127 3336

White Level                     : 16200
Default Crop Origin             : 0 0
DNG Version                     : 1.3.0.0
DNG Backward Version            : 1.3.0.0
Unique Camera Model             : Canon EOS 5D Mark II
Color Matrix 1                  : 0.4716 0.0603 -0.083 -0.7798 1.5474 0.248 -0.1
496 0.1937 0.6651
Analog Balance                  : 1 1 1
As Shot Neutral                 : 0.473635 1 0.624
Baseline Exposure               : undef
Baseline Noise                  : 1
Baseline Sharpness              : 1.333333333
Linear Response Limit           : 1
Calibration Illuminant 1        : D65
Frame Rate                      : 25
CFA Pattern                     : [Red,Green][Green,Blue]
Image Size                      : 3336x1127
Megapixels                      : 3.8


On to the next experiment .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 01, 2018, 08:15:24 AM
Got to full width "5632x1075" for a short time then lock up , but I was able to save a image dump only good thing I got was a Liveview .422
scrambled up image , I didn't get a chance to change 6008 before it locked up .

(https://image.ibb.co/ircMZe/LV_008_small.png) (https://imgbb.com/)

Here the LV-008.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-008.422) from the rez of 5632x1074

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on September 01, 2018, 11:51:04 AM
Hm, the DNGs should be clean, not sure what's missing. Did you start overriding from x5 zoom?

Anyway, back to full height.

CMOS[1]: 0xC00
C0F06014: 0xEDC (increased by 0x9DD)
C0F07150: need to increase by 0x9DD from 0x49C to 0xE79.
C0F0713C: need to increase by 0x9DD from 0x476 to 0xE5E.
C0F06088: 0x46a04ba + 0x9DD0000 = 0xE4704BA (a little lower than full height)
=> 2144x3600 in mlv_rec; the DNG from VRAM dumps looks clean.


Default Crop Size               : 2152 3600
Active Area                     : 52 160 3652 2312


Still need to reach 0xEDD04BA for full height (150 pixels left). Dialing this value doesn't work.

Let's increase everything by 150 units (0x9DD + 150 = 0xA73).

CMOS[1]: 0xC00
C0F06014: 0xF72
C0F07150: 0xF0F
C0F0713C: 0xEE9
C0F06088: 0xEDD04BA
=> 2144x3750 in mlv_rec, 9.6 FPS.

Let's add the full width:

CMOS[2] = 0x00E
C0F06008: 0x27705DB
C0F06084: 0x10037 (same as photo mode)
C0F06088: 0xEDD0B87 (same as photo mode)
=> 5632x3752 in mlv_rec, 4 FPS, DNG looks clean to me.

Default Crop Size               : 5632 3752
Active Area                     : 52 160 3804 5792


(https://a1ex.magiclantern.fm/bleeding-edge/5D2/fullres-lv.png)

May require some fine-tuning to match a CR2, maybe the frame rate can be slightly increased, but this completes the proof of concept: arbitrary resolutions, up to full-res LiveView, are possible on 5D2 and likely on all other models already running ML.

This experiment was actually a continuation of the 700D full-res LV PoC (https://www.magiclantern.fm/forum/index.php?topic=19300.msg205546#msg205546).

Will commit the changes and update the experimental builds to make sure the above can be reproduced by any of you.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on September 01, 2018, 01:49:44 PM
I have the same problem with C0F07150 and C0F0713C I can't override them correctly in my implementation of a 
crop_rec module for 5D2. Maybe that's why I'm limited in height

A few month ago i posted a dng with a promising size from my experiences in "dm-spy-experiments" branch.
It took a lot of time to implement this method in a module of crop_rec_4k branch.
I've seen that Reddeercity spend a lot of time time here, it's encouraged me to continue
 
for the moment only "CROP_PRESET_3X" and "CROP_PRESET_CENTER_Z".For those who want to test the code is there:

https://bitbucket.org/waza57/magic-lantern/src/crop_rec_4k_5D2/ (https://bitbucket.org/waza57/magic-lantern/src/crop_rec_4k_5D2/)
Download link:
magiclantern-Nightly.2018Sep02.5D2212.zip (https://bitbucket.org/waza57/magic-lantern/downloads/)
 
I hope it will be useful. It works for me but at your own risk (Digic poke is enable don't use it if you do not know what you are doing)

SETTINGS:
   
      Canon menu : NTSC 30
      ML menu    : set resolution to maximum ,  preview grayscal
      Modules to load: crop-rec, mlv play, mlv rec, silent
   
NOTES for developers:

I don't use CONFIG_EDMAC_RAW_SLURP.

I started from this:
https://www.magiclantern.fm/forum/index.php?topic=10111.msg123909#msg123909 (https://www.magiclantern.fm/forum/index.php?topic=10111.msg123909#msg123909)

https://www.magiclantern.fm/forum/index.php?topic=10111.msg125581#msg125581 (https://www.magiclantern.fm/forum/index.php?topic=10111.msg125581#msg125581)

And especially this where i decide to try increment height with patching ROM addresses.
https://www.magiclantern.fm/forum/index.php?topic=19336.msg183200#msg183200 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg183200#msg183200)

Thanks to GREG!

For crop_rec.mo works i need to make change in functions of:

raw.c

          "raw_lv_get_resolution" ->  ...uint32_t top_left  = shamem_read(0xC0F06800)... to retrieve the size like this
          "autodetect_black_level"->  disabled,  otherwise  i got "raw detect error"
          "#define BLACK_LEVEL 1792"  otherwise bad black level
         
edmac-memcpy.c , fio-ml.c
          I need to disable call to function "clean_d_cache()" otherwise i can't patch ROM addresses

lv-img-engio.c
          I need to add this otherwise impossible to compile : "if defined(FEATURE_EXPO_ISO_DIGIC) && defined(CONFIG_EDMAC_RAW_SLURP)"
         
Thanks to a1ex for this project, without him, I would never have discovered the joys of programming in C.

TODO list:

- Increase height frame.
- mlv_lite module compilation.
- Compressed raw activation. ( new idea here too :Danne Idea  (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205681#msg205681))
- Shutter speed question ( is this bad?)
- compatibility with Dual_iso module question.
- ISO adjustment activation (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205719#msg205719 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205719#msg205719))
- Faulty HDMI connection
- Bad preview solution

* And all i will notice in your feedback
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on September 01, 2018, 03:45:07 PM
Right, it took a lot of trial and error to figure out the meaning of all these registers. Back then, when I started to write the initial 4K hack, I didn't understand how this works well enough to make a generic implementation; that's why crop_rec ended up preset-based, with each resolution being manually tuned.

Now looking into 500D. This one is interesting because its 1080p mode is limited to 20 FPS. It does have 720p30 though. Let's compare them:


          1080p20  720p30  x5 zoom    photo
C0F06008  50F050F 5630563  58F058F  AF90AF9
C0F06014      4D1     304      3FF      C83
C0F06084    20040   20040    2004A    1004C
C0F06088  43D06C0 29506C0  360085A  C84132C
C0F0713C      440     294      370   unused
C0F07150      46E     2A1      388   unused


First mystery is at timer A: both 1080p20 and 720p30 use the same horizontal resolution, yet there are different values of timer A. Dialing down the value from 720p (0x563) to 0x50F does not work; image breaks below 0x543. Why?!

1080p: 1664x1083
720p: 1664x659

It's definitely one of the ADTG registers. Locked all of them to their values from 1080p, switched to 720p et voilà! 32 FPS with timer A = 50F!

Narrowed down to ADTG[100C] - the line skipping factor. Boo...




Full width (from x5):

FPS timer A: 1 unit = 2 pixels.
C0F06088: 1 unit = 1 pixel.
2 * 0x58F - 0x85A = 710
2 * 0xAF9 - 0x132c = 708
=>
C0F06008: 58F0AF9
C0F06084: 2004C
C0F06088: 360132C

Appears to work, but need to implement the display filters first in order to check the results. Will revisit later.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 02, 2018, 06:59:51 AM
Ok starting over , trying the first experiment again to get 3k (H) in 3x crop
Still can't get a clean dng out , even try a fresh iso-reserach branch clone no go , there must be something I'm missing!
I can't get it to work at all with mlv_rec , kind of works with mlv_lite  -- so a1ex are you using mlv_lite?
even with mlv_lite I'm still at max rez of 2144 even if I try the full width , I get the res in the menu 5623 but still say max 2144.

It work better with my very hacky "raw_slurp" with the default raw at 4096X4096 (the one I post come from that )

Do I need to add that patch code you posted ? to made it work so I can export clean dng's ?

I followed your procedure
- enable ADTG registers and (in the advanced tab) ENGIO registers
- go to x5 zoom, enable mlv_rec's grayscale preview
- override FPS timer A (C0F06008) from 0x2770277 to 0x2770377 (increase X by 0x100)
- this change lets you increase raw resolution register (C0F06088) by 0x200, i.e. from 0x46A04BA to 0x46A06BA
- this gives 3168 pixels horizontally in mlv_rec: (0x6BA - 0x36)*2 = 3336, minus 160px black bar = 3176, then 3168 is multiple of 32.


When I got it to show the right size 3168x1074 it was with mlv_lite & "raw_slrup" with default raw buffer 4096x4096 which I thing is 32MB

I also had Frame Ride disabled , and I did see with the full width 5632 It was at 12.5 fps but image was garbage .

Should I try the 4k crop record branch with ADTG_GUI.mo ?
This confusing me why I can't it to work , the image is always garbage/noise/distorted .

@waza57 , good to see you back here !
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on September 02, 2018, 07:12:21 AM
....some unnecessary deleted modes at the moment in the crop_rec module :

new link : magiclantern-Nightly.2018Sep02.5D2212.zip (https://bitbucket.org/waza57/magic-lantern/downloads/)

@ a1ex
Do you see a ADTG[100C] with adtg_ gui ? i'don't see it with 5D2.

@reddeercity
i have tried to simply copy the files from the adtg_gui folder of iso-research into the crop_rec_4k branch. it seems to work.


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 02, 2018, 08:36:14 AM
Great job @waza57 ! Tried you crop_record 4k build , and I can confirm 3xcrop 3.5k preset works @ 2880x1080 23.976 10bit out of the box !

Liveview image dump is good LV-011.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-011.422)
(https://image.ibb.co/kKBUNz/LV_011_small.png) (https://imgbb.com/)

Image dump dng also good RAW-003.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-003.DNG)
(https://image.ibb.co/ddR79e/RAW_003_3k_small.png) (https://imgbb.com/)

Default Crop Size               : 2884 1080
Active Area                     : 54 264 1134 3148
Black Level                     : 1792
White Level                     : 16200


File Size                       : 6.4 MB
File Modification Date/Time     : 2018:09:01 23:34:10-06:00
File Access Date/Time           : 2018:09:01 23:58:53-06:00
File Creation Date/Time         : 2018:09:01 23:58:53-06:00
File Permissions                : rw-rw-rw-
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Make                            : Canon
Camera Model Name               : Canon EOS 5D Mark II
Orientation                     : Horizontal (normal)
Software                        : Magic Lantern
Subfile Type                    : Full-resolution Image
Image Width                     : 3192
Image Height                    : 1200
Bits Per Sample                 : 14
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Strip Offsets                   : 33792
Samples Per Pixel               : 1
Rows Per Strip                  : 1200
Strip Byte Counts               : 6703200
X Resolution                    : 180
Y Resolution                    : 180
Planar Configuration            : Chunky
Resolution Unit                 : inches
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 0 1 1 2
Black Level                     : 1792
White Level                     : 16200
Default Crop Origin             : 0 0
Default Crop Size               : 2884 1080
Active Area                     : 54 264 1134 3148
DNG Version                     : 1.3.0.0
DNG Backward Version            : 1.3.0.0
Unique Camera Model             : Canon EOS 5D Mark II
Color Matrix 1                  : 0.4716 0.0603 -0.083 -0.7798 1.5474 0.248 -0.1
496 0.1937 0.6651
Analog Balance                  : 1 1 1
As Shot Neutral                 : 0.473635 1 0.624
Baseline Exposure               : undef
Baseline Noise                  : 1
Baseline Sharpness              : 1.333333333
Linear Response Limit           : 1
Calibration Illuminant 1        : D65
Frame Rate                      : 25
CFA Pattern                     : [Red,Green][Green,Blue]
Image Size                      : 3192x1200
Megapixels                      : 3.8


Recorded a short 12 seconds raw video @10bit 2880x1080 23.976 here the ProRes_2880x1080_24p (https://drive.google.com/file/d/1Yae_P2I-ORNUv3xuDENs6r3iqm9y5dV-/view?usp=sharing) version of it , around 600 MB
The original raw video M01-2333.MLV (https://drive.google.com/file/d/1FTtNPhspYlu8HQgrWGIZi9qdbwujwRoe/view?usp=sharing) , it's about a 1GB
(https://image.ibb.co/g3v9pe/mlvproduder_2880x1080_5d2_10bit.png) (https://imgbb.com/)

Here a frame from that
(https://image.ibb.co/b7dqvK/M01_2333_000113_small.png) (https://imgbb.com/)

Awesome job waza57 , works great so far .
I must of really messed up the raw.c somewhere in my builds , I have to check out your code and see where I when wrong .
Thanks again  :D

If anyone missed it 5D2 10 bit 3K raw video is a reality !
Another Pipe Dream comes true  8)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Ottoga on September 02, 2018, 09:10:34 AM
QuoteIf anyone missed it 5D2 10 bit 3K raw video is a reality !
Another Pipe Dream comes true  8)

So cool Reddeercity, persistence and perseverance really paid off.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: scotophorus on September 02, 2018, 09:16:06 AM
i've been reading your updates on this thread for the last year and a half and now i have to spam you some thanks for this great achiement: holy sh*t! that's amazing! thank you very much @alex @reddercity @waza57 @dfort and everyone involved
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on September 02, 2018, 10:09:25 AM
Quote from: waza57 on September 02, 2018, 07:12:21 AM
@ a1ex
Do you see a ADTG[100C] with adtg_ gui ? i'don't see it with 5D2.

Yes, I see it after getting out of LiveView and back. Works as expected.

Looking for a 720p mode on 5D2? Now we need to figure out how to reduce the resolution in order to increase the frame rate :)

On 700D I've ran into a restriction that appears to be related to the live preview: modified resolution could not have an area (WxH) smaller than in the original video mode. The behavior is likely there on earlier models as well. What I think it's going on: the image processing modules that create the preview image will not get enough input data, and they will just keep waiting, freezing the preview and eventually crashing the camera. If the modified resolution is larger, the modules get enough input data, possibly with incorrect dimensions, they discard the extra data and the camera ends up displaying something (a garbled image); and it doesn't lock up.

edit: reducing resolution seems to work, to some extent, LiveView is frozen, camera didn't crash after ~ 1 minute (didn't try more), there are some black level issues, mlv_rec shows 1856x704 at 50.004 FPS.

(https://a1ex.magiclantern.fm/bleeding-edge/5D2/720p.png)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: kwstas on September 02, 2018, 11:31:15 AM
it's alive! congrats guys!

I can confirm
20 sec mlv, 2880x1080 @ 10 bit, though a -116px black at the right
13 sec mlv, 2540x1080 @ 12 bit

my card does 77mb at average

--
edit: cropped mode a bit unstable yet, seems not to be working now
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on September 02, 2018, 02:42:24 PM
@ a1ex

OK, thanks. I have clone  crop-rec4k branch and work on it with a copy of the adtg_gui folder of iso-research branch.
The things compile and seems to work like this and permitted a minimal  crop_work module to work, but:

adtg[100c] is not present even if i refresh livevew or go to play mode and back or go to x5 an go to play mode and...and.. ...nothing ...nothing.

C0F0713C  and C0F07150  i pray for  it won't lock up but god was not in my build.....

it's why , i would like to know, if it's possible:
 
1 - what is exactly the reference of your commit.
2 - Also, if you are currently using your "raw_lv_setedmac_patch" solution?
3 - if yes , is this solution works only with CONFIG_EDMAC_RAW_SLURP
4- And because I'm a little lost , what this solution does concretely.

@reddeercity

happy birthday anyway!

@kwstas

Do you use the new download link ? what is the problem ? have you apply my recommanded settings ?


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: kwstas on September 02, 2018, 06:33:18 PM
@waza57

fixed, the problem occurs while FPS override is enabled.
Yet, 1920 1:1 does not work for me and 3.5k gives me a freezed display while recording, but I think that happened before.
Basically, I wish we don't have to zoom in 5x mode and have all this side effects (low fps, greyscale, freezes etc) and use just some basic crop rectangle in 1:1 instead.. but I guess there should be some limitations.

Nevertheless superb job considering this September mkII celebrates its 10-year birthday.. :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on September 02, 2018, 08:01:23 PM
@ kwstas

Thank you for the feedback.
I'm not sure to correctly understand but :

Quotethe problem occurs while FPS override is enabled
Yes I have not alerted about this because I think it is the case for all cameras model with crop_rec module . if someone can confirm ?

Quotefreezed display while recording
the preview is bad when recording but the recording is good , no ? same thing ,  if someone can confirm wit other cameras model ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: kwstas on September 02, 2018, 08:30:27 PM
Quote from: waza57 on September 02, 2018, 08:01:23 PM
the preview is bad when recording but the recording is good , no ? same thing ,  if someone can confirm wit other cameras model ?

exactly, recording is flawless, preview is freezed.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 03, 2018, 06:01:43 AM
The frozen Liveview did happen on the 5D3 in crop_rec 4K if memory serve me correctly ,
I do think a1ex fixed that in mlv_lite , that the one I've been working from .
if you can fix the compiling problem/error in mlv_lite , has something to do with compressed raw 10,12 bit ?
I think I had it compiling in d4 4k crop_rec  , I do believe I just commented out the 10-12bit compressed stuff .
With mlv_lite the preview should be better , it where the bulk of the crop_rec 4k stuff is .

I can get around 30 second (average 79-80MB/s) after playing around with the srm write mode , "4" seem to be the fastest until a frame skips .
I did enabled "frame skipping" and shot a 8GB 1:40 second 2880x1080 clip , I really did notice any missing frames (unless it was just 1 frame)
It look seamless after rendering to a prores , next I'll try a talking head and see if there's a noticeable gap -- even if there is run a "bcam"
and kept recording 3k , if it's just one or two frame I can live with that for the monument .

Now next we need compression , that's lossless compressed raw . We are very close on the that front I think , the problem is the cam lockup's upon
saving a lossless silent photo (when we can save a lossless silent photo we should have it for raw video)
Here where we are at right now with 5D2/D4 cam's ProcessTwoInTwoOutLosslessPath  (https://www.magiclantern.fm/forum/index.php?topic=18443.msg205333#msg205333)

There may be another way also to increase the CF card write speed --
I notice in one of my LOG's that for some reason 5d2 clocks the UDMA at 6 , where the cards are rated for UDMA 7
825FE>  CSMgrTask:00095f98:00:00: *** register_interrupt("CFDriver", 0x82, 0xffb8b8cc, 0x0), from ffb8bb58
      .............
834A0>  CSMgrTask:ffb8a92c:22:05: [ID:Model Number] = LEXAR ATA FLASH CARD                   
834C8>  CSMgrTask:ffb8aabc:22:05: IDE = 4, PCMCIA = 80, UDMA = 6
      .............
83572>  CSMgrTask:ffbdb8a0:22:01: cfDecideTiming: UDMA Mode 6 (CFA4.0)
      .............
83592>  CSMgrTask:ffbdbb3c:22:03: CF_GetAccessTiming : DatTim = 3, DatMod = 6

Quote from a1ex here (https://www.magiclantern.fm/forum/index.php?topic=12862.msg199697#msg199697)
Quote@reddeercity: more details after I'll get a new card (it *is* possible to overclock the 5D2 CF interface).
Quote from: a1ex on April 11, 2018, 07:48:49 AM
You can send ATA commands to the CF card (QEMU emulates them), so if the only difference between UDMA 6 and 7 is timing, it might even be possible to put the card in UDMA7.
If this could work , then we would have some write speed as the 5d3 or close to it (120MB/s) anything we could gain is better then 79MB/s
some related stuff --> https://www.magiclantern.fm/forum/index.php?topic=21311.msg204395#msg204395

Lots to do yet ,  but we are on the way  :D
Edit: just notice the crop factor when down from almost 3 (2.8 ) to 1.95 ,
which mean it's more usable  , e.g. 24mm -> 46.8 mm
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on September 03, 2018, 09:07:35 AM
@reddeercity
thanks for this detailed feedback!

Quoteif you can fix the compiling problem/error in mlv_lite
yes , I'm working on it.

QuoteNow next we need compression , that's lossless compressed raw
yes, I saw your work with A1ex but but it's not my speciality yet.

Quoteincrease the CF card write speed
yes , good idea . but compressed raw wouldn't be good enough  ?

Quotecrop factor when down from almost 3 (2.8 ) to 1.95 , which mean it's more usable
yes, so, don't you think that it remains also and above a problem with frame height?

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 04, 2018, 12:54:14 AM
Quote from: waza57 on September 03, 2018, 09:07:35 AM
@reddeercity
thanks for this detailed feedback!
yes , good idea . but compressed raw wouldn't be good enough  ?

No we need the higher write speed , compressed raw would give us (at .5 compression , which I read would the average)
from http://rawcalculator.netlify.com/calculator_desktop
basic 10bit just out of reach of cf/5d2-d4 cams

Resolution:2880x1080 10bit
Data rate:88.9MiB/s
Frame size:3.7MiB
Crop factor:1.96x crop
Field of view:20.8°
RECORD TIME: 5 minutes 43 seconds per 32GB card.

14bit compressed raw at .5 compression
Data rate:62.2MiB/s

So this would be the max if lossless worked on 5d2 , better crop factor

Resolution:3200x1080
Data rate:69.1MiB/s
Frame size:2.9MiB
Crop factor:1.76x crop
Field of view:23.1°


16x9 2880 with increased height & compressed raw , still out reach of the current CF D4 bandwidth (75-80MB/s)
need faster interface --  overclock the cf bus driver or find away to run cards at udm7 , theoretically @ umd7 the write speed should be 120-133MB/s

Resolution:2880x1556
Data rate:89.7MiB/s
Frame size:3.7MiB
Crop factor:1.96x crop
Field of view:20.8°

Max rez with compressed raw with increased height at the current cf bandwidth (75MB/s)

Resolution:2880x1200
Data rate:69.1MiB/s
Frame size:2.9MiB
Crop factor:1.96x crop
Field of view:20.8°


I prefer for more horizontal rez , thou there nothing wrong with increasing the height to the max .

This is what I'm hoping to reach as a goal for raw video , it could happen only with 2 things
compressed raw (.5) & faster UMD 6 ->7 & or overclock the cf interface on the write side

Resolution:3840x1556
Data rate:119.5MiB/s
Frame size:5MiB
Crop factor:1.47x crop
Field of view:27.5°


Now , I basic all this on continuous recording for raw video  @ 24p
If some user what to use the full width at lower frame rate with increase height at maybe 10 of 12 fps
for hyperlays etc... will that's great , so there's really should be no limit on resolutions just
limitations on the presets in the crop_rec.mo .

At least now I can say there's no more aliasing & moiré patterns  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on September 04, 2018, 08:18:42 AM
Quote from: waza57 on September 03, 2018, 09:07:35 AM
yes , good idea . but compressed raw wouldn't be good enough  ?
Idea. What are the analog gain registers for 5D mark II? Dialing them down will darken output but also reduce bitrate in a lossless manner. For the 100D:
switch (crop_preset)
{
case CROP_PRESET_2K_100D:
case CROP_PRESET_3K_100D:
case CROP_PRESET_4K_100D:
{
/* assuming FPS timer B was overridden before this */
                int fps_timer_b = (shamem_read(0xC0F06014) & 0xFFFF) + 1;
                int readout_end = shamem_read(0xC0F06804) >> 16;    /* fixme: D5 only */

                /* PowerSaveTiming registers */
                /* after readout is finished, we can turn off the sensor until the next frame */
                /* we could also set these to 0; it will work, but the sensor will run a bit hotter */
                /* to be tested to find out exactly how much */

/* will add 10bit through analog gain */
adtg_new[0] = (struct adtg_new) {2, 0x8882, 0x46};
adtg_new[1] = (struct adtg_new) {2, 0x8884, 0x47};
adtg_new[2] = (struct adtg_new) {2, 0x8886, 0x46};
adtg_new[3] = (struct adtg_new) {2, 0x8888, 0x45};

/* even lower bits */
/* adtg_new[0] = (struct adtg_new) {2, 0x8882, 0x14};
adtg_new[1] = (struct adtg_new) {2, 0x8884, 0x15};
adtg_new[2] = (struct adtg_new) {2, 0x8886, 0x14};
adtg_new[3] = (struct adtg_new) {2, 0x8888, 0x13}; */

                adtg_new[4]  = (struct adtg_new) {6, 0x8172, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (6D/700D) */
                adtg_new[5]  = (struct adtg_new) {6, 0x8178, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (5D3/6D/700D) */
                adtg_new[6]  = (struct adtg_new) {6, 0x8196, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (5D3) */

                adtg_new[7]  = (struct adtg_new) {6, 0x8173, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (6D/700D) */
                adtg_new[8]  = (struct adtg_new) {6, 0x8179, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (5D3/6D/700D) */
                adtg_new[9]  = (struct adtg_new) {6, 0x8197, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (5D3) */

                adtg_new[10] = (struct adtg_new) {6, 0x82B6, nrzi_encode(readout_end - 1) }; /* PowerSaveTiming ON? (700D); 2 units below the "ON" timing from above */

                /* ReadOutTiming registers */
                /* these shouldn't be 0, as they affect the image */
                adtg_new[11] = (struct adtg_new) {6, 0x82F8, nrzi_encode(readout_end + 1) }; /* ReadOutTiming */
                adtg_new[12] = (struct adtg_new) {6, 0x82F9, nrzi_encode(fps_timer_b - 1) }; /* ReadOutTiming end? */
}
                break;
}
    }


This part creates lossless:
/* will add 10bit through analog gain */
adtg_new[0] = (struct adtg_new) {2, 0x8882, 0x46};
adtg_new[1] = (struct adtg_new) {2, 0x8884, 0x47};
adtg_new[2] = (struct adtg_new) {2, 0x8886, 0x46};
adtg_new[3] = (struct adtg_new) {2, 0x8888, 0x45};

/* even lower bits */
/* adtg_new[0] = (struct adtg_new) {2, 0x8882, 0x14};
adtg_new[1] = (struct adtg_new) {2, 0x8884, 0x15};
adtg_new[2] = (struct adtg_new) {2, 0x8886, 0x14};
adtg_new[3] = (struct adtg_new) {2, 0x8888, 0x13}; */


Checking your code maybe put in the analog gain registers here?:
    /* these should work on all presets, on all DIGIC 5 models and also on recent DIGIC 4 */
    if (1)                  //waza57 here, I don't think this have an effect on 5D2 until we find proper regs
    {
        /* assuming FPS timer B was overridden before this */
        int fps_timer_b = (shamem_read(0xC0F06014) & 0xFFFF) + 1;
        int readout_end = shamem_read(is_digic4 ? 0xC0F06088 : 0xC0F06804) >> 16;

        /* PowerSaveTiming registers */
        /* after readout is finished, we can turn off the sensor until the next frame */
        /* we could also set these to 0; it will work, but the sensor will run a bit hotter */
        /* to be tested to find out exactly how much */
        adtg_new[4]  = (struct adtg_new) {6, 0x8172, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (6D/700D) */
        adtg_new[5]  = (struct adtg_new) {6, 0x8178, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (5D3/6D/700D) */
        adtg_new[6]  = (struct adtg_new) {6, 0x8196, nrzi_encode(readout_end + 1) }; /* PowerSaveTiming ON (5D3) */

        adtg_new[7]  = (struct adtg_new) {6, 0x8173, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (6D/700D) */
        adtg_new[8]  = (struct adtg_new) {6, 0x8179, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (5D3/6D/700D) */
        adtg_new[9]  = (struct adtg_new) {6, 0x8197, nrzi_encode(fps_timer_b - 1) }; /* PowerSaveTiming OFF (5D3) */

        adtg_new[10] = (struct adtg_new) {6, 0x82B6, nrzi_encode(readout_end - 1) }; /* PowerSaveTiming ON? (700D); 2 units below the "ON" timing from above */

        /* ReadOutTiming registers */
        /* these shouldn't be 0, as they affect the image */
        adtg_new[11] = (struct adtg_new) {6, 0x82F8, nrzi_encode(readout_end + 1) }; /* ReadOutTiming */
        adtg_new[12] = (struct adtg_new) {6, 0x82F9, nrzi_encode(fps_timer_b - 1) }; /* ReadOutTiming end? */
    }


White level will have to follow but this can be altered on the fly in Mlv App, or else some if statement in raw.c or maybe building a crop_rec.c header file...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yobarry on September 05, 2018, 07:54:01 AM
Quote from: reddeercity on August 31, 2018, 08:47:07 AM
Looking up now ,  :)
I'm taken baby steps , able to increase the height now with
c0f06088 , A & B timers , target height is 2160 -- I'm at 1760


I'm a big fan of the work you've been doing for the 5D2! When I check back into the ML forum it's basically just to see what progress you're making. Big thank you for the recent achievements! How's the working coming along with increasing the height? Have you been able to save any footage at 1760px?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 05, 2018, 08:07:41 AM
Don't take this the wrong way but ,
Like I said before Lossless is not working period so the above is useless at this point , only uncompressed 10,12 & 14 bit works
For this thread 10bit uncompressed is what discussed as the norm . 

On the crop_rec.mo , I notice you can't change the ISO (at least didn't re-fresh in liveview) had to go to 10x adjust ISO then back again
before it would take effect in the crop_rec.mo 3.5k preset -- also tried to connect my hdmi zacuto evf while the in 3.5k preset , the screen what back
on the evf and recorded a few seconds and it change rez to 1856x1006 by it self . Un plugged the hdmi the lcd screen was back also .
flip back & fouth in 3x to 1:1 thern back to 3x toggle the crop_rec.mo off and on a few  then is was back to normal .

I think I know my problem with raw.c & "raw_slurp" I'm still using 0x02 as the write channel like the 12bit I use in 1:1 and I see in the Edmac.mo
I need to be on 0x0 .
I want to reproduce the experiment's from a1ex , like full width (5632x1074) & a1ex said he had figured out a 720p 50fps mode (1856x704 at 50.004 FPS)
Quote from: a1ex on September 02, 2018, 10:09:25 AM
Yes, I see it after getting out of LiveView and back. Works as expected.

Looking for a 720p mode on 5D2? Now we need to figure out how to reduce the resolution in order to increase the frame rate :)
    ...................
edit: reducing resolution seems to work, to some extent, LiveView is frozen, camera didn't crash after ~
1 minute (didn't try more), there are some black level issues, mlv_rec shows 1856x704 at 50.004 FPS.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 05, 2018, 08:12:58 AM
Quote from: yobarry on September 05, 2018, 07:54:01 AM
I'm a big fan of the work you've been doing for the 5D2! When I check back into the ML forum it's basically just to see what progress you're making.
Big thank you for the recent achievements! How's the working coming along with increasing the height? Have you been able to save any footage at 1760px?
No not yet , test out the 2880x1080 24p preset from waza57 the least few days , I posted video links to prores files & test results here (https://www.magiclantern.fm/forum/index.php?topic=11205.msg205674#msg205674)
I'll be back at it in few days  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on September 05, 2018, 08:14:26 AM
@Danne
Thanks , I do not know this . I'll try to understand.
This is it is noted in TODO list (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205624#msg205624)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on September 05, 2018, 08:36:45 AM
@Reddeercity

Ok, so,  if I understand ( my English is difficult) correctly:

with crop_rec , you can't change ISO settings and you have a black screen when you connect HDMi object?

question:
Is this the same when we use a 5D3 (with his crop_rec enabled)?

I need to know this for investigation , thanks.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 06, 2018, 08:08:37 AM
Quote from: waza57 on September 05, 2018, 08:36:45 AM
@Reddeercity
with crop_rec , you can't change ISO settings and you have a black screen when you connect HDMi object?
Yes that right , ISO can only be adjusted outside of 3x crop_rec like 10x zoom and is applied correctly .
There seem to be no HDMI buffer , if you do a image_dump in the debug menu the HDMI dump is black  ,
there data in the dump image but all black . Once I connected the HDMI the Overlays where still there
I could go in to ML menu's etc. ..... when I disconnected the HDMI the LCD back screen was black .

If I remember right I disabled the crop_rec to get back the screen , once that happen things where back to normal
and could use crop_rec again without any problems .

Just looking at the Edmac.mo , your read buffer
4096x4096
which I thing is 32MB , that good
write channel "0" connection#5
with a buffer of
7200x1896

Now playing around with HDMI again -- will , sometime the hdmi image is pink noise and another had the frozen liveview
I think maybe there too much over head with HDMI , I did record a 170 frames with HDMI connected , the mlv recording was all messed up
I'm rendering a h264 version out M05-2314.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M05-2314.mov) (15Mb)
Liveview dump with HDMI
(https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-000_small.png)
This is what the HDMI was showing , I took a photo after I dump the liveview image
(https://image.ibb.co/eFNhje/hdmi_small.png) (https://imgbb.com/)
here the LV-000.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-000.422) with HDMI connected & the HD-000.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/HD-000.422) dumps

Another weird thing , if you disable the overlay to "Always" the Liveview freezes , but still able to record raw video
When you re-enable the Overlays the Liveview is back to normal .
got a idea , switch GD to allow is what happen , I thinking liveview will be not frozen no didn't work
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on September 07, 2018, 12:46:11 AM
OK, i added all this in my list of more and more things to do.
My version of crop_rec is essentially based on things known for a long time (see here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205624#msg205624) for reminder)
I also think that A1ex seemed to be about to rewrite a crop-rec module that will work for 5D2 and probably more academically than I could do it.
With changes I do not know:

ADTG [100C]                                      -> I still do not see it (I guess good for the size)
C0F07150 and C0F0713C                  -> which does not change in with the code of the current crop_rec (possibly using adtg_gui.  I suppose good for the height)
"raw_lv_setedmac_patch" solution? -> with or without CONFIG_EDMAC_SLURP? (I guess good for the memory)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 07, 2018, 06:49:39 AM
One thing I haven't tried yet was to "force hdmi to low rez."
There a option to force hdmi to 720x480 the same size as liveview , I'll try tomorrow .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 12georgiadis on September 07, 2018, 07:11:09 PM
Quote from: reddeercity on September 07, 2018, 06:49:39 AM
One thing I haven't tried yet was to "force hdmi to low rez."
There a option to force hdmi to 720x480 the same size as liveview , I'll try tomorrow .
For one project, I needed a monitor HDMI + raw rec on 7D and had a black screen. Force to VGA low rez solved the problem for liveview
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on September 07, 2018, 08:44:48 PM
Congtartiulations waza57 and reddeercity!!! Sorry I never got involved, magic lantern development/hacking just seems to be something I can not get the hang of even a tiny bit.

THis is amazing I will try and fix my 5D mark II as soon as I can. I am truly excited.

The PNG samples on 5D2 thread have really jagged edges, could you upload some dngs?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 08, 2018, 05:10:41 AM
Quote from: Ilia3101 on September 07, 2018, 08:44:48 PM
Congtartiulations waza57 and reddeercity!!! ....
The PNG samples on 5D2 thread have really jagged edges, could you upload some dngs?

Sure , here you go , I dumped the dng's to true 10bit .  I had to use a Old version of mlv_dump , the new one keeps upsampleing to 14bit for some reason .
the difference is size , 14bit 5.3MB & 10bit 3.7 MB
M02-1800_000000_2880x1080_10bit.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M02-1800_000000_2880x1080_10bit.dng)
M02-1810_000000_2880x1080_10bit.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M02-1810_000000_2880x1080_10bit.dng)
M02-1802_000010_2880x1080_10bit.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M02-1802_000010_2880x1080_10bit.dng)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 08, 2018, 05:20:26 AM
@waza57 , tried to "force hdmi to VGA" (720x480) just pink noise , but there is movement when recording raw video , the preview is not frozen .
Looks like frame rate (fps) is out or the size of the buffer that feed in to the encoder is messed up .
more low level investigation is needed .

Edit: checking the mlv file I recorded (2880x1080) with HDMI force to VGA (720x480)
No frame corruption as seen on the previous file I posted here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205719#msg205719) Made a quick h264 file of that recording here (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M07-2101-hdmi_forced-to-720x480.mov)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 08, 2018, 06:15:07 AM
Something interesting , I think this for D4 cams 5d2 etc. .... from http://magiclantern.wikia.com/wiki/Register_Map
0xC0F383D4 Preview area (y1 << 16) | (x1/4), similar to raw_info.active_area (5D3)
0xC0F383DC Preview area (y2 << 16) | (x2/4)

useful ? needs investigation
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 08, 2018, 06:47:27 AM
@waza57 , this could useful for HDMI preview , from my decompiled 5d2 rom
https://www.magiclantern.fm/forum/index.php?topic=21311.msg197060#msg197060
ff0316b4 FramTable == NULL
ff031b47 @DP_SetCroppingFrameLocation(X:%d Y:%d W:%d H:%d)
ff031b7c BaseW:%d BaseH:%d Angle:%d
ff031b9f @DP_SetCroppingFrameLocation FrameSize Exchange To HDMI
ff031bd8 Angle == 3600

Would be nice to know what the in the "framtable" 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on September 08, 2018, 08:11:28 PM
Thanks for the DNGs they look wonderful, highlights ont he first one are smooth 😍
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 10, 2018, 06:06:18 AM
Notice on the 3.5k preset the image is not centered , it looks like it's Left and down of center
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on September 11, 2018, 06:30:15 PM
Can you not move it like in normal crop recording?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 13, 2018, 03:31:56 AM
Quote from: Ilia3101 on September 11, 2018, 06:30:15 PM
Can you not move it like in normal crop recording?
No , I think because the crop windows has increased from default .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 14, 2018, 05:42:44 AM
I'm taking a break from magic lantern , so don't expect any update .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: IDA_ML on September 14, 2018, 08:03:27 AM
I am really sorry to hear that, Reddeercity.  Now, that you achieved such a remarkable progress with the 5D2, provided so much useful information for all D4 cameras and there is finally hope for 4K-crop recording on them.  I understand, you feel humiliated but please be aware that your efforts are greatly appreciated by many people on the forum.  That is why, I hope, you will reconsider your decision and will continue providing updates on your work. 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 12georgiadis on September 14, 2018, 09:55:21 AM
@Reddeercity, you helped us a lot with your research. Please don't give up. You've done an amazing work for D4 improvement.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on September 14, 2018, 08:19:20 PM
@reddeercity: sad to read that. I enjoyed every day reading your messages, what you found out. You are so close now - the goal should not be far! :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on September 15, 2018, 12:24:31 AM
@reddeercity Thank you for all you have done! I can imagine you are tired, even from the occasional checking I could see it was a lot you were doing. Really excited to get back to this stuff when I get my camera back :D :D :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 16, 2018, 07:05:41 AM
Quote from: waza57 on September 15, 2018, 11:39:57 PM
with the value of  1000102,  I have get a big improvement of around 30% speed write.
Sadly , this corrupt the card filesystem .  ( i can't read mlv file in card and i must format the card if i try to delete it)
........hey , reddeercity, do you hear this?  ;)

Yes , nice -- Can you do a benchmark in cam with out it corrupting the file system ?
would be nice to see a LOG file . either in dm-spy or mmio to see what the card connects at .
I'm wondering if there more to UDMA7 then the reg. e.g. CF Timing
83592>  CSMgrTask:ffbdbb3c:22:03: CF_GetAccessTiming : DatTim = 3, DatMod = 6
So I would expect to see
DatMod = 7
assuming this is UMDA but not too sure what to see for
DatTim = 3
maybe ?
DatTim = 4
A LOG file would confirm this
Or does the cam go to auto mode went UDMA7 is forced
This is from dm-spy log reviewing a cr2 or a h264 .mov file , not sure which one .

825FE>  CSMgrTask:00095f98:00:00: *** register_interrupt("CFDriver", 0x82, 0xffb8b8cc, 0x0), from ffb8bb58
834A0>  CSMgrTask:ffb8a92c:22:05: [ID:Model Number] = LEXAR ATA FLASH CARD                   
834C8>  CSMgrTask:ffb8aabc:22:05: IDE = 4, PCMCIA = 80, UDMA = 6
83572>  CSMgrTask:ffbdb8a0:22:01: cfDecideTiming: UDMA Mode 6 (CFA4.0)
83592>  CSMgrTask:ffbdbb3c:22:03: CF_GetAccessTiming : DatTim = 3, DatMod = 6



To all the core supporters  , THANKS
I'll keep checking in .
Later I'll post a explanation for my decision to pull back from ML 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on September 16, 2018, 09:01:07 AM
The hardware only cares about what gets written to MMIO registers (or DMA channels). What hardware we've got here?
- CF controller (from camera); likely that's where the data transfer clocks are generated
- CF card itself (this one may also have to know about UDMA mode; it's configured with IDE commands, also sent via MMIO)

What gets into debug messages (DatTim, DatMod, UDMA Mode) are just some variables in Canon code that only live in the ARM processor, i.e. they don't reach the hardware. Sure, their value decides what reaches the hardware, but in many cases we can overwrite that later (and that will happen without changing the original debug messages). The hardware doesn't look at debug messages - these are just human-readable messages written by Canon engineers for themselves.

So, you won't necessarily see DatTim = 4 or DatMod = 7, unless you patch Canon code to print that. Either you change that message (easy, but the hardware won't listen), or you change the message AND add the code to handle these values (not present in the original Canon firmware), or - sometimes easier - you just write a different value to these registers later. That last approach (i.e. changes done via DIGIC Poke) won't show up in a regular DebugMsg log, but will show up in a MMIO log.

So, the first half of the story is probably covered by the registers waza tried. We know they are somehow related to clock generation, lower numbers appear to result in higher frequencies, but that's pretty much it.

The second half of the story may or may be necessary. Some cards are able to work at (slightly) different speeds than what was declared in the IDE commands, others maybe not.

Sorry I'm not always able to give a complete answer quickly enough. Some topics, such as this one, require experimenting, as I don't know the answer in advance. I'm physically unable to do a fair "round robin" over all the active ML topics during let's say one week, as that would require too many "context switches" for me (and I'm not good at that), so inevitably some topics are going to be delayed. Seeing a message (with new useful content) on some topic reminds me to look into it; seeing the same message in 3 different places... triggers a priority drop on my side. Others may react in different ways; no need to get mad at them just for that.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on September 16, 2018, 09:31:47 AM
@a1ex +1

There is nothing personal about getting a little forum reminder now and then. I like the fact that Audionut is checking in on things. Keep up the good work everyone.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 17, 2018, 05:17:29 AM
I did a head shot to see how bad with frame skipping enable is , usable but the frame drops are noticeable
had to cut up the audio a lot to make it sync up , it's still not 100% but close (be kind please)

I what to show the concept of near UHD (3k) 10bit raw video (thanks waza57  :D)

2880x1080 23.976p 10bit , EF 24-70mm @ 24x 1.95 crop factor = 48.6mm , 1/42th sec shutter , 400 ISO
mlv_dump to true 10bit  exported from Resolve 12.5 to rec709 DNxHR 444 , finished off in FCPX uploaded file ProRes422Hq
Hopefully this will encourage other 5d2/d4 cam user to join in


Since Youtube doesn't have 2k or 3k options it only play at 1080p
so I upload a mp4 to my goggle drive at the native resolution (2880x1080p) 600MB
https://drive.google.com/file/d/1obTz9AyEglhv9u3wjKS6x2mDWVxdkU79/view?usp=sharing

@a1ex thanks , but I feel I need to clear up something here first before I continue on (It's too important now to drop this development)
I think there's being a big miss understanding here ,
My original intent was not spam you with new threads & questions , My logic with to put all the CF card research for d4 cams in one place without
trying find little bit of info here & there and the UHS-I / SD cards investigation thread , to me that was the wrong thread to continue on
the discussion being it not cf card . So I could not understand why that first thread was locked , to me it was a road block for some reason towards me .
specially after I thought I made my self clear in the second thread I start with the same topic heading , and that turn in to a shit show as you know .

I still think a separate thread for CF card development is needed , but that my own opinion .
I'll just work from this thread for now on .
Sorry for the confusion .


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Audionut on September 17, 2018, 07:16:42 AM
Quote from: reddeercity on September 17, 2018, 05:17:29 AM
specially after I thought I made my self clear in the second thread I start with the same topic heading , and that turn in to a shit show as you know .

I believe that I need to set a standard, publicly.

http://www.catb.org/esr/faqs/smart-questions.html#keepcool
QuoteMuch of what looks like rudeness in hacker circles is not intended to give offense. Rather, it's the product of the direct, cut-through-the-bullshit communications style that is natural to people who are more concerned about solving problems than making others feel warm and fuzzy................it may help you cope with our eccentricities if you think of us as being brain-damaged....

I attempted (I am not a robot, I am fallible) in the first thread to show some friendliness.
Quote from: Audionut on September 13, 2018, 03:55:04 PM
Sorry my reply wasn't useful.

Regards,
Audionut.

The second thread, was pure emotion.  It was made public, and it was dealt with publicly (see first line of reply in this post).  Taken to PM would have been significantly better.

I have critically maintained this public standard with you in the past, but having said that, at any point in time..........I could have also handled the matter in a more private nature, showing more respect to your contributions to the community.

Emotions take time to process from this keyboard, so I (unfortunately on some occasions) revert simply to,
QuoteExaggeratedly "friendly" (in that fashion) or useful: Pick one.

I'm sure somehow, at some stage in the future, that I can be better at being useful, and also a little (more) friendly at the same time.  Maybe we could start some banter in general chat or something, where we can all just shoot some shit (https://www.urbandictionary.com/define.php?term=shoot%20the%20shit).  We could discuss topics in this thread, such as, hey Audionut, I'm trying to start a new thread regarding CF cards, not SD cards, you dumb shit!  Why you close my thread for?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on September 17, 2018, 06:28:54 PM
Quote from: reddeercity on September 17, 2018, 05:17:29 AM
I still think a separate thread for CF card development is needed , but that my own opinion .

A while back my opinion was that in order to get crop_rec_4k working on Digic 4 cameras we needed to figure out 10bit/12bit (https://www.magiclantern.fm/forum/index.php?topic=5601.0), lossless compression (https://www.magiclantern.fm/forum/index.php?topic=18443.0), get the basic crop_rec module (https://www.magiclantern.fm/forum/index.php?topic=17021.0) working then finally tackle crop_rec on steroids (https://www.magiclantern.fm/forum/index.php?topic=19300.0) and speeding up the SD/CD read/write speeds (https://www.magiclantern.fm/forum/index.php?topic=12862.0). Instead of trying to follow so many development discussions that had only a few posts related to Digic 4 cameras, I thought it might be better to keep all Digic 4 advanced development on this topic but I got chased off (https://www.magiclantern.fm/forum/index.php?topic=19336.msg194212#msg194212).

Hey, you did a great job with the theory and @waza57 figured out the code and posted a test build for 5D2 users (https://bitbucket.org/waza57/magic-lantern/downloads/). There are several other cameras that could benefit from your work and there is still a lot more to do before Digic 4 can be merged into the crop_rec_4k branch so let's not hold a grudge and keep the momentum going.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 23, 2018, 07:54:33 AM
Find a little more info on CF cards , https://www.compactflash.org/cf-cards
(https://image.ibb.co/iQwRZ9/CF_Card_Spec_small.png) (https://imgbb.com/)

83572>  CSMgrTask:ffbdb8a0:22:01: cfDecideTiming: UDMA Mode 6 (CFA4.0)
So there the limit on the 5D2 , the card connects at udma6 @ CF4.0 = about 90MB/s

Does the 5D3 @ udma7 run at CF6.0 ?
Should be able to tell from a dm-spy log , I found mind after reviewing a cr2 file .
Can any 5d3 user confirm this please .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on September 23, 2018, 08:01:00 AM
Quote from: reddeercity on September 23, 2018, 07:54:33 AM
Does the 5D3 @ udma7 run at CF6.0 ?

I bet it does 6.0 or 6.1:
(http://i781.photobucket.com/albums/yy98/blitzkrieg669/5D/BENCH_SDCFXPS-064G_zpsf97r5i3e.png)
Screenshot by blitzkrieg666 in message https://www.magiclantern.fm/forum/index.php?topic=12630.msg161435#msg161435
Close numbers to firmware 1.1.3 as seen in https://www.magiclantern.fm/forum/index.php?topic=12630.msg144600#msg144600
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 23, 2018, 08:41:49 AM
Thanks  Walter , yes more then likely .
Found a very complete CF Card specifications pdf from 2005 , a little old (CF+ & CF SPECIFICATION REV. 3.0)
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/cfspc3_0.pdf
On page 67 there is Timing Modes definitions
(https://image.ibb.co/gWFH7U/CF_Card_Spec_timing_mode_small.png) (https://imgbb.com/)
83592>  CSMgrTask:ffbdbb3c:22:03: CF_GetAccessTiming : DatTim = 3, DatMod = 6
From this it looks like "DatTim3" = 100 ns  , So does that mean 5d3 run in Mode 4 @ 80ns ?
I maybe wrong here , but it sure looks like it -- did waza57 test with udma7 on 5d2 need to adjust the mode time to shorter  e.g. 80ns ?
I my need to run some test in forced udma7 and see if I can get some logs

Edit:On Page 88-- noticed that CF cards can be run @ 8bit data & or 16bit data from the pdf , so maybe the d4 cams are in 8bit data write mode instead of 16bit data write mode

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on September 23, 2018, 08:54:48 AM
IMO (see page 2) all specs by CFA are copyrighted (outdated or not) and links to it might be able to harm ML site owner.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 23, 2018, 09:17:21 AM
Ok , I put the PDF file on my bitbucket downloads & if that's not ok then I'll remove the link
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on September 25, 2018, 01:00:29 PM
Some notes on fixing the live preview on 60D, after resolution overrides:

x5, x10:

C0F08518: 0x045109D7
C0F0851C: 0x00D601E6
C0F08520: 0x038A05FE

To find them: register diff with adtg_gui between two x5 configurations shifted horizontally by one position
(careful not to shift the captured image position; only the preview must be shifted).

Interpretation:
0x38A05FE - 0x00D601E6 => 1048x692.
"HD" YUV422 buffer size in these modes: 1024x680.

=> C0F0851C/C0F08520: output resolution, with some margins; these might be trimmed later.

Changing the lower half of C0F0851C and C0F08520 by the same amount: horizontal translation.
Same for the upper half: vertical translation.

C0F08518: 0x045109D7 = 2519x1105.
From C0F06084/88: 2520 x 1105. So, register C0F08518 specifies the "input" resolution (i.e. captured Bayer data). Subtract 1 on the X axis.

Increase X resolution by 1 unit from C0F06088: 0x45205B6 -> 0x45205B7 (1 unit = 2 pixels)

Fix the preview by adding 2 units to C0F08518: 0x45109D7 -> 0x45109D9.

Register C0F08518 is not used in 1x. It also appears in 640x480 crop mode, with value 0x026F0397. Raw resolution in this mode is 920 x 623. Check.

Register C0F08518 also appears on 5D2, 5x, with value 0x04670907. Raw resolution 2312 x 1127. Check. To fix the preview, C0F08188 has to be updated as well.

It also appears on 700D, CR2 capture, value 0x0DC80A4F. Raw resolution 5280 x 3529. Check, after adding 1 and multiplying by 2 on the X axis.

It does not appear on 700D in LiveView. It does not appear at all on 5D3.

Related: Bilal notes for 700D / mine for 60D: https://www.magiclantern.fm/forum/index.php?topic=9741.msg203674#msg203674

TLDR: looks like we've got a way to:
- specify the RAW (Bayer) input buffer size (for the image processing path that creates the live preview image in x5)
- pan the preview window around (without resizing it)

For image resizing: use adtg_gui to "diff" the registers between x5 and x10, then lock their values. Preview in x5 and x10 will stay the same. Figure out the meaning of these registers to find out how to apply arbitrary scaling to the preview image.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on September 25, 2018, 10:18:28 PM
Please, What are the problems fixed here?

1 - The size of the preview (before or during the recording)?
2 - The preview is frozen or very slow ((before or during recording)?
3 - Bad previewing with artefacts (before or during recording)?

(On 5D2 the only issue (for me) is that the preview is too slow )

Thanks.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on September 25, 2018, 10:35:22 PM
That's right; CPU-based preview is slow, hardware-based real-time preview is... well... hard.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 26, 2018, 05:28:43 AM
In the 5d2 & 5d3.113 rom disassembly found the same "cfDecideTiming" ,
Looking for the parameter that decide timing  & UDMA mode , still no luck yet
5D2 rom disassembly
ff3db894: e28f2f55 add r2, pc, #340 ; ff3db9f0: (65446663)  *"cfDecideTiming: UDMA Mode %d (CFA4.0)"
ff3db8b8: e28f2f56 add r2, pc, #344 ; ff3dba18: (65476663)  *"cfGetRegisterTiming: I/O %dnS (CFA3.0)"
ff3db8d8: 328f2e16 addcc r2, pc, #352 ; ff3dba40: (65476663)  *"cfGetRegisterTiming: I/O 250nS"
ff3db9a4: e28f20ec add r2, pc, #236 ; ff3dba98: (65476663)  *"cfGetRegisterTiming: I/O 120nS"
ff3db9bc: e28f20f4 add r2, pc, #244 ; ff3dbab8: (65476663)  *"cfGetRegisterTiming: I/O 250nS (PIO Mode4)"


5D3 113 rom disassembly
ff727f18: e28f2f56 add r2, pc, #344 ; ff728078: (65446663)  *"cfDecideTiming: UDMA Mode %d (CFA4.0)"
ff727f3c: e28f2f57 add r2, pc, #348 ; ff7280a0: (65476663)  *"cfGetRegisterTiming: I/O %dnS (CFA3.0)"
ff727f5c: 328f2f59 addcc r2, pc, #356 ; ff7280c8: (65476663)  *"cfGetRegisterTiming: I/O 250nS"
ff72802c: e28f20ec add r2, pc, #236 ; ff728120: (65476663)  *"cfGetRegisterTiming: I/O 120nS"
ff728044: e28f20f4 add r2, pc, #244 ; ff728140: (65476663)  *"cfGetRegisterTiming: I/O 250nS (PIO Mode4)"

Interesting 5d2 nearly Identical to 5d3 113 . There's no reference to any mode higher then "cfa4.0" on 5D3.

Edit: Looks little a closer (5D2 disassembly)
ff38affc: e28f2f7d add r2, pc, #500 ; ff38b1f8: (64496663)  *"cfIdentifyDrive: Set UDMA( Mode=%d )"

(5D3 113 disassembly)
ff6aa9dc: e28f2f77 add r2, pc, #476 ; ff6aabc0: (64496663)  *"cfIdentifyDrive: Set UDMA( Mode=%d )"
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 26, 2018, 06:06:49 AM
I may have found the "CF card Configuration base"
RequestConfiguration: pLStorage=0x884e5c
81F67>  CSMgrTask:ffb8bcc8:22:01:                       ConfigBase=0x200, StatusReg=0x0
81F90>  CSMgrTask:ffb8bce0:22:01:                       PinRepReg=0x0, CopyReg=0x0
81FB6>  CSMgrTask:ffb8bcf8:22:01:                       ConfigIndex=0x1, Present=0xf
81FE2>  CSMgrTask:ffb8bd80:22:01: RequestConfiguration: Base = 200, Data = 41
8200E>  CSMgrTask:ffb8bde0:22:01: RequestConfiguration: SUCCESS
82031>  CSMgrTask:ffb8c688:22:01: CF_RequestConfiguration: err=0x0
82058>  CSMgrTask:ffb8c6a4:22:01: ConfigIndex = 1 Match


The "ConfigBase=0x200" sure looks very close to waza57 test (https://www.magiclantern.fm/forum/index.php?topic=12862.msg206010#msg206010)
0xC062850C --> 0x1213 and I obtain UDMA 0 speed (around 15MB/s)
                     --> ...........
                     --> ...........
                     -->0x0202 and I obtain UDMA 6 speed (around 80MB/s) i suppose original value

So could 0x200 be the same as 0x0202 or close , if so them maybe 0x100 will work instead 0x1000102 ? Just a thought  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on September 26, 2018, 09:30:57 AM
I have not 5D2  ROM disassembly to compare but when i see this :

QuoteThere's no reference to any mode higher then "cfa4.0" on 5D3.

I think it's a good news.

Could we think that "500" for 5D2 and "496" for 5D3 can be interpret as  parameters passed to function  "Set UDMA( Mode=%d )" ?

For 0xC062850C -> 100 , I have already tested and 5D2 freeze.



Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on September 27, 2018, 01:54:16 AM
Quote from: reddeercity on September 23, 2018, 08:41:49 AM
Found a very complete CF Card specifications pdf from 2005 , a little old (CF+ & CF SPECIFICATION REV. 3.0)
Also revision 4.0 (2006) can be found just by googling, but apparently no rev.4.1a neither 6.0

Instead latest revision can be bought from CFA website (https://www.compactflash.org/cfa-specifications)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 27, 2018, 04:52:50 AM
@waza57 I sent you a pm , should help with post #269
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 01, 2018, 03:43:47 AM
Found some info on the CF5.0 specification , looks like it was implemented in 2010
Key Feature Benefits :
Capacity points beyond current limitation of 137GB (up to 144PB) (1,024 TB = 1 PeteByte)
& more efficient data transfer (32MB per transfer versus 128K per transfer)
Data Set Management Command/Trim (Mandatory)
More efficient cleanup of unused space on memory card (LBA's)
Update ATA References to ATA-6 & ATA-8/ACS-2

QuoteMr. Shigeto Kanda of Canon and the CFA chairman of the board said
"The higher capacity and higher performance of CF cards enabled by the 48-bit addressing feature in the CF5.0 specification
will further increase the value of DSLR cameras. The Video Performance Guarantee feature of the CF5.0 specification
will help CF cards to expand into new markets such as high-speed movie equipment like professional video camcorders".

So it seems that there 48-bit addressing on CF5.0 vs CF4.0 32bit addressing
FAT File Allocation Tables https://en.wikipedia.org/wiki/File_Allocation_Table#FAT32
so can we trick 5d2 to cf5.0 for 48bit addressing , or is 48bit addressing only for exfat format ?
Still researching this .

Edit: Here the PDF file where I got the info about CF5.0
cf5_0_Spec100222.pdf (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/cf5_0_Spec100222.pdf)

found the CF6.0 Spec PDF news release
cf6_0_101118.pdf (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/cf6_0_101118.pdf)
This seems important information
QuoteCompactFlash cards are capable of 600x or 90MB/second throughput performance using the current Ultra DMA Mode 6.
CF 6.0 Ultra DMA Mode 7 along with 48-bit addressing defined in the CF 5.0 specification enables the development of
CompactFlash cards with up to 25% more throughput performance.
@waza57 This looks very close to your 30% (https://www.magiclantern.fm/forum/index.php?topic=12862.msg206061#msg206061) increase in write speed
So I guess we know why 5D2 is limited to around 79MB/s (UDMA6.0 CF4.0)

QuoteThe CF 6.0 specification includes a new Sanitize Command, Trim Usage Guidelines and an Operating Temperature Range function
along with adding Ultra DMA Mode 7 which supports 167 MB/second speed.
Benefits: Ultra DMA Mode 7
Key Feature Enhancement:
Provides an interface speed of 167MB/s.
This speed enhancement enables a new generation of higher performance cards while providing complete backward compatibility.

Benefits: Sanitize Command
Key Feature Enhancement: Provides an efficient NAND Block Erase of
the entire user data area to return the CF card to "fresh" state before reuse or repurposing.
Leverages a command defined in INCITS T13 ACS-2.

Benefits: Trim Usage Guidelines
Key Feature Enhancement: Provides improved write performance consistency.

Benefits: Operating Temperature Range
Key Feature Enhancement: An optional card capability to report the
operating temperature range of the card.
This allows card/camera combinations to enable use in extreme temperatures.code]

So from this
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 02, 2018, 09:04:06 AM
Trying to reproduce a1ex's full width experiment on 5d2 , It not totally right but close
I had a problem with cmos[2]  for some reason I couldn't get 0x00e 
QuoteFor full width LiveView:
CMOS[2] = 0x00E
C0F06008: 0x27705DB (only the lowest half appears to matter on this camera)
C0F06084: 0x30037 (copied lowest half from photo mode)
C0F06088: 0x46A0B87 (copied lowest half from photo mode)
=> 5632x1074 in mlv_rec, 12.5 FPS.
everything else work out , not sure why I can't , the best I could get is 0xe , So left it at default "0x10e" (will try later again on "cmos[2]" )
Did a Image dump , it did save a successful DNG & Liveview 422 image

I was able to save a mlv in 10bit 12.5fps 3520x1074 about 100 frames (max size in normal 3x crop_mode)
Here a sample frame M02-0002_00040.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M02-0002_00040.png)
(https://preview.ibb.co/kzD2uK/M02_0002_00040_small.png) (https://ibb.co/mXpaEK)

The DNG dump was 5632x1074 , 5.239 aspect ratio . RAW-004.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-004.DNG) tif file RAW-004.tif (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-004.tif)
(https://image.ibb.co/dxthTe/RAW_004_small.png) (https://imgbb.com/)
Active Area    :52 160 1127 5792
File Name                       : RAW-004 (2).DNG
File Size                       : 11 MB
File Modification Date/Time     : 2018:10:02 00:01:54-06:00
File Access Date/Time           : 2018:10:02 00:51:47-06:00
File Creation Date/Time         : 2018:10:02 00:51:47-06:00
Camera Model Name               : Canon EOS 5D Mark II
Orientation                     : Horizontal (normal)
Software                        : Magic Lantern
Subfile Type                    : Full-resolution Image
Image Width                     : 5792
Image Height                    : 1127
Bits Per Sample                 : 14
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Strip Offsets                   : 33792
Samples Per Pixel               : 1
Rows Per Strip                  : 1127
Strip Byte Counts               : 11423272
X Resolution                    : 180
Y Resolution                    : 180
Planar Configuration            : Chunky
Resolution Unit                 : inches
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 0 1 1 2
Black Level                     : 1277
White Level                     : 16200
Default Crop Origin             : 0 0
Default Crop Size               : 5632 1075
Active Area                     : 52 160 1127 5792
Unique Camera Model             : Canon EOS 5D Mark II
Color Matrix 1                  : 0.4716 0.0603 -0.083 -0.7798 1.5474 0.248 -0.1
496 0.1937 0.6651
Analog Balance                  : 1 1 1
As Shot Neutral                 : 0.473635 1 0.624
Calibration Illuminant 1        : D65
Frame Rate                      : 25
CFA Pattern                     : [Red,Green][Green,Blue]
Image Size                      : 5792x1127
Megapixels                      : 6.5


The liveview dump
(https://image.ibb.co/bTN6Mz/LV_012_small.png) (https://imgbb.com/)

I what to reproduce all of a1ex crop_rec experiment on the 5d2 , specially the 50p 1856x704 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205641#msg205641) ;D
I would like to put these resolution's in the crop_rec module at some point , even though the write speed on the 5d2 is slow right now it would be nice to have this part done.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 02, 2018, 09:27:23 AM
This is interesting , after recording that short mlv file @ 3520x1074 @12.5fps I was getting very high right speeds and I
don't change anything else
# Config file for module mlv_rec (MLV_REC.MO)
mlv.video.enabled = 1
mlv.res.x = 9
mlv.bpp = 0
mlv.aspect_ratio = 16
mlv.write_speed = 8527
mlv.preview = 2
mlv.display_rec_info = 2

So 85.27MB/s  :o , not sure why maybe 12.5 fps ? can't see it
this is a mystery why , but I like it !
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: theBilalFakhouri on October 02, 2018, 09:37:49 AM
Not really a mystery it's same in all other cameras (https://www.magiclantern.fm/forum/index.php?topic=12862.msg202507#msg202507) :P
Quote from: a1ex on October 01, 2018, 04:42:56 PM
Yes, it's normal.

If capture speed is greater than writing speed (i.e. recording is not continuous): memory bus is the main limit. If you capture at a higher data rate (e.g. higher resolution and/or FPS), there won't be enough bandwidth left for file I/O at full speed. I'm not sure how DMA priorities are working, or whether there's any way to adjust them. This is why the hacks that freeze LiveView are able to achive slightly better speed - they turn off some "useless" image streams, freeing some memory bandwidth.

Otherwise... the buffer will be mostly empty and the speed will be limited by image capture speed. In this case, mlv_lite will try to "benchmark" only when the card is actively writing, and may print the "idle" time as well. However, this method is just an approximation: there will be some overhead caused by small buffer sizes (as the raw recording task starts writing to card as soon as it has completed frames in the buffer, rather than waiting for a huge contiguous chunk to be completed). No big deal, as in this case, recording will be continuous anyway. It may affect the estimated time for the next recording.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 03, 2018, 09:59:30 AM
Yes finally 50p on the 5D2  8)

(https://image.ibb.co/cwR66z/VRAM3_small.png) (https://imgbb.com/)

Got a1ex's experiment to run on my 5d2 , it was a little tricky , had a few freezes with c0f0713c & c0f07150
and take me a while to figure out that I need to start with 30p NTSC in the Canon Menu  :-[

(https://image.ibb.co/kLLute/VRAM2_samll.png) (https://imgbb.com/)

but after that I needed to adjust the registers in a certain order or else it would lockup/freeze
So it go like this :
CMOS[1]
c0f06088
c0f06008
ADTG12[100c]
c0f0713c
c0f07150
c0f06014

Got not too bad of a image , needs work yet to clean up totally , did a image dump , got DNG, 422 .
I was able to record a squeezed 1856x704 10bit @50p for about 1000 frames , after unsqueezed I get 1856x1172
then crop off the bottom 266 lines to get 1856x906 .Very smooth frame rate , lots of detail , can't wait to try  some slow-mo shots  :)

(https://image.ibb.co/eWLBrz/50p_1856x704_mlvproducer_file_info.png) (https://imgbb.com/)

Here the RAW-1880x705.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-1880x705.DNG) from the image dump ,  squeezed (1880x704)
(https://image.ibb.co/gH3jde/RAW_004_3_small.png) (https://imgbb.com/)

Un-squeezed to 1880x906
(https://image.ibb.co/jjkajK/RAW_004_3_1880x902_small.png) (https://imgbb.com/)

here a short 56Mb I exported from MLVProducer Un-Squeezed with 3x5 option M03-0028_1856x906_50p_h264.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M03-0028_1856x906_50p_h264.mov)

(https://image.ibb.co/cAoT4K/50p_1856x704_mlvproducer_Small.png) (https://imgbb.com/)

(https://image.ibb.co/egwWPK/50p_1856x1172_mlvproducer_unsqeezed_small.png) (https://imgbb.com/)

exported frame from mlvproducer un-squeezed & cropped
(https://image.ibb.co/d6nKde/50p_1856x1172_after_crop1856x906_unsqeezed_small.png) (https://imgbb.com/)

Now to get Full width working properly , that's my next challenge .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ch_d on October 03, 2018, 10:42:01 PM
OMG! ML team - You are the best!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 04, 2018, 08:22:11 AM
This is a continuation of post #278 -- Seeing how far I push the frame rate , got to 71.737fps before it froze/Locked up , had to pull battery.
I was looking for 75fps , I thinks it there just need to play around more , even thinking I can push it to 100 fps  :D 

(https://image.ibb.co/b3NTpK/5d2_71fps_small.png) (https://imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on October 04, 2018, 11:46:15 AM
Really? 70FPS on 5d mk II? Guys i'm following you here on this topic but never write. So this my first time. You are awesome.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: CITY-U1001 on October 04, 2018, 04:00:46 PM
reddeercity , show test video in slow mo :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 06, 2018, 05:33:06 AM
Will it looks like I made a mistake on the 50p experiment from a1ex & I'm happy I did !
After going thought it again , I made a error by not refreshing Liveview after setting CMOS[1]
So after the 3x5 unsqueezed  I get 1856x1172  :o  not 1856x906 .
M05-2017_1856x1172_50p_h264.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M05-2017_1856x1172_50p_h264.mov)
Got a really clean image now but black level is off a little as per a1ex post .

Here the squeezed RAW-007_1880x705_50p.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-007_1880x705_50p.DNG) from image dump
(https://image.ibb.co/hV2YuK/RAW_007_1880x705_50p_small.png) (https://imgbb.com/)

RAW-007_1880x705_50p_Unsqueezed_1880_1172_50p.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-007_1880x705_50p_Unsqueezed_1880_1172_50p.png)
(https://image.ibb.co/nOxcMz/RAW_007_1880x705_50p_Unsqueezed_1880_1172_50p_small.png) (https://imgbb.com/)

Exported frame with some CC from MLVProducer
(https://image.ibb.co/db9DuK/M05_2017_00036_1856x1172_50p_small.png) (https://imgbb.com/)

Liveview 422 image dump
(https://image.ibb.co/meB9ZK/LV_016_small.png) (https://imgbb.com/)

Screen shot of Liveview
(https://image.ibb.co/hRa08e/VRAM9_small.png) (https://imgbb.com/)

In short , This is As exciting As UHD (2880x1080) on the 5D2 or maybe better in some cases .
To sum it up we are within reach of stable 50p @ 1856x1127 (records @ 1856x705) that's better then the standard
1856x1044 @ 10 & 14bit 24fps , maybe it's time to look in to 3x1 & 1x3 binding to reduce aliasing & moire pattern .
8)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: IDA_ML on October 06, 2018, 12:48:35 PM
A nice step forward, Reddeercity!  Keep going!

For those of you who still question if it is worth the effort to use Magic Lantern in these old Digic 4 cameras, please watch this video:

https://www.youtube.com/watch?v=WTr2L3uCYts

Imagine, we also have 4K crop recording working ...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: goldenchild9to5 on October 06, 2018, 03:44:37 PM
@reddeercity That's unbelievable.. the 5D Mark 2 capable of doing slowmotion video WOW.  Great progress Team  8)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 07, 2018, 07:05:45 AM
Dialed down the frame rate to 48.015 fps have a really clean image now ,


black levels are very close "1654-16200" , seems to clean up at "1730" in mlvproducer
(https://image.ibb.co/e2sN69/48fps_info.png) (https://imgbb.com/)

8 second test clip [email protected] h264.mov  (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M06-2218_1856x1172_48p.mov)

(https://image.ibb.co/jgqLtp/M06_2218_00001_small.png) (https://imgbb.com/)
Frame from mlvproducer with good black levels @ 1730 with some CC ,
Looks almost perfect M06-2218_000011856x1172_48p.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M06-2218_000011856x1172_48p.png)

So I think 48fps is a better fit for the 5D2 (black levels are better mainly), one issue off the bat I notice I was locked to 1/333 on the shutter plus had a frozen Liveview.
Thou this could be because of ADTG_GUI taken up resource's slowing down liveview I guess .
The 2 frames different is not noticeable 50->48 and I have a better black level . More work needed of course , getting closer  ;)   

Edit: here's the Cdng from mlvfs M06-2218_000000_1856x704_48p.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M06-2218_000000_1856x704_48p.dng)

Found something interesting about global shutter on CMOS sensors https://www.qimaging.com/ccdorscmos/triggering.php
Just wondering if I'm close to a global shutter with 48fps with 1/333 of a sec ?
I never thought it could be possible on a CMOS , just know that CCD could be global shutter , Thing you learn while researching  :D

(https://preview.ibb.co/edY1G9/Untitled.png) (https://ibb.co/j3cTw9)

from my file I'm at 3ms @ 48fps so theoretically 70-75fps is maybe a possibility or at least 60fps
Unless I'm miss understand this .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Levas on October 07, 2018, 12:57:41 PM
Interesting article about global shutter on CMOS.
But if I understand the stuff in the article right ??? they're using external light source for global shutter on CMOS.
They're using a LED or Laser which can be switched on and of fast enough for controlling exposure.
CMOS is exposed, but about only half of the shutter time is used for exposure, the external light source kicks in every frame and exposing for a shorter time then shutter time is set.
So no global shutter for normal cinema situations unfortunately, more for like fabrication/quality control and stuff.

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 08, 2018, 07:35:06 AM
Good news I got near 100fps :D Bad the image is all noise distortion  but didn't crash !

(https://image.ibb.co/b3MrJp/99_781fps_small.png) (https://imgbb.com/)

I still looking for stable 60fps , we do have 48p & 50p with black levels issue that dose work .

Changed Timers
c0f06008 0x23b023b -> 0x23b011d   **this alone gave me 59.94** (looks familiar , never try just that alone)
c0f06014 0x577 ->   0x348


(https://image.ibb.co/cDMGJp/VRAM14_small.png) (https://imgbb.com/)
Liveview is mess , I did record a few MLV's


Image Width                     : 1856
Image Height                    : 704
Camera Model Name               : Canon EOS 5D Mark II
Strip Offsets                   : 65536
Rows Per Strip                  : 704
Strip Byte Counts               : 2613248
Planar Configuration            : Chunky
Software                        : MLVFS
Exposure Time                   : 1/250
F Number                        : 4.0
ISO                             : 800
Black Level                     : 1792
White Level                     : 16200
Default Crop Origin             : 0 0
Default Crop Size               : 1856 704
Calibration Illuminant 1        : D65
Active Area                     : 0 0 704 1856
Frame Rate                      : 99.781

Showing the right frame rate & black level (1792)
Ok , back to finding stable 60fps .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on October 08, 2018, 12:30:28 PM
Incredible progress!

Also what are the next steps to getting 3k centered?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on October 08, 2018, 07:59:19 PM
If you guys need help with one more 5d mark II for your tests, let me know
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 09, 2018, 07:41:24 AM
Quote from: Ilia3101 on October 08, 2018, 12:30:28 PM
Incredible progress!
Thanks , a1ex figured it out originally now I'm hooked  on the hi-frame rate  :)
Quote from: Ilia3101 on October 08, 2018, 12:30:28 PM
Also what are the next steps to getting 3k centered?

I think the c0f0713c & c0f07150 is not set right , from my understanding (which is very limited)
I believe this controls the window size in Liveview (I maybe wrong) but from a1ex experiment with full width
and height liveview those reg's where needed .
This should be addressed in future updates to the crop_rec.module
I what to add same different presets anyways .
Also we need to implement a1ex's new "CONFIG_EDMAC_RAW_SLURP/CONFIG_EDMAC_RAW_PATCH" into the 5d2 4k branch
that how we have 10-12bit full frame liveview builds on the experiments page.

I'm starting to come to the end of my experiments with Hi-Frames rate , I'm starting to see really clean images now (black level not as bad now)
I increased the vertical to 720p from 704 this give me after unsqueezed 1856x1200 , but it has better black levels -- there still are not right
10bit yields the best image so far, black level are 98 (should be around 112) image cleans up at black level 75
and 14bit is 1559 and cleans up at 1200 black level , before at 704 black level where all over the place and couldn't clean up .
changed
c0f06088 0x4f40432 ->0x2e40432 =>720p
Before it was 0x2d40432 => 704
FYI -- the default is 0x4f40432 =>1250x1856 , so take the first set of numbers "4f4" that's the vertical resolution
"432" the horizontal
hence 0x2e40432 =>720x1856 ,  we read this as 1856x720
I encourage other with d4 cam not just 5d2 to try out a1ex experiments , there are very simple & will help you understand
how the camera works or at least with different resolutions & frame rates
Edit: this was in 50fps , didn't try 48 yet
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 11, 2018, 05:21:34 AM
Found a module on the crop_rec 4k 5d2 branch that "cf_acc.mo , added it to the module list to complied
Did that ok now what is it ? is it for over clocking the CF card interface ?
I haven't loaded it yet just trying to fine some back ground info , looking at the code
seem to be able to bypass canon code and access the cylinder/blocks directly or something like that .
Can anybody enlighten me please  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on October 11, 2018, 07:43:58 AM
https://bitbucket.org/hudson/magic-lantern/branch/hdparm

Might be useful for overclocking, too.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 11, 2018, 07:50:01 AM
Ok , loaded it up and it reads the CF Card info
Printed this on the screen
(https://image.ibb.co/bxObPU/TEST.png) (https://imgbb.com/)

Some useful info at least , this is a Lexar 1066x 32GB
I see the card runs as a PCMCIA & the UDMA in Advance i/o --0x0003  , UDMA6 ? More then likely from the dm-spy logs
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 11, 2018, 07:51:24 AM
Quote from: a1ex on October 11, 2018, 07:43:58 AM
https://bitbucket.org/hudson/magic-lantern/branch/hdparm

Might be useful for overclocking, too.

Thanks a1ex , I'll have a look at this . :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 13, 2018, 07:08:48 AM
Quote from: a1ex on October 11, 2018, 07:43:58 AM
Might be useful for overclocking, too.
It going to take little time to get my head around "hdparm" branch for overclocking , a lot of stuff there .

Also got my new replacement CF card today (Lexar 1066x 64GB) the old one couldn't format just came up as "RAW"
So I did a fresh benchmark speed test in a few different modes to see where it at before I start to
adjust reg's to force it to UMDA 7 (I'm doing this on a new card because it I smoke it , it has warranty  :D )
1 min bench test , Movie Liveview , first run
(https://image.ibb.co/gbMn3p/bench0-small.png) (https://imgbb.com/)

Photo Liveview , better speed
(https://image.ibb.co/cRgRw9/bench2-small.png) (https://imgbb.com/)

Great Speed !! Play mode GD on
(https://image.ibb.co/mdJvpU/bench3-small.png) (https://imgbb.com/)

There is quite a bit of difference between those 3
So why is there so much overhead in Movie Liveview ?
Or is it just Liveview taken up that extra resources
Normally I record raw video at about 72-77MB/s on my other Lexar 1066x CF cards .
Even if we could get the write speed in play mode (87.4MB/s) for movie liveview , That's enough to record 2880x1080 continuously .

Need to do more investigation .

Edit: this was in 1080 30p fps , just checking raw video write speed --
1:1 averages 74MB/s & 3x cropmode 2144x1074 12bit  averages 75MB/s and pecks at 79MB/s
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 14, 2018, 06:35:22 AM
Working on increasing height in 3x Crop_mode  , I'm up to 2152x1331
Default Crop Size               : 2152 1331
Active Area                     : 52 160 1383 2312
Image Size                      : 2312x1383

(https://image.ibb.co/fbEGg9/RAW-002-2152x1330-small.png) (https://imgbb.com/)
RAW-002_2152x1331.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-002_2152x1331.DNG)
The last 300 line at the bottom is because I haven't implemented a1ex's redirect buffer for larger image in the edmac capture
(that's coming very soon , along with a updated crop_rec with added presets up to full width )

Also got 3200x1200 working , did have good success  with 3168x1152

(https://image.ibb.co/dB3j19/M13-2030-000000-3168x1152-small.png) (https://imgbb.com/)
M13-2030_000000_3168x1152.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M13-2030_000000_3168x1152.dng)

Did get 3168x1330 , still needs more work to get 24fps (at 21 fps right now)
(https://image.ibb.co/cnmq8p/M13-2141-00001-3168x1330-small.png) (https://imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 14, 2018, 10:02:47 AM
Try to force CF card to UDMA7 on Lexar 1066x from this post here (https://www.magiclantern.fm/forum/index.php?topic=12862.msg206010#msg206010) and here (https://www.magiclantern.fm/forum/index.php?topic=12862.msg206061#msg206061)
The Reg is only accessible with "DIGIC POKE" , there no 0xC062850C in the ADTG_GUI
0xC062850C --->1000102
As waza57 said -- corrupt the file system , the file are hidden .
I'm in the process of recovering the files ( it's not a fast process that's for sure) didn't hurt the card so far
the card is still in FAT32 , and can read -- file explorer says the card is 64GB with 7 GB in space used up with data(files)
So before I format again and install magic lantern I what to recover the setting files .
before it lost the file system I took a photo with my iPhone , and there's a increase for sure with some interesting info

(https://image.ibb.co/ihaLr9/photo-udma7-small.png) (https://imgbb.com/)
70.7MB/s write in the other test a few post up it was 57-63MB/s
but the read is crazy 2260.4MB/s  :o
So maybe that reg affects the "read only" ?
more investigation
Edit: Strange thing when it lost the file system it changed the name of the card to
"EOS_S_GIGIL" not sure why something with canon I guess
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on October 14, 2018, 01:47:23 PM
@ reddeercity

Great job with fps !

Quote"EOS_S_GIGIL"

Same thing for me...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 20, 2018, 07:57:46 AM
Quote from: waza57 on October 14, 2018, 01:47:23 PM
@ reddeercity
Great job with fps !
Thanks , a1ex did all the heavy  lifting I'm trying get to fine tune it.
There still a black level problem (green cast) , I can't fix yet -- it happens when ADTG_12[100c] 0x2 -> 0x4  is applied
(It's the one that centers the image in liveview) . If you change to 0x3 or 0x5 the image is has blue cast
So on my quest to find reg's that would help I did find the reg for black level in liveview  , but sadly it only for liveview & not for raw image in the buffer
C0F08034 0x1c00 (default)
C0F08034 0x1c00 ->1b00 (green cast)
C0F08034 0x1c00 ->1d00 (red cast)


C0F08034 0x1c00
(https://image.ibb.co/g0Uk30/VRAM7-small-0x1c00.png) (https://imgbb.com/)
C0F08034 0x1b00
(https://image.ibb.co/gWaEwL/VRAM5-small-0x1b00.png) (https://imgbb.com/)
C0F08034 0x1d00
(https://image.ibb.co/kAMtqf/VRAM6-small-0x1d00.png) (https://imgbb.com/)

This what happen after ADTG_12[100c] 0x2->0x4 is applied 
but before I refresh liveview which will center the image
(https://image.ibb.co/b6A030/VRAM11-small.png) (https://imgbb.com/)


I think I may have kind of figured out the "EOS_S_GIGIL"
after recovering some of the data with diskinternals.com/partition-recovery  (https://www.diskinternals.com/partition-recovery/)
the CF card came up as "EOS_DEVELOPFAT32"  , I made a disk image of the card and loaded it up in "HxD" hex editor and the beginning
of the image I have this
ëX.CanonEOS..@ ......ø..?...?...±9t.Ÿ;..........................€.)
....EOS_DEVELOPFAT32   ..BOOTDISK.FA.........................................
ëX.CanonEOS..@ ......ø..?...?...±9t.Ÿ;..........................€.)....EOS_DIGITALFAT32   ......

I'm thinking by setting
0xC062850C --->1000102
We may have accessed a canon service mode or canon development mode for maybe internal testing .
Al this only happen when I tried to record raw video , I first ran the bench.mo and it seem ok at the point.



@a1ex I'm trying to compile "hdparm" branch , getting a fatal error .
I first  tried to complied the 5d2 platform on my new VM " Up to date Ubuntu"
didn't work  , it was looking for a older tool chain "gcc-arm-nne-eabi-4-8-2013q4"
Tried to install but I don't know how to set it in the path for compiling , lucky I still have the Old Clone of
Ubuntu with all the old  tool chain pre-installed . So I complied it and all worked except  the cf_acc module fail to build
So I tried  to compile the module separately and got the following 


ml@ml-pc:~/CF_Card-Over-Clocking_magic-lantern/modules/cf_acc$ make
Updated HGVERSION
[ README   ]   module_strings.h
[ CC       ]   cf_acc.o
[ CC       ]   hdparm/identify.o
hdparm//identify.c:7:25: fatal error: linux/types.h: No such file or directory
#include <linux/types.h>
                         ^
compilation terminated.
make: *** [hdparm/identify.o] Error 1
ml@ml-pc:~/CF_Card-Over-Clocking_magic-lantern/modules/cf_acc$


Seems to look for this "linux/types.h (https://bitbucket.org/hudson/magic-lantern/src/1af20c52fbe4cc48312af61d596967a018283eeb/modules/cf_acc/hdparm/identify.c?at=hdparm&fileviewer=file-view-default#identify.c-7)" and from what I can tell there is nothing related to "Linux" anywhere
Is there something missing ? do I need to add something or should I just comment it out ?
As a test , I tried to compile on the 5D3 113 platform -- still the same error .

I notice this is on the old core , still has raw_rec.mo .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 20, 2018, 09:08:28 AM
I knew it !
There a Service menu I found in the rom
I don't know how to access it but maybe that "EOS_DEVELOPFAT32" did something to the card .
Something along the lines like magic lantern does when it boots

"********** FACTORY ADJUSTMENT MENU VER 0.01 **********":
"0. Exit from Factory Adjustment":
"1. Leak Check":
"2. SDRAM Check":
"3. ROM Check":
"4. SDIO Implementation Check":
"5. Video Adjustment":
"6. A Adjustment":
"7. USB Check":
"8. Adjustment Data Display and Change":
"9. Check Flag Display and Initialization":
"A. ALL Check":
"D. Debug Command":
"R. Data Receive":
"Y. Input Unique":
"Z. Input_Device_Unique":
"invalid input":
"CheckPro> ":
"--- LeakCheck ---":
"--- Finish ---":
"K218 System & Display Check & Adjustment program has started.":

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on October 20, 2018, 04:50:26 PM
I shot my cat with your build. She liked 2880:1080 so much. Everything was ok but froze grayscale preview.
https://drive.google.com/file/d/1YiBTSvS1JYzhThcNcmEZbBaBjnLibsjX/view?usp=sharing
Reddeercity, you told that you finally objective here is 4096:1550.
Look, 2.64:1 - it's a very special cinematic aspect ratio.
Do you plan to bring some options to choose between 2.64:1, 2.35:1, 2:1 or maybe 16:9?
I think 2.35:1 is the most optimal option here. It could be 3642:1550 (2.35:1) for example.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 21, 2018, 05:30:51 AM
Quote from: Igor_Braun on October 20, 2018, 04:50:26 PM
............. Everything was ok but froze grayscale preview.
right now that one of the limitation , press half shutter before recording will give a full color preview in liveview


Quote from: Igor_Braun on October 20, 2018, 04:50:26 PM
Do you plan to bring some options to choose between 2.64:1, 2.35:1, 2:1 or maybe 16:9?
Yes , but the choices will be limited to CF card write speed . 3k to maybe 3.5k (currently at 3168x1330 @ 21 fps)
no sense trying to record 4k @ 24p for 2 seconds .
I can implement full width @ 12.5 fps (5632x1074) and full height @ 4 fps (5632x3752)
Which I plan on doing later on (in a few week or so)
I'm really focused  right now on CF card overclocking (increasing write speed beyond 80MB's)

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: IDA_ML on October 21, 2018, 12:32:48 PM
Quote from: reddeercity on October 21, 2018, 05:30:51 AM
I can implement full width @ 12.5 fps (5632x1074) and full height @ 4 fps (5632x3752)
I'm really focused  right now on CF card overclocking (increasing write speed beyond 80MB's)

Reddeercity,

If you could implement 5632x3752@4 fps that would be fantastic!  This means very smooth timelapse videos at 1 to 4 fps which really look very beautiful, especially in combination with Dual ISO. 

What about the 7D?  It writes at 90 MB/s as it is.  No overclocking would be necessary to get reasonably long recording times at the high croped resolutions.  2560x1440 at 10bits lossless and 24 fps would be a dream come through for this fantastic old camera.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 23, 2018, 09:01:32 AM
Calling all 5D2 users --  8)
magiclantern-crop_rec-4k-5D2-eXperimental.2018Oct22.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/)

Updated File
magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212.zip)

I implemented  a1ex's redirect buffer (https://bitbucket.org/hudson/magic-lantern/commits/3e3fac1e9c8e3b2ae956a0613c1e49c705831257?at=raw_video_10bit_12bit_LVState)  that's used in the official 10-12bit "raw_slurp" in the  https://builds.magiclantern.fm/experiments.html for 5D2.
Work very well with 10bit 1:1 FHD 1856x1044 @ 30fps & 24fps with out issue with mlv_rec  (did a short 30 second test at both frame rates).

More important , it works great with the "Crop_Rec" module , no problem recording 2880x1080@24fps
This was the first step to bring 5D2 in to the main "crop_rec 4k branch"

There's a little interesting side benefit from using the new redirect buffer from a1ex ,
I can get 3008x1138 with the 2880x1080 crop_rec preset , just need to set aspect ratio to 1:1
go to 10x zoom then back to 5x zoom (3x crop_mode) basically refresh liveview
that resolution is a little too unstable , but it proves that the buffer is working correctly
Now I can push forward with some different preset .

So now you have stable 10-12bit in 1:1 Full HD with 3k crop_rec , more usable now .
To come full hd 48fps or maybe 50 & Full resolution 5.6k @4 fps  to start with.
Enjoy

@IDA_ML a developer with 7d really needs to implement the code ,
I don't have one so it would be nearly  impossible for me .
Sorry .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on October 23, 2018, 09:06:39 AM
Do you have your code uploaded? Would like to check it out.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 23, 2018, 09:13:02 AM
Quote from: Danne on October 23, 2018, 09:06:39 AM
Do you have your code uploaded? Would like to check it out.
Here's the source code Crop_Rec-4k-5d2-magic-lantern_redercit_buffer_a1ex-code-10-22-2018.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Crop_Rec-4k-5d2-magic-lantern_redercit_buffer_a1ex-code-10-22-2018.zip)
The changes are in
platform/5D2.212/internals.h
platform/5D2.212/stubs.S
src/raw.c


Edit: I edited the first post in thread with the latest build ,
I'll update it with information & new build when necessary
This will help make it easy to find the new builds for 3k,4k crop_rec
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on October 23, 2018, 09:29:02 AM
Thanks. By the way. Could you tell me the ISO ADTG registers for the 5D mark 2?
ADTG2[8882]
ADTG2[8884]
ADTG2[8886]
ADTG2[8888]

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on October 23, 2018, 09:24:27 PM
@reddeercity: thanks for your build! I played a bit around with it. 10 and 12 bit seems to work very stable for me. I got no issues so far, @25fps (PAL) and 30fps (NTSC), FHD and 2144x1074 working nicely. The only thing I did not get to work is 3k. What I tried:
- activate mlv_rec, mlv_snd, crop_rec
- restart camera
- start liveview
- goto 5x zoom
- select resolution 2880
- select aspect ratio 1:1
- leave ml menu
- goto 10x zoom
- goto no zoom
- goto 5x zoom
If I now record, I get 2144x1074. Or if I go into ml menu I see the same resolution.
What am I doing wrong? I tried with Canon menu PAL (25fps) and NTSC (30fps).


Edit: forget what I wrote... so stupid... never worked with crop_rec... I found it now! ;)
Edit2: oh yeah... after getting the right settings to work, the quality of the resulting clips are brilliant at 2880x1080!!! Great work!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 24, 2018, 03:49:30 AM
Thanks, as always I'm only a part or a bigger picture  ;)
Most of the credit for the crop_rec goes to waza57 for porting it to 5d2 .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 24, 2018, 06:22:12 AM
Now that I have the redirect buffer working I can test some full width in 3x crop_mode  from a1ex's experiments .
I modify the resolutions in mlv_rec , added 3008, 3840, 4096, 5632 (H)
I successfully reproduced the full width 5632x1074 @ 12.5fps , can record continuously in 10bit
For full width LiveView:
CMOS[2] = 0x00E
C0F06008: 0x27705DB (only the lowest half appears to matter on this camera)
C0F06084: 0x30037 (copied lowest half from photo mode)
C0F06088: 0x46A0B87 (copied lowest half from photo mode)
=> 5632x1074 in mlv_rec, 12.5 FPS.


(https://image.ibb.co/c1pNrV/5632x1074-12-5fps-M23-2045-small.png) (https://imgbb.com/)

5632x1074-12.5fps-M23-2045.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5632x1074-12.5fps-M23-2045.mov) 40Mb , Short 12 second H264 test

selected 4096 from menu to record 4k
4096x1074-12.5fps-M23-2047.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4096x1074-12.5fps-M23-2047.mov) 30Mb , 8 second H264 test

@Danne there's no
ADTG2[8882]
ADTG2[8884]
ADTG2[8886]
ADTG2[8888]

on the 5D2 , those reg's are for Digic 5 cams only
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 24, 2018, 07:26:21 AM
Next to get full height  as per a1ex experiment , in 3x crop_mode -- 2144x1330 @ 9.98fps (hopefully I can increase frame rate)
C0F06014: 0xEDC
C0F06088: 0x0x56A04BA
CMOS[1] => F6A
C0F0713C from 0x476 -> 0x576
C0F07150 from 0x49C -> 0x59C
9.98fps


(https://image.ibb.co/cYVCrV/M23-2259-00001-small.png) (https://imgbb.com/)
2144x1330-9.9fps-M23-2259.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2144x1330-9.9fps-M23-2259.mov) 40Mb H264 clip
2144x1330-M23-2259_00001.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2144x1330-M23-2259_00001.png) PNG frame export from mlvproducer
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 25, 2018, 07:14:14 AM
Continuing from yesterday with full height exercise ,

2144x1586
(https://image.ibb.co/dDv7RV/2144x1586-M24-2225-00001-small.png) (https://imgbb.com/)
2144x1586-9.9fps-M24-2225.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2144x1586-9.9fps-M24-2225.mov) 30Mb h264 clip
2144x1586-M24-2225_00001.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2144x1586-M24-2225_00001.png) Frame exported from mlvproducer

C0F06014: 0xEDC
C0F06088: 0x0x66A04BA
C0F0713C from 0x476 -> 0x676
C0F07150 from 0x49C -> 0x69C


2144x1842
(https://image.ibb.co/g4ZFfq/2144x1842-M24-2227-00001-small.png) (https://imgbb.com/)
2144x1842-9.9fps-M24-2227.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2144x1842-9.9fps-M24-2227.mov) 30Mb h264 clip
2144x1842-M24-2227_00001.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2144x1842-M24-2227_00001.png) Frame exported from mlvproducer

C0F06014: 0xEDC
C0F06088: 0x0x76A04BA
C0F0713C from 0x476 -> 0x776
C0F07150 from 0x49C -> 0x79C
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on October 25, 2018, 03:15:17 PM
@reddeercity ->  Calling all 5D2 users --  8)
Congratulations, the first step in crop_rec is here !
I tested a 10bits 2752x1080 file and it is ok but liveview is frozen during recording.
I tested a dual_iso_crop_rec mixed video but i got an "oldapi" error message on loading dualiso module. I know I'm asking a little too much for a first step ...
But it's was so existing that I tested !
Thanks
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on October 25, 2018, 03:50:10 PM
Quote from: benoit on October 25, 2018, 03:15:17 PM
I tested a 10bits 2752x1080 file and it is ok but liveview is frozen during recording.
Japp same here. But somewhere I read "liveview is for wimps", so I thought this is state of the art... 8)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on October 25, 2018, 04:48:38 PM
Perhaps I'm a wimps ...   ??? I want to see what i'm filming  ;) ! I'm not only use my cam with a tripod  !
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on October 26, 2018, 05:24:19 PM
I must be doing something very wrong, so I'll take the hit. Please breakdown what our setting should be, I only see static lines when I try to use 3.5K 1.1 Crop setting.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 28, 2018, 07:23:05 AM
Updated the First Post with very basic screen shot tutorial
Short Tutrial for Setting up Crop_Rec Module (3.5k 1:1 Preset) on the 5D Mark ii (https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476)
Hopefully this should clear up any confusion
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on October 28, 2018, 03:53:10 PM
I don't know what I'm doing wong but I can't wait to find out, I've follow all tutorial steps. I attached photos to represent setting, also showing that in 1856x1044 1:1 and 10x zoom everything looks OK, but x5 3.5K 2880x1080 is all static lines. PLEASE HELP ME SOLVE THIS!
(https://thumb.ibb.co/mgbJrV/IMG-1348.jpg) (https://ibb.co/mgbJrV) (https://thumb.ibb.co/eNkpyA/IMG-1349.jpg) (https://ibb.co/eNkpyA) (https://thumb.ibb.co/hcK4WV/IMG-1350.jpg) (https://ibb.co/hcK4WV) (https://thumb.ibb.co/feoRdA/IMG-1351.jpg) (https://ibb.co/feoRdA) (https://thumb.ibb.co/dyphJA/IMG-1352.jpg) (https://ibb.co/dyphJA) (https://thumb.ibb.co/bFLYQq/IMG-1353.jpg) (https://ibb.co/bFLYQq) (https://thumb.ibb.co/dvBJrV/IMG-1354.jpg) (https://ibb.co/dvBJrV) (https://thumb.ibb.co/nd6JrV/IMG-1347.jpg) (https://ibb.co/nd6JrV)
 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 29, 2018, 04:51:22 AM
You don't follow the tutorial , I don't know how to make it any simpler .
This is bleeding edge stuff , Go back to the tutorial and match your screen shots to the
ones in the first post , Do there match exactly in the Movie Tab  ?
Hint: something to do with fps

1:For best results start with 30 fps ntsc in the canon Liveview menu
2:Load crop_rec  & mlv_rec , mlv_snd can be load but reduces the amount of frames recorded .
3:Next setup Liveview preview , This is the most important part without this set correctly it will not work !
-Liveveiw needs to be set to ML Gray Scale-
4:Next go in to the crop_rec menu select 3.5k preset .
-Note ! this only work when you enter 5x zoom/3x crop_mode as per the hi-lighted texts says-

Did you do this exactly ?
The frame shots tell a different story .
Only load & enable the bear minimum to run crop_rec
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on October 29, 2018, 06:23:20 AM
Wow, @Reddeercity you're affirmative I use both FPS setting, Thanks for the smack on the back of my head. So is it necessary to use Exact FPS on 10bit 12bit? When using 5x is it recommended to use 24mm to get a 50mm framing? Thanks Again to Reddeercity, Alex, Waza57 for the phenomenal job of making my trusty 5D Mark II at least a FHD RAW recorder, with full frame sensor RAW our images will meets up to wide range of end Presentations Format.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 30, 2018, 06:52:42 AM
Working on presets , Got a nice clean 3168x1330 but sadly only 12fps at the moment (looking to get 23.976 fps)
I need to understand the A & B Timers better , this was just luck -- started at 7fps and worked my way up to 12 fps

3168x1330 @ 12fps
(https://image.ibb.co/hOYek0/3168x1330-12fps-frame-M29-2249-00001-small.png) (https://imgbb.com/)

3168x1330_12fps-M29-2249.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3168x1330_12fps-M29-2249.mov) h264 10 second 30Mb clip
3168x1330-12fps-frame-M29-2249_00001.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3168x1330-12fps-frame-M29-2249_00001.png)
3168x1330-RAW-017.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3168x1330-RAW-017.DNG) from image dump

Default Crop Size               : 3176 1331
Active Area                     : 52 160 1383 3336

These are the overridden reg's ** Note** the times (6008 & 6014) are for 7fps I didn't save the timers for 12 fps

(https://image.ibb.co/e3apJL/VRAM5-small.png) (https://imgbb.com/)

I did find that "CMOS[1] need to set early -- if not , thing don't work right

and you get this 2.38 A.R. ( that what I'm looking for)

(https://image.ibb.co/goATCf/VRAM6-small.png) (https://imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 31, 2018, 06:51:57 AM
working on a 4k preset (4096)
c0f06088 - 46a088a =>4096x1074
c0f06008 - 2770477 => 16.389 fps

Clean dng from image_dump plus clean dng's in .mlv's

4104x1075-RAW-019.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4104x1075-RAW-019.DNG) from image_dump
(https://image.ibb.co/hLOYf0/4104x1075-RAW-019-small.png) (https://imgbb.com/)

After cropping off the  black space it's only 3718x1074 , needs more reg's added
(https://image.ibb.co/hsLof0/3718x1075-RAW-019-small.png) (https://imgbb.com/)
3718x1075-RAW-019.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3718x1075-RAW-019.png)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 03, 2018, 07:09:49 AM
Ok got 4096x1074 @ 18fps working -- It's almost useable as is  :D

Needed to add CMOS[2] 0x10E ->0xE (same as full width 5632)
(https://image.ibb.co/kCkHyL/4104x1074-18fps-VRAM9-small.png) (https://imgbb.com/)

A & B Timers setting for 18 fps
(https://image.ibb.co/hJT6Q0/4104x1074-18fps-setting-VRAM10-small.png) (https://imgbb.com/)

For same reasons mlv_rec in crop_mode cut off the 48 pixels from 4096 to 4048 , not sure why .
(https://image.ibb.co/m6xi50/4104x1074-18fps-M02-2240-00001-small.png) (https://imgbb.com/)

4048x1074-18fps-M02-2240_00001.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4104x1074-18fps-M02-2240_00001.png)

If you look at the dng from the image _dump it's 4104x1074 and there no black space on the right side like the PNG .
4104x1075-18fps-RAW-020.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4104x1075-18fps-RAW-020.DNG)
4104x1074_image_dump-RAW-020.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4104x1074_image_dump-RAW-020.png) used dcraw to export a ppm them saved it to png with Irfan View

4096x1074-M02-2240_000000.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4096x1074-M02-2240_000000.dng) Cdng exported from MLVFS (quick mount)
there a 48pixel black area on the right , thinking the problem is mlv_rec  , I see in the code there's a 160 pixel crop , so may be change that to 100 maybe

Subfile Type                    : Full-resolution Image
Image Width                     : 4264
Image Height                    : 1127
Bits Per Sample                 : 14
       ..........
Default Crop Size               : 4104 1075
Active Area                     : 52 160 1127 4264


Black Level                     : 1001
White Level                     : 16200

This is also interesting , can the black level really be 1001 ?
Image looks great , so I guess this is ok , but strange -- I recorded  at 10bit .

Recorded a short .mlv about 10 seconds exported it to h264 from mlvproduder 4096x1074-18fps-M02-2240.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4096x1074-18fps-M02-2240.mov)

So close now to 24 just need 6 more frames per second .

Now it time to work on Lossless compression , since I'm close to having the presets done.
So far I have ,  full liveview , 2880x1080, 3168x1330, 5632x1074 ,4096x1074
I'm thinking of added some of the near full height in 2k mode e.g. 2144x1842 or 2144x1586.
I would like to add 48 FPS in FHD (1856x720) but it's too unstable right now , need a lot of work yet.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on November 03, 2018, 11:08:45 AM
I'm waiting for yours presets with hope.
I don't know the right number you can reach with yours registers but do you think you can include standard preset 24p with 16/9 ratio and vertical resolution like 1080, 1160, 1320, 1450, 1586 ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 04, 2018, 04:40:03 AM
Quote from: benoit on November 03, 2018, 11:08:45 AM
......  do you think you can include standard preset 24p with 16/9 ratio and vertical resolution like 1080, 1160, 1320, 1450, 1586 ?
All presets will be dependent on Card write speed and whether or not lossless compression can be implemented .
So currently write is around 77-79MB's with the odd burst at 85MB/s , the preset will reflect that .

I guess the answer is no , I will not have standard presets . But I will have one's that are close .
The main reason is that the sensor is in a non-conformed size to Video ( that why FHD is only 1880x1054 , unlike 5d3 1920x1080)
and being raw , it can be resized as much as 150% before any image break down .
There will be some at 24p , like 1080 & 1330 -- I plan on 2:1 but not sure if I will go 16/9
It takes a long time to get this sizes to work @ 24p and the more the height is increased the more difficult it becomes
But nothing is written in stone ! things can change at any time as new discoveries can happen .

After we have this 100% working then I plan on moving to Mjpeg Compression , which I hope to be able
to have Standardize Frame resolutions up to 5.6k in a .AVI format.
But don't hold your breath  :D that could take a year or more , or maybe less who knows for sure .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on November 04, 2018, 10:27:31 AM
ok, I didn't know it takes a long time to get this sizes to works.
Could you write a brief summary (a few lines) on how to find these values for 5D2 ?
I read yours threads with a1ex about the registers, but i need a summary.
Perhaps I could understand and make my own values to share with others users and increase the preset database.

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 05, 2018, 05:48:44 AM
Everything is all there in the posts , I just reproduced a1ex experiments
read this first (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205556#msg205556) then this next (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205559#msg205559) , finish off with reproducing a1ex experiments  (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205614#msg205614) and here too (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205622#msg205622) . 
Now you know what I know , Next you need to setup a environment to compile magic lantern . Depending on your OS there different ways to setup a environment
Compiling Magic Lantern on Windows 10 (using its Linux subsystem) (https://www.magiclantern.fm/forum/index.php?topic=20214.msg188142#msg188142) , Compiling Magic Lantern with Cygwin/MinGW-64  (https://www.magiclantern.fm/forum/index.php?topic=15894.msg154435#msg154435) , Compiling Magic Lantern on a Macintosh (https://www.magiclantern.fm/forum/index.php?topic=16012.0) etc. ...
I use Ubuntu in a VM (Virtual Machine) on windows 7Pro . Next Clone waza57 source code (crop_rec_4k_5D2) link is here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205624#msg205624) ,
now you need to add "raw_slup" redirect buffer to the 5d2 4k source from waza57 to get the extended image .
This committed 3e3fac1 (https://bitbucket.org/hudson/magic-lantern/commits/3e3fac1e9c8e3b2ae956a0613c1e49c705831257?at=raw_video_10bit_12bit_LVState) is the one I used , now compile . That it .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on November 05, 2018, 10:24:37 AM
Thanks for the links.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 11, 2018, 08:10:58 AM
Looking in to the HDMI resolution limitations , if everyone doesn't know on the 5D2 the HDMI output @ 1650x1080 60i 4.2.2 uncompressed (1500MB/s)
It anyone wondering why the 5d2 has this res. when standard HDMI is normally 1920x1080 or 1280x720
From what I can figure out there was no need for 16x9 format (After all 5D2 was the first to introduce Video to DSLR's) as a second thought .
So the main object was Photo's and a 3x2 format was needed for photos in  Liveview  so hence 1650x1080 .

So I connected my HDMI Evf & loaded ADTG_GUI.mo  and saved a log file
So I may have found some reg's that control this .
c0f11314: 4d70653 .... LiveViewMgr pc=ffa07c28 addr=ffca88e0 LV resolution (raw.j.height | lv.width) before upsampling?
If you convert this to decimal you get 4d7=1239 & 653=1619  so 1619x1267 , look very close
c0f0713c:     4f5 ......  LiveViewMgr pc=ff8e1830 addr=8328 HEAD3 timer (ticks?)
c0f07150:     681 ..... LiveViewMgr pc=ff8e1844 addr=8370 HEAD4 timer (ticks?)

convert this to decimal  4f5=1269 & 681=1665 so 1665x1269
I could be totally wrong but that so close , now there's more then likely more reg's evolved here , it could be the hd buffer is the limitation.

There a few reason why I what to hopfully fix this issue ,
First: to get real full 1920x1080 so it would be usable without resizing .
Second: the HDMI spec 1.3a (https://en.wikipedia.org/wiki/HDMI) allowed up to 2560×1440 at 60 Hz & 1920 × 1080 at 120 Hz
Since there's 4k monitor's & recorder out there and getting cheaper everyday, well I thinks it time to look in to this to see it it's possible .

Edit: Ok I forgot to mention the hdmi 1.3a supports 8bit(24 bit/px) , 10bit(30 bit/px), 12bit(36 bit/px), 16bit(48 bit/px) up to 2560x1440 60i
Color Space: RGB, Y′CBCR 4:4:4 & Y′CBCR 4:2:2
and apparently hdmi 1.3a supports UHD 8bit(24 bit/px) 3840x2160 30p 4.2.2 & 60p @4.2.0 and also 5k 8bit(24 bit/px) 5120 × 2880 30p 4.2.2
Maybe there's hope yet for real-time hdmi UHD up to 5k preview   :))
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on November 11, 2018, 07:47:30 PM
Wow... it looks so interesting
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 12, 2018, 07:22:00 AM
I should mention that the 5D2 HDMI signal is 8bit 4.2.2 Full range (0-255) , Check it on my AJA Kona LHi video capture card .
So by this theoretically  "UHD 8bit(24 bit/px) 3840x2160 30p 4.2.2" would be in the realm of possibility .
Here's hoping   :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 15, 2018, 06:00:44 AM
Having some fun we presets ,
5632x1844 @ 4fps , got up 6.5fps then froze liveview , unresponsive so I just rebooted .
Sorry , overexposed it too much @ 800 ISO  ::)  as frame rate came down , exposure became normal

(https://image.ibb.co/fRjmkL/M14-2138-00050-small.png) (https://imgbb.com/)
5632x1844_M14-2138_00050.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5632x1844_M14-2138_00050.png)
Default Crop Size               : 5632 1845
Active Area                     : 52 160 1897 5792

5632x1845_dump_RAW-026.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5632x1845_dump_RAW-026.DNG)

(https://image.ibb.co/g6Xvef/VRAM3-small.png) (https://imgbb.com/)

Modified Reg's
(https://image.ibb.co/jU7aC0/VRAM2-small.png) (https://imgbb.com/)

5632x1844_4fps_M14-2138_h264.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5632x1844_4fps_M14-2138_h264.mov)
Had problem playing it in QT player , work fine in  MPV player (https://mpv.io/) -- cross platform  ;)

I sure wish I could get 24fps at this resolution , it would be prefect  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on November 15, 2018, 05:25:08 PM
Cool -) Thanks for keeping going
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 17, 2018, 07:50:18 AM
Keeping with the same height (1842) as above , but @ 4096 and see how far I can push it
So 4096x1842 I could reach 10.5fps with a little image corruption (needed to adjust 5x frame blanking to clean up)

Shown below is  4096x1842@4fps
(https://image.ibb.co/jJYJLL/VRAM4-small.png) (https://imgbb.com/)

notice for 4096 c0f06084->30036 , where 5632 @ c0f06084->10037 -- (for anyone that's following this and what's to reproduce)
Raw resolution c0f06088-> 0x76ao88a

Changed A & B timers from image posted

10.501 fps
C0f06008 ->0x2770462
c0f06014 ->0x7f2


4104x1843_dump_RAW-028.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4104x1843_dump_RAW-028.DNG)
Default Crop Size               : 4104 1843
Active Area                     : 52 160 1895 4264


(https://image.ibb.co/krj00L/4096x1842-M16-2255-000076-small.png) (https://imgbb.com/)

4096x1842_ppm_dcraw-M16-2255_000076.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4096x1842_ppm_dcraw-M16-2255_000076.png)
exported ppm file from Cdng with dcraw -- with a quick color correction in irfanview and saved as a PNG
4096x1842_mlvfs_M16-2255_000076.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4096x1842_mlvfs_M16-2255_000076.dng)

A short 36Mb h264 clip @ 7 FPS 4096x1842_7fps-M16-2255.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4096x1842_7fps-M16-2255.mov)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on November 17, 2018, 10:29:33 AM
Wow, i saw the h264 video.
Maybe reducing vertical resolution till 1742 (4096x1742 (2.35:1 aspect ratio) will help to increase a couple of frames per second?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 18, 2018, 08:20:37 AM
I don't thing it will help , I thinking the sensor readout may be too slow for that resolution .

Trying 3072x1200 see if I can get 24fps
CMOS[1] 0x6ea->0xc0a
c0f0713c 0x476->0x576
c0f07150 0x49c->0x59c
c0f06008 0x2770277->0x2770346  ** A-Timer **
c0f06014 0x4ff->5cd  ** B-Timer **
c0f06088 0x46a04ba->0x4e7068a  **raw resolution **


This gives me only 19.419 fps with some frame blanking corruption (not totally clean) , seem that I can't get any more .
5d2 maybe limited to only 1080 vertically , as all my efforts seem to point to this .
I keep dropping the horizontal to get as much height as possible ,
so I guess next back to 2880 and see if I can increase the height to 1200 at 24fps
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on November 18, 2018, 07:25:36 PM
reddeercis,

is it possible, in your build, to shoot 2880x1080, and shoot normal, almost 1080p in full sensor 1:1, too (in the same time, i mean using same build)?

i want to try your build, but find some test on YT, where guy sad that full sensor mode does not work when using experimental build that crops the center of frame....

maybe i did not understand it correctly..

tnx 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on November 18, 2018, 11:04:48 PM
i just try it!

2880p is so great>))

Thank you very much for such a progress!!

I dont have preview while recording. Preview is frozen and black and white, once i press record button.

Is that normal?

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 19, 2018, 04:17:24 AM
Quote from: banertop on November 18, 2018, 07:25:36 PM
reddeercis,

is it possible, in your build, to shoot 2880x1080, and shoot normal, almost 1080p in full sensor 1:1, too
Yes , the crop rec is only active in 5x zoom mode . So you can shoot 10bit raw in 1:1 FHD also .


Quote from: banertop on November 18, 2018, 11:04:48 PM
i just try it! 2880p is so great>))
    ........
I dont have preview while recording. Preview is frozen and black and white, once i press record button.
Is that normal?
Yes , That's one of the limitation at the moment , until liveview preview is more understood more , that's the way it is .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 19, 2018, 08:01:29 AM
Trying a more modest size to see where the limits are .

2144x1200 @ 23.976 fps
(https://image.ibb.co/duoHKf/VRAM11-small.png) (https://imgbb.com/)

(https://image.ibb.co/kEszX0/VRAM10-small.png) (https://imgbb.com/)

(https://image.ibb.co/dS53s0/M18-2327-000000-small.png) (https://imgbb.com/)
2144x1200-M18-2327_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2144x1200-M18-2327_000000.png)

A short h264 2144x1200_24fps_M18-2327.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2144x1200_24fps_M18-2327.mov) 75Mb 25 seconds

It does record continuously @ 73.5MB/s  :D , the preview is Not broken so you can see what's being recorded
It took a while to get it work , the sensor is not centered I had to move the image to the lower half of the sensor
to clean up the image , use CMOS[1] 0xe6a -> 0xc14 , 0xc0a centers the image but has bad corruption (frame blanking issue).

I guess to sum it up ,  1080 maybe the maximum vertical resolution with extended horizontal e.g. "2880 or 3168" that can achieved .
Every effect above 1080 is meet with failure @ 3k and greater .

It Maybe that the Sensor readout is just to slow to get higher resolution @ 23.976 fps anything less is a waste of time ,
ok maybe full res 5632x3750 @4 fps .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on November 19, 2018, 10:24:00 AM
Usable 2880x1080p, with live view, so we can see what we record, is just fine. Maybe we do not need more resolution..... :P
This is still such a big progress for 5d2
2880 horizontal is really a sweet spot!

thx for responding
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on November 19, 2018, 04:12:48 PM
Great reddeercity !
Perhaps with a continuous bit rate, the same vertical 1200 pixels, the same liveview not frozen, we can find horizontal resolution for 1.85, 2 and 2.35 aspect ratio ? like 2220, 2400 and 2820. I'm afraid the last one won't be continuous. Who knows ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: poromaa on November 19, 2018, 09:40:51 PM
Impressive!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on November 20, 2018, 02:38:57 AM
Absolutely agree with Banertop, hehe..
Usable 2880x1080p, with live view, so we can see what we record, is just fine. Maybe we do not need more resolution..... :P
It's a big progress. Crop Factor is like 1.9 but it's ok for this case with 5d mk 2, i think.
Anyway, thanks for keeping going Reddercity! And of couse we are waiting for any news. When you rich 2880x1080 24fps, I try to inspire you with little videos, shot with the new build, hehe
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on November 20, 2018, 02:45:45 AM
Also we can try to fix Metabones SpeedBuster on 5dMK2 to "reduce" the crop factor.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 20, 2018, 05:51:13 AM
I might I have a fix for the frozen Liveview in crop rec 3k preset .
https://www.magiclantern.fm/forum/index.php?topic=19336.msg206539#msg206539
I haven't tried it yet , in theory it should work

Quote from: benoit on November 19, 2018, 04:12:48 PM
Great reddeercity !
Perhaps with a continuous bit rate, the same vertical 1200 pixels, the same liveview not frozen, we can find horizontal resolution for 1.85, 2 and 2.35 aspect ratio ? like 2220, 2400 and 2820. I'm afraid the last one won't be continuous. Who knows ?
Thanks ,
That's my next step to increase horizontal resolution , I might get 2400 or maybe even 2560 with 1200 vertical @ 23.976 fps
Don't count out 2880x1080 @ 24fps to be continuous  , I being working with others on Lossless Compression ( Canon's compression for Raw Images in CR2)
Read this ProcessTwoInTwoOutLosslessPath (https://www.magiclantern.fm/forum/index.php?topic=18443.msg205333#msg205333) and here too (https://www.magiclantern.fm/forum/index.php?topic=18443.msg198487#msg198487) , from other cameras that support this, there about 40-50% data compression
https://rawcalculator.netlify.com/calculator_desktop
2880x1200 @23.976 fps with 14bit compressed raw =>69.1MB/s -- continuous
2880x1200 @23.976 fps with 14bit uncompressed raw =>98.8/s
2880x1080 @23.976 fps with 10bit uncompressed raw =>88.9MB/s

I'm also looking in to CF Card Overclocking , seems that the bus interface can be pushed not sure how much,
hoping to get at least 120 MB/s write speed .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on November 20, 2018, 09:35:43 AM
Here's a simple task for you guys: find out how far you can push FPS timer A.

Steps:
- download raw_diag.mo and adtg_gui.mo (https://builds.magiclantern.fm/modules.html#iso-research) and load them on top of the build from first post
- enable RAW Diagnostics (Debug menu)
- enable OB zones, disable other types of analysis
- in LiveView, press shutter halfway for 1-2 seconds to run the analysis
- enable ADTG Registers, enable Advanced -> ENGIO Registers, override FPS timer A (C0F06008) and decrease its value
- notice only odd raw values (i.e. same parity as with Canon firmware) are going to give clean image on this camera
- find the lowest value of timer A that still gives clean image (pay attention to the right border in the raw_diag screenshot)
- upload the raw_diag screenshot with the optimal value (clean image) and with the first bad value (that is, optimal value minus 2)

Run this in 1080p24/25/30 (during standby) and also in x5 zoom mode.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 20, 2018, 09:45:13 AM
It's was only a matter of time before things start to came together , once I started to understand the imaging pipeline (will at least in my mind I started to understand :P)

Ok now to the detail:
2400x1200 @ 23.976 fps
With a Clean Preview No Frozen Preview !! (will mostly , still have a little frame blanking issue -- I'll it fix later)

(https://image.ibb.co/jTiFPf/VRAM15-small.png) (https://imgbb.com/)

@a1ex it works , no more frozen Liveview when recording raw video  -- Thank you 8)
It took a while to find c0f08518 , it was deep towards the end of all the reg's
Wasn't should if I needed to add c0f08184 , but I did
c0f08518 0x4670907 ---------->0x4e20a07
c0f08188 0x907 -------------->0xa07
c0f08184 0x467 -------------->0x4e2

0xa07=>2567 -- horizontal
0x4e2=>1250 -- vertical





(https://image.ibb.co/f3CCn0/Un-Frozen-preview-IMG-2671-small.png) (https://imgbb.com/)
iPhone video showing unfrozen Liveview when recording raw video -- 2.4kx1.2k-Un-Frozen-preview-IMG_2671.MOV (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2.4kx1.2k-Un-Frozen-preview-IMG_2671.MOV)

Raw Resolution 1253*2568 (2568x1253)
c0f06088 0x46a04ba ---> 0x4e8053a

Default Crop Size               : 2408 1201
Active Area                     : 52 160 1253 2568


Here is a screen shot from the frame over ride menu showing timing info
(https://image.ibb.co/kPBKjf/VRAM14-small.png) (https://imgbb.com/)

Proof , but still need to fix black levels -- h264 5 seconds clip 2400x1200-24fps-bad-black-levels-M20-0131.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2400x1200-24fps-bad-black-levels-M20-0131.mov)

2400x1200-bad black levels M20-0131_00001.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M20-0131_00001.png)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 20, 2018, 09:48:27 AM
@ a1ex I posting when you posted this , I try this tomorrow -- should have time then  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 21, 2018, 06:40:58 AM
Quote from: a1ex on November 20, 2018, 09:35:43 AM
Here's a simple task for you guys: find out how far you can push FPS timer A.

Steps:
- download raw_diag.mo and adtg_gui.mo (https://builds.magiclantern.fm/modules.html#iso-research) and load them on top of the build from first post
- enable RAW Diagnostics (Debug menu)
- enable OB zones, disable other types of analysis
- in LiveView, press shutter halfway for 1-2 seconds to run the analysis
- enable ADTG Registers, enable Advanced -> ENGIO Registers, override FPS timer A (C0F06008) and decrease its value
- notice only odd raw values (i.e. same parity as with Canon firmware) are going to give clean image on this camera
- find the lowest value of timer A that still gives clean image (pay attention to the right border in the raw_diag screenshot)
- upload the raw_diag screenshot with the optimal value (clean image) and with the first bad value (that is, optimal value minus 2)

Run this in 1080p24/25/30 (during standby) and also in x5 zoom mode.

Mission accomplished
ob-zones.rar (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/ob-zones.rar)

default                                                                                                                     
(https://image.ibb.co/c4VJCA/small-ob-zones-30p.png)1080 30p (https://imgbb.com/)  (https://image.ibb.co/jY7Rzq/small-ob-zon-1-24p.png)1080 24p (https://imgbb.com/) (https://image.ibb.co/hJRx5V/small-ob-zon-3-25p.png)1080 25p (https://imgbb.com/) (https://image.ibb.co/ee4Oeq/small-ob-zon-2-640x480-30p.png)480 30p (https://imgbb.com/) (https://image.ibb.co/dCc9Kq/small-ob-zon-4-3xcrop-30p.png)3xcrop-30p (https://imgbb.com/) (https://image.ibb.co/mNqeKq/small-ob-zon-5-3x-Crop-24p-over-ride.png)3xcrop-24p (https://imgbb.com/)
------------------------------------------------------------------------------------------------------------------------------------------------------

Modified Timer A c0f06008
NTSC 30p
1st: 23b0219=>31.864 fps -- good    2nd: 23b0217=>31.982 fps -- bad
(https://image.ibb.co/kQRN5V/small-ob-zon-6-30p-31-864fps-timer-A-23b0219-good.png)1080-30p (https://imgbb.com/) (https://image.ibb.co/iEWpkV/small-ob-zon-6-30p-31-982fps-timer-A-23b0217-bad.png)1080-30p (https://imgbb.com/)

NTSC 24p
1st: 23b0219=>25.491 fps -- good       2nd: 23b0217=>25.586 fps -- bad
(https://image.ibb.co/cRM3eq/small-ob-zon-8-24p-timer-A-23b0219-25-491fps-good.png)1080-24p (https://imgbb.com/) (https://image.ibb.co/ngqUKq/small-ob-zon-8-24p-timer-A-23b0217-25-586fps-bad.png)1080-24p (https://imgbb.com/)   

3x Crop_Mode 30p NTSC
1st: 277025d=> Good (fought to write down fps)     2nd:277025b=> Bad
(https://image.ibb.co/eA01XA/small-ob-zo-10-3x-Crop-30p-timer-A-277025d-good.png)3xCrop-30p (https://imgbb.com/) (https://image.ibb.co/kHbKKq/small-ob-zo-10-3x-Crop-30p-timer-A-277025b-bad.png)3xCrop-30p (https://imgbb.com/)

PaL 1080 25p
1st: 2570219=>27.881 fps Good    2nd:  2570217=>27.985 fps Bad
(https://image.ibb.co/cOXWXA/small-ob-zo-12-25p-27-881fps-timer-A-2570219-good.png)1080-25p (https://imgbb.com/) (https://image.ibb.co/ktK6zq/small-ob-zo-12-25p-27-985fps-timer-A-2570217-bad.png)1080-25p (https://imgbb.com/)

NTSC 640x480 30p
1st: 23b0219=>31.864 fps Good   2nd:  23b0217=>31.980 fps Bad
(https://image.ibb.co/dEs1FV/small-ob-zo-14-640x480-30p-31-864fps-timer-A-23b0219-good.png)480-30p (https://imgbb.com/) (https://image.ibb.co/jj7JvV/small-ob-zo-14-640x480-30p-31-980fps-timer-A-23b0217-bad.png)480-30p (https://imgbb.com/)


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on November 21, 2018, 07:43:42 AM
Pretty sure the modified images labeled as "good" are actually bad. Double-check the right side; if in doubt, check the DNG as well. An image is good if and only if all the columns contain valid image data (from the original subject, not noise). It doesn't have to be totally broken to be "bad".

480p is no different from 1080p on this camera; no need to include it.

x5 is only one, regardless of FPS. No need to use FPS override.

The x5 24p (ob-zon-5-3xCrop-24p-over-ride.png) probably was done using FPS override, and looks bad to me. There are 24 bad columns, so timer A - whatever it was in that mode - was wrong by 6 units. That means, FPS override should be fixed as well (FPS_TIMER_A_MIN in fps-engio.c).

In the other images labeled as "good", there are more than 50 bad columns (the chart displays only the last 50), so I'm unable to tell the exact number.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 22, 2018, 05:42:12 AM
Ok ,
I used the "auto Preview" in all the tests , when the image got scrambled when I pushed timer A (decreased the number value)
I would increase the value until I got a clean image , noted the timer-A setting then decrease the number value by "2" this
would scrambled the image , so I called it "bad" .

I'll re-run the tests again with just the ML B/W low res . preview instead of "auto" preview .
with 1080 30p , 24, 25, & 3x crop 30p

Yes I ran 3x crop in 24p with frame over ride enabled .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 22, 2018, 06:40:56 AM
Alright just did 24p in 1080 with ML gray preview ,
Is it right now ? 1080-24p-ob-zones_11-21-2018.rar (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1080-24p-ob-zones_11-21-2018.rar)

(https://image.ibb.co/dMMXsA/small-ob-zon-1080-24p-Good-timer-A-23b21c-25-349-fps.png)1080 24p Good (https://imgbb.com/)
c0f06008 23b021c =>25.349 fps **good**

(https://image.ibb.co/iPEDzq/small-ob-zon-1080-24p-Bad-timer-A-23b21a-25-443-fps.png)1080 24p Bad (https://imgbb.com/)
c0f06008 23021a=>25.443 fps **bad**

Basically , I adjusted timer A (increase fps) until I saw a white line/bar in the image on the right side of Liveview and
dialed it back until the white line/bar was gone , (23b021c) that's the one I call "good"

On the one I call "bad" I dialed the value down by 2 (23b021a) .

If this is the right procedure , let me know and I'll finish the rest tomorrow .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on November 22, 2018, 07:14:16 AM
Nope, timer A still seems to be too low.

If in doubt, check or upload the DNG as well (either "dump raw buffer" from raw_diag, or plain silent pictures). Look closely at the last columns (they should not be black or white or otherwise static noise; they should have details from the test scene). These columns may not be obvious on the low-res preview.

Also, please pay attention to parity. Both of these examples show different parities, e.g. Canon value 0x23b is an odd number, modified value 0x21c is even. This causes some non-uniform noise (look closely at the images; it's no longer Gaussian).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 22, 2018, 07:31:16 AM
Ok , I thought I my have messed up . Now I understand sorry slow learner  :)
Actually I did do a Image_Dump on both tests , just incase there were needed.
Good-RAW-013.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Good-RAW-013.DNG)  & Bad-RAW-014.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Bad-RAW-014.DNG)

I'll re-run the test again & I'll  pay attention to parity
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on November 22, 2018, 08:16:08 AM
These artifacts all over the image are caused by different parity of timer A (compared to Canon value).

From the DNG, I can now do the math. Number of bad columns: 52 in the "good" image, 60 in the "bad" image. Difference of 8. Timer difference: 2 units. One unit of timer A reads out 4 pixels, so it adds up. The "good image" is off by 13 units, the "bad" one is off by 15 units.

The screenshot shows the last 50 columns, and all of them were bad, so I could not do the math from there.

0x21c + 13 = 0x21a + 15 = 553 units. Adding 1 since hardware register is written with timer-1.

Therefore, the minimum timer A value in 1080p is 554 (when doing the math for frame rate) or 553 in the hardware register. Canon default is 0x23b (hardware register) or 572 (when doing the math).

That means, in 1080p24 and 30, we can reduce timer A by no more than 18 units (starting from default value of 572). In 25p, default value will be 600, so you'll be able to reduce it by 46 units.

This value (554) will give the lowest rolling shutter and the highest resolution at any given frame rate. Why not just using this? When aiming for a certain frame rate, timers can be set to integer values, so it may not be possible to get the requested frame rate with 3 decimal places. For example, 23.976 => timer B would be 1807 => 23.974 (closest approximation). For 25p => timer B would be 1733 => 24.998 fps. Or, 1732 => 25.012 fps. For this reason, we should try slightly higher timer A values that may result in frame rates closer to what we expect, but these higher timer A values will require lower values for timer B (which might limit the vertical resolution).

If you'll ever want to reduce horizontal resolution in 1080p (e.g. to get higher frame rates or smaller rolling shutter), you may reduce timer A even more. For example, if you are OK with 1872 pixels, you may use A = 552 (2 units = 8 pixels). If you want higher resolutions, you will need to add 2 units for every 8 pixels; however, on 5D2, you won't be able to increase active area beyond 1880 in 1080p; there are not enough pixels on the sensor. You can still request higher resolutions and get a "4K" frame good for April 1st announcements, if you want :D

Now it's your turn to do the same with x5.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 23, 2018, 03:18:10 AM
Great ! thanks for the math ,

Quote from: a1ex on November 22, 2018, 08:16:08 AM
Now it's your turn to do the same with x5.
Ok , Weekend project .

Quote from: a1ex on November 22, 2018, 08:16:08 AM
You can still request higher resolutions and get a "4K" frame good for April 1st announcements, if you want :D
Yes , I like that idea ! :))
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 25, 2018, 07:32:04 AM
Continue on with the Timer A experiment , to see how far it can be push in 5x Zoom (3xCrop) .
but before I start I did a 4k frame in 1:1 (FHD) for kicks and
QuoteYou can still request higher resolutions and get a "4K" frame good for April 1st announcements, if you want :D
to see what happens , will something I thought that could not work .
(https://i.ibb.co/MGWLJff/4104x1249-RAW-015-Small-dng.png) (https://imgbb.com/)
1:1-FHD-4104x1249_RAW-015.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4104x1249_RAW-015.DNG)
4104x1249_RAW-015_uncroped.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4104x1249_RAW-015_uncroped.png)
Default Crop Size               : 4104 1249
Active Area                     : 18 160 1267 4264



So I crop off the bottom part & ended up with 4104x594 , as you can see it's vertically compressed
(https://i.ibb.co/6ts1crt/4104x594-compressed-RAW-015-Small.png) (https://imgbb.com/)
4104x594_compressed_RAW-015.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4104x594_compressed_RAW-015.png)

In Irfanview I vertically stretched the image 300% , pixel binding maybe "1:3" ?

I ended up with 4104x1782  :))
(https://i.ibb.co/ph14gK8/RAW-015-small-4104x1782.png) (https://imgbb.com/)
RAW-015_Unsqueezed_4104x1782.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-015_Uncompressed_4104x1782.png)

This was done in 1:1 FHD not 3x crop(5x zoom) that's what makes this so very interesting
Setting: 9.980 FPS
COMS[2] 0x40e ->0xe
c0f06008 0x23b023b ->0x23b080b **not too sure but very close to this**
c0f06088 0x4f40432 -> 0x4f4088a

So when I set CMOS[2] from 0x40e to 0xe that changed the image form the Full HD (pixel binding  & line skipping) to center crop at 1856x1249
then I slowed the frame rate to 9.980 fps and increased the raw image buffer to 0x4f4088a=>4096(4104)
I'm still wonder  how this possible , it shouldn't work unless I stumbled upon pixel binding mode  :o

Oh yea here the 4k ob zone that started this all
(https://i.ibb.co/7jB5JBR/4k-ob-zon-7-small.png) (https://imgbb.com/)
4K-FHD-ob-zon-7.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4K-FHD-ob-zon-7.png)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on November 25, 2018, 09:52:22 AM
Yeah, binning factors on X and Y are pretty much*) independent. With the notation from crop_rec, this is the 3x1 binning mode (read 1 line, skip 2, read every column). You probably want to experiment with the opposite, i.e. 1x3.

Binning factor is controlled like this:
- horizontally: from CMOS registers (model-specific, CMOS[2] on 5D2); apparently doing proper binning on all models;
- vertically: from ADTG[1000/100C] or [8000/800C]; one register switches between 5 = read out every line and 6 = enable line skipping/binning, the other selects the number of lines skipped/binned.

*) On 5D3, in this mode (3x1) there are some artifacts I don't know how to fix; register ADTG[8806] appears to control these.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on November 29, 2018, 07:43:11 PM
When i was shooting with this build, i had problem with opening 12 files in mlv producer
I got run-time error 52 - bad file name or number.

Did not have such a problem in the past.

Any ideas?

Tnx
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 30, 2018, 04:42:38 AM
It more then likely a problem with mlv producer , remember there was a issue with too many files open and causing a crash.
Report it on mlv producer thread , and post your log file from mlv producer also try deleting all your "idx" (index files) in the "data" subdirectory folder "IDX"


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 30, 2018, 04:48:26 AM
Quote from: a1ex on November 25, 2018, 09:52:22 AM
Yeah, binning factors on X and Y are pretty much*) independent. With the notation from crop_rec, this is the 3x1 binning mode (read 1 line, skip 2, read every column). You probably want to experiment with the opposite, i.e. 1x3.

Binning factor is controlled like this:
- horizontally: from CMOS registers (model-specific, CMOS[2] on 5D2); apparently doing proper binning on all models;
- vertically: from ADTG[1000/100C] or [8000/800C]; one register switches between 5 = read out every line and 6 = enable line skipping/binning, the other selects the number of lines skipped/binned.

*) On 5D3, in this mode (3x1) there are some artifacts I don't know how to fix; register ADTG[8806] appears to control these.

That's great , I gave 3x1 a try but unsuccessful -- It seem I can't find that long vertical with CMOS[2]
more investigation needed , I must be missing something .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 30, 2018, 07:08:09 AM
I'm starting to finalize the crop_rec preset's , I'll keep 2880x1080 as is but it will not have the broken preview
as I found a fix with a1ex's help (as always  :) ) here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg208497#msg208497) is the post with proof.

2400x1200 @23.976 is next , It's 90% , just getting some image ghosting a little , but clean black levels .
5x frame blanking & CMOS[1] reg's were needed
CMOS[1] was changed to 0xe6a ->0xa28 (I think , I forgot to write it down)
(https://i.ibb.co/xh6Z3yc/VRAM2-23-9762-4kx1-2k-almost-clean-small.png) (https://imgbb.com/)

2144x1200 @ 23.976 fps is stable and ready to go , post is here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg208478#msg208478) with samples
4096x1074 @ 18fps is working , and I will include it in the presets , I may be able to squeeze a few more frames/sec to 20fps
full res 5632x3750 @4 fps
I do plan on including a squeezed 48/50fps 1856x704 but there's still a black level issue , so that need to be fix before I can include it


This is not the final list of crop_rec preset's , but these are the ones I can get to work right now


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on November 30, 2018, 02:16:45 PM
reddeercity,

you were right!
the problem was with mlv producer. I did not use latest version. No problems with latest one.

very excited about your future crop build.

tnx
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on December 01, 2018, 09:21:59 PM
Quote from: reddeercity on November 30, 2018, 07:08:09 AM
I'm starting to finalize the crop_rec preset's , I'll keep 2880x1080 as is but it will not have the broken preview
as I found a fix with a1ex's help (as always  :) ) here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg208497#msg208497) is the post with proof.

2400x1200 @23.976 is next , It's 90% , just getting some image ghosting a little , but clean black levels .
5x frame blanking & CMOS[1] reg's were needed
CMOS[1] was changed to 0xe6a ->0xa28 (I think , I forgot to write it down)
(https://i.ibb.co/xh6Z3yc/VRAM2-23-9762-4kx1-2k-almost-clean-small.png) (https://imgbb.com/)

2144x1200 @ 23.976 fps is stable and ready to go , post is here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg208478#msg208478) with samples
4096x1074 @ 18fps is working , and I will include it in the presets , I may be able to squeeze a few more frames/sec to 20fps
full res 5632x3750 @4 fps
I do plan on including a squeezed 48/50fps 1856x704 but there's still a black level issue , so that need to be fix before I can include it


This is not the final list of crop_rec preset's , but these are the ones I can get to work right now




Sounds amazing.

Are any of these available to test yet?

Also do you have a code repository/upload it somewhere?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 02, 2018, 05:06:33 AM
No , not yet , thou you could load the adtg_gui.mo (https://builds.magiclantern.fm/modules.html#iso-research) and manual set it .
I'm not the fastest or best coder  :D so may take me a little time , plus I would like to update 5d2 4k branch in to the main 4k branch
since we have "raw_slurp" working now ( 10bit_12bit (https://builds.magiclantern.fm/experiments.html) FHD) like 5d3/D5 cams , there will be no need for waza57 edmac hack (to extend higher resolutions)

I may also add a cheap & dirty 4k/3.6k squeezed 3x1 in 1:1 FHD , there will be some Aliasing but not as bad as 3x3 .
I'm close to getting 24 fps (currently 20.5 fps unstable ,  17.3 fps stable no image artifacts) @ 3648x660 squeezed ( un-squeezed 3648x1980) 1.85:1 A.R. almost 16x9  :)
I'm most likely will have to reduce the vertical to 2.35 A.R. (1550) (squeezed 519) to get more fps , or more -- still experimenting .

@Ilia3101 , being using your app (mlv app 1.3) to get the right final size with 3x1 , thanks
It's 500% better then the last time I use it , (6 months ago)

From Image_Dump
(https://i.ibb.co/TtfX9D9/3672x661-squeezed-RAW-018-small.png) (https://imgbb.com/)
3672x661-squeezed-RAW-018.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3672x661-squeezed-RAW-018.DNG)
3672x661-squeezed_RAW-018.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3672x661-squeezed_RAW-018.png)


(https://i.ibb.co/TmZHkxq/3672x1983-RAW-018-small-360.png) (https://imgbb.com/)
3672x1983-RAW-018.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3672x1983-RAW-018.png)

Here a short 13Mb clip 14 seconds 3648x1980-17.3fps-M01-0004.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3648x1980-17.3fps-M01-0004.mp4)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 02, 2018, 07:58:56 AM
Ok got it !  :)) 24 fps !! 3.6k Squeezed !!!
(https://i.ibb.co/5LkrV1T/Squeezed-RAW-020-small.png) (https://imgbb.com/)

3672x513-RAW-020.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3672x513-RAW-020.DNG)
3672x513-SqueezedRAW-020.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3722x513-SqueezedRAW-020.png)

3672x1539 @ 24 fps A.R. 2.375:1
(https://i.ibb.co/yQfrQbt/Un-Squeezed-RAW-020-Small-360.png) (https://imgbb.com/)
3621x1539-UN-Squeezed_RAW-020.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3621x1539-UN-Squeezed_RAW-020.png)

(https://i.ibb.co/k07x1JG/VRAM11-small.png) (https://imgbb.com/)


2 clips  -  3648x1536 @ 24.003 fps
No-frame_blanking-M01-2328.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/No-frame_blanking-M01-2328.mp4)
Frame_blanking-M01-2343.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Frame_blanking-M01-2343.mp4)

3672x513 @ 24.004fps
ADTG1[1061] frame blanking was add to try and get the frame ghosting to stop , still need more work.  :-\

(https://i.ibb.co/THFWhCT/VRAM12-small.png) (https://imgbb.com/)

A & B Timer info
(https://i.ibb.co/wrvS3s1/VRAM10-small.png) (https://imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on December 02, 2018, 08:20:37 PM
wow...cant wait!!!

You are the man.

This is more than a  dream for 5d2
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 03, 2018, 08:38:59 AM
Going back to Hi-Frame rate 50/48fps to see if I can clean up the black levels.
Got side tracked , wanted to see how far I could push the frame rate passed 50 fps
in a smaller resolution (squeezed-1600x500=>un-squeezed-1600x832) (don't think 1856 will work , but didn't try)
I could push it to 64.020 fps but Liveview seemed to lockup/freeze -- battery pull.

But I can record at a little less frame rate 63.050 fps !! :D
Thou 62.5fps is more stable and has at least good black levels (101-1013 10bit) and mostly if not all artifact free.

Setting for 63.051fps @ 1600x500 (squeezed)
(https://i.ibb.co/j3wXbT3/VRAM18-63fps-small.png) (https://imgbb.com/)

From .mlv file @ 62.50fps Squeezed
(https://i.ibb.co/W037mWQ/M02-2327-000000-small.png) (https://imgbb.com/)
1600x500-M02-2327_000000.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1600x500-M02-2327_000000.dng)
1600x500-M02-2327_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1600x500-M02-2327_000000.png)

Un-Squeezed 1600x832 @ 62.50fps
(https://i.ibb.co/MRSgHPC/1600x832-M02-2327-000000-small.png) (https://imgbb.com/)
1600x832-M02-2327_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1600x832-M02-2327_000000.png)

Here a short 5 second h264 clip -- 1600x832 @ 62.50fps
1600x832-62.5fps-M02-2327.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1600x832-62.5fps-M02-2327.mov)

Photo from my iPhone with the camera @ 60.024fps
(https://i.ibb.co/VtXvzJ4/60-024fps.png) (https://imgbb.com/)

There one thing really puzzles me , as I increase the frame rate with timer "B" the image clean up ,
the black levels are better and no corruption . If you pan fast to a low light area then back to a Hi-Constat (bright Object) then there's the
odd corrupted frame or frames .

Wondering about sensor readout , 5d2 as noted by a1ex is 100 Mpixel/s (Mega pixel per second) on 4 Channels or 24 per channel
so could the bad black level/corruption be that at lower fps  (48/50) the channel band width is too wide as the 5d3 has twice this (at 8 channels) so
a narrower band width . Not too sure how to read this , I guess this sensor is too slow for what I'm expecting it to do.
Still trying to get my head around this sensor readout stuff to understand the limitation correctly .

Edit: found this about readout on CMOS sensor , Interesting !
https://www.researchgate.net/figure/a-Readout-direction-in-one-channel-and-b-the-timing-for-sensor_fig2_238541020
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on December 03, 2018, 11:10:56 AM
Quote from: reddeercity on December 02, 2018, 07:58:56 AM
Ok got it !  :)) 24 fps !! 3.6k Squeezed !!!
Could you explain me the reel pro of a high horizontal resolution squeezed at 24fps ? I can understand at 62fps to minimize the data rate per picture and getting better fps.
In this test image the aliasing is awful once unsqueezed, so I'm thinking I'm miss something.
David
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 04, 2018, 01:18:07 AM
Quote from: benoit on December 03, 2018, 11:10:56 AM
Could you explain me the reel pro of a high horizontal resolution squeezed at 24fps ? I can understand at 62fps to minimize the data rate per picture and getting better fps.
In this test image the aliasing is awful once unsqueezed, so I'm thinking I'm miss something.
David

Those are all experiment's for investigation purposes .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 04, 2018, 01:43:11 AM
Ok I think I have found 3x1 pixel binding for reduce aliasing .

1856x1248 3x3 @ 29.97 fps (I haven't increased the raw height yet to 3750)
(https://i.ibb.co/19JV6dj/M03-1806-00001-3x3-small.png) (https://imgbb.com/)
M03-1806_00001-3x3pinning.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M03-1806_00001-3x3pinning.png)

1856x412 3x1 
(https://i.ibb.co/HPBdTMt/M03-1806-00001-3x1pinning-small.png) (https://imgbb.com/)
M03-1806_00001-3x1pinning.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M03-1806_00001-3x1pinning.png)


CMOS[1] 0xc00->0xbe0
ADTG12[100c] 0x2->0x100
c0f06084 0x10036 ->0x36 **I think , have to check again**


Need to clean up ,  just my very first try & test with this binning mode , still experimenting .
In theory , 3750/3=>1250
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 05, 2018, 07:10:15 AM
Finding different pining modes on 5d2 in 1:1 FHD (3x3) with
ADTG12[100c] 0x2 ->100
Compresses the image vertical 3x - maximum raw height 402=>1206
1856x402-M04-2332_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1856x402-M04-2332_000000.png)
1856x1206-300%-M04-2332_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1856x1206-300%25-M04-2332_000000.png)

The image pretty bad , so I won't post a screen shot

ADTG12[100c] 0x2 ->107
Compresses the image vertical 6x - maximum raw height 210=>1260
1856x210-M04-2336_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1856x210-M04-2336_000000.png)
1856x1260-600%-vertical-M04-2336_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1856x1260-600%25_vertical-M04-2336_000000.png)

(https://i.ibb.co/RHYVdk8/VRAM19-small.png) (https://imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 05, 2018, 09:54:56 AM
Ok looks like I found 3x1 I think , As I thought ADTG12[100c] does a lot of the pinning/line skipping

(https://i.ibb.co/Z1SzY2j/VRAM21-small.png) (https://imgbb.com/)
So 0x0 put the vertical in stretch mode , it seem 33.3% of the original height put the image in proper aspect ratio again
ADTG12[100c] 0x2->0x0

1888x1248 ( I changed c0f060084 0x10036->0x10026=> 1888)
(https://i.ibb.co/8YfscBj/1888x1248-M05-0138-000000-small.png) (https://imgbb.com/)
1888x1248-M05-0138_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1888x1248-M05-0138_000000.png)


Re-sized the vertical (33.3% of original height) to 416
(https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1888x416-33%25-M05-0138_000000_small.png)
1888x416-M05-0138_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1888x416-M05-0138_000000.png)

Still trying to get the ghosting to stop with cmos[1]
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on December 05, 2018, 02:25:38 PM
Quote from: reddeercity on December 05, 2018, 09:54:56 AM
Ok looks like I found 3x1 I think , As I thought ADTG12[100c] does a lot of the pinning/line skipping
If it's vertical, you found the 1x3 binning ?  ;) The same as theBilalFahkouri in "Magic Lantern Cinema Camera - Dual ISO without aliasing & without quality loss!" thread ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 06, 2018, 04:12:28 AM
It's 3x1 (1856 horizontal is still in 3x pinning/line skipping)  & yes but with out dual ISO
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 06, 2018, 06:50:27 AM
Pushing 1280x720 to 72.053fps  :o
A few artifact but I'm more interested in how far the timers can be pushed , with different reg. configuration .

(https://i.ibb.co/JzkTpcy/VRAM26-small.png) (https://imgbb.com/)

(https://i.ibb.co/F4WvT36/VRAM25-small.png) (https://imgbb.com/)

A short clip 1280x722-72.05fps-M05-2306.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1280x722-72.05fps-M05-2306.mov) -- h264 26Mb

One thing I notice , when I started to pushing "B"timer over 55fps the back level started to clean up & over 65fps black level seem to stabilize .
It's almost like in multiplies of 30 fps gives good black level , e.g. 60fps

Edit: Here are some short hi-frame rate clips that got me to 72fps , the best one is 67fps .
It seems the sweet spot is around 60-62.5fps , I get this with 1600x832 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg208955#msg208955)

1280x722-70.086-fps-M05-2305.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1280x722-70.086-fps-M05-2305.mov)
1280x722-69.094fps-M05-2304.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1280x722-69.094fps-M05-2304.mov)
1280x722-68.034fps-M05-2303.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1280x722-68.034fps-M05-2303.mov)
1280x722-67.006fps-M05-2302.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1280x722-67.006fps-M05-2302.mov)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 07, 2018, 08:42:37 AM
Finally got It 2560x1200 @ 23.976fps with Clean preview while recording raw and say Goodbye to Frozen Preview forever :))

(https://i.ibb.co/yFtc8D9/2560x1200-VRAM30-small.png) (https://imgbb.com/)

(https://i.ibb.co/st4ksDt/2560x1200-M07-0020-00001-small.png) (https://imgbb.com/)
2560x1200-M07-0020_00001.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2560x1200-M07-0020_00001.png)
2560x1200-M07-0020_000000.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2560x1200-M07-0020_000000.dng)
Default Crop Size               : 2568 1200
Active Area                     : 52 160 1252 2728


(https://i.ibb.co/ydQHx5k/i-Phone-320-small.png) (https://imgbb.com/)
iPhone video of 5d2 showing un-frozen liveview while recording Clean_preview_recording-IMG_2690.MOV (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Clean_preview_recording-IMG_2690.MOV)
Half shutter works for full color preview in standby & recording raw video -- I show this in the video from my iPhone.


Here a short h264 clip 2560x1200-23.976fps-M07-0020.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2560x1200-23.976fps-M07-0020.mov) 36Mb

One thing the bug me is in the raw image dump I can see corruption line on the right side (like timer "A" pushed to far) but it's not in the Raw .mlv video @ 10bit
The image dump is in 14bit , could that be the issue ? but clean in 10bit , strange .
Exiftool black & white level from 14bit dump
Black Level                     : 1027
White Level                     : 16200

10bit dng report correct level
Black Level                     : 112
White Level                     : 1013


(https://i.ibb.co/1XpZpsJ/2560x1200-RAW-038-small.png) (https://imgbb.com/)
2568x1200-RAW-038.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2568x1200-RAW-038.DNG)

Can also be seen in the Liveview_dump from the LCD/422
(https://i.ibb.co/Svn6GHP/2560x1200-LV-045-small.png) (https://imgbb.com/)
2560x1200LV-045.422 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2560x1200LV-045.422)

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 07, 2018, 08:55:59 AM
There´s no source code to check? Why not post stuff on bitbucket? Very easy to follow and check into things.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 07, 2018, 09:08:55 AM
It's the same code I pointed out to you before ,  I'm doing everything with ADTG_GUI.mo
so there no special crop_rec module , just plain old mlv_rec in 3x Crop_ mode (5xZoom)

The next step is to code it in to the crop_rec module .
Everything can be reproduce , that why I post of those images so other can fellow along .
It you what to get other D4 cams working with this , first the redirect buffer("raw_slurp") need to be
figured out and I also pointed this out before .

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 07, 2018, 09:14:13 AM
Ok I see. Is the generic power time code in crop_rec.c applied to 5d2? Might fix the stripes if so or at least there would be a place to look for fixing.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 09, 2018, 12:32:11 AM
No , 5d2 doesn't use those. The image is clean in 10bit and that what really counts as 14bit would be only 1-2 second of record time
At least 10bit I can get 15-20 seconds , until I can get working on Lossless & CF interface overclocking .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 09, 2018, 02:21:38 AM
Ok I'm starting to code this (any help/advice would be great) 
I'm not the greatest at this so please be patient :)
All the new valve's I'm using are in the previous post #379
I'm working off crop_rec_4k_5d2 branch , waza57 source

I'm coding a 2560x1200 @ 23.976 preset

crop_rec.c-1436 (https://bitbucket.org/waza57/magic-lantern/src/eff925c034ff006014d2f026bba3f8259884a220/modules/crop_rec/crop_rec.c?at=crop_rec_4k_5D2&fileviewer=file-view-default#crop_rec.c-1436)
if (is_5D2)
    {
       int timerA = 727;
       int timerB = 1374;
       int a = reg_override_fps_nocheck(reg, timerA, timerB, old_val);
       if (a) return a;
       
       switch (reg)


crop_rec.c-1684 (https://bitbucket.org/waza57/magic-lantern/src/eff925c034ff006014d2f026bba3f8259884a220/modules/crop_rec/crop_rec.c?at=crop_rec_4k_5D2&fileviewer=file-view-default#crop_rec.c-1684)
   for (uint32_t * buf = (uint32_t *) regs[0]; *buf != 0xFFFFFFFF; buf += 2)
    {
        uint32_t reg = *buf;
        uint32_t old = *(buf+1);
        if (is_5D2)
        {
          if (reg == 0xC0F06088) 
            {
               engio_vidmode_ok = (crop_preset == CROP_PRESET_CENTER_Z)
                ? (old == 0x46A04BA)                        /* x5 zoom */
                : (old == 0x4E7058A);   /* 1080p or 720p */
            }
        }
        else


But my question is this :
I need to add
c0f0713c
c0f07150
c0fo8184
c0f08188
c0f08518

Do I add them after the raw resolution like this
case 0xC0F0713c:
                return (old_val & 0xFFFF0000) + 0x500

or
if (reg == 0xC0F0713c) 
            {
              ? (old == 0x476)                     
                : (old == 0x500)
             }
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 09, 2018, 08:52:46 AM
I built a preset from your registers 2560x1200 in here:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec.mo

Wonder if would work with 5D2 at all?

Working in this branch. Didn´t commit any changes yet:
https://bitbucket.org/Dannephoto/magic-lantern/branch/crop_rec_4k_mlv_snd_isogain_1x3_presets

I don´t have the 5D2 so take it with a grain of salt...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 09, 2018, 09:32:07 AM
Ok , can you post your code please
I need to look at the code first before I try it ,
Don't what to brick my camera .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 09, 2018, 09:43:58 AM
code:
https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec.c
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 09, 2018, 09:48:12 AM
Thanks , I'll have a look later tonight , early here - 2am
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 09, 2018, 09:57:31 AM
If it doesn't work here it should be fairly easy to put it into waza57 code.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 10, 2018, 04:05:06 AM
Yea it's different then the D4 branch , anyways I mainly was looking for the code structure
you gave me some good example to follow , that helps a lot .
 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 11, 2018, 09:34:33 AM
Look in to HDMI output Reg's
c0f11314
2eb045f->2eb0dof

In Canon Preview
Changes the liveview HDMI HD Width only not raw , I was able to have proper field of view in Crop_Mode 2144x1074 on the HDMI
instead of having a center crop on the hdmi  (still a little distorted & compressed vertically) . Normally with HDMI while in Crop_mode
Let say at 2144x1074 the image gets centered crop to 1920x1080  so you loss the field of view unless you are at 1920 all ready .
I still have to find the HD height yet the first set of valve didn't do anything.
I was mainly only trying to get rid of the black bar on the side of the hdmi ( this cause it to be 1650x1080 and I looking for 1920x1080 or more)
But I think I may have to increase the HD buffer some how , still early in this investigation/development .
I had my Ninja connected to hdmi while I adjusted the reg's and recorded a short 2min clip 85Mb
First I set the reg . then at the 20 second mark I compare it to the reg turn off

(https://i.ibb.co/ynvSf5Z/5d2-HDMI.png) (https://imgbb.com/)

H264-5d2-hdmi-00208.MOV (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/H264-5d2-hdmi-00208.MOV)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Aperture Science on December 15, 2018, 05:06:45 AM
Is there any way to boost 5D2's CF slot speed up to 90m/s? Seems like the 4k recording is about to success, but the slot is dragging all the things slow. By changing the speed of the "Bus speed?" (I have no ideal what is in the camera). I am interested in the "cf_acc.mo" could anyone help me explain what should I change in the "cf_acc.mo"? I will test the value and the result (Hopefully I will not brick my camera lol)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 16, 2018, 05:15:28 AM
Yes ,A1ex experimented with this before . But not sure that the total answer -- We are very close to getting Lossless compression working
It a matter of finding the correct "TwoInTwoOut" r/w addresses .
Currently I can capture a lossless FRSP  (https://www.magiclantern.fm/forum/index.php?topic=18443.msg198044#msg198044) (5632x3752) but can't save it to the CF card , cam freezes .
So from my understanding , the module just needs to be compiled -- I looked in to it a few weeks ago but never got around to compiling it
read this here (https://www.magiclantern.fm/forum/index.php?topic=12862.msg206061#msg206061)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 16, 2018, 08:44:31 AM
just a update , fix 2560x1200 @ 23.976 preset , got rid of the corruption on the right side of the 14bit raw_dump image
Pushing this preset , and the max I can get is 2712x1200 @ 23.976 , this close to the limits for vertical resolution @ 23.976
Unless I can get more resolution  @ 23.976 playing around with the Head Timer 3 & 4 . The problem isn't getting higher resolution
it's getting higher frame rate (23.976) instead of 12 or 18 fps @ 4k ( I could use a vertical compression height trick to get 4k)

Almost got 3008x1080 @ 23.976 working  , still have some frame corruption , (frames are moving sideway & out of order)
need to find a better CMOS[1] reg.

Hoping to have a crop_rec module for Xmas or by the New Year for tester's
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 16, 2018, 09:20:45 AM
Quote from: reddeercity on December 16, 2018, 08:44:31 AM
just a update , fix 2560x1200 @ 23.976 preset , got rid of the corruption on the right side of the 14bit raw_dump image
Pushing this preset , and the max I can get is 2712x1200 @ 23.976 , this close to the limits for vertical resolution @ 23.976
Unless I can get more resolution  @ 23.976 playing around with the Head Timer 3 & 4 . The problem isn't getting higher resolution
it's getting higher frame rate (23.976) instead of 12 or 18 fps @ 4k ( I could use a vertical compression height trick to get 4k)

Almost got 3008x1080 @ 23.976 working  , still have some frame corruption , (frames are moving sideway & out of order)
need to find a better CMOS[1] reg.

Hoping to have a crop_rec module for Xmas or by the New Year for tester's

which register are you setting for these last presets? So if you go abit lower let´s say 3008x1040 it should be corruption free?

Vertical compression height trick? How?


EDIT:
Cool. Got 3096x1320 24 fps on my 100D now.


        case 0xC0F06804: return 0x5490331; // 3072x1320  x5 Mode;

        case 0xC0F06824: return 0x3ca;
        case 0xC0F06828: return 0x3ca;
        case 0xC0F0682C: return 0x3ca;
        case 0xC0F06830: return 0x3ca;
       
        case 0xC0F06010: return 0x34b;
        case 0xC0F06008: return 0x34b034b;
        case 0xC0F0600C: return 0x34b034b;

        case 0xC0F06014: return 0x5f7;
        case 0xC0F07150: return 0x428;
        case 0xC0F0713c: return 0x555;

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 16, 2018, 05:37:37 PM
Quote from: Danne on December 16, 2018, 09:20:45 AM
Cool. Got 3096x1320 24 fps on my 100D now.

That's a Digic V camera. Somewhat off topic but good to know this stuff applies to all ML enabled cameras.

What would be sweet is taking reddeercity's findings on the 5D2 and applying it to other Digic IV cameras. IDA_ML has been pushing to get this on the 7D but I keep getting lost trying to figure it out. The 50D might be easier to port -- @aprofiti are you following this?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: IDA_ML on December 16, 2018, 07:22:36 PM
Quote from: dfort on December 16, 2018, 05:37:37 PM
That's a Digic V camera. Somewhat off topic but good to know this stuff applies to all ML enabled cameras.

What would be sweet is taking reddeercity's findings on the 5D2 and applying it to other Digic IV cameras. IDA_ML has been pushing to get this on the 7D but I keep getting lost trying to figure it out. The 50D might be easier to port -- @aprofiti are you following this?

Oh yeah, can you imagine having these insane resolutions on the 7D with its 90 MB/s write speed?  3k continuous recording at 24 fps and 10-bit lossless, no aliasing, no VAF filter, no focus pixels, small crop factor, wonderful colors, unique filmic look - that would be a dream come true for all 7D shooters!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 17, 2018, 03:47:51 AM
Hi all. First thanks to Reddeercity and other guys for keeping working on it all. I check this thread every day.
I have a question. Maybe someone else has this problem. My 5d2 has a bad feature. When I magnificate the image with the 5x or 10x mode, it jumps back to 1x after a few seconds (from 1 to 30 second approx. It depends...) And if I don't have time to press the recording mode, the mode changes from crop to standard and the resolution goes down again to 1856 ... very annoying glitch. In theory, is there any program way to force magnification?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 17, 2018, 04:28:00 AM
@Igor_Braun , No I haven't had this issue , have you tried to clear your setting folder ?
Are you using the build from the first page (https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476) ?
Or you could make a short cell phone video and post it here , showing this problem .
Are you using the crop_rec 3.5k preset (2880x1080) when this happens ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 17, 2018, 04:34:12 AM
Quote from: Danne on December 16, 2018, 09:20:45 AM
Vertical compression height trick? How?
Look here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg208930#msg208930) & also here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg208931#msg208931)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 17, 2018, 04:52:28 AM
Quote from: IDA_ML on December 16, 2018, 07:22:36 PM
Oh yeah, can you imagine having these insane resolutions on the 7D with its 90 MB/s write speed? 
3k continuous recording at 24 fps and 10-bit lossless, no aliasing, no VAF filter,
no focus pixels, small crop factor, wonderful colors, unique filmic look - that would be a dream come true for all 7D shooters!
Just to be clear 7D dose not have focus pixels (https://www.magiclantern.fm/forum/index.php?topic=16054.msg155923#msg155923) if you see any that's would be hot pixels.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 17, 2018, 05:24:17 AM
@dfort , the redirect buffer needs to be the very first thing to be figured out , all the rest will fall in place .
Being it's a dual processor , makes it that more difficult .
I do have my eye on a 7d with a battery grip at my local canon camera store in there used/trade in's dept. for $300 cad = $224 usd
If it's still there in the new year , I may buy it to help port this but not to sure .
I'm actually looking in to a use Red Scarlet X (https://www.ebay.ca/itm/Red-Scarlet-X-DSMC-4K-Camera-Kit-with-Accessories/292816232987?hash=item442d35221b:g:ev8AAOSw1IVb52A~:rk:2:pf:0) right now , has ef mount plus everything to get to work .
I'm a big fan of the FCPX Red camera support , fit my pipeline perfectly.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 17, 2018, 08:00:40 AM
Truth be known I've been shooting mostly with a Canon C300 lately, mostly with EF-S lenses that work nicely with the dual pixel autofocus. No pipeline or playback issues with these files.

It seems that one of the most important tools, adtg_gui module, doesn't work on the 7D. I just dug into it (https://www.magiclantern.fm/forum/index.php?topic=10111.msg209402#msg209402). I've got a love/hate relationship with this camera. One day I'm thinking about getting back into the Portable ROM dumper (https://www.magiclantern.fm/forum/index.php?topic=16534.msg192336#msg192336) to figure out how to pry into the master processor so I can properly finish the firmware update to 2.0.6 and the next day I'm looking for someone who will take it off my hands so I can regain my sanity. Depending on my mood I might let you have it for a bottle of Canadian maple syrup.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 17, 2018, 11:40:37 AM
@reddeercity, you know, I think this is my camera glitch, because it doesn't depend whether it has Magic Lantern installed or not. I will make a small phone video...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 18, 2018, 05:58:05 AM
Quote from: dfort on December 17, 2018, 08:00:40 AM
Truth be known I've been shooting mostly with a Canon C300 lately, mostly with EF-S lenses that work nicely with the dual pixel autofocus. No pipeline or playback issues with these files.
Nice , the close one to me is 200 mile away , a real pain -- thou they will ship it to me
But they do have a c300 markii $350 day/weekend (fri-monday) or $850 a week , not too badly priced for body rental
If I was doing something serious , I'd be all over that  :)
Quote from: dfort on December 17, 2018, 08:00:40 AM
It seems that one of the most important tools, adtg_gui module, doesn't work on the 7D.
adtg_gui module was broken for d4 cam up until a few mouths ago , I discovery this when I could  made changes with digic poke (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205062#msg205062) and not adtg_gui module .
a1ex fixed adtg_gui here (https://www.magiclantern.fm/forum/index.php?topic=10111.msg205480#msg205480) for d4 cams , as I can confirm for 5d2 , the adtg_gui.mo on the
module download page (https://builds.magiclantern.fm/modules.html) is the one I have been using of late . So it should work on 7d.

Quote from: dfort on December 17, 2018, 08:00:40 AM
I've got a love/hate relationship with this camera.
One day I'm thinking about getting back into the Portable ROM dumper (https://www.magiclantern.fm/forum/index.php?topic=16534.msg192336#msg192336) to figure out how to pry into the master processor so I can properly finish the firmware update to 2.0.6 and the next day
I'm looking for someone who will take it off my hands so I can regain my sanity.
Depending on my mood I might let you have it for a bottle of Canadian maple syrup.
Sure I'll sent you 2 ,  It like water up here  :P 
which do like better  , Grade A: Light Amber, Medium Amber, Dark Amber.  ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 18, 2018, 07:04:47 AM
What branch are you using for compiling for the 5D2? Checking your autoexec.bin it says:
Magic Lantern crop_rec-4k-5D2-eXperimental.2018Oct22.5D2212
Camera   : 5D2
Firmware : 212
Changeset: 8bfbb0ca228e+ (crop_rec_4k_5D2) tip
Built on : 2018-10-23 05:49:51 by david@reddeercity


Tried downloading a 7D version from  Branch: lua_fix but it from will not work with adt_gui.mo from iso-research branch.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 18, 2018, 08:38:47 AM
If you turn on the adtg_gui module on a 7D you'll see that it won't load. Take a look at adtg_gui.c you'll see that the code for the 7D is missing. I was able to find the missing pieces (https://www.magiclantern.fm/forum/index.php?topic=10111.msg209402#msg209402) so the module will load but it still doesn't work. Maybe you need to poke the 7D some other place?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 18, 2018, 09:32:50 AM
Forgot to mention i updated 7D with above code before loading it.
I could of course start erasing error codes but it would be the better starting point to get a branch and a blessed  :P adtg_gui module like the one for 5D2.
Then again I am maybe only opening up a can of worms...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: IDA_ML on December 18, 2018, 01:20:27 PM
Quote from: Danne on December 18, 2018, 09:32:50 AM
Then again I am maybe only opening up a can of worms...

Ha ha, Danne, there is nothing bad opening a can of worms.  You can catch a big fish using them as a bate.  Without worms you catch nothing!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on December 18, 2018, 06:13:57 PM
Quote from: Danne on December 18, 2018, 09:32:50 AM
I could of course start erasing error codes but it would be the better starting point to get a branch and a blessed  :P adtg_gui module like the one for 5D2.

The current iso-research branch should work on the 5D2.

Quote from: reddeercity on December 18, 2018, 05:58:05 AM
I can confirm for 5d2 , the adtg_gui.mo on the
module download page (https://builds.magiclantern.fm/modules.html) is the one I have been using of late . So it should work on 7d.

Not on the 7D without this:

modules/adtg_gui.c
    else if (is_camera("7D", "2.0.3"))
    {
        ADTG_WRITE_FUNC = 0xFF2C0944; //"[REG] @@@@@@@@@@@@ Start ADTG[CS:%lx]"
        CMOS_WRITE_FUNC = 0xFF2C0B3C; //"[REG] ############ Start CMOS"
        ENGIO_WRITE_FUNC = 0xFF1F6B20;  // from stubs
        ENG_DRV_OUT_FUNC = 0xFF1F675C;
        SEND_DATA_TO_DFE_FUNC = 0xFF32B800; //"[REG] DFE:[%#lx]"
    }


I'm pretty sure these are the missing pieces but it doesn't work. Maybe double check it?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 19, 2018, 08:52:58 AM
Quote from: Danne on December 18, 2018, 07:04:47 AM
What branch are you using for compiling for the 5D2? Checking your autoexec.bin it says:
Magic Lantern crop_rec-4k-5D2-eXperimental.2018Oct22.5D2212
Camera   : 5D2
Firmware : 212
Changeset: 8bfbb0ca228e+ (crop_rec_4k_5D2) tip
Built on : 2018-10-23 05:49:51 by david@reddeercity

Tried downloading a 7D version from  Branch: lua_fix but it from will not work with adt_gui.mo from iso-research branch.
I was explaining to another user , the source link is here with all the bits to work with 10-12bit FHD
https://www.magiclantern.fm/forum/index.php?topic=19336.msg208026#msg208026

Have you tried "Digit Poke" ?
You manually input any reg. number & change it's parameters
https://www.magiclantern.fm/forum/index.php?topic=19336.msg205062#msg205062
Look like 7d not define , for some reason.
will needs to be fix , I can't help much with this as a 7d is needed .
when I catch up on the my coding of the crop_rec module for 5d2
then I can help more .

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on December 20, 2018, 11:41:04 PM
Quote from: dfort on December 16, 2018, 05:37:37 PM
The 50D might be easier to port -- @aprofiti are you following this?
Yes, I kept reading what reddeercity is posting and maybe I can jump doing some tests in the next weeks, but I need some help from you guys.

Looking at register map (http://magiclantern.wikia.com/wiki/Register_Map) to have a better idea of what is going on here, but I feel the need of more documentation about...

From what I understood, first I need to figure out which registers are responsible of height and width of liveview/capture image and then figure out how much fps can be pushed on each resolution, using raw_diag to check bounds for timer values. Is this right?

I tried to poke some registers with adtg_gui in iso-reaserch branch, but no visible effects on image size...
Even changing resolution from raw video menu and log values doesn't seems to show any useful info (well no info at all...)

Modified Cmos[1] and Cmos[2] but nothing changed, leaving these register: C0F06008, C0F06084, C0F06088... after figured out how to increase resolution

Do I need to switch to crop before make anything?
If I remember right I need to push the "zoom" button to switch to 5x, but it doesn't do anything. I think I'm forgetting something about....

Started to read again from the beginning of this thread and I would like to know if is still needed to find/adapt buffer to accommodate larger image size.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 21, 2018, 06:09:21 AM
Experimenting with line skipping & pixel binding (1:1 FHD) with 1x3 to reduce moire and aliasing

Un-stretch 1856x416
(https://i.ibb.co/g6H06ZY/1856x416-1x3-small.png) (https://imgbb.com/)
1856x416_1x3.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1856x416_1x3.png)

Here the stretch version 1856x1248
(https://i.ibb.co/JnKqH8F/M20-1741-000000-small.png) (https://imgbb.com/)
M20-1741_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M20-1741_000000.png)
M20-1741_000000.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M20-1741_000000.dng)

setting
(https://i.ibb.co/Wp7jcmf/VRAM41-small-a.png) (https://imgbb.com/)

Short 2Mb h264 clip - M20-1741.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M20-1741.mp4)

I don't really see this to be too useful , as it's too narrow (416 h) , if the vertical was 2400 etc.. then maybe , but record time would only be 1-2 seconds
if your lucky , so unless something changes , I won't be continuing with this 1x3 .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 21, 2018, 07:29:28 AM
@aprofiti I'll try explain the basics here ,
1st: need to have a redirect raw buffer , You are in luck  :) because 50D along with the 5D2 now have CONFIG_EDMAC_RAW_SLURP/CONFIG_EDMAC_RAW_PATCH (https://builds.magiclantern.fm/experiments.html)
Why is this important , it all starts with extending the EDmac to have a large raw buffer e.g. =>full res . (10-12bit doesn't hurt either)
read me (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205614#msg205614)
The import reg's are:
CMOS[1] = Sensor Vertical (vertical start/stop)
CMOS[2] = Sensor Horizontal (horizontal pos & binning mode)
C0f06008 = FPS timer A (1 unit = 4 pixels horizontally)
C0f06014 = FPS timer B (1 unit = 1 pixel vertically)
C0F06084 = start row/column
C0F06088 = stop row/column (column: 1 unit = 2 pixels)


Rule of thumb , slow down timer "A" to increase horizontally resolution (start around 8-12fps)
All extended resolutions are in 3x Crop_mode (5x zoom) that the staring point
Then increase Timer "B" to bring back frame rate to 24 etc. ....
c0f06088 adjusts the raw image size , here a hex to decimal converter is real handy .

5d2 FHD default raw size
4f40432

first 3 number "4f4"= vertical resolution
last 3 number "432"=horizontal resolution x2

CMOS[2] 0x10e (for 5d2 crop_mode) horizontal offset up to 3520 .
0xe => up to full width.

Now how do get this to work & extend the resolution ,
Start with the 10-12bit build on the experimental page 50D.109/magiclantern-raw_video_10bit_12bit.2018Oct10.50D109.zip (https://builds.magiclantern.fm/jenkins/job/raw_video_10bit_12bit/51/artifact/platform/50D.109/magiclantern-raw_video_10bit_12bit.2018Oct10.50D109.zip)
then go to modules (https://builds.magiclantern.fm/modules.html)  download page and get adtg_gui/adtg_gui.mo (https://builds.magiclantern.fm/jenkins/job/iso-research/14/artifact/modules/adtg_gui/adtg_gui.mo)  , That's the one I being using of late .

Once you got it all loaded up , you need to activate the ENGIO registers in the advanced tab of adtg_gui ,
then you need to refresh the reg's in adtg to bring up the hidden ones , I usually go to liveview and play a h264  .mov for a few seconds
then brings up the other 900 other registers . Please note this is all experiences from my 5d2 so 50d may have some differences .
Be ready for hundreds of battery pulls  :P
I hope I didn't confuse you , or send you down any rabbit holes .
Make sure you pay attention to any a1ex's post on this thread , actually anything that a1ex posts is all realavent to all cams , though there may be small
little differences , but in the big picture (no pun in tented) all the camera work the same .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 24, 2018, 12:33:00 AM
@aprofiti , Have try it this yet ? I'm courous to see it this work with the 10-12bit build from experimental download page .
If not let me know and I can build it on my 4k 5d2 d4 branch , you may need some of the code from there.
I locally updated waza57 source to include the new raw_slurp patch & redirect  buffer.
You can also download my source code Crop_Rec-4k-5d2-magic-lantern_redercit_buffer_a1ex-code-10-22-2018 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Crop_Rec-4k-5d2-magic-lantern_redercit_buffer_a1ex-code-10-22-2018.zip) ,
but you need to add the 50d raw_slurp & redirect buffer code I haven't updated it yet for 50d.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 24, 2018, 06:01:26 AM
Looking like I got a usable 48fps @ 1888x704 (1888x1174)
The black level is off 10bit (127-1013) usually it's (112-1013) but I can clean it up in MLV App 1.3v windows
Have a little issue with MLV App 1.3v -- Hi-Lights ,when pushed turn pinkish
If I set the black level to 78-80 , clean up really nice , I would say almost perfect , I may have too
code in a black level fix , but still a little more work -- need to adjust preview reg's to un-freeze liveview while record raw .

(https://i.ibb.co/6mfm6XF/VRAM42-small.png) (https://imgbb.com/)


(https://i.ibb.co/6ZzCgby/1880x1174-48fps.png) (https://imgbb.com/)
Frame grab from the h264 from MLV App 1.3v win.

Short h264 clip from mlv app 1.3v win
1888x1174_48fps-M23-2150.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1888x1174_48fps-M23-2150.mp4)
1880x704_48fps-M23-2150_000000.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1880x704_48fps-M23-2150_000000.dng)

If any one wants to play with this 48 fps raw clip I loaded it on my google drive , 487 MB
shot @ 800 ISO , shutter 1/59th of second
1880x704_48fps_M23-2150.MLV (https://drive.google.com/file/d/1u-bmXVgluiMQIA__VEgs2mUXVhE60rUw/view?usp=sharing)
Best to use MLV App , set black level to around 78-80
:)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on December 24, 2018, 10:48:06 AM
Quote from: reddeercity on December 24, 2018, 12:33:00 AM
@aprofiti , Have try it this yet ? I'm courous to see it this work with the 10-12bit build from experimental download page .
If not let me know and I can build it on my 4k 5d2 d4 branch , you may need some of the code from there.
No, not exactly... I tried only to load your linked adtg_gui on top of 10-12bit build, but wasn't working due to linker unable to reference function; probably it's because wasn't compiled for that branch, so I have to retry when I have some spare time.

Crop_rec_4k build could be useful to work with, let me know if you add support for 50D

Edit: Downloaded your source code and applied CONFIG_EDMAC_RAW_PATCH support for 50D. Put in 5x mode and trying to understand what regs are related to resolution.
If I change timer A, image will be altered as expected, but if check resolution from movie tab it doesn't change, even by modifing CMOS regs
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on December 24, 2018, 07:42:32 PM
Quote from: reddeercity on December 24, 2018, 06:01:26 AM
(https://i.ibb.co/6ZzCgby/1880x1174-48fps.png) (https://imgbb.com/)
Frame grab from the h264 from MLV App 1.3v win.

Short h264 clip from mlv app 1.3v win
1888x1174_48fps-M23-2150.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1888x1174_48fps-M23-2150.mp4)
1880x704_48fps-M23-2150_000000.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1880x704_48fps-M23-2150_000000.dng)


Wow that's a lot of pink moire!!! do you get less with MLVProducer?

What happens if you choose a debayer algorithm other than AMaZE?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 25, 2018, 05:29:53 AM
Quote from: Ilia3101 on December 24, 2018, 07:42:32 PM
Wow that's a lot of pink moire!!!
Yea thought so , but remember it's vertically compressed & 5d2 as you know have pixel binding & line skipping
and by compressing it I bet is making the pixel binding worse , I good case for you guys over on the MLVApp1.4 thread
to analyzes and make better algorithm .
Quote from: Ilia3101 on December 24, 2018, 07:42:32 PM
What happens if you choose a debayer algorithm other than AMaZE?
It's worse , AMaZE give the best image
Quote from: Ilia3101 on December 24, 2018, 07:42:32 PM
do you get less with MLVProducer??
I did a test to see , there a Alias Filter & False Color Filter .
I let the false color filter enabled and toggled off and on the alias filter , mlvproducer has his own
debayer algorithm "MLVPDM" .

MLVProducer-Alias-Filter & False-Colors-Filter enabled
(https://i.ibb.co/Mfn7VMY/MLVProd-Yes-Alias-Filter-Yes-False-Colors-Filter-1880x704-48fps-M23-2150-00001-small.png) (https://imgbb.com/)

mlvprod_alias+falsecolor filter_1888x1174_48fps.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/MLVProd-Yes_Alias%20Filter_Yes%20False%20Colors_Filter-1880x704_48fps_M23-2150_00001.png)
h264 clip -- mlvprod_alias+falsecolor filter_1888x1174_48fps.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/MLVProduc-Yes_Alias%20Filter_Yes%20False%20Colors_Filter-1880x704_48fps_M23-2150.mov)

MLVProducer-No Alias Filter & Yes False Colors Filter
(https://i.ibb.co/dQpdkR7/MLVProd-No-Alias-Filter-Yes-False-Colors-Filter-1880x704-48fps-M23-2150-00001-small.png) (https://imgbb.com/)

mlvprod_no_alias_filter_yes_false_color_filter.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/MLVProd-No%20Alias%20Filter_Yes%20False%20Colors_Filter-1880x704_48fps_M23-2150_00001.png)
h264 clip mlvprod_no_alias_filter_yes_false_color_filter_1888x1174_48fps.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/MLVProd-No%20Alias%20Filter_Yes%20False%20Colors_Filter_1888x704_48fps_M23-2150.mov)

Don't judge by the small images I posted , there down sized to 16% of the original
I think the it better , what do you guy think ?
Maybe mlvapp 1.4v is better with this , didn't try it yet .

Merry Xmas every one  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 26, 2018, 01:35:46 AM
Quote from: aprofiti on December 24, 2018, 10:48:06 AM
No, not exactly...
Edit: Downloaded your source code and applied CONFIG_EDMAC_RAW_PATCH support for 50D.
Put in 5x mode and trying to understand what regs are related to resolution.
If I change timer A, image will be altered as expected,
but if check resolution from movie tab it doesn't change, even by modifing CMOS regs
Sorry I may have forgot to mention that c0f06088
Is the number 1 reg you need to be concern about mostly , it's what set the Raw Resolution
the rest reg's adjust image , e.g. cmos[1] centers the image etc. ...

Can you post a screen shot of the 50d's "capturing Reg's in agtd_gui ?
I'm mostly interested in c0f06088 values , let start there .

Just as a refresher ,
this is what I'm looking for (in the show capture reg's)
but with the 50D valves as this from my 5D2 in default FHD 1:1 29.97fps

(https://i.ibb.co/J2FQKM9/VRAM43-small.png) (https://imgbb.com/)
(https://i.ibb.co/ZGsdZ56/VRAM44-small.png) (https://imgbb.com/)

If there are not showing up ,
make sure you enable "ENGIO Registers" in the advanced tab
refresh liveview ( play a h264 .mov) should see  around 900 reg's
(https://i.ibb.co/WphWCZV/VRAM46-crop-small.png) (https://imgbb.com/)

And when this successful you should see Memory Patches like this , I have 7
(https://i.ibb.co/TtYpV9m/VRAM45-small.png) (https://imgbb.com/)

does this help ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on December 26, 2018, 10:26:21 PM
Yesterday I tried to understand what happen if I poke some regs, this is what I get in 5x mode with default settings:

(https://i.ibb.co/h86ywWx/VRAM25.jpg) (https://ibb.co/h86ywWx) (https://i.ibb.co/fYLmFq6/VRAM21.jpg) (https://ibb.co/fYLmFq6) (https://i.ibb.co/hmTJsFq/VRAM22.jpg) (https://ibb.co/hmTJsFq) (https://i.ibb.co/k8j1B5P/VRAM23.jpg) (https://ibb.co/k8j1B5P) (https://i.ibb.co/NSgLJMr/VRAM26.jpg) (https://ibb.co/NSgLJMr) (https://i.ibb.co/tMNJrRB/VRAM24.jpg) (https://ibb.co/tMNJrRB)

Let's compare behaviour of 50d with 5d2:

If I increase Timer A it will lower FPS, if I decrease value I get more FPS, similar behaviour with TIMER B.

If I try to change image size using c0f06088, it allow me to increase by 1 (value => 0x4530452) and get 1080 vertical but it get upper bounded; tried to lower FPS value (10.572) changing TIMER A => 0x27b070b but no increase in resolution.
Instead if I try to change horizontal resolution (increasing or decreasing) I get distorted pink images...

I saw you modified also other regs (ADTG12[100c], c0f07XXX...) from the one you listed a couple of post above.
How did you find they were necessary? Did you randomly tried register or did you observed what register were changed by canon firmware using adtg_gui?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 27, 2018, 07:43:23 AM
Thanks for the screen shots , but something when wrong I was looking for just the capture size reg's

(https://i.ibb.co/Bw3bdwm/VRAM49-small.png) (https://imgbb.com/)
(https://i.ibb.co/R9M8q82/VRAM47-small.png) (https://imgbb.com/)

You should see the same list of reg's I posted not just cmos reg's  , that tells me either
didn't enabled the "ENGIO Reg's" in the advanced tab of adtg_gui.mo or you didn't refresh liveview
e.g. load a h264 of review a photo , this a must or nothing will work and you will not get access to the hidden reg's .

There over 900 reg's you can see when it right but if not there only around 16 or 18 reg's visible
(https://i.ibb.co/fS4KdYf/VRAM48-small.png) (https://imgbb.com/)

The other think that help's with development is the dump_image_buffer
(https://i.ibb.co/P1xpk38/VRAM50-small.png) (https://imgbb.com/)

This help to analyze the raw image for true raw size before cropping with exiftool

There was 3 reg's that I saw that where useful
c0f06088
c0f06008
c0f06014

First let understand what going on here
let look at reg c0f06088 =>0x4520452
What does this tell us ?
Will for starter , this is the raw resolution and we read this in decimal , so let spilt the number up .
452=>1106 that the vertical res & the back half is horizontal 452=> 1106x2 =>2212 -160 (crop off boarders) =>2052
that should close to the Image dump .

Next Timer A =>c0f06008 & Timer B =>c0f06014
So timer A =>0x27b27b , we only use the last 3 (27b) ,
Let do some math , 27b hex = 635 decimal x4 => 2540
So this tell us that the max horizontal resolution that Timer"A" can do , including black boarder crop off area offset etc. ...
On the 5d2 raw image dump exiftool tell me the crop off area or offset  e.g. 52 /160 (52=vertical  & 160=horizontal) in crop_mode

Timer B =>c0f06014=>0x5e5 =1509 ,  max vertical resolution
So a Image dump in crop_mode would be very useful right now to see the offset , not sure if it's the same as 5d2

So if I was you , I would start with a small horizontal increase , from 1920 -- theoretically you shouldn't have to mess with timers
unless you exceed 2540 H , thou the very first thing I would do is slow timers A & B to get 23.976fps of very close to that (24).

This what I do on 5D2 , start @ 29.97fps -- reduce timer A by only a few units (0x2270227=>0x227027b) Increase timer B (0x4ff=>0x59f)=>23.976fps
then increase vertical to 1200 (my case) then increase horizontal in very small increments to 2400 H stopping between changes to check liveview .

The best way to approach this in my opinion , is to understand what each reg does in relation to raw size & preview size.

Also Liveview will messed up or freeze something pink , that canon liveview everything has to done with ML B/W (Gray) Preview.
A half shutter button press should give clean preview plus that how I refresh liveview between resolution changes .

Sorry for the long post and If this seem too simplistic , I'm writing this not only for you @aprofiti  but for everyone else , so it can be understood by all
and get more people to develop .

Quote from: aprofiti on December 26, 2018, 10:26:21 PM
I saw you modified also other regs (ADTG12[100c], c0f07XXX...) from the one you listed a couple of post above.
How did you find they were necessary?
That was found or discovered by a1ex from what I understand
ADTG12[100c] squeezes the image vertically , that's the 720p 50/60p mode on other cams ,
yes that right 5d2 can do 720p 50/60 all the way up to 72fps (that's where I stop don't push it any more but there was still room to go)
The how I got 1888x704 @ 48fps -- unsqueezed (1.66) I get 1888x1174 @48fps .
c0f0713c & c0f007150 open up the vertical window , it a must .
Explained here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205614#msg205614) , I do belive I posted this before for you to read -- It's a1ex explaining every thing you what to know .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on December 27, 2018, 08:30:50 PM
Quote from: reddeercity on December 25, 2018, 05:29:53 AM
Yea thought so , but remember it's vertically compressed & 5d2 as you know have pixel binding & line skipping
and by compressing it I bet is making the pixel binding worse , I good case for you guys over on the MLVApp1.4 thread
to analyzes and make better algorithm .It's worse , AMaZE give the best imageI did a test to see , there a Alias Filter & False Color Filter .
I let the false color filter enabled and toggled off and on the alias filter , ...

I think the it better , what do you guy think ?
Maybe mlvapp 1.4v is better with this , didn't try it yet .

Great progress @reddeercity! 48fps on a 5D2 is great!
With MLVApp v1.4 we introduced "Hue vs. Saturation" elements. That means not, that moiree will get any better, but maybe you can make it a bit more invisible by desaturating "a small pink peak". Sure - if you have much (real) pink in the clip you're lost... ;)

Edit: whoa... not only a peak... nearly all pink tones... and still not great.
(https://i.ibb.co/XVqqCgf/Bildschirmfoto-2018-12-27-um-20-41-57.png)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 28, 2018, 04:20:10 AM
Not too bad , The 1.4v looking good , I gave it a try and compared it to MLVProducer & After Effect -- though I corrected black level
and un-squeezed in MLVApp 1.4v then exported as a Uncompressed Cdng for After Effects .

So from left to right:  After Effect CS6 , MLVProducer , MLVApp 1.4v
(https://i.ibb.co/cyQCnSS/A-E-MLVPro-MLVApp1-4v-small.png) (https://ibb.co/Twtb3xx)
A.E.-MLVPro-MLVApp1.4v.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/A.E.-MLVPro-MLVApp1.4v.png)

Here are all 3 PNG's from the above image screen shot
MLVProducer (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/MLVProd-Yes_Alias%20Filter_Yes%20False%20Colors_Filter-1880x704_48fps_M23-2150_00001.png)
AfterEffects_CS6 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/AfterEffects_CS6_1880x704_48fps_M23-2150_00000.png)
MLVApp1.4v (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1880x704_48fps_M23-2150_000000_MLVApp1.4v.png)

Here's the h264 from though 3 App's
MLVProducer_h264 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/MLVProduc-Yes_Alias%20Filter_Yes%20False%20Colors_Filter-1880x704_48fps_M23-2150.mov)
MLVApp1.4v_h264 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/MLVApp1.4v_1880x704_48fps_M23-2150.mp4)
After.Effects.CS6_h264 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/AfterEffects_CS6-1880x704_48fps_M23-2150.mp4)
A.E. rendered out the file as 50fps where the other 2 keep the correct frame rate of 48fps

I tried to grade it as a flat film/log like image across all 3 App's
A.E. seem to be the worst on the test image (lost must of the detail in the center plus data missing/holes)
but the rest of the image is killer thou .
MLVProducer seemed to retain all the detail with very good motion blur , but with pink hi-lights in the almost blowout
hi-lights  , the rest of the image looks similar to A.E. in quality , clean shadows (no noise)
MLVApp 1.4v , I used the Smooth Aliasing filter set to "Last Resort"
Lost detail in the center of the test photo image , has less pink moiré then mlvproducer but more then A.E.
but didn't loose as much detail as A.E. with better motion then A.E. but not as good as mlvproducer , thou it very close.

So I would really be happy with either MLVApp 1.4v Or MLVProducer with a small lean towards mlvProducer .
As far as A.E. is concerned , It was the least desirable image with aliasing & moiré patterns .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on December 28, 2018, 12:57:19 PM
it's not a good workflow, but for curiosity's sake, you might try rawtherapee.

the latest version is quite powerful with dual demosaicing capability.

it would be an interesting test, even just on a single frame...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on December 28, 2018, 03:49:45 PM
@reddeercity
Is it correct to say you are working a 48fps release, what will that resolution be and is it Squeeze or Unsqueezed? Also is the most stable 10/12bit the Oct10 release in MagicLantern Experimental Nightly Build?

@70MM13
RawTherapee sound interesting, I use MLV App but I'm using Camera Raw in AfterEffects the most. I just installed RawTherapee, what is the workflow you use with Magic Lantern? I'm filming a Feature Film exclusively with Magic Lantern, I want the best option to process Magic Lantern footage,  so my Question is what's you favorite stable firmware, and if you were risky and brave to film a feature with Magic Lantern what would be your workflow? Please anyone feel free to answer above questions, I'm very competent but I love to have options and keep my options open so every time I ask a question on this forum its for real productivity. Sometimes I ask question on this forum then it gets loss in the clouds and ignore. Thanks for Any Advice or Guidance
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 29, 2018, 08:00:38 AM
Can 5D2 be set to mv720p 50 or 60 fps? If so it should be possible to set 100c to 3x3 binning and give mv1080p 50 or 60 fps with a little reduced height.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 29, 2018, 08:48:56 AM
There's no real 720p 50/60 mode on 5d2 , it's come about though reg's , actually a1ex figured it out
I'm just fine tuning it ,It's really FHD with squeezed reduced height 1888x704 @48fps I can push it to 750 so it's 3/5
I can do 3x3 with hi-frame rate 72+ @ 1280x720 (raw size) not vertical squeezed.
I did the squeezed version to get full height that's centered. Never really tried to do a un-squeezed FHD at reduced height
I don't think I'll get 48fps more like 40 if that , the sensor it too slow I think for that 5d2 is only 96MP/s where the 5d3 is 192MP/s
and that's the big difference between D4 & D5 . The 5d3 has 8 channels off the sensor where the 5d2 has only 4 channels .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 29, 2018, 09:48:29 AM
So how about 10-12bit 4096x1770 @ 23.976 fps (A.R. 2.31) in 1:1 FHD sound ?
I got a clean 4096x590(squeezed 3x) version I'm thinking of add to the crop_rec
I can record 10bit continuously @ 69MB/s  :D and MLVApp can process then easily
There still a little aliasing , but not that bad you really have to push it to produce it.
Never notice any moiré pattern at all , In this mode the sensor is 1:1 pixel horizontally hence 4096
and the vertical is line skipping every third line from what I can see so 1x3.

(https://i.ibb.co/SV644HY/VRAM58-small.png) (https://imgbb.com/)(https://i.ibb.co/SfxWnwV/VRAM57-small.png) (https://imgbb.com/)

(https://i.ibb.co/jwVs6jJ/4096x1770-M29-0050-frame-1-small.png) (https://imgbb.com/)
4096x1770-M29-0050_frame.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4096x1770-M29-0050_frame_1.png)
4096x1770-M29-0050_frame_1.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4096x1770-M29-0050_frame_1.dng) un-sqeezed from mlvapp 1.4v
4104x591-RAW-033.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4104x591-RAW-033.DNG) squeezed , from the image dump
h264 clip 4096x1770-M29-0050.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4096x1770-M29-0050.mov) 70Mb

Pretty much works flawlessly , not issue at all I can see only thing I don't like is there more rolling shutter then I would like .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on December 29, 2018, 10:50:01 AM
Hell Fkin' Yeaaaaah David! :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on December 29, 2018, 02:15:36 PM
Quote from: reddeercity on December 28, 2018, 04:20:10 AM
...
So I would really be happy with either MLVApp 1.4v Or MLVProducer with a small lean towards mlvProducer .
As far as A.E. is concerned , It was the least desirable image with aliasing & moiré patterns .
I again played a bit around with some freestyle code and your 48fps MLV. The pink in on the chart looks like the green channel is clipped from the low side, not from the high side. So I implemented just for testing a stupid algorithm which reconstructs this part of the green channel. Then I switched to IGV debayer and this is what you'll get then (+ Sharpen=50 to see even more detail):
(https://i.ibb.co/xGcwtt2/short1880x704-48fps-M23-2150-frame-1.png)
The problem is: this is not at all scientific, so we'll kill some areas color with that...

Code raw_processing.c (first the old highlight reconstruction, second the "lowlight reconstruction"):
...
                /* Check if its the highest green value possible */
                if (tmp1 == processing->highest_green)
                {
                    pix[1] = (pix[0] + pix[2]) / 2;
                }
                /* reconstruct low green */
                if (tmp1 < 15000)
                {
                    if( pix[1] < 1.1*pix[0] && pix[1] < pix[2] )
                    {
                        pix[1] = (pix[0] + pix[2]) / 2;
                    }
                }
...



Quote from: reddeercity on December 29, 2018, 09:48:29 AM
So how about 10-12bit 4096x1770 @ 23.976 fps (A.R. 2.31) in 1:1 FHD sound ?
Great!  8) Wanna have! ;)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on December 29, 2018, 06:00:37 PM
simply amazing!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on December 29, 2018, 06:03:19 PM
i mean simply amazing - for 4096x1770 @ 23.976 fps....
just amazing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 31, 2018, 03:05:01 AM
Thanks , did a short clip test in a real world situation (outdoors , shot a skyline @ sunset) .
Didn't like the artifact and saw some aliasing that look not good .
Timers need to be reworked to reduce artifacts before I could use it .
So what did I do to get this , I basically stop the vertical columns (every third one) CMOS[1] 0x40=>0xe
if you do the math 5632/3=1877.333 so round it off the same
as the silent picture or the liveview raw image dump 1880x1248 . By increasing the horizontal to 4096 I still have line skipping
vertically (every 3rd line) , but if this was not so I wouldn't be able to record 4k squeezed . So we can call this 4K with trickery
Or 4K line skipping vertically , need to think able this some more .

On the other hand seeing how far I can push Hi-Frame rate squeezed -1280x432 (1280x720 un-squeezed).
Seems I can get to 83.815fps pretty cleanly , thou I can push it to 85.881fps but the image break ups badly , (image moves sideways) 
More then likely if I reduce horizontal from 1280 to less then 1000 I'm sure to get more but this doesn't interest me below 1280x720
So I should be able to give the 5d2 82.5fps safely up to 83.88fps   



Edit: Update -- I push the 4k compressed vertical to 29.970 FPS ,
So that's 4096x590 @29.97 (Un-compressed 4096x1776 @ 29.97 fps)
(https://i.ibb.co/pnv8vPG/VRAM0-4096x591-29-97fps-small.png) (https://imgbb.com/)

Unfortunately it's not continuous at 10bit 29.97fps , I can get around 5000 frames (2.5min) where at 23.976 is continuous .
So that's a plus  :) , it's making this more attractive (even with it's flaws) a little slow-mo in 4k even if it's line skipping
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 31, 2018, 05:20:18 PM
Got a 4k squeezed preset working for eosm. Good work RDC:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg209967#msg209967
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 31, 2018, 07:48:22 PM
Reddeercity, you do amazing things... look, after I wrote you about the zoom (magnification) jumping out automaticaly, it stopped to do so. Now it works properly. I actually don't know what happened (mystic). So, everything is fine now, but it have been jumping out for almost 7 years till that moment... hm.... So if it happen again I'll show you. Thanks
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 01, 2019, 02:12:31 AM
Quote from: reddeercity on December 29, 2018, 08:48:56 AM
I can do 3x3 with hi-frame rate 72+ @ 1280x720 (raw size) not vertical squeezed.

You can do 1280x720 unsqueezed, but can you not increase horizontal resolution? I think it's worth it even if vertical resolution or framerate has to be reduced. An 1880x700 48fps preset would be really good.

Also does anyone else with a 5D2 find that when shooting in 2880x1080 the rolling shutter is really extreme?

It is not possible to get usable footage handheld at all. I'm asking because my camera is together by only one screw after repair, so it may just be that.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 01, 2019, 06:24:17 AM
I check in the Frame over ride menu in the advance tab and compared to the other mode ,
Normal crop_mode at 23.976 & 30 plus in FHD at 23.976 (frame override)

2880x1080 crop preset  33.5 -- rolling shutter
(https://i.ibb.co/SBg68ys/VRAM17-small-crop-preset3-5k.png) (https://imgbb.com/)

Normal 3x crop_mode 2144x1074 @ 23.976 frame override -- 25.6 rolling shutter
(https://i.ibb.co/MV7jXwC/VRAM16-small-crop-mode-24-over-ride.png) (https://imgbb.com/)

FHD 1:1 1856x1044 @ 23.976 frame override -- 23.8 rolling shutter (best)
(https://i.ibb.co/TmnRGzq/VRAM15-small-FHD-24-over-ride.png) (https://imgbb.com/)

So yes this more rolling shutter ,
I've being trying to reduce this in normal FHD (1856x1044) I have rolling shutter down to 22.8
Looking to be around 15 but sensor too slow I bet .

FHD 1:1 1856x1044 Low rolling shutter
(https://i.ibb.co/pyZc5Sr/VRAM52-small-FHD-22-8-rolling-shutter.png) (https://imgbb.com/)

Have you try a higher shutter speed  e.g. 1/60th , or  1/90 this should help a little (never tried)
the reason I say 1/60th because timer A is set to 31fps  and timer B just slows it down to 23.976
This is needed to extend the resolution , even in 4k line skipping  (4096x590) to get 23.976 timer A
was pushed pretty high (58 fps) then timer B reduce it to 23.976 .

Unless the timer over head can be reduced (remember a1ex did some test) (https://www.magiclantern.fm/forum/index.php?topic=23040.msg208479#msg208479) he was able to reduce more rolling shutter and here (https://www.magiclantern.fm/forum/index.php?topic=19300.msg207723#msg207723) too
I think on the 5d3 the rolling shutter is around 18 I think .
So I hope to have less rolling shutter , but that work in process .

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 01, 2019, 08:37:12 AM
Pushing to full width (5632) with the vertical compressed so 5632x416 @ 23.99fps = 5632x1248 @23.99fps
Works good , but the full width I can only get 1248 vertical which give an A.R. of 4.5:1  :o
If you are wondering why I can't get a higher vertical resolution , from what I can see the width is
the main factor , as I reduce the horizontal raw size e.g. 4096 I get increased vertical e.g. 1776 etc. ...
So the width & height is in direct proportional to it's self , if I was in 3x crop_mode I wouldn't have this limit .

(https://i.ibb.co/xf7q2gW/M01-0022-frame-224-small.png) (https://imgbb.com/)
M01-0022_frame_224.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M01-0022_frame_224.png)
5632x1248_23.99fps_M01-0024.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5632x1248_23.99fps_M01-0024.mp4)

(https://i.ibb.co/9sWGF2h/VRAM2-small-5632x416.png) (https://imgbb.com/)
(https://i.ibb.co/qFFg3VL/VRAM1-small-full-width.png) (https://imgbb.com/)  (https://i.ibb.co/FBBL5rq/VRAM3-small-frame-override-full-width-24.png) (https://imgbb.com/)

It's just tooooo narrow , I would have to get at least 1877 (3:1 A.R.) or 2440 (2.30:1 A.R.) vertical .
So I'll just keep 4096x590 (4096x1776) and develop it further  That seems to be the Sweet Spot for 5d2  :))

Happy NewYear everyone !!
It should be only a few more weeks before my crop_rec module is ready for testing .

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on January 01, 2019, 10:03:21 AM
Quote from: reddeercity on January 01, 2019, 06:24:17 AM
Unless the timer over head can be reduced (remember a1ex did some test) (https://www.magiclantern.fm/forum/index.php?topic=23040.msg208479#msg208479) he was able to reduce more rolling shutter and here (https://www.magiclantern.fm/forum/index.php?topic=19300.msg207723#msg207723) too
I think on the 5d3 the rolling shutter is around 18 I think .
So I hope to have less rolling shutter , but that work in process .

In 1080p 3x3, the minimum achievable rolling shutter would be...

Quote from: a1ex on November 22, 2018, 08:16:08 AM
the minimum timer A value in 1080p is 554

... 554 / 24 MHz = 23.08 μs/line.

You can already get this with FPS override, by manually changing timer A manually to 554 (in the Advanced menu).

Values without FPS override (i.e. with stock Canon firmware):
- 1080p24/30: 576 / 24 MHz = 23.8 μs/line
- 1080p25: 600 / 24 MHz = 25 μs/line

Theoretical speed: 5D2 reads out 4 columns at a time (i.e. 4 analog channels), with a pixel clock of 24 MHz; that would give 96 MPix/second. You just can't get more than that without overclocking (no idea whether this frequency is adjustable by software or not).

That means, the absolute minimum, assuming one can find a way to somehow get rid of the horizontal overheads (which is likely impossible), would be 1880 / 4 / 24 MHz = 19.58 μs/line. Keeping the left OB area (C0F06084: 0x10036,  C0F06088: 0x4f40432) would require 0x432 * 2 / 4 / 24 MHz = 22.38 μs/line.

The remaining (unexplained) overhead is a whopping 23.08 - 22.38 = 0.7 μs/line. Even if that can be reduced somehow, I'm pretty sure it's not worth the effort. You simply won't get significant rolling shutter improvements without overclocking the sensor (or without reducing the horizontal resolution).

1100D, on the other hand, is an interesting beast:

Quote from: a1ex on November 27, 2018, 09:09:31 AM
720p25: A = 1000, B = 1280, readout size 1496 x 967 (this includes black borders)
720p30: A = 960, B = 1112, same readout size
Timer A can be pushed to 872 in both modes.

It runs at 32 MHz x 4 channels (i.e. pixel clock is the same 700D & co.), but for some reason, it has a huge overhead. In regular (24/25/30p) movie mode, C0F06084/88 is set to 0x200c3/0x3c903af, i.e. xmax = 1886 (likely rounded to 1888), out of which 390 pixels are cropped from the left side. ML still has to crop 68 pixels of left OB; that gives 1428 active horizontal pixels, 458 (!) for OB width and 2 pixels for rounding (guess); these require "only" 14.75 μs/line. The unexplained overhead would be 12.5 μs/line, or exactly 400 clock ticks. This one is worth fine-tuning, if you ask me.

Why?
- 720p24/30: 960 / 32 = 30 μs/line
- 720p25: 1000 / 32 = 31.25 μs/line
- FPS override: 872 / 32 = 27.25 μs/line
- theoretical value: (872 - 400) / 32 = 14.75 μs/line! (without altering OB areas)

For a 16:9 frame, which would be just 1428x804, it would require only 11.86 ms!

Assuming one can somehow drop 400 pixels from the left OB, that would free another 100 clock ticks, i.e. 11.63 μs/line, or 9.35 ms for a 16:9 LiveView frame.

5D3 also operates at 24 MHz, but has 8 channels. In 1080p 3x3, timer A values are (see here (https://www.magiclantern.fm/forum/index.php?topic=21379.0)):

- 24/30p: 440
- 25p: 480
- minimum value: 398

That gives 18.3 and 20 μs/line with Canon firmware, and 16.58 μs/line with FPS override.

C0F06800/C0F06804 are set to 0x10017/0x528011b; that would require 0x11b * 8 / 8 / 24 MHz = 11.8 μs/line (unexplained overhead: 4.8 μs/line, i.e. 115 clock ticks). Optical black area: Canon crops away 0x17 * 8 = 184 pixels, ML skips 146 + 2, active area width is 1932, total 2264 = 0x11b * 8.

That way, should we find out how to remove that overhead, a 16:9 frame (1920x1080) would require only 12.75 ms.

If we could also get rid of 300 OB pixels (out of 332), that would cut about 1.56 microseconds per line.

Of course, these are just theoretical values; I don't know how to reduce these overheads, I just tried to estimate whether it's worth trying to figure them out, or not. On 5D2, the potential improvement would be very small (the sensor is already used at close to its full speed in LiveView). On 5D3 and 1100D... it's worth taking a closer look.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 02, 2019, 05:32:54 AM
Thanks, much clearer now I'm starting understand this . Yes I see where the 5d3 would be good there.
Now I thinking about 5D4 since this  (https://www.magiclantern.fm/forum/index.php?topic=17695.msg209985#msg209985) happened  :D
I'm keeping my eye's open for a good deal on one , starting to see a few used ones out for sale now .
After I'm done with the 5d2 development , I looking forward to the 5d4.

Now about the 5d2 , I got a new preset UHD Vertical Compressed 3840x624 @ 23.976 (3840x1872 2.0:1 A.R.)
I'm likening this a little more then 4k for the better vertical height , it's a hard to get continuous @ 10bit (more height)
I will keep both presets (3840 & 4096) , the rest will be from the 3x crop mode .

UHD 3840x1872
(https://i.ibb.co/dmNMwMZ/M01-2117-frame-170-3840x1872-small.png) (https://imgbb.com/)

3848x625_RAW-040.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3848x625_RAW-040.png)
M01-2117_frame_170_3840x1872_.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M01-2117_frame_170_3840x1872_.png)
3840x1872_23.976_400ISO_M01-2117.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3840x1872_23.976_400ISO_M01-2117.mp4)
3840x1872_23.976_800ISO_M01-2122.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3840x1872_23.976_800ISO_M01-2122.mp4)

(https://i.ibb.co/zHbGRcc/VRAM5-small.png) (https://imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on January 03, 2019, 03:11:56 PM
Can't wait... =))

"After I'm done with the 5d2 development , I looking forward to the 5d4."
This also sounds inspiring.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on January 03, 2019, 11:56:03 PM
Quote from: reddeercity on December 27, 2018, 07:43:23 AM
You should see the same list of reg's I posted not just cmos reg's  , that tells me either
didn't enabled the "ENGIO Reg's" in the advanced tab of adtg_gui.mo or you didn't refresh liveview
e.g. load a h264 of review a photo , this a must or nothing will work and you will not get access to the hidden reg's
Sorry, forgot to say that 50D doesn't have movie playback.
Already enabled "ENGIO Regs" and apparently I can find all relevant regs including adtg12[100c] that were mentioned here, just need to switch category to avoid headache after switching to LV end 3X mode :)

Anyway after some days of swpping batteries (damn LV drawn them quickly...) and trying to match full width, I'm still stuck on solving some problems:
1. There is a pink bar at the bottom of the frame when I increase reg 0xc0f06088 (more resolution, more pink area), no matther how TImer A is modified
2. Increasing width will make room for rigth side of framebut will result also in black bar... In other words image is not centered
    Expected to fix with CMOS[2] but no apparently usefull (Need to redo some testing about) and with 0xc0f06084 (small diff from default appears so leave room for left data but then increasing more will only result in moving image to rigth)
3. a couple of gray/green vertical line in the second half of the frame

(https://i.ibb.co/g60K3Pd/RAW-072.jpg) (https://ibb.co/g60K3Pd) (https://i.ibb.co/LkLDbQ1/RAW-073.jpg) (https://ibb.co/LkLDbQ1)

First DNG (https://drive.google.com/file/d/1N2xFoEHiVTGkQUSBl7NXSth6CjO37mrd/view?usp=sharing) is default ML preset, second DNG (https://drive.google.com/file/d/12_yolOKlVtt_OWxzoDFNj1nh3QzQU0bo/view?usp=sharing) is attempt of full width:

0xc0f06008 : 0x27b04f5 (from 0x27b27b)
0xc0f06084 : 0x1004b (from 0x1004a)
0xc0f06088 : 0x45209bb (from 0x450452)


Not completely sure about photo mode register's values for FUll Resolution LiveView:

0xc0f06008 : 0x4f504f5
0xc0f06014 : 0xc9c
0xc0f06084 : 0x1004b (from default 1004a)
0xc0f06088 : 0xc9d09bb

They were retrived first time taking FRSP and watch previus values of regs in adtg_gui and then after taking a regular picture while not in LV.
Timing values and resolution appears more higher than needed, is this normal?

What is the relation between Timer A/B regarding FPS and rolling shutter?
I read somewhere on forum one time and can't find anymore (Also still need to finish read half of this thread)

Here is a spreadsheet (https://docs.google.com/spreadsheets/d/1pXZGJov8V-5K-ESiRTn5Z7zgydLhvNhUMqLdLMpEKVQ/edit?usp=sharing) where I'm trying to summarize what tested and trying not to forget their outcomes
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 04, 2019, 12:51:35 AM
Great Job !!
That expected , did you try and rec a .mlv file ? Looks like a image dump
more then likely the pink lower bar with not be in the raw video
It was the same for the 5D2 , I had the same thing on full width (5632) & 4k (4096)
https://www.magiclantern.fm/forum/index.php?topic=19336.msg207777#msg207777


CMOS[2] should center it , check photo mode and see what it is.
On the 5d2 CMOS[2]0xe centers for full width (5632) and in
default 3xcrop_mode CMOS[2]0x10e centers it (2144) but in 4096(4k)
CMOS[2]0x6e centers it , I check to see by refreshing liveview with the half shutter button 
Seems the image is always right justified , and it can only be moved to the left from what I seen on 5d2 .

That all for now , I'll answer the rest of your questions later tonight
in the middle of some 4k line skipping experiments in 5x zoom (3xcrop_mode) . :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on January 05, 2019, 10:31:08 AM
Hi Reddeercity,

The squeezed mode is interesting. I'm thinking about use it in fullhd+ like (1920 x 360 up to 2400 x 432) at a high frame rate like 72 to optimise aliasing.
With an offset of 1 line in height every 1/72 second, we can reconstruct with dual-iso a good fullhd+ (1920x1080 up to 2400x1296 fps24 ) ?
I know we need to adapt mlv_dump or mlv_app to take in account the height shift but do you think it's doable in crop.c ?
Here is a schema to explain my thoughts :

(https://i.ibb.co/vVNqCfp/schema-ml.jpg) (https://ibb.co/vVNqCfp)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on January 05, 2019, 10:35:34 AM
Height shifts and metadata area already under control through metadata and taken care of in Mlv App. You reach the metadata parts in crop_rec.c in here:
static unsigned int raw_info_update_cbr(unsigned int unused)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on January 05, 2019, 10:59:34 AM
Whaou , thanks Danne for the quick and precise answer !
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on January 05, 2019, 11:11:53 AM
Maybe I'm missing the point, but I don't see any connection between the previous 3 replies. Metadata has nothing to do with benoit's diagram; maybe Danne's reply was meant for some other post?

One has to control the phase of the line skipping sequence (i.e. from a 3-line group, which line gets captured and which two lines get skipped). This is not under our control at the time of writing. If it is controllable by software, it would be from one of the ADTG registers.

To find it, one requires a way to change ADTG registers without moving the camera (such as, some sort of USB remote interface, or some register brute-forcing script running unattended). Why? You would have to identify an image shift of exactly 1/3 pixels vertically (after the line skipping).




In the mean time, the easiest way to use all physical pixels on the sensor (including 5D2) is to read every single line, and bin every 3 columns (https://www.magiclantern.fm/forum/index.php?topic=16516.msg209999#msg209999) (that is, full height 1x3 readout).

Simulation to show the difference between 3x1 (with line skipping) and 1x3 (with column binning):
Quote from: a1ex on January 01, 2019, 05:31:20 PM
binning-modes.html (https://a1ex.magiclantern.fm/bleeding-edge/lv-binning/binning-modes.html) (18.5 MB)

However, recovering the full resolution from a 1x3 readout is not exactly straightforward.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on January 05, 2019, 12:13:17 PM
Yes, 1x3 will be the way so I assume 5D mark III and maybe 700D would be the better start off cams to work on as they can can expand height with less restrictions than eosm and 100D for instance.
My metadata reply was simply referring to info passed into mlv through crop_rec.c which Mlv app follows.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on January 05, 2019, 04:12:19 PM
Quote from: reddeercity on January 04, 2019, 12:51:35 AM
CMOS[2] should center it , check photo mode and see what it is.
I check to see by refreshing liveview with the half shutter button 
Seems the image is always right justified , and it can only be moved to the left from what I seen on 5d2 .
Found CMOS[4] is responsible for horizontal position on 50D, that's why i wasn't noticing any change in value while checking cmos[2]

Strange thing: after poking around cmos registers set to found what said above, for uknow reason now I get automatic centered image while entering liveview and setting full width:

Previuous automatic value:
CMOS[4] = 484
Centered Full Wide:
CMOS[4] = 404
Another Value fond early while logging photo mode, but not sure about
CMOS[4] = 40c


Quote from: reddeercity on January 04, 2019, 12:51:35 AM
That expected , did you try and rec a .mlv file ? Looks like a image dump
more then likely the pink lower bar with not be in the raw video
It was the same for the 5D2
First attempt result in a standard 2000x1079 mlv video....
Discovered that to increase image size while recording, I have to manually increase resolution in movie Tab -> raw video

Max available is 3520 which give a almost good video of 3520x1078 (a couple of vertical dark lines as wrote defore) from a dump of 4768x1078

Is this how is suppose to do or I'm missing something?
@reddeercity Mlv_lite and crop_rec doesn't compile for 50D in Crop_Rec-4k-5d2-magic-lantern_redercit_buffer_a1ex-code-10-22-2018 (//http://)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 06, 2019, 12:37:16 AM
Quote from: benoit on January 05, 2019, 10:31:08 AM
Hi Reddeercity,
The squeezed mode is interesting. I'm thinking about use it in fullhd+ like (1920 x 360 up to 2400 x 432) at a high frame rate like 72 to optimise aliasing.
With an offset of 1 line in height every 1/72 second, we can reconstruct with dual-iso a good fullhd+ (1920x1080 up to 2400x1296 fps24 ) ?
I know we need to adapt mlv_dump or mlv_app to take in account the height shift but do you think it's doable in crop.c ?
Here is a schema to explain my thoughts :
Interesting idea ,
but 1280x434 is the max resolution I can go at Hi-Frame rate (https://www.magiclantern.fm/forum/index.php?topic=19336.msg209935#msg209935) e.g. 72 (FYI - max frame rate to date is 82.5fps @ 1280x722 un-squeezed)

This the Hi-Frame rate limitations squeezed :
1888x704 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg209658#msg209658) @ 48fps
1600x500 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg208955#msg208955) @ 62.5fps
1280x434 @ 82.5fps


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 06, 2019, 01:56:59 AM
Quote from: aprofiti on January 05, 2019, 04:12:19 PM
Found CMOS[4] is responsible for horizontal position on 50D ......
Strange thing: .... now I get automatic centered image while entering liveview and setting full width:

Previuous automatic value:
CMOS[4] = 484
Centered Full Wide:
CMOS[4] = 404
Another Value fond early while logging photo mode, but not sure about
CMOS[4] = 40c

Great , good to know it may help other d4 cams
Quote from: aprofiti on January 05, 2019, 04:12:19 PM
First attempt .... mlv video....
Discovered that to increase image size ...... increase resolution in movie Tab -> raw video
Max available is 3520 which give a almost good video of 3520x1078 (a couple of vertical dark lines as wrote defore) from a dump of 4768x1078
Is this how is suppose to do or I'm missing something?
No,  I forgot to add the extended resolutions to the source (3008, 3840,4096 & 5632) -- sorry
Add that to mlv_rec.c @ line 136 e.g. like this
static uint32_t resolution_presets_x[] = {  640,  960,  1280,  1600,  1920,  2240,  2560,  2880, 3008,  3200,  3520, 3840, 4096, 5632 };
#define  RESOLUTION_CHOICES_X CHOICES(     "640","960","1280","1600","1920","2240","2560","2880","3008","3200","3520",3840","4096","5632")

Even though your 50d can't  do 5632 it will go to the  next max. available size to you
Quote from: aprofiti on January 05, 2019, 04:12:19 PM
@reddeercity Mlv_lite and crop_rec doesn't compile for 50D in Crop_Rec-4k-5d2-magic-lantern_redercit_buffer_a1ex-code-10-22-2018 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Crop_Rec-4k-5d2-magic-lantern_redercit_buffer_a1ex-code-10-22-2018.zip)
I know mlv_lite didn't but wasn't sure about crop_rec  , I meaning to look in to it .
I can't even remember what error there where for mlv_lite , I need sound & HDMI support so I just kept with mlv_rec .
mlv_lite doesn't work very well with HDMI and there no sound surport in the 4K branch yet for 5D2 in mlv_lite
but I guess for the 50d there's no sound recording so you my have too look in to it your self , I my not get to it for a long time.

It took waza57 some time to get the crop_rec to work with the 5d2 , I would need to see the errors to understand
why it's not compiling .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 06, 2019, 02:09:03 AM
Quote from: reddeercity on January 06, 2019, 12:37:16 AM
1888x704 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg209658#msg209658) @ 48fps
Very nice!!! (edit: before I somehow missed that you got that resolution despite looking at the images 🤦)

Is it possible to disable the squeeze and have that same resolution without it?

Sorry if my spam is getting annoying, but is it possible to get higher vertical than the 2880x1080 preset if you lower the fps? Something like 2880x1200 at 21fps
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 06, 2019, 06:28:10 AM
Quote from: Ilia3101 on January 06, 2019, 02:09:03 AM
Is it possible to disable the squeeze and have that same resolution without it?
No, at 48fps the max vertical is 704 , maybe 750
The nice think about the 5D2  with 3x5 , there no more aliasing and moire then you see in 3x3
plus at 48fps the frames are cleaner I think , good enough for smooth slow-mo @ 24fps
Quote from: Ilia3101 on January 06, 2019, 02:09:03 AM
......  but is it possible to get higher vertical than the 2880x1080 preset if you lower the fps? Something like 2880x1200 at 21fps
In crop_mode I'm planning on 3008x1200 @ 24fps , it's almost there got sidetrack by the 4k vertical compressed height dev.
currently I have 2712x1200 @ 23.976fps & 2560x1200 @ 23.976fps (https://www.magiclantern.fm/forum/index.php?topic=19336.msg209086#msg209086)
I'm also thinking of adding some square presets , 2144x1330 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg207515#msg207515) , 2144x1586 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg207553#msg207553) & 2144x1842(maybe) currently there are at 9.9fps but hope to get 24
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 06, 2019, 09:08:03 AM
I got (compressed vertical) 4416x1608 @ 23.98 working (A.R. 2.74:1) , would this resolution be useful ?
Or I can go to 4250x1722 @ 23.976 (A.R. 2.46:1) , I'm still keeping 3840x1872 (A.R. 2.05:1) & 4096x1776 (A.R. 2.30:1) @ 24fps
I just don't want too many presets or I can have both . **Note these are un-compressed vertical resolution**
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on January 06, 2019, 09:13:32 AM
No 2.35:1 preset?
By the way. Are you getting any real-time preview on any of your squeezed presets or are all of them 'framing' preview based?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 06, 2019, 05:19:29 PM
Quote from: reddeercity on January 06, 2019, 06:28:10 AM
No, at 48fps the max vertical is 704 , maybe 750

I didn't mean unsqueezed equivalent of 1880x704, I meant just 1880x704 but not squeezed, so basically a wide aspect ratio. 1880x704 matches pretty well to 2.67 if it were 3x3.

Would that be possible?

Quote from: reddeercity on January 06, 2019, 06:28:10 AM
The nice think about the 5D2  with 3x5 , there no more aliasing and moire then you see in 3x3

There's got to be some difference, so I would use the 3x3 just to be safe as I like wide aspect ratios anyway.


Quote from: reddeercity on January 06, 2019, 06:28:10 AM
I'm planning on 3008x1200 @ 24fps , it's almost there got sidetrack by the 4k vertical compressed height dev.
currently I have 2712x1200 @ 23.976fps & 2560x1200 @ 23.976fps (https://www.magiclantern.fm/forum/index.php?topic=19336.msg209086#msg209086)
I'm also thinking of adding some square presets , 2144x1330 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg207515#msg207515) , 2144x1586 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg207553#msg207553) & 2144x1842(maybe) currently there are at 9.9fps but hope to get 24

This is good news! I tried changing fps override timers in the 2880x1080 build in first post, also the registers crop_rec lets you change, but it seems locked to height 1080. Guess I'd need to use whatever branch you are using to actually come up with these values. The name of it is not coming to my head right now.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on January 06, 2019, 09:44:43 PM
Quote from: reddeercity on January 06, 2019, 09:08:03 AM
I got (compressed vertical) 4416x1608 @ 23.98 working (A.R. 2.74:1) , would this resolution be useful ?
Or I can go to 4250x1722 @ 23.976 (A.R. 2.46:1) , I'm still keeping 3840x1872 (A.R. 2.05:1) & 4096x1776 (A.R. 2.30:1) @ 24fps
I just don't want too many presets or I can have both . **Note these are un-compressed vertical resolution**
That all sounds so promising! The release date for all this will be a perfect day! 8)
If I think about what I use most, then this should be A.R.s between 16:9 and 2.35:1. But maybe 2.74:1 is useful for someone out there...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on January 08, 2019, 11:58:20 PM
let's send from each person one good of the place where he lives to Reddeercity? =) hehe.. That's absolutely incredible.

I absolutely agree with Masc about 16:9 and 2.35:1
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on January 09, 2019, 01:34:25 AM
I'm trying to get full heigth from vanilla 5x just like a1ex (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205622#msg205622) and reddeercity already did, but doesen't appears to extend image stright off like previous full width experiment.

I'm getting that pink bottom like this (https://www.magiclantern.fm/forum/index.php?topic=19336.msg207127#msg207127) and can't get clean image like this (https://www.magiclantern.fm/forum/index.php?topic=19336.msg207553#msg207553) by just increasing Timer B and Head timers.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 09, 2019, 04:33:30 AM
Quote from: aprofiti on January 09, 2019, 01:34:25 AM
I'm trying to get full heigth from vanilla 5x just like a1ex (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205622#msg205622) and reddeercity already did,
but doesen't appears to extend image stright off like previous full width experiment.

I'm getting that pink bottom like this (https://www.magiclantern.fm/forum/index.php?topic=19336.msg207127#msg207127) and can't get clean image like this (https://www.magiclantern.fm/forum/index.php?topic=19336.msg207553#msg207553) by just increasing Timer B and Head timers.

That's a CMOS[1] adjustment (Vertical Offset) & Head Timers c0f0713c & c0f07150 (vertical window size for liveview)
not sure if these are the same in 50d . I get this for time to time also but sometimes refreshing liveview fixes it (half shutter)
Good that you posted , I'm currently working on those square presets for 24p (2144x1586)
Got it work at 2144x1448 @ 23.976 ( I wanted some tall 4/3 preset - I like that look)
So I took that experiment (https://www.magiclantern.fm/forum/index.php?topic=19336.msg207553#msg207553)  and built around it , getting these extended height can be difficult .

Here a procedure I use to get extended height in 5x zoom(2144x1586) on 5d2 (default 2144x1074)
1-Slow Timer'B" down to 9.98 fps (C0F06014: 0xEDC)
2-Adjust Raw Height C0F06088: 0x66A04BA (1586x2144) we read it as 2144x1586
3-Adjust the Vertical Liveview window (head timers) C0F0713C from 0x476 -> 0x676  & C0F07150 from 0x49C -> 0x69C
Need to do so math for the C0F07150 head timers e.g. 0x49c=1180 vertical default for 5d2 (1074) we need 1586+52(OB offset)=1638
0x69c=1692 , C0F0713C 0x476=1142 (default) need 1586+52(OB offset)=1638  0x676=1654 more then we need but I think you get I the idea .
4-CMOS[1] Vertical Offset adjustment , 0xE6A (default)  change to 0xBOD (just need to play around cmos[1] trial & error)
5-Increase frame rate , c0f06014 0xEDC to 0x4something etc...

That's the basics , adjust slowly -- if you flip thought the adjustments too fast you can either lockup the cam or loose the image (liveview turns black)
At best most or this is just trial & error plus a few lucky breaks  :D

Edit: Are you using the 50d 10-12bit build from the experimental download page
or did you use a build compiled from my source file with the add parts (redirect buffer for extended res.) for 50d ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 09, 2019, 07:12:24 AM
Still adjust different presets , trying to get more height in the 2k range
Figured out 2400x1330 @ 23.976fps real time color preview (same as canon preview) while recording raw video
record times are short , about 20 seconds at the moment (I haven't starting on Over Clocking the CF card Bus yet that's next)

(https://i.ibb.co/3Mq5zKz/VRAM28-crop-small.png) (https://imgbb.com/)
(https://i.ibb.co/54W99qD/VRAM29-small-2400x1330-24fps.png) (https://imgbb.com/)

You may wonder what the last three reg's are for , they are for the liveview preview while recording instead of a frozen liveview
c0f08184 = preview height,  I set this to the raw height +52 (the OB area)  0x467(1127)=>0x56a(1386)
c0f08188 = preview width , I set this to the total raw width+(OB Area 160) 0x907(2311)=>0xa07(2567)
c0f08518 = both preview height & width 0x4670907->0x56a0a07 ( 1127x2311)->(1386x2567) and of course we read this as 2567x1386    ;)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on January 09, 2019, 11:01:50 AM
Quote from: reddeercity on January 09, 2019, 07:12:24 AM
Figured out 2400x1330 @ 23.976fps real time color preview (same as canon preview) while recording raw video

Great Reddeercity, "color realtime preview" I can't believe it ! the wimps will be glad when you release it !
Can't wait the date !  ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on January 09, 2019, 06:00:11 PM
Quote from: reddeercity on January 09, 2019, 04:33:30 AM
That's a CMOS[1] adjustment (Vertical Offset) & Head Timers c0f0713c & c0f07150 (vertical window size for liveview)
not sure if these are the same in 50d
Not quite sure abut CMOS[1] on 50D... his default value is 0x1 and doesn't seems to change from photo mode.
Tryed to increase by little steps but no feedback from LV image.

Instead CMOS[5] looks more interesting, because it's default value is 0x628 after switching to 5x mode and increasing to 0x629 will move image up, decreasing will move image down till image get duplicated/distorted.

Unfortunately small modifications from that value doesn't cover pink are with valid data, just transalate sensor image in upper area untill it gets white...
Didn't tryed increasing/decreasing by larger value, but if this is what you get from CMOS[1] on 5d2, maybe is what I'm looking for.

Quote from: reddeercity on January 09, 2019, 04:33:30 AM
Here a procedure I use to get extended height in 5x zoom(2144x1586) on 5d2 (default 2144x1074)
1-Slow Timer'B" down to 9.98 fps (C0F06014: 0xEDC)
2-Adjust Raw Height C0F06088: 0x66A04BA (1586x2144) we read it as 2144x1586
3-Adjust the Vertical Liveview window (head timers) C0F0713C from 0x476 -> 0x676  & C0F07150 from 0x49C -> 0x69C
Need to do so math for the C0F07150 head timers e.g. 0x49c=1180 vertical default for 5d2 (1074) we need 1586+52(OB offset)=1638
0x69c=1692 , C0F0713C 0x476=1142 (default) need 1586+52(OB offset)=1638  0x676=1654 more then we need but I think you get I the idea .
4-CMOS[1] Vertical Offset adjustment , 0xE6A (default)  change to 0xBOD (just need to play around cmos[1] trial & error)
5-Increase frame rate , c0f06014 0xEDC to 0x4something etc...
Yes, basically it's what I'm doing, except for CMOS. From photo mode 0xc0f06088 = c9d09bb, 0xc0f06014 = c9c and 0xc0f06008 = 4f504f5

Noticed that timers appears to have a little overhead based from math (https://www.magiclantern.fm/forum/index.php?topic=19336.msg209780#msg209780) you pointed to me before, should I try to reduce timer and resolution (to mach cr2 and reduce black area) later when I get Full Resolution working?

Quote from: reddeercity on January 09, 2019, 04:33:30 AM
That's the basics , adjust slowly -- if you flip thought the adjustments too fast you can either lockup the cam or loose the image (liveview turns black)
At best most or this is just trial & error plus a few lucky breaks  :D
Cycling through video mode will usually help to get LV working when gets completely dark.

Quote from: reddeercity on January 09, 2019, 04:33:30 AM
Edit: Are you using the 50d 10-12bit build from the experimental download page
or did you use a build compiled from my source file with the add parts (redirect buffer for extended res.) for 50d ?
I'm using your source files with the addittion of 50D's stubs (https://bitbucket.org/hudson/magic-lantern/commits/db4ee396f4a261d33688b528c39447682f871a07?at=raw_video_10bit_12bit_LVState) for CONFIG_EDMAC_RAW_PATCH support.
I still need to recompile with what you told me before (https://www.magiclantern.fm/forum/index.php?topic=19336.msg210179#msg210179) to add extendeds resolutions.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 09, 2019, 07:33:26 PM
@reddeercity are you using the iso-research branch with adtg gui module to come up with these settings? I'm wanting to try... I see you posting screenshots of your settings which seem like a good starting point.

Also have you been able to create any presets (that can be selected in crop_rec menu) since that time Danne tried to put it in crop_rec.c?

I have been reading the code and competely confused how a preset is added to the menu. Why can't it just be a struct with some values...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 10, 2019, 06:21:16 AM
@Ilia3101 I'm working on the code , lots of parts to deal with .
if you what to try , use the build on the first page of this thread then
download the ADTG_GUI.mo from https://builds.magiclantern.fm/modules.html
that's the one I use . Only load mlv_rec.mo & adtg_gui.mo -- Do Not load the crop_rec.mo

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 10, 2019, 06:54:30 AM
Can somebody check this please , I'm not sure about the "case cmos_new[1]:"
not sure if I need to add it here or somewhere else
I'm using waza57 crop_rec source 5d2_crop_rec.c (https://bitbucket.org/waza57/magic-lantern/src/8bfbb0ca228e11f091869df09a1b8f16dd36b508/modules/crop_rec/crop_rec.c?fileviewer=file-view-default)
Starting at Line 1508 of crop_rec.c (https://bitbucket.org/waza57/magic-lantern/src/8bfbb0ca228e11f091869df09a1b8f16dd36b508/modules/crop_rec/crop_rec.c?fileviewer=file-view-default#crop_rec.c-1508) is where I made the changes , I'm using the values from this post here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg210327#msg210327)
static inline uint32_t reg_override_zoom_fps(uint32_t reg, uint32_t old_val)
{
    /* attempt to reconfigure the x5 zoom at the FPS selected in Canon menu */
    if (is_5D2)
    {
     int timerA = 679 ;         //waza57 good values for 23.976: 804- 1245  and 807 - 1237
     int timerB = 1471 ;     
     int a = reg_override_fps_nocheck(reg, timerA, timerB, old_val);
        if (a) return a;
           switch (reg)
            {
                /* raw resolution (end line/column) */
                /* X: (3072+140)/8 + 0x17, adjusted for 3072 in raw_rec */
               
                case cmos_new[1]:
                    return (old_val & 0x0000000) + 0xb0d  ;

                case 0xC0F06084:
                    return (old_val & 0x0000000)  ;
               
                case 0xC0F06088:
                    //return (old_val & 0xFFFF0000) + 0x630 ;
                    return (old_val & 0x00000000) + 0x56a053a;   //0x4b00648 or origin value chang nothing

                 /* HEAD3 timer */
                case 0xC0F0713c:
                    return 0x566 + YRES_DELTA + delta_head3;

                /* HEAD4 timer */
                case 0xC0F07150:
                    return 0x58c + YRES_DELTA + delta_head4;

                case 0xC0F08184:
                    return (old_val & 0x0000000) + 0x56a ;

                case 0xC0F08188:
                    return (old_val & 0x0000000) + 0xa07 ;

                case 0xC0F08518:
                     return (old_val & 0x0000000) + 0x56a0a07 ;                   
            }


Here Line-594 (https://bitbucket.org/waza57/magic-lantern/src/8bfbb0ca228e11f091869df09a1b8f16dd36b508/modules/crop_rec/crop_rec.c?fileviewer=file-view-default#crop_rec.c-594) there's a cmos[1] change , I thinking that's where I should put the new cmos[1] value
  if (is_5D2)
                {
                cmos_new[1] = PACK12(42,57);//0xB0D;    /* pink highlights without this */
                cmos_new[2] = 0x10E;    /* read every column, centered crop */     //waza57 cmos 6
                break;



Edit: I updated my source with the 50d stubs for "CONFIG_EDMAC_RAW_PATCH" support
Crop_Rec-4k-5d2-50d-magic-lantern_1-9-2019.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Crop_Rec-4k-5d2-50d-magic-lantern_1-9-2019.zip)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: theBilalFakhouri on January 10, 2019, 10:52:00 AM
Quote from: reddeercity on January 09, 2019, 07:12:24 AM
Figured out 2400x1330 @ 23.976fps real time color preview (same as canon preview) while recording raw video
Quote from: reddeercity on January 09, 2019, 07:12:24 AM
You may wonder what the last three reg's are for , they are for the liveview preview while recording instead of a frozen liveview
c0f08184 = preview height,  I set this to the raw height +52 (the OB area)  0x467(1127)=>0x56a(1386)
c0f08188 = preview width , I set this to the total raw width+(OB Area 160) 0x907(2311)=>0xa07(2567)
c0f08518 = both preview height & width 0x4670907->0x56a0a07 ( 1127x2311)->(1386x2567) and of course we read this as 2567x1386    ;)

This is great! Do you mean also the real-time preview now is uncropped ? It shows the entire frame in real-time?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 11, 2019, 04:08:45 AM
Quote from: theBilalFakhouri on January 10, 2019, 10:52:00 AM
This is great! Do you mean also the real-time preview now is uncropped ?
Yes , until recording raw , then just like the default crop_mode it center crops
but I never tried with the overlays enabled as I have "kill overlays on recording"
in mlv_rec , get me a extra 25% write speed .
So it may with overlays  :) I'll check next time I have it loaded up.
Here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg209086#msg209086) I posted a iPhone video or the lcd screen showing the Liveview as I record 2400x1200 @ 23.976fps .

I know before with default crop_mode (2144x1074) if you press half shutter , you get a color preview with correct full framing
while recording , so I'm not sure it there's ML code that does the resizing or if it's a reg , maybe in the "preview regs"
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on January 11, 2019, 06:14:58 AM
Quote from: reddeercity on January 09, 2019, 07:12:24 AM
Still adjust different presets , trying to get more height in the 2k range
Figured out 2400x1330 @ 23.976fps real time color preview (same as canon preview) while recording raw video
record times are short , about 20 seconds at the moment (I haven't starting on Over Clocking the CF card Bus yet that's next)

(https://i.ibb.co/3Mq5zKz/VRAM28-crop-small.png) (https://imgbb.com/)
(https://i.ibb.co/54W99qD/VRAM29-small-2400x1330-24fps.png) (https://imgbb.com/)

You may wonder what the last three reg's are for , they are for the liveview preview while recording instead of a frozen liveview
c0f08184 = preview height,  I set this to the raw height +52 (the OB area)  0x467(1127)=>0x56a(1386)
c0f08188 = preview width , I set this to the total raw width+(OB Area 160) 0x907(2311)=>0xa07(2567)
c0f08518 = both preview height & width 0x4670907->0x56a0a07 ( 1127x2311)->(1386x2567) and of course we read this as 2567x1386    ;)

Are these the complete set of registers changed? Could you recheck if any setting is missed when selecting overridden regs only?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 14, 2019, 10:58:13 PM
@reddeercity Is it techninally possible to get slow motion 48+ fps in crop mode? If I were to copy your slow motion registers while in 5x would it be effective? Or does that need something completely different


@a1ex We have been discussing with Danne about making presets for crop_rec out of the sets of registers that reddeercity has been finding. It seems really difficult, can there not be a way to just have them in some c structs/arrays?

soething like

preset_struct presets[ ] =
{
    {
        .name = "preset1",
        .num_registers = 2,
        .registers = {register1, value1, register2, value2}
    },
    {
         .name = "preset2",
...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 15, 2019, 06:35:22 AM
Quote from: Ilia3101 on January 14, 2019, 10:58:13 PM
@reddeercity Is it techninally possible to get slow motion 48+ fps in crop mode? If I were to copy your slow motion registers while in 5x would it be effective?
Or does that need something completely different
Yes needs something different, 48fps is dependent on line skipping & vertical compressed (3x5) which only work will in 3x3
you can force line skipping  in 3x crop but can't produce a useable image (being trying for 2 weeks now)
The best to hope for in crop mode would be a very small crop window e.g. 1280x430 (crop factor of about 4.5 which I have no interest in)
The simple fact that the sensor is very slow (96MP/s) kept it from any meaning full resolutions & or hi-frame rate .
This is a fact that can't be changed at present , Overclocking the sensor frequency would be the only way (24->32 maybe ?)
a1ex would the only one that would know for sure.
But you got to ask yourself , is worth it ? there would need to be more voltage among other variable (just like overcocking PC CPU)
Can it handle it or will it burn up ?

But you can certainly explore the possible , All my note are here on my thread .


Quote from: Ilia3101 on January 14, 2019, 10:58:13 PM

..... discussing with Danne about making presets for crop_rec out of the sets of registers that reddeercity has been finding.
It seems really difficult, .....
No it not difficult , I got it figured out just been too busy to finish coding it .
D4 cams work a little different then d5 cams in crop_rec . (this is no small thing , it took @waza57 some time to get it to work , more then a mirror hack)
beside that , I not very impressed with 4k compressed vertical (4096x590)
It seems to produce more artifact then standard 3x3 (1856x1248)  , I guess because of the different pixel binding then d5 -- it's worst then d5 cams .
So I guess just because you can implement something doesn't mean you should .  :-\
I'm looking at this from a video production side (that why I won't have anything that doesn't record @ 24fps - besides Full Res ([email protected])
I'm re-evaluating all my presets , sorry for the delay but life come first  :)
As this part of the development is coming to a close for the 5D2 , other d4 cams with users need to catch up on there own (I'll still help where I can).

After I finish the crop_rec for the 5d2 , I'll spend my time now on overclocking CF card bus & Lossless compression .
I know the limitations of the sensor now  , I pushed it as far as it can go (I can see a cam upgrade for me in the future maybe 5d3 or 5d4)


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 16, 2019, 12:25:47 AM
Quote from: reddeercity on January 15, 2019, 06:35:22 AM
Yes needs something different, 48fps is dependent on line skipping & vertical compressed (3x5) which only work will in 3x3
you can force line skipping  in 3x crop but can't produce a useable image (being trying for 2 weeks now)
The best to hope for in crop mode would be a very small crop window e.g. 1280x430 (crop factor of about 4.5 which I have no interest in)
The simple fact that the sensor is very slow (96MP/s) kept it from any meaning full resolutions & or hi-frame rate .

Assuming 96 is a reachable limit, here's what we would be able to get at 60fps:






Aspect         Max resolution
2.67:12064x774
2.35:11936x824
16:91680x945

This sounds really appealing if possible to me. I'm sure it's probably not possible to reach that limit so perfectly, but those resolutions might be doable at something lower like 50fps.

Quote from: reddeercity on January 15, 2019, 06:35:22 AM
But you can certainly explore the possible , All my note are here on my thread .

What you have been doing is definitely very helpful.


Quote from: reddeercity on January 15, 2019, 06:35:22 AM
No it not difficult , I got it figured out just been too busy to finish coding it .
D4 cams work a little different then d5 cams in crop_rec . (this is no small thing , it took @waza57 some time to get it to work , more then a mirror hack)

:(

It is difficult to understand though, I still have no clue how to make presets.

Quote from: reddeercity on January 15, 2019, 06:35:22 AM
I'm looking at this from a video production side (that why I won't have anything that doesn't record @ 24fps - besides Full Res ([email protected])

I would personally like to make lower fps ones just for higher, but not full, resolution recording. It works if there's no speech or fast movement.

Quote from: reddeercity on January 15, 2019, 06:35:22 AM
I'm re-evaluating all my presets , sorry for the delay but life come first  :)

Understandable :)

Quote from: reddeercity on January 15, 2019, 06:35:22 AM
After I finish the crop_rec for the 5d2 , I'll spend my time now on overclocking CF card bus & Lossless compression .

Any of those two things would be AMAZING 😍

Quote from: reddeercity on January 15, 2019, 06:35:22 AM
I know the limitations of the sensor now  , I pushed it as far as it can go (I can see a cam upgrade for me in the future maybe 5d3 or 5d4)

You definitely deserve the upgrade.

I am starting not to want to upgrade ever though, 5D mark 2 is too much fun. How endlessly can the 11 year old camera be pushed?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on January 16, 2019, 12:55:47 PM
Quote from: reddeercity on January 15, 2019, 06:35:22 AM
I pushed it as far as it can go (I can see a cam upgrade for me in the future maybe 5d3 or 5d4)

get the 5D4!!!  let's not wait until it is 11 years old before it is supported!
you're the man to make it happen!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ItsMeLenny on January 17, 2019, 02:16:20 AM
96 isn't possible on any camera https://docs.google.com/spreadsheets/d/16cgnRivbUv7nA9PUlCLmLdir3gXdIN3pqzCNAAybepc/edit#gid=5
in digic IV. Unless anybody's tried overclocking the SD card speed.
Probably wonldn't need to if RAW compressed (or lossless whatever stupid name people were calling it) could work.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 17, 2019, 06:36:05 PM
Quote from: ItsMeLenny on January 17, 2019, 02:16:20 AM
96 isn't possible on any camera https://docs.google.com/spreadsheets/d/16cgnRivbUv7nA9PUlCLmLdir3gXdIN3pqzCNAAybepc/edit#gid=5
in digic IV. Unless anybody's tried overclocking the SD card speed.
Probably wonldn't need to if RAW compressed (or lossless whatever stupid name people were calling it) could work.

This isn't MB/s card speed, it's sensor speed- Megapixels per second. I was just coming up with some theoretically possible resolutions/framerates that I want to try and get on the 5D2 in crop mode. They wouldn't record continuously of course, about 6 seconds in 10 bit (but it is 60fps, so a 14 second long shot once slowed down to 24)

There's also a chance that none of this is possible at all in crop mode.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 19, 2019, 06:45:39 AM
I'm looking at keeping 4k (4096x590=>4096x1770 23.976fps) or as I call it
Vertical compressed , don't like the name 1x3 or 3x1 or what ever it's called (less confusing)
I did a short 4 min talking head clip (me talking about 4k on the 5d2  :) ) Yea that continuous @69MB/s write speed @ 10bit uncompressed
I found it to be not as bad as I thought , yes there a very small amount of aliasing
but with some heavy grading and anti-aliasing filters is more then useable
thou you must select your shots wisely to avoid this little issue .(fine patterns etc. .. you know the drill)
Here a frame of the close to final grade , still needs a little tweaking .

(https://i.ibb.co/V9DwCbZ/5-D-Mark-ii-Real-4-K-small.png)
5D Mark ii Real 4K.tiff (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D%20Mark%20ii%20Real%204K.tiff)

Workflow:
MLV App 1.4 wins , to exact dngs to full height (590->1770)
Imported the dng's in to Adobe CS6 A.E. ACR set project to 32bit float
(I tried mlv app first , but I can't get the image to look right the skin tone where really bad & had had too much noise in the shadow ,
sadly I when back to A.E. so slow 6Hrs to generate 6300 4k tiff's  :( compared to 90 mins on mlv app maybe it better on the MAC never tried).
Imported the tiffs in to Apple compressor , made a real ProRes4444XQ (Apple ProRes is much better , 16bit+alpha)
The real ProRes4444XQ can be real pushed hard for grading , take a lot to break the image .
Imported in to FCPX edit & final grade , added my LUT there , "ArrilInterneg_KodakVision_sRGB.cube .

So do the other 5D2 users think I should keep 4K preset in compressed vertical ?
The sample above should be noted that I was a little out of focus (hard to do by yourself  :D )

FYI: 4mins of 4096x590@ 23.976fps = 17.7GB (mlv_rec file) that got blown up to 350GB of tiffs = 40GB ProRes4444XQ
So at 4096x1770 , 1 dng = 9MB , 1 Tiff = 58MB x6300 frames
Cam settings 400 ISO 1/50th 180-degree shutter , Canon Ef24-70 f2.8L @ 24mm f4.0
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on January 19, 2019, 01:30:04 PM
Quote from: reddeercity on January 19, 2019, 06:45:39 AM
So do the other 5D2 users think I should keep 4K preset in compressed vertical ?
Any of your posted presets would be nice to try out for me. But unfortunately I don't get one single preset to work over here...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 19, 2019, 02:16:26 PM
Quote from: reddeercity on January 19, 2019, 06:45:39 AM
So do the other 5D2 users think I should keep 4K preset in compressed vertical ?

First impression when I opened it: wow this looks sharp, second impression: Oh there's a lot of alising (even on the out of focus monitor).

I still think the 1:1 presets look better, though you could keep this one for a wider field of view. Also a softer lens might reduce aliasing.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on January 19, 2019, 02:58:16 PM
As presets need to be frequently updated, I like the Ilia idea of creating a struct to store different presets.
If it's possible to do it, it's more suitable, it's open, it's simple (no need to compile every time).
Indeed, we can exchange an xml file (or whatever file format)  that store 5D2 presets between users.
IMHO, I think 3K/UHD raw developpement and registers discover will be more easy and quickly spread over users.

David
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 20, 2019, 01:59:50 AM
Quote from: benoit on January 19, 2019, 02:58:16 PM
As presets need to be frequently updated, I like the Ilia idea of creating a struct to store different presets.
If it's possible to do it, it's more suitable, it's open, it's simple (no need to compile every time).
Indeed, we can exchange an xml file (or whatever file format)  that store 5D2 presets between users.
IMHO, I think 3K/UHD raw developpement and registers discover will be more easy and quickly spread over users.

David

Well with my idea (https://www.magiclantern.fm/forum/index.php?topic=19336.msg210543#msg210543) (C structs in the source code) you'd still need to compile it each time, but it would still be massively easier than what it is now (loads of random bits spread all over the code where you need to add stuff).

And your idea of having it in a file is amazing if it could come true, but it seems unlikely to me that anyone able to do that will find time (and for my idea). It also requires a format simpler than xml, maybe csv, or even a binary format that would need to be edited with a hex editor.

I tried recreating reddeercity presets with adtg_gui (crop rec disabled), no success yet, can't find some of the registers 🙁
@reddeercity Also are the adtg_gui screeenshots showing all the registers you changed?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 20, 2019, 06:26:50 AM
@Ilia3101 , Your right too much aliasing , check out the sample below  -- better ?
Yes when I post Reg's screen shots it all the reg's  needed for the what ever preset I was working on.
No sure why you can't reproduce , 5D2 is really touchy , if you adjust reg's out order the cam will lock up or will not respond .
You have to enable (in the advanced tab) ENGIO registers , then refresh , view or play a h264 or cr2 file to update the reg's
Rule of thumb : First thing Slow down Timer"A" first then Timer"B" , I usually do all my work around 10fps then after I set raw rez.
and head timers I increase timers "B" to the correct frame rate (23.976) (but slowly) .
Yes "C" code can be hard to understand sometimes , but there is a reason for everything  ;D
If you read the crop_rec code you will see that it  needs calls from adtg to apply the modified regs.


Ok I'm very close to getting 3xcrop (5xzoom) to work with Lineskipping 
I had to start in 1:1(3x3 FHD) set the timers , resolution & cmos[2] reg
Then switch to 3x crop_mode (5x zoom) set to adtg12[100c] to line skipping (0x5->0x6)
Adjust frame blanking adtg1[106] 0x301->0x1 lastly cmos[5]
Seem to be cleaner , haven't notice any aliasing yet but I'm still adjusting
here a sample to pixel peep

(https://i.ibb.co/k2T4Cby/M19-2238-frame-47-small.png) (https://imgbb.com/)
3x_crop_mode_Line-Skipping_4096x1776_M19-2238_frame_47.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3x_crop_mode_Line-Skipping_4096x1776_M19-2238_frame_47.png)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on January 20, 2019, 07:42:49 PM
yep!

definitely looks much better!!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 20, 2019, 08:09:44 PM
does look better
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 21, 2019, 10:50:00 AM
Quote from: reddeercity on January 20, 2019, 06:26:50 AM
Yes when I post Reg's screen shots it all the reg's  needed for the what ever preset I was working on.

good

Quote from: reddeercity on January 20, 2019, 06:26:50 AM
No sure why you can't reproduce , 5D2 is really touchy , if you adjust reg's out order the cam will lock up or will not respond .
You have to enable (in the advanced tab) ENGIO registers , then refresh , view or play a h264 or cr2 file to update the reg's
Rule of thumb : First thing Slow down Timer"A" first then Timer"B" , I usually do all my work around 10fps then after I set raw rez.
and head timers I increase timers "B" to the correct frame rate (23.976) (but slowly) .

Thank you for this, really useful! I had enabled ENGIO, but had no idea about the playing a file trick. Also had no idea about the order of setting them. I think this will help a lot next time I try.

Quote from: reddeercity on January 20, 2019, 06:26:50 AM
Yes "C" code can be hard to understand sometimes , but there is a reason for everything  ;D
If you read the crop_rec code you will see that it  needs calls from adtg to apply the modified regs.

I still think it could be modified to make those calls based on some data in a simple to understand struct. I would like to do this if I ever get an understanding of how crop rec presets work right now.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on January 21, 2019, 03:03:34 PM
Well one way to proceed is to create actual presets in crop_rec.c. From there one can alter registers from within a sub menu structure. Then include all relevant registers and connect them to the presets so it´s possible to modify anything but from a starting point where all registrys are already set.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 22, 2019, 07:37:06 AM
Ok I got crop_rec to work with my new preset 2.4k (2400x1330 @ 23.976)
But I do have a problem , waza57 added a rom hack to increase height in 5x Zoom
before a1ex implemented "Raw_Slurp" Redirect Buffer for extended resolution in 5D2/D4 Cam .
As a result my preset is  2368x1266 @24.028fps  :(
Active Area     : 0 0 1266 2368
Frame Rate      : 24.028


So I'm currently going thought the crop_rec code line by line to figure out how to disable it/remove it
as it's no longer needed as a result the 2 hacks are fighting each other causing conflict

@waza57 is there a quick way to disable the rom hack you added ?

So once that rom hack is disable , I can implement all my preset including
4K line skipping in 5x zoom , got that to work now @ 23.976fps (4096x590=>4096x1770)

(https://i.ibb.co/p1ZQ7r6/VRAM1-small.png) (https://imgbb.com/)

We are almost thought the forest now and I can see a clearing with rolling grasslands to play in  :))
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 23, 2019, 07:20:17 AM
Boy this ROM hack from waza57 for height is really hard to disable , more I think I'm close the more error's I get  & can't compile  :(
On the Bright side  :D My preset code structure works , after I found where to add the code.

2400x1330 23.976fps +unfrozen liveview
line_592-598 
                  if (is_5D2)
                {
                cmos_new[1] = 0xB0D;    /* pink highlights without this */
                cmos_new[2] = 0x10E;    /* read every column, centered crop */     //waza57 cmos 6
                break;
                }
                break;   
   
 

****Line_685-693****
/* raw buffer centered in zoom mode */
            case CROP_PRESET_CENTER_Z:
                cmos_new[1] = (!is_5D2)
                  ? 0xB0D          /* vertical (first|last) */
                  : -1;
                cmos_new[2] = (!is_5D2)
                ? 0x10E            /* horizontal offset (mask 0xFF0) */
                : -1;
                break;


****Line_1508-1544***
static inline uint32_t reg_override_zoom_fps(uint32_t reg, uint32_t old_val)
{
    /* attempt to reconfigure the x5 zoom at the FPS selected in Canon menu */
    if (is_5D2)
    {
     int timerA = 680 ;         
     int timerB = 1472 ;     
     int a = reg_override_fps_nocheck(reg, timerA, timerB, old_val);
        if (a) return a;
           switch (reg)
            {
                /* raw resolution (end line/column) */
                /* X: (3072+140)/8 + 0x17, adjusted for 3072 in raw_rec */
               
                 case 0xC0F06084:
                    return (old_val & 0x0000000)  ;
               
                case 0xC0F06088:
                    //return (old_val & 0xFFFF0000) + 0x630 ;
                    return (old_val & 0x00000000) + 0x56A053A;   //0x4b00648 or origin value chang nothing

                case 0xC0F0713C:
                    return (old_val & 0x0000000)  + 0x566;

                case 0xC0F07150:
                    return (old_val & 0x0000000) + 0x58C;
               
                case 0xC0F08184:
                    return (old_val & 0x0000000) + 0x56A;

                case 0xC0F08188:
                    return (old_val & 0x0000000) + 0xA07;

               case 0xC0F08518:
                    return (old_val & 0x0000000) + 0x56A053A;

               }


Plus another Good thing I discovered tonight , after testing my Reg's with ADTG_gui to make sure I have all of them right
I thought I would test HDMI to my Zacuto EVF and see if it works & Yes! I can preview in real time over the HDMI
while recording raw @ 2400x1330 23.976fps  :D
Even pass it thought to my Ninja HDMI recorder , thou the image is gray scale when recorded but Half Shutter press give
a color preview . It's still center crop but at least you can have finer focus level special if you have diopter as I do for the EVF
Still haven't figured out how to scale it to LCD(720x480) or HDMI(1920x1080) yet .

I'll start fresh tomorrow on the crop_rec.c , hopefully I'll has a better luck .


Edit: I see there maybe a quick way to disable the ROM hack
Quote from: waza57 on September 01, 2018, 01:49:44 PM
edmac-memcpy.c , fio-ml.c
          I need to disable call to function "clean_d_cache()" otherwise i can't patch ROM addresses
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on January 23, 2019, 01:07:42 PM
I really hope you get a 5D4...  Your persistence and determination are badly needed for that unfortunate unappreciated BEAST!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 24, 2019, 01:08:03 AM
Ok that didn't work , there more that's needed to override this going to have to take a drive deep .
It may be more then just Crop_Rec.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 24, 2019, 08:00:01 AM
Found the ROM_Hack from waza57 , now I open up a can of worms  :(
just freezes in x5 zoom with crop_rec enable , before at least I had a image , thou it was all messed up.
going to have to back track again , this maybe over my head , I thought by disabling the hack I could get this to work.
not too sure now .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on January 24, 2019, 03:01:11 PM
Quote from: reddeercity on January 23, 2019, 07:20:17 AM
Boy this ROM hack from waza57 for height is really hard to disable , more I think I'm close the more error's I get  & can't compile  :(
On the Bright side  :D My preset code structure works , after I found where to add the code.
I was wondering if isn't better to make Digic 4 camera build on top of official crop_rec_4k branch? This is something should be done before or after.

At the moment I can remember @dfort for lossless compression here (https://bitbucket.org/daniel_fort/magic-lantern/commits/branch/crop_rec_4k_Digic4) and a new one here (https://bitbucket.org/daniel_fort/magic-lantern/commits/branch/compressed_raw_digic4) , which is a fork of crop_rec_4k if I'm not wrong; then there is waza57 repository (https://bitbucket.org/waza57/magic-lantern/branch/crop_rec_4k_5D2), which I'm not of what is capable of doing and I imagine that @reddeercity is working on top of that because is the one which compile, is it right?

Can someone make me an overview of the difference on those repository and what is needed to make d4 compile on top of crop_rec_4k?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on January 24, 2019, 05:13:38 PM
Hi, Reddeercity, this is about zoom jumping out (the problem of my camera I described)
https://drive.google.com/file/d/1aa8kzCQueoRzHM1PoiVRr0jjbkSrG8Za/view?usp=sharing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 25, 2019, 06:53:25 AM
Success !! disabled the "Rom Hack" and I can use the "raw_slurp" redirect buffer for extended rez.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on January 25, 2019, 08:30:41 AM
Adding above gives this when compiling:
           if (!patch_active)   waza57 changes to effective update for 5d2
                                       ^
crop_rec.c:1660:71: error: invalid suffix "d2" on integer constant
           if (!patch_active)   waza57 changes to effective update for 5d2
                                                                       ^
crop_rec.c: In function 'crop_rec_init':
crop_rec.c:2313:9: error: 'RAWROMHEIGHT_WRITE' undeclared (first use in this function)
         RAWROMHEIGHT_WRITE = 0xFFCAA1F4;
         ^
crop_rec.c:2314:9: error: 'MEM_RAWROMHEIGHT' undeclared (first use in this function)
         MEM_RAWROMHEIGHT = 0x00000467;
         ^
crop_rec.c:2316:9: error: 'RAWROMWIDTH_WRITE' undeclared (first use in this function)
         RAWROMWIDTH_WRITE = 0xFFCAA1EC;   // waza57 patch ROM for more height
         ^
crop_rec.c:2317:9: error: 'MEM_RAWROMWIDTH' undeclared (first use in this function)
         MEM_RAWROMWIDTH = 0x00000FCE;


Did you also disable this?
       // RAWROMHEIGHT_WRITE = 0xFFCAA1F4;
       // MEM_RAWROMHEIGHT = 0x00000467;
       
       // RAWROMWIDTH_WRITE = 0xFFCAA1EC;   // waza57 patch ROM for more height
       // MEM_RAWROMWIDTH = 0x00000FCE;


Please upload a working crop_rec.c version that you know compiles.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 29, 2019, 07:18:52 AM
Ok Finally got Crop_Rec working correctly on 5D2 with "raw_slurp" redirect buffer !
I started fresh with a simple  3k @ 23.98fps preset (3008x1074 @ 23.98fps)
(https://i.ibb.co/GnKz6KB/VRAM16-small.png) (https://imgbb.com/)

magiclantern-crop_rec-3k-5D2-eXperimental.2019Jan28.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/)
Updated file
magiclantern-crop_rec-3k-5D2-eXperimental.2019Jan31.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/)
Updated File
magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212.zip)

Unfrozen Liveview while recording raw video  :D

I was having problem with c0f0713c & c0f07150 , as I have this all wrong in the code
and figured out that if you reduce 7150 more then 713c
you actually reduce over head for more frame FPS , that's how I squeezed out 23.98 @ 3008x1074 ,
if you leave those reg's alone the best you can do is 18fps at the same resolution .

The other big problem I have (haven't solved yet) is I need different "CMOS[1]" vertical offsets
setting then the one in "crop_rec" seems it just call's the one vertical offset in default 5x Zoom mode (0xE6A)
where in fact I need to have at least 3-4 different vertical offset (one per preset)
Unless there's something I missed , I just worked around the default CMOS[1] offset
and adjust from there . That why I haven't increase the vertical from default.

So this is the first test build for everyone to try , You only need to load
5D2_212.sym
crop_rec.mo
mlv_rec.mo


Same as before , canon menu 30fps -- enable 3.5k preset then enter in to
5x zoom , should take a few seconds before it comes up , half shutter will give a full color preview.
when you start recording , there should be a short 1-2 second delay then the preview will switch to real
time (canon preview) in b/w , as noted in my other posts it a center crop , but it is in real time  8)

Do Not ! enable "frame override" as this will lockup/freeze cam .
FYI - there's no mlv_lite.mo right now ,I need to add lossless to it before it can compile .
Test it as normal , let me know what breaks if anything . I'll add more preset after I figure out how
to code CMOS[1] offset's correctly .

@danne not sure where you when wrong , I checked the code I posted and it looks right
but I check it againist the source . when I have every thing done I'll post my code , it far too messy right now. :P
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 29, 2019, 10:46:01 AM
Incredible work @reddeercity!!! Can't wait to try it out.

3008x1074x24 is only 77Mpix/s, so there should still be plenty of  room for height increases if the overhead can be reduced more.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on January 29, 2019, 03:54:02 PM
Thank for Anything that you give us RedDeerCity but What's the best 1:1 Resolution? I do notice if I'm going to use x5 then to get the equlavate of a 50mm I need to use 24-35mm right?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on January 29, 2019, 08:11:51 PM
Thank you @reddeercity! I tried it out:
so far it is working over here. Mostly it is working with superb picture quality. In one test clip the first 2 pictures were corrupted and after some seconds maybe 10 frames came in as "corrupted bundle". Other test clips were 100% fantastic.
This might not be of interest: I can't quit the crop mode by using zoom button. I have to use menu button. But I think, to make this safe was not your goal ;) (no real problem so far)
Keep up the great work!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 29, 2019, 08:16:53 PM
About to test it.

@masc you are right, leaving 5x stopped working. It did work some time ago though... used to be able to get to 10x
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on January 30, 2019, 01:19:37 AM
My tests went well! I had one freeze/battery pull but it happened when I was bumping my camera around a lot getting it onto a tripod shoe so I'm not exactly sure what caused it.

I was surprised at how long I could record with my Lexar 1066x. The icon was red but it just kept going. I got to 2 minutes before eventually stopping it myself.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 30, 2019, 05:45:08 AM
@ masc & Ilia3101 thanks , yea not too worried about safety right now , you know Bleeding Edge stuff  :D
The best way to exit crop_rec is to press the play button twice (once to enter in player mode & once to exit player mode)
this will exit to 3x3 (1:1 FHD) without any cam freeze/lockup that needs a battery pull . Then you can enter into crop_rec
again by pressing the 5x zoom .


Quote from: jpegmasterjesse on January 30, 2019, 01:19:37 AM
I was surprised at how long I could record with my Lexar 1066x. ........  got to 2 minutes before eventually stopping it myself.

Will that cool ! I only tested on my slow 90MB/s Sandisk extreme pro  , just check on my 32GB Lexar 1066x card .
Holy Cow ! your right , ml is only expecting around 300 frames , I stop at 10GB 3300 frames(2:19 min) 
WOW that's continuous  :o , never thought 3008x1074 @ 23.98fps would be special when 2880x1080 was only good for 20-30 seconds .

Must be because of reducing the head timers 3 & 4 which reducers the over head .
ML reports it takes 92.3MB/s @ 3008x1074 23.98 but in the setting file I see the write speed reported was 75.8MB/s , strange .

Another bonus  - HDMI works while recording raw  in crop_rec 3k (3008x1074) , check with my HDMI Zacuto EVF .
But you need to enable "Force HDMI to VGA" so that changes the HDMI out put from 1920x1080 to 720x480.
It really very surprising that HDMI works as well as it does in 3k plus recording continuous , that just blow me away :))

Edit:
For others who may have missed it , 3008x1074 build is available now , details here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg211130#msg211130)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on January 30, 2019, 01:52:39 PM
well done!!!

Again>
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: coco770108 on January 30, 2019, 04:53:21 PM
That is Incredible !!!
THX @reddeercity!!!
cannot wait to try it~~
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: stokis on January 31, 2019, 11:57:24 AM
I tested it, filmed many clips. Strange, but when i play the clips on my pc it plays in fast motion. It looks like in reality 3008x1074 is filmed in ~17 fps, but compressed in 23.976 and plays back in fast motion.
Try it yourself. Film some animal or human, you will see it will play back not in normal pace but in fast motion.

I calculated real file size vs time, and it is ~99 MB/s. But when i film, the camera display shows me ~72 MB/s.

p.s. 2880x1080 10bit worked perfect for me, with normal motion as 23.976 supposed to work.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on January 31, 2019, 05:14:32 PM
I have tried the new Preset. In the beginning i must say perfekt...

But... the record timer is to slow, maybe x0.5 -> in the postproduktion the soundline is in normal speed, but the moviespeed maybe x2.  I thing there are not 23.98 FPS, but ~ 12 FPS.

sry for bad english
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 31, 2019, 09:00:08 PM
Thank for the report
I'll check it out and see what's going on
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 01, 2019, 06:07:49 AM
Found the problem in the Head Timers , had c0f07150(Head timer 4) below the valve( of c0f0713c (Head Timer 3)
I relaxed Head Timer c0f07150 to the same valve a c0f0713c (0x467) now everything is running at the right speed .

Here the updated build , 3008x1080 @ 23.98 fps
magiclantern-crop_rec-3k-5D2-eXperimental.2019Jan31.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/)
Updated File
magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212.zip)
Updated , see  post #512  (https://www.magiclantern.fm/forum/index.php?topic=19336.msg211379#msg211379)

Ran a few tests , looks ok but no more continuous write speed , sorry  :-[
that was side effect of the messed up head timers .
I'm getting around 300-400 frames right now , until I get on with cf card over clocking & lossless compression .

Everything should be the same as before with real time preview while recording raw @ 3k 24p
HDMI should also work , with hdmi forced to VGA (720x480) .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on February 01, 2019, 05:46:17 PM
Just stumbled onto something that might help Digic IV.

src/propvalues.c
        //#ifdef CONFIG_DIGIC_IV - fixme
        return version[0] == '4';
        //#endif


If you look at the code for any Digic V camera you'll find this in the platform/[camera]/internals.h file:

/** This camera has a DIGIC V chip */
#define CONFIG_DIGIC_V


This is not in the internals.h for Digic IV cameras. Code like this could be simplified:

src/raw.c
#if defined(CONFIG_5D2) || defined(CONFIG_50D) || defined(CONFIG_60D) || defined(CONFIG_550D) || defined(CONFIG_500D) || defined(CONFIG_600D) || defined(CONFIG_1100D) || defined(CONFIG_7D)
#define RAW_PHOTO_EDMAC 0xc0f04208
#endif

#if defined(CONFIG_5D3) || defined(CONFIG_700D) || defined(CONFIG_6D) || defined(CONFIG_EOSM2) || defined(CONFIG_EOSM) || defined(CONFIG_650D) || defined(CONFIG_70D) || defined(CONFIG_100D)
#define RAW_PHOTO_EDMAC 0xc0f04008
#endif


To this:

#ifdef CONFIG_DIGIC_IV
#define RAW_PHOTO_EDMAC 0xc0f04208
#endif

#ifdef CONFIG_DIGIC_V
#define RAW_PHOTO_EDMAC 0xc0f04008
#endif


Just a suggestion that might simplify getting other Digic IV cameras up to speed with the great work reddeercity has been doing on the 5D2.

BTW--the "fixme" comment in propvalues.c probably still applies for Digic 4+ cameras like the 1300D.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: stokis on February 01, 2019, 10:43:50 PM
I tested latest 31.jan update on 3008x1074 @ 23.98 fps.
Now the motion is ok, as 23.98 fps should be. I can record ~18 sec without dropped frames.
And mlv sound also goes one on one with picture.

I hope you can somehow make crop preset with increased vertical resolution more than 1074 :)

Thank you reddeercity for your work!
I can use my old canon 5D mkII for filming my kids in 3k now :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on February 02, 2019, 01:58:44 AM
Curious about 30 fps in the Canon menu - why is this necessary? I tried it today with 24 and 30 and didn't notice any difference in my results.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 02, 2019, 06:35:55 AM
New Update: 3008x1080 @ 23.98 fps
magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212.zip)

Update Notes:
Pushing the vertical without having to code CMOS[1] offset plus more user friendly preview.

I added 6 more vertical lines to 1080 from 1074 , this took some time to get right .
This is a far as I can go vertically without changing CMOS[1] offset valve .
Also you no longer need to select ML "gray scale" (B/W) preview ,
Just select "Auto Preview" and magic lantern will do the rest  :))
You still need to press half shutter for color preview  but it acts just like non crop_rec .
HDMI should be still working .

FYI on Head Timers : if you adjust both 3 & 4 head timers to the same vertical raw size + the offset
you get more fps - e.g. on 5d2 in 5x zoom (1080+52-offset =1132) =>0x46c
this is still below the default of 0x476 & 0x49C which canon uses for 1074
seem there put too much overhead in there for some reason.

@ jpegmasterjesse , I know it works either way but just for safety reasons .
It is bleeding edge . ;)
   
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on February 02, 2019, 08:10:31 AM
@reddeercity - Could you please commit your local changes and push to the remote?

You are working on this repository, right?

https://bitbucket.org/reddeercity/magic-lantern_10-12bit_raw/commits/all
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on February 02, 2019, 06:47:44 PM
Quote from: dfort on February 02, 2019, 08:10:31 AM
@reddeercity - Could you please commit your local changes and push to the remote?

Great call, @dfort!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 02, 2019, 07:51:46 PM
I work locally only , so until I'm done .......
Read though my post and you will see all the changes
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on February 02, 2019, 11:14:14 PM
Open source projects like these are never "done."

The last time you posted your local repository was in August (https://www.magiclantern.fm/forum/index.php?topic=18443.msg205303#msg205303) and from the looks of it, you were working from a clone of my repository.

(https://farm5.staticflickr.com/4899/39998071683_42595a6fea_c.jpg) (https://flic.kr/p/23Wuzcz)

Hey, not a problem. I'm glad that you're building off some of the stuff that I pushed out there. If you want to continue working locally only, that's your choice. I'm just asking that you please reciprocate.

By the way, your binaries show the uncommitted changes in plain view. Opening your latest autoexec.bin in a text editor reveals this (I masked out your email address):

Magic Lantern crop_rec-4k-5D2-eXperimental.2019Feb01.5D2212
Camera   : 5D2
Firmware : 212
Changeset: 8bfbb0ca228e+ (crop_rec_4k_5D2) tip
Built on : 2019-02-02 04:40:34 by david@***

diff -r 8bfbb0ca228e src/raw.c
--- a/src/raw.c Sat Sep 08 11:26:34 2018 +0200
+++ b/src/raw.c Fri Feb 01 21:40:35 2019 -0700
@@ -32,6 +32,11 @@
#include "fps.h"
#include "platform/state-object.h"

+#ifdef CONFIG_EDMAC_RAW_PATCH
+#include "patch.h"
+#include "edmac.h"
+#endif
+
#undef RAW_DEBUG        /* define it to help with porting */
#undef RAW_DEBUG_DUMP   /* if you want to save the raw image buffer and the DNG from here */
#undef RAW_DEBUG_BLACK  /* for checking black level calibration */
@@ -221,7 +226,7 @@

#define SHAD_GAIN_REGISTER 0xC0F08030

-static int lv_raw_type = PREFERRED_RAW_TYPE;
+//static int lv_raw_type = PREFERRED_RAW_TYPE;//
static int lv_raw_gain = 0;

/**
@@ -1831,6 +1836,10 @@
     #endif
}

+#if defined(CONFIG_EDMAC_RAW_SLURP) || defined(CONFIG_EDMAC_RAW_PATCH)
+static int lv_raw_type = PREFERRED_RAW_TYPE;
+#endif
+
#ifdef CONFIG_EDMAC_RAW_SLURP

void FAST raw_lv_vsync()
@@ -1867,7 +1876,31 @@
     /* overriding the buffer is only valid for one frame */
     redirected_raw_buffer = 0;
}
+#endif

+#ifdef CONFIG_EDMAC_RAW_PATCH
+static void raw_lv_setedmac_patch(uint32_t* regs, uint32_t* stack, uint32_t pc)
+{
+    /* R0: EDMAC channel */
+    /* R1: output buffer */
+    /* R2: EDMAC info (geometry) */
+    /* R3: flags */
+
+   int width, height;
+   int ok = raw_lv_get_resolution(&width, &height);
+   if (ok)
+   {
+      /* update EDMAC image size */
+      int pitch = width * raw_info.bits_per_pixel / 8;
+      static struct edmac_info dst_edmac_info;
+      dst_edmac_info.xb = pitch;
+      dst_edmac_info.yb = height - 1;
+      regs[2] = (uint32_t) &dst_edmac_info;
+
+     /* we can override this here */
+     EngDrvOut(RAW_TYPE_REGISTER, lv_raw_type);
+  }
+}
/* integer gain used to fix the image darkening caused by lv_raw_gain */
/* this gain must not (!) change the raw data */
int _raw_lv_get_iso_post_gain()
@@ -2158,6 +2191,12 @@
}

#ifdef CONFIG_RAW_LIVEVIEW
+
+#ifdef CONFIG_EDMAC_RAW_PATCH
+extern thunk StartImagePass_x1_SetEDmac;
+extern thunk StartImagePass_x5_SetEDmac;
+#endif
+
static void raw_lv_enable()
{
     /* make sure LiveView is fully started before enabling the raw flag */
@@ -2168,6 +2207,10 @@

#ifndef CONFIG_EDMAC_RAW_SLURP
     call("lv_save_raw", 1);
+#ifdef CONFIG_EDMAC_RAW_PATCH
+   patch_hook_function((uint32_t) &StartImagePass_x1_SetEDmac, 0xE3A03202, raw_lv_setedmac_patch, "RAW LV x1");
+   patch_hook_function((uint32_t) &StartImagePass_x5_SetEDmac, 0xE3A03202, raw_lv_setedmac_patch, "RAW LV x5");
+#endif
#endif

#ifdef DEFAULT_RAW_BUFFER
@@ -2210,6 +2253,10 @@

#ifndef CONFIG_EDMAC_RAW_SLURP
     call("lv_save_raw", 0);
+#ifdef CONFIG_EDMAC_RAW_PATCH
+   unpatch_memory((uint32_t) &StartImagePass_x1_SetEDmac);
+   unpatch_memory((uint32_t) &StartImagePass_x5_SetEDmac);
+#endif
#endif

#ifdef CONFIG_ALLOCATE_RAW_LV_BUFFER
diff -r 8bfbb0ca228e platform/5D2.212/internals.h
--- a/platform/5D2.212/internals.h Sat Sep 08 11:26:34 2018 +0200
+++ b/platform/5D2.212/internals.h Fri Feb 01 21:40:35 2019 -0700
@@ -131,6 +131,9 @@
#define CONFIG_RAW_DISABLE_IN_10X_ZOOM
#define CONFIG_RAW_DISABLE_IN_10X_ZOOM_WEAK

+/** experimental - patch raw buffer parameters in Canon code */
+#define CONFIG_EDMAC_RAW_PATCH
+
/** Use joystick for one-finger menu navigation */
#define CONFIG_LONG_PRESS_JOYSTICK_MENU

diff -r 8bfbb0ca228e platform/5D2.212/stubs.S
--- a/platform/5D2.212/stubs.S Sat Sep 08 11:26:34 2018 +0200
+++ b/platform/5D2.212/stubs.S Fri Feb 01 21:40:35 2019 -0700
@@ -222,6 +222,10 @@
NSTUB(0xFF9D8250,  DispSensorStart)
NSTUB(0xFF9B1BE0,  LightMeasure_n_Callback_r0)

+/** LiveView RAW patches **/
+NSTUB(0xFFA08008, StartImagePass_x1_SetEDmac)               /* right before the SetEDmac call from StartImagePass_x1/x5 CrawAddr / KindOfCraw */
+NSTUB(0xFFA08B24, StartImagePass_x5_SetEDmac)               /* FIXME: why it fails when the hook is placed on the BL instruction?! */
+
/** Making the card bootable **/
NSTUB(   0x20890,  cf_device)
NSTUB(   0x208D8,  sd_device)


Running the module_hginfo_dump.sh script on your compiled modules also show your uncommitted changes:

./module_hginfo_dump.sh /Users/rosiefort/Downloads/magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212/ML/modules/crop_rec.mo
Name        : Crop mode recording
Author      : a1ex
License     : GPL
Summary     : Turn the 1080p and 720p video modes into 1:1 sensor crop modes
Description : This alters the 1080p and 720p video modes, transforming them
              into 3x (1:1) crop modes, by tweaking the sensor registers.
               [...]
Last update : 8bfbb0c on 2018-09-08 09:26:34 UTC by waza57:
              now we can change the ISO settings in crop x5
Build date  : 2019-01-27 21:31:15 UTC
Build user  : david@***

modules/crop_rec/Makefile
modules/crop_rec/README.rst
modules/crop_rec/crop_rec.c
diff -r 8bfbb0ca228e modules/crop_rec/crop_rec.c
--- a/modules/crop_rec/crop_rec.c
+++ b/modules/crop_rec/crop_rec.c
@@ -187,10 +187,10 @@
static uint32_t MEM_ADTG_WRITE  = 0;
static uint32_t ENGIO_WRITE     = 0;
static uint32_t MEM_ENGIO_WRITE = 0;
-static uint32_t RAWROMHEIGHT_WRITE = 0;
-static uint32_t RAWROMWIDTH_WRITE = 0;
-static uint32_t MEM_RAWROMHEIGHT = 0;
-static uint32_t MEM_RAWROMWIDTH = 0;         
+//static uint32_t RAWROMHEIGHT_WRITE = 0;
+//static uint32_t RAWROMWIDTH_WRITE = 0;
+//static uint32_t MEM_RAWROMHEIGHT = 0;
+//static uint32_t MEM_RAWROMWIDTH = 0;         

/* from SENSOR_TIMING_TABLE (fps-engio.c) or FPS override submenu */
static int fps_main_clock = 0;
@@ -343,10 +343,10 @@
         case CROP_PRESET_CENTER_Z:
               if (is_5D2)                       
             {
-              skip_left       = 264;    //265 269bad value create "black meean too..." and bug in hack suite
-              skip_right      = 44;
-              skip_top        = 54;      // waza57 pourquoi 54 pour avoir une image correcte alors que dmspy que 2
-              skip_bottom     = 66;        //52
+              skip_left       = 160;    //265 269bad value create "black meean too..." and bug in hack suite
+              skip_right      = 0;
+              skip_top        = 52;      // waza57 pourquoi 54 pour avoir une image correcte alors que dmspy que 2
+              skip_bottom     = 0;        //52
             }
             
            break;
@@ -591,7 +591,7 @@
                 }
                 if (is_5D2)
                 {
-                cmos_new[1] = PACK12(42,57);//0xBB3;    /* pink highlights without this */
+                cmos_new[1] = PACK12(42,57);  //0xBB3; /* pink highlights without this */
                 cmos_new[2] = 0x10E;    /* read every column, centered crop */     //waza57 cmos 6
                 break;
                 }
@@ -685,10 +685,10 @@
             /* raw buffer centered in zoom mode */
             case CROP_PRESET_CENTER_Z:
                 cmos_new[1] = (!is_5D2)
-                  ? PACK12(9+2,42+1) /* vertical (first|last) */
+                  ?  PACK12(9+2,42+1) /* vertical (first|last) */
                   : -1;
                 cmos_new[2] = (!is_5D2)
-                ? 0x09E            /* horizontal offset (mask 0xFF0) */
+                ? 0x9E            /* horizontal offset (mask 0xFF0) */
                 : -1;
                 break;
         }
@@ -1510,8 +1510,8 @@
     /* attempt to reconfigure the x5 zoom at the FPS selected in Canon menu */
     if (is_5D2)
     {
-     int timerA = 804 ;         //waza57 good values for 23.976: 804- 1245  and 807 - 1237
-     int timerB = 1245 ;     
+     int timerA = 834 ;         //waza57 good values for 23.976: 804- 1245  and 807 - 1237
+     int timerB = 1200 ;     
      int a = reg_override_fps_nocheck(reg, timerA, timerB, old_val);
         if (a) return a;
            switch (reg)
@@ -1520,11 +1520,32 @@
                 /* X: (3072+140)/8 + 0x17, adjusted for 3072 in raw_rec */
                 
                  case 0xC0F06084:
-                    return (old_val & 0x0000000)  ;
+                    return (old_val & 0x0000000)  + 0x30036 ;
                 
                 case 0xC0F06088:
                     //return (old_val & 0xFFFF0000) + 0x630 ;
-                    return (old_val & 0x00000000) + 0x4b0063c;   //0x4b00648 or origin value chang nothing               
+                    return (old_val & 0x00000000) + 0x46F066A;   //0x4b00648 or origin value chang nothing
+
+                   /* HEAD3 timer */
+                   /* 467 in 30p */
+                case 0xc0f0713c:
+                   return 0x46C ; 
+
+                   /* HEAD4 timer */
+                   /* 49C in 30p */
+                case 0xC0F07150:
+                   return 0x46C ;
+
+                case 0xC0F08184:
+                   return 0x46C ;
+
+                case 0xC0F08188:
+                   return 0xC67 ;
+
+                case 0xC0F08518:
+                   return 0x46C0C67 ;
+
+
             }
     }
     else
@@ -1640,25 +1661,25 @@

static int patch_active = 0;
static int lv_dirty = 1; //waza57 changes to effective update for 5d2
-static void update_patch(int ROMhack) 
+static void update_patch() 
{
     if (CROP_PRESET_MENU && !patch_active)
     {
         /* update preset */
         crop_preset = CROP_PRESET_MENU;
-        if (is_5D2 && crop_preset == CROP_PRESET_CENTER_Z )       // need to patch ROM for increase height for 5D2
-        {
-            patch_memory(RAWROMWIDTH_WRITE, MEM_RAWROMWIDTH, 0x00001C70, "crop_rec: rawromwidth hook");                                                                                                                   
-            patch_memory(RAWROMHEIGHT_WRITE, MEM_RAWROMHEIGHT, 0x00000767, "crop_rec: rawromwidth hook");             
-            if (ROMhack)
-              {
-                 patch_active = 1;
-                 return;                                                                                     
-              }
-        }                                                                               
+        //if (is_5D2 && crop_preset == CROP_PRESET_CENTER_Z )       // need to patch ROM for increase height for 5D2
+       // {
+            //patch_memory(RAWROMWIDTH_WRITE, MEM_RAWROMWIDTH, 0x00001C70, "crop_rec: rawromwidth hook");                                                                                                                   
+            //patch_memory(RAWROMHEIGHT_WRITE, MEM_RAWROMHEIGHT, 0x00000767, "crop_rec: rawromwidth hook");             
+            //if (ROMhack)
+              //{
+                 //patch_active = 1;
+                 //return;                                                                                     
+              //}
+        //}                                                                               
         /* install our hooks, if we haven't already do so */
-      //  if (!patch_active)   waza57 changes to effective update for 5d2
-      //  {
+         if (!patch_active)   //waza57 changes to effective update for 5d2
+        {
             patch_hook_function(CMOS_WRITE, MEM_CMOS_WRITE, &cmos_hook, "crop_rec: CMOS[1,2,6] parameters hook");
             patch_hook_function(ADTG_WRITE, MEM_ADTG_WRITE, &adtg_hook, "crop_rec: ADTG[8000,8806] parameters hook");
             if (ENGIO_WRITE)
@@ -1666,8 +1687,8 @@
                 patch_hook_function(ENGIO_WRITE, MEM_ENGIO_WRITE, engio_write_hook, "crop_rec: video timers hook");
             }
             patch_active = 1;
-            //return ; //waza57 present in my version not in  standard
-      //  }
+            return ; //waza57 present in my version not in  standard
+        }
       
     }
     else
@@ -1675,16 +1696,16 @@
         /* undo active patches, if any */
         if (patch_active)
         {
-           if (is_5D2 && crop_preset == CROP_PRESET_CENTER_Z)
-            {
-              unpatch_memory(RAWROMHEIGHT_WRITE);           
-              unpatch_memory (RAWROMWIDTH_WRITE);
-              if (ROMhack)
-              {   
-                  patch_active = 0;
-                  return;
-              }
-            }
+           //if (is_5D2 && crop_preset == CROP_PRESET_CENTER_Z)
+            //{
+              //unpatch_memory(RAWROMHEIGHT_WRITE);           
+              //unpatch_memory (RAWROMWIDTH_WRITE);
+              //if (ROMhack)
+              //{   
+                  //patch_active = 0;
+                  //return;
+              //}
+            //}
             unpatch_memory(CMOS_WRITE);
             unpatch_memory(ADTG_WRITE);
             if (ENGIO_WRITE)
@@ -2310,11 +2331,11 @@
         ENGIO_WRITE = 0xFF9A5618;
         MEM_ENGIO_WRITE = 0xE92D407C;
         
-        RAWROMHEIGHT_WRITE = 0xFFCAA1F4;
-        MEM_RAWROMHEIGHT = 0x00000467;
+       // RAWROMHEIGHT_WRITE = 0xFFCAA1F4;
+        //MEM_RAWROMHEIGHT = 0x00000467;
         
-        RAWROMWIDTH_WRITE = 0xFFCAA1EC;   // waza57 patch ROM for more height
-        MEM_RAWROMWIDTH = 0x00000FCE;
+        //RAWROMWIDTH_WRITE = 0xFFCAA1EC;   // waza57 patch ROM for more height
+        //MEM_RAWROMWIDTH = 0x00000FCE;
         is_5D2 = 1;
         
         crop_presets                = crop_presets_5d2;


Hum--looks like you are working off of the repository that waza57 updated back in September (https://bitbucket.org/waza57/magic-lantern/commits/all). So using the tools built into ML we can reverse engineer your local repository--or you can do us a big favor and:

hg commit
hg push


You are doing some great work that will benefit all Digic IV ML users. Keep it up!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on February 02, 2019, 11:54:12 PM
dfort is right

I was also going to say you should make the source code easily accessible. So we don't have to add changes by looking trough your posts. What you're doing may also be breaking the GPL license, it states that you should give someone the code if you give them the software. But it's not the end of the world yet.

At least upload the source in a zip! if you really don't want to mess around with installing mercurial, which is very easy btw. Would make us all a lot happier and I don't mean this to sound mean with this post :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 03, 2019, 03:06:43 AM
If any of you would look at my downloads page you will see the source code I use , It been there from the start !
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Crop_Rec-4k-5d2-50d-magic-lantern_1-9-2019.zip
I use waza57 source (and in my posts I state this , if anyone would have cared to read a few)
I have stated many times I work locally and don't work online as I don't know anything about bitbucket , of which I stated
many times also , sorry my work method are unconventional but that's the way I work & is the most comfortable for me .

So if it makes people feel better , I won't post code until I'm done all the presets in a working crop_rec module
Quote from: dfort on February 02, 2019, 11:14:14 PM
Open source projects like these are never "done."
Well for me I'm almost "done" , after cf card over clocking & lossless I have no more goals .

Edit: updated first Post (https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476) with the new 3008x1080 23.98fps build & details
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on February 03, 2019, 03:19:04 AM
after you're done with the cf overclocking and lossless, it's time to grab a 5d4!!!

please!! :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 03, 2019, 06:15:01 AM
Fix the black level problem with 48fps in 3x3 1.67 squeezes (1856x688->1856x1146)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on February 03, 2019, 08:55:58 AM
Quote from: 70MM13 on February 03, 2019, 03:19:04 AM
...time to grab a 5d4!!!

And a beer! I'd suggest a strong Canadian brew like Unibroue La Fin du Monde.

Quote from: reddeercity on February 03, 2019, 03:06:43 AM
So if it makes people feel better , I won't post code until I'm done all the presets in a working crop_rec module Well for me I'm almost "done" , after cf card over clocking & lossless I have no more goals .

Doesn't make me feel any better and having no more goals only happens when you're dead.

In any case, keep at it in whatever way works best for you. Just for fun, I downloaded that repository (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Crop_Rec-4k-5d2-50d-magic-lantern_1-9-2019.zip) that you said was there from the start and it has uncommitted changes. Yep, you cloned the waza57 repository (https://bitbucket.org/waza57/magic-lantern/src/unified/) and made these changes on the crop_rec_4k_5D2 branch (https://bitbucket.org/waza57/magic-lantern/branch/crop_rec_4k_5D2):

diff --git a/Makefile.user.default b/Makefile.user.default
--- a/Makefile.user.default
+++ b/Makefile.user.default
@@ -202,7 +202,7 @@
# magiclantern-0.2.0.rc1.550d.fw109.zip
#~ VERSION=0.2.0.rc1.550d.fw109
BUILDVER=$(shell whoami).$(shell hg id -i -r .)
-VERSION = $(call eval_once,VERSION,Nightly.$(shell LC_TIME=EN date +'%Y%b%d').$(MODEL)$(FW_VERSION))
+VERSION = $(call eval_once,VERSION,crop_rec-4k-5D2-eXperimental.$(shell LC_TIME=EN date +'%Y%b%d').$(MODEL)$(FW_VERSION))

# Build configuration - untested and debug stuff handy for devs
# these can be specified either in Makefile.user, or in the command line:
diff --git a/modules/Makefile.modules.default b/modules/Makefile.modules.default
--- a/modules/Makefile.modules.default
+++ b/modules/Makefile.modules.default
@@ -19,4 +19,6 @@
adv_int \
crop_rec \
edmac \

\ No newline at end of file
+        adtg_gui \
+        cf_acc \

diff --git a/modules/mlv_rec/mlv_rec.c b/modules/mlv_rec/mlv_rec.c
--- a/modules/mlv_rec/mlv_rec.c
+++ b/modules/mlv_rec/mlv_rec.c
@@ -133,8 +133,8 @@
  * => if my math is not broken, this traslates to resolution being multiple of 32 pixels horizontally
  * use roughly 10% increments
  **/
-static uint32_t resolution_presets_x[] = {  640,  960,  1280,  1600,  1920,  2240,  2560,  2880,  3200,  3520 };
-#define  RESOLUTION_CHOICES_X CHOICES(     "640","960","1280","1600","1920","2240","2560","2880","3200","3520")
+static uint32_t resolution_presets_x[] = {  640,  960,  1280,  1600,  1920,  2240,  2560,  2880,  3008,  3200,  3520,  3840,  4096,  5632 };
+#define  RESOLUTION_CHOICES_X CHOICES(     "640","960","1280","1600","1920","2240","2560","2880","3008","3200","3520","3840","4096","5632")

static uint32_t aspect_ratio_presets_num[]      = {   5,    4,    3,       8,      25,     239,     235,      22,    2,     185,     16,    5,    3,    4,    12,    1175,    1,    1 };
static uint32_t aspect_ratio_presets_den[]      = {   1,    1,    1,       3,      10,     100,     100,      10,    1,     100,      9,    3,    2,    3,    10,    1000,    1,    2 };
diff --git a/platform/50D.109/internals.h b/platform/50D.109/internals.h
--- a/platform/50D.109/internals.h
+++ b/platform/50D.109/internals.h
@@ -138,6 +138,9 @@
#define CONFIG_RAW_DISABLE_IN_10X_ZOOM
#define CONFIG_RAW_DISABLE_IN_10X_ZOOM_WEAK

+/** experimental - patch raw buffer parameters in Canon code */
+#define CONFIG_EDMAC_RAW_PATCH
+
/** Use joystick for one-finger menu navigation */
#define CONFIG_LONG_PRESS_JOYSTICK_MENU

diff --git a/platform/50D.109/stubs.S b/platform/50D.109/stubs.S
--- a/platform/50D.109/stubs.S
+++ b/platform/50D.109/stubs.S
@@ -231,6 +231,10 @@
NSTUB(0xFF9ADD28,  DispSensorStart)                         // d
// NSTUB(    ???,  LightMeasure_n_Callback_r0)              /* present on 7D.203, 5D2.212 */

+/** LiveView RAW patches **/
+NSTUB(0xff9db254, StartImagePass_x1_SetEDmac)               /* right before the SetEDmac call from StartImagePass_x1/x5 CrawAddr / KindOfCraw */
+NSTUB(0xff9dbce8, StartImagePass_x5_SetEDmac)               /* FIXME: why it fails when the hook is placed on the BL instruction?! */
+
/** Making the card bootable **/
NSTUB(   0x1D6FC,  cf_device)                               // in cfReadBlk
NSTUB(   0x1D744,  sd_device)                               // in sdReadBlk
diff --git a/platform/5D2.212/internals.h b/platform/5D2.212/internals.h
--- a/platform/5D2.212/internals.h
+++ b/platform/5D2.212/internals.h
@@ -131,6 +131,9 @@
#define CONFIG_RAW_DISABLE_IN_10X_ZOOM
#define CONFIG_RAW_DISABLE_IN_10X_ZOOM_WEAK

+/** experimental - patch raw buffer parameters in Canon code */
+#define CONFIG_EDMAC_RAW_PATCH
+
/** Use joystick for one-finger menu navigation */
#define CONFIG_LONG_PRESS_JOYSTICK_MENU

diff --git a/platform/5D2.212/stubs.S b/platform/5D2.212/stubs.S
--- a/platform/5D2.212/stubs.S
+++ b/platform/5D2.212/stubs.S
@@ -222,6 +222,10 @@
NSTUB(0xFF9D8250,  DispSensorStart)
NSTUB(0xFF9B1BE0,  LightMeasure_n_Callback_r0)

+/** LiveView RAW patches **/
+NSTUB(0xFFA08008, StartImagePass_x1_SetEDmac)               /* right before the SetEDmac call from StartImagePass_x1/x5 CrawAddr / KindOfCraw */
+NSTUB(0xFFA08B24, StartImagePass_x5_SetEDmac)               /* FIXME: why it fails when the hook is placed on the BL instruction?! */
+
/** Making the card bootable **/
NSTUB(   0x20890,  cf_device)
NSTUB(   0x208D8,  sd_device)
diff --git a/src/backtrace.c b/src/backtrace.c
--- a/src/backtrace.c
+++ b/src/backtrace.c
@@ -12,7 +12,7 @@
#include "tasks.h"
#include "backtrace.h"

-#define qemu_log_mask(...)
+#define qemu_log_mask(level, fmt, ...) { qprintf(fmt, ## __VA_ARGS__); }
#define qemu_loglevel_mask(x) 0
#define EOSState void

diff --git a/src/raw.c b/src/raw.c
--- a/src/raw.c
+++ b/src/raw.c
@@ -32,6 +32,11 @@
#include "fps.h"
#include "platform/state-object.h"

+#ifdef CONFIG_EDMAC_RAW_PATCH
+#include "patch.h"
+#include "edmac.h"
+#endif
+
#undef RAW_DEBUG        /* define it to help with porting */
#undef RAW_DEBUG_DUMP   /* if you want to save the raw image buffer and the DNG from here */
#undef RAW_DEBUG_BLACK  /* for checking black level calibration */
@@ -221,7 +226,7 @@

#define SHAD_GAIN_REGISTER 0xC0F08030

-static int lv_raw_type = PREFERRED_RAW_TYPE;
+//static int lv_raw_type = PREFERRED_RAW_TYPE;//
static int lv_raw_gain = 0;

/**
@@ -1831,6 +1836,10 @@
     #endif
}

+#if defined(CONFIG_EDMAC_RAW_SLURP) || defined(CONFIG_EDMAC_RAW_PATCH)
+static int lv_raw_type = PREFERRED_RAW_TYPE;
+#endif
+
#ifdef CONFIG_EDMAC_RAW_SLURP

void FAST raw_lv_vsync()
@@ -1867,7 +1876,31 @@
     /* overriding the buffer is only valid for one frame */
     redirected_raw_buffer = 0;
}
+#endif

+#ifdef CONFIG_EDMAC_RAW_PATCH
+static void raw_lv_setedmac_patch(uint32_t* regs, uint32_t* stack, uint32_t pc)
+{
+    /* R0: EDMAC channel */
+    /* R1: output buffer */
+    /* R2: EDMAC info (geometry) */
+    /* R3: flags */
+
+   int width, height;
+   int ok = raw_lv_get_resolution(&width, &height);
+   if (ok)
+   {
+      /* update EDMAC image size */
+      int pitch = width * raw_info.bits_per_pixel / 8;
+      static struct edmac_info dst_edmac_info;
+      dst_edmac_info.xb = pitch;
+      dst_edmac_info.yb = height - 1;
+      regs[2] = (uint32_t) &dst_edmac_info;
+
+     /* we can override this here */
+     EngDrvOut(RAW_TYPE_REGISTER, lv_raw_type);
+  }
+}
/* integer gain used to fix the image darkening caused by lv_raw_gain */
/* this gain must not (!) change the raw data */
int _raw_lv_get_iso_post_gain()
@@ -2158,6 +2191,12 @@
}

#ifdef CONFIG_RAW_LIVEVIEW
+
+#ifdef CONFIG_EDMAC_RAW_PATCH
+extern thunk StartImagePass_x1_SetEDmac;
+extern thunk StartImagePass_x5_SetEDmac;
+#endif
+
static void raw_lv_enable()
{
     /* make sure LiveView is fully started before enabling the raw flag */
@@ -2168,6 +2207,10 @@

#ifndef CONFIG_EDMAC_RAW_SLURP
     call("lv_save_raw", 1);
+#ifdef CONFIG_EDMAC_RAW_PATCH
+   patch_hook_function((uint32_t) &StartImagePass_x1_SetEDmac, 0xE3A03202, raw_lv_setedmac_patch, "RAW LV x1");
+   patch_hook_function((uint32_t) &StartImagePass_x5_SetEDmac, 0xE3A03202, raw_lv_setedmac_patch, "RAW LV x5");
+#endif
#endif

#ifdef DEFAULT_RAW_BUFFER
@@ -2210,6 +2253,10 @@

#ifndef CONFIG_EDMAC_RAW_SLURP
     call("lv_save_raw", 0);
+#ifdef CONFIG_EDMAC_RAW_PATCH
+   unpatch_memory((uint32_t) &StartImagePass_x1_SetEDmac);
+   unpatch_memory((uint32_t) &StartImagePass_x5_SetEDmac);
+#endif
#endif

#ifdef CONFIG_ALLOCATE_RAW_LV_BUFFER


BTW--I'm just sharing what I discovered about some tools included in the ML source code to look into ML binaries. I think it is possible to reconstruct your local repository provided you keep working the way you're working--and keep publishing test builds. Eventually you really should create a fork and make a pull request so you can get credit for your work (https://bitbucket.org/waza57/magic-lantern/annotate/8bfbb0ca228e11f091869df09a1b8f16dd36b508/modules/crop_rec/crop_rec.c?at=crop_rec_4k_5D2&fileviewer=file-view-default).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on February 03, 2019, 09:03:58 AM
haha, dfort  :-*
Anyway. The code ought to be continuously published, gpl is all about collaboration.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Audionut on February 03, 2019, 10:52:37 AM
This project looks to be dangerously close to tragic lantern (https://www.magiclantern.fm/forum/index.php?topic=11080.0).

Is there any sort of commit history?

A dump of all of your code changes into a single commit, is likely to never ever be checked or pushed into the mainline.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on February 03, 2019, 08:45:06 PM
Quote from: reddeercity on February 03, 2019, 03:06:43 AM
If any of you would look at my downloads page you will see the source code I use , It been there from the start !
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Crop_Rec-4k-5d2-50d-magic-lantern_1-9-2019.zip
I use waza57 source (and in my posts I state this , if anyone would have cared to read a few)
I have stated many times I work locally and don't work online as I don't know anything about bitbucket , of which I stated
many times also , sorry my work method are unconventional but that's the way I work & is the most comfortable for me .

This is good! But I downloaded the source you have uploaded(https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Crop_Rec-4k-5d2-50d-magic-lantern_1-9-2019.zip) and compiled it, it is only the 2880x1080 version. For any compiled version you release, you should also release the code specific to it. Could you upload a zip of the current 3008x1080 build's code please?

Quote from: reddeercity on February 03, 2019, 03:06:43 AM
So if it makes people feel better , I won't post code until I'm done all the presets in a working crop_rec module Well for me I'm almost "done" , after cf card over clocking & lossless I have no more goals .

Definitely keep posting code, especially if you are uploading a compiled build. If you really don't want to use mercurial (hg), please just upload a zip file of your code every now and then. It will really make the ML developers a lot happier.

Quote from: reddeercity on December 09, 2018, 09:32:07 AM
Ok , can you post your code please
I need to look at the code first before I try it ,
Don't what to brick my camera .

This
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on February 19, 2019, 09:25:46 PM
The new Preset works great, but...

Is it possible to switch the live view on and off, as the recording time is reduced by ~1/4? Or implement the old one too?

And is it possible to change the area of the live view before recording? -> left middle position is not perfect sometimes. You can change it with the joyspad (during the recording), but only limited. Too much -> error
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 20, 2019, 06:46:14 AM
Quote from: Bender@arsch on February 19, 2019, 09:25:46 PM
Is it possible to switch the live view on and off
No , that's how video works thought liveview . No liveview no video

Quote from: Bender@arsch on February 19, 2019, 09:25:46 PM

... is it possible to change the area of the live view before recording? ......
No , that's the way it's in crop_rec.mo
If you want to move the crop window around  you must use the ML default 3x crop_mode without the crop_rec.mo enabled .

Quote from: Bender@arsch on February 19, 2019, 09:25:46 PM
left middle position is not perfect sometimes.

That will be fixed in updated crop_rec.mo , to come very soon , just working out a few bugs right now.

Just some notes: for developers only
Figured out why/how to get canon liveview (real time)while recording raw with extended or reduced resolutions .
Simple ,  the final frame rate must be 23.976fps -- 23.98 , 24.004 or even below will give frozen liveview
I tested on a (1x3) Anamorphic 5k preset in 3xcrop I'm working on (1664x1842 @ 23.976fps = 4992x1842 )
If I chose canon liveview I get full canon color liveview in standbye or recording raw that's real time!
Plus this seems to fix HDMI , you no longer need to force HDMI -> VGA(480p)
Works now at 1080i , even when recording , but this does take resources from recording raw (less frames + the chance of corrupted frames)
I kept adjusting  A & B timer to fine the sweet spot , also noticed that at 23.976 the sensor run way cooler & liverview refresh is fast no delays
special panning the cam , everything just works smoother at 23.976.
I guess because the sensor runs at 24 MHz
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on February 20, 2019, 11:53:01 PM
Thank you so much, we are so waiting for it... Stupid question, but how to adjust that final framerate?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on February 21, 2019, 12:10:37 AM
Quote from: Igor_Braun on February 20, 2019, 11:53:01 PM
Thank you so much, we are so waiting for it... Stupid question, but how to adjust that final framerate?

FPS overrride is not working with these builds as the timers need to be at special values... but you can hack it:

1. go in to crop mode
2. go to fps override, but do not turn it on
3. read the timer A and B values in advanced section
4. turn fps override on, and set the A and B values to what you read in step 3
5. increase timer A to reduce fps
6. (you may have to leave live view and go back to get rid of artifacts created while adjusting timers)
7. done

This is assuming you're using 5D2 and the 3k crop preset, as that is what worked for me
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 21, 2019, 05:33:39 AM
Sorry those note where intended for developers not end users
I was thinking out loud
I would not recommend adjusting timers a & b in crop_rec
can cause soft brick , or other possible issue/damage .
I'll have to disable that in future builds for end users

   
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on February 21, 2019, 04:57:59 PM
Okay, Thank you so much Ilia, actually, it's very interesting for me... But I think now I won't touch the timer values. Just in case will follow the advice of Reddeercity, because my 5DII is the only camera I have and I still work using it. But thank you anyway.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on February 21, 2019, 06:19:33 PM
Timer values can be altered to match crop rec and put into fps-engio.c but I don´t see why. Better to just profile them for a certain preset. I alter timers all the time for my eosm. Sometimes a tweak can lead to less corruption and so on. The thing is. If code would have been continuously updated I could have put in those tools myself but I assume code is coming one of these days...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on March 03, 2019, 05:35:40 PM
@reddeercity - Thanks for your notes on the registers for the 7D (https://www.magiclantern.fm/forum/index.php?topic=5601.msg212812#msg212812). I've got things pretty much working on a merge of the raw_video_10bit_12bit_LVState and RscMgr_memory branches so I pulled in the Digic Poke (https://magiclantern.fandom.com/wiki/Register_Map/Brute_Force) changes (https://bitbucket.org/waza57/magic-lantern/commits/db30a11dca7b9e531d69a74e0f5ff9947fe05068?at=crop_rec_4k_5D2) @waza57 made and it seems to be working.

(https://farm8.staticflickr.com/7846/32325680587_90874d46a4.jpg) (https://flic.kr/p/RfvyCt)

The default is to adjust 0xc0f08008 so I take it that this is an interesting register?

  0xc0f00000 Register family
+ 0x00008000 Register base
+ 0x00000008 Register offset
  --------------
  0xc0f08008 Value


Could you please give us a simple example of how to use this tool using something that you have already tested on the 5D2? I have a feeling that much of your work on that camera will translate to other Digic 4 cameras.

Thanks!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on March 03, 2019, 06:15:25 PM
Quote from: dfort on March 03, 2019, 05:35:40 PM
Could you please give us a simple example of how to use this tool using something that you have already tested on the 5D2?

+1

Quote from: dfort on March 03, 2019, 05:35:40 PM
I have a feeling that much of your work on that camera will translate to other Digic 4 cameras.

Likewise. Big time!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 04, 2019, 01:22:49 AM
Quote from: dfort on March 03, 2019, 05:35:40 PM
.......
Could you please give us a simple example of how to use this tool using something that you have already tested on the 5D2? I have a feeling that much of your work on that camera will translate to other Digic 4 cameras.
Thanks!
Sure , have to dig it up & refresh my memory .

I'm slowly moving away from D4 (5d2) and moving in to D5 (5D3) for now (need the stable 4k raw for up coming projects)
So I'm actively looking (if any one knows of a good deal PM me).
If not then it's looking like a EOS R (4k 10bit canon log HDMI) or EOS RP(4k HDMI but in only 8bit & no Canon Log - $1000 cheaper)
I thought of a 5D4 but is so close to what the 5D3 can do internally and still it's only 1080p from the HDMI .
But I will keep my 5D2 for backup & further development's , still need to get Lossless & cf card bus overclocking working yet .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on March 06, 2019, 04:24:48 AM
Reddeercity, I understand you, but how sad it is... Do you still plan to finish your work with turning 5d2 into a UHD cinema camera and finalize your presets (including your awesome vertical squeezed preset) and lifeview?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 06, 2019, 05:24:20 AM
Quote from: Igor_Braun on March 06, 2019, 04:24:48 AM
Reddeercity, I understand you, but how sad it is...
Do you still plan to finish your work with turning 5d2 into a UHD cinema camera and finalize your presets
(including your awesome vertical squeezed preset) and lifeview?
Yes , don't worry  ;) all that plus a surprise or two

Quote from: Igor_Braun on March 06, 2019, 04:24:48 AM
....
(including your awesome vertical squeezed preset) and lifeview?
Oh ,  something farrrrr better ,
hint : has something to do with 5k  :P :P
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: honza123 on March 06, 2019, 10:48:45 AM
Great news !!
:)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on March 07, 2019, 03:31:58 AM
oh.... can't wait!!)) uh
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 07, 2019, 08:04:14 AM
A little preview of things to come ,
I just putting the finishing touches on 1x3 5k Anamorphic setting in 3xcrop_mode(5x zoom) (well not totally 5k, but  closer to 4.9K)  :D
Start with 1632x1828 @ 23.976 fps with less then 20us/line of rolling shutter , that's better FHD (around 24us/line)
Once you stretch the horizontal (3x) you get 4896x1828 @ 23.976 fps without any aliasing or moiré pattern  :o
That's not the best part ! I have real time liveview plus with corrected aspect ratio that not distorted !!

Before without any liveview A.R. Correction
(https://i.ibb.co/tJDKcH3/Small-No-N-Anamophic.png) (https://imgbb.com/)

After with liveview A.R. Correction
(https://i.ibb.co/myFq2z3/Small-Anamophic.png) (https://imgbb.com/)

How did I do this you may ask ?
Well I just used the Anamorphic correction that's all ready there in magic lantern under display settings
Thou I may update that to 3:1 , which is the real corrected A.R. but it's close enough for now (that really cool)
(https://i.ibb.co/QXYfcg2/Small-Anamophic-settings.png) (https://imgbb.com/)>

Shot a quick video on my iPhone , showing this option being enabled and recording raw video .
I posted it on my google drive IMG_2710.MOV (https://drive.google.com/file/d/16EnP5D3kbbY4pHQO5zoJG7BFmA7WI-HI/view?usp=sharing)
Thou the correction does not work while recording raw video , it does work in standby mode only , great for framing you shot .
Also HDMI work great too , but correction does not work though the HDMI just on the LCD screen .

I'll try and push this one though for a test crop_rec module for tester , so keep a look out for it.

@dfort , I haven't forgot about you and the digital poke on 7D , just trying to wrap things up on the 5D2 crop_rec .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on March 07, 2019, 07:55:37 PM
Eager to test this out! Fantastic work!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on March 09, 2019, 12:25:19 AM
o man.....

this is not normal any more :) :) :) :) :) :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on March 09, 2019, 12:34:30 AM
Really exciting,

But it should be compared to the original '3.5K' preset (3008x1080), to make sure it actually gives a more detailed image. I'm still uncertain about 3x squeeze.

Can we see some samples maybe?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on March 09, 2019, 05:30:29 PM
Quote from: Ilia3101 on March 09, 2019, 12:34:30 AM
Can we see some samples maybe?

+1
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 10, 2019, 02:14:08 AM
Ok , how this
(https://i.ibb.co/rmTS4ZM/1x3-M07-0007-000000-small-320x358.png) (https://imgbb.com/)
1x3_M07-0007_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1x3_M07-0007_000000_Small_color_correction.png)
1x3_M07-0007_000000.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1x3_M07-0007_000000.dng)

Exported Tiff from Adobe CS6 A.E.
(https://i.ibb.co/nDHByK3/4896x1828-M007-0007-00000-small-490x183.png) (https://imgbb.com/)>
4896x1828_M007-0007_00000.tif (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4896x1828_M007-0007_00000.tif)

The only way to process this is in Adobe After Effect , non of the app on the form do 3x horizontal resolution I've only seen 2x , I did make a  request on the mlv app thread
but got no response . Beside that I like ACR anyways , I just set the project size e.g. 4896x1828 then just stretch out the horizontal , done .
Exported a ProRes 422LT ,  217MB 4896x1828_23.976fps_ProRes_422LT_M07-0007.mov (https://drive.google.com/file/d/1CNOXcG0fCuHkirdslmpN8Ffux1HG5iD3/view?usp=sharing)

Also here a mp4  from A.E.  , A.E. will not export at 4896x1828 h264 instead it exports at 4096x1828 , so it looks like it just crops the side .
4096x1828_A.E._cropped_M07-0007.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4096x1828_A.E._cropped_M07-0007.mp4) 22MB
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on March 10, 2019, 02:50:00 AM
Thanks, actually looks much better than I expected! Can't say it's sharper than the 3008x1080 preset, but close, will be useful as it has less crop factor.

Quote from: reddeercity on March 10, 2019, 02:14:08 AM
The only way to process this is in Adobe After Effect , non of the app on the form do 3x horizontal resolution I've only seen 2x , I did make a  request on the mlv app thread
but got no response .

You did get a response...

Quote from: masc on February 10, 2019, 11:10:36 AM
But now back to your request: 3.0x horizontal stretch is there since v1.5. If the file has the right metadata the setup works automatically. If not, you have to setup height stretch factor to 0.33x. MLVApp always upsamples since v1.5, so a height stretch factor of 0.33x is realized as width stretch factor of 3.0x. In older versions you can do this manually for ffmpeg export by entering your desired output resolution (frame is stretched once only on export, so no quality loss). Maybe we should add a information label somewhere, which shows the resolution after stretching.

Basically do this: set the vertical stretch factor to 0.33x in Transformation section, and MLV App will upsample horizontally by 3x.


I will admit that is confusing. @masc could we make it nicer and have it as horizontal 3.0x? Or even allow the user to enter custom value?


Also an shortened MLV would be nice if you could? to test out in MLV App :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on March 10, 2019, 07:06:49 AM
How large is the crop factor? I love the high resolution 5x stuff but the closer we can get to full frame with higher rez is obviously desirable.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on March 10, 2019, 07:49:18 AM
Quote from: Ilia3101 on March 10, 2019, 02:50:00 AM
I will admit that is confusing. @masc could we make it nicer and have it as horizontal 3.0x? Or even allow the user to enter custom value?

Agreed. Good call.


Quote from: Ilia3101 on March 10, 2019, 02:50:00 AM
Also an shortened MLV would be nice if you could? to test out in MLV App :)

+1
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 10, 2019, 11:14:28 AM
Quote from: Ilia3101 on March 10, 2019, 02:50:00 AM
You did get a response...
Correct. (Works officially since december 2017 in MLVApp (v0.12), when entering resolution manually.)
Quote from: Ilia3101 on March 10, 2019, 02:50:00 AM
I will admit that is confusing. @masc could we make it nicer and have it as horizontal 3.0x? Or even allow the user to enter custom value?
I already explained that somewhere else: this is close to impossible because we automate correct stretching in dependency to metadata. Now we set one UI element in dependency to this metadata. Doing it to horizontal stretching would make it far more difficult without advantage. Quite the contrary: when using 1x3 with horizontal stretching + anamorphic lens, it becomes impossible to stretch.
If I disable automation completely - no problem. But it won't be easier.
If it is easier for you, I could name the elements different: "anamorphic lens correction" (=horizontal stretching) and "sensor readout correction" or something like that (=vertical stretching), with 1x1 -> 1x1, 3x1 -> 1x1, 5x3 -> 1x1 and 1x3 -> 1x1.
On the other side, since a very long time in export options you can resize the export to any resolution without quality loss.

Edit:
I now implemented a label which shows what you'll get.
(https://i.ibb.co/NYtJ5t3/Bildschirmfoto-2019-03-10-um-11-39-30.png)

Tip: if you set metadata in camera correctly, MLVApp shows the right AR automatically, and you also can fast export to DNG with correct AR. Adobe programs show correct AR out of the box then.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on March 10, 2019, 01:38:19 PM
Quote from: masc on March 10, 2019, 11:14:28 AM
Edit:
I now implemented a label which shows what you'll get.
(https://i.ibb.co/NYtJ5t3/Bildschirmfoto-2019-03-10-um-11-39-30.png)

Tip: if you set metadata in camera correctly, MLVApp shows the right AR automatically, and you also can fast export to DNG with correct AR. Adobe programs show correct AR out of the box then.

Thanks! I like this

Quote from: jpegmasterjesse on March 10, 2019, 07:06:49 AM
How large is the crop factor? I love the high resolution 5x stuff but the closer we can get to full frame with higher rez is obviously desirable.

Divide 5616 (sensor width) by the video width, for example: 5616/3008 = 1.86x crop

@reddeercity, I know you have been increasing the width mostly, but would it be possible to increase the height and ignore width? something like a 2000x1500 preset for example, instead of 3008x1080. It would be nice for anamorphic lenses or people who just enjoy aspect ratios.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 11, 2019, 04:54:45 AM
@ Ilia3101 & masc
Just tested mlv app V1.5 on windows , seem to work now , but I sure it didn't work before
that why ask on your thread . I don't get the same Info that you posted @ masc  when I select 0.33x .
That not on my windows version and I guess that the confusion right there , If I would have seen the
pixel stretch final resolution , I would have used mlv app .
Quote from: masc on March 10, 2019, 11:14:28 AM
Tip: if you set metadata in camera correctly, MLVApp shows the right AR automatically, and you also can fast export to DNG with correct AR. Adobe programs show correct AR out of the box then.
No it doesn't , at least on windows -- I can't get it to export more then 1 dng (only works with export frame) "fast export to DNG" does nothing
and the dng that's does export with "single frame export with 0.33x set is only 1632x609 not 4896x1828 , that only
happens with compressed file format like "PNG" or "H264" etc. ..... no pass though cdng's to 4.9k etc. ....

Even MLVFS with Quick Mount  ( which I use 99%  in After Effect & Resolve) doesn't show corrected a.r.
I guess as long as I have 16bit tiff's I'm happy , would like to have the right sized cdng (4896x1828) to work with .
Maybe mlv_dump can export them correctly never looked in to it yet .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on March 11, 2019, 06:58:26 AM
Can we expect to see a build to test soon? Are there any specific kinks you're still working out?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 11, 2019, 10:49:31 AM
@reddeercity:
The MLV and DNG code is identical on Win/Linux/OSX. But I tested on Windows now. Unfortunately I can't find a bug. I tried with a 1x3 MLV file from EOS-M (which has correct stretching metadata) and it works (AR is correct in viewer and in all possible DNG (single compressed/uncompressed, batch uncompressed, compressed, fast) / PNG exports without changing one single parameter in MLVApp).
https://bitbucket.org/Dannephoto/magic-lantern/downloads/M19-1215.MLV
I opened the DNGs / PNGs in LR6.14 for testing. In my case I got always 3552x2000 from originally 1184x2000. You'll get something else if you override resolution by enabling "resize" box in export settings.

How did you test? Would be interesting - maybe somewhere are still bugs (I am sure there are).
If you like to see the new resolution label from my screenshot you should compile the latest commit - just pushed the code to the repos for now. But all the resizing should work in v1.5 as described.

The metadata which is used for correct AR are:
RAWC.binning_x, RAWC.skipping_x, RAWC.binning_y, RAWC.skipping_y

Somehow Resolve can't interpret stretching metadata - so here you'll get wrong AR out of the box.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: OlRivrRat on March 12, 2019, 06:04:31 AM
                     jpegmasterjesse

       If You Read the last 100 or so replies, I suspect You will find what You are Wanting ~
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on March 12, 2019, 01:37:51 PM
masc,
I have had the occasional mlv from my 5d3 that doesn't get automagically stretched.  It may have been only once.  I just recall it happening about a week or two ago.
Unfortunately I don't have the mlv's, as it was during my ongoing testing, and those files get ditched when I format the card.
I'll keep an eye out for it and if it happens again I'll send you the file...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on March 12, 2019, 02:03:03 PM
Quote from: masc on March 11, 2019, 10:49:31 AM
@reddeercity:
How did you test? Would be interesting - maybe somewhere are still bugs (I am sure there are).
If you like to see the new resolution label from my screenshot you should compile the latest commit - just pushed the code to the repos for now. But all the resizing should work in v1.5 as described.
No bug. metadata has to be set in code, so where´s the code?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on March 12, 2019, 10:41:49 PM
Quote from: Danne on March 12, 2019, 02:03:03 PM
No bug. metadata has to be set in code, so where´s the code?
+1

@reddeercity if you released code when you come up with more changes, all of your presets would be added to the menu by now, and the camera would set stretch metadata correctly. Danne is very keen on helping.

I appreciate that you have the original code uploaded, but it's important to have modified newer versions too.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 14, 2019, 04:04:41 AM
There's no new code yet , I'm doing everything though adtg_gui_.mo
Now that should clear up things about code etc. ....
and the 1x3 files display correctly in mlv app when exported to compressed format
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on March 14, 2019, 09:51:49 PM
Quote from: reddeercity on March 14, 2019, 04:04:41 AM
There's no new code yet , I'm doing everything though adtg_gui_.mo
Now that should clear up things about code etc. ....

Thanks it does clear stuff up. Then could we just have the code for the build that can do 3008x1080 resolution? Last time I checked, the uploaded code was the 2880x1080 version.

Quote from: reddeercity on March 14, 2019, 04:04:41 AM
and the 1x3 files display correctly in mlv app when exported to compressed format

Good to hear it works.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 14, 2019, 10:02:22 PM
Quote from: Ilia3101 on March 14, 2019, 09:51:49 PM
Then could we just have the code for the build that can do 3008x1080 resolution?
Have you tried this one from first post? https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on March 16, 2019, 01:06:33 AM
confirmed here, uncompressed doesn't display correct aspect ratio automagically in mlvapp.  lossless does.
i remember now how it happened here.  i was troubleshooting a shot a while back and tried uncompressed.  i was focused on the issue and didn't pay much attention to the whole stretching thing...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 16, 2019, 09:52:59 AM
I don't think it has something to do with compression, because these metadata variables are always there. If they are used by ML in camera, MLVApp can use them as well, if not then not. But if you have such a file I could have a look.

Edit: quick test with EOS-M footage 10bit lossless vs. uncompressed: no difference in those parameters and behaviour.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on March 16, 2019, 01:14:48 PM
i'll PM you the file...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: kwstas on March 16, 2019, 02:02:44 PM
Hi reddeercity and guys,

I've made some quick test with experimental release. Followed the steps from p1. I'm facing these issues.

- 5x mode is magenta, when recording turns grayscale, yet crop factor is irrelevant to what it records
- when pressing to 10x mode camera freezes

And a hypothetical scenario.
Just wondering if it possible when 5x mode pressed the viewfinder remains in 1x with crop lines displayed based on chosen ratio.

cheers,
K.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 16, 2019, 07:30:13 PM
Quote from: 70MM13 on March 16, 2019, 01:14:48 PM
i'll PM you the file...
Thank you. The metadata is definitively wrong in your file - so it is not set correct in camera. This is how your file looks inside (look at the very right, I marked metadata blue):
(https://i.ibb.co/WkxxwRD/Bildschirmfoto-2019-03-16-um-19-26-19.png)
Your file is 1x3, but 3x3 is set in the MLV.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on March 16, 2019, 09:39:08 PM
as soon as i switch to lossless in camera, there's no problem.
uncompressed only.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: mlfan on March 17, 2019, 03:33:57 AM
Quote from: kwstas on March 16, 2019, 02:02:44 PM
Hi reddeercity and guys,

I've made some quick test with experimental release. Followed the steps from p1. I'm facing these issues.

- 5x mode is magenta, when recording turns grayscale, yet crop factor is irrelevant to what it records
- when pressing to 10x mode camera freezes

I'm a new 5D mk II user and I'm seeing this too. IIRC 10x mode doesn't always freeze, but I consistently have a very pink 5x mode.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 17, 2019, 05:01:30 AM
What preview mode are you using ?
remember this is bleeding edge stuff , sometimes things that
normally work on the nightly builds don't always work on bleeding edge stuff .
crop_rec is only meant to record extended resolutions in 3x crop_mode (5x zoom)
and not to be used with the regular nightly build feature set .
Meaning 10x zoom freezes the camera .

I expect to have normal canon previews in future releases of the crop_rec  in 3x crop_mode (5x Zoom)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: mlfan on March 18, 2019, 12:36:22 AM
Quote from: reddeercity on March 17, 2019, 05:01:30 AM
What preview mode are you using ?
remember this is bleeding edge stuff , sometimes things that
normally work on the nightly builds don't always work on bleeding edge stuff .
crop_rec is only meant to record extended resolutions in 3x crop_mode (5x zoom)
and not to be used with the regular nightly build feature set .
Meaning 10x zoom freezes the camera .

I expect to have normal canon previews in future releases of the crop_rec  in 3x crop_mode (5x Zoom)

I'm not quite sure what you mean by preview mode -- I'm in manual mode in Live View. Could you explain?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on March 18, 2019, 03:19:46 AM
Please read first post of this thread with care. Esp. what is written about preview mode there.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on March 21, 2019, 05:29:45 AM
I've been experimenting around with the 7D to see if I can get it working with reddeercity's latest crop_rec settings. Still a long way off but I have managed to merge the current crop_rec_4k_mlv_snd branch with waza57's crop_rec_4k_5D2 along with raw_video_10bit_12bit_LVState -- but wait, there's more -- RscMgr_memory and the changes need to get the 7D working with 10/12bit. But that's not all, also the latest lua_fix. The 50D should be working on this branch. In fact all of ML cameras should be functional on this branch. Ok--they all compile but I haven't tested them. Need some help with that.

There's a test build for the 5D2 on my downloads page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/). It should work the same as the waza57 version. I didn't include reddeercity's unpublished changes at this time.

Here's the source code if you want to roll your own and start experimenting:

https://bitbucket.org/daniel_fort/magic-lantern/src/crop_rec_4k_mlv_snd_experiments/

@reddeercity -- I could use some tips from you on how to get started with Digic Poke to change the resolution.

[EDIT] Sorry, looks like I didn't merge waza57's code before I posted this.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 21, 2019, 05:58:32 AM
I don't really see why you merged in to "crop_rec_4k_mlv_snd branch"
I can't see any advantage with it , specially with lua_fix . That I can't see to be any use at all at least on 5d2 in Video .

I thinks it simpler & easier to keep on the waza57's crop_rec_4k_5D2 branch , in my own opinion .

Quote from: dfort on March 21, 2019, 05:29:45 AM
@reddeercity -- I could use some tips from you on how to get started with Digic Poke to change the resolution.
Sorry , I been busy with getting a few projects up and running ( it spring time and warm weather finally has come to the great white north  :D)
and writing code for crop_rec 5d2 . I'll see if I can put something together in the next day of two .

Edit: just read though the crop_rec code and there's no code for waza57 crop_rec branch
There no reference to 5D2 in the code at all , so I would say it didn't merger any think for waza57 source "crop_rec_4k_5D2 " branch
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on March 21, 2019, 12:44:20 PM
I think it's better to work on a clean fork of crop_rec_4k_mlv_snd.

It will allow to prepare for merging back into the "official" codebase when things will be ready.

Regarding code from Waza, is it still necessary or a better way to allow digic4 to work on this branch was found?

I remember reddeercity has to reverse back some of the code to make it work as current state
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on March 21, 2019, 09:55:27 PM
Quote from: reddeercity on March 21, 2019, 05:58:32 AM
I don't really see why you merged in to "crop_rec_4k_mlv_snd branch"

Why? Because it is there! Oh wait, that's a mountain climber's answer. Well, in a way this is quite a mountain to climb.

If you take a look at the experiments downloads page (https://builds.magiclantern.fm/experiments.html)--

4K raw video recording; lossless compression
crop_rec module with higher resolutions (4K, 1080p48 etc):
(https://farm8.staticflickr.com/7816/46521454375_88479116ab_z.jpg) (https://flic.kr/p/2dSWBAX)

Quote from: aprofiti on March 21, 2019, 12:44:20 PM
I think it's better to work on a clean fork of crop_rec_4k_mlv_snd.

Yes, I did that. Basically the various development branches went into the Cuisinart and they blended together quite nicely with very few conflicts. Then I added some work I did go get the 7D working and voilà, a branch where the old 5D2 and 7D can live side by side with newer 5D3, 70D and the rest of the family too! @aprofiti - that means your 50D and eventually your manual_lens_info pull request (https://bitbucket.org/hudson/magic-lantern/pull-requests/940/manual-lens-info-support-to-crop_rec_4k/diff) should also be working on all cameras.

Ok--so much for the sales pitch, it is not quite ready for prime time. There's some build script problem that prevents "make zip" from working properly though "make install" works fine. Go figure. I'm also sure there are a few bugs that need to be addressed. Right now let's call it more a proof of concept.

Quote from: aprofiti on March 21, 2019, 12:44:20 PM
Regarding code from Waza, is it still necessary or a better way to allow digic4 to work on this branch was found?

Quote from: reddeercity on March 21, 2019, 05:58:32 AM
Edit: just read though the crop_rec code and there's no code for waza57 crop_rec branch
There no reference to 5D2 in the code at all , so I would say it didn't merger any think for waza57 source "crop_rec_4k_5D2 " branch

:-X

Oops, that's the one branch that I haven't merged in yet. Thought I did it earlier. Need to resolve a few conflicts.

Well, it does answer the question whether Digic 4 can work on this branch but in order to get the crop_rec module working it does need some more work and since waza57 and reddeercity have already done that let's see if it can be merged in with the mainline experimental branches without breaking anything.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on March 22, 2019, 02:09:25 AM
Ok--still a little ruff around the edges but I got something that might work. Tested on the 7D and it does 10/12bit, the 50D should work the same. Now on the 5D2 I'm interested if the crop_rec module is working, it is from waza57's crop_rec_4k_5D2 branch. If it does work it would be interesting to see if reddeercity's crop_rec module works in this build.

Posted test builds on my downloads page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/). Look for the builds from the magiclantern-crop_rec_4k_mlv_snd_experiments_waza57 branch.

@reddeercity - waza57's changes are showing up now:

https://bitbucket.org/daniel_fort/magic-lantern/commits/branch/crop_rec_4k_mlv_snd_experiments_waza57?page=5
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: IDA_ML on March 22, 2019, 12:40:31 PM
Hello Dfort,

Been following all your posts with great excitement!  Thank you so much for not giving up on the fantastic 7D and trying to move forward with Digic IV cameras.

I have now briefly tested the build called:

magiclantern-crop_rec_4k_mlv_snd_experiments_waza57.2019Mar21.7D203

that I downloaded from your download area.  It works partially but in fact, worse than your March 2-nd build which works very well with all 3 bit depths with sound.  Here is what I found:

1)  I found only the mlv_rec.mo (RAW video (MLV)) in the modules directory.  MLV_Lite.mo is missing.

2)  Crop preview at all preview modes provides 5x and not 3x magnification.

3)  Once started, recording at all 3 bit depths does not stop upon pressing the rec button.  You get a red dot and a "stopping" message in the RAW video (MLV) menu and camera freezes.  Turning camera off and pulling battery out resolves the issue.

4)  In-camera playback does not work any more, not even after adding the raw_twk module.

5) Every mlv file gets recorded with 7 additional M00 through M07 files of 1 kB size each.  MLVApp does open and play the files though.  Sound works also.

Hoping that some of the other developers may be able to help you with useful hints, I keep my thumbs pressed for further progress on your part.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on March 22, 2019, 06:04:24 PM
Looks like it is one step forward and two steps back.

Main thing I'd like to find out is if the crop_rec module is working on the build I posted for the 5D2. Anyone?

The 7D has many issues. If I leave the battery in it overnight it wakes up dead--what I mean is that the battery drains even when the camera is turned off. I think this was reported before.

Quote from: IDA_ML on March 22, 2019, 12:40:31 PM
It works partially but in fact, worse than your March 2-nd build which works very well with all 3 bit depths with sound.

The March 2 build is strictly raw_video_10bit_12bit_LVState so if that's working nicely we should probably talk about how to integrate it into the main repository but let's continue with that topic on the 12-bit (and 10-bit) RAW video development discussion (https://www.magiclantern.fm/forum/index.php?topic=5601.new#new).

Quote from: IDA_ML on March 22, 2019, 12:40:31 PM
1)  I found only the mlv_rec.mo (RAW video (MLV)) in the modules directory.  MLV_Lite.mo is missing.

Right -- mlv_lite on the crop_rec_4k branch works with lossless compression so it won't even compile for Digic IV on my experimental branch. We're a ways off from getting that working on Digic 4. In the meantime most reports indicate that mlv_rec is working better than mlv_lite on Digic IV.

Quote from: IDA_ML on March 22, 2019, 12:40:31 PM
2)  Crop preview at all preview modes provides 5x and not 3x magnification.

Not sure what you mean by that. Maybe some pictures or video will help show the issue? I do have an issue when going into zoom mode I can't get out of it. Hey, this is highly experimental but I was able to record 233 frames of 2496x1198 10-bit at 24fps on the 7D.

(https://farm8.staticflickr.com/7886/32500800867_7e7597371e.jpg) (https://flic.kr/p/RvZ6MD)

Ok--not the most exciting shot but it does work over here.

Quote from: IDA_ML on March 22, 2019, 12:40:31 PM
3)  Once started, recording at all 3 bit depths does not stop upon pressing the rec button.  You get a red dot and a "stopping" message in the RAW video (MLV) menu and camera freezes.  Turning camera off and pulling battery out resolves the issue.

Again--can't reproduce that issue over here.

Quote from: IDA_ML on March 22, 2019, 12:40:31 PM
4)  In-camera playback does not work any more, not even after adding the raw_twk module.

I never played around much with in-camera playback. @reddeercity's - do you have that working on the 5D2?

Quote from: IDA_ML on March 22, 2019, 12:40:31 PM
5) Every mlv file gets recorded with 7 additional M00 through M07 files of 1 kB size each.  MLVApp does open and play the files though.  Sound works also.

Strange, I'm not seeing that either. What happens with mlv_rec is that it often leaves an MLV_REC.TMP file on the card but it doesn't seem to have any negative consequences.

Quote from: IDA_ML on March 22, 2019, 12:40:31 PM
Hoping that some of the other developers may be able to help you with useful hints, I keep my thumbs pressed for further progress on your part.

That's why I'm posting on this topic. I'd like to get the crop_rec module working on the 7D (probably hard) and the 50D (probably easier) and maybe other Digic IV cameras.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on March 22, 2019, 06:15:34 PM
It seems to freeze when in 3X mode and also have black lines vertical in the frame.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on March 22, 2019, 06:28:52 PM
I don't think I can experiments in the next days, but will follow and try something if possible.

Is the new branch cleaner compared to what you posted yesterday?
I mean only strictly necessaries (apart lua_fix which is easy to see) to understand difference between crop_rec_4k and 10-12 branch.

Can you make a "fake PR" to your bitbucket to evidence the difference with the main repository?

I need firstly understand what was changed with the code from Waza57 to make 5d2 works (I saw some commit which "revert" some previous one) and then start investigating on 50D (just " nearly standard" resolution because I'm stuck with incrementing heigth (https://www.magiclantern.fm/forum/index.php?topic=19336.msg210344#msg210344)).

Regarding compilation of ml_lite due to lossless.. can't remember if by adding the non definitive changes posted in lossless thread, will allow us to record raw videos... I was trying with silent pictures mainly.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: IDA_ML on March 22, 2019, 06:46:14 PM
Quote from: dfort on March 22, 2019, 06:04:24 PM

The 7D has many issues. If I leave the battery in it overnight it wakes up dead--what I mean is that the battery drains even when the camera is turned off. I think this was reported before.

This was not the case with the March 02-nd build.  No battery drain here even after I left the battery for two weeks in the camera. Will check tonight if the battery gets drained with the March 22 build.

Quote from: dfort on March 22, 2019, 06:04:24 PM
Not sure what you mean by that. Maybe some pictures or video will help show the issue? I do have an issue when going into zoom mode I can't get out of it. Hey, this is highly experimental but I was able to record 233 frames of 2496x1198 10-bit at 24fps on the 7D.

I mean that there is still no WySiWyG preview on today's build.  What you see on the screen while recording corresponds to 5x magnification while in fact the image is recorded at the correct 3x magnification.  5x-magnification does not allow for correct framing when you move the camera during recording.

Quote from: dfort on March 22, 2019, 06:04:24 PM
Again--can't reproduce that issue over here.

Probably we have different camera settings.  Will keep testing tomorrow.  When I revert back to the March 02 build, recording is perfectly fine and stops nicely when I press the record button.  By the way, at 2496x1198 10-bit and 24fps, with the March 02 build, I get continuous recording on the 7D.  Please try that build again and compare it with today's one!

Quote from: dfort on March 22, 2019, 06:04:24 PM
I never played around much with in-camera playback. @reddeercity's - do you have that working on the 5D2?

Please try the raw_twk.mo on your March 02 build.  Just add it to the Modules directory and activate it in the Modules menu.  It plays back all recorded files nicely in camera after they have been recorded. Very useful!

Quote from: dfort on March 22, 2019, 06:04:24 PMStrange, I'm not seeing that either. What happens with mlv_rec is that it often leaves an MLV_REC.TMP file on the card but it doesn't seem to have any negative consequences.

The  MLV_REC.TMP  always gets recorded on an empty card, no question about that.  I am talking about these 7 files (M00 through M07) short files that get recorded with every MLV file too. 

Quote from: dfort on March 22, 2019, 06:04:24 PM
That's why I'm posting on this topic. I'd like to get the crop_rec module working on the 7D (probably hard) and the 50D (probably easier) and maybe other Digic IV cameras.

Can anyone help with this, please?  We badly need some help here and every hint that could guide us in the right direction will be greatly appreciated. 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on March 22, 2019, 08:12:45 PM
Quote from: dfort on March 22, 2019, 02:09:25 AM
Now on the 5D2 I'm interested if the crop_rec module is working, it is from waza57's crop_rec_4k_5D2 branch. If it does work it would be interesting to see if reddeercity's crop_rec module works in this build.

I am getting a Raw Detect Error with 3.5k 5x mode on Crop_Rec. When adjusting values in the menu, at first it acted like it was in the 1920 1:1 mode, with 1984 as the max H resolution - after some bumbling around the menu now it expects to be able to get up to 2880x1080 - and I recorded a few seconds with accurate framing in liveview but frozen during recording. The black level is totally off in the clip, but it seems to be valid 2880 x 1080. (Uploaded the MLV here, M22-1357.MLV: https://drive.google.com/open?id=1vfodz-Dsk8T0NppBxFIQaCt59NSDEyZP)

And then, randomly, it now gives me a Raw Detect error again and I can get it to record again.

1920 1:1 works fine, with accurate framing until you press record. 12 bit has slightly incorrect black level, green tinted shadows.

Generally this build runs a little slow - the UI is slightly unresponsive here and there.

EDIT:
Quote from: dfort on March 22, 2019, 02:09:25 AM
If it does work it would be interesting to see if reddeercity's crop_rec module works in this build.

It does. And much better. Adding the crop_rec module from reddeercity's magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212 to magiclantern-crop_rec_4k_mlv_snd_experiments_waza57.2019Mar21.5D2212 works like a charm.

Max resolution in crop_rec 3.5k 5x mode is 3008 x 1080. All bit depths work. Will keep messing around a little more.

Notes:
Sound Recording works, but after stopping recording the camera is stuck at "Stopping" for a long time - requires a restart but no battery pull necessary.

Is there a raw_twk module I could be trying on 5d2? MLV_PLAY hasn't worked for a long time with any of these 3k 5x builds, and the raw_twk I tried adding just gave me an oldAPI error.



Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on March 23, 2019, 01:52:16 AM
Ok--good to hear that reddeercity's crop_rec is working better. All hail to the 5D2 guru!

Merged in the unpublished reddeercity code which bumped everything up to a more current code base. This means that the lossless compression in mlv_play and some other places will be a problem until lossless is working on Digic IV cameras. Note that raw_twk does compile but I'm not sure what good it will do without mlv_play. Maybe start mixing and matching modules from other builds?

This time my camera did reproduce the issue where it wouldn't stop. Found a temporary fix for that.

src/raw.c
static void raw_lv_disable()
{
// ASSERT(!lv_raw_gain); // dfort - 7D cannot stop recording if this is enabled
    lv_raw_enabled = 0;
    raw_info.buffer = 0;


Quote from: aprofiti on March 22, 2019, 06:28:52 PM
Can you make a "fake PR" to your bitbucket to evidence the difference with the main repository?

Good idea, I was thinking about that myself. Here's a fake pull request (https://bitbucket.org/daniel_fort/magic-lantern/pull-requests/27/crop_rec_4k-for-digic-iv-cameras/diff) that shows the changes between my latest experimental branch and the crop_rec_4k_mlv_snd branch. Looks like lots of extra stuff that could probably be eliminated though it would still be huge and most likely not acceptable to merge into the main repository.

By the way, on these latest test builds adtg_gui should be working on the 50D so you should be able to start tweaking the registers to extend the resolution. On the 7D that should be possible using Digic Poke. (Once reddeercity wraps up some projects and takes a Spring break maybe he'll teach us how he uses that tool.)

Test builds on my downloads page (https://bitbucket.org/daniel_fort/magic-lantern/downloads/).

Quote from: aprofiti on March 22, 2019, 06:28:52 PM
Is the new branch cleaner compared to what you posted yesterday?
I mean only strictly necessaries (apart lua_fix which is easy to see) to understand difference between crop_rec_4k and 10-12 branch.

As you can see from the fake pull request, it is getting complicated. The cleanest by far is the stuff I did on the raw_video_10bit_12bit_LVState branch but even there a PR with my changes to get the 7D working probably won't cut the mustard.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 23, 2019, 02:57:37 AM
Well it looks like I'm being pushed off my thread.
I have no intention of supporting  crop_rec_4k_mlv_snd branch on 5d2
I'll be finishing of the crop_rec in "crop_rec_4k_5D2" branch , Sorry it others disagree .
If there too many branches nothing will get done , so I will only help with development on that branch.

As for as other d4 go e.g. 7d,  go ahead and use "crop_rec_4k_mlv_snd branch" as I have no interest
in that camera or other d4's any more .

After I have finished code for 5d2 crop_rec then I may consider "crop_rec_4k_mlv_snd branch" but not before .
and that's a big maybe .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on March 23, 2019, 04:35:09 AM
I don't think anyone is trying to minimize your efforts - I'm very excited by everything you've added to the 5d2, and I'd hate to see development stall!

Could you explain the pros or cons between crop_rec_4k_mlv_snd /crop_rec_4k_5D2 in your opinion? When I added your crop_rec to crop_rec_4k_mlv_snd it was very stable and capable of all the same great stuff.

My understanding is that crop_rec_4k_mlv_snd more closely resembles development on other cameras and thus will be easier to keep the 5d2 up to date moving forward? It would be a shame if all your hard work was the last act for this legendary camera.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: theBilalFakhouri on March 23, 2019, 05:07:31 AM
Don't worry about the branches, crop_rec code will change completely to the new method of it, maybe at 1st April for this year for both D4 and D5.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on March 23, 2019, 05:27:56 AM
Quote from: dfort on March 23, 2019, 01:52:16 AM
This time my camera did reproduce the issue where it wouldn't stop. Found a temporary fix for that.

src/raw.c
static void raw_lv_disable()
{
// ASSERT(!lv_raw_gain); // dfort - 7D cannot stop recording if this is enabled
    lv_raw_enabled = 0;
    raw_info.buffer = 0;



This issue was still occurring for me on your latest test build - I'm assuming this code was included? This issue does not occur on reddeercity's Feb01 build.

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 23, 2019, 06:35:36 AM
Here is one of my first Test builds -- 2.6k crop_rec for the 5d2
2.6k_5d2_crop_rec.rar (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2.6k_5d2_crop_rec.rar)
I still have a vertical offset  problem in the code , it not applying cmos[1] correctly so there's a white bar at the bottom .
I'll work on it tomorrow , it's getting late .

So everything is working at 2624x1200 @ 23.976fps , realtime canon preview when preview set to "Auto"
It all works the same as nightly builds , HDMI works @ 480p the best , 1080i works but causes a odd flicker on screen
you can record raw video with HDMI connected . I quickly check that you can change shutter speed but didn't check ISO changes .
It should work on top of the build I have posted on the first page of this thread .

So check it out and let me know how it works please and yes there will be a white bar at the bottom of the image
that should be fixed tomorrow hopefully , I just want to know if everything else works correctly .

I will also be posting test crop_rec's for 48fps 3x3 & 1x3 anamorphic 4.9k @ 2.66 A.R. 23.976fps .
Once all that is tested and confirmed working then I'll have it all together in one "crop_rec"
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on March 23, 2019, 09:18:06 AM
Everything seemed normal! - besides what you mentioned.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: honza123 on March 23, 2019, 12:51:51 PM
I confirm the same.
:)

There is a white bar (120 pixels) at the bottom of the image.
On the right side is a 16 point wide black bar.
The active area is 2608x1080 pixels.


(https://i.ibb.co/hXbjY4Z/M23-1102-frame-172.png) (https://ibb.co/hXbjY4Z)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on March 23, 2019, 05:53:09 PM
Hum--looks like I painted myself into a corner. Recording is still not stopping properly on my experimental branch:

Quote from: jpegmasterjesse on March 23, 2019, 05:27:56 AM
This issue was still occurring for me on your latest test build - I'm assuming this code was included? This issue does not occur on reddeercity's Feb01 build.

There are some other issues too so I should either back up to find where the problem started, start all over or wait for reddeercity to publish his code.

Quote from: reddeercity on March 23, 2019, 02:57:37 AM
Well it looks like I'm being pushed off my thread.

Didn't know you could own a thread on this forum. How about being positive and seeing this as continuing the discussion on a topic that you started?

Quote from: reddeercity on March 23, 2019, 02:57:37 AM
I have no intention of supporting  crop_rec_4k_mlv_snd branch on 5d2

Fair enough, this is an open source project and we're all volunteers.

QuoteEvery good work of software starts by scratching a developer's personal itch (https://opensource.com/article/17/4/itch-to-scratch-model-user-problems).
                                                                                       -- Eric Raymond

I only picked crop_rec_4k_mlv_snd because that seemed to be the right branch to eventually merge in your experimental work on the 5D2.

(https://farm8.staticflickr.com/7816/46521454375_88479116ab_z.jpg) (https://flic.kr/p/2dSWBAX)

Now will your work eventually get merged in to this branch or another branch or will it become a new branch on the main repository? Something to think about. The worst-case scenario is that it doesn't get into the main repository at all.

Quote from: reddeercity on March 23, 2019, 02:57:37 AM
I'll be finishing of the crop_rec in "crop_rec_4k_5D2" branch , Sorry it others disagree .
If there too many branches nothing will get done , so I will only help with development on that branch.

I would agree that there are too many open branches but I'm confused. It seems are you proposing a new branch only for the 5D2. Wouldn't it be better to have the 5D2 working on the same branch as the other ML supported cameras?

Quote from: reddeercity on March 23, 2019, 02:57:37 AM
As for as other d4 go e.g. 7d,  go ahead and use "crop_rec_4k_mlv_snd branch" as I have no interest
in that camera or other d4's any more .

Sorry, I took your topic heading literally - 3K/UHD 5D2 Raw development and Other Digic IV Cams

Note that the crop_rec_4k branch at first only worked properly with the 5D3. Eventually the EOSM then other Digic V cameras were working on it. Maybe I'm the pushing to get the other Digic IV cameras started a little too early.

Quote from: reddeercity on March 23, 2019, 02:57:37 AM
After I have finished code for 5d2 crop_rec then I may consider "crop_rec_4k_mlv_snd branch" but not before .
and that's a big maybe .

You know Magic Lantern is a work in progress and will probably never be "finished" -- right?  :)

As far as crop_rec_4k_mlv_snd and crop_rec_4k_mlv_lite_snd branches, those were started in order to get sound recording working with mlv_lite. At this point my understanding is that the Digic IV cameras are working better on mlv_rec. However, if we can get lossless compression working on Digic IV we will need to switch to the newer mlv_lite code.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on March 24, 2019, 12:55:08 AM
Not all that important, but I'm wondering if digital dolly works for any versions of crop_rec?  I think it'd be pretty great to scan vertically to get something like a full-sensor scan for landscapes. It hasn't worked for any of these builds we've been testing.

I noticed you can't move the preview window in 5x mode at all once digital dolly is enabled, wheras it does move without.

Here is what Alex said in another thread regarding digital dolly and crop_rec:

Quote from: a1ex on March 18, 2018, 09:30:49 PM
Modifying digital dolly to cover the full sensor is possible, but non-trivial (Canon recalibrates the vertical noise correction factors with each video mode reconfiguration, including when shifting the scanned area in x5 zoom mode). By default, Canon code scans a non-centered wide area in x5 zoom mode (3584x1320 usable area on 5D3) and moves it around as you move the focus box - notice a slight pause at certain positions on the screen, when it reconfigures the sensor.

You can, however, try something with the crop_rec_4k experimental build (caveat: requires fiddling)

- load crop_rec and mlv_lite
- set crop_rec preset to full-res LiveView
- reduce the vertical resolution in crop_rec submenu until you get the desired FPS (Target YRES = 1039 will give 23.973 FPS)
- recalibrate the CMOS[1] register to get good and centered image (trial and error)
- enable raw video and digital dolly - now it should cover the full width of the sensor.

If useful, I'll consider adding a full-width 1080p preset to crop_rec (maybe it can be pushed to about 5784x1080 at 23.976 FPS, or it might require slightly lower horizontal resolution).

Side note: the MLV spec also has metadata for 1-pixel increments for digital dolly, but would require special support from the MLV converter.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 24, 2019, 02:06:21 AM
Ok moving on .

Thanks to all tester  :)
I've removed that crop_rec from yesterday & updated with
2.6k_5D2_3-23-2019_crop_rec.rar (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2.6k_5D2_3-23-2019_crop_rec.rar)
Just use it with the build on the first page .
I fixed the cmos[1] reg in the code , now it works correctly with the full vertical height .
So now you have 2624x1200 @ 23.976fps with Canon liveview in real time !
There is about 10 or 12 line missing on the right side , I need to increase timer "A"
a little bit more , that I will take care of later on . So in reality horizontally ,  it's closer to 2605 or something like that .

Test it as if you are in a video production  :D
Let me know of any problems .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on March 24, 2019, 08:44:17 AM
Quote from: reddeercity on March 24, 2019, 02:06:21 AM
Let me know of any problems .

Seems to work ok at all bit depths in non-crop and 5x crop. No issues with sound enabled.

I did try going to Canon 24p and it worked ok for a few shots and then I encountered issues with 5x preview mode being broken; on auto and on ML greyscale. A battery pull returned things to normal.Going in to 10x mode seems to trigger the same wonkiness no matter the Canon FPS, actually.

FPS override does not work in 5x mode.

Changing ISO's in 5x mode does work with the ISO button on the top of the camera. Changing the ISO in the ML menu with the joystick however, results in an infinite scroll where the ISO keeps increasing or decreasing depending on the direction you pushed. I believe this was also the case on crop_rec_4k_mlv_snd, but I will have to double check. Using the top scroll wheel in this menu does not cause this.

Using any method to change iso while in 5x zoom mode creates a white bar on the bottom of the frames of some ISOs - exiting to menu and back fixes this. Results for which ISOs are effected seems to depend on your starting ISO.

Gradual Exposure seems to be broken - there is no smooth transitions, and the briefly gets darker before brighter if you raise the iso

This is probably already known, but arrow/set shortcuts don't work in 5x mode.

ML digital ISO seems to have no effect with or without Raw modules loaded.

@dfort: One thing I'm noticing in your latest build magiclantern-crop_rec_4k_mlv_snd_experiments_waza57.2019Mar22.5D2212 is that there are 8 black pixels on the right side of the frame at 5x mode.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 25, 2019, 04:28:23 AM
Ok , thanks for testing .
All this is known issue , ISO on 5d2 works best anyways set to manual as I only use that.
I think I have a  fix for changing ISO in ML Menu.
10x Zoom is known to break liveview , Workaround : (in fact any liveview breakage)
Press play button twice (that refreshes the liveview ) (help if you have a h264 .mov or CR2 there to be reviewed)
That put you in 3x3 (FHD) Liveview should be ok now , press 5x zoom(3xcrop_mode) again to enable crop_rec and liveview should good now.

You can't use frame override , this is again known . The preset framerate is locked/fine tuned to that resolution size (besides why would what to change 23.976 ?)
You must expect the little feature sometimes will not work , What you can expect is to have a basic recordable extended resolution e.g. 2600x1200
Think of the 5d2 as a film camera , you have no auto functions  everything is manually set .  After all it is bleeding edge stuff .

My objective is to have a basic recordable extended res. in crop_rec that can be adjustable e.g. ISO & Shutter speed with
realtime liveview & HDMI support (at least at 480p) , which is useable now .

So I can conclude this preset is working then , I'm now adding more presets to crop_rec .
Of course that's easier said then done  :P 
I do have it figured out by the way just my C code skill are not the best (I'm slow)
But just like the tortoise and the hare I will finish the race , just not in record time  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on March 26, 2019, 09:27:12 PM
I also tested the new crop module - it works fine, the image is awesome. I was trying to see the picture via HDMI monitor for adjust the aspect ratio to see the borders of the image I shoot, but I couldn't... and all I see pink solid color with some kind of transparent image.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: honza123 on March 26, 2019, 09:54:49 PM
After a short test ...
Recording RAW 2624x1200pixels, 10 bit is OK
:)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on March 26, 2019, 11:08:25 PM
I just was thinkin about crop factor reducing and thought that if we have money for buying sigma art 12-24 f4 or just Laowa 12mm f2.8 (which are zero distortion lenses) we can achieve 25mm in our crop module with 2.8 - 4 aperture, if we use sigma art 24mm 1.4 we get 50mm 1.4. Those lenses are costly, but have absolutely razor sharp and beautiful picture... So thanks Reddercity we have absolutely amazing 2624x1200 2:1 with full sensor bokeh style... Just wanna shoot cinema lol... The rest is to get a liveview (I would help you, Reddeer, but don't know how) and write a screenplay (also need to get some life experience for that, hehe...).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 27, 2019, 11:47:04 AM
@Igor_Braun: you won't get full sensor bokeh style. You'll get a cropped bokeh - same "problem" as using a crop camera, because the used sensor area is cropped. 5568/2426=2.29. So your f/1.4 used in this crop mode will get a bokeh like f/3.2 on the full sensor.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 28, 2019, 05:32:30 AM
@dfort here a quick tutorial on DiGic Poke

from 5d2
(https://image.ibb.co/muotkp/VRAM3_small.png) (https://imgbb.com/)
Here a example of setting up Reg C0F06084 and of course this would be the same as c0f06088 (raw resolution)
The default valve for C0F06084 is 10036 which would mean 3x3 (FHD) & 30036 valve is for 5x zoom (3x crop_mode)

There Two ways to explore the poke , manually  setting valves  in the Valve box and see the results .
Second , set valve in valve box then with half shutter press in liveview , you can increase valve or decrease valve
This done with the Altering mode box , you can see the "x++" this increase the valve and you can change that to "x--"
to decrease , still with me ?

Now what can be adjusted with the poke , will all reg's to my knowledge but I could not see a way to adjust the cmos's reg's
so if you need a new offset e.g. vertical of horizontal not sure how to do that other then coded it in crop_rec.
You mainly need new offset's when you increase vertical resolution but not so much on horizontal .
When I coded 3008x1080 I didn't have to use any new cmos vertical offset's just timers & raw resolution reg's

So the important Reg's for basic extended raw resolution in D4 cams are
Timer"A" (C0F06008)
Timer"B" (C0F06014)
raw resolution (C0F06088)

A1ex gave me great explanation here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205614#msg205614) after that I understood how it worked  :)

This is for the 5D2 , I would think it's very close to 7D . I know on the 50D everything is the same except the CMOS reg's  where different .

              x1     x5 centered        photo    description
C0F06084:   0x10036     0x30036        0x10037   start row/column
C0F06088: 0x4F40432   0x46A04BA      0xEDD0B87   stop row/column (column: 1 unit = 2 pixels)
C0F06008: 0x23B023B   0x2770277      0x5DB05DB   FPS timer A (1 unit = 4 pixels horizontally)
C0F06014:     0x6D5       0x4FF          0xEDC   FPS timer B (1 unit = 1 pixel vertically)
CMOS[1] :     0xC00       0xE6A          0xC00   vertical start/stop
CMOS[2] :     0x40E       0x10E            0x8   horizontal pos & binning mode
CMOS[3] :       0x5         0x5            0x7   ???
CMOS[4] :     0x244       0x244          0x244   ???
CMOS[5] :       0x1        0x1           0xC05   ???

Not sure if you know how to read the  raw resolution reg (c0f06088)
let take the 5x valve for example 0x46a04ba , take the first half 46a= vertical  last half 4ba = horizontal .
The only other thing that you need to know is the head timers 3 & 4
c0f0713c
c0f07150

there control the vertical window in Liveview , so if you aren't increasing vertical res. then you don't really need to bother with it .
but it does help with reducing liveview overhead for increasing fps .

Still there ? I know there a lot to take in .
One last thing (I think  :D )
When trying to increase horizontal res. , you need to slow down timer"B" (c0f06008 )to around 9 fps first
then increase timer"A"(c0f06014) for extended resolution (1 unit = 4 pixel) be mindful to don't get in to the OB area by
reduce timer"a" too much .

I hope I didn't confuse you with tooooo much info  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on March 29, 2019, 02:13:22 AM
Thanks!

Ok--here we go. My version looks a little different but it has all the necessary fields.

(https://farm8.staticflickr.com/7898/46574056415_f6acc26e5a.jpg) (https://flic.kr/p/2dXAdmF)

On the 7D the default value for that register is 100c3 so assuming it works like the 5D2 if it is changed to 300c3 it should be 5x zoom. However, if I put the camera in 5x mode that's not the value that shows up:

(https://farm8.staticflickr.com/7859/40523392343_c5c37894a1.jpg) (https://flic.kr/p/24JUYJk)

Quote from: reddeercity on March 28, 2019, 05:32:30 AM
There Two ways to explore the poke , manually  setting valves  in the Valve box and see the results .
Second , set valve in valve box then with half shutter press in liveview...

How do you manually set the values? The only way it seems to work over here is using the half shutter press. I tried changing the value of C0F06084 and for the most part all I get is a garbled screen. I thought that maybe I needed to "Apply" option but that doesn't apply the value in the field, it applies a different value - it always changes to 10002.

Quote from: reddeercity on March 28, 2019, 05:32:30 AM
...you can see the "x++" this increase the valve and you can change that to "x--"
to decrease , still with me ?

No, can't say that I'm still with you:

(https://farm8.staticflickr.com/7834/47436486432_44e65dcd95.jpg) (https://flic.kr/p/2fgNoKJ)

So far I only figured out how to increment the value, can't seem to find the button to decrement.

Well, looks like this is going to be a bit more challenging. I'll check out that post from a1ex and see if anything sinks in.

Thanks for the lesson--I'll see if I can get any further with it a little later.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 29, 2019, 09:22:00 PM
How do you guys get the new crop_rec to work? Tried it with the build from first page. After enabling the neccessary modules I get a error message "tcc: error: undefined symbol 'lossless_decompress_raw' [E] failed to link modules". No RAW recording possible here.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on March 30, 2019, 05:01:48 AM
Sounds like you have MLV_Play enabled?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 30, 2019, 09:27:06 AM
Oh yes... thanks. This one was still enabled and I didn't recognize. Now it seems to work fine. The picture coming in look fantastic. On the right side I get some black lines. Is it as expected that the liveview has wrong framing? It is realtime as @reddeercity wrote, but recorded image has ~ double width and height to what I see in liveview.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 30, 2019, 02:22:40 PM
Oh I love this resolution and the quality you get out of the old 5D2. Went out and did some test shots. Most of them look awesome. But some other shots are a bit strange and I don't know why yet. Some colums seems to have right blacklevel, others haven't. After around 50 to 200 frames all strips disappear.

First frame looks like that:
(https://i.ibb.co/dtfvZ35/M30-1246-frame-1.png)
Frame 90 with same settings:
(https://i.ibb.co/p4XrmKr/M30-1246-frame-90.png)

The black bar on the right should be easily cut somehow?!

And it seems that the frame is not exactly centered on the sensor - it is a bit more on the right, so the left part of the picture is a bit sharper than the right (right is a bit more to the lens corner).

Let me know if you need a shortend MLV for analysis.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on March 30, 2019, 03:46:13 PM
that looks like agtg amplifier "settling" to me.
i see if often when shooting using the reduced gain mode i love so much.  it takes about 30 seconds to go away, and then it's stable for shooting.

maybe something is being set in the registers every time recording, instead of once only?

i'm just guessing, but it looks very similar to me...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on March 30, 2019, 05:18:43 PM
Hey, I am new to the magic Lantern group !!
I own a 5d mark ii, and I decided to try 3k raw video on it.
All steps from the first page of this topic work, but there is one thing i don't understand


(https://i.ibb.co/yY3Dbj1/Full-Size-Render.jpg) (https://ibb.co/yY3Dbj1)


I think it is just because I am a little bit stupid, but what does that mean ?

Again sorry for my stupid question  :-X , and thanks for your amazing work... (who would have thought of 3k raw and a 10 year old camera...)

And also, is a 800x card enough to record 3k raw ? (and maybe 4k in the future) from magic lantern ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on March 30, 2019, 06:01:00 PM
The video modes discussed here (3K and others) are interlinked to liveview mode. Entering liveview and pressing magnify/loupe button (=called crop mode) enables recording in higher resolutions.

800x is for read mode only and has nothing to say for write performance. Lexar's 800x cards are doing very differently for different card sizes. 16-32 GB performs about 45 MByte/s, 64 GB does about 75 MByte/s. See www.cameramemoryspeed.com
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on March 30, 2019, 06:12:16 PM
thanks for your fast answer Walter Schulz, the button to press is the top right one, right ? and is it supposed to display something on the screen to see if its anbled or not ? I don't see a way to see if its on or off..., I might do something wrong, sorry  :-X

and I didn't know 800x was only for reading, thanks . If  I use a 120mb/s card, is it enough ?
Thanks again !
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on March 30, 2019, 06:29:12 PM
Consult manual about how to operate your cam! You may have to press magnify button twice.

Rough calculation:
Horizontal resolution x vertical resolution x bit depth x frame rate x compression gain / 8 = Byte/s
4000 x 4000/2.35 x 14 bit x 24 x 0.5 / 8 = 136 MByte/s.
Cam might do about 75 MByte/s continuously due to limited storage interface.
https://rawcalculator.netlify.com/calculator_desktop
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on March 30, 2019, 06:34:30 PM
thanks i will consult my manual  ;) sorry for disturbing you  8)




It works !!!! Thanks a lot ^^?
is it normal if I can't go to 2880 ? (I am stuck to 2560)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on March 31, 2019, 07:26:32 AM
Quote from: Galterius on March 30, 2019, 06:53:07 PM
It works !!!! Thanks a lot ^^?
is it normal if I can't go to 2880 ? (I am stuck to 2560)

Use the build in the first post of the thread. Cheers!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: joanllm on April 01, 2019, 09:55:09 AM
Great work goes on like this! the quality of this camera's image is incredible! Just a question. Is it possible to increase the 1200 pixels in the future or is this really the limit? It's just out of curiosity.

Regards!

https://serving.photos.photobox.com/646633975cb85bff654605ffbada15d8acfd544241fa3c31b157ccba9d1f620fec945235.jpg

https://serving.photos.photobox.com/221122633a58bf77a484541c81752f7d1d8765f053386da7e7dc8fb680dcd6e0a707030b.jpg
https://serving.photos.photobox.com/05994318716afdbafebf049b4584fee8a200eff2a6cd83a20ee4138725ce64dc91c3e7c4.jpg

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on April 02, 2019, 05:31:29 PM
Quote from: masc on March 30, 2019, 02:22:40 PM
Oh I love this resolution and the quality you get out of the old 5D2. Went out and did some test shots. Most of them look awesome. But some other shots are a bit strange and I don't know why yet. Some colums seems to have right blacklevel, others haven't. After around 50 to 200 frames all strips disappear.

First frame looks like that:
(https://i.ibb.co/dtfvZ35/M30-1246-frame-1.png)
Frame 90 with same settings:
(https://i.ibb.co/p4XrmKr/M30-1246-frame-90.png)

The black bar on the right should be easily cut somehow?!

And it seems that the frame is not exactly centered on the sensor - it is a bit more on the right, so the left part of the picture is a bit sharper than the right (right is a bit more to the lens corner).

Let me know if you need a shortend MLV for analysis.

I had the same problem sometimes but thought that the sensor's time is coming to its end. Hope I'm wrong.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on April 02, 2019, 07:54:41 PM
thanks for your answers, I just got some sandisk 160mb/s card that should do the job ^^
My question is now, what is the best way, according to you, to import those .mlv files into Premiere Pro ? (to not loose the raw advantage too much, I don't really know what to do, there are lots of different technics on youtube but I don't know which one to pick  :)
Thanks !!!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: OlRivrRat on April 02, 2019, 09:33:36 PM
          @Galterius > Try Switch to Convert to Usable ~

https://www.magiclantern.fm/forum/index.php?topic=15108.0
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 03, 2019, 04:54:26 AM
Thanks @masc & @Igor_Braun for feed back , interesting never happen doing testing
but that what you guys are for  :P  I think know the issue . I'll post a fixed test build  a little tonight .

By the way what was the camera setting?
e.g. ISO , Liveview preview setting,  SR memory setting (e.g. 0-4) etc. ...
did you use any Auto setting? e.g. auto ISO or auto WB ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 03, 2019, 05:10:58 AM
Quote from: joanllm on April 01, 2019, 09:55:09 AM
......  Is it possible to increase the 1200 pixels in the future or is this really the limit? It's just out of curiosity.
Yes (1330) , at the cost of the reduced horizontal resolution -- manly because the sensor is too slow , main clock is 24 MHz
and reads 4 channels at a time so 24x4=96MP/s (mega pixels per second)
where the 5d3 has 8 channels at 24 MHz (24x8=192 MP/s). If I did the math right 2624x1200@ 24fps = 75MP/s so near the limit of
the image sensor 96 MP/s.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 03, 2019, 05:58:52 AM
This off topic but I will answer it this time , better to post in the raw processing threads
Quote from: Galterius on April 02, 2019, 07:54:41 PM
...... what is the best way, according to you, to import those .mlv files into Premiere Pro ?
(to not loose the raw advantage too much, I don't really know what to do, there are lots of different technics on youtube but I don't know which one to pick  :)
Thanks !!!
Will it all depends on the computer hardware you have .
e.g. fast SSD , lot's of Ram 16-32GB & fast CPU ( at least a 8core i7 or AMD FX)
In all workflows , the best way is to use MLVFS (https://www.magiclantern.fm/forum/index.php?topic=13152.msg127218#msg127218) (Cdng on the fly , Virtual Cdng's) specially with 5D2
once you have mlvfs installed load MLV on to SSD ( does work will off the CF card but a little slow)
The big advantage is you don't have to export Cdng & use up hard drive space , just load them in to mlvfs start grading/editing no wait time .
Works in MAC , PC  & Linux , I've used it with Blackmagic Resolve & CS6 Adobe After Effects .
My preferred workflow on my PC (Windows) is Adobe After Effects (ACR) -> dynamic Link to Premiere Pro
Make a After Effects Composition , do your Camra Raw adjustment with ACR save it
In Adobe Premiere Pro, choose File > Adobe Dynamic Link > New After Effects Composition
You will need a very fast computer to do this .

My Mac workflow , is a little different depending on what I'm doing .
I all ways start with Adobe After Effect CS6 & MLVFS
If I'm doing heavy FX in Autodesk Smoke for MAC , I'll export either 12bit DPX or 16 bit Tiff image sequence
after I'm done I save as a ProRes 4444 XQ or 16 bit tiff sequence .
If I'm doing lite color grade & editing , I export 16bit tiff from AF+MLVFS to Apple compressor and save
it to ProRes4444 XQ+ alpha and finish off in FCPX ( it's truly my real favorite all around video App) IMO  ;)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 03, 2019, 08:47:00 AM
Updated 2.6k_5d2_4-3-2019_crop_rec.rar (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2.6k_5d2_4-3-2019_crop_rec.rar) , adjusted the right side offset .
So now it's 2592x1200 @ 23.976fps , the raw dump is 2618 but because of the "Mod 8" thing for resolution size it goes to 2592 .
Canon liveview (pinkish) blacklevel is off for some reason more then likely it's the liveview preview reg's are not right ,
I'll need to play around with them , for now just use Auto or ML gray scale preview .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on April 03, 2019, 02:23:06 PM
thanks a a lot reddeercity and OlRivrRat.
My pc should handle it (I have a i5 6600k and 32gb of ram), I will try the new crop_rec in a few minutes !
and sorry for the offtopic, I won't do the mistake again !!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on April 04, 2019, 02:27:55 AM
Quote from: reddeercity on April 03, 2019, 04:54:26 AM
Thanks @masc & @Igor_Braun for feed back , interesting never happen doing testing
but that what you guys are for  :P  I think know the issue . I'll post a fixed test build  a little tonight .

By the way what was the camera setting?
e.g. ISO , Liveview preview setting,  SR memory setting (e.g. 0-4) etc. ...
did you use any Auto setting? e.g. auto ISO or auto WB ?

Where can I see "SR memory settings?"
I remember ISO was 1600 (I've tried with 800 also, and in my case the lower I go, the less likely this glitch happens), I didn't use any auto settings, WB was something like 5400... (actually, I don't remember exactly, but usually I use this WB when shooting in my room)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 04, 2019, 04:17:24 AM
Quote from: Igor_Braun on April 04, 2019, 02:27:55 AM
Where can I see "SR memory settings?"
Near the bottom of MLV Raw Video Tab or in the "mlv_rec.cfg" file
This is my mlv_rec.cfg file
# Config file for module mlv_rec (MLV_REC.MO)
mlv.video.enabled = 1
mlv.res.x = 7
mlv.bpp = 0
mlv.aspect_ratio = 17
mlv.write_speed = 6969
mlv.display_rec_info = 2
mlv.buffer_fill_method = 0  *** this is the SRM***


Quote from: Igor_Braun on April 04, 2019, 02:27:55 AM
I remember ISO was 1600 (I've tried with 800 also, and in my case the lower I go, the less likely this glitch happens)
Ok that's what I'll looking for , High ISO seems to have problem with these experimental builds even the old
dual ISO video module did this , maximum ISO I use is 800 (rarely thou) , I mainly use 100 -> 400  & if that doesn't get
correct exposure then I just add artificial light .
I test these experiments  between 100 & 400 ISO , so I can say at those ISO setting I have no issues .

If you when to use 800 & 1600 ISO I would recommend that you first set it in 3x3 (FHD) then reboot the cam
this also will refresh liveview , (that's part of the problem "Liveview not refreshing with HI ISO)
A thought , the play button should also refresh Liveview , so maybe try that .

In the mean time I will look in to this further , but no guarantees that I can fix it .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on April 04, 2019, 08:48:44 PM
Thanks @reddeercity. The black bar at the right is gone. But is there any possibility to center the frame? It is still a bit shifted to the right. So when recording full open, the left side is perfect, while in the right side you see if a lens is not good (enough for this resolution). :)
If standard 3x3 is that:
(https://i.ibb.co/xM8Wqwg/M04-1936-frame-1.png)
the crop frame is that:
(https://i.ibb.co/wgQW51g/M04-1938-frame-1.png)

If I use the joystick to move around, the next step to the left is also not centered (more or less the same as here, but to the other direction).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 05, 2019, 12:56:35 AM
@masc , yes should be able to do that
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on April 05, 2019, 10:21:24 AM
Quote from: reddeercity on April 04, 2019, 04:17:24 AM
Near the bottom of MLV Raw Video Tab or in the "mlv_rec.cfg" file
This is my mlv_rec.cfg file
# Config file for module mlv_rec (MLV_REC.MO)
mlv.video.enabled = 1
mlv.res.x = 7
mlv.bpp = 0
mlv.aspect_ratio = 17
mlv.write_speed = 6969
mlv.display_rec_info = 2
mlv.buffer_fill_method = 0  *** this is the SRM***

Ok that's what I'll looking for , High ISO seems to have problem with these experimental builds even the old
dual ISO video module did this , maximum ISO I use is 800 (rarely thou) , I mainly use 100 -> 400  & if that doesn't get
correct exposure then I just add artificial light .
I test these experiments  between 100 & 400 ISO , so I can say at those ISO setting I have no issues .

If you when to use 800 & 1600 ISO I would recommend that you first set it in 3x3 (FHD) then reboot the cam
this also will refresh liveview , (that's part of the problem "Liveview not refreshing with HI ISO)
A thought , the play button should also refresh Liveview , so maybe try that .

In the mean time I will look in to this further , but no guarantees that I can fix it .

Understood.

I've tried to achieve that glitch again, but no result... But from this moment i will be attaching my videos.  This shot with ISO 1600, WB 4500, everything was ok...
https://drive.google.com/open?id=15gcACm5Wyj0x2MUcQd0BU66baNNeATuA

My  mlv_rec.cfg is:

# Config file for module mlv_rec (MLV_REC.MO)

mlv.video.enabled = 1
mlv.res.x = 13
mlv.bpp = 0
mlv.aspect_ratio = 16
mlv.write_speed = 7394

Thank you for the advise, and i understood you about 100-400 iso range.

Also wanted to ask you about your plans for squeezed 4096 videos. I just think it's a good way to cover a larger sensor space for reducing the crop factor to 1.37 and get more cinematic look. Or am i wrong? And also remember that aspect ratio, in that case, was close to 2.34:1, which is awesome...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on April 07, 2019, 05:08:23 AM
Today when i was shooting a video without the crop mode for youtube it came again. Vertical lines and something strange with colors... it disappears with time.

(https://i.ibb.co/vPyjgSq/M07-0350-mov-snapshot-00-18-628.jpg) (https://ibb.co/vPyjgSq)
 
Iso 400
WB 5400
https://drive.google.com/open?id=1GaTK9QzHNCzCmXwiMNXKOxxuQrFHw8ms
If it necessary i can upload the MLV
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on April 07, 2019, 08:20:19 PM
I used the last crop_rec module and exported in h.265 (2560x1200), and colors change while I am recording, image is green at the beginning, and then it becomes normal (plus there is a lot of color noise). (it is out of focus at the beginning because of me ^^ but we can see some weird lines); On some clips I don't have this problem at all...
I will try with an older crop_rec.

And most of my clips have a very green tint
here it is : https://drive.google.com/open?id=1DDOEYHpd3js1NWW8jdeKUp0cFdbLr7-5

I can upload the mlv if needed.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on April 07, 2019, 08:40:25 PM
Probably power timing regs needs tuning. Code, code, code...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on April 07, 2019, 09:35:57 PM
i just tried the crop_rec from the old version (it does 3008x1080) and it seems to work
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 08, 2019, 04:17:38 AM
Ok thanks for the results , it looks like the real time preview reg's are the issue .
I'll look in to this further , later tonight I'll post a non real time preview for 2600x1200
so it can be used until I fix the problem .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 08, 2019, 06:17:05 AM
Ok let give this new test crop_rec 2.6k_5d2_R_T_PreView_4-7-2019_crop_rec.rar (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2.6k_5d2_R_T_PreView_4-7-2019_crop_rec.rar) a try .
I re-worked the preset a little more , reduced the raw horizontal resolution from 2632 to 2600 for a final of 2592 in mlv_rec (mod8) so same output as before
just different way of getting that same size & re-adjusted the raw preview Reg's . The horizontal raw preview reg was missed matched to the total raw width with the OB area .   




Worked best with grey scale preview ( still have a issue with full canon color) it's still realtime (buy the way) but in B/W .
I tested it at 800 ISO & 1/64th shutter speed at 29.97fps in 3x3 (FHD) when I entered in to crop_rec for 5x zoom
the shutter speed changed to 1/48th of second   ( that's by design  :D )

Also check it with the HDMI EVF , @ 480p & 1080i once again 480p work the best but the canon color is not right
I think there something in  mlv_rec preview code I need to check in to .

Let me know of any issue as before , I'm mainly want to know if that strange color/pattern show up , it shouldn't  but ......
   
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on April 08, 2019, 08:32:35 PM
once again, thank you for your work. Amazing things you done for 5d2.

In general, what would be a real benefit of shooting 2.6k, rather than 3k....beside a bit more vertical resolution?

We have now, thanks to you, very nice and stable according to my tests at least, 3k on 5d2.....15 or more seconds, but looks very nice...close to 5d3 max resolution.....less crop then 2.6k....i like it very much......but i think maybe i missed something about 2.6k build, because you guys talk about that build a lot......

Am i missing something?

tnx


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on April 09, 2019, 08:25:40 PM
Thanks for the update. But I think the problems are still the same.

The crop frame is not centered on the sensor. See the picture 2nd row 1st column, this is the 3x3 full sensor. 2nd row 2nd column is the croprec frame. 1st row 1st column I painted the used area.

Then I again get some strange problems with black level: by standard it looks a bit greenish... okay, lower black level in post.
When now scrolling through all the frames, I saw that sometimes some area gets orange blacks... see 2nd row 2nd column carefully, on top it is more orange then bottom. The orange area changes over time.

And last problem: when zooming to 100% I get a kind of bayer pattern in the debayered picture - never saw something like this before... see 1st row 2nd column.

(https://i.ibb.co/dcx4yT1/Unbenannt-1.jpg)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 09, 2019, 09:47:41 PM
I don't fix the centering yet , what to see if that odd color thing was
Still there .

I don't have any issue with debaying , I do not see those problem your reporting.
To me it looks like lens color distortion/sharpness problem.
I use MLVFS ( quick mount ) plus After Effect or MLV Producer
I get correct clean black level every time .

Try it on real scene ( not charts) and see it you get the same result
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on April 10, 2019, 09:17:10 PM
Yes, you also see it in "real world" footage. But with the chart you see it more clearly. With ACR and MLVProducer the black is off too (here uncorrected in the pictures) and the moiree looks even worse:
ACR:
(https://i.ibb.co/3d6C6WL/M09-1858-frame-138-l1.jpg)
ACR, moiree detail
(https://i.ibb.co/WPsMMdV/Bildschirmfoto-2019-04-10-um-20-55-45.png)
MLVProducer:
(https://i.ibb.co/gJmZXML/Bildschirmfoto-2019-04-10-um-20-57-01.png)
MLVProducer, moiree detail
(https://i.ibb.co/DW3jMjb/Bildschirmfoto-2019-04-10-um-21-14-03.png)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on April 10, 2019, 09:42:17 PM
Check black level auto detection code in raw.c. Or timer tweaks. No code, only guesswork.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on April 11, 2019, 01:31:04 AM
i experienced greenish blacks in my tests, to.

but that was with previous build. Did not test the latest one jet.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 11, 2019, 04:59:44 AM
Quote from: masc on April 10, 2019, 09:17:10 PM
Yes, you also see it in "real world" footage. But with the chart you see it more clearly. With ACR and MLVProducer the black is off too (here uncorrected in the pictures) and the moiree looks even worse:
Interesting , can upload a .mlv file with the problem & also a DNG please ?
The reason is I want to see the debayed image from you so I can compare on my system to see if I'm missing something .
Can you if you have time to take a short 2144x1074 .mlv in 10bit in normal 3xCrop_mode with out the crop_rec enabled , in fact can compare between 10, 12 & 14bit with the crop_rec .
Do you see black level problem in normal 3x3(FHD) 10,12,14bit ? same in normal 3x crop_mode (not crop_rec) 10,12,14bit , is it normal black levels ?

Also does 3008x1080 preset have ok black level on your system ?
Oh yea also you cam setting , preview mode , etc. ....

Edit:
I'll do some more test here and compare
Edit:#2 by chance did you use "Calibrated Colors" ?
If so Don't !!! it screw up the all the raw levels big times , I never use it .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 11, 2019, 07:49:28 AM
@masc -- 2592x1200 , exported dng's from mlv app , mlv_dump & mlvfs(quick mount) on windows7 pro
all have identical black levels & look the same  no bad black level here , can't reproduce you're problematic  results .

No W/B adjustment or any adjustment at all , just pass though from mlv file.
shot at 200 ISO , f4.0 @ 24mm on 24-70mm f2.8L canon EF , w/b 5200k manual
on B/W (Grey Scale) ML Preview Liveview .

MLV App , no correction -- dng export uncompressed
(https://i.ibb.co/59sC9Tj/mlv-app-dng-small.png) (https://imgbb.com/)

MLV_Dump , 10bit dng export
with this
mlv_dump --dng --no-fixcp M10-2118.MLV
(https://i.ibb.co/9vS2s2w/mlv-dump-10bit-dng-small.png) (https://imgbb.com/)

MLVFS (Quick Mount) 16bit dng export
(https://i.ibb.co/MVLfYGX/mlvfs-quick-mount-16bit-dng-small.png) (https://imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on April 11, 2019, 08:07:00 PM
Note, that not every clip looks like that. In average maybe 2 of 3 are okay. One of 3 has wrong blacklevel and very strange behavoir over time and much more moiree.
Standard 3x3 is always fine, at least I never had a bad clip and I can't see anything wrong there.
Here the frame as MLV, you can use MLVFS to get dng. It is not perfectly focused, it was just a quick test, when I got this. But I think it doesn't matter for your anaysis.
https://www.dropbox.com/s/gwim47xsqdbo65z/M09-1858.MLV?dl=0

My setting was the one you posted in first post in this thread. I don't know where to find "calibrated colors"... so I think I did not use it.

Edit:
Recorded some testclips again, but all fine at 2144x1074 10/12/14 bit or 1856x1044 10/12/14 bit. 2.6K setting @ 14bit kills camera -> battery pull. With todays clips I don't got the green again - last time I got several clips with wrong blacks. Same when I tried your 1st 2.6K module: 20 from 33 clips were fantastic, the rest had "changing blacks over time".
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on April 13, 2019, 08:43:37 PM
Hi, my test was okay. I'll be doing tests with high iso.
WB 5400, ISO 1600, the shutter i set to 1/33, it automatically as you said changed to 1/48 (but on the display it showed me 1/24, something like that)

(https://i.ibb.co/98RVkPL/M13-2053-mov-snapshot-00-02-421.jpg) (https://ibb.co/98RVkPL)

https://drive.google.com/open?id=1awSg_e5ifXgLB2J-xXCxpnJ8Qmo0RVZF

Here the shutter i set was 1/64  (i'm just playing for only test reasons), same as before it changed to 1/48, but i got a big horizontal line, very stylish tho...

(https://i.ibb.co/vY8F9GP/M13-2052-mov-snapshot-00-00-743.jpg) (https://ibb.co/vY8F9GP)

https://drive.google.com/open?id=1HJuuYfkMawudk-Wdf8qdaiy8SNglVpEC

Can't wait for the vertical squeezed preset for achieving less cropping by increasing horizontal resolution  :D :D :D :D :D

Also, i wanted to say, that with this build i can't format the card from the camera.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 16, 2019, 07:45:23 AM
Quote from: masc on April 11, 2019, 08:07:00 PM
Note, that not every clip looks like that. In average maybe 2 of 3 are okay. One of 3 has wrong blacklevel and very strange behavoir .......
Here the frame as MLV, you can use MLVFS to get dng. It is not perfectly focused, it was just a quick test, when I got this. But I think it doesn't matter for your anaysis.
https://www.dropbox.com/s/gwim47xsqdbo65z/M09-1858.MLV?dl=0

Thanks for the file ,
before I start to comment on this file , 
To All   :D  This is a big FYI
if ever liveview is messed up of not responding correctly (scrambled , lines or green cast)
You need to refresh liveview (this the cause of 99% of the crop_rec liveview issue reported here)
Press the PLAY Button twice (2 times , 1 to enter to play mode "view a file" 1 to exit)
this will bring you back to 3x3 (FHD) press 5x zoom to re-enter crop_rec.
the odd time you may have too repeat that process a couple a times .

Ok this file was kind of strange , all mlv->dng converters reported different things
It came down to having a corrupted frame , the only convertor that found it was mlvfs(quickmount) .
mlv producer , mlv_dump & mlv app didn't catch it , so this produce wrong white & black levels 118 1013
should be 112(black level) & 1023 (white level)

MLVFS(QuickMount)
(https://i.ibb.co/mqVmL13/mlvfs-screen-shot-small.png) (https://imgbb.com/)

MLV_Dump 10bit
(https://i.ibb.co/WW3G5wG/screen-shot-mlv-dump-10bit-small.png) (https://imgbb.com/)

The other thing I found to be a problem the shutter speed , the file report 1/40th and with
the preset , I have tuned it to 1/48th if you start at 1/64th at 29.97fps in 3x3 (FHD) .
I did find if you don't use 1/48th shutter this has cause strange liveview behavior .

One other thing that will cause corruption is having overlays enabled while recording raw video
It's far too much load on the cam .

So I don't know your setting but this is what I commend people use to be issue free .
Full manual setting (Important! No Auto setting)
3x3 (FHD) 1/64th 29.97 fps (this will change to 23.976fps @ 1/48th in crop_rec)
ISO 100,200,400 (always refresh liveview after any ISO change)
Do not enabled frame per second  override !!
Do not enable overlays in mlv_rec (default is disabled while recording raw video)
SRM (memory fill rate) "0" seem to be the best on Lexar 1066x cards (optional , shouldn't cause corruption)
Liveview preview , grey scale (B/W) low res.

this part is kind of off topic but anyways .....
The other strange thing was mlv app exported a total pink dng frame ,.

(https://i.ibb.co/YRH3Vc5/pink-dng-small.png) (https://imgbb.com/)

the png frame export was good .
I used ver. 1.6 64bit on windows7 pro  , thou I could see the frame in the preview window fine in the app.
when adjusted the black & white level (112-1023) & unchecked the "don't use camera color matrix " tab it exported a pretty good looking png frame
(https://i.ibb.co/3Y6zYHT/M09-1858-frame-1-small.png) (https://imgbb.com/)

 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 16, 2019, 08:16:25 AM
Updated the first page post with 2592x1200 test build link
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on April 16, 2019, 09:40:21 AM
Black level inconsistencies are most probable related to automation. This can be fixed since a1ex put in two options for settling black level. First one needs to know what and how regs were applied.
Seriously, I can't understand your logic with holding open source code that needs to be shared.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: benoit on April 16, 2019, 10:43:55 AM
Quote from: Danne on April 16, 2019, 09:40:21 AM
Seriously, I can't understand your logic with holding open source code that needs to be shared.
Me too. 6 months of personal coding on one preset with bugs that other people have already see on other plateforms ...
IMHO It's non-sense on an open source forum. >:(
Share is pure power, just think what will be Magic Lantern if a1ex post .mo files with as explanation "i'm busy just read my posts" ...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on April 16, 2019, 11:05:49 AM
Quote from: reddeercity on April 16, 2019, 07:45:23 AM
...
You need to refresh liveview (this the cause of 99% of the crop_rec liveview issue reported here)
Press the PLAY Button twice (2 times , 1 to enter to play mode "view a file" 1 to exit)
this will bring you back to 3x3 (FHD) press 5x zoom to re-enter crop_rec.
the odd time you may have too repeat that process a couple a times .
I did.
Quote from: reddeercity on April 16, 2019, 07:45:23 AM
Ok this file was kind of strange , all mlv->dng converters reported different things
It came down to having a corrupted frame , the only convertor that found it was mlvfs(quickmount) .
mlv producer , mlv_dump & mlv app didn't catch it , so this produce wrong white & black levels 118 1013
should be 112(black level) & 1023 (white level)
Black level is different very often, mostly something between 110 and 118.
Quote from: reddeercity on April 16, 2019, 07:45:23 AM
...
The other thing I found to be a problem the shutter speed , the file report 1/40th and with
the preset , I have tuned it to 1/48th if you start at 1/64th at 29.97fps in 3x3 (FHD) .
I did find if you don't use 1/48th shutter this has cause strange liveview behavior .
Okay...  I sometimes forget this, but I'll take care in future.
Quote from: reddeercity on April 16, 2019, 07:45:23 AM
One other thing that will cause corruption is having overlays enabled while recording raw video
It's far too much load on the cam .
Hm... I just had this B/W realtime preview with nothing else shown when recording.
Quote from: reddeercity on April 16, 2019, 07:45:23 AM
Full manual setting (Important! No Auto setting)
3x3 (FHD) 1/64th 29.97 fps (this will change to 23.976fps @ 1/48th in crop_rec)
ISO 100,200,400 (always refresh liveview after any ISO change)
Do not enabled frame per second  override !!
Do not enable overlays in mlv_rec (default is disabled while recording raw video)
SRM (memory fill rate) "0" seem to be the best on Lexar 1066x cards (optional , shouldn't cause corruption)
Liveview preview , grey scale (B/W) low res.
Shutter... okay... I sometimes forget about, but the other settings should be as you described.
Quote from: reddeercity on April 16, 2019, 07:45:23 AM
The other strange thing was mlv app exported a total pink dng frame ,.
...
the png frame export was good .
I used ver. 1.6 64bit on windows7 pro  , thou I could see the frame in the preview window fine in the app.
Thanks. I found the bug. It was only when exporting the last frame in a MLV as DNG using "Single Frame Export". "Normal Export" (DNG) was working.
Quote from: reddeercity on April 16, 2019, 07:45:23 AM
when adjusted the black & white level (112-1023) & unchecked the "don't use camera color matrix " tab it exported a pretty good looking png frame
For me it is still not really good looking. E.g. the blacks get still orange on top. Just the contrast and saturation is a bit lower.

What exactly was changed with the new build?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 18, 2019, 05:19:42 AM
Quote from: masc on April 16, 2019, 11:05:49 AM
What exactly was changed with the new build?
https://www.magiclantern.fm/forum/index.php?topic=19336.msg214773#msg214773

I'll test record a bunch a clips (20) and see if I can reproduce your odd black & white levels




@masc Ok I can't re-produce your problem , I recorded 13 clips(17.6GB) in a row with high & low contrast .
I recorded until It skipped a frame & stop , then I press record again & so on until I had 13 clips .
I was getting around 400 frames on average before it stopped .

Here are my setting for the test .
(https://i.ibb.co/xjQzPnJ/VRAM5-small.png) (https://imgbb.com/)

So at this point I'm going to say it's a cam setting issue e.g. shutter etc. ...
I'll center the image better & if I have time I'll fix the pinkish canon full color liveview .

I want to move on to finishing coding the other presets e.g. hi frame rate (48fps 3x3 FHD) & 4k anamorphic ,
plus I need to get back on CF card over clocking .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: honza123 on April 18, 2019, 08:56:14 AM
Thumbs up  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on April 28, 2019, 11:03:06 PM
Guys -- You should be testing and reporting on the builds that reddeercity has posted so he can continue coding knowing what he has done so far is working:

Quote from: reddeercity on April 18, 2019, 06:58:33 AM
I want to move on to finishing coding the other presets e.g. hi frame rate (48fps 3x3 FHD) & 4k anamorphic ,
plus I need to get back on CF card over clocking .

Repeating, "Are we there yet?" won't get you anywhere. Believe me!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on April 29, 2019, 02:48:21 PM
Look, actually I think, I can't be useful for Reddeercity more with this new preset, because already posted a small test and told that everything works as described.
But also have problems... But they are of the camera i guess

1. Sometimes there are vertical lines and crazy temperature (WB) changing. There it is:

(https://i.ibb.co/j3X5gpq/Screenshot-34.png) (https://ibb.co/j3X5gpq)


That's the video:
https://drive.google.com/file/d/1a6PLP2R4e1wLgdBBx8JNwb3OKRDcGV7h/view?usp=sharing

It disappears when I press set button for STOP rolling, and then, when i press REC it appears again. So there are two ways to avoid it: 1. restart the camera 2. wait damn long time watching crazy color changing and vertical lines count increasing, so after a few minutes regularly they are going away.
It happens not only when shooting raw, but I noticed that it took place when the color profile set to cinema style (downloaded from the Technicolor webpage if I remember right)

2. Zoom jumping out.
I had this problem since bought the camera. Already described it here. When I magnify to x5, it jumps out to x1 in most of the cases, so it's truly hard to enable crop rec module and start recording in it.
Guys, maybe you know why is it happening?
That's how it looks from the side (shot on my phone):
https://drive.google.com/file/d/1RSBxPpdYgM38a9qRJfi9IoZbfb3zlUrs/view?usp=sharing

But sometimes everything is ok:

https://drive.google.com/file/d/1CPDy5NCb8kvK4Jhgeh9gWG3VFfBm3K8g/view?usp=sharing (from side)
https://drive.google.com/file/d/1lqBY8uhzRFfAD9oecv_VeNhSvlkJYloO/view?usp=sharing (actual footage)

So, i'm waiting for 4k squeezed preset to compare it with regular 1856 (2.34:1). Just wonder if we really can spot the resolution difference  :D :D. And 4k - it's a good way to decrease the crop factor also.

MLV_rec.cfg of that test:
# Config file for module mlv_rec (MLV_REC.MO)

mlv.video.enabled = 1
mlv.res.x = 10
mlv.bpp = 0
mlv.write_speed = 7227

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on April 29, 2019, 07:04:48 PM
you have this problem (red lines) only with 2.6k build?

I have the same issue with 1:1 mode in h26h (it is in 3.5k centered module).....it si mode for shooting in 3xcrop mode in h264.....it is useless...but your video (with red vertical lines) reminds me on it
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 30, 2019, 06:23:21 AM
Quote from: Igor_Braun on April 29, 2019, 02:48:21 PM
But also have problems... But they are of the camera i guess
......
1. Sometimes there are vertical lines and crazy temperature (WB) changing. There it is:

(https://i.ibb.co/j3X5gpq/Screenshot-34.png) (https://ibb.co/j3X5gpq)

It disappears when I press set button for STOP rolling, and then, when i press REC it appears again.
So there are two ways to avoid it: 1. restart the camera
.....
It happens not only when shooting raw, but I noticed that it took place when the color profile set to cinema style (downloaded from the Technicolor webpage if I remember right)
Thanks for feedback , which crop_rec built are you using ? there 3 different builds I have on my download page by the way .
Similar problem that  @ masc had , camera setting issue , Technicolor Cinestyle picture style has nothing to do with it ( I've had installed on my cam since 2012 & do all my tests with it)
It came to the shutter speed , I've tuned/locked the preset to be 180 degrees 1/48th second @ 23.976 fps , and any other setting mess up liveview
here the fix https://www.magiclantern.fm/forum/index.php?topic=19336.msg215200#msg215200
I see in the video you have 1/38th shutter speed in crop_rec 2.6k & 1/50th in 3x3 (FHD) .
You Must set the cam to 1/60th (1/64th close enough) 29.97 fps in 3x3 (FHD) when crop_rec enabled it will change to 1/48th 23.976 fps automatically
If you change this you will have problems , as seen in your post.
Think of the 5d2 as a Manual 24p film camera
(there's a long answer , but I don't have the time right now , it has to do with the Headtimers 3 ,4 & sensor's 4 channel readout @ 24Mhz ) 
Quote from: Igor_Braun on April 29, 2019, 02:48:21 PM
2. Zoom jumping out.
I had this problem since bought the camera. Already described it here. When I magnify to x5, it jumps out to x1 in most of the cases, so it's truly hard to enable crop rec module and start recording in it.
Guys, maybe you know why is it happening?
I believe it's the same as problem #1 see answer above ,
plus I'll add a remember here about cleaning up liveview problems with simply pressing the play button twice (2 times)
Once to view a image (h264 or Cr2) press button once more to exit in to 3x3 (FHD) then re-enter crop_rec
no need to restart the camera .
So to recap , shutter in crop_rec needs to be 1/48th seconds , No exception's period .
Set 3x3 FHD to 1/60th & in the canon menu set 1080p to 29.97 fps , if you want 24p in 3x3 just enabled "frame override" and set it 24 from 30 .
By side that why would you want to change shutter from 180 degrees @ 24p ?

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on April 30, 2019, 08:49:38 AM
Thanks for reply. 

@Banertop
About red lines. They began to apper relatively recently when i installed one of the experimental builds. No matter if shooting raw or h264, if enabled raw module or no.  Actually if I delete magic lantern (waiting 60 seconds after install complete) it appears anyway... but really, noticed it with only cinestyle enabled. By the way, Banertop, how do you use (where did you find it) 3x3 crop module for h264 for 5dmkii? Before I was using it with 650d when needed to zoom in (shooting concerts for example), so when bought 5dmkii I could not use it anymore.

@Reddeercity
I use magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212.zip build with 2.6k_5d2_R_T_PreView_4-7-2019_crop_rec.rar crop_rec module.
Understood you about the shutter speed. Actually didn't pay attention to it this time... regularry I use 24 fps (25 for TV, 29,90 with your build) with 1/50 shutter speed when shooting and never change it. But 1/38 it's something that happened without my participation. Will set it to 1/64 next time.
And about zoom jumping out. Actually I has it since I bought the camera no matter if it has ML installed... understood you about "manual film camera". Will wait for it anyway
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on April 30, 2019, 11:57:27 PM
@igor_braun
about 3x crop mode in h264......in latest experimental builds, under crop_rec mode, there is 3 options, 1:1, 3.5k centered, and can not remember the third one.....Just choose the 1:1, and disbale the raw_rec.....

hope this help
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 01, 2019, 07:15:32 AM
Quote from: banertop on April 30, 2019, 11:57:27 PM
@igor_braun
about 3x crop mode in h264......in latest experimental builds, under crop_rec mode, there is 3 options, 1:1, 3.5k centered, and can not remember the third one.....Just choose the 1:1, and disbale the raw_rec.....

hope this help
What the heck are you talking about ?
Show me proof that this works on 5D2.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on May 01, 2019, 10:18:41 AM
Banertop, can you please show how you do that?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dfort on May 01, 2019, 02:42:33 PM
I don't know how it works on the 5D2 but on other cameras some of the crop_rec settings still work in H.264.

Check out the first post of the Crop mode recording (crop_rec.mo) (1:1, RAW/H.264, 25/30/50/60 fps) (https://www.magiclantern.fm/forum/index.php?topic=17021.0) topic.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on May 01, 2019, 05:17:46 PM
Yes, that is it>))

thank you dfort!!

i will send a prooff as a video, soon




ok guys,

about shooting h264 in 3x crop on 5d2....

so, it is 1080 1:1 mode , under 3.5 crop mode.....it is nothing new,  I use it with 5d3 couple of times, and i wish it works with 5d2 (without red lines) if i need some aliasing free shoots in h264...

video as a proof, as you asked for it, on this link
https://youtu.be/o6XSPyI9lrk
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 01, 2019, 07:15:41 PM
Like I said it dosn't work , I know about this and records garbage
Image !! There nothing here . Show me the H264 file with clean images
So this is BS plain simple .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on May 01, 2019, 07:58:16 PM
If you share builds, share code. Plain and simple:
https://www.gnu.org/licenses/gpl-faq.html#GPLRequireSourcePostedPublic
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on May 01, 2019, 09:33:11 PM
@reddeercity

I never said it works.....I mention it when red lines problem come into discussion, because the red lines in that mode (1080 1:1), reminded me on sample that Igor_Braun shared....(his red lines problem)

As i mention before, it is not a new thing, i used this preset couple of times with 5d3

By the way, it works just fine with 5d3....no red vertical lines, or other artifacts.....But in general, no real use of it....to much croped




to reddeercity

one more thing...no need to be rude....(BS things.....)we are on the same side here...

We are thankful for everything you did with 5d2.....
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: theBilalFakhouri on May 01, 2019, 10:11:24 PM
@banertop

What happens if you record H.264 in 1920 1x1 on 5D2 ? Can you share a video recorded with that mode or it's a fully corrupted frames in H.264?

I think these red lines can be fixed with CMOS registers,  try to refresh LiveView manfully press Menu button then get back to LiveView after enabling 1920 1x1 preset if nothing improved I am fairly sure it can fixed with CMOS registers, I wish I have that camera to work on since it's from the golden days and history.


Not sure why reddeercity is getting angry very fast, it doesn't worth it. We all trying to help here.

Quote from: reddeercity on December 06, 2017, 06:39:28 PM
I'll ask nicely one more time , please stay ON TOPIC !!!
If that can not happen then I will remove this thread all
Together , as I started this thread and all information will be
Lost .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on May 01, 2019, 10:18:43 PM
Of course it can be fixed. Co-operation level is non existent. Simple gpl rules don't apply here evidently.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Audionut on May 02, 2019, 05:22:33 AM
I need some clarification about what is going on here.

Quote from: reddeercity on March 14, 2019, 04:04:41 AM
There's no new code yet , I'm doing everything though adtg_gui_.mo
Now that should clear up things about code etc. ....

Then two posts later.

Quote from: reddeercity on March 21, 2019, 05:58:32 AM
and writing code for crop_rec 5d2.

I would remind the OP regarding these two posts.
Quote from: Official statement about ML's GPL license and its violation on September 17, 2014, 09:32:58 PM

  • If you use the code, or parts of it and distribute it (or even sell it), you must release this code (per the GPL).

About forking Magic Lantern and the support that can be provided on this forum (https://www.magiclantern.fm/forum/index.php?topic=11080.0)


I've seen some builds being posted here by the OP.  Now, for the sake of the argument, even if no modification was being made to said code, you still need to ensure that the readers are exactly aware of the source code being used in these builds being posted, so that they may compile their own builds based on the exact same source (it's just an open source thing).

However, given that builds are being distributed, and the OP is not simply sharing ADTG settings, I have to assume that there has been modification of Magic Lantern code.  In which case, you must share your source tree.  Not just because the GPL says so, but because of respect for this open source community.




I have also cleaned some completely unnecessary duplicate posts from this thread.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 02, 2019, 07:13:21 AM
Not to offend any body 
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Crop_Rec-4k-5d2-50d-magic-lantern_1-9-2019.zip
Source has been there since Jan. 1 2019  If any one would have care to check .
But the main troller here never thought to check my bitbucket downloads , I was seeing how far he would go.

I have to question why I'm doing this , with all this grief I getting .
I have better thing to do with my time then this , (I do promise to finish crop_rec before I say goodbye to magic lantern)
And I'll say this one last time , every thing I've done (including code bits & agtd_gui stuff) is all in this thread
the problem is every one is too lazy to read a few pages .
I have also said that after I'm done the crop_rec for 5d2 that I would post the code for everything ,
though It may take me a few mouths  or more but that life , I'm in no rush to finish this .
I guess that the problem , after I spend the last 18 mouths figuring this all out every one want my knowledge NOW !
So this will be my last and only words on this . 

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on May 02, 2019, 07:32:09 AM
Your branch is lastly update in october? We want clearance for the last versions of crop_rec.mo you are posting.
Please do everyone a favour and start by next to every build you are posting simply add the crop_rec.c itself. Can´t be hard to do. This way we can help out with things. I am not even being personal about this.
Consider following. Everyone stopped sharing code and only posted compiled builds? Not only would progression completely stop but the level of uncertainty would increase to the level that no coder/programmer would even touch the compilations and that would be the end of it.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Audionut on May 02, 2019, 10:33:25 AM
Quote from: reddeercity on May 02, 2019, 07:13:21 AM
Not to offend any body

No offense taken.  I mean no offense with my post style either.

Quote from: reddeercity on May 02, 2019, 07:13:21 AM
Source has been there since Jan. 1 2019  If any one would have care to check .

Which is now four months old.  The last change to that source provided appears to be 24/10/2018, which is over six months old.  Given the ease with which one can zip a directory and upload it, I really don't think that anybody is asking that much, when they ask for an updated copy of the current working source.

Magic Lantern does not follow this style of collaboration.  When a new build is released, the source code for that release is made available at the same time. 

Quote from: reddeercity on May 02, 2019, 07:13:21 AM
I have also said that after I'm done the crop_rec for 5d2 that I would post the code for everything ,
though It may take me a few mouths  or more but that life , I'm in no rush to finish this .

No one is rushing you to finish anything.  This is a hobby project.

Quote from: reddeercity on May 02, 2019, 07:13:21 AM
I guess that the problem , after I spend the last 18 mouths figuring this all out every one want my knowledge NOW !

I guess that the problem, we waited to long before we ask you to adhere to the open source ethos, and our Official statement about ML's GPL license (https://www.magiclantern.fm/forum/index.php?topic=13335.msg128757#msg128757).

Your work is a derivative of code that has been provided to you for free (free as in beer (http://en.wikipedia.org/wiki/Gratis_versus_libre)), which is "a result of several thousand hours of work, either researching or programming", by others.

Quote from: reddeercity on May 02, 2019, 07:13:21 AM
So this will be my last and only words on this .

Quite frankly, your post translates to, "fuck you, fuck all of the time all you guys spent on this, I'll do whatever the fuck I want with the code all of you guys spent countless hours developing, because fuck what you guys think is appropriate". 

This is not conductive to a healthy open source development based community.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on May 02, 2019, 02:04:57 PM
 :(
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on May 02, 2019, 10:13:04 PM
@theBilalFakhouri

i will try it, but in general, that mode is not very interesting at all...it was more about red lines problem that one user have with his 5d2.....but, thanks for your input, i will try to refresh live view...

@reddeercity

we all know that you give yourself for this build, try not to me emotional....everything we said is in best fate....we want to help with comments...sometimes the comments are stupid, but we are not all experts for coding....

again, we are thankful, just keep it on, and try to understand that all comments here are in sake of best build for 5d2.....and you are so close to it
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on May 02, 2019, 11:32:02 PM
I don't have a working 5D2 right now, not sure if I will again, but I really want this to work out!!! pls calm down everyone

@reddeercity We only want code for uploaded builds,  that's all. You only change crop_rec.c right?, so uploading just that is enough.

Quote from: Danne on May 02, 2019, 07:32:09 AM
Please do everyone a favour and start by next to every build you are posting simply add the crop_rec.c itself.

I am sorry for filling up the thread with this, I will delete this post once everything is resolved
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on May 03, 2019, 01:44:57 AM
EDIT - I don't need to contribute to bad vibes. Appreciate everyone's hard work.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on May 03, 2019, 12:11:57 PM
Hacking the hack. Let's take one of the builds from first post:

Quote from: reddeercity on April 06, 2017, 12:22:27 AM
April 16 2019
2.6k_5d2_R_T_PreView_4-7-2019_crop_rec.rar (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2.6k_5d2_R_T_PreView_4-7-2019_crop_rec.rar)

It contains a single file: crop_rec.mo. Put it in your magic-lantern/modules directory and run:

./module_hginfo_dump.sh crop_rec.mo


Result:

Build date  : 2019-04-05 04:19:36 UTC
Build user  : david@reddeercity

diff -r 8bfbb0ca228e modules/crop_rec/crop_rec.c
...
< what follows is the complete source code of crop_rec, displayed as diff from changeset 8bfbb0ca228e >


Changeset 8bfbb0ca228e is public (https://bitbucket.org/waza57/magic-lantern/branch/crop_rec_4k_5D2) (exactly where reddeercity indicated). The diff embedded in the binary module (crop_rec.mo in our case) contains all changes to that module, since the reference changeset.

Et voilà, you've got the complete source code.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on May 03, 2019, 12:34:04 PM
Nice ;)

Maybe we should make an agreement with reddeercity - keep releasing as before, and we will extract the source using that method.

Thoughts ? (@Danne/@dfort/@Audionut/@anyone who's said anything about this)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on May 03, 2019, 12:43:35 PM
Quote from: a1ex on May 03, 2019, 12:11:57 PM
Hacking the hack. Let's take one of the builds from first post:

It contains a single file: crop_rec.mo. Put it in your magic-lantern/modules directory and run:

./module_hginfo_dump.sh crop_rec.mo


Result:

Build date  : 2019-04-05 04:19:36 UTC
Build user  : david@reddeercity

diff -r 8bfbb0ca228e modules/crop_rec/crop_rec.c
...
< what follows is the complete source code of crop_rec, displayed as diff from changeset 8bfbb0ca228e >


Changeset 8bfbb0ca228e is public (https://bitbucket.org/waza57/magic-lantern/branch/crop_rec_4k_5D2) (exactly where reddeercity indicated). The diff embedded in the binary module (crop_rec.mo in our case) contains all changes to that module, since the reference changeset.

Et voilà, you've got the complete source code.

dfort did the same to complete unreleased code as well but frankly, who cares? And if this is the important part of this, to me, very strange "collaboration" approach, it can´t take long to backtrack several places where nothing is returned when asking for it. Even if everything is published(i know it isn´t and frankly I don´t care about the code), it´s faul play in my book, he´s even admitting that the very reason for keeping posts coming is about testing "trollers"? Come on. Is this what this forum is about? Honestly, I thought more of this place...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on May 03, 2019, 12:56:01 PM
I agree, things could be a lot better. What I mean is that reddeercity did not actively take steps towards hiding the source code. Rather, I believe he just had some difficulties getting started with Mercurial.

TLDR - yes, it's not the preferred way of sharing things, but I don't see it as a GPL violation.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on May 03, 2019, 01:01:43 PM
Quote from: a1ex on May 03, 2019, 12:56:01 PM
TLDR - yes, it's not the preferred way of sharing things, but I don't see it as a GPL violation.
That's a good thing though.
I would label it communication breakdown. Leaning towards either getting this cam myself and do all presets or take active measures to stay off this topic completely(no threat intended)  8).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on May 03, 2019, 01:05:55 PM
No need, my plan is to make crop_rec fully generic on all models (arbitrary resolutions, binning factors and frame rates, within the sensor readout speed limits).

That is, I'm going to use the current code (preset-based) only as documentation.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on May 03, 2019, 01:17:16 PM
Didn't see that one coming. Looking forward to this coded artwork.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Audionut on May 03, 2019, 01:30:21 PM
Ah.....thanks a1ex.

I did check back through several pages, hence the merged duplicate posts, and I did check the OP, but it was a speed-read of the OP.  No apologies though, because.....

One could have taken the high road (https://www.urbandictionary.com/define.php?term=take%20the%20high%20road) and simply linked to the correct current source, everything sorted.  Pats on the back, apologies.  I mean, you could even link to the current source, and still take a jab at the lack of reading skills of the participants of the conversation.

.......the op got what he was after!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on May 03, 2019, 10:30:23 PM
Ugh.. That was the "coming of gods" time... So, i just wanna say to @banertop, that my 1x1 on h264 is absolutely distorted, but not with lines... It was absolutely wrecked.
I do not know if this will be any use in this thread but anyway just in case.

(https://i.ibb.co/SdYJj2n/MVI-9967-MOV-snapshot-00-01-979.jpg) (https://ibb.co/SdYJj2n)


Video from phone (sorry for shaky footage):
https://drive.google.com/file/d/1Rwb-UcuOI658NBkzb3biFNm-PGJScFrk/view?usp=sharing

Footage from camera:
https://drive.google.com/open?id=1Lye4j21sme_KfOt8bBoIVQy-tIphahHT

Also, my 5d2 started to have some problems with turning on, I don't know if it's related to the ML build:
https://drive.google.com/file/d/1vGMVn1rD5uO7QnOF6xHv0GOPU8Zh7h1f/view?usp=sharing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on May 03, 2019, 11:38:48 PM
@Igor_Braun
Dont be sad about 1:1 mode.....it is really not worth it.....just an interesting idea, to shoot h264 in crop mode :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on May 04, 2019, 01:49:34 AM
@banertop, Yeah... And as i said, it was very very useful on my previous 650d when shooting some kind of events
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on May 04, 2019, 04:05:43 PM
I tested the new crop module and at the beginning I thought everything was great, no stripes or anything and I can change the Shutterspeed as I want, but ... later i see "sometimes" (not everytime or always) false greenish blacklevel for some sec.
Yes, I record ALL in manuel mode and i have testet the 1/48 shuterspeed too -> same resultat

Btw. if i set shuterspeed to 1/33, it set to 1/26 -> good for low light

Here a short testfootage, no correction, F11,3, iso 100, 1/48, 10bit (112 black, 1013 white), temp. 5800 -> false blacklevel here at the beginning :
https://www.dropbox.com/s/n11skw92t0fo6et/M04-1357.mov?dl=0
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on May 04, 2019, 09:40:20 PM
By the way, i can confirm that, i have the same thing that Bender@arsch has
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Macke1 on May 06, 2019, 07:52:41 PM
it´s this the latest try out? https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/2.6k_5d2_R_T_PreView_4-7-2019_crop_rec.rar
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on May 06, 2019, 07:59:23 PM
See first post in this thread.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Macke1 on May 06, 2019, 09:23:04 PM
ok, i forgot it that the 1 is in the first post, sorry but i can't see 30 only 24 25. this image shows 30
(https://i.ibb.co/yNPr96M/VRAM6-Small.png) (https://ibb.co/yNPr96M)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: kitor on May 07, 2019, 08:23:38 AM
Change PAL/NTSC region in camera.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Macke1 on May 07, 2019, 05:31:14 PM
#kitor Thanks will try when i coming home! it worked thanks!

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 12, 2019, 06:26:34 AM
 @a1ex yes, you're exactly right .

Just pick up a cheap 50d body from my local camera store with 90 day warranty  :D
when I have time I want to build on @aprofiti early work & porting 3k/UHD crop_rec to 50d
Report : Canon EOS Digital INFO - v1.4 - CANON EDSDK v2.14
-------------------------------------------------------------------------
Camera Product Name : Canon EOS 50D
Serial Number : XXXXXXXXXXX
Firmware Version : 1.0.3
Shutter Counter : 21006
Battery Level : Battery Level : 80%
Camera Date/Time : Saturday, May 11, 2019 , 11:48:43 PM

Still need to be updated to firm 1.09v , only has 21k shutter activations
Very good shape , just like new almost , $300 cad ($225 usd) .

So I'm wondering now after all that did I waste my time getting this 50D ?
Is the drama over now , can I get back to work ?

FYI , I have always known that a1ex was going to make a generic crop_crop for all cams
but I couldn't wait that long , and this thread is the result of the effort .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on May 12, 2019, 09:03:27 PM
well guys,

after some testing, i can say that 2.6k build is the best thing ever happened to 5d2!!!

More useful than anything before -......very filmic, nice resolution, very nice aspect ratio, not so terrible crop with lens like sigma 18-35....

For now, only con is aliasing

5d2 is finally so close to be a true film camera....

BUT....is it possible to deal with aliasing and moire problem? In other crop modes there is no aliasing issues
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 13, 2019, 05:08:32 AM
Quote from: banertop on May 12, 2019, 09:03:27 PM
BUT....is it possible to deal with aliasing and moire problem?
show me please , upload a dng or short mlv file .
It can't happen in 5x zoom , because you are in true 1:1 pixel so there no pixel binding or line skipping .
If I open up the horizontal & vertical to the maximum (5632x3752) which is = to the full res cr2
https://www.magiclantern.fm/forum/index.php?topic=19336.msg205622#msg205622

I have canon "L" lens and I never seen aliasing and moire pattern in crop_rec ,
I use my EF 24-70 f2.8L at 24 in crop_mode and no issue .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 13, 2019, 06:13:52 AM
ran a adtg_gui log in 1080 mode & 5x zoom mode on the 50d
Looks almost like the 5d2 exactly , just different cmos numbers
shouldn't be too hard to get this in the D4/5D2 crop_rec code .
50D-adtg_gui_1080mode.log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D-adtg_gui_1080mode.log)
50D-adtg_gui_5xZoom.log (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D-adtg_gui_5xZoom.log)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on May 13, 2019, 12:59:22 PM
ok...on this link....aliasing an moire in 2.6k  mode

https://youtu.be/UH_scf54tpY

link to download mp4
https://wetransfer.com/downloads/55fca1b8ee5dad964331171119c1c0d420190513112737/849be0
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 13, 2019, 11:55:32 PM
No a original dng & or a short mlv file , all  I see is bad h264 compression (blocky) & hi ISO noise 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 14, 2019, 02:14:58 AM
Got some interesting results from the 50D today with adtg_gui

(https://i.ibb.co/88fhCPR/50-D-M13-1935-00001-small.png) (https://imgbb.com/)

4640x1078 M13-1935_00001.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M13-1935_00001.png)
4640x1078 M13-1935_00000.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M13-1935_00000.dng)
Short 11 second h264 clip 4640x1078_8fps-M13-1935.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4640x1078_8fps-M13-1935.mov)

The only real differences I found was sensor pinning is done by CMOS[4] instead of CMOS[2] on 5D2

(https://i.ibb.co/MCyPzG8/VRAM4-small.png) (https://imgbb.com/)
in 5x zoom default = 0x484 ,  to center the image to the right decrease the second valve (0x424)
I could have move the image a little more to the right in the 4640 test but I have only 1 battery so time is limited
and I wanted to get as much done as possible before battery dies .
So that's close to the maximum horizontal resolution of 4752

I did notice with the mlv's the write speed was 83MB/s , very nice more then 5d2 (75MB/s)
less overhead in the 50d . and with the faster sensor speed 28Mhz I think the 50D
will get a higher resolution then 5D2. But time will tell .


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on May 14, 2019, 02:34:57 AM
Some old notes which can come in handy on horizontal (https://www.magiclantern.fm/forum/index.php?topic=19336.msg210099#msg210099) and vertical (https://www.magiclantern.fm/forum/index.php?topic=19336.msg210344#msg210344), plus a spreadsheet (https://docs.google.com/spreadsheets/d/1pXZGJov8V-5K-ESiRTn5Z7zgydLhvNhUMqLdLMpEKVQ/edit#gid=0) of the initial analysis.

Cmos[5] may be responsible of vertical crop area position; expected full-res up to 7 FPS compared to 5 FPS 5d2, due to faster clock speed.

Haven't tried yet to increase vertical resolution and reduce to minimum timing, because I'm currently experimenting in other areas.
I expect to see some interesting results from you, reeddercity. Good luck! :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 14, 2019, 07:16:47 AM
Yes that's right cmos[5] does the vertical offset , tried to increase vertical to 1200 , got it close but can't find the right vertical offset 
have to do it by trial & error and that eat up battery time .

On a happier note , the 50D is a beast ! without trying too hard
I could get 3200x1080 @ 23.965fps (battery running low so I stopped , did have it close to 25fps at times)
Thinking I my get close to UHD(3840) or 3520 with the 50D , thou the vertical is a different story , but the future looks bright for the 50D
So don't sell your 50D yet it my be one of the best for crop_rec.

Here are some test images & H264 files
(https://i.ibb.co/zsdBs1R/3200x1080-23-965fps-M14-0016-00001-small.png) (https://imgbb.com/)
50D_3200x1080_23.965fps_M14-0016_00001.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_3200x1080_23.965fps_M14-0016_00001.png)
50D_3200x1080_23.965fps-M14-0016_00000.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_3200x1080_23.965fps-M14-0016_00000.dng)
50D_3200x1080_23.965fps_M14-0016.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_3200x1080_23.965fps_M14-0016.mov)

 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 15, 2019, 08:08:01 AM
Looks like I found the horizontal limited @ 23.976fps

(https://i.ibb.co/NN4Yxj7/50-D-3744x1080-23-976fps-VRAM10-small.png) (https://imgbb.com/)

I can get about 7 seconds (177 frames) write speed needed is 115MB/s , I get 75-80MB/s .
But I'm really liken this APS-C (https://en.wikipedia.org/wiki/APS-C) small (22.3×14.9 mm 4752×3168 ) sensor image , very clean & sharp  :D

Some more samples images & h264 clip
(https://i.ibb.co/9w26FYz/50-D-3744x1080-23-976fps-M15-0046-00001-small.png) (https://imgbb.com/)
50D-3744x1080_23.976fps-M15-0046_00001.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D-3744x1080_23.976fps-M15-0046_00001.png)
50D-3744x1080_23.976fps-M15-0046_00001.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D-3744x1080_23.976fps-M15-0046_00000.dng)

Left hand side has a 200w incandescent light bulb - cam setting 100 ISO , 1/27th shutter, f8 24mm , crop factor 2.0 =48mm (Canon EF24-70mmf2.8L)
Has great dynamic range .
(https://i.ibb.co/3d8fNZw/50-D-3744x1080-23-976fps-M15-0046-00154-small.png) (https://imgbb.com/)
50D-3744x1080_23.976fps-M15-0046_00154.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D-3744x1080_23.976fps-M15-0046_00154.png)
50D-3744x1080_23.976fps-M15-0046_00154.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D-3744x1080_23.976fps-M15-0046_00154.dng)

h264 clip,  7seconds 50D-3744x1080_23.976fps-M15-0046.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D-3744x1080_23.976fps-M15-0046.mov)

So after this test I think I should be able to get 3200x1300 @ 23.976fps being I'm at 3744x1080 right now .
I may look at getting another 50D ( they're dirt cheap) at this rate , if I can get CF card over clocking working. :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on May 15, 2019, 01:08:48 PM
@Reddeercity I missed why we are talking about a 50D on a 5D2 Thread, I'm game but I just want to see what's the relevance to the 5D2. Have we reach the full width of 3x3 5D2 at 1856, is it official that the most possible stable for 1x1 and 3x3 is 2.6k_5d2_R_T_PreView_4-7-2019_crop_rec and the one thing we have to remember is that we need to push play twice to return to FHD 3x3 mode from 1x1 crop mode, additional we have to delete files in computer because reformat in camera delete firmware also. Thanks for everything you do Magic Lantern & Reddeercity for us 5D2 devotee.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on May 15, 2019, 04:05:17 PM
Very impressive how quickly you have made progress on the 50D!

Looks like good dynamic range and very clean, maybe it has better quality than 5D2 ??? Sensor a bit small though...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on May 15, 2019, 08:19:24 PM
Quote from: Bender@arsch on May 04, 2019, 04:05:43 PM
I tested the new crop module and at the beginning I thought everything was great, no stripes or anything and I can change the Shutterspeed as I want, but ... later i see "sometimes" (not everytime or always) false greenish blacklevel for some sec.
Yes, I record ALL in manuel mode and i have testet the 1/48 shuterspeed too -> same resultat

Btw. if i set shuterspeed to 1/33, it set to 1/26 -> good for low light

Here a short testfootage, no correction, F11,3, iso 100, 1/48, 10bit (112 black, 1013 white), temp. 5800 -> false blacklevel here at the beginning :
https://www.dropbox.com/s/n11skw92t0fo6et/M04-1357.mov?dl=0

I fixed my own problem by changing only one setting:
QuoteDo not enable overlays in mlv_rec (default is disabled while recording raw video)
Now all works great, thanks Reddeercity!

Your 50D Projekt sounds realy good;)

btw. can you make a Build with both latest crop moduls in one?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on May 16, 2019, 08:43:58 PM
@Reddeercity

You asked for the original mlv file to see aliasing and moire in 2.6k crop mode...

her is the original mlv file to download

https://wetransfer.com/downloads/cf61b9e2e0f7948bf1e42fea2b7ce82f20190516175713/87be95

Side note, with this and with 3k build, when formating card in camera, it delete ml from card

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: LEVISDAVIS on May 17, 2019, 09:25:42 AM
Downloaded 04-07-2019 CROP_REC.MO for 5D II.  Un-zipped the .RAR and installed to Module to the CF card.  I'm getting an error when loading.

Question.  Does Crop_Rec.MO not go to the Modules Folder?

I get an Err for all Modules when loading Crop_Rec.MO.  When powering off, the Error says it failed to link modules.

-----

Proudly own a 50D...  Will test functionality of further Crop_Rec.MO developments...  With your help.  Thanks.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 18, 2019, 06:23:17 AM
Quote from: banertop on May 16, 2019, 08:43:58 PM
@Reddeercity
You asked for the original mlv file to see aliasing and moire in 2.6k crop mode...
her is the original mlv file to download
Thanks got it , Ok now I see it in the Blinds in the windows .
After carefully examining the file I found user/cam setting issue problems.
Your shutter setting is far too high (very short exposure time)

(https://i.ibb.co/jRJr5yn/Bad-shutter-5-D2.png) (https://imgbb.com/)

You have 1/362th second @ 24 degrees !
This well cause aliasing big time , you should have 1/48th second @ 180 degrees .

This the metadata from my camera set correctly , note shutter angle & aperture .
(https://i.ibb.co/VwWD4gv/Good-shutter-5-D2.png) (https://imgbb.com/)

Please take this with a grain of salt , you need to understand better shutter angle & aperture
because they work hand in hand , you set the lens to f2.8 and reduce shutter angle to get a more in focus (1/362th)
where you should have closed down the aperture to let say f8 increase shutter angle ( 1/48th) = the same focus .
and if you have 1/362th shutter you didn't set the camera up right .
If you follow the instruction on the first post of this thread you should end up @ 1/48th when you enable the crop-rec 2.6k preset.
If not you did something wrong , and that's why you have aliasing problem .
To get these sizes I had to increase the rolling shutter from 25.6 to 30.6 ms (milliseconds)

To recap the set up: canon menu - NTSC , 1080p30 (29.97fps)
In magic lantern: set shutter to 1/60th second or as close to it as possible (1/63th is good)
enable crop_rec and go to 5xzoom & it should say 23.976fps & 1/48th shutter .
I label this as user/cam setting error.

Quote from: banertop on May 16, 2019, 08:43:58 PM
Side note, with this and with 3k build, when formating card in camera, it delete ml from card
Yes I know about this it's a bug that's very low priority right now , well get addressed in do course.

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on May 19, 2019, 10:18:09 PM
Yes,
i thought it was something about shutter speed...
Also, i remember once you said that all problems in 2.6k build (like red stripes) are related to improper shutter use....

I understand how shutter works, just did not think it will cause aliasing while i was shooting...

good to know

thanks

Any chance to test 48p soon?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 20, 2019, 01:54:16 AM
3200x1143 @ 23.973fps on the 50D  :)
Almost got the vertical height regs figured out (not as simple as 5d2 unfortunately), still need to push the image down 52 pixels .
So , yea I can do at least 3200x1143 @ 24fps  :D
I do think I can push it to 3200x1330 @ 24fps , tried 3744x1143 , got to 22fps so 1300 should be doable at 24pfs if not a little more

Short out doors (my back yard) clip in my google drive , 3200x1143 @ 23.973fps
Exported with mlv app. , could have use some defringing in upper part of the tree branches near sky
but if I needed to defringe ,  I would just use Adobe A.E. with ACR , but still not a bad job.
The focus could have been better , it's just a quick test for increased vertical resolution   
H264_M19-1835.mov (https://drive.google.com/file/d/1pWamZ_Kbez_c7p6JfWkn09D6WoZXBAts/view?usp=sharing)
ProRes4444-M19-1835.mov (https://drive.google.com/file/d/1IVZeiibqkvE8HK-YSW8A7T9chYON9nqe/view?usp=sharing)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 20, 2019, 07:24:07 AM
Ok the on the 50D the maximum resolution is 3200x1268 @ 23.98 fps but the image is not totally clean yet
still looking for the right vertical offset in cmos[4] . So at this point I do have clean image vertically at 1090 @ 24fps .

Some interesting information about the sensor on the 50D , it has 4.69µm(micro meter) pixel size
where the 5D2 pixel size is 6.41µm , no wonder the 50D is sharper but the 5D2 is more cinematic . 

Now I'll see if I can add the 50D to the existing crop_rec_4k_5d2  branch and have at least 1 preset (3744x1090 @ 23.98) for now
for the user to experiment with, but don't expected to see it for a few weeks or so . I need to get back on the 5d2 stuff .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 21, 2019, 06:12:18 AM
Didn't what to double post , updated the 50D thread with my updates & video sample of UHD Test on youtube
https://www.magiclantern.fm/forum/index.php?topic=9852.msg216783#msg216783

I can confirm that the preview liveview regs work on the 50D same as the 5D2
c0f08184
c0f08188
c0f08518
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on May 22, 2019, 06:58:32 PM
"I need to get back on the 5d2 stuff " like a music to my ears, hehe.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 23, 2019, 08:11:15 AM
Checking on Hi-Frame rate for 50D ,
In 3x3 maximum 1568x528 = 63.042fps thou it's a little unstable ,  60.058fps is stable .
in 5x Zoom(3xcrop_mode) 1920x760 = 48fps , I can't figure out yet how to put 3x3 in to 5x3
(720p mode) , adtg12[100c]0x2 -> 0x4 doesn't work like 5d2 , it may be a different reg
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 24, 2019, 07:25:18 AM
I'm very close to having the 50D code in to the 5d2/4k Crop_rec  ,
there one piece of information I need to have a working crop_rec on 50D . I'm looking for the address of
EngDrvOutLV
Can't seem to find it , all hints where to look ?
Edit: for 5xZoom

Ok I think I got this part but  ......

found more code I need to change for 50d
else if(is_camera("50D", "1.0.9"))
    {
        CMOS_WRITE = 0xFFA12190;
        MEM_CMOS_WRITE = 0x;
       
        ADTG_WRITE = 0xFFA11FDC;                                             
        MEM_ADTG_WRITE = 0x;
       
        ENGIO_WRITE = 0xFF97D904;
        MEM_ENGIO_WRITE = 0x;

I found the CMOS , ADTG & ENGIO Write address  in the adtg_gui.c
but I can't found the MEM Write address , do it need to look in the 50d rom ? hints ?

Just need to figure out the address for mem write (may have to run some logs on 50d)
and I need the  A & B Timing table for frame rate ( but should be easy enough I think)
that should be the last bits to work over .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Apxub on May 24, 2019, 11:17:33 AM
Hello people!
I`ve tested the latest stuff on my 5dmkii + 64gb sandisk extreme pro 160mb/s (magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212.zip + 2.6k_5d2_R_T_PreView_4-7-2019_crop_rec.rar)
After a "card warmup" and 5x zoom button on 12bit mlraw i got max. 8sec rec with 2592x1200 | 2.16:1 | 1.12x | needed speed 106mb/s
And the recorded on any crop modes and resolutions .mlv files do not play via file manager mode.
Also navigation in the "modified" tab of ml menu got very long latency between button pushes and that`s strange. On the latest ML Nightly Builts no latency at all.
After formating cf card with "keep ml" via canon menu camera shows info ("card not formated in camera, format it"). After you`ve done that second formating ML firmware totally disapears. That hapened on 2 different cards.
It`s possible to fix it?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on May 24, 2019, 07:41:19 PM
@Apxub
This is a experiment build, not finished yet. There are many problems at the moment and the priority is currently elsewhere.

@reddeercity
I have to revise my last entry, the problem persists. I've try formating and new installt, different settings, different aspect ratio (less writing speed), but nothing works (black level intermittent).

On one day i've testet static-in-Room-shoots and some static-outside-shoots, all without blacklevel problem...
Now i've testet stativ (no changing Camera angle, dynamik objekt like cars is ok) vs dynamik shoots (changing camera angle, especially between hard contrast, much wiggle^^) -> Result: all dynamik shoots have intermittently black level problem! The problem is only in the new crop build.

I hope that will helps you.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 25, 2019, 04:27:37 AM
Quote from: Bender@arsch on May 24, 2019, 07:41:19 PM
.......
Now i've testet stativ (no changing Camera angle, dynamik objekt like cars is ok) vs dynamik shoots (changing camera angle, especially between hard contrast, much wiggle^^) -> Result: all dynamik shoots have intermittently black level problem! The problem is only in the new crop build.
I hope that will helps you.
Thank you , yes very helpful !
I wish I had more user like you that are on the ball.
Can you upload a short .mlv with the problem please ?
I'll prepare a special 2.6k build (shortly)with the preview regs disabled for testing and see if the problem persists .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 25, 2019, 05:25:06 AM
Updated the first page/post of this thread with new test build for 5d2 2.6k , hopefully to fix the problem that Bender@arsch reported above on Reply #713
preview_regs_modified-crop_rec-2.6k-5D2-eXperimental.2019May24.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/preview_regs_modified-crop_rec-2.6k-5D2-eXperimental.2019May24.5D2212.zip)
I loaded the CF card with a fresh ml build with crop_rec.mo and no problem here yet .
When testing , don't just copy the new crop_rec.mo over to CF card , do a fresh install to be sure
there nothing leave from previous build to cause issues .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 25, 2019, 06:00:00 AM
Found the CMOS , ADTG & ENGIO MEM Write address  :D
sad part there where right under my nose all along , all I had to do is look at the memory patches in the debug menu.
look at the green text at the bottom of the screen shot image.
(https://i.ibb.co/BykBztC/VRAM24-small.png) (https://imgbb.com/)

One step closer  ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: LEVISDAVIS on May 25, 2019, 07:00:44 AM
5D II
Komputerbay 128 CF 1066x

Link to image showing "Failed to Link Modules."  https://ibb.co/Hqj2bZp

This is the latest 2.6K modified crop_rec.  Fresh "ML install" as instructed.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 25, 2019, 07:32:44 AM
No problem here , just to be sure I downloaded the build again from my bitbucket downloads and every works correctly .
Only load 5d2.sys , mlv_rec.mo & crop_rec.mo 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Apxub on May 25, 2019, 03:58:06 PM
Quote from: Apxub on May 24, 2019, 11:17:33 AM
Hello people!
I`ve tested the latest stuff on my 5dmkii + 64gb sandisk extreme pro 160mb/s (magiclantern-crop_rec-3k-1080p-5D2-eXperimental.2019Feb01.5D2212.zip + 2.6k_5d2_R_T_PreView_4-7-2019_crop_rec.rar)
After a "card warmup" and 5x zoom button on 12bit mlraw i got max. 8sec rec with 2592x1200 | 2.16:1 | 1.12x | needed speed 106mb/s
And the recorded on any crop modes and resolutions .mlv files do not play via file manager mode.
Also navigation in the "modified" tab of ml menu got very long latency between button pushes and that`s strange. On the latest ML Nightly Builts no latency at all.
After formating cf card with "keep ml" via canon menu camera shows info ("card not formated in camera, format it"). After you`ve done that second formating ML firmware totally disapears. That hapened on 2 different cards.
It`s possible to fix it?
Buy the way all lags stays the same with a reddeercity 24.05.19 built on my stuff.
After i found this Danne`s built: https://bitbucket.org/Dannephoto/magic-lantern/downloads/crop_rec_4k_5D2_2019Jan09.5D2212.zip
And it works perfect on my stuff without previous lags. Only the canon formatting problem not fixed.
And i got 2880x1080 or 2.67:1 with crop mode 3.5k 1:1 centered x5:
10bit=26sec.rec
12bit=9sec.rec
14bit=6sec.rec
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on May 25, 2019, 07:38:25 PM
Here a short 10sec. MLV file:
https://drive.google.com/open?id=1i2MHDMm5FwymQrADLuMiPLxzp_2OI_iS


I've tested the modified crop module (format and new install) -> same result...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: LEVISDAVIS on May 25, 2019, 11:19:20 PM
All good.  Was a card issue and an FPS issue in the Canon Menu.  ( Strange combination of errors as both were previously set accordingly. )

Link to modules working - https://ibb.co/0mhjYrn
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 26, 2019, 12:35:32 AM
Quote from: Bender@arsch on May 25, 2019, 07:38:25 PM
Here a short 10sec. MLV file:
I've tested the modified crop module (format and new install) -> same result...
Well it look ok here , black & white level are correct (112-1013) check with exiftool & mlv app  ,
I notice you are recording audio also . I never tested crop_rec test build with audio , and that will push the limits of the 5d2 beyond it's limits .
I only intended the preset to be used as video only not with audio yet , if you want to use audio reduce resolution to 2400 etc. ....
If you need audio real bad , use a audio recorder . When d4 crop_rec get in to the main 4k branch , audio is supported there but until then
you will have to made do as is .

Check a little further , and I see a light change (sun come out in the image on the right)
I see a more contrasts frames near the end of the clip but all dng are correct levels .
The light in the image start to be flat then after the bright wall the sun comes out (just coincidence ?) .
There my also be a issue in the raw backend with auto blacklevel detection
Below are dng screen shoot from my desktop , left is the second frame(0002) & the right is frame# 199
(https://i.ibb.co/bHJLTGg/M25-1855-small.png) (https://imgbb.com/)

Try without audio , different ISO's etc. ...  , keep all light in scene equal (to validate the problem) & if you do a ISO change of White balance change (I use only manual Kelvin setting no auto)
refresh liveview , press the play button twice then re-enter 5xzoom for crop_rec .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on May 26, 2019, 05:11:00 AM
compare Frame 26 (blacklevel 112) and 199 (change blacklevel to 107) -> is nearly equal, no sun changing  (all in MLV APP)

in crop mode i change nothing, but i will test it without audio later.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on May 26, 2019, 06:19:14 PM
CropTest without sound or without all, only rec.mo and crop.mo with the modified build -> same result
btw. audio works great with all experimental crop builds, with my extern stereo microphon too (not used in the updloaded test MLV)

workflow: i change all settings in "normal" liveview, then i press + for crop

Quote
if you want to use audio reduce resolution to 2400 etc. ....
I already tested different aspect ratio (less resolution) -> no different
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 28, 2019, 06:35:50 AM
Had time to test 50D with UHD (3744x1080 23.976) , I did a better setup on the tripod , filmed my flowering plum tree (before there gone) Canon EF24-70mmL 2.8f @ F8 800ISO 1/48th second .
Process thought After Effect ACR , I wanted to use mlv app but it kept clipping the highlight & turning them Pink so I use A.E. , export to ProRes4444 and uploaded that same file to YouTube (2GB)
Very impressive image this may be the best crop_record next to the 5D3  :) watch on chrome for the full UHD/4k stream . I don't have the crop_rec working yet for the 50D I just used the ADGT_GUI.mo   

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on May 28, 2019, 04:05:01 PM
Nice work. I had concerns about 50D sensor size, but with 3744 resolution, the crop factor is actually pretty good (36/22.3)*(4752/3744) = 2.049x, same width as micro four thirds, only a tiny bit smaller than what you get on 5D2 with 3008 pixels, 1.867x

I think it may be more practical if you increase height and decrease width though.

I see 50Ds for 99 pounds :o
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on May 28, 2019, 10:01:53 PM
Very nice @reddeercity. Let us know when you release your code. I have a 50D so I could help testing.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 29, 2019, 05:00:14 AM
sure , I'll let you know when it's ready 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 29, 2019, 05:17:27 AM
Quote from: Ilia3101 on May 28, 2019, 04:05:01 PM
I think it may be more practical if you increase height and decrease width though.
Yea I plan on it , I can do 3200x1200 @ 23.976fps  but I have a problem with the vertical offset right now (still working on it)
3744x1080 @ 23.976 was the easiest to achieve with minimum reg's being applied & of course I needed a simple preset
to code to make it easier for me .
Quote from: Ilia3101 on May 28, 2019, 04:05:01 PM
I see 50Ds for 99 pounds :o
Yea 50D are cheap and a lot of them have very low shutter count , they came out in mid 2008 to 2010
but in the time 5d2 came out & less then a year later the 7D so I bet a lot did a upgrade to 7D or 60D etc. ..
specially since the other had factory video option  https://en.wikipedia.org/wiki/Canon_EOS_50D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 31, 2019, 09:22:13 AM
Back Working on 5D2 3x1 4k (4096x590(4096x1770) 23.976fps)
But it's super buggy right now , but if you feel brave check it out .
3x1 4K is the (1920 1:1) in the crop_rec menu (I haven't change the name yet)
Please note !! you need to adjust the cmos[1]hi & cmos[2] in the advanced menu of crop_rec  like the image below
(https://i.ibb.co/5v7BDcc/VRAM7-small.png) (https://imgbb.com/)
this needs to be set before you enable the crop_rec , after that start in 5xZoom (without crop_rec enabled) you should be  @ 2144x1078 that this point .
press the half shutter to refresh , now go to the crop_rec menu , select 1920 1:1 , press half shutter  , then right after that cycle thought the 5xZoom button
until it's at 3x3 then the 4096x590 should come up . Yea I now it's a kind of a pain to get to ,  but this is just my first daft .
The cmos's for some reason are not being applied correctly , I need to go over it again .
 
Tomorrow I'll make a quick setup video on how to enabled it ,
Remember ! this is Very Very Buggy !!! and my crash , the screen may turn black etc. ... you guy know the routine , if breaks you get to keep all the parts .

Also note , that the advanced crop_rec menu setting for 3x1 4k will not work with the 2.6k 1x1 , you need to disable those 2 setting back to "0'
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on May 31, 2019, 07:01:04 PM
For now i have this bug:

https://drive.google.com/open?id=1PkNXq2j6tPX7U9_2yzlzqn_oI2ZuJWE4

The same story if I off the crop rec and raw modules and after that press liveview button
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on May 31, 2019, 07:37:27 PM


Quote from: reddeercity on May 31, 2019, 09:22:13 AM
Back Working on 5D2 3x1 4k (4096x590(4096x1770) 23.976fps)
magiclantern-crop_rec2.6k-1x1-4k-3x1-5D2-eXperimental.2019May30.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec2.6k-1x1-4k-3x1-5D2-eXperimental.2019May30.5D2212.zip)
But it's super buggy right now , but if you feel brave check it out .


My bug: MLV rec module isn't even loading now. https://ibb.co/5FRytgG
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 01, 2019, 05:30:17 AM
Ok , here the video to set it up 4k anamorphic on 5D2 , *Note *only Load mlv_rec & crop_rec that's it , just fellow the video & like I said it's very buggy . Even show how to get out of the black screen


Shot this on my new/used 50D in crop_mode @ 1920x1078 24fps  ;)


Edit:Here a short 11 second/30Mb 5D2_4096x1770_24fps_M31-1952.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2_4096x1770_24fps_M31-1952.mp4)

(https://i.ibb.co/zFy5gXV/M31-1952-frame-1-small.png) (https://imgbb.com/)
Frame export from MLV App. 5D2_4K_M31-1952_frame_1.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M31-1952_frame_1.png)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 01, 2019, 11:46:35 AM
@Reddeercity
Can't understand what i'm doing wrong:
https://drive.google.com/file/d/11dcQtvpvP02H49nuaTKMZdL_Z8Hd_0X0/view?usp=sharing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 01, 2019, 02:14:04 PM
You need to select 4096 in the raw movie/mlv_rec sub menu .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 01, 2019, 02:47:00 PM
I think i found the key=) just get the 4096... It needs to set the maximum resolution possible in raw module before you go x5 the first time
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 01, 2019, 02:47:46 PM
@Reddeercity, Yes, thanks =)) That's my 4096x1770 footage. Looks so cool...
https://youtu.be/hFjuebwv26w
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on June 01, 2019, 06:51:56 PM
First test looks great, but there is a black bar on the right side in all test footages.

Continuous recording is really nice, so is there room for more?


I also have good news:

I've tested with 10bit (continouous), 12bit (around 630 frames, ~26sec.)
and 14bit (around 250 frames, ~12sec. and with real-time liveview!! (grayscale) -> low frame, but it works!)
And all tests with audio;)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on June 01, 2019, 07:09:21 PM
Impressive resolution numbers, but it looks aliasy and not that sharp. The best mode for quality is still the 3008x1080, so I think squeezing more out of that would be the most effective investment of time on 5D2 (just my opinion).

3.008*1.080*23.976 = 77.9 MPix/sec, there's still a bit of vertical resolution to squeeze if the limit is 96 ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 01, 2019, 07:32:01 PM
@Bender@arsch
Wow... You have real-time liveview? Only when shooting 14 bit?
@Ilia3101
You right, but i think even though it doesn't look too sharp, it is very cinematic because of less crop factor when shooting 4096...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on June 01, 2019, 07:44:06 PM
I agree it's cinematic from having less crop, definitely good for people shots with bokeh, but other things will moire a lot.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: honza123 on June 01, 2019, 09:40:15 PM
It works!
My first 4K footage on 5Dmk2 :

https://drive.google.com/file/d/1jsUXLNozpeqCS-1WjM87B2BJOr9NTrsy/view?usp=sharing
https://youtu.be/xLpVYGDsRpc
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 02, 2019, 12:44:09 AM
@Ilia3101 You are right... Maybe, i think the best way is to combine 4096 x 1770 (when you need Shallow Depth Of Field) with 3008 x 1080 cutting it to 2494 x 1080  (when you need more detailed and quality picture). But is there any sense? I'm gonna make a comparison side to side, shooting 4096x590 and 1856x790
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 02, 2019, 06:42:29 AM
https://youtu.be/ObU4x8f5i0Y
Another shot. I've tried to imitate 16mm film (starts from the 9th second). After, understood that it was a bad idea,  but render time is too much for do it again =)
unfortunately, i don't have any ND filters, so the aperture here is 11....

And of course, i noticed that the image is captured from the top of the sensor (it's not centered).
@reddeercity, Do you think that the best way to unsqueeze the preview (when/if it will be unfrozen) is to use an external monitor that is able to stretch the image vertically?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 02, 2019, 07:49:22 AM
Quote from: Igor_Braun on June 02, 2019, 06:42:29 AM
@reddeercity, Do you think that the best way to unsqueeze the preview (when/if it will be unfrozen) is to use an external monitor that is able to stretch the image vertically?

Yes there a unsqueeze anamorphic preview mode , but only work in canon liveview , I had it working in old test a few mouths ago .
Need to adjust the preview reg's agian to get it to work , I'll look in to it . 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 02, 2019, 08:00:42 AM
Quote from: Ilia3101 on June 01, 2019, 07:09:21 PM
3.008*1.080*23.976 = 77.9 MPix/sec, there's still a bit of vertical resolution to squeeze if the limit is 96 ;D
You need to include the OB range , even thou the final resolution is 3008x1080 there more area to consider here .
Default Crop Size               : 3016 1080
Active Area                     : 52 160 1132 3176

as you can see the real size is 3016x1080 but because of mod8 rules the mlv_rec only save as 3008 .
Now  in the raw image dump to dng you can see the OB area included now  3176x1132 , the 52 160 is the crop offset
52lines for vertical & 160 lines horizontal = 3016x1080 (mlv_rec cropped 3008x1080) .

So let figure out the math for the over all size with OB to MP/s
3176*1132*23.976 =  86.19 MP/s so there less then 10 MP/s left for total available of 96 MP/s so there
must be a little overhead there in headtimers , so I think that really the limit unless there a way to increase the frequency
from 24MHz to let say 32MHz , that would give us 128MP/s good enough for 3520x1350 @ 23.976(114 MP/s) with OB it would be (3680*1402*23.976=123.7 MP/s)
let push it to 48MHz per channel (wishful thinking) 48x4=192 MHz now this is in 5D3 territories 4096x1770 24fps (with OB area -4264*1822*23.976=186MP/s)
With the 3x1 4096x590 in mlv_rec before mod8 rule 4120*591 , with OB area it's 4280*609 (4280*609*23.976=62.5MP/s)
So yea I can go wider but as I go wider the vertical get reduced in proportional to the wide e.g. 5632x416 =5632x1248 (4.5A.R.) , too wide

(https://i.ibb.co/xf7q2gW/M01-0022-frame-224-small.png) (https://imgbb.com/)
M01-0022_frame_224.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M01-0022_frame_224.png)

But I can increase fps , I had 4k anamorphic up to 29.97 fps maybe there a little more there like 35fps
did the math and 4280*609*33=86MP/s , so  33fps looks like the limit , I maybe able to push it to 34 or 35 fps
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 02, 2019, 10:25:03 AM
Quote from: reddeercity on June 02, 2019, 07:49:22 AM
Yes there a unsqueeze anamorphic preview mode , but only work in canon liveview , I had it working in old test a few mouths ago .
Need to adjust the preview reg's agian to get it to work , I'll look in to it .

Do you think that the only way to choose the composition and see what you're filming is to use x5 canon liveview (without understanding the borders of the frame? ) maybe there is a way to unfrozen the liveview even if it will be 2 or 3 fps?  :P OR maybe use canon preview but in no magnification mode (for make exactly the same anamorphic frame from black paper and stick it to an external monitor for example, so u can understand your composition. Hope I wrote understandable, heh. And how'd @Bender@arsch get low frame real-time liveview?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: shiro_hh on June 03, 2019, 04:43:02 PM
Awesome work you guys do !

I also get inspired by all the good videos, especially by Ilia Sibiryakov.
So I bought a used 5D and went for holidays to fool around a little bit.
The newest build worked pretty well for me.

I hope you guys enjoy

https://www.youtube.com/watch?v=8_ud5lXRJDg&t=10s
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 04, 2019, 12:42:40 PM
@shiro_hh

Hi, here at 2:34 - this footage with kite in sky also shot in 4k? And what was the FPS and shutter speed did you use? Every moving objects are so blurred. Also what was the lens? Think you should buy another one if want to shoot in 4k, because in my opinion the lens resolution here is not enough for it.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Dmytro_ua on June 04, 2019, 01:17:20 PM
Quote from: shiro_hh on June 03, 2019, 04:43:02 PM
I hope you guys enjoy

https://www.youtube.com/watch?v=8_ud5lXRJDg&t=10s

I see some strong "ghosting/blurring" on moving objects. Why is it so?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on June 04, 2019, 01:45:44 PM
Quote from: shiro_hh on June 03, 2019, 04:43:02 PM
Awesome work you guys do !

I also get inspired by all the good videos, especially by Ilia Sibiryakov.
So I bought a used 5D and went for holidays to fool around a little bit.
The newest build worked pretty well for me.

I hope you guys enjoy

https://www.youtube.com/watch?v=8_ud5lXRJDg&t=10s

What a wonderful video. Good colour and composition. And nice lens choice, I once had a sigma 15-30 :D

Quote from: Igor_Braun on June 04, 2019, 12:42:40 PMAlso what was the lens? Think you should buy another one if want to shoot in 4k, because in my opinion the lens resolution here is not enough for it.

I thought the Helios 44's softness looked really nice actually, a pleasing glow. The shots at 0:44 1:49 and 3:03 in my opinion are examples of that.

Quote from: Dmytro_ua on June 04, 2019, 01:17:20 PM
I see some strong "ghosting/blurring" on moving objects. Why is it so?

If MLV App was used, "smooth aliasing" filter was probably enabled on export. I have seen a few videos affected by it before. @masc it should have a warning label so people don't accidentally use it on shots with moving objects. It doesn't even do much for tripod shots anyway.




Quote from: reddeercity on June 02, 2019, 08:00:42 AM
You need to include the OB range , even thou the final resolution is 3008x1080 there more area to consider here .
Default Crop Size               : 3016 1080
Active Area                     : 52 160 1132 3176

as you can see the real size is 3016x1080 but because of mod8 rules the mlv_rec only save as 3008 .
Now  in the raw image dump to dng you can see the OB area included now  3176x1132 , the 52 160 is the crop offset
52lines for vertical & 160 lines horizontal = 3016x1080 (mlv_rec cropped 3008x1080) .

So let figure out the math for the over all size with OB to MP/s
3176*1132*23.976 =  86.19 MP/s so there less then 10 MP/s left for total available of 96 MP/s so there
must be a little overhead there in headtimers , so I think that really the limit unless there a way to increase the frequency
from 24MHz to let say 32MHz , that would give us 128MP/s good enough for 3520x1350 @ 23.976(114 MP/s) with OB it would be (3680*1402*23.976=123.7 MP/s)

I see. So OB is just black area? Maybe that could be reduced somehow. Sad to know that maybe it really is reaching the limit now.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: domasa on June 04, 2019, 02:04:11 PM
Quote"ghosting/blurring" on moving objects
Probably bad project settings and export...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: andy kh on June 04, 2019, 04:34:56 PM
i think its the shutter speed problem. double of your frame rate is all you need
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: shiro_hh on June 04, 2019, 07:55:35 PM
Quote from: domasa on June 04, 2019, 02:04:11 PM
Probably bad project settings and export...

No its not, i just messed up the shutter speed in most of the shots ;) Cam resets every time to 38 after writing, i dont know why.... And i didnt realize this most of the time in the sunny environment.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: shiro_hh on June 04, 2019, 07:56:24 PM
Quote from: andy kh on June 04, 2019, 04:34:56 PM
i think its the shutter speed problem. double of your frame rate is all you need

thats right i just messed up the shutter speed.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: shiro_hh on June 04, 2019, 08:00:25 PM
Quote from: Ilia3101 on June 04, 2019, 01:45:44 PM
What a wonderful video. Good colour and composition. And nice lens choice, I once had a sigma 15-30 :D

I thought the Helios 44's softness looked really nice actually, a pleasing glow. The shots at 0:44 1:49 and 3:03 in my opinion are examples of that.

If MLV App was used, "smooth aliasing" filter was probably enabled on export. I have seen a few videos affected by it before. @masc it should have a warning label so people don't accidentally use it on shots with moving objects. It doesn't even do much for tripod shots anyway.




I see. So OB is just black area? Maybe that could be reduced somehow. Sad to know that maybe it really is reaching the limit now.

@ Ilia : Thanks mate i appreciate that.

Regarding the Blur... It is messed up shutter speed...  dooh :/
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: shiro_hh on June 04, 2019, 08:21:03 PM
Quote from: Igor_Braun on June 04, 2019, 12:42:40 PM
@shiro_hh

Hi, here at 2:34 - this footage with kite in sky also shot in 4k? And what was the FPS and shutter speed did you use? Every moving objects are so blurred. Also what was the lens? Think you should buy another one if want to shoot in 4k, because in my opinion the lens resolution here is not enough for it.

I shot in 2624x1192 and pumped it up to 3840x fps=24 Shutter Speed 38 (messed up most of the time) and yes you right, the lenses arent probably enough (helios 44, sigma 15-30)
But i will be striving for better results next time. It was a first atempt with ml and this setup
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 05, 2019, 01:57:59 AM
Thanks guy for the sample of 4k Anamorphic in the real world  :) looks good , since no one has any big problems
I  updated (https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476) the first page/post with 4k anamorphic build & setup video for more users to test out ( *note* not for the faint of heart , real bleeding edge stuff)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 05, 2019, 05:34:11 AM
Playing around with the 1:1 3.5k centered crop_rec preset , I can push the vertical to 2784x1160 @ 23.976fps
I have to reduced the vertical to 1160 from 1200 , so you lose 40 lines vertical but gain 192 lines horizontal .
A.R. is 2.40 , so is this something that would be used ? I'll put up a test build later on .
So it seems that 86 MP/s is as far I can go currently , unless I can find where the overhead is limiting the total output of 96 MP/s
So for the one's follow alone , the raw dump image is 2952x1213 with offset crop 52 vertical & 160 horizontal = (2792x1161) mod8 rules =2784x1160
FYI 2.952*1.213*23.976=85.85 MP/s .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on June 05, 2019, 01:25:55 PM
Quote from: reddeercity on May 29, 2019, 05:00:14 AM
sure , I'll let you know when it's ready
I'm going to clean up my 50d tonight ...wowowow.. I'm so excited about this

Sent from my Redmi 4A using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 05, 2019, 06:25:19 PM
@Reddeercity, so, maybe you already knew it, but i just noticed that if you select 14 bits and global Draw is "on" with Ml Grayscale preview, then the lifeview become lifeview =) It's around 1 fps, but better then nothing. And it freezes when selecting 12 or 10 bpp. Why is it happening?
I've also tried to connect an external hdmi monitor, but it doesn't work (no signal) if enabling anamorphic 4k preset.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on June 05, 2019, 10:13:35 PM
Quote from: reddeercity on June 05, 2019, 05:34:11 AM
Playing around with the 1:1 3.5k centered crop_rec preset , I can push the vertical to 2784x1160 @ 23.976fps
I have to reduced the vertical to 1160 from 1200 , so you lose 40 lines vertical but gain 192 lines horizontal .
A.R. is 2.40 , so is this something that would be used ? I'll put up a test build later on .
So it seems that 86 MP/s is as far I can go currently , unless I can find where the overhead is limiting the total output of 96 MP/s
So for the one's follow alone , the raw dump image is 2952x1213 with offset crop 52 vertical & 160 horizontal = (2792x1161) mod8 rules =2784x1160
FYI 2.952*1.213*23.976=85.85 MP/s .


2792x1161 is a much improved resolution I think, better balance of width and height (to my taste). Definitely worth releasing a build. Not than I can test with my broken 5D :(
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on June 06, 2019, 12:47:51 AM
I was able to get continuous 4k with my Lexar 32gb 1066x!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 06, 2019, 06:51:17 AM
Quote from: Igor_Braun on June 05, 2019, 06:25:19 PM
@Reddeercity, so, maybe you already knew it, but i just noticed that if you select 14 bits and global Draw is "on" with Ml Grayscale preview, then the lifeview become lifeview =) It's around 1 fps, but better then nothing. And it freezes when selecting 12 or 10 bpp. Why is it happening?
I heard something about it a while ago , yea I kind of figured in 14bit it would (it's the camera's backend bit rate) but never thought to test it , as 14bit is not feasible ,
unless I get lossless working (14bit at about 50-60% compression)  .
10 & 12bit comes from what I understand comes from a different area (A1ex could explain better then me) That's why I tried to get 3.5k centered preset
with at least the same preview as 3x crop_mode .

1x3 4k anamorphic will be better with canon reatime preview with almost correct aspect ratio , that's because 1x3 is in 5x Zoom mode (3xcrop_mode)
https://www.magiclantern.fm/forum/index.php?topic=19336.msg213010#msg213010
https://www.magiclantern.fm/forum/index.php?topic=19336.msg213265#msg213265
1632x1828 @ 23.976 fps (1x3) --> 4896x1828 @ 23.976 fps without any aliasing or moiré pattern
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 08, 2019, 12:56:28 AM
@reddeercity, thanks for answering.
Ugh.. I've been trying to get into all that sourcecode for learn it (i'm not a programer at all) and maybe help you with some little tweaks (my objective was to center the 4096x590, I actually LOVE this preset, It looks so quality and cinematic). Actually my brain has been exploded :o I was following your video about compiling ML via Ubuntu in virtual machine and got a bunch of errors in the way to open the croprec code. Was googling how to fix. So, now i'm trying to clone the source code and getting errors about that in the terminal. If I overcome all that shit i'll try to get inside  :D :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on June 08, 2019, 12:26:27 PM
Yes, 4096x590 is very cinematic!!!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 09, 2019, 07:53:06 AM
*Note's for me for later investigation*
looking in to (M)JPEG encoder  (https://www.magiclantern.fm/forum/index.php?topic=2803.0) , it seems that there was some development on 5D2
well , really some jpeg code execution on 5D2 , a1ex had some success back 2012 with 5D2
https://www.magiclantern.fm/forum/index.php?topic=2803.msg12793#msg12793

https://www.magiclantern.fm/forum/index.php?topic=2803.msg17885#msg17885
On 5D2, the syntax is: call("lv_save_jpeg", 1)

this maybe of interest to 550D user for lossless compression , thou it's was mainly for Jpeg but that where lossless come from , so still hope for you guys yet !
https://www.magiclantern.fm/forum/index.php?topic=2803.msg12793#msg12793

Maybe possible to play with HD buffer too (hopefully to increase for crop_rec res. for hdmi)
https://www.magiclantern.fm/forum/index.php?topic=2803.msg12100#msg12100
for 5D2
c0f04008:  4000080 <-- HD buffer


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 09, 2019, 08:34:41 PM
Quote from: shiro_hh on June 03, 2019, 04:43:02 PM
Awesome work you guys do !

I also get inspired by all the good videos, especially by Ilia Sibiryakov.
So I bought a used 5D and went for holidays to fool around a little bit.
The newest build worked pretty well for me.

I hope you guys enjoy

https://www.youtube.com/watch?v=8_ud5lXRJDg&t=10s

Wow very nice video !! I also have a 5d2, but I have some issues with the last builds, so I wanted to know if you had a big green tint in your videos when you opened them in mlv app ?
and also which format did you choose to export ? thnaks !
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 09, 2019, 08:46:49 PM
I don't know where that green tint comes from, but in some shots all shadows are completely green and I see some sort of "green chromatic aberration" on some others...
here are 2 clips, the edge of the flower is weird in one and again heavy green tint on both when I open them in MLV app (v1.7)

https://drive.google.com/file/d/1LW5hHsyuWd_gyprptBxXp75Cx_Z_k-yV/view?usp=sharing
https://drive.google.com/file/d/1ka62BY_UILq3TZy81gav9BXXDI6xE1PX/view?usp=sharing

I used the build from May 24 2019 (2592x1200) and I had the right shutter speed

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 10, 2019, 03:27:30 AM
@Galterius , I checked  your files , black level is off by a few numbers (120 & 119) should be 112 your can correct that in MLV App. or MLVProduder by adjusting the raw black level slider.
To know why this happen , I need to know the content of the ml setting folder & the exact build date ( this can seen in the autoexec.bat file with a simple text editor.

(https://i.ibb.co/x8DBZrs/Untitled.png) (https://imgbb.com/)
Magic Lantern crop_rec-4k-5D2-eXperimental.2019May30.5D2212
Camera   : 5D2
Firmware : 212
Changeset: 8bfbb0ca228e+ (crop_rec_4k_5D2) tip
Built on : 2019-05-31 05:50:42 by david@reddeercity

diff -r 8bfbb0ca228e src/raw.c
--- a/src/raw.c Sat Sep 08 11:26:34 2018 +0200
+++ b/src/raw.c Thu May 30 23:50:42 2019 -0600
@@ -32,6 +32,11 @@
 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 10, 2019, 02:13:59 PM
Thanks @reddeercity ! I will try to fix the black level
here is the ml settings file :

https://ibb.co/hy3H8pg
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on June 10, 2019, 02:17:45 PM
Can somenone point me to the exact branch and code used here. I could then take a look at the black level issue and probably fix it. I need crop_rec.c implementation of the preset in question and the contextual raw.c from the actual branch used.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on June 10, 2019, 05:57:46 PM
Hello Danne,

Quote from: a1ex on May 03, 2019, 12:11:57 PM
Hacking the hack. Let's take one of the builds from first post:

It contains a single file: crop_rec.mo. Put it in your magic-lantern/modules directory and run:

./module_hginfo_dump.sh crop_rec.mo


Result:

Build date  : 2019-04-05 04:19:36 UTC
Build user  : david@reddeercity

diff -r 8bfbb0ca228e modules/crop_rec/crop_rec.c
...
< what follows is the complete source code of crop_rec, displayed as diff from changeset 8bfbb0ca228e >


Changeset 8bfbb0ca228e is public (https://bitbucket.org/waza57/magic-lantern/branch/crop_rec_4k_5D2) (exactly where reddeercity indicated). The diff embedded in the binary module (crop_rec.mo in our case) contains all changes to that module, since the reference changeset.

Et voilà, you've got the complete source code.

So , it's seem that  reddeercity worked on a clone of my repository. As apparently we can not speak about this subject with him we must do archaeology of code.
And to avoid triggering his anger, I suggest you do not talk to him about it. :P

I believe but I am not sure that since then, the only changes it makes is found in the crop_rec module.
This implementation of crop_rec (https://bitbucket.org/waza57/magic-lantern/branch/crop_rec_4k_5D2) for 5D2 is so the last time we have trace of a complete source code.
I do not have enough time to dig through the whole thread but it would be really nice to have a summary of the registry changes to get the progress that reddeercity  found:

2624x1200 @ 23.976 fps with the full vertical heigh
4096x590 (Anamorphic 4096x1770) @ 23.976 fps
....and other things that don't remember.

These only in these conditions I could easily update the source code.


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on June 10, 2019, 06:09:24 PM
Hopefully he respects your wishes here Waza57. And welcome back!
I could probably track and put the pieces together here but it's such a waste of time...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 10, 2019, 06:43:41 PM
@reddeercity, its really weird because the black level is set correctly, but its still very green...

https://ibb.co/2347cMx
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 11, 2019, 01:03:28 AM
@reddeercity, I'm keeping break my brain with configuring my VB Ubuntu for ML, heh.. Now I have all the source files (thank you for helping me on that thread). But still, have a lot of errors.
Look, actually, i started to do it all only for "centering" you 4k anamorphic preset for comparing it with "regular" 1856-790.
The reason I can't do that now is that in 4096 preset the image is being captured from the left top corner so we have distortion and the image is not as sharp as if it would be captured from the center at apertures like 1.4, 2.0, 2.8.
Could you please center the image capturing in the future?

I really DO believe in this awesome 4096x590, and actually believe more than in 1632x1828 @ 23.976 fps (1x3), because in my personal opinion, in this case, the true horizontal resolution is more important (but would like to try 1632 also, of course).

You, David, do really great things...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 11, 2019, 06:43:36 AM
@Igor_Braun , to center the 3x1 4096x590 on cmos[2] in the advanced crop_rec menu
instead of
0x1E
select
0x6E
that will center it from what I see here

(https://i.ibb.co/BcbnPGR/VRAM9-Center-small.png) (https://imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 11, 2019, 07:12:15 AM
Quote from: Galterius on June 10, 2019, 06:43:41 PM
@reddeercity, its really weird because the black level is set correctly, but its still very green...
Using MLV App. ? turn off the "Camera Matrix" and use "Don't use Camera Matrix"

Also use the latest built , that one was a test build with preview reg's turn off with 2.6k 
this is the newest build magiclantern-crop_rec2.6k-1x1-4k-3x1-5D2-eXperimental.2019May30.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec2.6k-1x1-4k-3x1-5D2-eXperimental.2019May30.5D2212.zip)
Read the first post of this thread for instruction on how to set it up correctly
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 11, 2019, 08:50:46 AM
Found a interesting thing today , I can record H264 (.MOV)  when I change the column pinning in 3x3
CMOS[2] 0x40E -> 0x10E 
so I end up with 3x1 1920x1080 -> 1920x3240 @ 29.97fps

(https://i.ibb.co/Kj9T585/1920x1080-MVI-8796-00000-small.png) (https://imgbb.com/)
1920x1080_MVI_8796_00000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1920x1080_MVI_8796_00000.png)
Original H264 file I started with from my 5D2 1920x1080_MVI_8836.MOV (https://drive.google.com/file/d/17baWTTLL8egG3KisUPH9B2aYj2hrwH6Y/view?usp=sharing)

(https://i.ibb.co/LP8pmyp/A-1920x3240-MVI-8796-00000-small.png) (https://imgbb.com/)
1920x3240_MVI_8796_00000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/A_1920x3240_MVI_8796_00000.png)
Adobe After Effect I stretched the vertical 3x , saved as a Black Magic 8bit(M)JPEG 1920x3240_MVI_8796.avi (https://drive.google.com/file/d/1-mBouWDrklSEykRIpiScElp34R6jxCsH/view?usp=sharing)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 11, 2019, 11:43:53 AM
Quote from: reddeercity on June 11, 2019, 06:43:36 AM
@Igor_Braun , to center the 3x1 4096x590 on cmos[2] in the advanced crop_rec menu
instead of
0xE
select
0x6E
that will center it from what I see here

(https://i.ibb.co/BcbnPGR/VRAM9-Center-small.png) (https://imgbb.com/)

Oh, thanks, and what about CMOS [1]lo for vertical centering?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 11, 2019, 09:47:26 PM
Quote from: reddeercity on June 11, 2019, 07:12:15 AM
Using MLV App. ? turn off the "Camera Matrix" and use "Don't use Camera Matrix"

Also use the latest built , that one was a test build with preview reg's turn off with 2.6k 
this is the newest build magiclantern-crop_rec2.6k-1x1-4k-3x1-5D2-eXperimental.2019May30.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec2.6k-1x1-4k-3x1-5D2-eXperimental.2019May30.5D2212.zip)
Read the first post of this thread for instruction on how to set it up correctly

yea using mlv app

thanks @reddercity ! I will try it and keep you updated ! thanks for your work (my 5d2 is having like a 2nd life  :) )
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 12, 2019, 02:37:49 AM
@reddeercity
I think I could find out the settings for center the image vertically too.
There are some black bars at the sides (especially at the right side), but not too much thou.
So, the settings to center 4k 4096x590 preset horizontally and vertically are:
(http://i63.tinypic.com/2w3wb2p.jpg)

And it need to apply them before switching the crop mode on as you said.
There is a  lot of moire, but personally me I don't really care about that.

Will try to find combination for avoid the black bars


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 12, 2019, 05:21:25 AM
@ Igor_Braun , Good job ! yes that better but I think CMOS[1]lo is better at "9" .
the left hand border black bar , never notice it before , that I need to fix with code .
Timer "A" needs to be reflexed a little more , then I need to re-adjust Timer "B" to get 23.976fps .
I fix that later on the weekend at the latest .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 12, 2019, 07:20:22 AM
Ok it seem that I can put the 5D2 in to center crop for H264(AVC) (.MOV) and record it just as if I was in FHD(3x3)  :D
These are the reg's to make it work , I set canon menu frame rate to 24 ntsc

(https://i.ibb.co/3hMGbSQ/VRAM124-small.png) (https://imgbb.com/)

I dump the image buffer and this what it cam out
(https://i.ibb.co/6vpq1bv/Center-Crop-HD-015-small.png) (https://imgbb.com/)
Center_Crop_HD-015.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Center_Crop_HD-015.png)

(https://i.ibb.co/2kF7hRx/Center-Crop-LV-015-small.png) (https://imgbb.com/)
Center_Crop_LV-015.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Center_Crop_LV-015.png)

DNG converted to PNG with dcraw
(https://i.ibb.co/9NNHtHX/Center-Crop-1920x1080-RAW-015-small.png) (https://imgbb.com/)
Center_Crop_1880x1247_RAW-015.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Center_Crop_1920x1080_RAW-015.png)

Center_Crop_1880x1247_RAW-015.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Center_Crop_1920x1080_RAW-015.DNG)
Center_Crop_1920x1080_MVI_8842.MOV (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Center_Crop_1920x1080_MVI_8842.MOV)

Why is this important ? First I can put the camera in 1:1 pixel , same as 3x crop_mode but in 3x3 (FHD) for AVC (H264 .MOV) recording not Raw .
This is the first step in getting a bigger res with native canon compression (h264) and hopefully that will help get (M)JPEG (.AVI) working.
Next is how to increase the horizontal & vertical (well mainly horizontal) so we can equal the raw resolutions , for UDH/4k 8bit AVC or (M)JPEG .   
Two problem I can see right off the start ,
1st ; change the HD Buffer for increase resolutions (if this can be done , we should have full color realtime preview in extended resolutions as it limited to HD size)
2nd ; the H264/AVC encoder is preset to 1920x1080 in the rom , even thou it resizes the h264 will always be 1080p unless we can get control over it .
This is where (M)JPEG .AVI could save us , there no preset size from what I can see in the rom , so theoretically we should be able to record the extended raw resolutions
which could be great benefit for the low speed write camera's plus we can start to use picture style profiles again like Technicolor Cinestyle but in a 422 color space .
Not sure how all this could be implemented , maybe a custom H264 module or a (M)JPEG Module ?
Maybe a1ex has a idea for that  :)   
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on June 12, 2019, 10:50:26 AM
man,
that would be nice

better h264!

good work!

Thanks again
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 12, 2019, 10:58:06 PM
@reddeercity I did what you told me and the image is very nice now ! Thanks !

https://www.youtube.com/watch?v=huFwnUV46Po

what I did : import in MLV app, change to "don't use camera matrix" and choose the "film" preset, export in CinemaDNG Uncompressed, import in AE, color grading with Camera Raw, dynamic link with Pr and export in 4k so youtube doesn't destroy it ^^$

It still looks a liiiitle bit too green but thats okay
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on June 12, 2019, 11:21:31 PM
Quote from: Galterius on June 12, 2019, 10:58:06 PM
what I did : import in MLV app, change to "don't use camera matrix" and choose the "film" preset, export in CinemaDNG Uncompressed, import in AE, color grading with Camera Raw, dynamic link with Pr and export in 4k so youtube doesn't destroy it ^^$
That makes no sense: if you export in MLVApp to dng, all settings (except RAW corrections) will get lost, because dng is a RAW format. Camera Matrix and Profile Film will only be visible in a non-RAW format like ProRes, H264, ...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 13, 2019, 07:18:32 AM
Playing around with 3x3 H264 & different pinning mode , set the cam same as in post #784 but left the pinning at default  CMOS[2] 0x40E
Ended up with a full view (3x1 binning) H264 1920x1080 -> 5568x1080 , Yea Anamorphic .MOV ! thou it's not as good as in 5x Zoom (3x Crop_mode)
the edges are jagged , but it does work , so any Reg other then the raw buffer ones are applied to H264 , as long as there's a clean canon preview
the encoder will save it to h264 .

DNG from image dump , converted to png with dcraw
(https://i.ibb.co/YXMHRnJ/RAW-017-small.png) (https://imgbb.com/)
RAW-017_1880x1247.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-017.png)

(https://i.ibb.co/JcNv5Zx/RAW-017-5640x1247-small.png) (https://imgbb.com/)
RAW-017_5640x1247.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-017_5640x1247.png)

Nice thing about 3x1 I can use the 2:1 Anamorphic preview option , thou it's should be 3:1 ,
I'll have to look in to the code and see if I can add a 3:1 preset view mode
(https://i.ibb.co/Xkv6wQz/VRAM126-small.png) (https://imgbb.com/)


Here the original 5D2 H264 MVI_8849.MOV (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/MVI_8849.MOV) file .
H264 frame from Adobe A.E.
(https://i.ibb.co/Q84trCY/5568x1080-MVI-8849-1-00000-small.png) (https://imgbb.com/)
5568x1080_MVI_8849_1_00000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5568x1080_MVI_8849_1_00000.png)

Here the export from A.E. , Blackmagic 8bit (M)JPEG 5568x1080_24fps_MVI_8849.avi (https://drive.google.com/file/d/15RTyEJU2x2gSSXCXmzaFW-AwoPUCKy7t/view?usp=sharing)
Like I said the edges are jagged , but the real test was if I could record 3x1 in AVC(H264) and I can so if we could record AVC in 5xZoom (3x Crop_mode)
in 3x1 we would have a very clean aliasing and moiré free H264 4k Anamorphic file  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 13, 2019, 12:00:02 PM
Quote from: masc on June 12, 2019, 11:21:31 PM
That makes no sense: if you export in MLVApp to dng, all settings (except RAW corrections) will get lost, because dng is a RAW format. Camera Matrix and Profile Film will only be visible in a non-RAW format like ProRes, H264, ...

yea thats true... Then I will try with Prores
The problem is that when using "don't use camera matric" colors are really flat, and its hard to get good skin tones
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 13, 2019, 12:03:39 PM
By the way I will try the 4096x590 anamorphic build
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 13, 2019, 12:56:13 PM
So, I just did a comparison test between the build from May 24 (preview reg's modified) and the one from May 30 that @reddeercity sent me to see if there is a difference in colors.

resolution : 2560x1200

So here is a google drive folder with 2 folders (one folder for each build) : each one contains an mlv file, the ml setting file and a png exported with mlv app, so you can tell if there is a difference.

https://drive.google.com/drive/folders/1j8EoJQy6sFCdp9x_SbhC7moSpwwfmANY?usp=sharing


I did that to see if there was a problem with the black levels, but the difference is very small, maybe the build from May 30 is a little bit less green, I don't know...

I don't really know if its useful, but maybe ^^

I will now test the 4096x590 build
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 13, 2019, 01:53:54 PM
So I just tried 4096x590 : it works, but when its enabled, I can't change the shutter speed, iso...
i also get a lot of aliasing and moiré a big vertical line on the right, and in some clips vertical lines, weird pixels... and overall it looks really unsharp which is weird...


this is prores 422 exported from MLV app : https://drive.google.com/file/d/1p8rOlaJ0X28Heth8Bsyxcnq-YYAE9vZQ/view?usp=sharing


(https://i.ibb.co/7gZw3Xn/bug-ml.jpg) (https://ibb.co/gvf503S)
4096x590 test (https://fr.imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on June 13, 2019, 02:27:15 PM
Quote from: Galterius on June 13, 2019, 12:00:02 PM
yea thats true... Then I will try with Prores
The problem is that when using "don't use camera matric" colors are really flat, and its hard to get good skin tones

I don't recommend using the "don't use camera matrix" option unless colours are clipping, it's just less accurate. Also you showed a different shot, processed with other software, to show the issues were fixed, what happened with the first one though?

The only reason "don't use camera matrix" might seem to make the "very green" issue go away, is the fact that it desaturates the image, but it will become visible as soon as you add saturation.

I think it's better for the main developers here to fix the black level issues in their magic lantern branch® than deny it or blame MLV App.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 13, 2019, 08:29:35 PM
Quote from: Ilia3101 on June 13, 2019, 02:27:15 PM
I don't recommend using the "don't use camera matrix" option unless colours are clipping, it's just less accurate. Also you showed a different shot, processed with other software, to show the issues were fixed, what happened with the first one though?

The only reason "don't use camera matrix" might seem to make the "very green" issue go away, is the fact that it desaturates the image, but it will become visible as soon as you add saturation.

I think it's better for the main developers here to fix the black level issues in their magic lantern branch® than deny it or blame MLV App.

yea true, i didn't know that "don't use camera matrix" just desaturated the colors, but there is definitely an issue with the black level. We can almost solve it with color grading but the problem is still here.

The first shot I showed here is still really green...

Sorry, I am still very new to this "world" so I don't know a lot of things  8)  ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on June 14, 2019, 12:31:45 AM
@Galterius
That Problem is not new for me, i Posted this problem many days before, and so far there is no solution.

The main problem is that it behaves intermittently, so it is not possible to edit the RAW settings easily. And this probelm is only on all 2.6k builds.

But I found out that it is reducible, if one takes a static video (camera + tripod + no movement). If you move the Camera (especially between hard contrasts) the greenish blacklevel cames out.

maybe someone understands why?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on June 14, 2019, 11:29:26 AM
Here is a short test with the Anamorphic build.



There is much aliasing but better crop-factor than 3K, 2.6K build. And with 14bit setting (and global draw alow) I have a full-low-frame-live-preview. So it is useful.

this is not the centered version.

After cutting the blackbar I have 4065 pixels. I also have a few stripes at the beginning, but after waiting 10 seconds the problem disappears.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 14, 2019, 02:07:29 PM
Quote from: Bender@arsch on June 14, 2019, 12:31:45 AM
@Galterius
That Problem is not new for me, i Posted this problem many days before, and so far there is no solution.

The main problem is that it behaves intermittently, so it is not possible to edit the RAW settings easily. And this probelm is only on all 2.6k builds.

But I found out that it is reducible, if one takes a static video (camera + tripod + no movement). If you move the Camera (especially between hard contrasts) the greenish blacklevel cames out.

maybe someone understands why?


ah thats sad... We will wait
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 14, 2019, 02:11:27 PM
By the way I did some tests in 4096x590 this morning but I have a lot of vertical lines in my shots, in the 4 I did... Maybe it has something to with the shutter speed, I had tp set it on 1/40 on the camera, and on the screen it was written 1/42 (this is the closest to 1/48 I can get)


Bender@arsch which shutter speed did you choose ? An you recorded in 14 bit ? What difference does it make ? IDoes it turns "on" some sort of live view while recording with this resolution ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 15, 2019, 08:33:15 AM
Ok playing around with 2784x1160 @ 23.976 Real Time liveview preview , I think I figured out the issue with black level in 2.6K
and applied then fix to 2784x1160 . Now I'm having issue with the cmos[1] vertical offset so you will have to use the advanced
crop_rec menu and adjust cmos[1]lo to 18 & cmos[1]hi to 62
(https://i.ibb.co/yBfMhFR/VRAM12-small.png) (https://imgbb.com/)

issue seems to be with the "cmos_new[1] PACK12(9+2 ,42+1)"
I'm not sure what "pack12" comes from , if I could understand what the number are calling (some kind of centering) for vertical offset
if I disable the pack12 the image cleans up but it pushes up with a white bar at the bottom , so you have to fine adjust it with the advanced
crop_rec menu like the image above until I figure out how to code it centered vertically .
So remember , expect problems , lockup's or anything odd as it could happen at any time !
This build is for test proposes only !! I really looking for any black level changes while recording .
magiclantern-crop_rec-2784x1160_24p_11.39pm-5D2-eXperimental.2019Jun14.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-2784x1160_24p_11.39pm-5D2-eXperimental.2019Jun14.5D2212.zip)

Edit: so if you add the cmos[1] lo18 & hi62  together you get 80 now that's the same as the white area from the image height
Original vertical is 1160 if you crop off the white area it's 1080 ,  so does this mean "pack12etc. ...." is just the vertical centering .
So tomorrow I'll change the "pack12 (9+2 ,42+1)" to "pack12 (18+62)" and see what happens  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on June 15, 2019, 08:47:24 AM
Do an exception in raw.c where auto black level is done...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 15, 2019, 09:27:24 AM
Got "pack12 stuff" figured out , and yes it's for centering vertical offset just need to do some math .
https://www.magiclantern.fm/forum/index.php?topic=19336.msg217565#msg217565
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 15, 2019, 07:45:55 PM
Quote from: reddeercity on June 15, 2019, 08:33:15 AM
Ok playing around with 2784x1160 @ 23.976 Real Time liveview preview , I think I figured out the issue with black level in 2.6K
and applied then fix to 2784x1160 . Now I'm having issue with the cmos[1] vertical offset so you will have to use the advanced
crop_rec menu and adjust cmos[1]lo to 18 & cmos[1]hi to 62
(https://i.ibb.co/yBfMhFR/VRAM12-small.png) (https://imgbb.com/)

issue seems to be with the "cmos_new[1] PACK12(9+2 ,42+1)"
I'm not sure what "pack12" comes from , if I could understand what the number are calling (some kind of centering) for vertical offset
if I disable the pack12 the image cleans up but it pushes up with a white bar at the bottom , so you have to fine adjust it with the advanced
crop_rec menu like the image above until I figure out how to code it centered vertically .
So remember , expect problems , lockup's or anything odd as it could happen at any time !
This build is for test proposes only !! I really looking for any black level changes while recording .
magiclantern-crop_rec-2784x1160_24p_11.39pm-5D2-eXperimental.2019Jun14.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-2784x1160_24p_11.39pm-5D2-eXperimental.2019Jun14.5D2212.zip)

Edit: so if you add the cmos[1] lo18 & hi62  together you get 80 now that's the same as the white area from the image height
Original vertical is 1160 if you crop off the white area it's 1080 ,  so does this mean "pack12etc. ...." is just the vertical centering .
So tomorrow I'll change the "pack12 (9+2 ,42+1)" to "pack12 (18+62)" and see what happens  :D


Thats awesome ! i will try it ASAP to see if the black level issue is fixed ! Thanks a lot !
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 15, 2019, 08:15:12 PM
So I just tried it (2784x1160), here are some clips, colors are clipping every 3/5 seconds (maybe its because of the shutter speed, I could not set it to 1/48)
It still seems a little bit too green (but maybe I am wrong), I will try on skin tones


https://drive.google.com/drive/folders/1gfgZUH7ruZd4HlZC5amwb1lDWoKYg1P3?usp=sharing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 15, 2019, 09:11:29 PM
In canon menu set to 1080 30p
NTSC .
In magic lantern set shutter to
1/60th of second or close to it (1/63th) while your in 3x3 (FHD)
and it will change to 1/48th to 1/52th of a second . When you enable crop_rec
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 15, 2019, 10:51:39 PM
Quote from: reddeercity on June 15, 2019, 09:11:29 PM
In canon menu set to 1080 30p
NTSC .
In magic lantern set shutter to
1/60th of second or close to it (1/63th) while your in 3x3 (FHD)
and it will change to 1/48th to 1/52th of a second . When you enable crop_rec

ok thats what I did  :)

by the way you just wrote your 2000th message here  :o
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 16, 2019, 07:05:26 AM
2784x1160 @ 23.976 real time Liveview preview
I was right about the "PACK12 Stuff"(vertical fine offset adjustment)  just needed to put in the correct offset number's
So now there no need to go in to the advanced crop_rec menu , should be very close to correct vertical centering now .

Caveat if you load Snd_mo (sound module) you may/will get corrupted frames !!
I highly recommend only loading crop_rec.mo & mlv_rec.mo that's it ! I'm running very close to the edge on this preset .
I've squeezed every bit of performance I know how to get to make this work .

The Raw dump size is 2792x1160 but with mod8 rules it get shaved down to 2784x1160
The total active area is 2952x1212 so if we do some simple math to find out Mega pixel per second (MP/s) the speed of the sensor
2.952x1.212x23.976= 85.78 MP/s , is this the limit ? maybe but I haven play around with the other headtimers (1&2) yet so
there might be some more room to reduce the overhead even more , time will tell but until then........

magiclantern-crop_rec-2784x1160_24p_10.24pm-5D2-eXperimental.2019Jun15.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-2784x1160_24p_10.24pm-5D2-eXperimental.2019Jun15.5D2212.zip)

Edit: when you are set to 1/60th (ML 1/63th) of a second for shutter speed the crop_rec will be 1/52th second shutter
that's the closest I can get right now to 1/48th of second .

Edit#2: I updated the first post on the first page of this thread with the new test build .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 16, 2019, 11:26:41 AM
Awesome !! I will also try it ASAP !!

By the way, is it normal if I get lots of vertical lines when using 4096x590 ? Weird thing is that these lines appear mostly on the blurry background, and where its focussed, there are no lines...
I will try to send some clips so you can see what i am talking about ;)

#Edit : here is a clip, no color correction, exported to Apple Prores 4444 (yea thats overkill but I wanted to try it)

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: RH42 on June 16, 2019, 11:19:51 PM
@Galterius Have you switched off the Auto Lighting Optimizer in the Canon menu? Just a thought - you probably have it switched off, but worth checking. I had problems caused by that under certain lighting conditions back in the day.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 17, 2019, 05:01:54 AM
Yea I know the problem , it's because of 2 thing
1) The black bar on the right side is part of the problem
2) The override vertical fine adjustment (cmos[1]lo & hi  in crop_rec advanced menu is
fighting the with the code
PACK12 (9+1 , 42+1)
which is doing the small thing , this need to be changed ,
I haven't done that yet (as I just discovered how to code it correctly last night )
Soon , I may look at it tonight yet .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 18, 2019, 02:23:14 PM
Quote from: RH42 on June 16, 2019, 11:19:51 PM
@Galterius Have you switched off the Auto Lighting Optimizer in the Canon menu? Just a thought - you probably have it switched off, but worth checking. I had problems caused by that under certain lighting conditions back in the day.

I will try that !! Hope it will fix it !

Edit 1 : it is switched off...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 18, 2019, 02:24:07 PM
Quote from: reddeercity on June 17, 2019, 05:01:54 AM
Yea I know the problem , it's because of 2 thing
1) The black bar on the right side is part of the problem
2) The override vertical fine adjustment (cmos[1]lo & hi  in crop_rec advanced menu is
fighting the with the code
PACK12 (9+1 , 42+1)
which is doing the small thing , this need to be changed ,
I haven't done that yet (as I just discovered how to code it correctly last night )
Soon , I may look at it tonight yet .

Ok, thanks a lot !  (I can wait, I have other builds to wait  :P)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 19, 2019, 05:21:28 AM
I'm very close to having 4k Anamorphic (10bit raw 4064x590->4064x1770) coded , still having a problem with "pack12" applying to cmos[1] , but there no problem with
the 3x crop preset 2784x1160 , I need to write some more yet , almost there  :)
Here a quick test , from my back yard yesterday , MLV App -> to 10bit dng's ->import in to CS6 Adobe A.E. , grade with ACR ->export ProRes4444->upload to YouTube
To see the 4k stream watch on chrome ,
FYI: ISO 100 1/50th , Canon EF24-70mm f2.8L , crop factor 1.38 shot at 24(33.12)mm & 70(96.6)mm @ f8  & ProMaster Variable ND for video set to ND3

Sorry for the double post of the Video  ::) thought it should be here also .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 19, 2019, 09:00:47 AM
Quote from: reddeercity on June 19, 2019, 05:21:28 AM
I'm very close to having 4k Anamorphic (10bit raw 4064x590->4064x1770) coded , still having a problem with "pack12" applying to cmos[1] , but there no problem with
the 3x crop preset 2784x1160 , I need to write some more yet , almost there  :)
Here a quick test , from my back yard yesterday , MLV App -> to 10bit dng's ->import in to CS6 Adobe A.E. , grade with ACR ->export ProRes4444->upload to YouTube
To see the 4k stream watch on chrome ,
FYI: ISO 100 1/50th , Canon EF24-70mm f2.8L , crop factor 1.38 shot at 24(33.12)mm & 70(96.6)mm @ f8  & ProMaster Variable ND for video set to ND3

Sorry for the double post of the Video  ::) thought it should be here also .

amazing ! Did you notice any issue with black level ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on June 19, 2019, 02:27:34 PM
aliasing on that bricks?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on June 19, 2019, 08:09:57 PM
Quote from: waza57 on June 10, 2019, 05:57:46 PM
I do not have enough time to dig through the whole thread but it would be really nice to have a summary of the registry changes to get the progress that reddeercity found:

2624x1200 @ 23.976 fps with the full vertical heigh
4096x590 (Anamorphic 4096x1770) @ 23.976 fps
....and other things that don't remember.

These only in these conditions I could easily update the source code.
@waza57, it would be sooo cool to have the source code updated with all this nice stuff, to make the 5D2 fly finally! I was searching through the whole thread, searching for the registers reddeercity found. Here is the summary (hope I don't forgot something important):

3x1 4k (4096x590(4096x1770) 23.976fps)
https://www.magiclantern.fm/forum/index.php?topic=19336.msg209935#msg209935
https://www.magiclantern.fm/forum/index.php?topic=19336.msg210017#msg210017
https://www.magiclantern.fm/forum/index.php?topic=19336.msg217151#msg217151
https://www.magiclantern.fm/forum/index.php?topic=19336.msg217439#msg217439

3x1 3.6K
https://www.magiclantern.fm/forum/index.php?topic=19336.msg208931#msg208931

3x3 48/50/60fps
https://www.magiclantern.fm/forum/index.php?topic=19336.msg206767#msg206767
https://www.magiclantern.fm/forum/index.php?topic=19336.msg206897#msg206897
https://www.magiclantern.fm/forum/index.php?topic=19336.msg208955#msg208955
https://www.magiclantern.fm/forum/index.php?topic=19336.msg211441#msg211441

2400x1200
https://www.magiclantern.fm/forum/index.php?topic=19336.msg208497#msg208497
https://www.magiclantern.fm/forum/index.php?topic=19336.msg208879#msg208879

2400x1330
https://www.magiclantern.fm/forum/index.php?topic=19336.msg210327#msg210327
https://www.magiclantern.fm/forum/index.php?topic=19336.msg210865#msg210865

2560x1200
https://www.magiclantern.fm/forum/index.php?topic=19336.msg209086#msg209086

3008x1080
https://www.magiclantern.fm/forum/index.php?topic=19336.msg211276#msg211276
https://www.magiclantern.fm/forum/index.php?topic=19336.msg211379#msg211379

Unfortunately I did not find registers for 1x3 which should give the best quality (when comparing footage from other camera's).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 20, 2019, 12:29:13 AM
Quote from: reddeercity on June 19, 2019, 05:21:28 AM
I'm very close to having 4k Anamorphic (10bit raw 4064x590->4064x1770) coded , still having a problem with "pack12" applying to cmos[1] , but there no problem with
the 3x crop preset 2784x1160 , I need to write some more yet , almost there  :)
Here a quick test , from my back yard yesterday , MLV App -> to 10bit dng's ->import in to CS6 Adobe A.E. , grade with ACR ->export ProRes4444->upload to YouTube
To see the 4k stream watch on chrome ,
FYI: ISO 100 1/50th , Canon EF24-70mm f2.8L , crop factor 1.38 shot at 24(33.12)mm & 70(96.6)mm @ f8  & ProMaster Variable ND for video set to ND3

Sorry for the double post of the Video  ::) thought it should be here also .

@Reddeercity, sorry for living one of your message without an answer. Had difficult days...
This is an absolutely cool resolution on your video, very noticeable on the bricks . Just tested out 4kanamo preset compared with 1856 one in real life and noticed that 4kanamo is really much more detailed than 1856. Actually, before i didn't believe that there is a real difference...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 20, 2019, 04:17:00 AM
Even though there maybe a little aliasing , (I think the brick is really me bumping the camera) resolution always wins out !
1856 , is every third column (5616/3=1872) so you are missing data really . 3x1 is every column just with 4k it's at a crop factor of 1.38 .
If I had to choose between 3x3(FHD) 1856x1248  & 3x1 4064x590->4064x1770 , pick 3x1 4k Anamorphic all day long .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 20, 2019, 04:38:59 AM
New test build crop_rec-4kAnamorph_12.36am-5D2-eXperimental.2019Jun17.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-4kAnamorph_12.36am-5D2-eXperimental.2019Jun17.5D2212.zip)
I'm dropping 2600x1200 in favor of 2784x1160 @ 23.976fps realtime preview , just too many issue with 2.6k beside that
2784x1160 is at 2.4 A.R.  and the sensor runs cooler temperature wises also .
I reduced the 4k Anamorphic resolution from 4096 to 4064 , to maintain 23.976fps at 4096 I can only get 24.001fps or 23.98fps
but I didn't change the vertical it's still 590->1770 , I still have a problem with vertical centering so you have to use the
advanced crop_rec menu and set
cmos[1]lo->11
cmos[1]hi->61

then do the normal thing to get to it , make sure you refresh liveview before recording raw video .
Best way I found is to use the play button twice then that get to 3x3 , then enable crop_rec
should good from there , once I get those centering regs working you won't have to go though
all that refreshing of live view . This is the build I used to make the video in the above test video
of my back yard.

Edit: I'm slowly removing all those test builds on the first page of the this thread , I'll leave 3008x1080
but all the 2.6k build are gone , if you really need it just request it and I'll look at it .
So 2784x1160 will be the new standard for 3x crop_rec & 4k Anamorphic 4064x590 for now .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 21, 2019, 04:39:26 AM
New test build , magiclantern-crop_rec-4k-3x1_2784x1160_1920centered_7.54pm-5D2-eXperimental.2019Jun20.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-4k-3x1_2784x1160_1920centered_7.54pm-5D2-eXperimental.2019Jun20.5D2212.zip)

Ok in 4k 3x1 Anamorphic , you now longer have to go to the advanced crop_rec menu to set the centered vertical , got that figured out now.
I notice when you enter in to that preset (4k 3x1), you will see some vertical lines at first , you need to refresh liveview !
best way , is to turn off "Raw Video" tab press the half shutter button , that will take you a frozen liveview , press trash button to enter ML menu
turn on "Raw Video" tab (should say 4064x590) and press half shutter to refresh and now you are good to go should have clean liveview now .

2784x1160 is the new 5x zoom preset .

I also add 1920x1200 1:1 centered in 3x3 , this one is still having some problems , one of the reg's didn't apply 
ADTG12 0x1000 0x6 ->0x5
need to do some more work on it .

Have to add several new lines of code to  crop_rec to get 3x1 4k to work , basically 3x1 preset structure was missing
took me a little while to wrap my head around what was missing . Once I debug the problem with "ADTG Reg's" not applying
all the rest of the preset will come very quickly , since all the rest of the preset rely on adtg12 reg's working . 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 21, 2019, 05:21:02 PM
Quote from: reddeercity on June 21, 2019, 04:39:26 AM
New test build , magiclantern-crop_rec-4k-3x1_2784x1160_1920centered_7.54pm-5D2-eXperimental.2019Jun20.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-4k-3x1_2784x1160_1920centered_7.54pm-5D2-eXperimental.2019Jun20.5D2212.zip)

Ok in 4k 3x1 Anamorphic , you now longer have to go to the advanced crop_rec menu to set the centered vertical , got that figured out now.
I notice when you enter in to that preset (4k 3x1), you will see some vertical lines at first , you need to refresh liveview !
best way , is to turn off "Raw Video" tab press the half shutter button , that will take you a frozen liveview , press trash button to enter ML menu
turn on "Raw Video" tab (should say 4064x590) and press half shutter to refresh and now you are good to go should have clean liveview now .

2784x1160 is the new 5x zoom preset .

I also add 1920x1200 1:1 centered in 3x3 , this one is still having some problems , one of the reg's didn't apply 
ADTG12 0x1000 0x6 ->0x5
need to do some more work on it .

Have to add several new lines of code to  crop_rec to get 3x1 4k to work , basically 3x1 preset structure was missing
took me a little while to wrap my head around what was missing . Once I debug the problem with "ADTG Reg's" not applying
all the rest of the preset will come very quickly , since all the rest of the preset rely on adtg12 reg's working .

Awesome !! I will try again 4064x590, to see if I still have some issues with vertical lines !

Edit : I just tried what you said (turn off raw, refresh, turn it back on), and I don't have vertical lines ! Black level is still off I think, even in 2784x1160, sadly. (I did not try the new build yet)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 22, 2019, 03:53:38 AM
Quote from: Galterius on June 21, 2019, 05:21:02 PM

........  Black level is still off I think, even in 2784x1160, sadly. (I did not try the new build yet)
What ? you say black level is off ,but you didn't try the new build ?
Then how do you know the black level is off ?
black level is ok here no problems
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 22, 2019, 03:43:40 PM
Quote from: reddeercity on June 22, 2019, 03:53:38 AM
What ? you say black level is off ,but you didn't try the new build ?
Then how do you know the black level is off ?
black level is ok here no problems

Yea sorry thats a bit stupid ^^
Did you have problems with black levels on previous builds ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: OlRivrRat on June 23, 2019, 12:59:55 AM
   Since almost none of this HiRes RawVid Fancy Dancing ever seems to work for Me,

I get more & more hesitant to take the time to try it > Finally talked Myself into giving

the latest build a shot on a newly acquired SanDisk ExtremePro 160MB/s 64GB card &

all I can seem to get out of the 3.5k 1:1 centered x5 is about 106 to 125 Frames before

it Quits (MLV Files on card range 432-537MB) & the Cam warms up to 50ºC pretty quick ~

Haven't yet tried to process the MLV to see what I actually got ~
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 23, 2019, 04:58:54 AM
@ OlRivrRat , what is the raw bit rate ?
I have a older Lexar 1066x 32GB (bought it when the cards just when from 1000x to 1066x)
I get up to 15+ seconds 2784x1160 @ 10bit , just did I quick test again and I got 311 frames (12.9 seconds)
then I stop it . I usually set the "Buffer Fill Method" in the Raw Video Tab  to  "0" , this can make  big difference 
in write speed , I see an average write speed of 78.1MB/s which peaks to 80 MB/s . You may what to play around with those setting
and see if there's a different (0->4)
   
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on June 23, 2019, 07:18:07 AM
@Reddeercity
In the 3,5K Preset I can use audio without corrupt frames (~12sec.) and I have only in 14bit a unfrozen low-frame-real-time liveview (grayscale) too, like 4K Preset. Canon live view works with all 10-14bit. No black level problems! But the 3.5K Preset is centered on bottom. 14bit stops working after 5 sec. (hang off - > restart) - >maybe over the limit^^ but it will save correctly.

Good Job!

If you complete lossless compression, it will changed only CF Card space (in minutes) or write speed too?

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 23, 2019, 08:51:25 AM
Quote from: Bender@arsch on June 23, 2019, 07:18:07 AM
the 3.5K Preset is centered on bottom.
Yea I just notice that  ,  tomorrow I see if I have time to fix that  .
Quote from: Bender@arsch on June 23, 2019, 07:18:07 AM
If you complete lossless compression, it will changed only CF Card space (in minutes) or write speed too?
It will reduce the data rate to around 50% of the file so 10bit uncompressed 2784x1160 @ 23.976 need about 90MB/s
so with compression that should be around 60MB/s see for your self
https://rawcalculator.netlify.com/calculator_desktop

(https://i.ibb.co/kcfs6x0/5-D2-record-time.png) (https://imgbb.com/)

The lossless compression is in 14bit also it the same a CR2 photo's
and if you look at the frames size 14 bit lossless is 2.7MB where the 10bit uncompressed is 3.9Mb .
So yea 65MB/s is great , you could record all day long at14bit lossless 2784x1160 @ 24fps
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: OlRivrRat on June 23, 2019, 05:40:58 PM
   @RDC

"@ OlRivrRat , what is the raw bit rate ?" > 10

"I usually set the "Buffer Fill Method" in the Raw Video Tab  to  "0" > I'll try it & report ~
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: OlRivrRat on June 23, 2019, 06:26:31 PM
   @RDC

"Buffer Fill Method" in the Raw Video Tab  to  "0" > Still only 110 frames

Also, if I choose anything other than "3.5k 1:1 centered x5" in CropRec the Cam becomes

totally unresponsive while in LiveView Mode ~
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 23, 2019, 10:19:44 PM
@Galterius

I'm getting those vertical lines as well as "white and black long time balance (for like a minute) auto adjustment color effect" for a few months already since I started to test all these experimental builds. And not only in raw mode. And also when i remove the ML bootflag i keep getting it sometimes with regular h264. Yesterday i got it during my work. I think for now there is no issue to resolve the problem. Only play button double click, or press rec for a second and stop... or restart the camera...

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 23, 2019, 11:30:28 PM
I made the cropmark, that allows you to view the composition without vertical stretching before you turn 4k anamorphic on with certain cmos hi and lo settings.
https://drive.google.com/file/d/1iOvXj_5aFVZO01stEWZTNP7M1AB6b_qX/view?usp=sharing

Works well with the "magiclantern-crop_rec-4k-3x1_2784x1160_1920centered_7.54pm-5D2-eXperimental.2019Jun20.5D2212" build.
The order is:
1. Turn on the "4kanamo_01" cropmark
2. Set the composition
3. Turn off cropmarks
4. Set cmos[1]lo to 9 cmos[1]hi->63
5. Enter the 4k anamorphic preset
And you can shoot now with exactly the same composition
^^

PS: @reddeercity, i found out that without setting the Hi and Lo CMOS you get almost bottom centered image... and more to the left. cmosLo =9 it's also not perfect but more or less centered as well as 63. CmosLo 11 is too much to the down.
Also i'm trying to do something to make the liveview work well but without any success.
Theoretically is there any way to use regular canon liveview during shooting 4k anamorphic? We can simply glue a piece of black cardboard cut out in the center in the form of the preset. I'm talking about that because saw the 4k anamorphic preset description "To use this mode, exit ML menu and press the zoom button (set to x1). So, does 4kanamo work in x1 zoom mode? Or this description is not correct because it's from any old preset?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 24, 2019, 01:04:42 AM
@Igor_Braun , there's a  new build (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-4k-3x1_2784x1160_1920centered_7.54pm-5D2-eXperimental.2019Jun20.5D2212.zip) , sorry I should have updated the first post , there no need to adjust cmos[1]lo & hi read here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg217774#msg217774)
Thanks for the Crop Marks  :)
The 1x3 4k (https://www.magiclantern.fm/forum/index.php?topic=19336.msg213010#msg213010) Anamorph is far better user experience , (coming very soon) 
Just in case everyone miss this (https://www.magiclantern.fm/forum/index.php?topic=19336.msg217774#msg217774)
Quoteyou will see some vertical lines at first , you need to refresh liveview !
best way , is to turn off "Raw Video" tab press the half shutter button , that will take you a frozen liveview , press trash button to enter ML menu
turn on "Raw Video" tab

@OlRivrRat , not sure what's the problem ,  you should benchmark your card to make sure it up to speed .
Did you set up crop_rec  as pre the picture tutorial on the first post of this thread ?
Maybe make quick cell phone video on how you set It up .

I think it time to make a new tutorial for the first page . 


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on June 24, 2019, 01:06:42 AM
Very exciting topic to read, a lot of people want the sourcecode but Dave keeps the code non public, i totally understand that guys are so excited and want to take a look at the code, but somehow its misunderstood, i get you prefere to polish everything , people want to learn cause Youre doing great things here squeezing the camera to the extremes but sometimes teamwork creates issues, i personally dont like it and prefere to work 3 months alone than split work with someone and do it in a month , mostly because of bad past experiences, and unequal skills.Fixing after them later and it takes 5 months now.
What could we do to help with lossless compression testing ? Recently some of the big resolutions are at maximum and lossless would greatly help to cut datarate in half so people wont hit the cardspeed limit and more of them could do tests. Is lossles raw in a state where it could be tried out somehow ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 24, 2019, 01:45:40 AM
This is as far as I got , the problem is the cam locks up when trying to save the lossless just need to figure that out and lossless works .
https://www.magiclantern.fm/forum/index.php?topic=18443.msg198487#msg198487
That the last step , I can create a lossless silent dng but can't save It , freezes the cam needs a battery pull.
I have a good idea what's the problem is but haven't had the time to test out my theory

You can start reading from here (https://www.magiclantern.fm/forum/index.php?topic=18443.msg197462#msg197462) to see what's being going on
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 24, 2019, 05:52:22 AM
@Igor_Braun  , I found the code to for Anamorphic preview (In tweaks.c) and it shouldn't be too big of a problem to add a 3x1 preview plus I'll add one for 1x3 also
Edit: That didn't work on 3x1, because there's a scrambled Canon liveview (I haven't figured out all the reg's for clean canon liveview in 3x3(FHD) with 3x1 4k .
But it will work with 1x3 .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on June 24, 2019, 08:08:36 AM
Quote from: 2blackbar on June 24, 2019, 01:06:42 AM
... i totally understand that guys are so excited and want to take a look at the code, but somehow its misunderstood, i get you prefere to polish everything , people want to learn cause...
That is not the point. The point is, people know how to fix the bugs we still have here since so a long time. But they only can fix it with source code. If just one guy is working on this, it will take a very long time, if it gets ready at all. With team work, this could all have been solved since months.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 24, 2019, 10:36:30 AM
I tried the new build, in 4064x590, and black level is far better than it was, I just need to make some tweaks with the green curve.

This is an awesome resolution, because we can shoot way longer than 2,7k, small crop factor, only downside is the preview, but for static subjects its ok.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 24, 2019, 10:52:17 AM
@reddeercity
That sounds so exciting... Want to try 1x3 so much.
So, for now, this evening-night (because I haven't ND filters yes) i'm going to test out 3x1 in a city park and compare it with 1856 again =)
Will share the results.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on June 24, 2019, 06:00:07 PM
Quote from: reddeercity on June 24, 2019, 01:45:40 AM
This is as far as I got , the problem is the cam locks up when trying to save the lossless just need to figure that out and lossless works .
https://www.magiclantern.fm/forum/index.php?topic=18443.msg198487#msg198487
That the last step , I can create a lossless silent dng but can't save It , freezes the cam needs a battery pull.
I have a good idea what's the problem is but haven't had the time to test out my theory

You can start reading from here (https://www.magiclantern.fm/forum/index.php?topic=18443.msg197462#msg197462) to see what's being going on
Thanks for the info Dave, so youre actuall very close to make it working, looking up how it works on other cams is a good way to crack it, today i finally have decent card for 5D2, maybe not the best but works ok and wasnt that expensive(canvas focus 128GB udma7)
I tested croprec builds and high resolutions are very impressive, On M and 5D2 i use fps override to 23.976 exact FPS and shutter 1/45 because it gives me nicest motion blur that works good evenwith handheld footage(much less micro jitters) , enabling fps override changes how shutter blur looks like and theres less rolling shutter from what i see on these cams.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: OlRivrRat on June 24, 2019, 07:00:47 PM
   @RDC

Pretty sure I followed 1st Post Tut' correctly ~

CFCard BnchMrks @ about 65MB/s LiveView On & 87MB/s LiveView Off

This CF Tests about 109MB/s in a 5DM4
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 25, 2019, 01:12:26 AM
Today I was late to the park and before sunset was able to shoot only one single shot using the new build...  Like the resolution so much and there are no side bars more.
https://youtu.be/nM7hfO5dkcM
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Ottoga on June 25, 2019, 12:32:57 PM
@Igor_Braun - Getting video unavailable error message.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 25, 2019, 06:11:57 PM
Quote from: Ottoga on June 25, 2019, 12:32:57 PM
@Igor_Braun - Getting video unavailable error message.

Error fixed.
PS: I set the Blackmagic profile in MLV Producer, decreased saturation from 100 to 56, exported to png, then added a little zoom in effect in editing program and rendered that to Avi uncompressed with 4064x1770 res
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 25, 2019, 10:19:41 PM
@reddeercity, correcting the preview in 3x1 preset using anamopric feature would be a good solution. I was looking for the code location but did't find it. Will check the tweaks.c, thanks! It's interesting for me too.

And also I noticed that if you select any of anamorphic view corrections it doesn't affect the image in the case of 4k anamorphic preset
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on June 25, 2019, 11:23:21 PM
With cheap Kingston 128GB canvas focus im getting 13 seconds in 2784x1160 but fps override has to be disabled.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 26, 2019, 10:04:48 PM
Did a comparison of 3 presets: regular 1856x790, 2784x1160 and 4064x590.
Before shooting i zoomed with the lens according to the crop factor of each preset.
Critically focused each time.
Lens: Nikkor 70-210 f/4.0
Aperture: 5.6
Iso: 400
In MLVP i set the saturation to 0, exposure to +3 and chose Blackmagic profile.

The whole images looks like this:

1856x790:
https://drive.google.com/file/d/1QSR48nk95eIA3N5FO9TyDnDCkwvcTU6W/view?usp=sharing
(http://i68.tinypic.com/ivyf7o.png)

2784x1160:
https://drive.google.com/file/d/11W53Eortc3acTAsksi-iJruOVoJ43nRE/view?usp=sharing
(http://i64.tinypic.com/34q4jv9.png)

4064x590
https://drive.google.com/file/d/1UDhqYLxRsc8OZlxGvqg-udXNkPeowwHB/view?usp=sharing
(http://i66.tinypic.com/e707xd.png)

Zoom in:
(http://i66.tinypic.com/ngt3m0.jpg)

2784 one came brighter than others, so let's level them by brightness:
(http://i66.tinypic.com/jq44ys.jpg)

4064 is my favorite although it has a lot of artifacts including a bit of digitalizing effect because of squeezing. But it's sharper than 2784 and has less crop factor, so it's more cinematic.

PS:
@Reddeercity
You said that the image is centered in 4k now, but look, when you set 9 and 34 with your latest build it looks like this:
(http://i68.tinypic.com/juaipy.png)
And if you set nothing:
(http://i63.tinypic.com/v4b0nc.png)

When did the cropmark I was using this latest build.


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on June 27, 2019, 08:12:14 AM
Something wrong with the test. 1160p seems softest of all 3. I think something in the brightness adjustment in post  made it softer.

Sent from my Redmi 4A using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 27, 2019, 08:19:11 AM
2784x1160 is not centered vertically yet , it on the lower half of the sensor so not in the center of the lens .
I've being  working on coding 48p in 3x3 (can of worms  :o) will not that bad a few bugs
I'll see if I can fix 1160p in the next few days  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 28, 2019, 07:19:13 PM
Understand what you mean. But if take a look more carefully it's noticeable that 1160 is more detailed while 790 is more "digitalized" so 790's one looks more sharp because of it. I think this happens because of squeezing the full frame image to 1856x790. And also as Reddeercity said, the image of 1160 is captured from the lower half of the sensor. The center of a lens is sharpest place...

@Reddeercity, 48 3x3 it's a full frame image with 48 fps?
PS:Waiting for the possibility to compare 3x1 with 1x3 4k anamos side to side, hehe
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on June 28, 2019, 08:17:24 PM
3x1 (tall one) has much better perceived resolution and less aliasing than 1x3(wide one) from previouis croprec tests
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 28, 2019, 10:30:16 PM
@Reddeercity, also would like to ask you. I just came from shooting event (was a college party). And got a bunch of videos with vertical lines like this:
(http://i65.tinypic.com/11aan0n.jpg)
To solve the problem i needed to press rec and stop several times.
Is there any way to avoid this problem?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on June 28, 2019, 10:39:43 PM
Power timing regs. Hand me the code, I take a look.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 29, 2019, 12:33:10 AM
@Danne, Hope it's the problem, that you mentioned. But which code should I hand you and where is it located? It's regular h264, not MLV.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 29, 2019, 06:11:15 AM
No you didn't refresh Liveview , I mentioned this many times before
https://www.magiclantern.fm/forum/index.php?topic=19336.msg217882#msg217882
turn off raw video "tab" exit to liveview , pan the cam around or just move it then let sit for a few seconds (10 or so)
re-enable raw video "tab" exit to Liveview all should be good now .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: OlRivrRat on June 29, 2019, 05:07:47 PM
           @Igor_Braun

Best Way to Avoid Might Be to Not Use Experimental ML when shooting an Event ~ {:~))
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 29, 2019, 07:35:06 PM
I don't know why but I get some heavy aliasing when shooting 4064x590...

https://drive.google.com/file/d/1xzii_J8viXX3q8RF9deiEayK97lrQY05/view?usp=sharing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on June 29, 2019, 09:05:00 PM
Its normal because 590 is not much when width is 4064, so upscaling will get a lot of aliasing, if 590 is skipping its more aliasing, if binning, there still gonna be some but less.
Read this about 1x3 and 3x1  https://www.magiclantern.fm/forum/index.php?topic=16516.25
Also the file is about 400MB, i dont think many guys have time to download this and check, its better to post the frame.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 29, 2019, 10:35:44 PM
@Reddeercity
@OlRivrRat

The thing is i don't use experimental build when shooting events... And don't turn on raw modules. It started to appear from the moment i installed an experimental build the first time (including when don't use ML or when i use stable version). Also with it, i have strange problems with turn on lifeview
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 29, 2019, 10:52:50 PM
Quote from: 2blackbar on June 29, 2019, 09:05:00 PM
Its normal because 590 is not much when width is 4064, so upscaling will get a lot of aliasing, if 590 is skipping its more aliasing, if binning, there still gonna be some but less.
Read this about 1x3 and 3x1  https://www.magiclantern.fm/forum/index.php?topic=16516.25
Also the file is about 400MB, i dont think many guys have time to download this and check, its better to post the frame.

sorry I will post the frame, and I will read what you sent me !
But its weird because I did not have these problems before
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 29, 2019, 10:56:08 PM
Quote from: Galterius on June 29, 2019, 07:35:06 PM
I don't know why but I get some heavy aliasing when shooting 4064x590...

https://drive.google.com/file/d/1xzii_J8viXX3q8RF9deiEayK97lrQY05/view?usp=sharing
I don't see the problem , quick export to h264 M27-1836.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M27-1836.mov)
Yes there will be some aliasing (Very little) , but not nearly as bad as 3x3 (FHD) and in some cases you will see no issue at all .
It comes down to how to you process (color grade) you file .

Just a FYI; I don't mind downloading a test .mlv file up to 1GB ( I'm on a commercial connection 300 download 15 up)
But you can shorten them in MLV App by selecting the number of frames and save as a .mlv .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 29, 2019, 11:12:57 PM
@ Igor_Braun , don't think it's a ML issue , so you are saying even on the nightly builds (https://builds.magiclantern.fm/#/) or 5D2_10-12bit  (https://builds.magiclantern.fm/jenkins/job/raw_video_10bit_12bit/53/artifact/platform/5D2.212/magiclantern-raw_video_10bit_12bit.2019Mar24.5D2212.zip) stable build you have that problems  ?
Do you have any customs or use any customs setting (e.g. C1 ,C2, etc. ...) other then Manual mode on the setting dial on the left hand side top of camera .
If problem persist maybe reset all camera setting  .
I don't have any of those issue , so I hard to figure where you problem is .
 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 29, 2019, 11:29:29 PM
Quote from: reddeercity on June 29, 2019, 10:56:08 PM
I don't see the problem , quick export to h264 M27-1836.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M27-1836.mov)
Yes there will be some aliasing (Very little) , but not nearly as bad as 3x3 (FHD) and in some cases you will see no issue at all .
It comes down to how to you process (color grade) you file .

Just a FYI; I don't mind downloading a test .mlv file up to 1GB ( I'm on a commercial connection 300 download 15 up)
But you can shorten them in MLV App by selecting the number of frames and save as a .mlv .

Maybe the mlv app didn't launch correctly, I will try again.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 30, 2019, 08:24:08 AM
@reddeercity i use only manual mode and yes, only safe stable night build for 5d2 when shooting events. I also think, that it's because of the camera, not ML. Today was keeping testing ml and bricked my lexar 1066... Just turned off and on again the camera, and "card not formatted"   text appeared. Cant open and format it on camera and PC, i'm totally unlucky... But will keeping testing every new stuff anyway
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 30, 2019, 05:05:24 PM
Here are some screenshots of 2 clips exported to ProRes 422HQ, I Don't know why I get this issue.. Its not really aliasing

(https://i.ibb.co/ZxXc5FT/ML-pb2.png) (https://ibb.co/cQ320ZT)


(https://i.ibb.co/3W27rjQ/ML-pb3.png) (https://ibb.co/VDRtjz5)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on June 30, 2019, 05:17:53 PM
Quote from: Galterius on June 30, 2019, 05:05:24 PM
Here are some screenshots of 2 clips exported to ProRes 422HQ, I Don't know why I get this issue.. Its not really aliasing
Sure, this is 3x1 aliasing. Somewhere there is a thread in the forum, where some guys compared 3x1 with 1x3, where 1x3 is way better. In this thread there was also the explanation why. From my own tests I can tell you, 3x1 always produces the artifacts you show here - also on other cameras like the EOSM. That's why I don't use it. ;)

Edit: just found this, but there was a more academic explanation and research post somewhere:
Quote from: theBilalFakhouri on April 26, 2019, 11:49:09 PM
Lines x Columns

1x1 Sampling is read every line (vertical pixels) and every column (horizontal pixels) without any skipping or binning, no aliasing and moire and you will see more noise in this mode , also Movie Crop Mode and x5 Mode are using 1x1 Sampling.

3x3 Read one line skip two every 3 lines/Bin every 3 columns. And skipping lines causes aliasing and moire.

1x3 Read all lines/Bin every 3 columns. No skipping at all only Binning pixels in columns, no aliasing/no moire and better details than 3x3.

3x1 Read one line skip two every 3 lines/Read all columns. Better details than 3x3 but with same alisaing and moire since we are skipping lines in this mode.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 30, 2019, 06:59:46 PM
Quote from: masc on June 30, 2019, 05:17:53 PM
Sure, this is 3x1 aliasing. Somewhere there is a thread in the forum, where some guys compared 3x1 with 1x3, where 1x3 is way better. In this thread there was also the explanation why. From my own tests I can tell you, 3x1 always produces the artifacts you show here - also on other cameras like the EOSM. That's why I don't use it. ;)

Edit: just fount this, but there was a more academic explanation and research post somewhere:

But on other clips I did not get these problems and now its on all clips...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on June 30, 2019, 07:07:58 PM
Moiree depends also on the lens used, the aperture used,... I also get 3x3 clips without moiree, and others with a lot.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 30, 2019, 07:42:37 PM
Quote from: masc on June 30, 2019, 07:07:58 PM
Moiree depends also on the lens used, the aperture used,... I also get 3x3 clips without moiree, and others with a lot.

Yea but are you sure this is moire ?? With so big green and pink pixels ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on June 30, 2019, 08:13:37 PM
Exactly these colors is moiree, yes. They are produced at contrast edges in all line skipping modes. With some luck, you can exchange them in MLVApp against a little noise, choosing IGV debayer (sometimes it gets a little better this way).

Edit: another comparison of the modes: https://www.magiclantern.fm/forum/index.php?topic=16516.msg210023#msg210023
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on June 30, 2019, 09:06:38 PM
https://youtu.be/p2jvRNGMUzc
Got those vertical lines in 4k, but did everything that @Reddeercity said for eliminate it before shooting. I think it's my camera issue.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 30, 2019, 09:19:07 PM
Quote from: Igor_Braun on June 30, 2019, 09:06:38 PM
https://youtu.be/p2jvRNGMUzc
Got those vertical lines in 4k, but did everything that @Reddeercity said for eliminate it before shooting. I think it's my camera issue.

I had those lines too, but I found out that turning off then back on raw video recording (after 4k crop rec is on)
2 times is not enough, I have to do it 3/4 times until its ok, and Don't forget te refresh liveview before you turn raw recording back on.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on June 30, 2019, 09:40:36 PM
The aliasing is sooo heavy... and I had no problem before with the same build... and its completely unsharp


(https://i.ibb.co/HtKrqKZ/ML-pb4.png) (https://ibb.co/CbwnQwL)

Here is another clip after I reinstalled ML (June 20th build) : https://drive.google.com/file/d/1NWaXzlWmFDpgGzRqZnWe3Xr5sYC4ilQm/view?usp=sharing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 01, 2019, 04:50:36 AM
Will , I see no aliasing in the clip I downloaded but it is at least 2-3 stops underexposed (try to use ETTR technique)
I always use Adobe After Effects CS6 to process my MLV files , so I hardly ever come across aliasing issues .
What I find that help with aliasing is to use "Edge Defringe" in A.E.
(https://i.ibb.co/wRYzcCD/A-E-defringe.png) (https://imgbb.com/)

(https://i.ibb.co/vdvf2vH/A-E-defringe-1.png) (https://imgbb.com/)

here is the XMP file from adobe after effects to adjust the dng's M30-2057_000000.xmp (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M30-2057_000000.xmp)
and here's the exported h264 mp4 file 4096x1784_h264-A.E._M30-2057.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4096x1784_h264-A.E._M30-2057.mp4)
Not sure why you are using 12bit , 10bit should be enough
FYI: I use Technicolor's CineStyle Profile for my Liveview  to judge my exposure with & I always get very clean shadows .
It's never fail me yet !
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on July 01, 2019, 08:34:19 AM
Hm... this reduces the colors, but all the moiree stairs are still there.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 01, 2019, 08:41:24 AM
No I don't see it
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 01, 2019, 08:56:29 AM
New test build
crop_rec-4k3x1_2784x1160_FHD48p_12.18am-5D2-eXperimental.2019Jul01.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-4k3x1_2784x1160_FHD48p_12.18am-5D2-eXperimental.2019Jul01.5D2212.zip)
Got 48p in 3x3 @ 1856x688 working (kind of) , should have been 1.67x vertical but adtg reg don't apply .
the whole adtg structure seem not to work with 5d2/d4 , I've look in to adtg_gui.mo and the adtg hooks are different then crop_rec
so I'll have to try and use the adtg_gui hooks the way they are code in there . 
Also 2784x1160 should be centered vertically & horizontal now even when zooming in & out with a (zoom len), should stay in the center now .

Short test 9 second with the crop_rec in FHD 48p 1856x688_48fps-M01-0032.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1856x688_48fps-M01-0032.mov)

To recap;
48p @ 1856x688 (3x3)
2784x1160 centered vertically & horizontally
3x1 4K @ 23.976p

Same as before , it may lockup , freeze , etc. ... you guy know the routine
Edit: 48p can be hard to enable , what I did was first put the cam in 1:1 (3x crop_mode)
enable 4k Anamorphic for a few second then switch to "FHD 48p" press the play but twice .
It will say in the left corner 1:1 but it's really 3x3 FHD , need to change that to say 3x3 .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on July 01, 2019, 10:11:00 AM
Quote from: reddeercity on July 01, 2019, 04:50:36 AM
Will , I see no aliasing in the clip I downloaded but it is at least 2-3 stops underexposed (try to use ETTR technique)
I always use Adobe After Effects CS6 to process my MLV files , so I hardly ever come across aliasing issues .
What I find that help with aliasing is to use "Edge Defringe" in A.E.
(https://i.ibb.co/wRYzcCD/A-E-defringe.png) (https://imgbb.com/)

(https://i.ibb.co/vdvf2vH/A-E-defringe-1.png) (https://imgbb.com/)

here is the XMP file from adobe after effects to adjust the dng's M30-2057_000000.xmp (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M30-2057_000000.xmp)
and here's the exported h264 mp4 file 4096x1784_h264-A.E._M30-2057.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4096x1784_h264-A.E._M30-2057.mp4)
Not sure why you are using 12bit , 10bit should be enough
FYI: I use Technicolor's CineStyle Profile for my Liveview  to judge my exposure with & I always get very clean shadows .
It's never fail me yet !

Well then my MLV app has a problem because the image is awful, completely unsharp and werd colors everywhere... I will try with the 1.6 version... and also in AE.
I will also send you another clip so you can tell me what you think.
btw, the clip is underexposed but i exposed for the highlights (maybe its not the right way to do things ^^)



Here is a png exported from MLV app without any correction (converted to jpeg because it was too big), its unusable...
(https://i.ibb.co/0JbkR9N/M30-2056-000021.png) (https://ibb.co/88ZJLx3)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on July 01, 2019, 03:25:57 PM
Quote from: Galterius on July 01, 2019, 10:11:00 AM
Well then my MLV app has a problem because the image is awful, completely unsharp and werd colors everywhere... I will try with the 1.6 version... and also in AE.
I will also send you another clip so you can tell me what you think.
btw, the clip is underexposed but i exposed for the highlights (maybe its not the right way to do things ^^)

Here is a png exported from MLV app without any correction (converted to jpeg because it was too big), its unusable...
MLVApp has no problem with that. The problem is still the 3x1 readout with line skipping. Current official version is v1.7. The way how you expose (+/-) doesn't matter, as long it doesn't clip. The used aperture is more important (open aperture brings less moiree). The picture you posted shows the same like all 3x1 test pictures. Have you tried another debayer algorithm? The jpg/png/dng is not important. A shortend MLV would be more important (1 or 2 frames is enough).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on July 01, 2019, 03:35:27 PM
Here is a shortened MLV (I chose Fast Pass in export settings)

https://drive.google.com/file/d/1xKjikSsYZnvyqrkQI4BWmStF-5AVwOto/view?usp=sharing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on July 01, 2019, 03:37:34 PM
Quote from: Galterius on July 01, 2019, 03:35:27 PM
Here is a shortened MLV (I chose Fast Pass in export settings)

https://drive.google.com/file/d/1xKjikSsYZnvyqrkQI4BWmStF-5AVwOto/view?usp=sharing
117MB? Shortend? Did you set the CutIn / CutOut?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on July 01, 2019, 04:50:10 PM
Here it is sorry (20mb)
https://drive.google.com/file/d/14tazB5SekYdMNuBOmREfc3DqNrZDIW8a/view?usp=sharing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on July 01, 2019, 07:12:53 PM
Quote from: Galterius on July 01, 2019, 10:11:00 AM
Here is a png exported from MLV app without any correction (converted to jpeg because it was too big), its unusable...
(https://i.ibb.co/0JbkR9N/M30-2056-000021.png) (https://ibb.co/88ZJLx3)
You should ETTR (expose to the right) with raw , otherwise you will get what you got.You have histogram which could tell you how much you can bump exposure up.
Recording with RAW is different than recording with h264, what you see on the screen is not what you will get, in liveview you have picture style applied and much more contrast than actual RAW frame.You should almost overexpose with raw.
I filmed with 5d2 today and dont have what you got, use magic zoom or peaking to get sharp focus
Id say all problems with this picture is your mistakes and filming like regular h264.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on July 01, 2019, 07:48:40 PM
Quote from: 2blackbar on July 01, 2019, 07:12:53 PM
You should ETTR (expose to the right) with raw , otherwise you will get what you got.You have histogram which could tell you how much you can bump exposure up.
Recording with RAW is different than recording with h264, what you see on the screen is not what you will get, in liveview you have picture style applied and much more contrast than actual RAW frame.You should almost overexpose with raw.
I filmed with 5d2 today and dont have what you got, use magic zoom or peaking to get sharp focus
Id say all problems with this picture is your mistakes and filming like regular h264.

Ok I will try that, it should fix that aliasing ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on July 01, 2019, 08:10:57 PM
Quote from: Galterius on July 01, 2019, 04:50:10 PM
Here it is sorry (20mb)
https://drive.google.com/file/d/14tazB5SekYdMNuBOmREfc3DqNrZDIW8a/view?usp=sharing
This is better, thanks. See some statement below...
Quote from: 2blackbar on July 01, 2019, 07:12:53 PM
You should almost overexpose with raw.
I would not recommend this, because if one channel clips just a little, it will look really bad. I always record at ~EV-2/3 for not getting crushed clips.
Quote from: 2blackbar on July 01, 2019, 07:12:53 PM
I filmed with 5d2 today and dont have what you got, use magic zoom or peaking to get sharp focus
Did you try 3x1 4K mode? If not I agree you won't get this. ;) And you just will get those artifacts if the focus was correct. If you watch the MLV, it is very sharp - for a 3x1 file. Use a sharpener and you'll see the focus is okay. If you use the sharpener combined with "Chroma Separation" checkbox in MLVApp, you desaturate the color artifacts. But again... the moiree is still there. Moiree / aliasing will always be there when recording with line skipping. No matter what program used for processing and no matter what the exact settings were when recording.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on July 01, 2019, 08:20:40 PM
So I have to expose looking at the histogram, just before it says OVER ? (so it says E0.3, or E0.4...)

the weirdest thing is that I had no issue before, check this : https://www.youtube.com/watch?v=BiQI63nJJ5g
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on July 01, 2019, 08:26:28 PM
Quote from: Galterius on July 01, 2019, 08:20:40 PM
So I have to expose looking at the histogram, just before it says OVER ? (so it says E0.3, or E0.4...)

the weirdest thing is that I had no issue before, check this :...
Here, no hard contrast edge is in focus... no chance for moiree.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on July 01, 2019, 08:31:33 PM
Quote from: masc on July 01, 2019, 08:26:28 PM
Here, no hard contrast edge is in focus... no chance for moiree.

ok, thanks. I definitely need to learn a lot but I am so excited about that. I will try again and again
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on July 01, 2019, 09:24:45 PM
E0.3, is too high already, try to keep it with at least 1.0 or something but you need to test this yourself and get a hang of it so you wont have to rely on histogram that much, in some scenarios(under tree for example) you have to choose if you want to keep bright areas not clipped or avoid noise in shadows, i go for second option because overexposing in raw does not look as bad for my taste but underexposing is a killer for some shots , you cant save them to look decent and have to do denoising which will steal the details.
Second clip is underexposed too.
Maybe youre like me and you like when its dark and moody but in raw you should not try to get that in camera , expose to get most dynamic range from the image and pick whats most important part of the image, sometimes some guy in far background with bright sunglasses will overexpose your histogram( it will say overexposed even if your close subject is underexposed in dark shadow) , hes not important, let him clip and preserve dynamic range of your important subject.
By writing "You should amost overexpose" i meant , i go for overexposed and if iso200 is overexposed then i go to iso 100 if its not overexposed and its about one stop till overexposed, if i expose for liveview to make it look nice on the screen then image is way too dark later in MLVApp, thats why i dont judge by looking at the liveview, mostly histogram and past experiences help me to judge what ISO should be used but its best to use ISO100 because this one has most dynamic range, higher ISO's will lose a bit from dynamic range.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 02, 2019, 01:51:41 AM
Awesome, awesome...
Tested 48FPS

48 fps original:
https://youtu.be/FtUpYJEYAkM

48 fps 2 time slower (23,98):
https://youtu.be/IwVu60D5N8A

Also, i was excited about less rolling shutter but noticed that it's pretty the same as in regular 1856 23.98 fps
Want to test 1x3 4k so much. And compare it with 3x1.

@Reddeercity Why do you use 1/130 instead of something like 1/100. I see a kind of strobe/rolling effect on the video. I think you may avoid it with 1/100
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on July 02, 2019, 10:09:57 AM
Quote from: 2blackbar on July 01, 2019, 09:24:45 PM
E0.3, is too high already, try to keep it with at least 1.0 or something but you need to test this yourself and get a hang of it so you wont have to rely on histogram that much, in some scenarios(under tree for example) you have to choose if you want to keep bright areas not clipped or avoid noise in shadows, i go for second option because overexposing in raw does not look as bad for my taste but underexposing is a killer for some shots , you cant save them to look decent and have to do denoising which will steal the details.
Second clip is underexposed too.
Maybe youre like me and you like when its dark and moody but in raw you should not try to get that in camera , expose to get most dynamic range from the image and pick whats most important part of the image, sometimes some guy in far background with bright sunglasses will overexpose your histogram( it will say overexposed even if your close subject is underexposed in dark shadow) , hes not important, let him clip and preserve dynamic range of your important subject.
By writing "You should amost overexpose" i meant , i go for overexposed and if iso200 is overexposed then i go to iso 100 if its not overexposed and its about one stop till overexposed, if i expose for liveview to make it look nice on the screen then image is way too dark later in MLVApp, thats why i dont judge by looking at the liveview, mostly histogram and past experiences help me to judge what ISO should be used but its best to use ISO100 because this one has most dynamic range, higher ISO's will lose a bit from dynamic range.

Ok thanks for the information, I will try different exposures to see how it works.
By the way what do you turned on in your display settings ? Are vectorscope, or other diagrams necessary ? I don't know which ones I should use to help me expose correctly.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on July 02, 2019, 01:42:27 PM
I used to use histogram and waveform with h264, with raw i use histogram and expose at one stop before oveexposure  because i didnt tested if waveform reads raw image(which would be good) or what is on liveview(which would be bad),taking into account most important subject in shot and trying to not over-under expose it , even if i want moody darker image, i prefere to darken in post because with raw i see more in shadows than h264 which means i see more noise in shadows, underexposed raw footage on my 5D2 and M looks quite bad.
Histogram informs you about brightest part in the frame so if you are underexposed by 2 stops and someone in background holds in hand tiny piece of mirror that reflects the sun then you will get overexposed info which is not correct, i mean technically it is but youre still underexposed and shouldnt look at the tiny overexposed parts of the image that dont really matter, sometimes i just pan/tilt camera until that small thing disappears from frame so it wont overexpose.
Ffor example if small cloud in the sky is overexposing, i tilt down so sky is not in the frame anymore and i adjust proper exposure, then i tilt up so sky is in the frame, if its overexposed then well.. too bad.But with sky its not a great example because i tend to care about it not being overexposed.
In some shots its kinda important subject.
Last time i remember being under tree , i had to adjust for under tree lighting/brightness, not brightness thats far away which was overexposed, if i go midway and underexpose under tree while having not overexposed far background then i still have crap image because im underexposed under tree where my main subject is.
I hope its not too confusing .Just expose for your subject, not for background, its great if you have both properly exposed but if you cant then go for properly exposing subject.There are exceptions from all this and its up to you to decide, i just found out that underexposing with raw is not a good idea if you dont like noise.I can brighten in post about half-one stop before noise is unacceptable while i can darken the image SOOOOO much its insane with RAW.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 02, 2019, 02:02:58 PM
Ok guys this is really off topic here , can you please talk about this on the many other raw video processing threads this is for development of crop_rec for D4 cameras . Clearly the problems are user errors/cam setup . Please learn your camera setting and what there are for , search the forum all your questions have already been answered many times .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 02, 2019, 02:11:47 PM
Today i was testing FHD in the streets but after several recorded clips all the modules disappeared.
(http://i67.tinypic.com/1zxbaq9.jpg)

So i got to my place, installed 48 fps build to my other card, went to the street again and got this:
https://youtu.be/oTB7ibj028U
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on July 02, 2019, 02:21:59 PM
Is 12fps the limit of anamorphic mode or is it just me? Cant get it to work with 23.976 override.If i dont use override it locks at 12fps
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on July 02, 2019, 06:12:46 PM
@Igor_Braun
Use default setting on ML. I see you use Magic zoom on both cards - > don't work with FHD48.
Format card complete from camera, re- install and see what happens. Or maybe battery out for 10 sec.

@2blackbar
12fps is not correct. Don't use fps override in crop mode.
Maybe same problem. Format re-install, try default setting

@Reddeercity
For me all new builds work great. Without failer.
Thanks for your work!

But:
- 3.5K is centered better, but not perfectly, -> little bit up more we need.
- I think Temperature is a little bit to fast to high.
- The vertical lines disappear with me after approx. 10 seconds
- can you implementat greyscale automatically in 4K its a little bit user-friendly

What build comes next?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 03, 2019, 05:52:29 AM
Quote from: Bender@arsch on July 02, 2019, 06:12:46 PM
- 3.5K is centered better, but not perfectly, -> little bit up more we need.
- I think Temperature is a little bit to fast to high.
- The vertical lines disappear with me after approx. 10 seconds
- can you implementat greyscale automatically in 4K its a little bit user-friendly
Thanks for the feed back ,
Ok , I'll look at 3.5k centered again , temperature wise as long as your under 68°c but 48p will heat up a bit I was seeing around 58°c but normally it's around 48-53 in 4k Anamoph.
About those lines , I had too disable a pause between cmos's 1 & 2 when being applied because the  statement (! is_5D2) wasn't always working but I'm working on a solution .
Until then just wait a few second before you recording . The greyscale preview should be automatic unless you are in auto mode preview , choose ml b/w/ greyscale for preview
but I'll look in to it at some point too .
Quote from: Bender@arsch on July 02, 2019, 06:12:46 PM
What build comes next?
More likely this --> 4k (4096x1770) half framerate of 24p (12fps) in 1:1 (should be close at least 10fps I think)
Then this -- being back 3K (3008x1080 maybe with a little more res)
and this 1920x1080 @ 45p in 1:1(3x crop_mode) 
The last one will be 1x3 4k , as this need my full attention (adtg hooks need to fixed for d4 cams)
Then I can move on to Lossless .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on July 03, 2019, 10:21:04 AM
4k (4096x1770) 12 bit lossless compression may be enough for the card speed limit. Fingers crossed

Sent from my Redmi 4A using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on July 03, 2019, 10:59:31 AM
4096 x 1770 x 12 x 24 /8 = 249 MByte/s.
Pipe dream about required compression rate IMO.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: OlRivrRat on July 03, 2019, 05:44:20 PM
      @Walter

   I'm not @all knowledgeable about calculations

so would like to know >

4096 x 1770 x 12 x 24 /8 = 260997120

How does 260997120 = 249MB/s
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on July 03, 2019, 06:14:15 PM
Because ML uses
1 MB = 1024 kB = 1024 x 1024 B
(virtually nobody uses the correct term KiB to tell binary based k (= 210 = 1024) from decimal k (= 103 = 1000) and its the same for MiB and GiB.
And that's why an empty 32 GB card is showing 29.8 GB (=GiB) available in ML's screen. Storage manufacturers are using decimal G = 1000 x 1000 x 1000 for some time (was different when I started in IT).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: OlRivrRat on July 03, 2019, 08:58:58 PM
Ahhaaaa > Just wasn't registering the 260997120 to be Bytes ~

& why did You write X12x24/8 instead of x36 ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on July 03, 2019, 09:09:26 PM
Horizontal resolution x vertical resolution x bit depth x frame rate  / 8 = uncompressed data rate in Bytes/s
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 03, 2019, 10:45:00 PM
1/48 preset (i used a very cheap lens, so it's not very sharp)

https://youtu.be/Zb0I6CIcRzM

Look, once again I would like to tell about 1/130 shutter and rolling strobe effect and also, look at the cat, it seems that on the video everything is a bit faster than in real life.
1st footage - x2 slower
2nd footage - original speed
and then the same footage, but x2 slower

And these are 4k footages.
By default, the shutter speed in 4k preset is 1/62 now, and in my case it unchangeable. Strobe effect is terrible (look at the last footage)
https://youtu.be/e83C51ZCw8A

Quote from: reddeercity on July 03, 2019, 05:52:29 AM
More likely this --> 4k (4096x1770) half framerate of 24p (12fps) in 1:1 (should be close at least 10fps I think)
Then this -- being back 3K (3008x1080 maybe with a little more res)
and this 1920x1080 @ 45p in 1:1(3x crop_mode) 
The last one will be 1x3 4k , as this need my full attention (adtg hooks need to fixed for d4 cams)
Then I can move on to Lossless .

Can't wait for the last one with canon preview...

And maybe it will be better if you decrease the horizontal res of 48fps to 1610 to achieve the same 2.34 aspect ratio? Maybe it will give you even more fps..
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on July 04, 2019, 07:44:36 PM
Quote from: Walter Schulz on July 03, 2019, 10:59:31 AM
4096 x 1770 x 12 x 24 /8 = 249 MByte/s.
Pipe dream about required compression rate IMO.
Compressed dng is 3x compression. So you are looking at 81MB

Sent from my Lenovo TB-8703F using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on July 04, 2019, 08:08:56 PM
2784x1160 definitely looks really cool, I made a short film using it, and I love it (its not perfect, I had sometimes bad exposure and out of focus clips but its okay, I am just starting)

https://www.youtube.com/watch?v=-5Fv7VY6rrU
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on July 05, 2019, 12:17:37 AM
Quote from: zcream on July 04, 2019, 07:44:36 PM
Compressed dng is 3x compression. So you are looking at 81MB

As a1ex wrote: Depends on scene. See
Quote from: a1ex on April 01, 2017, 11:15:41 AM
*) Continuous recording for the above resolutions can be achieved as long as you can get a LJ92 (http://www.magiclantern.fm/forum/index.php?topic=18443.msg182074#msg182074) compression ratio (compressed / 14-bit uncompressed) of about 50-55%, with preview set to Frozen LV (previously known as Hacked Preview) for an additional speed boost. Otherwise, you'll have to reduce the resolution or the frame rate.

The following table shows how compression rate changes with ISO and bit depth; please check the figures for your particular scene in the raw video submenu, as they can vary a lot, depending on the scene content.
Quote from: a1ex on April 15, 2017, 01:12:36 PM
Bits per pixel      14  12  11  10   9   8
ISO  100 1/100     61% 53% 50% 48% 46% 43%
ISO  200 1/200     62% 54% 51% 49% 47% 44%
ISO  400 1/400     63% 54% 51% 49% 47% 45%
ISO  800 1/800     65% 55% 52% 50% 48% 46%
ISO 1600 1/1600    67% 56% 53% 50% 48% 46%
ISO 3200 1/3200    70% 57% 53% 50% 49% 47%
ISO 6400 1/6250    76% 60% 55% 52% 50% 48%
ISO 12800 1/12500  79% 63% 57% 53% 50% 49%


I suppose you can get 33% compression rate shooting white sheets at ISO 100...
Prove me wrong and show some footage with that ratio.

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 05, 2019, 05:02:46 AM
@Reddeercity
I also wanted to ask you, what do you think about 3840 × 2160 (16:9) 8 fps 1x1 preset? It must be very very useful for real estate shootings with tripods, sliders, or gimbals.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 06, 2019, 05:39:05 AM
Updated the first post/page with new test build  (https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476)
what new; 48p in FHD (1856x688) should be able to increase vertically a little more to 800
Still have 4k 3x1 & 2784x1160 . Working on adding 4k (4096x1700) 1:1 half frame rate (12fps)
Got it to work with adtg_gui.mo , write speed @ 10bit is 83MB/s very close to being continuous .
If you double the playback speed 12->24 (reduces overall time length to half 10 sec.-> 5 sec.)
You get a smooth play back (like a hyper timelapse but better) , just need to sit down and finish coding it .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on July 06, 2019, 07:06:23 PM
Quote from: Walter Schulz on July 05, 2019, 12:17:37 AM
As a1ex wrote: Depends on scene. See
I suppose you can get 33% compression rate shooting white sheets at ISO 100...
Prove me wrong and show some footage with that ratio.
I was thinking cineform raw. But that's not implemented. 10bit could work then.

Sent from my Lenovo TB-8703F using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on July 06, 2019, 07:07:27 PM
Quote from: zcream on July 06, 2019, 07:06:23 PM
I was thinking cineform raw. But that's not implemented. 10bit could work then.

Sent from my Lenovo TB-8703F using Tapatalk
Ignore the post. Won't work. My bad.

Sent from my Lenovo TB-8703F using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 07, 2019, 01:51:59 AM
FYI about 48p & shutter speed , to get correct shutter speed for 48p you should be double
your frames rate e.g. 24fps = 1/48th , 48fps = 1/96th of second ideally so the best I can get is 1/91th of second
to get that I set shutter speed in 3x3 without crop_rec enabled to 1/41th or 1/40th of a second then enable FHD 48p
should be 1/91th of a second , this give very good motion blur for slow-mo shots down to 12fps
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 07, 2019, 07:11:45 PM
@Reddeercity, I've tried everything i know to activate the 48fps mode but couldn't get into it. Will try again tomorrow. And could you tell how to change shutter speed from 1/62 to 1/50 (or 1/48) in 4k mode, and if it's possible to change iso in 4k?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on July 07, 2019, 08:16:01 PM
@Igor_Braun

My way to activate FHD 48p is (preview Options: Auto):
1. enable live view
2. set your Settings (manuel mode: shutter, aperture and iso)
3. press play 2 times (wait 2-3 sec.)
4. press "+" (wait 2-3 sec.)
5. repeat step 3. and 4.
6. enter crop rec 48p in ML (press half shutter to exit ML) -> and you will see a half frozen live view (top is liveview and bottom frozen) -> this is correct
(7). sometimes it doesn't work and you must disable 48p first (press half shutter to exit ML), and repeat step 4. - 5. or go back to step 2.

If you want change shutter, aperture or iso -> disable crop rec first and go back to step 2.
-> if you forgot disable crop rec first (blackscreen!!) -> disable live view, disable crop rec, enable live view and go back to step 2.

Set correct shutterspeed is important!  1/33 --> 1/58,       1/41 --> 1/86,       1/45 --> 1/100(ML menu)       1/50 --> 1/130,       1/64 --> 1/320,       1/82 --> error
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 07, 2019, 08:32:53 PM
Quote from: Bender@arsch on July 07, 2019, 08:16:01 PM
@Igor_Braun

My way to activate 48p is (preview Options: Auto):
1. enable live view
2. set your Settings (manuel mode: shutter, aperture and iso)
3. press play 2 times (wait 2-3 sec.)
4. press "+" (wait 2-3 sec.)
5. repeat step 3. and 4.
6. enter crop rec 48p in ML (press half shutter to exit ML) -> and you will see a half frozen live view (top is liveview and bottom frozen) -> this is correct
(7). sometimes it doesn't work and you must disable 48p first (press half shutter to exit ML), and repeat step 4. - 5. or go back to step 2.

If you want change shutter, aperture or iso -> disable crop rec first and go back to step 2.
-> if you forgot disable crop rec first (blackscreen!!) -> disable live view, disable crop rec, enable live view and go back to step 2.

Set correct shutterspeed is important!  1/33 --> 1/58,       1/41 --> 1/86,       1/45 --> 1/100(ML menu)       1/50 --> 1/130,       1/64 --> 1/320,       1/82 --> error

Ah.. I saw a half frozen live view everytime. Is it correct? I'm going to try to shoot something

Edit:
It works
https://youtu.be/aSK1YYAwWuo
It's wonderful! I mean the real-time preview in 48fps!!

But in my case it works if i select 1/62 before enabling crop_rec, if i select 1/50 it freezes, but anyway camera captures normal life video
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 08, 2019, 02:58:25 AM
What the heck , no it's far easier then that , read the first post/page of the thread and I tell you how to very easily enable 48p
Quote48p can be hard to enable , what I did was first put the cam in 1:1 (3x crop_mode)
enable 4k Anamorphic for a few second then switch to "FHD 48p" press the play but twice .
It will say in the left corner 1:1 but it's really 3x3 FHD , need to change that to say 3x3
I'll post a video later.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 08, 2019, 08:23:00 AM
Here you go , how to enable 48p on 5D2 (Shot on my 50D) , to get close to correct shutter set to 1/41th in 3x3 then enable crop_rec ,
note on liveview it says 1/86th in 48p FHD when you check the dng it's 1/91th of second , so I'm not sure why there's a different between them but I believe the dng is correct here .

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on July 08, 2019, 10:21:25 AM
@Reddeercity
With my method you have a live-colorfully real-time preview;)

See my last Post.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 08, 2019, 02:43:21 PM
@Reddeercity, please could you port this halfscreen liveview to the 4k preset? It's so cool
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 09, 2019, 02:43:47 AM
Will that's not how it works guy's , basically  the preview buffer is not clearing it self from 3x3 non crop_rec .
In Crop_Rec the Canon Liveview should be Scrambled @ 10->14bit . This is a result of some of my hacks
in crop_rec where I had to remove a few checks & pauses to get the preset to work , When these hacks are reversed
the preview buffer should be back normal .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 09, 2019, 02:58:36 AM
Yesterday I shot same 48p in FHD (1856x688) and then slowed it down to 23.976 & 12 fps  Looks not bad even 12fps looks good , Workflow; extracted dng to the native 10bit -> Adobe After Effect CS6 cam setting; 48fps preset , ISO 400 , EF 24-70mm f2.8L @ f8 set to 61mm , shutter 1/91th of second . Notice that the cam temperature never when over 53° C. when in 48p preset .

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 09, 2019, 11:21:03 PM
Cool shot!
So, you tell that half screen canon preview is not correct? Or it was missunderstood by me. Anyway it works good in 3x3. So it could be used in 4k.
Wanted to ask, does it influent to crop_rec or raw module which kind of battery or card do you use? I use a battery grip and notice (or it just seems to me) that your build work different ways depending on which battery slot do I use, or use I an external power or not.
And also by the way I killed another Lexar 32 1066x :) I don't know why. I changed the battery and the camera refused to turn on. No blinks, screen doesn't work.. i've changed the battery several times and got the massage: card not formatted. So I unable to format it from any devices. Just test report... :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 10, 2019, 07:42:11 AM
Quote from: Igor_Braun on July 09, 2019, 11:21:03 PM
So, you tell that half screen canon preview is not correct?
That's right , it's a bug right now , It's down on my list of thing to do .
Quote from: Igor_Braun on July 09, 2019, 11:21:03 PM
Wanted to ask, does it influent to crop_rec or raw module which kind of battery or card do you use? I use a battery grip and notice (or it just seems to me) that your build work different ways depending on which battery slot do I use, or use I an external power or not.
I Think there maybe a issue with your battery grip (dirty contact etc. ...) you seem to be the only one that's having issue with crop_rec .
Can you test without the battery grip ? Do you have a Canon Battery ? if so please test and see if you are still having these issue .
I also did a search on the forum for "battery grip" came back with over 100 results , and some did have issue with ML & battery grip .

Quote from: Igor_Braun on July 09, 2019, 11:21:03 PM
And also by the way I killed another Lexar 32 1066x :) I don't know why. I changed the battery and the camera refused to turn on. No blinks, screen doesn't work.. i've changed the battery several times and got the massage: card not formatted. So I unable to format it from any devices. Just test report... :)
They do burn out from time to time  , FYI I don't restore my card in cam (Format in camera & replace ML build) unless there's a good reason too
I just delete the files or "move" them from the CF card to a local drive , I feel the more time you format CF card you reduce the life of the NAND chips (where the data is stored on the card)
There is only so many cycles (read/write function's) before it stops working .
A good description here how-does-flash-drive-work (https://howflashdriveworks.wordpress.com/how-does-flash-drive-work/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: histor on July 10, 2019, 10:47:53 AM
Quote from: Igor_Braun on July 09, 2019, 11:21:03 PM
card not formatted. So I unable to format it from any devices.
Please, do make sure that you can't format it. Try HP USB Disk Storage Format Tool on Windows or some other special utility. There was a problem with file system corruption long time ago (reported and discussed here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg203743#msg203743) ). I'm still trying to narrow down the search. My prime suspect is writing MLV without space preallocation, possibly writing it from Silent.mo.
Looks like this (two different crashes). You can almost read the filenames...
(https://i.ibb.co/FmjGQvT/2019-07-06-17-00-51.jpg) (https://ibb.co/0mdP0Nb)(https://i.ibb.co/jJT02L7/2019-03-25-21-46-39.jpg) (https://ibb.co/0sc4SfP)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 10, 2019, 07:52:39 PM
Thanks for answering guys! Will try to follow your advises and text you back
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 14, 2019, 03:16:38 AM
@reddeercity, but half-frozen preview would be a good way to see what you are filming without any delay in 48fps.
Quote from: reddeercity on July 10, 2019, 07:42:11 AM
I Think there maybe a issue with your battery grip (dirty contact etc. ...)
Will try to clean the contacts, maybe it's an issue, but not sure, because anyway, the camera behaves badly. And yes, i have original canon batteries.
Quote from: reddeercity on July 10, 2019, 07:42:11 AM
They do burn out from time to time  , FYI I don't restore my card in cam (Format in camera & replace ML build) unless there's a good reason too
I will be more careful and follow your advice to don't format it without the reason to actually do it to save NAND's life :) Thanks for the link.

And good news for me. The Lexar card started to work again as if nothing had happened... What is it? I don't really know. The interesting thing is i low level formatted it with Hard Disk Low Level Format Tool, the format was done successfully (but anyway it still wouldn't open, the same process i done with the 1st lexar), but all the files are still there on the card... And i can format it now.

And i'm sorry for disturbing and waste your time. Don't sure i wrote in right threat.
Also need to say that bought the camera like it was a new one, but later noticed some bugs:
1. Magnify jumping out to 3x3 automatically after randomly count of seconds
2. The sensor was "covered" with a lot of artifacts and strange patterns. All of these disappeared after i cleaned it with a special swab.
3. The serial number on EXIF metadata was different from the serial on the camera body. And i ever read the name of the previous owner.

Of course, i bought it for the price like it was new. In the store, they told me that they didn't know about it, and also told that if i had any problem they repair the camera by themselves...

That time i needed the camera so much, because my 650d's sensor deteriorated after a trip, so i decided to leave the cam with me.
It was a kind of wrong way =)
Now i thing those bugs appear with the new force (all this i write because is started to appear with Exp. build and including because maybe it could be a kind of useful for the development):
Mirror doesn't go up and the camera doesn't switch on (with magic lanter uninstalled). After several (regularly 3-4 tryings) it starts to work. But step by step... look:
https://youtu.be/PsNHjv31loY
pressing trash button because of the habbit  :D
https://youtu.be/DdbDzuy2GI0

Also, sometimes i have color changing on video or more frequently on photos (especially first 20-60 seconds when i activate preview)

Look, this photoshoot i did three days ago. (i cant show the model here on this forum including because it's a nude photo)
one photo has this color:
(http://i64.tinypic.com/2ntd314.jpg)
And after several times it has "normal" colors:
(http://i65.tinypic.com/28rpid3.jpg)

But from time to time the color changes slightly thats why i wrote "normal" (i newer use AWB)

Also, there are vertical stripes sometimes:
(http://i63.tinypic.com/14w53dy.jpg)

Sometimes i got a mix of it's all.  :o :o

Another thing...
After this photoshoot, i notices a strange texture, zoomed it, and was slightly shocked.
Just look a that
(http://i64.tinypic.com/fmouom.jpg)
(http://i65.tinypic.com/2nqzk0k.jpg)
(http://i67.tinypic.com/dbkt9l.jpg)
(http://i63.tinypic.com/2ajpcag.jpg)
I have no idea what it is.
That was a non-commercial shooting  ;D

This all are so strange and started to appear after i installed experimental build the 1st time, but i think it's only coincidence, or maybe this story will be useful for development of this experimental build

For my work, i decided to rent another cam for now :-\, and don't know what to do with imy one. Maybe i need to bring it to any repair service, but I am not sure they understand all of this.
or buy a "new" used 5dii when have enough for it.

But while i'm thinking i'd like to share some files CRASH00.log and log000.log (not on lexar card, but after it stopped to work iinserted another one (sandisk extreme) and saw some crash log files). I don't know if it can be usefulm but anyway...
https://drive.google.com/file/d/1N5gaSjCsRfeDB9eJgfI9LEc4Ci0TKKB6/view?usp=sharing
https://drive.google.com/file/d/1K1K6mvRW6O2mhQOTDGgZXkrb1b0Cxpe_/view?usp=sharing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 16, 2019, 08:20:50 AM
@Igor_Braun , I don't have time right now to investigate your camera's issue -- but it looks like a setting & corrupted images , I've seen this before (I think it was a card write issue)
I'll need to dig thought some of my past research to find info for you .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 16, 2019, 08:41:10 AM
New test build , Increased vertical in FHD 48p preset from 688 to 774 , (1856x774  2:40 A.R. @ 48.028fps) write speed @ 10bit is 82MB/s so close to continuous  :)
crop_rec-4k3x1_2784x1160-1x1_FHD48p3x3_11.59pm-5D2-eXperimental.2019Jul15.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-4k3x1_2784x1160-1x1_FHD48p3x3_11.59pm-5D2-eXperimental.2019Jul15.5D2212.zip)
I did reduces the rolling shutter in 48p from 29.7ms to 17.8ms  :D  Timer A has a lot of overhead in FHD I reduce the valve from 23B (default 1880) to 229 .
Still have 4k 3x1 & 2784x1160 1x1 , same as always check it out & let me know if 48p is centered correctly ( FYI , this the max height for FHD 48fps)

Edit: updated the First post/page with new test build
https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on July 16, 2019, 12:05:54 PM
Congratulations on your beautiful work!
Are you still getting a 5d3?  We need your breakthroughs too!
:)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on July 16, 2019, 12:27:49 PM
Quote from: 70MM13 on July 16, 2019, 12:05:54 PM
Congratulations on your beautiful work!
Are you still getting a 5d3?  We need your breakthroughs too!
:)
What do you miss on 5D3? It's all there already.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 16, 2019, 06:19:21 PM
Quote from: reddeercity on July 16, 2019, 08:20:50 AM
@Igor_Braun , I don't have time right now to investigate your camera's issue -- but it looks like a setting & corrupted images , I've seen this before (I think it was a card write issue)
I'll need to dig thought some of my past research to find info for you .

David, that was for info purposes, please don't waste time for helping me:) heh, do what your doing, and i'd be happy if that what I wrote in any way can help in further development
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on July 16, 2019, 06:30:25 PM
Dont know what doing wrong, with latest build (16 july)

Can enable only 2.7k

can not enable 4k anamorphic or 48p......i watched tutorial....but nothing...

thanks for help
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on July 16, 2019, 08:44:00 PM
I get the feeling that RDC would push the limits more than is currently possible.  He's clearly enthusiastic about what he is doing, and it certainly wouldn't hurt to have him working on that camera!
As for what is missing for me, it is mentioned in my prior posts.  I won't repeat what those things are, as it always results in some form of backlash.
If the 5d3 was working at its full potential I think there would be longer recording times.  Bright scenes stop recording just when the action is right.  Not very nice.
I like RDC and think it would be great for him to be working on the 5d3!
It's just my opinion...  please be kind.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on July 16, 2019, 08:53:41 PM
I was only curious to hear what was missing.
I keep my opinion to myself about the other stuff mentioned.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 17, 2019, 07:29:31 AM
Quote from: banertop on July 16, 2019, 06:30:25 PM
Dont know what doing wrong, with latest build (16 july)
Can enable only 2.7k
can not enable 4k anamorphic or 48p......i watched tutorial....but nothing...
thanks for help
You have to start in 3x crop (5x zoom) first , so liveview should say "2144x1074" default crop_mode
then enable 4k 3x1 preset next , refresh liveview again , stay there for a few second (record a few second it you like)
then go directly to FHD48p preset with out refreshing liveview , crop menu will say 48p & frame size of 4064x590 .
next press half shutter then the zoom button once , should now be in FHD 48p (1856x774)
Here a quick flow chart showing the process .

(https://i.ibb.co/fFf7hMg/48p-5d2-setup-full-size.png) (https://ibb.co/sRnr0jf)
 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 17, 2019, 07:48:00 AM
Quote from: 70MM13 on July 16, 2019, 12:05:54 PM
Congratulations on your beautiful work!
Are you still getting a 5d3?  We need your breakthroughs too!
:)
Thanks ,
I'm still looking for a good deal on a 5D3 , everyone wants too much $$$ for the cam's here in western Canada
I can get a EOS RP for the same price as a used 5D3 ($1500-1600) (The EOS R & RP have the greatest potential for ML with a 176MB/s SD card interface)
But at some point I do plan on getting one  , just not sure when it may be after I get a EOS RP but there again I may get the 5D3 first  :D

Yes there still a lot of stuff to discover on both D4/5 cameras and I would love nothing better then to fine tune the 5D3 .
After reading thought a1ex's code for 5D3 I see sooooo much more that this camera can do , we are just scratching the surface .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on July 17, 2019, 11:38:16 AM
Now you're just teasing!

Get the 5d3 first!  We need your help!

;)

What about collaborating with other ML people who have a 5d3?  Maybe there's one in red deer city?!

Or over the internet, which of course would be very slow and not fun...  or just get a 5d3 :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Dmytro_ua on July 17, 2019, 04:25:01 PM
Quote from: reddeercity on July 17, 2019, 07:48:00 AM
I can get a EOS RP for the same price as a used 5D3

I'm still on 5dIII, but you're right that it's much better to get a newer system R or RP as it has more potential and power! If there is a ML for EOS R, I would switch to that system without hesitation.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on July 17, 2019, 06:13:42 PM
@reddeercity

Thank you....

After yours latest instruction post, i was able to record 48p and 4k anamorphic  :)

Is it possible to see what you record in those two modes? Or the live view is frozen?...no real time, or any buggy preview?

I set greyscale preview, but after pressing record button it is frozen

tnx

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on July 17, 2019, 06:26:54 PM
and, one more question, please...

How to unstretch in post 4k squeezed footage?

thanks
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on July 17, 2019, 07:56:02 PM
Quote from: banertop on July 17, 2019, 06:26:54 PM
How to unstretch in post 4k squeezed footage?
Load into MLVApp, streching is done automatically. Or use Resolve and stretch 300% manually vertically.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on July 18, 2019, 12:02:37 AM
@masc

thank you :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 18, 2019, 04:59:20 AM
Quote from: banertop on July 17, 2019, 06:26:54 PM
and, one more question, please...

How to unstretch in post 4k squeezed footage?

thanks
Just run them thought MLV App , set the vertical stretch to 300% and export as Cdng (which will be at the native bit depth)
and then you will have the 1x3 tag in the dng and any program like Adobe After Effect & Balckmagic Resolve will read them
correctly without having to manually stretching even windows file explorer display the frame correctly (at least in Windows7 pro)
Info from Exiftool
Default Scale                   : 1 3
Default Crop Origin             : 0 0
Default Crop Size               : 4064 590
   
Exported from mlv app with 300% vertical , and you can see the tag "Default Scale" 1 3 (1x3) so any app that read Cdng or DNG's  will scale it correctly to 4064x1770
this how I work in Adobe After Effect , plus it keeps the 10bit raw frames @  2.86MB , where a 10bit 4096x1706 frame = 8.33MB

Default Scale                   : 1 1
Default Crop Origin             : 0 0
Default Crop Size               : 4064 590

This one was export with mlv app. without setting the 300% vertical stretch and you can see the "Default Scale" say 1 1 (1x1)
and this one will not be displayed the correct frame scale size , so all app will see it as 4064x590.

So you need to set the scale with all mlv file from "mlv_rec.mo" with vertical squeeze files .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on July 18, 2019, 08:04:44 AM
Quote from: reddeercity on July 18, 2019, 04:59:20 AM
... and any program like Adobe After Effect & Balckmagic Resolve will read them
correctly without having to manually stretching ...
Unfortunately Resolve is not able to read the metadata (at least up to v15.x, did not try v16). You have to strech manually in Resolve. But no big deal.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: marcelolippi on July 18, 2019, 12:00:32 PM
Hi David, but it's possible see well in the liveview or only streched?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 19, 2019, 01:15:42 AM
(http://i68.tinypic.com/2yvw0v5.jpg)
Look, David, i marked the center with the red circle.
So, it's not centered in 4k.
PS: can't activate the 48p preset... but now i'm scary trying to get into it if it's not enabling because my cam is half-alive right now and i have a project to shoot, heh... i'll attempt a little later or wait till next build if you don't mind
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on July 19, 2019, 02:06:10 AM
@reddeercity Any news about the 50D build?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: banertop on July 20, 2019, 03:30:29 PM
Is preview during recording frozen in 48p and 4k anamorphic mode?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 21, 2019, 12:27:09 AM
In my case yes
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: marcelolippi on July 22, 2019, 01:00:27 AM
Quote from: reddeercity on July 17, 2019, 07:29:31 AM
You have to start in 3x crop (5x zoom) first , so liveview should say "2144x1074" default crop_mode
then enable 4k 3x1 preset next , refresh liveview again , stay there for a few second (record a few second it you like)
then go directly to FHD48p preset with out refreshing liveview , crop menu will say 48p & frame size of 4064x590 .
next press half shutter then the zoom button once , should now be in FHD 48p (1856x774)
Here a quick flow chart showing the process .

(https://i.ibb.co/fFf7hMg/48p-5d2-setup-full-size.png) (https://ibb.co/sRnr0jf)

.
I uploaded the July 15th build, the same thing happened with a build about a month ago: all the modules I enable don't download, they give "error".Ideas? Thanks
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 22, 2019, 02:18:29 AM
Try to use another CF
Edit: @reddeercity, tomorrow i have my final shooting with 5d2 for the long period and after that will try to get into all the features in your last build. A also found an interesting thing about Lexar 1066. You remember i told you that it became a brick without any possibility to format it with any program or camera? So, the one thing i needed to do with it is insert it to 5d2 with magic lantern bootflag activated, then plug it again to the PC et... voilà, it works! And all the files are in their places. So i think will it crash/restore again if doesn't have ML folder on it
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on July 25, 2019, 10:39:47 PM
There seems to be a rly weird bug in this branch, when I press the joystick, it starts recording, which is really annoying as I'm just trying to center the zoom window.

(yes, got another mark 2)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on July 26, 2019, 11:12:31 AM
https://drive.google.com/file/d/17DyCmp6m20Ge0DGGr1JJOeFfnhAYanLo/view?usp=sharing

:o
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: histor on July 26, 2019, 11:50:44 AM
Quote from: Ilia3101 on July 26, 2019, 11:12:31 AM
:o
Fantastic resolution! 3968x1800. Unfortunately the top right corner stays dead.
Frame difference, exaggerated
(https://i.ibb.co/XXpgsJW/2019-07-26-12-45-00.png) (https://ibb.co/0qZSMJn)
So, what was the trick? Was it fair? :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on July 26, 2019, 01:03:03 PM
Quote from: histor on July 26, 2019, 11:50:44 AM
Fantastic resolution! 3968x1800. Unfortunately the top right corner stays dead.
Frame difference, exaggerated
(https://i.ibb.co/XXpgsJW/2019-07-26-12-45-00.png) (https://ibb.co/0qZSMJn)
So, what was the trick? Was it fair? :)

Ok, I will admit, you can't use this shooting method for a shot with a lot of action, but a reveal is coming very soon. (also I genuinely have no idea why the corner is stuck)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on July 27, 2019, 12:14:34 AM
@reddeercity, would it be possible to have a square/tall aspect ratio? (in the same builds that also contain 3008x1080 mode)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: marcelolippi on August 05, 2019, 05:03:12 PM
help me please, my 5d no longer loads the modules, everyone gives error, I changed tabs, build without being able to solve the problem. Thanks to those who can help me

https://photos.app.goo.gl/r1ALhemFcBB4VUgx9
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: histor on August 05, 2019, 05:36:55 PM
You haven't removed the previous build before copying the latest one. There is no mlv_lite.mo in that branch. Remove ML folder and copy one from the downloaded archive. Does it work?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: theBilalFakhouri on August 05, 2019, 05:56:06 PM
Quote from: marcelolippi on August 05, 2019, 05:03:12 PM
help me please, my 5d no longer loads the modules, everyone gives error, I changed tabs, build without being able to solve the problem. Thanks to those who can help me

https://photos.app.goo.gl/r1ALhemFcBB4VUgx9

Disable mlv_lite, Load mlv_rec instead .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: marcelo.lippi on August 05, 2019, 07:05:00 PM
THANK YOU, removed lite, loaded rec, works ... GREAT. What did I miss? because before I could choose between lite and rec and now no more? I'll try again with the reddercity builds
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: KirbyLikes525 on August 05, 2019, 08:06:52 PM
Quote from: Ilia3101 on July 26, 2019, 11:12:31 AM
https://drive.google.com/file/d/17DyCmp6m20Ge0DGGr1JJOeFfnhAYanLo/view?usp=sharing

:o

@Ilia3101, I hope you don't mind. I'm finding a workflow for 3k and 4k MLV video so your share allowed me to play some more with post processing. The main variable for me is HEVC (h265) settings but found a couple of "got ya's" that you probably already know about. 1) Uncompressed cDNG export cuts off at 3016x1080, hence, the ProRes and 2) MLV App did not confirm overwrite ('Are you sure?' message) for Uncompressed cDNG export

(looped for continuity)


123MB for 22 frames, holy cow!

MLV App
Exposure: (-1.21)
Rec.709
Prores 4444

Premiere Pro
{Color Grading}
HEVC
VBR, 1 Pass
Target: 4Mbps
Max: 6Mbps
Quality: Highest (Slowest)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: marcelo.lippi on August 05, 2019, 08:13:21 PM
confirmed, works with the "5D Mark II version
Canon Firmware Version: 2.1.2
Latest Build: 2018-07-02 "raw rec, but not with versions of David ...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: thebailey on August 13, 2019, 07:54:22 AM
Thank you for your awesome work, I've been having a blast with 1920 x 1080 (ish) 10bit 24fps RAW

I was wondering, since Avisynth has such great deinterlacing modules, would it be possible to do a 1080i at up to 30p (60 frames) per second?

The only reference to interlacing I could find on this site was to do with doing HDR and interlacing the light and dark exposures.

This would mean that effectively you were capturing 1920 x 540 worth of bandwidth, but easily able to interpolate it to double frame rate full resolution.

Also, I understand that LJ92 lossless compression isn't yet unlocked in 5Dii, but I was wondering what the current status is or whether it is off the development map (if I've missed the ML thread, let me know).

Until I can afford a BMPCC this has been fantastic
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on August 18, 2019, 01:43:40 AM
Quote from: KirbyLikes525 on August 05, 2019, 08:06:52 PM
The main variable for me is HEVC (h265) settings but found a couple of "got ya's" that you probably already know about. 1) Uncompressed cDNG export cuts off at 3016x1080, hence, the ProRes

Hmm why does it cut off, what do you mean?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on August 21, 2019, 09:54:07 AM
I don't understand why I can't get the 48p mode to work. It worked once, and now it doesn, work anymore. Basically, when I turn on the 4k crop mode (before turning on the 48p mode), the screen is blinking, so maybe the problem comes from the 4k crop mode... Thats really weird, I will try to reinstall the build
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 22, 2019, 06:16:51 AM
Some low hanging fruit I thought I would pick  :P
5D2_dual_iso_video-updates_8-21-2019.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dual_iso_video-updates_8-21-2019.zip)

Works with my crop_rec builds (not too sure about the nightly builds didn't test them) at 3x3 FHD(1856x1248) & 5xZoom(2144x1074) (3x crop_mode) @ 10bit raw ,
I didn't check it with the Crop_Rec Preset , so if you do try it with Crop_Rec it may freeze you cam and may need I battery pull .
Best to start out at 100 ISO with the recovery ISO at 800 , so 100/800 ISO , I wouldn't recommend a recovery ISO higher then 1600 ISO (200/1600 ISO Max.)
Anything beyond  that is just noise -- remember 5D2 has only 5 native analog ISO's (100 , 200 , 400 , 800 , 1600) the rest is just digital push or pulled ISO (e.g. 160 ISO = 100 ISO digital pushed)
The Zip file is just the dual_iso.mo (module) not the full build , if you think you need the full build with dual_iso.mo let me know , but it should work on top of
my builds on the first page , in theory it should work with the 5D2 10-12bit  magic lantern experiments (https://builds.magiclantern.fm/experiments.html)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on August 22, 2019, 02:20:19 PM
Wow, personally i was waiting for it a lot...
In my case, it doesn't work with Nightly Builds, only with your builds.
Could you please tell me how to process it? I checked the Dual Iso thread and found that

- For RAW video files, use the following commands in a terminal:
mlv_dump clip.mlv --dng --no-fixcp --no-stripes
cr2hdr --same-levels *.DNG

So, how to start cr2hdr.exe with this code in windows?

I already have experience of dual ISO photography postprocessing with cr2hdr.exe, but dont know about the video.
When try to process it in MLV app or MLVP i get a lot of ugly noise in the shadows.
ISO 100/800

(https://i.ibb.co/Z1T9rHz/Screenshot-50.png) (https://ibb.co/qF1680Y)

The noise is noticeable, look... I'm i doing it the right way? Or maybe it's because of low light condition?

-
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on August 22, 2019, 02:56:19 PM
@Igor_Braun: I think you should play around with your exposure settings in camera. In MLVApp I see you set Exposure very high... this brings noise. The clips I got for testing were fine at Exposure=0.0 + Lighten=+50..+100. There was only very few noise visible. (Clip from EOS M, ISO100/1600, 10bit, 1600x2040->4800x2040)
(https://i.ibb.co/86Ypmp5/Bildschirmfoto-2019-08-22-um-14-55-06.png)
(pic by Danne)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on August 22, 2019, 03:01:32 PM
@masc
Understood... Also, i still think that my flat is too dark.
Got another handheld picture in the same light condition but near the window, and with another lens which is faster, but less sharp and foggy, because it's 50 years old Nikkor.
This is it:
https://youtu.be/QQXpnerAsco
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 22, 2019, 07:42:21 PM
Ok guys this is off topic , please discuss this on the MLV App thread . Thanks
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on August 23, 2019, 11:53:40 PM
@Reddeercity, ok, anyway, this last video is the good working dual iso module. The best effect of this module i think is less noise in the shadows in the video. I couldn't even dream of such a thing for my 5d2...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: KirbyLikes525 on August 26, 2019, 06:34:50 PM
Quote from: Ilia3101 on August 18, 2019, 01:43:40 AM
Hmm why does it cut off, what do you mean?

The frame capture of the comment below is 3016x1080, not the full 3968x1800 which I'm only pointing out because cDNG export gave me that exact same frame, 3016x1080, and NOT the full 3968x1800 which I only got via ProRes export without resizing to do so. So for post processing purposes, MLV App did not render the full resolution of the 4k mlv file as cDNG. The HEVC portion of my comment didn't have anything to do with the resolution.

Quote from: histor on July 26, 2019, 11:50:44 AM
Fantastic resolution! 3968x1800. Unfortunately the top right corner stays dead.
Frame difference, exaggerated
(https://i.ibb.co/XXpgsJW/2019-07-26-12-45-00.png) (https://ibb.co/0qZSMJn)
So, what was the trick? Was it fair? :)

Actual frame:
https://imgur.com/uTBvWxk (https://imgur.com/uTBvWxk)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 26, 2019, 07:41:13 PM
Off topic please take this to MLV App thread .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 27, 2019, 07:29:07 AM
Starting to update the source code on bitbucket , should I be doing this or should I be looking at GitHub for ML source code ?

Anyways I fork the source from waza57 and started a new repository magic-lantern_reddeercity_5d2-50d_4k-crop_rec (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/src/crop_rec_4k_5D2/)
So far I've removed waza57 "rom hack" for height & width and I added the parts for "raw_slurp" (needed for extended res. & 10-12bit )
No updated crop_rec presets commits .
commit e7bfe64 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/e7bfe646352f287a6ae0c01f533644d1d380feac) disables rom hack
commit 1a2efe8 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/1a2efe89e394d9b59766881d873083a116fd6fd5) enable EDMAC_RAW_PATCH for raw buffer 5d2 Crop_Rec
commit 9704e3b (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/9704e3b8c02734e7b99923c777bced725cde6e68) in stubs.S , Added Liveview Raw patches 5d2 4k crop_rec
commit c6cb9b4 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/c6cb9b4bb1cdb30377d328d4b5404341d76152ed) Raw backend: overriding raw stream (CONFIG_EDMAC_RAW_PATCH) 5d2 4kcrop_rec extented res & 10-12bit support added

After this , I'm not posting any more commit here ,  going forward everybody will have to check the magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/)

Also I haven't added the parts for 50D raw_slurp yet in the next days ..... plus I'm close on the crop_rec module for 50D , just a few errors to fix so it can compile
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: york824 on August 30, 2019, 12:12:23 PM
I installed the July 15th build and found one issue: the mlv_play module cannot be enabled. Once I enable it and reboot the camera, I got some errors and all modules were disabled. Had to disable the mlv_play to have other modules working. I tried uninstalling the Magic Lantern on the CF card, re-formatting it and installing it back, still not working. When the module disabled, all other modules seem working fine.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on August 30, 2019, 12:55:35 PM
Non-working mlv_play is a known issue. Use search field on upper right corner for more info.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on September 01, 2019, 04:21:50 PM
@Reddeercity
I've testet your new Dual iso.mo and postproduction only works in MLV App.

- Can't export cDNG after force to dual iso in MLV App -> large Artefact in cDNG (in MLV App preview is correct)
- Export as cDNG without force to dual iso in MLV App -> lightromm plug/ CR2hdr  says "Doesn't look like interlaced ISO" / no Dual iso
- Open with  MLVFS -> "Doesn't look like interlaced ISO"

Maybe false Metadata?
-----------
Any new news to 4096x1770 half framerate?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 01, 2019, 08:58:23 PM
Ok I'll check in to it , did you check MLVFS ? I use quick mount on Win7Pro looks ok also did you check MLVProducer ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 02, 2019, 02:07:59 AM
checking in dual iso metadata with ExifTool I get a warning from mlv producer with cdng's
Bad IFD1 directory
I can't see any dualiso metadata right now , either a problem with raw.c or dng.c  not writing correct metadata for dual iso , need to investigate further .
Thanks for the report .

Edit:MLVFS Web GUI (win7), set to dual iso 20bit etc. ....
from cmd when I refreshed web gui with dual iso setting 
Doesn't look like interlaced ISO
ISO pattern     : dBBd RGGB
White levels    : 10000 5000
Noise levels    : 8.00 8.00 8.00 8.00 (14-bit)


So yea there a problem , I'll need to debug this -- in the mean time your will have too force "dual iso" in mlv app. & mlv producer .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 02, 2019, 02:59:18 AM
found some more problems , mlvfs web gui gave me the dual iso pattern and did not recognize the patterns
Doesn't look like interlaced ISO
ISO pattern     : ddBB RGGB
ISO pattern     : BBdd RGGB
ISO pattern     : dBBd RGGB
ISO pattern     : BddB RGGB
ISO pattern     : BBdd RGGB

these patterns alternate on each of the frames , need to look in the dual iso code and see what pattern are being used

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: york824 on September 02, 2019, 03:52:38 AM
So I shoot some footage using the 3K build and overall I am very impressed by the image quality and resolution. Only issue seems to be the aliasing and the very short recording time. I get 10~11 seconds with a Sandisk 120MB/s CF and 14 seconds with a Toshiba 150MB/s CF card.
I am very new to shooting videos but it feels really fun.
Thank you for your amazing work.

https://www.youtube.com/watch?v=hSqJzyp5DY8
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 02, 2019, 05:48:21 AM

Quote from: york824 on September 02, 2019, 03:52:38 AM
... Only issue seems to be the aliasing and the very short recording time. I get 10~11 seconds with a Sandisk 120MB/s CF and 14 seconds with a Toshiba 150MB/s CF card.
First , there no aliasing in 3x crop mode (5xZoom) it's 1:1 pixel , that why there a crop factor .
The aliasing in your video comes from your workflow (special the 8bit  Jpegs)
(From your Youtube post )
QuoteWorkflow: MLV App -- CinemaDNG -- Lightroom for Coloring -- JPEGs -- Davinci Resolve 16 -- Final output
Plus I see it's also upscaled to 4k  :( , personal I think up scaling to 4k is a bad think , I never seen a good one yet .
It always introduces artifacts & aliasing issues

You go from RAW Cdng's (10bit ,Linear color space) to Jpeg (8bit , 4.2.2 , 16-235 color space) then grade it (coloring) in Light Room .
Why grade the 8bit Jpegs ? you are loosing all that data .
Best to grade the cdng's , you have Resolve so you can grade (color) the cdng right there , and if you don't like resolve
to grade your image you can use MLV App to color the raw cdng's .

The data rate for 2784x1160 23.976 fps is around 90MB/s & 5D2 CF Card bus speed is limited to around 75-80MB/s ( but this not a hard limit , working on overclock the bus speed)
So if you get around 15 second a footage that's about right , I have Lexar 1066x 64GB and what I get on average.
If you what longer record times use 4k Anamorphic preset or lower horizontal resolution in centered 3.5k crop to 2.5k etc. ...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: york824 on September 02, 2019, 08:48:30 AM
Quote from: reddeercity on September 02, 2019, 05:48:21 AM
First , there no aliasing in 3x crop mode (5xZoom) it's 1:1 pixel , that why there a crop factor .
The aliasing in your video comes from your workflow (special the 8bit  Jpegs)
(From your Youtube post ) Plus I see it's also upscaled to 4k  :( , personal I think up scaling to 4k is a bad think , I never seen a good one yet .
It always introduces artifacts & aliasing issues

You go from RAW Cdng's (10bit ,Linear color space) to Jpeg (8bit , 4.2.2 , 16-235 color space) then grade it (coloring) in Light Room .
Why grade the 8bit Jpegs ? you are loosing all that data .
Best to grade the cdng's , you have Resolve so you can grade (color) the cdng right there , and if you don't like resolve
to grade your image you can use MLV App to color the raw cdng's .

The data rate for 2784x1160 23.976 fps is around 90MB/s & 5D2 CF Card bus speed is limited to around 75-80MB/s ( but this not a hard limit , working on overclock the bus speed)
So if you get around 15 second a footage that's about right , I have Lexar 1066x 64GB and what I get on average.
If you what longer record times use 4k Anamorphic preset or lower horizontal resolution in centered 3.5k crop to 2.5k etc. ...

Thank you so much for your advises. I use Lightroom because I am new to Davinci Resolve and don't know how to get the color grading done. The MLV App seems to render the images badly at least they don't look half as good when exported as DNG and opened in Lightroom so I don't really know how to grade in MLV App. Guess I have to learn how to grade Davinci Resolve now :D.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on September 02, 2019, 10:53:40 AM
Quote from: york824 on September 02, 2019, 08:48:30 AM
don't look half as good when exported as DNG and opened in Lightroom

DNG is a raw format. None of the changes you make in it will take effect on Lightroom.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: thebailey on September 02, 2019, 12:24:23 PM
Hey, just wondering if there is any way to get an interlaced mode enabled for 5Dii 1:1 pixel 3 x crop mode (10bit raw), without dual ISO? It would give us 1920 x 1080 @ 50fps for half the data rate?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on September 02, 2019, 05:10:23 PM
Quote from: york824 on September 02, 2019, 08:48:30 AM
...The MLV App seems to render the images badly at least they don't look half as good when exported as DNG and opened in Lightroom so I don't really know how to grade in MLV App.

If you don't mind me asking... what were your export settings set to? Would be happy to get yourself squared up on MLV App to get the most out of your 5D2 while ur at it.

Please reply within MLV App (https://www.magiclantern.fm/forum/index.php?topic=20025.msg186965#msg186965) to keep this thread on topic per @reddeercity. Thanks!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 03, 2019, 05:46:27 AM
Ok back to Dual ISO patterns , I when back to the old March 2017 dual iso video module
and loaded it on  "magiclantern-10bit_12bit_raw_twk_crop_rec.2016Dec01.5D2212" and it works ok.
I did find the ISO pattern  ISO pattern     : BBdd RGGB

The updated crop_rec dual video iso module has 4 different ISO patterns for crop_rec

Doesn't look like interlaced ISO
ISO pattern     : ddBB RGGB
ISO pattern     : BBdd RGGB
ISO pattern     : dBBd RGGB
ISO pattern     : BddB RGGB

I just need one ,
ISO pattern     : BBdd RGGB
not sure how this happen , basically I used waza57 dual video iso code and replace the non dual video iso module in the  5d2 4k crop_rec branch with the Video enabled dual iso .

But I find that MLV App still doesn't recognize it as dual iso ,
but MLVFS Web Gui does ! see below , it export as 20bit from 13.8 to 16.82EV
AMaZE interpolation ...
Edge-directed interpolation...
Smoothing alias map...
Final blending...
Noise level     : 8.00 (20-bit), ideally 8.00
Dynamic range   : 16.82 EV (cooked)
Semi-overexposed: 0.38%
Deep shadows    : 61.15%
Full-res reconstruction...
ISO overlap     : 4.7 EV (approx)
Half-res blending...
Building alias map...
Filtering alias map...
Amaze took 0.65 s
Edge-directed interpolation...
Smoothing alias map...
Final blending...
Noise level     : 8.00 (20-bit), ideally 8.00
Dynamic range   : 16.82 EV (cooked)
Semi-overexposed: 0.47%
Deep shadows    : 61.19%
Full-res reconstruction...
ISO overlap     : 4.7 EV (approx)
Half-res blending...
Building alias map...
Filtering alias map...
Smoothing alias map...
Final blending...
Noise level     : 8.00 (20-bit), ideally 8.00
Dynamic range   : 16.82 EV (cooked)
ISO pattern     : BBdd RGGB
White levels    : 16219 14819
Noise levels    : 8.00 8.00 8.00 8.00 (14-bit)
ISO difference  : 3.13 EV (875)
Black delta     : -4.94
Dynamic range   : 10.82 (+) 10.67 => 13.80 EV


So For now you can used the old 5D2212_Video_dual_iso.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2212_Video_dual_iso.zip) with magiclantern-10bit_12bit_raw_twk_crop_rec.2016Dec01.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-10bit_12bit_raw_twk_crop_rec.2016Dec01.5D2212.zip) together and you sure be able to use lightroom plug/ CR2hdr .
That build has only 10-12bit in crop_mode not 3x3(FHD) FYI

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on September 03, 2019, 12:43:58 PM
Thanks like always RedderCity, but just for full operation option which build allow 3x3 FHD Dual ISO?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on September 06, 2019, 02:51:01 AM
@Reddeercity. Sorry, could you please tell with simple words? As I understood, you did your work, and uploaded the source code for others?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 23, 2019, 05:11:55 AM
Hey all , I'm back from my summer projects  :)
I have some free time now for the next 8weeks , I hope to get more work done on both 5D2 & 50D crop_rec .
I'll need a few days to refresh & catch up on things .

A little more investigation in to dual ISO patterns from above posts

(https://i.ibb.co/phww7X1/dual-iso-frame0-1-2-3-small.png) (https://imgbb.com/)

from left to right , frame 0 , 1 , 2 & 3 , first 2 frames are different then the next 2 ( frame 2 & 3)  are the same
So by blending all 3 different iso patterns , would that give a better image without artifacts ?
I remember a post by @Ilia3101  about having different iso lines on the same image for a better image interpolation .
how about different iso pattern on each frame ? I guess we would need to add different patterns to mlv_dump (not sure if I can do that , may be beyond my abilities)
and then blend them , not sure if that would be better or not .
Here the 4 dng's I used to compare Different_ISO_Patterns.rar (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Different_ISO_Patterns.rar)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 23, 2019, 07:01:18 AM
from this post https://www.magiclantern.fm/forum/index.php?topic=21447.msg220835#msg220835
here the link to the 3xcrop_mode check I having issue with
crop_rec.c#lines-1588 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/src/crop_rec_4k_5D2/modules/crop_rec/crop_rec.c#lines-1588)
any suggestions ? , I'll post tomorrow the code pieces I've tried that didn't work .
I started to update the source code to a new bitbucket repository. magic-lantern_reddeercity_5d2-50d_4k-crop_rec
but I havn't updated the crop_rec module code yet with the new presets , in the next days .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on September 24, 2019, 03:27:56 PM
Didn't understood well where is the problem and what you are trying to achieve, but the highlighted line of code is never taken with the 5d2, because the condition for true statement as it is written now is: "go inside the true branch if is not a 5d2 and current register is 0xC0F06804".
Or are you working in local with a different code at that line?

Maybe try to manipulate the register 0xC0F06804 inside the false branch like waza57 did (or remove !is_5d2), if you want to follow his route.
If I mislead something just ignore, didn't digged inside crop_rec code yet :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 25, 2019, 12:43:28 AM
@aprofiti thanks for input , sorry here the code I'm working from locally
if (!is_5D2 &&(reg == 0xC0F06084))         
        {
            engio_vidmode_ok = (crop_preset == CROP_PRESET_3K)
                ? (old == 0x46A04BA)                        /* x5 zoom */
                : (old == 0x4F40432);                       /* 1080p */
        }                 
        else
        {
          if (reg == 0xC0F06014)      // waza57 for 5D2 without this reg check,  0xC0F06084 is not seen 
            {                 
               if (crop_preset == CROP_PRESET_3K)
                {
                  if ((old == 0x4ff) || (old == 0x4d5 ))
                    {
                      engio_vidmode_ok = 1;
                    }               
                  else
                    {
                      engio_vidmode_ok = 0;
                    }
                 }   
               if (crop_preset != CROP_PRESET_3K)
                {
                   if (old == 0x577)
                    {
                      engio_vidmode_ok = 1;
                   }
                 else
                    {
                      engio_vidmode_ok = 0;
                    }
                } 
            }                     
        } 
    }
    if (!is_supported_mode() || !engio_vidmode_ok)
    {
        /* don't patch other video modes */
        return;
    }

    for (uint32_t * buf = (uint32_t *) regs[0]; *buf != 0xFFFFFFFF; buf += 2)
    {
        uint32_t reg = *buf;
        uint32_t old = *(buf+1);
       
        int new = reg_override_func(reg, old);
        if (new || reg == 0xC0F06084)            // waza57 fixme! reg_override_func(0xC0F06084 return 0 so new don't exist)
        {
//            dbg_printf("[%x] %x: %x -> %x\n", regs[0], reg, old, new);
            *(buf+1) = new;
        }
    }
}

This works right now , problem is adding in a second or third 3xcrop_mode (5xZoom) preset .
If I have 2 presets e.g.
engio_vidmode_ok = (crop_preset == CROP_PRESET_3K || CROP_PRESET_CENTER_Z)
.........
if (crop_preset == CROP_PRESET_3K || CROP_PRESET_CENTER_Z)
........
if (crop_preset != CROP_PRESET_3K || CROP_PRESET_CENTER_Z)

This causes both preset to default back to 3x3(FHD)
I did have a little better success with
engio_vidmode_ok = ((crop_preset == CROP_PRESET_3K) (CROP_PRESET_CENTER_Z))
the first preset applied correctly (in 5xZoom) the next one loaded in 3x3(FHD) but before it loaded , it flash
for a split second in 5xZoom then fell back to the default 3x3 (FHD)
I can change the name to any 5xZoom preset & it loads correctly , just can't have more then one preset name that a time .
I'm not a real coder , so this may be fix easy for a real programmer  :D
I work best from examples .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: alexboum on September 27, 2019, 04:54:04 PM
Hello,

"if" CROP_PRESET_CENTER_Z = 1 the two last "if" will always be executed.

Quote

engio_vidmode_ok = (crop_preset == CROP_PRESET_3K || CROP_PRESET_CENTER_Z)
.........
if (crop_preset == CROP_PRESET_3K || CROP_PRESET_CENTER_Z)
........
if (crop_preset != CROP_PRESET_3K || CROP_PRESET_CENTER_Z)



Is it what you are looking for? If not you can maybe try:
Quote
engio_vidmode_ok = (crop_preset == CROP_PRESET_3K || CROP_PRESET_CENTER_Z)
.........
if (crop_preset == CROP_PRESET_3K || CROP_PRESET_CENTER_Z)
........
else if (crop_preset != CROP_PRESET_3K || CROP_PRESET_CENTER_Z)
........

With this synthax only one "if" can be executed. (If the first "if" is true the "else if" is ignored).

Hope it can help you.
Anyway thanks for your amazing work.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 28, 2019, 04:11:10 AM
@alexboum thanks for the help , I'll give it a try
Yes that helps me under the synthax  better  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 02, 2019, 08:10:28 AM
Hey 50D users I've started to port 50D to crop_rec , first step was to code a simple preset in ADTG_GUI.mo where a1ex first tested the 5d3 code.
Reasons why I put the 50D crop_rec in adtg_gui is to make sure it work or course & a simple coding task for me ! Since I very weak on that I
being teaching myself (with the help of the internet  :D ) c code syntax , I'm slowing making good process , much to learn but it's worth it in the end  ;D

adtg-gui_crop_rec-3.7k-50D-eXperimental.2019Oct01.50D109.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/adtg-gui_crop_rec-3.7k-50D-eXperimental.2019Oct01.50D109.zip)

How to get started:
Load mlv_rec.mo & adtg_gui.mo
Enable adtg_gui go to the advance menu and enable ENGIO Registers

(https://i.ibb.co/VvfN2g6/VRAM0-small.png) (https://imgbb.com/)

Press half shutter to exit to liveview then press the play button (make sure to have a cr2 there to view ,this refreshes liveview & enables the other 1500 reg's)
go back to the advance tab in the adtg_gui.mo and now enable the 3k 1:1 50D preset

(https://i.ibb.co/48Wwqzk/VRAM2-small.png) (https://imgbb.com/)

Press half shutter to exit to liveview , then you may have to cycle thought the 5xzoom button ( but you should only have to press it once)
Make sure you set the shutter to as close to 1/48th of second as you can , I was able to set it to 1/49th and when the preset is enabled it will
go to 1/48th of a second  for 23.9876 fps @ 3744x1080

(https://i.ibb.co/6XXtmBW/VRAM3-small.png) (https://imgbb.com/)

Don't worry about the line on the right had side , there will not be in the raw image (this is the same thing that happen with the 5D2 , still working on a fix )
To make sure everything gets loaded , you can check the memory patches in the debug menu you should have 7 like this

(https://i.ibb.co/xJFSvcv/VRAM4-small.png) (https://imgbb.com/)

On a Lexar 1066x 64GB CF card was getting around 10 second in 10bit .

This is very Bleeding Edge Stuff , so expect lockups , cam liveview freezes , etc. .... if it brakes you get to keep both halves  :P
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: thebailey on October 02, 2019, 08:40:31 AM
Standard interlacing would take 2 lines (to make sure you have RGBG included within that line) per stripe, and then break a 25 frame/s (40ms) timeline into a 50 field/s timeline, packing an 'upper' and 'lower' fields (20ms) taken sequentially but packed into one 40ms frame as far as the codec is concerned.

If you want max HDR and framerate in low data rate (ignoring compression), then you would have a different gain or exposure time on each stripe. Since I don't think the 5D2 can do different exposure times for individual stripes, then you have a few options:

1) In frame 1, you have the up field low gain, the down field high gain, then in frame 2 you have the up field high gain and down field low gain. This minimises the time that errors can accumulate that are hard to correct for (motion).

2) It may be possible to actually interlace using single (incomplete colour) lines, RG lines and GB lines, as long as you invert which one has the gain per field each frame

3) If you have the option to sample the pixels individually, you could effectively interlace in stripes of 2 lines (complete colour data) in horizontal and vertical directions, with each RGBG subset alternating high-gain, low-gain, then low gain, high gain, for 4 fields / 2 frames. This would minimise the chance for errors while maximising dynamic range and frame rate for the lowest data rate.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: AF-OFF on October 02, 2019, 07:38:36 PM
@reddeercity
can confirm it works on my 50D.
could record several clips with 2560x1080 10bit uncompressed and 23.9 fps
Because of a slow card only 12seconds each, one pink glitch in one clip
GD should be set to allow in raw video (mlv) menu and preview to auto

cheers
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: aprofiti on October 02, 2019, 07:48:34 PM
It's normal if the pink clip is on the 2nd or 3rd frame (link (https://www.magiclantern.fm/forum/index.php?topic=5601.msg212869#msg212869) to old post for reference) it's from the 10-12bit experimental method for D4 cameras

@reddeercity is this with your current repo codebase?
Do you think the problem (https://www.magiclantern.fm/forum/index.php?topic=19336.msg220836#msg220836) you where having with 5d2 will pretty much likely happen also with 50D?

Was thinking to have a look to understand crop_rec_4k code using 50D for testing, but I'm a bit busy these days...

Would like just try to use as much of the main codebases as possible compared to current waza changes, but due to not being familiar with those changes I would like to know if are they all necessary?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on October 02, 2019, 09:56:53 PM
Hey @reddeercity , thanks for coming back to work on 50D. I will test later your build and report any issues.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 03, 2019, 05:12:39 AM
Quote from: aprofiti on October 02, 2019, 07:48:34 PM
It's normal if the pink clip is on the 2nd or 3rd frame (link (https://www.magiclantern.fm/forum/index.php?topic=5601.msg212869#msg212869) to old post for reference) it's from the 10-12bit experimental method for D4 cameras
I don't have that here ,
I only use mlv_rec (need audio) not mlv_lite (no audio support for d4 cams) ,
there too many issue with D4 cams with mlv_lite  -- mainy the liveview preview side of the code (more resources are needed etc. .... )

Quote from: aprofiti on October 02, 2019, 07:48:34 PM
@reddeercity is this with your current repo codebase?
Do you think the problem (https://www.magiclantern.fm/forum/index.php?topic=19336.msg220836#msg220836) you where having with 5d2 will pretty much likely happen also with 50D?
Yes more then likely ,  but it's a syntax code issue I'm having after reading some info online (I think I may have resolved it )

Quote from: aprofiti on October 02, 2019, 07:48:34 PM
Would like just try to use as much of the main codebases as possible compared to current waza changes,
but due to not being familiar with those changes I would like to know if are they all necessary?
I think yes , from waza comments there more then just the crop_rec code that was changed .
I'm still updating my source slowly , really the only changes I made was adding raw_slurp for 5d2 & 50d
and add the preset reg's to the crop_rec menu . Thou it did some time to understand the crop_rec code , just because
there was no preset example's to follow for D4 cams (other then the one that waza made but it was incomplete ) , I had to figure it out for myself   
I basically fellow the code structure for the 5D3 and just add the D4(5D2) code needed , so yea I'm very close to the main code base .

Next thing for me is to add the 50D in to the crop_rec module code ,
In fact that's what I'm doing tonight  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: waza57 on October 03, 2019, 06:38:10 AM

QuoteI think yes , from waza comments there more then just the crop_rec code that was changed .

Here , in "NOTES for developers: " You can find where i change the code other than in crop_rec module:

https://www.magiclantern.fm/forum/index.php?topic=19336.msg205624#msg205624 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg205624#msg205624)

Sorry, always without more time to help.... :-[
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Jackeatley on October 06, 2019, 05:40:26 PM
Quote from: reddeercity on October 02, 2019, 08:10:28 AM
Hey 50D users I've started to port 50D to crop_rec , first step was to code a simple preset in ADTG_GUI.mo where a1ex first tested the 5d3 code.
Reasons why I put the 50D crop_rec in adtg_gui is to make sure it work or course & a simple coding task for me ! Since I very weak on that I
being teaching myself (with the help of the internet  :D ) c code syntax , I'm slowing making good process , much to learn but it's worth it in the end  ;D

I can report working as expected with my 50d, have only experienced one freeze when the buttons wouldn't respond and the card led was solid, some reduced responsiveness but I expect that's due to the first version.

i'm getting around 7 seconds of 10bit onto my Komputerbay 1050x 128gb card.  Image looks pretty much as i'd expect, very impressed.

Thanks for all the hard work, lots of fun revisiting the 50d.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: york824 on October 10, 2019, 04:35:39 AM


I have been using this build for a few months now. Really enjoying it. Can't say thank you enough for your work.
It's also a learning process for me as I am really new to video. One small question though: I saw that there is an app called DSLR Controller on Google Play Store that seems to allow Android phones to control Canon DSLRs. My smartphone has a much better screen so I am wondering if it is compatible with Magic Lantern. I did some Google search on this but most information was from 2013 or 2014 so I am not sure how it is today.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on October 10, 2019, 07:34:28 AM
You can expect problems using DSLR Controller and ML. Especially crop mode. In standard Raw recording you MAY be able to expect RAW video to work ok.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on October 11, 2019, 07:19:51 PM
Looking forward to 1x1 2784x1160 on 50d. That is about the same as micro 43 crop due to lower Res sensor. I think FHD will be limited to 1600x1000 (approx) in 3x3 binning mode sadly.

Sent from my Lenovo TB-8703F using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on October 13, 2019, 09:20:32 AM
Quote from: reddeercity on September 04, 2018, 12:54:14 AM
No we need the higher write speed , compressed raw would give us (at .5 compression , which I read would the average)
from http://rawcalculator.netlify.com/calculator_desktop
basic 10bit just out of reach of cf/5d2-d4 cams

Resolution:2880x1080 10bit
Data rate:88.9MiB/s
Frame size:3.7MiB
Crop factor:1.96x crop
Field of view:20.8°
RECORD TIME: 5 minutes 43 seconds per 32GB card.

14bit compressed raw at .5 compression
Data rate:62.2MiB/s

So this would be the max if lossless worked on 5d2 , better crop factor

Resolution:3200x1080
Data rate:69.1MiB/s
Frame size:2.9MiB
Crop factor:1.76x crop
Field of view:23.1°


16x9 2880 with increased height & compressed raw , still out reach of the current CF D4 bandwidth (75-80MB/s)
need faster interface --  overclock the cf bus driver or find away to run cards at udm7 , theoretically @ umd7 the write speed should be 120-133MB/s

Resolution:2880x1556
Data rate:89.7MiB/s
Frame size:3.7MiB
Crop factor:1.96x crop
Field of view:20.8°

Max rez with compressed raw with increased height at the current cf bandwidth (75MB/s)

Resolution:2880x1200
Data rate:69.1MiB/s
Frame size:2.9MiB
Crop factor:1.96x crop
Field of view:20.8°


I prefer for more horizontal rez , thou there nothing wrong with increasing the height to the max .

This is what I'm hoping to reach as a goal for raw video , it could happen only with 2 things
compressed raw (.5) & faster UMD 6 ->7 & or overclock the cf interface on the write side

Resolution:3840x1556
Data rate:119.5MiB/s
Frame size:5MiB
Crop factor:1.47x crop
Field of view:27.5°


Now , I basic all this on continuous recording for raw video  @ 24p
If some user what to use the full width at lower frame rate with increase height at maybe 10 of 12 fps
for hyperlays etc... will that's great , so there's really should be no limit on resolutions just
limitations on the presets in the crop_rec.mo .

At least now I can say there's no more aliasing & moiré patterns  :)
I know this was an old post, but is there a possibility of overclock of CF interface? Something similar to sd_uhs for Eos m?

Sent from my Lenovo TB-8703F using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 13, 2019, 10:07:37 AM
Quote from: zcream on October 13, 2019, 09:20:32 AM
I know this was an old post, but is there a possibility of overclock of CF interface? Something similar to sd_uhs for Eos m?
Short answer , Yes . read links below
https://www.magiclantern.fm/forum/index.php?topic=19336.msg207053#msg207053
https://www.magiclantern.fm/forum/index.php?topic=19336.msg207054#msg207054
https://www.magiclantern.fm/forum/index.php?topic=19336.msg207128#msg207128
https://www.magiclantern.fm/forum/index.php?topic=19336.msg207348#msg207348
But  I haven had time to go back ,
Lossless Compression is closer to be do-able , I just have a few error to fix read this post
https://www.magiclantern.fm/forum/index.php?topic=18443.msg198487#msg198487
there again I haven had time to go back  :(
hoping to get there soon , if you like you can jump in and help , you will need to do some reading , ok a lot of reading  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: york824 on October 13, 2019, 11:54:32 AM
Quote from: jpegmasterjesse on October 10, 2019, 07:34:28 AM
You can expect problems using DSLR Controller and ML. Especially crop mode. In standard Raw recording you MAY be able to expect RAW video to work ok.
Thanks for your reply.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 14, 2019, 08:03:02 AM
doing some UDMA clocking on the CF Card interface with DIGIC POKE , I'm able to push the write speed to a average of 75.5MB/s  (I'm using a will used Lexar 1066x 32GB from 2014)
normally it's around 68MB/s , so peak write speed that I saw was around 80MB/s + . Seems to be a 10% increase ,
I don't think I have the right reg's yet or I need more reg's , I can change the reg's for write speed to as low as 5.5MB/s then I increase it slowly to 12 , 15, 20, 25, 33, 48, 60, 68MB/s to 75.8MB/s with different reg's valves .

(https://i.ibb.co/sFtnVGs/VRAM1.png) (https://imgbb.com/)
So far
0x101
seem to be the fastest at the moment  , I'll have to start working in the HDParm branch to see if I can clock it higher
I have the branch cloned just need to compile for 5d2 and investigate what can be done to increase write speed .

Edit: My test resolution for this was 2144x1078 @ 29.66 FPS in 14bit raw ( default ML 3x crop_mode)
so around 7-8 seconds .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on October 15, 2019, 11:11:57 PM
Tested now the 50D build. Thanks to the @AF-OFF tip about "GB Allow" and "Preview Auto", I'm not getting pink frames. With other configs there is half-screen glitchs.

Using KomputerBay 64gb 1000x:
- 8 seconds on 10bit 3744x1080px
- 7 seconds on 14bit 2160x1080px
- 16 seconds on 12bit 2160x1080px
- More than 1min on 10bit 2160x1080px

This is awesome!

Don't know if that's expected or not, but it fails to load mlv_play.mo:


tcc: error: undefined symbol 'lossless_decompress_raw'
   [E] failed to link modules
UILock: 00000000 -> 41000000 => 00000000


Will it be possible to have live preview? And the 'anamorphic mode' from Danne's EOS M would be possible, eventually? The crop factor is too big for what I need, unfortunatelly :(
Thanks for all the work @reddeercity and everyone that contributed!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 16, 2019, 07:38:45 AM
Quote from: Luther on October 15, 2019, 11:11:57 PM
Don't know if that's expected or not, but it fails to load mlv_play.mo:
error message hints to the problem
mlv_play is no supported right now in crop_rec d4 branch , until I finish implementing lossless compression  (almost there) , been jumping around between different 5d2 & 50d
crop_rec problems . Trying to workout the bugs for crop_rec module on 50D ,(cmos hooks , are different for 50d then the 5d2 or 5d3) 50D uses "cmos3 & cmos4 for height & width
respectively , where all other cams to my knowledge uses cmos1 & cmos2 . So I need to change the code to use cmos 3&4 hooks (just opened up a big can of worms  :o )
Quote from: Luther on October 15, 2019, 11:11:57 PM
Will it be possible to have live preview? And the 'anamorphic mode' ?
Should be possible , but not as good as the d5/5d3/EOSm , d4 cam 50d & 5d2 don't have as much processing power , so real time liveview will be limited
to the center crop of the resolution e.g. 3744x1080 =>1920x1080 b/w which gets resize for the lcd sceen to 720x480.
but the 1x3 Anamorphic just like the 5d2 has a de-squeezes function for liveview already in ML display features (2:1) close , should be (3:1)
see here https://www.magiclantern.fm/forum/index.php?topic=19336.msg213010#msg213010
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 16, 2019, 08:08:32 AM
Still having trouble with adtg hooks for d4/5d2 in crop_rec (needed for 1x3 anamorphic) there just don't work .
this is what I have
   
              /* ADTG2/4[0x1000] = 0: read every line */
              /* ADTG2/4[0x100C] = 2: vertical binning factor = 3 */ 
              if (is_5D2)
              {             
              adtg_new[2] = (struct adtg_new) {6, 0x1000, 5};
              adtg_new[2] = (struct adtg_new) {0, 0x100c, 2};
              }
              break;

where "0x1000" default valve is "5" 0x5 (3xcrop_mode) and "6" 0x6 the line skipping mode that it's get changed to ,
and "0x100c" is the pinning , default valve is "2" 0x2 -> "0" 0x0
and add "1000" &  "100c" to the adtg_hooks
patch_hook_function(ADTG_WRITE, MEM_ADTG_WRITE, &adtg_hook, "crop_rec: ADTG[1000,100c,8000,8806,] parameters hook");

It I put the 5d2 1x3 preset in to the adtg_gui.mo like the 50D 3.7k preset I can get one of the adtg reg to load "adtg[12] 0x1000" but not "adtg[12] 0x100c"
this what I did to get it to work , commented out the first line and then it loaded the first case 0x1000.
//else if (regs[reg].dst == 2)        /* ADTG 12 */
    {
        switch (regs[reg].reg)
        {
            case 0x1000: return 0x6;      /* it's 5 in zoom mode and 6 in 1080p; this also overrides ADTG4 */
            case 0x100c: return 0x0;
            case 0x8806: return 0x6088; /* without this, you get some weird artifacts; this should only go to ADTG2, not 4 */


So I'm kind of lost , @a1ex I know you are not really working much here on the forum , but do you have any ideas where I should start to look to fix this  :D
Any help would be appreciated .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on October 17, 2019, 12:11:09 AM
Quote from: reddeercity on October 16, 2019, 07:38:45 AM
see here https://www.magiclantern.fm/forum/index.php?topic=19336.msg213010#msg213010
Wow. That seems good.
Got some pink frames while testing again today with 50D, but nothing too bad. It's working nicely with in 2560x1072px 10-bit.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on October 17, 2019, 12:40:01 AM
One frame from the tests. Quite nice, considering the lens I'm using is 50 years old :) Also, the iso is 200, there's some noise. Processed with MLVApp.

(https://pictshare.net/wta4tv.png) (https://pictshare.net/wta4tv.png)
Click for full-res.
Title: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on October 18, 2019, 06:16:12 PM
Beautifully rendered @Luther and the contrast looks amazing especially coming from a 50 year old glass... :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Volumetrik on October 20, 2019, 04:07:47 AM
I think I may be doing something wrong with the  crop_rec-4k3x1_2784x1160-1x1_FHD48p3x3_11.59pm-5D2-eXperimental.2019Jul15.5D2212  build. I can't get the preview or the recorded video to be centered.

Following instructions on the first post : 1080p30 --> crop_rec & mlv_rec only --> 10bit & 1:1 --> x5 zoom --> 3.5k preset --> 2784x1160 --> liveview and record.

Here is a normal photo of the scene. Full frame 3:2 :

(https://i.ibb.co/f1bqFmH/IMG-8844.jpg)

And here is the framing I get out of the settings mentioned :

(https://i.ibb.co/QQPrmjf/non-center.jpg)

And here are these two pictures overlaid to illustrate the difference in framing.

(https://i.ibb.co/0YP1MXC/overlaid.jpg)

Here is a picture of what I see when recording :

(https://i.ibb.co/q9mtqqT/IMG-3034.jpg)

Am I doing something wrong ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on October 20, 2019, 03:44:36 PM
Quote from: DeafEyeJedi on October 18, 2019, 06:16:12 PM
Beautifully rendered @Luther and the contrast looks amazing especially coming from a 50 year old glass... :)
Thanks. Some pixel pipping using mlvapp always helps the low contrast of the lens :p
Maybe I will do a video test, got inspired by what @york824 did.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 23, 2019, 06:54:21 AM
Updating my source (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/src/crop_rec_4k_5D2/modules/)  code &  comments
https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/
8eae4f2 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/8eae4f2d0ab9cdef59c2969afcec11413512ed05) & 2ebc7c7 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/2ebc7c7070eed6fa7d85d5f661ad02032c9cafb2)
add the 50D raw_slurp for extended rez & 10/12bit raw in crop_rec
still have issue with 50d crop_rec compiling right now , but the rest of ml compiles ok .
hope to the rest of the 5d2 code updated by the weekend  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 23, 2019, 07:15:47 AM
I think there a way to get 7D (1st ver.) in to crop_rec for extended resolution in 5xZoom .
By way of "Rom Hack" this by-passes the raw_slurp in the EDmac to push the vertical
that how waza57 first got the crop_rec to work on the 5d2 at 2880x1080 . Now the 5d2 doesn't need this any more as we found how to get raw_slurp working .
See this commit e7bfe64 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/e7bfe646352f287a6ae0c01f533644d1d380feac) for detail (currently it's disable for 5d2 but wouldn't take long to enable it for 7D)

//static uint32_t RAWROMHEIGHT_WRITE = 0;
//static uint32_t RAWROMWIDTH_WRITE = 0;
//static uint32_t MEM_RAWROMHEIGHT = 0;
//static uint32_t MEM_RAWROMWIDTH = 0; 
etc. .....

It may be possible , but don't have a 7d to test on
an then it would have to be added to the crop_rec code -- thou I don't have the time but if someone with some coding ability & 7D wants to
get involved I can help push it alone .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 23, 2019, 07:47:19 AM
Updated the first page with the 50D adtg_gui crop_rec experiment 3744x1080 @ 23.976 fps
https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on October 23, 2019, 12:59:30 PM
Hey !
I have been trying to get 4k anamorphic and 48p to work for a few days, but it keeps showing me weird flickering black and white liveview, which shows on the mlv file when opening it in mlv app, the weird thing is that I can see that its enabled (it shows 4064x590 when using 4k anamorphic, and 48p crop_rec seems to load correctly). So if you have any ideas, please help me ^^
Thanks a lot for your awesome work !!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: stokis on October 27, 2019, 07:50:10 AM
It would be great to have 4k anamorphic working without bugs. I would say 4k anamorphic is second most  important thing after losless compression for 5d mkii. At least for me.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 27, 2019, 10:48:54 PM
Quote from: Galterius on October 23, 2019, 12:59:30 PM
Hey !
I have been trying to get 4k anamorphic and 48p to work for a few days, but it keeps showing me weird flickering black and white liveview, which shows on the mlv file when opening it in mlv app, the weird thing is that I can see that its enabled (it shows 4064x590 when using 4k anamorphic, and 48p crop_rec seems to load correctly). So if you have any ideas, please help me ^^
Thanks a lot for your awesome work !!
did you do this ?
https://www.magiclantern.fm/forum/index.php?topic=19336.msg218753#msg218753
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on October 27, 2019, 11:11:32 PM
@stokis, absolutely agree. Even though my 5d2 stopped working (i've tried to dissasemble it, but made a mistake.... so, I can't participate en testings).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 27, 2019, 11:14:50 PM
Quote from: stokis on October 27, 2019, 07:50:10 AM
I would say 4k anamorphic is second most  important thing after losless compression for 5d mkii. At least for me.
I disagree , 5k Anamorphic in x5 Zoom is the most important thing to me (adtg hook problem), Lossless compression (https://www.magiclantern.fm/forum/index.php?topic=18443.msg198487#msg198487) is way down on my priority list .
This is my priorities :Get 50D in to the crop_rec code at the same level as the 5D2 (including Anamorphic)
5D2:  5k Anamorphic in x5 Zoom (works in the adtg_gui module) adtg hook write problem in crop_rec code
CF Card interface bus Overclocking write speed (looking for at least 100MB/s write speed , 30% increase to be useful)
Lossless compression for 5d2 & 50d
And at the bottom ....
7D with limited "rom hack" extended  horizontal  resolutions (maybe 3k)

Edit: FYI 50D maximum resolution @ 24fps ( with a good A.R.)  in crop_mode 3200x1254 @ 24.011 fps = to 100.7 Mpixel/s ( Mega Pixels per second)
Sensor speed on the 50D is 28.8 MPixel/s x 4 channel readout = 115.2MPixel/s
Still have a vertical offset problem , only 1087 vertical pixels are visible out of 1254 (the other 167 lines are black no image data)
may need to do a dm_spy log for cr2/full res liveview height to see what regs are being applied .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on October 28, 2019, 05:01:09 AM
Quote from: reddeercity on October 27, 2019, 11:14:50 PM
This is my priorities :Get 50D in to the crop_rec code at the same level as the 5D2 (including Anamorphic)
Nice! 50D is a 12 years old camera that still rocks so much. Anamorphic mode will be great. Thanks for all your work on this @reddeercity.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 28, 2019, 05:35:30 AM
Quote from: Luther on October 28, 2019, 05:01:09 AM
Nice! 50D is a 12 years old camera that still rocks so much. Anamorphic mode will be great. Thanks for all your work on this @reddeercity.

Thanks , here the start of 3x1 anamorphic on 50D in 3x3 Line Skipping.

(https://i.ibb.co/7n1gb5Q/50-D-3x1-4096x1968-M27-2334-frame-2-small.png) (https://imgbb.com/)
4096x1968 @ 23.976
50D_3x1_4096x1968_M27-2334_frame_2.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_3x1_4096x1968_M27-2334_frame_2.png) Un-Squeezed Vertically
50D_3x1_4096x656-M27-2334_000002.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_3x1_4096x656M27-2334_000002.dng) Vertical Squeezed
So the crop factor on this is 1.76 of full frame (@ 4k(4096) there a crop factor of 1.16 of the 50D sensor plus a 1.6 crop factor of full frame if using full frame lens that is)
So just to refresh every one memory , the 50D senor resolution is 4752x3168 .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on October 28, 2019, 12:48:27 PM
Quote from: reddeercity on October 28, 2019, 05:35:30 AM
Thanks , here the start of 3x1 anamorphic on 50D in 3x3 Line Skipping.
4096x1968 @ 23.976
50D_3x1_4096x1968_M27-2334_frame_2.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_3x1_4096x1968_M27-2334_frame_2.png) Un-Squeezed Vertically
50D_3x1_4096x656-M27-2334_000002.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_3x1_4096x656M27-2334_000002.dng) Vertical Squeezed
So the crop factor on this is 1.76 of full frame (@ 4k(4096) there a crop factor of 1.16 of the 50D sensor plus a 1.6 crop factor of full frame if using full frame lens that is)
So just to refresh every one memory , the 50D senor resolution is 4752x3168 .

This will be very useful for me. I use 50D MLV professionally (marketing videos). It works very well, but the resolution is not ideal. With your last build I was able to record in very high res, but with too much crop factor. This anamorphic mode will enable me to get the best out of 50D, without cropping that much. Amazing.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on October 28, 2019, 06:06:46 PM
Can someone explain how to use 48fps firmware, its doesn't have setting like on the video on front page, I will use firmware that correspond with video, maybe that'll work. Thanks again Redeercity, are you done with 5DM2 firmware and are you getting better images with 50D?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Galterius on October 28, 2019, 07:49:20 PM
Quote from: reddeercity on October 27, 2019, 10:48:54 PM
did you do this ?
https://www.magiclantern.fm/forum/index.php?topic=19336.msg218753#msg218753
Thanks, I had not seen this message, I will see if that works !
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on October 28, 2019, 09:00:20 PM
Guys, could you please upload some more 4k anamo videos from your 50d? And is the liveview frozen like on 5d2?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: stokis on October 29, 2019, 08:14:03 AM
Quote from: reddeercity on October 27, 2019, 11:14:50 PM
I disagree , 5k Anamorphic in x5 Zoom is the most important thing to me (adtg hook problem), Lossless compression (https://www.magiclantern.fm/forum/index.php?topic=18443.msg198487#msg198487) is way down on my priority list .
This is my priorities :Get 50D in to the crop_rec code at the same level as the 5D2 (including Anamorphic)
5D2:  5k Anamorphic in x5 Zoom (works in the adtg_gui module) adtg hook write problem in crop_rec code
CF Card interface bus Overclocking write speed (looking for at least 100MB/s write speed , 30% increase to be useful)
Lossless compression for 5d2 & 50d
And at the bottom ....
7D with limited "rom hack" extended  horizontal  resolutions (maybe 3k)

Edit: FYI 50D maximum resolution @ 24fps ( with a good A.R.)  in crop_mode 3200x1254 @ 24.011 fps = to 100.7 Mpixel/s ( Mega Pixels per second)
Sensor speed on the 50D is 28.8 MPixel/s x 4 channel readout = 115.2MPixel/s
Still have a vertical offset problem , only 1087 vertical pixels are visible out of 1254 (the other 167 lines are black no image data)
may need to do a dm_spy log for cr2/full res liveview height to see what regs are being applied .

Ofcourse, working 5K would be even better than 4K :)
Title: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on October 29, 2019, 09:55:52 PM
Quote from: DeafEyeJedi on April 06, 2017, 12:45:30 AM
I'll be standing by you with all of my support for this...

I came across a deal that turned out to be too good to pass on and decided to scoop up on a decent 50D body with only less than 5K shutter counts directly from an original owner on OfferUp.com. It even came w a 4GB CF card. Ha!

I personally felt that @reddeercity has done a tremendous job with an enormous amount of work already put onto this very project for the Digic 4 bodies which literally got me to jump onto this wagon. Rightfully so. Thank you, David!

Never thought I'd ever buy an 11-year old digital camera body but hey never say never, right? ;)

More to come...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 04, 2019, 07:13:54 AM
50D Update:
adtg_gui-crop_rec-4k3x1-50D-eXperimental.2019Nov03.50D109.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/adtg_gui-crop_rec-4k3x1-50D-eXperimental.2019Nov03.50D109.zip)
refined 3x1 4k preset experiment in the adtg_gui module -- (4096x720 ->4096x2160 @ 23.976FPS)
Preview is all scrambled at the moment , because I haven't figured out the correct preview reg's yet (keep in mind this is a work in progress)

(https://i.ibb.co/3CYgKZm/M04-0119-frame-1-small.png) (https://imgbb.com/)
50D_4096x2160-M04-0119_frame_1.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M04-0119_frame_1.png)
50D_4096x720-M04-0119_frame_1.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M04-0119_frame_1.dng)
Short 20 second clip with no real setup 50D_4096x2160_23.976fps-M04-0119.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_4096x2160_23.976fps-M04-0119.mp4)

So it just like before , follow the this (https://www.magiclantern.fm/forum/index.php?topic=19336.msg221074#msg221074) post for how to install and get running if you're a new user .
A FYI -- @ 4k we have a 1.76 crop factor of full frame which is = to 3.2k on full frame sensor , it work out to be a little under a ASP-C of a little bit bigger then a M4/3ths
full sensor is said to be 35x24 mm e.g. 5D2 or 5d3 , the 50D is said to be a APS-C  22.3x14.9mm with the 4k(4096x720) crop factor  = 19.22x13 mm .
There some aliasing I noticed but I didn't have time to really play around with as I just used mlv.app , if I was going to spend time grading etc. ... I would have
Used Adobe After Effects CS6 with camera raw , as this is the best work flow as it can fix aliasing issue the best . 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: nikfreak on November 04, 2019, 08:15:35 PM
Quote from: DeafEyeJedi on October 29, 2019, 09:55:52 PM
....a decent 50D body with only less than 5K shutter counts directly from an original owner...

Wow, Sean, wish 'ya lots of fun with it. Should last forever with that shutter count cause it's also built like a tank.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on November 05, 2019, 03:37:01 PM
Are the columns of pixels are even in the correct order?

(https://i.ibb.co/zVzrvCF/Screenshot-2019-11-05-at-14-35-21.png)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on November 06, 2019, 06:19:26 PM
Hi. Just to confirm 3x3 mode on both cameras is still 14-bit hd right? We are using 10-bit and 12-bit for 3k and above..

Sent from my Redmi 4A using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 07, 2019, 04:43:19 AM
Quote from: zcream on November 06, 2019, 06:19:26 PM
Hi. Just to confirm 3x3 mode on both cameras is still 14-bit hd right? We are using 10-bit and 12-bit for 3k and above..
All three bit rates (10,12,14) work in all video modes including crop_rec on 5D2 & 50D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 07, 2019, 05:30:04 AM
Experimenting with 1x3 pixel binding/anamorphic on the 50D (still very early in the development , don't expect too much yet it could all change.)
currently I got 12 fps(for testing) thou I have a black bar in the middle of the image
(cmos5, vertical offset is not set correctly yet, I was more interested in to see if I could get more height then 1200 lines) then I fell in to the 1x3 anamorphic rabbit hole  :D
So I have almost full width (4704) cr2 width is 4752 ,
I should be able to get 1365x2160 @ 23.976fps = 70.76Mpixel/s (Max useable sensor speed is 105 Mpixel/s out of 115 (4 channel readout @ 28.8mhz)
So Un-Squeezed size will be 4096x2160 @ 23.976fps , if I get the vertical offset corrected .
(https://i.ibb.co/N1qtD01/1568x2192-1x3-M05-2326-frame-1-small-1.png) (https://imgbb.com/)
1568x2192_1x3_M05-2326_frame_1.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1568x2192_1x3_M05-2326_frame_1.png)

(https://i.ibb.co/cvjMhTd/4704x2192-1x3-M05-2326-frame-1-small.png) (https://imgbb.com/)
4704x2192_1x3_M05-2326_frame_1.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4704x2192_1x3_M05-2326_frame_1.png)

(https://i.ibb.co/hVwc50R/1568x2192-1x3-M05-2326-frame-56-small-1.png) (https://imgbb.com/)
1568x2192_1x3_M05-2326_frame_56.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1568x2192_1x3_M05-2326_frame_56.png)

(https://i.ibb.co/n1Qgb16/4704x2192-1x3-M05-2326-frame-56-small.png) (https://imgbb.com/)
4704x2192_1x3_M05-2326_frame_56.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4704x2192_1x3_M05-2326_frame_56.png)

Liveview , has correct A/R with the Anamorphic display filter in ML drop down menu .
(https://i.ibb.co/s5ynY14/1x3-Anamorphic-VRAM24-small.png) (https://imgbb.com/)

2 short 20 second video chips , 1st is the squeezed @  1568x2192 mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1568x2192_12FPS_1x3_M05-2326.mp4) & then the un-squeezed at 4704x2192 mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4704x2192_1x3_12FPS_M05-2326.mp4)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on November 08, 2019, 10:50:08 PM
Wow....
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on November 09, 2019, 12:28:59 PM
Ok I really want to get the image centered on the mark II in the 3008x1080 mode, how do I get started. Is it a difficult change?

Also... wow! I'm super impressed that you almost have proper anamorphic working on 50D,  exciting.

Edit: it looks like there is still some weird aliasing, could we see MLV  file?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Volumetrik on November 09, 2019, 08:04:16 PM
Quote from: Ilia3101 on November 09, 2019, 12:28:59 PM
Ok I really want to get the image centered on the mark II in the 3008x1080 mode, how do I get started. Is it a difficult change?

Also... wow! I'm super impressed that you almost have proper anamorphic working on 50D,  exciting.

Edit: it looks like there is still some weird aliasing, could we see MLV  file?

I love using the 3.5k crop mode with 2784x1160 resolution! I also agree on the live-view centering. It's the only thing left on my wish-list, everything else works great for my use !
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 10, 2019, 02:29:54 AM
Quote from: Ilia3101 on November 09, 2019, 12:28:59 PM
Ok I really want to get the image centered on the mark II in the 3008x1080 mode, how do I get started.
First , take a cr2 (or silent full res DNG) of a grid pattern (horizontal & vertical)
then with out moving the camera or focus length record a second or 2 in the 3k preset crop_rec , upload a link to the DNG or PNG here .
Quote from: Ilia3101 on November 09, 2019, 12:28:59 PM
Is it a difficult change
yes & no , if the horizontal is out (left or right of center) then in the advance crop_rec menu you can change
cmos[2] , just key in 0x10E etc. ... being 0x10E is the default centering for 3xcrop_mode e.g. 2144x1076
if I remember right I had it set to 0x9E , (there no real fine adjustment on the horizontal)  it's moves at more then 1 columns at a time from my finding
I thing it's like 3-5 columns per valve , so it may never be right on .
If it's a vertical issue then yes it's way more work , vertical centering done by
base vertical offset cmos[1] + cmos[1]Lo & cmos[1]Hi in the code , it called "PACK12" so there's 2 value's in the pack  that need to be adjusted .
/* raw buffer centered in zoom mode */
            case CROP_PRESET_CENTER_Z:
                cmos_new[1] = 0xB0E  /* vertical (first|last) */
                  ? PACK12(12,11)
                  : -1;

this give us a per line/pixel vertical movement up & down to center .
Quote from: Ilia3101 on November 09, 2019, 12:28:59 PM
..... you almost have proper anamorphic working on 50D,
Edit: it looks like there is still some weird aliasing, could we see MLV  file?
I'm not 100% sure it was in 5x Zoom , it may have change to 3x3 , I'll have check again

Google Drive M05-2326.MLV (https://drive.google.com/file/d/1iYO0Sf9lA_m5IYcnhgL5oYsXa7R29SMZ/view?usp=sharing) 815MB
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on November 10, 2019, 11:50:48 AM
Quote from: reddeercity on November 10, 2019, 02:29:54 AM
First , take a cr2 (or silent full res DNG) of a grid pattern (horizontal & vertical)
then with out moving the camera or focus length record a second or 2 in the 3k preset crop_rec , upload a link to the DNG or PNG here .yes & no , if the horizontal is out (left or right of center) then in the advance crop_rec menu you can change
cmos[2] , just key in 0x10E etc. ... being 0x10E is the default centering for 3xcrop_mode e.g. 2144x1076
if I remember right I had it set to 0x9E , (there no real fine adjustment on the horizontal)  it's moves at more then 1 columns at a time from my finding
I thing it's like 3-5 columns per valve , so it may never be right on .
If it's a vertical issue then yes it's way more work , vertical centering done by
base vertical offset cmos[1] + cmos[1]Lo & cmos[1]Hi in the code , it called "PACK12" so there's 2 value's in the pack  that need to be adjusted .
/* raw buffer centered in zoom mode */
            case CROP_PRESET_CENTER_Z:
                cmos_new[1] = 0xB0E  /* vertical (first|last) */
                  ? PACK12(12,11)
                  : -1;

this give us a per line/pixel vertical movement up & down to center .I'm not 100% sure it was in 5x Zoom , it may have change to 3x3 , I'll have check again

Google Drive M05-2326.MLV (https://drive.google.com/file/d/1iYO0Sf9lA_m5IYcnhgL5oYsXa7R29SMZ/view?usp=sharing) 815MB

Here I shot some easy to match patterns: https://drive.google.com/open?id=1QdR9tKKupaleX6fjUdRiN06d-07djl0l

Looks like its almost vertically centered (good news):

(https://i.ibb.co/K0JH4yN/Screenshot-from-2019-11-10-10-45-59.png)

But quite off horizontally...

I will try what you said in crop rec menu and see if I can center it a bit more.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on November 10, 2019, 12:18:05 PM
Quote from: Ilia3101 on November 09, 2019, 12:28:59 PM
Edit: it looks like there is still some weird aliasing, could we see MLV  file?
Hm, when I zoom-in into this file from @reddeercity, I remember a pattern we had half a year ago with the EOS M. I can't proof, but for me it does not look like every 3rd row was recorded. For me it looks like 2 rows were recorded, then 4 rows weren't. When you zoom in, you see that 2 rows look a bit similar, and then the next row looks very different. With EOS M we had that in 1080p mode, but maybe something like that is also possible in 1x3 mode. https://www.magiclantern.fm/forum/index.php?topic=16516.msg212270#msg212270
(https://i.ibb.co/KGx6cnQ/Bildschirmfoto-2019-11-10-um-12-08-36.png)
(https://i.ibb.co/D9spxgD/Bildschirmfoto-2019-11-10-um-12-21-31.png)
(Pictures are not 3x stretched vertically.)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on November 10, 2019, 03:05:15 PM
Ok seems like 0x9e makes the image centered, but I have to wait like 10 seconds for colourful vertical stripes to settle down each time I enter crop.

Could this be at all harmful for the sensor or to dynamic range?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 11, 2019, 06:27:39 AM
Quote from: Ilia3101 on November 10, 2019, 03:05:15 PM
..... I have to wait like 10 seconds for colourful vertical stripes to settle down each time I enter crop.
Could this be at all harmful for the sensor or to dynamic range?
Short answer , no
but ml dose put extra load & stress on the sensor so maybe at some point it may burn out (or Not) who knows for sure .
The lines you see are related to the power timing reg's , those don't work in the crop_rec because the adtg write hook stop working (there a note about it the adtg c code)
same reason why I can't 1x3 anamorphic working  on the 5D2 adtg write hooks stop working in the crop_rec.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 11, 2019, 06:33:15 AM
Quote from: Ilia3101 on November 10, 2019, 03:05:15 PM
Ok seems like 0x9e makes the image centered
Well it close , but not really I did some testing with full res cr2 then overlay the 3008x1080 image over it in After Effects .
I found the best fit for horizontal alignment is
0xBE 
cmos[2] 0xBE
(https://i.ibb.co/PCWLd5x/M10-2116-000001-screen-capture-AE-small.png) (https://imgbb.com/)

The default setting
(https://i.ibb.co/5WLqXmK/M10-2116-000002-screen-capture-AE-default-center-dark-small.png) (https://imgbb.com/)

cmos[2] 0x9E , close but not good enough 
(https://i.ibb.co/4pyLntY/M10-2116-000002-screen-capture-AE-default-center-dark-cmos0x9e-small.png) (https://imgbb.com/)

It seems that for each cmos[2] reg unit the image moves about 100 column ,
from default to 0xbE , that's around 6 units = 600 , Look at the white box , Irfan view says that box is 604 pixels wide.
(https://i.ibb.co/MN5SZmd/cmos-offset-default-0x-BE-600pixels-small.png) (https://imgbb.com/)

I'll update the code for the 3k preset later on tomorrow , in the mean time go to the advance crop_rec menu
and adjust cmos[2] to 0xBE , that's as close as we will get .   ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on November 11, 2019, 06:52:46 PM
Looks like 0xBE is much better. But the vertical lines seem even stronger and take a little longer to fade.

@a1ex do you know if these lines be made to not appear when entering centered crop?

1 more question: why does the camera start recording when I press the joystick? I just want to center the window automatically, but this makes it slithgly annoying. I don't think the normal ML builds do this.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on November 13, 2019, 09:00:45 AM
Dave theres better method to test this, you can get sensor size chart and then adjust values and pinpoint where center should be , thats what i used with canon m withiut entering photoshop, adjusting on busy background is too confusing, with rectangles/ sensor sizes filling frame its much easier
I made and used this for adjusting cmos 5 and 7 on apsc camera, it needs fullframe rectangle  overlay
http://ipic.su/img/img7/fs/senoc.1573593027.gif
Ill do new version with fullframe later
--
HEre it is, If you display this on your pc monitor and fillm on canon filling frame with it then you can easily pinpoint center with different crop modes and aspect ratios
(http://ipic.su/img/img7/fs/sensors.1573634342.png)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on November 13, 2019, 08:39:24 PM
Fantastic work and thanks for sharing @2blackbar!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 14, 2019, 07:14:52 AM
Working on 50D x1.67 stretch (720p Hi frame rate Mode)
Found the adtg reg to change the vertical to the 1.67 pixel (squeezed)
adtg[12]100c 0x2 ->0x4
same as the 5D2 , I also reduced rolling shutter .
Head times are a little touchy but was able to get 50p for a brief time then lock up ,
I should be able to get a stable 48p at some point but the math says I should be able to get 1568*576*96fps=86 Mpixel/s(mega pixels per second)
So 96fps should be possible I think if all thing are equal  ::) since the sensor speed is 115Mp/s
I short test with 1.67 stretch but with only 27 fps ( to be sure it work before I tried to increase frame rate to 50p+)

50D_1.67_Stretch_1568x576_M14-0117.MLV (https://drive.google.com/file/d/1rw0lljF5fbwMUPXIXO3Kw7_HP0xwHSxf/view?usp=sharing) 118MB file (4 second of raw)

(https://i.ibb.co/cyhqzvp/M14-0117-frame-103-1-small.png) (https://imgbb.com/)
1568x576_1.67_stretchM14-0117_frame_103_1.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1568x960_1.67_stretchM14-0117_frame_103_1.png)

(https://i.ibb.co/HG28BRV/M14-0117-frame-103-small-1-67-stretch.png) (https://imgbb.com/)
1568x960_1.67_stretch_M14-0117_frame_103.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1568x960_1.67_stretch_M14-0117_frame_103.png)

2 , 4second clips one squeezed & the other still squeezed
1568x576_27fps_1.67_squeezed_M14-0117.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1568x960_27fps_1.67_squeezed_M14-0117.mp4)  1568x960_27fps_1.67_un-squeezed_M14-0117.mp4] (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1568x960_27fps_1.67_un-squeezed_M14-0117.mp4)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 15, 2019, 12:42:06 AM
Updated the 3k preset (3008x1080) with the correct horizontal centering as per this post  here (https://www.magiclantern.fm/forum/index.php?topic=19336.msg222262#msg222262)
magiclantern-crop_rec-3k_Updated_Center_4.20pm-5D2-eXperimental.2019Nov14.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-3k_Updated_Center_4.20pm-5D2-eXperimental.2019Nov14.5D2212.zip)
Edit: updated the first post (https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476) on page one with updated 3k corrected centering
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: pyffi on November 15, 2019, 05:01:46 PM
hello! in days i saw the great news about 4k and FHD 48fps on 5dmk2
download latest build in this conversation
and modules doesnt work(
- format and install on clear CF NO WORKING
- replace modules NO WORKING
and i see this message
(https://sun9-55.userapi.com/c858224/v858224235/ee1e2/Yo8-2BL-seU.jpg)

can you help me? and what i doing wrong?

5dmk2+CF Transcend 32gb 1000x
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on November 17, 2019, 08:38:39 PM
Thanks Again,  Can we use 3x3 1856x1248 with magiclantern-crop_rec-3k_Updated_Center_4.20pm-5D2-eXperimental.2019Nov14.5D2212.zip, or do we only use it for crop and also crop rec is more useful with crop lens or 24mm, right. Its so much redundancy on the front page, so if you can so I won't need to ever ask these question again, please answer these questions: 1. which firmware & tutorial is for the most stable 48fps, 2. which firmware & tutorial is for the most stable 3x3 FHD 1856x1248 3. which firmware & tutorial is for the most stable Crop-Rec 3008x1080 4. which firmware & tutorial is for the most stable 3x1 and 5. which firmware & tutorial is for the most stable 1x3.

I know these questions might irratates you but PLEASE MAKE THIS CLEAR for user so we can better utilize as instructed, THANKS for any info you can clear up.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 18, 2019, 07:10:49 AM
Working on pixel binding 50D trying to remove & or fill that black bar (https://www.magiclantern.fm/forum/index.php?topic=19336.msg222150#msg222150) with data, but found something strange today .
I can some how get 1:1 pixel in 3x3 mode  :o
I set cmos[4] (horizontal pinning) on 50D cmos[2] on 5d2 & 5d3 to 3xcrop_mode
cmos[4]0x40c(default3x3) -->0x434 (3xcrop_mode 1:1)
Then I set the c0f06084 to 3xcrop_mode (default)
c0f06084 0x10045(default3x3) --> 0x1004a(3xcrop_mode default)
Finally I set adtg[12]1000 to default 3xcrop_mode
adtg[12]1000 0x6(default3x3) --> 0x05 (default 3xcrop_mode)
on other D4/5 cams you need to set adtg[1]100c (d4) or adtg[1]800c (d5) line skipping mode
but on the 50D it looks like you only need adtg[12]1000 to change line skipping to non line skipping
After setting raw image size to 4096x3166 @ 5.4 fps (locks up/freezes @ 8fps) ok at 7.5fps
I have a full height with near full width (I didn't want to go full width yet) and without any aliasing  8)
So it seem the 50D is a different kind of beast ! thou it similar to 5d2 & other D4 cams it has some
unique challenges .

(https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/VRAM27_small.png)

(https://i.ibb.co/d6Vdswn/M18-0012-00001-4096x3166-small.png) (https://imgbb.com/)
M18-0012_00001_4096x3166.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M18-0012_00001_4096x3166.png)
M18-0012_4096x3166_5.4fps_00001.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M18-0012_4096x3166_5.4fps_00001.dng)

100% zoom in mlvProducer viewer , no  aliasing
(https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/MLVProcduce_100Zoom_small.png)

200% zoom in mlvProducer viewer and again no aliasing
(https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/MLVProcduce_200Zoom_small.png)

Edit: here the MLV I was testing 4096x3166 @ 5.4fps  M18-0012_4096x3166_5.4fps.MLV (https://drive.google.com/file/d/1qMZ2nMMUnDuspu0OepLdgNs7QjMctobU/view?usp=sharing)
The upper portion before the black bar is equal to the liveview in 3x3 (1067 lines or vertical pixels) and in 3xcrop_mode the upper portion is again = to Liveview which is 1089 .
it seem this has something to do with the 50D video canon hack I think , but I can't proof it the silent simple dng's have the same black bar but the Full Res silent Dng's are fine , no black bar .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: SKecorius on November 18, 2019, 04:26:11 PM
Dear reddeercity (Dave?),
fantastic work! Thanks a lot for all your time and efforts. I have recently purchased well preserved 5D2, and now discover the world of Magic Lantern.
I would like to ask, if "magiclantern-crop_rec-3k_Updated_Center_4.20pm-5D2-eXperimental.2019Nov14.5D2212.zip" is the last working build that would allow me to use (assuming I have fastest lexar cf) the following:

1. continuous 1044 raw at 23.9 fps
2. 12-bit and 10-bit full 1080 raw at 23.9 fps
3. higher resolution (3K, 2.5K) using cropping, at 12-bit, 10-bit 23.9 fps, that would reduce moire and aliasing
4. dual iso video at 23.9 fps (I saw some videos online that say dual iso can only be used for static shots (due to ghosting)).

I am eager to test the mentioned functions and perhaps to conclude my experience in a short report here in forum.

I would also like to ask:

a) what is the biggest capacity CF that will work on 5DmII with ML?
b) how far is your eXperimental 2019Nov14 build from becoming a nightly build?
c) do you have any other plans that are one the pipeline for 5DmII (e.g. improvements on the existing features, rather than adding new ones)?
d) is there a list of known issues that comes with your latest 2019Nov14 build?

Looking forward hearing from you.

Sincerely,
Simonas
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 19, 2019, 06:21:19 AM
Looking in to that black bar on 50d , and there is data there
(https://i.ibb.co/xLjzPk1/300-percent-zoom-in-small.png) (https://imgbb.com/)
300_percent_zoom_in.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/300_percent_zoom_in.png)

M19-0024_00001_black_bar_full-res_4448x246.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M19-0024_00001_black_bar_full-res_4448x246.png)

so I thinking now that there's a corrupted & or incorrect address for that raw stream part of the sensor
If I'm not mistaken , there 3 different stream address , from the "consts.h"
consts.h#lines-44 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/src/crop_rec_4k_5D2/platform/50D.109/consts.h#lines-44)
#define YUV422_LV_BUFFER_1 0x41B00000
#define YUV422_LV_BUFFER_2 0x5C000000
#define YUV422_LV_BUFFER_3 0x5F600000

maybe buffer_2 ? is the problem
then I see this
consts.h#lines-54 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/src/crop_rec_4k_5D2/platform/50D.109/consts.h#lines-54)
// not 100% sure, copied from 550D/5D2/500D
#define REG_EDMAC_WRITE_LV_ADDR 0xc0f26208 // SDRAM address of LV buffer (aka VRAM)
#define REG_EDMAC_WRITE_HD_ADDR 0xc0f04008 // SDRAM address of HD buffer (aka YUV)

the LV addr is from the 5d2 etc... so how can I verify these address to be 100% correct
I never look for addresses before , so not too sure how fine them .

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: histor on November 19, 2019, 11:15:03 AM
No, that looks like actual data from the sensor. I get such bias frames adjusting analog iso register (between the main steps) (and I successfully use them as bais frames). Maybe you managed to set different iso for this part )
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 20, 2019, 06:19:41 AM
Ok I think I finally found on to fix the black bar (kind of) or I should say I found how to change the lower half
to black out a portion or the lower screen just like the upper half , see the raw image dumps below
So it looks like the ADTG1[1179]N
Controls the lower portion of the screen
(https://i.ibb.co/hgzC8Rc/RAW-014-small.png)     (https://imgbb.com/) (https://i.ibb.co/chjbyWn/RAW-015-small.png) (https://imgbb.com/)
ADTG1[1179]N 0x53A ->0x1A5 , black the whole lower portion                      ADTG1[1179]N 0x53A ->0x5BA       
RAW-014.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-014.png)                                                                                                         RAW-015.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-015.png)
(https://i.ibb.co/YNhcznS/RAW-016-small.png)     (https://imgbb.com/) (https://i.ibb.co/0mLscpm/RAW-017-small.png) (https://imgbb.com/)
ADTG1[1179]N 0x53A ->0x845                                                                   ADTG1[1179]N 0x53A ->0x9BA
RAW-016.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-016.png)                                                                                                            RAW-017.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-017.png)
(https://i.ibb.co/BZ4PzPY/RAW-018-small.png)     (https://imgbb.com/) (https://i.ibb.co/CV9tQ5T/RAW-019-small.png) (https://imgbb.com/)
ADTG1[1179]N 0x53A ->0x5A4                                                                   ADTG1[1179]N 0x53A ->0x45B
RAW-018.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-018.png)                                                                                                        RAW-019.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-019.png)

So I just need to find the upper portion ADTG1 Reg now .

Edit: Also here a very short MlV 450MB 4k-Dual_ISO_50D_M19-2349.MLV (https://drive.google.com/file/d/1zus1BD51IvxQ2XJ6wZKcV0actZQ6QicV/view?usp=sharing) -- found out that cmos[0] enables it ,
not to sure what recovery ISO is the base was 800ISO , it by luck I found it I was looking for reg's for the black bar. 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 21, 2019, 05:34:56 AM
Getting somewhere now on the 50D (black bar problem) seems I need to use 2 different "ADTG" Regs
adtg1[1172]0x665 -->0x465
adtg1[1178]0x665 -->0x465


(https://i.ibb.co/tKvnn4V/VRAM39-small.png) (https://imgbb.com/)

(https://i.ibb.co/1n13d2n/RAW-022cc-small.png) (https://imgbb.com/)
4134x2593_RAW-022.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4134x2593_RAW-022_cc.png)
So now I can get vertically 1950 out of the 2593 so 643 lines at the bottom ,
much better now at least I can get now 3100x1300 in 5xZoom

(https://i.ibb.co/zF9SpDr/RAW-022-croped-4108x1950-small.png) (https://imgbb.com/)
4134x2593_RAW-022_croped_to_4108x1950.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4108x1950_RAW-022_croped_4108x1950.png)
I just cropped off the bottom to view it without the black bottom .
I still need to find the other reg's to let me have full view height .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on November 21, 2019, 03:17:23 PM
Nice progress here @reddeercity ! I still need to try the last build, but the first one work well.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: DeafEyeJedi on November 21, 2019, 09:55:48 PM
I seem to have better luck with the latest build as opposed to the older one for the 50D. Been a few weeks since I've played.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Macke1 on November 22, 2019, 08:07:21 PM
New Update: November 14/2019
3008x1080 @ 23.98 fps Updated Corrected Horizontal Centering

I can't get this work on my 5d markii, has someone got it to work?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on November 23, 2019, 02:30:45 AM
Dave can You upload Your 5D2 build with source? Id like to tinker a bit with CMOS values for centering .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 23, 2019, 06:53:36 AM
Quote from: Macke1 on November 22, 2019, 08:07:21 PM
New Update: November 14/2019
3008x1080 @ 23.98 fps Updated Corrected Horizontal Centering
I can't get this work on my 5d markii, has someone got it to work?
All builds that I post are tested on my 5D2 & 50D , I don't post all thing that doesn't work or hasn't been test first by me
and I'm the one that coded it .

You need to explain you're self better , maybe some screen shots etc. .....
I can't help you if you don't help me by explaining the problem in detail .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 23, 2019, 07:30:09 AM
Quote from: SKecorius on November 18, 2019, 04:26:11 PM
Thanks a lot for all your time and efforts. ......
I would like to ask, .....
#1) yes 10bit  and maybe 12bit
#2) Yes in 3xcrop_mode , in 3x3 you can record 10bit 1856x1248 @23.976fps -- there no 1920x1080 in 3x3 or FHD (Full HD)
#3) there is no moire and aliasing in crop_mode it's 1:1 pixel no binding or line skipping . 3k is just a extension of 3x crop_mode.
#4) no , dual iso has been working on video for the last few years , I have a build on the first post of this thread that works with
all the crop_rec & non crop_rec video modes but it will not work on the nightly builds just my builds work with it  dual_iso_video_5D2.mo (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dual_iso_video-updates_8-21-2019.zip)

#a) I use 64GB but I hear 128GB cards work on the 5D2
#b) I don't think it will ever make the nightly , it still needs lossless compression implemented (it's close but missing something to be full functional)
More then likely the preset crop_rec will be obsolete soon , it will be replace with a resolution base interface , meaning you can
select any resolution and it will pick the max frame rate for that size e.g. 5632x3744 =5fps or 3008x1080 = 24fps etc. .....   
#c) No , I have not plan to maintain the basic feature set unless there something I need for me (I shot news stories , short docs & the odd low budget feature films)
I only have plans to implement Lossless compression & CF Card interface over clocking ( for higher write speeds)
#d) yes , read the first post of this thread (https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 23, 2019, 07:47:48 AM
On 50D got vertical res working in 3xcrop_mode more then the base 1078 , I can go up to 1950
so I did a test and I can now get 3136x1302 @ 24.003fps , but the bad new is if it's like the 5D2 (adtg reg's hooks don't work)
so I guess the next step is trying to get the crop_rec module to build for the 50D so I can code these preset in to it

(https://i.ibb.co/WDzcCcQ/3136x1302-24fps-M23-0028-000000-small.png) (https://imgbb.com/)
3136x1302_24fps_M23-0028_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3136x1302_24fps_M23-0028_000000.png)

(https://i.ibb.co/fMsDfxY/VRAM43-small.png) (https://imgbb.com/)

So the adtg1[1172] & adtg1[1178] are the key regs to get larger vertical resolution then liveview e.g. 1067 or 1078
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on November 23, 2019, 11:31:34 AM
Quote from: reddeercity on November 23, 2019, 07:30:09 AM
#1) yes 10bit  and maybe 12bit
#2) Yes in 3xcrop_mode , in 3x3 you can record 10bit 1856x1248 @23.976fps -- there no 1920x1080 in 3x3 or FHD (Full HD)
#3) there is no moire and aliasing in crop_mode it's 1:1 pixel no binding or line skipping . 3k is just a extension of 3x crop_mode.
#4) no , dual iso has been working on video for the last few years , I have a build on the first post of this thread that works with
all the crop_rec & non crop_rec video modes but it will not work on the nightly builds just my builds work with it  dual_iso_video_5D2.mo (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dual_iso_video-updates_8-21-2019.zip)

#a) I use 64GB but I hear 128GB cards work on the 5D2
#b) I don't think it will ever make the nightly , it still needs lossless compression implemented (it's close but missing something to be full functional)
More then likely the preset crop_rec will be obsolete soon , it will be replace with a resolution base interface , meaning you can
select any resolution and it will pick the max frame rate for that size e.g. 5632x3744 =5fps or 3008x1080 = 24fps etc. .....   
#c) No , I have not plan to maintain the basic feature set unless there something I need for me (I shot news stories , short docs & the odd low budget feature films)
I only have plans to implement Lossless compression & CF Card interface over clocking ( for higher write speeds)
#d) yes , read the first post of this thread (https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476)
#4b Is the current 10-bit and 12-bit video in 5d2 achieved without lossless compression? By truncated lower bits?

Sent from my Lenovo TB-8703F using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: alexboum on November 23, 2019, 11:49:22 PM
Quote from: zcream on November 23, 2019, 11:31:34 AM
#4b Is the current 10-bit and 12-bit video in 5d2 achieved without lossless compression? By truncated lower bits?

Sent from my Lenovo TB-8703F using Tapatalk

Yes read this post dedicated to the 10/12bit développement:
https://www.magiclantern.fm/forum/index.php?topic=5601.0
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 25, 2019, 08:49:42 AM
Looking like I'm very close to 1x3 (1376x1776) on the 50D some of the edges are a little jagged yet ,
my need a some more adtg_reg's like "adtg1[100c]"
but here's the first sample for the pixel peeper  :D

(https://i.ibb.co/vx2Lb66/4128x1776-24fps-M25-0243-frame-3-small.png) (https://imgbb.com/)
4128x1776_24fps_M25-0243_frame_3.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4128x1776_24fps_M25-0243_frame_3.png)

(https://i.ibb.co/m6wys8q/M25-0243-frame-0-small.png) (https://imgbb.com/)
1376x1776_M25-0243_frame_0.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1376x1776_M25-0243_frame_0.png)

If any one what to try to get the same results here the reg's (note it takes time , and can't be rushed , adjust reg very slooooowly )  ;)
(https://i.ibb.co/0JVtmky/VRAM46-small.png) (https://imgbb.com/)

Edit: this is the max resolution for 1x3 @ 24fps ,
thou it is continuous recording @75MB/s , good that the 50D can record at 80MB/s  8)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 27, 2019, 05:00:53 AM
Found the Liveview preview reg's for the 50D in 3x3 for 1x3 (1376x1777 @ 24.010fps)
(for the 50D you don't have to be in 3xcrop for it to work)

(https://i.ibb.co/tZzpYqf/VRAM52-Lv-preview-regs-small.png) (https://imgbb.com/)
C0f08184 = Height
c0f08188 = Width
C0f0d014 = Height & Width

the only difference from the 5D2 is "c0f08518" is = to raw height & width where as "c0f0d014" is the reg in 50D

I came upon it by chance  :) , just look for the height & width in hex ( that is the total raw area including the offset)
c0f0d014 = 0x43b0680 (default 3x3)
0x43b = 1083 (Decimal Height Value)
0x680 = 1664 (Decimal Width Value)

So to do the math ,
Lv height in 3x3 is 1056+27 offset = 1083
Lv width in 3x3 is 1591+73 offset =  1664
and mlv_rec uses mod8 rules so the closes we can get is 1568 .

I connected my Atomos Ninja hdmi recorder to the 50D with 1x3(1376x1777)
Got a good clean(mostly  :D) footage , wanted to see if the HD VRAM is good to monitor or capture .
Some work still needs to done , the early results look really good though some of the raw footage is getting corrupted  .
so I need to find out why to use this .

In Standby Mode
(https://i.ibb.co/q5mJyxq/Ninja-hdmi-50-D-screenshot-small2.png) (https://imgbb.com/)
footage from Ninja_hdmi_50D_720P_00240.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/Ninja_hdmi_50D_720P_00240.mp4) ,

While record raw @ 1376x1777 24.010 fps
(https://i.ibb.co/QDHZ1HV/Ninja-hdmi-50-D-screenshot-small1.png) (https://imgbb.com/)
originally it was 1080i (8bit 4.2.2 uncompressed) so I did a reverse telecine form 3-2 pull-down (https://en.wikipedia.org/wiki/Three-two_pull_down) (extracted the 24p frames from 60i stream)
then I resized it to 720p for a small file size conveniences .

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on November 27, 2019, 09:35:52 PM
Quote from: reddeercity on November 25, 2019, 08:49:42 AM
Looking like I'm very close to 1x3 (1376x1776) on the 50D some of the edges are a little jagged yet ,
my need a some more adtg_reg's like "adtg1[100c]"
but here's the first sample for the pixel peeper  :D

(https://i.ibb.co/vx2Lb66/4128x1776-24fps-M25-0243-frame-3-small.png) (https://imgbb.com/)
4128x1776_24fps_M25-0243_frame_3.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4128x1776_24fps_M25-0243_frame_3.png)

(https://i.ibb.co/m6wys8q/M25-0243-frame-0-small.png) (https://imgbb.com/)
1376x1776_M25-0243_frame_0.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1376x1776_M25-0243_frame_0.png)

If any one what to try to get the same results here the reg's (note it takes time , and can't be rushed , adjust reg very slooooowly )  ;)
(https://i.ibb.co/0JVtmky/VRAM46-small.png) (https://imgbb.com/)

Edit: this is the max resolution for 1x3 @ 24fps ,
thou it is continuous recording @75MB/s , good that the 50D can record at 80MB/s  8)
This is great news. The Resolution is closer to 2.7k and the human eye is less sensitive to reducing horizontal resolution.
We used to have 1440x1080 and not many could discern the difference with 1920x1080

Sent from my Lenovo TB-8703F using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on November 28, 2019, 06:31:32 AM
Quote from: zcream on November 27, 2019, 09:35:52 PM
This is great news. The Resolution is closer to 2.7k ......
Close , it really = to FF(full frame) 3.2K (it that's what you are referring to)
The crop factor on the 50D is 1.6 of FF as everyone know (APS-C) ,
with the 1x3 -> 4124x1776 the crop factor = to 1.15 of the full 50D sensor (4752 width)
so we get a crop factor of 1.75 of FF (full frame)  , 5D2 sensor = 5632 width (FF)
so 5632/1.75 = 3218
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on November 28, 2019, 09:04:29 AM
In practice 1376 px out of 4128 px columns will give about 2.7k resolution (as row resolution is unchanged). However as you are squashing horizontal it is less noticeable.
It's great news however you look at it. This is closest to 4k dci we can have.

Sent from my Redmi 4A using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on November 30, 2019, 06:56:30 PM
@reddeercity, I just bought a "new" 5d2. It works a lot better than the previous one which is died, unfortunately... so I'm exiting to test and get into some new stuff, hehe :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 05, 2019, 07:01:44 AM
Quote from: zcream on November 28, 2019, 09:04:29 AM
...... However as you are squashing horizontal it is less noticeable.
Sorry incorrect , I'm increasing vertical so the horizontal is unchanged , it all has to do with the horizontal pinning mode .
the only reason I'm at 1376 instead of 1568 is the sensor can't do 1568 @ 24fps with the vertical at 1776.
so there 4 different pinning mode know at the moment
1:1 (x5 Zoom)
3x3 = default FHD mode
3x1 (full res horizontal 1x1 with vertical line skipping) (same as the default 3x3 vertical)
1x3 (horizontal columns every third same as 3x3 horizontal with 1:1 vertical)
The field of view in 1x3 = 4128 from 4755 , so as I said before it's = to 3200 in full frame .
And the math supports this  ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 05, 2019, 08:16:51 AM
50D Fine tuning the 1x3 reg's , I'm starting with 5x Zoom this time instead of 3x3 because I can't clean edges without the ladder stairs step thing
Success ! the ladder stairs issue is gone , I have now clean edges  :D

(https://i.ibb.co/HVN58T6/M04-2318-frame-93-small.png) (https://imgbb.com/)
4128x1776_AR-2.35_1x3_M04-2318_frame_93.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/4128x1776_1x3_M04-2318_frame_93.png)

I also got the HD liveview working with a clean image that can used with HDMI EVF , 1080i give a the odd pink frame but forcing HDMI to VGA (780x480)
there not pink frames or corruption in the raw frames , I tested it with my Zacuto Z-Finder EVF Pro (https://www.zacuto.com/zfinderevf) even has a anamorphic setting will "un-squeeze"
mode (forgot to try it) next time .

From the image_dump
(https://i.ibb.co/vBmgFrx/HD-024-small.png) (https://imgbb.com/)
HD-024.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/HD-024.png)

I set the anamorphic mode in the display menu and it save it in the image _dump , cool
(https://i.ibb.co/Wp4xDM7/LV-024-small.png) (https://imgbb.com/)
LV-024.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/LV-024.png)

These the screen shot from LCD with anamorphic set to 2:1
(https://i.ibb.co/5RSRfyx/VRAM59-small.png) (https://imgbb.com/)

Here screen shot of Reg's  I used from adtg_gui , if anyone  is brave enough give it a try ,
remember if you break your cam you get to keep the pieces  :P
(https://i.ibb.co/WKvPYsP/VRAM57-small.png) (https://imgbb.com/)
(https://i.ibb.co/3sPVX1N/VRAM58-small.png) (https://imgbb.com/)
There's 2 reg's that I used that are not on the screen shot
adtg12[1000] 0x5 -> 0x6
adtg1[100c] 0x2 -> 0x0

I also reduced the rolling shutter in reg c0f06008
here a short 5 second mp4 @ 4128x1776 24fps 50D_4128x1776_24fps_M04-2318.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_4128x1776_24fps_M04-2318.mp4) in 5x Zoom

Also here footage from my atomos ninja , I resize it to 720p from 1080i
00241.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/00241.mp4)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 06, 2019, 10:05:11 AM
With the 50D discovery of the 3x3 Liveview reg's , I got my 5D2 working with the HDMI out in 3x1 4k  8)
c0f08184
c0f08188
c0f0d014

Will it not totally clean but a start , I need to find the HDMI reg's for size & reduce the vertical
It only work with HDMI -> forced to VGA (780x480) not 1080i , still that's good enough to monitor used my HDMI Zacuto Z-Finder EVF Pro to test .

Here a short 1 min. video I shot with my iPhone showing HDMI & 4k 3x1 raw video recording at the same time
5D2_Liveview_3x1_4k_IMG_2871.MOV (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2_Liveview_3x1_4k_IMG_2871.MOV)

Below are screen shots from my iPhone
(https://i.ibb.co/BzwRYHP/vlcsnap-2019-12-06-01h31m20s903.png) (https://imgbb.com/)  (https://i.ibb.co/HCctvMX/vlcsnap-2019-12-06-01h31m29s233.png) (https://imgbb.com/)
(https://i.ibb.co/VN6fdt7/vlcsnap-2019-12-06-01h31m38s258.png) (https://imgbb.com/)   (https://i.ibb.co/zn4NmY4/Untitled.png) (https://imgbb.com/)

In theory  this should work with 3x3 & 5x Zoom with all D4 & D5 Cams  :D .

Edit: Here the Image_dump for HD(HDMI) & Lv
5D2_hdmi480p_dump_HD-013.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2_hdmi480p_dump_HD-013.png)  and   5D2_ml_preview_LV-014.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2_ml_preview_LV-014.png)
(https://i.ibb.co/3M2ZmQd/HD-013-small.png) (https://imgbb.com/)  (https://i.ibb.co/0YpvZNZ/LV-014-small.png) (https://imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 06, 2019, 02:18:28 PM
So waiting to test that on 5d2...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 07, 2019, 05:04:41 AM
Quote from: Igor_Braun on December 06, 2019, 02:18:28 PM
So waiting to test that on 5d2...
You don't have to wait for me , give it a try with 3.5k centered presets 3008x1080 or 2784x1160
I tested it on 3008x1080 centered (latest upload Nov.14/2019) and yes indeed it dose work

I shot a quick 1 min. video with my old iPhone 
5D2_HDMI_1080i_crop_rec_3k_IMG_2882.MOV (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2_HDMI_1080i_crop_rec_3k_IMG_2882.MOV)

Note it's the center crop 1920x1080(HDMI) of the 3008x1080 Liveview , I like to figure out how to resize hdmi to get the full width
from 3008x1080 -> HDMI 1920x1080 , the aspect ratio is 2.78 so 1920x690 .
Need to find more reg to do with the HDMI out put , or is this controlled by the dryOS of canon ?
I do know that the HD image_dump is used by canon for the HDMI , so it a matter of finding them I think  ::)

Edit: some screen captures , first 1 is in stand by mode , next is recording raw video (3008x1080)
(https://i.ibb.co/0s4qdb4/crop-rec3k-hdmi.png) (https://imgbb.com/)   (https://i.ibb.co/Q9z5QQt/crop-rec3k-hdmi-recording-raw.png) (https://imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 07, 2019, 01:46:16 PM
Yes, i've already tested it with HDMI monitor, and got non-full width preview  :)

But what i'm really waiting for is 3x1 4k anamorphic full-width preview... The last time I tried to HDMI preview with 4k it just didn't work.
If i can try to do it myself without programming (maybe changing and figuring out some values even if i could spend a tonn of hours doing this) please tell me how and where. And maybe i could help you with it   ;)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on December 07, 2019, 07:59:28 PM
Quote from: Igor_Braun on December 07, 2019, 01:46:16 PM
But what i'm really waiting for is 3x1 4k anamorphic full-width preview...
I more still wait for 1x3 mode for 5D2, because this would be the real killer mode. There is no advantage with 3x1: you still have moiree, because we still skip lines. With 1x3 you don't have such moiree.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 09, 2019, 06:21:40 AM
Quote from: masc on December 07, 2019, 07:59:28 PM
I more still wait for 1x3 mode for 5D2, because this would be the real killer mode.
If it was simple to implemented , I would have done it a long time ago .
short answer , adtg reg hooks don't work which is needed for 1x3 , and I said as much mouths ago .

a1ex maybe the only one that knows how solve this problem , so don't hold you're breath
It I can solve it great , then I can get 1x3 up & running , but there's a very high possibility
that 50D will get 1x3 before the 5D2 as the adtg hooks seems to work ok .

Quote from: masc on December 07, 2019, 07:59:28 PM
There is no advantage with 3x1: you still have moiré, because we still skip lines. With 1x3 you don't have such moiré.
What are you talking about ? There major advantages , e.g. 4k resolution and using every column not skipping every third .
yes there a very small amount of moiré but far far less the 3x3 and there no substitute for resolution period .
The thing that I can't stand is when people upscale HD or 2k footage to 4k , it looks so very bad I never seen a good one to date !
and a FYI , there's absolutely no moiré artifact in 1x3 at least on the 5d2 , I've pixel peep the test image maybe on other 1x3 cams there is e.g. eosm or 100d .

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 09, 2019, 06:52:28 AM
Quote from: Igor_Braun on December 07, 2019, 01:46:16 PM
But what i'm really waiting for is 3x1 4k anamorphic full-width preview...
Ok , got it working now but it still the center crop 1920 of 4096 , the image is clean now
there still some distortion at the bottom half of the screen , it dose not affect the 4k raw video stream .

A couple of photo from my iPhone , first one is in standby mode & the next is recording raw video

(https://i.ibb.co/RB20g9Q/IMG-2883-small.png) (https://imgbb.com/)  (https://i.ibb.co/XjMFN4c/IMG-2893-small.png) (https://imgbb.com/)

Here are the 5D2 Preview Reg's for 4k 3x1 realtime with HDMI 1080i force to  -> VGA(480P)
this is the only way it works , it dose not work with 1080i HDMI ,
(https://i.ibb.co/sWSZnS2/VRAM6-small.png) (https://imgbb.com/)
C0f0d030 0x10 ->0x20 this is the one that clean the distorted image to what you see here in this post found it by luck !
just started to adjust different preview reg's until the image change for the better or worse  ;)

I also shot a short 45 second video with my iPhone to show this working 
5D2_3x1_4096x592_24fps_HDMI_Realtime_preview_IMG_2891.MOV (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2_3x1_4096x592_24fps_HDMI_Realtime_preview_IMG_2891.MOV)

Next I'll code this in to the 4k 3x1 crop_rec preset in the next few days .  :D
So now we will have realtime preview while recording raw video , no more frozen liveview preview
so you can actually focus & follow subject while recording raw video .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on December 09, 2019, 11:00:51 AM
Quote from: reddeercity on December 09, 2019, 06:21:40 AM
What are you talking about ? There major advantages , e.g. 4k resolution and using every column not skipping every third .
Quote from: masc on December 07, 2019, 07:59:28 PM
...you still have moiree, because we still skip lines.
As long as we skip, there will be moiree. Also in 3x1 4K. Also this mode will not look better than upscaling 2K. (3x1 4K is also upscaled)

1x3 does not skip. It has column binning. And this is the difference. We than use (not save) every pixel from the sensor for the final image.

Quote from: reddeercity on December 09, 2019, 06:21:40 AM
If it was simple to implemented , I would have done it a long time ago .
...
and a FYI , there's absolutely no moiré artifact in 1x3 at least on the 5d2...
Nobody told it was simple.
But this mode would be by far the most intersting update for 5D2. I thought you found the regs around a year ago already (didn't you write that? Dez2018) and it is "just some" implementation work to be done?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 09, 2019, 11:02:39 PM
Masc, i can mistake, but just think that the horizontal resolution is more important even if there's moire
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 09, 2019, 11:03:18 PM
Reddercity, i will be waiting for the preset, hope it will work for me as for you, because i have diy HDMI monitor
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on December 10, 2019, 07:19:46 AM
Quote from: Igor_Braun on December 09, 2019, 11:02:39 PM
Masc, i can mistake, but just think that the horizontal resolution is more important even if there's moire
With that you'll skip each 3rd line. This data is NOT in the image. Columns are binned when reading each 3rd: data is in the image! So 1x3 is better. You easily see it in all examples in this and other threads.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 10, 2019, 07:53:25 AM
1x3 is very low on my list of priority (at the bottom) on the 5D2 so don't ask , it will happen when happen not moment sooner.
and by the way there's data lost both ways , the only time there's not is in 1:1 pixel mode ! So don't be so hung up on 1x3 .

Will here we go  :) HDMI  480p realtime preview in 4k 3x1 , I made a few extra tweaks to the 4K preset
I made it true 4k (4096x592) @ 24.007fps so this is close as I can get to a 24fps film camera   :D

crop_rec-4k_hdmi480p-preview_10.33PM-5D2-eXperimental.2019Dec09.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-4k_hdmi480p-preview_10.33PM-5D2-eXperimental.2019Dec09.5D2212.zip)

Usage: first switch the HDMI from 1080i (the default) to "force VGA"(480p) in the display advanced menu .
always start in 3x3 without the HDMI connected , then plugin the cable , swicth to  non crop_rec  x5zoom mode (should be 2144x1078)
then enabled crop_rec select 4k preset , and cycle the zoom button until you get to 3x1 4k .

It doesn't work with the 2784x1160 yet , but 48p 3x3 works but you have to start over from the beginning then just enable the 48p preset instead of the 4k one .
but make sure you unplug the HDMI cable first to flush the HDMI buffer .

It look like I found most of the HDMI (HD image) Reg's from this , they all have the letter "D" in there name .
I play around a bit and I can get the HDMI image to move down on the screen (trying to center it on the monitor screen but the rest of the image get distorted
need to find more  reg , plus think I have identified the reg that limits HDMI to 1650 horizontal  on the 5D2  , ( which being goal of mind right from the beginning)

As always if you break your camera you get to keep all the pieces    ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on December 10, 2019, 11:44:45 AM
Quote from: reddeercity on December 10, 2019, 07:53:25 AM
1x3 is very low on my list of priority (at the bottom) on the 5D2 so don't ask , it will happen when happen not moment sooner.
...
I made it true 4k (4096x592)...
Sure, the best always comes last.
True 4K...  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 11, 2019, 04:13:35 AM
It worked for me!
Cool.
This is the picture where i kept 3 things together:
1 - Full Frame image with regular h264 mode
2 - 4k anamorphic mode from the same camera position inside the white border
3 - The area I saw via HDMI monitor in the blue border

(https://i.ibb.co/7kTCKwq/Screenshot-96.jpg)
https://ibb.co/7kTCKwq

It worked this way (phone video): https://drive.google.com/file/d/1FcdwtZUzQdPfIoX1151yZzOBU0jHkWas/view?usp=sharing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 11, 2019, 04:35:42 AM
seem the vertical is down , a little bit from center I'll see if I can move that up a bit .
Also You should have a clean standby mode , looks weird from the video .
I post a short video later tonight hopefully, on how I set it up .
On HDMI I get the center on 4k anamorph not to the left as you posted
but I'll check in to that,  not sure if there's anything I can do about it .

Edit: Ok you're right the HDMI is to the left of center on 4k Anamorphic
Strange 5xZoom 2784x1160 is centered in the hdmi , I have to do some more investigation to see if I can center it horizontally
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 11, 2019, 05:56:13 AM
Okay :)
This semi-white blinking preview appeared when I updated the camera to your latest build. With the regular nightly build the standby mode is clear.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 14, 2019, 10:04:48 PM
@Masc Man, I think true 4k is when you have true 4k pixels in a line (?), here we have 4096.
and the rolling shutter if understand right must be less in this mode because there are only 592 vertical lines.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on December 14, 2019, 10:29:05 PM
Quote from: Igor_Braun on December 14, 2019, 10:04:48 PM
@Masc Man, I think true 4k is when you have true 4k pixels in a line (?), here we have 4096.
and the rolling shutter if understand right must be less in this mode because there are only 592 vertical lines.
Yes... you're right. The definition tells 4000x1 is also 4K, while I would understand resolutions like 4096×2160 as "true 4K" (5D2 will never record true 4K RAW at ~24fps). When counting pixels 4096*592 (3x1) would be the same like 1365*1776 (1x3). But one produces much more moiree than the other (sry... I told now more than once). That's why this should be the last research in this 3x1 readout type, which has been mostly discontinued for other cams since some time, for getting the way better 1x3. Anyway... nice to see some progress on the old 5D2.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on December 15, 2019, 12:04:45 AM
Igor there was a lot of tests and examples/comparisons between 3x1 and 1x3 , masc is right , tall anamorphic 3x1 has better final resolution than wide anamorphic 1x3  because of skipping which produces nasty moire and aliasing vs binning which interpolates nicely.
Its beating dead horse, sadly its how these cameras are made, alex wrote that the solution would be writing binning patterns which are probably hardcoded.
Quote1x3 Read all lines/Bin every 3 columns. No skipping at all only Binning pixels in columns, no aliasing/no moire and better details than 3x3.

3x1 Read one line skip two every 3 lines/Read all columns. Better details than 3x3 but with same alisaing and moire since we are skipping lines in this mode.
Read this: ( should be sticky somewhere IMO cause this topic is important part of ML history)
https://www.magiclantern.fm/forum/index.php?topic=16516.0
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 15, 2019, 05:46:41 AM
Quote from: 2blackbar on December 15, 2019, 12:04:45 AM
Its beating dead horse .......
There no 1x3 on 5D2 just 3x1,  so you're wasting you're  time trying to compare 1x3 to 3x1 here .
And it may never be , to found out why just read this thread , I've posted many times on this
kind of sick about hearing about it , If you what to solve the problem figure out why adtg hooks stop working
in the adtg_gui.c adtg_gui.c#lines-5923 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/src/crop_rec_4k_5D2/modules/adtg_gui/adtg_gui.c#lines-5923)
crop_rec/crop_rec.c#lines-2307 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/src/crop_rec_4k_5D2/modules/crop_rec/crop_rec.c#lines-2307)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 15, 2019, 06:01:24 AM
Got a better Liveview for 5D2 4k 3x1 HDMI now , almost the full height in the screen of the EVF/monitor but I still can't get the image to
move to the center for center crop so just the left side at the moment , and it seems the lines are like dual ISO but in cluster of 3
still need more research to hdmi reg's . This does not effect the raw video stream in anyway just Liveview.
Reg that increased the vertical c0f11304 0x10 ->0x15

(https://i.ibb.co/42nQp5s/LV-018-small.png) (https://imgbb.com/)

This is the normal liveview of 4k 3x1 without the modified reg's

(https://i.ibb.co/16rqqY1/LV-016-small.png) (https://imgbb.com/)

Batteries are dead on the 5d2 & hdmi evf so I'll continue tomorrow .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on December 15, 2019, 10:34:04 AM
Quote from: reddeercity on December 15, 2019, 05:46:41 AM
There no 1x3 on 5D2 just 3x1,  so you're wasting you're  time trying to compare 1x3 to 3x1 here .
Really?

Quote from: reddeercity on December 21, 2018, 06:09:21 AM
Experimenting with line skipping & pixel binding (1:1 FHD) with 1x3 to reduce moire and aliasing

Un-stretch 1856x416
(https://i.ibb.co/g6H06ZY/1856x416-1x3-small.png) (https://imgbb.com/)
1856x416_1x3.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1856x416_1x3.png)

Here the stretch version 1856x1248
(https://i.ibb.co/JnKqH8F/M20-1741-000000-small.png) (https://imgbb.com/)
M20-1741_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M20-1741_000000.png)
M20-1741_000000.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M20-1741_000000.dng)

setting
(https://i.ibb.co/Wp7jcmf/VRAM41-small-a.png) (https://imgbb.com/)

Short 2Mb h264 clip - M20-1741.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M20-1741.mp4)

I don't really see this to be too useful , as it's too narrow (416 h) , if the vertical was 2400 etc.. then maybe , but record time would only be 1-2 seconds
if your lucky , so unless something changes , I won't be continuing with this 1x3 .
Aha. This could have been a nice starting point. Nearly the same height and +1.5K ( = 5.5K ) to your current 3x1 mode. Would there be the code for this revision and base idea, I am sure there would have been a lot of help making it real for the community. Processing this old DNG file with current RAW processors is just awesome.

Quote from: reddeercity on December 15, 2019, 05:46:41 AM
kind of sick about hearing about it
+1
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on December 15, 2019, 01:41:28 PM
So there is 1x3 mode and stretch for that is 5568x1248 not 1856x416
Once lossless raw recording would be finished then it would record longer like EOS M thats why i think lossless should have top priority so you cant say meh i only got 2 seconds uncompressed, with lossless it would record much longer in those high res modes with more data to push.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on December 15, 2019, 08:28:24 PM
Quote from: 2blackbar on December 15, 2019, 01:41:28 PM
... why i think lossless should have top priority so you cant say ...
Finding the right registers and values for such new functions is indeed not easy and you need much time, much effort and some luck. If you have more luck with one topic, I can understand doing more research there. I also tried to compile with those registers for 5D2 - I had success with compiling, but just got crashes with how I tried to integrate those registers. So I can also understand why someone does not want to go a certain way... It is not like implementing a new program.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on December 15, 2019, 08:47:52 PM
IMO there should be ability to hide ML menu when you change registers in realtime, it would speed up development by a lot.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on December 15, 2019, 08:54:29 PM
Quote from: 2blackbar on December 15, 2019, 08:47:52 PM
IMO there should be ability to hide ML menu when you change registers in realtime, it would speed up development by a lot.
Isn´t already in adtg_gui?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on December 15, 2019, 08:57:37 PM
Quote from: Danne on December 15, 2019, 08:54:29 PM
Isn´t already in adtg_gui?
Yes it is. But sometimes camera hangs up when entering those values.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 16, 2019, 03:58:07 AM
@masc , go slow start on something simple that only needs a couple of reg's , d4 cams are very sensitive to any
shock to the system , the reg need to be applied in stages.
Rule of thumb I start at very low frame rate , I do all my adjustments around 8-10fps .
A FYI : I've had all the 1x3 reg's plus preview for 5k(4992) ([email protected]) for the last year ,
but until the adtg hooks problem was solve no need to post anything about it . 

(https://i.ibb.co/g7Q4mzg/RAW-059-5016x1843-23-976-small.png) (https://imgbb.com/)
RAW-059_5016x1843_23.976.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-059_5016x1843_23.976.png)


(https://i.ibb.co/VJHbfY3/RAW-059-small.png) (https://imgbb.com/)
1672x1843-RAW-059.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-059.png)
1x3_1672x1843_24fps-RAW-059.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1x3_1672x1843_24fps-RAW-059.DNG)

Just a word on development to all newbies , just because there a focus on one thing at a time doesn't mean that discovery are not made all the time .
That's how I figured out the preview for HDMI at 4k 3x1 and all that knowledge gets applied to other preset .
And if a discovery or a flash of insite lead to lossless ok then , but that's how it goes sometimes , when you hit a brick wall on
one thing you make process on a another thing even if it something that was work on 2 years ago .

If one of the reasons I bought a 50D , it helps me understand the D4/5D2 cams better because it's less complicated , not so many rabbit holes  ;)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 16, 2019, 05:26:37 AM
New update for 5D2 3x1 4k HDMI 480p preview full screen now  :D
crop_rec-4k3x1hdmi480p-preview-5D2-eXperimental.8.08pm.2019Dec15.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-4k3x1hdmi480p-preview-5D2-eXperimental.8.08pm.2019Dec15.5D2212.zip)

How to setup
It a bit glitchy , Very first thing !!
HDMI must be switch to "Force ->VGA" (480P) before you enter in to 5xzoom !
and do not plug in the hdmi cable yet .
start off in non crop_rec  just the normal 5xZoom (3xcrop_mode) should be at 2144x1978
Then plug in the hdmi cable , should have a normal crop_mode image .
Next , press the trash can to enter in to crop_rec module , select 3x1 4k then press half shutter to enable
you should see this image below.

(https://i.ibb.co/PYkFFJb/vlcsnap-2019-12-15-20h54m13s163-small1.png) (https://imgbb.com/)

short 1 min video from my iPhone with this preset enabled & recording raw.
3x1_4k_hdmi_480p_full_screen_IMG_2895.MOV (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3x1_4k_hdmi_480p_full_screen_IMG_2895.MOV)

If you unplug hdmi cable you will need to refresh Liveview , you will have a black screen .
Press the play button to show a cr2 photo, press play again , then press the Info button until
ml overlay screen comes up ( but will have no image yet) then press trash can to bring up
the ml menu then disable Crop_Rec , press half shutter , it should be back to 3x3 1856x1248 .
Yea I know it's kind of a pain , but it a very new thing I found it will take time to smooth out the edges  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ZEEK on December 16, 2019, 01:44:31 PM
Great! New update works just fine :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on December 16, 2019, 08:46:58 PM
Quote from: reddeercity on December 16, 2019, 03:58:07 AM
@masc , go slow start on something simple that only needs a couple of reg's , d4 cams are very sensitive to any
shock to the system , the reg need to be applied in stages.
Rule of thumb I start at very low frame rate , I do all my adjustments around 8-10fps .
A FYI : I've had all the 1x3 reg's plus preview for 5k(4992) ([email protected]) for the last year ,
but until the adtg hooks problem was solve no need to post anything about it . 

(https://i.ibb.co/g7Q4mzg/RAW-059-5016x1843-23-976-small.png) (https://imgbb.com/)
RAW-059_5016x1843_23.976.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-059_5016x1843_23.976.png)


(https://i.ibb.co/VJHbfY3/RAW-059-small.png) (https://imgbb.com/)
1672x1843-RAW-059.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-059.png)
1x3_1672x1843_24fps-RAW-059.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1x3_1672x1843_24fps-RAW-059.DNG)
That looks great. Wow. What I have to do to get that? You also just entered some registers in adtg gui? And in which order? Or did you implement this in code testwise somewhere?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on December 16, 2019, 09:00:48 PM
lol its 5016x1843 anamorphic on 5D2  :D
How well it runs ? I suspect it wont record much even in 10 bits because of uncompressed ?
Can You share the build ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: RH42 on December 17, 2019, 05:44:24 PM
Quote from: reddeercity on December 16, 2019, 03:58:07 AM
@masc , go slow start on something simple that only needs a couple of reg's , d4 cams are very sensitive to any
shock to the system , the reg need to be applied in stages.
Rule of thumb I start at very low frame rate , I do all my adjustments around 8-10fps .
A FYI : I've had all the 1x3 reg's plus preview for 5k(4992) ([email protected]) for the last year ,
but until the adtg hooks problem was solve no need to post anything about it . 

(https://i.ibb.co/g7Q4mzg/RAW-059-5016x1843-23-976-small.png) (https://imgbb.com/)
RAW-059_5016x1843_23.976.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-059_5016x1843_23.976.png)

(https://i.ibb.co/VJHbfY3/RAW-059-small.png) (https://imgbb.com/)
1672x1843-RAW-059.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/RAW-059.png)
1x3_1672x1843_24fps-RAW-059.DNG (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1x3_1672x1843_24fps-RAW-059.DNG)

Just a word on development to all newbies , just because there a focus on one thing at a time doesn't mean that discovery are not made all the time .
That's how I figured out the preview for HDMI at 4k 3x1 and all that knowledge gets applied to other preset .
And if a discovery or a flash of insite lead to lossless ok then , but that's how it goes sometimes , when you hit a brick wall on
one thing you make process on a another thing even if it something that was work on 2 years ago .

If one of the reasons I bought a 50D , it helps me understand the D4/5D2 cams better because it's less complicated , not so many rabbit holes  ;)


Just amazed by the amount of potential that has been unlocked on these older cams - big thank you to @reddeercity and everyone else who has continued to push the boundaries!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 19, 2019, 03:02:36 AM
@reddeercity
Works well...
In this build in my case HDMI monitor gets the signal only in 1x zoom mode (then after the monitor switches on you need to go to x5 zoom)


And just in case again:
(https://i.ibb.co/NjsQdKs/positions.jpg) (https://ibb.co/hWyhJVy)
<a target='_blank' href='https://ru.imgbb.com/'>image host</a>

https://i.ibb.co/NjsQdKs/positions.jpg

The whole image is standard h264
Grey zone - is 4k recording zone
Blue zone is what i've seen through the HDMI

Also, I've tried to align recording zone vertically, fixing  hi and lo cmoses but didn't get the result, so i'm not sure i do it the right way

Edit:
Some videos shot on phone:
https://drive.google.com/file/d/1VKryn8bJag0e8h8Bu2ojMHrcTI4RiHjL/view?usp=sharing
https://drive.google.com/file/d/1sESwTTsWetm8dVMqwHNxDxBC5g_mlLB5/view?usp=sharing
https://drive.google.com/file/d/15CG1zm-gBCpFoegXJkj-I2xfIBxd7BRE/view?usp=sharing

@Mask and @2blackbar thanks for the explanations i'll check it all about the pixel binning methods
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 20, 2019, 08:09:43 AM
Ok , I'll check the vertical positioning in 3x1 4k . It should be easier then that to get monitoring in 3x1 .
I'll post a video on the setting etc.  ..... to get hdmi monitoring in the next few days hopefully
I've being busy trying to get 50D crop_rec out by xmas , but as always there's road blocks & rabbit holes slowing me down  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 21, 2019, 10:16:17 PM
BIG thanks  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ZEEK on December 23, 2019, 01:38:33 PM
@Reddeercity
Hey, I'm using the build for the 5D2:
November 14/2019 3008x1080 @ 23.98 fps Updated Corrected Horizontal Centering.
It's been working really awesome! Thanks! I have just come across two issues:
1.Mlv_Play Module appears as "error" when the module is loaded but works without it.
2. When you format the camera in the camera from Canon Menu, ML is removed completely (Even though it says if I would like to keep ML). Is there a way around this? 
Thanks heaps!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on December 24, 2019, 12:10:16 AM
Those are just facts of life. You can't have a camera this cinamatic AND playback.

Think about film cameras:
1. You can't play back what you shot until you process it
2. You can't format film
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ZEEK on December 24, 2019, 01:51:40 AM
@IIia3101
Ah right. Just thought I was doing something wrong! I can live with these standards! Thanks! 👍
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on December 24, 2019, 07:54:27 AM
New update for 5D2


magiclantern-crop_rec-4k-3x3-48p_new-centering_-5D2-eXperimental.11.27pm_2019Dec23.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-4k-3x1_3x3-48p_new-centering_-5D2-eXperimental.11.27pm_2019Dec23.5D2212.zip)

I corrected centering on 4k 3x1 & (3x3 48fps ,1856x774 which no one seem to notice that is was out more then the 4k  ??? )
please check all preset this helps me to know if I broke anything else , I do check everything before I upload , but sometimes I may miss something .
@Igor_Braun here you go , thanks for check the centering , I can't change the HDMI centering sorry , that's a lot of reverse engineering to figure it out and it may
not be doable , maybe a1ex has more insight  on the topic I haven't came across anything that works at the monument .

This will more then likely be the last update for 3x1 & 3x3 48p unless there some issue that needs to be addressed
I'll be focusing on getting ADTG hooks working in the crop_rec  along with lossless compression  and (over clocking CF card bus , hopefully) plus finish porting/compiling 50D crop_rec

merry xmas everyone  :D :D

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on December 24, 2019, 09:04:57 AM
2.7K and/or 3K build inclusiv?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on December 26, 2019, 08:27:19 AM
I would like to thank you for all the work that you have done reddeercity.

I have such a problem - on crop rec 4k anamorphic, and 48p, all I see on live view is a black screen.
Even going to the ML menu is then quite problematic, some of the buttons simply do not work.
I tried to refresh the screen in different ways, tried different settings, everything..

However, the video is being recorded ok, I even shot some clips in 48p slo-mo. I just don't see the preview when shooting.
Maybe there is some reason or am I missing something?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on December 26, 2019, 10:19:54 PM
@reddeercity, i understand. Thank you for your awesome work... Merry Christmas! Wishing you a magical holiday!
The day after tomorrow i'm coming back to my house and check the new build  :D :D :D

Edit: Everything works just fine in my case except of 3,5k profile. Think i missed something. Will try to do something with it and reply to you later :) 48p is very cool mode... In the meantime, i'm thinking of using mobile phone as a viewfinder for 4k mode..   :-\

https://youtu.be/vUcr2taVEGs

Hm... Also, i can connect hdmi monitor to this phone and make right framing there...

Question: What about adding full-resolution 5k with small fps mode? It may be very good for time-lapses or interior shooting using motorized sliders for example.

12.31.19
I got around 13 sec of 14-bit 4k raw with full range (without cropping) 1-2 fps squeezed preview in camera on Lexar 1066x with your previous build.
it,s really dope =)
Is there any way to in camera de-squeeze this preview image (3 times stretch vertically)?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 01, 2020, 05:21:38 AM
New up Date for 5D2 ,
crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip)

All crop_rec presets ( 4k 3x1 , 3x3 48p & 2.8k (2784x1160) are perfectly centered or as close I can get
It maybe a few pixel out , I check the centering in A.E. with a full CR2 (5616x3744) then overlay the crop_rec frame .

Updated the centered 3.5k preset (2784x1160) AKA as 2.8k
I reworked it to get a True 24p FPS (24.000) since the main clock is 24MHZ it runs cooler now too  :) plus it's also centered .

(https://i.ibb.co/R0Lv1HV/VRAM36-small.png) (https://imgbb.com/)

HDMI is working as long as you have  "HDMI->force to VGA" (480p) even if you don't have a HDMI cable plugged in
It's recommended to keep this enable for all crop_rec presets ( reduces the chance of pink/corrupted frames )

Happy New Year !! & happy testing  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 02, 2020, 06:08:05 AM
Quote from: Igor_Braun on December 26, 2019, 10:19:54 PM
Question: What about adding full-resolution 5k with small fps mode? It may be very good for time-lapses or interior shooting using motorized sliders for example.
Ok , I see the value in that -- give this a try  ,
a Very eXperimental  5632x2356 2.40 A.R. @ 4.221fps
crop_rec-5k-4fps-5D2-eXperimental.9.08pm.2020Jan01.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern/downloads/)

(https://i.ibb.co/vmkvCD3/5632x2356-M01-2227-000000-small.png) (https://imgbb.com/)
5632x2356_CC_M01-2227_000000.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/CC_M01-2227_000000.png)

It's at 4.22fps right now , I can push it to 6.5fps -- but I want to see if anyone is going to use it before I develop it further  , I'll know by how many downloads there is .
Same as before start in 3x crop_mode (2144x1078 29.66 fps default) then enable crop_rec 5k preset , Liveview is slow (4fps) .
you may have to cycle though the 5xzoom button , it may lockup of freeze liveview , if so just do a battery pull .
As before if it brick your camera you get to keep all the pieces . :P
 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on January 02, 2020, 01:16:35 PM
Hi, what about my problem? I can't see the preview in all modes except 1:1.
All I see on 48p and 3x1 anamorphic modes is just a black screen.
I tried everything I can, I read almost the entire topic from beginning to end, tried different builds - nothing helped.

Could this be because I use manual (non-chip) lens? Well, I don't even know.


Quote from: reddeercity on January 02, 2020, 06:08:05 AM
Ok , I see the value in that -- give this a try  ,
a Very eXperimental  5632x2356 2.40 A.R. @ 4.221fps
crop_rec-5k-4fps-5D2-eXperimental.9.08pm.2020Jan01.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-5k-4fps-5D2-eXperimental.9.08pm.2020Jan01.5D2212.zip)
I tried this build, works great! I'll definitely use this, because I've been shooting timelapses with ML for a very long time (with 550d raw video)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on January 02, 2020, 08:25:13 PM
Works great, Centered 1x1 crop is nice.
How did You change camera to 24HZ , is there a a way to change main clock in source code ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on January 03, 2020, 01:11:26 AM
@reddeercity
Divine...  :D
1 png frame from the video:
https://drive.google.com/file/d/1nB00fawDLep-4uCd0ean_9aJUUVe_1xY/view?usp=sharing

It works great. I think it can be useful only with timelapses or interior shootings with motorized sliders with very low-speed movement (when you set A point and B point and it slides extremely slow) and make it 6-8 times faster to get 25 or 30 fps. But for this, i think it should be 16:9. And it records only 15 sec now. Don't know if it's usable mode but it's very very interesting!


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on January 03, 2020, 02:39:04 AM
5.6K works realy nice!
I've been waiting for it for a long time. I record some Tests and the result is realy awesome. Great job;)
The only problem is the shutterspeed. I can only choose 1/4s, 1/5s and 1/6s. (Camera setting: 1/30, 1/50 and 1/100. below and above nothing changes). Ah, and audio doesn't work^^

@2blackbar
15sec? In 10bit i can record continuous at 70-74mb/s.

@Reddeercity
Is it possible to get some more selection of shutterspeed? It is much easier to record on daylight.

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 03, 2020, 04:18:35 AM
Quote from: Bender@arsch on January 03, 2020, 02:39:04 AM
The only problem is the shutterspeed. I can only choose 1/4s, 1/5s and 1/6s. (Camera setting: 1/30, 1/50 and 1/100. below and above nothing changes). Ah, and audio doesn't work^^
Yea I didn't think audio would work , maybe because liveview is at 4fps not sure .
I have a look at the shutter speed , just did a quick test to make sure it worked  but I didn't check all the shutters speeds .
Quote from: 2blackbar on January 02, 2020, 08:25:13 PM
How did You change camera to 24HZ , is there a a way to change main clock in source code ?
Not or at least not yet (?) a1ex has hinted at the possibility of overclocking the sensor for increased res & fps .
The 5D2 & 5D3 have main clock of 24MHZ , thou the 5D3 has 8 channel readout (8x24=192 MegaPixel per Second) (all the some time) where the 5D2 has only 4 channel readout (4x24=96MP/s) .
That's why these 2 camera have a very cinematic look to them and the only other camera that has 8 channel is 7D .
(I wouldn't mind poking at this one , as it maybe as good as the 5D3 in crop_rec , but sadly I can't find one cheap)
The EOSM as 4 channel but at a higher MHZ(30) same as the 50D (28.8MHZ) which allow for higher resolution & hi-speed frame rate.
But are not as cinematic in the motion blur , thou they are not bad as simulating film blur .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: T123 on January 03, 2020, 05:14:43 AM
@reddeercity

Any chance to get new updates on 50D?

Thank you for your hard work.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: IDA_ML on January 03, 2020, 05:35:19 PM
Quote from: reddeercity on January 03, 2020, 04:18:35 AM
That's why these 2 camera have a very cinematic look to them and the only other camera that has 8 channel is 7D .
(I wouldn't mind poking at this one , as it maybe as good as the 5D3 in crop_rec , but sadly I can't find one cheap)

It is great news that you are considering the 7D, Reddeercity! I still use mine for a significant part of my photo and video work and it works great!  The video quality and especially the filmic look coming out of this camera are fantastic.  Here is a video of mine that illustrates this:

https://cloud.mail.ru/public/3Yw3/MbAHmSoSk

The first 3:25 minutes of it were shot with the 7D at 1728x972 resolution.  You need to download the video with the "Скачать" button, (lower left of the above page), to watch it in its maximum quality. 

I wish you a Happy New Year and good luck with finding an inexpensive 7D body.  And keep up the good work!

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: cinematic on January 04, 2020, 04:17:08 AM
Hi Redeercity,

Thanks for all your efforts.

I tested dual iso video with iso 100/800 with crop_rec, mlv and dual_iso modules enabled. I am getting ISOless LV error 2 in all resolutions. Liveview goes black.

Is there something else that needs to be set?

Cheers,



Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: flofifull on January 04, 2020, 03:53:16 PM
Quote from: IDA_ML on January 03, 2020, 05:35:19 PM
It is great news that you are considering the 7D, Reddeercity!

+1
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 05, 2020, 03:46:02 AM
Ok here is a sample from the 5.6k preset (5632x2356 4.22fps)
Shot this yesterday , at sunset (4:30pm mst) , use my ef24-70mm 2.8L canon lens @ 24mm
F8 , ISO 100 , 1/5th second and because it was a sunset I had to use a Variable ND filter and stop it down 4 stops
I processed the 10bit cdng's thought Adobe After Effect CS6 with ACR ,I didn't spend much time on grading , just the basic .
I export the file to apple ProRes 422HQ and uploaded that same file (9.3GB)
I capture 7:00 min's continuously to a 32GB Lexar 1066x CF Card , I saw write speeds pecking around 87MB/s & the average was 79.3MB/s   :)
I set the A.E. project to 24fps and that made the time condensed to about 1:15 seconds from 7:00 min's .
Watch it @ 5K (2880p) for the best viewing , I might have to increase the frame rate to 6.5 to see if it's better film motion .
 


FYI: I was trying this out on the 50D and I can get 4752x1980 @ 10fps 10bit but only gets around 6-8 seconds , looks like I'm going to have to get
Lossless working on D4/5D2& 50D sooner then later  ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 05, 2020, 04:34:23 AM
Quote from: Skinny on January 02, 2020, 01:16:35 PM
Hi, what about my problem? I can't see the preview in all modes except 1:1.
All I see on 48p and 3x1 anamorphic modes is just a black screen.
I tried everything I can, I read almost the entire topic from beginning to end, tried different builds - nothing helped.
Sorry missed your question .
did you read this and try ?  Reply #933 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg218753#msg218753) plus I don't know the build date you are using or any other information , so with out at I can't help .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 05, 2020, 04:45:52 AM
Quote from: cinematic on January 04, 2020, 04:17:08 AM
I tested dual iso video with iso 100/800 with crop_rec, mlv and dual_iso modules enabled. I am getting ISOless LV error 2 in all resolutions. Liveview goes black.
Is there something else that needs to be set?
Same question to you , what ML build date are you using ,
experimental 10-12bit from magiclantern.fm/experiments (https://builds.magiclantern.fm/experiments.html) , Nightly main builds (https://builds.magiclantern.fm/#/) or my highly bleeding edge eXperimental
Also what version Dual ISO module & where from what download page

There's two different version of dual iso I have , 1 for the old API ( about a 1.5 years ago) or the new on for Crop_Rec only builds ( no supported on the main nightly build)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: cinematic on January 05, 2020, 06:54:17 AM
Quote from: reddeercity on January 05, 2020, 04:45:52 AM
Same question to you , what ML build date are you using ,
experimental 10-12bit from magiclantern.fm/experiments (https://builds.magiclantern.fm/experiments.html) , Nightly main builds (https://builds.magiclantern.fm/#/) or my highly bleeding edge eXperimental
Also what version Dual ISO module & where from what download page

There's two different version of dual iso I have , 1 for the old API ( about a 1.5 years ago) or the new on for Crop_Rec only builds ( no supported on the main nightly build)

Hi Reddeercity,


Build info as below:

crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip
dual_iso module in the zip file above

Cheers
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 05, 2020, 09:39:03 AM
Quote from: cinematic on January 05, 2020, 06:54:17 AM
Build info as below:
crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip
dual_iso module in the zip file above ......
No that one is for photo only , if you do search you will if all the info , and it's on my download page in the first post of this thread .
here is the link dual_iso_video-updates_8-21-2019.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dual_iso_video-updates_8-21-2019.zip)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on January 05, 2020, 10:37:11 AM
Quote from: reddeercity on January 05, 2020, 04:34:23 AM
Sorry missed your question .
did you read this and try ?  Reply #933 (https://www.magiclantern.fm/forum/index.php?topic=19336.msg218753#msg218753) plus I don't know the build date you are using or any other information , so with out at I can't help .
I tried this of course. And I tried all the builds starting July 5, 2019.
I notice some difference by the way. In your video, when you turn on the camera, you see a preview in black and white. And I see the preview as a regular video, although raw is on. And when crop-rec is turned on, in some modes I can't get out of the x5 zoom mode...
Also, my "decrease" button does not work, even without ML, I have to scroll through x10 zoom. Although it works when viewing a photo, I can zoom in and out there. I don't know if it's normal or not..
Maybe there are different versions of 5d2? Like different revisions or something?
I tried to reset the camera settings, no difference..
And my camera never gives high resolution through HDMI on any ML builds, official nightly, or any others, it's always VGA..


As for the new preset.. 5.6k is just awesome. The amount of detail is just phenomenal, now I really want it to work at 30 fps or, better 60 :D Of course I know it's impossible :) Anyway this is just an awesome build! Wow.

Although, it seems the frame is far from center. And it would be great to be able to change the FPS. Usually I use 1...2 fps and sometimes 0.33 for timelapses at 360 degrees shutter. 4 fps should be good for a motorized slider, but too fast for a timelapse..
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: cinematic on January 05, 2020, 11:55:27 AM
Quote from: reddeercity on January 05, 2020, 09:39:03 AM
No that one is for photo only , if you do search you will if all the info , and it's on my download page in the first post of this thread .
here is the link dual_iso_video-updates_8-21-2019.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/dual_iso_video-updates_8-21-2019.zip)


I tried your module. Getting the same issue.

Liveview went all dark when I enabled Dual ISO 100/800 under Expo menu and pressed half shutter and came back to Liveview.  After sometime getting the same error I mentioned in my first message. Resolution is 1856x1044. Tried with and without crop_rec module but getting the same issue.

Many Thanks

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on January 05, 2020, 01:37:09 PM
@cinematic interesting, I have the same problem but with crop_rec, not with dual iso. You could try to cycle trough the INFO button to unlock the screen and record something. In my case it works, although the liveview is black...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on January 05, 2020, 04:21:23 PM
@Skinny
Use the play button twice (refresh liveview) to go back from crop rec. 10x and decrease button doesn't work with crop rec enabled - > it's experimental
3. 5K/2.8K build works easy.
4K Anamorphic and 48p is a bit tricky to use. You must disable modul first before you press play button twice.
In 4K Anamorphic you must set the preview option to greyscale.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on January 05, 2020, 04:38:45 PM
Update!
I am using the latest build now, I managed to get a preview.
- I choose the "ML Gray" preview option
- Only the RAW module is activated
- I press the zoom x5 button (!)
- Now I immediately activate crop_rec 48p.
And the preview works. But if I update the live view or click something else - everything will fall into a black screen.

In the same way, I can choose 3x1 anamorphic instead of 48p, (the whole procedure needs to be done from the beginning) and the preview will work if I don't press the "play" button or "zoom" or something else....
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on January 05, 2020, 06:19:30 PM
Quote from: reddeercity on January 05, 2020, 03:46:02 AM
Ok here is a sample from the 5.6k preset (5632x2356 4.22fps)
Shot this yesterday , at sunset (4:30pm mst) , use my ef24-70mm 2.8L canon lens @ 24mm
F8 , ISO 100 , 1/5th second and because it was a sunset I had to use a Variable ND filter and stop it down 4 stops
I processed the 10bit cdng's thought Adobe After Effect CS6 with ACR ,I didn't spend much time on grading , just the basic .
I export the file to apple ProRes 422HQ and uploaded that same file (9.3GB)
I capture 7:00 min's continuously to a 32GB Lexar 1066x CF Card , I saw write speeds pecking around 87MB/s & the average was 79.3MB/s   :)
I set the A.E. project to 24fps and that made the time condensed to about 1:15 seconds from 7:00 min's .
Watch it @ 5K (2880p) for the best viewing , I might have to increase the frame rate to 6.5 to see if it's better film motion .
 


FYI: I was trying this out on the 50D and I can get 4752x1980 @ 10fps 10bit but only gets around 6-8 seconds , looks like I'm going to have to get
Lossless working on D4/5D2& 50D sooner then later  ;D

just amazing......
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 05, 2020, 08:11:33 PM
Quote from: reddeercity on January 05, 2020, 03:46:02 AM
Ok here is a sample from the 5.6k preset (5632x2356 4.22fps)
Shot this yesterday , at sunset (4:30pm mst) , use my ef24-70mm 2.8L canon lens @ 24mm
F8 , ISO 100 , 1/5th second and because it was a sunset I had to use a Variable ND filter and stop it down 4 stops
I processed the 10bit cdng's thought Adobe After Effect CS6 with ACR ,I didn't spend much time on grading , just the basic .
I export the file to apple ProRes 422HQ and uploaded that same file (9.3GB)
I capture 7:00 min's continuously to a 32GB Lexar 1066x CF Card , I saw write speeds pecking around 87MB/s & the average was 79.3MB/s   :)
I set the A.E. project to 24fps and that made the time condensed to about 1:15 seconds from 7:00 min's .
Watch it @ 5K (2880p) for the best viewing , I might have to increase the frame rate to 6.5 to see if it's better film motion .
 


FYI: I was trying this out on the 50D and I can get 4752x1980 @ 10fps 10bit but only gets around 6-8 seconds , looks like I'm going to have to get
Lossless working on D4/5D2& 50D sooner then later  ;D

Stunning.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 06, 2020, 06:25:02 AM
@Ilia3101 & @Igor_Braun , thanks  :)
FYI: I been using the Technicolor's CineStyle Profile to expose with and so far it's never been wrong
and of course I just let the hi-light almost clip , on other Picture Profiles it would be clipping badly .
What like the most is the shadows there are sooooo clean , there's no noise at all !
When you can use the whole sensor the image can be breathtaking  8)

@Bender@arsch thanks for jumping in and clearing things up for newbies , I don't have the time
to answer questions that are searchable . You have great handle on how to operate the crop_rec on 5D2  ;D

Now a new update ,
crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip)

I increased the vertical in this 5.6k 1:1 preset from 2354 --> 3124 and drop the frame rate to 3.8fps from 4.22.
also check the centering and it was off a bit , I drop it down should be right on center now check with a full res CR2
The new preset size is 5632x3124 @ 3.8fps 16x9 A.R. , the wright speed is 79.6MB/s so it's right on the edge of continuous/not continuous
if you drop the A.R. to 1.85 (5632x3044) write speed drops to around 71-72MB/s which is continuous

Later on I can add the crop_rec sensor timing to the "FPS override" so you lower of rise the frame rate .

Edit: I updated & cleaned up the first post of the this thread , removed old past builds & information that was confusing people
posted links to the new & latest crop_rec (dec,31,2019) plus the new 5.6k 1x1 preset from today .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on January 06, 2020, 10:56:12 AM
Sorry if I asked too many questions, or something. I just spent a lot of time to figure out how to do it right.
And I am very grateful for all the work and these builds. It breathes new life into this old camera :)

New 5.6k 16:9 works perfectly. full-frame 5.6k RAW video... This is just amazing
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: cinematic on January 06, 2020, 01:45:41 PM
Hi,

Sorry to be a party pooper. Unfortunately LV display refresh did not work for my dual iso issue. Tried recording video turned out to be all dark frames. Tried clearing all camera settings and reloading the ML again using a fresh card as well.

Getting ISOless LV error 2 error repeatedly. I am hoping someone can test 5D2 with the recent build and report back.

Thanks.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 07, 2020, 06:09:32 AM
Quote from: cinematic on January 06, 2020, 01:45:41 PM
....... Unfortunately LV display refresh did not work for my dual iso issue.
Tried recording video turned out to be all dark frames. .......
Getting ISOless LV error 2 error repeatedly. .....
You still haven't proofed to me that you downloaded the correct dual ISO module & you haven't shown any setting .
Take some screen shot in camera , is there a error Log on the card ?
Do you know how to use Dual Video ISO ? If not start searching the forum there many posts about it .

Well there no problem , I tested with the latest 5.6k crop_rec I just posted last night and it works as expected .
Works well with 5xZoom/3xcrop_mode even with crop_rec it work good , but in 3x3 it freezes liveview (again a expected result)
It very glitch·y can freeze liveview at any time , specially in fast pans .
So there is no issue here , and I use the link that I give you a few posts back

(https://i.ibb.co/4VKpB2S/Dual-ISO-Video-5-D2-Enabled.png) (https://ibb.co/nL6jhB1)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: cinematic on January 07, 2020, 11:28:09 AM
Quote from: reddeercity on January 07, 2020, 06:09:32 AM
You still haven't proofed to me that you downloaded the correct dual ISO module & you haven't shown any setting .
Take some screen shot in camera , is there a error Log on the card ?
Do you know how to use Dual Video ISO ? If not start searching the forum there many posts about it .

Well there no problem , I tested with the latest 5.6k crop_rec I just posted last night and it works as expected .
Works well with 5xZoom/3xcrop_mode even with crop_rec it work good , but in 3x3 it freezes liveview (again a expected result)
It very glitch·y can freeze liveview at any time , specially in fast pans .
So there is no issue here , and I use the link that I give you a few posts back

(https://i.ibb.co/4VKpB2S/Dual-ISO-Video-5-D2-Enabled.png) (https://ibb.co/nL6jhB1)






Hi Redeercity,

Screenshots from my 5D2.

https://ibb.co/N6mtvmF (https://ibb.co/N6mtvmF)
https://ibb.co/N6mtvmF (https://ibb.co/N6mtvmF)
https://ibb.co/z8j6qmn (https://ibb.co/z8j6qmn)
https://ibb.co/YfVNLK7 (https://ibb.co/YfVNLK7)
https://ibb.co/HTSnv4W (https://ibb.co/HTSnv4W)
https://ibb.co/Dwfbzct (https://ibb.co/Dwfbzct)
https://ibb.co/JvT2q5s (https://ibb.co/JvT2q5s)
https://ibb.co/YkWbSPP (https://ibb.co/YkWbSPP)

Couple of months ago, it was working. Processed MLV files using MLV App Dual ISO. All worked well that time.

Thanks for looking in to this issue.

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on January 07, 2020, 09:07:36 PM
Hi, @Reddeercity

I did some tests, it's dope...   :)

Centering is perfect, thanks
(https://i.ibb.co/D54QNnz/centering.jpg) (https://ibb.co/pJ2dgpb)
https://i.ibb.co/D54QNnz/centering.jpg

Unfortunately, I don't have any cool interiors or good looking landscape and motorized slider as well... so I shoot a 5k panorama with gimbal and downscaled it to 4k because my computer is toooo slow for 5k, or even 2,5k, and my software is unable to render videos that are more than 2880 vertically, so I had to render it with "avi ucompressed" to get 5632 x 3124 it requires a lot of space, so I rendered two videos: 1. only 3 sec of 5k and 2. whole panorama in 4k

4k https://www.youtube.com/watch?v=YmEGeIXQXv0&feature=youtu.be
5k 3 sec https://youtu.be/RSMCYftSGGY

But there is a problem (for me). Even in 14 bit with global draw allowed in 4k, I got a frozen preview.

Videos from phone:
5k https://youtu.be/sdDMUaP5HFE
4k  https://youtu.be/eLp6ubzA7k8 (frozen preview is here)

Also, got lines in shadows (i think it disappears with time)

(https://i.ibb.co/sgRVK3R/SHADOWS.jpg) (https://ibb.co/wdgzS6g)
https://i.ibb.co/sgRVK3R/SHADOWS.jpg

@Reddeercity, what do you think about 4k 16:9 (binned or regular) with small FPS? 

PS:
What do you think about 1dx mark iii. I was shocked by 5.5k 60 fps raw internally but it has only 20,1 mp in 2020 is it ok for photographers? We have 21 in 5d2 :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Dmytro_ua on January 08, 2020, 11:00:05 AM
Quote from: Igor_Braun on January 07, 2020, 09:07:36 PMit has only 20,1 mp in 2020 is it ok for photographers? We have 21 in 5d2 :)

Yes, it's more than enough for professional use. But it's not enough for cell phone users, that's why very soon we'll see 108-megapixel in Xiaomi  ;)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 09, 2020, 07:27:51 AM
Quotewhat do you think about 4k 16:9 (binned or regular) with small FPS? 
New Update: 4K 16x9 , 1:1 pixel  (4096x2160) at 8fps

@Igor_Braun  try this
magiclantern-crop_rec-4k16x9_8fps-5D2-eXperimental.11.01pm-2020Jan08.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-crop_rec-4k16x9_8fps-5D2-eXperimental.11.01pm-2020Jan08.5D2212.zip)

It's 4096x2160 1X1 @ 8fps , I can push it to 9.5fps but it's not a good  rate to made 24fps in hyper-lapse (8fps*3=24fps)
so the frame divide equally for good film motion .
Write speed is around 83MB/s , so limited number of frames , around 350-400 @ 10bit but if you drop the A.R. to 1.85 (4096x2048) this is continuous at 73MB/s or around there .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on January 09, 2020, 08:00:24 PM
Hi, Reddeercity, thanks for your amazing work!

My camera is happy every time a new firmware comes out...

It worked for me:

https://youtu.be/B6GvrqcPUAQ
8 fps - it's an ideal decision in my opinion.

Can't wait to test it. Will try to do it tomorrow or the day after, when it's daylight in the same place i shoot 5k panorama  ;)

And i still think it perfect for interior shooting.
Just look at tre first 3 seconds. This is what i'm talking about.
This shot with gimbal, but hope i'll have possibility to shoot a wide-open bright interior with motorized slider and 4k or 5k 16:9 preset
https://vimeo.com/319057484

Edit: Did a small 3 sec video in my place instead of going outside  :P:
https://youtu.be/3d-M7sQ3ALk
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 11, 2020, 05:57:34 AM
Here a sample frame of the sunset with 4096x1842 @ 8fps from my back door , dam near froze my finger off -25°C  :(

(https://i.ibb.co/DpgZh27/M09-1723-SMALL.png) (https://imgbb.com/)
4096x1842 M09-1723_0-00-00-09.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M09-1723_0-00-00-09.png)
4096x1842 M09-1729_000000.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/M09-1729_000000.dng)

Exported , cdng from mlv app , mlvfs quick mount doesn't reconizes the .mlv  say it "can't be mounted" I'll have to look in to mlvfs.dll I guess .
Imported in to After Effect CS6 , the dng I posted is the first frame from the sequence so A.E. applies the camera raw adjustment to it so
it will look like the post png plus it will have the metadata from the ACR embedded , FYI .   
Shot at 24mm with a 1.4 crop from full frame = to 34mm @ F8 + Variable ND at 4stops down , 100 ISO , 1/8th shutter .

Not sure yet but I think I like the 5.6k preset better @ 4fps , need to play around with it more .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on January 11, 2020, 08:01:29 PM
@reddeercity, would it be possible to make something like 5.6k but write only every second pixel to CF (or read only every second pixel, I don't know) so we would have a full sensor image with a ~2.8k resolution?
I think it would be almost free of aliasing when converted to 1080p in post, very low-noise and cinematic.. and it would save the card space and speed and everything, while still being full-frame..

But I don't know if this is technically possible at all :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 11, 2020, 08:07:30 PM
Quote from: reddeercity on January 11, 2020, 05:57:34 AM
Not sure yet but I think I like the 5.6k preset better @ 4fps , need to play around with it more .

Keep both!!!

Quote from: Skinny on January 11, 2020, 08:01:29 PM
@reddeercity, would it be possible to make something like 5.6k but write only every second pixel to CF (or read only every second pixel, I don't know) so we would have a full sensor image with a ~2.8k resolution?
I think it would be almost free of aliasing when converted to 1080p in post, very low-noise and cinematic.. and it would save the card space and speed and everything, while still being full-frame..

But I don't know if this is technically possible at all :)

You'd get a black and white image with aliasing, but I'd be interested to see how it really looks.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on January 11, 2020, 08:45:59 PM
@Ilia3101 :D :D I didn't think about that :) I mean... you know.. skip every 2 and write 2, or what needs to be done, then same with lines..  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ilia3101 on January 11, 2020, 08:50:39 PM
Oh that method. Probably too complicated to make 5d2 do it.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dlprod on January 12, 2020, 01:57:38 PM
Hi Zeek.  Can you please post a link to this build?  Thank you.

Quote from: ZEEK on December 23, 2019, 01:38:33 PM
@Reddeercity
Hey, I'm using the build for the 5D2:
November 14/2019 3008x1080 @ 23.98 fps Updated Corrected Horizontal Centering.
It's been working really awesome! Thanks! I have just come across two issues:
1.Mlv_Play Module appears as "error" when the module is loaded but works without it.
2. When you format the camera in the camera from Canon Menu, ML is removed completely (Even though it says if I would like to keep ML). Is there a way around this? 
Thanks heaps!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 13, 2020, 05:51:45 AM
Taken another shot at 1x3 on 5D2 , getting closer I got the adtg hooks working (at least in 3x3 , not sure about 5xzoom yet) plus figure out  how to code it .

/* ADTG12[0x100C] = 0: read every line */
/* ADTG12[0x1000] = 0: read every column */
if (is_5D2)
{
  adtg_new[2] = (struct adtg_new) {2, 0x100C, 0};
  adtg_new[3] = (struct adtg_new) {5, 0x1000, 6};
}

The problem I'm having now is it works in 3x3 but not 5xzoom , for some reason ( it's given me a headache  :( )
So I'm half ways there , adtg hook are working now just need to found out why it's not loading/working in 5xZoom .
Here a sample 1x3 frame from the .mlv , it 1856x1214 (3x3) @ 23.976fps and MLV App automatically detect it was 1x3 and displayed it correctly (5568x1214)
   
(https://i.ibb.co/HPFJzqC/M12-2034-frame-138-small.png) (https://imgbb.com/)
1856x1214_3x3_M12-2034_frame_138.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1856x1214_3x3_M12-2034_frame_138.png)

(https://i.ibb.co/Bjnc2Rs/M12-2034-frame-139-small.png) (https://imgbb.com/)
5568x1214_3x3_M12-2034_frame_139.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5568x1214_3x3_M12-2034_frame_139.png)

I didn't think this would work in 3x3 , the edges are smooth no nasty steps  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ZEEK on January 13, 2020, 09:33:32 AM
Quote from: dlprod on January 12, 2020, 01:57:38 PM
Hi Zeek.  Can you please post a link to this build?  Thank you.
I can, though isn't it up on Redeercity Build page in the discussion forum? Or has it been updated?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on January 13, 2020, 09:51:34 AM
Regarding adtg hooks not working. I can´t check sources but look for this place:
    if (!is_supported_mode() || !engio_vidmode_ok)
    {
        /* don't patch other video modes */
        return;
    }


Simply uncomment like this:
    if (!is_supported_mode() || !engio_vidmode_ok)
    {
        /* don't patch other video modes */
        //return;
    }


Will allow for adtg hooks to always run.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 14, 2020, 03:15:45 AM
Thanks , I've seen this before , I'll give that a try .

Quote from: Danne on January 13, 2020, 09:51:34 AM
.....  I can´t check sources .....
Here a link , crop_rec.c (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/src/crop_rec_4k_5D2/modules/crop_rec/crop_rec.c)
I'm way behind on commits (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/) just have the basic stuff , no crop_rec commits yet , I wanted to get 1x3 working before I update the crop_rec to the latest changes for 5D2 that I'm using now .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 18, 2020, 03:27:34 AM
Quote from: Danne on January 13, 2020, 09:51:34 AM
Regarding adtg hooks not working. ........
Simply uncomment like this:
    if (!is_supported_mode() || !engio_vidmode_ok)
    {
        /* don't patch other video modes */
        //return;
    }


Will allow for adtg hooks to always run.
No that didn't help , I moved 1x3 to the 3.5k centered preset (5xzoom) and it loads correctly but locks up with wrong fps .
Keeps it at 30fps in 5xzoom instead of 23.976fps as I coded it , the image looks right (1x3) with correct raw size , so
I think almost confident that the adtg hooks are working in both 3x3 & 5xzoom .

I have a solution , after looking in to the "fps-engio.c" I noted that the fps valves are limited for D4 cams (5D2 , 50D , 500D , 110D)

    // restrict max fps to 35 for 1100D, 5D2, 50D, 500D (others?)
    #if !defined(CONFIG_1100D) && !defined(CONFIG_5D2) && !defined(CONFIG_50D) && !defined(CONFIG_500D)
    36000, 37000, 38000, 39000, 40000, 41000, 42000, 43000, 44000, 45000, 46000, 47000, 48000,
    50000, 55000, 59940, 60000, 61000, 62000, 63000, 64000, 65000, 70000
    #endif
};


And the D5 cams have this
static int fps_values_x1000[] = {
    150, 200, 250, 333, 400, 500, 750, 1000, 1500, 2000, 2500, 3000, 4000,
    5000, 6000, 7000, 8000, 9000, 10000, 11000, 12000, 12500, 14000, 15000, 16000,
    17000, 18000, 19000, 20000, 21000, 22000, 23000, 23976, 24000, 25000, 26000, 27000,
    28000, 29000, 29970, 30000, 31000, 32000, 33000, 33333, 34000, 35000,


and since timerA is 440 = to 1376+160 (H) is a lot lower then the default @ 1880 (663)

I'll try the expanded valves and see if this stops it from locking up at the frame timers "A&B"

Thou I could almost get it to work if I set "frame override" to 8fps in 3x3 default mode then cycle
to 5xzoom and it enables for a 1-2 seconds then kick back out to 3x3 , seems Liveview fps is to fast
for 1x3 to enable , so still not there yet -- about 98% , 2% to go but that 2% could take some time  ::)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 18, 2020, 07:21:55 AM
Quote from: a1ex on August 16, 2018, 12:27:31 PM
.....  when it hits the lower limit for timer A, as declared in fps-engio.c. That is, the value sent to hardware won't be below the declared limit ...
To be able to dial a lower value, edit that limit (FPS_TIMER_A_MIN).
I think I found the answer  :D
I'm hitting a lower limit for timer"A" as define
#if defined(CONFIG_5D2)
    #define TG_FREQ_BASE 24000000
    #define FPS_TIMER_A_MIN MIN(fps_timer_a_orig - (ZOOM ? 0 : 20), ZOOM ? 0x262 : 0x228) // trial and error (with digic poke)

0x262 = 610 (crop_mode)
0x228 = 552 (3x3 non crop)
these valves are backup in  "frame override" menu and can not be lowered manually , so I have changed the lower limit
to 0x190 = 400 on both (3x3 & 5xzoom) , this should now let the 1x3 timer"A" load (I hope) as it's valve is 440 will below the min. of 610 in crop_mode .
It's late now so I'll try this tomorrow .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: domasa on January 18, 2020, 12:04:39 PM
It means that would be possible use "FPS Override" with more frames per seconds?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on January 21, 2020, 11:15:13 AM
Hi, i was doing some things these days and missed a 1x3 preset =) So I think i shouldn't try it now before it becomes part of new build version?

@Reddeercity - what do you think about FHD 48p binned for rich 16x9 aspect ratio? Or just lower the resolution?
For example, now you have 1856 x 774 with 48p working. So 1856x774=1,436,544 (pixels that work stable with 48p, so it's 68,954,112 pixels per second i bet..?)

So there are two options that are close to 1,436,544
1600*900=1,440,000 pixels
1598*898=1,435,004 pixels

16x9 48p I think would be very useful for b-rolls or things like that.

Meanwhile, I did some shot's:

4k 3x1
(aliasing is noticeable)
https://youtu.be/Giczkr4_yfg

5k
downgraded to 4k in editing program
(sorry, i forgot to switch off blending when increased video speed, you can notice it on birds, but my PC is too slow to rerender it again)
https://youtu.be/RUUzLdQH-5g

Reddeercity, understand you about "-25 degrees"  ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on January 23, 2020, 07:46:44 AM
Quote from: Igor_Braun on January 21, 2020, 11:15:13 AM
.... missed a 1x3 preset =)
No , still working on it . Locks up the cam . don't no why yet .
Quote from: Igor_Braun on January 21, 2020, 11:15:13 AM
@Reddeercity - what do you think about FHD 48p binned for rich 16x9 aspect ratio?
.......
16x9 48p I think would be very useful for b-rolls or things like that.
Since I have the adtg hooks working in 3x3 I can do 1856x680 48fps at 1.67 pixel stretch = 1856x1133 ,
There's some aliasing to be expected (scene depended)   It is continuous recording .

Sample frame Un-squeezed 
(https://i.ibb.co/RHXdKtx/1856x1134-M22-2355-frame-2-small.png) (https://imgbb.com/)
1856x1133_M22-2355_frame_2.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1856x1133_M22-2355_frame_2.png)
1856x1134_48fps_M22-2355_frame_1.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1856x1134_48fps_M22-2355_frame_1.dng)
Short 8 second video 1856x1134_48fps_M22-2355.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1856x1134_48fps_M22-2355.mp4)

1.67pixel Squeeze  , 1856x680 48fps
(https://i.ibb.co/x2K1xxm/1856x680-M22-2355-frame-small.png) (https://imgbb.com/)
1856x680_M22-2355_frame_1.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1856x680_M22-2355_frame_1.png)

I may be able to push it to 50fps
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on January 23, 2020, 08:10:25 AM
If lock up is because of to fast fps you can try and force timer inside the reg function. For instance, a workaround I use with 5d3 looks like this:
    /*  reduce to 30fps temporary to be able to go back from x10 zoom without freezes */
    if (lv_dispsize != 1) *(volatile uint32_t*)0xC0F06014 = 0x613;

You can reduce fps to whatever here, at least on digic v cams. Erase lv_dispsize !=1 if not needed of course.
Since I do not own any digic 4 camera I can´t guarantee anything here.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on January 24, 2020, 08:47:44 PM
@Reddeercity, i would like to test 1856x680 48fps  :D
Saw some aliasing, but i also think it depends on scene.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 11, 2020, 05:44:44 AM
Quote from: Igor_Braun on January 24, 2020, 08:47:44 PM
..... i would like to test 1856x680 48fps
Saw some aliasing, but i also think it depends on scene.
Not looking good for this , I have 3 problems
1st , The black level is all over the place (108 - 154) in 10bit thou black level can always be clean up at 78 every time.
2nd there's no lcd liveview (just a black screen) but it dose record correctly but with bad black levels .
and thirdly if you refresh liveveiw , either with the half shutter of the play button the adtg hook stop working
so there's no more 720p mode (1.67pixel) falls back to just vertical line skipping @ 3x3 .

So I'm going to leave this for now ( I'll came back on a later date) and more over to Lossless Compression
It work half ways , as in I can create a lossless fullres image but I can't save it , it just lock's up the camera
I need to debug the D4 lossless code to see where the problem is .

edit: sorry forgot to say also I'll be finishing the 50D port for crop_record .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 11, 2020, 05:57:51 AM
Shot a test clip of the updated 5D2 1:1 5632x3124 @ 3.8fps , made a hyper lapse at 24fps 6 mins at 3.8fps = 1 min at 24fps . mlv app for cdng's then ae with acr in cs6 uploaded prores 422hq to youtube best to watch at 5k (2880p) mode , I do like 16x9 ratio I think I will keep it  :)

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on February 11, 2020, 02:11:04 PM
Quote from: reddeercity on February 11, 2020, 05:57:51 AM
Shot a test clip of the updated 5D2 1:1 5632x3124 @ 3.8fps , made a hyper lapse at 24fps 6 mins at 3.8fps = 1 min at 24fps . mlv app for cdng's then ae with acr in cs6 uploaded prores 422hq to youtube best to watch at 5k (2880p) mode , I do like 16x9 ratio I think I will keep it  :)

Reddeer you are amazing! I was just thinking of shooting a music video, but shooting it at (0.25x) the speed. This was really cool to see, and for it to be 5K is insane!

I swear if you could zipfile your current CF card on those exact settings then I could sell it online and make a killing XD! (no seriously... ⚆ _ ⚆ )
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on February 18, 2020, 11:20:56 PM
Reddeercity. Keep it, please. That looks so crisp
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: jpegmasterjesse on February 19, 2020, 04:56:08 AM
Had some fun messing around with the 5k build last weekend. I didn't have any motion control system which is to me what really makes a time-lapse more interesting, but I liked the look of things.

One problem I had was pretty serious vertical stripes every time I entered 5x mode. I found that scrolling the shutter-dial - which messes up the frame - repeatedly until the frame returns to normal would sometimes fix this. I also believe that simply recording for 10 seconds or so can also slowly alleviate the problem.

Seems as if Dual-Iso at different bit-depths Just Works.

One thing I'd be very interested in would be even slower frame rates.

Thanks for your hard work RDC, we appreciate it.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: mark robinson audio on February 19, 2020, 06:03:12 PM
I am interested in this build   I have 2 anamorphic setups
one is a Kowa B&H with 50mm 1.2 canon lens and the Redstan collar with
the Rapido 16hva front variable diopter the other is a Kowa 16D with the Redstan
connecting collars and a FD 50 1.4 taking lens  ( for lighter weight)(who am I kidding
they both are heavy  and long)
both setups do vignette heavly on the 5dmk2     so I'm looking for a solution
for this critical shooting  problem
Just stumbled onto this site last night   my interest is glued
Mark Robinson
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on February 20, 2020, 12:00:53 AM
Quote from: mark robinson audio on February 19, 2020, 06:03:12 PM
I am interested in this build   I have 2 anamorphic setups
one is a Kowa B&H with 50mm 1.2 canon lens and the Redstan collar with
the Rapido 16hva front variable diopter the other is a Kowa 16D with the Redstan
connecting collars and a FD 50 1.4 taking lens  ( for lighter weight)(who am I kidding
they both are heavy  and long)
both setups do vignette heavly on the 5dmk2     so I'm looking for a solution
for this critical shooting  problem
Just stumbled onto this site last night   my interest is glued
Mark Robinson

Crop mode would probably help you extremely! That as a full frame setup sounds extremely interesting to me though. Anamorphic f/1.2? How much vignetting is there? Do you have any snapshots?

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 06, 2020, 12:22:27 AM
Almost got 50D to compile the crop_rec module , but getting a error
"Will not load on: 50D (fps_override_shutter_blanking) not checked"
?
Of course this stop the crop_rec module from compiling.
I know it has something to do with the video & or shutter hack on 50D as it doesn't have native video support .

Here the added support for 50D , it uses the same values as 5D2 (0x105f & 0x1061) , in fact everything is almost identical to the 5d2  :D
    const int blanking_reg_zoom   = (is_50D) ? 0x105F : 0x805F ;     // waza57 for test
    const int blanking_reg_nozoom = (is_50D) ? 0x1061 : 0x8061 ;           // :
    const int blanking_reg = (lv_dispsize == 1) ? blanking_reg_nozoom : blanking_reg_zoom;


shutter_hack movtweaks.c#lines-307 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/src/crop_rec_4k_5D2/src/movtweaks.c#lines-307)
#ifdef CONFIG_MOVIE_RECORDING_50D_SHUTTER_HACK

CONFIG_INT("shutter.btn.rec", shutter_btn_rec, 1);

void shutter_btn_rec_do(int rec)
{
    if (shutter_btn_rec == 1)
    {
        if (rec) gui_uilock(UILOCK_SHUTTER);
        else gui_uilock(UILOCK_NONE);
etc. ....

50d video_hack movtweaks.c#lines-466 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/src/crop_rec_4k_5D2/src/movtweaks.c#lines-466)
#ifdef FEATURE_MOVIE_RECORDING_50D

static MENU_UPDATE_FUNC(lv_movie_print)
{
    MENU_SET_VALUE(
        lv_movie_select != 2 ? "Disabled" :
        video_mode_resolution == 0 ? "1920x1080, 30fps" :
        video_mode_resolution == 2 ? "640x480, 30fps" : "Invalid"
    );
etc. ....


Is there anyone on the forum left that knows more able this ?
5D2 doesn't have any problem with this so I'm a little lost on how solve this .

Edit: found something that I think could be the issue
50D/EOSM: disabled FRAME_SHUTTER_BLANKING_WRITE  commit f68f196 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/f68f196807a3442a6bb09d40ecae89f00cb7697a) this different then the 5D2 which has this enabled 
**50D.109/consts.h**
-#define FRAME_SHUTTER_BLANKING_WRITE  (lv_dispsize > 1 ? &FRAME_SHUTTER_BLANKING_ZOOM : &FRAME_SHUTTER_BLANKING_NOZOOM)
+//~ #define FRAME_SHUTTER_BLANKING_WRITE  (lv_dispsize > 1 ? &FRAME_SHUTTER_BLANKING_ZOOM : &FRAME_SHUTTER_BLANKING_NOZOOM)


5D2.212/consts.h
#define FRAME_SHUTTER_BLANKING_WRITE  (lv_dispsize > 1 ? &FRAME_SHUTTER_BLANKING_ZOOM : &FRAME_SHUTTER_BLANKING_NOZOOM)


I see the EOSM has this also disabled , what would be the reason for 50D to have this disabled ?

Edit:#2
committed 6192a0d (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/6192a0d5aabb86db9ea23aa0f27da0439381141e#Lplatform/50D.109/internals.hF80T80)
Looks like a1ex disabled this from TL ,
/** We can also override shutter on a per-frame basis */
-#define CONFIG_FRAME_SHUTTER_OVERRIDE
+/* really? no other old camera can do this */
+//~ #define CONFIG_FRAME_SHUTTER_OVERRIDE

Not sure if this related but looking like it .
5D2 does not have this , so this seems it could be re-enabled , from what I understand it was disabled because it was a bleeding edge stuff
as no other camera has this & can do this .

Would be nice to find some history notes for this .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on March 07, 2020, 11:54:33 PM
I'm thinking now of buying 50d  :D :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 08, 2020, 06:30:29 AM
Fixed crop_rec module compiling problem on the 50d
in "platform/50D.109/consts.h" In this line

+//~ #define FRAME_SHUTTER_BLANKING_WRITE  (lv_dispsize > 1 ? &FRAME_SHUTTER_BLANKING_ZOOM : &FRAME_SHUTTER_BLANKING_NOZOOM)


I changed in to this

#define FRAME_SHUTTER_BLANKING_WRITE  (lv_dispsize > 1 ? &FRAME_SHUTTER_BLANKING_ZOOM : &FRAME_SHUTTER_BLANKING_NOZOOM)

Now it compiles but the module error out (module will not initiate startup) after loading .
So something is not letting the crop_rec startup , I guess I have to go over the code again and see what I missed .
I am closer now , just a matter of chasing down the bugs .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on March 19, 2020, 01:02:01 AM
@Reddeercity, look, i was playing with 48 fps today trying make some b roll transitions and noticed some strobing or flickering (i don't know how to say it correctly) effect when i was in the room with fluorescent lamps. So it because 1/125 shutter speed. Is there any way to change it in 48 fps mode?
Another question. How do you think, is it real to push fps even more just cutting edges from the right and the left side? It could become 1376x774 (from 1856x774) but with higher fps.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 19, 2020, 02:14:42 AM
Change your shutter speed to 1/90th - 1/100th of second for 48fps ,
just adjust the shutter speed before you enter into 48p you may have to lower it to 1/40th or lower in 3x3 (FHD) then when you which to crop_rec
it should be right.

I have no plans to increase the fps of reducing frame size for 48fps at this time on the 5D2 .
FYI is close to the edge for 5d2 in 3x3 Hi speed , the 50D will have more frame rate capability ( faster sensor) I can see at least 60 if not 70fps in 3x3
Unless there a problem with 5D2 builds , My time is dedicated now to 50D Crop_Rec & Lossless Compression on 50D & 5D2 .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: tankmanbrad on March 19, 2020, 05:44:27 PM
Quote from: reddeercity on April 06, 2017, 12:22:27 AM
Full width 5.6k 16x9 A.R. , 4K Anamorphic 3x1, 1x3,  UHD 3K Plus 10-12bit Full HD ,
this is the latest build as things process I'll keep updating this post

See below short video on how to enable 48fps 3x3 preset

Jan 5 2020
Experimental build:5632x3124 @ 3.8fps
crop_rec-5k-4fps-5D2-eXperimental.9.08pm.2020Jan01.5D2212.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-5k-4fps-5D2-eXperimental.9.08pm.2020Jan01.5D2212.zip)
This a 1:1 pixel 5632x3124 @ 3.8fps , Very eXperimental you need at least a 1066x CFcard for this .
Read the discussion in these posts
https://www.magiclantern.fm/forum/index.php?topic=19336.msg223937#msg223937
https://www.magiclantern.fm/forum/index.php?topic=19336.msg224036#msg224036


Shoot, tried to download this version...
I get this: "That link has no power here" and the link data in the browser tab says 404....

Am I being blocked from load or has it been pulled?
I blame wuhan virus! ;)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on March 24, 2020, 07:25:29 PM
Understood you, Reddeercity. Wish I could help you, but i'm too old to get into all this... oly can help with testing, checking, filming somthing etc..
Hope you take 50d to a new level :), and even 7D afterwards.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dariSSight on March 25, 2020, 10:16:36 PM
@reddeercity I Know your busy and you do not like redundancy, but this will be last time I'll ask question:
1. Which is your most stable build in FHD, 48fps, 5K, 4K time-lapse and 5x Crop?
2. For Best Image results is it best to use 14bit and 12bit or is 10bit Raw good enough?
3 Lastly, What is the best workflow that you use.
Thanks for your answers, I wanted to have your best recommendation as I test out builds in this time of population Quarantine.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on March 29, 2020, 02:48:30 PM
The "crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip" build is missed in the first Page and the link of the old one "crop_rec-5k-4fps-5D2-eXperimental.9.08pm.2020Jan01.5D2212.zip" is corrupt (51kb file).

I like the new 5.6K build. Works really good;)
Is it possible to implement the new 5.6K crop in the main Build?

Sidequest: what is the biggest CF Card for the 5D Mark II, can the 5D Mark II handle a 256GB CF Card?

Any new news about lossless compression?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: tankmanbrad on March 29, 2020, 07:53:29 PM
I pointed out the first page flaw as well, over ten days ago...
Seems the moderator doest like me enough to post my question "where did the file go?" as I kept getting a 404 error when I tried using the link.

I confirm a 128Gig 1066x Lexar card works on my 5DmkII, use it a lot for these huge mlv files,  if you get a 256G wouldn't mind knowing if it works or not.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on March 29, 2020, 08:38:48 PM
Quote from: Bender@arsch on March 29, 2020, 02:48:30 PM
what is the biggest CF Card for the 5D Mark II, can the 5D Mark II handle a 256GB CF Card?


Just don't do it. I use the 5D2 as my main camera and CF cards just aren't reliable consistently. I've had a couple die on me over the years. Get two 128GB's 1066x instead. 128GB is the sweet spot for me.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on March 31, 2020, 06:12:39 AM
Quote from: Bender@arsch on March 29, 2020, 02:48:30 PM
The "crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip" build is missed in the first Page and the link of the old one "crop_rec-5k-4fps-5D2-eXperimental.9.08pm.2020Jan01.5D2212.zip" is corrupt (51kb file).
Thanks for the heads up , I fixed the link , should be able to download now
I was cleaning up the 1st page post of downloads links to streamline things.

Quote from: Bender@arsch on March 29, 2020, 02:48:30 PM
I like the new 5.6K build. Works really good;)
Is it possible to implement the new 5.6K crop in the main Build?
This is a problem with all 5x crop presets on 5d2 , as the code is written now I can only have 1 5xcrop preset
on the module loaded , more then 1 e.g. 3.5k centered  & 5.6k FFV (Full Frame View) one of them will fallback to 3x3 mode
usually the second one in the crop_mode check in crop_rec.c , read the link , it explains more , if you can solve this I can add all the 5x crop preset in 1 build
https://www.magiclantern.fm/forum/index.php?topic=19336.msg220869#msg220869

Quote from: Bender@arsch on March 29, 2020, 02:48:30 PM
Sidequest: what is the biggest CF Card for the 5D Mark II, can the 5D Mark II handle a 256GB CF Card?
technically it dose since firmware 1.09v. but. .......... , so yea keep with 128GB , I just use 64GB cards , it safer I think .
If a 64GB card dies you may only loose 8-10min of footage , that bad enough but if a 128GB card dies will that would hurt bad :'(
Not to say it will , but you have to be prepared for the worst , as there say "plan for the worst - hope the best" words to live by  :D
https://www.magiclantern.fm/forum/index.php?topic=16045.0

Quote from: Bender@arsch on March 29, 2020, 02:48:30 PM
Any new news about lossless compression?
No , not yet I'm focused on getting the 50D crop_rec working without any workarounds right now ,
will look in to it later on ,
FYI , Lossless on the 5D2 is a lot closer then I'm letting on too
Why ?
Because having crop_rec working is more important , there's only one or 2 line of code that really my attention in lossless .
It's just a working philosophy I have , figure the hard stuff first & the easy stuff will fall in place , or so I hope  ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: JMS on April 03, 2020, 06:29:05 PM
I have been following and playing with ML for a number of years and reddeercity your work has been inspiring.
When you have the 50D crop ready I will gladly run any and all tests.
Thanks to all and Best wishes to everyone during these trying times.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: d!g!c on April 04, 2020, 11:21:28 PM
Something weird happend to my 50D. No, it did not break but when I tried the November experimental build for 3.5k crop, my vertical resolution is limited to 720p. So it's like 3.5k pixels across then 720 pixels down. I'm sure I followed all the steps correctly. The first experimental build for the 50D worked fine on my unit.

I'm more than willing to contribute to this effort however my sole CF card just died and I can't get a new one until the end of our community quarantine.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on April 06, 2020, 12:19:53 AM
Quote from: d!g!c on April 04, 2020, 11:21:28 PM
Something weird happend to my 50D. No, it did not break but when I tried the November experimental build for 3.5k crop, my vertical resolution is limited to 720p. So it's like 3.5k pixels across then 720 pixels down. I'm sure I followed all the steps correctly. The first experimental build for the 50D worked fine on my unit.

I'm more than willing to contribute to this effort however my sole CF card just died and I can't get a new one until the end of our community quarantine.
There's no November build posted on the 1st page/post download links just this one
''adtg-gui_crop_rec-3.7k-50D-eXperimental.2019Oct01.50D109.zip''
post a frame (dng) from that .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on April 26, 2020, 07:46:42 PM
4k picture is insane.
There is definitely something special in the 592-1776 stretching. I don't know how it works but it adds a certain kind of look to the picture:
https://youtu.be/aRxTGMj9Cxo
Processed with MLVP (BDM film profile, ''Lift'' green channels reconstruction mode, MLVPDM debayering method, some basic cc)
Very sharp (keep in mind that it's f/1.7)
And it feels sharper if compare it with 3k:
https://youtu.be/1-ykYj1ukBo
(1856 mode also feels sharper, but 3k is megacool zoom mode, and it contains a lot of information)

And this is color graded (in Vegas pro) 2:1 version of that 4k anamorphic image:
https://youtu.be/WoXb0fUDM1I

My dream is unsqueezed preview in 4k mode :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: RH42 on April 27, 2020, 08:38:19 PM
Which build did you use? - that 4k looks great!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on April 29, 2020, 07:26:22 AM
This build by Reddeercity:
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 02, 2020, 03:11:46 AM
Quick update , working on the 50D crop_rec module -- complied ok now but will not load on the camera just says error
stops at the module_init
MODULE_INFO_START()
    MODULE_INIT(crop_rec_init)
    MODULE_DEINIT(crop_rec_deinit)
MODULE_INFO_END()

So it won't start up  :(
So I need to trace it or log the startup to see where it fails , not sure if qemu can run Crop_Rec module on D4/50D
not even sure if 50D runs in qemu at all .

On a side note , I figured out 4k 3x1 4096x720 ->4096x2160 @23.976fps 
on 50D with continuous recording on Lexar 1066x
(https://i.ibb.co/3fLv3b1/50-D-4096x2160-3x1.png) (https://imgbb.com/)

a short mp4 10 second test here ->50D_4096x2160_24fps_3x1-M30-0002.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_4096x2160_24fps_3x1-M30-0002.mp4)
it's a little shaky  :D
used a full frame 50mm prime , the crop factor is 1.76 (1.6 for APS-C + .16 crop of the 50D full senor)(4096 from 4755)
50mm x 1.76 = 88mm

Head hurts now , I'll come back to this in a few days .

5D2: working on getting multiply 5xzoom crop_rec preset into one build with help from another forum member (You know who you are) :D
e.g. 3k (3008x1080) , 5.6k (5632x3124) & 2.8k (2784x1160) plus more in time .
still a work in process , chasing bugs  :(

Edit: here a ADTG_module build with 4k 3x1 if any one what to test it out
adtg_gui-crop_rec-4k3x1-50D-eXperimental.2019Nov03.50D109.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/adtg_gui-crop_rec-4k3x1-50D-eXperimental.2019Nov03.50D109.zip)
Just load it like the 3.7k build but in 3x3(FHD)(full width mode)
If you do get it in to 5xZoom mode , it will only be at 4096x720 without vertical line skipping.
So make sure you are in 3x3 mode to set the 3x1 pixel mode.
Have fun
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: JMS on May 02, 2020, 07:04:58 PM
I will load this up and give it a run. Thank you (and everyone else) for sharing your hard work.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Wlad81 on May 02, 2020, 11:33:13 PM
Quote from: reddeercity on May 02, 2020, 03:11:46 AM
On a side note , I figured out 4k 3x1 4096x720 ->4096x2160 @23.976fps 
on 50D with continuous recording on Lexar 1066x
Could you, please, be so kind to upload the original MLV file?

Am I to understand that by now 50D is better choice than 100D for real work?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 03, 2020, 03:03:18 AM
Quote from: Wlad81 on May 02, 2020, 11:33:13 PM
Could you, please, be so kind to upload the original MLV file?
I don't have that one any more ,  I did fine another one but with 1x3 pixel instead of 3x1 pixel

(https://i.ibb.co/yfkbkCs/M04-2317-frame-4-small.png) (https://imgbb.com/)
50D_1x3_4128x1776_M04-2317_frame_4.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_1x3_4128x1776_M04-2317_frame_4.png)

525MB file 50D_1x3_1376x1776-M04-2317.MLV (https://drive.google.com/file/d/17mMLQriyp4caeRteP1csXNr0VtaFJF_J/view?usp=sharing)

quick grade in mlv app. then exported to mp4
50D_1x3_4128x1776_M04-2317.mp4 (https://drive.google.com/file/d/16uOZ6IHWpiy3k6Dew0m1Zr1J3s0t0HTx/view?usp=sharing)

what the differences you may ask  ?
1x3 is in 5xZoom so no chance of aliasing & it's a horizontal stretch (1376x1776 = 4128x1776) which is a better mode
3x1 is 3x3 (FHD - Full HD) so vertical line skipping 4096x720 = 4096x2160) , not as good as 1x3 but still useable .

Quote from: Wlad81 on May 02, 2020, 11:33:13 PM
Am I to understand that by now 50D is better choice than 100D for real work?
Will , the 100d has more advance features e.g. 10,12,14bit lossless compression
A working crop_rec module , it d5 camera the d4 camera's (50D & 5D2) are behind in development .

The 50D has faster storage (CF Card 80MB/s+) can record 10,12bit uncompressed raw video along with 14bit .
So it really depends on your needs for your projects , as I don't have a 100D I can't really give the best advice on this
do a forum search for 100D are see what others are saying about it .
Though if the 50D is cheap enough buy both , and use as a"B"Cam.

My main cam is 5D2 & the 50D is "B"Cam/backup , FYI.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Wlad81 on May 03, 2020, 10:36:14 PM
reddeercity
Thank you so much! Got the file and graded it. Exporting to ProRes4444 1920x826 takes 2.5 minutes on my 2008 8-core Mac Pro using MLVApp.

Are there any other non-anamorphic hi-res modes on 50D currently?

I can get 50D body almost as cheap as used CF 1066x 64GB flash for it. The seller says it's shutter counter is almost 350000, is this a big value for 50D?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 04, 2020, 02:48:55 AM
Quote from: Wlad81 on May 03, 2020, 10:36:14 PM
.... Exporting to ProRes4444 1920x826 takes 2.5 minutes on my 2008 8-core Mac Pro using MLVApp.
? 1920x826 why , it a 4k file (4128x1776) 
Quote from: Wlad81 on May 03, 2020, 10:36:14 PM
Are there any other non-anamorphic hi-res modes on 50D currently?
Yes, but experimental right now .
I can get 3200x1250 @ 24fps , 3700x1080 24fps & in the
non experimental (nightly builds) 1920x1078 in 3xcrop_mode (high crop factor 3x).
Quote from: Wlad81 on May 03, 2020, 10:36:14 PM
I can get 50D body almost as cheap as used CF 1066x 64GB flash for it. The seller says it's shutter counter is almost 350000, is this a big value for 50D?
Yes , but for video it doesn't much matter . Canon says the shutter life span is 100,000 but I read that other 50d users had as much (350000+) counts and had no problems.
Unless you plan on doing a long time lapse with cr2 photos , the high shutter count shouldn't be any problem in Video mode .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 04, 2020, 03:09:10 AM
Successfully got the Crop_Rec Module to load on my 50D :D

(https://i.ibb.co/PNzmwvv/50d-crop-rec-loading-VRAM0-Small.png) (https://imgbb.com/)

(https://i.ibb.co/CHJ7cVC/50d-crop-rec-loading-VRAM1-Small.png) (https://imgbb.com/)

(https://i.ibb.co/55bPRYn/50d-crop-rec-loading-VRAM2-Small.png) (https://imgbb.com/)

I had to comment out a few line to do with the sensor timing

else if(is_camera("50D", "1.0.9"))
      ............
       // f = FIO_CreateFileOrAppend("ML/LOGS/console.log"); //waza57 log in file
      //  fps_main_clock = 28800000;
       
                                       /* 24p,  25p,  30p,  50p,  60p,   x5 */
       // memcpy(default_timerA, (int[]) { 0x000, 0x000, 696, 0x000, 0x000, 636 }, 30);     
       // memcpy(default_timerB, (int[]) { 0x000, 0x000, 1380, 0x000, 0x000, 1510}, 30);   
       // memcpy(default_fps_1k, (int[]) { 23976, 25000, 29985, 50000, 59940, 29988, 23976, 25000, 29985, 50000, 59940 }, 30);



/* default FPS timers are the same on all these models */
   ...........
    else if (!is_5D2)
    {
         memcpy(default_fps_1k, (int[]) { 23976, 25000, 29970, 50000, 59940, 29970, 23976, 25000, 29970, 50000, 59940 }, 24); 
    }
   // else if (!is_50D)
    //{
     //    memcpy(default_fps_1k, (int[]) { 23976, 25000, 29985, 50000, 59940, 29988, 23976, 25000, 29985, 50000, 59940 }, 30); 
    //}


I'm not sure why I had to do this but with it ,  it will not load on 50d .
I don't have full frame rate yet , just 1.5fps right now and that could be part of the timer issue I described  in the code I posted here.
more bug chasing , at least I'm almost there with this .
Don't hold you breath on a fully working Crop_Rec Module just yet , but you never know thou  ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: deanziyangyu on May 04, 2020, 04:09:30 AM
Quote from: reddeercity on May 03, 2020, 03:03:18 AM
I don't have that one any more ,  I did fine another one but with 1x3 pixel instead of 3x1 pixel

(https://i.ibb.co/yfkbkCs/M04-2317-frame-4-small.png) (https://imgbb.com/)
50D_1x3_4128x1776_M04-2317_frame_4.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_1x3_4128x1776_M04-2317_frame_4.png)

525MB file 50D_1x3_1376x1776-M04-2317.MLV (https://drive.google.com/file/d/17mMLQriyp4caeRteP1csXNr0VtaFJF_J/view?usp=sharing)

quick grade in mlv app. then exported to mp4
50D_1x3_4128x1776_M04-2317.mp4 (https://drive.google.com/file/d/16uOZ6IHWpiy3k6Dew0m1Zr1J3s0t0HTx/view?usp=sharing)

what the differences you may ask  ?
1x3 is in 5xZoom so no chance of aliasing & it's a horizontal stretch (1376x1776 = 4128x1776) which is a better mode
3x1 is 3x3 (FHD - Full HD) so vertical line skipping 4096x720 = 4096x2160) , not as good as 1x3 but still useable .
Will , the 100d has more advance features e.g. 10,12,14bit lossless compression
A working crop_rec module , it d5 camera the d4 camera's (50D & 5D2) are behind in development .

The 50D has faster storage (CF Card 80MB/s+) can record 10,12bit uncompressed raw video along with 14bit .
So it really depends on your needs for your projects , as I don't have a 100D I can't really give the best advice on this
do a forum search for 100D are see what others are saying about it .
Though if the 50D is cheap enough buy both , and use as a"B"Cam.

My main cam is 5D2 & the 50D is "B"Cam/backup , FYI.

Been following the thread for a while now and happily registered upon the news of 1x3 mode.
Wonderful progress on the 50D so far, reddeercity! Can't wait to try this when I get my 50D...

In the mean time I would like to hear about the progress on the 1x3 mode on the 5D2 since this is the only possible way to avoid the horrible moire in the 3x3 mode without compromising the crop factor "too much"... Correct me if I am wrong, but I think the camera bins horizontal pixels but skips vertical pixels in live view just like the Digic 5 cameras (except 5D3)?

Would like to hear from you about this! Also since I have some background in C and Arm assembly, maybe I can help with development a bit (looking at you, lossless compression)!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on May 04, 2020, 12:26:46 PM
Quote from: reddeercity on May 03, 2020, 03:03:18 AM
I did fine another one but with 1x3 pixel instead of 3x1 pixel

(https://i.ibb.co/yfkbkCs/M04-2317-frame-4-small.png) (https://imgbb.com/)
50D_1x3_4128x1776_M04-2317_frame_4.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_1x3_4128x1776_M04-2317_frame_4.png)

525MB file 50D_1x3_1376x1776-M04-2317.MLV (https://drive.google.com/file/d/17mMLQriyp4caeRteP1csXNr0VtaFJF_J/view?usp=sharing)
This looks awesome! Great step in the right direction!  8)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: RH42 on May 04, 2020, 03:12:39 PM
Quote from: Igor_Braun on April 29, 2020, 07:26:22 AM
This build by Reddeercity:
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip

Thank you :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Wlad81 on May 04, 2020, 08:26:28 PM
Quote from: reddeercity on May 04, 2020, 02:48:55 AM
? 1920x826 why , it a 4k file (4128x1776)  Yes, but experimental right now .
I've just tried to do it, because Full HD 16x9 is the final  form of the future project. But - yes, I can export it to 4K (after first grading) and re-frame it to 16x9 after the final grading in Final Cut Pro 7 or X.

Quote from: reddeercity on May 04, 2020, 02:48:55 AM
I can get 3200x1250 @ 24fps , 3700x1080 24fps & in the
non experimental (nightly builds) 1920x1078 in 3xcrop_mode (high crop factor 3x).Yes , but for video it doesn't much matter . Canon says the shutter life span is 100,000 but I read that other 50d users had as much (350000+) counts and had no problems.
Unless you plan on doing a long time lapse with cr2 photos , the high shutter count shouldn't be any problem in Video mode .
OK, so there's nothing to worry about.

I think that 50D awaits a big future, much bigger than 100D because 50D has a CF 70 MB/s slot. (Sorry for my English).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Frayfray on May 04, 2020, 08:41:59 PM
I am also shopping for few new oldie cameras i found this list about write speed
Write speed for some Canon cameras
D30 – 1,2 MB/s
D60 -1,8 MB/s
300D – 1,4 MB/s
10D – 1,4 MB/s
1Ds – 2 MB/s
1D – 3,3 MB/s
20D – 6 MB/s
350D – 6 MB/s
30D – 6 MB/s
40D – 11 MB/s
60D – 21 MB/s
600D – 21 MB/s
550D – 21 MB/s
500D – 21 MB/s
1100D – 21 MB/s
EOS M – 44 MB/s (Overclocked at SDR104 @ 160MHz: 69 MB/s)
EOS M2 – 44 MB/s (Overclocked at SDR104 @ 160MHz: 69 MB/s)
1200D – 44 MB/s
100D – 44 MB/s (Overclocked at SDR104 @ 160MHz: 71 MB/s)
650D- 44 MB/s (Overclocked at SDR104 @ 160MHz: 71 MB/s)
700D -44 MB/s (Overclocked at SDR104 @ 160MHz: 71 MB/s)
70D – 44 MB/s (Overclocked at SDR104 @ 160MHz: 71 MB/s)
6D – 44 MB/s (Overclocked at SDR104 @ 160MHz: 71 MB/s)
1D IV – 66 MB/s
50D – 70 MB/s
M50 – 80 MB/s
5D II – 85 MB/s
750D – 90 MB/s
760D – 90 MB/s
800D – 90 MB/s
77D – 90 MB/s
80D – 90 MB/s
7D – 90 MB/s
1Dx – 120 MB/s
7D II – 120 + 90 MB/s (CF+SD)
5D III – 120 + 21 MB/s (CF+SD) SD card write speed will limit CF slot write speed if you use CF + SD at the same time.  (Overclocked at SDR104 @ 160MHz: 51 MB/s)
5Ds – 120 + 90 MB/s (CF+SD)
5D IV – 130 + 90 MB/s (CF+SD)
EOS R – 200 MB/s
here is the link https://kameratrollet.se/canon-write-speeds/
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: kitor on May 04, 2020, 09:07:44 PM
When you steal information from website, it's nice to at least provide original source.

https://kameratrollet.se/canon-write-speeds/
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Wlad81 on May 04, 2020, 10:26:10 PM
Quote100D – 44 MB/s (Overclocked at SDR104 @ 160MHz: 71 MB/s)
But the SD card overheats quite a lot even without the cover.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Frayfray on May 05, 2020, 04:47:48 AM
Quote from: kitor on May 04, 2020, 09:07:44 PM
When you steal information from website, it's nice to at least provide original source.

https://kameratrollet.se/canon-write-speeds/
i am sorry that you feel that way i was just trying to help but your lack of understanding of the English language is clear .i did say that "i found this list about write speed",plus your website is not in all English and you did not list where your source of the list is from. so start from there before you say that i was stealing.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 07, 2020, 01:34:17 AM
@Frayfray this off topic here ,
Moderator can please move this to general chat
https://www.magiclantern.fm/forum/index.php?topic=19336.msg226795#msg226795
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 07, 2020, 02:12:30 AM
crop_rec-3744x1080_24fps_50D-eXperimental.4.57pm.2020May06.50D109.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/crop_rec-3744x1080_24fps_50D-eXperimental.4.57pm.2020May06.50D109.zip)

(https://i.ibb.co/2crx84b/VRAM3-small.png) (https://imgbb.com/)

(https://i.ibb.co/7QBrLmv/VRAM4-small.png) (https://imgbb.com/)

OK , 50D has a working Crop_Rec Module (just 2 preset for now , still debugging)
It's a little glitchy right now but works , there's 2 preset 3x1 4k (4096x720 but does not work fully)
and "3.5k Centered" preset 3744x1080 @ 23.976fps , this works will now .
So don't use 3x1 4k preset for now (until I can fix the cmos bug) , it's needs to be there so 1:1 preset can work
not sure why it needs a 2nd preset but with out it the "3.5k Centered" preset does not work right .
If may lockup getting out of the 5xZoom (3.5k Centered preset ) to FHD if so just pull the battery out .
Just load crop_rec module like on the other cameras that support crop-rec .

##Note## I have to say this !
This is very Bleeding edge stuff , so Test away with Caution  ,
and of course if it bricks your camera you get to keep all the pieces  :P

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on May 07, 2020, 02:29:12 AM
Very nice @reddeercity! I might test it next week and see how it goes. Amazing how 50D still rocks so much, after all those years.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Frayfray on May 07, 2020, 06:12:58 AM

Quote from: reddeercity on May 03, 2020, 03:03:18 AM

The 50D has faster storage (CF Card 80MB/s+) can record 10,12bit uncompressed raw video along with 14bit .
So it really depends on your needs for your projects , as I don't have a 100D I can't really give the best advice on this
do a forum search for 100D are see what others are saying about it .
Though if the 50D is cheap enough buy both , and use as a"B"Cam.

My main cam is 5D2 & the 50D is "B"Cam/backup , FYI.

@reddeercity
.so you record audio on different device ? cause 100d records  audio but / 50d does not right ...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: d!g!c on May 08, 2020, 04:20:04 PM
Quote from: Frayfray on May 07, 2020, 06:12:58 AM
@reddeercity
.so you record audio on different device ? cause 100d records  audio but / 50d does not right ...

If you plan on using a 50D, then an external audio device is required. One real problem is syncing. I haven't found any shortcuts to syncing as there is no dummy audio and no T/C in with the 50D so the only option is manually lining up the audio track to sync with the video...unless somebody has a better solution.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Frayfray on May 08, 2020, 04:40:10 PM
i don't have 50d yet i was thinking about getting one  because of the cost and latest developments i saw a video a few years ago about 50d shooting raw but  there was no audio and i see in earlier posts that the developer has a tascam or zoom recorder in his pics. i think if you use 50d as b roll camera or buy a 5d2 lets see how far the development go ... cause for me audio is important.. 8)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Luther on May 08, 2020, 07:30:54 PM
I record the audio with a Tascam and manually sync after. It is a pain, indeed. But the final quality if very nice. For long shots I use 600D instead and the 50D stays just for small inserts.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on May 08, 2020, 10:50:13 PM
Does the 50D have a beeper noise? Like the AF beep sound?

One way to possibly sync footage is to have a beep (or series of beeps depending on shot number) that start at frame 1 of the RAW footage. Now I'm not sure how easily that would be to implement, but that would at least give a starting point to sync footage more easily.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Wlad81 on May 08, 2020, 11:12:38 PM
I'm sorry... making beep with the camera that doesn't have any sound or even video modules and modes at all? Or, are you talking about the non-documented possibilities of Digic 4 processor? And how to synchronize the 50D's beep with the start of the audio recorder? I think the beep must also be in the audio recorder, or I'm mistaking?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on May 09, 2020, 01:37:41 AM
No I mean like an audio spike for your recorder to hear. Like when you focus lock and the camera makes a BEEP. That way, you will know the first frame goes here.
(https://i.ibb.co/8D2Rcmn/z.png) (https://imgbb.com/)


Then for shot 2 there can be two audio spikes

Then shot 3 there can be 3 audio spikes
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 09, 2020, 02:05:06 AM
OK guy's,  take this to 50D thread (https://www.magiclantern.fm/forum/index.php?topic=9852.0) please , it totally Off Topic this has nothing to Crop_Rec .
I have No plans on developing the 50D beyond crop_rec  & lossless compression.
There no Audio on the 50D and never will be there no speaker .
Just use a  Clapperboard (https://en.wikipedia.org/wiki/Clapperboard)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: deanziyangyu on May 09, 2020, 04:22:59 AM
Hello everyone, now that I have my hands on the 5Dii I would like some help on properly enabling the 3x1 anamorphic mode since it doesn't seem to work no matter what I try.

Build: crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip
What I did:
After enabling Raw Video (MLV) module, go into live view and toggle to 5x -> press half shutter -> press delete -> now I see 2144x1078
After that I Enable crop_rec and choose 3x1 4k anamorphic -> the option is greyed out and I am asked to go back to 1x mode in live view
That's when the live view goes horribly wrong with the top half showing a horizontally stretched image and bottom half sowing white space.
Now I press delete again -> RAW video MLV shows 1856x1044 but with a crop factor of 3.02. When I try to modify the resolution in the sub menu it says "4096 is not possible in current video mode (max 1856)"

My question is how do I manage to set 4096x590 in the RAW video MLV menu ? I followed the earlier posts in the thread and reddeercity's Youtube tutorial video but none of them worked. I also tried the the old CMOS[1] hi and CMOS [2] setting and also to no avail. I attempted this with basically all other builds with 3x1 4k built in as well.

In the pseudo-anamorphic mode I tried to record something and open in MlvApp. With a maximum horizontal resolution I can get at 1856px the image's left and right 1/3 are black, for the middle 1/3 only the top 1/2 has an actual output and the bottom 1/2 is white, just like in the live view.

Since I don't have access to imgur at this moment I will try to post some screenshots later, but please let me know if any of you have an idea of what is going on here.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Wlad81 on May 09, 2020, 11:49:21 AM
Quote from: reddeercity on May 09, 2020, 02:05:06 AM
There no Audio on the 50D and never will be there no speaker .
That's what I was trying to say.

reddeercity, is there'd really be the lossless compression on 50D? Seriously? Great then.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Macke1 on May 11, 2020, 10:35:08 PM
strange things have happened.. I'm using the sane card as I used to. But after I loaded reddeercity 14 juni build I can't record mlv files. everything becomes .mov. I tried to recall using info button but even if I tries the last nightlybuild the same happens. someone knows?!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on May 12, 2020, 04:45:36 PM
@Macke1
Try: change to photo mod and back to video mod. (LV Dysplay in ML settings)

@deanziyangyu
Can you record a short smartphone movie from your ml settings and how you try to enable the Anamorphic mod?

Usually way is (fresh install) :
1. enable crop_re, mlv_rec and if you want mlv_snd module -> restart

2. Raw video (MLV) set to "on" and set shutterspeed to maybe 1/50 (not 1/8000 or so - > maybe don't work)

3. Open liveview, open ml menu, set resolutions to the highest value. Close ml - > simply press half shutter.

4. Press "+", open ml, crop rec is set to "off", now go to 4k Anamorphic. Half shutter.

5. Have fun

Important:
if you want go back to 3x3 mod, set crop rec first to "off", half shutter, than simply press play button twice.
Don't change settings in crop mode (shutterspeed, ISO...) .

Preview:
usually (auto): you have Canon preview (small area) - > works while recording

ML grayscale: unstreched full preview, but frozen liveview while recording in 10 and 12bit. If you use 14bit and global draw on you have a unfrozen liveview but low frame.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Macke1 on May 12, 2020, 10:13:24 PM
Thanks! everything is good except + can't zoom in, but will try more
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on May 13, 2020, 09:44:11 AM
@maybe your lens is in autofocus mode?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Macke1 on May 13, 2020, 11:22:22 PM
Thanks! it worked, have some more problems to deal with.. hope I solve it myself, if I don't Im asking again
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 18, 2020, 07:27:59 AM
Update 50D , I have 48fps  preset working now just needs a few more tweaks in the preview regs .

(https://i.ibb.co/k1ZVJ6m/M17-1958-frame-1-small.png) (https://imgbb.com/)
1568x700_M17-1958_frame_1.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1568x700_M17-1958_frame_1.png)

Here a short 4 second 10bit .mlv sample from my Google drive 50D_48fps_M17-1958.MLV (https://drive.google.com/file/d/1rMXmIFMlOh6WzvjL72DKldfI5Kln7VyL/view?usp=sharing) 275MB
Exported a 4 second  h264 (mp4) from mlv app 1568x700_48fps_M17-1958.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1568x700_48fps_M17-1958.mp4)

Still working on the bug for 4k 3x1 , I may have a new crop_rec for 50d in a few days with 48p & center crop .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Wlad81 on May 18, 2020, 11:41:45 PM
Great! Is it from the whole sensor?

Can 50D work on 50fps, even in slightly less resolution?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on May 19, 2020, 10:25:05 AM
Quote from: Wlad81 on May 18, 2020, 11:41:45 PM
Great! Is it from the whole sensor?
See wikipedia: 50D resolution: 4752 × 3168
Divided by 3x3: 1584 x 1056
Now look again to the resolutions posted by reddeercity.
Quote from: Wlad81 on May 18, 2020, 11:41:45 PM
Can 50D work on 50fps, even in slightly less resolution?
If you just read this thread...
https://www.magiclantern.fm/forum/index.php?topic=19336.msg225537#msg225537

@reddeercity: nice work!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 26, 2020, 07:26:07 AM
50D update: I got a Full Res preset figured out in 5xZoom .
4768x2678 @ 6.5fps I did have it push to 8fps but it was too unstable .
Also I had the full height (4768x3166) @2.5fps (I think it should be able to get around 4-5fps but it was getting hot & lockup) 

(https://i.ibb.co/z8ct4rD/M25-2350-00001-small.png) (https://imgbb.com/)

Here a 16bit PNG from MLV Producer 50D-6.5fps-M25-2350_00001.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D-6.5fps-M25-2350_00001.png)
I had  problems using MLV App , It had bad pink hi-lights that couldn't be fixed.

I also export 2 h264 clips , 1 at the native frame rate (6.5fps) and the 2nd at 24fps ( would gave me 2.0 second from 8 seconds)
I didn't have the right shutter speed so it's a little jerky.
50D-M25-2350-4768x2678-6.5fps.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D-M25-2350-4768x2678-6.5fps.mov)
50D-M25-2350-4768x2678_6.5fps-to-24fps.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D-M25-2350-4768x2678_6.5fps-to-24fps.mov)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: heder on May 26, 2020, 11:19:50 AM
Regarding Oc'ing

Quote from: reddeercity on October 11, 2018, 07:50:01 AM
Ok , loaded it up and it reads the CF Card info
Printed this on the screen
(https://image.ibb.co/bxObPU/TEST.png) (https://imgbb.com/)

Some useful info at least , this is a Lexar 1066x 32GB
I see the card runs as a PCMCIA & the UDMA in Advance i/o --0x0003  , UDMA6 ? More then likely from the dm-spy logs

This image (upper) is really odd. Look at this image (lower - older cf_acc version) from the 40D.

(https://lh3.googleusercontent.com/pw/ACtC-3c-U6erZJHBmA-1r6y1izSiM1XOMfSkNOymdIUthkhq0B9raXW6YeFGMt6mfXwUJubCgzxfAFokQjkbg6sLJvW2fdQZfiTRcEX5wx1TSURzTjpc8ENGVt3TsghQ-LUq8H95gvA41KgyU5n3e-_gaA6y=w554-h510-no?authuser=0)

There are more questions than answers ..

Why are features supported the excat same ? 40D (digic 3) only runs in slow PIO mode, while digic 4 run lighting fast UDMA mode. The digic 4 code says UDMA , and the digic 3 code only says PIO mode.

This is really fishy ..

1. Features supported are the exact same (no can't be !).
2. No mention to UDMA support or enabled in upper image, but digic 4 runs UDMA
3. No mention to PIO mode 5,6 support in lower image, but digic 3 runs 80 ns cycle (pio mode 6)
4. Both upper and lower image, says support PIO 3+4 only.
5. lots of others words are exactly the same.

The Features Enable words are different, but the CFA3.0 or CFA4.0 specifications does not mention anything about these bits, they are reserved. Im talking about word 86. Sorry can't show the specs, they are "copyrighted". so need to use google yourself.

Seems like the CF card controller is a part of the DIGIC-SOC chip itself, and the cf card controllers "words" are just dummy data. Or am I missing something ? it not then this basically means that OC'ing can only be done with CFDMA I/O (0xC0620000 for digic 4) 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Wlad81 on May 26, 2020, 10:57:57 PM
Do you want to say that it may be possible to hack 40D CF controller to work in PIO 6 mode?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 27, 2020, 06:00:05 AM
Quote from: heder on May 26, 2020, 11:19:50 AM
Why are features supported the excat same ? 40D (digic 3) only runs in slow PIO mode, while digic 4 run lighting fast UDMA mode.
The digic 4 code says UDMA , and the digic 3 code only says PIO mode.
.........
it not then this basically means that OC'ing can only be done with CFDMA I/O (0xC0620000 for digic 4)
Seem 40D uses  (40D = 0xC0700000) instead of 0xC0620000
from here wiki/Register_Map#CFDMA (https://magiclantern.fandom.com/wiki/Register_Map#CFDMA)
When i did my investigation i used "digic poke" not sure if it's works on 40D , i could clock down the CF on the 5D2
to 20MB/s and rise it in steps (35,50 etc. ...) but i could not go to udma7  just 6 which is 80MB/s .
I followed this https://www.magiclantern.fm/forum/index.php?topic=12862.msg206010#msg206010
this may help also https://www.magiclantern.fm/forum/index.php?topic=12862.msg206591#msg206591

I guess a1ex did have his 5d2 cf card bus over clocked ,
https://www.magiclantern.fm/forum/index.php?topic=19336.msg207053#msg207053
I guess  /magic-lantern/branch/hdparm can help do this , i have the source but haven't had the time to compile on 5d2/50d yet.
not sure if this helps you  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Wlad81 on May 27, 2020, 12:18:13 PM
Quote from: reddeercity on May 27, 2020, 06:00:05 AM
i have the source but haven't had the time to compile on 5d2/50d yet.
Waiting for this.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 29, 2020, 01:14:25 AM
@heder see this https://magiclantern.fandom.com/wiki/Register_Map/40D

CF Interface block
+0x2000 [8]   PIO DATA (used to transfer data in polled mode)
+0x2001 [8]   Read: Error data, Write: Features
+0x2002 [8]   Sector count
+0x2003 [8]   Sector number
+0x2004 [8]   Cylinder low byte
+0x2005 [8]   Cylinder high byte
+0x2006 [8]   Drive/Head
+0x2007 [8]   Read: Status, Write: CF command
+0x200E [8]   Read: Alt Status, Write: Device control

Extern DMA Controller Interface (DMA PIO mode - not polled mode) (digic3<->cf controller ?)
+0x800C [32]  Data register to feed with data (usually 0x2000)
+0x8010 [32]  unknown, set to 0x100. maybe sector size?
+0x8014 [32]  unknown, set to PIO_W:0x1E, PIO_R:0x16, DMA:0x00,
+0x8024 [32]  PIO: CF Command register to use, OR'ed with 0xA000
+0x8028 [32]  PIO: set to ?
+0x8030 [32]  PIO: set to ?
+0x8034 [32]  PIO: set to ? or ? depending on command reg
+0x8038 [32]  PIO: set to ? or ? depending on command reg
+0x8040 [32]  Enable interrupt when flag changed? 0, 1, 0x2000000
+0x8044 [32]  Interrupt reason, write with negated value upon interrupt
+0x8048 [32]  set to 1, maybe transfer count?

Tranfer mode control registers ? (cf controller <-> cf card ?)
+0x8100 [32]  0x0 (init) or 0x2 (software OR's with 0xFFFFFFF9 then AND with 0x2)
+0x8104 [32]  0x4

PIO access timing registers (cf controller <-> cf card ?)
+0x8108 [32]  access timing PIO mode - fastest mode = 0xC0 ~ 20mb/s
+0x810C [32]  access timing PIO mode - fastest mode = 0xC0 ~ 20mb/s
+0x8110 [32]  access timing PIO mode - fastest mode = 0xC0 ~ 20mb/s
+0x8114 [32]  access timing PIO mode - fastest mode = 0xC0 ~ 20mb/s

Notes:
When setting timing registers 0x8108-0x8114 incorrectly, the FIO_Write function will recover from any
errors and return a error to the caller via return value. The FIO_Read will not, and a call will FIO_read
will stall the camera.

UDMA Mode Settings ? (CF identify drive command says UDMA is not supported)
+0x8200 [32] 0x0     (?)
+0x8204 [32] 0x0     (?)
+0x8208 [32] 0x3FFFF (?)
+0x820C [32] 0x3FFFF (?)
+0x8210 [32] 0x3FFFF (register is changable)
+0x8214 [32] 0x3FFFF (register is changable)


SDCON Interface used by CF
0xC0C.....


Some useful info , even for 5d2/50d
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: heder on May 29, 2020, 10:18:56 AM
Quote from: reddeercity on May 29, 2020, 01:14:25 AM
@heder see this https://magiclantern.fandom.com/wiki/Register_Map/40D
...

I wrote that section on 40D  ;)

I have digged into the CF-1.4/2.0/3.0/4.0 specifications (gosh ... boring) and the words comming out of the CF from both 5D's and 40D controller is crap, and partly dummy data, no mention to UDMA or newer PIO modes. I have'nt disassemble a real canon camera and searched for a CF chip, so I can't for sure say if it is a real chip of SoC part of the digic processor. One thing is however clear,most chips inside the camera are specific for canon cameras,

It feels like :

I will keep digging into this for a while, the real timing values are in the controller area 0xc0628xxx (digic 4) 0xc0708xxx (digic 3).

I will create a new thread soon regarding CF investigation
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 01, 2020, 01:47:18 AM
Quote from: heder on May 29, 2020, 10:18:56 AM
I wrote that section on 40D  ;)
Great Job  :) looks like it will help me with 5d2/50d
Quote from: heder on May 29, 2020, 10:18:56 AM
I will create a new thread soon regarding CF investigation
Great ! I'll definitely be involved in that  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 01, 2020, 02:11:31 AM
Working on 50D Full Height in FLV preset , So the max full height is 3176 in the frame below I open up the raw height to 3185
if you crop off the bottom of the image to remove white/gray pixels you get 3176 vertical

1984x3185 in 5xzoom (FYI - Lens Canon EF24-70mm f2.8L @ 63mm =100mm cropfactor1.6 @ F4 ISO800 , shutter .5")

(https://i.ibb.co/kGzMWMS/50-D-Full-Height-M31-1853-frame-1-Small.png) (https://imgbb.com/)
1984x3185_50D_Full_Height_M31-1853_frame_1.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_Full_Height_M31-1853_frame_1.png)

I also recorded a few second 1984x3185 @ 3.885fps
50D_3.885fps_M31-1853.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_3.885fps_M31-1853.mp4)
and I exported a 24fps version , 3 second from 21
50D_24fps_M31-1853.mp4 (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_24fps_M31-1853.mp4)

I did have full width 4768x3185 @1.96fps but when I tried to save a raw image dump it lockup  >:(
I was able to save the LV dump

(https://i.ibb.co/XLnGqx7/LV-032-small-full-Liveview.png) (https://imgbb.com/) (https://i.ibb.co/dgMNBHK/VRAM24-small.png) (https://imgbb.com/)

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Wlad81 on June 01, 2020, 12:47:18 PM
Quote from: reddeercityWorking on 50D Full Height in FLV preset
Maybe there's something that I don't understand, but what is the need in this mode? Vertically oriented low frame rate video, when most screens are horizontally oriented? There are only smartphones that are vertically oriented, but what is the need of shooting raw to watch the final result on the small screen?

The very thing is needed about 50D is the compressed 10, 12 and 14 bit raw mode.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: d!g!c on June 01, 2020, 01:27:47 PM
This is to increase vertical resolution. Current limit is 1080 in 5x crop mode.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 02, 2020, 08:20:01 AM
Ok , I'm able to record Full Liveview on 50D 4768x3176 @ 1.96 fps 10bit raw video (at time of writing , but should be able to get 3.5-4fps)

(https://i.ibb.co/4VX1QQZ/50-D-FLV-Lut-M01-2254-00001-small.png) (https://imgbb.com/)
Here 16bit PNG from MLV Producer , used the "Fast Color" LUT that's provided in MLV Producer 
50D_FLV_Lut_M01-2254_00001.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_FLV_Lut_M01-2254_00001.png)

Export a 11Mb h264 clip @ 24fps , so 3second from 44 ( the original frame was 1.96fps)
50D_FLV_Lut_24fps_M01-2254.mov (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/50D_FLV_Lut_24fps_M01-2254.mov)

I Still need to adjust the horizontal position on the right hand side to remove the black bar  , then code in to the crop_rec .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: deanziyangyu on June 03, 2020, 07:20:51 PM
Quote from: Bender@arsch on May 12, 2020, 04:45:36 PM
@deanziyangyu
Can you record a short smartphone movie from your ml settings and how you try to enable the Anamorphic mod?

Usually way is (fresh install) :
1. enable crop_re, mlv_rec and if you want mlv_snd module -> restart

2. Raw video (MLV) set to "on" and set shutterspeed to maybe 1/50 (not 1/8000 or so - > maybe don't work)

3. Open liveview, open ml menu, set resolutions to the highest value. Close ml - > simply press half shutter.

4. Press "+", open ml, crop rec is set to "off", now go to 4k Anamorphic. Half shutter.

5. Have fun

Finally got around to take some screenshots. Here is the video: https://imgur.com/SEoLdZS (https://imgur.com/SEoLdZS)
I pressed "+" at 8 sec and it goes into x5 crop. At 17 sec, ML displays the aforementioned "To use this mode, exit ML menu and press the zoom button (set to x1)".
If I do nothing here and exit to live view to record a clip it looks like this: https://imgur.com/cge21XP (https://imgur.com/cge21XP) To me it looked as if the leftmost and rightmost 1/3 is letterboxed off with the middle having a 1.33x vertical stretch, at the resolution of 2144px.
If I re-enter x1 mode as suggested by ML, a recording would looks like this: https://imgur.com/tKsmqXF (https://imgur.com/tKsmqXF) This is like the the previous image but without the vertical stretch and at a lower resolution.
At this moment (30 sec) the Canon liveview looks as if the bottom is chopped off and the upper part having a 3x horizontal stretch; A ML screenshot here. https://imgur.com/hMCVdVa (https://imgur.com/hMCVdVa) You can see a horizontally-stretched laptop.
If i re-enter video menu here I see the 1856px resolution with x3.02 crop: https://imgur.com/HbObcEp (https://imgur.com/HbObcEp) and no way of selecting higher resolution than this: https://imgur.com/CPgrb0t (https://imgur.com/CPgrb0t)

My many attempts at simply enabling 4k anamorphic mode on 5D2 have had zero success. @reddeercity Would you hopefully shed some light here as well? I am using your December 31st build and I have tried many other builds as well.

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 04, 2020, 05:40:21 AM
Here a simple picture tutorial https://www.magiclantern.fm/forum/index.php?topic=19336.msg218753#msg218753
2 thing I saw from your screen shots ,
1st: NO "AF" , there can not be any Auto Focusing enabled in Crop_Rec this a limitation with exception
every thing must be manual and that include top mode dial .

2nd: No Auto White balance , & you must have the shutter set to 1/48th !!

The 5D2 must always be in manual mode for ML Crop_Rec .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: deanziyangyu on June 05, 2020, 10:32:36 AM
Quote from: reddeercity on June 04, 2020, 05:40:21 AM
Here a simple picture tutorial https://www.magiclantern.fm/forum/index.php?topic=19336.msg218753#msg218753
2 thing I saw from your screen shots ,
1st: NO "AF" , there can not be any Auto Focusing enabled in Crop_Rec this a limitation with exception
every thing must be manual and that include top mode dial .

2nd: No Auto White balance , & you must have the shutter set to 1/48th !!

The 5D2 must always be in manual mode for ML Crop_Rec .
Thank you for your tips. I set Mode dial, WB and focusing to manual, and followed your tutorial again, but wasn't able to get 4096x590 on the menu however I try. I can only get shutter speed at either 1/45 or 1/50 using Expo. override or SS fine tuning. And the weird 3.02 crop and live view white bar still persist.
Although I have no use for it, I can neither make the camera record 48fps.
May be there is something special with my particular 5D2? Are every variant built the same?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 06, 2020, 06:40:38 AM
did you go to the first post on the thread ?
If you did you would see the very big mistake you made
There's a reason Why I make these tutorial as simple as possible
QuoteNext setup Liveview preview , This is the most important part without this set correctly it will not work !
(https://image.ibb.co/gKpLmV/VRAM8-small.png) (https://imgbb.com/)
Liveveiw needs to be set to ML Gray Scale
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on June 06, 2020, 01:04:46 PM
@deanziyangyu

I see your video multiple times to understand what going wrong and finally a can replicate your problem.

There is one simple change you need to set:
Camera on, open Menu (Canon), go to the video settings and change PAL to NTSC.

Finished!

Another little think is, if you are in 5x zoom, don't press "+" again ;)

@Reddeercity
In your newest version you can use preview option auto. It works while recording;) grayscale is not necessary.

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: deanziyangyu on June 08, 2020, 10:08:30 AM
Quote from: Bender@arsch on June 06, 2020, 01:04:46 PM
@deanziyangyu

I see your video multiple times to understand what going wrong and finally a can replicate your problem.

There is one simple change you need to set:
Camera on, open Menu (Canon), go to the video settings and change PAL to NTSC.

Finished!

Another little think is, if you are in 5x zoom, don't press "+" again ;)

@Reddeercity
In your newest version you can use preview option auto. It works while recording;) grayscale is not necessary.

Great! Except I am in NTSC mode already... Using PAL mode with crop_rec leads to multiple lockups and battery pulls on my camera.
Either way thank you so much for the help! I might just have to stick with a combination of FF and 3K shots.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 10, 2020, 08:01:26 AM
Ok I made a short video on setup crop_rec.module for the 5D2 and enabling 3x1 4k .
I didn't spend too much time here on the video , it was more to test 1x3 4k on the 50d  :D
So Yea 1x3 4k (1376x1746 = 4128x1746 @ 24fps) , just did a quick grade & export from MLV App .
Looks good so far now the challenge is to code it without it locking up like 5D2   >:(



                                                                                                                                          Make sure you watch it in 4K  ;D

Back to the setup problem of @ deanziyangyu , follow this it should would work without a problem .
One last time You Can't have Crop Mark Enabled !! this will cause crop_rec not to work
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: okayokay on June 10, 2020, 11:32:59 AM
Hey, I just began trying Magic Lantern on my 5D II, I am using the 3K build to record at 2.39:1 at 10 bit, 1856x776, but I got a few problems

One, when recording, I don't have crop marks so I have to guess the framing, how can I enable it?

Also, I cant enable mlv_play in modules at the same time as mlv_rec and mlv snd or I get an

ttc: error: undefined symbol 'lossless_decompress_raw' [e] failed to link modules

That gets annoying, since I can't preview my RAW videos, which BTW, I find having some extreme noise all over the images in some cases, even in lit scenes, the shadows are full of that annoying noise, but I guess it is what it is, can't complain

Also, I am looking to buy a cheap monitor so I can nail focus easier, I have heard you can force HD signal off the 5DII, anyone has tried playing back a clip with a monitor attached and had no problems?



Thanks
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on June 13, 2020, 09:06:27 PM
Hey Dave was there any progress in lossless for 5D2 ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on June 13, 2020, 09:57:27 PM
@okayokay: this build is bleeding edge and work in progress. If it is anoying for you, I recommend to implement the features you need yourself, or you wait until it is "ready" or you use another build.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Wlad81 on June 15, 2020, 12:00:49 AM
Quote from: 2blackbar on June 13, 2020, 09:06:27 PM
Hey Dave was there any progress in lossless for 5D2 ?
The same question is about 50D. Needed a lot in September for work.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: okayokay on June 17, 2020, 07:16:15 PM
Quote from: masc on June 13, 2020, 09:57:27 PM
@okayokay: this build is bleeding edge and work in progress. If it is anoying for you, I recommend to implement the features you need yourself, or you wait until it is "ready" or you use another build.

Somehow the nightly build for the 5D2 doesn't offer an option for 10bit raw video. That selection is the only reason I am using this build, do you know of any other I could use? Thanks
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on June 17, 2020, 09:15:27 PM
Top of page -> Downloads -> Downloads (drop down menu) -> Experiments -> 10/12-bit RAW video
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: okayokay on June 17, 2020, 10:33:12 PM
Quote from: Walter Schulz on June 17, 2020, 09:15:27 PM
Top of page -> Downloads -> Downloads (drop down menu) -> Experiments -> 10/12-bit RAW video

Thanks! Also, I have noticed the cropmarks in the Overlay menu dont appear once I am recording, is there any way to get some marks for reference when recording 2:39, since I can use them before recording but once I start it just goes to full view. Also, is there any kind of timer or timecode available so I know for how long I've been recording? I see that appears sometimes, for example when I enabled HDR video just to test it, but after disabling everything but mlv_rec, play and snd it wont appear anymore

Thanks
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on June 17, 2020, 10:39:57 PM
Movie tab -> RAW video (MLV) -> [Q] -> Global Draw Allow
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 18, 2020, 02:01:03 AM
Yes but you reduce the record time & there will not be continuous recording in the higher res e.g. 1856x1044 , 1856x928 .
Best option is to use HDMI (forced to 480p) out to a cheap evf/monitor with crop marks (but i never use them) , this will give continuous recording in the higher res .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: okayokay on June 18, 2020, 12:46:43 PM
Quote from: Walter Schulz on June 17, 2020, 10:39:57 PM
Movie tab -> RAW video (MLV) -> [Q] -> Global Draw Allow

Thank you! I appreciate your help! I have been noticing I lose my modules tab when I format the CF card from the camera, so I just cut the videos off it for now without formatting, even though I am sure keep ML is selected

Quote from: reddeercity on June 18, 2020, 02:01:03 AM
Yes but you reduce the record time & there will not be continuous recording in the higher res e.g. 1856x1044 , 1856x928 .
Best option is to use HDMI (forced to 480p) out to a cheap evf/monitor with crop marks (but i never use them) , this will give continuous recording in the higher res .

I've been thinking about getting a monitor, I hear you could force 720p out though, don't know if that is true. I'm going to try in 1856x776, I always record that way (2.39:1) for now.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on June 18, 2020, 01:02:14 PM
Quote from: okayokay on June 18, 2020, 12:46:43 PM


I have been noticing I lose my modules tab when I format the CF card from the camera

Module tab should only disappear after abnormal shutdown. Debug -> Modules debug -> Load modules after crash OFF (default). 5D2 is among the cams able to harm cam and card if mishandled!

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on June 22, 2020, 08:44:17 PM
Quote from: Walter Schulz on June 18, 2020, 01:02:14 PM
Module tab should only disappear after abnormal shutdown. Debug -> Modules debug -> Load modules after crash OFF (default). 5D2 is among the cams able to harm cam and card if mishandled!

Walter, can you suggest good practices for us 5D2 users, to keep from harming our cameras?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on June 22, 2020, 08:48:00 PM
No, but a1ex already did:
https://www.magiclantern.fm/forum/index.php?topic=1455.msg3943#msg3943
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 23, 2020, 01:03:49 AM
No , this a bug in this branch that I have not address yet , know fact when formatting cf card on this branch.
FYI , I never re-format my CF Card ever , no need for that in my opinion I just move the mlv file off the CF Card
when needed and besides that ever time you re-format a flash drive (includes CF & SD Card) you take a more life out
of the card & get premature failure . I'm still using my original CF cards from 2013 (Lexar 1000x 64 & 32GB) with raw video
plus all the development was done on these cards , so they have been heavily abused for hours at a time until they are hot to the touch  :o

The procedure I use to shut down the camera to get the CF Card out is
1st: turn the power switch off
2nd: wait for the red led flash & the blue led light flash on the Lv button
3rd: open the CF Card door , wait for the red led light flash (1-2 second at most)
4th: remove CF card .

Now if you have a Lockup/frozen Liveview or non-responsive camera  etc. .... open the battery door wait for the red led light flash then drop/remove the battery out .
Then reinstall the battery (if the camera is not too hot , if so let it cool down) and start up the camera normally .
You may get a error message on the LCD screen at time of bootup saying
"Improper shutdown , modules will not load"
Just power off the camera with power switch wait until the red led flashes then just re-boot the camera .
It will startup with all the modules you had loaded before the Improper shutdown .
Don't worry , it a safety net in case there is a problem with the module to protect the camera ,
I run in to this all the time when I'm experimenting with crop_rec presets when I push the camera too far .   
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 06, 2020, 03:05:43 AM
Updated mlv_rec module (mlv_rec.mo) for max (canon default) resolution in 3x3 & 5x Zoom on the 5D2
Now you can get in 3x3 (FHD) 1880x1248 (old res 1856x1248) and 3xcrop_mode (5xZoom) 2152x1074 (old res 2144X1074)

Download Link
1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip)

It will load on top of my eXperimental Crop_Rec builds , did not test it with the nightly builds

(https://i.ibb.co/JvnTghP/VRAM103-small.png) (https://imgbb.com/) (https://i.ibb.co/0cWP49X/VRAM104-5x-Zoom-small.png) (https://imgbb.com/)

Frame exported from MLV App. 3x3_1880x1248_mlv_rec-M05-1821_frame_1.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3x3_1880x1248_mlv_rec-M05-1821_frame_1.png)
3x Crop_mode Frame exorped from MLV App. 3xcrop_2152x1074_mlv_rec-M05-1823_frame_1.png (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/3xcrop_2152x1074_mlv_rec-M05-1823_frame_1.png)


(https://i.ibb.co/2vLB7Mt/5d2-1880-mlv-rec-mlv-app-1.png) (https://imgbb.com/) (https://i.ibb.co/Mpmr6Zj/5d2-2152-mlv-rec-mlv-app2.png) (https://imgbb.com/)

So a side effect of this is in crop_rec 3k preset (3008x1080) now becomes 3016x1080 , so you gain 8 more pixel
In 3x3 you gain 24 more pixels (1856->1880) & in 3xcrop_mode(5xZoom) there a gain of 8 pixels (2144->2152)




Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on July 06, 2020, 03:24:12 AM
@Reddeercity, thanks so much! I'll get to testing straight away!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 06, 2020, 03:28:36 AM
If anyone is curious on what i change to the code (thanks @Levas for reminded me about the code in mlv_lite.c)  ;D
static void update_resolution_params()
{
    /* max res X */
    /* make sure we don't get dead pixels from rounding */
    int32_t left_margin = (raw_info.active_area.x1 + 7) & ~7;
    int32_t right_margin = (raw_info.active_area.x2 + 0) & ~7;
    int32_t max = (right_margin - left_margin);

    /* horizontal resolution *MUST* be mod 32 in order to use the fastest EDMAC flags (16 byte transfer) */
    //max &= ~31;
   
    max_res_x = max;

    /* max res Y */
    max_res_y = raw_info.jpeg.height & ~1;


I changed 3 things so it could work with mlv_rec.c

Change these 2 lines in mlv_rec.c

int32_t left_margin = (raw_info.active_area.x1 + 7) / 8 * 8;
int32_t right_margin = (raw_info.active_area.x2) / 8 * 8;

To this from mlv_lite.c

int32_t left_margin = (raw_info.active_area.x1 + 7) & ~7;
int32_t right_margin = (raw_info.active_area.x2 + 0) & ~7;


Also commended out this line

/* horizontal resolution *MUST* be mod 32 in order to use the fastest EDMAC flags (16 byte transfer) */
    //max &= ~31;


From this in mlv_rec.c

/* horizontal resolution *MUST* be mod 32 in order to use the fastest EDMAC flags (16 byte transfer) */
    max &= ~31;

The last one , as you can see really tell the story about the horizontal resolution

Edit: if there any broken download links be sure let me know , as I'm still using bitbucket fot my unloads  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 06, 2020, 03:50:03 AM
Updated the first post of this thread with the updated mlv_rec.mo download link
https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: CITY-U1001 on July 07, 2020, 10:56:49 AM
@reddeercity you have any news about 50d in crop ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on July 07, 2020, 07:25:22 PM
Quote from: reddeercity on July 06, 2020, 03:05:43 AM
Now you can get in 3x3 (FHD) 1880x1248 (old res 1856x1248) and 3xcrop_mode (5xZoom) 2152x1074 (old res 2144X1074)

Download Link
1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip)


You are a god. Thank you!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 08, 2020, 05:11:47 AM
Quote from: CITY-U1001 on July 07, 2020, 10:56:49 AM
@reddeercity you have any news about 50d in crop ?
It's tough nut to crack , the cmos & atdg hooks are not work correctly , cmos = pinning of columns(3x3 or 1x1) + horizontal & vertical offset
adtg = 1x3 pixel binding , full sensor readout + many other feature that need the adtg hooks to work . So it still a work in process .

Brighter side of thing the new mlv_rec.mo I updated works on the 50D also ,
This give a increase in resolution in 3x3 & 1x1 (5xZoom)
1568x1056 -> 1584x1056 in 3x3 & 3xcrop_mode(5xZoom) 1984x1078 -> 2000x1078

in 3x3 you gain 16 more pixels & in 5xZoom you gain 16 pixels also

(https://i.ibb.co/yYvVQVG/VRAM47-small.png) (https://imgbb.com/)  (https://i.ibb.co/ydtn19s/VRAM48-small.png) (https://imgbb.com/)

Download link
1880_3x3-2152_5xZoom-5d2+50d-support__7-5-2020-5.59pm-mlv_rec.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 10, 2020, 08:15:08 AM
Update 5D2 Crop_Rec.mo
5D2_7-9-2020_11.31pm_crop_rec.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2_7-9-2020_11.31pm_crop_rec.zip)

I updated the 5d2 crop_rec module , it will switch to centered 3x_mode(5xZoom) a lot faster then before , Almost instant you push enable
crop_rec 3.5k centered loads very fast , this is first part of the 3x crop_mode check so i can add more 3x crop_mode preset's .
It will load on top of the my eXperimental builds , with the updated mlv_rec.mo , 48pFHD preset now is 1880x774 @48fps  :D instead of 1856x774 .
All the rest of the preset resolution's still the same .
I'm closer to getting more  5xzoom preset's working in the same build , I can get either centered preset to load or the 3K preset to load not both
One will always far back to 3x3  >:(

Here the changes i made the the 3x crop_mode check that work right now with the posted crop_rec module  , I back out my multiply 3xcrop_mode check ,
see below for the back out changes

  /* cmos_vidmode_ok doesn't help;
     * we can identify the current video mode from 0xC0F06804 */
    for (uint32_t * buf = (uint32_t *) regs[0]; *buf != 0xFFFFFFFF; buf += 2)
    {
        uint32_t reg = *buf;
        uint32_t old = *(buf+1);
        //if (!is_5D2 &&(reg == 0xC0F06804))
        if (reg == 0xC0F06804)
        if (crop_preset == !CROP_PRESET_CENTER_Z)
    {
        engio_vidmode_ok =  (old == 0x46A04BA);                        /* x5 zoom */
    }
    else if (crop_preset == CROP_PRESET_CENTER_Z)
    {
         engio_vidmode_ok =  (old == 0x4F40432);                       /* 1080p */
   }
    else // If the two previous conditions are wrong the last one is necessary true
    {
        engio_vidmode_ok =  (old == 0x46F066A);               /* 3xcrop_mode*/
    }         
       // {
          //  engio_vidmode_ok = (crop_preset == CROP_PRESET_CENTER_Z || crop_preset == CROP_PRESET_4K_HFPS)
           //     ? (old == 0x56601EB)                        /* x5 zoom */
           //     : (old == 0x528011B || old == 0x2B6011B);   /* 1080p or 720p */
       // }
        else
        {
          if (reg == 0xC0F06014)      // waza57 for 5D2 without this reg check,  0xC0F06084 is not seen 
            {                 
               if (crop_preset == CROP_PRESET_CENTER_Z)
                {
                  if ((old == 0x4ff) || (old == 0x4d5 ))
                    {
                      engio_vidmode_ok = 1;
                    }
                 //if (crop_preset == CROP_PRESET_4K_HFPS)
                 //{
                  //if ((old == 0x4ff) || (old == 0x712 ))
                    //{
                     // engio_vidmode_ok = 1;
                   // }
                  else
                    {
                      engio_vidmode_ok = 0;
                    }
                 }   
               if (crop_preset == !CROP_PRESET_CENTER_Z)
                {
                   if (old == 0x577)
                    {
                      engio_vidmode_ok = 1;
                    }
                    //if (crop_preset == CROP_PRESET_4K_HFPS)
                // {
                  //if  (old == 0x577)
                   // {
                     // engio_vidmode_ok = 1;
                   // }
                 else
                    {
                     engio_vidmode_ok = 0;
                    }
                } 
            }           
        } 
    }
    if (!is_supported_mode() || !engio_vidmode_ok)
    {
        /* don't patch other video modes */
        return;
    }


This what i'm can't get to work right
if (crop_preset == CROP_PRESET_3K && !CROP_PRESET_CENTER_Z)
    {
        engio_vidmode_ok =  (old == 0x46A04BA)                        /* x5 zoom */
    }
    else if (crop_preset == CROP_PRESET_3K && CROP_PRESET_CENTER_Z)
    {
         engio_vidmode_ok =  (old == 0x4F40432);                       /* 1080p */
    }
    else  // If the two previous conditions are wrong the last one is necessary true
    {
         engio_vidmode_ok =  (old == 0xnew_value);               /* 3xcrop_mode*/
    }


"engio_vidmode_ok =  (old == 0xnew_value);               /* 3xcrop_mode*/"
This part i just in put the 3k raw number 0x46F066A
code doesn't like "0xnew_value" , i think i need to made a new function for  "0xnew_value", not sure yet


I'll also undated the 50D Crop_Rec module with the faster 3xcrop_mode check in a few days .

Test the crop_rec.mo before i upload it in a full build .
48p fhd can be a little tough to get into , start of in 3xcrop_mode then enable 48p fhd
:)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 11, 2020, 05:46:52 AM
Success !!
I finally have more then one 3x crop_rec (5xZoom) preset load without falling back to 3x3  :D
Here the new Crop_Rec Module download link  5D2-4k3x1-3k1x1-2.8k1x1-48pfhd3x3_7-10-2020_8.42pm_crop_rec.zip (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2-4k1x3-3k1x1-2.8k1x1-48pfhd3x3_7-10-2020_8.42pm_crop_rec.zip)

(https://i.ibb.co/4pv6749/VRAM106-small.png) (https://imgbb.com/) (https://i.ibb.co/0Xbpx5G/VRAM105-small.png) (https://imgbb.com/)
(https://i.ibb.co/ZRrWZGB/VRAM107-small.png) (https://imgbb.com/)

I was so very close before , but i keep over looking the begin of the code for the 3xcrop check function and assumed that it was correct , it wasn't for the 5D2.
That been the story all along , close but not close enough .

Here the code for 5d2 3x crop_mode check , Thanks to @alexboum  for the help with this! This is a Big Thing for d4 cam.
/* cmos_vidmode_ok doesn't help;
     * we can identify the current video mode from 0xC0F06804 */
    for (uint32_t * buf = (uint32_t *) regs[0]; *buf != 0xFFFFFFFF; buf += 2)
    {
        uint32_t reg = *buf;
        uint32_t old = *(buf+1);
        //if (!is_5D2 &&(reg == 0xC0F06804))
        if (!is_5D2 &&(reg == 0xC0F06084))
        if (crop_preset == !CROP_PRESET_CENTER_Z)
    {
        engio_vidmode_ok =  (old == 0x46A04BA);                        /* x5 zoom */
    }
    else if (crop_preset == CROP_PRESET_CENTER_Z)
    {
         engio_vidmode_ok =  (old == 0x4F40432);                       /* 1080p */
   }
    else // If the two previous conditions are wrong the last one is necessary true
    {
        engio_vidmode_ok =  (old == 0x46F066A);               /* 3xcrop_mode*/
    }         
       // {
          //  engio_vidmode_ok = (crop_preset == CROP_PRESET_CENTER_Z || crop_preset == CROP_PRESET_4K_HFPS)
           //     ? (old == 0x56601EB)                        /* x5 zoom */
           //     : (old == 0x528011B || old == 0x2B6011B);   /* 1080p or 720p */
       // }
        else
        {
          if (reg == 0xC0F06014)      // waza57 for 5D2 without this reg check,  0xC0F06084 is not seen 
            {                 
               if (crop_preset == CROP_PRESET_CENTER_Z)
                {
                  if ((old == 0x4ff) || (old == 0x4d5 ))
                    {
                      engio_vidmode_ok = 1;
                    }
                  else
                    {
                      engio_vidmode_ok = 0;
                    }
                 }   
               if (crop_preset == !CROP_PRESET_CENTER_Z)
                {
                   if (old == 0x577)
                    {
                      engio_vidmode_ok = 1;
                    }                   
                 else
                    {
                     engio_vidmode_ok = 0;
                    }
                } 
            }           
        } 
    }
    if (!is_supported_mode() || !engio_vidmode_ok)
    {
        /* don't patch other video modes */
        return;
    }

So from yesterday this what i changed 
I commend out this line
//if (!is_5D2 &&(reg == 0xC0F06804))
and replace it with this and clean up the code a little .
if (!is_5D2 &&(reg == 0xC0F06084))

Usage:
Start out in 3x3 , enable the 3.5k Centered (2784x1160) , press 5xZoom button to fresh to just in to 2784x1160 .
Then after that you can go to 3k 1:1 (3016x1080) ,  fresh liveview , screen will black press the play button 1-2 times
then fresh liveview with the 5xZoom button ( you may have to cycle though with the zoom button at least twice )
then the 3k preset will come up .
next you can go to the 4k 3x1 from 3k , just enable it and refresh liveview .
Now 48p FHD  , you need to start off in default 3xcrop_mode , then enable the 48p preset , it will load from there .
There may be different ways to enable all 4 presets , but i found this seems to work the best .

Recap: 3k may a little hard to enable , just clyce thought the zoom button , Later I'll post a video on how to
fully enable the presets trouble free .  :D   

Edit: I deleted the previous crop_rec from yesterday .   
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 11, 2020, 08:27:32 AM
Quote from: reddeercity on July 11, 2020, 05:46:52 AMI'll post a video on how to
fully enable the presets trouble free .  :D   
sounds very good :)

Wow, I will be trying out this today
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Bender@arsch on July 11, 2020, 12:59:52 PM
Very nice!
Is it now possible with 5.6K Build too?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on July 11, 2020, 08:39:59 PM
Reddeercity, thanks again for all your hard work on this! I tested the new mlv_rec module "1880_3x3-2152_5xZoom-5d2+50d-support__7-5-2020-5.59pm-mlv_rec.zip" with the latest "crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip" build. I'm loving the new 3x3 resolution!

I noted four things for the 5D2:
1) In 3x3 1880, and in all bit depths, I'm getting pink spots in the sun, see .dng here > https://www.mediafire.com/view/ssal2p12hvreadi/C-12-BIT-111.dng/file (https://www.mediafire.com/view/ssal2p12hvreadi/C-12-BIT-111.dng/file) Is this something that needs to be repaired in MLV app?
2) In live view movie mode, ML reports 1/48s shutter speed, but the top LCD of my 5D2 reports 1/90s. And when switching out of live view, and capturing a still, it is captured at 1/90s.
3) Forgive me if you've already addressed this, but the fastest shutter speed I can achieve in mlv_rec (3x3 1880) is 1/100s despite the top LCD reporting all shutter speeds up to the max 1/4000s.
4) mlv_play isn't working and causes no modules to load.

Thanks again for your continued development!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on July 11, 2020, 09:25:20 PM
@Milk and Coffee:
1) you overexposed / clipped the green channel. This brings pink color. Try to reconstruct highlights!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on July 11, 2020, 11:17:11 PM
Thanks masc,

Even after highlight reconstruction, there is a clear pink spot right in the center of the sun. See images below.

https://www.mediafire.com/view/j0ahekfwxgnrz2p/Screen_Shot_2020-07-11_at_4.13.33_PM.png/file (https://www.mediafire.com/view/j0ahekfwxgnrz2p/Screen_Shot_2020-07-11_at_4.13.33_PM.png/file)

https://www.mediafire.com/view/lsgffzpvk0pg811/Screen_Shot_2020-07-11_at_4.13.59_PM.png/file (https://www.mediafire.com/view/lsgffzpvk0pg811/Screen_Shot_2020-07-11_at_4.13.59_PM.png/file)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 11, 2020, 11:29:44 PM
Can't help without the original  dng/mlv file shorten  , not jpeg compression (lossless) or png .
just use passthou when exporting dng's in mlv app. and it will export dng's at the original bit depth
Did you try Adobe ACR ? (Photoshop/After Effects)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on July 11, 2020, 11:32:13 PM
Original DNG here  https://www.mediafire.com/view/ssal2p12hvreadi/C-12-BIT-111.dng/file (https://www.mediafire.com/view/ssal2p12hvreadi/C-12-BIT-111.dng/file)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 12, 2020, 12:38:07 AM
Quote from: Milk and Coffee on July 11, 2020, 11:32:13 PM
Original DNG here  https://www.mediafire.com/view/ssal2p12hvreadi/C-12-BIT-111.dng/file (https://www.mediafire.com/view/ssal2p12hvreadi/C-12-BIT-111.dng/file)
Quote from: reddeercity on July 11, 2020, 11:29:44 PM
Can't help without the original  dng/mlv file shorten  , not jpeg compression (lossless) or png .
......

(https://i.ibb.co/s3wf1QD/lossless-no-default-5d2.png) (https://imgbb.com/)

As i said this is jpeg Lossless , i need the uncompressed dng , see what it say at the compression line
5D Mark ii Dose not shoot Lossless Compression
The file size of the dng should be around 4-5Mb not 1.23Mb
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on July 12, 2020, 06:15:09 AM
Oh! I thought you meant JPEG as a file type. Let me shoot another test as I don't have the original mlv. Or can I make another uncompressed DNG from this one?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 12, 2020, 08:15:14 AM
read this post to get up to speed  https://www.magiclantern.fm/forum/index.php?topic=19336.msg228941#msg228941
Ok , I haven't had time to make a video yet on the procedure to enable the new crop_rec.mo with multiplies 3x crop_mode preset .
So here is the way to enable crop_rec preset's

Note** Always start with No.1 Step**
1: Start off in 3x3 , enable the "3.5k centered  preset (2784x1160) , refresh liveview (half shutter)
then press the 5xZoom button "centered preset" will load from there .

2: To enable 3k 1:1 (3016x1080) , press trash can button , to bring up the ml menu , enable the 3K 1:1 preset
**Note** do not exit 3.5k centered preset before enable 3k preset , just go to the crop_rec menu & enable.
fresh liveview with half shutter , the lcd screen will be black , don't worry now press the play button twice (2x)
then cycle though the 5xZoom button (usually  press it 1 time , sometimes it needs to be press twice)  .
The most important part is Press the play button 2 times (but not too fast thou) .
Now once you are in 3k (3016x1080) you can go to 4k 3x1 (4096x590) from there.

3: Just enable the 4k1x3 preset while in the 3k 1:1 preset and refresh liveview (half shutter) the preset will load itself from there .

4: 3x3 48p FHD (Full HD) needs to load from the default 3xcrop_mode (2152x1078) , enable the 48p preset then refresh live
(you may have to cycle thou the 5xZoom button on the odd time) , but it should load as soon as you refresh liveview .

 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 13, 2020, 07:13:44 AM
I found some time today to shoot a short video on how to enable the new crop_Rec.mo (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2-4k1x3-3k1x1-2.8k1x1-48pfhd3x3_7-10-2020_8.42pm_crop_rec.zip)  with multiplies 3xcrop_mode preset's
Shot it on my 50D in 1x3 1376x1742 (4128x1742) @24fps , did a quick grade in mlv.app and exported and uploaded prores 444 to Youtube
Make sure you watch in 4k  ;)


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on July 17, 2020, 12:31:22 AM
Quote from: reddeercity on July 11, 2020, 11:29:44 PM
Can't help without the original  dng/mlv file shorten  , not jpeg compression (lossless) or png .
just use passthou when exporting dng's in mlv app. and it will export dng's at the original bit depth
Did you try Adobe ACR ? (Photoshop/After Effects)

New sample DNG's here (included both uncompressed/passthrough, and lossless compressed) http://www.mediafire.com/folder/lelofkxvrvyhw/Pink-Sun (http://www.mediafire.com/folder/lelofkxvrvyhw/Pink-Sun)

Different clip, but same problem; pink pixels in the center of the sun in Davinci Resolve. It's especially apparent with "highlight recovery" turned on.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 17, 2020, 01:56:18 AM
I don't use Resolve , it does a trouble job of raw from magic lantern specialty from 5D2 , I never seen anything good from it (there reason why it's free)
Your better off using MLV App to develop your dng's , I use Adobe After Effects CS6 .

I See many problems here , First you Badly Over Expose the image & there too much data loss in the high light to be reconstructed .
From the Metadata from the dng you use F2.0 @ 1/48th shooting the sun , plus your are out of focus .
Better set up would be , F8 plus a (Variable ND) and or ND 0.6-0.9 or F16 with a Variable ND 0.3-400 .
There no programs the can fix data loss , you need to trust the ML meters in liveview  and ETTR (expose to the right) .

I run the dng thought Adobe A.E. cs6 and i did get a not too bad image with out any pink sun , but it took some time .

(https://i.ibb.co/StQ0X76/1-Uncompressed-77-0-00-00-00.png) (https://imgbb.com/)
Here the exported tif from cs6 1-Uncompressed-77_0-00-00-00.tif (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1-Uncompressed-77_0-00-00-00.tif)
Also here is the 1-Uncompressed-77.dng (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/1-Uncompressed-77.dng) from After Effect with the ACR adjustment make
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on July 17, 2020, 02:08:04 AM
Thanks for taking the time! It was just a quick clip from my apartment patio. I intentionally overexposed to make sure the pink highlights would show in Davinci.

Your graded image looks great! Good highlight roll off!

Any Davinci users that can shed some light on the pink pixels happening in Davinci?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on July 17, 2020, 03:23:05 AM
it's just a matter of making sure the highlights actually reach the top of the signal range.

this took about 30 seconds in resolve.

boosted highlights with the log wheels, and then made a quick curve to extend the range to fill out the scopes as shown in the overlaid scopes.

https://ibb.co/zntbzhs
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on July 17, 2020, 05:06:08 AM
I appreciate all the reply's, and grading suggestions! Maybe I didn't clarify enough: The DNG is a sample to demonstrate the pink pixels in the sun. I didn't shoot it with intentions of using it. 

Seems to be a bug with Davinci? Or with ML? Didn't know if anyone had any insight as to why they are appearing.

70MM13, did you notice the pink pixels?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ZEEK on July 17, 2020, 08:47:50 AM
I haven't had that issue yet with the 5D2. It has to do I believe with the White Level of the Camera. The topic was discussed, you can check it out here [Reply #3289]:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg219884;topicseen#msg219884
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 17, 2020, 09:45:47 AM
Has nothing to do with it , read the thread where you posted from , it has to do with the eosm with lossless compression .
That does not apply here at all !
There no information in the high light at all from the posted sample , it a user setup problem.
As I posted the corrected image from ACR/A.E. , resolve can't handle the high light reconstruction like ACR , actually there no program/app that can reconstruct high light like ACR .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on July 17, 2020, 12:25:40 PM
i'm not sure if you guys looked at the image i sent...

i pushed it just enough to get rid of them for my taste.  i didn't want to push any further than necessary or else there would be even more useful visual information lost.  if any remained in my image, you must be able to see something i cannot see here.

so just try it yourself and push a little further if you are still seeing them.  and i had highlight recontstruction ON.  i like what it does to the halo of the sun.  there are definitely more pink pixels with reconstruction on.

if you are hypersensitive to such almost invisible things, you can simply create another layer, select that colour with the eyedropper and drop the saturation to zero on that layer.  gone without a trace.

i don't care what program anyone uses.  i'm just offering help with resolve as the poster asked.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on July 17, 2020, 01:54:44 PM
Pink dots in highlights is not related to lossless compression. On the eosm I modified one of the cmos settings which was causing the issue. Reverting the change fixed the issue. Maybe this could be fixed on 5d2 as well tweaking a cmos reg. Or maybe it's wrong in the preset you recorded in.
Resolve is superior to premiere in so many ways nowadays imo handling dng files which is the more fair tool comparison. Anyone can pull the highlight slider in acr on s single image. Try applying it to a sequence and the flicker party started. But, with some dcp tweaking acr actually works really good even for dng sequences but still. Cutting in after effects takes forever. Too bad adobe don't develop their own programs for their own dng format. Strange.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on July 17, 2020, 02:01:46 PM
do you remember what cmos setting exactly was involved?  it might be very useful to know this for other times this issue pops up!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on July 17, 2020, 02:11:53 PM
I don't. Dig around from the post from Zeek. Shouldn't be far from the fix.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 18, 2020, 04:56:23 AM
@70MM13 , yes i looked at your sample .
Quote from: Danne on July 17, 2020, 01:54:44 PM
.... Maybe this could be fixed on 5d2 as well tweaking a cmos reg. Or maybe it's wrong in the preset you recorded in.
I don't think you look at the sample "1-Uncompressed-77.dng" , it's a default 3x3 from mlv_rec  (1880x1058)
Don't asume  it a crop_rec issue or anything without totally check the sample .

easily fix in RawTherapee with one step , High Light Recovery - Luminance Recovery
That's because of the data loss from over exposing too much .
(https://i.ibb.co/16FZyt0/5d2-no-pink-sun-reconstructed-setting.png) (https://imgbb.com/)

(https://i.ibb.co/q01qnWJ/5d2-no-pink-sun.png) (https://imgbb.com/)

Ok i take it back , there is another program (RawTherapee) that handles high light as good if not better then ACR/After Effect CS6
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on July 18, 2020, 12:06:50 PM
that's the clip i used...

i agree about how great rawtherapee is.  in fact, i used it to process my first music video with the 5d3 in raw video back in 2017.  talk about a slow process, but it looked beautiful.  there were a few issues however in using rawtherapee for video:

first, it crops the image by 8 pixels on each axis.  not a fatal flaw, but not nice.

second, third, fourth etc, several of the tools can't handle changing light conditions.  unfortunately, it was a problem with some of the best tools.

if someone forked rawtherapee for video usage, it would be a very valuable asset for us all...  it does better debayering than anything i have tried.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on July 18, 2020, 04:16:30 PM
Thanks for all the useful replies! So it sounds like it's an issue with Davinci resolve. Hope they are still updating their cDNG functionality now with all the other raw formats out there.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 70MM13 on July 18, 2020, 04:23:29 PM
i wouldn't hold my breath.  their priority is the blackmagic cameras, and rightfully so.

but it would be nice!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 20, 2020, 04:08:29 AM
Being checking on my source code in bitbucket and notice if you update my branch to the latest commit 8eae4f2 (https://bitbucket.org/reddeercity/magic-lantern_reddeercity_5d2-50d_4k-crop_rec/commits/8eae4f2d0ab9cdef59c2969afcec11413512ed05)
you will get a raw.c error
../../src/raw.c:1840:12: error: redefinition of 'lv_raw_type'
static int lv_raw_type = PREFERRED_RAW_TYPE;

Before bitbucket shut down updating I forgot to comment out raw.c #line 229 like this Sorry  :-[
//static int lv_raw_type = PREFERRED_RAW_TYPE;//
This will fix the compiling error , I did a quick test and compiles fine now .

I wish i had updated the source more , but I'm looking in to git now to host my code unless a1ex has setup a new site to host ml code ?

If you're new to coding and wondering what's a good code editor , being using Visual Studio Code (https://code.visualstudio.com/)  (on win7 , cross platform on mac ,etc. ..) to review the changes from the hg
makes debugging real easy , show the diff between code changes , even helps with code syntax and show any error right now .  :D
(https://i.ibb.co/8cgSvdz/visual-stuidio-code.png) (https://ibb.co/16Gwyds)

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 20, 2020, 11:42:34 PM
updated the first post on page 1 with the new crop_rec.mo download link & short tutorial video on how to enable the presets
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on July 21, 2020, 08:54:49 AM
Hi, Reddeercity!
BIG Thanks for your work.
I tested the new Crop_Rec.Mo and did some videos.
Everything is perfect, except that I couldn't get 3k 1:1 mode work the right way.
Mobile phone video:
https://drive.google.com/file/d/1DYDZtlMTFckaUj5Pka_uxf1wpulynMzd/view?usp=sharing

Other moments:
* Is it possible to add 5k mode here to get all in 1 crop_rec.MO?
* Is it real to unsqueeze the 4k anamorphic preview somehow?
* I keep trying to get unfrozen preview somehow but only can get it in 14 bit... You already answered me on it, but I think it will be a good decision to figure out why there is a live preview in 14 bit mode, but not in 10 bit mode.


PS: sorry, had some unexpected things to do, so i did it only today.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on August 10, 2020, 10:58:44 PM
Per my calculations, Centered 3.5k 1:1 (2784x1160 24fps) gives approx the crop of micro 4/3 on 50d.
Could someone confirm this?

Sent from my Lenovo TB-8703F using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on August 11, 2020, 12:55:04 AM
Quote from: zcream on August 10, 2020, 10:58:44 PM
Per my calculations, Centered 3.5k 1:1 (2784x1160 24fps) gives approx the crop of micro 4/3 on 50d.
Could someone confirm this?


I think I did the math right:
2784 x1160 pixels divided by 50D's 4752 x 3168 pixels = 0.5858585858

So if 50D has a 1.6x crop vs full frame, and then you divide 0.58585858 that's a 2.73x crop

A micro 4/3 crop is 2.0x. So the crop would be closer to a Sony 1" crop of 2.7


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 11, 2020, 02:22:35 AM
It depends , 50d does 3744x1080 So ......
5d2 crop_rec 2784x1160 , 5632/2784= 2.019 crop factor (18mmx12mm used of 36mmx24mm) of full frame 
Micro 4/3rds , sensor (17mmx13mm) 2.11 crop factor or full frame
50D , sensor 22.3mmx14.9mm 1.6 crop factor of full frame , but in 3xcrop_mode you have 2000x1078 = (5.6mmx3.75mm used) 4752/2000 ->(2.37)+1.6=3.97 crop factor of full frame .
Though in Crop_Rec on 50D max res at the mount is 3744x1080 , so final crop factor changes to 4752/3744 = 1.2 crop factor + 1.6 crop factor of full frame = 2.8 crop factor .
So 22.3mmx14.9mm / 2.8 crop factor = 7.9mmx5.2mm used on 50D crop_rec .

So on the 5D2 center preset (2784x1160) (18mmx12mm) is just a bit bigger then 4/3rds sensors (17.3mmx13mm)
50D , 3xcrop_mode (3.97crop factor) (5.6mmx3.75mm) which is close to 1/2'' sensor (6.4mmx4.8mm)
50D , in crop_rec (2.8 crop_factor) (7.9mmx5.2mm) which close to 2/3'' sensor (8.8mmx6.6mm)


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on August 14, 2020, 07:44:26 PM
So far the center preset in only on 5d2. I forgot that part. If that works on 50d, it's about 2x crop.

Sent from my Lenovo TB-8703F using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on August 14, 2020, 07:45:05 PM
This center preset (2784x1160). That's what I was referring to.

Sent from my Lenovo TB-8703F using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 17, 2020, 08:17:15 AM
New updated 50D Crop_Rec
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-eXperimental-crop-rec_3.7k-1x1.11.30pm.2020Aug16.50D109.zip
New download Link from google drive.
magiclantern-eXperimental-crop-rec_3.7k-1x1.11.30pm.2020Aug16.50D109.zip (https://drive.google.com/file/d/)

New Updated Bitbucket Down Load Link(8-27-2020)
magiclantern-eXperimental-crop-rec_3.7k-1x1.11.30pm.2020Aug16.50D109.zip (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/magiclantern-eXperimental-crop-rec_3.7k-1x1.11.30pm.2020Aug16.50D109.zip)

Updated the 50D Crop_Rec with new 5xZoom check (which enables crop_rec with out delay , same as the 5D2)
plus other back-end tweaks for 50D model .
Also has the updated mlv_rec module for increase resolution in non crop mode's as will as Crop_Rec.
3744x1080 before , now 3760x1080 .
I changed how to exit Crop_Rec , after you'er done recording press play button twice and you should exit cleanly to 3x3 from Crop_Rec.
If you don't and you just turn off crop_rec or raw video the camera will lockup and need a battery pull .

It's just got the one Crop_Rec Preset (3.5k Centered (3760x1080) for now , need some tester to make sure it works ok , once that's done I'll add more crop_rec preset .
Enjoy
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 19, 2020, 12:21:12 AM
Looks like Bitbucket has finally Delete Magic Lantern project,  all my download link & source files are broken/gone  :-[
I have to re-upload my Crop_Rec Builds to my Google Drive and link from there .

I Hope @a1ex has got some place for Magic Lantern project so we can continue , or is this the end ?
I hope not , but there again nothing last forever .

Edit: I may have missed some links , so if you find any of my links are still broken please let me know .
Edit2: Ok I updated all the download links on the First post on the First page of this thread .
So now i have no way of tracking downloads to know if anyone being testing/using the experimental crop_rec builds .
That part of my feedback , so i know the build is working as intended , since no one really gives feed back any more .
I can only assume by the download number that thinks are Ok . Now I'll have to guess   :-\
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on August 19, 2020, 08:31:41 AM
I guess it means we have to report if everything is working as expected.. I usually download and test everything new for 5d2
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: T123 on August 20, 2020, 12:07:58 PM
Quote from: reddeercity on August 17, 2020, 08:17:15 AM
New updated 50D Crop_Rec
https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/magiclantern-eXperimental-crop-rec_3.7k-1x1.11.30pm.2020Aug16.50D109.zip
New download Link from google drive.
magiclantern-eXperimental-crop-rec_3.7k-1x1.11.30pm.2020Aug16.50D109.zip (https://drive.google.com/file/d/1VCk_Ir1P3srP4XEGXLb-ZJU3Wz3avKKC/view?usp=sharing)

Updated the 50D Crop_Rec with new 5xZoom check (which enables crop_rec with out delay , same as the 5D2)
plus other back-end tweaks for 50D model .
Also has the updated mlv_rec module for increase resolution in non crop mode's as will as Crop_Rec.
3744x1080 before , now 3760x1080 .
I changed how to exit Crop_Rec , after you'er done recording press play button twice and you should exit cleanly to 3x3 from Crop_Rec.
If you don't and you just turn off crop_rec or raw video the camera will lockup and need a battery pull .

It's just got the one Crop_Rec Preset (3.5k Centered (3760x1080) for now , need some tester to make sure it works ok , once that's done I'll add more crop_rec preset .
Enjoy


@reddeercity

Thank you for your hard work.

Here is my feedbacks to the build-2020Aug16

1. Format CF card, it deletes the whole maigc lantern files.
old nightly build can restore back ML after card format.


2. No adtg.mo & cfacc.mo compared to previous build.
with crop mode on, I can record the 3760x1080, but no preview, whole screen with pink colour.
The video output also full of pink colour.


3. So I copy adtg.mo & cfacc.mo from previous built
Now I can record the 3760x1080 resolution, with zoomed in preview, no full size preview while recording.


4. With crop mode on, the 3760x1080 files, shows red pink bar on the right end, tiny pink line at the bottom.
black object/dark shadow with vertical lines. Black colour with colour cast.
---See attached file, extracted from MLVFS
M20-1619_000005.dng 3760x1080


5. With crop mode on, for other resolutions, tiny pink line at the bottom.
---See attached file, fifth frame from various resolution videos.
M20-1620_000005.dng 3520x1080
M20-1621_000005.dng 3200x1080
M20-1622_000005.dng 3008x1080
M20-1623_000005.dng 2880x1080
M20-1624_000005.dng 2560x1080
M20-1625_000005.dng 2240x1080

6. with crop mode off, no colour cast/ colour lines.
M20-1617_000005.dng 1584x892 no crop
M20-1618_000005.dng 2000x1078 crop


7. when i adjust the shutter speed, it changes ISO instead.

8. Is the extra 2pixels caused the bottom colour line?

download link for the dng files.
https://we.tl/t-Z42VBPSTuj
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on August 20, 2020, 08:00:49 PM
Quote from: reddeercity on August 19, 2020, 12:21:12 AM
I can only assume by the download number that thinks are Ok . Now I'll have to guess   :-\

Look, you can maybe use some online link trackers. I found this one:
https://www.linkclickcounter.com/
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: SKecorius on August 21, 2020, 05:40:08 PM
How about making freely accessible (google docs) spread sheet, where everyone could put, which version he/she uses, what features are tested, what are the results, opinion, questions, requests, etc..?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: T123 on August 21, 2020, 08:15:04 PM
refer to my previous post,

we do not need to copy adtg.mo & cfacc.mo from other build, can just use the 2020Aug16 build.
The problem happened to me yesterday is due to the "fps override".
If i manually press on both the "fps override" and crope mode, problem happens.

the solution is, we do not need to manually on the "fps override".
It is already in 23.976 fps default in the build.

Here is the screenshot of my 3760x1080
https://we.tl/t-GQABked6sy
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on August 23, 2020, 06:18:33 PM
I like the google docs idea!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on August 24, 2020, 02:52:09 AM
If you need storage space Reddeercity on Google drive I'll buy it for you! Google drive is pure gold honestly.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: T123 on August 24, 2020, 03:25:36 AM

My earlier two posts are still waiting for approval..

@reddeercity
How do you activate 50D in 1x3 1376x1742 (4128x1742) @24fps ?
with anamorphic lens?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on August 27, 2020, 03:41:16 AM
I think, that the google docs idea would be ok in the case of at least 2 or 3 developers. David now works alone on the developments of 5d2 and 50d. I don't think he would like to read every little thought came to our heads described in a sheet of google docs. Personally me, I wouldn't. It's more like hobby for him.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: SKecorius on August 27, 2020, 03:36:14 PM
There is no need to read anything, if you do not want to. The numbers of people downloading 5DmII's code would reflect in the number of filled lines, which would not require too much attention..additionally, you would have information about tests in one place (e.g. card info, minutes recorded, max resolution, etc.). Perhaps after a while one could run some stats on the data gathered. To place 1 spread sheet file into Google drive is not that hard (also does not cost anything, as far as I know).

There are already google docs of ML (e.g. features, bitrates, supported cams, etc.). I do not know how frequently those are updated, or useful. Nevertheless, it could be a great tool providing feedback to developers.

E.g.: https://docs.google.com/spreadsheet/ccc?key=0AgQ2MOkAZTFHdFFIcFp1d0R5TzVPTVJXOEVyUndteGc

Also, just downloads does not tell much.. Myself including, one might download ML build, but would not do anything with it. Then loose it (delete it, change pc, etc.) and download again, without putting it to test..(getting different camera). Spread sheet, on the other hand, would require active input from people actually testing the thing, and not just downloading.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dlprod on August 27, 2020, 09:08:00 PM
Sorry to come off like a dummy but how do I install the crop rec module?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on August 27, 2020, 09:17:19 PM
Copy it to ML\modules\
just where the old one is located.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dlprod on August 27, 2020, 09:38:53 PM
Hi  I've done that but the module won't load on startup.  Is there a specific build I should be using?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on August 27, 2020, 09:43:02 PM
use the build on the first post of this thread
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dlprod on August 27, 2020, 09:52:00 PM
Full width 5.6k 16x9 A.R. , 4K Anamorphic 3x1, 1x3,  UHD 3K Plus 10-12bit Full HD?

then just drop the module in the modules folder correct?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dlprod on August 28, 2020, 02:08:11 AM
Never mind- got it working.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 28, 2020, 03:10:42 AM
Ok Solved my download tracking issue , I started a new Bitbucket Repository
https://bitbucket.org/reddeercity/crop_rec_5d2_50d/src
There's no ml source files yet (i guess i could have clone from Mercurial repos before there where deleted  :( )
But for now i have all the Crop_Rec Builds there , In the download area so i can see the downloads as before
and in time i will unload my source , when I figure that part out .

I've undated the first post of the this thread with the new updated links .

Thanks for the suggestions everyone .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: T123 on August 28, 2020, 06:58:23 AM

testing, testing, testing,

Why my previous post are still waiting for review, not visible yet?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: names_are_hard on August 28, 2020, 07:26:44 PM
That's a Git repo, so you couldn't have cloned from any Mercurial repo.  You'll have to use git if you want to put code in there.  I have a git copy of the old ML repo, you could clone that - but you'll still have to use git to work with it.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: T123 on August 31, 2020, 10:20:58 PM

@reddeercity

Thank you for your hard work.
My previous posts have been approved, but the attachement is expired.
So i attached again the link. Here is my feedbacks to the build-2020Aug16


1. With crop mode on, the 3760x1080 files,
-shows red pink bar on the right end,
-tiny pink line at the bottom.
-black object/dark shadow with vertical lines. Black colour with colour cast.

See attached file, extracted from MLVFS
M20-1619_000005.dng 3760x1080


2. With crop mode on, tiny pink line at the bottom.
---See attached file, fifth frame from various resolution videos.
M20-1620_000005.dng 3520x1080
M20-1621_000005.dng 3200x1080
M20-1622_000005.dng 3008x1080
M20-1623_000005.dng 2880x1080
M20-1624_000005.dng 2560x1080
M20-1625_000005.dng 2240x1080


3. with crop mode OFF, no colour cast/ colour lines.
M20-1617_000005.dng 1584x892 no crop
M20-1618_000005.dng 2000x1078 crop


Is the extra 2pixels caused the bottom colour line?

Here are the download link for the dng files.
https://we.tl/t-Qftxyjao2D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 04, 2020, 04:45:42 AM
@ t123, try a different  Cdng extraction software like MLV App.
The color vertical line are a know problem even on the 5D2 , just wait a few minutes or go thought the menu
then back to crop_rec liveview and the the lines will be gone .
The is just a crop_rec issue and not a problem with normal raw recording , e.g. 3x3 or crop_mode (5xzoom)
This is very Bleeding Edge for the 50D since it never had video in the first place , so little issue/problems need some work around
 
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 04, 2020, 04:51:29 AM
Updated the first post of this thread with the 50D Crop_Rec build , since there no big issue being reported
I can consider it to be safe for general user's now .
https://www.magiclantern.fm/forum/index.php?topic=19336.msg182476#msg182476
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on September 07, 2020, 04:59:46 PM
Any chance of 2560x1440 ? Its less than 3.7k and its standard 1440p, hence more usable with other video footage.

Sent from my Redmi 4A using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 08, 2020, 02:14:25 AM
Quote from: zcream on September 07, 2020, 04:59:46 PM
Any chance of 2560x1440 ? Its less than 3.7k ....
I have 3200x1330 @ 23.976 , but the problem I'm having is the adtg hooks (adtg[1]1172 & adtg[1]1178) are not enabling (this is needed for any height beyond 1080)
I'm working on it and a possible word around .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 08, 2020, 02:33:10 AM
50d_3x1_crop_rec_4118x720_24p_9-7-2020-6.04pm_adtg_gui.zip (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/50d_3x1_crop_rec_4118x720_24p_9-7-2020-6.04pm_adtg_gui.zip)

On the 50D I'm also having problems with the cmos hooks in 3x3 (there work Ok in 5xZoom),
I have a 3x1 4k & 48p preset but there crash the camera (lockup neededs a battery pull)

So I coded the 3x1 4k preset in to the adtg_gui module , just like the 3.7k from before .
So it will work the same , go in to the advance menu & enable "Digic Reg" first , refresh liveview
then go back the advanced adtg_gui menu and enable the 3k(50D) preset (I haven't changed the name in the menu)
then press the play button once it should then give you a scambled HD Liveview & or a frozen liveview , make sure you are in ML Black & white low res preview
I haven't coded the preview reg in to it , but you can put them in yourself .
Select the "Preview Regs Only" tab and input the reg in the image below , this will un-freeze liveview at standby but still frozen when you record raw video .

(https://i.ibb.co/8Y1qd9m/VRAM16-small.png) (https://imgbb.com/)

To process the 3x1 files use MLV App and make sure you enable the vertical 3x to get the full height (4118x2160)
and you may have to adjust the black level down to 105 in 10bit , i still need to adjust the regs
Give it a try , i'll try and update this later on .

I'm also looking in to a new camera for myself , maybe a 5d3 but i'm leaning towards a 5d4 or EOS R or R6
I do feel the pull toward the 5D4 as the basic ML has being ported (stubs mainly) but needs alot of work , switch I don't mind helping of course  :D
and the EOS-R with 4k 10bit log HDMI .

So if i do get new camera the update here maybe slow here , but as long as i have my D4/5D2 & 50D I"ll keep working to advance them to the best they can  ;D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: d1c1 on September 15, 2020, 09:50:30 PM
i'm getting these vertical lines in my crop rec videos

(https://ibb.co/q7YqPw4)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on September 18, 2020, 04:11:14 AM
I tested the new mlv_rec module "1880_3x3-2152_5xZoom-5d2+50d-support__7-5-2020-5.59pm-mlv_rec.zip" with the latest "crop_rec-5632x3124_3.8fps-5D2-eXperimental.9.42pm.2020Jan05.5D2212.zip" build. I'm loving the new 3x3 resolution!

mlv_play isn't working and causes no modules to load.

Any way to fix this easily?

Thanks again for your continued development!

Is there a way to get the 1880 res in the 5D2 nightly build?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on September 18, 2020, 05:32:44 AM
Quote from: Milk and Coffee on September 18, 2020, 04:11:14 AM

mlv_play isn't working and causes no modules to load.


mlv_play hasn't worked in ages on experimental builds. Everyone has this problem. There isn't much that can be done unless you can convince someone to develop mlv_play more.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 18, 2020, 08:14:14 AM
Quote from: Milk and Coffee on September 18, 2020, 04:11:14 AM
mlv_play isn't working and causes no modules to load.
Any way to fix this easily?
Yes,  here you go try this . mlv_play_9-17-20_11.29pm.zip (https://bitbucket.org/reddeercity/)
It should let you load your other modules now but i having a decoding error from "raw_twk module" now
the images are are all snow/noise , in 10 & 12bit but 14bit works a little better , I'll have to go thought the raw_twk module and see where the problem is at .

Should have some time here in the next few days


Quote from: Milk and Coffee on September 18, 2020, 04:11:14 AM
Is there a way to get the 1880 res in the 5D2 nightly build?
Not right now , need to wait until the source code with all the branches are updated on the new web platform .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 20, 2020, 02:31:24 AM
Did a quick video for the adtg_gui crop_rec 3x1 4k on the 50D. , I unloaded the file as prores 4444 5016x1832 but it seem to only play up to 4k , maybe latter there will be the 5k version



This gave me a opportunity to push the 5D2 to it limit for Mpixel/s readout , So in 1x3 1672x1832 @ 23.976 is the limit .
1672 x 1832 x 24 /8= 91.89 Mpixel/s , in theory the max read out is 24Mhz x 4 channel readout = 98 Mpixel/s
So i haven't been able to reach the max yet , or this may be it .
I'll explain , if you actually take in account the black boarder offset , horizontal offset 160 pixels & vertical offset 52 pixels
If you add the offset you get (1672+160)=1832 (1832+52)=1884 , 1832 x 1884 x 24/8 = 103.5 Mpixel/s  :D
So when we do raw size for raw video , should we take in account the offset or is this not read ?
Anyway I thought this was of interest .

Also as I'm having problem getting 1x3 to work in the crop_rec module on the 5D2 , so to please all the 5D2 user's , I'll give you a little treat  ;)
I'll post a video plus instructions on how to get 1x3 with the adtg_gui module .
Not sure if 1x3  will ever be coded to crop_rec so this will be the next best thing , stay tune  .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on September 22, 2020, 09:13:01 AM
I'm exciting
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dlprod on September 24, 2020, 04:05:46 PM
I can't wait!!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: kpn on September 25, 2020, 10:42:09 AM
Absolutely amazing work reddeercity! Glad to see this old but great camera being pushed to it's limits.

I've just got 5Dmk2 a few weeks ago and I've been experimenting with these raw builds, most of it seems to work fine but for the heck of me I can't figure out how to enable 48p mode. It would be really useful cause I've just got a glidecam and that would be a perfect combo. I tried following the video guides, guess I'll try again, I know it should work. Looking forward for future releases!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on September 29, 2020, 07:49:50 AM
Ok here we go , 5k ([email protected]) 1x3 setup with the adtg_gui module  and yes it can take up to 15min to setup , if you get it to work on the first try (without a battery pull) you're dong good  :D
The audio is a little low , sorry  :-[ forgot to gain the audio up when i captured it to my hdmi aja captured card from my HV20 .
let my know if it's problem , I can always upload a new version with gained up audio .   

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on October 02, 2020, 06:02:54 AM
This might be a stupid question, but is there a 3x1 (vertical stretch) anamorphic preset to use the full width of the sensor? Would this theoretically reduce aliasing and moire the same as 3x1 (horizontal stretch)?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on October 02, 2020, 10:54:39 AM
Quote from: Milk and Coffee on October 02, 2020, 06:02:54 AM
This might be a stupid question, but is there a 3x1 (vertical stretch) anamorphic preset to use the full width of the sensor?
No.
Quote from: Milk and Coffee on October 02, 2020, 06:02:54 AM
Would this theoretically reduce aliasing and moire the same as 3x1 (horizontal stretch)?
No. 3x1 horizontal stretch = 1x3. 5D2 does a lineskipping. As long as you just read each 3rd line, you'll get moiree. Reading all columns won't reduce that.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dlprod on October 02, 2020, 11:32:39 AM
Hi

Which specific 5D2 build is this for?

Thank you
Joseph
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on October 02, 2020, 11:38:52 AM
Quote from: dlprod on October 02, 2020, 11:32:39 AM
Which specific 5D2 build is this for?

Context missing. What are you asking and whom?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dlprod on October 02, 2020, 10:42:13 PM
Sorry anyone who can answer regarding Reedeercity's adtg_gui module in the 5D mk2.  Which build is he using to do the adjustment?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 03, 2020, 01:32:43 AM
Quote from: dlprod on October 02, 2020, 10:42:13 PM
Sorry anyone who can answer regarding Reedeercity's adtg_gui module in the 5D mk2.  Which build is he using to do the adjustment?
adtg_gui module from any of my full builds , I think i use the dec31/19 but can't remember right now.

Quote from: kpn on September 25, 2020, 10:42:09 AM
.....I can't figure out how to enable 48p mode.
It can be difficult yes , but what seem to work is start off in 3.5k centerd preset , then switch to the 48p preset .
seem to work ever time , if you couldn't get it in the first few tries then turn off crop_rec restart camera , enable crop_rec
try again . I'll post a video later , on how to enable 48p .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dlprod on October 03, 2020, 01:12:18 PM
Thank you Reddeercity!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 04, 2020, 04:52:17 AM
Quote from: Milk and Coffee on September 18, 2020, 04:11:14 AM
mlv_play isn't working and causes no modules to load.
Any way to fix this easily?
Done :)

5D2_mlv_play+raw_twk_10-3-2020_8.18pm.zip (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/5D2_mlv_playraw_twk_10-3-2020_8.18pm.zip)

(https://i.ibb.co/ggbjTJt/5d2-mlv-play-10-3-20-small.png) (https://imgbb.com/)

Fixed problems with raw_twk & mlv_play modules on my D4/5D2 4k Crop_Rec branch .
Tested it in 3x3(1880x1258) , crop_rec 3.5k centered preset(2784x1160) & 3x1 4k anamorphic (4096x590)
there's all view ok in camera , except for 3x1 4k there noise/snow on the bottom half of the screen .
(the anamorphic part of the code is not vertically stretching the image need to look further in the code) but still viewable .
Make sure you load both module's in the zip file , that's it . when you what to view a file just press the > play button 

edit:Updated the first post of this thread with the mlv_play & raw_twk module download links also
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on October 05, 2020, 12:47:30 AM
@Reddeercity
COOL!!!

Pulling out my red-hot СF card, I can tell that I had a sucsess and got 4896x1828 23.976 video after post processing.
https://youtu.be/0wZ3TA-6mLQ

I noticed that it's not a "proper framing", it's proper aspect ratio, because we see only this area in the preview:
(https://i.ibb.co/n1ZNSv1/M05-0832-00003.jpg)
In the video with the laptop i saw only the keyboard and the touchpad.

Also i was getting red blinking message through all the adjustment process:
[56]  light measure stack overflow free:0 used:512

And could not enable 180 degree shutter, only 1/45 or 1/50

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on October 05, 2020, 05:06:31 AM
Quote from: reddeercity on October 04, 2020, 04:52:17 AM
Done :)

5D2_mlv_play+raw_twk_10-3-2020_8.18pm.zip (https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/5D2_mlv_playraw_twk_10-3-2020_8.18pm.zip)

(https://i.ibb.co/ggbjTJt/5d2-mlv-play-10-3-20-small.png) (https://imgbb.com/)

Fixed problems with raw_twk & mlv_play modules on my D4/5D2 4k Crop_Rec branch .
Tested it in 3x3(1880x1258) , crop_rec 3.5k centered preset(2784x1160) & 3x1 4k anamorphic (4096x590)
there's all view ok in camera , except for 3x1 4k there noise/snow on the bottom half of the screen .
(the anamorphic part of the code is not vertically stretching the image need to look further in the code) but still viewable .
Make sure you load both module's in the zip file , that's it . when you what to view a file just press the > play button 

edit:Updated the first post of this thread with the mlv_play & raw_twk module download links also

Reddeercity, thank you so much sir!! 🙏🙏
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on October 05, 2020, 08:55:01 AM
Quote from: reddeercity on October 04, 2020, 04:52:17 AM
Done :)
Awesome!! Thank you so much! It's a really big usability improvement for this build
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: NHCD on October 05, 2020, 01:10:23 PM
hi
sorry im quite new to ML, is there a combined link (1 file rar only) for 10bit recording for 5d2 ? which i just have to unrar and make a boot in camera ?
i see an experimental version 10 12 bit but not sure it is the same.
a little confusing with patch files, as english is not my  native language also.
thank you so much and kudos for such effort :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 06, 2020, 03:14:59 AM
Here a little video about mlv_play & raw_twk module's on the 5D2 , shot on my 50D at10bit  1x3 1376x1742 ->4128x1742 @ 24fps , exported though A.E. C.S.6
youtube still processing the 4k stream but it up at 1080p right now
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on October 06, 2020, 03:55:17 AM
One  more video , Here's how to enable 48p preset every time , start off in 3.5k Centered preset (2784x1160) , re-fresh liveview then go directly to 48p preset , that's it .
shot on my 50D 1x3 4k (1376x1742 -> 4128x1742 in A.E. CS6 acr to prores 422hq , which is the uploaded file .

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: SKecorius on October 06, 2020, 12:24:45 PM
Thanks for the tutorials! It really helps.

I was reading that you are interested in 5Dmiv or other camera. Does this mean your development on 5DmII will stop/slow?
Can 5DmII users still expect lossless compression?

Best!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on October 11, 2020, 02:00:46 PM
Quote from: NHCD on October 05, 2020, 01:10:23 PM
i see an experimental version 10 12 bit but not sure it is the same.

I think if you are looking for a build able to record in 10 bit a build labeled to do 10 bit would do just fine.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Karmaschinken on October 27, 2020, 07:41:19 AM
Hello nice people.

I bought one of the first 5D MkII some years ago and tried Magic Lantern too, but then stopped working with my camera. Now due to Corona I need to use my 5D2 for filming again, since I can't afford a new camera.

I would like to do some tabletop clips and want to get the best quality. Now when I installed Magic Lantern some years ago, it was starting to be capable of recording RAW. But now it seems it can do much more?

If you don't mind, could anyone give me a short glance of what happened here? I have to get the camera ready until tomorrow. I installed the newest Magic Lantern but did not succeed to make it film e.g. Full HD RAW at 25 or 30 fps.

I obviously failed to find a complete install for all the new software that is developed here, I don't even understand what is possible. So my question is: Is there a short description here of what can be done with the software and is there a complete stable setup with all modules?

What is filming with crop, what is this 1:1? I saw the videos above, does crop read out e.g. only the upper part of the sensor?

Most important for me: Can someone please help me make at least Full HD filming in RAW work? I have a Komputerbay 1000x 64GB. I don't have the time to read though the 55 pages of this thread until tomorrow, so I'd appreciate help very much! Thanks!

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: thebailey on October 27, 2020, 07:56:40 AM
Quote from: Karmaschinken on October 27, 2020, 07:41:19 AM
Hello nice people.

I bought one of the first 5D MkII some years ago and tried Magic Lantern too, but then stopped working with my camera. Now due to Corona I need to use my 5D2 for filming again, since I can't afford a new camera.

I would like to do some tabletop clips and want to get the best quality. Now when I installed Magic Lantern some years ago, it was starting to be capable of recording RAW. But now it seems it can do much more?

If you don't mind, could anyone give me a short glance of what happened here? I have to get the camera ready until tomorrow. I installed the newest Magic Lantern but did not succeed to make it film e.g. Full HD RAW at 25 or 30 fps.

I obviously failed to find a complete install for all the new software that is developed here, I don't even understand what is possible. So my question is: Is there a short description here of what can be done with the software and is there a complete stable setup with all modules?

What is filming with crop, what is this 1:1?

Most important for me: Can someone please help me make at least Full HD filming in RAW work? I have a Komputerbay 1000x 64GB. I don't have the time to read though the 55 pages of this thread until tomorrow, so I'd appreciate help very much! Thanks!

This is a development thread, so general use questions probably won't be welcomed here. The most advanced 5Dii builds are to be found here, but most with only one aspect of the camera enhanced at the cost of useability and stability. If you 'need' a reliable camera, you're stuck with general builds or the default 5Dii video.

The camera sensor is 5000 X 4000 ish, and there's no chance of recording the values of all those pixels at 24+fps so you either lower the frame rate, or you throw out pixels. One way is to throw out pixels on the borders, so using the central ones (1:1) for a smaller sensor but a pretty perfect image. Or you can throw out every second or third pixel in the horizontal or vertical directions so you end up with stripes that span the whole sensor, which deinterlacing algorithms can deal with (but more image artifacts, moire especially).

In general, you can get 'reliable no dropped frames' for a few seconds, 'risk of some dropped frames' for a minute or so for 10bit 1080p(ish) at 24fps.

I'd check the amazing work done by Reddeercity recently, check his YouTube videos for links to the right firmware and guides, but only if you can re-film and spend time getting the settings right as this is bleeding edge.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Karmaschinken on October 27, 2020, 08:09:56 AM
Thanks for your reply. I see that this is the bleeding edge of the progress, yes. However I really would like to get the best out of the camera and it is quite hard to find the right download links from the first page, these are only the newest modules, right? Should I install the "official" Magic Lantern and then put these modules on top? All I need is e.g. 4 seconds of recording at Full HD, it would be ok to drop some frames. Could you maybe point me to a complete install of ML where the basic RAW recording capabilities are implemented?

The last one seems to be this (https://builds.magiclantern.fm/5D2-212.html) from 2018? Is this the basic install and do I have to drop the development modules on top of that? Thanks! Martin
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Dnhilos on October 27, 2020, 10:29:53 AM
Hello,

I've been trying out the 10/12-bit video on the 5dm2 but once converted with MLV Converter 1.9.2 and imported into resolve, all I get is noise.
The 14 bit works fine.
(https://ibb.co/tpL7GWn)

Am I missing something?


Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on October 27, 2020, 01:10:26 PM
Quote from: Karmaschinken on October 27, 2020, 08:09:56 AM
The last one seems to be this (https://builds.magiclantern.fm/5D2-212.html) from 2018? Is this the basic install and do I have to drop the development modules on top of that? Thanks! Martin

Just watch ZEEK's video. He explains everything and he has download links in the description https://www.youtube.com/watch?v=UQHPUMQQJx8&ab_channel=Zeek
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on October 31, 2020, 02:14:49 PM
Dave can You enable audio recording by default ? Ive lost important audio because of it today, i forgot that fresh install has disabled audio, im sure im not only one who thinks its probably just a joke to piss people off, its a weird choice to turn it off so you have to enable it... it happened too many times when ive lost audio on 5d2.

Theproblem is ,when camera shuts itaself or hangs then settings are reset to default, then sound is turned off again, thats super annoying and lost me important shots without realizing.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on November 16, 2020, 04:22:35 PM
Why do you need audio in 5d2?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on November 17, 2020, 12:00:29 AM
My daughter starts talking first words , these are important.
Its just strange that sound is disabled when switching to raw recording and you have to enable it.
Also i see its hard to find working version now, there are just forlders with modules and nothing else in them, main ML files are somewhere else in other links , i have no idea which version is most recent one for 5D2, maybe it has sound on from getgo with RAW.
First post is super confusing now.
I know it comes out whiney but i spent over hour just on figuring out which build i have to get from all the links in first post to have new modules working and i did not succeed cause they are all separated, i had to load some of the older builds onto cf card.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Igor_Braun on November 19, 2020, 05:30:44 AM
Man, I perfectly understand you, but because you told about personal details that are important to you I have to remind, you should remember that this is developer threat. I bet, that the most part of those who use these builds doesn't need in-camera sound recording.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Macke1 on November 20, 2020, 07:22:20 PM
i tried to install from Zeek´s download (Youtube). But im getting errors on all moviemodules. Im on 5d markii 2.1.2. Where can i find Another download link to the same build?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on November 20, 2020, 07:33:01 PM
The builds which are mentioned in this thread are still on the first post and never moved. Standard (nightly) builds can be found in the download section, as always.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ZEEK on November 20, 2020, 11:31:06 PM
Quote from: Macke1 on November 20, 2020, 07:22:20 PM
iBut im getting errors on all moviemodules. Im on 5d markii 2.1.2.
Errors will happen if the wrong Module has been enabled. E.g. MLV_Rec
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: SKecorius on November 26, 2020, 11:43:52 PM
Hi guys,

Although its most likely a rudimentary knowledge, I still would like to ask:

I have installed "magiclantern-raw_video_10bit_12bit.2019Mar24.5D2212.zip" from https://builds.magiclantern.fm/experiments.html onto my 5DmkII. It runs well, with 14bit video at max resolution lasting for over 20 sec (I stopped recording after this time).

I would like to try crop_rec module, thus I have downloaded "5D2-4k1x3-3k1x1-2.8k1x1-48pfhd3x3_7-10-2020_8.42pm_crop_rec" from reddeercity. When extracted, there is only one file - "crop_rec.mo". The question is - can I just place this module into modules folder on existing ML card with previously mentioned ML version? Or shall I completely re-install ML? If so, which ML version for installation shall I pick?

Also, I have two CF cards. I would like to have ML on both of them. Do I need to install (download ML files onto card, go through camera menu to update firmware) for each CF card? Since ML runs from CF card, I would assume "YES". However, I am not entirely certain about that. I would like to have one CF card with Nightly build, and another one with crop_rec module for higher resolutions.

Perhaps this helps someone in the future.

Thanks!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on November 27, 2020, 07:27:12 AM
you need to download this ML build from december:
https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip

then, download new mlv_rec.mo
https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip
and place it into "modules"

then download new crop_rec.mo
https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/5D2-4k1x3-3k1x1-2.8k1x1-48pfhd3x3_7-10-2020_8.42pm_crop_rec.zip
and also place it into modules

and finally download
https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/5D2_mlv_playraw_twk_10-3-2020_8.18pm.zip
and add it too.

All those links are from the first post.
This way you'll get all latest features.. Except the 5k build, which can only be used separately.

QuoteDo I need to install (download ML files onto card, go through camera menu to update firmware) for each CF card?
well you can use a program called EOScard, I use it all the time. Copy ML files to the card, launch this program and click ML, save. done. no need to go through camera update process.
https://pel.hu/eoscard/
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: pyffi on November 27, 2020, 01:32:03 PM
hi! have great experience with this build https://vimeo.com/469719016
thx reddeercity!

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: SKecorius on November 27, 2020, 04:31:28 PM
Thanks a lot, Skinny! One more thing - when installing ML through camera menu, it creates ROM, which is suggested to save in case there will be some problems in the future. Will EOScard utility do the same?

Also, if I replace my current CF card with ML on it with a new CF (without ML) - would camera function normally as if no ML had been installed? And in the same manner - could I have multiple CF cards with different ML builds and just swap them without any other actions to be taken?

Cheers!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on November 27, 2020, 04:48:00 PM
First chapter: During ML startup there is a check if ROM files are present on card. If not ROM will be written to card. If they are already there startup will proceed.

Second: Yes. Yes. Top of page -> User Guide -> Install Guide ->
Sections "Installing Magic Lantern on other cards" and "Updating Magic Lantern"
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Macke1 on November 29, 2020, 07:42:54 PM
I followed Zeek´s YouTube and everything works fine, but I wondering about what 3x3 means?! I'm on 5d mark ii
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on November 29, 2020, 08:54:38 PM
Quote from: Macke1 on November 29, 2020, 07:42:54 PM
but I wondering about what 3x3 means?!
3x3 modes: the camera records one value from 3x3 pixels. The cam records with line skipping and column binning.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Macke1 on November 30, 2020, 02:49:26 PM
ok! Thanks
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: SKecorius on November 30, 2020, 11:15:56 PM
Hi,

I have tried HDR video mode on 5DmII (in 10/12bit Experimental built). As you may know, after merging frames the video ends up to be 12 fps. I would like to ask, if anyone tried this mode more extensively (e.g. trying to interpolate missing frames in software)? Did you get smooth video with moving objects? From my initial test, a slow pan creates some nasty artifacts..

From the very first page, I also found "Dual ISO Video Module" by reddeercity (5d2_dual_iso_video_updated_21-8-2020.rar).
Did anyone test it with nightly build? If yes, could you share your experience?

Also, I do not quite understand the description: "Works with my crop_rec builds ..at 3x crop_mode @ 10bit raw,
I didn't check it with the Crop_Rec Preset,.. it may freeze you cam and may need I battery pull"...

From how I understand this, it seems that it works at 3x crop and it doesn't at the same time? Did anyone had success with mentioned "Dual ISO Video Module"?

Thank you for your thoughts.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on November 30, 2020, 11:20:01 PM
You can merge frames in consecutive order instead and just keep the same framerate. Technique used with tblend average filter in ffmpeg or with enfuse/hugin/align_image. It´s implemented as script(mac users) in Mlv app and also in mac based application "Switch".
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: SKecorius on November 30, 2020, 11:35:11 PM
Thanks, Danne. Is there a function in Windows version MLV APP?
--------
OK, I have looked through some forum threads, and it seems "Switch" is only for Mac and Linux. It's ok, I can still try it.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on December 01, 2020, 07:41:55 AM
Quote from: SKecorius on November 30, 2020, 11:35:11 PM
Thanks, Danne. Is there a function in Windows version MLV APP?
--------
OK, I have looked through some forum threads, and it seems "Switch" is only for Mac and Linux. It's ok, I can still try it.
Switch and MLVApp scripts are for OSX. But you can look into the script, find the ffmpeg command and try something similar on Windows. (ffmpeg is multi platform)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: pyffi on December 10, 2020, 07:05:47 PM
hey guys! is it real shoot in 48fps mode without freezing LV?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on December 11, 2020, 07:03:22 PM
@pyffi yes, it is possible.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: pyffi on December 11, 2020, 10:07:44 PM
Quote from: Skinny on December 11, 2020, 07:03:22 PM
@pyffi yes, it is possible.
if it no problem for you, how?

all is good, but when i start recording it's freezing
files ok
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on December 11, 2020, 11:44:09 PM
It could be somewhat tricky... try first start in 3.5k with 5x zoom, make sure everything is ok. then switch to crop-mode "off" (and 5x zoom still active), make sure the preview is working, and then select 48fps. If you still don't get preview, switch again to "off" with 5x zoom, and then back to 48.. try it couple of times, it should work.. maybe it could be done easier, I don't know.

And make sure selected type of preview is "auto"
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: pyffi on December 12, 2020, 06:02:39 AM
Quote from: Skinny on December 11, 2020, 11:44:09 PM
maybe it could be done easier, I don't know.

it works! thank you
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Audionut on January 28, 2021, 12:08:27 AM
OP appears offline since October, including at his youtube channel, hope all is well.

Going to lock this for now. PM me if status changes.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: a1ex on January 28, 2021, 07:44:42 AM
Also hoping all is well, but locking is probably not needed - the small community already formed around these builds can still continue the discussion, if you ask me.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: pyffi on February 01, 2021, 07:24:51 PM
hello, it's me again ;D
and i have one more question

how enable 48fps when i use external monitor? is IT possible?

when i on crop mode all works, but when 48fps is on - all monitors (external and dslr) in black(
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 08, 2021, 05:00:24 AM
Yes that's the way it is at the moment , Liveview only run at max 32fps and i haven't look further in to it.
Its low priority for me .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on February 11, 2021, 05:35:33 PM
From @reddeercity youtube channel 


Was there ever a clean 3200x1268 @ 23.976fps mode ? Thats closer to 16:9 than 3744x1080
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: far.in.out on March 26, 2021, 03:17:20 PM
Can anyone please show me what kind of preview is available when recording with 1x3 pixel binning on the latest build on 50D (or 5D2 if it's similar)? Maybe you can record with your phone what's on the screen of the camera while you record? Is preview available on external monitor via HDMI? I only care about 24FPS.

Also, can 10/12 bit RAW introduce additional moire or aliasing when compared to 14 bit? Or does it only depend on binning skipping?

I think I'm gonna get the 50D again soon and maybe even try to help with testing/development.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 26, 2021, 05:12:42 PM
There is no 1x3 recording on 5D2 or 50D yet.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: far.in.out on March 26, 2021, 06:53:52 PM
Are you sure?
Please check the DESCRIPTION for those videos on youtube pages (OP's channel)

Says:
Shot on my 50D in 10bit Raw 4k Anamorphic 1x3 pixel format (1376x1742 @ 24fps ) Converted to 4128x1742 with MLV App.

Shot on my 50D experimental Crop_Rec 1x3 1376x1746 @ 24fps
the 1x3 pixel when converted to 1x1 turns in to 4128x1746 .
There's no Line Skipping here , no Aliasing/Moire pattern or artifacts . You Can look at it as a Extreme Anamorphic

https://www.youtube.com/watch?v=FmDoJCmBqpA
https://www.youtube.com/watch?v=536-Bj9yAyA
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 26, 2021, 07:24:34 PM
Can you find a 1x3 build on the first post of this thread? If not, it doesn't exist for public.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: far.in.out on March 26, 2021, 08:08:16 PM
Is this not IT ?

(https://i.imgur.com/3AZ5Fsk.png)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 26, 2021, 08:21:13 PM
In your screenshot I read croprec 1x1.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: far.in.out on March 26, 2021, 08:29:35 PM
How can you do 3760x1080 @ 23.976fps with 1x1 on 50D? Am I missing something? That's how many MBs per second?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: masc on March 26, 2021, 08:36:27 PM
Nobody wrote it is continous. I would expect around 115MB/s.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on March 26, 2021, 09:15:11 PM
I suggest an eos m if you want a cheap way to test out 1x3 binning @far.in.out https://www.magiclantern.fm/forum/index.php?topic=25781.0
If you want to help developing digic 4 I guess the 50D and would be a good choice. Prepare for some heavy digging.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: far.in.out on March 26, 2021, 09:49:17 PM
@masc Damn, you're right.

I wonder why there's so little interest in 50D.
It's cheap (around $125), it writes at 70MBs per sec out of the box and it's 15MP, so it has the lowest res sensor and therefore a chance of having the lowest crop factor.
Yet there seems to be more action around other cameras.
Again, am I missing something? What are the advantages of the newer cams?

@Danne Thanks for the hint. The thing is, the cheapest EOS M that I can find is around $300 while 650D/700D is around $200. Is there any particular reason why the M is a better choice? Is it the most active port?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on March 26, 2021, 09:54:12 PM
Eos m $300. They sure gone up price wise.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: far.in.out on March 26, 2021, 10:09:31 PM
Maybe that's just my location, maybe it's different elsewhere.
What about the other questions?
What is the difference between those cams? What are the advantages? I thought they all were pretty similar. 40MB SD bottleneck, 18MP, similar CPUs. Is it the newer Digic that makes them better then 50D? Lossless compression? What else?
I don't know which cam to get now)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Danne on March 26, 2021, 10:24:19 PM
5D3  and eos m are solid to me.
Check this section:
https://www.magiclantern.fm/forum/index.php?board=66.0

Usually when someone works hard on a cam it gets better and better. digic 4 lacks lossless and digic 5 works generally better.
Shouldn´t be too hard to get a picture of which cams are good at what simply by following forum posts.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on March 27, 2021, 02:11:47 AM
yup Danne is responsible for M price bump  :P , its because of great magic lantern support for it, people are discovering it does almost 3k raw in crop.
I remember buying M for around 150 euro with kit lens in 2014 and i sold kit lens, i think like 2 years ago ive seen m going for 100 as well but now they want as much as for 5DII.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on March 27, 2021, 11:40:06 AM
Actually 5D2 can record 1x3 video, here is a way to enable 1x3 for it

Quote from: reddeercity on September 29, 2020, 07:49:50 AM
Ok here we go , 5k ([email protected]) 1x3 setup with the adtg_gui module  and yes it can take up to 15min to setup , if you get it to work on the first try (without a battery pull) you're dong good  :D
The audio is a little low , sorry  :-[ forgot to gain the audio up when i captured it to my hdmi aja captured card from my HV20 .
let my know if it's problem , I can always upload a new version with gained up audio .   



But it was never made into a preset, apparently it's not that simple...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Markjinli on April 23, 2021, 04:25:58 AM
I am new user,can you pack full file in one zip/rar :'( :'(
Mo file is dispersion :'( :'( :'( :'(
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 20, 2021, 10:29:30 PM
Hi all , i'm just about ready to resume development , in the process of moving/buying a house in a different community (small town, no stress  :D ) 
Plus my main development computer mother board burned up  :( (FX8350 over clock to 4.9Ghz)
So instead trying to find a new/used FX board i bought (in the process of assembling) Intel 10th Gen. i7-10700K (LGA1200) (16 thread CPU)
on a Gigabyte Z490 Vision G mother board with 32GB DDR4 3600MHz Hyperx Predator Kingston Ram .
Yea i when to the dark side  :P :P :P , but the new AMD's don't impress me .

So soon i'll be up & running , i still have a dev environment on my old i5 dell laptop (M430) but it's slow specially when compiling ML.
My foucus is CF Card over clocking (75/80 to 120Mb/s) on D4 Cams  https://www.magiclantern.fm/forum/index.php?topic=19336.msg207053#msg207053
Using ://bitbucket.org/hudson/magic-lantern/branch/hdparm before bitbucket stop.

After that I'll take a run at Lossless compression/MLV_Lite again , I was close before but i know i missed something .

There some small thing i would like to fix with the crop_record module on D4's (ATDG reg's are not being applied & or locks up cam)
on both 5D2 & 50D.

After that i what to jump in to the D6 Cams mainly 5D4 and get ML running so it can be used (no including raw video as this is very difficult right now)

I'll try and catch up on the older post questions , so hang in there D4 user's  ;)

One last thing , the new intel mother board has 2 M.2 PICe4x SSD slots
just wondering what other people are using , eg. brand & size
I what to load win's on it and make it a working drive for Raw Video & Adobe After Effect CS6, i hoping it will read in realtime or close to it.
Or is it better just to boot to a Sata3 SSD for wins and just use the M.2 ssd as just a Volume drive.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Volumetrik on May 21, 2021, 05:07:37 PM
Glad to have you back ! Congratulations on the house ! That's a pretty big machine you got there. After a lot of research, I figured that the SN750 NvMe drives from WD were the top option considering price. I have a 1TB SN750 and it's blazing fast. I also have a regular 1TB SATA SSD for my ML scratchpad files. The Samsung NvMe 970 EVO plus use more power than the SN750 and can heat up quicker. There are frequent deals for SN750's for 150-155$ CAD and I think that's a steal for that kind of performance and storage.

You'll see that MLVFS is slow for 5.7K ana and 3.5k files (5D3) because it looks like it's a single-threaded process and is outputting low amounts of data per sec. Also using almost 100% of one CPU thread. However, regular 1080p, I had no issues with scrubbing. If you dump the MLV contents via MLVAPP instead and access the folder with DNG's, even 5.7K and 3.5k DNG's load up fine and have no scrubbing issues with a SATA SSD.

I became frustrated with the lack of GPU accel on after effects even though I much prefer ACR's rendering of raw files vs Resolve. Even with a 3900x (12 cores & 24 threads), the export was 1-2 fps with 50% core utilization in AE. In resolve though, I could export ridiculously fast at 24-30fps with GPU accel on my 3070 and made things so much easier like scrubbing and color correction. Curious to see what you think about that
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: yourboylloyd on May 21, 2021, 06:58:27 PM
Quote from: reddeercity on May 20, 2021, 10:29:30 PM

One last thing , the new intel mother board has 2 M.2 PICe4x SSD slots
just wondering what other people are using , eg. brand & size

Any M.2 NVME that had the SABRENT brand on it is amazing. I own so many of those drives now and they have reliably performed for me. The 4TB Q version is my go to, but the 2TB is perfect as well.

Quote from: Volumetrik on May 21, 2021, 05:07:37 PM

I became frustrated with the lack of GPU accel on after effects even though I much prefer ACR's rendering of raw files vs Resolve. Even with a 3900x (12 cores & 24 threads), the export was 1-2 fps with 50% core utilization in AE. In resolve though, I could export ridiculously fast at 24-30fps with GPU accel on my 3070 and made things so much easier like scrubbing and color correction. Curious to see what you think about that

I love ACR too but I hated ACR as the bottleneck. It would take wayy too long to render. Resolve is faster, but I'm using the R6 now with 10bit log so I don't really have those problems anymore. 5D2's Raw is still sooooo good though and it's hard NOT to shoot with it sometimes on more artsy shoots. Would love to see more GPU accel on AE.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 22, 2021, 04:57:49 AM
Quote from: yourboylloyd on May 21, 2021, 06:58:27 PM
Any M.2 NVME that had the SABRENT brand on it is amazing.
Unfortunately SABRENT M.2 are not on the supported list for my Gigabyte mother board

Quote from: yourboylloyd on May 21, 2021, 06:58:27 PM
I love ACR too but I hated ACR as the bottleneck. It would take wayy too long to render. ..... Would love to see more GPU accel on AE.
When i was on the Adobe cloud "CC" there was GPU accel for After Effects & Premiere Pro but it cost monthly and that add up
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 22, 2021, 05:32:00 AM
Quote from: Volumetrik on May 21, 2021, 05:07:37 PM
That's a pretty big machine you got there. After a lot of research, I figured that the SN750 NvMe drives from WD were the top option considering price. I have a 1TB SN750 and it's blazing fast. I also have a regular ....
There are frequent deals for SN750's for 150-155$ CAD 
Thanks
This one is  supported by Gigabyte
I just order 1 from amazon ($154.00CAD)  10min before i wrote this post 
with tax came to $161.00 CAD , should be here by next Friday.   :)


Quote from: Volumetrik on May 21, 2021, 05:07:37 PM
I became frustrated with the lack of GPU accel on after effects even though I much prefer ACR's rendering of raw files vs Resolve. Even with a 3900x (12 cores & 24 threads), the export was 1-2 fps with 50% core utilization in AE. In resolve though, I could export ridiculously fast at 24-30fps with GPU accel on my 3070 and made things so much easier like scrubbing and color correction. Curious to see what you think about that
With CS6 AE On my 2013 MacPro Hackintoss  i7 3770k (overclocked to 4.7GHz)  16GB ram , on After Effect I was averaging 3-4 frames/s @ 1080p/24fps
That was from the (4x2TB disk) 6TB Raid5 box thought  ATTO PCIe-x4 SaS card to the SSD
I had .75GB per thread of the CPU (x8) in the multi threading configtion  , it only limited by how much ram you have 
AE loves ram the more the better & the faster it works.
I hope i can get more with the new computer  :D

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Volumetrik on May 23, 2021, 05:36:35 AM
Quote from: reddeercity on May 22, 2021, 05:32:00 AM
With CS6 AE On my 2013 MacPro Hackintoss  i7 3770k (overclocked to 4.7GHz)  16GB ram , on After Effect I was averaging 3-4 frames/s @ 1080p/24fps
That was from the (4x2TB disk) 6TB Raid5 box thought  ATTO PCIe-x4 SaS card to the SSD
I had .75GB per thread of the CPU (x8) in the multi threading configtion  , it only limited by how much ram you have 
AE loves ram the more the better & the faster it works.
I hope i can get more with the new computer  :D

Interesting, I'll try messing in the multi threaded option in AE. I have 24 threads, so I guess I'll do 1.25gb/thread.

I'll try the 1080p/24 with my current rig : 5900x + 32GB Dual Rank 3733Mhz CL16 + SN850 Pcie4.0 SSD to see what frame rate I get. It would be interesting to compare our results. Maybe I can download your MLV so we can render the same file directly to compare ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 25, 2021, 05:09:24 AM
All my files and gear is all pack away right (getting ready to move) i'll see if i can find a older mlv on my work laptop for on site shoots.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on May 25, 2021, 05:57:10 AM
Quote from: far.in.out on March 26, 2021, 08:29:35 PM
How can you do 3760x1080 @ 23.976fps with 1x1 on 50D? Am I missing something? That's how many MBs per second?
It in the Crop_Record module just like the 5D2 , You need to download the Crop_Record module
https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/magiclantern-eXperimental-crop-rec_3.7k-1x1.11.30pm.2020Aug16.50D109.zip
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: theBilalFakhouri on May 26, 2021, 07:17:39 PM
Hello there!

Any source code for crop_rec? I need to know what stubs do 5D2 & 50D use for MEM_ENGIO_WRITE
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: thebailey on June 08, 2021, 06:04:11 PM
Yay reddeercity, you're back!

If you get lossless compression working for 5Dii, that would be such a game changer.

I've gone through posts related to HDMI out, and it seems like although it syncs to 1920x1080, because it is linked to what the cam screen shows, it will always have black bars, even in crop mode. Such a shame. In all your deep dives into registers, you've never found something that would allow full 1:1 pixel 1920x1080 with no bars? If black bars are compulsory, would it be possible to anamorphically squeeze 1920x1080 into 1650x1080 for less wasted pixels in final 16:9 footage? For livestreaming concerts, that would be a godsend.

Not the right forum, but if anyone has hints on good affordable HDMI to USB capture cards or what 256Gb CF card you can still buy with 5Dii raw level write speeds, I'd love recommendations.

Lastly reddeercity, for all the useful stuff you do, I'm sure lots of people would donate to your coffee / computer parts fund by Paypal if you have one?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 14, 2021, 04:55:37 AM
Quote from: thebailey on June 08, 2021, 06:04:11 PM
Yay reddeercity, you're back!
........ If black bars are compulsory, would it be possible to anamorphically squeeze 1920x1080 into 1650x1080 for less wasted pixels in final 16:9 footage?
No , short answer .
It maybe possible , but I need to go thought the UART so I can research HDMI configurations and how to manipulate it
(I did get some good DM log (https://www.magiclantern.fm/forum/index.php?topic=21311.msg194750#msg194750%5B/url) file from previous research and this too
Understanding MotionJpeg & h264 routines for Digic 4 (https://www.magiclantern.fm/forum/index.php?topic=21593.msg197068#msg197068%5B/url) but need to go on a deep drive in the the motion jpeg chip where the HDMI comes from)
if I can understand the mjpeg chip calls totally with out guess work then we should have control & be able to write code to override the limitation
Quote from: thebailey on June 08, 2021, 06:04:11 PM
Lastly reddeercity, for all the useful stuff you do, I'm sure lots of people would donate to your coffee / computer parts fund by Paypal if you have one?
No , just donate to Magic Lantern so a1ex can come back sooner  :D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on June 14, 2021, 05:04:13 AM
Quote from: theBilalFakhouri on May 26, 2021, 07:17:39 PM
Hello there!

Any source code for crop_rec? I need to know what stubs do 5D2 & 50D use for MEM_ENGIO_WRITE
in the process of moving right now , so all development  files are packed up , it was on bitbucket before they deleted it
Plus my FX pc mother board blow up , and it have all my files in a VM so it could be difficult getting them out of the old SSD .
Could be many mouths , not sure maybe fall .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: theBilalFakhouri on June 14, 2021, 10:41:15 AM
No worries reddeercity, good luck!

I found a workaround for MEM_ENGIO_WRITE, not needed anymore . .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: An147 on July 11, 2021, 07:22:36 PM
Hi guys! I have a problem with 48fps mode - I managed to shoot it without working LiveView (just black screen and debug overlay that shows recording process, estimated  frames etc.).
But when I opened the video in MLV App - every even frame is empty (2nd, 4th, 6th etc..):
https://prnt.sc/1aavrxg (https://prnt.sc/1aavrxg)
What am I doing wrong? MLV Preview is Auto, Aspect Ration 1:1, entered 48fps mode with 3.5K 1:1 right before it (like in instruction).
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on August 05, 2021, 03:15:25 AM
Turn off Auto preview and use Grey preview and refresh Liveview
User "Skinny" in this post explains how to enable 48fps preset easily 
https://www.magiclantern.fm/forum/index.php?topic=19336.msg224056#msg224056
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ShittyWebsite on September 26, 2021, 01:50:22 PM
Is there anything for 7D? i have one but could not find anything
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on September 26, 2021, 02:11:43 PM
7D may be described as an oddball in ML world: Dual processor design and therefore some features are still missing. And ATM there is no serious development to solve dual Digic 4 processor issues.
Don't hold your breath!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ShittyWebsite on September 26, 2021, 03:01:42 PM
I see, thank you, i understand, old camera, not many people interested
i'm using Daniel Fort's magiclantern-raw_video_10bit_12bit_LVState.2019May15.7D203

It's quite good, 2.5k for about 450 frames,

I'm like this build for photos (rec trigger, half shutter pre only)
Silentpics: slow
Burst: good but low resolution (and many purple frames using crop)
Crop video: good resolution, no purple frames (but many useless frames)
Crop video + rec trigger: Good resolution, frames as needed
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on September 26, 2021, 03:18:53 PM
To speed up Silent Pics use MLV, not DNG. See https://wiki.magiclantern.fm/camera_help#silent_pictures
Quote
Storing into MLV is considerable faster than writing DNG. Example: 7D, burst mode (94 pics). Writing DNG: 80 seconds Writing to MLV: 24 seconds
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: ShittyWebsite on September 26, 2021, 03:23:59 PM
Yeah, I'm using mlv, I mean the shutter speed, maybe I should buy a t5i and shoot 1 second of 4K and use as photo
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: dlprod2014 on September 26, 2021, 06:03:08 PM
Hi
Can anyone recommend a stable build for 2.5k on the 5D2?

Thx in advance.
Joseph
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: kitor on September 27, 2021, 08:36:32 AM
Quote from: ShittyWebsite on September 26, 2021, 03:01:42 PM
i understand, old camera, not many people interested

Nope, that's not what Walter said. 7D has a very different hardware design than any other supported ML camera.

In general, features of given generation can be easily, often even blindly ported into other models.

7D has two CPUs, each run it's own OS and they communicate with each other. Think of it not like your dual core / dual CPU computer running Windows/Linux/OSX, but more like two computers linked via a very fast network. CPUs share responsibilities that in "normal" circumstances are done by single OS on single CPU.

Thus porting features to 7D is significantly harder, requires knowledge specific to that one model and usually having one in hands to test if given idea works at all.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Steadycamer on January 02, 2022, 12:44:38 PM
Hello

From the build of October 03 2020, I can't make digic iso working ( negative or positive value do not affect the picture ) Is there something Im missing to do ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: zcream on January 06, 2022, 07:39:59 PM
Quote from: reddeercity on June 14, 2021, 05:04:13 AM
in the process of moving right now , so all development  files are packed up , it was on bitbucket before they deleted it
Plus my FX pc mother board blow up , and it have all my files in a VM so it could be difficult getting them out of the old SSD .
Could be many mouths , not sure maybe fall .
GitHub is a better option than bitbucket.

Sent from my Lenovo TB-8703F using Tapatalk

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on March 14, 2022, 10:56:43 AM
@reddeercity hi! if you are still here and following this thread, could you take a look at this https://www.magiclantern.fm/forum/index.php?topic=26414.msg238174#msg238174
if you can make a build with this option enabled I will gladly test all values from 0 to 0xFF :D

Or maybe some other guys who have all necessary tools installed can do this?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: 2blackbar on March 29, 2022, 07:55:09 PM
Hey anyone knows what address i neeed to adjust to finetune shutterspeed on this camera  like on EOS M ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on March 30, 2022, 04:25:51 AM
Similar question here: I like having 1/48 shutter speed, but it keeps me from getting 1/4000. How do I get 1/4000 back?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: kwstas on May 01, 2022, 03:53:50 PM
Hi guys,
trying to use 48fps, seems like I have a white area on bottom part I can't avoid. Any ideas? Not sure in which build should I place the modified modules.
K.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Skinny on May 02, 2022, 12:54:07 PM
Is this area exist on the captured footage also? 48 fps preset uses less height so you have part of the screen frozen, it is ok..

Modules will work with builds from the first page:
Quote from: Skinny on November 27, 2020, 07:27:12 AM
you need to download this ML build from december:
https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/crop_rec-4k3x1-2.8k24p_3x3-48p-all-centered_8.18pm-5D2-eXperimental.2019Dec31.5D2212.zip

then, download new mlv_rec.mo
https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/1880_3x3-2152_5xZoom-5d2_7-5-2020-5.59pm-mlv_rec.zip
and place it into "modules"

then download new crop_rec.mo
https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/5D2-4k1x3-3k1x1-2.8k1x1-48pfhd3x3_7-10-2020_8.42pm_crop_rec.zip
and also place it into modules

and finally download
https://bitbucket.org/reddeercity/crop_rec_5d2_50d/downloads/5D2_mlv_playraw_twk_10-3-2020_8.18pm.zip
and add it too.

All those links are from the first post.
This way you'll get all latest features.. Except the 5k build, which can only be used separately.

how to enable 48 fps with realtime preview:
Quote
It could be somewhat tricky... try first start in 3.5k with 5x zoom, make sure everything is ok. then switch to crop-mode "off" (and 5x zoom still active), make sure the preview is working, and then select 48fps. If you still don't get preview, switch again to "off" with 5x zoom, and then back to 48.. try it couple of times, it should work.. maybe it could be done easier, I don't know.

And make sure selected type of preview is "auto"
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: kwstas on May 03, 2022, 03:17:09 PM
thank you Skinny!  :) :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: kolodiazniy on May 20, 2022, 01:39:02 PM
Guys, I heed help:
just can`t get a 4096 X 592 resolution in 4K Anamorphic crop mode!
How to switch it on?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Vector54 on May 27, 2022, 12:41:44 AM
Why you cant switch it on? You just need to select the option in the crop menu.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: MalcolmB on July 12, 2022, 12:45:04 AM
I just cannot get this to work.
Doing everything suggested and I can only get 2784x1160.
4K anamorphic crop mode just does not work either. I can't turn it on. Selecting it I just get 1180x1248 and lots of lines down the image.
When 'unanamorphing' it in post the image ends up a lot smaller.

Sometimes the display on the camera goes black and I need to take out the battery to fix it.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on July 12, 2022, 04:00:53 AM
Not enough infos to tell what you are doing wrong. Here is someone able to tell how to do it right:
https://www.youtube.com/watch?v=UQHPUMQQJx8
You will find a preconfigured download there and some other links to optimized handling and settings.
And read Skinny's post above, too.
HTH
If not: Come back with more details like card used, benchmark numbers ...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on July 12, 2022, 04:19:09 AM
Quote from: reddeercity on July 17, 2019, 07:29:31 AM
You have to start in 3x crop (5x zoom) first , so liveview should say "2144x1074" default crop_mode
then enable 4k 3x1 preset next , refresh liveview again , stay there for a few second (record a few second it you like)
then go directly to FHD48p preset with out refreshing liveview , crop menu will say 48p & frame size of 4064x590 .
next press half shutter then the zoom button once , should now be in FHD 48p (1856x774)
Here a quick flow chart showing the process .

(https://i.ibb.co/fFf7hMg/48p-5d2-setup-full-size.png) (https://ibb.co/sRnr0jf)

Did you do this ?
It's the simplest preset to enable
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: MalcolmB on July 12, 2022, 03:19:38 PM
Thanks for your reply.
I went through all the steps in a video posted at the start of this thread.

I will now attempt the steps reddeercity posted above.

Live view on. Picture black and white with a pink box along the bottom.
x5 times zoom Liveview menu says: 2152x1074 2.60x
Enable 4k 3x1 2152x1074 (need to set to x1)
Refeshed live view. Now states 1880x1248 2.98x
Selected FHD 48p 1880x1248
Pressed shutter half way and zoom. Now says FHD 48p 2152x1108

That's with the resolution set to highest. If I start with 1920 the numbers are all lower.
In the end I endup with FHD48p 1880x1248

I will watch the video posted by Walter Schulz.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: MalcolmB on July 12, 2022, 03:36:08 PM
The video Walter Schulz posted. I downloaded the files he posted on the Youtube video and installed them and at first I get similar results to him!
Why are his files different from the ones posted here?
His work the ones here do not.
One of the crop options are now missing FHD 48P results in liveview not working. I need to exit it, go into the ML menu and change the crop option.
Now when I try 3.5K 1:1 centered x5 I get lines down the picture and 4k breaks the liveview now too.
It's very inconsistent. I am stuck.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: badro999 on August 15, 2022, 12:40:46 AM
Quote from: reddeercity on July 12, 2022, 04:19:09 AM
Did you do this ?
It's the simplest preset to enable
here is what i get when i used this preset:
(https://i.ibb.co/rscwRjB/20220814-233617.jpg) (https://ibb.co/M6GCxvz)

Just a black screen no preview, it can record on 48fps no issues but i cant make the preview screen working so i cant frame/focus my shot, any solutions?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: cedricp on October 14, 2022, 05:43:30 PM
Is there a repository for the source code ? I've searched everywhere, but found nothing. Could someone upload it (even as an archive) ?
I've just bought a 5D2 for a very good price (80$), and I'd like to do some tweaks in the code from that version.
I could do a repo for it if needed.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: MalcolmB on October 17, 2022, 02:49:04 AM
This is what I get now too.
3.5k 1:1 centered totally pink in liveview then it goes all snowy.
48fps and 3X1 4K Anamorphic totally black screen. No GUI at all. Need to exit live view to change out of those..
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: SoulState on April 30, 2023, 05:36:08 AM
(https://i.ibb.co/8Yfd6Mc/111.jpg) (https://ibb.co/SfMdRNr)

I want to ask - is this normal in 3x1 mode? (50d) Looks like terrible artifacts...
It's "Zoom 100%" view in latest MlvApp with 3x vertical desqueezing...
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: SoulState on April 30, 2023, 05:42:48 AM
And this is without 3x desqueezing:

(https://i.ibb.co/Yhp3y4M/222.jpg) (https://imgbb.com/)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: theBilalFakhouri on April 30, 2023, 09:13:28 AM
Quote from: SoulState on April 30, 2023, 05:36:08 AM
I want to ask - is this normal in 3x1 mode? (50d) Looks like terrible artifacts...

It's not normal, it's coming becuase of bad configuration . .
The dev of this thread wasn't active for years.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Milk and Coffee on May 30, 2023, 03:00:19 AM
Hi all, been getting occasional pink frames when filming with an HDMI monitor plugged in. Any ideas?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Lis_Rudy_PLR on May 30, 2023, 09:34:17 PM
hello, i got eos 1200d and i really want go fhd48fps or even 45fps. The problem is that i actually complete nooby in programming etc. Could please someone help me to somehow port crop_rec to this cam? I was looking for it everywhere on forum but there's almost nothing for this camera bout cropping etc. im sending screenshot with error that shows when trying to load crop_rec
(https://i.postimg.cc/4xqymPgb/image.png)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on May 31, 2023, 03:46:57 AM
Mission impossible.
1200D's SD-card interface is locked at 25 MByte/s and ATM there is no way to tweak it.

Rough calculation (with all the bells and whistles not available for Digic 4):

1920 x 1080 x 45 fps x 10 bit x 0.4 (compression) / 8 = 46.7 MByte/s

Far beyond technical specs for continuous recording.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Lis_Rudy_PLR on May 31, 2023, 06:39:23 PM
oh, so that's how it is. so basically the last thing for me to do is wait for someone to break the card write limit on 1200d somehow or just get cash for newer camera. thanks for reply anyway.
Also the continous recording is something that i don't need, if the video recording would be stable for around 2 minutes or so it would be great (it is like this with 1.7x bitrate and it's enough for me)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on June 02, 2023, 07:02:14 AM
We are talking about different things.
Bitrate manipulation is for native H.264/MOV recording only and doesn't apply to raw/mlv recording.
Required bandwidth without tweaking is around 5.5 MByte/s. 1.7x should give around 9.4 MByte/s and that's within cam's limits.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 02, 2024, 06:35:05 AM
After seeing considerable work on D6 (7d mrkii heder)  &  D7& D8(names_are_hard)   8)
Its inspired me to finish some unfinished magic lantern business on the 5D2 & 50D
I find that i have some free now (6-8 weeks , after that Drag racing Season(ProMod's) start around Mid May (canada)
being i build race car chassis & parts (I'm in the metal fabricator business) i need to be at the track & late nights in the shop  :)

Anyway i'm going to take a stab at CF card over clocking first, I have all the code (thanks a1ex!) and it work on the old
ml core platform (.raw video) optimize for D4 cams with no restrictions on performance , That not why I'm using the old core ,
mainly because it work or should i say it will once i can compile it , then i can that the info and implement it in to the newer core that
D4 works on crop_rec, if i can get the CF card controller to switch to UDMA 7 from UDMA 6 i should go from 80-85MB/s to 120MB/s or around there.
But here lay the problem , see screen shot , how do i add "gcc 4.8.3 or check for it , i'm on Ubuntu Bionic 18.04
I do all my crop_rec development with it so no issue there just this old core stuff ,
do know for a fact that a1ex had it working on the 5D2 in the very early days of ML
(https://i.ibb.co/6mNhY22/Untitled.jpg) (https://ibb.co/HVXv700)

Any help or advice would appreciated , i'm not big Linux user mainly PC(windows7 some windows 10 if i have to)
and some mac. its been about 3 years since i did any development so just need some refreshing  :D

A few other things i what to finish/complete on 5D2 & 50D
Compressed 14bit raw (I now know what the problem is "ResLock address issue" 5d2 would lock up on saving the compressed raw so
I've already had it running serval years ago but drop it because i couldn't find the issue .
a fully funtional 1x3 crop_rec perset without adtg_gui module, the problem is the adtg reg are not applying for pixel binning "more then likely just a coding problem" I'm not a professorial coder , but i can get by.
Better Preview in crop_rec & general code cleanup (sometimes there a race condition in crop_rec between 3x1 4k (4096x590->4096x1770 2.3AR)
and High speed frame rate 1x1 48FPS 1880x774 .
Just for the 50D :
Get the ADTG Reg to apply , seems to be a problem on D4's ? or my code .
In crop_rec on 50D i can get 3760x1080 , the problem is the vertical , I need to have an ADTG reg applied to get a full vertical
(5d2 dosen't have this more then likely because it all ready does video out of the box)
the 50d never had Video even thou it was always there , Magic Lantern code enabled it so crop_rec (5xzoom) was always limited to 1080 vertical
until i found the Reg (only but luck) so i do 1x3 on 50d but i need to fix the ADTG reg first then everything will fall in place fast on the 50d.

That all for now , i hope all development continues on the forum for everyone to see publicly , i don't like when the ML discussion is behind closed doors
that one of the reason i stop doing any development , even thou the development is on D6 ,7 & 8 it still helps me a lot discover thing on the D4 cams.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on February 02, 2024, 07:38:25 AM
Good to have you back on tracks. Can't help with your dev environment issue. The only thing I know (IIRC) Ubuntu is not the prefered dev platform (to put it into friendly words).

Quote from: reddeercity on February 02, 2024, 06:35:05 AM
That all for now , i hope all development continues on the forum for everyone to see publicly , i don't like when the ML discussion is behind closed doors
that one of the reason i stop doing any development , even thou the development is on D6 ,7 & 8 it still helps me a lot discover thing on the D4 cams.

Not getting the "closed door" bit. Most of dev talk today takes place on our Discord server. Only essential findings will get documented in forum which serves as some kind of "long term memory". I think apart from Danne all current devs and contributors are present over there. You will find a discord invite on top of this page: https://discord.gg/uaY8akC

There is a restricted area but it is for administration.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: heder on February 02, 2024, 09:15:37 AM
Quote from: reddeercity on February 02, 2024, 06:35:05 AM
After seeing considerable work on D6 (7d mrkii heder)  &  D7& D8(names_are_hard)   8)
Its inspired me to finish some unfinished magic lantern business on the 5D2 & 50D

Yes, come back and help lighting up the development  :)


Quote from: reddeercity on February 02, 2024, 06:35:05 AM
see screen shot , how do i add "gcc 4.8.3 or check for it ,

Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: names_are_hard on February 03, 2024, 03:20:59 PM
Quote from: reddeercity on February 02, 2024, 06:35:05 AM
After seeing considerable work on D6 (7d mrkii heder)  &  D7& D8(names_are_hard)   8)
Its inspired me to finish some unfinished magic lantern business on the 5D2 & 50D
I find that i have some free now (6-8 weeks , after that Drag racing Season(ProMod's) start around Mid May (canada)
being i build race car chassis & parts (I'm in the metal fabricator business) i need to be at the track & late nights in the shop  :)

Cool, would be good to have you back :)

Quote
That all for now , i hope all development continues on the forum for everyone to see publicly , i don't like when the ML discussion is behind closed doors

Keeping devs involved is the most important thing for me, so I'd like to understand this more.  I'm not aware of anything being closed up, apart from the way Alex used to work.  Is there anything in particular that you don't like?  Maybe we can imake things better.

I would say the dev work has been more public since Alex left than before.  Alex did a lot of work, but he didn't like sharing things until they were finished.  We had to relearn lots of things about D678 cams because it wasn't documented anywhere.  E.g. the M50 "fishy" build - the source for this was never published, or even shared with other devs.  This was a largely working Digic 8 build, I could have learned a lot from it.  Instead I had to work it all out myself.

See updates from me since at least 2020, here:
https://www.magiclantern.fm/forum/index.php?topic=19737.350
Kitor on edmac, here:
https://www.magiclantern.fm/forum/index.php?topic=26249.0

Summaries of work here and here:
https://www.magiclantern.fm/forum/index.php?topic=26814.0
https://www.magiclantern.fm/forum/index.php?topic=26852.0

Also see a large number of commits over the last few years, all open :)  https://github.com/reticulatedpines/magiclantern_simplified/commits

Discord is just as open as the forums, anyone can join.  If I posted all my small findings to the forums it would be useless spam.  Instead, I post summaries, with links to interesting commits.  We treat Discord as useful for quick discussion, but useless as an archive, so we document things properly on forum and in the repo.

PS I did lots of work on D6 too :P  Heder started from a working ML build for 7D2, because I did that port.  750D and 5D4, too.  There's been lots of background work getting ML booting on many new cams!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: names_are_hard on February 03, 2024, 03:33:39 PM
Separating out code / repo stuff.

Quote from: reddeercity on February 02, 2024, 06:35:05 AM
Any help or advice would appreciated , i'm not big Linux user mainly PC(windows7 some windows 10 if i have to)
and some mac. its been about 3 years since i did any development so just need some refreshing  :D

The repo I maintain should work on old cams, and has many branches merged.  It contains crop_rec, lua_fix and other branches: https://github.com/reticulatedpines/magiclantern_simplified/commits

I mention this because some of the work I've done means this repo is much easier to build on modern systems.  You don't need to find gcc 4.8.  You can use any gcc from 8 to 13.  Ubuntu is an okay distro to use, Debian is better (this makes some things easier with qemu testing).  It compiles cleanly, with no warnings and no errors.  If any compilation warnings are introduced, I fix them.

Unlike the heptapod repo, I want to keep the number of active branches very low: only "dev".  All other branches are temporary, with the intention they will be deleted or merged into dev.  I found using heptapod very confusing - hard to find what you wanted.  Hard to know if a given branch would build, or what features it had.

I've also done similar work with qemu code, making this easier to build, and easier to run.  No patching things, no needing weird old versions of tools.

If you want to try this repo, I will happily talk you through how to get it working.  It is likely to become the official repo.  Nobody with access to the current official repo is active anymore, and making a working dev environment with it is difficult.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 04, 2024, 03:50:01 AM
Quote from: heder on February 02, 2024, 09:15:37 AM
Yes, come back and help lighting up the development  :)
I do my best  :)
I'm in catch up & refresh mode , being a while starting to get my head around everything .
Quote from: heder on February 02, 2024, 09:15:37 AM

  • Download the arm gcc packages https://launchpad.net/gcc-arm-embedded/4.8/4.8-2014-q3-update
  • Unzip it somewhere
  • Add the path to Makefile.user (if needed update "ARM_ABI", "GCC_VERSION" & "PREFERRED_PATH")
Doing this right now , thanks its coming back to me now .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 04, 2024, 06:35:10 AM
@names_are_hard , thanks , glad to be back .
I guess Discord just sound unappealing , having to join a different forum/chat , as I'm not very good at chatting/typing fast/spelling  :D
Take me forever to type up these posts on magic lantern (spell checker etc. ...)
I have dyslexia so it's a challenge but funny thing about It I have no problem with reading & write code (go figure) .

Quote from: names_are_hard on February 03, 2024, 03:20:59 PM
If I posted all my small findings to the forums it would be useless spam. ...... We treat Discord as useful for quick discussion, but useless as an archive, so we document things properly on forum and in the repo.
Ok I don't know that , I've always found all the info needed here on the forum for the stuff I did in crop_rec , Actually a lot of information came from
[CMOS/ADTG/Digic register investigation on ISO] (https://www.magiclantern.fm/forum/index.php?topic=10111.0) and others related.

I've being reading all of your posts of late & have tried to keep up with the 200d from you(great work) .
I actually was very interested in the 5D4 (D6) a few years back but the cost was just too much , even today these still around $2,000.0 (Canadian) used.

So 2 Thing that really brought back to do more work was 1st: Heder 7DMrk2 work simple great work at getting access to the raw stream & the different ways he achieved it plus
the quick access to the system Then he summarized on how it was done . Besides that the 7D2 Mrk2 is a very good cam 3x3 liveview no line skipping , clean HDMI with audio & 3K 5x zoom
I would say it may be better then the 5D3  ;D specially it raw vide is fully functional .Oh yea it a lot cheaper too , I can pick one up used from camera rental store for $599.00 (Canadian).

The second thing that got me back was when to posted in the ResLock thread https://www.magiclantern.fm/forum/index.php?topic=6740.msg245655#msg245655
got me thinking about 5D2 address's,  so I was Hooked again! I have a need to finish this plus a few more thing I would like get working if possible
E.G. MJpeg 4.2.2 (color space) (AVI) compression (same as 1DC)
Change the Uncompressed liveview stream from 4.2.2 to 4.4.4 also if its in the Jpeg encoder chip change liveview from 8bit to 10bit .
I do know the HDMI has 3 different color space out put , ok 4 their a 4th "Unknow color mode" maybe Linear color similar to EXR's or DPX's files or the Raw color space .
Something to need to used a 2.20 lut curve to see the image etc. .....
here a short summary out of the 5D2 rom disassemble , just did a "HDMI" search with visual Studio on win7
475223: "Pana_TurnOnHDMI (Audio = %d)":
   475247: "Pana_TurnOnHDMI : EDID_COLOR_444":
   475257: "Pana_TurnOnHDMI : EDID_COLOR_RGB":
   475267: "Pana_TurnOnHDMI : EDID_COLOR_422":
   475277: "Pana_TurnOnHDMI : Unknown ColorMode (%d)":
   475289: "Pana_TurnOnHDMI : FULL_HD_50":
   ....................
   475417: "Pana_TurnOnHDMI : FULL_HD_60":
   475426: "Pana_TurnOnHDMI : DEFAULT_NTSC":
   475435: "Pana_TurnOnHDMI : DEFAULT_PAL":
   475444: "Pana_TurnOnHDMI : Unknown VideoCode (%d)":
   475457: "Pana_TurnOnHDMI : Av_hdmi_avreq_init failed":
   475469: "Pana_TurnOnHDMI : End":


I would like to get in to the Jpeg chip , on the 5D2 its the JP62 if I'm not mistaken , there so much I thing in there that would help us .
I could go on and on , time to get back to it  :)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: names_are_hard on February 04, 2024, 11:30:54 AM
If you never used IRC (or never knew that ML devs used IRC!), then you definitely don't need to care about Discord.  It is better for quick things, so feel free to use it if you just want to ask a question and get an answer (it's like WhatsApp or similar in this regard).  Can also do screensharing / voice chat, sometimes useful.

I like the 7D2.  Good build quality, metal body, even has weather-sealing.  It feels like an APS-C sized 5D3.  It's faster with card spanning than 5D3.  I'd recommend it if you see one at a good price :)

5D2 plans also make sense to investigate.  I'd especially appreciate somebody with more experience working with the hardware, who could try to verify my theory that the reslock structures are a linked list holding (block num, device ID) pairs.  If I'm right, it means the way ML locks devices is wrong.  So if some cams don't work the way they "should", that could be the cause.  And the read_edmacs[] / write_edmacs[] can be simplified quite a bit.  We might also be able to determine what individual devices are for - the current way we do this means we're using the wrong numbers!

Interesting HDMI strings :)  Good luck!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: SKecorius on February 07, 2024, 12:12:27 AM
This is fantastic news - welcome back, reddeercity!
Must admit that I bought 5D mark ii only because of ML and your developments. Was always keen on trying some extended features, with less interest in actual scenes to capture  :-[.
After some months of silence in 3K/UHD 5D2 thread, also Zeek disappearing from youtube, I was about to let my camera go...
So you can imagine how happy I am now :)

Looking forward to exciting developments and news from you!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 10, 2024, 08:17:07 AM
Making some progress i guess , got it to compile . Took the cf_acc module code and put it in my new crop_rec source that seem to work until it didn't .
Got a error i had years ago when i was working it activity , didn't understand the problem then but i think I understand now but don't know how to solve.
Here what i got so for see image below:

(https://i.ibb.co/ZxjZRgq/error-cf-acc.jpg) (https://ibb.co/jDNp9wn)

so its looks for <linux/types.h>
and there's nothing in the magic lantern source , then i looked in the "Makefile" in the module and found this
# define the module name - make sure name is max 8 characters
MODULE_NAME=cf_acc
MODULE_OBJS=cf_acc.o hdparm/identify.o hdparm/hdparm-routines.o hdparm-ml-shim.o

# include modules environment
include ../Makefile.modules

LINUX_PATH=$(HOME)/src/linux-3.19

CFLAGS += -Ihdparm
CFLAGS += -Idietlibc/include
CFLAGS += -I$(LINUX_PATH)/arch/arm/include/uapi
CFLAGS += -I$(LINUX_PATH)/arch/arm/include
CFLAGS += -I$(LINUX_PATH)/include/uapi
CFLAGS += -I$(LINUX_PATH)/include

# fixme: properly handle generic-y from Linux makefiles
# (hack: copy and rename folders around manually)
CFLAGS += -I$(LINUX_PATH)/generic

test: FORCE
gcc hdparm-test.c hdparm-ml-shim.c hdparm/identify.c hdparm/hdparm-routines.c -Ihdparm -o hdparm-test
./hdparm-test


So it needed something from Linux or it needs the uapi from arms ? i don't understand this never seen this before , something a1ex needed to run
the overclocking on the cf card . i hope somebody could enlighten my please , I'll do more research in the mean time.
:D
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 10, 2024, 08:38:08 AM
Did a search for all references to "linux" in the "cf_acc module to see what its calling etc. ....
hdparm-ml-shim.c:
  7: #include <linux/types.h>

Makefile:
   9: LINUX_PATH=$(HOME)/src/linux-3.19
  13: CFLAGS += -I$(LINUX_PATH)/arch/arm/include/uapi
  14: CFLAGS += -I$(LINUX_PATH)/arch/arm/include
  15: CFLAGS += -I$(LINUX_PATH)/include/uapi
  16: CFLAGS += -I$(LINUX_PATH)/include
  18: # fixme: properly handle generic-y from Linux makefiles
  20: CFLAGS += -I$(LINUX_PATH)/generic

hdparm\hdparm-routines.c:
  6: #include <linux/types.h>

hdparm\hdparm.c:
    27: #include <linux/types.h>
    28: #include <linux/fs.h>
    29: #include <linux/major.h>
   825: * The Linux kernel IDE driver (until at least 2.6.12) segfaults on the first
   830: * We would like to issue these commands consecutively, but since the Linux
  1632: " Due to bugs in older Linux kernels, use of these commands may even\n"

hdparm\hdparm.h:
  91:  * instead of using the system def from <linux/fs.h>

hdparm\identify.c:
  7: #include <linux/types.h>

hdparm\sgio.h:
  3: #include <linux/types.h>

Or should just comment out all these references ?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 14, 2024, 06:21:42 AM
Ok I thing i figured this out , i need to put the Linux parts in my "Path" for compiling .
Been learning all about the "Linux Path" environment for developing, etc. .....
Finally found a list of command for terminal command line , it's really not that different then old DOS or 4DOS  :)
It all coming back now  :D , had to deep dive in to the "Path" stuff now i can move on and make some progress .
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: names_are_hard on February 14, 2024, 10:36:51 AM
Sorry, I missed some of this as I was busy with digic 7 raw video.

You almost certainly don't want <linux/types.h>.  That's typically used for writing linux device drivers.  And we want ML to build on Mac and Windows, too.  You probably want <stdint.h>.  Swap that over and see if anything breaks.

You definitely don't want "LINUX_PATH=$(HOME)/src/linux-3.19".  That's just wrong.  Can you link to the source for this module?
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 15, 2024, 05:29:41 AM
Quote from: names_are_hard on February 14, 2024, 10:36:51 AM
Sorry, I missed some of this as I was busy with digic 7 raw video.
No worries , raw video on D7 that great !!  8)
Quote from: names_are_hard on February 14, 2024, 10:36:51 AM
You definitely don't want "LINUX_PATH=$(HOME)/src/linux-3.19".  That's just wrong.  Can you link to the source for this module?
Ok , i found the source I'm using in foss.heptapod.net
Here's the link to the module , it looks the same as my source i downloaded about 4 years ago
cf_acc hdparm (https://foss.heptapod.net/magic-lantern/magic-lantern/-/tree/branch/hdparm/modules/cf_acc?ref_type=heads)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 15, 2024, 06:15:02 AM
@ names_are_hard
Here other post that references cf card sppeds & cf_acc hdparm
CF Card Info screen shot https://www.magiclantern.fm/forum/index.php?topic=19336.msg207054#msg207054 ,
a1ex links the "hd_parm" build https://www.magiclantern.fm/forum/index.php?topic=19336.msg207053#msg207053 ,
direct reg for UDMA7? https://www.magiclantern.fm/forum/index.php?topic=19336.msg207128#msg207128 ,
some useful info , its where i got the idea to over clock the CF card bus from a1ex https://www.magiclantern.fm/forum/index.php?topic=12862.msg199697#msg199697
From a1ex on cf speed & overclocking
QuoteI was doing sector-level benchmarks with Canon routines from bootflags.c, filling the entire card with 0xFFFFFFFF, 0x00000000, 0x55555555, 0xAAAAAAAA, 0x5A5A5A5A etc,
to see if that makes any difference in benchmarks. It didn't, but some blocks were written at ~120MB/s, others at ~85MB/s, and a very small percentage of them were written at lower speeds.
That pattern wasn't exactly repeatable, but the histograms were pretty much the same .....
Quote@reddeercity: more details after I'll get a new card (it *is* possible to overclock the 5D2 CF interface).
https://www.magiclantern.fm/forum/index.php?topic=12862.msg199759#msg199759 a1ex quoted"You can send ATA commands to the CF card (QEMU emulates them), so if the only difference between UDMA 6 and 7 is timing, it might even be possible to put the card in UDMA7."
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: Walter Schulz on February 15, 2024, 07:58:19 AM
I'm not sure if my information is correct but UDMA-7 may come with a lower voltage. No CF 6.0 spec available here, though.
Maybe ted ramasola (http://www.magiclantern.fm/forum/index.php?action=profile;u=238) can help. He seems to be (or was) on good terms with Komputerbay.

EDIT: Found a piece of contradicting info claiming UDMA-5, 6, 7 all using 3v3.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: names_are_hard on February 15, 2024, 04:57:11 PM
Quote from: reddeercity on February 15, 2024, 06:15:02 AM
Here other post that references cf card sppeds & cf_acc hdparm

Thanks, I understand what's going on now.  This module is a port of the existing linux hdparm util so we can run it on cam.  How successfully, or how to even interface with it, I didn't look into :)  I assume the intent was to learn what control was possible over CF cards, then make nicer (and importantly, smaller) code to do the parts we care about.

The code is obviously quickly hacked up, possibly why it's in cf_acc module (which already existed).  That combined with it being a low-level linux util explains the refs to linux/types.h.

I've removed the linux specific stuff, here: https://github.com/reticulatedpines/magiclantern_simplified/tree/hdparm_hack

That should build more easily.  Because it's based on my repo, you can use a more modern linux.  I recommend Debian Testing.  It might not work on whatever old linux you're using, mainly because we converted everything to python3.  Try it and see if you want.

I haven't tested the code in any way.  Will it destroy your cam?  Maybe!
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 16, 2024, 06:08:07 AM
Quote from: Walter Schulz on February 15, 2024, 07:58:19 AM
Found a piece of contradicting info claiming UDMA-5, 6, 7 all using 3v3.
I found in the documents that the CF cards  are 5v tolerate 
https://www.transcend-info.com/support/product/memory-card/compactflash-1000#SpecificationsModal
says there cards are 3.3v & 5V , i think the 5v is from the older ide PC interface like cd roms & 3.5' floppy disc etc. ....
when CF cards where used for backup soooo long ago  :P
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 16, 2024, 06:29:16 AM
Quote from: names_are_hard on February 15, 2024, 04:57:11 PM
Thanks, I understand what's going on now.  ... I assume the intent was to learn what control was possible over CF cards, then make nicer (and importantly, smaller) code to do the parts we care about.
Yes that's right , i hope to learn where the timing control regs are to manually enable UDMA 7 plus maybe bypass some canon routines to limit the overhead.
Quote from: names_are_hard on February 15, 2024, 04:57:11 PM
.... I've removed the linux specific stuff, here: https://github.com/reticulatedpines/magiclantern_simplified/tree/hdparm_hack
Thanks , i'll give it a try
Quote from: names_are_hard on February 15, 2024, 04:57:11 PM
That should build more easily.  Because it's based on my repo, you can use a more modern linux. 
I recommend Debian Testing.  It might not work on whatever old linux you're using, mainly because we converted everything to python3. 
Try it and see if you want.
Ok , I'm on Ubuntu 18.4x something , it use gcc arms 5.3 i think.
Quote from: names_are_hard on February 15, 2024, 04:57:11 PM
I haven't tested the code in any way.  Will it destroy your cam?  Maybe!
"With Great Risk Comes Great Reward"
at least that's what they say :D
I'll more then likey try it on my 50D first ! , just in case something goes south  :P
only paid $150 5 years ago , that may sacrificial lamb
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: names_are_hard on February 16, 2024, 02:14:02 PM
That's an old linux distro, I don't advise it.  My repo will build cleanly up to gcc 13.  You can just use the distro gcc, no need to build a special one.

It's worth noting also that my repo has had very little testing on Digic 4 and 5 cams.  It's been used lightly on a range of cams with no significant problems reported.  But please do use some caution, and let me know if there are any new problems.
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 19, 2024, 07:36:05 AM
Ok trying to compile , getting a error , but i don't know how to solve , Just compile the module by it self not a full build
did some warning but that's to be expected , being code from the old core.

(https://i.ibb.co/bdgZkpB/2-18-2024-error-cf-acc.jpg) (https://ibb.co/7yJ5q7W)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: reddeercity on February 19, 2024, 08:07:49 AM
Just for kicks i try to build the 50D & 5D2 with @names_are_hard  magiclantern_simplified but failed tried comment the error but make it worse  :(
here a few screen shots . Do i need to update my Linux ? to the what is being used by you ? will i have problem compile my crop_rec builds ?
Anyways here the screen shots :

5D2
  (https://i.ibb.co/gd5YCRt/2-18-2024-error-full-build.jpg) (https://ibb.co/0Kg53GY)
50D
(https://i.ibb.co/NCnrVMk/2-18-2024-error-full-build-50-D.jpg) (https://ibb.co/fFG91sK)
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: kitor on February 19, 2024, 09:10:28 AM
What is your distro? Old Ubuntu required for old repo?

permission denied is weird, try to run that script directly, maybe you are missing python3 or git. Or somehow it is missing executable bit.
This is literally a python3 script that runs two git commands.

kitor@kitor-p73:5D2.212$ ../../build_tools/git_summary.py
268224668 raw_draft_v2%
Title: Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
Post by: names_are_hard on February 19, 2024, 10:05:59 AM
Quote from: reddeercity on February 19, 2024, 08:07:49 AM
Do i need to update my Linux ? to the what is being used by you ? will i have problem compile my crop_rec builds ?

You need to work out why your system can't execute the python script listed in the error.

My repo has crop_rec code included.

For future reference, please don't post pictures of text - you can just paste the text!  I can't copy stuff out of an image to explain it or quote it.