Magic Lantern Forum

Developing Magic Lantern => Reverse Engineering => Topic started by: waza57 on March 13, 2016, 11:32:10 AM

Title: Video - dual iso for 5D2
Post by: waza57 on March 13, 2016, 11:32:10 AM
@reddeercity

http://www.magiclantern.fm/forum/index.php?topic=6751.msg163491#msg163491 (http://www.magiclantern.fm/forum/index.php?topic=6751.msg163491#msg163491)

I make some changes in dual-iso.c and this seems to work for the 5D2 in video mode .

I would like to share it's changes for test.
I don't know exactly how to proceed to respect the forum's rules.

Can I publish here a new module dual_iso.mo or his code?
Thanks
Title: Re: Video - dual iso for 5D2
Post by: Danne on March 13, 2016, 11:34:51 AM
Nice work! Hope you get some testers trying it out. I don,t own a 5D mark 2 unfortunately.
Title: Re: Video - dual iso for 5D2
Post by: Walter Schulz on March 13, 2016, 11:44:09 AM
Proper procedure will take place here:
https://bitbucket.org/hudson/magic-lantern/overview
Create a branch, put your changes there and assign some peers to look over it.
http://www.magiclantern.fm/forum/index.php?topic=9524.0
Title: Re: Video - dual iso for 5D2
Post by: nikfreak on March 13, 2016, 11:51:18 AM
70D is affected, too. So either do like Walter instructed or share the code (not module) here.
Title: Re: Video - dual iso for 5D2
Post by: waza57 on March 13, 2016, 12:47:12 PM
@Walter Schulz
I just tried but I do not see how to create a branch.
So I created a fork. It's Ok?
I make changes in this fork?

@nikfreak
Why would 70D also affected?
Title: Re: Video - dual iso for 5D2
Post by: nikfreak on March 13, 2016, 12:49:54 PM
cause 70D only has dualiso working for photo mode - like 7D / 50D / 5D2
Title: Re: Video - dual iso for 5D2
Post by: DeafEyeJedi on March 13, 2016, 05:40:42 PM
Dude ... This is HISTORY in the making!!!
Title: Re: Video - dual iso for 5D2
Post by: dfort on March 13, 2016, 06:45:13 PM
Quote from: waza57 on March 13, 2016, 12:47:12 PM
I just tried but I do not see how to create a branch.
So I created a fork. It's Ok?
I make changes in this fork?

Read dmilligan's excellent Mercurial Tips: http://magiclantern.fm/forum/index.php?topic=9524

Basically if you created a fork of the main repository that's step #1.

(https://farm2.staticflickr.com/1696/25133644223_043235ff91_n.jpg)

Now in your bitbucket workspace make sure you're in the unified branch and create a new branch.

(https://farm2.staticflickr.com/1655/25133644413_dfe06c54eb_n.jpg)

When you're done making your changes create a pull request.

(https://farm2.staticflickr.com/1701/25734304016_b271426776_z.jpg)

You can do this all on the bitbucket website or use the sourcetree app (https://www.sourcetreeapp.com/).

Hope this helps. Looking forward to your contribution.
Title: Re: Video - dual iso for 5D2
Post by: waza57 on March 13, 2016, 07:23:09 PM
So...

I made  a repository on bitbucket with a specific branch : https://bitbucket.org/waza57/magic-lantern/branch/dual_iso_video_5d2 (https://bitbucket.org/waza57/magic-lantern/branch/dual_iso_video_5d2)

My changes in dual_iso.c:
https://bitbucket.org/waza57/magic-lantern/src/7be4cd7f975ef25e98143d10da26712e5ec8ca77/modules/dual_iso/dual_iso.c?at=dual_iso_video_5d2&fileviewer=file-view-default (https://bitbucket.org/waza57/magic-lantern/src/7be4cd7f975ef25e98143d10da26712e5ec8ca77/modules/dual_iso/dual_iso.c?at=dual_iso_video_5d2&fileviewer=file-view-default)

I think it's not yet perfect because sometimes the Screen freezes.
when it's appear I must to restart and disable dual iso to restart again.
I have to continue to improve this certainly but I would really appreciate if someone could tell me if it's already the right direction.

@ nikfreak

The 5D2 problem is/was that there is/was only one register to store the values iso.
Unfortunately I do not know what is the problem for the 70D. I do not have any.

Regarding the 50d, someone should try to manually compare (with ADTG-gui) the actual values CMOS[0] for the iso photo mode with those of the video mode(for the same iso).
iso 100, 200, 400, 800, 1600.

@dfort
Thanks , I think it's ok.
Title: Re: Video - dual iso for 5D2
Post by: dfort on March 13, 2016, 09:36:16 PM
@waza57

You might consider uploading your compiled module to the Downloads area of your bitbucket project for users that aren't able to compile Magic Lantern. You could also do a pull request to the main unified branch -- hudson/magic-lantern/branch/unified to get developers' attention.

Like others that have commented on your posts I don't have a 5D2 so you might want to advertise your module on the 5D2 camera specific topic (http://magiclantern.fm/forum/index.php?topic=11205.0) to pick up some testers.
Title: Re: Video - dual iso for 5D2
Post by: waza57 on March 13, 2016, 11:32:58 PM
@dfort
thank you very much, excellent advice.
I was wondering exactly how to do next.
so, I'm going to do all this.

Title: Re: Video - dual iso for 5D2
Post by: reddeercity on March 14, 2016, 12:54:31 AM
Quote from: waza57 on March 13, 2016, 11:32:10 AM
@reddeercity
http://www.magiclantern.fm/forum/index.php?topic=6751.msg163491#msg163491 (http://www.magiclantern.fm/forum/index.php?topic=6751.msg163491#msg163491)
I make some changes in dual-iso.c and this seems to work for the 5D2 in video mode .
Thanks
Great stuff , thanks for the PM . Downloaded the module and will report here , as I said in the PM thanks for pushing this thought
and once we get all the bugs worked out if any , then hopefully it will be included in the nightly builds and the rest of the Non video dual iso
camera will be supported finally  :)
Title: Re: Video - dual iso for 5D2
Post by: jpegmasterjesse on March 14, 2016, 01:55:21 AM
I would love to help test this - but I'm afraid I don't know how to start.  Where can I download the module?
Title: Re: Video - dual iso for 5D2
Post by: reddeercity on March 14, 2016, 05:16:37 AM
Cool Works , No lockup of freezes yet used magiclantern-Nightly.2016Feb13.5D2212 with the Dual ISO Video module.
1856x928 23.976p + audio ,  ISO 100/800 on Lexar 1066x 64GB CF card Records continuous , at least up to 1:00 from the initial test.
Title: Re: Video - dual iso for 5D2
Post by: reddeercity on March 14, 2016, 07:00:37 AM
Well that impressive ! :D
Just processing the dual iso now , So far MLVFS for Mac seems to the one can process (didn't Try MLP from Danne Yet I downloaded it thou)
Just toggle the dual iso 20bit on with default setting then Loaded the Virtual file folder in to Resolve 12 adjusted the raw base setting and export to prores4444xq.
I didn't notice any excessive  aliasing and moiré pattern in fact I didn't see any at all , but I wasn't really try to produce any .
It's only a 60 second test , but it look killer . So if normally on the 5d2 you get up to 11.92 EV of dynamic range (http://www.magiclantern.fm/forum/index.php?topic=10111.msg98281#msg98281) dual ISO at 100/800
(I only tested those number)  adds 2.5-3 stops for a total of up to about 15 stops of usable DR , that's amazing for a 6 year old camera .
I'll post some images in day or two , when I had more time to do a more comparative test .
Title: Re: Video - dual iso for 5D2
Post by: waza57 on March 14, 2016, 07:51:42 AM
@jpegmasterjesse

https://bitbucket.org/waza57/magic-lantern/downloads/dual_iso.mo (https://bitbucket.org/waza57/magic-lantern/downloads/dual_iso.mo)
Title: Video - dual iso for 5D2
Post by: DeafEyeJedi on March 14, 2016, 05:53:52 PM
I have a friend whom has a 5D2 ... I can test this on it for y'all ... @reddeercity I'd suggest to try @Danne's MLP for Dual ISO footage. Not only it's faster but it seems to spit out results about 1-stop brighter than if used w MLVFS which is closer than original exposure from the shot.

Still trying to investigate and figure out why Dual ISO comes out about 1-stop underexposed.

Try it out and you'll see what I mean.
Title: Video - dual iso for 5D2
Post by: DeafEyeJedi on March 15, 2016, 11:54:16 PM
According to my friend's 5D2 test footage that I shot in Dual-ISO (100/1600) which shows a lot of green cast (or possible incorrect metadata for Blacklevels and/or more others?) ... Any of you guys getting those as well?

Dual-ISO
(https://farm2.staticflickr.com/1528/25818932995_c60a7d61b2.jpg) (https://flic.kr/p/FkwMga) https://vimeo.com/159129291

Non Dual-ISO
(https://farm2.staticflickr.com/1569/25792865616_4374713a9f.jpg) (https://flic.kr/p/FiebkA) https://vimeo.com/159129292

Files were converted with @Danne's MLP and here are the samples files (MLV plus a single DNG from each) from 5D2 below for those that want to play.

DNG's -- https://mega.nz/#F!qgFDVIYY!Qe6XZc8FyVnbM6C4_OtdsA

MLV's -- https://mega.nz/#F!Wo8SDLaR!KD98tGfnzJMHlLy-cllB1A
Title: Re: Video - dual iso for 5D2
Post by: reddeercity on March 16, 2016, 01:46:36 AM
Ok here is my Screen shots with dual iso 100/800 from 5d2
First one I used MLVFS Mac ver. basic raw camera setting in resolve add 3+
for exposure , color space BMFilm that it.
(https://farm2.staticflickr.com/1707/25191231354_f703d0f659_z.jpg) (https://flic.kr/p/Eo4Dr5)
ScreenShot Windows7 Resolve12MLVFS_dualISO (https://flic.kr/p/Eo4Dr5) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr


This one I use BarracudaGUI for cr2hdr dual iso raw post-processing (http://www.magiclantern.fm/forum/index.php?topic=9102.0) and you can see a green cast , or black level issue , same setting as the first
I like it on windows , it's use 100% of my over clocked 4.9Ghz AMD FX8350 ,
(don't laugh it killer fast like my over clocked  i7 Hackintoss) and process way faster then MLVFS

(https://farm2.staticflickr.com/1650/25821718165_02ef652690_z.jpg) (https://flic.kr/p/FkM4ck)
ScreenShot Windows7 Resolve12_CR2HDR_dualISO (https://flic.kr/p/FkM4ck) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr
Title: Re: Video - dual iso for 5D2
Post by: Danson Delta-40 on March 16, 2016, 04:23:14 AM
You guys are amazing. Keep it up. I downloaded it myself, but I only have a 7D. Will it fry my 7d or should I got ahead and test it out anyways?
Title: Re: Video - dual iso for 5D2
Post by: Danne on March 16, 2016, 09:07:15 AM
Tried the files you uploaded deafeyejedi and compared the files that seems to be working with MLVFS but not with cr2hdr. Tested changing the black levels to the same as in mlvfs but it seems more things are off the charts. Might be good to post the testfiles in the dual iso thread and ask if the necessary changes in cr2hdr can be made. That is, if dual iso in 5D mark II is here to stay.
Title: Re: Video - dual iso for 5D2
Post by: waza57 on March 16, 2016, 03:13:52 PM
@DeafEyeJedi


In my case it' works  with :
extracting dng files from Mlrawviewer and doing a cr2hdr-20bit.exe or cr2hdr.exe.

And make exiftool.exe -TAG -Blacklevel=7400 -Whitelevel=64000 (or values around this)on each DNG files.

I don't know  why I must do exiftool yet.
Title: Re: Video - dual iso for 5D2
Post by: nikfreak on March 16, 2016, 04:25:53 PM
@waza57:

can you please post output values of ADTG CMOS (either written in text or as screenshot) from your 5D2 for ISO100 and ISO200? Examining your code the address doesn't change for all ISO's?
Title: Re: Video - dual iso for 5D2
Post by: waza57 on March 16, 2016, 06:36:15 PM
@nikfreak

movie mode -->  cmos[0] at 0x404b4590

                          iso100 cmos[0]= 0x203 (nrzi=0x3fd)
                          iso200 cmos[0]= 0x207 (nrzi=0x3fa)           ....only cmos[0] values changes , not the address.
                         
photo mode -->                at 0x404b3b5c
                       
                          iso100                 0x203 (nrzi=0x3fd)
                         
                                        at 0x404b3b6a                               Then in photo mode in contrary the cmos[0] address changes to according with different iso !!

                          iso200                  0x207 (nrzi=0x3fa)

In dual iso photo mode:
                                        at 0x404b3b5c                                the same address that iso100 in normal photo mode but with the dual value of cmos[0]
                         iso100-1600           0x383 (nrzi=0x2fd)

So, To have dual iso video with 5d2 my idea is to put these different values of "cmos[0] photo-dual iso" in cmos[0] of movie mode always at the same address 0x404b4590.
Ouf......I do not know if this is useful and I'm quite understandable
             
Title: Re: Video - dual iso for 5D2
Post by: nikfreak on March 16, 2016, 07:13:02 PM
@waza57: thank you very much.
@a1ex: is the output with the method described really dual-iso?
Title: Re: Video - dual iso for 5D2
Post by: reddeercity on March 16, 2016, 09:52:12 PM
Further examination of the Cdng from MLVFS with the dual iso option enable and CR2HDR with Exiftool

This one was process though MLVFS With dual iso enabled/default setting
ExifTool Version Number         : 10.09
File Name                       : M13-2146_000000.DNG
Directory                       : C:/Users/FX8350/Desktop/New folder
File Size                       : 3.3 MB
File Modification Date/Time     : 2016:03:13 22:46:43-06:00
File Access Date/Time           : 2016:03:16 13:57:24-06:00
File Creation Date/Time         : 2016:03:16 13:57:24-06:00
File Permissions                : rw-rw-rw-
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Subfile Type                    : Full-resolution Image
Image Width                     : 1856
Image Height                    : 928
Bits Per Sample                 : 16
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Fill Order                      : Normal
Make                            : Canon
Camera Model Name               : Canon EOS 5D Mark II
Strip Offsets                   : 65536
Orientation                     : Horizontal (normal)
Samples Per Pixel               : 1
Rows Per Strip                  : 928
Strip Byte Counts               : 3444736
Planar Configuration            : Chunky
Software                        : MLVFS
Modify Date                     : 2016:02:13 21:46:43
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 0 1 1 2
Exposure Time                   : 1/53
F Number                        : 1.4
ISO                             : 100
Sensitivity Type                : ISO Speed
Exif Version                    : 0230
Subject Distance                : 180 m
Focal Length                    : 50.0 mm
Lens Model                      : 50mm
DNG Version                     : 1.4.0.0
Unique Camera Model             : Canon EOS 5D Mark II
Black Level                     : 7176
White Level                     : 60000
Default Crop Origin             : 0 0
Default Crop Size               : 1856 928
Color Matrix 1                  : 0.5309 -0.0229 -0.0336 -0.6241 1.3265 0.3337 -
0.0817 0.1215 0.6664
Color Matrix 2                  : 0.4716 0.0603 -0.083 -0.7798 1.5474 0.248 -0.1
496 0.1937 0.6651
As Shot Neutral                 : 0.4580008629 1 0.5775139181
Baseline Exposure               : 0
Calibration Illuminant 1        : Standard Light A
Calibration Illuminant 2        : D65
Active Area                     : 0 0 928 1856
Forward Matrix 1                : 0.8924 -0.1041 0.176 0.4351 0.6621 -0.0972 0.0
505 -0.1562 0.9308
Forward Matrix 2                : 0.8924 -0.1041 0.176 0.4351 0.6621 -0.0972 0.0
505 -0.1562 0.9308
Time Codes                      : 00:00:00.00
Frame Rate                      : 23.976
Baseline Exposure Offset        : 0
Aperture                        : 1.4
CFA Pattern                     : [Red,Green][Green,Blue]
Image Size                      : 1856x928
Megapixels                      : 1.7
Shutter Speed                   : 1/53
Focal Length                    : 50.0 mm
Light Value                     : 6.7
-- press RETURN --



This was process thought BarracudaGUI for cr2hdr dual iso
ExifTool Version Number         : 10.09
File Name                       : M13-2146_000000 (2).DNG
Directory                       : C:/Users/FX8350/Desktop/New folder
File Size                       : 3.6 MB
File Modification Date/Time     : 2016:03:15 15:48:31-06:00
File Access Date/Time           : 2016:03:16 14:00:28-06:00
File Creation Date/Time         : 2016:03:16 14:00:28-06:00
File Permissions                : rw-rw-rw-
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Image Description               :
Make                            : Canon
Camera Model Name               : Canon EOS 5D Mark II
Orientation                     : Horizontal (normal)
Software                        : MLVFS
Modify Date                     : 2016:02:13 21:46:43
Artist                          :
Subfile Type                    : Full-resolution Image
Image Width                     : 1856
Image Height                    : 928
Bits Per Sample                 : 16
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Strip Offsets                   : 4330
Samples Per Pixel               : 1
Rows Per Strip                  : 928
Strip Byte Counts               : 3444736
X Resolution                    : 180
Y Resolution                    : 180
Planar Configuration            : Chunky
Resolution Unit                 : inches
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 0 1 1 2
Black Level                     : 8192
White Level                     : 40000
Default Crop Origin             : 0 0
Default Crop Size               : 1856 928
Active Area                     : 0 0 928 1856
Opcode List 1                   : (Binary data 28 bytes, use -b option to extrac
t)
XMP Toolkit                     : Image::ExifTool 10.02
Subject                         : Dual-ISO
Copyright                       :
Exposure Time                   : 1/53
F Number                        : 1.4
Exposure Program                : Not Defined
ISO                             : 100
Sensitivity Type                : ISO Speed
Exif Version                    : 0230
Date/Time Original              :
Shutter Speed Value             : 1
Aperture Value                  : 1.0
Exposure Compensation           : 0
Max Aperture Value              : 1.0
Subject Distance                : 180 m
Metering Mode                   : Unknown
Flash                           : No Flash
Focal Length                    : 50.0 mm
Sub Sec Time                    :
Sub Sec Time Original           :
Focal Length In 35mm Format     : 0 mm
Lens Model                      : 50mm
TIFF-EP Standard ID             : 1 0 0 0
Serial Number                   :
DNG Version                     : 1.3.0.0
DNG Backward Version            : 1.3.0.0
Unique Camera Model             : Canon EOS 5D Mark II
Color Matrix 1                  : 0.4716 0.0603 -0.083 -0.7798 1.5474 0.248 -0.1
496 0.1937 0.6651
Analog Balance                  : 1 1 1
As Shot Neutral                 : 0.3895824105 1 0.4863276282
Baseline Exposure               : undef
Baseline Noise                  : 1
Baseline Sharpness              : 1.333333333
Linear Response Limit           : 1
Calibration Illuminant 1        : D65
Frame Rate                      : 23.976
Aperture                        : 1.4
CFA Pattern                     : [Red,Green][Green,Blue]
Image Size                      : 1856x928
Megapixels                      : 1.7
Shutter Speed                   : 1/53
Focal Length                    : 50.0 mm
Light Value                     : 6.7
-- press RETURN --


There a few thing that are different , White level , Black Level are off

mlvfs-- Black Level   : 7176
                 White Level   : 60000
                 Strip Offsets : 65536
                  Forward Matrix 1 : 0.8924 -0.1041 0.176 0.4351 0.6621 -0.0972 0.0505 -0.1562 0.9308
                  Forward Matrix 2  : 0.8924 -0.1041 0.176 0.4351 0.6621 -0.0972 0.0505 -0.1562 0.9308

cr2hdr---Black Level  : 8192
                 White Level  : 40000
                  Strip Offsets  : 4330
                   Color Matrix 1 : 0.4716 0.0603 -0.083 -0.7798 1.5474 0.248 -0.1496 0.1937 0.6651
No time code , only one forward matrix & it not right.

So I don't think its the dual iso module problem , it some think to do with cr2hdr as I extracted the cr2hdr process one with mlvfs on windows7 with Pismo quick mount
Title: Re: Video - dual iso for 5D2
Post by: DeafEyeJedi on March 17, 2016, 12:31:05 AM
Excellent work @reddeercity!
Title: Re: Video - dual iso for 5D2
Post by: jpegmasterjesse on March 17, 2016, 06:29:25 AM
I went looking for proof that this gets us the kind of dynamic range we're looking for. With MlRawViewer the MLV file looks like what you'd expect - interlaced, but data-rich.  I exported as DNG's from MlRawViewer from there and used Cr2HDR to convert.  Perhaps that's a bad workflow, because the resulting dngs were missing any information about shadows or blacks.

I imported the original, interlaced DNG's into lightroom. Added some grain and noise reduction.  The interlacing isn't so noticeable.
https://gfycat.com/WastefulGraciousAndeancat

The point of that being that YES, this is a step in the right direction.  Even before the conversion, I definitely felt as if I was working with the ISO's I specified in Dual-ISO.

THANKS!  If I can provide some more objective information for anyone, please let me know.
Title: Re: Video - dual iso for 5D2
Post by: Danne on March 18, 2016, 08:35:56 AM
Feel free to test out A1ex experimental cr2hdr build for the 5D mark 2 in MLP. (thanks dfort for compiling)

From discussion here
http://www.magiclantern.fm/forum/index.php?topic=7139.msg164375#msg164375

Here is a version of MLP with the experiment cr2hdr build for 5D mark 2 (and other cams to it seems). It will work right out of the box for developing the dng files. Havn,t had time to test it out and don,t own a 5d mark 2.

MLP_5D2
https://www.dropbox.com/s/vawstdji3yif8sk/MLP_5D2.zip?dl=0

Works like this
1 - Download and then double click MLP(.workflow) to install
2 - Right click and select MLP from the contextual menu on a folder containing dual iso MLV or RAW or MLV-lite footage. Do this twice cause first time MLP creates a folder structure then quits. The second time will start processing. Same white level(avoid flicker) will be applied to all files and it works multithreaded.
Title: Video - dual iso for 5D2
Post by: DeafEyeJedi on March 18, 2016, 08:43:41 AM
Great progress everyone on board so far and this is why I love you guys. Humans are amazing. Especially in this Community. Thanks for the quick update on MLP aiming for 5D2 users (Hello there we didn't forget about you!) and honestly none of this would have happened this promptly if it weren't for your suggestion to post my test files from my friend's 5D2. Bold move, @Danne!

[emoji106]
Title: Re: Video - dual iso for 5D2
Post by: reddeercity on March 18, 2016, 08:47:50 AM
Just tested update CR2HDR process the dual iso without any problem , there is a little difference in the mid tone area
there seems to more mid detail with MLVFS  but more shadow detail with the update CR2HDR.

One think I notice is with cr2hdr process file in resolve I had to use less exposure then mlvfs process file
by about half , e.g. cr2hdr was about 2.5 where mlvfs was 4.6 , interesting .

Link to my dropbox with Dual ISO Cdng if any one would like check them out

Non process dual ISO --> https://www.dropbox.com/s/qnzst1571q766pu/Non_process_M13-2146.MLV.frame_000000.dng?dl=0
New CR2HDR dual ISO       https://www.dropbox.com/s/3w24ifc4opurmgi/New_CR2HDR_M13-2146.MLV.frame_000000.DNG?dl=0
Old CR2HDR dual ISO        https://www.dropbox.com/s/3al9oe8oerahtzv/Old_CR2HDR_M13-2146_000000.DNG?dl=0
MLVFS dual ISO (PC ver.)   https://www.dropbox.com/s/c7punwsgezdmopt/M13-2146_000000.dng?dl=0

Non process Dual ISO 100/800 1856x928  from 5D2
(https://farm2.staticflickr.com/1564/25247778443_1ff8070722.jpg) (https://flic.kr/p/Et4sUT)
Non_process_M13-2146.MLV.frame_000000 (https://flic.kr/p/Et4sUT) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

New CR2HDR process though BarracudaGUI
(https://farm2.staticflickr.com/1712/25575943190_1eed090cb4.jpg) (https://flic.kr/p/EY4oRL)

New_CR2HDR_M13-2146.MLV.frame_000000 (https://flic.kr/p/EY4oRL) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

MLVFS process dual iso on windows7
(https://farm2.staticflickr.com/1630/25243826364_6a4db33c1a.jpg) (https://flic.kr/p/EsHd6A)
M13-2146_000000 (https://flic.kr/p/EsHd6A) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr

Old CR2HDR , wrong white & black levels
(https://farm2.staticflickr.com/1680/25876518485_5f39f24790.jpg) (https://flic.kr/p/FqBVqr)

Old_CR2HDR_M13-2146_000000 (https://flic.kr/p/FqBVqr) by RedDeerCityTV (https://www.flickr.com/photos/67942440@N06/), on Flickr


Title: Re: Video - dual iso for 5D2
Post by: tecgen on March 18, 2016, 12:44:04 PM
QuoteRegarding the 50d, someone should try to manually compare (with ADTG-gui) the actual values CMOS[0] for the iso photo mode with those of the video mode(for the same iso).
iso 100, 200, 400, 800, 1600.

As far as I understand the implementation for the 5D2 and 50D is exactly the same, just with different address values.

Could someone tell us in more detail, how we can find out the values for the 50D? When we know the value for other cameras next to the 5D2, that also use only one register, we could change the implementation to a more generic "one register" solution, right?
Title: Re: Video - dual iso for 5D2
Post by: waza57 on March 18, 2016, 07:34:30 PM
For 50D :

QuoteAs far as I understand the implementation for the 5D2 and 50D is exactly the same, just with different address values.

I don't know at all but if my little method can be applied for 50D:

Without dual iso:
We do learn what is the CMOS[0] address when LiveView is ON. (start at iso 100 for example)
is it the same as photo mode (probably not)?
What is his value and is it the same than photo mode (for same iso 100) ?

and make the same thing for iso 200.... it would be enough to have an idea about 50D.

To do this :
We must have a 50d (I have not).
Enable adtg_gui module after compiled it for your nightly builds installed( except if a precompiled one exist, I don't know that).
Then we could investigate the addresses of CMOS[0] and its values.




Title: Re: Video - dual iso for 5D2
Post by: DeafEyeJedi on March 18, 2016, 09:10:11 PM
Quote from: Danne on March 18, 2016, 08:35:56 AM
MLP_5D2
https://www.dropbox.com/s/vawstdji3yif8sk/MLP_5D2.zip?dl=0

Works like this
1 - Download and then double click MLP(.workflow) to install
2 - Right click and select MLP from the contextual menu on a folder containing dual iso MLV or RAW or MLV-lite footage. Do this twice cause first time MLP creates a folder structure then quits. The second time will start processing. Same white level(avoid flicker) will be applied to all files and it works multithreaded.

Question though ... if I were to test 5D2 files after installing this version ... Do I need to reinstall back to the original MLP from your thread? Wasn't sure if this was designed to be used ONLY with 5D2 atm? If so, would be nice if we could add 'MLP_5D2' under the service list unless you think you will eventually make this into the classic MLP?

Thanks, D!
Title: Re: Video - dual iso for 5D2
Post by: Danne on March 18, 2016, 09:47:22 PM
You can run it as usual use the command to transfer the other stuff. Or grab the cr2hdr(I renamed it cr2hdr20bit) from the 5D2 version and put it in regular MLP if you know where to do it.
Title: Video - dual iso for 5D2
Post by: DeafEyeJedi on March 18, 2016, 10:41:40 PM
Got it -- Will do just that. Thanks!

However, we may have a problem with the new cr2hdr binary (I ran this twice just to be sure) and both converted DNG's and ProRes4444 show a bunch of random dropped/pinkish frames (or something else strange is going on) but when I then took the original MLV's (same 5D2 test files that I posted from earlier) and rendered through MLVFS (which still uses the older Binary -- not the one that @dfort uploaded) via DR12 which spitted out fine like normal as it seems.

https://vimeo.com/159560127
Title: Re: Video - dual iso for 5D2
Post by: dfort on March 18, 2016, 11:06:38 PM
Humm--looks like people are posting the same comments on this topic and the main Dual ISO topic (http://magiclantern.fm/forum/index.php?topic=7139). Makes things a bit confusing.

In any case, that cr2hdr I posted is for testing so it looks like you got to be the crash test dummy  :D

I don't have any 5D2 Dual ISO MLV files but the posted tests do look very promising. Maybe the 7D and 50D are next?
Title: Re: Video - dual iso for 5D2
Post by: Danne on March 18, 2016, 11:20:50 PM
Isn,t David using his own created dual iso algoritm in MLVFS?
Sean, can you try the same clip with the regular MLP and check for dropped frames?
Title: Video - dual iso for 5D2
Post by: DeafEyeJedi on March 18, 2016, 11:38:08 PM
Quote from: dfort on March 18, 2016, 11:06:38 PM
Humm--looks like people are posting the same comments on this topic and the main Dual ISO topic (http://magiclantern.fm/forum/index.php?topic=7139). Makes things a bit confusing.

In any case, that cr2hdr I posted is for testing so it looks like you got to be the crash test dummy  :D

I don't have any 5D2 Dual ISO MLV files but the posted tests do look very promising. Maybe the 7D and 50D are next?
It's still online from my earlier post (http://www.magiclantern.fm/forum/index.php?topic=16854.msg164286#msg164286post) if you would like to play with. I'm also trying to get ahold of my co worker to shoot some more Dual-ISO footage from his 5D2 just to have options for us to troubleshoot this bad boy down.

Quote from: Danne on March 18, 2016, 11:20:50 PM
Isn,t David using his own created dual iso algoritm in MLVFS?
Sean, can you try the same clip with the regular MLP and check for dropped frames?
Ah, well the one thing I'm surprised about is the fact that @dmilligan hasn't updated his cr2hdr binary yet (which I thought was needed in order for 5D2 Dual ISO to work?) however I then ran another test running with just MLVFS and default settings for cr2hdr20-bit and applied Cinelog-C in AE so here's what it looks like...

https://vimeo.com/159564896

Also here's the quick comparisons of in between MLP and MLVFS...

https://vimeo.com/159564926

@Danne -- I am currently rendering the same test files from 5D2 with the latest MLP from original post of your thread just to see what happens with the random dropped/pinkish frames on both the DNG's and ProRes4444 ... Stand by!

*EDIT*

Just finished spitting out DNG's and ProRes4444 which showed these same 'Green Cast' issues from few days ago BUT no dropping/pinkish frames.

https://vimeo.com/159566907

It seems with old cr2hdr which shows green cast and new cr2hdr which shows random pinkish frames but no green cast ... Weird?

How exactly can we narrow down the culprit on the latest cr2hdr?

Also let me know if I need to report this in the orignal Dual-ISO thread for @a1ex to see or not just yet?
Title: Re: Video - dual iso for 5D2
Post by: reddeercity on March 19, 2016, 03:38:03 AM
Seems to be ok on the PC side with the new updated cr2hdr
No green cast
Title: Re: Video - dual iso for 5D2
Post by: reddeercity on March 19, 2016, 05:40:05 AM
Just a quick report , I did have a Liveview freeze up with dual iso module loaded,
basically I was doing very quick pan without recording raw video and needed a battery pull
re-start cam every back to normal.
so watch out for any quick motion with cam as this can freeze the 5D2 Liveview with dual ISO.
Title: Re: Video - dual iso for 5D2
Post by: Danne on March 19, 2016, 07:44:54 AM
The problem seems to be with the experimental cr2hdr build @deafeyejedi. Maybe good ide to continue this discussion in the other dual iso thread?

If you report green cast still going on you have to clarify what you are testing. If you run the official non experimental build of MLP of course there will be a green cast. If you run the experimental 5D2 build it should not.
The test were after was the dropframe issue. If it really is an issue. Hard to say. Maybe let reddeercity try out the file with the barracuda workflow.
Title: Re: Video - dual iso for 5D2
Post by: tecgen on September 01, 2016, 01:05:45 PM
Quote from: reddeercity on March 19, 2016, 05:40:05 AM
Just a quick report , I did have a Liveview freeze up with dual iso module loaded,
basically I was doing very quick pan without recording raw video and needed a battery pull
re-start cam every back to normal.
so watch out for any quick motion with cam as this can freeze the 5D2 Liveview with dual ISO.


Do you still use the dual iso module on your 5d2? Did your Liveview freeze just once or did it happen more often?

@all
Did anyone except Nikfreak already tried to implement dual iso video for the 50d?
Title: Re: Video - dual iso for 5D2
Post by: reddeercity on September 02, 2016, 04:36:39 AM
No I don't use it much and no I haven't had a lockup but I don't pan fast so I don't have problems
Works great but a nightmare in processing , unless you have a very fast Computer & Raid0 or storage SSD .
Title: Re: Video - dual iso for 5D2
Post by: tecgen on September 02, 2016, 08:48:33 AM
> No I don't use it much and no I haven't had a lockup but I don't pan fast so I don't have problems

Good to hear that it the approach works somehow reddeercity. Maybe your freeze of the Liveview on the was related to something else. I'll test it on the 5d2 by myself later.
First I want to check out if we can get dual iso video on the 50D. I'm sure the implementation for the 50D is very similar to the 5d2. We know already the address of CMOS[0] in movie mode on the 50D. I used the implementation that works for the 5d2 and tried to get it working for the 50D.
https://bitbucket.org/tecgen/magic-lantern/branch/5D2-50D-dualiso-video#diff

I don't know why, but dualiso does not yet work when Liveview is turned on. I checked the CMOS[0] address for ISO 100 in movie mode. It's correct. I played around a bit with atdg_gui. By changing the value to 8, 18 or 28 I was able to record a raw video of something that looks similar to dualiso. I need to check the DNGs if these are just horizontal stripes or not.

> Works great but a nightmare in processing , unless you have a very fast Computer & Raid0 or storage SSD .
Sure it needs more time and steps in postprocessing. But it's the same on every dualiso video supported camera model. No matter if your using a 5d3 or 550d. It just takes some time ;) I think it would be cool to have this dualiso video feature on all ML cameras. I don't want to waste the effort waza57 put into the implementation for the 5d2. At least this implementation should be in the nightly build when its stable enough.
Title: Re: Video - dual iso for 5D2
Post by: Deadcode on October 03, 2016, 07:23:37 AM
Please help me!

I tried the dual ISO module above. It worked but once the camera freezed and i couldn't do anything. So i removed the battery. I know it's possibile to cause permanent damage on the camera or CF card, maybe that happened this time.

After restoring the camera to the original firmware, i still get unusable raw files and unusable .mov-s.
Check it out: https://drive.google.com/drive/folders/0Bwoum3ZKc_jNR3lLbmV3S1RodUk?usp=sharing

The strange thing is: ML Raw video is not affected! only still picture raw and compressed video. (jpg's are not perfect, but OK)
What can i do? If i have to send the camera to repair what module should be changed?

Thank you for your help!


edit: 5D markII, Lexar 64GB 1066x
Title: Re: Video - dual iso for 5D2
Post by: a1ex on October 03, 2016, 08:10:15 AM
Quote from: Deadcode on October 03, 2016, 07:23:37 AM
If i have to send the camera to repair what module should be changed?

The card.
Title: Re: Video - dual iso for 5D2
Post by: dfort on April 15, 2018, 09:20:28 PM
Quote from: TrEK on April 15, 2018, 04:42:25 PM
how you use DUAL ISO in video recording ?

Quote from: jpegmasterjesse on April 15, 2018, 07:04:12 PM
It's an old module that was being tested on the 5D2.  I don't have any documentation on who created it.

Here's how to find out more about that old module. In the modules directory there is a shell script called module_hginfo_dump.sh. Run it on the module--I ran it from the modules directory:

./module_hginfo_dump.sh /[PATH_TO]/dual_iso.mo
Name        : Dual ISO
Author      : a1ex
License     : GPL
Summary     : Dynamic range improvement by using two ISOs in one picture
Forum       : http://www.magiclantern.fm/forum/index.php?topic=7139.0
Description : Increases dynamic range by sampling the sensor at two different
              ISOs, switching ISO for every other line pair. This trick cleans up
              shadow noise, resulting in a dynamic range improvement of [...]
Help page 1 : Quick start
             
               *  Start at ISO 100 in Canon menu [...]
Help page 2 : Tips and tricks
             
               *  Do not use dual ISO for regular scenes that don't require a very [...]
Last update : ab1e90c on 2015-11-24 09:50:19 UTC by a1ex:
              cr2hdr: moved safeguard from median_int_wirth to kth_smallest_i...
Build date  : 2016-03-13 16:24:44 UTC
Build user  : cmoi@kiranfou7


The "Last update" is the changeset so knowing that you can search for it on Bitbucket (https://bitbucket.org/hudson/magic-lantern/commits/ab1e90c88b9515856f77bff0a88714d900918f39) and if you have a development environment setup you can roll back to that version like this:

hg update ab1e90c

Now you can look at the source code for that old module.

So how different is it from the current unified version?

hg diff -r ab1e90c:unified modules/dual_iso/dual_iso.h

Not much though there is one significant change:

diff -r ab1e90c88b95 -r 36acb0f9c3e2 modules/dual_iso/dual_iso.c
--- a/modules/dual_iso/dual_iso.c       Tue Nov 24 11:50:19 2015 +0200
+++ b/modules/dual_iso/dual_iso.c       Fri Feb 02 22:07:41 2018 +0100
@@ -556,8 +556,12 @@

     int mvi = is_movie_mode();

-    int raw = mvi ? FRAME_CMOS_ISO_START && raw_lv_is_enabled() : ((pic_quality & 0xFE00FF) == (PICQ_RAW & 0xFE00FF));
+    /* default checks will not work here - we need full-sized raw in photo mode */
+    int raw = mvi ? raw_lv_is_enabled() : ((pic_quality & 0xFE00FF) == (PICQ_RAW & 0xFE00FF));

+    if (mvi && !FRAME_CMOS_ISO_START)
+        MENU_SET_WARNING(MENU_WARN_NOT_WORKING, "Dual ISO does not work in movie mode on your camera.");
+   
     if (!raw)
         menu_set_warning_raw(entry, info);
}
@@ -832,7 +836,7 @@
         CMOS_FLAG_BITS = 2;
         CMOS_EXPECTED_FLAG = 0;
     }
-    else if (is_camera("700D", "1.1.4"))
+    else if (is_camera("700D", "1.1.5"))
     {
         is_700d = 1;   


Why was that block put in there? Let's run "blame" on dual_iso.c -- this can be done on Bitbucket (https://bitbucket.org/hudson/magic-lantern/annotate/unified/modules/dual_iso/dual_iso.c?fileviewer=file-view-default). Looks like blocking dual_iso on the 5D2 was put there for a reason, though the reason isn't clear in the commit message (https://bitbucket.org/hudson/magic-lantern/commits/e609c143d3f1d13888fd41972a2f1842a7e44b9b).

Time to search the forum. Looks like someone got quite far with it here: Video - dual iso for 5D2 (https://www.magiclantern.fm/forum/index.php?topic=16854.0) (recursive link) However, no pull request was submitted.
Title: Re: Video - dual iso for 5D2
Post by: reddeercity on April 15, 2018, 10:36:59 PM
https://bitbucket.org/waza57/magic-lantern/commits/7be4cd7f975ef25e98143d10da26712e5ec8ca77?at=dual_iso_video_5d2
Yea it works just fine  ;)  (https://bitbucket.org/reddeercity/magic-lantern_10-12bit/downloads/5D2212_Video_dual_iso.zip)
Title: Re: Video - dual iso for 5D2
Post by: dfort on April 15, 2018, 11:03:18 PM
Great! Thanks reddeercity

@waza57 -- What are you up to these days? I see you're still active on the forum and Bitbucket.
Title: Re: Video - dual iso for 5D2
Post by: reddeercity on April 16, 2018, 03:33:40 AM
waza57 has put some amazing work into my 3K/UHD 5D2 Raw development thread , I guess he has been able to increase resolution without the adtg_gui.mo
https://www.magiclantern.fm/forum/index.php?topic=19336.msg198809#msg198809
He mention that when he has a chance he'll post his code  :)
Title: Re: Video - dual iso for 5D2
Post by: waza57 on April 16, 2018, 08:33:24 PM
Hello , I need your help for that because i'm not sure.
For dual-ISO on 5D2, how to make a pull request and to which branch ?
Thanks.
Title: Re: Video - dual iso for 5D2
Post by: Danne on April 16, 2018, 08:56:31 PM
Do you have a bitbucket account? If not I think creating one would be an easy way to fork, branch and submit a pull request. What branch did you work from to begin with when starting changing dual iso code?
Title: Re: Video - dual iso for 5D2
Post by: waza57 on April 17, 2018, 12:21:46 AM
Thanks,
yes, I have an account.
I had created a new branch (dual_iso_video_5d2 ) from a fork the 2016-03-13
I  only made changes in dual_ISO.c file.
What next for a pull request ?
Title: Re: Video - dual iso for 5D2
Post by: dfort on April 17, 2018, 12:47:51 AM
@waza57 - Push your changes to Bitbucket then in your web browser click on "Pull requests" in the left panel and pick the appropriate branch in the hudson repository. You didn't mention which branch you started with, unified? cr2hdr? I see several changes on a crop_rec_4k_5D2 branch in your repository but that doesn't have the dual_iso changes, does it?

Perhaps you should ask a1ex about an appropriate branch for this pull request.
Title: Re: Video - dual iso for 5D2
Post by: waza57 on April 19, 2018, 09:50:39 AM
Thanks,

Changes are in branch: dual_iso_video_5d2 in my repository. (crop_rec_4k_5D2 i s for experiment but not complete)

I made pull request on "unified" but perhaps a mistake, i don't know.
Title: Re: Video - dual iso for 5D2
Post by: Danne on April 19, 2018, 09:59:22 AM
Great waza57. If you started working from unified there´s where you should send your pull request.
You seem to now what you´re doing. I wanted to ask if you perhaps wants to take a look at this problem?
https://www.magiclantern.fm/forum/index.php?topic=7139.msg200188#msg200188
Title: Re: Video - dual iso for 5D2
Post by: waza57 on April 19, 2018, 10:13:47 AM
Sorry but i don't understand nothing about pink highlight.
it's comes with 5D2 ?
Title: Re: Video - dual iso for 5D2
Post by: Danne on April 19, 2018, 10:26:40 AM
Nope 100D bit I think it's rekated to 12bits lossless rather than camera.