Author Topic: 3K/UHD 5D2 Raw development and Other Digic IV Cams  (Read 93039 times)

Igor_Braun

  • New to the forum
  • *
  • Posts: 26
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #625 on: April 05, 2019, 10:21:24 AM »
Near the bottom of MLV Raw Video Tab or in the "mlv_rec.cfg" file
This is my mlv_rec.cfg file
Code: [Select]
# Config file for module mlv_rec (MLV_REC.MO)
mlv.video.enabled = 1
mlv.res.x = 7
mlv.bpp = 0
mlv.aspect_ratio = 17
mlv.write_speed = 6969
mlv.display_rec_info = 2
mlv.buffer_fill_method = 0  *** this is the SRM***
Ok that's what I'll looking for , High ISO seems to have problem with these experimental builds even the old
dual ISO video module did this , maximum ISO I use is 800 (rarely thou) , I mainly use 100 -> 400  & if that doesn't get
correct exposure then I just add artificial light .
I test these experiments  between 100 & 400 ISO , so I can say at those ISO setting I have no issues .

If you when to use 800 & 1600 ISO I would recommend that you first set it in 3x3 (FHD) then reboot the cam
this also will refresh liveview , (that's part of the problem "Liveview not refreshing with HI ISO)
A thought , the play button should also refresh Liveview , so maybe try that .

In the mean time I will look in to this further , but no guarantees that I can fix it .

Understood.

I've tried to achieve that glitch again, but no result... But from this moment i will be attaching my videos.  This shot with ISO 1600, WB 4500, everything was ok...
https://drive.google.com/open?id=15gcACm5Wyj0x2MUcQd0BU66baNNeATuA

My  mlv_rec.cfg is:

# Config file for module mlv_rec (MLV_REC.MO)

mlv.video.enabled = 1
mlv.res.x = 13
mlv.bpp = 0
mlv.aspect_ratio = 16
mlv.write_speed = 7394

Thank you for the advise, and i understood you about 100-400 iso range.

Also wanted to ask you about your plans for squeezed 4096 videos. I just think it's a good way to cover a larger sensor space for reducing the crop factor to 1.37 and get more cinematic look. Or am i wrong? And also remember that aspect ratio, in that case, was close to 2.34:1, which is awesome...

Igor_Braun

  • New to the forum
  • *
  • Posts: 26
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #626 on: April 07, 2019, 05:08:23 AM »
Today when i was shooting a video without the crop mode for youtube it came again. Vertical lines and something strange with colors... it disappears with time.


 
Iso 400
WB 5400
https://drive.google.com/open?id=1GaTK9QzHNCzCmXwiMNXKOxxuQrFHw8ms
If it necessary i can upload the MLV

Galterius

  • New to the forum
  • *
  • Posts: 9
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #627 on: April 07, 2019, 08:20:19 PM »
I used the last crop_rec module and exported in h.265 (2560x1200), and colors change while I am recording, image is green at the beginning, and then it becomes normal (plus there is a lot of color noise). (it is out of focus at the beginning because of me ^^ but we can see some weird lines); On some clips I don't have this problem at all...
I will try with an older crop_rec.

And most of my clips have a very green tint
here it is : https://drive.google.com/open?id=1DDOEYHpd3js1NWW8jdeKUp0cFdbLr7-5

I can upload the mlv if needed.

Danne

  • Hero Member
  • *****
  • Posts: 5106
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #628 on: April 07, 2019, 08:40:25 PM »
Probably power timing regs needs tuning. Code, code, code...

Galterius

  • New to the forum
  • *
  • Posts: 9
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #629 on: April 07, 2019, 09:35:57 PM »
i just tried the crop_rec from the old version (it does 3008x1080) and it seems to work

reddeercity

  • Hero Member
  • *****
  • Posts: 1942
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #630 on: April 08, 2019, 04:17:38 AM »
Ok thanks for the results , it looks like the real time preview reg's are the issue .
I'll look in to this further , later tonight I'll post a non real time preview for 2600x1200
so it can be used until I fix the problem .

reddeercity

  • Hero Member
  • *****
  • Posts: 1942
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #631 on: April 08, 2019, 06:17:05 AM »
Ok let give this new test crop_rec 2.6k_5d2_R_T_PreView_4-7-2019_crop_rec.rar a try .
I re-worked the preset a little more , reduced the raw horizontal resolution from 2632 to 2600 for a final of 2592 in mlv_rec (mod8) so same output as before
just different way of getting that same size & re-adjusted the raw preview Reg's . The horizontal raw preview reg was missed matched to the total raw width with the OB area .   
   

reddeercity

  • Hero Member
  • *****
  • Posts: 1942
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #632 on: April 08, 2019, 06:27:41 AM »
Worked best with grey scale preview ( still have a issue with full canon color) it's still realtime (buy the way) but in B/W .
I tested it at 800 ISO & 1/64th shutter speed at 29.97fps in 3x3 (FHD) when I entered in to crop_rec for 5x zoom
the shutter speed changed to 1/48th of second   ( that's by design  :D )

Also check it with the HDMI EVF , @ 480p & 1080i once again 480p work the best but the canon color is not right
I think there something in  mlv_rec preview code I need to check in to .

Let me know of any issue as before , I'm mainly want to know if that strange color/pattern show up , it shouldn't  but ......

banertop

  • New to the forum
  • *
  • Posts: 48
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #633 on: April 08, 2019, 08:32:35 PM »
once again, thank you for your work. Amazing things you done for 5d2.

In general, what would be a real benefit of shooting 2.6k, rather than 3k....beside a bit more vertical resolution?

We have now, thanks to you, very nice and stable according to my tests at least, 3k on 5d2.....15 or more seconds, but looks very nice...close to 5d3 max resolution.....less crop then 2.6k....i like it very much......but i think maybe i missed something about 2.6k build, because you guys talk about that build a lot......

Am i missing something?

tnx



masc

  • Hero Member
  • *****
  • Posts: 963
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #634 on: April 09, 2019, 08:25:40 PM »
Thanks for the update. But I think the problems are still the same.

The crop frame is not centered on the sensor. See the picture 2nd row 1st column, this is the 3x3 full sensor. 2nd row 2nd column is the croprec frame. 1st row 1st column I painted the used area.

Then I again get some strange problems with black level: by standard it looks a bit greenish... okay, lower black level in post.
When now scrolling through all the frames, I saw that sometimes some area gets orange blacks... see 2nd row 2nd column carefully, on top it is more orange then bottom. The orange area changes over time.

And last problem: when zooming to 100% I get a kind of bayer pattern in the debayered picture - never saw something like this before... see 1st row 2nd column.

5D2.212 | EOSM.202

reddeercity

  • Hero Member
  • *****
  • Posts: 1942
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #635 on: April 09, 2019, 09:47:41 PM »
I don't fix the centering yet , what to see if that odd color thing was
Still there .

I don't have any issue with debaying , I do not see those problem your reporting.
To me it looks like lens color distortion/sharpness problem.
I use MLVFS ( quick mount ) plus After Effect or MLV Producer
I get correct clean black level every time .

Try it on real scene ( not charts) and see it you get the same result

masc

  • Hero Member
  • *****
  • Posts: 963
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #636 on: April 10, 2019, 09:17:10 PM »
Yes, you also see it in "real world" footage. But with the chart you see it more clearly. With ACR and MLVProducer the black is off too (here uncorrected in the pictures) and the moiree looks even worse:
ACR:

ACR, moiree detail

MLVProducer:

MLVProducer, moiree detail
5D2.212 | EOSM.202

Danne

  • Hero Member
  • *****
  • Posts: 5106
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #637 on: April 10, 2019, 09:42:17 PM »
Check black level auto detection code in raw.c. Or timer tweaks. No code, only guesswork.

banertop

  • New to the forum
  • *
  • Posts: 48
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #638 on: April 11, 2019, 01:31:04 AM »
i experienced greenish blacks in my tests, to.

banertop

  • New to the forum
  • *
  • Posts: 48
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #639 on: April 11, 2019, 01:32:56 AM »
but that was with previous build. Did not test the latest one jet.

reddeercity

  • Hero Member
  • *****
  • Posts: 1942
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #640 on: April 11, 2019, 04:59:44 AM »
Yes, you also see it in "real world" footage. But with the chart you see it more clearly. With ACR and MLVProducer the black is off too (here uncorrected in the pictures) and the moiree looks even worse:
Interesting , can upload a .mlv file with the problem & also a DNG please ?
The reason is I want to see the debayed image from you so I can compare on my system to see if I'm missing something .
Can you if you have time to take a short 2144x1074 .mlv in 10bit in normal 3xCrop_mode with out the crop_rec enabled , in fact can compare between 10, 12 & 14bit with the crop_rec .
Do you see black level problem in normal 3x3(FHD) 10,12,14bit ? same in normal 3x crop_mode (not crop_rec) 10,12,14bit , is it normal black levels ?

Also does 3008x1080 preset have ok black level on your system ?
Oh yea also you cam setting , preview mode , etc. ....

Edit:
I'll do some more test here and compare
Edit:#2 by chance did you use "Calibrated Colors" ?
If so Don't !!! it screw up the all the raw levels big times , I never use it .

reddeercity

  • Hero Member
  • *****
  • Posts: 1942
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #641 on: April 11, 2019, 07:49:28 AM »
@masc -- 2592x1200 , exported dng's from mlv app , mlv_dump & mlvfs(quick mount) on windows7 pro
all have identical black levels & look the same  no bad black level here , can't reproduce you're problematic  results .

No W/B adjustment or any adjustment at all , just pass though from mlv file.
shot at 200 ISO , f4.0 @ 24mm on 24-70mm f2.8L canon EF , w/b 5200k manual
on B/W (Grey Scale) ML Preview Liveview .

MLV App , no correction -- dng export uncompressed


MLV_Dump , 10bit dng export
with this
Code: [Select]
mlv_dump --dng --no-fixcp M10-2118.MLV

MLVFS (Quick Mount) 16bit dng export


masc

  • Hero Member
  • *****
  • Posts: 963
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #642 on: April 11, 2019, 08:07:00 PM »
Note, that not every clip looks like that. In average maybe 2 of 3 are okay. One of 3 has wrong blacklevel and very strange behavoir over time and much more moiree.
Standard 3x3 is always fine, at least I never had a bad clip and I can't see anything wrong there.
Here the frame as MLV, you can use MLVFS to get dng. It is not perfectly focused, it was just a quick test, when I got this. But I think it doesn't matter for your anaysis.
https://www.dropbox.com/s/gwim47xsqdbo65z/M09-1858.MLV?dl=0

My setting was the one you posted in first post in this thread. I don't know where to find "calibrated colors"... so I think I did not use it.

Edit:
Recorded some testclips again, but all fine at 2144x1074 10/12/14 bit or 1856x1044 10/12/14 bit. 2.6K setting @ 14bit kills camera -> battery pull. With todays clips I don't got the green again - last time I got several clips with wrong blacks. Same when I tried your 1st 2.6K module: 20 from 33 clips were fantastic, the rest had "changing blacks over time".
5D2.212 | EOSM.202

Igor_Braun

  • New to the forum
  • *
  • Posts: 26
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #643 on: April 13, 2019, 08:43:37 PM »
Hi, my test was okay. I'll be doing tests with high iso.
WB 5400, ISO 1600, the shutter i set to 1/33, it automatically as you said changed to 1/48 (but on the display it showed me 1/24, something like that)



https://drive.google.com/open?id=1awSg_e5ifXgLB2J-xXCxpnJ8Qmo0RVZF

Here the shutter i set was 1/64  (i'm just playing for only test reasons), same as before it changed to 1/48, but i got a big horizontal line, very stylish tho...



https://drive.google.com/open?id=1HJuuYfkMawudk-Wdf8qdaiy8SNglVpEC

Can't wait for the vertical squeezed preset for achieving less cropping by increasing horizontal resolution  :D :D :D :D :D





Igor_Braun

  • New to the forum
  • *
  • Posts: 26
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #644 on: April 13, 2019, 09:06:48 PM »
Also, i wanted to say, that with this build i can't format the card from the camera.

reddeercity

  • Hero Member
  • *****
  • Posts: 1942
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #645 on: April 16, 2019, 07:45:23 AM »
Note, that not every clip looks like that. In average maybe 2 of 3 are okay. One of 3 has wrong blacklevel and very strange behavoir .......
Here the frame as MLV, you can use MLVFS to get dng. It is not perfectly focused, it was just a quick test, when I got this. But I think it doesn't matter for your anaysis.
https://www.dropbox.com/s/gwim47xsqdbo65z/M09-1858.MLV?dl=0

Thanks for the file ,
before I start to comment on this file , 
To All   :D  This is a big FYI
if ever liveview is messed up of not responding correctly (scrambled , lines or green cast)
You need to refresh liveview (this the cause of 99% of the crop_rec liveview issue reported here)
Press the PLAY Button twice (2 times , 1 to enter to play mode "view a file" 1 to exit)
this will bring you back to 3x3 (FHD) press 5x zoom to re-enter crop_rec.
the odd time you may have too repeat that process a couple a times .

Ok this file was kind of strange , all mlv->dng converters reported different things
It came down to having a corrupted frame , the only convertor that found it was mlvfs(quickmount) .
mlv producer , mlv_dump & mlv app didn't catch it , so this produce wrong white & black levels 118 1013
should be 112(black level) & 1023 (white level)

MLVFS(QuickMount)


MLV_Dump 10bit


The other thing I found to be a problem the shutter speed , the file report 1/40th and with
the preset , I have tuned it to 1/48th if you start at 1/64th at 29.97fps in 3x3 (FHD) .
I did find if you don't use 1/48th shutter this has cause strange liveview behavior .

One other thing that will cause corruption is having overlays enabled while recording raw video
It's far too much load on the cam .

So I don't know your setting but this is what I commend people use to be issue free .
Full manual setting (Important! No Auto setting)
3x3 (FHD) 1/64th 29.97 fps (this will change to 23.976fps @ 1/48th in crop_rec)
ISO 100,200,400 (always refresh liveview after any ISO change)
Do not enabled frame per second  override !!
Do not enable overlays in mlv_rec (default is disabled while recording raw video)
SRM (memory fill rate) "0" seem to be the best on Lexar 1066x cards (optional , shouldn't cause corruption)
Liveview preview , grey scale (B/W) low res.

this part is kind of off topic but anyways .....
The other strange thing was mlv app exported a total pink dng frame ,.



the png frame export was good .
I used ver. 1.6 64bit on windows7 pro  , thou I could see the frame in the preview window fine in the app.
when adjusted the black & white level (112-1023) & unchecked the "don't use camera color matrix " tab it exported a pretty good looking png frame


 

reddeercity

  • Hero Member
  • *****
  • Posts: 1942
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #646 on: April 16, 2019, 08:16:25 AM »
Updated the first page post with 2592x1200 test build link

Danne

  • Hero Member
  • *****
  • Posts: 5106
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #647 on: April 16, 2019, 09:40:21 AM »
Black level inconsistencies are most probable related to automation. This can be fixed since a1ex put in two options for settling black level. First one needs to know what and how regs were applied.
Seriously, I can't understand your logic with holding open source code that needs to be shared.

benoit

  • New to the forum
  • *
  • Posts: 26
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #648 on: April 16, 2019, 10:43:55 AM »
Seriously, I can't understand your logic with holding open source code that needs to be shared.
Me too. 6 months of personal coding on one preset with bugs that other people have already see on other plateforms ...
IMHO It's non-sense on an open source forum. >:(
Share is pure power, just think what will be Magic Lantern if a1ex post .mo files with as explanation "i'm busy just read my posts" ...

masc

  • Hero Member
  • *****
  • Posts: 963
Re: 3K/UHD 5D2 Raw development and Other Digic IV Cams
« Reply #649 on: April 16, 2019, 11:05:49 AM »
...
You need to refresh liveview (this the cause of 99% of the crop_rec liveview issue reported here)
Press the PLAY Button twice (2 times , 1 to enter to play mode "view a file" 1 to exit)
this will bring you back to 3x3 (FHD) press 5x zoom to re-enter crop_rec.
the odd time you may have too repeat that process a couple a times .
I did.
Ok this file was kind of strange , all mlv->dng converters reported different things
It came down to having a corrupted frame , the only convertor that found it was mlvfs(quickmount) .
mlv producer , mlv_dump & mlv app didn't catch it , so this produce wrong white & black levels 118 1013
should be 112(black level) & 1023 (white level)
Black level is different very often, mostly something between 110 and 118.
...
The other thing I found to be a problem the shutter speed , the file report 1/40th and with
the preset , I have tuned it to 1/48th if you start at 1/64th at 29.97fps in 3x3 (FHD) .
I did find if you don't use 1/48th shutter this has cause strange liveview behavior .
Okay...  I sometimes forget this, but I'll take care in future.
One other thing that will cause corruption is having overlays enabled while recording raw video
It's far too much load on the cam .
Hm... I just had this B/W realtime preview with nothing else shown when recording.
Full manual setting (Important! No Auto setting)
3x3 (FHD) 1/64th 29.97 fps (this will change to 23.976fps @ 1/48th in crop_rec)
ISO 100,200,400 (always refresh liveview after any ISO change)
Do not enabled frame per second  override !!
Do not enable overlays in mlv_rec (default is disabled while recording raw video)
SRM (memory fill rate) "0" seem to be the best on Lexar 1066x cards (optional , shouldn't cause corruption)
Liveview preview , grey scale (B/W) low res.
Shutter... okay... I sometimes forget about, but the other settings should be as you described.
The other strange thing was mlv app exported a total pink dng frame ,.
...
the png frame export was good .
I used ver. 1.6 64bit on windows7 pro  , thou I could see the frame in the preview window fine in the app.
Thanks. I found the bug. It was only when exporting the last frame in a MLV as DNG using "Single Frame Export". "Normal Export" (DNG) was working.
when adjusted the black & white level (112-1023) & unchecked the "don't use camera color matrix " tab it exported a pretty good looking png frame
For me it is still not really good looking. E.g. the blacks get still orange on top. Just the contrast and saturation is a bit lower.

What exactly was changed with the new build?
5D2.212 | EOSM.202