Danne's crop_rec_4k experiments for EOS M

Started by Danne, December 03, 2018, 06:10:17 PM

Previous topic - Next topic

0 Members and 3 Guests are viewing this topic.

2blackbar

chrispwilliams use mcm rewire mode, the one above it is the only one thats tricky to get focus pixels removed.

dfort

If there are any settings where the focus pixels can't be removed using my map files please upload a short MLV file and post on the Dealing with Focus Pixels in raw video topic.

CaseClosedTV

Hey guys i just wanted to thank everyone for working on these constant updates.

mangurian

I started with page one of this thread and have been going through a page at a time.
I am trying to see if the shutter bug has been fixed.
I will try to short circuit my search by just asking.  Is it fixed ?
also,
What is the general state of ML for the EOS-M as of today.

Many thanks,


Danne


kuchka94

Hi! I'm in your family now=)
Thank you for your work, that's amazing!

I have problem on all latest builds - I can't make photo!
Is this problem that called SHUTTER BUG?

On Nightly.2018Jul03 everything is ok with photo.

So am I right, that latest builds can be used only for video mode?
And no chance to use it in photo?

On this latest builds not working SD UHS module out of the box, seems like it doesn't load, I turned on it, but it doesn't appear in debug menu.
So i used ftom another builds. And on new builds i don't see script for loading sd module. So i took from older build)
Is it ok? Or something wrong on my side?

Just have my cam few days, sorry for novice questions...
Thank you for reply!

Danne

Please more info about not being able to take photos. Tried reinstall ml? Clear camera settings? Are you even in photo mode?
Sd_uhs don't need any script anymore. Enabling the module is enough.
To get out of the novice corner do read though the last ten to twenty pages of this thread. Most has been covered.

Danne

New version:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg208959#msg208959

- Possible fix for corrupted frames in anamorphic modes

For all "anamorphers" out there you may wanna test this version since it seems we can get corruptionless footage. Not that there was a lot of it but still.
Shortly. I am testing to short circuit the real time buffer while we are in framing preview. Since we are in framing it doesn´t matter anyway. I then free buffer again when pressing halfshutter to get back into real time framing.

Freeze with these two regs:
          EngDrvOutLV(0xc0f383d4, 0x4efffc);
  EngDrvOutLV(0xc0f383dc, 0x42401b2);

Then set them back to original numbers when going into real time again. I guess it works something like a "dynamically" hacked liveview setting.

Commit:
https://bitbucket.org/Dannephoto/magic-lantern/commits/bbdec706c04f2e9bf443d47d4590356d03c52b99

For now only works with preview set to framing and with anamorphic modes.

baladev

God bless your heart Danne for your relentless work on pushing forward and fixing/improving stuff. Without you and a few other people we would be in the dark ages in terms of ML development.

I will test your latest build in the next couple of days. This corrupted frames nonsense is time consuming to troubleshoot because of its random nature. Sometimes I get 10 corrupted frames and sometimes none in a clip of the same length.

Thank you!

DarkCAL

First of all I would like to thank Danne and the rest of the ML team for helping us all out !
I was and am reading a ton here on the forum specifically in this subforum and finally decided to become a member
Also I became the owner of a black EOS-M in the meantime
I'll post some pictures of my run & gun rig later on

@Danne,
I am running / testing ML May14 and ML May15
Just downloaded ML May 16 and gonna try it in a minute
What is actually the difference between these 3 versions please ?
ML May14 improved the pink frames a lot in all settings for me
Thanks
Dark

kuchka94

Quote from: Danne on May 15, 2019, 06:48:02 AM
Please more info about not being able to take photos. Tried reinstall ml? Clear camera settings? Are you even in photo mode?
About photo:
Tried to clear camera settings, reinstall ML, I'm in photo mode of course.
On latest builds camera can't took photos with kit lens 18-55I'm.
When I try to make photo, camera start try to make focus AF, after pressing shutter button - no reaction. Photo doesn't make.
I tried it in all focus modes, if, af, mf+af,  tried to change different settings and it didn't work.
When I enable shoot by taping on screen - camera seems try to make and stuck on that moment. Live View work, show real time picture, but camera stuck and didn't react on any buttons or screen taps.
After pulling out battery, I restarted camera, it shows that modules not loaded, and in this mode after few times suddenly pulling out battery, camera in some times can make photo as everything is ok. IMPORTANT - it happen not every time, and it very hard to catch that moment.
I thought, maybe problem in modules - disabled every module, but it didn't work. As at the start.
On fully manual lens photo start to work, and seems everything to be ok. I just turn on shutter without lens option, in C.Fn settings. But it don't work for kit lens.

On nightly 2018 Jul 03 everything works perfectly out of the box, after install, without any manipulation.

So, I think that maybe this is problem with working together with kit lens, but I don't understand why, what's a problem.

Video works ok on that lens.

dfort

@kuchka94 - Welcome to the infamous shutter-bug. This has been bugging the EOSM for years. It only affects EF-M lenses that have Image Stabilization and it only happens in still photo mode.

Current workarounds include:

  • Untwist lens to break the lens/body connection while camera is on and reseat the lens.
  • If your lens has the collapse feature, collapse and uncollapse the lens or make sure to start the camera with the lens collapsed.
  • Use a standard SD card, 2GB or lower capacity.
  • SanDisk Extreme Pro cards 32GB and under usually aren't affected by the bug.

baladev

Hmm.. the last couple of days I've been taking my M with me for walks around the bay area here, taking some short clips and an occasional photo in between. Didn't notice any strange behavior like that.
Apr 11 build
18-55 kit lens
SanDisk Extreme Pro 128Gb card

Could it be the card?

dfort

Quote from: baladev on May 17, 2019, 04:40:58 AM
Apr 11 build
18-55 kit lens
SanDisk Extreme Pro 128Gb card

Could it be the card?

I've got that card and it shows the shutter-bug. Try turning off all the modules, start camera in photo mode and shoot a still with that kit lens attached. Chances are the shutter won't fire.

Which April 11 build are you using? Maybe one of Danne's experiments squashed the bug?

kuchka94

Great :)
But do you know, why everything works fine on nightly 2018 Jul 03?
This bug was dated in 2014... Why it is now here in latest builds?

baladev

Quote from: dfort on May 17, 2019, 05:08:22 AM
I've got that card and it shows the shutter-bug. Try turning off all the modules, start camera in photo mode and shoot a still with that kit lens attached. Chances are the shutter won't fire.

Which April 11 build are you using? Maybe one of Danne's experiments squashed the bug?

Nope, can't reproduce. Disabled all modules, started in photo mode, takes pictures fine.

I'm on this at present:
crop_rec_4k_mlv_snd_isogain_1x3_presets_2019Apr11.EOSM202 by Danne

Danne

Quote from: dfort on May 17, 2019, 05:08:22 AM
I've got that card and it shows the shutter-bug. Try turning off all the modules, start camera in photo mode and shoot a still with that kit lens attached. Chances are the shutter won't fire.

Which April 11 build are you using? Maybe one of Danne's experiments squashed the bug?
I got the same set up and it´s not happening here. My colleague experienced the bug though a week ago, so it´s there alright. Not gonna open up that can of worms...

a1ex

Quote from: kuchka94 on May 17, 2019, 05:38:45 AM
This bug was dated in 2014... Why it is now here in latest builds?

Are you aware of how many man-years went into diagnosing just this specific bug (i.e. "that can of worms") ?

So far, the current hypothesis is that it's primarily caused by what happens before our code even has a chance to run (yes, before even loading autoexec.bin). If my theory is valid, the shutter bug happens if and only if the time required by Canon bootloader to mount the SD card and load autoexec.bin (details), is "too long" (whatever that means for Canon's firmware that runs on the MPU, in parallel with the main CPU).

Therefore...

Quote from: dfort on May 17, 2019, 05:08:22 AM
Maybe one of Danne's experiments squashed the bug?

Fixing it would require one of these, if my theory is valid:

- reprogramming the MPU (unfamiliar CPU architecture; unexplored territory; currently, the MPU is a black box for us)
- resetting lens communication somehow via MPU messages (good luck figuring out how exactly...)
- reprogramming (reflashing) Canon's bootloader (pretty extreme)

So... zero chances of getting this fixed (even by pure luck) with video-related experiments ;)




TLDR: the outcome (whether you are experiencing the shutter bug or not) depends on card contents, card speed, size, filesystem, autoexec.bin size, fragmentation... you get the idea. It does not depend on what build you are using. The only things under our control are:

- autoexec.bin size (reducing it to a few K - from roughly half of MB - is going to result in a *minor* improvement)
- card filesystem (FAT32 vs exFAT; this one is worth exploring, as a workaround; formatting with a large cluster size might help)

Danne

Quote from: a1ex on May 17, 2019, 07:19:04 AM
Are you aware of how many man-years went into diagnosing just this specific bug (i.e. "that can of worms") ?
Yes, my statement doesn´t adress any time span or what others did before. I simply realize it´s not within reach on a personal level. Respecrt for work done here all the way.

While you´re around. Could you share some thoughts on this:
https://www.magiclantern.fm/forum/index.php?topic=9741.msg216553#msg216553

What I notice is a constant bitrate when recording lossless while liveview is frozen. Meaning, even if pointing towards very bright areas bitrate seems to stay the same where as we know that too bright areas usually means early stops.

a1ex

That reply was not for you, but for the question in the quote ;)

QuoteWhat I notice is a constant bitrate when recording lossless while liveview is frozen. Meaning, even if pointing towards very bright areas bitrate seems to stay the same where as we know that too bright areas usually means early stops.

Is the resulting video valid, or it's encoding the same LiveView frame (or a small group of frames) over and over?

dfort

Got a few grey hairs myself trying to track down the shutter-bug. This bug is very easy to reproduce, I can't believe you're not seeing it. Any card over 32GB should show it as long as you are using a native EF-M lens with image stabilization -- not an EF/EF-S with the adapter.

I loaded up Danne's changeset a8ccb36 (April 11) on a SanDisk Extreme Pro 128GB card and -- sorry, shutter-bug is present. Didn't think it was going to work but needed to see for myself.

[EDIT] Tried this too:

Quote from: baladev on May 17, 2019, 04:40:58 AM
..taking some short clips and an occasional photo in between...

Switching from video to photo mode or shooting stills while in photo mode doesn't clear out the shutter-bug.

a1ex--I knew it was a long shot that the shutter-bug would disappear on this build but since Danne was doing some tweaking on the sd_uhs module I thought maybe it would do something with the card that would delay the boot process just enough--but no such luck.

Danne

Quote from: a1ex on May 17, 2019, 07:42:15 AM
That reply was not for you, but for the question in the quote ;)
My own mishap as usual then  8).

Back to the question. Files recorded are perfectly valid. Cancelling out(freezing) real time buffer while running framing mode has a signifcant impact on recordings. On 100D it went from plenty pink corruption party to zero pinkys.

Anyway. Going from dark to extreme bright does show a very slight increase in compression handling(50.6 to 51.3mb/s, 0.7mb/s neglible right?), check vid:
https://www.youtube.com/watch?v=AhKaFIwKtgA&feature=youtu.be


Well, I redid test without the liveview hack freeze and not really consistent. First darker input yields around 52mb and brighter slightly increased bitrate to around 53. So my hack is slightly reducing overhead but the effective part is by far reduction of corrupted frames. Whatever that means. SOmething with liveview running is causing the corruption. Some regs maybe?

baladev

Quote from: dfort on May 17, 2019, 07:47:02 AM
I can't believe you're not seeing it. Any card over 32GB should show it as long as you are using a native EF-M lens with image stabilization -- not an EF/EF-S with the adapter.

I loaded up Danne's changeset a8ccb36 (April 11) on a SanDisk Extreme Pro 128GB card and -- sorry, shutter-bug is present. Didn't think it was going to work but needed to see for myself.

[EDIT] Tried this too:

Switching from video to photo mode or shooting stills while in photo mode doesn't clear out the shutter-bug. Danne himself said it's working for him.

a1ex--I knew it was a long shot that the shutter-bug would disappear on this build but since Danne was doing some tweaking on the sd_uhs module I thought maybe it would do something with the card that would delay the boot process just enough--but no such luck.

Do you believe me it works the way I described or should I make a video? Configuration is exactly as I described above. Native EF-M kit lens used.

BTW I have 3 EOS M cameras, one is a Japanese version. The screen on it is a bit different (it has bluish anti-glare coating) maybe some internals are different too? Maybe there are slightly different versions of the camera for different markets? The one I use with ML atm is original Aussie variety.

niono

Hi,guys
I testing my lots of sd cards for my eos m, toshiba 64g sdxc is best,it can be wrating in 55mb/s,when i change to 14 bit,is recording few seconds then stoped.
  that is not point!,i find the inel m.2 have a sdio route,so i google it, woooo,look!!!!