SDcard doubts and "hot" pixels appearing only when recording raw video on EOS-M

Started by ricardopt, August 04, 2018, 02:38:09 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

ricardopt

Hi everyone!

Since im on holidays i decided to test the "new" capabilities of the EOS-M specially the mv1080 mode, i read that i need a fast card, one of the recomended cards is the Sandisk extreme pro:
https://www.amazon.com/Tarjeta-Memoria-SanDisk-Extreme-hasta/dp/B01J5RHBQ

but it seems its only available in the normal sd card size, i wont be able to use it in action cams and other devices that use micro sd cards, i found one micro sd card from the same brand, it seems it has the same caracteristics but its sdxc instead of sdhc:
https://www.amazon.com/SanDisk-Extreme-microSDXC-Memory-Adapter/dp/B06XYT37NK

Will the micro sd card work on the EOS-M?

What micro sd card do you recomend?

Also when trying out the new builds i've encountered a problem with red/pink/white/ pixels apperaing on the lcd screen/footage when using raw video... if i disable raw video the pixels dont appear neither in the lcd screen or footage.

can anyone help me understand where this pixels problems come from? only appear when using raw video, this problem only appeared after trying the july build from "here", i loaded the version i was using before but the problem doesnt go away.

thanks to anyone that could help


dfort

Quote from: ricardopt on August 04, 2018, 02:38:09 AM
can anyone help me understand where this pixels problems come from?

You mean like this?

https://www.magiclantern.fm/forum/index.php?topic=16054.0

You didn't say how you are processing your raw video. The newer MLV processing apps can handle this problem without the user having to think about it.

ricardopt

Hi Dfort, no its not like that, i dont use auto focus, it's the same "hot" pixels that appear when using long exposure...for example im recording a video and if i cover the lens for 2 seconds those red/black pixels appear on the lcd screen/footage, if i disable raw video nothing happens.

i tried some builds and i reseted the eos-m before using the build i normally used, thats when i saw it happening, i must have some option turned on/off that shouldn't be, i never had this problem before.

im using the mlvproducer latest alpha and the latest mlvapp.

i will try o record some footage so that you can see the problem, at first sight it reminds me of long exposure hot pixels, one thing i noticed is that altough i have auto focus off i hear the lens making noise but i dont see them moving.

thanks for any help...will post a video later

Levas

Souns like they are indeed hot pixels.
Which many MLV processing apps have an option for, fix hot pixels.

That you can't see them withou raw video is because Canon standard movie mode isn't in raw format, Canon fixes the hot pixels in the proces.

That hot pixels appear in raw video is very standard when using high iso's.
On the 6d going higher than iso 6400 is most certainly gonna give hot pixels.

What helps in reducing the amount of visible hot pixels is doing hot pixel remapping, google for it: Canon hot pixel remapping.
It is done by performing the sensor cleaning in Canon menu, while you don't have a lens on your camera but the body cap. Before doing this, you could warm up the sensor a little by using liveview with iso 6400 for a minute or so.

ricardopt

Thanks for the input, while recording some test footage that i will upload in a bit i found out that by disabling 3x3 720p in the magic lantern menu the red pixels disappear on the lcd screen, i cant stop noticing the lens noise, it didnt do that before. while googling i found out that there's a setting in the customs functions called shutter lock/af/ae lock, since i reseted my eos-m i lost all the configuration, i remember setting something in that menu (custom functions) but cant remember what it was...

ricardopt

here's the test video, first clip in normal h264/cinestyle profile, second clip recorded in raw (1632x964- 3x3 720p) stretched to 1920x1080, i used mlvproducer and mlvapp but enabling the removal of hot/focus pixels doesnt work. In case it helps before i reseted the eos-m i never had this problem



here's some videos i recorde in raw before without this problem:
https://www.magiclantern.fm/forum/index.php?topic=22464.0

ricardopt

Quote from: Levas on August 04, 2018, 12:29:12 PM
What helps in reducing the amount of visible hot pixels is doing hot pixel remapping, google for it: Canon hot pixel remapping.
It is done by performing the sensor cleaning in Canon menu, while you don't have a lens on your camera but the body cap. Before doing this, you could warm up the sensor a little by using liveview with iso 6400 for a minute or so.

there's no option for manual sensor cleaning on the EOS-M

ricardopt

can anyone help? Should i ask in a more suitable/specific thread?

Thanks

Levas

Looks indeed like stuck hot pixels, so hot pixel remapping is probably gonna work.
For hot pixelremapping you need to manually 'activate' sensor cleaning from the Canon menu, which is available on the EOS-M
(Manual sensor cleaning, moves the mirror up in DSLR's so you can see the sensor and clean it physically, no mirror in the EOS-M, so no need for that option.)

So go find your body cap, take your lens of the Eos-M, put the body cap on your Eos-m, go to video mode and select iso 3200 or 6400, leave video mode on for a minute or so to be sure hot pixels are indeed hot and visible.
And then switch to photo mode, without liveview and select automatic sensor cleaning from Canon menu, like this:
http://support-th.canon-asia.com/contents/TH/EN/8202643800.html

After that, shutdown the camera and restart it and see if those hot pixels still pop up in your raw video.

EDIT: after googling some more, and seeing there is indeed a manually cleaning option on the eos-m:
It seems that pixel remapping is done this way:
You need to select the manually cleaning option, with a body cap on the camera, and wait for at least 30 seconds


From the internet:
Canon Hot(dead) pixel fix:

Step 1 - Detach the lens from your Canon camera
Step 2 - Put on the camera cap (do not leave your camera open).
Step 3 - Turn your camera on. Go to the menu and select "Sensor cleaning" under the utilities menu.
Step 4 - Turn the dial to select "Clean manually" under the sensor cleaning option.
Step 5 - Turn the dial to select "OK" and then press the "Set" button. The mirror will lock up and the shutter will open. Allow the shutter to remain open for at least 30 seconds. In your case I'd leave it sitting like that for 60 seconds.
Step 6 - Turn your camera off and reattach the lens.

Youtube video:
https://www.youtube.com/watch?feature=player_embedded&v=WJBuGhMnvFo

Danne

 If they still don't vanish you can build a pixel map list and use it in Switch or batch_mlv...

dfort

@ricardopt - What would really help is if you could upload a short MLV file for us to examine. Since these are also showing up in H.264 they are not focus pixels.

I agree with Levas, you should first try the manual sensor cleaning which is available on the EOSM:





If that doesn't work you might have to physically clean the sensor. There are several cleaning kits available. Try just the air blower first, it might be all that's needed.



There is also a way to map out those pixels using the Canon DPP software. As you can see I have never tried this myself.



There are several easy to find tutorials, here's one from the Canon USA website:

http://learn.usa.canon.com/resources/blogs/2013/20131105_stoner_dustdelete_blog.shtml

Finally if none of the above options works you can map out the problem pixels as Danne suggested. In fact the first experiments I did mapping out the focus pixels is using a feature in dcraw that was designed to hide so called "bad" pixels. How to do this varies depending on the MLV processing app you're using.

Quote from: ricardopt on August 04, 2018, 08:35:44 AM
...i dont use auto focus...

I believe that comment is about the focus pixels, they show up whether or not you use auto focus.

Quote from: ricardopt on August 04, 2018, 12:53:13 PM
...i cant stop noticing the lens noise, it didnt do that before. while googling i found out that there's a setting in the customs functions called shutter lock/af/ae lock, since i reseted my eos-m i lost all the configuration, i remember setting something in that menu (custom functions) but cant remember what it was...

This setting allows you to focus using the '*' button instead of the half-shutter press. By the way you need this set when shooting silent stills:



The noise you're hearing might be either the camera autofocusing while shooting movies:



Or maybe it is the image stabilization? EF and EF-S lenses have a switch on the lens but EF-M lenses need to be switched on/off through the Canon menu:



I prefer to keep this feature turned on. None of my lenses make enough noise to bother me when IS is turned on.

Quote from: ricardopt on August 04, 2018, 02:38:09 AM
Will the micro sd card work on the EOS-M?

What micro sd card do you recomend?

I don't use Micro SD cards unless the hardware requires it because they are just too hard to fiddle with in the field but my understanding is that the Micro sized cards perform the same as the standard ones. The adapter is just a straight connection so it shouldn't slow down the performance. I can't give you a recommendation on something I haven't tried myself but the Sandisk Extreme Pro cards have worked fine for me. Look for the ones with the fastest write speed. The EOSM has a UHS-I controller so there's no point in paying extra for a UHS-II card.

ricardopt

Quote from: Levas on August 05, 2018, 02:14:55 PM
Step 1 - Detach the lens from your Canon camera
Step 2 - Put on the camera cap (do not leave your camera open).
Step 3 - Turn your camera on. Go to the menu and select "Sensor cleaning" under the utilities menu.
Step 4 - Turn the dial to select "Clean manually" under the sensor cleaning option.
Step 5 - Turn the dial to select "OK" and then press the "Set" button. The mirror will lock up and the shutter will open. Allow the shutter to remain open for at least 30 seconds. In your case I'd leave it sitting like that for 60 seconds.
Step 6 - Turn your camera off and reattach the lens.

i dont have the eos-m camera cap anynore, but i used a lens cap and holded it against the eos-m body while following "your" guide, the menu only shows "clean automatically" or clean now, using the dial or other keys doesnt show "clean manually" but i think that might be because the camera is not detecting anything attached to it, i hear the shutter noise when selecting "clean now" and but it doesnt work, red pixels show up on lcd like before in the same position, ill order a cap to test it properly


Quote from: Danne on August 05, 2018, 02:42:42 PM
If they still don't vanish you can build a pixel map list and use it in Switch or batch_mlv...
will try that after cleaning the sensor with a "proper" cap attached to the eos-m



Quote from: dfort on August 05, 2018, 04:22:24 PM
@ricardopt - What would really help is if you could upload a short MLV file for us to examine. Since these are also showing up in H.264 they are not focus pixels.

the dots dont appear on footage using normal/h264 video recording...thanks for your suggestions and the sdcard advice aswell, i will upload 2 samples...mlv and h264...so that you can have alook if you have the time

Thanks a lot to everyone

ricardopt

Hi everyone
here's 2 files, mlv and normal h264, the h264 looks clean no hot pixels, the mlv one has red and a black pixel, the black pixel can be seen around the door handle region and the red above the handle, i covered the lens so that you can follow the red pixels in to the normal video after i stop covering the lens...

sometimes the pixels show up more than others on mlv files but the h264 video is always free of them

h264 file;
https://www.dropbox.com/s/svh9u3h4vktm1ko/MVI_2274.MOV?dl=0

mlv file
https://www.dropbox.com/s/yv1vv9mhzdudu9x/M05-1825.MLV?dl=0

thanks for helping

ricardopt

with mlv app black pixel disapears red pixels are barelly seen
with mlvproducer both are seen

Danne

Very hard to see the pixels. Try a cs 2x2 (chroma smooth) or even add a darkframe.

ricardopt

Hi Danne, thanks for your insight on this, i'm using a 40 inch monitor perhaps i spot them quicker because of that,  i've reached a few conclusions, they might seem obvious to some more experienced users:

1: the red pixels appear more often when high iso is selected by the camera, the automatic video mode was selecting 6400 ISO on dark scenes, hot pixels are connected to high iso so i made a few tests starting from 100 to 6400 and the intensity of the red pixels increased as the iso was higher.

2: before doing the sensor clean without lens on it red pixels were visible on both mlvapp/mlvproducer both with hot pixels option turned on,

3: after the sensor clean mlv app removes the red pixels if i select 3x3 chroma smooth

4:this whole red pixels thing only appeared after i tried some newer magic lantern builds, both mlv processing apps worked perfectly, whatever caused this it taught me to never "mess" with a winning "combination/team" if it works dont mess with it

i personally prefered the look/output of the mlv producer app, i'll have to spend some time learning mlvapp options

thanks everyone for your invaluable help!
Greetings from Portugal

Danne

Hot pixels will come and go with higher isos. Has nothing todo with ml build versions.
If shooting at higher isos a lot I recommend including a darkframe to fix the issue.

dfort

I see focus pixels! (For me this is scarier than, "I see dead people")

So what is this?

Block: RAWI
...
    Res:  1632x694
    raw_info:
...
      height           727
      width            1808
...
Block: RAWC
...
      sampling        3x3 (read 1 line, skip 2, bin 3 columns)


Looks like the crop_rec module was used on the EOSM. I tried mlv_dump and MLVFS and they both showed focus pixels but Danne's Switch app handled it perfectly. Need to dig into this further to figure out what is going on.

ricardopt

Hi, can i do something to help? Am i not using magic lantern correctly? i use the following build "magiclantern-crop_rec_4k.2017Dec19.EOSM202", i activate croprec and mlv lite modules, restart, activate raw video, activate crop rec (3x3 720). i use the 22mm lens.

i have some mlv's that i can share from before and after trying new builds if it helps

ricardopt

here's 2 samples from "before" the pixels started to show, if i load the mlv's into mlv producer, i only check "false color" and "alias filter", remove focus pixels is enabled by default, after that i export to prores, i find the "default colour scheme" from mlvproducer a lot more pleasing to the eye, i dont need to do a lot of color correction after.

if i load the mlv's into mlvapp, i dont need to check any options but the default color scheme it presents needs a lot of tweaking, too "blown out", i know raw means color tweaking but its easier with mlvproducer.

here's the 2 files

https://www.dropbox.com/s/e2bfzluequn5psb/M04-1815.MLV?dl=0

https://www.dropbox.com/s/6ym2fonsz99cu1n/M04-1947.MLV?dl=0

MLVProducer:  3354 alpha
MLVapp: 0.17 alpha 64bit

hope it helps