Author Topic: Danne's crop_rec_4k experiments for EOS M (Read 416458 times)

gabriielangel · « **Reply #3525 on:** February 02, 2023, 11:36:19 PM »

Tested Feb 02 2023 Build:

Framing Preview:
The framing preview is now fast enough to track moving subjects (Like someone walking or a squirrel)

Dual ISO:
Dual ISO works best with 5k frtp up to 100/400. Fewer artifacts and no flickering.

With other presets (Especially 1080p, where I would not use it) artifacts are visible on very contrasting objects (Like a black on white address sign), and on very thin objects and lines.

5k anamorphic 1304x1636 exhibits bad pixels when using dual ISO

5k Ana FLV shows more flicker, but no visible aliasing artifacts.

Suggestion: For shadow recovery, it would be nice to have a way to lower the pivot point between highlights / shadows (in MLV App). Right now it is around 75% brightness which is in a very visible range. Maybe having it around 25-30% would bring the flicker down to a less noticeable region?

MLV app doesn't show dual ISO metadata, but all clips (except 2.8k sometimes) now show the dual iso stripes on frame 1.
The 2.8K preset recorded dual iso at frame 2 on 1 occasion, But the last frames are always ok now.
The 2.8k preset sometimes stays on preview and you can't go back to framing unless you switch presets.
2.8k gave me an "Unpatched error" once. I could not reproduce it.

It seems that MLV app is having a hard time with dual ISO and focus pixels maps (Some focus pixels are left behind) and with the fix bad pixels option ( which makes cyan, magenta and yellow pixels appear, where the non dual ISO recording won't.) See image below, the rainbow pixels only appear when switching on the "Fix Bad Pixels" option, regardless of the interpolation method.

image is from 1736x2180, the problem is a lot less visible in other modes.

iaburn · « **Reply #3526 on:** February 03, 2023, 12:30:41 AM »

Just tried the 2023-02-02 22:32 build and it changed the way I see hires modes

I would normally choose 2.5K frtp over 2.8K just because of the live view and 14bit continuos, but now the preview in 2.8K is so responsive, even while recording, that together with the seamless zoom for focus during recording makes me want to use it a lot more!
The same goes for 5K 1736x2180, can't wait for dawn to go trying outside <3
Thanks Danne!

iaburn · « **Reply #3527 on:** February 03, 2023, 03:27:47 PM »

@gabriielangel could you share an MLV file with a few frames from that video with the artifacts?

BatchGordon · « **Reply #3528 on:** February 03, 2023, 03:31:47 PM »

The latest changes appear to be a big improvement over the version from just a month ago!
The shot preview is pretty much usable in my opinion, so even with the highest resolutions I don't have to be blind frame anymore!
Danne, you did a great job once again!

I have other cameras, but I honestly find using Magic Lantern easier and more rewarding than other companies' native shooting functions.

Only one detail bothers me, and that's the shutter button. Isn't there a way to block the photo taking functionality?
Personally I only use the camera for video and sometimes it happens that I press the button too much while focusing, ruining the shot.
Does this happen to anyone else or am I the only wimp?

iaburn · « **Reply #3529 on:** February 03, 2023, 04:00:40 PM »

Quote from: BatchGordon on February 03, 2023, 03:31:47 PM

Only one detail bothers me, and that's the shutter button. Isn't there a way to block the photo taking functionality?
Personally I only use the camera for video and sometimes it happens that I press the button too much while focusing, ruining the shot.
Does this happen to anyone else or am I the only wimp?

I asked the exact same question a few days ago on the "MLV RAW ON EOS M" Facebook group! No answers so far...

Danne · « **Reply #3530 on:** February 03, 2023, 04:16:33 PM »

I was looking into this before and had a semiworking solution with unpress halfshutter but buggy as hell. If someone finds a good function here please let me know.

gabriielangel · « **Reply #3531 on:** February 03, 2023, 04:50:35 PM »

Quote from: BatchGordon on February 03, 2023, 03:31:47 PM

Personally I only use the camera for video and sometimes it happens that I press the button too much while focusing, ruining the shot.
Does this happen to anyone else or am I the only wimp?

You can try using the * (Star Button) to focus instead. The shutter button behaves a lot better than last year, where it were a lot more sensitive, but I also press it too far every now and then.

Try using the method Danne outlined here: https://www.magiclantern.fm/forum/index.php?topic=25781.msg241458#msg241458

BatchGordon · « **Reply #3532 on:** February 03, 2023, 05:22:46 PM »

Nice to know I am not alone!

Quote from: gabriielangel on February 03, 2023, 04:50:35 PM

The shutter button behaves a lot better than last year, where it were a lot more sensitive, but I also press it too far every now and then.

Yes, it definitely behaves alot better. I will try your suggestions this weekend, thank you very much!

@Danne
As soon as I get some free time I'll starting looking too, but I don't have big hopes since you know the code so much better than anyone else!

Zag · « **Reply #3533 on:** February 03, 2023, 06:18:43 PM »

For some reason this version doesn't work on my Canon eos-m. When i install it it just freezes the camera after restart. Older version from january works fine tho.

Walter Schulz · « **Reply #3534 on:** February 03, 2023, 08:33:50 PM »

Latest changes to Danne's builds will not work with all cards anymore. Which one are you using?

Danne · « **Reply #3535 on:** February 03, 2023, 10:35:44 PM »

I am enabling 240Mhz blindly upon install. Should be possible to remove sd_uhs.mo before install if we suspect card issues.
Also format card to fat32 if on later mac os systems.

iaburn · « **Reply #3536 on:** February 03, 2023, 11:17:59 PM »

Danne since the improvements on dual ISO, files have a "0" on dualMode and isoValue. Any idea why? Not a big deal, but it's nice when MLVApp detects it

Code: [Select]

typedef struct {
    uint8_t     blockType[4];    /* Dual-ISO information */
    uint32_t    blockSize;
    uint64_t    timestamp;
    uint32_t    dualMode;    /* bitmask: 0=off, 1=odd lines, 2=even lines, upper bits may be defined later */
    uint32_t    isoValue;
}  mlv_diso_hdr_t;

Danne · « **Reply #3537 on:** February 03, 2023, 11:41:16 PM »

I have no reference file or source to what it should look like to be able and autodetect in Mlv App but if it´s not working properly let´s fix it. You reference to mlv.h and here is what to check in dual_iso.c:

Code: [Select]

/* callback routine for mlv_rec to add a custom DISO block after recording started (which already was specified in mlv.h in definition phase) */
static void isoless_mlv_rec_cbr (uint32_t event, void *ctx, mlv_hdr_t *hdr)
{
    /* construct a free-able pointer to later pass it to mlv_rec_queue_block */
    mlv_diso_hdr_t *dual_iso_block = malloc(sizeof(mlv_diso_hdr_t));
    
    /* set the correct type and size */
    mlv_set_type((mlv_hdr_t *)dual_iso_block, "DISO");
    dual_iso_block->blockSize = sizeof(mlv_diso_hdr_t);
    
    /* and fill with data */
    dual_iso_block->dualMode = dual_iso_is_active();
    dual_iso_block->isoValue = isoless_recovery_iso;
    
    /* finally pass it to mlv_rec which will free the block when it has been processed */
    mlv_rec_queue_block((mlv_hdr_t *)dual_iso_block);
}

I tested dual iso prior to my fixes in the code but metadata was not in there to autodetect in Mlv App. Do you have any reference file which autodetects?

EDIT: Mode 1 is on when I take out my fixes. So some race condition. Let´s have a look.

iaburn · « **Reply #3538 on:** February 04, 2023, 12:14:08 AM »

Sure, in this video the structure looks like this after parsing:

Code: [Select]

DISO	@0x25156def9cb	mlv_diso_hdr_t
		blockSize	24		uint32_t
		blockType	"DISO"	uint8_t[4]
		dualMode	1		uint32_t
		isoValue	4		uint32_t
		timestamp	17445940	uint64_t

Danne · « **Reply #3539 on:** February 04, 2023, 12:48:35 AM »

WHat build was used? Code source? I probably fkd up my dualiso stuff somehow

Danne · « **Reply #3540 on:** February 04, 2023, 12:56:38 AM »

Should be fixed now. Uploaded a new build.
Still some race condition with 2.8k where the first 2 frames are non dualiso. Need to check into that.
Anyway. Should Mlv App autoenable dualiso when mode 1 flag is on?

Danne · « **Reply #3541 on:** February 04, 2023, 02:02:18 AM »

New build:
To fix dualiso when using framing prwview it will apply dualiso as it was working before showing scanned lines. Real time preview will choose base iso when not recording. This should work with all metadata and all frames will be correctly recorded.

EDIT: More fixes. Previewing in x10 zoom and when using halfshutter if in framing will show base iso.

gabriielangel · « **Reply #3542 on:** February 04, 2023, 07:51:04 AM »

Tested 2023Feb03 and 04 version 1 build:

Card: Sandisk Ultra SD 200 MB/s 256GB

Max Recording Time

Interesting, when using my ef-m 32mm f1.4, The 2.8k 12bits and 2.5k 14 bits took a performance hit.

Example:

2.8k/12 Feb 04 2023: 7-9 seconds (Stopped by itself)
2.8k/12 Feb 03 2023: 7-9 seconds (Stopped by itself)
2.8k/12 Jan 13 2023: 43-54 Seconds (Stopped by itself)

I noticed the shorter recording duration, so I recorded, Installed a different build on the card, restarted the camera twice, used identical settings, record. Did the same thing to move back to the latest build, then again to an older build.

I shot a chart lit by a LED panel, with the tripod. So there are no variables other than the change of build.
(Details in the screenshot)

When I switched to the EF 35mm F2, I got 1 minute+ like before...

Dual ISO

The metadata shows as expected and the ISO stripes show on the 1st frame.
2.8k Dual ISO Clips are ok from 1st to last frame.
No need to force Dual ISO anymore, you just have to set it to "On"
The focus dots / Bad pixels issue I reported earlier is not as bad, but it is still there:

With a Single ISO Clip: Works as expected

With a Dual ISO clip: Enabling fix focus dots will make pixels appear (Less pronounced with 5k frtp).
Using Fix Bad Pixels leads to some pretty nasty artifacts.

I linked 3 clips at the end of the post.

Sample MLV (Will remove after 1 week, drive needs cleaning): http://bit.ly/3HxcSs7

iaburn · « **Reply #3543 on:** February 04, 2023, 09:26:43 AM »

Hi, just tried the build from 2023-02-04 06:27, things that I've noticed:
- Dual ISO is now detected by MLVApp!

- Didn't noticed a significant drop (something that I can notice) in recording times on my side.
- In Framing preview (tested in 2.8K) once you activate dual ISO, it stays on even after switching it off. I can make it back to normal by switching to zoom focus view (real time) a few times (another race condition?)
- In 2.8K sometimes it switches to Real time preview and there is no way to go back to Framing. Not pressing the button, not going to the menu and back... just switch off and on works. I cannot replicate this every time, so it could be something that I'm doing and I shouldn't do

iaburn · « **Reply #3544 on:** February 04, 2023, 10:22:05 AM »

Quote from: gabriielangel on February 04, 2023, 07:51:04 AM

The focus dots / Bad pixels issue I reported earlier is not as bad, but it is still there:

With a Single ISO Clip: Works as expected

With a Dual ISO clip: Enabling fix focus dots will make pixels appear (Less pronounced with 5k frtp).
Using Fix Bad Pixels leads to some pretty nasty artifacts.

Thanks for the samples, very good tests for debugging the focus pixels issue

I tried moving the pixel fixing stuff after the Dual ISO processing and doing the fixing ignoring that it's a Dual ISO video works very well, looks almost identical to the non-dual ISO:
single:

dual:

I just need to find out why once I switch fix focus pixels OFF, cannot turn it ON again

Let's discuss this on the MLVApp thread if you don't mind, it's not related to Danne's builds

Danne · « **Reply #3545 on:** February 04, 2023, 11:04:51 AM »

Quote from: iaburn on February 04, 2023, 09:26:43 AM

Hi, just tried the build from 2023-02-04 06:27, things that I've noticed:
- Dual ISO is now detected by MLVApp!
- Didn't noticed a significant drop (something that I can notice) in recording times on my side.
- In Framing preview (tested in 2.8K) once you activate dual ISO, it stays on even after switching it off. I can make it back to normal by switching to zoom focus view (real time) a few times (another race condition?)
- In 2.8K sometimes it switches to Real time preview and there is no way to go back to Framing. Not pressing the button, not going to the menu and back... just switch off and on works. I cannot replicate this every time, so it could be something that I'm doing and I shouldn't do

Thanks. Reproduced the issues. Will fix maybe this evening 👍.

gabriielangel · « **Reply #3546 on:** February 04, 2023, 02:42:33 PM »

Quote from: iaburn on February 04, 2023, 10:22:05 AM

I just need to find out why once I switch fix focus pixels OFF, cannot turn it ON again Let's discuss this on the MLVApp thread if you don't mind, it's not related to Danne's builds

I noticed that with Dual ISO Clips, I need to be patient

Try switching to another clip, moving to a different spot in the file and wait 5-15 seconds after making a change (Focus Dots / Pixel Stuff).
The change will eventually appear.

Danne · « **Reply #3547 on:** February 04, 2023, 04:22:56 PM »

Turning off dualiso issue fixed. New build uploaded.

iaburn · « **Reply #3548 on:** February 04, 2023, 05:00:08 PM »

Quote from: Danne on February 04, 2023, 04:22:56 PM

Turning off dualiso issue fixed. New build uploaded.

Rock solid now, I couldn't break it

It records non-stop at 12 bits on 2.8K and 5K 1736x with pretty fast preview!

gabriielangel · « **Reply #3549 on:** February 04, 2023, 11:45:09 PM »

Tested Build Feb 04 2023 4:18 PM

-Presets 1080p, 5k frtp and 2k register all clips as Dual ISO, even when Dual ISO isn't engaged. The clips record properly though.

-The Bulk of the flicker issue is gone. I recorded with the same settings as yesterday and there is slight flickering with the 5K Ana and 5k Ana flv presets, but the other presets are fine.

1080p could cause issues when handheld though (aliasing+Dual Iso).
See video file below. Pay attention to the eyes and neck of the Blonde Haired Doll (shiny plastic). I forgot to add more reflective surfaces in the frame to make that case more obvious.

There is also a download of 2 MLV files from yesterday and 2 recorded with today's build, for reference.

-The issue with shorter recording times at 2.8 and 2.5k / 14bits and the 32mm f1.4 lens remains. I will try all my lenses later this week to see what's going on there.

-The 2.8k and 2.5k presets won't stay stuck in preview mode anymore. Seems to be fixed.

Here is a video file where I put every preset through a worst-case Single vs Dual ISO test (I don't think anyone would underexpose that much just to save the sky!).
You can see how much improvement in DR/noise can be achieved, especially when recording at 10 bits.
Dual ISO can be very helpful to "Reach into dark corners" when it's working well.

http://bit.ly/3le61ww

MLV Files which exhibit some flicker: https://bit.ly/3jBxozR

News:

Author Topic: Danne's crop_rec_4k experiments for EOS M (Read 416458 times)