Danne's crop_rec_4k experiments for EOS M

Started by Danne, December 03, 2018, 06:10:17 PM

Previous topic - Next topic

0 Members and 3 Guests are viewing this topic.

gabriielangel

Tested Feb 02 2023 Build:

Framing Preview:
The framing preview is now fast enough to track moving subjects (Like someone walking or a squirrel)

Dual ISO:
Dual ISO works best with 5k frtp up to 100/400. Fewer artifacts and no flickering.

With other presets (Especially 1080p, where I would not use it) artifacts are visible on very contrasting objects (Like a black on white address sign), and on very thin objects and lines.

5k anamorphic 1304x1636 exhibits bad pixels when using dual ISO

5k Ana FLV shows more flicker, but no visible aliasing artifacts.

Suggestion: For shadow recovery, it would be nice to have a way to lower the pivot point between highlights / shadows (in MLV App). Right now it is around 75% brightness which is in a very visible range. Maybe having it around 25-30% would bring the flicker down to a less noticeable region?

MLV app doesn't show dual ISO metadata, but all clips (except 2.8k sometimes) now show the dual iso stripes on frame 1.
The 2.8K preset recorded dual iso at frame 2 on 1 occasion, But the last frames are always ok now.
The 2.8k preset sometimes stays on preview and you can't go back to framing unless you switch presets.
2.8k gave me an "Unpatched error" once. I could not reproduce it.

It seems that MLV app is having a hard time with dual ISO and focus pixels maps (Some focus pixels are left behind) and with the fix bad pixels option ( which makes cyan, magenta  and yellow pixels appear, where the non dual ISO recording won't.) See image below, the rainbow pixels only appear when switching on the "Fix Bad Pixels" option, regardless of the interpolation method.

Fix-Bad-Pixels" border="0

image is from 1736x2180, the problem is a lot less visible in other modes.


iaburn

Just tried the 2023-02-02 22:32 build and it changed the way I see hires modes  :o
I would normally choose 2.5K frtp over 2.8K just because of the live view and 14bit continuos, but now the preview in 2.8K is so responsive, even while recording, that together with the seamless zoom for focus during recording makes me want to use it a lot more!
The same goes for 5K 1736x2180, can't wait for dawn to go trying outside <3
Thanks Danne!

iaburn

@gabriielangel could you share an MLV file with a few frames from that video with the artifacts?

BatchGordon

The latest changes appear to be a big improvement over the version from just a month ago!
The shot preview is pretty much usable in my opinion, so even with the highest resolutions I don't have to be blind frame anymore!
Danne, you did a great job once again!  ;)

I have other cameras, but I honestly find using Magic Lantern easier and more rewarding than other companies' native shooting functions.  :)

Only one detail bothers me, and that's the shutter button. Isn't there a way to block the photo taking functionality?
Personally I only use the camera for video and sometimes it happens that I press the button too much while focusing, ruining the shot.
Does this happen to anyone else or am I the only wimp?  :P


iaburn

Quote from: BatchGordon on February 03, 2023, 03:31:47 PM
Only one detail bothers me, and that's the shutter button. Isn't there a way to block the photo taking functionality?
Personally I only use the camera for video and sometimes it happens that I press the button too much while focusing, ruining the shot.
Does this happen to anyone else or am I the only wimp?  :P

I asked the exact same question a few days ago on the "MLV RAW ON EOS M" Facebook group! No answers so far...

Danne

I was looking into this before and had a semiworking solution with unpress halfshutter but buggy as hell. If someone finds a good function here please let me know.

gabriielangel

Quote from: BatchGordon on February 03, 2023, 03:31:47 PM
Personally I only use the camera for video and sometimes it happens that I press the button too much while focusing, ruining the shot.
Does this happen to anyone else or am I the only wimp?  :P

You can try using the * (Star Button) to focus instead. The shutter button behaves a lot better than last year, where it were a lot more sensitive, but I also press it too far every now and then.

Try using the method Danne outlined here: https://www.magiclantern.fm/forum/index.php?topic=25781.msg241458#msg241458


BatchGordon

Nice to know I am not alone!

Quote from: gabriielangel on February 03, 2023, 04:50:35 PM
The shutter button behaves a lot better than last year, where it were a lot more sensitive, but I also press it too far every now and then.

Yes, it definitely behaves alot better. I will try your suggestions this weekend, thank you very much!

@Danne
As soon as I get some free time I'll starting looking too, but I don't have big hopes since you know the code so much better than anyone else!

Zag

For some reason this version doesn't work on my Canon eos-m. When i install it it just freezes the camera after restart. Older version from january works fine tho.

Walter Schulz

Latest changes to Danne's builds will not work with all cards anymore. Which one are you using?

Danne

I am enabling 240Mhz blindly upon install. Should be possible to remove sd_uhs.mo before install if we suspect card issues.
Also format card to fat32 if on later mac os systems.

iaburn

Danne since the improvements on dual ISO, files have a "0" on dualMode and isoValue. Any idea why? Not a big deal, but it's nice when MLVApp detects it

typedef struct {
    uint8_t     blockType[4];    /* Dual-ISO information */
    uint32_t    blockSize;
    uint64_t    timestamp;
    uint32_t    dualMode;    /* bitmask: 0=off, 1=odd lines, 2=even lines, upper bits may be defined later */
    uint32_t    isoValue;
}  mlv_diso_hdr_t;

Danne

I have no reference file or source to what it should look like to be able and autodetect in Mlv App but if it´s not working properly let´s fix it. You reference to mlv.h and here is what to check in dual_iso.c:
/* callback routine for mlv_rec to add a custom DISO block after recording started (which already was specified in mlv.h in definition phase) */
static void isoless_mlv_rec_cbr (uint32_t event, void *ctx, mlv_hdr_t *hdr)
{
    /* construct a free-able pointer to later pass it to mlv_rec_queue_block */
    mlv_diso_hdr_t *dual_iso_block = malloc(sizeof(mlv_diso_hdr_t));
   
    /* set the correct type and size */
    mlv_set_type((mlv_hdr_t *)dual_iso_block, "DISO");
    dual_iso_block->blockSize = sizeof(mlv_diso_hdr_t);
   
    /* and fill with data */
    dual_iso_block->dualMode = dual_iso_is_active();
    dual_iso_block->isoValue = isoless_recovery_iso;
   
    /* finally pass it to mlv_rec which will free the block when it has been processed */
    mlv_rec_queue_block((mlv_hdr_t *)dual_iso_block);
}

I tested dual iso prior to my fixes in the code but metadata was not in there to autodetect in Mlv App. Do you have any reference file which autodetects?

EDIT: Mode 1 is on when I take out my fixes. So some race condition. Let´s have a look.

iaburn

Sure, in this video the structure looks like this after parsing:

DISO @0x25156def9cb mlv_diso_hdr_t
blockSize 24 uint32_t
blockType "DISO" uint8_t[4]
dualMode 1 uint32_t
isoValue 4 uint32_t
timestamp 17445940 uint64_t



Danne

WHat build was used? Code source? I probably fkd up my dualiso stuff somehow :)

Danne

Should be fixed now. Uploaded a new build.
Still some race condition with 2.8k where the first 2 frames are non dualiso. Need to check into that.
Anyway. Should Mlv App autoenable dualiso when mode 1 flag is on?

Danne

New build:
To fix dualiso when using framing prwview it will apply dualiso as it was working before showing scanned lines. Real time preview will choose base iso when not recording. This should work with all metadata and all frames will be correctly recorded.

EDIT: More fixes. Previewing in x10 zoom and when using halfshutter if in framing will show base iso.

gabriielangel

Tested 2023Feb03 and 04 version 1 build:

Card: Sandisk Ultra SD 200 MB/s 256GB

Max Recording Time

Interesting, when using my ef-m 32mm f1.4, The 2.8k 12bits and 2.5k 14 bits took a performance hit.

Example:

2.8k/12 Feb 04 2023: 7-9 seconds (Stopped by itself)
2.8k/12 Feb 03 2023: 7-9 seconds (Stopped by itself)
2.8k/12 Jan 13 2023: 43-54 Seconds (Stopped by itself)

I noticed the shorter recording duration, so I recorded, Installed a different build on the card, restarted the camera twice, used identical settings, record. Did the same thing to move back to the latest build, then again to an older build.

I shot a chart lit by a LED panel, with the tripod. So there are no variables other than the change of build.
(Details in the screenshot)

When I switched to the  EF 35mm F2, I got 1 minute+ like before...

Dual ISO

The metadata shows as expected and the ISO stripes show on the 1st frame.
2.8k Dual ISO Clips are ok from 1st to last frame.
No need to force Dual ISO anymore, you just have to set it to "On"
The focus dots / Bad pixels issue I reported earlier is not as bad, but it is still there:

With a Single ISO Clip: Works as expected

With a Dual ISO clip: Enabling fix focus dots will make pixels appear (Less pronounced with 5k frtp).
Using Fix Bad Pixels leads to some pretty nasty artifacts.



I linked 3 clips at the end of the post.

Dual-ISO-Test-Jan-04-2023" border="0

Sample MLV (Will remove after 1 week, drive needs cleaning): http://bit.ly/3HxcSs7


iaburn

Hi, just tried the build from 2023-02-04 06:27, things that I've noticed:
- Dual ISO is now detected by MLVApp! :)
- Didn't noticed a significant drop (something that I can notice) in recording times on my side.
- In Framing preview (tested in 2.8K) once you activate dual ISO, it stays on even after switching it off. I can make it back to normal by switching to zoom focus view (real time) a few times (another race condition?)
- In 2.8K sometimes it switches to Real time preview and there is no way to go back to Framing. Not pressing the button, not going to the menu and back... just switch off and on works. I cannot replicate this every time, so it could be something that I'm doing and I shouldn't do  ;D

iaburn

Quote from: gabriielangel on February 04, 2023, 07:51:04 AM
The focus dots / Bad pixels issue I reported earlier is not as bad, but it is still there:

With a Single ISO Clip: Works as expected

With a Dual ISO clip: Enabling fix focus dots will make pixels appear (Less pronounced with 5k frtp).
Using Fix Bad Pixels leads to some pretty nasty artifacts.

Thanks for the samples, very good tests for debugging the focus pixels issue :)
I tried moving the pixel fixing stuff after the Dual ISO processing and doing the fixing ignoring that it's a Dual ISO video works very well, looks almost identical to the non-dual ISO:
single:


dual:


I just need to find out why once I switch fix focus pixels OFF, cannot turn it ON again  ??? Let's discuss this on the MLVApp thread if you don't mind, it's not related to Danne's builds

Danne

Quote from: iaburn on February 04, 2023, 09:26:43 AM
Hi, just tried the build from 2023-02-04 06:27, things that I've noticed:
- Dual ISO is now detected by MLVApp! :)
- Didn't noticed a significant drop (something that I can notice) in recording times on my side.
- In Framing preview (tested in 2.8K) once you activate dual ISO, it stays on even after switching it off. I can make it back to normal by switching to zoom focus view (real time) a few times (another race condition?)
- In 2.8K sometimes it switches to Real time preview and there is no way to go back to Framing. Not pressing the button, not going to the menu and back... just switch off and on works. I cannot replicate this every time, so it could be something that I'm doing and I shouldn't do  ;D
Thanks. Reproduced the issues. Will fix maybe this evening 👍.

gabriielangel

Quote from: iaburn on February 04, 2023, 10:22:05 AM

I just need to find out why once I switch fix focus pixels OFF, cannot turn it ON again  ??? Let's discuss this on the MLVApp thread if you don't mind, it's not related to Danne's builds

I noticed that with Dual ISO Clips, I need to be patient :)

Try switching to another clip, moving to a different spot in the file and wait 5-15 seconds after making a change (Focus Dots / Pixel Stuff).
The change will eventually appear.

Danne

Turning off dualiso issue fixed. New build uploaded.

iaburn

Quote from: Danne on February 04, 2023, 04:22:56 PM
Turning off dualiso issue fixed. New build uploaded.
Rock solid now, I couldn't break it  :D
It records non-stop at 12 bits on 2.8K and 5K 1736x with pretty fast preview!

gabriielangel

Tested Build Feb 04 2023 4:18 PM

-Presets 1080p, 5k frtp and 2k register all clips as Dual ISO, even when Dual ISO isn't engaged. The clips record properly though.

-The Bulk of the flicker issue is gone. I recorded with the same settings as yesterday and there is slight flickering with the 5K Ana and  5k Ana flv presets, but the other presets are fine.

1080p could cause issues when handheld though (aliasing+Dual Iso).
See video file below. Pay attention to the eyes and neck of the Blonde Haired Doll  (shiny plastic). I forgot to add more reflective surfaces in the frame to make that case more obvious.

There is also a download of 2 MLV files from yesterday and 2 recorded with today's build, for reference.

-The issue with shorter recording times at 2.8 and 2.5k / 14bits and the 32mm f1.4 lens remains. I will try all my lenses later this week to see what's going on there.

-The 2.8k and 2.5k presets won't stay stuck in preview mode anymore. Seems to be fixed.

Here is a video file where I put every preset through a worst-case Single vs Dual ISO test (I don't think anyone would underexpose that much just to save the sky!).
You can see how much improvement in DR/noise can be achieved, especially when recording at 10 bits.
Dual ISO can be very helpful to "Reach into dark corners" when it's working well.

http://bit.ly/3le61ww

MLV Files which exhibit some flicker: https://bit.ly/3jBxozR