Messages

Is there any way to fix this? Such a problem on many source videos
(Highlight Reconstruction and White Level Change - do not help)

I am not familiar with the peculiarities of ML Dual ISO nor FRSP.

However, would the recently added Sobatka AgX trasnsform help in reducing this clipping/black-sun problem?

Troy Sobatka also made a "Filmic" Blender module, which has been adapted to Darktable, and which seems to deal well with clipping and tonal shifting in the highlights:

https://www.youtube.com/watch?v=bnfBgK3j1Bg&t=165s
https://www.youtube.com/watch?v=bnfBgK3j1Bg&t=165s


It would be great if MLV App featured a similarly versatile "filmic" module!

Beautiful!  Nice work!

What resolution did you shoot (1080, 2.5k, 2.8k, 5k-frtp, etc.)?

Also, how did you grade it (MLV, Resolve, Premiere, etc.)?

I am trying to shoot a photo with long exposure 30 seconds at daylight.
The model uses the light source to draw something up in the air.
Exposure finishes the shot with rear-flash.

Flash power is finite and if background is very distant and bright (e.g. mountings) then
catching background at the moment of the rear flash shines, my camera should increase ISO, IMHO.

It sounds like you are trying to balance the exposure for three different elements in a single frame:

• the flash exposure for the foreground and for the model;
• the exposure of the model's handheld light source;
• and the exposure of the background.

If so, why does the ISO/gain need to change during the exposure?  Just set the desired exposure for the background, based on a 30-second shutter speed.  Then, set the flash to give the proper exposure for the foreground.  Then, adjust the output of the model's handheld light source so that its brightness balances with the foreground and background.

That's why I would like to see the parametric ISO feature.
Use case:
Shutter speed: 15 seconds
Flash mode: rear
ISO(t): if t < 14.9s then 100 else 8000

There could be a few ways to "effectively" change the ISO gain during exposure.  Not sure if "parametric" is the proper term for such a method.

The simplest way to increase the "gain" during such a long exposure would be to set the camera to the higher gain/ISO and then simply hold an ND filter in front of the lens for the first half of the exposure, and then remove the ND filter for the last half of the exposure.

Another possible method would be to use the Dual ISO feature during a 30-second video with a 360-degree shutter speed (i.e. 1/24th of a second at 24fps).  Then, "exposure blend" all of the video frames into a single still photograph, but use the lower ISO scan lines from frames of the first 15 seconds and use the higher ISO scan lines from the frames of the last 15 seconds.

The "black sun" effect is a well known phenomenon with CMOS camera sensors -- it has nothing to do with Magic Lantern.

Here is one way to fix it in post.

Any idea why I can't turn IS on on my eos m? My settings are pretty standard, I'm shooting in 5k frtp and 2.8k, tried different settings with/without fps override/shutter fine tuning or MF/AF. IS is of course enabled in Canon menu.

Which lens are you using?

There is no IBIS (In-Body Image Stabilization) with the original EOSM -- stabilization is in the lens.

Perhaps it is a corrupt frame or file.

Did you try selecting "Skip Frame?"

Are you doing a batch export of multiple files?  If so, did you select "Abort Current Export?"

Also, try exporting different individual files, and see if any of them work.

Now if we want to go back to ExFat after installing ML, this doesn't seem possible?

Since we would prefer to record while in ExFat for obvious reasons.

Is there some sort of a workaround for this so called bug?

Unless I need to install Catalina onto one of my hackintoshes on a spare drive? Ha.

I have no problems mounting and writing to (or copying from) ExFat drives with Linux, with two ExFat SD cards containing ML.

As a Mac user, you can just boot a Linux live USB.  Pick a newbie Linux distro, like Mint, Ubuntu, Pop!_OS, etc.

Of course, format the SD card in-camera prior to copying the ML files.

It might be prudent to put the ML files onto the USB stick (after you have burned the Linux live OS onto it), and then mount it, and then copy those files onto the SD card -- instead of trying to mount any of your Mac partitions.  The Mac filesystem is notoriously fragile.

Here is a video by @ZEEK on using monitors with the EOSM:
https://youtu.be/UCc4VNzivwE

Here is another one by @ZEEK:
https://youtu.be/dtnZAjh7kpQ

In his other EOSM instructional videos, he often mentions how monitors work with the video settings that he is showing.

Does the Menu button not work at all or does it work intermittently?

There could be many causes of this problem.  There are threads on humidity/condensation and liquid intrusion possibly causing contacts to corrode on Canon switches.  Here is one.  Here is another.

There are potential solutions posted in those threads, but you might try rapidly pressing the button about 10-20 times to see if it starts working again.

... 2.8k raw, 12 bit. Exposure was done correctly, right in the center of the histogram and utilized zebras.

As was suggested in the Reddit thread, a little ETTR ("Expose To The Right" side of the histogram) helps -- don't expose to the center of the histogram.

ETTR works especially well when shooting raw in a higher bit depth.  You can really push things far to the right side with raw, as long as nothing clips (or as long as any clipping is kept to an acceptable amount).  ETTR doesn't work as well with compressed codecs, but one can do it a little, perhaps a stop or two to the right.

Imagine that the noise is a vertical band on the left end of the histogram.  Any detail that you would like to see without noise should lie to the right side of that noise band.

Additionally, it probably wouldn't hurt to add a little more contrast in grading your footage.  That would make it look more "snappy."  One way to add contrast In MLVapp, is to use the Gradation Curve in the processing tab.   Boost the right side of the Gradation Curve and lower the left side of the curve to taste.

By the way, the noise isn't too apparent, likely due to the denoise and the other processing you did in MLVapp.

As a somewhat experienced cinematographer and photographer, I would urge you to work on any one of the cameras with an EF-M mount, that also has viewfinder capability and dual pixel autofocus, and the potential for continuous, 4K, raw video.

There are plenty of ML cameras with EF and EF-S mounts that work great -- more of those are not needed.

On the other hand, EF-M mount cameras (the EOSM line) have a shallow mount that accepts almost all lenses via mount adapters, speedboosters, and tilt/swing and tilt-shift adapters.  Also, the EOSM cameras are relatively cheap (unlike the shallow mount, full frame, EOSR line, with some models already featuring raw video capability).

The EOS M50 is a good candidate that has a built-in viewfinder (allowing one to use radio slaves along with the viewfinder), but I am not sure if ML would work on it.

An EOSM with full-sensor, 4K, raw video capability and a viewfinder would be amazing!

[IS]

Dual ISO is not the same as Frame Stacking

Agreed, but "HDR Video" IS the same as frame stacking.

"HDR Video" is a different ML feature from "Dual ISO."

Did you watch the video that I linked above?  Go to a 07:05, and you will see the playback of alternating frames with differing gain (iso).  ML HDR Video creates those alternating frames so that they can be stacked.

Using dual ISO on this sequence would avoid the lights being clipped, but it would not help reduce noise, because there is very little light hitting the scene as a whole. You would end up with either a lot of noise in the shadows, or a very underexposed image overall.

I do not agree here.  Although I have never tried Dual ISO, I have seen the work of others, and Dual ISO and HDR Video (frame stacking) give similar noise reduction and dynamic range results.

It is conceivable that Dual ISO reduces detail, because it necessarily halves the vertical resolution.

Nevertheless, my post above referred to the "HDR Video" feature of ML -- it didn't refer to "Dual ISO."

Dual ISO is used to preserve highlights. It lowers the sensitivity to properly expose the highlights and then uses a normal sensitivity to properly expose the rest. When you have plenty of light available to begin with, you can expose everything properly using Dual ISO (But there are often artifacts at the transition points).

Artifacts and noise are two different things.

The primary drawbacks that Dual ISO suffers are aliasing/moire and the halving of the vertical resolution.  The main problems with HDR Video are motion smearing and the halving of the frame rate.

It is my understanding that Dual ISO and HDR Video are based on the same principle.  Dual ISO "stacks" "interlaced" frames while HDR Video stacks consecutive frames.

Frame Stacking increases the Signal-to-Noise ratio.

So does the ML Dual ISO feature, as it increases dynamic range (very similar to signal-to-noise ratio), thus reducing the relative noise.

Again, what I was referring to in the above post is the old ML "HDR Video" feature, not "Dual ISO."

The noise is random and is completely different across frames.
On a still shot, the "Signal" (the subject) doesn't move or at least varies at a much lower pace than the noise itself.
Therefore, when you average, the "Signal" average is higher than the "Noise" average, so you end up with an image having a lot more "Signal" and a lot less "Noise", as you can see in the example above.

Yes.  The same dynamic range increase happens with Dual ISO and with HDR Video.

By the way, the two "interlaced" frames which are "stacked" in Dual ISO have different noise patterns (barring fixed-pattern noise).

Again, I was referring in the above post to the ML feature of "HDR Video."  I was not referring to "Dual ISO."

[Frame Staking Noise Reduction used on Video]

Frame Staking Noise Reduction used on Video
[snip]
I have been running a lot of tests lately to devise an optimal pipeline for a project. Noise being something to always take into consideration, I were wondering if Frame Stacking, used regularly when taking stills, could be applied to video. There is a built-in script in MLV App based on this idea.
[snip]
After processing. The noise is completely gone on the Recycling bin, and a little movement is left in the shadows which were lifted to the right of the image.

Isn't that basically what the ML "HDR Video" feature does?  It "stacks" two consecutive video frames, each with differing gain (iso), to yield higher capture dynamic range.  Hence, lower noise from combining those two consecutive frames.

Here is an old ML HDR Video tutorial that shows the DR increase along with the motion smearing.

What really puzzles me is why people ask this types of questions on forum? So then they wait for 3 days before someone will answer or what if no one will answer at all.. when you can just google and watch any first youtube video with installation process and clearly see the "fw update" step there.

OP is evidently a veteran open source contributor who likely uses the official documentation first.

No doubt that after the initial negative response he received, he did just what you suggested and watched a YouTube video.

I think sometimes it just happens. Sometimes even a very smart person can stuck on something that even 5-years old can figure out in 5 seconds. I had that moments too and then when you realise it - big facepalm

In this case, the installation instructions seem to lack the details of an important step.

About attitude - well.. do you need that thing on your camera or not?

About developers -- well.. does ML need them or not?

OP stated that he works on many open source projects and was, accordingly, diligent with his presentation of details.  He was met with discouragement.

When you say "ML community should do this, and shouldn't do that" - I don't think somebody will actually listen.  Because there is no "team" or something, or company, or corporate ethics. It's just people. Individuals. So someone may listen and someone may not.

Who said anything about "community?"

Please enlighten me on how things work with open source projects.

You'll never make docs that everyone finds clear, either.

Perhaps, but evidently there is a significant "air-gap" in the official ML installation instructions.

I try to give polite answers to questions, in a way that lets people know how to ask it sensibly the next time.  I take this attitude so that the *next* generation of noobs is more likely to do things the sensible way.

Please note that OP does not come across as a "noob" with his detailed presentation of the problem, and he even stated that he works on many open source projects.

I know it won't work 100%, but I hope it will approach the limit over time.  Bear in mind you're not just replying to the questioner, you're also replying to the much larger audience of lurkers.  Giving a positive, helpful impression seems better to me than a negative one.  Yes, it's frustrating answering the same questions every time - but that specific person is asking it for the *first* time, so they don't understand why they're now getting shouted at.  We can do better than that.

Agree 100%.  Good point about the lurkers.

I think that replying in a hostile, or grumpy way is *understandable* for people that have been here a long time (or worked with similar attitudes from noobs in different circumstances), but not *helpful*.  It gives a hostile impression about ML forums, discord etc, and is off putting.  I definitely get annoyed sometimes.  Sometimes I even reply when I'm annoyed, although I try to simply wait it out.  It's okay to let someone else answer if you're not in the right mood.

Grumpy is one thing.  Withholding the solution is another thing.

Re the specific question of documentation; we should try to improve it so it makes the most sense to the most people.

The problem is not that complex.  It simply seems to be an error of omission within the installation instructions, that can be remedied by adding one or two senteces.

It's unfortunate that very early ML docs called ML firmware, and it's unlucky that we use Canon's firmware update *menu* to do something that's not a firmware update; it's pretty obvious why this can cause confusion.

That is a separate topic from OP's problem -- OP just needed to know where to go in the menus to start the install.

Regardless, I am not sure how such "confusion" over what is essentially a semantics challenge could be a serious issue.  That problem can be easily fixed with one or two sentences.

It's interesting that you think most devs use proprietary OS - I'm fairly sure most code was written using Linux.

Coming from an almost purely open source experience for the last 20 years, it is apparent that a majority of the devs (and almost all of the users) run proprietary OS's and proprietary software.  Any consideration for Linux and the BSD's seem like an afterthought here.

Magic Lantern is lacking everything:
* user friendliness
* user documentation
* support
* code review
* developer documentation (!)
* manpower to continue this project

I mostly agree here.

As far as I'm concerned, due to its hackish nature it was always designed for "technology nerds", not for "general public".

Everyone has a right to their opinion.  My opinion is different from yours.

What open source developers do is what they consider fun. If project assumes user had to have at least a bit of skill, then technically it is user fault if he doesn't.
https://mikemcquaid.com/2018/03/19/open-source-maintainers-owe-you-nothing/

I know all about open source.  I have been almost exclusively running open source software for 20 years (since 2002) and first contributed code to an open source project 19 years ago (2003).

Please note that my first open source contribution occurred 15 years prior to the Mike McQuaid post which you linked and note that my first open source contribution occurred three years prior to the first coding project that McQuaid lists on his CV

There's a reason it is almost a year since Alex last logged in. User expectations are different than authors intentions, and it is devs who give you something to play with zero promises.

There are many reasons why mods/devs come and go on forums.

In the instance of this thread, the OP stated that he works on several open source projects.  So, OP obviously has the "skill" that you seem to think is required for users of open source software, and, accordingly, he gave (without prompting) more than the necessary, clear details needed to assess and solve his problem.

The appropriate response to OP is to give the simple solution -- just like it is done in most other forums of open source projects.  Instead, OP was met with condecension and misplaced blame.

Please explain what is the purpose of this "General Help" section of the ML forum, if all users are expected to already have the "skill" to know the answers to their questions.

On the other hand - wiki is publicly editable. Feel free to improve it.

The lack of documentation is only part of the problem, but, with that in mind, we should be more helpful and less condescending in the "General Help" section of the forum.

And this is a different view from experienced dev vs user.

It appears to be a difference in views between the ML devs and the devs of other open source projects.  For the last 19 years, I have been involved in the development of open source software, and have  used open source software almost exclusively.  I see a general difference in attitude at ML.

I have given examples of the simple sentences that it would take to almost entirely correct the problem (and I have given OP the answer that he needs to proceed with the "installation").

There is no reason to be condescending nor kurt with OP, who is an experienced open source contributer who gave more than the necessary information to solve his problem.

No, it shouldn't.  Reason is simple - even now people think that Magic Lantern is installed into a camera.

That's not surprising, considering that OP's post has exposed a fundamental gap in the ML installation instructions.

However, I fail to see how that notion by new users is relevant.

And whenever camera fails (99% of the time cause being not related to ML), we are accused of failure.

I see no way to explain to a technology noob why "firmware update" is not a firmware update, and what bootflag setting means + why it just can't brick / damage a camera.
If you can - please add a section to wiki. Everyone will benefit from that.

Well, I haven't seen many of such "accusations" here over the years, but such notions are instantly negated by single sentence explanation and by a two-sentence disclaimer.

If you state, "ML software runs on the camera, but it isn't actually installed on the camera," 99% of the folks will comprehend that simple, compound sentence.

You can also disclaim any problems caused by ML and declare that the user proceeds with ML at their own risk (which I think has been already stated somewhere on the site).

Problem solved.

I see no way to explain to a technology noob why "firmware update" is not a firmware update, and what bootflag setting means + why it just can't brick / damage a camera.

Not sure why it is necessary to explain the firmware point, but how about this:
"Go to the firmware update selection in the menus (usually found in the 'gold/tools' section), and start the firmware update process.  Please note that the firmware is not actually being updated -- ML runs on top of the Canon firmware."

What more needs to be explained?

In regards to the bootflag (which by the way, is not part of the topic of this thread), the language from the wiki FAQ seems to cover the topic.  Perhaps a condensed version of that text could be added as an "incidental" in the installation instructions.

If you can - please add a section to wiki. Everyone will benefit from that.

I am not sure that contributing to the wiki works as smoothly as you think, but I have already provided language that can prevent OP's problem.  Perhaps someone who is already thoroughly involved with the wiki can insert those few sentences.

This is a difference between developer and user. The thing you see is "user friendly" may be actually damaging for project reputation. Yes, I'm probably exaggerating in that particular case, but when you know how it works from the other side, you tend to choose the words more carefully.  You see - those people do a great work. But they do it on their own, in their own medium.

Again, no lecture is needed on developers vs. users nor on open source projects.  I think that there is a difference in attitude between ML seniors on this forum and mods of forums on most other open source projects.

I agree you are exaggerating.  I fail to see how "damage" could result from the concise explanations (accompanied by apropriate disclaimers) I have given, nor from the simple solution that I have given to OP.

Coming from the open source world, I have noticed an occasional kurtness and obstinance in this forum.  I have also observed a palpable fear of retribution from Canon, which is likely unfounded.

In my experience, most forums for open source projects are helpful and encouraging.  One notable exception would be the mods of the Arch Linux forum, but I hear that things have improved in the years since I posted there.

In regards to open source documentation, I find that most projects have extensive and thorough manuals and help pages.

Almost every installed package on Linux/BSD's is accompanied by a detailed man page.  For example, here is one man page for vi.  With such man pages, one can usually learn and take advantage of the full functionality of terminal programs.

Of course, apps on Linux/BSD that utilize a GUI might have additional help pages.  Even many terminal programs offer addtional help pages.  Here is the configuration help page for the NeoMutt mail client.

Note the detail and thorough explanations given in these man/help pages.  They give everything, and usually do not assume that the user is a newbie.

A majority of the open source projects in the Linux/BSD realm treat users differently than here.  Perhaps it is because most of the ML develpers use mostly proprietary OS's and proprietary software.  Likewise, with most of the ML users, who expect some sort of support similar to the proprietary software/OS's that they run.

You highlighted a very important issue here - there are people with knowledge, but there's not enough people who contribute that knowledge back to the project.

Of course, like many other open source projects, ML is short-handed.  However, I have given the language that fixes the install instructions.

Regardless, there is no need for condescension nor blame towards new users who ask questions in good faith in the "General Help" section, especially towards users as diligent and detailed as OP.

No, they shouldn't - because there's no actual firmware update performed.

Yes, it should -- because of the problem with the instructions that is the cause of this thread.

This is just a way to execute a program on stock camera.

A new user is not a pedant on what is occurring technically within the process.  The new user just wants to install ML.

If it is important to be more accurate, the instructions could read:
"Go to the 'firmware' selection in the menus (usually in the gold 'tools' section of the menus), and proceed with the firmware update process."  However, it is imperative to tell the new user where to go to find the firmware update selection in the menus.

Because that may depend on model.

In which models is the firmware selection not in the menus?  ... or not in the gold "tools" menus?  It is important to let the new user know that they have to go to the firmware selection in the menus and to suggest where in the menus that they might find it.

This is a fundamental instruction that is lacking.  OP's post highlights the problem.

It, in fact, didn't - however ML (unfortunately) is not a beginners tool, so finding how to execute firmware update is quite an easy task.

Not sure what your point is, but OP (and others) have problems with ML documentation.  OP came to the forum to find an answer to a problem caused by incomplete ML documentation, and instead of getting straightforward help, it was suggested that the problem was his fault.

The only reason why executing a firmware update is "quite an easy task" is because most folks go to YouTube videos by @Zeek and others that show them how to do so -- they don't use the vague/incomplete ML instructions.

This should be covered in your manual.

The instructions in the ML wiki state:
"Launch the Firmware Update process and follow the messages on the screen."

Those instructions should probably add:
"Go to the 'firmware' selection in the menus (usually in the gold 'tools' section of the menus), and update the firmware."

There is no hint this step should start automatically. Quite the opposite.

It's not the "opposite" -- it just doesn't give any clue on how to "Launch the Firmware Update process..."

In about 9.5 years you are the first person having trouble to make sense of it.

OP is just being diligent by following the incomplete ML documentation and then asking for a solution directly on the forum.  Most people find a YouTube video with instructions, when they encounter ML problems.  OP is definitely not the first in 9.5 years to have problems with ML instructions.

OP's difficulty highlights problems with ML documentation.  Let's not assume that this is OP's fault.

@JohnC,
Here is a YouTube video on installing ML on a 6D, cued to the point at which the firmware is "updated":
https://www.youtube.com/watch?v=XWf8289_34U&t=240s

Not sure what you are seeing, but what I see isn't bit depth banding.  It would help if we could see the same setup and camera angle, both with and without the phenomenon.

If I do the same trick with an Alexa XT (preparing for a feature) there seems to be some banding on both sides of the image:

... but is it actually the same?  There seems to be only a single shot with a single camera in one video,  but the other video shows completely different shots with a variety of setups.

Were both camera shot under the exact same conditions (at the same time, with the same lighting, from the same camera angle)?

Was the reverse lens configuration "light tight" on both cameras, or could light leak in, around the back of the lens?

Did one setup use a matte box or lens hood (or a lenser flag) while the other setup didn't use a matte box or lens-hood?

You mentioned the possibility of internal reflections.  If the Alexa setup was light tight, could the lens be sending light off to the side internally, that reflects off of shiny parts of the lens adapter/mount?

So, i recorded some footage in 10bits raw in my good and trust Canon 5D iii
         1- I Use the MLV App, that is amazing, to output to ProRes 4444. Then wend i open in premiere i saw banding in the footage. If
         it's in 10 bits and in ProRes 4444 shouldn't see any kind of banding in the footage, right? What im doing wrong?

Banding occurs all the time in 10-bit footage, but it usually isn't perceptible.  Please post/link the banded 10-bit footage.

    2- After this, I export the same footage in CinemaDNG Lossless, and the banding disappeared. Great, very happy. But... wend i import the CinemaDNG footage in Premiere the colores are one thing. Wend i import the CinemaDNG in after effects they are other thing.
        I see in premiere the colores are a little bit more washed and looks like the image are not so sharp. In after effects the colores are more saturated and looks like the footage are more sharp.

The difference in colors between Premiere and After Effects could be caused by a number of things, and they could probably be matched by changing settings in one or both of the programs.  The difference in sharpness could be sharpening and/or contrast settings.  The "washed" look means less contrast and, likely, less sharpness and less saturation. 

Perhaps After Effects is applying filtration/effects.

35mm f 2.0 not good for aps c sensor it is hardly a good focal lens for full frame sensor , its a uphill battle from there with ml crop walking around with telephoto and shooting video i am not trying to be negative but composition is very important if your using crop senor camera you almost always have to go wide but not full frame wide.that is why most high end zoom start even wider than 35mm  it is a very common mistake that happens investing in a fast lens that was made for full frame camera but trying to use on crop sensor and when you mix ml crop on top not a very good look.

It's probably unnecessary to reiterate, but OP used a focal reducer (speedbooster) with his 35mm lens.  So, with APS-C crop he was getting around a 40mm full frame equivalent view, and with 3x crop he was getting around a 75mm full frame equivalent view (similar to a full frame "portrait" lens).

How would I go about adding motion blur that doesn't look weird? Any suggested tutorials?

I found this motion blur tutorial for Resolve, and this video shows a few ways to apply motion blur in Premiere Pro.
No doubt, similar motion blur methods can be used in other NLE's and compositors, and I am fairly sure that there are special plugins and stand-alone programs available that do the same.

Thanks for the tip. Posted!

Thank you for inspiring us with your work!

Nice work!

That lens/focal-reducer combo yields some splendid looks on the EOSM.

The little bit of jitter during the fast pans and high shutter speed might be smoothed with simple motion blur applied to those frames.

Since some of the shots are 3x crop, it would be fitting to post this video in this recently active eoshd.com thread on the EOSM ML crop modes.

1. I have particular requirements for a dedicated camera rig (a camera pair). The Canon EOS R5 would be ideal if I could dial up slower frame rates in 8K mode, say, down to 1fps. Are there any technical reasons why this might not be possible?

The R5's internal intervalometer can shoot at one second intervals in raw or jpeg.  An external intervalometer might allow shorter intervals.

2. It would also be advantageous if I could select a narrower vertical pixel range. So for example, 8192 wide by 500 pixels high. Again,  are there any technical reason why this might not be possible?

Why not just crop into the frame in post?

It looks good to me!

To eliminate the hand-held micro-jitters, it might be good to add some weight to the camera or use a shoulder mount, gimbal or some other stabilizer.  @ZEEK sometimes uses the neck strap tension trick to stabilize his hand-held shots.

By the way, what is a Canon 600DEF?  Do you mean a 600D, which happens to only come with an EF mount?