Messages

Sounds interesting - what's the code change?  I don't see any updates to the repo (makes it hard to apply to other cams )

Trying to find the specific code for rendering that one graph is likely to take more than a weekend   But it is fun if you're in the mood for it.

Human perception of brightness is a weighted sum of red, green and blue components, pretty much.  So that makes sense.

The initial post wasn't confusing, it just doesn't ask anything.  Hence, I was trying to confirm what you wanted to know.

More important was *why* you wanted to know this, because that informs *what you actually want to know*.  People are on average quite bad at asking questions and often don't ask in the right way to get the answer they want.

If the real reason truly is just because you're curious what Canon use, then there's only one way to answer the question: check the code to find out what they do.  You don't want to do anything with this information once you have it?  It's hard to imagine that you really care that much about this particular histogram.  If they're approximating it in order to be fast, they won't use the approximation for anything important, so it doesn't matter.

Maybe not by Canon itself, but for example for computer graphics, there are pages describing efficient algorithms for various things using only integer arithmetic.

Why do you want this?

In fact, can I get some more detail on what you want to find out, and why?  You don't actually ask a question in your first post.  The closest you get is "I wondered what the luminance/brightness histogram of Canon cameras actually show" - for this, you'll need to check the rom code.

Do you want any somewhat fast alg?  Do you want Canon's exact alg?  Why?

Documentation of how Canon implements it?  None that I know of, but I haven't looked at all.

Okay, so it's not busy before it sleeps, or during.

I tested 70D waking from sleep by holding full shutter until it took a pic, then immediately entering ML menu.  Dual ISO menu options were present.  Didn't time it, but less than 2s.

The time until the module becomes available seems to vary, though, might be related to how "busy" the camera is with taking and storing photos, bu that is a wild guess of mine. Haven't had that much time to do thorough tests.

Sounds strange.  How could it be busy with taking or storing photos, if it was asleep?  Are you doing something unusual before or after it goes to sleep?

Works fine on 70D.  I can't think of many reasons why you'd see this behaviour, one possible is if you have multiple modules enabled and one is slow to initialise.

Do you see the problem if you:
- disable all modules
- enable only dual_iso
- restart cam
- wait for cam to sleep and test as before

Assuming that works okay, enable other modules *one at a time*, and re-test each time.  This should let you find which other module is causing the problem.

If you really want to know, you'll probably have to dig around in the rom code to find out.

It makes sense to me that it could be an artifact of integer maths; these cams don't have hardware for floating point.

Walter regularly helps me with programming, otherwise how would I know how features should work, or what cams have them?

Fast preview looks good!  Haven't looked at the code but it doesn't sound like it should be too hard to integrate into any other repo, or make it work on more than the 5d3.  You might be able to get around the large single alloc problem by segmenting it.  Try say, 6 x 256kB allocs.  You'd need to segment the LUT too, and do perhaps some bitmasking ops to quickly determine which segment to use.  Only benchmarks can tell if this is worthwhile

Re my opinion on "where", see my recent post: https://www.magiclantern.fm/forum/index.php?topic=26814.msg244762;topicseen#msg244762

My repo now has crop_rec_4k_mlv_snd code (and, unlike other crop_rec_4k_mlv_snd forks, has all the *other* code, too).  I'd like to start bringing in code from other popular repos, so we can have a single place to work from again.  I've fixed a significant number of bugs (especially in the module system) and improved build time performance greatly.  It builds easily on modern systems, with modern compilers etc (this makes the ML binary smaller as an additional bonus).

I can't make people use it, but I think most people would like to have a single repo that's useful for everyone, and I'm willing to put in the work to merge code.

There are lots of good cameras.  Porting to any cam takes a lot of work.  I'd like to see ML on the 6D too, but I don't own one.

Since you want the 6D to get more attention, you could consider learning to code, you'd want to learn C and some Arm assembly.  After that, get some experience with common reverse engineering tools, definitely including Ghidra.  We can help you at any stage.

If that sounds difficult or time consuming to you, now you should understand why not every camera has a port, and why not all cameras have the same features.

[Updates for 2023 Jun - Aug]

Updates for 2023 Jun - Aug

Most of this period was spent integrating popular branches from official repo into mine.  The goal here is to have a modern repo with all of the popular branches merged into one; this allows anyone who was working with e.g. crop_rec_4k_mlv_snd to easily port their work into my repo, if they want.  All work is merged down into one branch.  All fixes and improvements go in the same branch; everyone benefits, fixes don't end up living in some minor branch for years.

I don't understand the branching strategy used by official, but it clearly allowed branches to remain alive for very long periods, often years.  This made integrating the work very difficult.  This was something like 900 commits, many requiring complex merge conflict resolutions.  I didn't enjoy this work but it was repeatedly causing great pain, and the more we made changes to the base repo to improve support on new cams, the harder it became to merge with official.  There's still some minor branches that may want to be merged in (e.g. iso_research), but the merge work looks easier for those that remain.

Lua_fix and crop_rec_4k_mlv_snd are now fully merged, initial testing effort (thanks Walter and Bilal!) suggests there are some minor problems but no major bugs have been found.

Some long standing bugs were fixed after these were merged, see my additions here (for lua_fix): https://github.com/reticulatedpines/magiclantern_simplified/commits/b7b45b3512a89b278f53d401bca3b2304393a141
And here (for crop_rec): https://github.com/reticulatedpines/magiclantern_simplified/commits/b4a40cf13e53a064049b09fd8639665e285c15c1

Build speed has been improved again.  Module builds in official had been artificially limited to build sequentially due to a known bug.  I fixed that bugs months ago, so I've re-enabled parallel builds.  Also during this period I found more bugs in the build system But they're old bugs, so, I guess we can tolerate them a little longer...

Current reticulated_pines repo status
Ready to integrate third-party repos if desired.

If people are using your builds, they're valuable and I want to collect all the good parts into one place, while keeping builds useful for everyone.  Improve raw video but break features for stills shooters?  We need to find a compromise.  In exchange you get a base repo that builds easily on modern systems, builds quickly, supports modern cams, has bug fixes, and is actively maintained.

If you have a repo based on lua_fix or crop_rec_4k_mlv_snd (or many other branches), your changes should apply over the top of my repo.  This won't be completely trivial because we've changed things on that repo to support new cams, this includes the boot process for all cams, and e.g. memory subsystem changes.  If you'd like to try this, fork reticulated_pines and work in a branch.  If you'd like to integrate code but don't want to do it yourself, there's a good chance I'll do it if you ask.

Upcoming work
Community build testing!  Private testing on Discord suggests this merged code is good enough for wider testing.

Automated regression test system.  With more cams expected to work, getting regression testing useful again becomes more important.

Every camera needs unique work to get features working.  There are very few devs.  If no dev owns a 6D, it's not going to get worked on.

Title say CF to SSD SD to SSD and I asked for the same thing in SSD too with similar answer that's why

Does it?  I only see CFast.  That is not CF.  It's easy to adapt CFast at high performance because of the protocol.  But 5D3 doesn't support CFast.  It's hard to adapt CF or SD at high performance.

OP specifically says CFast.  This is a different standard to Compact Flash.  5D3 does not support CFast.  OP's idea cannot work.

Might it be possible to plug together some set of cables and adapters?  Probably.  With high performance?  Probably not for CF or SD.  But I'd be happy to be proven wrong, then we'd all know a combination that actually worked!

Have you tried clearing camera settings from Canon menu?

Okay.  MLVFS already exists and does transparent MLV to CDNG conversion, possibly that's all you need: https://www.magiclantern.fm/forum/index.php?topic=13152.0

MLVApp is open source and can convert to multiple other formats.  It's not CLI, but adding CLI options would be useful and likely not too hard.

There's modules/mlv_rec/mlv_dump, which has many options (I haven't used this tool, just know it exists):

Code Select Expand


Usage: ./mlv_dump [options] <inputfile>
Parameters:
  -o output_file      write video data into a MLV file
  -v                  verbose output
  --version           print version information
  --batch             format output message suitable for batch processing
  --relaxed           do not exit on every error, skip blocks that are erroneous
  --no-audio          for DNG output WAV not saved, for MLV output WAVI/AUDF blocks are not included in destination MLV

-- DNG output --
  --dng               output frames into separate .dng files. set prefix with -o
  --no-cs             no chroma smoothing (default)
  --cs2x2             2x2 chroma smoothing
  --cs3x3             3x3 chroma smoothing


Many options trimmed, it's pages of possible flags!  Best to check yourself.

The ffmpeg code for working with mlv wasn't written by ML team, so you're asking in the wrong place - you probably need to ask ffmpeg (or check their source, it's available).

You don't need to use ffmpeg to work with mlv files, so, if frame extraction is your end goal, there should be an easier way.

Can you link to the tutorial you're currently using?  Are you using dual ISO, or HDR recording?

I'm guessing you're using HDR recording (which captures alternating light and dark frames) and trying to follow a dual ISO tutorial.

Thinking about it, you should always be able to unroll the access to p so you write an aligned amount into the struct in one go.  This will avoid needing to do complicated shifted accesses through the p pointer, which is likely to be quite slow.  Accumulate writes into a local var until you have some amount of contiguous bits that is a nice round number.  So, 32-bit minimum, in which case your local var is a register, or, probably better, some multiple of sizeof(raw_pixblock) that is itself a multiple of 32 bits in length.  Multiple of 2 would work, higher is probably more efficient.  Your local var is then some buffer and you memcpy() into your dest buffer, no need to do any shifts during the write, no need to do any expensive sub-32-bit length writes, and you reduce loop overhead.

Separately, you possibly could use the UNCACHED() address for p.  This may avoid expensive sync operations in between writes.  On the other hand, this may already be happening in which case it's pointless, or, it may be unsafe - depends what the code is doing.

It'll interact with caching, might be that.

I don't want to review code unless I can be sure of what changes are made - can you host this as a repo somewhere with "fast" and "slow" being different commits?

(You also probably want to avoid division since that's done in software, and avoid unaligned memory access since a) it can be slow on ARM and b) it may return different data than you expect)

Meaning while the HDMI is plugged in, switch MLV RAW off and be in normal recording mode and err 80 goes away after a few reboots and a clean install of ML.

Did you test whether restarting cam a few times without an ML card in also stopped err 80 occuring?

Switch MLV RAW on and get err 80 and locked HDMI screen.

Here, do you mean the very first time MLV RAW was enabled with HDMI connected, err 80 would always happen?  Or was it only sometimes?
After err 80 had "stopped", did you ever test whether restarting cam a few times without an ML card, with HDMI connected made err 80 happen again?

Cool, glad to hear it's mostly fixed.  I'd guess you might be hiding file extensions in Explorer, combined with not having anything installed that can sensibly open CR2.  Some quick searches suggest Windows can handle older format CR2 but not newer.

Could ML have caused a problem?  Maybe, but there's not much information to go on.  ML will print its own errors to screen and log some stuff to card.  It doesn't sound like you saw anything like that?

I'd say it's more likely just a random hardware failure, the err 80 error reports I found elsewhere (unrelated to ML) were mostly board or cable failures.  But I'm not confident either way - could be anything.

According to Canon, Err 80 is when "An image or electric control malfunction has been detected".  The only thing you haven't yet tried from their official advice is taking it to a Canon Service Center.  Worth a try for an assessment if there's one near you.

Largely guessing, this sounds like a hardware error to me.  There are very few reports of Err 80 on magiclantern.fm previously, so I'd assume ML is unlikely to be the cause.  The pattern of "intermittent but getting more frequent" feels more like hardware than software, too.

If you know what a UART is, you might be able to get more information that way.  USB might work too, is it recognised when you plug it in?