Canon 100D / SL1

Started by nikfreak, October 19, 2015, 10:41:29 PM

Previous topic - Next topic

0 Members and 2 Guests are viewing this topic.

dfort

Good to confirm that the audio issue in mlv_lite/mlv_snd has been there all this time and mlv_rec/mlv_snd doesn't have the issue. Next thing to do is to find out what mlv_rec/mlv_snd is doing differently.

Quote from: IDA_ML on January 26, 2018, 12:52:22 PM
...in the normal uncropped lossless modes, at 1736x976 resolution, the focus pixels are back.  So, if we stay with that 2520x1080 max. resolution, the focus pixel maps will have to be corrected again.  I hope, this will not be a problem.

The only thing I can think of that might be causing this is the height adjustment in the raw buffer. I'm also seeing focus pixels on the 700D and EOSM. I can adjust the map files to deal with that. a1ex made a suggestion in the pull request:

QuoteThe offsets affecting your focus maps may indicate another problem - centering the maps (which should be done with CropPosX/Y) probably doesn't work well. Torture test: does the focus pixel fix work with digital dolly enabled, on both directions?


IDA_ML

Good job, Danne and Dfort!

Sound finally works with the 5x-magnification mode at 2520x1080 resolution without hiccups !!!  It gets recorded until video stops due to the buffer full condition.  We are now getting one stop closer to the perfect 100D experimental build.

Here is what remains to be done:
======================

RAW video mode:
-------------------

1) Movie Crop Mode at 10, 12 and 14 bits uncompressed has to be fixed.  Now just a few frames in the beginning are OK.  All the rest is corrupt;

2) In-camera playback of 10 and 12-bit uncompressed RAW video does not work - black screen;

RAW video (MLV) mode:
--------------------------

3) This mode does not work at all although it used to work with older builds.  It used to provide 5, 8 and 16 s. recording with sound at 14, 12 and 10 bits uncompressed, respectively.  Now it hangs and needs several times camera restarting and battery pulls before it can record a clip.  Sound does not get recorded at any of the modes.  In my opinion, it would be useful to fix this mode since it provides fixed record times, regardless of how the scene is lit or exposed, so you know exactly how long you can record before recording stops.  Personally, I find the 10-bit and 12 bit with sound modes perfectly usable with their 16 s. and 8 s. recording times, respectively  at the 1728x972 resolution both in the uncropped and the Movie Crop modes.

4) As a final step, shooting new test clips with the Pixel Scanner pattern and correcting the focus pixel maps after the latest maximum resolution changes (2520x1080).  Now, focus pixels are seen at the edges and high contrast even without using Dual ISO.

Silent module:
-----------------

5) Adding the latest trigger modes to that module that allow taking single FRSPs with making efficient use of the  image stabilization (IS) of your IS lenses would be extremely useful.  This feature is now available only on the latest lua_fix experimental build (Jan. 20-th, 2018), thank you A1ex!

EDIT:  The above tests were performed with the following build:
--------------------------------------------------------------------------

https://bitbucket.org/Dannephoto/magic-lantern/downloads/magiclantern-Nightly.2018Jan27.100D101_audio.zip

Danne

Quote from: IDA_ML on January 29, 2018, 01:06:04 PM
1) Movie Crop Mode at 10, 12 and 14 bits uncompressed has to be fixed. Now just a few frames in the beginning are OK.  All the rest is corrupt;

I tried just now and mlv_lite 10/12/14 bit works with or without sound enabled both in regular or 5xzoom mode. I set fps to around 24fps. What are you getting?

Are you using this build?
https://bitbucket.org/Dannephoto/magic-lantern/downloads/magiclantern-Nightly.2018Jan27.100D101_audio.zip

IDA_ML

Yes Danne,

The above tests were made with your January 27-th build. 

In my previous post I listed just the functions that are not working and need a fix.  One of them - listed under #1, is the Movie Crop Mode at the UNcompressed 10, 12 and 14-bit rates available in the "RAW video" mode.  Either it needs some work to get it fixed or it should be removed (greyed out) altogether from that mode to avoid confusion.

As far as the mlv_lite modes are concerned, I can confirm that they really work well with sound at 10, 12, 14-bits losslessly compressed with the normal and 5x-magnifications and with FPS override set to 23,976 fps.  Congratulations!

Danne

Ah, Movie crop mode, sorry. So many modes   :P

Danne

Can confirm uncompressed 10/12/14 bit yields corrupted frames while compressed lossles 10/12/14 does not with Movie crop mode...

IDA_ML

Danne, Dfort,

Do you think, these uncompressed bit rates can be fixed in the Movie Crop Mode and work with sound too?  In this way, the entire RAW video mode will work fine and the the RAW video (MLV) mode will become redundant !

Danne

Why wouldn't it work? It needs attendance and coding skills. When did this Movie crop mode stop working with uncompressed files? Anyway. Could you test movie crop mode on a regular crop_rec_5k expermental version? Maybe it works there and then we can start looking...

IDA_ML

Quote from: Danne on January 29, 2018, 05:20:24 PM
When did this Movie crop mode stop working with uncompressed files? Anyway. Could you test movie crop mode on a regular crop_rec_5k expermental version? Maybe it works there and then we can start looking...

I was asking myself that same question.  I will look through my old builds and try to find out which one worked with uncompressed modes.

OlRivrRat

       Danne wrote "sorry. So many modes"

I'll 2nd that for sure > & in some cases the Modules have Diff' Names than the way They appear in the Menus.

Can be Quite Confusing to the Not So Savvy > T'is why I often ask for people to provide Their Settings Folders ~

                                                                                   ORR ~ DeanB
ORR~DeanB  ~~  80D-ML  &  SL1+ML  &  5D2+ML  &  5DC+ML  &  70D+ML(AliveAgain)

theBilalFakhouri

Quote from: IDA_ML on January 29, 2018, 03:42:10 PM
...UNcompressed 10, 12 and 14-bit rates...
Quote from: IDA_ML on January 29, 2018, 05:15:08 PM
... these uncompressed bit rates...
::)
Bit depth, not bit rates  8)

dfort

Quote from: OlRivrRat on January 29, 2018, 06:29:10 PM
... T'is why I often ask for people to provide Their Settings Folders ~

In addition, the MAGIC.CFG file shows which build was used.

@IDA_ML - You would be able to resolve some of these issues "like when did it break" if you could compile. I've gone over this with several users and the ones who took the plunge found out that the water is fine. You don't need to know how to code in order to compile ML.

Ok--I'll get off my soap box now.

On my downloads page you'll find a file named "crop_rec_4k_original_100D_merge.2017Oct02.100D101.zip" that was when the 100D was merged into the crop_rec_4k branch. My theory is that the uncompressed Movie Crop Mode issue was there from the beginning but nobody noticed because the default is set to lossless compression. You are probably the first user who has looked into this.

fbaux

last release is from october 2017 that can be found in post 1 or here https://builds.magiclantern.fm/100D-101.html ?
EOS 100D.101 (2018-02-27) + Canon EF-S 18-135mm f/4-5.6 IS + Canon EF-S 55-250 f/4.0-5.6 IS II + Eye-fi card

IDA_ML

Calm down, guys, everything is under control !

Danne, Dfort,

I found a build with the Movie Crop Mode working properly with the 10, 12 and 14-bit depths UNcompressed in the RAW-video mode at 1800x1008 maximum resolution.  NO SOUND, of course, in the "RAW video" mode.  This build is:

magiclantern-raw_video_10bit_12bit.2017Dec18.100D101

and you can find it on the Experimental page:

https://builds.magiclantern.fm/experiments.html

This mode works also at 5x-magnification where maximum resolution is 2520x1080.

As a bonus, this build provides also flawless operation in the "RAW video (MLV)"  mode at 10, 12 and 14-bit UNcompressed WITH SOUND in both: the  normal uncropped mode at 1728x972 resolution, the MOVIE CROP MODE at 1792x1008 resolution and at 5x-magnification at 2496x1080 resolution.

ATTENTION !!!
===========
For this build to work properly, the RAW_tweak module that it contains has to be deactivated!
------------------------------------------------------

I think, this build would be a good starting point for fixing the issues listed in my post #1052, moreover it is quite up to date.

Danne

Cool. So we are going back to:
https://bitbucket.org/hudson/magic-lantern/branch/raw_video_10bit_12bit

And checking that branch last thing happening there was merging of unified 2017-12-07. So merging into crop_rec_4k would be the cause of the issue, introducing lossless and so on?

dfort

I'm thinking that the problem existed since the 100D was merged into the crop_rec_4k branch but nobody has verified it. Re-read the last paragraph of Reply #1062. Another thing to check is if uncompressed is working on all of the other cameras that have the CROP_MODE_HACK (Movie Crop Mode).

Danne

Yes, you are probably right. Tracked down to october as well. Should be tested. Tomorrow  :P.

dfort

Quote from: dfort on January 30, 2018, 01:54:37 AM
Another thing to check is if uncompressed is working on all of the other cameras that have the CROP_MODE_HACK (Movie Crop Mode).

I was able to reproduce the Movie Crop Mode uncompressed issue on both the 700D and EOSM with crop_rec_4k using mlv_lite. It only happens on the first clip recorded after starting up the camera. Subsequent clips are fine. This doesn't happen with the raw_video_10bit_12bit build.

In order to keep the variables at a minimum I used the crop_rec_4k and raw_video_10bit_12bit builds from the Experimental downloads page.

Try it out on the 100D.

Danne

mlv_lite is actually printing out that it causes data corruption on liveview display when recording scrambled frames in movie crop mode. this doesn´t seem to be an issue with raw_video_10bit_12bit builds. What´s causing this. Hard to find out...

a1ex

That calls for a "hg bisect"; let's try this range:


hg bisect --reset
hg up 295426810352 -C
make install
# make sure this build is bug-free (easiest on EOSM; in this changeset, 700D is at 1.1.4 and 100D is not included)
hg bisect --good
hg up crop_rec_4k -C
make install
# make sure this build is buggy
hg bisect --bad

dfort

Ran the test on the EOSM. Spoiler alert--might have smoked my EOSMs (yes, all of them) but that's a different topic.

Changeset 295426810352 indeed worked fine on the EOSM. Interesting that this is from a different branch.

changeset:   14611:295426810352
branch:      raw_video_10bit_12bit
user:        alex@thinkpad
date:        Wed Jan 11 01:38:23 2017 +0200
summary:     mlv_dump: fix long exposure metadata in DNG


Just to make sure we're talking apples to apples I used a clone of the main "hudson" repository instead of my fork. Here's the results:

The first bad revision is:
changeset:   15103:8f76df23cfdc
branch:      crop_rec_4k
user:        alex@thinkpad
date:        Wed Apr 12 14:50:39 2017 +0300
summary:     Experiment: prevent Canon code from saving RING and RASEN settings when removing the battery


However, several of the changesets I marked as "bad" were because they didn't build. I searched for a working version and found this one--the one prior to this didn't build:

changeset:   15884:ebf206a3a2d1
branch:      crop_rec_4k
user:        Daniel Fort <[email protected]>
date:        Fri Aug 18 18:15:35 2017 -0700
summary:     Lossless compression working on EOSM. mlv_play doesn't build.


So I reset hg bisect, marked this as the "good" changeset and the latest crop_rec_4k as the "bad" and ran hg bisect again. This time the results were:

The first bad revision is:
changeset:   15886:024351174cb1
branch:      crop_rec_4k
user:        alex@thinkpad
date:        Sat Aug 19 10:16:44 2017 +0300
summary:     lossless.c: minor cleanups


So the difference between the good and the bad changeset is only two commits. Just to double check I decided to try these three changesets and this is where the plot starts to thicken.

Remember this warning?



Yeah, it looks scary but when it came up on the EOSM a while back it seemed that the null pointer bug wasn't affecting the EOSM. However, when I started on this morning's adventure the "good" changeset was working fine and didn't show this warning but after running hg bisect it showed up on both the "good" and the "bad" changeset and the changeset in-between too. Ugh--one EOSM possibly smoked so I grabbed another then another and guess what, they all came up with that null pointer bug warning on the "good" changeset that was working fine on the first EOSM body.

[EDIT] Did a test on the unified branch and thought it saved an assert log but it turned out to be a leftover file on the card. I deleted this section to avoid confusion.

a1ex

That null pointer warning is very likely a false alarm; the pattern it's looking for is also present in vanilla Canon firmware.

Yeah, marking changesets as bad because they did not build is not the best idea; I think the best way to proceed is to either fix the build error manually if it's not hard to do so (just be aware the changes will be temporary and erased at next step), or try hg bisect --skip.

http://hgbook.red-bean.com/read/finding-and-fixing-mistakes.html#x_126 (scroll up one notch from there)

It is my understanding that we are not talking about a bug in lossless compression, but something in the uncompressed modes; that means, you can ignore lossless.c (you can even comment out these calls if they are preventing ML from building, and skip testing lossless presets).

dfort

Thanks for the tips. I'll get back to testing soon but this issue seems like Déjà vu -- Oh yeah:

https://www.magiclantern.fm/forum/index.php?topic=19300.msg192266#msg192266

I noticed it not working in mlv_rec a while back. Just checked and it is still broken. So whatever is going on is probably somewhere in the backends and not in the mlv_lite or mlv_rec modules. Or maybe I'm wrong--I've been wrong before. Like at least 10 times today!

dfort

Spent the better part of the day learning how hg bisect works and getting familiar with the history of crop_rec_4k. Narrowed down the Movie Crop Mode uncompressed record issue to something between commit 3ce63f4 (not working, my bad) and commit 41e7183 (works but needs some changes to compile on EOSM/700D/100D). Lots of commits in between that don't compile out of the box for these cameras.

Doing "hg bisect --skip" on the commits that didn't compile on these cameras seemed like a short-cut but believe me, it wasn't.

So whatever caused Movie Crop Mode uncompressed to break on these cameras is becoming like that horror movie --