Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - a1ex

Pages: [1] 2 3 ... 352
General Help Q&A / Re: Eos m series
« on: Today at 10:34:52 AM »
Correction: they are identified and already used on 100D and 70D (would be nice to get better feedback from 6D); I just want to merge them into mainline before continuing on M2. I need some help with testing, please check the pull requests page.

edit: to be clear, I'm primarily looking for help with testing the updates to ML build process and the QEMU workflow on the PC, on various operating systems, not with testing ML binaries on the camera.

Camera-specific discussion / Re: Canon 6D
« on: Yesterday at 05:16:47 PM »
It can be worked around in post, or - preferably - it can be fixed in lossless.c. The only issue - I have no idea where to start. Maybe this can help. Or running Canon's compression routine in QEMU and comparing register configurations with our own. BTW, is the defect present on a full-res silent picture? What about different resolutions?

The issue from 650D is not that obvious, but you can look into it if you like math ;)

Camera-specific discussion / Re: Canon 6D
« on: Yesterday at 04:12:27 PM »
Alright, so the defect appears to be a per-channel offset. Let's check the two images that are supposed to be identical - will look at the 4 Bayer channel differences (R, G1, G2, B):
Code: [Select]
octave:1> a = imread('2-normal.pgm');
octave:2> b = imread('2-lossless.pgm');
octave:3> for i = 1:2, for j = 1:2,
>           chd = (a-b)(i:2:end,j:2:end);
>           disp([min(chd(:)), max(chd(:))])
>         end, end
  128  128
  256  256
  512  512
  1024  1024

Result: each channel differs from the ground truth by a constant value (that happens to be power of 2!)

There are no other differences (that was the role of min and max).

Let's check the other pair of images (slightly different scene, so you can't draw many conclusions from their raw difference). Will check only their optical black area; in this case, min and max will no longer be relevant, as we will be looking at two different samples; let's take a look at median differences (median is statistically robust):
Code: [Select]
octave:4> a = imread('Silent-14bit.pgm');
octave:5> b = imread('Silent-lossless.pgm');
octave:6> for i = 1:2, for j = 1:2,
>           chd = (a-b)(i:2:end,j:2:50);
>           disp([min(chd(:)), median(chd(:)), max(chd(:))])
>         end, end
    0  128  214
  151  256  329
  397  512  599
  961  1024  1092

Result: median differences are the same as before!

This defect is likely predictable and can be easily worked around in post. Let's check the actual black levels (if you didn't do the math already):
Code: [Select]
octave:7> for i = 1:2, for j = 1:2,
>           obch = b(i:2:end,j:2:50);
>           disp(median(obch(:)))
>         end, end

Dial these values in darktable (the 4 black level sliders) and the image is back to normal :)

Homework for you: hardcode the above black levels in the DNG with exiftool / exiv2 / whatever you prefer.

Camera-specific discussion / Re: Canon 6D
« on: Yesterday at 10:07:43 AM »
There might be some unhandled case in our lossless decoder and maybe also in dcraw, or there may be some encoder misconfiguration.

Test #1 - you can use any build with lossless compression enabled:

To find out whether the fault is in our decoders, or on the encoder side, try loading a lossless silent DNG in pic_view from file_man. Only run this if mlv_dump or dcraw output is obviously wrong (maybe as in the above example, maybe with other defects).

- Looks fine? The problem is in our PC-based decoders (dcraw, mlv_dump).
- Image from pic_view has the same defects as with dcraw/mlv_dump? The problem is in the encoder configuration (lossless.c).
- Image from pic_view broken, or playback doesn't work? Look at Canon's lossless decoding stubs, or at the way we call it from pic_view/mlv_play.

Test #2 - you need to compile a custom

To check whether this bug affects your camera (applies to any model that supports lossless compression), or to investigate and hopefully fix it, follow these steps:

Step 1: save some image as both 14-bit lossless *and* 14-bit uncompressed (it *must* be the same image, not just the same static scene):
Code: [Select]
diff -r 8ee7858f0d7e modules/silent/silent.c
--- a/modules/silent/silent.c
+++ b/modules/silent/silent.c
@@ -560,3 +560,5 @@
             if (!ok) bmp_printf( FONT_MED, 0, 83, "DNG save error (card full?)");
-            return ok;
+            extern void reverse_bytes_order(char*, int);
+            reverse_bytes_order(raw_info->buffer, raw_info->frame_size);
+            /* fall through */

Step 2: compare the raw data and check whether there are any differences (on 6D and 650D, there will be; on other models, hopefully not):
Code: [Select]
dcraw -4 -E 12340000.DNG 12340001.DNG
md5sum 12340000.pgm 12340001.pgm

Step 3: if the two checksums are different, investigate the differences (tutorial):
Code: [Select]
octave:1> a = imread('12340000.pgm');
octave:2> b = imread('12340001.pgm');
octave:3> manual inspection, imshow(a - b, []) etc

Step 4: adjust the lossless decoder (either the one from dcraw, or lj92.c from mlv_dump) or the encoder configuration (lossless.c) until these differences disappear (how? I don't know, you need to experiment)

Step 5: repeat for 650D and all other models (I've ran this test on 5D3, got identical checksums at step 2)

Good luck!

Camera-specific discussion / Re: Canon 650D / T4i
« on: Yesterday at 08:28:28 AM »
ResumeLiveView goes back to LiveView from the "paused LiveView" state (an artificial state not present in Canon firmware, but used by ML for various things, such as powersaving, refreshing LiveView or full-res silent pictures).

On the other hand, force_liveview is much older - it goes to LiveView from any camera state (photo mode, Canon menus, whatever).

edit: got another test for you to run, for the lossless compression routines.

Camera-specific discussion / Re: Canon 650D / T4i
« on: October 17, 2017, 01:06:14 PM »
- ISO far from anything set in cam. ISO 100 (in cam) -> ISO 380 in Exif Or 235. Or else.

If you are in photo mode, with any non-fullres pictures, that's the right behavior. It records the ISO, shutter and aperture used for the captured LiveView frame, which are different from what you have set in Canon menu. Try setting the exposure time to 1 second - is LiveView refreshing once per second? Of course not.

In movie mode, the values should match those printed by ML (but not necessarily those printed by Canon).

- Screen frozen with message "Saving [...]". Unlockable by pressing half shutter

Full-res or simple? edit: simple. Normally it should return to LiveView...

Try this in 'don't click me':
Code: [Select]
static void run_test()

With Simple and Full-res works as assigned: AE button runs AF, Half Shutter actuates silent pic capture. But not with Full-Res LV. Now *both* buttons trigger silent pics.

AF detection code is common to simple and full-res versions. Don't know why it works with one, but not the other, but short AF presses (if focusing doesn't have time to start) are likely to be mistaken for half-shutter presses. All these buttons send the same GUI event - ML tells the difference by looking whether autofocus starts or not, and this always happens some time *after* the half-shutter press event (100ms or so). So you need long press for autofocus.

Camera-specific discussion / Re: Canon 650D / T4i
« on: October 16, 2017, 08:07:27 PM »
I'm just using the 700D SFDATA. This camera doesn't even have a ROM0 - QEMU asks for it because it's one size fits all (you can supply any 32MB file).

The biggest issue is missing lossless encoder emulation (shouldn't be very difficult, as the hard part was already done in mlv_dump, but it's not a beginner project either). It's on my list, but after getting qemu in mainline.

Camera-specific discussion / Re: Canon 650D / T4i
« on: October 16, 2017, 03:14:11 PM »
One way would be running the lossless compression routine in QEMU and comparing I/O traces to 700D/100D ones to see if there's anything obvious missed. That should also be useful for diagnosing the full-res lossless bug - is our code running out of memory, or maybe Canon's encoder has limits on maximum image size?

Compressing some arbitrary*) data and comparing it to the original, outside LiveView, may also be helpful. Maybe you'll find it works only before taking a CR2 picture, or only after, or maybe after a JPG one, or only after entering LiveView, or...

*) Synthetic data may help (e.g. a small matrix filled with 1 2 3 4 ... or any other known pattern).

In other words, trial and error (detective work).

A startup log (dm-spy-experiments branch) with interrupts, engio messages and ResLock messages enabled, covering one still CR2 picture, can be also helpful (background reading). This branch doesn't compile out of the box on 650D though (some stubs may be missing).

Side note: 5D3 has an interesting bug: if you take a CR2 image while the camera is in movie mode, our lossless compression functions stop working. If you take it in photo mode (LiveView or not), the lossless functions are not affected. I wasn't able to diagnose it with current tools (e.g. adtg_gui or DebugMsg logging).

Camera-specific discussion / Re: Canon 650D / T4i
« on: October 16, 2017, 01:30:42 PM »
Ok--we're good then? There seems to be a problem with the lossless DNG from Walter's 650D test. Did I get an address wrong?

Or maybe the 650D isn't nearly the same as 700D - answered in #1943.

Camera-specific discussion / Re: Canon 650D / T4i
« on: October 16, 2017, 12:01:19 AM »
So this DNG is the same size as the firs (uncompressed) DNG. Can't be.

That's OK - raw_diag only saves uncompressed 14-bit. Look at Walter's samples - both RAW.DNG's are fine (same size, different levels).

Latest qemu branch on a fresh Mac VM:

Virtualization  8)

Code: [Select]
ruby -e "$(curl -fsSL"
brew install mercurial

hg clone
cd magic-lantern
hg update qemu -C
cd contrib/qemu

Thanks kichetof 8)

Camera-specific discussion / Re: Canon 650D / T4i
« on: October 15, 2017, 08:27:43 PM »
Uncompressed files are clean; the banding defect is only on the losslessly compressed DNG. It may require additional fine-tuning on registers related to resolution (tuning that apparently is not needed on other cameras).

The 12-bit DNG is fine (no white level issues).

For skip offsets, the 700D/100D ones are a better match - we should unify them.

Can you also try a full-res silent lossless picture?

Camera-specific discussion / Re: Canon 650D / T4i
« on: October 15, 2017, 09:44:53 AM »
Hm... okay, you can patch it in mlv_lite.c:raw_digital_gain_ok (for a quick test, just return 1).

Does it help if you disable lossless compression?

Just a guess: Canon's image compression module might be in use by the remote control app (to my knowledge, the camera sends the LiveView image as JPEG). As a result, ML might not be able to use lossless compression in this mode.

Please try to answer the questions from #2.

Are things any better with the regular nightly build?

The crop_rec_4k build uses a bunch of memory management trickery, including using some memory without telling Canon code that we have allocated it. I've checked whether these areas are unused with built-in LCD and with HDMI monitor, but not with USB apps.

Are these issues reproducible with any smartphone app that controls the camera via USB?

Camera-specific discussion / Re: Canon 600D / T3i
« on: October 14, 2017, 03:52:00 PM »
Yes, a video would be nice.

Unfortunately, the audio side of 600D is different from all other modes, so it requires a 600D owner with programming skills to look into it. Also check the new-sound-system branch, maybe the issue is fixed there. I'm afraid diagnosing this issue without a camera is going to be more difficult than asking a tester whether it works.

Maybe if QEMU will ever be able to emulate the audio chip and Canon's LiveView. Docs are available, manpower is always welcome.

Camera-specific discussion / Re: Canon 100D / SL1
« on: October 14, 2017, 12:04:57 AM »
Interesting - I have a feeling overriding this to 30 or lower might give better battery life in photo mode.

Camera-specific discussion / Re: Canon 100D / SL1
« on: October 13, 2017, 11:34:58 PM »
Quick question: how many FPS do you have in photo mode by default?

(look in the FPS override submenu, without enabling the main entry - just read the grayed out values, including FPS timers)

The correct skip offsets are already in raw.c - your approach requires duplicating them, does not reduce the number of branches and... you have a typo in one of the offsets ;)

The current solution does not override the offsets for these models (these offsets happen to be the same as without crop_rec). Compare these two screenshots side by side.

This particular sequence works fine here. However...

sometimes the screen remains black when switching modes around (not sure how to reproduce); in this case, you don't need to go to photo mode; just press MENU twice (see first post).

On 5D3, the 50/60 3x3 preset is indeed 1x, but also with higher vertical resolution (unlike the smaller cameras).

Camera-specific discussion / Re: Canon 100D / SL1
« on: October 13, 2017, 08:38:31 PM »
Regarding the preferred raw type. Is this way to scroll trough raw types still valid?
in raw.c
Code: [Select]
#define RAW_DEBUG_TYPE   /* this lets you select the raw type (for PREFERRED_RAW_TYPE) from menu */It sure shows in the menu and I am able to change the register number but it seems it has no effect to the live view window.

That's right - it only changes the raw stream that gets recorded as MLV. It doesn't change Canon's image pipeline (so the end result is the same); it only picks some Bayer stream from a different location in their pipeline (at least that's my understanding). The raw type "CCD" (default) is probably the earliest in the pipeline, and the "DEFCORRE" one (used with 8...12-bit lossless) is after some preprocessing (such as digital gain with the SHAD module*), or defect correction, or whatever else Canon does).

*) I use SHAD_GAIN to darken the image (reducing the number of useful levels and therefore the bit depth).

There are many raw types, but where exactly they are in the pipeline, it's not known. What I know is summarized here.

The easiest way to test is to enable the Framing preview in mlv_lite (or raw_diag, if you prefer).

Camera-specific discussion / Re: Canon 600D / T3i
« on: October 13, 2017, 07:39:18 PM »
Hi I find that I have to go into the normal canon menu and tweak the sound level there then when I come out to live view the level metering is working but it is so un reliable [...]

Did a quick test, comparing ML audio meters to Canon's (5D3):

Unable to see what's wrong with them.

Additionally, a long time ago, a 60D user compared ML audio meters with a calibrated reference and found nothing wrong:
A -12dB tone recorded with my 60D (only the tip of the meters was yellow) resulted in a perfect -12dB tone. I checked all signals on our calibrated system.

Therefore, if you are looking for help, it's your duty to document the issue properly.

By the way, I've been using Danne's Switch app to process the MLVs and at one point I tried mlv_dump on the command line and Adobe Camera Raw stretched out the image--guess it was adjusting it as if it were a regular mv720 clip. However, this is also how it looks like on the LiveView.

Pretty sure it does that. Suggestion: if the MLV has RAWC metadata, this can tell whether the image requires any stretching (and if so, how much). The aspect ratio would be (binning_x + skipping_x) / (binning_y + skipping_y).

Camera-specific discussion / Re: Canon 650D / T4i
« on: October 13, 2017, 08:51:16 AM »
Do you see these artifacts on mlv_lite's Framing preview? If yes, that's some misconfiguration of the LiveView registers; to fix it, a 650D owner will have to use adtg_gui (Downloads - Modules) and fiddle with it to figure it out.

Does it make a difference if you select different data formats? (in particular, 14-bit lossless and 12-bit lossless).

To diagnose, I need you to run a little test (edited - the original scenario didn't work):
- prepare a static scene with some overexposed highlight in it (any amount will do)
- load mlv_lite, silent and raw_diag (the latter is at Downloads - Modules)
- set camera to movie mode, 720p
- enable RAW diagnostics, Optical Black + DR, Optical Black Zones, Dump RAW Buffer, Auto Screenshot
- don't enable raw video yet
- in LiveView, press shutter halfway for 1 second to capture the image; copy RAW.DNG to PC now
- enable RAW video, disable SRM memory, enable 12-bit lossless compression
- disable and re-enable RAW video just in case
- in LiveView, press shutter halfway for 1 second to capture a second image (also named RAW.DNG)
- disable RAW video and enable the Silent Picture option (simple)
- capture one regular DNG and one lossless DNG (these are auto-numbered, no need to fiddle)

So far, you've got two RAW.DNG, two numbered DNGs and four screenshots (PPM).

Repeat the above with crop_rec 3x3.

In total, you'll get 8 DNG images and 8 screenshots (way below 300MB).

Pages: [1] 2 3 ... 352