Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - a1ex

#1
Other than recording the sequence of button presses required to achieve the config... probably not. The ADTG menus are generated dynamically, as the registers are discovered; internally, they are an array of menu entries with the same name. As such, they are not accessible from Lua's menu API (at least, not at the time of writing).

The easiest way is to hardcode the preset directly in adtg_gui.c. There are a few examples in the source code, including a 3K preset for 5D3.
#2
The buffer is valid, but... for some reason, cr2hdr is trying to access some pixels at negative coordinates:


buf:0x4771028 x:-3616 y:0
==11898== Invalid read of size 2
==11898==    at 0x804F2E8: raw_get_pixel16 (cr2hdr.c:473)
...
buf:0x4771028 x:-3601 y:15
==11898== Invalid write of size 4
==11898==    at 0x804F360: white_balance_gray (cr2hdr.c:3589)
...


The problem comes from here (output size larger than full image size):

dcraw -i -v M22-1343_000000.dng
...
Pixel Aspect Ratio: 3.000000
Full size:   1808 x 2268
Image size:  1808 x 2268
Output size: 5424 x 2268


Note that exiftool identifies the active area correctly:

Active Area                     : 0 0 2268 1808
Image Size                      : 1808x2268


As cr2hdr attempts to detect the active area by parsing dcraw output, that's where it fails (that's the reason for trying to access pixels at negative coordinates). Using exiftool for autodetection is technically possible, and probably a better approach, but somebody has to write the code. I'm not able to help with this for the time being, but I'll take a look if somebody comes up with a patch.

Edit: apparently, using the "Image size" field from dcraw instead of "Output size" fixes the issue. Will commit the fix later this week.

In the meantime, you may want to try mlv_dump for preprocessing, as recommended in the Dual ISO topic:

mlv_dump clip.mlv --dng --no-fixcp --no-stripes
#3
You might be looking for this feature: https://wiki.magiclantern.fm/faq#where_is_movie_mode_remap

Currently not available in ML, but feel free to investigate.
#4
Reverse Engineering / Re: Battery grip pins / UART
April 10, 2021, 04:23:14 PM
EOS M2 has the same UART connector as 1300D (photo from Walter's; TX pins matching the above description from deviousfusion):




MON>>>
E1OFF



SD Detect High
K355 READY
128K Sector FROM From BL 0xffff
[SF] InstallSerialFlash 6 0xc022c0d4 0x0 0x1000000 1

[SF] Bufcon Base 0xc022c0d4
SerialFlash Initialize
     0:    55.279 [STARTUP]
K355 ICU Firmware Version 1.0.3 ( 6.0.6 )
     4:    57.627 [PROPAD] PROPAD_CreateFROMPropertyHandle DRAMAddr 0x416d5b00
     5:    58.313 [PROPAD] SerialFlash Packages!! 0x2d0000
...


760D appears to be similar (photo from igoro00 on Discord):


Matching cable also available on Digikey:
https://www.digikey.com/en/products/detail/jst-sales-america-inc/A06SUR06SUR32W51A/6009351
#5
Scripting Q&A / Re: Dual ISO and Lua
March 23, 2021, 10:11:04 AM
To some extent, yes, some menus will not toggle correctly from Lua. A proper fix requires major change in the menu code + testing every single menu entry (there are hundreds of these, if not one thousand, at the time of writing).

Long answer: https://www.magiclantern.fm/forum/index.php?topic=19932.0

Workaround: for Dual ISO, Canon settings are changed from shoot_task, shortly after changing the menu setting. As such, a delay of 1 second or so after changing the menu might do the trick (not tested). Other menu entries (for other features) might require different tricks.
#6
Alright, please ping me on IRC or Discord during the weekend.
#7
ROM files: keep them safe for now.

Just to double-check: the boot flags you have enabled in EosCard are EOS_DEVELOP and BOOTDISK, right? Also, is the boot flag still enabled in the camera ROM?

Do you happen to have (access to) a second camera with CF slot, in order to test the card? A card prepared with portable ROM dumper should run on any camera that has the boot flag enabled - from DIGIC 2 to DIGIC 8 at the time of writing.

I can also take a closer look over USB, as the camera is obviously alive, but let's exclude any user errors first.
#8
Just in case, make sure you have a backup of your ROM files (from ML/LOGS on any recent ML card that used to work).

Then, assuming you still have the boot flag enabled (that is, you didn't uninstall ML since the error happened), you should be able to make your card bootable and run the portable ROM dumper. If it doesn't work, try with a different card, and make sure there are no bent pins in the card slot.

"Update file cannot be found." might be the bootloader not recognizing the card. Try formatting it in the camera, then make it bootable (with EosCard or make_bootable.sh).

Once you are able to run the portable ROM dumper, use these builds to get a diagnostic log:
https://builds.magiclantern.fm/jenkins/job/startup-log/
https://builds.magiclantern.fm/jenkins/job/startup-log-mpu/

If possible, try to enter LiveView while the log is being captured. That should display any error messages from Canon firmware. Also try to capture a log while taking a picture.

Unusual colors in still photos could indicate a hardware issue on the image sensor side - but it doesn't explain why the card failed at the same time.
#9
There are a few problems with your request.

First, there is a dedicated thread for MLV App. It's linked on their Github page, too - no need to open another one:
Quote from: https://github.com/ilia3101/MLV-App
Ask questions on the Magic Lantern forum thread.

Second, the change you are talking about was made less than 24 hours ago. A build including that change will be, very likely, made as soon as the authors decide to do that - they don't seem to offer nightly builds, so they might prefer to do some kind of testing before releasing, or whatever (I didn't follow this closely). There's no need to rush the process, just because you want this feature... yesterday.

On the contrary - lack of patience, in particular, demanding/pushing attitudes, might have the opposite effect. "Remember that people do free work because it is fun, not to get stressed by strangers" ;)

Recommended reading:

https://teejeetech.medium.com/etiquette-in-open-source-projects-f302b6e19c38
https://snarky.ca/setting-expectations-for-open-source-participation/
#10
General Help Q&A / Re: Disable Error Logs
March 13, 2021, 09:58:44 AM
Lens errors, at least on DIGIC <= 5, come from the MPU. We don't know how to suppress them yet - I tried to work around ERR20 several years ago, but it didn't work.

I happen to have a faulty lens that also gives ERR01 and only works wide-open - so I took a quick look. Apparently, the MPU message related to this error is:

*** mpu_recv(06 05 03 29 01 00), from ff1bf420                     ; PROP 80030023
PropMgr:ff01300c:8b:16: DispError 1 ERROR Time 2020/11/10 13:42:40


The 60D appears to ignore ERR01 and continue shooting after half-shutter press, but the 5D3 does not. So, at least on 60D, it might be possible to somehow block that MPU message in order to block the error. However, I had no success doing that - display goes black and camera becomes unresponsive, but background tasks are still working.

Disabling the error logs saved by ML is straightforward, but won't help much, as these logs are saved in background, without delaying normal operation. Feel free to edit the source code (debug.c, ErrForCamera_handler) and customize it as needed.




Edit: after pressing half-shutter on the ERR01 dialog, what happens is that camera actually reboots. As such, ignoring this error must be done on the MPU side - possibly by reprogramming it. Short answer becomes: not possible to ignore this error.

The only workaround I can think of, is to use the camera without ML, so the camera can reboot as fast as possible...
#11
General Chat / Re: Applying for fiscal hosting
March 13, 2021, 08:08:37 AM
Right - Open Collective would have taken the same percentage. It's not something I'm worried about.

On the contrary, I believe the services offered by Conservancy are worth a lot more than that, as discussed earlier in this thread.
#12
General Chat / Re: Applying for fiscal hosting
March 13, 2021, 07:25:55 AM
Update: received an answer from Conservancy, regarding our application.

We still have some things to fix, the most glaring issue being project governance (or lack thereof), but the outcome is - in their words - "we're committed to get there" :)

Quote
Our first next step is to discuss the governance of the project and its representation in Conservancy to make sure we understand how decisions are made on behalf of Magic Lantern and to make sure Conservancy is set to interface well with the project. I know from the application you submitted that there's currently no formal decision-making on behalf of the project, so we'll have to design a process that fits with the way you all currently collaborate. Some of the easiest ways to do this are having an elected or self appointed committee, which we can discuss in greater detail.

As such, they invited us to a virtual meeting, together with the new and/or currently active developers. Trammell will be able to join us on Thursday evening. Meeting time wasn't fixed yet, but - my guess - it will be most likely between 18:00 and 21:00 GMT. For the meeting, we will use BigBlueButton, which works directly in browser - there's no need for the participants to install any custom software.

For reference, here's the agreement we'll have to complete - but after the meeting, of course:

https://sfconservancy.org/docs/sponsorship-agreement-template.pdf
#13
General Help Q&A / Re: App for viewing Code
March 09, 2021, 01:20:08 PM
Correction: for Bayer files, dcraw will output a PGM file (the grayscale version of PPM). But yes, they are reasonably easy to work with, for example with Octave scripts. There are a few examples on the forum - look up "pgm2dng" and maybe also "read_raw".

This file format is simple enough to be read/written with a few lines of C, without any dependencies - one of the reasons ML saves PPM screenshots. It doesn't handle metadata, though, but if you want to compare the image data from CR2/DNG files (for example, to make sure lossless is actually lossless), converting both files to PGM via "dcraw -4 -E" and comparing the contents works reasonably well. Caveat: these files might have embedded comments.

PGM file format: http://netpbm.sourceforge.net/doc/pgm.html
#14
General Help Q&A / Re: App for viewing Code
March 09, 2021, 07:52:33 AM
By "Bayer data", OP means Bayer image data, that is, image data which wasn't demosaiced. Look at e.g.:

Imatest docs -> "undemosaiced (Bayer) data"
Processing-RAW-Images-in-MATLAB-Sumner.pdf -> "Bayer CFA data"
Picamera Raw Bayer data captures -> "raw Bayer data recorded by the camera's sensor"
"bayer data" site:magiclantern.fm ... ;)

This kind of image data is what you get with "dcraw -4 -E". For a CR2, the pixel values output by that dcraw command are actually the exact values encoded in the CR2 file, after lossless decompression. For an uncompressed DNG (such as one saved by ML's silent picture module), the pixel values obtained by using that command are the exact values saved in the DNG. Same for a lossless DNG - pixels output by that dcraw command are the actual pixel values that were encoded when creating the DNG - without any image processing applied.

The only thing "dcraw -4 -E" does is to extract the image data (by parsing and unpacking/decompressing the input file), without applying any kind of image processing. That command will output the actual image data encoded in the CR2 file, into a 16-bit PPM PGM (which you can further process in Octave, for example). Or a 16-bit TIFF if you also use -T.

Quote from: man dcraw
[...] Use dcraw -E -4 to get the raw pixel values. [...]

-d     Show  the  raw  data as a grayscale image with no interpolation.
        Good for photographing black-and-white documents.

-D     Same as -d, but with the original unscaled pixel values.

-E     Same as -D, but masked pixels are not cropped.

With LibRaw, the above options were removed from dcraw_emu - apparently the unprocessed_raw command does what OP wants, but I didn't try it.

Now, the data that can be encoded into a CR2 or DNG might be either unprocessed / minimally processed by the camera (which is what Imatest calls "Camera RAW"), or it might be further processed - for example, our Dual ISO conversion tool (cr2hdr) outputs non-demosaiced (Bayer) image data as DNG. In the latter case, the image data is obviously not RAW.

So, if you know a better name for the nonexistent thing that we used to call "Bayer data", please enlighten us ;)




To summarize:
- to extract the unprocessed image data from a CR2 or DNG: "dcraw -4 -E" or "unprocessed_raw"
- to view the CR2/DNG metadata: exiftool (as answered previously) or exiv2
- to interpret the low-level CR2/DNG file contents yourself: hex editor, or... highly recommended: "exiftool -htmlDump".
#15
In other words, if you unload the modules while capturing a timelapse, you expect it to work well, right?

Testing this hypothesis would take a long time - but if it consistently works without error while the modules are not loaded (let's say no errors within a week), and it consistently fails with these two modules loaded (let's say at least 2-3 failures within a week), it's probably safe to blame the modules - or perhaps the module system itself.

Bulb timer itself could very well be cause problems - I haven't tested it extensively. Plain intervalometer - not expecting such surprises there.
#16
Got it. Are you able to reproduce the issue? Were older builds working any better under the same scenario? Can you share your exact settings (contents of ML/SETTINGS from the card), maybe the issue can be reproduced on a different camera model?

Those logs don't appear on their own, either - but generally, when a crash happens, a CRASH*.LOG file is expected to appear.

You may create a crash log from selftest.mo: Debug -> Fault Emulation -> AllocateMemory 1MB - click it a few times, until it crashes. It works in QEMU on 6D; expecting it to work on the camera as well.
#17
No obvious error. Are there any crash logs saved?

Auto Power Off (a setting in Canon menu) might turn off the camera while intervalometer is running (you'd get a warning in menu, if that's the case). Preventing this kind of shutdown from ML is possible, but it's not implemented yet for the intervalometer.
#18
Camera-specific Development / Re: Canon 70D
February 27, 2021, 09:43:55 PM
I've enabled it 3 years ago.

To date, I haven't received any confirmation on whether it works or not, except for your message. Mind providing some more details? I don't own this camera, and QEMU doesn't emulate this functionality, so I cannot check it myself.
#19
Camera-specific Development / Re: Canon 1100D / T3
February 27, 2021, 07:39:15 AM
Thanks, you did the tests correctly.

ERR70 from the stubs test appears to come from here (first fail):

       m0 = GetFreeMemForAllocateMemory() => 0x6361c
[FAIL] p = (void*)_AllocateMemory(128*1024) => 0x0


The memory backend reports 0x6361c = 407068 bytes available for AllocateMemory, but... attempting to allocate 128K (131072 bytes) fails for some reason. Best guess: memory fragmentation. Most of the subsequent failures are caused by the initial crash.

If you try to load only selftest.mo, and run the stubs test only with that module loaded, does it still crash?

Lua test worked for the most part, but crashed near the end (at "Setting aperture to random values"). Is this crash repeatable? If you run the same test a few times, does it crash in the same place?

Menu becoming transparent... is a feature, if you press LiveView while in ML menu (which might help when adjusting settings that affect the image).

When asking for M mode change, api_test.lua actually does attempt to beep, in the same way as when asking for half-shutter. If it didn't beep, there might be a problem with beeps as well. Beeps are not yet emulated in QEMU.
#20
Scripting Q&A / Re: Extra delay script for intervalometer
February 26, 2021, 04:47:34 PM
Quote from: surami on February 26, 2021, 03:05:35 PM
Will Auto ETTR take care of the shutter and ISO settings automatically?

In theory, yes. In practice, I don't remember trying anything similar during the past few years, so YMMV.

Something to keep in mind:

Before 2017: ETTR will ignore intervalometer settings, and might choose exposures longer than interval time.
After 2017: ETTR will not choose exposures longer than interval time.

Since you use long exposures, I'd recommend operating outside LiveView (in plain photo mode). There, Expo. override and ExpSim will have no effect.
#21
General Chat / Re: Any thoughts on this idea?
February 20, 2021, 11:50:59 PM
Something like this?


while true do
  if lens.focal_length == lens.focal_length_max then
    set_pixel_binning_mode(1,1)
  else
    set_pixel_binning_mode(3,3)
  end
  sleep(1)
end


The script won't run out of the box - one still has to expose crop_rec API to Lua and to hardcode (or find out) the focal length limits.

Since changing the pixel binning mode is not quite the same as digital zoom, the above script probably won't infringe the patent - but IANAL.

The following script - which implements KR's original idea, might be infringing, though:

while true do
  -- map: https://www.arduino.cc/reference/en/language/functions/math/map/
  zoom = map(lens.focal_length, lens.focal_length_max * 3/4 + lens.focal_length_min * 1/4, lens.focal_length_max, 1.00, 2.00);
  -- constrain: https://www.arduino.cc/reference/en/language/functions/math/constrain/
  zoom = constrain(zoom, 1.0, 2.0);
  -- 600D, maybe also 70D: Canon's "digital zoom" is either 1x (3x3 pixel binning with line skipping), or 3...10x (1:1 crop)
  -- set_digital_zoom can be implemented as: property.DIGITAL_ZOOM_RATIO:request_change(math.floor(zoom * 300), 4);
  -- it can be hacked to work continuously from 1x to 10x, but... it's kinda pointless ;)
  set_digital_zoom(zoom)
  sleep(1)
end


;)
#22
Scripting Q&A / Re: Extra delay script for intervalometer
February 18, 2021, 08:25:21 PM
Quote from: surami on February 18, 2021, 08:17:26 PM
It means that eg. at interval 5s and force delay 2s you get a 7s long extended interval?

... if your exposure time is 5 seconds, yes.

The script only enforces a minimum delay of 2 seconds between two pictures (with the above settings). Nothing more, nothing less.

If you want to take a picture every 7 seconds, simply dial 7 seconds in the intervalometer menu ("Take a picture every 7 seconds").




BTW, when used together with intervalometer, AutoETTR will limit its maximum exposure time to intervalometer delay minus 2 seconds. This is in the AutoETTR help text, but only appears when you enable both options. This change was merged into mainline in early 2017.

In other words, your idea of taking a picture every 5 seconds, and extending the exposure time to 32 seconds when it gets dark, will not work out of the box - the only way would be to write a custom intervalometer script in Lua.
#23
Scripting Q&A / Re: Extra delay script for intervalometer
February 18, 2021, 07:15:42 PM
That might work. The intervalometer ramping module (adv_int) might be useful as well.

Quote from: surami on February 18, 2021, 06:43:15 PM
AutoETTR always on, longest shutter 32s, max ISO 1600, fixed aparture 2.8 and interval 6s

To maintain a constant frame rate during the timelapse, I'd suggest a fixed interval of about 35 seconds between 2 pictures. ETTR is not going to ramp the shutter speed nicely, so with the above settings, you'll end up with a sudden jump in the frame rate.

In this case, you will have at least 3 seconds delay between two exposures out of the box (without the script).

Quote from: surami on February 18, 2021, 06:43:15 PM
As the exposure time exceeds the interval time there will be a problem, because ML would take a pic as quick as it can. I need the gap to have enough time for moving the pant/tilt head and enough time to chill the whole setup on the tripod.

In this case, the script works as expected, by enforcing a minimum delay between two pictures (detailed answer in my earlier post). I have tested it on 5D3 with both lua_fix and regular builds. I don't know what kind of signal you get through the hotshoe port, though - is it a logic signal active during the entire exposure, or something else?

For further help, please provide more details:

Quote from: garry23 on February 17, 2021, 12:01:10 AM
And, what's not working ?
#24
Scripting Q&A / Re: Extra delay script for intervalometer
February 18, 2021, 09:54:20 AM
Quote from: surami on February 18, 2021, 08:47:14 AM
If I set the interval to 5s and the force delay to 3s, there isn't 3s extra gap before the next shot, so the shot happens at the 5th second instead of at the 8th second.

That's how the intervalometer works - "Take a picture every 5s" means just that, regardless of how long it might take to capture a picture (plus any extra delays you might have set in the script). As long as that total time (image capture time + extra delays) doesn't exceed 5 seconds, of course.

Just like when recording video e.g. at 25 fps, the frame rate is constant, regardless of the exposure time you use for each frame, or whether you change the shutter speed while recording.

You may want to set the intervalometer to 8 seconds and to limit the exposure time to 5 seconds (e.g. from AutoETTR menu). That way, you'll have an additional delay of 3 seconds out of the box, without any trickery.

BTW, if you set "Take a picture every 5s" and "Force Delay: 10s", a picture will be taken every - roughly - 10 seconds (actually every 10 seconds + time required to capture the image + a small overhead). So, the script works as expected, enforcing the minimum delay between two pictures to 10 seconds, no matter what. Obviously, in this case, the intervalometer won't be able to keep up with the requested frame rate of 1 picture every 5 seconds.
#25
General Help Q&A / Re: Start up messages
February 08, 2021, 07:35:34 PM
These are some integrations between core ML code and some modules. For example, the intervalometer needs to know whether Auto ETTR is enabled (for some additional delay), and the raw histogram also asks Auto ETTR in order to display the ETTR hint. The ISO display in the LiveView info bars needs to know whether Dual ISO is enabled, in order to display it properly (e.g. 100/1600). And so on.

Pretty sure there are nicer ways to implement these, with less coupling, but I didn't know any better back in 2013. I'm no expert in software engineering, so any help in this direction is usually welcome ;)