Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - dfort

Pages: [1] 2 3 ... 93
Please note that some of the files (e. g. the ones shot at 10 and 12 bit uncompressed in the Movie Crop Mode) contain corrupt frames...

Add to that 14 bit uncompressed. That's strange, all of the uncompressed settings show corrupted frames in Movie Crop Mode. You should probably report that on the 100D topic. Maybe that "hiccup" build I made also fixes this issue?

I just processed everything into movie files and need to take a closer look but first impression is very good--at least as far as focus pixels are concerned.

...Now you understand why I am so eager to get the 100D working properly. 

Maybe a little less eager on the 7D? Getting 10/12bit working on that camera is a tough nut to crack.

Camera-specific discussion / Re: Canon 100D / SL1
« on: Today at 08:03:32 PM »
Could you do more tests? My experience shows that frames are missing.

We know there's a problem with the Dec19 build. Try the Hiccup_Test.2018Jan22.100D101 build I posted on my Bitbucket downloads page.

Camera-specific discussion / Re: ML on EOS-M2
« on: Today at 06:42:06 PM »
Thanks @Danne - Not much is working and it has a long way to go but it feels like we reached a milestone.

@a1ex - The dm-spy-experiments merge broke a few things so I think I'll make that a separate branch and keep the wip branch in its current somewhat working state so I can cherry pick the pieces to put into a new clean branch.

I was able to create a startup log with mpu events that might help with QEMU. The camera locked up on startup and I had to do a battery pull but here's the log it created. The lock up problem also happened on the EOSM so it isn't just an EOSM2 issue, it worked fine on the 700D.

Ok. This is a Dual ISO image? I'm surprised it looks this good. My understanding of how MLVFS fills in the focus pixels for Dual ISO is that it only samples the neighboring pixels on the horizontal axis. This is because Dual ISO has a different ISO every 2-pixels vertically. That's why it is difficult to remove the focus pixels on high contrast vertical lines. Someone please correct me if I'm wrong on this.

Camera-specific discussion / Re: ML on EOS-M2
« on: Today at 01:31:00 PM »
@a1ex -- A question about porting a new camera.

I would think that the goal is to eventually make a pull request in order to add a new camera to the unified branch so I created a "dummy" pull request in my ML fork:

Looks like I've got lots of things in there that aren't a part of the unified branch yet. Should I continue merging in branches to my work in progress branch or start over with a clean branch from unified and add in my changes? My gut feeling is that I should continue on my wip branch. The reason I'm asking is because I believe that the next step I should take is to merge in the latest dm-spy-experiments and make an in camera startup log. This might resolve issues we're having with QEMU like not recognizing some of the buttons and losing the GUI after going into Live View.

Then again maybe the goal is to make a pull request to the "bleeding edge" crop_rec_4k branch and bring in all the good stuff from the various branches as long as it doesn't break the other platforms?

I know we're far from a pull request, much less getting it approved and merged but thought I'd ask anyway. After all it is working--sort of. Let's just say it is crawling for now.

[EDIT] Speaking of QEMU issues, when running "./ EOSM2 -d debugmsg -s -S & arm-none-eabi-gdb -x EOSM2/debugmsg.gdb" it works fine but running "./ EOSM2 -d debugmsg" I'm getting an infinite loop like this on the Mac:

Code: [Select]
   252:  6418.176 WARN [I2C] localI2C_Write : 317 (Task : Startup)
[     Startup:ff349fd8 ] (14:02) I2C > [01] 02
[     Startup:ff356f04 ] (00:26) [I2C] localI2C_Write : 317 (Task : Startup)
[     Startup:ff34a020 ] (14:06) I2C abort 0(1020000)
   253:  6466.816 WARN [I2C] localI2C_Write : 317 (Task : Startup)
   254:  6466.816 [AUDIO] ERROR I2C abort 0(1020000)
[     Startup:ff349fd8 ] (14:02) I2C > [01] 02
[     Startup:ff356f04 ] (00:26) [I2C] localI2C_Write : 317 (Task : Startup)
[     Startup:ff34a0a4 ] (14:03) I2C_Write retry 1 1 c3

No issues with pausing if the terminal window is "too small" though.

Camera-specific discussion / Re: Canon 100D / SL1
« on: Today at 05:16:33 AM »
Uploaded a test build to my Bitbucket downloads page. Look for the "Hiccup Test" file.

What I did was the same as was done for the 5D3:

Code: [Select]
#ifdef CONFIG_100D

How did I come up with this?

Code: [Select]
    /* height may be a little different; 5D3 needs to subtract 1,
     * EOS M needs to add 1, 100D usually gives exact value
     * is it really important to have exact height?
     * for some raw types, yes! */

It looks like 1 wasn't added to the EOSM so I tried it and was able to get the full 2520x1080 resolution out of zoom mode but it had a "hiccup" at about the same rate as the 100D examples. It wasn't exactly the same kind of "hiccup" but let's try it out. If it fixes the "hiccup" issue maybe it could be refined so that it is only applies to zoom mode. Then again if it doesn't affect the other video modes we could consolidate the 100D with the 5D3.

Camera-specific discussion / Re: ML on EOS-M2
« on: Today at 02:13:42 AM »
It's Alive!

I've just refactored it to use CONFIG_NEW_DRYOS_TASK_HOOKS in internals.h.

That's all it needed. The Trash button brings up the ML menu on the camera, no need to hunt down the button code.

Of course this isn't the end of the adventure, there's still lots to do. The ML menus look a little rough, the screenshot feature doesn't work and [EDIT - Didn't work on the first boot but now it works] there are other obvious things that need fixing but hey--it is finally running on the camera.

BTW--Yesterday was my birthday and this is the best present!

Camera-specific discussion / Re: Canon 100D / SL1
« on: Yesterday at 04:25:06 PM »
how are you getting that resolution?

Getting closer to answering this question.

On the 100D zoom mode files the full raw buffer is a different height by 1-pixel and because the hight needs to be rounded to an even number the image size is different by 2-pixels.

100D zoom mode
Code: [Select]
Block: RAWI
    Res:  1920x1080 <-- image size
      height           1108 <-- full raw buffer
      width            2592

EOSM zoom mode
Code: [Select]
Block: RAWI
    Res:  1920x1078 <-- image size
      height           1107 <-- full raw buffer
      width            2592

I'm not sure if this is the cause of the "hiccup" issue but if I have a choice I'd rather have 1920x1080 than 1920x1078.

By the way, what I thought might have been happening was something that was discussed in this comment from the pull request that unified the 100D/650D/700D/EOSM skip offsets but all these resolutions are valid:

Code: [Select]
In [1]: 1920*14//8*1078 % 16
Out[1]: 0

In [2]: 1920*14//8*1080 % 16
Out[2]: 0

In [3]: 2520*14//8*1072 % 16
Out[3]: 0

In [4]: 2520*14//8*1080 % 16
Out[4]: 0

[EDIT] If you're wondering what's going on with resolutions between 2520x1072 and 2520x1080:

Code: [Select]
In [5]: 2520*14//8*1074 % 16
Out[5]: 4

In [6]: 2520*14//8*1076 % 16
Out[6]: 8

In [7]: 2520*14//8*1078 % 16
Out[7]: 12

These are not valid.

Camera-specific discussion / Re: ML on EOS-M2
« on: Yesterday at 07:39:37 AM »
Took another step forward -- got a SFDATA.BIN dump from the camera. I still can't get to the ML menu so here's how I did it.

In order to load modules on startup simply add an empty file with the name of the module in the ML/SETTINGS directory. For the sf_dump module it has to be named like this:

Code: [Select]
Now in order to get the module to auto run without having to start it from the Debug menu I did this hack to sf_dump_init():

Code: [Select]
//    menu_add("Debug", sf_dump_menu, COUNT(sf_dump_menu));
    return 0;

It also worked in QEMU but since we've been using a 100D SFDATA.BIN all that happened was that it copied the 100D file to the card image. Now I've got a "real" EOSM2 SFDATA.BIN to work with.

My main development system is a Mac PowerBook and since much of what I've been running in QEMU is not on the qemu branch I've had to unmount the disk image manually. Doing "make install" would trigger the "Error: please unmount the SD image." even though the disk didn't appear as a mounted volumes. Turns out that the fix was to simply "eject" instead of "unmount" -- this works great with both physical cards and disk images. According to the hdiutil man page:

NOTE: unmount does NOT detach any disk image associated with the volume.
          Images are attached and detached; volumes are mounted and unmounted.

Pull request submitted on the qemu branch.

Great news @martinhering -- you've been missed!

Are you referring to the colored area under the knobs?

These aren't focus pixels. You should get similar artifacts on the 7D (camera no focus pixels) as you do on the 100D (camera with focus pixels). That's why some ML users won't shoot without a VAF filter on their camera. Well, except for 5D3 users, note that you can't even get a VAF filter for the 5D3.

Camera-specific discussion / Re: ML on EOS-M2
« on: January 21, 2018, 10:30:33 PM »
Fixed task_dispatch_hook so it matches the changes on the 1300D and 6D. I couldn't follow the comments on the task_dispatch_hook for those cameras but I used my best pattern matching skills and also checked against the 100D and 70D so I'm pretty confident that I've got this right. By the way, the EOSM doesn't fall into the same pattern as the other cameras. Is new-dryos-task-hooks working properly on the EOSM?

In the process I also found a typo on task_trampoline and fixed it.

Code: [Select]
/** Tasks **/
NSTUB(   0x8FBE0,  task_dispatch_hook)                      // changed from 0x8FAB4 - now it doesn't work on the camera.
NSTUB(0xFFD24B44 - RAM_OFFSET,  task_create)
NSTUB(0xFFD2A1D8 - RAM_OFFSET,  task_trampoline)

However, as you can see from my comment it no longer works on the camera. LED turns on and camera won't boot up. So there must be something else that needs fixing. I know there are other things to fix. The lens focal length is displayed as 0 thought the f/stop and shutter speed is fine.

So it is one step forward and two steps back.

Still to do:

...a tiny hack in menu.c to bring menu in QEMU (not needed for camera), and if there are any buttons not recognized, just print their codes.

Big discovery for today is that adding this line to MAGIC.CFG the camera boots up with the console open.

Code: [Select]
module.console = 1

Now how to get the console to show button events? I'll bet the answer is to get the task hooks working properly.

Camera-specific discussion / Re: ML on EOS-M2
« on: January 21, 2018, 04:06:15 AM »
Next steps were covered before: new-dryos-task-hooks (done for 1300D, some stubs apparently wrong on M2)

I tried merging new-dryos-task-hooks again and this time it looks like it worked. At least "Hello World" is running--tested on the camera too.

You can now see DryOS tasks switching if you compile with CONFIG_QEMU=y and you enable DEBUG_TASK_HOOK in boot-hack.c. Without the latter, only new tasks will be displayed.

Did that too. Don't know what exactly to look for but there are a lot more messages flying through the terminal when run in QEMU.

So--one small step.

Camera-specific discussion / Re: Canon 100D / SL1
« on: January 20, 2018, 09:56:14 PM »
...high-quality video recording is possible at resolutions up to 2520x1080....

Just realized--how are you getting that resolution? The zoom mode file @canneloni did is 1920x1080. The maximum horizontal resolution I'm seeing is 2520 and the max vertical is 1078 using mlv_lite:

EOSM = 2520x1072, 1920x1078
700D = 2520x1072, 1920x1078

I'm not sure how the update_resolution_params function in mlv_lite works but it seems to me that the 100D should have the same combination of image sizes as the EOSM and 700D.

Camera-specific discussion / Re: Canon 100D / SL1
« on: January 20, 2018, 08:36:55 PM »
Hum--I was in the unified branch when I looked up the skip values and it seems that Matthew Kelly's commit for the 700D has already been applied to the other cameras in the crop_rec_4k branch. Edited my previous post.

[EDIT] Shoot -- I made the pull request that made that happen. No wonder it looked familiar. Must have had a senior moment when I wrote that post.

I couldn't reproduce the "hiccup" issue on the EOSM or 700D.

Camera-specific discussion / Re: Canon 100D / SL1
« on: January 20, 2018, 07:05:49 PM »
What were the exact settings you used?

I know that it could get confusing because there are several "crop" modes but you're using zoom mode which is set when the Canon menu is at 1920x1080 and you press the magnifying glass button until you get into 5x magnification. However, to record the largest sized image you need to use one of the reduced bit settings and lossless compression. Is this happening at 14bit lossless or only 8...12bit lossless? Maybe this has something to do with it?

...8...12-bit lossless modes (they use raw type 0x12 DEFCORRE, other modes use 0x10 CCD aka the first step in the pipeline).

Maybe there is a problem using 0x12 DEFCORRE in zoom mode? Maybe experimenting with a different PREFERRED_RAW_TYPE?

Code: [Select]
#define RAW_DEBUG_TYPE   /* this lets you select the raw type (for PREFERRED_RAW_TYPE) from menu */

There's a "warning FIXME" in the skip values so maybe this needs to be changed?

[EDIT] I was looking at the unified branch, this was fixed in the crop_rec_4k branch but I'll leave what I posted here because maybe this is causing some issues? Though I doubt it.

Code: [Select]
        #if defined(CONFIG_650D) || defined(CONFIG_EOSM) || defined(CONFIG_100D)
        #warning FIXME: are these values correct for 720p and crop modes?
        skip_top    = 28;
        skip_left   = 74;
        skip_right  = 0;
        skip_bottom = 6;

From the files I've been looking at for the cameras with the focus pixel issue it seems to me that it should be:

Code: [Select]
       skip_top    = 28;
        skip_left   = 72;
        skip_right  = 0;
        skip_bottom = 0;

Note that the skip values are different on the 700D. These cameras are very similar so maybe this should also apply to the 100D/650D/EOSOM?

Code: [Select]
        #ifdef CONFIG_700D
        skip_top    = 28;
        skip_left   = 72;
        skip_right  = 0;
        skip_bottom = zoom ? 0 : mv1080crop ? 0 : 4;

Could it be that the skip_bottom in zoom mode that might be causing the "hiccup" issue?

Camera-specific discussion / Re: Canon EOS M
« on: January 20, 2018, 05:42:41 PM »
Short answer, the shutter-bug isn't persistent.

The shutter-bug only happens on certain cards when ML is loaded and you are using an EF-M zoom lens (don't know if it is the zoom or the IS that is causing this). There are a few workarounds but the easiest is to use a SanDisk Extreme 32GB or smaller SD card. Others might or might not work. It is important that the card not be over 32GB.

Camera-specific discussion / Re: Canon 100D / SL1
« on: January 20, 2018, 05:29:42 PM »
Whoa--the Dual ISO lines on the highlights are egregious!

It would also be nice to get rid of the focus pixels that are on the edges of the frame on your shot:

Did some testing on the 100D. The 1:1 video modes, mv1080crop and zoom, could also have the focus pixels offset 8-pixels to the right. Discussion on the 100D topic.

Updated the map files for MLVFS, updated the MLVFS pull request and updated the script.

Camera-specific discussion / Re: Canon 100D / SL1
« on: January 20, 2018, 03:04:17 AM »
Ok, this is what is going on with the 12bit zoom mode and most likely also on the mv1080crop mode. [EDIT] This also applied to mv1080crop.

It doesn't matter if the current focus pixel map is used or the --no-fixfp in mlv_dump or if I remove all of the map files from MLVFS, the results are the same. Focus pixels pop up near contrasty borders.

These aren't quite as obvious as they are on the mv1080 test but they are there. They are usually green or magenta and fall into a pattern. (Click on the image for a closer look.)

It turns out that on these tests the pattern was shifted 8 pixels to the right. On the 100D the pattern repeats every 12 pixels so you can also say that it was shifted 4 pixels to the left. Is it something that happens all of the time or can it be in the "normal" position on one shot and shifted on another shot using the same settings? Since we've seen this happen before on other lossless compressed footage let's make a second pass shifting the map 8 pixels to the right:

I'll check the mv1080crop shots and run some tests on the files from @canneloni and @IDA_ML submitted for testing and if everything checks out I'll update the focus pixel map files.

By the way, I'm using both mlv_dump (the latest version on steroids) and MLVFS on these tests along with dcraw and ACR so if you notice a color shift, it is probably just me using different apps.

You mustn't do that.

Most of what I do I mustn't do.

Ok, so the download link on the website took me to a Russian page but the next link went back to English--that was weird. I opened a QT account in order to run the installer, figured out what I wanted to install (there are lots of options) and when I finally got it working--

Code: [Select]
:-1: error: No rule to make target `FFmpeg/ffmpeg', needed by `MLV'.  Stop.
Oh yeah--go back and read the MLV App instructions. Ah ha, - unpack in `platform/qt/FFmpeg` and - go to tab project, add command line argument -j4 (for quad core)


Camera-specific discussion / Re: Canon 100D / SL1
« on: January 19, 2018, 12:58:26 AM »
Great test @canneloni !

I need to take a closer look but right away look at the psychedelic display of focus pixels on mv1080:

Compared to mv1080crop -- without fixing the focus pixels:

Thanks for this.

It even works with this EOSM crop_rec pre-RAWC shot if you play around with the options:

The binaries listed on the first post haven't been updated in a while and I wanted to check out the new focus pixel commits so I decided to compile the qt version on my Mac.

A couple of important steps:

Code: [Select]
brew install qt caskroom/cask/qt-creator
echo 'export PATH="/usr/local/opt/qt/bin:$PATH"' >> ~/.bash_profile

You also have to tweak the preferences:


Pages: [1] 2 3 ... 93