Author Topic: Development Thread: Footage for Mac (New Mac app for MLV)  (Read 54261 times)

cpc

  • New to the forum
  • *
  • Posts: 37
Re: New Mac app for MLV (Alpha 1)
« Reply #50 on: February 27, 2017, 12:53:00 PM »
Looks like the white level is too high, you should be able to remedy this by pulling it down.

martinhering

  • Member
  • ***
  • Posts: 120
Re: New Mac app for MLV (Alpha 1)
« Reply #51 on: March 01, 2017, 07:41:08 AM »
@cpc

Quote
Looks like the white level is too high, you should be able to remedy this by pulling it down.

What do you exactly mean with "white level"? Can you give an example?
5D Mark III, EOS M, 700D

cpc

  • New to the forum
  • *
  • Posts: 37
Re: New Mac app for MLV (Alpha 1)
« Reply #52 on: March 01, 2017, 02:54:07 PM »
Cameras have nonlinearities in the signal above some level. This usually leads to tints and improperly reconstructed highlights when this part of the range is used for recovery. You'd need to restrict the upper limit and discard some of the signal. This is what the white level metadata is used for, it tells the raw processor where to cut. Ideally this is calibrated per camera, but you can usually get the approximately correct values by running some stills through exiftool and seeing what's supplied by the camera manufacturer.

martinhering

  • Member
  • ***
  • Posts: 120
Re: New Mac app for MLV (Alpha 1)
« Reply #53 on: March 02, 2017, 09:25:18 AM »
@cpc @budafilms

Unfortunately I can't fix problems in the RAW module myself, since I am using Apple's RAW engine that I have no control over. I sent a bug report to Apple however. I tried changing the white level and clamping raw values to lower levels, but that only makes the problem worse. I found this post (http://www.darktable.org/2016/10/raw-overexposed/) and developed a custom Core Image filter that does the trick. From the raw data, I am now creating another raw image that only contains luminance data for the highlights above 0.9 (normalized). I then take this image as a mask and desaturate the highlights.

Before: https://www.dropbox.com/s/fmybr0szer04geg/magenta_before.png?dl=0
After: https://www.dropbox.com/s/ayqp3vsq510nxz3/magenta_after.png?dl=0

This gives a descent result until Apple fixes its raw engine.
5D Mark III, EOS M, 700D

reddeercity

  • Hero Member
  • *****
  • Posts: 1634
Re: New Mac app for MLV (Alpha 1)
« Reply #54 on: March 02, 2017, 09:37:48 AM »
Nice work around , looks good to me .

Sameto

  • New to the forum
  • *
  • Posts: 1
Re: New Mac app for MLV (Alpha 1)
« Reply #55 on: March 02, 2017, 12:50:36 PM »
Wow!!! I'm downloading right now!!! Great work man[emoji2]

aschille84

  • Freshman
  • **
  • Posts: 72
Re: New Mac app for MLV (Alpha 1)
« Reply #56 on: March 02, 2017, 02:31:22 PM »
Couple of export suggestions: change fps (i.e timelapse, slomo), up-/downscaling (4k, 720p etc.)

martinhering

  • Member
  • ***
  • Posts: 120
Re: New Mac app for MLV (Alpha 1)
« Reply #57 on: March 02, 2017, 04:01:21 PM »
I am also making progress in dealing with focus pixels. Please see http://www.magiclantern.fm/forum/index.php?topic=16054.msg180682#msg180682. Would be great if anybody can post recordings so that I can map out the focus pixels for different cameras. EOS M I already have.
5D Mark III, EOS M, 700D

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3081
  • 5D3 / M1 / 7D / 70D / SL1
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

martinhering

  • Member
  • ***
  • Posts: 120
Re: New Mac app for MLV (Alpha 1)
« Reply #59 on: March 02, 2017, 05:37:20 PM »
Thanks. I checked out a few files. But they are not very suitable for focus pixel map extraction.

Short tutorial:
Remove the lens and point the camera to the sky or put a white paper in front of the camera. Then expose the image so that you have 98% white on screen. The white should not clip. I just need 1 frame. You can stop after 1 second. Also I need the highest resolution possible with the camera in fullframe and crop mode.

UPDATE:
I found a few suitable files. But is this the highest possible resolution? Can you tell me the resolution and mode you have been recording which files with?
5D Mark III, EOS M, 700D

dfort

  • Developer
  • Hero Member
  • *****
  • Posts: 3030
Re: New Mac app for MLV (Alpha 1)
« Reply #60 on: March 02, 2017, 06:29:59 PM »
The 70D doesn't show focus pixels but the 100D has a boatload.
5D3.* 7D.206 700D.115 EOSM.203 EOSM2.103 500D.112

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3081
  • 5D3 / M1 / 7D / 70D / SL1
New Mac app for MLV (Alpha 1)
« Reply #61 on: March 02, 2017, 06:36:45 PM »
Here's another quick run at their largest resolutions possible for each mode per your request. Used a beige colored wall.

70D: (Regular, Movie Crop-Mode, 5x zoom) https://mega.nz/#!DpkW3TgJ!-Cvy86JmmPg2TjgiOqgIim7RfQUxdq5ny2u_SGkodnE

100D: (Regular & 5x zoom) https://mega.nz/#!K0dAWQLD!CcDM6Fw0_YTFUIN4sp6IAHGPKIhGIPIAbmpmApcC_u0

The 70D doesn't show focus pixels but the 100D has a boatload.

D'oh and thanks for the friendly reminder, @dfort as I haven't been playing w the 70D as of late!  :)
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

martinhering

  • Member
  • ***
  • Posts: 120
Re: New Mac app for MLV (Alpha 1)
« Reply #62 on: March 02, 2017, 08:30:53 PM »
Quote
The 70D doesn't show focus pixels but the 100D has a boatload.

Yes, I can confirm that. However your 70D has lots of hot/dead pixels in all channels. Here are some images of the channels as their 8bit linear representation.

R: https://www.dropbox.com/s/6vc80iqc7igafq8/raw_red.tiff?dl=0
G: https://www.dropbox.com/s/t7i1min1830x8ez/raw_green.tiff?dl=0
B: https://www.dropbox.com/s/d9f8pdxxnvne3kp/raw_blue.tiff?dl=0

The good news is that their value is way below the black_level, which should be detectable fairly reliably.

The 100D has indeed a lot of focus pixels. I'll check your MLVs and see if I can reconstruct the pixel map.
5D Mark III, EOS M, 700D

fvanlint

  • New to the forum
  • *
  • Posts: 2
Re: New Mac app for MLV (Alpha 1)
« Reply #63 on: March 04, 2017, 04:37:03 PM »
Wow, this is awesome! This could app be a breakthrough for MLV workflow ease! I will keep close tabs on the development of this app. Will you start selling this app in the future?

martinhering

  • Member
  • ***
  • Posts: 120
Re: New Mac app for MLV (Alpha 1)
« Reply #64 on: March 05, 2017, 03:41:28 PM »
Quote
Will you start selling this app in the future?

Yes, I intend to. But only as soon as the app has enough compelling features. Until then the app will be free.
5D Mark III, EOS M, 700D

Teamsleepkid

  • Member
  • ***
  • Posts: 209
Re: New Mac app for MLV (Alpha 1)
« Reply #65 on: March 05, 2017, 10:51:57 PM »
Anything new to download with focus pixel fix?
EOS M

martinhering

  • Member
  • ***
  • Posts: 120
Re: New Mac app for MLV (Alpha 1)
« Reply #66 on: March 06, 2017, 03:54:59 PM »
There is a new version out: 0.1.1 (77)
https://rink.hockeyapp.net/manage/apps/477081/app_versions/29

Change log:
- Improved white balance handling
- Added white balance picker
- Added filter for reducing discolored highlights
- Added 'Copy Image' menu item to copy current image into clipboard
- Added support for 10 and 12 bit RAW
- Added correction of dead pixels
- Added correction of focus pixels for EOS-M and EOS 100D
- Backported to 10.11 El Capitan

Please try it out. Especially correction of focus pixels and dead pixels. Thank you.

Also I need sample files (preferably silent images) from the 650D and 700D. If anybody can provide that, that would be nice. Thank you.
5D Mark III, EOS M, 700D

kontrakatze

  • Freshman
  • **
  • Posts: 72
Re: New Mac app for MLV (Alpha 1)
« Reply #67 on: March 06, 2017, 06:05:02 PM »
Works nice here (5d Mark III), good work!
A play button would be great and a way to save projects. Do you have a "vertical stripes fix" in mind?

Regards.

martinhering

  • Member
  • ***
  • Posts: 120
Re: New Mac app for MLV (Alpha 1)
« Reply #68 on: March 06, 2017, 06:08:11 PM »
Quote
Do you have a "vertical stripes fix" in mind?

My 5D Mark III does not show the problem of vertical stripes. Do you have this? Can you send me a sample MLV file that shows the problem?
5D Mark III, EOS M, 700D

Ilia3101

  • Senior
  • ****
  • Posts: 302
Re: New Mac app for MLV (Alpha 1)
« Reply #69 on: March 06, 2017, 06:30:54 PM »
Thanks for back porting, but as soon as I opened(I'm on 10,11), it only let me to open a video clip, then just froze up after that, without displaying it. Now when I open it, nothing works, and full screen mode just goes black most times I try, sometimes individual gui elements show up tho. I think something is broken :-X Will be incredible to have a fast app for raw video, also I have a question, as it uses the apple RAW engine and all that(I'm guessing same as Apple Photos), and I've found from experience that Apple's demosaicing has never been that great compared to Adobe and amaze, what do you think about implementing amaze demosaicing?
5D2

martinhering

  • Member
  • ***
  • Posts: 120
Re: New Mac app for MLV (Alpha 1)
« Reply #70 on: March 06, 2017, 06:48:38 PM »
@Ilia3101

Fixed that. Please use build 78:
https://rink.hockeyapp.net/apps/3ed6ecf60e684239a6aba3d407cf3935/app_versions/29

Quote
I have a question, as it uses the apple RAW engine and all that(I'm guessing same as Apple Photos), and I've found from experience that Apple's demosaicing has never been that great compared to Adobe and amaze, what do you think about implementing amaze demosaicing?

I already thought about that. Since 10.12 it is actually possible to add your own CPU code to Core Image filters, so implementing Amaze should be possible. But that's not on the priority list at the moment. But at some point, I will try to do that.
5D Mark III, EOS M, 700D

a1ex

  • Administrator
  • Hero Member
  • *****
  • Posts: 11475
  • 5D Mark Free
Re: New Mac app for MLV (Alpha 1)
« Reply #71 on: March 06, 2017, 07:07:09 PM »
I already thought about that. Since 10.12 it is actually possible to add your own CPU code to Core Image filters, so implementing Amaze should be possible. But that's not on the priority list at the moment. But at some point, I will try to do that.

Yay, so this app will eventually become open source :)

https://github.com/Beep6581/RawTherapee/blob/dev/rtengine/amaze_demosaic_RT.cc

DeafEyeJedi

  • Hero Member
  • *****
  • Posts: 3081
  • 5D3 / M1 / 7D / 70D / SL1
Re: New Mac app for MLV (Alpha 1)
« Reply #72 on: March 06, 2017, 07:31:27 PM »
My 5D Mark III does not show the problem of vertical stripes. Do you have this? Can you send me a sample MLV file that shows the problem?

Here's a shortened MLV file that shows stripes @ ISO 3200 zipped up (19.5 MB):
https://mega.nz/#!65lGmAbJ!K4QIKbv2yvwVIzYJyKNKSHLkSx-btiefptbm2W2okxI

Here's a shortened MLV DF file w caps on in same settings (20.9 MB):
https://mega.nz/#!31dWhZCS!pz2sEkRP0zjmEfvfyBtP95Jy0zHUefCtm98BT3gD1is

Also would you consider including some kind of a DarkFraming/FullFraming average processing within your app or no?

Just in case here are some sample DNG's of Pre DF, Post DF and Post DF + FF for comparisons for those are curious and want to play (8 MB):
https://mega.nz/#!SslF0TxS!fF44DQaStWkriM38wlWqwXHowQoSsX8qn2ZqySyFZbY
5D3.113 • 5D3.123 • EOSM.203 • 7D.203 • 70D.112 • 100D.101

reddeercity

  • Hero Member
  • *****
  • Posts: 1634
Re: New Mac app for MLV (Alpha 1)
« Reply #73 on: March 06, 2017, 10:50:43 PM »
There is a new version out: 0.1.1 (77)
https://rink.hockeyapp.net/manage/apps/477081/app_versions/29
Change log:
- Backported to 10.11 El Capitan
Yosemite 10.10.5 in the near future ? , I would  really like to try it out  :)

dia3olik

  • New to the forum
  • *
  • Posts: 23
Re: New Mac app for MLV (Alpha 1)
« Reply #74 on: March 07, 2017, 12:09:52 PM »
Martin, this is great!

Is it possible to implement a way to copy and paste developing settings from one file to another of the list?

It would help with multiple takes or clips taken in the same lighting conditions...