Messages

1

Post-processing Workflow / Re: MLVToMovie: MLV file converter for Mac OS X

« on: October 04, 2016, 02:03:43 PM »

Version 1.2.0 available:

October 4, 2016: Release 1.2.0:
- Flat dark support added.
- Calibration diagram added for help.
- Ask user for flat, dark and flat dark overwrite after averaging frames to save disk space.
- Wrong frame number solved with splitted files (>4GB if exFAT not supported).

I've added the diagram (see "Vertical stripes revisited (5D Mark III)" topic) as help to understand image calibration concept...


I'm now working on preset editor and HDR support.

2

General Development / Re: Vertical stripes revisited (5D Mark III)

« on: October 03, 2016, 12:08:21 AM »

Yes it's a second dark. Mathematically, each dark must respect the raw image it will be subtracted from:
- the dark "classic" must respect ISO and Exposure of the raw mlv you want to process.
- the flat dark must respect ISO and Exposure of the flat mlv.

I wish this diagram helps...


When you shot your flat mlv with a light diffusor, you get the same repeatable noise that you get in the raw mlv. If you subtract the dark from your raw mlv you get no more repeatable noise. Now, if you apply a simple flat you inject the repeatable noise it also contains. As a result, the best way is to subtract a "flat dark" to remove repeatable noise from flat before the flat division.

I hope my explanation (and my English) is good enough. Lol 

3

General Development / Re: Vertical stripes revisited (5D Mark III)

« on: September 29, 2016, 10:04:44 PM »

Danne, yes it's a simple dark. flat dark is a dark shooted with the same parameters (Speed and Iso) as your flat. It's the same concept as the dark witch must use the same parameters as the raw images. It's really easy to do. After recording your flat as usually, you put the lens cover and launch a second record.

Vignetting issue with my shoot is only due to the fact that my focal reducer on my telescope limits the field so there is no more light to calibrate in the borders. You do not encounter such an effect with standard lens.

If you look to this video (raw-dark)/(flat-dark):

The field of view is quite perfect in the image circle. There is only random noise (no more vignetting and no more vertical or horizontal stripped lines). It's not the case with the other videos.

4

General Development / Re: Vertical stripes revisited (5D Mark III)

« on: September 29, 2016, 08:26:36 PM »

Everything works fine to me with mlv_dump. I don't think this job has to be handle by mlv_dump. We just need to know we have to be aware for the flat dark subtraction. I'll update my GUI.

I've processed a movie from this morning (very poor seeing but it doesn't matter for tests). Here is the first frame with different calibration...
http://astronomy.eliotis.com/tests/m29-0657-6400Iso-calibration-test.zip
- (raw-dark)/(flat-dark): tmp000000_raw-dark-div-by-flat-dark.tiff
- (raw-dark)/flat: tmp000000_raw-dark-div-by-flat.tiff
- raw-dark: tmp000000_raw-dark
- raw/flat: tmp000000_raw-div-by-flat
- raw (No calibration): tmp000000_raw

And here are the movies to see more easily random and constant noise:
- (raw-dark)/(flat-dark):

- (raw-dark)/flat:

- raw-dark:

- raw/flat:

- raw (No calibration):

Do not take care of corners. My focal Reducer/Corrector reduces the field. As a result, flat can only improve noise in the borders.

In my opinion, I do not recommend using flat without flat dark. But as you say a1ex, It can be less significant with high lights.

Danne, if you want to test. Flat dark could be the ultimate trick to remove focus pixels perfectly (or not).

5

General Development / Re: Vertical stripes revisited (5D Mark III)

« on: September 29, 2016, 12:55:08 PM »

A grab from "The Handbook of Astronomical Image Processing" by Richard Berry & James Burnell to illustrate my last message...

6

General Development / Re: Vertical stripes revisited (5D Mark III)

« on: September 29, 2016, 09:43:27 AM »

I've just tested flat calibration this morning on the crescent moon. As far as I can see, the dark signal is not removed automatically from the flat field when applying -s and -t options together. It's not really a mistake but the good mathematical process should be:

Calibrated raw = (raw-dark)/(flat-dark)*(average value of flat-dark)

As a result, take care when using flat calibration. We have to remove dark from flat with mlv_dump before using flat...

mlv_dump -s dark.mlv -o flat-dark.mlv flat.mlv

Otherwise noise will be reintroduced by the flat.

-I hope my explanation is clear enough.-

7

Post-processing Workflow / Re: MLVToMovie: MLV file converter for Mac OS X

« on: September 22, 2016, 04:55:08 AM »

Workaround published...

September 22, 2016 - Release 1.1.1:
Security update: adds a subdirectory MLVToMovieTmp when user want to use a custom temporary directory. It'll avoid bad surprises if user selects accidentally its Desktop directory or other personal directories. Cause temporary directory is automatically deleted, user could loose personal data. Now, only the MLVToMovieTmp directory will be deleted.

8

Post-processing Workflow / Re: MLVToMovie: MLV file converter for Mac OS X

« on: September 22, 2016, 04:25:15 AM »

khuong, I'm so sorry. temp folder, as its name says, is a temporary folder. The application needs to delete it automatically because MLV file extraction uses a lot of disk space.

Temp files do not go into the user's trashcan. To restore data, you'll need your Time Machine backup or a recovery application like Mac Data Recovery.

I'll publish immediately a workaround to avoid user's manipulation error.

9

Post-processing Workflow / Re: MLVToMovie: MLV file converter for Mac OS X

« on: September 18, 2016, 10:26:50 PM »

Thank you guys.

reddeercity, yes my first need was to render lossless for astronomical process so It’s big and QT have synchronization issues when displaying. Very bad flicker with DualIso? With lossles it should be at least beautiful flicker! I joke of course. Dual Iso is not supported yet but I'll try in a next release.

According to your feedbacks, I’ve spent a little bit of time on release 1.1.0 today so here it is:
-Drag & drop support of MLV files and directories (recursive scan) into the sources-list.
-Drag & drop support of MLV files and directories (recursive scan) on app icon.
-Allow directories selection with open file panel too -> selected directories are automatically scanned recursively for MLV files.
-Temporary directory can be edited in preference panel.
This release is already available on my website.

Now I'll work on news features soon. Here is my road map:
- a preset editor for FFmpeg exports so we should be able to export other movie formats.
- more DCRaw options (color space, saturation, brightness, gamma curve, Linear mode, white balance, 8 or 16 bits export for RGB conversion).
- DualIso support.

Few questions (I'm not a video master at all as I said):
- What's LUT and CINELOG?
- Lars Steenhoff, what do you mean by "Debayering to 4k"? Is there a Canon camera able to produce 4K with Magic Lantern yet?

10

General Development / Re: Vertical stripes revisited (5D Mark III)

« on: September 15, 2016, 07:16:32 PM »

Thank you guys! I've just created a new topic for MLVToMovie here...
http://www.magiclantern.fm/forum/index.php?topic=17893

Danne, yes I'm using MLV_Dump, DCRaw and FFmpeg. I don't think to make it open source cause it's a signed application and I'm using personal proprietary code for multi-tasking but I can send you code source for study. Send me a private message with you mail if you want.

Lars Steenhoff, I'm not a video master but if your needs can be done with FFmpeg I can look at it. Feel free to switch on the MLVToMovie topic to talk about It.

11

Post-processing Workflow / MLVToMovie: MLV file converter for Mac OS X

« on: September 15, 2016, 07:08:35 PM »

Let me introduce my own little software available today for the community... 
http://mlvtomov.eliotis.com



Specifications:
- Native Mac OS X application written in Objective-C.
- Queue management to process as many files as you want..
- MLV files can be added to the queue from different directories.
- Movie files are build in the same directory as their original MLV files.
- DNG extraction with MLV_Dump.
- RGB conversion with DCRaw (parallelized for better performances).
- Movie conversion with FFmpeg.
- Dark and flat calibration with automatic averaging of MLV files.
- Lossless export AVI (codec rawvideo; yuv420p) and MOV (codec h264; qp=1; yuv420p).
- Preview export AVI (codec mpeg4; qscale=1; yuv420p) and MOV (h264; qp=17; yuv420p).
- Sound export if available (PCM S16 LE for AVI file; AAC for MOV file).
- Prevent computer sleep when working.
NB: MOV quality is not perfectly lossless (qp parameter should be set to 0 and not 1) to be readable by Quicktime player.

Software requirements:
- Mac OS 10.9 or higher.
- 64 bit processor (Intel Core 2 Duo or higher).

My first goal was to produce AVI lossless videos to process astronomical images with Registax, AutoStackkert, or AviStack. RAW videos and 3x crop allow to produce very nice high resolution images with telescopes and Canon cameras...



Full size: http://astronomy.eliotis.com/gallery/lune-26-08-16-cn-212-red10-ml3x-100perc-v3.jpg

I'm not a video master at all but if you like MLVToMovie and need some improvements (according to MLV_Dump, DCRaw and FFmpeg capabilities) let me know. I'll do my best to help the community.

Wish it helps,
Sebastien.

12

General Development / Re: Vertical stripes revisited (5D Mark III)

« on: September 15, 2016, 01:15:41 PM »

Thank you for your feedback a1ex. 

So I've just made some dark tests with my 5D Mk III. As far as I can see, thermal noise has no real impact with short exposures. Here is a video showing dark with growing temperature (22°c to 52°c according to ML display)...

-dark signal was stretched for visualization-
It's a good news for calibration. I don't think we really need to take care about bias. Vertical stripes seem to be constant so dark and flat should do the job. I will test on astronomical images as soon as possible.

I've just finished my own Mac GUI to convert easily MLV to lossless AVI (needed for astronomical softwares). -It can also export mov files-
Dark and flat calibration are already integrated...

I'll post a download link quickly.
Wish it helps. 

13

General Development / Re: Vertical stripes revisited (5D Mark III)

« on: September 11, 2016, 01:15:37 AM »

Thank you very much a1ex for flat processing. I was just searching for such a feature! It will be useful for high-resolution planetary astronomical imaging. 

First shots on Moon with MLV files and crop 3x (without dark and flat yet) make me really enthusiast...
http://www.magiclantern.fm/forum/index.php?topic=17825.0

Thank you GutterPump.  I'm working on a Mac application to produce avi files for astronomical tools like Registax, AviStack 2 and AutoStakkert 2. You're binary will be useful.

I would have a question about processing dark frame. Cause there is no temperature stabilization with Canon cameras, can we compensate the temperature difference between main video and dark?

14

Share Your Photos / Re: Fly me to the moon with ML / Raw Video / 3x Crop

« on: September 07, 2016, 08:02:18 PM »

Thank you very much! 

DeafEyeJedi, the build is: Nightly.2015Dec20.5D3123

I've just finished to process the next nights (August 26,27,28). We can see that the diameter of the moon slightly decreases (around 3% from 25 to 28)...


And here is an animation to see the progress of the terminator (the line between day and night)... 

August 26:
           
Full size: http://astronomy.eliotis.com/gallery/lune-26-08-16-cn-212-red10-ml3x-100perc-v3.jpg

August 27:
           
Full size: http://astronomy.eliotis.com/gallery/lune-27-08-16-cn-212-red10-ml3x-100perc-v3.jpg

August 28:
           
Full size: http://astronomy.eliotis.com/gallery/lune-28-08-16-cn-212-red10-ml3x-100perc-v3.jpg

15

Share Your Photos / Fly me to the moon with ML / Raw Video / 3x Crop

« on: September 06, 2016, 06:46:03 PM »

My first moon quarter with Magic Lantern using RAW video and 3x crop... 

CN-212 Takahashi - F/10 - 5D MK III - 1/250s - Iso 200

Full size: >>>>here<<<< 

Processed with:
- mlv_dump, dcraw and ffmpeg: image extraction and AVI.
- AutoStackkert2, Iris : stack, lucky imaging, deconvolution.
- Pixelmator: create panoramic manually and final adjustments.

I love Magic Lantern!!! 

16

Hardware and Accessories / Re: Komputerbay 1000x Cards - Benchmarks & Uncompressed video resolution

« on: April 07, 2016, 12:25:55 PM »

I received my Komputer Bay 64GB 1066x yesterday.

Benchmark with my 5D Mk III...

17

Modules Development / Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)

« on: January 21, 2014, 01:20:11 AM »

So much to say about astronomy! Enough to spend many nights but my english is not good enough I think!

To stay on topic, my first test is pretty conclusive even if I have to improve many things...

My approach is may be a little bit different: I extract the two frames and inject them in my HDR fusion.

18

Modules Development / Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)

« on: January 13, 2014, 06:35:20 PM »

100 bracketed pictures? Come on... take a look at my experiments with CeroNoice

In normal "human" conditions it has no interest of course. It's more a marketing tool. 

But for more specifics conditions (High Iso, very low signal)...

EOS 20D - 160x30s - 1600 Iso

And of course, some astronomical subjects have also a very high dynamic range: Andromeda Galaxy, Orion nebula, etc.

19

Modules Development / Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)

« on: January 13, 2014, 03:36:18 PM »

Thank you a1ex and congratulations for your brilliant idea!!!
I've already downloaded the first link (Orange Juice Bottle). Unfortunately many links are dead. I'll still take the time to check all the pages for cr2 files.

I'm working on a software called MarScaper HDR for Mac. For more details, you can have a look at here: MarScaper HDR web site.

I hope I'll be able to add Dual Iso support to my roadmap. I have already started studying your PDF to understand all the machinery. Very smart approach. If I find an idea to fight against artifacts, I'll share it with pleasure.

20

Modules Development / Re: Dual ISO - massive dynamic range improvement (dual_iso.mo)

« on: January 11, 2014, 03:47:52 PM »

Hi all! 

Can I have access to some raw images (Dropbox or anything else)? I would like to do some tests with a HDR software that I designed.

If I can provide Dual Iso support, that would be great!