Switch for macOS Catalina/Linux (former cr2hdr.app)

Started by Danne, May 05, 2015, 04:32:38 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Danne

I can´t really say what you should do here. Switch is exporting Log-C through piping in dcraw and ffmpeg. What do you see when you export these files and wht is wrong with them coming from Switch @whysodifficult?

togg

Came back to magic lantern and switch (1.2? latest) after kind of a long break, I have a couple of new questions to get in touch again with it :)

- Batched 22 clips, 174GB. Simple dng export with jpeg compression, it took 2 hours. Is it in line with what you're having? I remembered it being faster mmm

- Audio of the last clip doesn't export, M29-1323.

- Skipped naming or exporti a frame in a moment where there're corrupted frames in the original file, M28-1741.

- The countdown was completelly off at the end of the process, negative numbers or something like that.

the log: https://pastebin.com/fWaHdWyM

Danne

Time sounds about right. Compression will take longer then without. The indicator can of course be counting better. Not very reliable and I´ll see if I can fix it. The other stuff is too hard to tell anything about. Seems it doesn´t have to do with Switch...

Alefrient

I wonder if it is possible to output dualiso stills without any interpolation in Switch? I am ok with resolution loss and if i get it right it will help get rid of banding even at extreme settings. Is it so?
Cheers! :)

Danne

I am sorry but I don't really get what you want? Please feel free to elaborate.

Alefrient

I found some lines in dualiso pdf:
5. Mixing the two exposures: "So, why not just use the
ISO 1600 data everywhere, besides the top 4 stops? Because resolution matters too, and so does
aliasing and moire."
5.1 Recovering the full resolution in midtones:
"As you can see from Figure 7, the result with the smooth transition is a little on the soft side (but
without any banding). The midtones created without any interpolated pixels (Figure 7, bottom)
have the same resolution as a normal picture (without dual ISO), but has some slight banding
artifacts."
"...in this case you'll have to live
with a softer image, obtained only with curve (9), which does not have any banding."
That makes me think that banding in shadows and highlights appears while recovering full resolution midtones. Is it correct?
Is it possible to trade half resolution to have zero banding?
I hope I managed to make it a bit more clear. :)

Danne

I don't think you can get around this(havn't checked the pdf) but you could ask in the dualiso thread about this or simply read it through. I think most of the stuff has been tested and discussed more than twice already :)

Alefrient


freely

Hello.

I am writing this in Google Translator because I am not good at English.

I think the switch is a great program. Thank you for your development.

I am using 5d mark3 and magiclantern-crop_rec_4k.2018Mar10.5D3113.

In the meanwhile, the monitor of 5d is small, and it is inconvenient to shoot. So I recently connected my smartphone to the camera and tried shooting with the DSLR Controler app. But there are various problems. An error often occurs during shooting, which makes stable shooting difficult. And there is also a problem with dng conversion.

After shooting today, I converted my mlv file to dng using the latest version of the switch. I reset the switch to default by using (R) first. Then, after selecting (ms), I ran (r). Some files were successfully converted, but some did not.


This is my mlv file.

case (1)
https://drive.google.com/open?id=1G43xO-7ni3ynm997KmB_utRPoOb1-W9t
After converted to dng, with an oblique oblique line on the screen and strange colors.





case (2)
https://drive.google.com/open?id=1IuOQTsPbHdSBZDEFBp3b--rxaWrDKtQW
This file has only a small wav file (626bytes) in the folder after the conversion and no dng file at all.

Here in my log
https://drive.google.com/open?id=1xgPV8Oegmb0zFtxmkJFctPB5TdDU7Mwc


I have played files with the above problem with MlRawViewer.app (1.5.0). Files play well in MlRawViewer.app.
I have converted the file to dng from MlRawViewer.app (1.5.0). I imported it from Da Vinci Resolv, and the screen looks clean. But the sound can not be played back. Import wav separately and place it on the timeline to hear the sound.

Today I took 35 mlv files and converted them into switches, of which there are 3 files with strange colors like case (1), and 9 files with no dng file like case (2).
I think it is caused by linking an app called DSLR Controler with 5d mark3, but I would like to use a smartphone to use focus picking and zebra on a bigger screen. Or I would prefer to see a larger magic lantern screen directly on my smartphone. Is there anyone who can suggest a good way I do not know?

Thank you for reading and I wish you all happy. Thank you.

Danne

Cool. Have a little less time atm. Could you try exporting your troubling files again but this time set the switch to relaxed in (ms) menu?

freely

Thank you.

I tried relaxed mode export. Here is screenshots.

case 1 is same. Nothing changed.

This is first frame of case 1.



This is second frame of case 1.


MlRawViewer 1.5.0 plays well.



case 2 is little better. It converted. but video and audio is separated.




MlRawViewer 1.5.0 plays well.



PS: I noticed a few moments ago, but the number of frames displayed by MlRawViewer is different from the one converted to a switch. If possible, I would like to know how to convert when converting from MlRawVeiwer to dng, which is able to import the latest version of DaVinci Resolves without problems.


I'm using DaVinci Resolve 15 beta 2 now.
Beta 3 and 4 has has trouble, green and magenta cast when import dngs.



Thank you.

Danne

Hi freely.Thanks for posting clear examples.

I tried your file M10-1749.MLV. It exports if selecting (15) relaxed(skip blocks that are erroneous) in ms menu.



The error is related to this:
File /Users/dan/Desktop/Timer_A/freely/M10-1749.M00 not existing.
Processing...
[ERROR]     LJ92: Open failed (-1)

Reached end of chunk 1/1 after 229 blocks
Processed 89 video frames at 23.98 FPS (3.71 s)
Done


In mlv_dump.c you can see that it has to to do with decompressing. Some block that is erroneous, setting relaxed mode skips abortion when that block comes along:
#ifdef MLV_USE_LJ92
                            lj92 handle;
                            int lj92_width = 0;
                            int lj92_height = 0;
                            int lj92_bitdepth = 0;
                            int lj92_components = 0;

                            int ret = lj92_open(&handle, (uint8_t *)frame_buffer, read_size, &lj92_width, &lj92_height, &lj92_bitdepth, &lj92_components);

                            /* this is the raw data size with 16 bit words. it's just temporary */
                            size_t out_size = lj92_width * lj92_height * sizeof(uint16_t) * lj92_components;
                           
                            if(ret == LJ92_ERROR_NONE)
                            {
                                if(verbose)
                                {
                                    print_msg(MSG_INFO, "    LJ92: Decompressing\n");
                                    print_msg(MSG_INFO, "    LJ92: %dx%dx%d %d bpp (%d bytes buffer)\n", lj92_width, lj92_height, lj92_components, lj92_bitdepth, out_size);
                                }
                            }
                            else
                            {
                                print_msg(MSG_ERROR, "    LJ92: Open failed (%d)\n", ret);
                                if(relaxed)
                                {
                                    goto skip_block;
                                }
                                goto abort;
                            }

                           
Why are you getting this error? Proabably the reason you mention yourselves. Now I´ll check your other file if it gets downloaded today  :P.

By the way. I can also recommend Mlv App for previewing exporting to dng etc:
https://www.magiclantern.fm/forum/index.php?topic=20025.0

Last I tried MlRawViewer dng exports metadata wasn´t top notch (cdng specwise) so maybe that relates to issues in DaVinci Resolve.



freely

Thank you for your reply.

I have tried several things first with M10-1746, including what you recommended. The method I have found so far is to import the mlv file from the MLV App and uncheck "Enable RAW Correction".

Then select "Export Selected Clips" from the File menu. Then, when I import the file from Da Vinci Resol, both video and audio are played back simultaneously. I do not see anything like color casts.


I also tried M10-1749, which is exported but the message in the conversion process is different from M10-1746. When the converted dng is imported into Da Vinci, the resolution information is "2048x1556".

When I check the information of the dng file in the finder, it appears as "1920x818", and it seems that some misinformation is transmitted.

In the last few weeks, as I was searching and testing various information, what I was feeling was that I wanted to know how to stably shoot with a magic lantern on a bigger screen. I feel it is unstable to shoot a magic lantern with a DSLR controller. I wonder how you can shoot with a bigger screen like a smartphone, with the focus adjusted exactly.

Danne Thank you for your kindness and good luck.


Danne

QuoteI wonder how you can shoot with a bigger screen like a smartphone, with the focus adjusted exactly.
Something out of the hdmi output? A small monitor maybe?
I use magic zoom on the actual liveview window. For me it works really good and it can be hide and unhide with half shutter pressing.

freely

Danne Thank you for your suggestion.

I tried Magic Zoom. It will be quite helpful when taking fixed shots. However, when shooting a slight motion shot, I could not move the zoom box during shooting, so it seemed a bit difficult to control. Still useful information thanks.

Purchasing an hdmi monitor is burdensome for now, so I hope to find a way to fix it with my smartphone if possible.

I'm testing DSLR Coltroller again. When you shoot with DSLR Coltroller, these characters appear on the camera lcd. I do not know the meaning of this code, but I do not know if I can get help from this.




Thank you very much.


Danne

Mainly means something isn't optimal, I can't say what exactly. I would probably stay off dslr controller and raw recording but it's your call. Good luck!

jpegmasterjesse

I've messed with DSLR Controller and Raw... definitely not ideal.  I believe it wasn't possible to trigger video via the app, but it was possible to see the phone screen during recording. Could be wrong as this was a long time ago, on my 5d2.

Normal video recording, however, works fine. I believe DSLR controller has a 1:1 pixel feature, too.

12georgiadis

[/url]



Hello Danne, I'm cleaning 10 hours of h264 proxy from 5DMKIII, and switch is running. But it shows me this parse error message (see the picture) 

What does it mean ?

log :
https://pastebin.com/sYV9guek

Edit : I batch renamed h264 files to correspond to the MLV's name.  It's still showing errors...
Edit2 : just 5 shots worked and now they are named like this : FY5A0516_1_2017-08-30_0001_C0000
It's good if we generate DNG files via MLVFS. But if we relink with the new XML option from MLV, we could keep only the original name. I don't understand what's wrong with my files.


12georgiadis

Good morning, everyone,

I have a problem with the amazing workflow H264 proxy + MLV.
I have to process a lot of h264 proxy files to remove the black frames/pink frames at the beginning of the shot in order to get clean h264 files that start with the same 1st frame as the MLVs. With Switch, this function worked on some files and I had done a lot of tests in September 2017 (see: https://www.magiclantern.fm/forum/index.php?topic=15108.msg189532#msg189532). The workflow consisted of:

1) run switch to remove the black/pink frames from the beginning of the MOV H264 proxy by generating a new file without transcode and whose TC is reset to 00:00:00:00:00 to match the MLV TC
2) edit offline with the h264 proxy.
3) At the end of the editing, select the MLV files that correspond to the files edited in H264 proxy. Then develop them with MLVFS to generate on the fly CNDG
4) import via XML and grading in Resolve

This saved a lot of disk space and time.

The first issue is that step 1) does not work on all files, whether with an old build or the latest build. The file keeps the pink/black frame from the beginning even after it has been processed.

In addition, with Danne, we tried to set up an even sexier workflow when it is combined with MLV app. It would be:

1) run switch to remove the black/pink frames from the beginning of the MOV H264 proxy by generating a new file without transcode and whose TC is reset to 00:00:00:00:00:00 to match the MLV TC
2) edit in offline with the h264 proxy.
3) At the end of the offline editing, Export an FCPXML from the timeline (or convert via Resolve a Premiere/AAFAvid XML to FCPXML), import it into MLV app via the FCPXML import option. This will output the list of mounted shots and import the MLV files corresponding to the edited h264 proxy.
4) do the pre-correction via MLV app and export to ProRes 4444 C-log
5) import into Resolve via FCPXML which will reconnect to ProRes 4444 C-log.
6) Grading and export your master
7) take some popcorn and admire the masterpiece 😎

The time, performance and disk space savings would be enormous!

So the problem for now is really how to detect properly the black/pink frames from the beginning and cut them by an automation. Danne thinks there's no need to cut the black frames at the end of the clip to reconnect the clips during the conformation phases (just the black frame/pink frames at the begining). This is why the latest switch versions allow you to skip frame counting to cut the black/pink frames of the MOV H264 Proxy, which speeds up the process enormously.
If anyone has any idea of the problem, don't hesitate!
@Dfort maybe?


jpegmasterjesse

I'm trying to run Switch in ChromeOS as a Linux container. I'm running into this error:

(yad:708): Gdk-WARNING **: /build/gtk+3.0-NmdvYo/gtk+3.0-3.22.11/./gdk/x11/gdkwindow-x11.c:5573 drawable is not a native X11 window

It successfully opens up a file-picker, but I just get a long list of "No such file or directory"s

Any thoughts? Would appreciate any help. Getting this running smoothly in a container would make my life on the go so much easier.

Danne

Sorry, can't help atm but you should try google the issue and maybe you'll fix it yourself.

jpegmasterjesse

I get it! I'm a linux n00b so a lot of basic stuff goes over my head.

Looking around a bit, it seems someone solved a similar issue by launching their program with something like
GDK_BACKEND=x11 ~/Switch/bin/Switch_MAIN.sh

This fixed the error I was having, I was prompted to pick a file like before, but then pretty much the same stuff followed. I'm only pasting a few lines because it goes on for a while:
ls: cannot access '*.MLV': No such file or directory
ls: cannot access '*.mlv': No such file or directory
/home/username/Switch/bin/Switch_MAIN.sh: line 80: /home/username/Switch/bin/mlv_dump: Permission denied
ls: cannot access '*.MLV': No such file or directory
ls: cannot access '*.mlv': No such file or directory
/home/username/Switch/bin/Switch_MAIN.sh: line 85: /home/username/Switch/bin/mlv_dump: Permission denied


Appreciate any ideas anyone may have.

Danne

@12georgiadis

Quote from: 12georgiadis on October 05, 2018, 10:03:26 AM
latest switch versions allow you to skip frame counting to cut the black/pink frames of the MOV H264 Proxy, which speeds up the process enormously.
Pink frames will not count as black frames so naturally they will not be cut out. Do you get a lot of files with pink starts?
You say that some files works and others don´t in your workflow. So which ones are working fully and which ones are not? Only the pink proxy files or are there others? If possible upload one file that is fully working, and one file that is not for comparison testing.

Is the two first steps working with all proxy scenarios on a working file?
Quote1) run switch to remove the black/pink frames from the beginning of the MOV H264 proxy by generating a new file without transcode and whose TC is reset to 00:00:00:00:00 to match the MLV TC
2) edit offline with the h264 proxy.

Please test following with a working proxy file and corresponding MLV:
There are three workflows in Switch. One is automated and activated simply without doing anything in any menu settings(First test). Next processing test would be activating (24) MLV+proxy(outputs only MOV(default MOV+MLV) in mlv_dump on steroids then run Switch menu and in the following menu select (r)  Continue without modifications. Last test would be selecting (S)  Skip MLV indexing(faster but proxy end part is not cut)

Danne

@jpegmasterjesse

Quote from: jpegmasterjesse on October 06, 2018, 06:49:53 AM
This fixed the error I was having, I was prompted to pick a file like before, but then pretty much the same stuff followed. I'm only pasting a few lines because it
Are you selecting a folder with MLV files to develop into dng or what are you trying to do? I don´t think you are supposed to select the MLV file itself but the actual folder.

You could try fix the permission error. In terminal write:
cd /home/username/Switch/bin/
sudo chmod o+r mlv_dump

enter

rerun Switch again on a folder with files.

12georgiadis

Quote from: Danne on October 06, 2018, 07:53:33 AM
@12georgiadis
Pink frames will not count as black frames so naturally they will not be cut out. Do you get a lot of files with pink starts?
You say that some files works and others don´t in your workflow. So which ones are working fully and which ones are not? Only the pink proxy files or are there others? If possible upload one file that is fully working, and one file that is not for comparison testing.

Is the two first steps working with all proxy scenarios on a working file?
Please test following with a working proxy file and corresponding MLV:
There are three workflows in Switch. One is automated and activated simply without doing anything in any menu settings(First test). Next processing test would be activating (24) MLV+proxy(outputs only MOV(default MOV+MLV) in mlv_dump on steroids then run Switch menu and in the following menu select (r)  Continue without modifications. Last test would be selecting (S)  Skip MLV indexing(faster but proxy end part is not cut)
@Danne,
I had my 5DmkIII back to make some tests. I used the last build and the new workflow is working well ((S)  Skip MLV indexing(faster but proxy end part is not cut)), very fast and it cuts the black. I think the pink frames belongs to a bugged build but it's fixed in newer versions.
I still have to test the offline/online MLV app conformation and see if it's ok or a problem to have the black at the end. I expect no problem. Thank you again for the support and to have helped with an even faster workflow with the new skip MLV indexing option !