Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - mlrocks

#101
Raw Video / Re: 5d3 3x3 1080p raw 60 fps not working?
February 15, 2023, 05:45:51 PM
A window popped up on the LCD and messages are rolling, something like memory allocating etc.
#102
Raw Video / 5d3 3x3 1080p raw 60 fps not working?
February 15, 2023, 05:37:10 PM
I just tried 3x3 modes, 14bit lossless, card spanning off, sd card overclocking off (I tried both on, the same results for the 1080p 60 fps):

Canon Q menu ALL-I 1920x1080p 24p mode, 1920x1280p, 3x3, 24 fps, 55 MB/s, continuous, green
Canon Q menu ALL-I 1920x1080p 30p mode, 1920x1280p, 3x3, 30 fps, 70 MB/s, continuous, green
Canon Q menu ALL-I 1280x720p 60p mode, 1920x672p, 3x3, 60 fps, 75 MB/s, continuous, yellow
Canon Q menu ALL-I 1280x720p 60p mode, 1920x1080p 50 fps preset, 85 MB/s, 5 seconds, red, automatically off
Canon Q menu ALL-I 1280x720p 60p mode, 1920x1080p 60 fps preset, red, several seconds, automatically off

I assume that for 1920x672p, I need to destretch to get 1920x800p.
1920x1080p 60 fps at ar 2.4 supposes to work, it is 120 MB/s.

The reason that I am interested in 1080p vista vision mode, is that the lcd is realtime color and can output through hdmi to evf, and can go to 60 fps. At 24 fps or 30 fps, I can turn off the card spanning and sd card overclocking, so that there is no potential risk of overheating. The system is very robust in these modes. These modes are good for doc, news, events, corporate interview, etc.

#103
Comparing to jpeg+raw, raw seems not improving the recording time. What did I miss, except using the second latest ver?
#104
Quote from: Walter Schulz on July 31, 2022, 08:09:04 PM
Please specify build used. "Second latest" is not a proper designation!

Overexposure (=sensor data blown out) and losless compression are rumoured to not get along well. What is your intention to verify that?

I will update to the latest and retest. :)
I intentionally overexposedthe scene to increase the writing stream to 85 MB/s, normal exposure was 70 MB/s.
#105
I tried 14 bit 14 bit lossless UHD 1x3 on 650 D using the ml version the second latest. photo image quality in canon menu set to raw (not raw+jpeg I had before). Overexposure the test scene intentionally to get 85 MB/s, it is about 9 seconds recording time. Debugging memory crushes the camera.

Update:
After unloading all the modules, I could check the memory 95 MB+2x31MB in raw mode.
#106
amazing, super 35 10bit lossless 5.2k 1x3 raw over 6 min! continuous!!
#107
Quote from: honza123 on June 08, 2022, 04:58:00 PM
Overlay two footages into one.
- time laps - Full-res Silent picture
- RAW 1920x1290, 25fps,
  (5DMkIII, Danne's Build)

https://youtu.be/Nm4epfUE0NU

This is insane. The DR and the resolution are unbelievable.
How can you do this overlay? in camera or in post?
Is it possible that you elaborate on your workflow?
Congratulation!
#108
Anyone has MLV App workstations built with AMD Threadripper CPUs, or EPYC CPUs? How do they perform comparing to Intel or ARM opponents?
#109
SanDisk 170 mb/s sd card can only write 90 mb/s. maybe 300 mb/s card is needed for 99 mb/s write speed?
#110
Quote from: theBilalFakhouri on May 24, 2022, 10:44:43 PM
I don't know how MLVApp code works, never looked into it.
But in theory if we re-write the code somehow so it could utilize more threads for decoding and raw correction then we may have faster playback, would be that possible or *some parts of processing must be single threaded?

*Even if it's must be single threaded, I guess we can process multiple frames at the same time (multi processing), e.g. each frame uses one thread or two frames per thread:
First frame would be processed on CORE 0 while second frame is being processed on CORE 1 and so one for other cores, when CORE 0 finishes, it would load the next frame which we need to process.



I have ~16 FPS playback speed using MLVApp 1.13 (default settings, Bilinear debayer) on Ryzen 3900x, Clip used: UHD 1280x2160 10-bit lossless 23.976 FPS
CPU utilization: is only ~17%
Using Amaze debayer it's ~33% (~11 FPS)

I opened two copies of MLVApp, opened same clip, used default settings (Bilinear debayer), playback for each copy ~16 FPS, CPU utilization is ~35%, multi processing theory works, total ~32 FPS playback speed :D and there is 65% of CPU power left.

What I am saying in short:
There is a room for enhancement, even if this mean re-writing MLVApp from scratch. I bet someone from MLVApp team would do that :P (seems a lot of work).

That's why we should start dedicated Patreon account for MLVApp, if one (or two/all) of MLVApp team feels he can improve MLVApp but can't do it for free (fully understandable), funding is a solution for that. I think a lot of users would fund such thing beside there is no legal concerns. if MLVApp team don't want to work on MLVApp for whatever reason (again, it's fully understandable), the idea of creating Patreon account for MLVApp lives, but instead we may hire a freelancer.

I did not mean that, just meant some beautiful MLV clips and used 5D3/EOSM as example (masc seems to have nice MLV clips, and looks good out of the box in MLVApp I guess), I forgot about masc owned 5D2 before :P, I don't think I have watched 5D2 clips from masc (or I just don't remember how it was). Of course clips from 5D2 would be ideal too, to be used for uncompressed MLV benchmarking :D

Blender adds render farm feature and multithreading and maybe 10 bit color depth editing in its latest VSE version. Blender has improved dramatically in the last five years, making it a serious option for video editing and VR creation. MLV App may consider adding render farm and improving multithreading in its future versions. ML coders and heros can set up your own render farm companies to create your own business for yourself and for the good of ML.
#111
Computer: MBA (entry model)
CPU: M1
RAM: 8GB
GPU: 7 core
OS: Latest
App: mlv app apple silicon version
https://www.dropbox.com/s/ej6ufca61ijeeyf/MLV.App.v1.13.macOS.arm64.zip?dl=0
All default settings

Footage: UHD 1X3 12 bit lossless from 650D
Scene: The same test scene

Codec Optimization
Export to H264 MP4 HQ, ffmpeg, Export time: 15 minutes. Used as the baseline here.

Export to Prores 422 LT, ffmpeg, Export time: 12 minutes.
Export to Prores 422, ffmpeg, Export time: 8 minutes.
Export to Prores 422 HQ, ffmpeg, Export time: 10 minutes.
Export to Prores 4444, ffmpeg, Export time: 18 minutes.

All of the above codecs can be opened by the Blender VSE. If export to Prores 422, it is another two folds improvement. Maybe ProRes 422 is the most widely used and therefore the best optimized and the most mature one.


More:

AVID's codecs:
Export to DNxHD, 10 bit 1080p, frame rate override to 23.976, Export time: 6 minutes.
Export to DNxHR, 444 10 bit, Export time: 12 minutes.
Export to DNxHR, HQX 10 bit, Export time: 7 minutes.
Export to DNxHR, HQ 8 bit, Export time: 8 minutes.
Export to DNxHR, SQ 8 bit, Export time: 8 minutes.
Export to DNxHR, LB 8 bit, Export time: 7 minutes.

All of AVID's codecs can be imported in the Blender VSE.


Seems to me the best workflow for UHD 1x3 is to have camera frame rate override at 23.976 when recording, MLV App on Linux or Mac M1, Export to DNxHD 10 bit 1080p or DNxHR HQX 10 bit or LB 8 bit without even considering frame rate override to 23.976, edit in Blender VSE 3.1. For high resolution cinematic projects, DNxHR 444 10 bit encodes faster than H264 420 8 bit. AVID is the still the king of video editing.


In conclusion, using export codecs of AVID DNxHR HQX 10 bit, increases the encoding speed twice, and improves final image quality to 10 bit 422, comparing to the H264 codec. Apple's ProRes 422 is a good second choice, about 20% slower than the DNxHR HQX 10 bit. In general, AVID's codecs are efficient.



This is what Avid recommended the codecs were suitable for:

DNxHR LB - Low Bandwidth (8-bit 4:2:2) Offline Quality

DNxHR SQ - Standard Quality (8-bit 4:2:2) (suitable for delivery format)

DNxHR HQ - High Quality (8-bit 4:2:2)

DNxHR HQX - High Quality (12-bit 4:2:2) (UHD/4K Broadcast-quality delivery)

DNxHR 444 - Finishing Quality (12-bit 4:4:4) (Cinema-quality delivery)
#112
Just tried another workflow on MBA M1.
In MLV App Apple Silicon, export setting to cdng uncompressed, dvr file name format. Export 1 minute UHD footage is about 1 minute. The cdng folder can be opened in Davinci Resolve 17. This way DVR's CUDA capability can be utlized to shorten the process time significantly.
I know many people here already know this workflow long time ago. Just a learning experience for me.
The cdng files cannot be opened in Blender 3.1 VSE. So this workflow does not help Blender.
#113
Quote from: Danne on May 24, 2022, 07:13:17 AM
Plenty to read about M1 cpu progression which explains why a "simple" iphone will outperform most(not all) intel processing units. Compiling mlv app with arm 64 makes all the difference in playback. Still mostly rosetta fallback when exporting with ffmpeg but apps are following. Resolve is already working quite nicely too on m1.

Maybe this explains that why M1 has 3 times better playback but the same export speed.
#114
It is interesting to see that my MBA performs similarly to my Xeon E3-1270V2 desktop in exporting UHD 1X3 footage. Makes me feel good that my old desktop is not outdated yet. LOL.
#115
Computer: MBA (entry model)
CPU: M1
RAM: 8GB
GPU: 7 core
OS: Latest
App: mlv app apple silicon version
https://www.dropbox.com/s/ej6ufca61ijeeyf/MLV.App.v1.13.macOS.arm64.zip?dl=0
All default settings
Export to H264 MP4 HQ

Footage: UHD 1X3 12 bit lossless from 650D
Scene: The same test scene

Performance
Playback: 18 fps most of the time, not stable
Export time: 15 minutes, calculating at the 50% of the timeline. The first several minutes the speed was not stable and time was changing. I always use the time at 50% and times 2 to get the total time.

This version is about 50% better than the MLV APP Image version.

#116
Quote from: bouncyball on May 23, 2022, 12:31:57 PM
Absolutely agree.

Once in 2018 I benched mlvapp playback (no export) on 160core (4CPU) Xeon system and ... well have a look yourself:
Link

160 threads xeon quad cpu server/workstation only gave 8 fps playback with default settings? What was the video card on this system?
it seems to me that playback is more video card dependent than cpu dependent. Exporting is for sure cpu dependent.
Otherwise, how can we explain 160 threads xeon machine only gave 8 fps, whereas mba m1 gives faster fps? I don't think m1 has so vastly improved on cpu speed, but on power saving.
#117
Quote from: names_are_hard on May 21, 2022, 08:40:09 PM
This is not true.  The article you linked to was about a specific *Intel* library that *Intel* deliberately made to be poorly optimised for AMD CPUs.  It has no relevance to "low level compilers".  Only one specific Intel library.  MLV App isn't even using that library!  Compilers optimise just fine on AMD.

Benchmarking is always quite tricky.  Since we can run MLVApp anywhere, when you're interested in MLVApp performance, it's the obvious best choice to use.  Ideally, script running MLVApp, always with the same input file, processing options etc.  That way you can get reproducible results, and you can share the script with other people so they can test on their systems.  It would be cool to see a set of results across a lot of different systems.

For MLV App parameters, for benchmark testing, I set all for default, export to h264 mp4 high quality, which can be imported by Blender VSE.
#119
Quote from: names_are_hard on May 21, 2022, 05:16:01 PM
MacOS has never had a Linux kernel.
Please don't give completely unfounded advice.  This is just untrue.  It's probably the *opposite* of true for MLVApp, which is heavily threaded; current Zen based AMD CPUs normally beat Intel (and Apple M1) based systems in multi-threaded workloads.

Don't guess.  Don't assume benchmarks *on a different piece of software* will be representative.  Benchmarking is notoriously complicated.  Benchmark using MLVApp on different systems; it's free and available on all OSes!  If somebody gives me an MLV file and instructions, I'll bench on my modern 8c/16t AMD Linux system.

I agree with you that maybe passmark is not very suitable for the mlv app. Maybe using MLV App is a better way for benchmarking. I am converting almost of all of my home systems for MLV App now, so it makes sense to use it as the benchmarking criteria.

I DIYed my first video/gaming AMD desktop about 8 years ago, with Sabertooth 990FX tough mo, Athlon FX8350 overclocked, AMD R9 295X2 8GB video card, 32GB DDR3 RAM, SATA3 SSD. At the time it was extremely powerful. I enjoyed 2k gaming long before it becomes main stream. Rawtherapee raw photo editing was totally real time on this system. The set up will cost twice higher if using Intel ecosystem, 5 times higher if using Mac Pro. The system went dead after several years of intensive use. I think that it was my fault, not optimizing the air flow well enough, even there were a liquid cooling system for the cpu and a separated one for the video card.

Anyways, I like AMD's lower price. I am very happy that AMD is leading Intel now. But I recently realize that AMD upgrading path overhead room is not as much as Intel based mother board. Also, I just realize that AMD CPU and GPU is not well supported by low level compilers. As an end user, I really hope AMD can address these two main issues, so that I can build my new system Zen4 or Zen 5 based. Cheers,
#120
Better avoid AMD cpus and video cards. Something I was not aware before.

https://news.ycombinator.com/item?id=21628713
#121
Quote from: masc on May 21, 2022, 12:52:29 PM
Rendering on macOS has always been faster than on Windows. When I compare some of my systems: a dualcore i5 2.4GHz with macOS has a similar performance like a quadcore i7 3.4GHz with Win10.
There is no optimization for certain systems in MLVApp. The code is identical (appart from AVFoundation export which is Apple only and post export scripts) for all OS. The only difference are the compilers and the operating system.
On Linux I saw very similar performance to macOS, but I don't use this really.

Maybe the MacOS and Linux kernels are more efficient. I remember that MacOS later versions use Linux kernel, not sure if it is still so on Arm M1 MBA.
#122
Apple ARM M1 is an 8 core cpu with a typical tdp of 15w, passmark 15000, single thread 4000.
Intel Xeon E3-1270 V2 is a 4 core 8 thread cpu with a tdp of 70w, passmark 6000, single thread 2000.
The hardware does not explain the difference of 4 folds in full hd exporting speed, about 2 folds to 2.5 folds.
Maybe the mlv app is more optimized for the Apple hardware?


Intel i3-3240 is a 2 core 4 thread cpu with a tdp of 55w, passmark 2300, single thread 1800.
AMD Athlon 5370 is a 4 core 4 thread cpu with a tdp of 25w, passmark 2000, single thread 750.
Both has similar passmark and 4 threads, yet i3 3240 is about twice faster than athlon 5370. The single thread performance is 2.5 folds, is the single thread performance the key here? If so, it can not explain the two folds difference between E3-1270v2 and i3-3240.
Probably the low level C libraries are more tuned to the Intel structure than to the AMD's?

Probably the passmark is not a good criteria here?

#123
Quote from: masc on May 20, 2022, 10:58:23 PM
I recommend to sharpen 1x3 footage (if neccessary at all) in NLE after MLVApp export. For an improvement inside MLVApp we don't need a 1x3 optimization option - we would need a complete rewrite with a different conecpt.

OK. It is not as simple as I thought. Thanks for the explanation.
#124
Quote from: masc on May 20, 2022, 09:29:59 PM
What do you mean with "optimized"?! What should be different for 1x3? MLVApp is optimzied for all options we have - as good as we can ;)

The sharpening right now is good for 1x1, not for 1x3?
#125
Hello, MASC. Just a quick thought.
Is it possible to add a new tab in MLV App for 1x3 processing. This way the workflow is optimized for 1x3 footage, the sharpening can be put anywhere it is proper.
Not sure if this is practical.