Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Messages - Milk and Coffee

#1
https://github.com/dnglab/dnglab

This is a new DNG conversion tool for linux!

This is not my repo, but I am excited for development!

I thought I'd mention it for any devs who want another project to work on! There is a need for Cinema DNG support, if anyone has knowledge to contribute!
#2
Quote from: names_are_hard on January 02, 2024, 05:56:07 PM
It can be good to have an end goal in mind when learning to program, but from experience teaching people, you don't want to start with a challenge.  Learning to program is enough challenge on its own, initially.

Start with standard boring tutorials (text, not video).  Learn how to use python first, write a range of simple but functioning programs, then consider starting on your actual goal.  If you're confused by python *and* your goal, it's much harder to learn.  Keep the problems simple so you're sure you understand them, while you're still getting over the confusion around programming itself.

I appreciate all the advice! And I agree.

Quote from: Danne on January 02, 2024, 06:37:23 PM
Chatgpt is a good friend explaining functions and what is going on in syntax and code in general.

I thought about doing this to help! - What did you prompt ChatGPT with? A question, then pasted the code?
#3
To help clarify why i'm so set on coding this myself: I'm wanting to learn python so that I can start creating DCTL's in DaVinci Resolve. I thought this would be a good challenge to help with that process.
#4
Quote from: a.sintes on January 02, 2024, 09:14:39 AM
why are you specifically searching for LJ92 compression?
does any alternative lossless compression algorithms supporting 12bits depth fits your initial need?
do you really need to work with lossless media or a professional lossy compression enough for you post-production requirements?

depending of these answers, I guess you can maybe avoid a lot of struggle just by using standard tools like ffmpeg/Resolve, alongside JPEG2000/x265/AppleProRes4444 algorithms?

I want to preserve RAW data as after lossless compression, it's file size is not much larger than ProRes422.

Quote from: Danne on January 02, 2024, 09:30:30 AM
This is also working:
https://www.slimraw.com/

Danne, this is extremely helpful! But was hoping to have a free solution.
#5
Hey all!

I've recently purchased a Sigma FP and am shooting UHD 12bit RAW cDNG. I use my Canon 5DIII with Magic Lantern shooting RAW as a B-Cam. The Sigma FP does not apply lossless LJ92 compression out of the camera, and I love that MLV APP allows me to export Lossless DNG for my Canon MLV RAW. I have processed the Sigma FP files with Adobe DNG Converter to apply lossless compression, but I lose all cinemaDNG-associated metadata in the process (timecode, color matrix, profile hue sat map, and bit depth.) Also, the audio must have embedded timecode, and thus does not sync with my Adobe-DNG-converted sequence. TLDR, Adobe DNG Converter does the trick to apply lossless compression, but comes with many caveats for cDNG.

Thus I'm determined to learn enough coding to write a script that can import cDNG, apply lossless LJ92 compression, and thats it. Leaving all metadata intact, and no conversion to higher bit depths.

I found this to be helpful in understanding the compression: https://thndl.com/how-dng-compresses-raw-data-with-lossless-jpeg92.html

I've spoken with a member of this forum on tips how to do this. The following was their instructions:

"First you would need to make a python extension allowing you to use the LJ92 library: https://github.com/ilia3101/MLV-App/tree/09a726ef66f693f29dd551534f7340cadb1c1955/src/mlv/liblj92. MlRawViewer had code for this, but it is for Python2 (now deprecated) and includes extra functions for bayer->RGB conversion. You would need to start from this file: https://bitbucket.org/baldand/mlrawviewer/src/master/bitunpack.c and cut it down to just the LJPEG parts + update for Python3 as required.

Python setup.py file: https://bitbucket.org/baldand/mlrawviewer/src/master/setup.py At this point I would also suggest updating the LJ92 to the version in mlv-app, but it may have some slight interface changes. Then you will need to update the DNG.py file for Python3: https://bitbucket.org/baldand/mlrawviewer/src/master/DNG.py After that you should have the needed parts to make a script that can read existing cDNG files, compress them, and write out a copy."


I have started a repo here: https://github.com/devonstanczyk/Digital-Negative-Lossless-Compression.git

I am very new to this, so please any help/collaboration is appreciated! Also, if I'm stepping on any toes, please know it's unintentional and let me know!

Thanks all!
#6
Thank you! What's the process to re-run ding's through MLV App? I cannot seem to import them. 🙏
#7
This looks amazing! Any discounts for the magic lantern family?
#8
masc, where can I find a list of MLV App's supported metadata fields when exporting lossless cDNG?
#9
Quote from: masc on August 16, 2023, 08:55:17 PM
Yes, that should all work. Note: just supported metadata will survive... (not many fields).

Great! Any tools/apps to view/edit metadata used on current DNG's? Adobe Bridge?
#10
Can MLV App apply lossless compression to already exported cDNGs/Can I reopen a DNG sequence and apply lossless compression? Would this work for other camera models like the Sigma FP? Been looking into SlimRAW www.slimraw.com, but realized maybe MLV App can already do this?

Thanks all!
#11
Hi all, been getting occasional pink frames when filming with an HDMI monitor plugged in. Any ideas?
#12
Anybody know how they go from 6k sensor to 4K raw? Pixel binning? Line skipping? What other "downscaling" methods are there that allow RAW recording (true RAW that is, no BRAW crap...lol)?
#13
Similar question here: I like having 1/48 shutter speed, but it keeps me from getting 1/4000. How do I get 1/4000 back?
#14
Raw Video / Re: History of ML cracking RAW
October 27, 2021, 05:51:08 AM
I know that there were two formats of Magic Lantern RAW video files. The current, .MLV. Are both formats just as RAW as the other? Any benefits or caveats to either format?

Also, is the data stored RAW Sensor Data (not debayered)? Or is it a "flavor" of RAW similar to Blackmagic's .BRAW (already debayered)?
#15
Does MLV store RAW data in a linear gamma?

I'm reading that cameras that store RAW sensor data linearly, the dynamic range is limited to the bit depth. Whereas cameras that store RAW sensor data logarithmically, the dynamic range is not limited by the bit depth.

https://www.avsforum.com/threads/fake-raw-braw-proresraw-z-raw-whats-going-on.3157980/

Example: In cameras that store sensor data linearly, capturing a 10bit signal, limits the available dynamic range to be captured to 10 stops.
#16
General Help Q&A / Re: App for viewing Code
March 09, 2021, 04:31:31 PM
Thank you all for the responses! I'm slowly catching up! Appreciate your patience 🙏
#17
General Help Q&A / Re: App for viewing Code
March 09, 2021, 03:37:25 AM
Quote from: names_are_hard on March 08, 2021, 08:15:20 PM
I can tell you that I don't think there is Bayer *data* in the files, simply because there isn't a need.

Thank you! Could you be more specific about what you mean by "Bayer Data"? I thought bayer data was present in all RAW sensor data? If there is no Bayer Data, does that mean it's not RAW?

And thanks Walter! I'll give that one a go!
#18
General Help Q&A / Re: App for viewing Code
March 08, 2021, 04:33:51 PM
I want to open my own CR2 images and compare the data to that of the same CR2 converted to DNG. I also would like to compare the Bayer data of a CR2 and a frame from an MLV. And if I'm feeling crazy, I'd also like to look at Canon's SRAW since it looks like it doesn't contain any Bayer data.

Thank you for your patience! - So to see the text format example, the file needs to be parsed. To parse the file, I should use DCraw (I see that's what the original article is using.)?
#19
General Help Q&A / Re: App for viewing Code
March 08, 2021, 03:40:02 AM
So I downloaded "Hex Fiend" for mac. From this link http://lclevy.free.fr/cr2 I'm seeing code examples like this:

(5D Mark II RAW) 
JPEG_SOF3: length=20
bits=14, wide=1448, high=3804, nb comp=4
  index=1, h=1, v=1, qt=0
  index=2, h=1, v=1, qt=0
  index=3, h=1, v=1, qt=0
  index=4, h=1, v=1, qt=0


I'm not seeing any code like that in the Hex Editor. I see the Hexidecimals of course, but the plain text view is nothing like the above. I tried Hexed.it as well. Is there a "view" setting I need to change? Is a Hex editor the correct program to view code in the format above / is there another app out there that shows me code similar to the above?

Let me know if you need clarification
#20
General Help Q&A / Re: App for viewing Code
March 03, 2021, 01:55:10 AM
Quote from: names_are_hard on March 02, 2021, 05:01:04 AM
You want a hex editor, there are many options.  010 Editor is very good, and has a free trial.

This is SO helpful! Thank you!
#21
General Help Q&A / App for viewing Code
March 02, 2021, 04:06:25 AM
This is a total newb question, but I just need some help to get started on the right foot:

What app can I use to view the "code" that makes up a .CR2 or .DNG file?

Context: http://lclevy.free.fr/cr2/
That is a link that provides an explanation of the code inside a .CR2 file. BUT I cannot figure out how to open my own .CR2 files to view the "code."

What apps allow me to do this on macOSX?

Thanks so much!
#22
Nevermind, I see we can debayer straight to linear with CDNG in Davinci Resolve 👌
#23
Thanks kharak! I prefer to use unmanaged YRGB. I'm not by my computer, but I believe YRGB requires debayer to either be 709, P3, or BMD Film. If I debayer with BMD film, then CST to linear, is that any different than starting with linear as an input with YRGB Color managed?
#24
What is the benefit of first converting to linear? In Davinci, is that using the OFX Color Space Transform to linear, then again from linear to 709?
#25
Raw Video / Re: Raw video on 5DMK2
February 23, 2021, 05:05:16 AM
Quote from: Boygetslost on January 12, 2021, 06:40:56 PM
Thanks everyone.

I actually went back to the last nightly build from 2018, for stability issues, one of the experimental builds I was using seems to corrupt files every now and then. I am only interested in using 14bit raw anyway.

I am having issues with my external monitor, sometimes I get pink flickers for a second and my last batch if clips had flickering issues that ruined them.

Is there a problem with the 5d2 and using an external monitor in raw?
(Both the experimental build and the older nightly build, seemed to cause some issues when I plugged my Cinieroid HDMI EVF/monitor).

Thanks
Chris

I have the same issue recording in both 10 and 12 bit using an external monitor.