Building A Camera From Scratch, Converting the Raw Data to an Image

[DavidP]

Hello guys, it's been a long time! I eventually jumped from the 5d mk iii raw, to a stint with sonys a7sii (vomit), and now I have the Ursa Mini Pro... but now my next project!

I'm looking into building a high speed camera from an existing sensor and custom FPGA. Just as a hobby for studio use but hey if it works well I could possibly produce more! The affordable projects I've seen have been pretty disappointing, awful color, poor resolution, jagged edges you name it. In fact I recommended one of them get in touch with guys here to try and get help with his demosaicing algorithm but he never listened and it just looks rubbish.

So my questions to people who've walked the path of working with canon and getting such great results already.

1) Is the sensor generally responsible for a certain color reproduction or is color science purely down to the software? So for example are canon sensors just awesome or is it cause of the post processing the color is so nice? Did you guys have to somehow hack this when unlocking the raw video?
2) Can I use software such as IMATEST, to shoot My ursa mini pro next to my Frankencamera with color charts and then try to match the color science coming out of it in that fashion?

Thanks, I hope this starts an interesting discussion

[Levas]

1) Is the sensor generally responsible for a certain color reproduction or is color science purely down to the software? So for example are canon sensors just awesome or is it cause of the post processing the color is so nice? Did you guys have to somehow hack this when unlocking the raw video?

I find this an interesting subject and have wondered this myself too, does something like Canon or Nikon colours really exist?
I think it all comes down to this, the only thing that can really influence the color science on a CMOS is the bayer filter / color filter aray (CFA) in front of it.
CMOS is monochrome, so all color response depends on the bayer filter in front of it.
Ofcourse a lot can be tweaked in post proces, but post proces depends on how the CMOS sees colours through the bayer filter.
Having played with raw files from different camera brands, I can say there are difference in color behaviour, I notice especially difference with setting white balance and using the Tint slider (magenta to green), I think Canon has more room to play with this slider then some other brands.

I remember reading somewhere that for getting better high iso performance, camera brands developed other color filter arrays, the filters got more overlaps between the 3 basic colours (Green, Blue and Red), so for example the green pixels also get some extra light from the blue and red spectrum, so more light and less noise, but also lower color accuracy.

Not sure, but I think the mayor camera brands have different bayer filters in their camera line up. Bayer filter can be different with each new camera model.

[mothaibaphoto]

the only thing that can really influence the color science on a CMOS is the bayer filter / color filter aray (CFA) in front of it.

Bingo!!! And, by the way, this is why sony can't do 15000K WB as Canons...

[DavidP]

Ok so basically a bayer filter could influence poor color in footage no matter how much post? For example could it cause jagged edges if it has bad "Nyquist Frequency"?

[50mm1200s]

See the Axiom project: https://www.apertus.org/axiom

[Levas]

Ok so basically a bayer filter could influence poor color in footage no matter how much post? For example could it cause jagged edges if it has bad "Nyquist Frequency"?

Bayer filter influences color, the difference is there, but with a little bit of post proces you can get good results with any of them.
As far as I know jagged edges are not caused by bayer filter.
Jagged edges can come from multiple things.
 -Debayering method used.
 -Is there an optical low-pass filter (OLPF) in front off the sensor and how strong is it ?
 -How is the sensor read out, does it do any tricks for high speed fps ?

Since you're talking about high speed cameras, I think the problem of the jagged edges is caused by how the sensor is read out, to achieve the high fps.
Sensors can do some trickery to achieve high fps, most of the time, line skipping.
Canon does it too, even in normal video mode, because the sensors are high resolution for photo (about 20 Megapixels) and video is 'only' 2 megapixel for Full HD.
Full HD video on the Canons is done by reading out every third line. So read one line, skip two lines etc. (except for the 5dIII)
For 50/60 fps, Canon skips 4 lines, so read out 1 line and skip 4 of them.
This line skipping causes jagged edges.

So choosing a sensor for high FPS, it's important to know how the sensor does this. Can it really do high fps or does it use line skipping.