Raspberry Pi High Quality Camera

Started by katrikura, May 01, 2020, 05:16:17 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.


2blackbar

Technical specs are fine and dandy but so far nobody posted any decent looking raw video from these in at least HD resolution.
There are some CCTV cameras that use dual exposure that are more exciting for me achieving over 14 stops easily but they only sell with subscriptions when i asked half year ago.

Anaconda_

I have one that I'm currently putting inside an old 8mm body - hoping to have it ready by the end of the week, so I can post some footage when it's all hooked up.

I could upload some footage already if you like, but it won't be anything more than a tripod shot filming a leaf or something... Also my knowledge of coding etc. is very limited. This is my first Pi project, so I don't know how useful I can be other than sharing footage.

2blackbar

Great, please record in 24fps in highest resolution you can achieve, on infinity focus so we could check aliasing, so maybe some bricks and roofs and patterns from distance.
Also If You can upload some DNG frames , can be even one from 24fps clip.

juliagomez2244



[/quote]

Super cool this new camera I think the same as other colleagues, it will not be a device that will replace the DSLR but it looks very tempting to buy it, the technological future is getting better.
A few days ago I saw a film that was very futuristic in the technological field and I think we are going there, of course after we leave the covid-19

Anaconda_

Sorry it took a while, here's two test shots out my window. I'm really not sure how helpful these are as the only c-mount lens I have has sever blurring and CA especially on the edges. It's 25mm 1.4 which is very tele on this sensor. I tried it at a number of f-stops to hopefully give some variety in the detail. The issues mentioned above get worse the more you open up, and the more you close down the more ISO has to work, so I'm not sure what the happy medium is just yet. Hopefully there's at least 1 frame in here that can be of some use.

Anyway, here's the original files, which come out .264 and I also converted them to h265.mkv for easier viewing.

1920x1080
24p
mixed aperture f1.4 - f8(ish)
auto shutter speed
auto ISO
auto WB

https://www.dropbox.com/sh/seddezwwymwnh4v/AAAU5es-RCWOXCj1Fpkbz3Dba?dl=0

I'm planning to use this with D-mount lenses, but until I've finished my build, I have no way to mount them to test with nicer glass.

2blackbar

Thanks, looks fine, can You output DNG files, raw files from it ?

Anaconda_

Premiere won't accept the .264 files, and I feel like taking a still from a converted file defeats the point right?

The originals are there, so feel free to take your own frames and stills.

I've not looked into taking a photo on the pi, but I believe they'd be jpeg.

2blackbar

Im not sure You understand, we are excited about these sensors and cameras because of RAW capability, phones can record h.264 since over a decade, its very easy to rip out original lens and glue back C mount to every smartphone.
So You cant get RAW DNG frames from the camera ? Only h.264 ? Those are subsampled at 4:2:0 , nowhere near at what RAW is.

Anaconda_

Yes I get that, and I'm aware of how h264 is used. I highly doubt the camera will record DNG as a video stream, as there is a patent out stopping that from being possible. If you could record a DNG video stream, the biggest part of Magic Lantern, raw video, is already taken care of right?

The documentation says:

Picture formats               JPEG (accelerated), JPEG + RAW, GIF, BMP, PNG, YUV420, RGB888
Video formats               raw h.264 (accelerated)
10-bit RAW RGB data       Yes - format conversions available via GPU

I'm unclear as to what 'raw h.264' really means, but since it's not in a wrapper (mp4 / mov) I took that to mean it's as unprocessed as you can get. Would love to be proven wrong though.

Happy to do more testing, but I'm far from a coder, so let me know what to type into terminal and I'll report what happens. (No blowup codes please)

EDIT:
I've tried using some YUV commands which, as far as my understanding goes, should bypass any conversion. I'm unable to look at either a still or a video using this command though. I've added them to the Dropbox folder... again, if they're not helpful, let me know what else I can try.

Danne

How can h.264 ever be raw  :P.

What does it mean? Some format available but how?
Quote10-bit RAW RGB data       Yes - format conversions available via GPU

Edit: How to decode the h.264 files? Tried viewing but fails.

Walter Schulz

Quote from: Danne on May 20, 2020, 02:29:03 PM
How can h.264 ever be raw  :P.

Hi444PP profile does predictive losless coding. Don't know if it applies here.

Anaconda_

Quote from: Danne on May 20, 2020, 02:29:03 PM
Edit: How to decode the h.264 files? Tried viewing but fails.

That's why I converted them to h265 :)

If you didn't see me edit above, there's a few more files in Dropbox.

I've also tried out a few 4x3 resolutions, since that's what I'll be shooting. The largest I could go with that is 1600x1200 - which is fairly reasonable. However I did read previously I should be able to get up to 30fps at 2028x1520, which would be really nice... so far it just gives me errors with that though.

2blackbar

we have to get to that raw 10 bit rgb data stream and debayer it , maybe try some raw to dng converters
Digital super 8 eh? id do it too with this setup

Anaconda_

I've tried a few, like SlimRaw, but they're not recognising the file. I feel like it needs an extension, but it's anybodies guess as to what extension it needs. I'll keep reading about it and offer more footage as and when more info becomes available.

Quote from: 2blackbar on May 20, 2020, 06:07:24 PM
Digital super 8 eh? id do it too with this setup

Yeah, the sensor size is almost exactly the same as 8mm. and the Pi fits very nicely inside my (previously broken and unusable) Bolex B8.

EDIT:

You can playback the native pi files (that end in .264) in VLC - but I've not found any other app that can open them.

Interestingly, the app Scoop! has this to say about those files:

200520075652.h264
Format                                   : AVC
Format/Info                              : Advanced Video Codec
File size                                : 5.10 MiB

Video
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 1 Ref Frames
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 1 frame
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Color space                              : YUV
Chroma subsampling                    : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive

I don't know if any of that is useful to anyone, but it certainly can't hurt to post it.

2blackbar

yeah post all findings, as you see its 4:2:0, typical smartphone vid specs

reddeercity

@Anaconda_  , .h264 is a basic avc/mp4/mov without the wrapper of mp4/mov .
It's consider raw AVC , because of no wrapper . I've use this format many times in windows.   
https://extension.nirsoft.net/h264
That profile you posted is the same profile canon 5d3/d5 cams  .mov/avc use .

Luther

The software a1ex posted before, raspiraw, added initial support for this camera's sensor:
https://github.com/6by9/raspiraw
https://github.com/6by9/raspiraw/commit/dbe1acf64cba221787080ad06f79d3a5bccd171a

Technically, it would be possible to record raw 24fps. RPi 4 has USB 3.0 ports, you could attach a SDD using an adapter.
They could add support for MLV format in rapiraw using Ilia's lib, so we could process using MLVApp... just an idea :P

Anaconda_

In theory, the files 'rawvid' and 'rawtest' in the Dropbox link should be raw video, not processed or converted into anything... but I have no idea how to go about looking at them, or looking at whatever information they might hold.

( https://www.dropbox.com/sh/seddezwwymwnh4v/AAAU5es-RCWOXCj1Fpkbz3Dba?dl=0 )

With my limited knowledge, Raspiraw helps decode this and then another program (dcraw) wraps it in a ppm container. I understand the concept and everything in theory, but actually getting it to work in practice is beyond my comprehension. In the words of the developer:

'If you expect everything spoon-fed to you, then raspiraw is not the program for you.'

I love ML on my Canon cameras. Thanks to the efforts of the people behind it, it's easy to understand, set up and work with. Raspiraw seems to be very different from a user's perspective... maybe the backend of ML is similar, but it's hidden behind the UI - I have no idea.

Not wanting to sound selfish, but my camera setup is working as is, albeit with .264 outputs... My switches run the codes I need them to, and I very much hope that in the future I can record some sort of raw, be it DNG, or MLV.

I'd love to be able to help out here, but without being instructed to try specific things, I don't know what I can do. I'll continue to do my own testing and post results, so please feel free to recommend certain (and specific) things, but keep in mind I am not a programmer and beyond the basics, learning python is a pain in my ass.

With all of that said, I do have a VNC on my pi and it's setup to work with remote connections, so I'm happy to connect with people and let them control my system and test scripts and stuff to give someone with more knowledge a chance to make some progress...

Anaconda_

I've added a .rgb file, which should be fairly raw. I've read you can decode it with dcraw, I have dcraw installed, and am just working out how to use it.

I'm not having any luck and need to stop working on this project for now.. Anyone else can feel free to try and get things working :)

Let me know how you get on!

yellowguy

Was about time somebody got around to this right?  ;)

Sample footage shot from the Raspberry Pi HQ Camera, 12-bit DNG 2028x0180 @ 24fps



You can read more here: https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=296776

If you can/want to help contribute, stay tuned I'll be posting guides/source to how I did it. Hopefully it can grow to a working camera system eventually.

shaun26


Kharak

Can someone correct me if I am wrong, but is this not the exact same concept as Axiom Camera? Open Source, all schematics on the table, if you got the skill you change any part you see fit? Or is there a fundamental difference, if so, what is it?

I must admit I personally only know rasperry pi as a TV computer.
once you go raw you never go back

2blackbar

Kharak its "just" a sensor with c mount
forget it, all pi card writing speeda are crap even if sensor outputs raw its a waste pf time until they upgrade components but its gonna be awhile , got to milk these old components first.
Use phone, its just a funky webcam sold separstely
raw in 1080 is fne i guess but card readers on pi are bootlenecking the process, there wont be brewkthrough witjout faster ones but id love to be proven wrong  of coures , read specs and its all clear.


Yellowguy whats the highest resolution in 24fpa this sony sensor can output to usb3.0 ssd? theory i mean.

the idea of working on rapberry softwareto enable raw recording is great but current specs of components isnt good enough for above 2k
Is it 1080 skipped or binned ? If its possible to get 4k raw with current components on raspi biards , ill buy one or two.
You should kickstart it, fundraise it.
In order to get 24fps@4056 (H) × 3040 (V) with 12bit RAW, the total bandwidth requires 4056x3040x24fpsx12/8=443.89MB/s

--
Soooo... i did some digging and usb 3.0 on raspberry 4 (best newest one) is able to push 620 megabits per second at max...
620 Megabit/Second (Mbit/s) = 77.5 Megabyte/Second (MB/s)

Which means its not enough to intercept 4K frames at 24fps , maybe if raw stream would be compressed but that would reduce fps unless done by hardware.
OR is it 5 Gbps   which means 625 MB/s ? Cause thats definitely capable of 4k raw acquisition.

OK some guys are able to get around 340MB per second with SSD drives on pi 4
So thats barely enough for 4k raw stream.
2:30:



So... sigma FP camera outputs 12 bit raw frame at 2400Mbps which is 300MB per sec data rate , there might be a chance , depending how heave one dng 4k frame from pi is.
So i might be proven wrong eventually which is amazing.

Kharak

Thanks for clarifying.

I borrowed a Pi once from a friend, but it was only for watching films on the TV.
once you go raw you never go back