Messages

There's nothing to gain creating 16bit images from 8bit, it just gets padded with zeros, unless you use something like a denoiser for example to make interpolated values going 8 to 16bit image files or even denoise at 32bit in memory within the app, then there might be some value in generating and storing 12mb per frame 16bit image sequences compared to 2mb 8bit frames. But the best place to do that is in memory within a 32bit float workspace.

I get what you're saying but I don't agree with you're point, sorry.  As long as the approach is consistent on both 'streams' either apply the LUT first or after which as said is best done in a 32bit workspace, non linear even for the difference it'll make and how / where else can a LUT be applied on an image sequence other than in a suitable app? Then preferably at higher precision than 8bit.

Again all this LUT stuff just applies to Cinestyle, can just grade it like any other source, don't have to apply the LUT. How would anyone treat flaat10 say, just grade it and that would be after merging?

There are no 'correct brightness' Cinestyle like Neutral, flaat or whatever is just a gamma encoded representation of the scene in sent to the camera encoder as 8bit 0 - 255 JFIF raw 4:2:2 YCC and  twisted into a BT709 primaries, full range luma, BT601 matrixed (T2i, 5D MKII) compressed h264 file. :-)

The closest we get to a 'physical / realistic way' is to blend and merge in linear light after first making an attempt to correctly decompress and more difficult to correctly linearize that twisted h264 source, going back to my earlier comment about more importance on doing the initial YCC to RGB conversion as 'best' we can. For example I don't like the way AE manages that.

Consistent approach applying LUT or grade prior or after treating the two 'streams' the same is all that is necessary.

(Because even if you're working in 32 bit linear you can alter the pixel's in a non-linear way

Not if your app linearizes the source correctly and that using non 32bit aware plugins, filters and effects is avoided. But if you mean you will see a different result compared to applying the LUT in a gamma encoded workflow, sure.

(baking the gamma) and it's still linear workspace and if you interpolate "wrong" values in a non-linear way (for example gamma 2.2) you will get a totally wrong result.

'Wrong' is subjective, the whole process is based on interpretation of the source anyway. It's also dependent on how the application works whether just blending in linear light, color processing may not be done in linear light. There are many steps to just getting an RGB image sequence out that can produce 'wrong' results to get picky about before being concerned about merging Cinestyle or LUT applied version.

That's why I think it may be important to deal with the exact time to apply the LUT to get the correct color values)

Fair enough, but 'correct' is subjective, like said earlier just test a 'before'  and an 'after' merging, see which looks the best but that has nothing to do with producing 16bit image sequences which make no improvement.

Just apply the LUT at 32bit precision in the NLE, Compositor before or after. Again this is just to do with Cinestyle, it's just a RGB curve to take a LOG looking image to a more typical rec709 image nothing more, just a grade, nothing to do with 'physically correct brightness'

Applying the 1D LUT would be via an NLE or AE or Nuke or something, so that's best done at higher than 8bit precision yes, but no point creating 16bit images to apply the LUT just work at 32bit float in the NLE or AE.

And if talking about 16bit precision, 32bit is more the norm as a working space and taking an app like AE for example, importing an 8bit file into 16bit workspace differs from importing into 32bit, in a 32bit workspace the 8bit level 0 is centered in the 65536 range so there's room for negative values too, 16bit maps the 8bit levels into 32768 levels prorata.

http://blogs.adobe.com/VideoRoad/2010/06/understanding_color_processing.html

But it's academic really, it's the way the initial conversion from YCC video to RGB image sequences is done that perhaps matters more as I'd assume Photomatrix will be working at 32bit precision linear light anyway on even 8bit image sequences.

Personally I'd just test a few picture styles. Applying the 1D LUT only applies to Cinestyle anyway. Test applying before and after merging see which you like best.

16bit won't gain you anything, nor will EXR and color space is going to be BT709 / sRGB. When you create your EXRs or other image format, really EXRs should be linear light, ie sRGB with no 'gamma' curve. Although Photomatrix will probably handle the linearizing anyway just will expect pngs or jpgs to be 2.2 gamma, EXRs 1.0.

How you produce the image sequences is up to you, how do you intend doing that?

Use silent pictures - that's the best quality you can get. Jpegs are 422.

Cheers, but I've been there, other threads ;-) I'm interested in comparisons between the three in other regards.

Is there any chance of a jpg via your work (whilst not recording), a short mov and a jpg from photo mode of the same static scene for comparison?

I had done on a number of silent pics whilst recording / not recording, is why I said what I said. :-)

To my eyes the larger of the two resolutions looks softer but ok I stand corrected. :-)

re T2i feed to encoder.

1056x704 when not recording uprezzed whilst recording and as a result softened to 1728x972 to feed the encoder? Would that be more accurate assumption?

For a T2i the feed into the encoder is assumed to be 1056x704, it then gets upsampled/uprezzed to 1920x1088. have a look at a uncompressed 4:2:2 silent pic captured in movie mode, but whilst not recording and see how much sharper it is.

And on a T2i the stream is actually 1088 in the MOV, not 1080. The decompressing codec or media player crops 8 rows off the bottom. VLC screenshot will give 1088 and show the bottom 8 is actual usable not so called black padding for mod16.

Let me share with you my findings with the H264 parameters in the .ini file, I did tons of experiments with Alex back in May, during the quiet days after the "Hello World" :-)

Code Select Expand

SarWidth = 0
SarHeight = 0
AspectRatioIdc = 1
VideoFmtAndVspFlag = 81
[b]VideoFullRangeFlag = 1[/b]
TimingInfoPresentFlag = 0

not very important parameters, doesn't affect the video much.

Hope this information is helpful and can save you some time, good luck folks!

Personally would like the 'VideoFullRangeFlag' on/off as an option for an ini file. :-)

With the fullrangeflag set on as it is with Canon MOV's it signals to the decompressing codec to squeeze luma levels 16 - 235 in any import into the NLE or any transcode / general encoder output. So it can be normalized into 0-1 ie: 0 - 255 in 8bit RGB terms.

I currently use a patched build of MP4Box to be able to switch this flag off in the h264 stream which involves a remux to mp4.

Although the feed into the camera encoder appears to be raw uncompressed JFIF ie: full luma/chroma levels over 0 - 255, the option to switch the flag off for the 32bit float < 0.0  > 1.0, so overbrights exist may be useful.

I kind of thought that would be the intention with 4:2:2 anyway as mentioned on the silent pic thread, maybe better to resize in post, I've been looking at resizing methods with the 4:2:2 silent pic 1056x704 from T2i and Avisynth with ResampleHQ filter provides a heap of different algos, does it a 32bit precision and in linear light, also the Dither tools plugin has uses too.

Admittedly this was done on uncompressed 4:2:2 rather than h264 but maybe theres some milage in the resize after anyway apart from upping bitrate, perhaps decent 720P (after resize) at 60fps or higher frame rates at 1056x704 for T2i maybe?

Cheers, sounds good.

Andy600, a quick look at your LOG PS stills, they look good, apart from no sky detail, but assume that was an exposure choice. Looking forward to hearing more about your PS and seeing some more.

What NLE and LUT format?

ha ha that's what my thinking was looking at the uprezz in video mode, if silent pics are bigger in photo mode, then phot mode could be the new video mode. :-)

Wonder what fps can be sustained with .422's even at Simple in movie mode and why so many variations in rez?

Not directly related with regard to encoding but post deblocking h264 with a 'decent' deblocker + a little temporal denoising an give a 'better' grain appearance subjectively but a little loss in detail. Fine line between keeping the detail and losing sharp edges to blocks.

T2i / 550d is 1056x704 when not recording, when recording movie silent pic is 1700xwhatever but quality is not as sharp, looks uprezzed, comparing 4:2:2 1056x704 to MOV shows MOV is soft too but that's no secret I guess, silent pic is not exactly new.

Agree would be interesting to get stream unuprezzed and do it on the PC after.

Yes, same here on a T2i with 2.3 everytime I first swith on, the first recording stops a second or two.
1.3x CBR on a high speed Sandisk Extreme, very rarely did a recording stop at 1.3x CBR prior to 2.3

I'm new to the silent pic feature so its probably already well known but it appears the raw yuv is JFIF that is chroma over full 8bit range, along with the luma and part of JFIF is that chroma is stored +- 128 centered about 0.

http://www.jpeg.org/public/jfif.pdf

Where as Rec BT 709 chroma is stored over limited range 16 - 240 so in avisynth shifting the chroma to 16 - 240 range which is what is assumed the output would be if unsigned.

Comment by um3k at Doom9:

Strangely, the chroma is offset and wrapped. My guess is that whatever program made it used a signed integer when it should have used unsigned.

My response:

Wrapped? The mt_lut seems to be shifting the chroma? Is the source in that case JFIF? Where chroma is full range and I think described as centred at 0 +128 & - 127? If that makes any sense.

um3k:

I'm having trouble finding a reference for JFIF chroma storage, but if I understand your post correctly, that could indeed be the case, and would make sense in the context of a (primarily) still camera. Regardless, a modulo addition works quite nicely to fix it.

http://forum.doom9.org/showthread.php?t=165831

What is maybe interesting is if you were able to change to unsigned whether chroma in the raw yuv would then be stored rec709 and if firmware more up stream still fed it full range as if JFIF then full chroma rec709 + full luma would be xvYCC extended gamut.

http://www.sony.net/SonyInfo/technology/technology/theme/xvycc_01.html

I believe the Digic3 & 4 supports xvYCC anyway, somewhere.

JFIF is a bit of a ugly YCC format and poynton is quite critical.

http://books.google.co.uk/books?id=dSCEGFt47NkC&pg=PA175&lpg=PA175&dq=chroma%2BJFIF%2Bvideo&source=bl&ots=OL4zeJ7Vyr&sig=aoNstetOv-HN7PpZ3i-dOj2JxVs&hl=en#v=onepage&q=chroma%2BJFIF%2Bvideo&f=false

I've had problems getting a correct frame via Avisynth, so asked the Doom9 community for help, they found a solution and suggest that the silent pic may not be being written correctly as it uses signed integer rathet than unsigned?

Any dev comment regarding this? I know tools exist that give a usable image from silent pic but...

Rehaan, first I'd say there is no issue regarding the Canon MOV's, as DFM has described above the process is simple.

But there are a couple of things I'd throw in for discussion that appear to me not quite right but others more knowledgeable may disagree. Then you can decide if it's an issue or not. :-)

The first step DFM describes is to use AE for importing your MOV's to export to 32bit RGB OpenEXR. Linear sRGB. So we'd assume our AE project will be 32bit sRGB and linearized working space as any blending of light such as merging exposures is best done in linear light and although we could use the older 'blend with linear', we are going to be exporting linear EXR's so a linear work space is not unreasonable.

In that process the Canon MOV's will be linearized, ie: the gamma removed, based on the icc profile assigned to the MOV's. Our Canon MOV's are BT709 color primaries, BT709 transfer curve and depending on Canon model either BT709 or BT601 color matrix. Typically described as 'rec709', but they have full range levels like a number of cameras such as Sony NEX5n, FS100 etc so they are not strictly Rec BT709.

AE assigns a display referred icc profile with sRGB 2.2 gamma by default for the MOV's on import, but our MOV's are scene referred and have a rec709 gamma curve. So we could use the option for compensating for scene referred source files. However I've found this still doesn't resolve the problem that follows.

Looking at the histogram, ie: Levels filter on the source files. Linearizing the source with the inverse of a 2.2 gamma curve appears to result in combing at the lower levels, suggesting to me, the wrong calculation is being used to remove the gamma ie: based on the source being 2.2 sRGB rather than rec709. The gamma curves differ between the two.

An alternative would be to tell AE to 'interpret the footage' as rec709 rather than gamma encoded sRGB in order to get the 'correct' inverse curve for the MOV's but on doing this AE assumes a levels range of 16 - 235 luma 16 - 240 chroma where as our MOVs are full range levels so we appear to again get the wrong linearizing of the source files as the histogram looks combed again. Also as mediacore handles rec709 files all options are greyed out.

An alternative is to remove the full range flag set 'on' in the h264 stream by the camera firmware / encoder. This can be done in a quick remux to mp4 with a h264 VUI Options enabled tool like MP4Box.

When we import the remuxed mp4 AE defaults to rec709 and handles the file as limited range even though it has full range levels so the linearizing appears incorrect again.

That's the point of the full range flag set on in the MOV, to signal to the decompressing codec to squeeze the full levels into 16 - 235 when decompressed ready for the NLE etc to convert to 8bit RGB. if the full levels were used RGB channels would almost certainly clip, at 8bit.

With a 32bit float RGB project the whole YCC h264 full range source can be held and manipulated in RGB, generally without clipping channels and losing data.

So when importing MOV's into AE or Premiere the full levels are scaled into 16 - 235 luma, 16 - 240 chroma, and normalized to fit within 0 - 1, ie: 0 - 255 8bit RGB so no overbrights / underbrights can exist.

Where as importing the remuxed mp4 into a 32bit space,  the RGB conversion done over the full range generates overbrights as the source is not normalized.

Which way is the right way is what I'd like to establish, for this camera source anyway.

Sound like someone with a technical knowledge of AE. Excellent. :-) A query or two.

Isn't it rather academic though because unless using Dynamic Linking and not everyone does, the input into AE with regard to grading after an edit won't be the original MOVs off camera anyway but an intermediate format of some description?

Further, that intermediate format unless RGB output will almost certainly be limited range. Canon chose to encode using full luma and chroma ie: JFIF as many DSLR camera manufacturers do in order to make the most of the data. Also flagging the h264 stream to signal 'fullrange' data, but typical output from an encoder receiving full range is to scale it into 16 - 235.

I'd also suggest one reason to transcode or convert to EXR or similar is if importing MOV's directly into a 32bit linearised AE project is that it appears impossible to get a correct degamma'd result because AE guesses sRGB iec61966 for the MOV's, so to linearize,  AE appears to apply a reverse sRGB 2.2 gamma which is incorrect on a rec709 source.

Tell AE to interpret footage as rec709 and it assumes 16 - 235 as range on a full range source which again results in an incorrect result, shown by a combed histogram.

So removing the full range flag and import remuxed mp4 into AE and mediacore imposes rec709 as format, correct but again uses 16-235 range as input on a full range source, so there appears no route other than to feed RGB perhaps as OpenEXR or a range limited YCC intermediate transcode to produce correct results for a linearised 32bit workflow?

Ok, thanks again A1ex that was one of the formats I'd tried as it appears to be the usual for raw generally, will look into it further.

I'd also like to know a bit more about the .422. I've looked at the wiki, used the couple of tools to create jpegs/pngs which don't appear very optimum.

So .422's are uncompressed raw YCC, but what about pixel format? Planar or interleaved, what I'd like to do is open them in Avisynth, but trying raw import plugins like rawsource or sashimi with any YCC pixel formats results in errors.

Silent pic was simple mode. Image size assumed was the 17--x9-- whatever exact dims are.

Cheers

Cheers for the reply. :-)

I'm not a dev but interested in the flow path and state of the data at entry into the h264 encoder.

Is the .422 data (silent pics) considered the state of the feed for the h264 encoder? I've never bothered with them are they really compressed or just subsampled to 4:2:2?