Messages

You'll see the resulting output is no different than the profile with a log curve in the User RGB gamma block - I'm still trying to work out what that log curve is for?

I replaced the RGB gamma blocks but the points will not show in PSE because there is/was no editable Tone Curve in a pf2. Cinestyle is a pf2 and has a tone curve so it's just a PSE omission. The editable Tone curve was added later (presumably under a different tag?) as part of the pf3 format.

You've been doing inspiring research! So you came to a conclusion that User RGB gamma block is specific to pf3 format only and is not supported/conveyed by pf2 format?

Wow! I don't know where did you get that list but that really covers a whole lot of things. Awesome!!

Yep, makes sense. By testing with PSE, the tags affected by making adjustments on PSE's Basic tab and saving a file are the following (Tone Curve not included):

- 0x1002: Caption (type 0x0002 ~ ASCII)
- 0x1009: Base Picture Style (type 0x0003 ~ Short)
- 0x100a: Copyright (type 0x0002 ~ ASCII)
- 0x1030: Strength (type 0x0004 ~ Long)
- 0x1031: Contrast (type 0x0004 ~ Long)
- 0x1032: Saturation (type 0x0004 ~ Long)
- 0x1033: Color tone (type 0x0004 ~ Long)

These are not verified, but I guess semantics for the following tags that seem to be of Long type (0x0004) as well:
- 0x1034: Fineness
- 0x1035: Threshold

Here are some actual tested hexadecimal responses for changing basic adjustments in PSE.

Base Picture Style:
  1009 0003 0000 0002 0081  # Standard
  1009 0003 0000 0002 0082  # Portrait
  1009 0003 0000 0002 0083  # Landscape
  1009 0003 0000 0002 0084  # Neutral
  1009 0003 0000 0002 0085  # Faithful

Strength:
  1030 0004 0000 0004 0000 0000  # Strength: 0
  1030 0004 0000 0004 0000 000a  # Strength: 10

Contrast:
  1031 0004 0000 0004 ffff fffc  # Contrast: -4
  1031 0004 0000 0004 0000 0000  # Contrast: 0
  1031 0004 0000 0004 0000 0004  # Contrast: 4
 
Saturation:
  1032 0004 0000 0004 ffff fffc  # Saturation: -4
  1032 0004 0000 0004 0000 0000  # Saturation: 0
  1032 0004 0000 0004 0000 0004  # Saturation: 4

Color tone:
  1033 0004 0000 0004 ffff fffd  # Color tone: -3
  1033 0004 0000 0004 0000 0000  # Color tone: 0
  1033 0004 0000 0004 0000 0003  # Color tone: 3

Here is the caption field from Example-Decoded.pf3 (provided by @chris_overseas in a previous message) and guesses for the structure:

  1002 # tag?
  0002 # type: ASCII?
  0000 0020  # length?
  4341 5054 494f 4e5f 4341 5054 494f 4e5f  # "CAPTION_CAPTION_"
  4341 5054 494f 4e5f 4341 5054 494f 4e00  # "CAPTION_CAPTION<NUL>"

@chris_overseas: Thanks for the clear explanation! You have clever ideas on further steps to resolve the key behind picture style files!

@Danne, @DeafEyeJedi: Btw, on Windows 10 machine, these example XOR calculations can be quite easily repeated with basic Calculator application in programmer/hex mode. There one can type in hexadecimal characters (number digits and letters between A-F) and perform XOR operations between two hexadecimal numbers.

I think in order to fully reverse-engineer .pf2/.pf3 format we should probably start with the lowest hanging fruits and then make progress to the harder parts (e.g. encoding of 6 color-axes data, possibly in form of ICC profile). One of the easiest things to do (still probably not very easy), as chmee presented in a previous post, is to reverse-engineer encoding of caption and copyright since by focusing on those two fields we can apply a so-called "chosen plaintext attack" as the exact input is known and can be freely selected. As I mentioned in my previous post, I think one component of encoding caption/copyright is applying a mathematical bitwise "Exclusive or" (XOR) operation on plaintext (https://en.wikipedia.org/wiki/Exclusive_or). XOR takes two operands; in this case one operand seems to be the plaintext string given by the user while saving a file in Canon Picture Style Editor and the other one is an opaque bunch of bytes the semantic of which is unknown. Resolving/understanding that unknown is IMO one step towards reverse-engineering this file format.

Hi!

I think there seems to be XOR operation taking place in encoding of picture style files. Take a look at the latter pair of chmee's example files having title and copyright fields consisting of series of single letters: A and B, respectively.

(1) In the first file the title consists of 10 x 'A' and it is encoded in hexadecimal as: 580F 2FC2 552F 2E76 D23D. When XOR'ed with 10 x 0x41h ("AAAAAAAAAA" in hexadecimal) we get: 194E 6E83 146E 6F37 937C

(2) In the second file the title consists of 10 x 'B' and it is encoded in hexadecimal as: 5B0C 2CC1 562C 2D75 D13E. When XOR'ed with 10 x 0x42h ("BBBBBBBBBB" in hexadecimal) we get: 194E 6E83 146E 6F37 937C. The same as with the first file!

Last not least, if saving/restoring metadata doesn't do it, maybe it's worth a try to remove the dng from the catalog and then re-add it, maybe not all "internal" tags Adobe doesn't expect to change are updated when reading an exif-modified dng (just a guess).

Thanks for the tip, that worked for me! After removing the photo from the catalog and reimporting it, the white level seems to be again at a normal level.

You can check with "exiftool IMG_1234.dng -WhiteLevel=3000"; if you can recover the highlights, all is fine. With ufraw you can't.

Tried this briefly with one DNG image file in Adobe Lightroom 5 and, indeed, the highlights were lost. After setting the white level to 3000 in the EXIF metadata, one cannot recover the lost highlights using the conventional Exposure, Highlights, White or Tone Curve adjustments.

In fact, my attempts to restore the original white level back by executing:

Code Select Expand

exiftool IMG_1234.dng -WhiteLevel=15000

on the command line did not succeed completely even after reading the metadata back from the file. Changing DNG camera profile from Adobe Standard to Camera Standard on the Camera Calibration tab fixed the issue temporarily but after returning from the Library module back to Develop module, the highlights seemed to be lost again. I should better comprehend myself with the Lightroom's metadata management since I don't quite understand how reverting the white level setting (at EXIF level) should be done in Lightroom. Anyway, it seems to me that the most recent release of this software does not resolve the white level from the actual image data.