I have this implementation issue, which I'm having trouble applying good judgement, I'd like to survey opinions...

So, RGB colours depend on this 'normalised integer' concept, that is:
  unsigned: luminance = val / IntType.max
  signed: luminance = max(val / IntType.max, -1.0)

So I introduce NormalizedInt(T), which does that.

The question is, what should happen when someone does:
  NormalisedInt!ubyte nub;
  NormalizedInt!byte nb;
  auto r = nub + nb;

What is typeof(r)?

There are 3 options that stand out, and I have no idea which one is correct.
1. Compile error, mismatching NormalisedInt type arithmetic shouldn't
work; require explicit user intervention.
2. 'Correct' result, ie, lossless; is(typeof(r) ==
NormalisedInt!short). Promote to type that doesn't lose precision,
type conversion loses efficiency, but results always correct.
3. Do what normal int types do; is(typeof(r) == NormalisedInt!int) ie,
apply the normal integer arithmetic type promotion rules. Classic
pain-in-the-arse applies when implicitly promoted result is stored to
a lower-precision value. Probably also slow (even slower) than option
#2.

Are there other options?
I'm tempted by #1, but that will follow right through to the colour
implementation, which will lead to colour type cast's all over the
place.

Am Wed, 31 Aug 2016 15:58:28 +1000
schrieb Manu via Digitalmars-d <digitalmars-d@puremagic.com>:

> I have this implementation issue, which I'm having trouble applying good judgement, I'd like to survey opinions...
> 
> So, RGB colours depend on this 'normalised integer' concept, that is:
>   unsigned: luminance = val / IntType.max
>   signed: luminance = max(val / IntType.max, -1.0)
> 
> So I introduce NormalizedInt(T), which does that.
> 
> The question is, what should happen when someone does:
>   NormalisedInt!ubyte nub;
>   NormalizedInt!byte nb;
>   auto r = nub + nb;
> 
> What is typeof(r)?
> 
> There are 3 options that stand out, and I have no idea which one is correct.
> 1. Compile error, mismatching NormalisedInt type arithmetic shouldn't
> work; require explicit user intervention.
> 2. 'Correct' result, ie, lossless; is(typeof(r) ==
> NormalisedInt!short). Promote to type that doesn't lose precision,
> type conversion loses efficiency, but results always correct.
> 3. Do what normal int types do; is(typeof(r) == NormalisedInt!int) ie,
> apply the normal integer arithmetic type promotion rules. Classic
> pain-in-the-arse applies when implicitly promoted result is stored to
> a lower-precision value. Probably also slow (even slower) than option
> #2.
> 
> Are there other options?
> I'm tempted by #1, but that will follow right through to the colour
> implementation, which will lead to colour type cast's all over the
> place.

I'd suspect #1 to be the best option, too. However, I don't know when users will deal with these calculations. Surely adding sRGB(22,22,22) + sRGB(11,11,11) gives sRGB(28, 28, 28), with a higher precision while performing the addition and then rounding back. Anything requiring multiple operations on an image should use a higher precision linear color space from the start.

-- 
Marco

On 2 September 2016 at 06:09, Marco Leise via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> Am Wed, 31 Aug 2016 15:58:28 +1000
> schrieb Manu via Digitalmars-d <digitalmars-d@puremagic.com>:
>
>> I have this implementation issue, which I'm having trouble applying good judgement, I'd like to survey opinions...
>>
>> So, RGB colours depend on this 'normalised integer' concept, that is:
>>   unsigned: luminance = val / IntType.max
>>   signed: luminance = max(val / IntType.max, -1.0)
>>
>> So I introduce NormalizedInt(T), which does that.
>>
>> The question is, what should happen when someone does:
>>   NormalisedInt!ubyte nub;
>>   NormalizedInt!byte nb;
>>   auto r = nub + nb;
>>
>> What is typeof(r)?
>>
>> There are 3 options that stand out, and I have no idea which one is correct.
>> 1. Compile error, mismatching NormalisedInt type arithmetic shouldn't
>> work; require explicit user intervention.
>> 2. 'Correct' result, ie, lossless; is(typeof(r) ==
>> NormalisedInt!short). Promote to type that doesn't lose precision,
>> type conversion loses efficiency, but results always correct.
>> 3. Do what normal int types do; is(typeof(r) == NormalisedInt!int) ie,
>> apply the normal integer arithmetic type promotion rules. Classic
>> pain-in-the-arse applies when implicitly promoted result is stored to
>> a lower-precision value. Probably also slow (even slower) than option
>> #2.
>>
>> Are there other options?
>> I'm tempted by #1, but that will follow right through to the colour
>> implementation, which will lead to colour type cast's all over the
>> place.
>
> I'd suspect #1 to be the best option, too. However, I don't know when users will deal with these calculations.

Neither do I, it's just that NormalizedInt is a type, it's a dependency, it exists, it needs an api, so it needs to be well defined I guess.

I wonder, should NormalizedInt be a module beneath the color package,
or should it be a separate module in its own right?
I don't know of uses for that type outside packed colours, but it
could theoretically be used for anything...

> Surely adding sRGB(22,22,22) + sRGB(11,11,11) gives sRGB(28, 28, 28),
> with a higher precision while performing the addition and then
> rounding back.

Umm, no. I think operations should be within their own colourspace, otherwise what's the point of selecting your working colourspace? You need to be able to do operations in gamma space too. If you want to do a linear operation, cast to a linear type before doing the operation.

> Anything requiring multiple operations on an
> image should use a higher precision linear color space from
> the start.

Right, I think you would typically cast from the storage type to the
working type before doing some work. But there are so many cases, and
various levels of tradeoff between efficiency and correct-ness, the
lib can't make presumptions.
The way it is currently, the operations are done in the typed
colourspace. Simple as that. If you want to do linear operations, cast
to a linear type first.

On 8/31/16 1:58 AM, Manu via Digitalmars-d wrote:
> I have this implementation issue, which I'm having trouble applying
> good judgement, I'd like to survey opinions...
>
> So, RGB colours depend on this 'normalised integer' concept, that is:
>   unsigned: luminance = val / IntType.max
>   signed: luminance = max(val / IntType.max, -1.0)
>
> So I introduce NormalizedInt(T), which does that.
>
> The question is, what should happen when someone does:
>   NormalisedInt!ubyte nub;
>   NormalizedInt!byte nb;

Is it s or z ? :)

>   auto r = nub + nb;
>
> What is typeof(r)?
>
> There are 3 options that stand out, and I have no idea which one is correct.
> 1. Compile error, mismatching NormalisedInt type arithmetic shouldn't
> work; require explicit user intervention.
> 2. 'Correct' result, ie, lossless; is(typeof(r) ==
> NormalisedInt!short). Promote to type that doesn't lose precision,
> type conversion loses efficiency, but results always correct.
> 3. Do what normal int types do; is(typeof(r) == NormalisedInt!int) ie,
> apply the normal integer arithmetic type promotion rules. Classic
> pain-in-the-arse applies when implicitly promoted result is stored to
> a lower-precision value. Probably also slow (even slower) than option
> #2.
>
> Are there other options?
> I'm tempted by #1, but that will follow right through to the colour
> implementation, which will lead to colour type cast's all over the
> place.

In the case that you are unsure, #1 is the only one that leaves room to make a decision later. I think you should start with that and see what happens.

What may turn out to happen is that most people only use one type, and then casts aren't going to be a problem.

-Steve

> Is it s or z ? :)

It's an alias to forego any language bike shedding.

Am Fri, 2 Sep 2016 15:58:05 +1000
schrieb Manu via Digitalmars-d <digitalmars-d@puremagic.com>:

> On 2 September 2016 at 06:09, Marco Leise via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> > I'd suspect #1 to be the best option, too. However, I don't know when users will deal with these calculations.
> 
> Neither do I, it's just that NormalizedInt is a type, it's a dependency, it exists, it needs an api, so it needs to be well defined I guess.

*nods*

> I wonder, should NormalizedInt be a module beneath the color package,
> or should it be a separate module in its own right?
> I don't know of uses for that type outside packed colours, but it
> could theoretically be used for anything...

I guess audio sample would be a perfect match for signed and
unsigned normalized integers.

> > Surely adding sRGB(22,22,22) + sRGB(11,11,11) gives sRGB(28, 28, 28),
> > with a higher precision while performing the addition and then
> > rounding back.
> 
> Umm, no. I think operations should be within their own colourspace, otherwise what's the point of selecting your working colourspace? You need to be able to do operations in gamma space too. If you want to do a linear operation, cast to a linear type before doing the operation.

So it is not a library that unifies color spaces into working
with light intensities, but more of a library to perform
linear blending directly in the color-space.
That means when I add two HSV colors of the same ubyte hue
= 200, the result will have hue == 144, right ?

> > Anything requiring multiple operations on an
> > image should use a higher precision linear color space from
> > the start.
> 
> Right, I think you would typically cast from the storage type to the
> working type before doing some work. But there are so many cases, and
> various levels of tradeoff between efficiency and correct-ness, the
> lib can't make presumptions.
> The way it is currently, the operations are done in the typed
> colourspace. Simple as that. If you want to do linear operations, cast
> to a linear type first.

Ok, might be worth it for people new to color spaces to highlight the fact that RGB is not linear and what that means for operations like brightness or contrast change. I've seen a hobbyist write a 3D engine and after he had deferred rendering, flash lights & shadows, scripting and animations, he stumbled over an article about screen gamma and why it matters. My first attempts at image manipulation show the same ignorance.

-- 
Marco

On 2 September 2016 at 22:36, Steven Schveighoffer via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> On 8/31/16 1:58 AM, Manu via Digitalmars-d wrote:
>>
>> I have this implementation issue, which I'm having trouble applying good judgement, I'd like to survey opinions...
>>
>> So, RGB colours depend on this 'normalised integer' concept, that is:
>>   unsigned: luminance = val / IntType.max
>>   signed: luminance = max(val / IntType.max, -1.0)
>>
>> So I introduce NormalizedInt(T), which does that.
>>
>> The question is, what should happen when someone does:
>>   NormalisedInt!ubyte nub;
>>   NormalizedInt!byte nb;
>
>
> Is it s or z ? :)

Oh piss off! ;)
I have to think *REALLY HARD* to write 'z', and when I type it without
thinking I accidentally write it correctly.
It's SO HARD to constantly spell wrong in this code!

On 3 September 2016 at 03:25, Marco Leise via Digitalmars-d <digitalmars-d@puremagic.com> wrote:
> Am Fri, 2 Sep 2016 15:58:05 +1000
> schrieb Manu via Digitalmars-d <digitalmars-d@puremagic.com>:
>
>> I wonder, should NormalizedInt be a module beneath the color package,
>> or should it be a separate module in its own right?
>> I don't know of uses for that type outside packed colours, but it
>> could theoretically be used for anything...
>
> I guess audio sample would be a perfect match for signed and unsigned normalized integers.

Perfect! I moved out outside the colour package. Contention resolved :)


>> > Surely adding sRGB(22,22,22) + sRGB(11,11,11) gives sRGB(28, 28, 28),
>> > with a higher precision while performing the addition and then
>> > rounding back.
>>
>> Umm, no. I think operations should be within their own colourspace, otherwise what's the point of selecting your working colourspace? You need to be able to do operations in gamma space too. If you want to do a linear operation, cast to a linear type before doing the operation.
>
> So it is not a library that unifies color spaces into working with light intensities

That's a design goal, and it's very easy to cast to a linear type to do that work.


> but more of a library to perform
> linear blending directly in the color-space.

This is another goal.
A colour implementation in the std library is a little tricky, it
needs to be both correct AND efficient for the different use cases
where the trade-off lines are drawn differently.
The only reasonable way I can think to support this is to do
operations within the typed colourspace, and let the user cast to the
colourspace desired for their operations.
So the meat of the lib is really about convenient colourspace
conversion, and making the colourspaces easy to describe, particularly
for non-experts.

I'm feeling like it's getting pretty close to my intent.

Naturally, image processing libs would tend to do linear conversions
internally. So it shouldn't be a detail that reaches the end-user very
often.
Remember that when doing linear conversions of colours, you tend to
also need to use float colour channels, since floats naturally express
the same non-linear precision curve that the gamma curves exist to
compensate for. Linear RGB8 will severely lose precision and band very
badly.


> That means when I add two HSV colors of the same ubyte hue = 200, the result will have hue == 144, right ?

Yes, it would necessarily be that. If it didn't behave that way, you
wouldn't be able to express a lerp in HSV space.
Additive colour blending in HSV doesn't really make sense. It works as
expected if you're adding deltas.

Ideally, hue would be expressed with a circular angle type (rather than NormalizedInt as I've used for the other channels), which is something we don't have in phobos, and I'm not motivated to write one just for this one color channel...


>> > Anything requiring multiple operations on an
>> > image should use a higher precision linear color space from
>> > the start.
>>
>> Right, I think you would typically cast from the storage type to the
>> working type before doing some work. But there are so many cases, and
>> various levels of tradeoff between efficiency and correct-ness, the
>> lib can't make presumptions.
>> The way it is currently, the operations are done in the typed
>> colourspace. Simple as that. If you want to do linear operations, cast
>> to a linear type first.
>
> Ok, might be worth it for people new to color spaces to highlight the fact that RGB is not linear and what that means for operations like brightness or contrast change. I've seen a hobbyist write a 3D engine and after he had deferred rendering, flash lights & shadows, scripting and animations, he stumbled over an article about screen gamma and why it matters. My first attempts at image manipulation show the same ignorance.

I understand this pattern very, very well. I've worked in games my
whole career ;)
This is precisely why the std library needs to be batteries-included
with respect to knowledge about colourspaces and conversions, but that
doesn't mean every operation can do it properly behind the scenes.
Colours are a fairly low-level primitive, and appear in
low-level/high-frequency code. The std library offering wouldn't be
very useful if it were so slow from doing linear conversions every
time you do any operation. That's a lot of int<->float conversions,
and pow's!

I think the presence of all this colour space information as type arguments should nudge users in the right direction. They'll be all "I've never seen this parameter before..." and google it... maybe. I don't think it's the std lib doco's job to give users a lesson in colour theory...? :/

On 03/09/2016 12:17 PM, Manu via Digitalmars-d wrote:
...snip...

> I think the presence of all this colour space information as type
> arguments should nudge users in the right direction. They'll be all
> "I've never seen this parameter before..." and google it... maybe.
> I don't think it's the std lib doco's job to give users a lesson in
> colour theory...? :/

Something[0] along this line perhaps?

Overview of the choices and scope along with reasoning.

[0] https://github.com/rikkimax/alphaPhobos/blob/master/source/std/experimental/graphic/image/specification.dd

Am Sat, 3 Sep 2016 16:01:26 +1200
schrieb rikki cattermole <rikki@cattermole.co.nz>:

> On 03/09/2016 12:17 PM, Manu via Digitalmars-d wrote: ...snip...
> 
> > I think the presence of all this colour space information as type arguments should nudge users in the right direction. They'll be all "I've never seen this parameter before..." and google it... maybe. I don't think it's the std lib doco's job to give users a lesson in colour theory...? :/

:D Just a short paragraph. "Note: The common sRGB color
space used in computer screens or JPEGs from digicams does not
evenly distribute brightness along the pixel values.
Multiplying an sRGB pixel by two, wont double its intensity!
To perform accurate image manipulations, you are advised
to always convert to a high precision linear color-space like
[insert name] first.

More information on the history of sRGB and the formulas used can be found here: https://www.w3.org/Graphics/Color/sRGB"

I believe sRGB is the only color-space that you "feel" you are already familiar with as a computer person, because you see and use it all the time.

If I really wanted to make you pull your hair I'd say: "Man this NormalizedInt stuff would so benefit from MMX. Imagine we don't use float but ushort as linear RGB value and then use a single PADDUSW to add two colors."

> Something[0] along this line perhaps?
> 
> Overview of the choices and scope along with reasoning.
> 
> [0] https://github.com/rikkimax/alphaPhobos/blob/master/source/std/experimental/graphic/image/specification.dd

I have not found text about color spaces there, but it is an interesting collection of API design rationales. "If it mutates it may throw. If it doesn't mutate it shouldnt." I never thought about it that way.

-- 
Marco