On Tue, 07 Jan 2014 12:13:18 -0800, Ola Fosheim Grøstad <ola.fosheim.grostad+dlang@gmail.com> wrote:

> On Tuesday, 7 January 2014 at 19:31:38 UTC, Adam Wilson wrote:
>> True enough, but for the moment we are trying to keep the scope of the project manageable. If someone wants to add it great! But until then we need to focus on what is required.
>
> Yes, that is true, and then we should define a set of applications which it will be suitable for and what the basic model is: Should it be immediate mode, retained mode or scene graph based?
>

I think Immediate Mode offers the greatest flexibility and performance. It's also the easiest to implement since you don't have to build the mountains of data structures needed to retain the scene info.

> High level graphic frameworks tend to loose momentum fast, even the good ones. Example: SGI's Open Inventor is actually a neat scene graph framework, and open source, but dead. SGI's GL survived because it was low level and immediate mode.
>

Cinder, Processing, etc are High-Level Immediate Mode, I think that offers the best chance of succeeding.

> So, I really think D is better off providing the basics in phobos first, staying true to the virtue of providing independent modules that are focused:
>
> - OS application abstraction: graphics context, input stream, audio playback
> - generally useful vector path datatype compatible with phobos-collections and SVG
> - vector/matrix library with competitive SSE performance and features such as clamping

To a large degree I agree with this. Getting some basics into Phobos is an excellent idea and most of the community seems to agree. The biggest problem I can see is that windows are usually tied to the graphics framework the implements them. There are numerous reasons for that, all of which make sense. For example, I don't know if it would be possible to pass the Phobos window to Qt or DirectX, it might be, but we'll have to be very careful in our API design if it is.

-- 
Adam Wilson
IRC: LightBender
Aurora Project Coordinator

On 2014-01-07 19:12, "Ola Fosheim Grøstad" <ola.fosheim.grostad+dlang@gmail.com>" wrote:

> I honestly think it will be easier to maintain and keep up to date with
> changes in OS-X if done entirely in Objective-C. If you are interested
> in that then I'd be interested in cooperating on a runtime wrapper that
> can be used both from C++ and D for OS-X 10.x+ and ios5.1+.

I don't know, I have quite a lot on my plate currently.

-- 
/Jacob Carlborg

On Tuesday, 7 January 2014 at 20:06:47 UTC, Adam Wilson wrote:
> Right, but Mike Parker has experience doing this, his opinion

(I don't know Mike, but it doesn't matter, I never care about technical opinions anyway. I care about arguments, so name dropping has zero effect on me. Even Carmack have opinions that are wrong.)

> counts for quite a bit. His biggest point however is that the high-level API should be completely independent of the low-level API's.

That's not possible. The GPU pipeline defines a design space. For 2D graphics it consists of texture atlases, shaders and how to obtain "context coherency" and reduce the costs of overdraw. If you stay in that design space and do it well, you get great speed and can afford to have less efficient higher-level structures creating a framework that is easier to use. The more low level headroom you have, the more high level freedom you get.

The more speed you waste on the lower levels the more constrained and annoying using the high level api becomes, because you have to take care to avoid lower level bottlenecks.

Which is a good argument for retained mode at the cost of latency for high level frameworks.

> The high-level API describes what the user wants and it's up to the graphics API implementor to get it right.

That is the scene-graph approach: Cocos2D, HTML, SVG, VRML, Open Inventor etc

> to the trouble of making OpenGL render 2D shapes in 3D space, I've done that before, it's not easy. One of the more difficult problems is converting 2D pixels into the Cartesian coordinates system while accounting for DPI. It's doable, but it's more a

Well, not sure why DPI is a problem, but managing dynamic atlases (organizing multiple images on a single texture) in an optimal and transparent manner requires an infrastructure. Sure.

On Tuesday, 7 January 2014 at 20:23:57 UTC, Adam Wilson wrote:
> I think Immediate Mode offers the greatest flexibility and performance.

But only if you do culling yourself and then render all your butterflies first, then all your trees, then all your stones etc in order front to back... So it isn't really high level unless you only have a few objects to render, but it is the easiest to implement, I agree.

> To a large degree I agree with this. Getting some basics into Phobos is an excellent idea and most of the community seems to agree.

Yes, I think looking at 3-4 medium size applications that use the basics is a good starting point. Allows you to ask yourself: what framework would have made it easier to create these applications? What would a particular framework design have made more difficult?

> The biggest problem I can see is that windows are usually tied to the graphics framework the implements them.

One could start with a very simple window abstraction as pointed out earlier in the thread and only allow the program to instantiate a subclass of that abstract window called "RenderWindow" or something like that, then throw an exception if the program tries to instantiate more than one. It could be made reasonable forward compatible by keeping the functionality in the abstract window class to a bare minimum initially.

I also kind of think that having an immediate mode high level API is a bit pointless. It would allow some nice abstractions, but I would still favor retained mode and maybe some quick helper functions that could partially imitate immediate mode (though that should really be a lower priority).

I understand that immediate mode maybe is more intuitive if you are new to graphics, and that it may also be easier to write for Aurora, but I feel like in the long run it would be a detriment.

> So, I really think D is better off providing the basics in phobos first, staying true to the virtue of providing independent modules that are focused:
>
> - OS application abstraction: graphics context, input stream, audio playback
> - generally useful vector path datatype compatible with phobos-collections and SVG
> - vector/matrix library with competitive SSE performance and features such as clamping

That is pretty much what I was thinking as well when I was talking about avoiding breaking things into some std.aurora package. Things that phobos could already stand to have such as vectors, audio, better netcode, and so on, may as well be integrated in phobos and then used in Aurora, rather than Aurora becoming some weird extension of phobos.

On Tuesday, 7 January 2014 at 23:08:42 UTC, Ross Hays wrote:
> I also kind of think that having an immediate mode high level API is a bit pointless. It would allow some nice abstractions, but I would still favor retained mode and maybe some quick helper functions that could partially imitate immediate mode (though that should really be a lower priority).

I've looked a bit at the Open Inventor spin-offs: vrml, x3d, Coin3d. The latter is actually a BSD clause updated Open Inventor 2.1 implementation, but it is big and not really useful since it is C++. Except as a source for ideas, perhaps. I really wish I knew more about SGI's Performer, which was a performance oriented scene graph toolkit which could run in parallell.

One possible solution that makes a trade off between ease-of-use and performance:

If one accepts an extra latency of 1 frame, then a "buffered" mode with intelligent caching might be possible, as a programmer friendly effort. That is:

Frame 0:
Thread A: collect draw calls and update accelerator structures, cache where possible
Thread B: sleeping

Frame 1:
Thread A: collect draw calls and update accelerator structures, cache where possible
Thread B: flush optimized and reordered draw calls to GPU (collected in frame 0)

> That is pretty much what I was thinking as well when I was talking about avoiding breaking things into some std.aurora package. Things that phobos could already stand to have such as vectors, audio, better netcode, and so on, may as well be integrated in phobos and then used in Aurora, rather than Aurora becoming some weird extension of phobos.

Yes, I think there are many useful building blocks that can be useful in batch mode (for conversion tools) and for game-servers that can later be useful for building a render engine.

- Spatial acceleration structures
- ray intersection tests
- object-object intersection tests
- various geometric algorithms (Graphic Gems series)
- mesh algorithms

A lot of this stuff could play nicely together with the collection apis, I think.

What would be nice to have in phobos is a idiomatic COLLADA DOM and helper functions, because it is a quite intricate file format. Maybe that could be a starting point for an engine.

On 1/8/2014 5:35 AM, "Ola Fosheim Grøstad" <ola.fosheim.grostad+dlang@gmail.com>" wrote:
> On Tuesday, 7 January 2014 at 20:06:47 UTC, Adam Wilson wrote:

>
>> counts for quite a bit. His biggest point however is that the
>> high-level API should be completely independent of the low-level API's.
>
> That's not possible. The GPU pipeline defines a design space. For 2D
> graphics it consists of texture atlases, shaders and how to obtain
> "context coherency" and reduce the costs of overdraw. If you stay in
> that design space and do it well, you get great speed and can afford to
> have less efficient higher-level structures creating a framework that is
> easier to use. The more low level headroom you have, the more high level
> freedom you get.

It is very much possible. There are a number of high-level graphics APIs out there designed in just such a way, many of which predate shader APIs. Java2D is a prominent one. For the sort of package we're discussing here, you should never define the high-level API in terms of low-level API features. Shaders shouldn't even enter into the equation. That's the sort of backend implementation detail that users of the API shouldn't have to worry about.

What you're talking about is fine when you have the luxury of supporting a limited number of platforms, or are working in a specific application space (games, for instance). In those cases, performance is going to trump generality and you can model your API more closely to the hardware. But for a general-purpose graphics API that needs to be available on as many devices as possible, that's not going to work. And anything that's going to be considered for inclusion into Phobos probably should favor generality over performance. Ideally, the default backend should be software, so that the API can be used even when graphics hardware is not available (though I'm not saying that's a realistic target to start out with).

I realize that we're in an age where GPU programming is becoming more common and even cell phones have good hardware acceleration, but another consideration is the target audience. If you want an API that a non-graphics programmer can more easily get up to speed with, then that's another reason that you can't let the low-level influence the design. The closer you get to the details, the more difficult it is to learn.

It's quite possible to put together a reasonably performant, easy-to-use, general purpose graphics API that can take advantage of the best renderer on a given platform without letting the low-level details leak into the high-level design.

On 2014-01-08 03:38, Mike Parker wrote:
> On 1/8/2014 5:35 AM, "Ola Fosheim Grøstad"
> <ola.fosheim.grostad+dlang@gmail.com>" wrote:
>> On Tuesday, 7 January 2014 at 20:06:47 UTC, Adam Wilson wrote:
>
>>
>>> counts for quite a bit. His biggest point however is that the
>>> high-level API should be completely independent of the low-level API's.
>>
>> That's not possible. The GPU pipeline defines a design space. For 2D
>> graphics it consists of texture atlases, shaders and how to obtain
>> "context coherency" and reduce the costs of overdraw. If you stay in
>> that design space and do it well, you get great speed and can afford to
>> have less efficient higher-level structures creating a framework that is
>> easier to use. The more low level headroom you have, the more high level
>> freedom you get.
>
> It is very much possible. There are a number of high-level graphics APIs
> out there designed in just such a way, many of which predate shader
> APIs. Java2D is a prominent one. For the sort of package we're
> discussing here, you should never define the high-level API in terms of
> low-level API features. Shaders shouldn't even enter into the equation.
> That's the sort of backend implementation detail that users of the API
> shouldn't have to worry about.
>
> What you're talking about is fine when you have the luxury of supporting
> a limited number of platforms, or are working in a specific application
> space (games, for instance). In those cases, performance is going to
> trump generality and you can model your API more closely to the
> hardware. But for a general-purpose graphics API that needs to be
> available on as many devices as possible, that's not going to work. And
> anything that's going to be considered for inclusion into Phobos
> probably should favor generality over performance.

I would say that we should try and built the API in many different levels and layers. The top layer would be for ease of use to quickly get something to show on the display. If more performance is needed it should be possible to access the lower levels of the API to get more control of what you need to do.

> Ideally, the default
> backend should be software, so that the API can be used even when
> graphics hardware is not available (though I'm not saying that's a
> realistic target to start out with).

Ideally the default backend should detect if graphics hardware is available or not at runtime and choose the best backend for that.

> I realize that we're in an age where GPU programming is becoming more
> common and even cell phones have good hardware acceleration, but another
> consideration is the target audience. If you want an API that a
> non-graphics programmer can more easily get up to speed with, then
> that's another reason that you can't let the low-level influence the
> design. The closer you get to the details, the more difficult it is to
> learn.
>
> It's quite possible to put together a reasonably performant,
> easy-to-use, general purpose graphics API that can take advantage of the
> best renderer on a given platform without letting the low-level details
> leak into the high-level design.


-- 
/Jacob Carlborg

On Tue, 07 Jan 2014 23:52:48 -0800, Jacob Carlborg <doob@me.com> wrote:

> On 2014-01-08 03:38, Mike Parker wrote:
>> On 1/8/2014 5:35 AM, "Ola Fosheim Grøstad"
>> <ola.fosheim.grostad+dlang@gmail.com>" wrote:
>>> On Tuesday, 7 January 2014 at 20:06:47 UTC, Adam Wilson wrote:
>>
>>>
>>>> counts for quite a bit. His biggest point however is that the
>>>> high-level API should be completely independent of the low-level API's.
>>>
>>> That's not possible. The GPU pipeline defines a design space. For 2D
>>> graphics it consists of texture atlases, shaders and how to obtain
>>> "context coherency" and reduce the costs of overdraw. If you stay in
>>> that design space and do it well, you get great speed and can afford to
>>> have less efficient higher-level structures creating a framework that is
>>> easier to use. The more low level headroom you have, the more high level
>>> freedom you get.
>>
>> It is very much possible. There are a number of high-level graphics APIs
>> out there designed in just such a way, many of which predate shader
>> APIs. Java2D is a prominent one. For the sort of package we're
>> discussing here, you should never define the high-level API in terms of
>> low-level API features. Shaders shouldn't even enter into the equation.
>> That's the sort of backend implementation detail that users of the API
>> shouldn't have to worry about.
>>
>> What you're talking about is fine when you have the luxury of supporting
>> a limited number of platforms, or are working in a specific application
>> space (games, for instance). In those cases, performance is going to
>> trump generality and you can model your API more closely to the
>> hardware. But for a general-purpose graphics API that needs to be
>> available on as many devices as possible, that's not going to work. And
>> anything that's going to be considered for inclusion into Phobos
>> probably should favor generality over performance.
>
> I would say that we should try and built the API in many different levels and layers. The top layer would be for ease of use to quickly get something to show on the display. If more performance is needed it should be possible to access the lower levels of the API to get more control of what you need to do.
>

As useful as that may be I want to keep the scope focused for now. A high-level API allows us the most flexibility in terms of implementation and user accessibility. We're not trying to be triple-A performant so building out extra API levels for more performance is a low priority. Let's do something focused and do it well before we expand the scope. This project is already huge. :-)

>> Ideally, the default
>> backend should be software, so that the API can be used even when
>> graphics hardware is not available (though I'm not saying that's a
>> realistic target to start out with).
>
> Ideally the default backend should detect if graphics hardware is available or not at runtime and choose the best backend for that.
>
>> I realize that we're in an age where GPU programming is becoming more
>> common and even cell phones have good hardware acceleration, but another
>> consideration is the target audience. If you want an API that a
>> non-graphics programmer can more easily get up to speed with, then
>> that's another reason that you can't let the low-level influence the
>> design. The closer you get to the details, the more difficult it is to
>> learn.
>>
>> It's quite possible to put together a reasonably performant,
>> easy-to-use, general purpose graphics API that can take advantage of the
>> best renderer on a given platform without letting the low-level details
>> leak into the high-level design.
>
>


-- 
Adam Wilson
IRC: LightBender
Aurora Project Coordinator

On 1/8/2014 4:52 PM, Jacob Carlborg wrote:

>
> I would say that we should try and built the API in many different
> levels and layers. The top layer would be for ease of use to quickly get
> something to show on the display. If more performance is needed it
> should be possible to access the lower levels of the API to get more
> control of what you need to do.

It's only necessary to have an abstract renderer interface with concrete backend implementations at the lowest level. Then user-facing API (std.gfx.geometry, std.gfx.svg, or whatever) operates through the renderer interface. For this sort of package, I don't believe the renderer interface should be exposed to the user at all. Doing so would greatly inhibit the freedom to refactor it down the road. And that freedom, I think, is important for long-term maintenance.

>
>> Ideally, the default
>> backend should be software, so that the API can be used even when
>> graphics hardware is not available (though I'm not saying that's a
>> realistic target to start out with).
>
> Ideally the default backend should detect if graphics hardware is
> available or not at runtime and choose the best backend for that.

By "default", I mean "fallback", for cases when there's no, or problematic, hardware acceleration available. As to which and how many hardware-accelerated backends ship along with that, that's very much open to debate. Which versions of OpenGL to support out of the box, whether or not a D3D renderer should be included and, if so, which version, and so on.

Honestly, I'd prefer not to see this package in Phobos at all. That implies a number of constraints, both design time and run time, that would not be necessary if it were left as a third-party dub-enabled package. Anything in Phobos has to work where D works, be it on a headless server or a tweaked-out gaming rig, or on an old system or a future one. Maximum portability and maintainability need to take priority over performance. If the graphics API can't work in all of those possible environments, then it has no business being part of the standard library.