October 11, 2012
On 2012-10-11 10:12, Manu wrote:

> Sorry, I do know what a package manager is. I was being facetious, in
> that windows has no such concept, and most people use windows.
> It's not practical to depend on anything like that.

RubyGems is working perfectly fine on Windows. Just because most Windows does not have a built in package manager and most of these open source language seems to lean towards Posix than Windows doesn't mean that you can't have a working package manager in Windows.

> I've never seen the 'custom build tool' option in Visual Studio (and if
> there were one, certainly people don't use it). And no associated
> package manager that automatically fetches dependencies...

What's stopping anyone from adding support for this in VisualD?

> Additionally, you're insisting build tool 'X' upon people, and I presume
> it's not an established 'standard' build tool that's accepted/agreed on
> all platforms. You'll never convince people to use it, and certainly not
> in in proprietary situations where your tools are dictated.

Then tell me what established standard build tool that works cross platform. They're just pain in the ass to use or have too many dependencies to use/too complicated to install.

I'm not forcing anyone upon my tools. But you're trying to force features into D that is not particular useful.

> This is indeed ridiculous, but trivial. I'm sure it can be fixed in minutes.
> I don't think the solution is complicated, given a lib name, on linux it
> first looks for a lib*.so, then for a .a. Windows uses import libs,
> which will be resolved exactly the same as a static lib, and I don't
> know anything about OSX, is it not similar/same as linux?

No, Mac OS X uses .dylib files. In addition to that it uses frameworks and other kind of bundles. A framework is a folder that the linker and Finder (file browser) treats differently. It contains a dynamic library, headers and other resources like images. It has the .framework extension.

>     Another point is that I don't want to separate the build script.
>     Compile and link flags in one file and then the libraries to link
>     with in a different file? That's just stupid.
>
>
> What's a flag? you mean those little options in the property grid when
> you Right Click->Properties on the project? ;)

No, I mean when you left click. How the hell would I know what tools you're using?

> People don't like doing that, and they really hate micro-managing a
> massive list of libraries in there. Also those flags aren't
> configurable, you can't just add one for your new feature of interest.

So how do you chooses what libraries to link with. If you want to build in a release or a debug build. Regardless what tools you have, in the end it will add compile and link flags somewhere.

> I also maintain that it's not stupid. The build script doesn't know what
> libs the code will link to. I guess you're arguing that your
> build-script should exclusively define the features/libs, not the other
> way around?

So what does know which libraries to link with if not the build script. Something needs to know that.

> This is annoying in a cross-platform environment, because all platforms
> tend to link a totally different set of libraries, and that means your
> build script is cluttered with annoying manually maintained lists for
> each platform, and the less commonly used/built platform ALWAYS seems to
> fall out of date.

The information needs to be somewhere.

> In D's context, D has very powerful (and more manageable) static logic,
> I can see it being even more useful to have code-driven selection of
> dependencies in D than it already is in C.
>
>
>     With the approach with the build tool and the package manager
>     working together the build tool can also automatically specify the
>     path to the import files. How would you do that in a source file?
>     Either the compiler would always need to read a special file first.
>     Or it would need to scan all files for a particular pragma to get
>     the import paths. Then it would rescan the files again during the
>     actual compilation.
>
>
> The linker would just need to be smart enough to gather the deps from
> the objects while linking.

The linker doesn't know anything about the import paths.

> I've said before, my question is: is it _possible_ to do it in a cross
> platform way currently? I see some other comments with ideas, maybe it is?
>
>
>     This is a specification/description of a package manager I'm working on:
>
>     https://github.com/jacob-__carlborg/orbit/wiki/Orbit-__Package-Manager-for-D
>     <https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D>
>
>
> Does it work in windows and integrate with Visual Studio?
> If not, sadly, it's irrelevant.

Of course, I build all by software to work cross-platform, what is pointing to anything else.

You're talking about "working on all platforms" but then you end with this comment making the rest of your comments not very believable.

I can end with the same comment. Does Visual Studio work on other platforms than Windows? No, then it's irrelevant.

I care about cross-platform not just about Windows and/or Visual Stdio. I build my tools as libraries make it easy to both create command line tools or plugins to integrate in whatever IDE/application you want to use. Not just a plugin specifically for Visual Studio.

> But aside from that, I am curious, what makes it better/more useful than
> apt?

It's specific to D and can be made a lot more easier to work with. apt doesn't know anything about D, DMD or how to build a D application. apt is also only works on Debian, Ubuntu and a few other Linux distributions.

-- 
/Jacob Carlborg
October 11, 2012
On 2012-10-10 10:31, Manu wrote:
> Percect, thanks!

As far as I recall, that doesn't work with import files.

-- 
/Jacob Carlborg
October 11, 2012
On Thursday, 11 October 2012 at 07:18:45 UTC, Arjan wrote:
> On Wed, 10 Oct 2012 16:32:05 +0200, Paulo Pinto <pjmlp@progtools.org> wrote:
>
>> On Wednesday, 10 October 2012 at 14:44:20 UTC, Walter Bright wrote:
>>> On 10/10/2012 4:49 AM, Paulo Pinto wrote:
>>>> On Wednesday, 10 October 2012 at 11:50:29 UTC, Manu wrote:
>>>>> Really? Is it an MS thing? I'm amazed the other compilers haven't adopted
>>>>> that in the last 10 years or whatever.
>>>>
>>>> Yes, it is a Microsoft extension. I never saw it in any other C or C++ compiler.
>>>
>>> Digital Mars C and C++ !!
>>
>> I only became aware of Digital Mars thanks to D, I must confess.
>>
>> When I moved away from Turbo Pascal, I started using Turbo C and Turbo C++, followed by Borland C++ and eventually Visual C++.
>>
>> Then at the university I started to use vendor's C and C++ compilers of the multiple UNIX systems we had access to.
>>
>> I used to see adverts for Watcom C/C++, High C/C++ and Zortech C/C++ in computer magazines, but never knew anyone that had access to them.
>>
>
> You really missed something. For years Zortech/Sysmantec/DigitalMars C++ has been my preferred compiler. Generated fast code very fast! With being STLport'ed it also had decent support for STL. It often times barked about issues with Type checking other compilers did not even mention. Which saved me from quite a few bugs.
> When porting big C++ libraries like wxWidgets (then wxWindows) it became apparent to me DMC++ did need the least special treatment to make it compile the code. Also ported various Boost libs to it.
> It wasn't util VS2005 came along before I started shifting away...
>
> About Watcom? Well a complete opposite experience...
>
> Arjan

I guess the Portuguese market was too small for having those compilers available.
October 11, 2012
On 11 October 2012 12:04, Jacob Carlborg <doob@me.com> wrote:

> On 2012-10-11 10:12, Manu wrote:
>
>  Sorry, I do know what a package manager is. I was being facetious, in
>> that windows has no such concept, and most people use windows. It's not practical to depend on anything like that.
>>
>
> RubyGems is working perfectly fine on Windows. Just because most Windows does not have a built in package manager and most of these open source language seems to lean towards Posix than Windows doesn't mean that you can't have a working package manager in Windows.


Okay, so I'll stop being a dick for a minute, I'm actually curious to know
how you imagine it working with a tool like VisualStudio.
It sounds like you're not just talking about a tool to fetch libs and
dependencies, but also perform the build? And the dependencies are detailed
in the build script?
An inflexible build system like Visual Studio doesn't really handle that
very well.

A package manager which collects libs and their dependencies into common include/lib paths sounds extremely useful. But to me it sounds even more useful when combined with #pragma lib! That conveniently eliminates the lib path issue.

I can imagine a situation where libraries would imply their associated lib just by being imported into your module. With a package manager as you describe, and source-embedded lib linkage statements, your average user would be in a position to never concern themselves with libs at all. Sounds almost as good as Java/C#.


I also maintain that it's not stupid. The build script doesn't know what
>
> libs the code will link to. I guess you're arguing that your
>> build-script should exclusively define the features/libs, not the other way around?
>>
>
> So what does know which libraries to link with if not the build script. Something needs to know that.


The source -> object files would ideally know, and the linker would extract that information its self. I think that's the topic of the conversation :)


The linker would just need to be smart enough to gather the deps from
>> the objects while linking.
>>
>
> The linker doesn't know anything about the import paths.
>

The linker knows standard places to search, and non-standard places would still need to be configured... but this marry's very nicely with a package manager as you describe, since there'll never be confusion about where to look under that system.


Does it work in windows and integrate with Visual Studio?
>> If not, sadly, it's irrelevant.
>>
>
> Of course, I build all by software to work cross-platform, what is pointing to anything else.
>
> You're talking about "working on all platforms" but then you end with this comment making the rest of your comments not very believable.
>
> I can end with the same comment. Does Visual Studio work on other platforms than Windows? No, then it's irrelevant.
>

The point I'm trying to make is that a solution which is only convenient
when working with a particular configurable/flexible build script isn't a
sufficient solution.
#pragma lib is very convenient, especially in the absence of a
comprehensive build environment (such as Visual Studio). I maintain that
there is absolutely nowhere that understands library dependencies better
than the code that produces that dependency. I'd love to be able to
describe all linker requirements in that way if I could, regardless of the
toolchain/operating system.

So I am actually all for a package manager for libs, particularly in
conjunction with #pragma lib. That beautiful union eliminates the linker
from the things that a programmer cares about almost entirely.
I have my suspicions though that unless it gains universal acceptable by
the D community, the libs either won't be up to date, or just not available
- which may be a worse case, in that the management of libs would be
fractured across multiple management mechanisms.


October 11, 2012
On Thursday, 11 October 2012 at 11:46:39 UTC, Manu wrote:
> On 11 October 2012 12:04, Jacob Carlborg <doob@me.com> wrote:
>
>> On 2012-10-11 10:12, Manu wrote:
>>
>>  Sorry, I do know what a package manager is. I was being facetious, in
>>> that windows has no such concept, and most people use windows.
>>> It's not practical to depend on anything like that.
>>>
>>
>> RubyGems is working perfectly fine on Windows. Just because most Windows
>> does not have a built in package manager and most of these open source
>> language seems to lean towards Posix than Windows doesn't mean that you
>> can't have a working package manager in Windows.
>
>
> Okay, so I'll stop being a dick for a minute, I'm actually curious to know
> how you imagine it working with a tool like VisualStudio.
> It sounds like you're not just talking about a tool to fetch libs and
> dependencies, but also perform the build? And the dependencies are detailed
> in the build script?
> An inflexible build system like Visual Studio doesn't really handle that
> very well.
>
> A package manager which collects libs and their dependencies into common
> include/lib paths sounds extremely useful. But to me it sounds even more
> useful when combined with #pragma lib! That conveniently eliminates the lib
> path issue.
>
> I can imagine a situation where libraries would imply their associated lib
> just by being imported into your module. With a package manager as you
> describe, and source-embedded lib linkage statements, your average user
> would be in a position to never concern themselves with libs at all. Sounds
> almost as good as Java/C#.
>
>
> I also maintain that it's not stupid. The build script doesn't know what
>>
>> libs the code will link to. I guess you're arguing that your
>>> build-script should exclusively define the features/libs, not the other
>>> way around?
>>>
>>
>> So what does know which libraries to link with if not the build script.
>> Something needs to know that.
>
>
> The source -> object files would ideally know, and the linker would extract
> that information its self. I think that's the topic of the conversation :)
>
>
> The linker would just need to be smart enough to gather the deps from
>>> the objects while linking.
>>>
>>
>> The linker doesn't know anything about the import paths.
>>
>
> The linker knows standard places to search, and non-standard places would
> still need to be configured... but this marry's very nicely with a package
> manager as you describe, since there'll never be confusion about where to
> look under that system.
>
>
> Does it work in windows and integrate with Visual Studio?
>>> If not, sadly, it's irrelevant.
>>>
>>
>> Of course, I build all by software to work cross-platform, what is
>> pointing to anything else.
>>
>> You're talking about "working on all platforms" but then you end with this
>> comment making the rest of your comments not very believable.
>>
>> I can end with the same comment. Does Visual Studio work on other
>> platforms than Windows? No, then it's irrelevant.
>>
>
> The point I'm trying to make is that a solution which is only convenient
> when working with a particular configurable/flexible build script isn't a
> sufficient solution.
> #pragma lib is very convenient, especially in the absence of a
> comprehensive build environment (such as Visual Studio). I maintain that
> there is absolutely nowhere that understands library dependencies better
> than the code that produces that dependency. I'd love to be able to
> describe all linker requirements in that way if I could, regardless of the
> toolchain/operating system.
>
> So I am actually all for a package manager for libs, particularly in
> conjunction with #pragma lib. That beautiful union eliminates the linker
> from the things that a programmer cares about almost entirely.
> I have my suspicions though that unless it gains universal acceptable by
> the D community, the libs either won't be up to date, or just not available
> - which may be a worse case, in that the management of libs would be
> fractured across multiple management mechanisms.

For those of us doing .NET development, the answer is NuGet.
http://visualstudiogallery.msdn.microsoft.com/27077b70-9dad-4c64-adcf-c7cf6bc9970c

And there is already a package manager for Windows that uses NuGet as infrastructure.

http://chocolatey.org/

--
Paulo
October 11, 2012
On 10/10/2012 11:49 PM, Jacob Carlborg wrote:
> Maybe it's time to update the old tool chain that we inherited form the 70s, but
> this is a totally different discussion.

Writing a new linker for each platform is a major undertaking.


October 11, 2012
On 11 October 2012 14:35, Walter Bright <newshound2@digitalmars.com> wrote:

> On 10/10/2012 11:49 PM, Jacob Carlborg wrote:
>
>> Maybe it's time to update the old tool chain that we inherited form the
>> 70s, but
>> this is a totally different discussion.
>>
>
> Writing a new linker for each platform is a major undertaking.
>

Well Microsoft's linker is obviously capable of finding lib details in the
object files.
Iain almost sounded like he had a working theory for ld.
I know nothing of OSX. LLVM?


October 11, 2012
On 2012-10-11 13:22, Manu wrote:

> Okay, so I'll stop being a dick for a minute, I'm actually curious to
> know how you imagine it working with a tool like VisualStudio.
> It sounds like you're not just talking about a tool to fetch libs and
> dependencies, but also perform the build? And the dependencies are
> detailed in the build script?
> An inflexible build system like Visual Studio doesn't really handle that
> very well.

I'm talking about three separate tools that work close together. The package manager, the build tool and the compiler. The package manager handles packages, i.e. collection of files. The build tool tells the compiler what files to build, what compile and link flags to use. The build tool asks the package manager of any external dependencies.

If it would be possible to interact with the package manager directly from the compiler or source code, i.e. pragma(package) that would be fine. But it's always a bigger step to modify the compiler than creating a separate tool. I also don't really think that the compiler should handle this.

The compiler works with single files, the package manager works with collection of files, that is, packages.

> A package manager which collects libs and their dependencies into common
> include/lib paths sounds extremely useful. But to me it sounds even more
> useful when combined with #pragma lib! That conveniently eliminates the
> lib path issue.

I don't think that pragma(lib) works that way. It doesn't for for headers/import files.

Say you create library "foo" and an import file, foo.di. Compiling, say main.d, together with foo.di will not link with foo.lib.

> I can imagine a situation where libraries would imply their associated
> lib just by being imported into your module. With a package manager as
> you describe, and source-embedded lib linkage statements, your average
> user would be in a position to never concern themselves with libs at
> all. Sounds almost as good as Java/C#.

Exactly.

> The source -> object files would ideally know, and the linker would
> extract that information its self. I think that's the topic of the
> conversation :)

Yeah, but I don't think it's there it belongs. I would also say that a package manager can be more flexible then the current state of pragma(lib). When we start to mix in versions and other type of features.

> The linker knows standard places to search, and non-standard places
> would still need to be configured... but this marry's very nicely with a
> package manager as you describe, since there'll never be confusion about
> where to look under that system.

Note that when I say "import path" I'm referring to the path where to search for header (.h) files/import files (.di). The linker has nothing to do with this.

> The point I'm trying to make is that a solution which is only convenient
> when working with a particular configurable/flexible build script isn't
> a sufficient solution.
> #pragma lib is very convenient, especially in the absence of a
> comprehensive build environment (such as Visual Studio). I maintain that
> there is absolutely nowhere that understands library dependencies better
> than the code that produces that dependency. I'd love to be able to
> describe all linker requirements in that way if I could, regardless of
> the toolchain/operating system.

Well, if it's possible to add flags for the compiler and linker it should be possible. If it's possible to create a plugin for VisualStudio that can compile D code then I'm sure that plugin needs to be able to add these kind of flags.

Instead of specifying libraries and link flags I want to specify packages. These packages know which libraries, link flags and other dependencies are needed to build with.

> So I am actually all for a package manager for libs, particularly in
> conjunction with #pragma lib. That beautiful union eliminates the linker
> from the things that a programmer cares about almost entirely.

Actually, this works quite nicely in Ruby. In Ruby the package manager overrides the standard "require" function, that would be the same as overriding "import" in D or "#include" in C/C++. But that's not possible for a library to do, so I'm trying my best here.

In Ruby you also don't need any link flags or other compile flags for that matter.

> I have my suspicions though that unless it gains universal acceptable by
> the D community, the libs either won't be up to date, or just not
> available - which may be a worse case, in that the management of libs
> would be fractured across multiple management mechanisms.

Absolutely, that is a real problem. But we have to start somewhere. I'm drying to do what I think is best and then collect feedback from the newsgroups.

-- 
/Jacob Carlborg
October 11, 2012
On 2012-10-11 13:47, Manu wrote:

> Well Microsoft's linker is obviously capable of finding lib details in
> the object files.
> Iain almost sounded like he had a working theory for ld.
> I know nothing of OSX. LLVM?

I think that Mac OS X is still using ld, but I'm not sure. LLVM do have a linker but I don't remember if it's the standard system linker. The LLVM linker most likely has a better chance of handling these kind of things if ld doesn't already do that.

-- 
/Jacob Carlborg
October 11, 2012
On 2012-10-11 13:35, Walter Bright wrote:

> Writing a new linker for each platform is a major undertaking.

Of course, but isn't writing a new compiler for each platform that as well?

-- 
/Jacob Carlborg