On Tuesday, 26 November 2013 at 19:23:54 UTC, Dmitry Olshansky wrote:
> 26-Nov-2013 23:00, Andrei Alexandrescu пишет:
>>> There may be other problems, but I think that those are the two bigs
>>> ones, and
>>> they've come up several times (particularly the issues with libcurl and
>>> Windows).
>>
>> I've asked the question "what exactly is the problem" several times and
>> you came closest to answering it, but a bunch of stuff is still unclear
>> to me.
>
> I thought I'd chime in on this.
>
> I find that if we actually worked on getting curl integrated better it may just as well be an all right network client lib.
>
> For the moment it looks as if half people are pissed off because of the work they would need to do on it (workaround, shipping, maintenance, etc.), and the other half (using it) are having usability problems on all fronts.
>
> Now to my limited experience with it.
> When I work with it on Linux I have to always jump through hoops because of:
> https://d.puremagic.com/issues/show_bug.cgi?id=7044
>

This is a constant annoyance of mine using std.net.curl on Linux.

> Now if that doesn't set off the alarms. Win32/64 curl libraries are NOT shipped with dmd zip or whatever. The better thing is that last time I checked there is no proper which means that for Win32 one had to get the rare OMF object file format (a custom compilation no less).

Nah, it was as simple as generating an import library from the DLL. There's no need to build libcurl yourself to do that. The installer automatically downloads one I prepared a long time ago already.

@brocolis recently prepared an updated libcurl zip with the OMF import library as well as 64-bit support. It also uses WinSSL instead of OpenSSL so using it is even easier than it was before. The installer has already been updated for it: https://github.com/D-Programming-Language/installer/pull/27

With that libcurl should just work on Windows for both 32 and 64 bits. Linux problems are basically all that remains of the libcurl issues.

>
> You may or may not find them at the VARY BOTTOM of downloads page, and figure it out how to fit them, what flags to use etc. Last but not least it's Win32 only. No links for Win64.

The installer handles downloading it.  End users don't need to find it themselves unless they download the dmd zip file (I still don't understand why anyone does that).

@brocolis wrote up a guide for how he made prepared the updated curl zip and put it up on the wiki. End users shouldn't need to do that though. It's just documented for the phobos developers. http://wiki.dlang.org/Curl_on_Windows

On 11/26/13 11:23 AM, Dmitry Olshansky wrote:
> I find that if we actually worked on getting curl integrated better it
> may just as well be an all right network client lib.

Great.

> For the moment it looks as if half people are pissed off because of the
> work they would need to do on it (workaround, shipping, maintenance,
> etc.), and the other half (using it) are having usability problems on
> all fronts.

I don't think that's the whole set. Don't forget the forum participation is a very small fraction of people who actually use D out there. So there are a bunch of people who don't use curl and aren't hurt, and presumably quite a few people who do use std.net.curl without troubles and don't know we plan to pull the plug on it.

> Now to my limited experience with it.
> When I work with it on Linux I have to always jump through hoops because
> of:
> https://d.puremagic.com/issues/show_bug.cgi?id=7044

Interesting, I ran into that too with ldc. That, fortunately, is fixable on our end.

> Because of that in the end I tend to dodge it completely and roll some
> shell script scaffolding that calls `curl`, as in the cmd-line client.
>
> See some hilarious things I had to do to link it properly:
> https://github.com/blackwhale/tools/blob/master/posix.mak#L35

Interesting. Never knew of the pattern syntax in there.

> Now if that doesn't set off the alarms. Win32/64 curl libraries are NOT
> shipped with dmd zip or whatever. The better thing is that last time I
> checked there is no proper which means that for Win32 one had to get the
> rare OMF object file format (a custom compilation no less).

Is that file needed if one does NOT use std.net.curl? Would it be a bear for us to distributed that particular file?

> You may or may not find them at the VARY BOTTOM of downloads page, and
> figure it out how to fit them, what flags to use etc. Last but not least
> it's Win32 only. No links for Win64.

Which downloads page? Ours or others'?

> All in all, it feels as if somebody actually wanted to sabotage the
> thing, he could score an A+ job as it stands already.

What I see above is it's in our power to fix https://d.puremagic.com/issues/show_bug.cgi?id=7044 and that would benefit not only curl but other libs as well.



Andrei

On 11/26/13 11:39 AM, Brad Anderson wrote:
> On Tuesday, 26 November 2013 at 19:23:54 UTC, Dmitry Olshansky wrote:
>> https://d.puremagic.com/issues/show_bug.cgi?id=7044
>>
>
> This is a constant annoyance of mine using std.net.curl on Linux.
>
>> Now if that doesn't set off the alarms. Win32/64 curl libraries are
>> NOT shipped with dmd zip or whatever. The better thing is that last
>> time I checked there is no proper which means that for Win32 one had
>> to get the rare OMF object file format (a custom compilation no less).
>
> Nah, it was as simple as generating an import library from the DLL.
> There's no need to build libcurl yourself to do that. The installer
> automatically downloads one I prepared a long time ago already.
>
> @brocolis recently prepared an updated libcurl zip with the OMF import
> library as well as 64-bit support. It also uses WinSSL instead of
> OpenSSL so using it is even easier than it was before. The installer has
> already been updated for it:
> https://github.com/D-Programming-Language/installer/pull/27
>
> With that libcurl should just work on Windows for both 32 and 64 bits.
> Linux problems are basically all that remains of the libcurl issues.

This is pretty awesome. Is @brocolis around on this forum?

Andrei

On Tuesday, 26 November 2013 at 17:58:46 UTC, Iain Buclaw wrote:
> sqllite3.amalgamation.c which I thought *was* in phobos at some
> point... maybe not?

I had done work in that regard, but it was decided that it shouldn't be added so Phobos didn't ship with old versions that had known security bugs.

On Tuesday, 26 November 2013 at 19:27:07 UTC, Andrei Alexandrescu wrote:
> On 11/26/13 11:13 AM, Brad Anderson wrote:
>> To use 32-bit curl you need to generate an OMF import library from the
>> libcurl DLL. It's not terribly hard. I did it originally for when I
>> added downloading of libcurl to the Windows installer.
>
> When is the OMF library needed?
>
> 1. While building dmd
>
> 2. While building druntime
>
> 3. While building phobos
>
> 4. While building client code that does NOT use std.net.curl
>
> 5. While building client code that DOES use std.net curl
>

5. std.net.curl actually existed for several releases of dmd before I went to use it one day and got linker errors and decided to fix it with the installer change.

>> It's not at all
>> obvious to users that they need to distribute libcurl.dll et al when
>> they distribute their application but that's a documentation issue.
>
> Or they could kick the can down the road by requiring their customers to have it.
>

The MSVCRT.dll hell of yore has shown this is a terrible option. Windows applications almost universally ship with everything needed to run. There is no package manager on Windows to handle dependencies like there is on Linux.  You always include the DLLs or installers in your application (or instruct the user to download themselves if licensing forbids redistribution but this is avoided as much as possible).

>> If I remember correctly Walter didn't want to distribute in the dmd zip
>> anything that wasn't boost licensed which is why curl on Windows comes
>> as a separate download. zlib is a notable exception but I seem to recall
>> Walter regretted including zlib.
>
> On the other hand we've hardly heard one peep from anyone in literally years that their zip code has trouble building.
>
> We can ask the libcurl author if it comes down to redistributing a libcurl binary. But we want to make sure this is the solution of choice.

Walter actually already did that and the author of libcurl said redistributing it was fine.

Here's an old thread discussing the licensing: http://forum.dlang.org/post/wgnsetqkzbzgulccphvu@forum.dlang.org

On 2013-11-26 20:13, Brad Anderson wrote:

> If I remember correctly Walter didn't want to distribute in the dmd zip
> anything that wasn't boost licensed which is why curl on Windows comes
> as a separate download. zlib is a notable exception but I seem to recall
> Walter regretted including zlib.

The zip already contains tools which the source code isn't available. Like "shell", "dumpobj" and "obj2asm". But I guess Walter, DigitalMars or Symantec has created those.

> I can't remember off the top of my head if libcurl's license allows it
> to be statically linked with a boost licensed library but that would
> certainly simplify things for end users (but complicate the building of
> phobos a bit).

Walter contacted the author and asked about that, here's the post:

http://forum.dlang.org/thread/i0dlde$59$1@digitalmars.com#post-i0eaoc:2410bs:241:40digitalmars.com

-- 
/Jacob Carlborg

On Tuesday, 26 November 2013 at 19:29:02 UTC, Andrei Alexandrescu wrote:
>> There is no such thing as _The_ curl library. Different distros may have
>> versions with incompatible symbol versioning installed - this is the
>> very issue with Fedora which was actively discussed in bugzilla.
>
> But can't we say "to use std.net.curl with dmd 2.xxx you must have libcurl 7.10 or higher"?
>
> Andrei

It is not that simple. Well, I remember you taking part in https://d.puremagic.com/issues/show_bug.cgi?id=10710 so you should be aware of it. It is not about version number of curl but about transitive dependency - openssl vs gnutls.

We can say "you must have this version of libcurl built with this flags" but it is not much different from saying "use Debian-likes or get out".

On Tuesday, 26 November 2013 at 19:43:02 UTC, Andrei Alexandrescu wrote:
> This is pretty awesome. Is @brocolis around on this forum?
>
> Andrei

I'm not sure. He goes by dnewbie/dnwbie on IRC.

On 2013-11-26 20:27, Andrei Alexandrescu wrote:

> When is the OMF library needed?
>
> 1. While building dmd
>
> 2. While building druntime
>
> 3. While building phobos
>
> 4. While building client code that does NOT use std.net.curl
>
> 5. While building client code that DOES use std.net curl

Not that I have tried it but I'm pretty sure it is 5.

> My reading of http://curl.haxx.se/docs/copyright.html is that the author
> does his best to open up his library.

See this post:

http://forum.dlang.org/thread/i0dlde$59$1@digitalmars.com#post-i0eaoc:2410bs:241:40digitalmars.com

-- 
/Jacob Carlborg

On 11/26/13 11:56 AM, Brad Anderson wrote:
> On Tuesday, 26 November 2013 at 19:27:07 UTC, Andrei Alexandrescu wrote:
>> On 11/26/13 11:13 AM, Brad Anderson wrote:
>>> To use 32-bit curl you need to generate an OMF import library from the
>>> libcurl DLL. It's not terribly hard. I did it originally for when I
>>> added downloading of libcurl to the Windows installer.
>>
>> When is the OMF library needed?
>>
>> 1. While building dmd
>>
>> 2. While building druntime
>>
>> 3. While building phobos
>>
>> 4. While building client code that does NOT use std.net.curl
>>
>> 5. While building client code that DOES use std.net curl
>>
>
> 5. std.net.curl actually existed for several releases of dmd before I
> went to use it one day and got linker errors and decided to fix it with
> the installer change.

Awesome. So can we assume https://github.com/D-Programming-Language/installer/pull/27 takes care of Windows for good?

Thanks for your work!!! I'll make sure I'll get a word of thanks to @broccolis as well.

>> We can ask the libcurl author if it comes down to redistributing a
>> libcurl binary. But we want to make sure this is the solution of choice.
>
> Walter actually already did that and the author of libcurl said
> redistributing it was fine.
>
> Here's an old thread discussing the licensing:
> http://forum.dlang.org/post/wgnsetqkzbzgulccphvu@forum.dlang.org

Terrific, thanks.


Andrei