Hi there!

I wrote a small utility library to read configuration parameters from both command-line arguments (using std.getopt) and SDLang files (using sdlang-d package).
The main library defines a struct ConfigParams whose fields are themselves structs defined in sub-libraries (set as dependencies), each responsible for reading one kind of parameter (BindAddress, BindPort, etc) from both command-line and SDL files. Each sub-library has its own unit tests which run successfully in isolation.
Then, when putting everything together in the main struct CofigParams I get the compile-time error:

/usr/include/dlang/dmd/std/socket.d(1195,9): Error: static variable getaddrinfoPointer cannot be read at compile time
called from here: parseAddress(addressString, null)

while using std.socket.parseAddress to validate IPv4/IPv6 bind addresses.
I'm using dmd v2.071.2 and dub v1.0.0.
The relevant piece of code where this happens is the following:

struct BindAddress {

  import std.socket: parseAddress, SocketException;

  // Private members
  private string addressString = "0.0.0.0";

  // Construct from address string
  this(string addressString) {
    try {
      auto address = parseAddress(addressString);
    } catch(SocketException ex) {
      throw new BindAddressException("Invalid bind address " ~ addressString);
    }
    this.addressString = addressString;
  }

}

As said, this works fine when tested in isolation, and the compiler only complains when using BindAddress as a member of ConfigParams.
Any idea what the problem may be?
Or is there maybe a ready to use, high-level library for parsing parameters from command-line arguments and config files of some kind?

On Tuesday, 27 September 2016 at 09:04:53 UTC, Dsciple wrote:
> As said, this works fine when tested in isolation, and the compiler only complains when using BindAddress as a member of ConfigParams.
> Any idea what the problem may be?
> Or is there maybe a ready to use, high-level library for parsing parameters from command-line arguments and config files of some kind?

I assume your ConfigParams variable is global or static? Can you show how you initialize it, and how it's declared?

You're probably using it in a way that requires it to be evaluated at compile time. That's the case for initializers of global/static variables, as well as default values of struct members.

On Tuesday, 27 September 2016 at 14:02:25 UTC, Marc Schütz wrote:
> On Tuesday, 27 September 2016 at 09:04:53 UTC, Dsciple wrote:
>> As said, this works fine when tested in isolation, and the compiler only complains when using BindAddress as a member of ConfigParams.
>> Any idea what the problem may be?
>> Or is there maybe a ready to use, high-level library for parsing parameters from command-line arguments and config files of some kind?
>
> I assume your ConfigParams variable is global or static? Can you show how you initialize it, and how it's declared?
>
> You're probably using it in a way that requires it to be evaluated at compile time. That's the case for initializers of global/static variables, as well as default values of struct members.

Yes I think so.
I use static default values for all members of ConfigParams and I instantiate ConfigParams in my unit tests, so I assume that the variable would be global there.
The code looks like:

unittest {

  string[] args = [
    "binaryFileName",
    "--bindAddresses=0.1.2.3;4.5.6.7",
    "--bindHTTPPort=80",
    "--bindHTTPSPort=443",
    "--configFile=testfiles/test.conf.sdl",
    "--verbosityLevel=detailed",
  ];

  ConfigParams configParams;  // default values for parameters
  configParams.readFromAll(args); // values read from command-line arguents

  // assertion checks here
}

What do you suggest? Should I move all default initializations to a constructor?
Thank you for your response.

On Tuesday, 27 September 2016 at 14:39:10 UTC, Dsciple wrote:
> On Tuesday, 27 September 2016 at 14:02:25 UTC, Marc Schütz wrote:
>> On Tuesday, 27 September 2016 at 09:04:53 UTC, Dsciple wrote:
>>> As said, this works fine when tested in isolation, and the compiler only complains when using BindAddress as a member of ConfigParams.
>>> Any idea what the problem may be?
>>> Or is there maybe a ready to use, high-level library for parsing parameters from command-line arguments and config files of some kind?
>>
>> I assume your ConfigParams variable is global or static? Can you show how you initialize it, and how it's declared?
>>
>> You're probably using it in a way that requires it to be evaluated at compile time. That's the case for initializers of global/static variables, as well as default values of struct members.
>
> Yes I think so.
> I use static default values for all members of ConfigParams and I instantiate ConfigParams in my unit tests, so I assume that the variable would be global there.
> The code looks like:
>
> unittest {
>
>   string[] args = [
>     "binaryFileName",
>     "--bindAddresses=0.1.2.3;4.5.6.7",
>     "--bindHTTPPort=80",
>     "--bindHTTPSPort=443",
>     "--configFile=testfiles/test.conf.sdl",
>     "--verbosityLevel=detailed",
>   ];
>
>   ConfigParams configParams;  // default values for parameters
>   configParams.readFromAll(args); // values read from command-line arguents
>
>   // assertion checks here
> }
>
> What do you suggest? Should I move all default initializations to a constructor?
> Thank you for your response.

I forgot to show declaration. Here it is:

struct ConfigParams {

  import ConfigParamsCLAMixin: ConfigParamsCLA;
  import ConfigParamsSDLMixin: ConfigParamsSDL;
  import ConfigParamsAllMixin: ConfigParamsAll;
  import BindAddress: BindAddress, BindAddressException;
  import BindAddresses: BindAddresses, BindAddressesException;
  import BindPort: BindPort, BindPortException;
  import PosixPath: PosixPath, PosixPathException;
  import VerbosityLevel: VerbosityLevel, VerbosityLevelException;

  // Define configuration parameters' static default fields
  static immutable BindAddresses defaultBindAddresses = BindAddresses([
    BindAddress("192.168.2.10")
  ]);
  static immutable BindPort defaultBindHTTPPort = BindPort(8080);
  static immutable BindPort defaultBindHTTPSPort = BindPort(4430);
  static immutable PosixPath defaultConfigFile =
    PosixPath("/etc/ras/ras.conf.sdl");
  static immutable VerbosityLevel defaultVerbosityLevel =
    VerbosityLevel("quiet");

  // Define configuration parameters fields with default values
  BindAddresses bindAddresses = defaultBindAddresses;
  BindPort bindHTTPPort = defaultBindHTTPPort;
  BindPort bindHTTPSPort = defaultBindHTTPSPort;
  PosixPath configFile = defaultConfigFile;
  VerbosityLevel verbosityLevel = defaultVerbosityLevel;

}

I don't understand why the unit tests of all individual members of ConfigParams (BindAddresses, BindAddress and so on) would work in isolation using the same kind of initialization, whereas some of them would fail as members of ConfigParams.
I suspect this may also be a problem with settings in dub.json for ConfigParams:

{
  ...
  "sourcePaths": [
    "sources"
  ],
  "importPaths": [
    "sources"
  ],
  "configurations": [
    {
      "name": "library",
      "targetName": "ConfigParams",
      "targetType": "library"
    }
  ],
  "dependencies": {
    "bind-address": {
      "version": "*",
      "path": "./libraries/BindAddress"
    },
    "bind-addresses": {
      "version": "*",
      "path": "./libraries/BindAddresses"
    },
    "bind-port": {
      "version": "*",
      "path": "./libraries/BindPort"
    },
    "posix-name": {
      "version": "*",
      "path": "./libraries/PosixName"
    },
    "posix-path": {
      "version": "*",
      "path": "./libraries/PosixPath"
    },
    "verbosity-level": {
      "version": "*",
      "path": "./libraries/VerbosityLevel"
    }
  }

if I include bind-address as a dependency (as above), dub refuses to run with:

Sub package bind-address: doesn't exist.

although the package is there and used by bind-addresses and confi params too...

On Tuesday, 27 September 2016 at 14:57:26 UTC, Dsciple wrote:
> struct ConfigParams {
>   // ...
>   // Define configuration parameters' static default fields
>   static immutable BindAddresses defaultBindAddresses = BindAddresses([
>     BindAddress("192.168.2.10")
>   ]);
>   // ...
> }

Yepp, that's definitely the culprit.

>
> I don't understand why the unit tests of all individual members of ConfigParams (BindAddresses, BindAddress and so on) would work in isolation using the same kind of initialization, whereas some of them would fail as members of ConfigParams.

Because in your unit tests, your BindAddress variables are probably normal, local variables. These are initial at runtime as you would expect. Only in places where a static initialization is required, the compiler will evaluate the initializer at compile time. Examples:

    // global, but no initializer => ok
    BindAddress global1;
    // global with initializer => needs to be evaluated at compile time
    BindAddress global2 = BindAddress("127.0.0.1");
    struct Config {
        // default value for members => evaluate at compile time
        BindAddress bindAddress = BindAddress("0.0.0.0");
    }

    void main() {
        // assignment to globals at runtime works (of course)
        global1 = BindAddress("192.168.1.2");
        // local with initializer => works at runtime
        BindAddress local1 = BindAddress("10.0.0.1");
        // static => same as global, evaluated at compile time
        static BindAddress static1 = BindAddress("127.0.0.1");
    }

You could solve the problem as you suggested by moving the initializations into a constructor, but if your default values are only ever IPs (i.e., no hostname resolution necessary), you could also add an additional function that parse them without calling into the C library, e.g.

    struct BindAddress {
        // ...
        static BindAddress fromIP(string addr) {
            // ...
        }
    }

    struct Config {
        // ...
        BindAddress bindAddress = BindAddress.fromIP("127.0.0.1");
    }

You can try whether InternetAddress.parse() [1] works at compile time, otherwise you would have to implement it by hand.

[1] https://dlang.org/phobos/std_socket.html#.InternetAddress.parse

On Wednesday, 28 September 2016 at 09:56:06 UTC, Marc Schütz wrote:
> You could solve the problem as you suggested by moving the initializations into a constructor, but if your default values are only ever IPs (i.e., no hostname resolution necessary), you could also add an additional function that parse them without calling into the C library, e.g.
>
>     struct BindAddress {
>         // ...
>         static BindAddress fromIP(string addr) {
>             // ...
>         }
>     }
>
>     struct Config {
>         // ...
>         BindAddress bindAddress = BindAddress.fromIP("127.0.0.1");
>     }
>
> You can try whether InternetAddress.parse() [1] works at compile time, otherwise you would have to implement it by hand.
>
> [1] https://dlang.org/phobos/std_socket.html#.InternetAddress.parse

Indeed, getting rid of all static initializations in ConfigParams did finally work!
Now I'm left with parameters' definitions without default values, and an ugly constructor taking a long list of initialization values as arguments and storing them in their respective private members :/ .

struct ConfigParams {
  BindAddresses bindAddresses;
  BindPort bindHTTPPort;
  BindPort bindHTTPSPort;
  PosixPath configFile;
  VerbosityLevel verbosityLevel;

  this(BindAddresses bindAddresses,
      BindPort bindHTTPPort = BindPort(8080),
      BindPort bindHTTPSPort = BindPort(4430),
      PosixPath configFile = PosixPath("/etc/ras/ras.conf.sdl"),
      VerbosityLevel verbosityLevel = VerbosityLevel("quiet")) {
    this.bindAddresses = bindAddresses;
    this.bindHTTPPort = bindHTTPPort;
    this.bindHTTPSPort = bindHTTPSPort;
    this.configFile = configFile;
    this.verbosityLevel = verbosityLevel;
  }
}

Luckily I can set default values in constructor's signature (except for the very first argument, otherwise I get a deprecation warning for defaulting all arguments which does not make sense), so I save some typing when calling it.
Any idea about how to improve this?

In the end everything boils down to not being able to use static initializations with std.socket.parseAddress and friends. I don't understand what prevents such function (in turn calling some OS-level C function) from doing its job at compile time too. Guess it's a very challanging thing to do at compile-time, or should I open an issue with the std.socket lib developers?

By the way, out of despair, I also tried to validate addresses myself using regular expressions but:
1) parsing generic IPv6 addresses is not trivial at all!
2) even skipping IPv6, I get similar compiler errors complaining again about malloc not being able to compute some value at compile time, so std.regex also conflicts with static initializations somehow..

All in all, I can live with the current solution and move on.
Thank you a lot for your support Marc!

On Wednesday, 28 September 2016 at 15:34:56 UTC, Dsciple wrote:
> I don't understand what prevents such function (in turn calling some OS-level C function) from doing its job at compile time too. Guess it's a very challanging thing to do at compile-time, or should I open an issue with the std.socket lib developers?

std.socket simply passes the request on to the operating system. Enabling parsing IP addresses at compile time would necessitate duplicating the logic already present in the operating system libraries, which would not be future-proof at best, and buggy at worst.

It's not possible to implement by invoking C functions like at runtime, since that is forbidden during compile-time for a number of reasons, mainly safety.

On Wednesday, 28 September 2016 at 16:49:54 UTC, Vladimir Panteleev wrote:
> On Wednesday, 28 September 2016 at 15:34:56 UTC, Dsciple wrote:
>> I don't understand what prevents such function (in turn calling some OS-level C function) from doing its job at compile time too. Guess it's a very challanging thing to do at compile-time, or should I open an issue with the std.socket lib developers?
>
> std.socket simply passes the request on to the operating system. Enabling parsing IP addresses at compile time would necessitate duplicating the logic already present in the operating system libraries, which would not be future-proof at best, and buggy at worst.
>
> It's not possible to implement by invoking C functions like at runtime, since that is forbidden during compile-time for a number of reasons, mainly safety.

I suspected there must have been a very good reason for this not being possible at compile time and this definitely looks like one: thanks Vladimir for tracking this discussion and clarifying it to me!
So I guess this answers all my questions and. as a beginner, I can also say I learned something in the process: thank you again!