Walter Bright wrote:
> Georg Wrede wrote:
> 
>> Uh-oh, after having read what Kris and others have posted as
>> replies to your post, I can't push D for embedded development. At
>> least until the issues they've brought up are resolved.
> 
> 
> Which particular issue?

Well,

> For what it's worth, the D spec doesn't require any of the behavior
> Kris has been talking about.  I'd consider most of it specific to the
> DMD implementation.
> 
> Sean

when Sean pointed this out, those issues of mine vanished, Poof!

Since DMD is Win+Lin _only_ (IIUC), none of the embedded woes apply. It's happened to me before, too, that I forget that DMD is not the same as the D spec. And that Phobos is not part of the spec.

So, actually, any discussions about embedded systems belong to D.gnu instead.

---

Still, I do agree with Kris on separation of concern, clear and consistent compartmentalization of dependencies, low level dependencies on high level code, modularization, etc., in Phobos.

Not addressing those issues, at least makes it harder to maintain and develop Phobos in the long run. IMHO.

Walter Bright skrev:
> Georg Wrede wrote:
>> I admit this is a "feelings based" thing with most people I've talked with. It seems that on embedded platforms, many expect to write all the needed code themselves. It's also felt (possibly unduely??) that Phobos (or whatever general Win+*nix standard library) is mostly useless in embedded applications.
> 
> I'd like to get to the bottom of this feeling. For example, Kris was unhappy that typeinfo imported std.strings. I can't figure out what the problem with that is.
> 
I think it is the perceived bloat, not what is in practice done. Importing std.string while only using a single function still gives the impression of needing the whole module.


Perhaps having a module scope of hmm... sys where typeinfo, object and anything needed by compiler, and runtime resides is a good idea. Totally forbid anything in "sys" to import/depend on anything from the outside. That way there would be no question for anyone about "how much is safe to strip"?

And besides, is it wise to depend on what a linker "should do"? If current build chain nicely throws out what is not needed, does that make it right to assume that all build chains should behave as such?


As I see it each module in std should be as self contained as ever possible. I know the std.date I proposed imports std.conv, std.stdio, std.string and std.c.time, but my intent is to not import any of them when finished.

But then my intention was never to bring up the internals to much debate, I wanted to have input on the externals, how you as developers use the code. That was perhaps futile, but I still think my approach of a few but flexible, and overloaded functions is the best approach.


// Fredrik

kris wrote:
> Yes, they are useful. And I'm not trying to be a jerk by pointing out that the D runtime is missing some much needed TLC (to put it very nicely). Why do you think Ares exists anyway?

I'm trying to understand.


>> All those other names are are the static data. Things like:
>>
>> const dchar LS = '\u2028';      /// UTF line separator
>> const dchar PS = '\u2029';      /// UTF paragraph separator
>>
>> I submit that they aren't significant. 
> Yes, you're right. Unless they're huge tables (such as the Unicode character map; oh wait; is that linked by default? :)

If you're talking about the one in std.uni, it took me a minute to figure that one out. The reason it's pulled in is  because of an error in the compiler, where lambda functions should be generated as comdats, but aren't. It is not a problem with the library design.

> What's all this about necessary re-engineering and rewriting of Phobos? Where the heck did that come from?

You asked what the motivation for Ares was.


>> Why does the C library need replacing? I honestly don't get it.
> Who said it /needed/ replacing? I'm simply talking about applying a small sprinkling of decoupling dust

You write: "If I have, as you say, a system that can't have C's IO subsystem, the above will hardly help me at all."


> Forgive me, but that's just printf(). If I have, as you say, a system that can't have C's IO subsystem, the above will hardly help me at all. I suspect you're well aware the C IO subsystem consists of significantly more than just printf? Say, perhaps 50-ish functions? You're going to suggest I stub them all out just because Object.print() calls printf()?

You only need to stub out the functions that pull in the rest. If it's only printf, then that's all you need to stub out. If you implement your own printf, then it will 'hook' any other calls to printf, and you can use it to call your own stuff. It's a common technique - heck, I've done it to 'hook' printf to write to a window instead of stdout.


> Given Derek's example:
> 
> void main() {}
> 
> The .map for that is a fine specimen of "unexpected" coupling. It does seem as though much of that is actually in the C library, but then you're arguing most fervently against doing anything to fix any such things.

It is mostly the C runtime library. And I've lived with this for a very long time, don't you think I've looked at it to try to reduce the minimum required to be pulled in?


> Walter Bright wrote:
>  > Phobos doesn't require floating point support from the processor
>  > unless one actually uses floating point in the application code.
> 
> That turned out to be somewhat less than truthful.

It will be in the next update. I had assumed it was working properly when it wasn't - however, printf had nothing to do with the problem. I don't know why you call it "futility" - the problem will get fixed. The solutions you proposed wouldn't have fixed it.

>  > I also really don't understand why anyone using D would require
>  > not using Phobos. What's the problem?
> 
> With much respect, the 'problem' is perhaps that you don't see any?

Here's the issue I have - postings that adamantly offer solutions without identifying the actual problem. Here are some examples:

Solution: remove printf
Alleged problem: printf pulls in floating point formatting code
Actual problem: std.string pulls in floating point formatting code due to reference to __fltused. printf does not pull in floating point formatting code.
Correct solution: fix compiler to not generate __fltused references in library code

Solution: implement separate itoa() for typeinfo
Alleged problem: calling one function in std.string pulls in everything in std.string
Actual problem: only a small portion of std.string is actually linked in because the free functions are implemented as COMDATs, but due to an error in the compiler, lambda functions are not written as COMDATs. This causes a reference to std.uni to still be pulled in, pulling in a large table in std.uni
Correct solution: fix compiler to generate COMDATs for lambda functions.


I'm glad these two problems came to light, and I'm happy to fix them. It's why I ask "What's the problem?" rather than just applying solutions without understanding what problem the solution is aimed at. In the two cases above, the real problems were found only after I kept asking seemingly stupid questions about what problem you were trying to solve by removing printf and the call to std.string.toString.

I know you're still bothered by the issue of C's IO being pulled in. I have another utility for you - \dm\bin\libunres. Libunres will identify any symbols unresolved by a library. Running libunres on phobos.lib gives:

Unresolved externals:
??2@YAPAXI@Z
??2@YAPAXIPAX@Z
??3@YAXPAX@Z
?__stl_throw_length_error@std@@YAXPBD@Z
?__stl_throw_out_of_range@std@@YAXPBD@Z
_ExpandEnvironmentStringsA@12
_FreeLibrary@4
_GetFileType@4
_GetProcAddress@8
_GetVersion@0
_IID_IUnknown
_LoadLibraryA@4
_QueryPerformanceCounter@4
_RegCloseKey@4
_RegCreateKeyExA@36
_RegDeleteKeyA@8
_RegDeleteValueA@8
_RegEnumKeyExA@32
_RegEnumValueA@32
_RegFlushKey@4
_RegOpenKeyA@12
_RegOpenKeyExA@20
_RegQueryInfoKeyA@48
_RegQueryValueExA@24
_RegSetValueExA@24
_WSACleanup@0
_WSAGetLastError@0
_WSAStartup@8
__Ccmp
__Dmain
__LCMP@
__LDIV@
__U64_LDBL
__ULDIV@
___alloca
___fp_lock
___fp_unlock
___pfloatfmt
__assert
__beginthreadex
__end
__except_list
__fltused
__fputc_nlock
__fputwc_nlock
__global_unwind
__imp__CloseHandle@4
__imp__CopyFileA@12
__imp__CopyFileW@12
__imp__CreateDirectoryA@8
__imp__CreateDirectoryW@8
__imp__CreateFileA@28
__imp__CreateFileMappingA@24
__imp__CreateFileW@28
__imp__CreateWindowExA@48
__imp__DeleteCriticalSection@4
__imp__DeleteFileA@4
__imp__DeleteFileW@4
__imp__DuplicateHandle@28
__imp__EnterCriticalSection@4
__imp__FileTimeToSystemTime@8
__imp__FindClose@4
__imp__FindFirstFileA@8
__imp__FindFirstFileW@8
__imp__FindNextFileA@8
__imp__FindNextFileW@8
__imp__FlushViewOfFile@8
__imp__FormatMessageA@28
__imp__GetCurrentDirectoryA@8
__imp__GetCurrentDirectoryW@8
__imp__GetCurrentProcess@0
__imp__GetCurrentThread@0
__imp__GetCurrentThreadId@0
__imp__GetFileAttributesA@4
__imp__GetFileAttributesW@4
__imp__GetFileSize@8
__imp__GetFullPathNameA@16
__imp__GetLastError@0
__imp__GetModuleFileNameA@12
__imp__GetProcessTimes@20
__imp__GetSystemInfo@4
__imp__GetSystemTime@4
__imp__GetThreadContext@8
__imp__GetThreadTimes@20
__imp__GetTickCount@0
__imp__GetTimeZoneInformation@4
__imp__GetVersionExA@4
__imp__InitializeCriticalSection@4
__imp__InterlockedDecrement@4
__imp__InterlockedExchange@8
__imp__InterlockedIncrement@4
__imp__LeaveCriticalSection@4
__imp__LocalFree@4
__imp__MapViewOfFileEx@24
__imp__MoveFileA@8
__imp__MoveFileW@8
__imp__MultiByteToWideChar@24
__imp__QueryPerformanceCounter@4
__imp__QueryPerformanceFrequency@4
__imp__RaiseException@16
__imp__ReadFile@20
__imp__RemoveDirectoryA@4
__imp__RemoveDirectoryW@4
__imp__ResumeThread@4
__imp__SetCurrentDirectoryA@4
__imp__SetCurrentDirectoryW@4
__imp__SetFilePointer@16
__imp__SetThreadPriority@8
__imp__Sleep@4
__imp__SuspendThread@4
__imp__UnmapViewOfFile@4
__imp__VirtualAlloc@16
__imp__VirtualFree@12
__imp__WaitForSingleObject@8
__imp__WideCharToMultiByte@32
__imp__WriteFile@20
__imp__lstrcatA@8
__imp__lstrcmpA@8
__imp__lstrcpyA@8
__imp__lstrlenA@4
__iob
__local_except_handler
__snprintf
__vsnprintf
__xi_a
_accept@12
_acosl
_asinl
_atan2l
_atanl
_atoi
_bind@12
_calloc
_cbrtl
_ceill
_closesocket@4
_connect@12
_coshl
_erfcl
_erfl
_errno
_execv
_execve
_execvp
_execvpe
_exit
_exp2l
_expl
_expm1l
_fclose
_fdopen
_feof
_fflush
_fgetc
_floorl
_fopen
_fprintf
_fputc
_fread
_free
_fseek
_ftell
_fwide
_fwrite
_gethostbyaddr@12
_gethostbyname@4
_gethostname@8
_getpeername@12
_getprotobyname@4
_getprotobynumber@4
_getservbyname@8
_getservbyport@8
_getsockname@12
_getsockopt@20
_ilogbl
_inet_addr@4
_inet_ntoa@4
_ioctlsocket@12
_lgammal
_listen@8
_log10l
_log1pl
_log2l
_logbl
_logl
_malloc
_memchr
_memcmp
_memcpy
_memicmp
_memmove
_memset
_modfl
_nanl
_nearbyintl
_powl
_printf
_realloc
_recv@16
_recvfrom@24
_remainderl
_roundl
_select@20
_send@16
_sendto@24
_setsockopt@20
_shutdown@8
_sinhl
_socket@12
_spawnvp
_sprintf
_strcat
_strcmp
_strcpy
_strerror
_strlen
_strncpy
_strrchr
_strtod
_strtof
_strtold
_system
_tanhl
_tgammal
_toupper
_truncl
_ungetc
_vsnprintf
_wcscmp
_wcslen


And that is the *entirety* of what phobos.lib needs for everything in phobos. A big chunk of it is Windows API imports. A quick look through it shows the following related to C I/O:

_printf
___fp_lock
___fp_unlock
__fputc_nlock
__fputwc_nlock
__iob
_exit
_fclose
_fdopen
_feof
_fflush
_fgetc
_fopen
_fprintf
_fputc
_fread
_free
_fseek
_ftell
_fwide
_fwrite
_ungetc

Applying grep tells us where these are used: modules like std.cstream, which is "C" streams, no surprise there. gzio.c, a C function that's part of zlib, no surprise there, either. trace's file logging function, no big deal, that wouldn't get shipped with a released application. std.stdio, no surprise there either, it needs to sync with C's stdio.

There isn't anywhere near 50 routines, and if you don't use cstream, zlib, trace, or writef, it's hard to see any difficulty at all. In fact, I find it hard to find any other uses of stdio other than printf (perhaps I overlooked some?). So why can't 'hooking' printf, as outlined above, work?

BTW, internal\dmain2.d also uses printf to print out an error message relating to any uncaught exceptions.

On Fri, 07 Apr 2006 14:05:02 +1000, Walter Bright <newshound@digitalmars.com> wrote:


> If you want to use a system that for some reason can't have C's IO subsystem, then just include the one liner:
>
> extern (C) int printf(char* f, ...) { return 0; }
>
> somewhere in your code, and it's gone.

So I did this, and it reduced the object size by 118 bytes. However, it still seems all the C I/O system is linked in as the only difference was that _vprintf was not linked in. Every other symbol was still linked.

-- 
Derek Parnell
Melbourne, Australia

Walter Bright wrote:
> kris wrote:
>> Why do you think Ares exists anyway?

> I'm trying to understand.

Ah. That makes some sense now ~ so, you're saying you don't see any need for a better library? That you're trying to understand what such a need might be?


> Here's the issue I have - postings that adamantly offer solutions without identifying the actual problem. Here are some examples:
> 
> Solution: remove printf
> Alleged problem: printf pulls in floating point formatting code
> Actual problem: std.string pulls in floating point formatting code due to reference to __fltused. printf does not pull in floating point formatting code.
> Correct solution: fix compiler to not generate __fltused references in library code
> 
> Solution: implement separate itoa() for typeinfo
> Alleged problem: calling one function in std.string pulls in everything in std.string
> Actual problem: only a small portion of std.string is actually linked in because the free functions are implemented as COMDATs, but due to an error in the compiler, lambda functions are not written as COMDATs. This causes a reference to std.uni to still be pulled in, pulling in a large table in std.uni
> Correct solution: fix compiler to generate COMDATs for lambda functions.


If you'll read the posts again, sans prejudice, I hope you'll find that they are about decoupling (like the title says). If you haven't heard of it before, it's a generally and widely applicable concept in software design. Been applied for probably 40 years now. If that thrust is not clear from my writing, then I've been totally lacking in my own conviction.

The 'solutions' you note above are being rather frugal with the truth:

Let's face it; printf should probably be removed from object.d purely because it represents poor design judgement (yes; my opinion. I have one). That aside; you make much of the fact that it can be stubbed out, whilst staunchly "refusing" to examine why it should be present in the first place. Instead, there's the persistent "it doesn't make any difference to remove it" stonewall. That approach inevitably leads to a less than fruitful discourse, and fits the description of futile (since you asked).

TypeInfo et. al. should most probably strive to be as isolated as they can be from higher level modules (which includes printf). Hooking it up contrary to this manner is sometimes called "tight coupling", and it's what this topic is about. Implementing a local itoa() is one way to decouple TypeInfo; linking to the C lib itoa() is another. You made it implicitly clear there was no way you'd consider isolating object.d from std.string in order to make the former more amenable to alternate libraries. Thus, that aspect was completely ignored also.

Looking again at your recital of adamant examples, I'm rather sorry, and entirely disappointed that's all you got from this exchange. Yes, there's certainly truth there; but it apparently makes a point of purging all mention of decoupling ~ that's where frugality lies.


> And that is the *entirety* of what phobos.lib needs for everything in phobos. A big chunk of it is Windows API imports. A quick look through it shows the following related to C I/O:
> 
> _printf
> ___fp_lock
> ___fp_unlock
> __fputc_nlock
> __fputwc_nlock
> __iob
> _exit
> _fclose
> _fdopen
> _feof
> _fflush
> _fgetc
> _fopen
> _fprintf
> _fputc
> _fread
> _free
> _fseek
> _ftell
> _fwide
> _fwrite
> _ungetc
> 
> Applying grep tells us where these are used: modules like std.cstream, which is "C" streams, no surprise there. gzio.c, a C function that's part of zlib, no surprise there, either. trace's file logging function, no big deal, that wouldn't get shipped with a released application. std.stdio, no surprise there either, it needs to sync with C's stdio.
> 
> There isn't anywhere near 50 routines, and if you don't use cstream, zlib, trace, or writef, it's hard to see any difficulty at all. In fact, I find it hard to find any other uses of stdio other than printf (perhaps I overlooked some?). So why can't 'hooking' printf, as outlined above, work?


Yeah ~ I posted the same lists last year, gained simply by hiding the C library.

You've again omitted a crucial part. The C runtime itself apparently has all kinds of interdependencies (the console startup/exit code is a prime example). Thus, the lists you show are simply the tip of the iceberg. I imagine you already know this quite intimately, so will suggest you do a step-by-step examination of the .map file for Derek's example:

void main() {}

and ask yourself just why and where the kitchen-sink is linked? I'm not telling you this to be a jerk ~ it would surely be of benefit to D if you were to understand where the dependencies actually lie.


> 
> BTW, internal\dmain2.d also uses printf to print out an error message relating to any uncaught exceptions.


I had not forgotton it. There's a time for that one also.

kris wrote:

>> BTW, internal\dmain2.d also uses printf to print out an error message relating to any uncaught exceptions.
> 
> I had not forgotton it. There's a time for that one also.

Maybe then the bug with it printing to the wrong stream can be fixed ?

Errors should be printed on stderr (fprintf), not on stdout (printf)...

--anders

Walter Bright wrote:
> 
> BTW, internal\dmain2.d also uses printf to print out an error message relating to any uncaught exceptions.

Ares currently uses fprintf for this purpose, which is essentially the same thing.  And as you say, it would be easy enough to hook the function if this behavior isn't desirable.

I'm glad the compiler issues came to light however.  I don't suppose this will have any impact on the template code generation problem in libraries?


Sean

Fredrik Olsson wrote:
> Walter Bright skrev:
>> Georg Wrede wrote:
>>> I admit this is a "feelings based" thing with most people I've talked with. It seems that on embedded platforms, many expect to write all the needed code themselves. It's also felt (possibly unduely??) that Phobos (or whatever general Win+*nix standard library) is mostly useless in embedded applications.
>>
>> I'd like to get to the bottom of this feeling. For example, Kris was unhappy that typeinfo imported std.strings. I can't figure out what the problem with that is.
>>
> I think it is the perceived bloat, not what is in practice done. Importing std.string while only using a single function still gives the impression of needing the whole module.

As I don't do much embedded programming, the issue for me is somewhat different, though the goals are similar: I want there to be a clean separation or clearly defined interaction between the runtime and standard library code to allow for the link-time integration of third-party standard libraries and garbage collectors.  I have little problem with using C library calls in the runtime however, as I see that as largely unavoidable.  Rather, my primary concern is that the runtime should not have any compile-time dependencies on standard library or GC code.

> Perhaps having a module scope of hmm... sys where typeinfo, object and anything needed by compiler, and runtime resides is a good idea. Totally forbid anything in "sys" to import/depend on anything from the outside. That way there would be no question for anyone about "how much is safe to strip"?

See Ares for another possible solution.

> And besides, is it wise to depend on what a linker "should do"? If current build chain nicely throws out what is not needed, does that make it right to assume that all build chains should behave as such?

I think this is a reasonable assumption, as to do otherwise necessitates design compromises to keep modules as small and isolated as possible. And while this may be reasonable for small projects, I can't see it working very well for large ones.

> As I see it each module in std should be as self contained as ever possible. I know the std.date I proposed imports std.conv, std.stdio, std.string and std.c.time, but my intent is to not import any of them when finished.

Again, I don't see much of a problem with calling C library functions in general, though I would prefer not doing so if it brings in an entire subsystem that is unrelated to anything done by the client code.

> But then my intention was never to bring up the internals to much debate, I wanted to have input on the externals, how you as developers use the code. That was perhaps futile, but I still think my approach of a few but flexible, and overloaded functions is the best approach.

That leaves a lot of room for interpretation, but I tenatively agree.


Sean

Anders F Björklund wrote:
> kris wrote:
> 
>>> BTW, internal\dmain2.d also uses printf to print out an error message relating to any uncaught exceptions.
>>
>> I had not forgotton it. There's a time for that one also.
> 
> Maybe then the bug with it printing to the wrong stream can be fixed ?
> 
> Errors should be printed on stderr (fprintf), not on stdout (printf)...

An easy change.  Just use fprintf and specify stderr as the output file.  Replacing printf entirely with something a bit less complex would be quite easy as well, as the error messages are just strings--there's no need for all the fancy stuff printf does.  I'll do this at some point for the DMD runtime in Ares, but no one's complained about it yet so I've put it off for now.


Sean

Sean Kelly wrote:

>> Errors should be printed on stderr (fprintf), not on stdout (printf)...
> 
> An easy change.  Just use fprintf and specify stderr as the output file. 

For some reason this change has been rejected earlier. I don't know why.
Just thought that if the file is revised, then maybe it can be included?

http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D.bugs/2001
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D.bugs/4368

--anders