View mode: basic / threaded / horizontal-split · Log in · Help
April 07, 2006
Re: On processors for D
Walter Bright wrote:
> Georg Wrede wrote:
> 
>> Uh-oh, after having read what Kris and others have posted as
>> replies to your post, I can't push D for embedded development. At
>> least until the issues they've brought up are resolved.
> 
> 
> Which particular issue?

Well,

> For what it's worth, the D spec doesn't require any of the behavior
> Kris has been talking about.  I'd consider most of it specific to the
> DMD implementation.
> 
> Sean

when Sean pointed this out, those issues of mine vanished, Poof!

Since DMD is Win+Lin _only_ (IIUC), none of the embedded woes apply. 
It's happened to me before, too, that I forget that DMD is not the same 
as the D spec. And that Phobos is not part of the spec.

So, actually, any discussions about embedded systems belong to D.gnu 
instead.

---

Still, I do agree with Kris on separation of concern, clear and 
consistent compartmentalization of dependencies, low level dependencies 
on high level code, modularization, etc., in Phobos.

Not addressing those issues, at least makes it harder to maintain and 
develop Phobos in the long run. IMHO.
April 07, 2006
Re: On processors for D
Walter Bright skrev:
> Georg Wrede wrote:
>> I admit this is a "feelings based" thing with most people I've talked 
>> with. It seems that on embedded platforms, many expect to write all 
>> the needed code themselves. It's also felt (possibly unduely??) that 
>> Phobos (or whatever general Win+*nix standard library) is mostly 
>> useless in embedded applications.
> 
> I'd like to get to the bottom of this feeling. For example, Kris was 
> unhappy that typeinfo imported std.strings. I can't figure out what the 
> problem with that is.
> 
I think it is the perceived bloat, not what is in practice done. 
Importing std.string while only using a single function still gives the 
impression of needing the whole module.


Perhaps having a module scope of hmm... sys where typeinfo, object and 
anything needed by compiler, and runtime resides is a good idea. Totally 
forbid anything in "sys" to import/depend on anything from the outside. 
That way there would be no question for anyone about "how much is safe 
to strip"?

And besides, is it wise to depend on what a linker "should do"? If 
current build chain nicely throws out what is not needed, does that make 
it right to assume that all build chains should behave as such?


As I see it each module in std should be as self contained as ever 
possible. I know the std.date I proposed imports std.conv, std.stdio, 
std.string and std.c.time, but my intent is to not import any of them 
when finished.

But then my intention was never to bring up the internals to much 
debate, I wanted to have input on the externals, how you as developers 
use the code. That was perhaps futile, but I still think my approach of 
a few but flexible, and overloaded functions is the best approach.


// Fredrik
April 07, 2006
Re: On processors for D ~ decoupling
kris wrote:
> Yes, they are useful. And I'm not trying to be a jerk by pointing out 
> that the D runtime is missing some much needed TLC (to put it very 
> nicely). Why do you think Ares exists anyway?

I'm trying to understand.


>> All those other names are are the static data. Things like:
>>
>> const dchar LS = '\u2028';      /// UTF line separator
>> const dchar PS = '\u2029';      /// UTF paragraph separator
>>
>> I submit that they aren't significant. 
> Yes, you're right. Unless they're huge tables (such as the Unicode 
> character map; oh wait; is that linked by default? :)

If you're talking about the one in std.uni, it took me a minute to 
figure that one out. The reason it's pulled in is  because of an error 
in the compiler, where lambda functions should be generated as comdats, 
but aren't. It is not a problem with the library design.

> What's all this about necessary re-engineering and rewriting of Phobos? 
> Where the heck did that come from?

You asked what the motivation for Ares was.


>> Why does the C library need replacing? I honestly don't get it.
> Who said it /needed/ replacing? I'm simply talking about applying a 
> small sprinkling of decoupling dust

You write: "If I have, as you say, a system that can't have C's IO 
subsystem, the above will hardly help me at all."


> Forgive me, but that's just printf(). If I have, as you say, a system 
> that can't have C's IO subsystem, the above will hardly help me at all. 
> I suspect you're well aware the C IO subsystem consists of significantly 
> more than just printf? Say, perhaps 50-ish functions? You're going to 
> suggest I stub them all out just because Object.print() calls printf()?

You only need to stub out the functions that pull in the rest. If it's 
only printf, then that's all you need to stub out. If you implement your 
own printf, then it will 'hook' any other calls to printf, and you can 
use it to call your own stuff. It's a common technique - heck, I've done 
it to 'hook' printf to write to a window instead of stdout.


> Given Derek's example:
> 
> void main() {}
> 
> The .map for that is a fine specimen of "unexpected" coupling. It does 
> seem as though much of that is actually in the C library, but then 
> you're arguing most fervently against doing anything to fix any such 
> things.

It is mostly the C runtime library. And I've lived with this for a very 
long time, don't you think I've looked at it to try to reduce the 
minimum required to be pulled in?


> Walter Bright wrote:
>  > Phobos doesn't require floating point support from the processor
>  > unless one actually uses floating point in the application code.
> 
> That turned out to be somewhat less than truthful.

It will be in the next update. I had assumed it was working properly 
when it wasn't - however, printf had nothing to do with the problem. I 
don't know why you call it "futility" - the problem will get fixed. The 
solutions you proposed wouldn't have fixed it.

>  > I also really don't understand why anyone using D would require
>  > not using Phobos. What's the problem?
> 
> With much respect, the 'problem' is perhaps that you don't see any?

Here's the issue I have - postings that adamantly offer solutions 
without identifying the actual problem. Here are some examples:

Solution: remove printf
Alleged problem: printf pulls in floating point formatting code
Actual problem: std.string pulls in floating point formatting code due 
to reference to __fltused. printf does not pull in floating point 
formatting code.
Correct solution: fix compiler to not generate __fltused references in 
library code

Solution: implement separate itoa() for typeinfo
Alleged problem: calling one function in std.string pulls in everything 
in std.string
Actual problem: only a small portion of std.string is actually linked in 
because the free functions are implemented as COMDATs, but due to an 
error in the compiler, lambda functions are not written as COMDATs. This 
causes a reference to std.uni to still be pulled in, pulling in a large 
table in std.uni
Correct solution: fix compiler to generate COMDATs for lambda functions.


I'm glad these two problems came to light, and I'm happy to fix them. 
It's why I ask "What's the problem?" rather than just applying solutions 
without understanding what problem the solution is aimed at. In the two 
cases above, the real problems were found only after I kept asking 
seemingly stupid questions about what problem you were trying to solve 
by removing printf and the call to std.string.toString.

I know you're still bothered by the issue of C's IO being pulled in. I 
have another utility for you - \dm\bin\libunres. Libunres will identify 
any symbols unresolved by a library. Running libunres on phobos.lib gives:

Unresolved externals:
??2@YAPAXI@Z
??2@YAPAXIPAX@Z
??3@YAXPAX@Z
?__stl_throw_length_error@std@@YAXPBD@Z
?__stl_throw_out_of_range@std@@YAXPBD@Z
_ExpandEnvironmentStringsA@12
_FreeLibrary@4
_GetFileType@4
_GetProcAddress@8
_GetVersion@0
_IID_IUnknown
_LoadLibraryA@4
_QueryPerformanceCounter@4
_RegCloseKey@4
_RegCreateKeyExA@36
_RegDeleteKeyA@8
_RegDeleteValueA@8
_RegEnumKeyExA@32
_RegEnumValueA@32
_RegFlushKey@4
_RegOpenKeyA@12
_RegOpenKeyExA@20
_RegQueryInfoKeyA@48
_RegQueryValueExA@24
_RegSetValueExA@24
_WSACleanup@0
_WSAGetLastError@0
_WSAStartup@8
__Ccmp
__Dmain
__LCMP@
__LDIV@
__U64_LDBL
__ULDIV@
___alloca
___fp_lock
___fp_unlock
___pfloatfmt
__assert
__beginthreadex
__end
__except_list
__fltused
__fputc_nlock
__fputwc_nlock
__global_unwind
__imp__CloseHandle@4
__imp__CopyFileA@12
__imp__CopyFileW@12
__imp__CreateDirectoryA@8
__imp__CreateDirectoryW@8
__imp__CreateFileA@28
__imp__CreateFileMappingA@24
__imp__CreateFileW@28
__imp__CreateWindowExA@48
__imp__DeleteCriticalSection@4
__imp__DeleteFileA@4
__imp__DeleteFileW@4
__imp__DuplicateHandle@28
__imp__EnterCriticalSection@4
__imp__FileTimeToSystemTime@8
__imp__FindClose@4
__imp__FindFirstFileA@8
__imp__FindFirstFileW@8
__imp__FindNextFileA@8
__imp__FindNextFileW@8
__imp__FlushViewOfFile@8
__imp__FormatMessageA@28
__imp__GetCurrentDirectoryA@8
__imp__GetCurrentDirectoryW@8
__imp__GetCurrentProcess@0
__imp__GetCurrentThread@0
__imp__GetCurrentThreadId@0
__imp__GetFileAttributesA@4
__imp__GetFileAttributesW@4
__imp__GetFileSize@8
__imp__GetFullPathNameA@16
__imp__GetLastError@0
__imp__GetModuleFileNameA@12
__imp__GetProcessTimes@20
__imp__GetSystemInfo@4
__imp__GetSystemTime@4
__imp__GetThreadContext@8
__imp__GetThreadTimes@20
__imp__GetTickCount@0
__imp__GetTimeZoneInformation@4
__imp__GetVersionExA@4
__imp__InitializeCriticalSection@4
__imp__InterlockedDecrement@4
__imp__InterlockedExchange@8
__imp__InterlockedIncrement@4
__imp__LeaveCriticalSection@4
__imp__LocalFree@4
__imp__MapViewOfFileEx@24
__imp__MoveFileA@8
__imp__MoveFileW@8
__imp__MultiByteToWideChar@24
__imp__QueryPerformanceCounter@4
__imp__QueryPerformanceFrequency@4
__imp__RaiseException@16
__imp__ReadFile@20
__imp__RemoveDirectoryA@4
__imp__RemoveDirectoryW@4
__imp__ResumeThread@4
__imp__SetCurrentDirectoryA@4
__imp__SetCurrentDirectoryW@4
__imp__SetFilePointer@16
__imp__SetThreadPriority@8
__imp__Sleep@4
__imp__SuspendThread@4
__imp__UnmapViewOfFile@4
__imp__VirtualAlloc@16
__imp__VirtualFree@12
__imp__WaitForSingleObject@8
__imp__WideCharToMultiByte@32
__imp__WriteFile@20
__imp__lstrcatA@8
__imp__lstrcmpA@8
__imp__lstrcpyA@8
__imp__lstrlenA@4
__iob
__local_except_handler
__snprintf
__vsnprintf
__xi_a
_accept@12
_acosl
_asinl
_atan2l
_atanl
_atoi
_bind@12
_calloc
_cbrtl
_ceill
_closesocket@4
_connect@12
_coshl
_erfcl
_erfl
_errno
_execv
_execve
_execvp
_execvpe
_exit
_exp2l
_expl
_expm1l
_fclose
_fdopen
_feof
_fflush
_fgetc
_floorl
_fopen
_fprintf
_fputc
_fread
_free
_fseek
_ftell
_fwide
_fwrite
_gethostbyaddr@12
_gethostbyname@4
_gethostname@8
_getpeername@12
_getprotobyname@4
_getprotobynumber@4
_getservbyname@8
_getservbyport@8
_getsockname@12
_getsockopt@20
_ilogbl
_inet_addr@4
_inet_ntoa@4
_ioctlsocket@12
_lgammal
_listen@8
_log10l
_log1pl
_log2l
_logbl
_logl
_malloc
_memchr
_memcmp
_memcpy
_memicmp
_memmove
_memset
_modfl
_nanl
_nearbyintl
_powl
_printf
_realloc
_recv@16
_recvfrom@24
_remainderl
_roundl
_select@20
_send@16
_sendto@24
_setsockopt@20
_shutdown@8
_sinhl
_socket@12
_spawnvp
_sprintf
_strcat
_strcmp
_strcpy
_strerror
_strlen
_strncpy
_strrchr
_strtod
_strtof
_strtold
_system
_tanhl
_tgammal
_toupper
_truncl
_ungetc
_vsnprintf
_wcscmp
_wcslen


And that is the *entirety* of what phobos.lib needs for everything in 
phobos. A big chunk of it is Windows API imports. A quick look through 
it shows the following related to C I/O:

_printf
___fp_lock
___fp_unlock
__fputc_nlock
__fputwc_nlock
__iob
_exit
_fclose
_fdopen
_feof
_fflush
_fgetc
_fopen
_fprintf
_fputc
_fread
_free
_fseek
_ftell
_fwide
_fwrite
_ungetc

Applying grep tells us where these are used: modules like std.cstream, 
which is "C" streams, no surprise there. gzio.c, a C function that's 
part of zlib, no surprise there, either. trace's file logging function, 
no big deal, that wouldn't get shipped with a released application. 
std.stdio, no surprise there either, it needs to sync with C's stdio.

There isn't anywhere near 50 routines, and if you don't use cstream, 
zlib, trace, or writef, it's hard to see any difficulty at all. In fact, 
I find it hard to find any other uses of stdio other than printf 
(perhaps I overlooked some?). So why can't 'hooking' printf, as outlined 
above, work?

BTW, internal\dmain2.d also uses printf to print out an error message 
relating to any uncaught exceptions.
April 07, 2006
Re: On processors for D ~ decoupling
On Fri, 07 Apr 2006 14:05:02 +1000, Walter Bright  
<newshound@digitalmars.com> wrote:


> If you want to use a system that for some reason can't have C's IO  
> subsystem, then just include the one liner:
>
> extern (C) int printf(char* f, ...) { return 0; }
>
> somewhere in your code, and it's gone.

So I did this, and it reduced the object size by 118 bytes. However, it  
still seems all the C I/O system is linked in as the only difference was  
that _vprintf was not linked in. Every other symbol was still linked.

-- 
Derek Parnell
Melbourne, Australia
April 07, 2006
Re: On processors for D ~ decoupling
Walter Bright wrote:
> kris wrote:
>> Why do you think Ares exists anyway?

> I'm trying to understand.

Ah. That makes some sense now ~ so, you're saying you don't see any need 
for a better library? That you're trying to understand what such a need 
might be?


> Here's the issue I have - postings that adamantly offer solutions 
> without identifying the actual problem. Here are some examples:
> 
> Solution: remove printf
> Alleged problem: printf pulls in floating point formatting code
> Actual problem: std.string pulls in floating point formatting code due 
> to reference to __fltused. printf does not pull in floating point 
> formatting code.
> Correct solution: fix compiler to not generate __fltused references in 
> library code
> 
> Solution: implement separate itoa() for typeinfo
> Alleged problem: calling one function in std.string pulls in everything 
> in std.string
> Actual problem: only a small portion of std.string is actually linked in 
> because the free functions are implemented as COMDATs, but due to an 
> error in the compiler, lambda functions are not written as COMDATs. This 
> causes a reference to std.uni to still be pulled in, pulling in a large 
> table in std.uni
> Correct solution: fix compiler to generate COMDATs for lambda functions.


If you'll read the posts again, sans prejudice, I hope you'll find that 
they are about decoupling (like the title says). If you haven't heard of 
it before, it's a generally and widely applicable concept in software 
design. Been applied for probably 40 years now. If that thrust is not 
clear from my writing, then I've been totally lacking in my own conviction.

The 'solutions' you note above are being rather frugal with the truth:

Let's face it; printf should probably be removed from object.d purely 
because it represents poor design judgement (yes; my opinion. I have 
one). That aside; you make much of the fact that it can be stubbed out, 
whilst staunchly "refusing" to examine why it should be present in the 
first place. Instead, there's the persistent "it doesn't make any 
difference to remove it" stonewall. That approach inevitably leads to a 
less than fruitful discourse, and fits the description of futile (since 
you asked).

TypeInfo et. al. should most probably strive to be as isolated as they 
can be from higher level modules (which includes printf). Hooking it up 
contrary to this manner is sometimes called "tight coupling", and it's 
what this topic is about. Implementing a local itoa() is one way to 
decouple TypeInfo; linking to the C lib itoa() is another. You made it 
implicitly clear there was no way you'd consider isolating object.d from 
std.string in order to make the former more amenable to alternate 
libraries. Thus, that aspect was completely ignored also.

Looking again at your recital of adamant examples, I'm rather sorry, and 
entirely disappointed that's all you got from this exchange. Yes, 
there's certainly truth there; but it apparently makes a point of 
purging all mention of decoupling ~ that's where frugality lies.


> And that is the *entirety* of what phobos.lib needs for everything in 
> phobos. A big chunk of it is Windows API imports. A quick look through 
> it shows the following related to C I/O:
> 
> _printf
> ___fp_lock
> ___fp_unlock
> __fputc_nlock
> __fputwc_nlock
> __iob
> _exit
> _fclose
> _fdopen
> _feof
> _fflush
> _fgetc
> _fopen
> _fprintf
> _fputc
> _fread
> _free
> _fseek
> _ftell
> _fwide
> _fwrite
> _ungetc
> 
> Applying grep tells us where these are used: modules like std.cstream, 
> which is "C" streams, no surprise there. gzio.c, a C function that's 
> part of zlib, no surprise there, either. trace's file logging function, 
> no big deal, that wouldn't get shipped with a released application. 
> std.stdio, no surprise there either, it needs to sync with C's stdio.
> 
> There isn't anywhere near 50 routines, and if you don't use cstream, 
> zlib, trace, or writef, it's hard to see any difficulty at all. In fact, 
> I find it hard to find any other uses of stdio other than printf 
> (perhaps I overlooked some?). So why can't 'hooking' printf, as outlined 
> above, work?


Yeah ~ I posted the same lists last year, gained simply by hiding the C 
library.

You've again omitted a crucial part. The C runtime itself apparently has 
all kinds of interdependencies (the console startup/exit code is a prime 
example). Thus, the lists you show are simply the tip of the iceberg. I 
imagine you already know this quite intimately, so will suggest you do a 
step-by-step examination of the .map file for Derek's example:

void main() {}

and ask yourself just why and where the kitchen-sink is linked? I'm not 
telling you this to be a jerk ~ it would surely be of benefit to D if 
you were to understand where the dependencies actually lie.


> 
> BTW, internal\dmain2.d also uses printf to print out an error message 
> relating to any uncaught exceptions.


I had not forgotton it. There's a time for that one also.
April 07, 2006
Re: On processors for D ~ decoupling
kris wrote:

>> BTW, internal\dmain2.d also uses printf to print out an error message 
>> relating to any uncaught exceptions.
> 
> I had not forgotton it. There's a time for that one also.

Maybe then the bug with it printing to the wrong stream can be fixed ?

Errors should be printed on stderr (fprintf), not on stdout (printf)...

--anders
April 07, 2006
Re: On processors for D ~ decoupling
Walter Bright wrote:
> 
> BTW, internal\dmain2.d also uses printf to print out an error message 
> relating to any uncaught exceptions.

Ares currently uses fprintf for this purpose, which is essentially the 
same thing.  And as you say, it would be easy enough to hook the 
function if this behavior isn't desirable.

I'm glad the compiler issues came to light however.  I don't suppose 
this will have any impact on the template code generation problem in 
libraries?


Sean
April 07, 2006
Re: On processors for D
Fredrik Olsson wrote:
> Walter Bright skrev:
>> Georg Wrede wrote:
>>> I admit this is a "feelings based" thing with most people I've talked 
>>> with. It seems that on embedded platforms, many expect to write all 
>>> the needed code themselves. It's also felt (possibly unduely??) that 
>>> Phobos (or whatever general Win+*nix standard library) is mostly 
>>> useless in embedded applications.
>>
>> I'd like to get to the bottom of this feeling. For example, Kris was 
>> unhappy that typeinfo imported std.strings. I can't figure out what 
>> the problem with that is.
>>
> I think it is the perceived bloat, not what is in practice done. 
> Importing std.string while only using a single function still gives the 
> impression of needing the whole module.

As I don't do much embedded programming, the issue for me is somewhat 
different, though the goals are similar: I want there to be a clean 
separation or clearly defined interaction between the runtime and 
standard library code to allow for the link-time integration of 
third-party standard libraries and garbage collectors.  I have little 
problem with using C library calls in the runtime however, as I see that 
as largely unavoidable.  Rather, my primary concern is that the runtime 
should not have any compile-time dependencies on standard library or GC 
code.

> Perhaps having a module scope of hmm... sys where typeinfo, object and 
> anything needed by compiler, and runtime resides is a good idea. Totally 
> forbid anything in "sys" to import/depend on anything from the outside. 
> That way there would be no question for anyone about "how much is safe 
> to strip"?

See Ares for another possible solution.

> And besides, is it wise to depend on what a linker "should do"? If 
> current build chain nicely throws out what is not needed, does that make 
> it right to assume that all build chains should behave as such?

I think this is a reasonable assumption, as to do otherwise necessitates 
design compromises to keep modules as small and isolated as possible. 
And while this may be reasonable for small projects, I can't see it 
working very well for large ones.

> As I see it each module in std should be as self contained as ever 
> possible. I know the std.date I proposed imports std.conv, std.stdio, 
> std.string and std.c.time, but my intent is to not import any of them 
> when finished.

Again, I don't see much of a problem with calling C library functions in 
general, though I would prefer not doing so if it brings in an entire 
subsystem that is unrelated to anything done by the client code.

> But then my intention was never to bring up the internals to much 
> debate, I wanted to have input on the externals, how you as developers 
> use the code. That was perhaps futile, but I still think my approach of 
> a few but flexible, and overloaded functions is the best approach.

That leaves a lot of room for interpretation, but I tenatively agree.


Sean
April 07, 2006
Re: On processors for D ~ decoupling
Anders F Björklund wrote:
> kris wrote:
> 
>>> BTW, internal\dmain2.d also uses printf to print out an error message 
>>> relating to any uncaught exceptions.
>>
>> I had not forgotton it. There's a time for that one also.
> 
> Maybe then the bug with it printing to the wrong stream can be fixed ?
> 
> Errors should be printed on stderr (fprintf), not on stdout (printf)...

An easy change.  Just use fprintf and specify stderr as the output file. 
 Replacing printf entirely with something a bit less complex would be 
quite easy as well, as the error messages are just strings--there's no 
need for all the fancy stuff printf does.  I'll do this at some point 
for the DMD runtime in Ares, but no one's complained about it yet so 
I've put it off for now.


Sean
April 07, 2006
Re: On processors for D ~ decoupling
Sean Kelly wrote:

>> Errors should be printed on stderr (fprintf), not on stdout (printf)...
> 
> An easy change.  Just use fprintf and specify stderr as the output file. 

For some reason this change has been rejected earlier. I don't know why.
Just thought that if the file is revised, then maybe it can be included?

http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D.bugs/2001
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D.bugs/4368

--anders
2 3 4 5 6 7 8 9
Top | Discussion index | About this forum | D home