torhu wrote:
> Kenny B wrote:
>> if you look later in my message... I have:
>>
>>  > gprof program gmon.out > profile_data && scite profile_data
>>
>> I haven't used windows in a while, but when I used it, it didn't have output redirection :D
> 
> I don't think there's ever been a windows version without output/input redirection and pipes.  That goes back to very old versions of DOS.
> 
> But I seem to recall that win98 didn't allow you to redirect stderr. Things like 2>&1 works in winxp, when using cmd.exe.
> 
> && and || doesn't work as far as i know.

&& works in XP.  I use it all the time.  Don't know about ||, but it's probably there too.

--bb

The idea of this builder is to obviate the need for all that manual stuff. Instead, it would be a fully-integrated way to click a button and  profile your project without needing a configuration.

Graham St Jack wrote:
> One other thing I like in a build system is to enforce design rules. The particular rules I use aren't important (dependency management) - the point is that a build system can help add a bit more rigour to a project, especially one with multiple developers. These rules would probably have to be described via some sort of extension mechanism, because there will never be agreement on what the rules should be.]

Can you give me an example of the sorts of rules you'd like to see?

On Mon, 25 Feb 2008 11:22:59 -0800, Robert Fraser wrote:

> Graham St Jack wrote:
>> One other thing I like in a build system is to enforce design rules. The particular rules I use aren't important (dependency management) - the point is that a build system can help add a bit more rigour to a project, especially one with multiple developers. These rules would probably have to be described via some sort of extension mechanism, because there will never be agreement on what the rules should be.]
> 
> Can you give me an example of the sorts of rules you'd like to see?

The rules I use in my own build system are:

Directory naming conventions determine what files are documentation, library source, executables and tests.

A test depends on its successful result file, and tests are run whenever their results file is out of date. If a test fails, the build fails.

A source module cannot depend on itself by any chain of imports.

A package depends on (from memory - I could have the details a bit wrong):
* all the packages it contains,
* all the sources it contains, and
* all the packages that its sources import, up to the common ancestor.

A package cannot depend on itself by any chain of dependencies.


I find the dependency rules make it a lot easier to keep the source well- factored, and results in a much cleaner design. It is far too hard to resist the temptation to sneak in a "bad" dependency between packages in the head of the moment - I like the build system to nip it in the bud.

The dependency rules also make it FAR easier to write reusable packages. Without rules like these (or a lot of willpower and only one developer), there is a strong tendency to end up with a monolith of tangled-together code.

The directory naming rules are just a way of eliminating a build configuration file - everything except compiler options can be worked out from where the files are and what they import.


These particular rules aren't for everyone, but especially on projects with multiple developers, automatic enforcement of rules of some kind can make things run a lot smoother. One thing that has stopped me from using Eclipse so far has been the difficulty of figuring out how to make it use a custom build system. It would be great if Descent had a build system that allowed rules to be added.

In Descent, this sort of thing isn't the builder's job (well it is, but the other builder - the Eclipse builder). There already is a partially-ported version of eclipse metrics which calculates things like dependencies and can help in resolving circular dependencies (if the circle is relatively small and all within the same package, I don't consider them bad -- but that's not the issue at hand :-)).

The sorts of rules you're talking about could easily be added as a plugin to Descent (well, not easily, but...) and seem to be more "Warnings on the project level" rather than anything to do with executable building.

Graham St Jack wrote:
> On Mon, 25 Feb 2008 11:22:59 -0800, Robert Fraser wrote:
> 
>> Graham St Jack wrote:
>>> One other thing I like in a build system is to enforce design rules.
>>> The particular rules I use aren't important (dependency management) -
>>> the point is that a build system can help add a bit more rigour to a
>>> project, especially one with multiple developers. These rules would
>>> probably have to be described via some sort of extension mechanism,
>>> because there will never be agreement on what the rules should be.]
>> Can you give me an example of the sorts of rules you'd like to see?
> 
> The rules I use in my own build system are:
> 
> Directory naming conventions determine what files are documentation, library source, executables and tests.
> 
> A test depends on its successful result file, and tests are run whenever their results file is out of date. If a test fails, the build fails.
> 
> A source module cannot depend on itself by any chain of imports.
> 
> A package depends on (from memory - I could have the details a bit wrong):
> * all the packages it contains,
> * all the sources it contains, and
> * all the packages that its sources import, up to the common ancestor.
> 
> A package cannot depend on itself by any chain of dependencies.
> 
> 
> I find the dependency rules make it a lot easier to keep the source well-
> factored, and results in a much cleaner design. It is far too hard to resist the temptation to sneak in a "bad" dependency between packages in the head of the moment - I like the build system to nip it in the bud.
> 
> The dependency rules also make it FAR easier to write reusable packages. Without rules like these (or a lot of willpower and only one developer), there is a strong tendency to end up with a monolith of tangled-together code.
> 
> The directory naming rules are just a way of eliminating a build configuration file - everything except compiler options can be worked out from where the files are and what they import.
> 
> 
> These particular rules aren't for everyone, but especially on projects with multiple developers, automatic enforcement of rules of some kind can make things run a lot smoother. One thing that has stopped me from using Eclipse so far has been the difficulty of figuring out how to make it use a custom build system. It would be great if Descent had a build system that allowed rules to be added.

On Tue, 26 Feb 2008 16:37:15 -0800, Robert Fraser wrote:

> In Descent, this sort of thing isn't the builder's job (well it is, but the other builder - the Eclipse builder). There already is a partially-ported version of eclipse metrics which calculates things like dependencies and can help in resolving circular dependencies (if the circle is relatively small and all within the same package, I don't consider them bad -- but that's not the issue at hand :-)).

I'm happy with circularities in the same module, but not so happy in the same package - but as you say, that isn't the issue.

> 
> The sorts of rules you're talking about could easily be added as a plugin to Descent (well, not easily, but...) and seem to be more "Warnings on the project level" rather than anything to do with executable building.

If there is a plugin mechanism that isn't too tough then I am happy.

My experience is that as soon as you make something optional, then people will turn it off and code away, leaving an intractable mess for someone who cares to clean up later. I had that experience with unit tests - most people didn't run them, so they got out of date and it became too hard to fix them later. The only time to fix them is as soon as they get broken. Its the same (or even worse) for dependency violations.

Robert Fraser wrote:
> Instead, an IDE is a _development_ environment, so builds performed within the scope of the IDE should work to support development and testing, not deployment.

i don't understand why there should be a distinction between building during development and for deployment. after all you trying to develop the software that you eventually are deploying, not something else.

> In particular, building within the scope of an IDE should have the following properties:
> 
> - It should be fast. Builds, generally with only small changes, are very frequent in a development or ad-hoc testing environment, and thus should run as quickly as possible. Rebuilding the project with all its dependencies for a one-line change is not an option.

re-building modules only when dependencies have changed is on of the most basic features of a build tool, regardless of whether it's embedded into an IDE or not.

> A word needs to be said on incremental compilation. Recently, there have been a number of posts related to incremental compilation being much slower, including some rather disturbing figures for DWT ;-P. Until this gets fixed, I will have an option to allow "all at once" compilation for projects. This may be faster at first for pathological cases such as DWT, but after a few hundred builds, the incremental process will likely be faster for most projects.

what is slow, is compiling a single module at a time. incremental compilation does not have to do this. you can determine the set of modules that need re-compilation and compile all of those with one call.


For your feature requirements, i fail to see how this is specific to an IDE build tool. After all, an IDE is supposed to be a frontend to all the tools required during development, including build tools. Once i defined how to build my program, i don't want to specify it twice. If it is so simple that a build tool can figure it out on it's own, i never want to do it - not for development, nor for deployment.

So what you're actually proposing is to introduce a secondary build tool that is positioned at an earlier stage in the workflow, manifested by some, yet to be defined features that make it more suitable for that stage. That implies that current build tools don't do a good enough job at some type of build. I'd rather like to see those shortcomings and the reasons why they cannot be addressed in the existing build tools first.


Besides that, here is what i personally find useful, that not all build tools have:

- Rebuild also if only the build-parameters changed
I frequently jump back and forth between debug and release mode. I don't want to clean explicitly each time. scons does that - it saves all the build parameters for each module and considers them when determining whether a module is dirty.

- Have all build parameters compactly editable in one place. All GUI based build tool interfaces i know only provide dialogs with all available parameters. With tools like scons or dsss OTOH, i have a single, small text file, that only specifies exactly what is relevant to my project. I don't have to click my way through endless tabs and dialogs to get to the settings that i want to change.

- I can parameterize my build arbitrarily (debug, release, with test-code for module group A, B or C, etc.). With selection of version-groups being just a single parameter to the build-call that i bind to some shortcut in the editor. It's merely a matter of one or two key/value entries in my project specific config files.

Hi Jascha. You make a number of valid points, but you may not understand exactly where I'm coming from. In particular, I think you're thinking of what I'm describing as a build tool, while it's actually the removal of a build tool from the user's perspective, and instead a way to set up run configurations from which the build follows. See inline:

Jascha Wetzel wrote:
> Robert Fraser wrote:
>> Instead, an IDE is a _development_ environment, so builds performed within the scope of the IDE should work to support development and testing, not deployment.
> 
> i don't understand why there should be a distinction between building during development and for deployment. after all you trying to develop the software that you eventually are deploying, not something else.
> 
>> In particular, building within the scope of an IDE should have the following properties:
>>
>> - It should be fast. Builds, generally with only small changes, are very frequent in a development or ad-hoc testing environment, and thus should run as quickly as possible. Rebuilding the project with all its dependencies for a one-line change is not an option.
> 
> re-building modules only when dependencies have changed is on of the most basic features of a build tool, regardless of whether it's embedded into an IDE or not.
> 
>> A word needs to be said on incremental compilation. Recently, there have been a number of posts related to incremental compilation being much slower, including some rather disturbing figures for DWT ;-P. Until this gets fixed, I will have an option to allow "all at once" compilation for projects. This may be faster at first for pathological cases such as DWT, but after a few hundred builds, the incremental process will likely be faster for most projects.
> 
> what is slow, is compiling a single module at a time. incremental compilation does not have to do this. you can determine the set of modules that need re-compilation and compile all of those with one call.

Can't this introduce spurious dependencies between the modules? Oh well, I'm relying on rebuild to do the dependency analysis/collection/compiler activation at this point, so I hope it takes care of that part.

> For your feature requirements, i fail to see how this is specific to an IDE build tool. After all, an IDE is supposed to be a frontend to all the tools required during development, including build tools. Once i defined how to build my program, i don't want to specify it twice. If it is so simple that a build tool can figure it out on it's own, i never want to do it - not for development, nor for deployment.

But what is a "development build"? If it's a run/debug configuration, you probably want --release off, optimization and inlining off (since it can screw up debugging), debug symbols on, etc. If it's a profile configuration, you may or may not want optimization/inlining, but you definitely want instrumentation for profiling so it provides profile output.

Further, deployment builds are often a lot more than simple builds. Where I work, the incremental build includes not only the build itself, but functional unit tests, a number of scripts running against the database, a push to test environments, e-mail notification to all the developers/testers, etc., etc. This is what a build tool like Ant is for. An IDE is for a different purpose, and I'm sure you'll agree an IDE should not support all of that.

> So what you're actually proposing is to introduce a secondary build tool that is positioned at an earlier stage in the workflow, manifested by some, yet to be defined features that make it more suitable for that stage. That implies that current build tools don't do a good enough job at some type of build. I'd rather like to see those shortcomings and the reasons why they cannot be addressed in the existing build tools first.

In a way (as I mentioned, I'm switching to rebuild to perform the actual compile/link). The features are defined as whatever features the relevant use case adds: features such as automated unit testing, automatic collection of profiling results and integration of these results into the editor, or debugger integration.

> Besides that, here is what i personally find useful, that not all build tools have:
> 
> - Rebuild also if only the build-parameters changed
> I frequently jump back and forth between debug and release mode. I don't want to clean explicitly each time. scons does that - it saves all the build parameters for each module and considers them when determining whether a module is dirty.

My original build system had that. Right now I'm using rebuild as-is, but I may tweak it a bit as well as give it a socket-based interface (if Gregor is cool with that) so it'll be better for builds without cleans.

> - Have all build parameters compactly editable in one place. All GUI based build tool interfaces i know only provide dialogs with all available parameters. With tools like scons or dsss OTOH, i have a single, small text file, that only specifies exactly what is relevant to my project. I don't have to click my way through endless tabs and dialogs to get to the settings that i want to change.

That's what the whole "use case" thing is about. For development builds, most of these parameters are determined by how you want to use it, so will be configurable as-needed in the use case configuration, or not at all (since if you click the "debug" button, you'll always want debug symbols, or if you click the "unit test" button you'll always want unit tests compiled in). The user will never need to think about the exact command-line options that are being passed to the tool, but instead how she plans to use the output, and the particular options will stem from these choices. I know this may make some users uncomfortable (if you're used to build tools as you describe them), which is why I'm planning to eventually add DSSS support.

> - I can parameterize my build arbitrarily (debug, release, with test-code for module group A, B or C, etc.). With selection of version-groups being just a single parameter to the build-call that i bind to some shortcut in the editor. It's merely a matter of one or two key/value entries in my project specific config files.

I'm not sure exactly what you mean by this.
- If you mean different build/run configurations for the same project, this is definitely supported, and I think your idea of adding key bindings is a good one (this may already be in Eclipse, if not, I'll add it).
- If you mean debug/version identifiers on a per-file basis, I didn't think as much about this... How important is this to people?