View mode: basic / threaded / horizontal-split · Log in · Help
May 02, 2008
Descent speed
I tried to create a test project to work with descent 0.5.2.

DwtSamples http://hg.dsource.org/projects/dwt-samples
additional include dirs:
- tango
- dwt-linux or dwt-win
[- dwt-addons]

Now i noticed that triggering the auto completion, makes Eclipse block 
for about 3-5 seconds. Did i miss something?
For me, this speed thing is still the most important blocker for using 
Descent.
Perhaps you can take the dwt-examples project as a speed indicator for 
doing optimations. Its easy to setup and if you want to test an even 
bigger thing, just try the jface examples and add the dwt-addons include 
path.

I also noticed that autocompletition does not work in anonymous classes, 
accessing external local variables. Like "folder" in 
dwtsnippets/ctabfolder/Snippet165.d line 72.
http://www.dsource.org/projects/dwt-samples/browser/dwtsnippets/ctabfolder/Snippet165.d#L72

Beside of this ... Great work, thanks for this Descent release.
May 03, 2008
Re: Descent speed
Frank Benoit escribió:
> I tried to create a test project to work with descent 0.5.2.
> 
> DwtSamples http://hg.dsource.org/projects/dwt-samples
> additional include dirs:
> - tango
> - dwt-linux or dwt-win
> [- dwt-addons]
> 
> Now i noticed that triggering the auto completion, makes Eclipse block 
> for about 3-5 seconds. Did i miss something?
> For me, this speed thing is still the most important blocker for using 
> Descent.
> Perhaps you can take the dwt-examples project as a speed indicator for 
> doing optimations. Its easy to setup and if you want to test an even 
> bigger thing, just try the jface examples and add the dwt-addons include 
> path.
> 
> I also noticed that autocompletition does not work in anonymous classes, 
> accessing external local variables. Like "folder" in 
> dwtsnippets/ctabfolder/Snippet165.d line 72.
> http://www.dsource.org/projects/dwt-samples/browser/dwtsnippets/ctabfolder/Snippet165.d#L72 
> 
> 
> Beside of this ... Great work, thanks for this Descent release.

Intersting. Ok, I'll try to optimize on that.

So I began. But first I wanted to compare it to something that works 
more or less well to me: DFL. (big note: I'm not comparing DFL with SWT, 
I'm just comparing two libraries I know).

Here is the DFL program I'll be trying:

---
module main;

import dfl.label;

int main(char[][] args) {
	Label label;
	return 0;
}
---

And here is the SWT one:

---
module main;

import dwt.widgets.Text;

int main(char[][] args) {	
	Text text;
	return 0;
}
---

Now, Descent needs to resolve the imports, and the imports of the 
imported modules recursively. You can't resolve on demand because 
symbols may be created because of compile-time funcionality, and we want 
here good support for that.

So below are the dependencies trees of those codes. That explains the 
time difference between the libraries with regards to autocompletion. Of 
course, I'll try to optimize further, but it's always a battle between 
speed and correct semantic resolution.

DFL Label (59 modules loaded)
------------------------------------------------------
main
  object
  dfl.label
    dfl.base
      dfl.internal.dlib
        std.thread
          std.c.windows.windows
        std.traits
        std.gc
          gcstats
        std.string
          std.stdio
            std.c.stdio
              std.c.stddef
              std.c.stdarg
            std.format
              std.stdarg
              std.utf
              std.c.stdlib
              std.c.string
          std.uni
          std.array
          std.ctype
        std.path
        std.outofmemory
        std.conv
          std.math
            std.c.math
      dfl.internal.clib
        std.stdint
      dfl.internal.winapi
        dfl.internal.wincom
          std.c.windows.com
      dfl.drawing
        dfl.internal.utf
          std.windows.charset
            std.windows.syserror
        dfl.internal.com
          std.stream
            std.system
            std.intrinsic
            std.file
              std.regexp
                std.outbuffer
                std.bitarray
              std.date
                std.dateparse
            std.mmfile
      dfl.event
    dfl.control
      dfl.menu
        dfl.application
          dfl.form
            dfl.collections
          dfl.button
          dfl.textbox
          dfl.environment
          dfl.resources
      dfl.data

SWT - Text (237 modules loaded)
------------------------------------------------------
main
  object
  dwt.widgets.Text
    dwt.DWT
      dwt.internal.Compatibility
        dwt.dwthelper.FileInputStream
          dwt.dwthelper.utils
            dwt.dwthelper.System
              tango.core.Exception
              tango.time.Clock
                tango.time.Time
                tango.sys.Common
                  tango.sys.win32.UserGdi
                    tango.sys.win32.Types
              tango.stdc.stdlib
                tango.stdc.stddef
                tango.stdc.config
            tango.math.Math
              tango.stdc.math
              tango.math.IEEE
              tango.stdc.stdio
                tango.stdc.stdarg
            tango.io.Stdout
              tango.io.Print
                tango.io.model.IBuffer
                  tango.io.model.IConduit
                tango.text.convert.Layout
                  tango.text.convert.Utf
                  tango.text.convert.Float
                    tango.text.convert.Integer
              tango.io.Console
                tango.io.Buffer
                tango.io.DeviceConduit
                  tango.io.Conduit
            tango.stdc.stringz
            tango.text.Util
            tango.text.Unicode
              tango.text.UnicodeData
          dwt.dwthelper.File
            tango.io.FileConst
            tango.io.FilePath
              tango.io.Path
            tango.io.FileSystem
          dwt.dwthelper.InputStream
          tango.io.FileConduit
          tango.io.protocol.Reader
            tango.io.protocol.model.IReader
              tango.io.protocol.model.IProtocol
          tango.text.convert.Format
        dwt.dwthelper.FileOutputStream
          dwt.dwthelper.OutputStream
        dwt.dwthelper.InflaterInputStream
        tango.sys.Process
          tango.sys.Pipe
          tango.stdc.string
      dwt.internal.Library
        tango.util.Convert
          tango.core.Traits
          tango.core.Tuple
          tango.text.Ascii
      dwt.internal.Platform
      dwt.DWTError
      dwt.DWTException
    dwt.events.ModifyListener
      dwt.internal.DWTEventListener
      dwt.events.ModifyEvent
        dwt.widgets.Event
          dwt.graphics.GC
            dwt.internal.gdip.Gdip
              dwt.internal.gdip.native
                dwt.internal.win32.WINTYPES
                  tango_sys_win32.Types
                tango.sys.SharedLib
                tango.util.log.Trace
              dwt.internal.win32.WINAPI
                tango_sys_win32.UserGdi
              dwt.internal.win32.OS
                dwt.internal.C
                tango.io.TempFile
                  tango.math.Random
                tango.io.File
                tango_sys_win32.CodePage
            dwt.graphics.Color
              dwt.graphics.Resource
                dwt.graphics.Device
                  dwt.graphics.Drawable
                    dwt.graphics.GCData
                      dwt.graphics.Image
                        dwt.graphics.ImageData
                          dwt.graphics.PaletteData
                            dwt.graphics.RGB
                              dwt.internal.SerializableCompatibility
                          dwt.graphics.ImageDataLoader
                            dwt.graphics.ImageLoader
                              dwt.graphics.ImageLoaderListener
                                dwt.graphics.ImageLoaderEvent
                                  dwt.internal.DWTEventObject
                              dwt.internal.image.FileFormat
                                dwt.internal.image.LEDataInputStream
                                dwt.internal.image.LEDataOutputStream
                                dwt.internal.image.GIFFileFormat
                                  dwt.internal.image.LZWCodec
                                    dwt.internal.image.LZWNode
                                dwt.internal.image.WinBMPFileFormat
                                  dwt.graphics.Point
                                  dwt.dwthelper.ByteArrayOutputStream
                                dwt.internal.image.WinICOFileFormat
                                dwt.internal.image.TIFFFileFormat
                                  dwt.internal.image.TIFFRandomFileAccess
                                  dwt.internal.image.TIFFDirectory

dwt.internal.image.TIFFModifiedHuffmanCodec
                                    dwt.dwthelper.Integer
                                dwt.internal.image.OS2BMPFileFormat
                                dwt.internal.image.JPEGFileFormat
                                  dwt.internal.image.JPEGFrameHeader

dwt.internal.image.JPEGVariableSizeSegment
                                      dwt.internal.image.JPEGSegment
                                  dwt.internal.image.JPEGScanHeader
                                  dwt.internal.image.JPEGHuffmanTable
                                  dwt.internal.image.JPEGAppn
                                  dwt.internal.image.JPEGComment

dwt.internal.image.JPEGArithmeticConditioningTable
                                  dwt.internal.image.JPEGRestartInterval
                                    dwt.internal.image.JPEGFixedSizeSegment
                                  dwt.internal.image.JPEGQuantizationTable
                                  dwt.internal.image.JPEGStartOfImage
                                  dwt.internal.image.JPEGDecoder
                                  dwt.internal.image.JPEGEndOfImage
                                dwt.internal.image.PNGFileFormat
                                  dwt.internal.image.PngIhdrChunk
                                    dwt.internal.image.PngFileReadState
                                    dwt.internal.image.PngChunk
                                      dwt.internal.image.PngPlteChunk
                                      dwt.internal.image.PngIdatChunk
                                      dwt.internal.image.PngIendChunk
                                      dwt.internal.image.PngTrnsChunk
                                  dwt.internal.image.PngChunkReader
                                  dwt.internal.image.PngEncoder
                                    dwt.internal.image.PngDeflater
                                  dwt.internal.image.PngInputStream
                                  dwt.internal.image.PngDecodingDataStream
                                    dwt.internal.image.PngLzBlockReader
                                      dwt.internal.image.PngHuffmanTables
                                        dwt.internal.image.PngHuffmanTable
                                  dwt.dwthelper.BufferedInputStream
                              tango.core.Array
                          dwt.internal.CloneableCompatibility
                        dwt.graphics.Rectangle
                      dwt.graphics.Pattern
                  dwt.graphics.DeviceData
                  dwt.graphics.FontData
                    dwt.dwthelper.Float
                    tango.text.Text
                  dwt.graphics.Font
                  dwt.dwthelper.Runnable
            dwt.graphics.FontMetrics
            dwt.graphics.Path
              dwt.graphics.PathData
            dwt.graphics.Region
            dwt.graphics.Transform
            dwt.graphics.LineAttributes
          dwt.widgets.Widget
            dwt.events.DisposeListener
              dwt.events.DisposeEvent
                dwt.events.TypedEvent
                  dwt.widgets.Display
                    dwt.graphics.Cursor
                    dwt.internal.ImageList
                    dwt.widgets.Control
                      dwt.accessibility.Accessible
                        dwt.accessibility.AccessibleControlListener
                          dwt.accessibility.AccessibleControlEvent
                        dwt.accessibility.AccessibleListener
                          dwt.accessibility.AccessibleEvent
                        dwt.accessibility.AccessibleTextListener
                          dwt.accessibility.AccessibleTextEvent
                      dwt.events.ControlListener
                        dwt.events.ControlEvent
                      dwt.events.DragDetectListener
                        dwt.events.DragDetectEvent
                          dwt.events.MouseEvent
                      dwt.events.FocusListener
                        dwt.events.FocusEvent
                      dwt.events.HelpListener
                        dwt.events.HelpEvent
                      dwt.events.KeyListener
                        dwt.events.KeyEvent
                      dwt.events.MenuDetectListener
                        dwt.events.MenuDetectEvent
                      dwt.events.MouseListener
                      dwt.events.MouseMoveListener
                      dwt.events.MouseTrackListener
                      dwt.events.MouseWheelListener
                      dwt.events.PaintListener
                        dwt.events.PaintEvent
                      dwt.events.TraverseListener
                        dwt.events.TraverseEvent
                      dwt.widgets.Composite
                        dwt.widgets.Scrollable
                          dwt.widgets.ScrollBar
                            dwt.events.SelectionEvent
                            dwt.events.SelectionListener
                            dwt.widgets.TypedListener
                              dwt.widgets.Listener
                              dwt.events.ArmEvent
                              dwt.events.ArmListener
                              dwt.events.ExpandEvent
                              dwt.events.ExpandListener
                              dwt.events.MenuEvent
                              dwt.events.MenuListener
                              dwt.events.ShellEvent
                              dwt.events.ShellListener
                              dwt.events.TreeEvent
                              dwt.events.TreeListener
                              dwt.events.VerifyEvent
                              dwt.events.VerifyListener
                        dwt.widgets.Layout
                        dwt.widgets.Menu
                          dwt.widgets.Decorations
                            dwt.widgets.Canvas
                              dwt.widgets.Caret
                            dwt.widgets.Button
                            dwt.widgets.Shell
                              dwt.widgets.ToolTip
                                dwt.widgets.TrayItem
                                  dwt.widgets.Item
                                  dwt.widgets.Tray
                            dwt.widgets.MenuItem
                      dwt.widgets.Monitor
                    dwt.widgets.EventTable
                    dwt.widgets.Synchronizer
                      dwt.widgets.RunnableLock
                        tango.core.Thread
                          tango.stdc.stdint
                        tango.core.sync.Condition
                          tango.core.sync.Mutex
                          tango.core.sync.Semaphore
May 03, 2008
Re: Descent speed
Ary Borenszweig escribió:
> Frank Benoit escribió:
> So below are the dependencies trees of those codes.

Great. I found a way to reduce the needed tree to a about a third of it. 
A lot of semantic errors appeared, but I'll try to see what's going on. 
Trying it on a SWT-example works pretty well now, but I guess I can 
furhter optimize things.

You can expect a new release one of this days, which will also include 
an UI to configure file imports, made by Robert, and maybe some 
"browsing" views, similar to the ones you get in the Java Browsing 
perspective. :-)

Thanks for the feedback!
May 03, 2008
Re: Descent speed
Why do you need to parse the private import in imported modules?
May 03, 2008
Re: Descent speed
Raynor escribió:
> Why do you need to parse the private import in imported modules?

---
module A;

class Bar {
	int property;
}
---
module B;

import A;

class Foo {
  Bar method() {
    return new Bar();
  }
}
---
module main;

import B;

void foo(Foo foo) {
  foo.method(). <-- just knowing B isn't enough to suggest "property"
}
---

But what's happening also, is that *every* import, even if it's not used 
in the current module, is being "parsed" (not parsed, but reconstructed 
from memory cache). That's because DMD works like that. DMD needs to 
resolve everything, because then an object file will be created. I 
already did some optimizations to cut down the semantic analysis needed, 
and I'm going for more... The hard thing is doing this correctly and not 
breaking semantic analysis (i.e.: reporting false errors, resolving 
symbols badly, etc.).
May 03, 2008
Re: Descent speed
Ary Borenszweig schrieb:
> But what's happening also, is that *every* import, even if it's not used 
> in the current module, is being "parsed" (not parsed, but reconstructed 
> from memory cache). That's because DMD works like that. DMD needs to 
> resolve everything, because then an object file will be created. I 
> already did some optimizations to cut down the semantic analysis needed, 
> and I'm going for more... The hard thing is doing this correctly and not 
> breaking semantic analysis (i.e.: reporting false errors, resolving 
> symbols badly, etc.).

Would it be a good strategy to prebuild all autocomplete information, so 
it is always available for every existing type?
May 03, 2008
Re: Descent speed
Frank Benoit escribió:
> Ary Borenszweig schrieb:
>> But what's happening also, is that *every* import, even if it's not 
>> used in the current module, is being "parsed" (not parsed, but 
>> reconstructed from memory cache). That's because DMD works like that. 
>> DMD needs to resolve everything, because then an object file will be 
>> created. I already did some optimizations to cut down the semantic 
>> analysis needed, and I'm going for more... The hard thing is doing 
>> this correctly and not breaking semantic analysis (i.e.: reporting 
>> false errors, resolving symbols badly, etc.).
> 
> Would it be a good strategy to prebuild all autocomplete information, so 
> it is always available for every existing type?

But a type's information changes even if you don't modify it.

---
module a;

import b;

class Foo {
	mixin Bar!();
}
---
module b;

import c;

template Bar() {
	static if (is(Baz)) {
		int property;
	} else {
		float property;
	}
}
---

If I change module "b", autocompletion for class Foo changes. If "c" 
doesn't have a Baz symbol and I define one, autocompletion for class Foo 
changes. Even if you change the current module's symbols, autocompletion 
for some other class might change. So you have to rebuild that 
information constantly, and for that you might want to take a look at a 
lot of modules... so it reduces to the same problem, more or less. 
(that's my reasoning, if I'm wrong, please tell me!)

Sorry... D is hard for autocompletion. :-(

I want to provide correct autocompletion always. I know it's hard, but 
it's also a challenge, and I think it's possible...
May 03, 2008
Re: Descent speed
Ary Borenszweig escribió:
> Frank Benoit escribió:
>> Ary Borenszweig schrieb:
>>> But what's happening also, is that *every* import, even if it's not 
>>> used in the current module, is being "parsed" (not parsed, but 
>>> reconstructed from memory cache). That's because DMD works like that. 
>>> DMD needs to resolve everything, because then an object file will be 
>>> created. I already did some optimizations to cut down the semantic 
>>> analysis needed, and I'm going for more... The hard thing is doing 
>>> this correctly and not breaking semantic analysis (i.e.: reporting 
>>> false errors, resolving symbols badly, etc.).
>>
>> Would it be a good strategy to prebuild all autocomplete information, 
>> so it is always available for every existing type?
> 
> But a type's information changes even if you don't modify it.
> 
> ---
> module a;
> 
> import b;
> 
> class Foo {
>     mixin Bar!();
> }
> ---
> module b;
> 
> import c;
> 
> template Bar() {
>     static if (is(Baz)) {
>         int property;
>     } else {
>         float property;
>     }
> }
> ---
> 
> If I change module "b", autocompletion for class Foo changes. If "c" 
> doesn't have a Baz symbol and I define one, autocompletion for class Foo 
> changes. Even if you change the current module's symbols, autocompletion 
> for some other class might change. So you have to rebuild that 
> information constantly, and for that you might want to take a look at a 
> lot of modules... so it reduces to the same problem, more or less. 
> (that's my reasoning, if I'm wrong, please tell me!)
> 
> Sorry... D is hard for autocompletion. :-(
> 
> I want to provide correct autocompletion always. I know it's hard, but 
> it's also a challenge, and I think it's possible...

More on this: if you save resolved information, you have to rebuild it 
from scratch when you change the debug/version idenfitiers, or when you 
change the active project. Descent currently stores unresolved 
information, which is a step between having nothing and having it fully 
resolved. So when you change the debug/version identifiers, you need to 
do nothing. But you need to re-do semantic analysis each time you open a 
file, but you already have precalculated the unresolved information (no 
reparsing of files needed). And now it's about optimizing that semantic 
pass, which I think it's possible.
July 05, 2008
Re: Descent speed
Ary Borenszweig wrote:
> 
> Sorry... D is hard for autocompletion. :-(
> 
> I want to provide correct autocompletion always. I know it's hard, but 
> it's also a challenge, and I think it's possible...

I'm not sure if I have mentioned this before already or not. It may be 
the case that achieving 100% may be to difficult, particularly 
performance wise. Doug Schaefer, lead developer of CDT, commented on 
this issue once, with regards to CDT's design:

http://cdtdoug.blogspot.com/2007/03/ego-less-development.html
"I totally changed the approach we had taken and threw away the 
requirement of being 100% accurate that we had be preaching in futility 
since we started."

Since D also has meta-programming (compile-time variance), a similar 
thing can also happen in Descent. On the other hand D is simpler and 
more sane than C++, so that goal is easier to achieve than in C++. But 
it may still be too difficult.
(I personally prefer auto-complete and other semantic features to be 
lightning fast even if only 99% accurate)


-- 
Bruno Medeiros - Software Developer, MSc. in CS/E graduate
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
July 05, 2008
Re: Descent speed
Bruno Medeiros a écrit :
> Ary Borenszweig wrote:
>>
>> Sorry... D is hard for autocompletion. :-(
>>
>> I want to provide correct autocompletion always. I know it's hard, but 
>> it's also a challenge, and I think it's possible...
> 
> I'm not sure if I have mentioned this before already or not. It may be 
> the case that achieving 100% may be to difficult, particularly 
> performance wise. Doug Schaefer, lead developer of CDT, commented on 
> this issue once, with regards to CDT's design:
> 
> http://cdtdoug.blogspot.com/2007/03/ego-less-development.html
> "I totally changed the approach we had taken and threw away the 
> requirement of being 100% accurate that we had be preaching in futility 
> since we started."
> 
> Since D also has meta-programming (compile-time variance), a similar 
> thing can also happen in Descent. On the other hand D is simpler and 
> more sane than C++, so that goal is easier to achieve than in C++. But 
> it may still be too difficult.
> (I personally prefer auto-complete and other semantic features to be 
> lightning fast even if only 99% accurate)

I just released a nightly build which should be very, very fast in 
comparison to older versions. I'm trying to retain correct semantic 
analysis while also doing it as fast as possible.
Top | Discussion index | About this forum | D home