Thread overview
dmdtags 1.0.0: an accurate tag generator for D source code
Aug 27
Dennis
Aug 28
evilrat
August 27

dmdtags is a tags file generator for D source code that uses the DMD compiler frontend for accurate parsing.

This release supports 100%-accurate parsing of arbitrary D code (tested on DMD and Phobos sources), as well as the most commonly-used command line options, -R, -o, and -a. The generated tags file has been tested for compatibility with Vim and is compliant with the POSIX standard for ctags, so any editor with ctags support should be able to use it.

What?

A tags file is a lightweight plain-text index of the symbols in a project. Editors that support tags files, such as Vim and Emacs, can use this index to help with things like project navigation and tab completion.

A tags file generator is, as you might expect, a program that reads source code and generates a tags file with entries for the symbols in that code.

Why?

universal-ctags, the current most-popular and best-maintained tags file generator, claims support for many programming languages, including D. However, its D parser is not well-maintained, and it often excludes large numbers of symbols from its output due to parsing failures.

Because dmdtags uses the DMD frontend for parsing, its results will always be accurate and up-to-date. For pure D projects, it can be used as a replacement for universal-ctags. For mixed-language projects, it can be used together with other tag generators with the --append option.

Where?

August 27

On Friday, 27 August 2021 at 21:38:58 UTC, Paul Backus wrote:

>

Editors that support tags files, such as Vim and Emacs, can use this index to help with things like project navigation and tab completion.

Cool! Now I'll have to look if I can make this work with Visual Studio Code, since code-d which uses Dsymbol tends to be unreliable, and this looks ideal for Phobos / Druntime symbol completion.

August 27

On Friday, 27 August 2021 at 22:01:59 UTC, Dennis wrote:

>

On Friday, 27 August 2021 at 21:38:58 UTC, Paul Backus wrote:

>

Editors that support tags files, such as Vim and Emacs, can use this index to help with things like project navigation and tab completion.

Cool! Now I'll have to look if I can make this work with Visual Studio Code, since code-d which uses Dsymbol tends to be unreliable, and this looks ideal for Phobos / Druntime symbol completion.

actually it uses D-Scanner and should be a fairly easy drop-in replacement.

I'm just worried about how the memory usage will grow with this, considering dmd never frees. Maybe I should make it run as external tool instead of a library so the OS cleans up, but for that get a performance penalty especially on Windows.

August 27

On Friday, 27 August 2021 at 22:45:15 UTC, WebFreak001 wrote:

>

I'm just worried about how the memory usage will grow with this, considering dmd never frees. Maybe I should make it run as external tool instead of a library so the OS cleans up, but for that get a performance penalty especially on Windows.

dmdtags does not run in the background, so any memory it uses will be freed at process exit, as soon as it has finished writing the tags file. And because it does not do any semantic analysis, only parsing, it does not use much memory while running in the first place.

According to /usr/bin/time, on my personal machine, using dmdtags to generate a tags file for all of Phobos takes slightly less than 1 second and has a peak resident set size of around 100MB.

August 27
On 8/27/21 2:38 PM, Paul Backus wrote:

> `dmdtags` is a tags file generator for D source code that uses the DMD
> compiler frontend for accurate parsing.

Thanks! I had stopped using tags files with Emacs for the reasons you give. With this, I will be more efficient. :)

And, I will be a happy puppy once Vladimir Panteleev finishes his D mode for Emacs, which he is writing from scratch.

Ali

August 28

On Friday, 27 August 2021 at 21:38:58 UTC, Paul Backus wrote:

>

dmdtags is a tags file generator for D source code that uses the DMD compiler frontend for accurate parsing.

This release supports 100%-accurate parsing of arbitrary D code (tested on DMD and Phobos sources), as well as the most commonly-used command line options, -R, -o, and -a. The generated tags file has been tested for compatibility with Vim and is compliant with the POSIX standard for ctags, so any editor with ctags support should be able to use it.

What?

A tags file is a lightweight plain-text index of the symbols in a project. Editors that support tags files, such as Vim and Emacs, can use this index to help with things like project navigation and tab completion.

A tags file generator is, as you might expect, a program that reads source code and generates a tags file with entries for the symbols in that code.

Why?

universal-ctags, the current most-popular and best-maintained tags file generator, claims support for many programming languages, including D. However, its D parser is not well-maintained, and it often excludes large numbers of symbols from its output due to parsing failures.

Because dmdtags uses the DMD frontend for parsing, its results will always be accurate and up-to-date. For pure D projects, it can be used as a replacement for universal-ctags. For mixed-language projects, it can be used together with other tag generators with the --append option.

Where?

# dmdtags -R ./source/ -o -
!_TAG_FILE_SORTED	1	/0=unsorted, 1=sorted, 2=foldcase/
App	./source/ui/app.d	13;"	T
App	./source/ui/app.d	15;"	a
App	./source/ui/app.d	19;"	T
App	./source/ui/app.d	21;"	s
__ctor	./source/ui/app.d	27;"	f
__ctor	./source/ui/app.d	30;"	f
_result	./source/ui/app.d	23;"	v
app	./source/ui/app.d	1;"	M
defaultEventLoop	./source/ui/app.d	48;"	f
defaultInitFunc	./source/ui/app.d	62;"	T
defaultInitFunc	./source/ui/app.d	62;"	f
window	./source/ui/app.d	26;"	v

Perfecto!

i can use it in Sublime Text editor...
Need to thinks, how it will be implemented...

August 28

On Friday, 27 August 2021 at 22:45:15 UTC, WebFreak001 wrote:

>

I'm just worried about how the memory usage will grow with this, considering dmd never frees.

Still not sure if it should be used in same process, but DMD definitely has -lowmem switch for turning on GC, and for DMD as a library there is initDMD() and deinitializeDMD() pair.

Haven't tested it myself wrt. to memory usage, but it is used by DMD tests.
Another concern would be code analysis time, even with skipping semantic passes every run is a new invocation - DMD simply wasn't designed to be a language server where it could just iteratively update compilation database.

IIRC VisualD with DMD frontend enabled has noticable long pauses on showing completion (from 100ms to few seconds on larger projects), which renders it unusable for me. Could be that same reason, no idea.

August 31
On Fri, Aug 27, 2021 at 09:38:58PM +0000, Paul Backus via Digitalmars-d-announce wrote:
> `dmdtags` is a tags file generator for D source code that uses the DMD compiler frontend for accurate parsing.
> 
> This release supports 100%-accurate parsing of arbitrary D code (tested on DMD and Phobos sources), as well as the most commonly-used command line options, `-R`, `-o`, and `-a`. The generated tags file has been tested for compatibility with Vim and is compliant with the [POSIX standard for `ctags`][posix], so any editor with `ctags` support should be able to use it.

This is AWESOME!!!  Thanks a ton for this... I'll definitely be using this in the near future!


[...]
> [`universal-ctags`][uctags], the current most-popular and best-maintained tags file generator, claims support for many programming languages, including D. However, its D parser is not well-maintained, and it often excludes large numbers of symbols from its output due to parsing failures.
> 
> Because `dmdtags` uses the DMD frontend for parsing, its results will always be accurate and up-to-date. For pure D projects, it can be used as a replacement for `universal-ctags`. For mixed-language projects, it can be used together with other tag generators with the `--append` option.

Is there any hope of merging this back to upstream ctags?

Regardless, this is awesome.


T

-- 
Long, long ago, the ancient Chinese invented a device that lets them see through walls. It was called the "window".
September 03

On Friday, 27 August 2021 at 21:38:58 UTC, Paul Backus wrote:

>

dmdtags is a tags file generator for D source code that uses the DMD compiler frontend for accurate parsing.
...

Where?

Wow, what perfect timing! I just switched sublime text only 4
days ago. Time to read some docs and figure out how to set this
up.

Thanks a lot!