Summary of Progress (October 14 – October 20)

This week, I focused on addressing feedback and adding tests for SARIF (Static Analysis Results Interchange Format) support in the DMD compiler. The current PR introduces a --sarif compiler flag to generate structured SARIF output for error messages.

PR: feat(errors): add SARIF report generation #16986

What I Worked On:

• Adding Tests and Addressing Feedback
  I spent time this week addressing the feedback provided on my PR, particularly refining the SARIF implementation. I added a test case to validate the output and ensure the correctness of SARIF reports.

• Handling Regex Matching for Variable Outputs
  I encountered issues with matching variable outputs in test pipelines versus local outputs. After multiple attempts to resolve these through regexes, I decided to implement a simpler version of the SARIF report, excluding certain complexities such as relative paths, to avoid complications.

• Milestone 1 Report Preparation
  I also dedicated time to finalizing and submitting my Milestone 1 report, which summarizes the progress from September 15 to October 14, 2024.

• Exploring Milestone 2 Tasks
  I began exploring the objectives of Milestone 2, which involve enhancing error message utilities such as adding more detailed squiggles and error codes. I reviewed the existing utilities and documentation to understand how error codes and squiggles are currently implemented in the DMD compiler.

Challenges:

• PR Merge
  I had hoped my PR would be merged this week, which would allow me to start working on additional updates like using MessageStyle for SARIF output instead of a compiler flag. However, the merge is still pending, which is currently blocking me from making further progress on SARIF-related updates. Although the PR has been approved by one maintainer, it still needs to be merged.

• Resolving Regex Issues
  The most significant challenge was handling regexes to match variable outputs between local tests and CI pipelines. Ultimately, I simplified the SARIF version to avoid complexities related to relative paths and other variable outputs.

Next Steps:

• MessageStyle Implementation
  Once the current PR is merged, I plan to work on implementing SARIF using MessageStyle, which will be done in a separate PR.

• Further Exploration of Milestone 2
  I will continue researching the error message utilities and explore how to integrate more detailed squiggles and error codes into the compiler. Any links to existing Bugzilla issues (specifically related to this) or resources would be really helpful.

On Monday, 21 October 2024 at 08:44:29 UTC, Royal Simpson Pinto wrote:

Hi,
It's nice that you are working on this!

I'd like to strongly urge you to put priority on these two things:

• Commandline interface. Please make this the same as GCC and Clang, such that tools do not have to implement a D-specific interface, and in may "just work" without any change needing in tooling. This includes having the option to output SARIF to a file.
• Output format. Mimic GCC and Clang as much as possible in what you output. Changes needed: (a) do not output errors to the same output stream as both normal and SARIF formats, because that immediately breaks tooling compatibility. (b) specify SARIF schema and adhere to it [1]. The output of your PR is not valid SARIF output [2]. For example, "invocation" is used wrong (should be inside a "runs" list).

The reason for this priority over anything else, is that both these are likely necessary to actually be able to use dlang+SARIF with external tools. So whatever other work you'll do will only be usable after these two items are done.

cheers,
Johan

[1] I again strongly advise to do what clang and GCC do, because it means that tooling will at least support that schema. They are using version 2.1.0 apparently. (https://docs.oasis-open.org/sarif/sarif/v2.1.0/cos02/schemas/sarif-schema-2.1.0.json)

[2] https://sarifweb.azurewebsites.net/Validation

On Monday, 21 October 2024 at 19:46:56 UTC, Johan wrote:

Hi Johan,

Thanks for your feedback!

I've made adjustments based on your suggestions:

• feat(errors): implement SARIF support as a MessageStyle #17016 – PR #17016
• fix(errors): adhere to SARIF 2.1.0 schema #17020 – PR #17020

These should align the command-line interface and output format with GCC and Clang, and ensure external tools can work seamlessly with DMD+SARIF.

Cheers,
Royal