Source Generator Performance: Incremental vs. Full Syntax Analysis

This example demonstrates the performance impact of using incremental syntax analysis versus full syntax analysis in a C# source generator. Incremental analysis can significantly reduce build times for larger projects by only re-analyzing changed code.

Concepts Behind the Snippet

Source Generators are powerful tools for generating code at compile time. However, inefficient source generators can significantly slow down build times. One key optimization is to use incremental syntax analysis. Instead of re-analyzing the entire codebase on every build, incremental analysis allows the generator to only process changes since the last build. The `IncrementalGeneratorInitializationContext` enables this behavior. When full syntax analysis is needed, ensure to use it wisely and as infrequently as possible.

Incremental Syntax Analysis Example

This `IncrementalHelloSourceGenerator` uses `IncrementalGeneratorInitializationContext`. It filters syntax nodes to find all `UsingDirectiveSyntax`. It efficiently gathers only the parts of the syntax that changed. The `CreateSyntaxProvider` method defines both a predicate (a filter) and a transform. The predicate `s is UsingDirectiveSyntax` checks if a syntax node is a `UsingDirectiveSyntax`. The transform casts it to the correct type. `RegisterSourceOutput` then creates the source file based on the collected `UsingDirectiveSyntax` node, but importantly, it only does this when the underlying syntax changes.

// Add this package to your project:  Microsoft.CodeAnalysis.CSharp.Workspaces

using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp.Syntax;
using Microsoft.CodeAnalysis.Text;
using System.Text;

namespace SourceGeneratorDemo
{
    [Generator]
    public class IncrementalHelloSourceGenerator : IIncrementalGenerator
    {
        public void Initialize(IncrementalGeneratorInitializationContext context)
        {
            // Find all syntax trees that contains using directives
            IncrementalValuesProvider<UsingDirectiveSyntax> usingDirectives = context.SyntaxProvider
                .CreateSyntaxProvider(
                    predicate: static (SyntaxNode s, CancellationToken ct) => s is UsingDirectiveSyntax, // filter to attribute syntax
                    transform: static (GeneratorSyntaxContext ctx, CancellationToken ct) => (UsingDirectiveSyntax)ctx.Node) // select the attribute syntax
                .Where(static m => m is not null);

            // Combine the selected syntax to create a compilation unit
            context.RegisterSourceOutput(usingDirectives.Collect(), static (spc, source) =>
            {
				StringBuilder sb = new StringBuilder();
				sb.AppendLine("// <auto-generated/>");
				sb.AppendLine("using System;");
				sb.AppendLine("namespace MyNamespace;");
				sb.AppendLine("public static class IncrementalGreetings");
				sb.AppendLine("{");
				sb.AppendLine("    public static string SayHello() => \"Hello from incremental source generator!\";");
				sb.AppendLine("}");
                
                spc.AddSource("IncrementalGreetings.g.cs", SourceText.From(sb.ToString(), Encoding.UTF8));
            });
        }
    }
}

Full Syntax Analysis (Less Performant)

The `FullHelloSourceGenerator` re-analyzes *all* syntax trees on every build. It fetches all syntax trees from the `Compilation`, then iterates through each tree, getting the root node and extracting all `UsingDirectiveSyntax` nodes. This approach is significantly slower, especially in large projects, because the same code is analyzed repeatedly even if it hasn't changed. Notice that the `Initialize` method uses a different type `GeneratorInitializationContext` because this is a standard `ISourceGenerator`.

// Add this package to your project:  Microsoft.CodeAnalysis.CSharp.Workspaces

using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.CSharp.Syntax;
using Microsoft.CodeAnalysis.Text;
using System.Text;
using System.Linq;

namespace SourceGeneratorDemo
{
    [Generator]
    public class FullHelloSourceGenerator : ISourceGenerator
    {
        public void Execute(SourceGeneratorContext context)
        {
            // Get the compilation from the context
            Compilation compilation = context.Compilation;

            // Find all syntax trees
            IEnumerable<SyntaxTree> syntaxTrees = compilation.SyntaxTrees;

            // Process all syntax trees to find all using directives
            List<UsingDirectiveSyntax> usingDirectives = new List<UsingDirectiveSyntax>();
            foreach (SyntaxTree tree in syntaxTrees)
            {
                var root = tree.GetRoot();
                usingDirectives.AddRange(root.DescendantNodes().OfType<UsingDirectiveSyntax>());
            }

			StringBuilder sb = new StringBuilder();
			sb.AppendLine("// <auto-generated/>");
			sb.AppendLine("using System;");
			sb.AppendLine("namespace MyNamespace;");
			sb.AppendLine("public static class FullGreetings");
			sb.AppendLine("{");
			sb.AppendLine("    public static string SayHello() => \"Hello from full source generator!\";");
			sb.AppendLine("}");

            // Add the generated source to the compilation
            context.AddSource("FullGreetings.g.cs", SourceText.From(sb.ToString(), Encoding.UTF8));
        }

        public void Initialize(GeneratorInitializationContext context)
        {
            // No initialization needed for this example
        }
    }
}

Real-Life Use Case Section

Imagine a source generator that generates boilerplate code for data transfer objects (DTOs) based on database schemas. If you re-generate all DTOs on every build (full syntax analysis), even if the database schema hasn't changed, the build time will increase unnecessarily. Using incremental syntax analysis, you can regenerate only the DTOs that correspond to modified tables, improving build performance.

Best Practices

• Prefer Incremental Analysis: Always strive to use `IIncrementalGenerator` and incremental syntax analysis unless you have a compelling reason not to.
• Cache Intermediate Results: If you perform complex computations in your source generator, cache the results so that they can be reused on subsequent builds.
• Profile Your Generator: Use profiling tools to identify performance bottlenecks in your source generator.
• Avoid String Concatenation in Loops: When building up large strings, prefer `StringBuilder` over repeated string concatenation.

Interview Tip

When asked about source generator performance, be prepared to discuss the importance of incremental analysis and how it improves build times. Explain the differences between `IIncrementalGenerator` and `ISourceGenerator`, and why the former is generally preferred for performance-critical scenarios.

When to Use Them

Use incremental source generators when performance is critical and you need to avoid re-analyzing unchanged code. This is especially important for large projects with many source files.

Memory Footprint

Incremental source generators typically have a smaller memory footprint than full syntax analysis generators because they only load and process the changed code. This can be beneficial in resource-constrained environments.

Alternatives

• T4 Templates: Text Template Transformation Toolkit (T4) is an older code generation technology that runs during development rather than at compile time. T4 templates can be easier to learn for simple scenarios, but they don't integrate as seamlessly with the compiler and can be less performant for complex code generation.
• Roslyn Analyzers: Analyzers can be used to enforce coding standards and provide suggestions for improving code quality. They don't generate code like source generators, but they can help reduce the amount of boilerplate code that needs to be written manually.

Pros

• Improved Build Performance: Incremental analysis reduces build times, especially for large projects.
• Better Integration with Compiler: Source generators integrate seamlessly with the C# compiler.
• Type Safety: Source generators operate on the compiler's syntax trees, providing type safety.

Cons

• Complexity: Source generators can be more complex to write than other code generation techniques.
• Debugging: Debugging source generators can be challenging.
• Learning Curve: Understanding the Roslyn APIs and the compiler pipeline can have a steep learning curve.