Proposal: Add --if-changed flag to speed up compilation of large files and folders in CI

[carsonjan]

Proposal: Add --if-changed flag to speed up compilation of large files and folders in CI

                ┌─────────┐                               
                │input.d2 │                               
                │         │                               
                └─────────┘                               
                     │                                    
                     ▼                                    
                 **********                               
       **********          **********                     
 ******  output.svg exits, has rev and id  ******                
      **********          **********                      
                **********                                
                                                          
        │                         │                       
        │                       true                      
        │                         │                       
        │                         ▼                       
        │                        ***                      
        │                      **   **                    
        │                    **       **                  
        │                  **           **                
        │                **┌───────────────┐              
        │              **  │get rev and id │**            
        │            **    │               │  **          
        │          **      └───────────────┘    **        
        │        **               │               **      
        │      **                 │                 **    
        │    **                   │                   **  
      false**                     │                     **
        │    **                equals?                **  
        │      **                 │                 **    
        │        **               │               **      
        │          **             │             **        
        │            ┌───────────────────────────────┐    
        │            │get current rev and compute id │    
        │            │   **               **         │    
        │            └───────────────────────────────┘    
        │                    **       **                  
        │                      **   **                    
        │                        ***                      
        │                     │     │                     
        │                   false   │                     
        │                     │     │                     
        └───────┐  ┌──────────┘     │                     
                │  │              true                    
                ▼  ▼                │                     
            ┌─────────┐             │                     
            │ render  │             │                     
            │         │             │                     
            └─────────┘             │                     
                 │                  │                     
                 │                  ▼                     
                 │               ┌───────┐                
           add metadata          │ skip  │                
                 │               │       │                
                 │               └───────┘                
                 │                  │                     
                 └──────┐   ┌───────┘                     
                        │   │                             
                        ▼   ▼                             
                   ┌───────────┐                          
                   │output.svg │                          
                   │           │                          
                   └───────────┘                          
                          │                               
                          ▼                               
               ┌───────────────────────┐                  
               │downstream (if needed) │                  
               │                       │                  
               └───────────────────────┘                  

Expected usage

original

command line

$ d2 file.d2
success: successfully compiled file.d2 to file.svg in 317.16995ms

file.svg

<?xml version="1.0" encoding="utf-8"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" data-d2-version="v0.7.1" preserveAspectRatio="xMinYMin meet" viewBox="0 0 255 434">
...more

.pre-commit-hooks.yaml

- id: d2
  name: d2
  description: Compiles and renders d2 files
  entry: sh -ec 'for f in "${@}"; do d2 --omit-version "${f}"; done' --
  language: golang
  files: \.d2$

.pre-commit-config.yaml

repos:
  - repo: https://github.com/terrastruct/d2
    rev: v0.7.1
    hooks:
      - id: d2

new

command line

$ d2 file.d2 --if-changed
skipped: no changes found for file.d2 to file.svg. skipped in 17.16995ms

file.svg

<?xml version="1.0" encoding="utf-8"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" data-d2-version="v0.7.1" preserveAspectRatio="xMinYMin meet" viewBox="0 0 255 434" data-d2-source-id-rev="1" data-d2-source-id="some-very-unique-id">
...more

.pre-commit-hooks.yaml

(unchanged)

.pre-commit-config.yaml

repos:
  - repo: https://github.com/terrastruct/d2
    rev: v0.7.1
    hooks:
      - id: d2
        args: ["--if-changed"]

Possible implementation

• Modify d2cli/main.go's _render
• Collect all args that affect the output file and hash it.
• Similar to d2target/d2target.go's func (diagram Diagram) HashID(salt *string) (string, error)

Backwards compatibility / robustness

• Optional flag
• Silently re-compiles if data-d2-source-id-rev or data-d2-source-id is missing