Slow Elixir.Credo.CLI.Task.PrepareChecksToRun

[zrzka]

We have been happily using Erlang 25.3.29, Elixir 1.14.5, and credo 1.6.1 for some time. It was super fast, mix credo runs in in a couple of seconds.

Analysis took 4.1 seconds (0.2s to load, 3.9s running 62 checks on 449 files)
3143 mods/funs, found no issues.

Recently, we've bumped Elixir & Erlang versions in one of our projects, and mix credo became incredibly slow (2s -> 80s on CI). We've tried following combinations when bumping Erlang, Elixir, and Credo, and all of them are super slow.

• Erlang 26.2.2, Elixir 1.16.2, credo 1.6.1
• Erlang 27.3.1, Elixir 1.18.3, credo 1.6.1
• Erlang 27.3.1, Elixir 1.18.3, credo 1.7.11

Analysis took 64.5 seconds (0.1s to load, 64.3s running 65 checks on 451 files)
3168 mods/funs, found no issues.

The difference between 62 & 65 checks can be ignored, we've added more checks and it's super slow even if we remove them.

When I run mix credo --debug locally, everything is fast except:

09:11:01.937 [info] Calling Elixir.Credo.CLI.Task.PrepareChecksToRun ...                                                
                                                                                                                        
09:13:05.141 [info] Finished Elixir.Credo.CLI.Task.PrepareChecksToRun in 123s ...

.credo.exs I'm using now (most of the checks removed)

%{
  configs: [
    %{
      name: "default",
      files: %{
        included: [
          "lib/**/*.{ex,exs}",
          "test/**/*.{ex,exs}",
          "mix.exs"
        ],
        excluded: []
      },
      plugins: [],
      requires: [],
      strict: false,
      parse_timeout: 5000,
      color: true,
      checks: %{
        enabled: [
          {Credo.Check.Readability.AliasOrder, []}
        ]
      }
    }
  ]
}

Execution struct

%Credo.Execution{                                           
  argv: ["--debug"],                                                                                                    
  cli_options: %Credo.CLI.Options{                                                                                      
    command: "suggest",                                                                                                 
    path: "/home/zrzka/projects/monorepo/work/hosted",                                                                  
    args: [],                                                                                                           
    switches: %{                                            
      debug: true,                                                                                                      
      files_included: [],                                                                                               
      checks_with_tag: [],                                                                                              
      checks_without_tag: [],                                                                                           
      files_excluded: []                                                                                                
    },                                                      
    unknown_switches: [],                                                                                               
    unknown_args: []                                                                                                    
  },                                                                                                                    
  cli_switches: [                                                                                                       
    debug: :boolean,                                                                                                    
    color: :boolean,                                        
    config_name: :string,                                                                                               
    config_file: :string,
    working_dir: :string,                                                                                               
    all_priorities: :boolean, 
    all: :boolean,                                                                                                      
    crash_on_error: :boolean,                                                                                           
    files_included: :keep,                                                                                              
    files_excluded: :keep,                                                                                              
    checks_with_tag: :keep,                                                                                             
    checks_without_tag: :keep,                              
    checks: :string,                                                                                                    
    enable_disabled_checks: :string,                        
    min_priority: :string,                                                                                              
    mute_exit_status: :boolean,                             
    first_run: :boolean,                                                                                                
    format: :string,                                                                                                    
    help: :boolean,                                                                                                     
    ignore_checks: :string,                                                                                             
    ignore: :string,                                                                                                    
    only: :string,                                                                                                      
    read_from_stdin: :boolean,                              
    strict: :boolean,                                                                                                   
    verbose: :boolean,                                                                                                  
    watch: :boolean                                                                                                     
  ],                                                                                                                    
  cli_aliases: [                                                                                                        
    C: :config_name,                                        
    D: :debug,                                                                                                          
    A: :all_priorities,                                                                                                 
    a: :all,                                                                                                            
    c: :checks,                                                                                                         
    h: :help,                                                                                                           
    i: :ignore                                              
  ],                                                                                                                    
  cli_switch_plugin_param_converters: [],                                                                               
  files: %{                                                                                                             
    excluded: [],                                                                                                       
    included: ["lib/**/*.{ex,exs}", "test/**/*.{ex,exs}", "mix.exs"]                                                    
  },                                                        
  color: true,                                                                                                          
  debug: true,
  checks: %{disabled: [], enabled: [{Credo.Check.Readability.AliasOrder, []}]},                                         
  requires: [],               
  plugins: [],                                                                                                          
  parse_timeout: 5000,                                                                                                  
  strict: false,                                                                                                        
  format: nil,                                                                                                          
  help: false,                                                                                                          
  verbose: false,                                           
  version: false,                                                                                                       
  all: false,                                               
  crash_on_error: true,                                                                                                 
  enable_disabled_checks: nil,                              
  ignore_checks_tags: [],                                                                                               
  ignore_checks: nil,                                                                                                   
  min_priority: 0,                                                                                                      
  mute_exit_status: false,                                                                                              
  only_checks_tags: [],                                                                                                 
  only_checks: nil,                                                                                                     
  read_from_stdin: false,                                   
  max_concurrent_check_runs: 22,                                                                                        
  pipeline_map: %{                                                                                                      
    Credo.CLI.Command.Explain.ExplainCommand.ExplainCheck => [                                                          
      print_explanation: [                                                                                              
        {Credo.CLI.Command.Explain.ExplainCommand.ExplainCheck, []}                                                     
      ]                                                     
    ],                                                                                                                  
    Credo.CLI.Command.Explain.ExplainCommand.ExplainIssue => [                                                          
      validate_given_location: [                                                                                        
        {Credo.CLI.Command.Explain.ExplainCommand.ExplainIssuePreCheck, []}                                             
      ],                                                                                                                
      load_and_validate_source_files: [                     
        {Credo.CLI.Task.LoadAndValidateSourceFiles, []}                                                                 
      ],                                                                                                                
      prepare_analysis: [{Credo.CLI.Task.PrepareChecksToRun, []}],                                                      
      run_analysis: [{Credo.CLI.Task.RunChecks, []}],                                                                   
      filter_issues: [{Credo.CLI.Task.SetRelevantIssues, []}],                                                          
      print_explanation: [                                  
        {Credo.CLI.Command.Explain.ExplainCommand.ExplainIssue, []}                                                     
      ]
    ],                                                                                                                  
    Credo.Execution => [      
      __pre__: [                                                                                                        
        Credo.Execution.Task.AppendDefaultConfig,                                                                       
        Credo.Execution.Task.AppendExtraConfig,                                                                         
        {Credo.Execution.Task.ParseOptions, [parser_mode: :preliminary]},                                               
        Credo.Execution.Task.ConvertCLIOptionsToConfig,                                                                 
        Credo.Execution.Task.InitializePlugins              
      ],                                                                                                                
      parse_cli_options: [                                  
        {Credo.Execution.Task.ParseOptions, [parser_mode: :preliminary]}                                                
      ],                                                    
      initialize_plugins: [],                                                                                           
      determine_command: [Credo.Execution.Task.DetermineCommand],                                                       
      set_default_command: [Credo.Execution.Task.SetDefaultCommand],                                                    
      initialize_command: [Credo.Execution.Task.InitializeCommand],                                                     
      parse_cli_options_final: [                                                                                        
        {Credo.Execution.Task.ParseOptions, [parser_mode: :strict]}                                                     
      ],                                                    
      validate_cli_options: [Credo.Execution.Task.ValidateOptions],                                                     
      convert_cli_options_to_config: [Credo.Execution.Task.ConvertCLIOptionsToConfig],                                  
      resolve_config: [Credo.Execution.Task.UseColors,                                                                  
       Credo.Execution.Task.RequireRequires],                                                                           
      validate_config: [Credo.Execution.Task.ValidateConfig],                                                           
      run_command: [Credo.Execution.Task.RunCommand],       
      halt_execution: [Credo.Execution.Task.AssignExitStatusForIssues]                                                  
    ],                                                                                                                  
    "diff" => [                                                                                                         
      load_and_validate_source_files: [                                                                                 
        {Credo.CLI.Task.LoadAndValidateSourceFiles, []}                                                                 
      ],                                                    
      prepare_analysis: [{Credo.CLI.Task.PrepareChecksToRun, []}],                                                      
      print_previous_analysis: [                                                                                        
        {Credo.CLI.Command.Diff.Task.GetGitDiff, []},                                                                   
        {Credo.CLI.Command.Diff.Task.PrintBeforeInfo, []}                                                               
      ],                                                                                                                
      run_analysis: [{Credo.CLI.Task.RunChecks, []}],       
      filter_issues: [                                                                                                  
        {Credo.CLI.Task.SetRelevantIssues, []},
        {Credo.CLI.Command.Diff.Task.FilterIssues, []}                                                                  
      ],                      
      print_after_analysis: [                                                                                           
        {Credo.CLI.Command.Diff.Task.PrintResultsAndSummary, []}                                                        
      ],                                                                                                                
      filter_issues_for_exit_status: [                                                                                  
        {Credo.CLI.Command.Diff.Task.FilterIssuesForExitStatus, []}                                                     
      ]                                                     
    ],                                                                                                                  
    "info" => [                                             
      load_and_validate_source_files: [                                                                                 
        {Credo.CLI.Task.LoadAndValidateSourceFiles, []}     
      ],                                                                                                                
      prepare_analysis: [{Credo.CLI.Task.PrepareChecksToRun, []}],                                                      
      print_info: [{Credo.CLI.Command.Info.InfoCommand.PrintInfo, []}]                                                  
    ],                                                                                                                  
    "list" => [                                                                                                         
      load_and_validate_source_files: [                                                                                 
        {Credo.CLI.Task.LoadAndValidateSourceFiles, []}     
      ],                                                                                                                
      prepare_analysis: [{Credo.CLI.Task.PrepareChecksToRun, []}],                                                      
      print_before_analysis: [                                                                                          
        {Credo.CLI.Command.List.ListCommand.PrintBeforeInfo, []}                                                        
      ],                                                                                                                
      run_analysis: [{Credo.CLI.Task.RunChecks, []}],       
      filter_issues: [{Credo.CLI.Task.SetRelevantIssues, []}],                                                          
      print_after_analysis: [                                                                                           
        {Credo.CLI.Command.List.ListCommand.PrintResultsAndSummary, []}                                                 
      ]                                                                                                                 
    ],                                                                                                                  
    "suggest" => [                                          
      load_and_validate_source_files: [Credo.CLI.Task.LoadAndValidateSourceFiles],                                      
      prepare_analysis: [Credo.CLI.Task.PrepareChecksToRun],                                                            
      __manipulate_config_if_rerun__: [Credo.CLI.Command.Suggest.SuggestCommand.ManipulateConfigIfRerun],               
      print_before_analysis: [Credo.CLI.Command.Suggest.SuggestCommand.PrintBeforeInfo],                                
      run_analysis: [Credo.CLI.Task.RunChecks],                                                                         
      filter_issues: [Credo.CLI.Task.SetRelevantIssues],    
      print_after_analysis: [Credo.CLI.Command.Suggest.SuggestCommand.PrintResultsAndSummary]                           
    ]
  },                                                                                                                    
  commands: %{                
    "categories" => Credo.CLI.Command.Categories.CategoriesCommand,                                                     
    "diff" => Credo.CLI.Command.Diff.DiffCommand,                                                                       
    "explain" => Credo.CLI.Command.Explain.ExplainCommand,                                                              
    "gen.check" => Credo.CLI.Command.GenCheck,                                                                          
    "gen.config" => Credo.CLI.Command.GenConfig,                                                                        
    "help" => Credo.CLI.Command.Help,                       
    "info" => Credo.CLI.Command.Info.InfoCommand,                                                                       
    "list" => Credo.CLI.Command.List.ListCommand,           
    "suggest" => Credo.CLI.Command.Suggest.SuggestCommand,                                                              
    "version" => Credo.CLI.Command.Version                  
  },                                                                                                                    
  config_files: [],                                                                                                     
  current_task: Credo.Execution.Task.ValidateConfig,                                                                    
  parent_task: nil,                                                                                                     
  initializing_plugin: nil,                                                                                             
  halted: false,                                                                                                        
  config_files_pid: #PID<0.203.0>,                          
  source_files_pid: #PID<0.205.0>,                                                                                      
  issues_pid: #PID<0.204.0>,                                                                                            
  timing_pid: #PID<0.206.0>,                                                                                            
  skipped_checks: nil,                                                                                                  
  assigns: %{},                                                                                                         
  results: %{},                                             
  config_comment_map: %{}                                                                                               
}

Any ideas what can be wrong, any recommendations? Anything I can provide to get to the bottom of this?

[zrzka]

So far, tracked it down to Credo.Check.ConfigCommentFinder and one problematic file icons.ex. It's 1.4MB file, generated, full of functions/icons like this:

  def merge(opts \\ []) do
    class = opts[:class]

    ~e"""
    <svg class="<%= class %>" viewBox="0 0 32 32" fill="none" xmlns="http://www.w3.org/2000/svg">
      <path d="M31 1H16V19H31V17.7143V6.46429V1Z" stroke="#777CF0" stroke-width="2"/>
      <path d="M16 1H1V19H16V17.7143V6.46429V1Z" stroke="#777CF0" stroke-width="2"/>
      <path d="M12 26L16 30L20 26" stroke="white" stroke-width="2" stroke-linecap="round"/>
      <line x1="16" y1="22" x2="16" y2="29" stroke="white" stroke-width="2"/>
      <path d="M27 7.1084C27 7.1084 16 12.1084 16 22.0003C16 12.1084 5 7.1084 5 7.1084" stroke="white" stroke-width="2"/>
    </svg>
    """
  end

[zrzka]

Keeping it open, just in case you see a way how to improve handling of such files. Otherwise I'm happy w/ the exclusion.

[rrrene]

Thx for reporting and your investigation!

I have an idea how to improve this, let me tinker a bit 👍

[zrzka]

Thanks, happy to test anything if you'll come up with something.

[rrrene]

Thanks again for reporting this 😀 It is now fixed on 1188-comment-finder-slow.

You should be able to try this by setting the Credo dep to

{:credo, github: "rrrene/credo", branch: "1188-comment-finder-slow"}

Please report back if your issue is solved! 👍

[zrzka]

master

• Elixir.Credo.CLI.Task.PrepareChecksToRun in 50s ...
• Elixir.Credo.CLI.Task.RunChecks in 52s ...

1188-comment-finder-slow:

• Elixir.Credo.CLI.Task.PrepareChecksToRun in 6ms ...
• Elixir.Credo.CLI.Task.RunChecks in 53s ...

PrepareChecksToRun is fast now, thanks for fixing it!

It also seems that I focused too much on PrepareChecksToRun and missed RunChecks. I've reconfigured Credo to include just the offending file and tracked it down to SpaceAfterCommas check. The part that is super-slow is Sigils.replace_with_spaces(" ", " ", source_file.filename):

• InterpolationHelper.replace_interpolations(interpolation_replacement, filename) - fast
• parse_code("", replacement, empty_line_replacement) - slow

Does it make sense to create a separate issue for it? When I open the offending icons.ex file, it has following properties:

• 207 functions
• each function returns ~e""" some-svg """
• file size 1354922 bytes (1.4MB)
• 5053 lines
• most of the functions are small, but some are quite big
  • found one that has 244 lines & 226kB

[rrrene]

@zrzka This was insightful, thank you! 🥳

I pushed an update to the branch, where I reworked some checks. They are currently producing some false positives/negatives, but maybe you could check the speed again?

(we will sort the false positives/negatives out eventually, now I am more interested if we are on the right path 😅)

[zrzka]

Whoa, here it is (the whole project, not just this one file):

Analysis took 1.1 seconds (0.3s to load, 0.7s running 44 checks on 447 files)

We're definitely on the right path ;)