ROB-2323 loki evals

[RoiGlinik]

No description provided.

[github-actions]

Results of HolmesGPT evals

• ask_holmes: 32/36 test cases were successful, 3 regressions, 1 setup failures

Test suite	Test case	Status
ask	01_how_many_pods	❌
ask	02_what_is_wrong_with_pod	✅
ask	04_related_k8s_events	✅
ask	05_image_version	✅
ask	09_crashpod	✅
ask	10_image_pull_backoff	✅
ask	110_k8s_events_image_pull	✅
ask	11_init_containers	✅
ask	13a_pending_node_selector_basic	✅
ask	14_pending_resources	✅
ask	15_failed_readiness_probe	✅
ask	17_oom_kill	✅
ask	18_oom_kill_from_issues_history	✅
ask	19_detect_missing_app_details	✅
ask	20_long_log_file_search	❌
ask	24_misconfigured_pvc	✅
ask	24a_misconfigured_pvc_basic	✅
ask	28_permissions_error	🚧
ask	39_failed_toolset	✅
ask	41_setup_argo	✅
ask	42_dns_issues_steps_new_tools	✅
ask	43_current_datetime_from_prompt	✅
ask	45_fetch_deployment_logs_simple	✅
ask	51_logs_summarize_errors	✅
ask	53_logs_find_term	✅
ask	54_not_truncated_when_getting_pods	❌
ask	59_label_based_counting	✅
ask	60_count_less_than	✅
ask	61_exact_match_counting	✅
ask	63_fetch_error_logs_no_errors	✅
ask	79_configmap_mount_issue	✅
ask	83_secret_not_found	✅
ask	86_configmap_like_but_secret	✅
ask	93_calling_datadog[0]	✅
ask	93_calling_datadog[1]	✅
ask	93_calling_datadog[2]	✅

Legend

• ✅ the test was successful
• :minus: the test was skipped
• ⚠️ the test failed but is known to be flaky or known to fail
• 🚧 the test had a setup failure (not a code regression)
• 🔧 the test failed due to mock data issues (not a code regression)
• 🚫 the test was throttled by API rate limits/overload
• ❌ the test failed and should be fixed before merging the PR