Understanding How TestZeus/Hercules Reduces Maintenance Efforts for Gherkin-Based Test Scripts

[Yeshwanth-Athmakuri]

Hello Team,

I’ve been exploring the features provided by TestZeus Hercules, and I must say the platform looks very promising. One aspect I’m keen to understand better is how Hercules helps reduce the maintenance efforts typically associated with Gherkin-based test scripts.

Specifically, I’m curious to know whether AI is involved in rewriting or updating Gherkin scripts in case of test failures. If so, how does the system distinguish whether the failure is due to an actual bug in the UI/backend or simply an outdated or incorrect test script?

If there's any documentation, video, or additional material that explains this process in more detail, I’d truly appreciate it if you could share it.

Thank you for your support!

Best regards,
Yeshwanth

[shriyanshagnihotri]

Thanks a lot, @Yeshwanth-Athmakuri, for the kind words!
1. Join our Slack Community: TestZeus Community
2. Explore real test cases on YouTube: TestZeus YouTube Channel

⸻

🧠 About Hercules

Hercules is designed as a state machine wrapped in a supervisor-agent pattern. Here’s how it works:
• It begins by generating a plan based on the Gherkin file you provide — this acts as the “base truth.”
• At runtime, it validates each executed step against this base plan using an LLM, which acts like a judge evaluating whether the current UI/App state aligns with what’s expected.
• If the step passes, it moves forward; otherwise, it stops and provides traceable insights.
• The platform version of Hercules adds strong guardrails around this base behavior to ensure robustness and traceability.

⸻

🚀 Try it out
• If you’re looking for hands-on experience, you can Start for Free with our SaaS version.
• If you’re a hardcore technologist, the open-source version is your playground. Feel free to raise questions or ideas on the community Slack.

⸻

🧬 On Deterministic Code Gen

We often get asked about Hercules generating deterministic test code.

Our belief: Deterministic code can be brittle — it often breaks when the UI or flows change, leading to high maintenance overhead.

Instead, we’re building smart feedback loops:
• If Hercules-generated code fails during a run, the system can autogenerate a fix and reattempt.
• If the agent-mode run still fails logically, we fail the test case — not silently, but with clarity.

This is a non-trivial problem at scale, and we’re treading carefully. Your feedback, ideas, and contributions from the community are not only welcome but essential.

Looking forward to seeing what we can build — together. 🙌

[Yeshwanth-Athmakuri]

Hi Team,

Thank you so much for the detailed and insightful explanation — it really helped clarify how Hercules operates, especially the state machine and supervisor-agent approach. We truly appreciate the transparency and the thoughtful design choices you’ve made around deterministic code and smart feedback loops.

We had one follow-up question regarding the application state evaluation during runtime. You mentioned that Hercules uses an LLM to evaluate whether the current UI/app state aligns with the expected outcome from the base plan. We're curious to understand this aspect a bit more:

Is this evaluation entirely based on AI decision-making?

What kind of system prompt or instruction set is used to guide the LLM?

Are there any defined heuristics or rules in place (e.g., “if element ID changes but label is same, consider it a match”) that the system uses to generate a fix or retry a failed step?

We're just trying to understand the boundaries between deterministic behavior and AI-driven adaptability — and where we, as users, can tune or influence the system’s decisions if needed.

Thanks again for building such a forward-thinking tool and for engaging so openly with the community. Looking forward to learning more!

Warm regards,
Yeshwanth

[generalrobin]

@Yeshwanth-Athmakuri - Our entire philosophy is around autonomy and agency. So we dont go towards the heuristics approach. As that brings us back to the "if-then-else" programming structure, rather than autonomy.
We would love to know more about your use cases, and answer any other questions on a quick call. Would you like to book a quick discussion here? https://cal.com/robin-gupta/testzeusdiscovery