chore: Enhance logging, validation handling, and HR workflow tests

[Harmanpreet-Microsoft]

Purpose

This pull request significantly enhances the robustness and reliability of the E2E test suite for the HomePage by improving error handling, validation logic, and reporting. The changes include smarter handling of clarification steps, more resilient validation of AI-driven responses, comprehensive error detection, and improved HTML reporting with screenshot support.

Key improvements include:

Test robustness and error handling

• Updated task plan approval methods (approve_retail_task_plan, approve_rfp_task_plan, approve_contract_compliance_task_plan) to treat clarification input as a non-error condition (expected for some workflows), returning a boolean instead of raising exceptions. This prevents unnecessary test failures for valid scenarios. [1] [2] [3]
• Enhanced error detection in validate_rai_error_message to check for a wider range of error messages and fallback states, including silent rejections, and to capture screenshots for easier debugging.

AI response validation and synchronization

• All major response validation methods (retail, product marketing, HR, RFP, contract compliance) now explicitly wait for the "AI Thinking Process" indicator to disappear before proceeding, reducing flakiness due to asynchronous UI updates. [1] [2] [3] [4] [5]
• Response validation now includes checks for expected content patterns (e.g., headings and key phrases) to ensure the AI-generated responses are not only present but also semantically correct. [1] [2] [3]

Reporting and test infrastructure

• Enabled HTML reporting for test runs by adding the --html=report.html option in pytest.ini.
• Added support for capturing screenshots for all tests by default, with configurable behavior, and introduced a utility for generating clean screenshot filenames.

Minor improvements

• Added a locator for the "AI Thinking Process" element to facilitate synchronization in tests.

These changes collectively make the test suite more stable, easier to debug, and provide richer feedback for failures.

• ...

Does this introduce a breaking change?

• Yes
• No

How to Test

• Get the code

git clone [repo-address]
cd [repo-name]
git checkout [branch-name]
npm install

• Test the code

What to Check

Verify that the following are valid

• ...

Other Information

[Copilot]

Pull request overview

This PR strengthens the Playwright-based E2E test harness under tests/e2e-test/ by adding HTML reporting + screenshot capture, improving UI synchronization around AI response completion, and adding a focused HR workflow smoke test to reduce debugging friction.

Changes:

• Added an HR-only E2E smoke test and updated RAI prompt sender method usage in the existing smoke suite.
• Enhanced BIABPage response validation to wait for an “AI Thinking Process” indicator and broadened RAI error-state detection.
• Enabled pytest-html reporting and added screenshot capture/attachment behavior via conftest.py.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.

File	Description
tests/e2e-test/tests/test_MACAE_Smoke_test.py	Adds an HR-only workflow test and updates calls to the renamed RAI prompt send method.
tests/e2e-test/tests/conftest.py	Adds screenshot capture/attachment to pytest-html reports and filename sanitization utilities.
tests/e2e-test/pytest.ini	Enables HTML reporting via --html=report.html.
tests/e2e-test/pages/HomePage.py	Adds AI “thinking” synchronization, expands response content checks, changes task-plan approval clarification handling, and updates RAI error validation behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

🎉 This PR is included in version 4.1.1 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀